2023-10-06T03:43:34.743 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-06T03:43:34.743 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/snaps}} --owner scheduled_yuriw@teuthology --verbose --archive /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412234 -- /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412234/orig.config.yaml 2023-10-06T03:43:34.790 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T03:43:34.872 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412234 branch: wip-yuri3-testing-2023-10-05-0943-quincy description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/snaps}} email: yweinste@redhat.com first_in_suite: false job_id: '7412234' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri3-testing-2023-10-05-0943-quincy ceph: cephfs: max_mds: 5 session_timeout: 300 conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: false mds op complaint time: 180 mds verify scatter: true mds_sleep_rank_change: 5000000.0 mon client directed command retry: 5 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds osd ms inject socket failures: 2500 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon client directed command retry: 5 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds ms inject socket failures: 2500 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - Replacing daemon mds sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri3-testing-2023-10-05-0943-quincy sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 4002 sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri3-testing-2023-10-05-0943-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 targets: smithi157.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBqwONQQ9SRgP7wl/eGNxCKZEU6YhxyvvULbzoPkEvm/SMYB0MAzt8PvgJdbr+t7Aux/At50nJ+Dt9rZoU+QuSw= smithi175.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMD1lY8k/JMCThp+vPfiNZh2sGw9juoPiQx7FaD+ZXMGbasjgL2nRuw9URi7/tDh0qjtjGWE635fMHv52/5FaFg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - kclient: null - mds_thrash: null - workunit: clients: all: - fs/snaps teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-05_23:54:57 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-06T03:43:34.873 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa; will attempt to use it 2023-10-06T03:43:34.874 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa/tasks 2023-10-06T03:43:34.874 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-06T03:43:34.875 INFO:teuthology.task.internal:Checking packages... 2023-10-06T03:43:34.899 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564' 2023-10-06T03:43:34.900 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-06T03:43:34.900 INFO:teuthology.packaging:ref: None 2023-10-06T03:43:34.900 INFO:teuthology.packaging:tag: None 2023-10-06T03:43:34.900 INFO:teuthology.packaging:branch: wip-yuri3-testing-2023-10-05-0943-quincy 2023-10-06T03:43:34.900 INFO:teuthology.packaging:sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T03:43:34.900 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri3-testing-2023-10-05-0943-quincy 2023-10-06T03:43:35.117 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1274.g68e53a4a 2023-10-06T03:43:35.118 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-06T03:43:35.124 INFO:teuthology.task.internal:no buildpackages task found 2023-10-06T03:43:35.124 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-06T03:43:35.138 INFO:teuthology.task.internal:Saving configuration 2023-10-06T03:43:35.155 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-06T03:43:35.166 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T03:43:35.193 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412234', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 03:37:55.989510', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBqwONQQ9SRgP7wl/eGNxCKZEU6YhxyvvULbzoPkEvm/SMYB0MAzt8PvgJdbr+t7Aux/At50nJ+Dt9rZoU+QuSw='} 2023-10-06T03:43:35.218 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412234', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 03:37:55.987883', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMD1lY8k/JMCThp+vPfiNZh2sGw9juoPiQx7FaD+ZXMGbasjgL2nRuw9URi7/tDh0qjtjGWE635fMHv52/5FaFg='} 2023-10-06T03:43:35.218 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-06T03:43:35.230 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2023-10-06T03:43:35.230 INFO:teuthology.task.internal:roles: ubuntu@smithi175.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2023-10-06T03:43:35.230 INFO:teuthology.run_tasks:Running task console_log... 2023-10-06T03:43:35.311 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7ce66acb80>, signals=[15]) 2023-10-06T03:43:35.312 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-06T03:43:35.323 INFO:teuthology.task.internal:Opening connections... 2023-10-06T03:43:35.324 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi157.front.sepia.ceph.com 2023-10-06T03:43:35.326 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T03:43:35.395 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi175.front.sepia.ceph.com 2023-10-06T03:43:35.396 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T03:43:35.464 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-06T03:43:35.474 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2023-10-06T03:43:35.492 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2023-10-06T03:43:35.492 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2023-10-06T03:43:35.549 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="Red Hat Enterprise Linux" 2023-10-06T03:43:35.549 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="8.4 (Ootpa)" 2023-10-06T03:43:35.549 INFO:teuthology.orchestra.run.smithi157.stdout:ID="rhel" 2023-10-06T03:43:35.549 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="fedora" 2023-10-06T03:43:35.549 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="8.4" 2023-10-06T03:43:35.549 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el8" 2023-10-06T03:43:35.549 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-06T03:43:35.549 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2023-10-06T03:43:35.549 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-06T03:43:35.550 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://www.redhat.com/" 2023-10-06T03:43:35.550 INFO:teuthology.orchestra.run.smithi157.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-06T03:43:35.550 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T03:43:35.550 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:43:35.550 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T03:43:35.550 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-06T03:43:35.550 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-06T03:43:35.550 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-06T03:43:35.551 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2023-10-06T03:43:35.578 DEBUG:teuthology.orchestra.run.smithi175:> uname -m 2023-10-06T03:43:35.595 INFO:teuthology.orchestra.run.smithi175.stdout:x86_64 2023-10-06T03:43:35.596 DEBUG:teuthology.orchestra.run.smithi175:> cat /etc/os-release 2023-10-06T03:43:35.652 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="Red Hat Enterprise Linux" 2023-10-06T03:43:35.652 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="8.4 (Ootpa)" 2023-10-06T03:43:35.652 INFO:teuthology.orchestra.run.smithi175.stdout:ID="rhel" 2023-10-06T03:43:35.652 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE="fedora" 2023-10-06T03:43:35.652 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="8.4" 2023-10-06T03:43:35.652 INFO:teuthology.orchestra.run.smithi175.stdout:PLATFORM_ID="platform:el8" 2023-10-06T03:43:35.653 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-06T03:43:35.653 INFO:teuthology.orchestra.run.smithi175.stdout:ANSI_COLOR="0;31" 2023-10-06T03:43:35.653 INFO:teuthology.orchestra.run.smithi175.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-06T03:43:35.653 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://www.redhat.com/" 2023-10-06T03:43:35.653 INFO:teuthology.orchestra.run.smithi175.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-06T03:43:35.653 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T03:43:35.653 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:43:35.654 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T03:43:35.654 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-06T03:43:35.654 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-06T03:43:35.654 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-06T03:43:35.654 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2023-10-06T03:43:35.678 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-06T03:43:35.693 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-06T03:43:35.713 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-06T03:43:35.713 DEBUG:teuthology.orchestra.run.smithi157:> test '!' -e /home/ubuntu/cephtest 2023-10-06T03:43:35.716 DEBUG:teuthology.orchestra.run.smithi175:> test '!' -e /home/ubuntu/cephtest 2023-10-06T03:43:35.731 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-06T03:43:35.740 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-06T03:43:35.740 DEBUG:teuthology.orchestra.run.smithi157:> test -z $(ls -A /var/lib/ceph) 2023-10-06T03:43:35.772 DEBUG:teuthology.orchestra.run.smithi175:> test -z $(ls -A /var/lib/ceph) 2023-10-06T03:43:35.786 INFO:teuthology.orchestra.run.smithi157.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-06T03:43:35.789 INFO:teuthology.orchestra.run.smithi175.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-06T03:43:35.790 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-06T03:43:35.844 INFO:teuthology.run_tasks:Running task kernel... 2023-10-06T03:43:35.863 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-10-06T03:43:35.864 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2023-10-06T03:43:35.864 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2023-10-06T03:43:35.864 DEBUG:teuthology.orchestra.run.smithi157:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T03:43:35.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:43:35.880 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2023-10-06T03:43:35.938 INFO:teuthology.orchestra.run.smithi157.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-06T03:43:35.938 INFO:teuthology.task.kernel:Running kernel on smithi157: 4.18.0-305.25.1.el8_4.x86_64 2023-10-06T03:43:35.938 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum install -y kernel 2023-10-06T03:43:37.381 INFO:teuthology.orchestra.run.smithi157.stdout:Updating Subscription Management repositories. 2023-10-06T03:43:37.381 INFO:teuthology.orchestra.run.smithi157.stdout:Unable to read consumer identity 2023-10-06T03:43:38.733 INFO:teuthology.orchestra.run.smithi157.stdout:Copr repo for python3-asyncssh owned by ceph 33 kB/s | 3.5 kB 00:00 2023-10-06T03:43:39.336 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 28 MB/s | 16 MB 00:00 2023-10-06T03:43:42.669 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 4.0 MB/s | 24 kB 00:00 2023-10-06T03:43:43.417 INFO:teuthology.orchestra.run.smithi157.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-10-06T03:43:43.418 INFO:teuthology.orchestra.run.smithi157.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-10-06T03:43:43.418 INFO:teuthology.orchestra.run.smithi157.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-10-06T03:43:43.574 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-10-06T03:43:43.575 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2023-10-06T03:43:43.575 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-10-06T03:43:43.690 DEBUG:teuthology.orchestra.run.smithi157:> echo no | sudo yum reinstall kernel || true 2023-10-06T03:43:44.034 INFO:teuthology.orchestra.run.smithi157.stdout:Updating Subscription Management repositories. 2023-10-06T03:43:44.034 INFO:teuthology.orchestra.run.smithi157.stdout:Unable to read consumer identity 2023-10-06T03:43:44.330 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:00:02 ago on Fri 06 Oct 2023 03:43:42 AM UTC. 2023-10-06T03:43:44.461 INFO:teuthology.orchestra.run.smithi157.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-06T03:43:44.461 INFO:teuthology.orchestra.run.smithi157.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T03:43:44.461 INFO:teuthology.orchestra.run.smithi157.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T03:43:44.487 INFO:teuthology.orchestra.run.smithi157.stderr:Error: No packages marked for reinstall. 2023-10-06T03:43:44.537 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum reinstall -y kernel || true 2023-10-06T03:43:44.879 INFO:teuthology.orchestra.run.smithi157.stdout:Updating Subscription Management repositories. 2023-10-06T03:43:44.879 INFO:teuthology.orchestra.run.smithi157.stdout:Unable to read consumer identity 2023-10-06T03:43:45.164 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:00:03 ago on Fri 06 Oct 2023 03:43:42 AM UTC. 2023-10-06T03:43:45.293 INFO:teuthology.orchestra.run.smithi157.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-06T03:43:45.293 INFO:teuthology.orchestra.run.smithi157.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T03:43:45.293 INFO:teuthology.orchestra.run.smithi157.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T03:43:45.307 INFO:teuthology.orchestra.run.smithi157.stderr:Error: No packages marked for reinstall. 2023-10-06T03:43:45.355 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T03:43:45.481 INFO:teuthology.orchestra.run.smithi157.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-06T03:43:45.481 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-06T03:43:45.481 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-10-06T03:43:45.481 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-06T03:43:45.481 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-06T03:43:45.481 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2023-10-06T03:43:45.482 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi157.front.sepia.ceph.com, path=None, version=distro) 2023-10-06T03:43:45.482 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T03:43:45.606 INFO:teuthology.orchestra.run.smithi157.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-06T03:43:45.607 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-06T03:43:45.607 DEBUG:teuthology.orchestra.run.smithi157:> sudo rpm -qi grub2-tools 2023-10-06T03:43:45.634 INFO:teuthology.orchestra.run.smithi157.stdout:Name : grub2-tools 2023-10-06T03:43:45.634 INFO:teuthology.orchestra.run.smithi157.stdout:Epoch : 1 2023-10-06T03:43:45.634 INFO:teuthology.orchestra.run.smithi157.stdout:Version : 2.02 2023-10-06T03:43:45.634 INFO:teuthology.orchestra.run.smithi157.stdout:Release : 99.el8_4.1 2023-10-06T03:43:45.634 INFO:teuthology.orchestra.run.smithi157.stdout:Architecture: x86_64 2023-10-06T03:43:45.634 INFO:teuthology.orchestra.run.smithi157.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-06T03:43:45.634 INFO:teuthology.orchestra.run.smithi157.stdout:Group : System Environment/Base 2023-10-06T03:43:45.635 INFO:teuthology.orchestra.run.smithi157.stdout:Size : 9264631 2023-10-06T03:43:45.635 INFO:teuthology.orchestra.run.smithi157.stdout:License : GPLv3+ 2023-10-06T03:43:45.635 INFO:teuthology.orchestra.run.smithi157.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-06T03:43:45.635 INFO:teuthology.orchestra.run.smithi157.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-06T03:43:45.635 INFO:teuthology.orchestra.run.smithi157.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-06T03:43:45.635 INFO:teuthology.orchestra.run.smithi157.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-06T03:43:45.635 INFO:teuthology.orchestra.run.smithi157.stdout:Relocations : (not relocatable) 2023-10-06T03:43:45.635 INFO:teuthology.orchestra.run.smithi157.stdout:Packager : Red Hat, Inc. 2023-10-06T03:43:45.635 INFO:teuthology.orchestra.run.smithi157.stdout:Vendor : Red Hat, Inc. 2023-10-06T03:43:45.635 INFO:teuthology.orchestra.run.smithi157.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T03:43:45.636 INFO:teuthology.orchestra.run.smithi157.stdout:Summary : Support tools for GRUB. 2023-10-06T03:43:45.636 INFO:teuthology.orchestra.run.smithi157.stdout:Description : 2023-10-06T03:43:45.636 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:43:45.636 INFO:teuthology.orchestra.run.smithi157.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T03:43:45.636 INFO:teuthology.orchestra.run.smithi157.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T03:43:45.636 INFO:teuthology.orchestra.run.smithi157.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T03:43:45.636 INFO:teuthology.orchestra.run.smithi157.stdout:hardware devices. 2023-10-06T03:43:45.636 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:43:45.636 INFO:teuthology.orchestra.run.smithi157.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T03:43:45.637 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T03:43:45.637 INFO:teuthology.task.kernel:Updating grub on smithi157 to boot 4.18.0-425.3.1.el8.x86_64 2023-10-06T03:43:45.638 DEBUG:teuthology.orchestra.run.smithi157:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T03:43:47.041 INFO:teuthology.orchestra.run.smithi157.stderr:Generating grub configuration file ... 2023-10-06T03:43:49.163 INFO:teuthology.orchestra.run.smithi157.stderr:done 2023-10-06T03:43:49.166 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2023-10-06T03:43:49.180 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.Noznvgilmb 2023-10-06T03:43:49.180 DEBUG:teuthology.orchestra.run.smithi157:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Noznvgilmb 2023-10-06T03:43:49.249 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.Noznvgilmb 2023-10-06T03:43:49.413 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.Noznvgilmb is 6KB 2023-10-06T03:43:49.431 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.Noznvgilmb 2023-10-06T03:43:49.445 DEBUG:teuthology.orchestra.run.smithi157:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T03:43:49.512 INFO:teuthology.orchestra.run.smithi157.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-06T03:43:49.513 INFO:teuthology.orchestra.run.smithi157.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-06T03:43:49.513 INFO:teuthology.orchestra.run.smithi157.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-06T03:43:49.513 INFO:teuthology.orchestra.run.smithi157.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-06T03:43:49.514 DEBUG:teuthology.orchestra.run.smithi157:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-10-06T03:43:49.697 DEBUG:teuthology.orchestra.run.smithi157:> sudo shutdown -r now 2023-10-06T03:44:19.701 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-06T03:44:19.701 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-10-06T03:44:19.702 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T03:44:38.270 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-10-06T03:44:47.272 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-10-06T03:44:47.272 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T03:44:50.334 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-10-06T03:45:02.356 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-10-06T03:45:02.357 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T03:45:05.434 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-10-06T03:45:20.435 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-10-06T03:45:20.436 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T03:45:38.942 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-10-06T03:45:56.944 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-10-06T03:45:56.944 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T03:45:57.955 DEBUG:teuthology.orchestra.run.smithi157:> true 2023-10-06T03:45:58.787 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-10-06T03:45:58.787 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "4.18.0-425.3.1.el8.x86_64"... 2023-10-06T03:45:58.787 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2023-10-06T03:45:58.845 INFO:teuthology.orchestra.run.smithi157.stdout:4.18.0-425.3.1.el8.x86_64 2023-10-06T03:45:58.845 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-10-06T03:45:58.845 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T03:45:58.845 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-06T03:45:59.846 DEBUG:teuthology.parallel:result is None 2023-10-06T03:45:59.847 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-06T03:45:59.860 INFO:teuthology.task.internal:Creating test directory... 2023-10-06T03:45:59.860 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T03:45:59.863 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T03:45:59.882 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-06T03:45:59.890 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-06T03:45:59.905 INFO:teuthology.task.internal:Creating archive directory... 2023-10-06T03:45:59.905 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T03:45:59.918 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T03:45:59.954 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-06T03:45:59.965 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-06T03:45:59.965 DEBUG:teuthology.orchestra.run.smithi175:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T03:46:00.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:46:00.004 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T03:46:00.007 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T03:46:00.076 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T03:46:00.082 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T03:46:00.095 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T03:46:00.096 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T03:46:00.098 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-06T03:46:00.109 INFO:teuthology.task.internal:Configuring sudo... 2023-10-06T03:46:00.109 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T03:46:00.112 DEBUG:teuthology.orchestra.run.smithi175:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T03:46:00.164 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-06T03:46:00.177 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-06T03:46:00.177 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T03:46:00.179 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T03:46:00.220 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T03:46:00.275 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T03:46:00.333 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T03:46:00.380 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T03:46:00.413 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-06T03:46:00.414 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T03:46:00.477 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T03:46:00.520 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T03:46:00.561 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T03:46:00.643 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T03:46:00.682 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-10-06T03:46:00.683 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T03:46:00.746 DEBUG:teuthology.orchestra.run.smithi157:> sudo service rsyslog restart 2023-10-06T03:46:00.748 DEBUG:teuthology.orchestra.run.smithi175:> sudo service rsyslog restart 2023-10-06T03:46:00.792 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T03:46:00.837 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T03:46:01.276 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-06T03:46:01.284 INFO:teuthology.task.internal:Starting timer... 2023-10-06T03:46:01.284 INFO:teuthology.run_tasks:Running task pcp... 2023-10-06T03:46:01.357 INFO:teuthology.run_tasks:Running task selinux... 2023-10-06T03:46:01.366 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-10-06T03:46:01.366 DEBUG:teuthology.orchestra.run.smithi157:> sudo service auditd rotate 2023-10-06T03:46:01.441 INFO:teuthology.orchestra.run.smithi157.stdout:Rotating logs: [ OK ] 2023-10-06T03:46:01.443 DEBUG:teuthology.orchestra.run.smithi175:> sudo service auditd rotate 2023-10-06T03:46:01.518 INFO:teuthology.orchestra.run.smithi175.stdout:Rotating logs: [ OK ] 2023-10-06T03:46:01.520 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-06T03:46:01.520 DEBUG:teuthology.orchestra.run.smithi157:> /usr/sbin/getenforce 2023-10-06T03:46:01.542 INFO:teuthology.orchestra.run.smithi157.stdout:Permissive 2023-10-06T03:46:01.542 DEBUG:teuthology.orchestra.run.smithi175:> /usr/sbin/getenforce 2023-10-06T03:46:01.573 INFO:teuthology.orchestra.run.smithi175.stdout:Permissive 2023-10-06T03:46:01.573 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi157.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi175.front.sepia.ceph.com': 'permissive'} 2023-10-06T03:46:01.573 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T03:46:01.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:46:01.607 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T03:46:01.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:46:01.637 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-06T03:46:01.638 DEBUG:teuthology.orchestra.run.smithi157:> sudo /usr/sbin/setenforce permissive 2023-10-06T03:46:01.672 DEBUG:teuthology.orchestra.run.smithi175:> sudo /usr/sbin/setenforce permissive 2023-10-06T03:46:01.702 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-06T03:46:01.711 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-06T03:46:01.711 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-06T03:46:01.734 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-06T03:46:01.735 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi157.front.sepia.ceph.com,smithi175.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-06T03:54:06.419 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi157.front.sepia.ceph.com'), Remote(name='ubuntu@smithi175.front.sepia.ceph.com')] 2023-10-06T03:54:06.420 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-10-06T03:54:06.421 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T03:54:06.494 DEBUG:teuthology.orchestra.run.smithi157:> true 2023-10-06T03:54:06.557 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-10-06T03:54:06.558 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2023-10-06T03:54:06.558 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T03:54:06.628 DEBUG:teuthology.orchestra.run.smithi175:> true 2023-10-06T03:54:06.651 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2023-10-06T03:54:06.651 INFO:teuthology.run_tasks:Running task clock... 2023-10-06T03:54:06.662 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-06T03:54:06.662 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T03:54:06.662 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T03:54:06.664 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T03:54:06.665 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T03:54:06.692 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T03:54:06.706 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T03:54:06.720 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T03:54:06.734 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T03:54:06.739 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: ntpd: command not found 2023-10-06T03:54:06.749 INFO:teuthology.orchestra.run.smithi157.stdout:506 Cannot talk to daemon 2023-10-06T03:54:06.762 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T03:54:06.767 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: ntpd: command not found 2023-10-06T03:54:06.776 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T03:54:06.777 INFO:teuthology.orchestra.run.smithi175.stdout:506 Cannot talk to daemon 2023-10-06T03:54:06.791 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T03:54:06.804 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T03:54:06.837 INFO:teuthology.orchestra.run.smithi157.stderr:bash: ntpq: command not found 2023-10-06T03:54:06.840 INFO:teuthology.orchestra.run.smithi157.stdout:210 Number of sources = 4 2023-10-06T03:54:06.840 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T03:54:06.840 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2023-10-06T03:54:06.840 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T03:54:06.840 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T03:54:06.840 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T03:54:06.840 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T03:54:06.881 INFO:teuthology.orchestra.run.smithi175.stderr:bash: ntpq: command not found 2023-10-06T03:54:06.884 INFO:teuthology.orchestra.run.smithi175.stdout:210 Number of sources = 4 2023-10-06T03:54:06.884 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T03:54:06.885 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2023-10-06T03:54:06.885 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T03:54:06.885 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T03:54:06.885 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T03:54:06.885 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T03:54:06.885 INFO:teuthology.run_tasks:Running task install... 2023-10-06T03:54:06.894 DEBUG:teuthology.task.install:project ceph 2023-10-06T03:54:06.895 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}} 2023-10-06T03:54:06.895 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'} 2023-10-06T03:54:06.895 INFO:teuthology.task.install:Using flavor: default 2023-10-06T03:54:06.900 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-06T03:54:06.900 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-06T03:54:06.901 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'tag': None, 'wait_for_package': False} 2023-10-06T03:54:06.901 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T03:54:06.903 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'tag': None, 'wait_for_package': False} 2023-10-06T03:54:06.903 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T03:54:07.143 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/ 2023-10-06T03:54:07.143 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1274.g68e53a4a 2023-10-06T03:54:07.149 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/ 2023-10-06T03:54:07.149 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1274.g68e53a4a 2023-10-06T03:54:07.277 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T03:54:07.277 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-06T03:54:07.278 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T03:54:07.304 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-06T03:54:07.305 DEBUG:teuthology.orchestra.run.smithi157:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T03:54:07.343 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T03:54:07.343 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-10-06T03:54:07.343 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T03:54:07.368 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-06T03:54:07.368 DEBUG:teuthology.orchestra.run.smithi175:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T03:54:07.370 DEBUG:teuthology.orchestra.run.smithi157:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T03:54:07.432 DEBUG:teuthology.orchestra.run.smithi175:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T03:54:07.446 DEBUG:teuthology.orchestra.run.smithi157:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T03:54:07.504 DEBUG:teuthology.orchestra.run.smithi175:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T03:54:07.509 INFO:teuthology.orchestra.run.smithi157.stdout:check_obsoletes = 1 2023-10-06T03:54:07.511 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum clean all 2023-10-06T03:54:07.568 INFO:teuthology.orchestra.run.smithi175.stdout:check_obsoletes = 1 2023-10-06T03:54:07.569 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum clean all 2023-10-06T03:54:07.903 INFO:teuthology.orchestra.run.smithi157.stdout:Updating Subscription Management repositories. 2023-10-06T03:54:07.951 INFO:teuthology.orchestra.run.smithi175.stdout:Updating Subscription Management repositories. 2023-10-06T03:54:08.226 INFO:teuthology.orchestra.run.smithi175.stdout:44 files removed 2023-10-06T03:54:08.236 INFO:teuthology.orchestra.run.smithi157.stdout:44 files removed 2023-10-06T03:54:08.277 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-06T03:54:08.288 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-06T03:54:08.621 INFO:teuthology.orchestra.run.smithi175.stdout:Updating Subscription Management repositories. 2023-10-06T03:54:08.641 INFO:teuthology.orchestra.run.smithi157.stdout:Updating Subscription Management repositories. 2023-10-06T03:54:09.482 INFO:teuthology.orchestra.run.smithi157.stdout:ceph packages for x86_64 194 kB/s | 78 kB 00:00 2023-10-06T03:54:09.566 INFO:teuthology.orchestra.run.smithi175.stdout:ceph packages for x86_64 151 kB/s | 78 kB 00:00 2023-10-06T03:54:09.877 INFO:teuthology.orchestra.run.smithi157.stdout:ceph noarch packages 26 kB/s | 9.2 kB 00:00 2023-10-06T03:54:09.968 INFO:teuthology.orchestra.run.smithi175.stdout:ceph noarch packages 25 kB/s | 9.2 kB 00:00 2023-10-06T03:54:10.253 INFO:teuthology.orchestra.run.smithi157.stdout:ceph source packages 4.2 kB/s | 1.4 kB 00:00 2023-10-06T03:54:10.306 INFO:teuthology.orchestra.run.smithi175.stdout:ceph source packages 4.7 kB/s | 1.4 kB 00:00 2023-10-06T03:54:12.254 INFO:teuthology.orchestra.run.smithi157.stdout:Copr repo for python3-asyncssh owned by ceph 1.8 kB/s | 3.5 kB 00:01 2023-10-06T03:54:12.256 INFO:teuthology.orchestra.run.smithi175.stdout:Copr repo for python3-asyncssh owned by ceph 1.8 kB/s | 3.5 kB 00:01 2023-10-06T03:54:13.708 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 11 MB/s | 16 MB 00:01 2023-10-06T03:54:14.083 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 8.8 MB/s | 16 MB 00:01 2023-10-06T03:54:16.989 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 4.3 MB/s | 24 kB 00:00 2023-10-06T03:54:17.392 INFO:teuthology.orchestra.run.smithi157.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 119 MB/s | 38 MB 00:00 2023-10-06T03:54:17.393 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 4.5 MB/s | 24 kB 00:00 2023-10-06T03:54:17.750 INFO:teuthology.orchestra.run.smithi175.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 120 MB/s | 33 MB 00:00 2023-10-06T03:54:22.886 INFO:teuthology.orchestra.run.smithi157.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 117 MB/s | 33 MB 00:00 2023-10-06T03:54:22.937 INFO:teuthology.orchestra.run.smithi175.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 128 MB/s | 38 MB 00:00 2023-10-06T03:54:30.767 INFO:teuthology.orchestra.run.smithi157.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T03:54:30.768 INFO:teuthology.orchestra.run.smithi157.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T03:54:30.778 INFO:teuthology.orchestra.run.smithi157.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-06T03:54:30.779 INFO:teuthology.orchestra.run.smithi157.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-06T03:54:30.780 INFO:teuthology.orchestra.run.smithi157.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-06T03:54:30.781 INFO:teuthology.orchestra.run.smithi157.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T03:54:30.782 INFO:teuthology.orchestra.run.smithi157.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T03:54:30.785 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-06T03:54:30.786 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-06T03:54:31.272 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-10-06T03:54:31.284 INFO:teuthology.orchestra.run.smithi157.stdout:====================================================================================================================== 2023-10-06T03:54:31.284 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2023-10-06T03:54:31.284 INFO:teuthology.orchestra.run.smithi157.stdout:====================================================================================================================== 2023-10-06T03:54:31.284 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2023-10-06T03:54:31.284 INFO:teuthology.orchestra.run.smithi157.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-06T03:54:31.285 INFO:teuthology.orchestra.run.smithi157.stdout: ceph x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 6.4 k 2023-10-06T03:54:31.285 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 5.2 M 2023-10-06T03:54:31.285 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 877 k 2023-10-06T03:54:31.285 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 162 k 2023-10-06T03:54:31.285 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 1.6 M 2023-10-06T03:54:31.285 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 121 k 2023-10-06T03:54:31.285 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 1.7 M 2023-10-06T03:54:31.285 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 7.4 M 2023-10-06T03:54:31.285 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 47 k 2023-10-06T03:54:31.285 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 17 M 2023-10-06T03:54:31.285 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 52 M 2023-10-06T03:54:31.286 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 267 k 2023-10-06T03:54:31.286 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 80 k 2023-10-06T03:54:31.286 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 255 k 2023-10-06T03:54:31.286 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 18 k 2023-10-06T03:54:31.286 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-06T03:54:31.286 INFO:teuthology.orchestra.run.smithi157.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-06T03:54:31.286 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-06T03:54:31.286 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 25 k 2023-10-06T03:54:31.286 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 797 k 2023-10-06T03:54:31.286 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 130 k 2023-10-06T03:54:31.286 INFO:teuthology.orchestra.run.smithi157.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-06T03:54:31.287 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-06T03:54:31.287 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-06T03:54:31.287 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 190 k 2023-10-06T03:54:31.287 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 371 k 2023-10-06T03:54:31.287 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 358 k 2023-10-06T03:54:31.287 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 107 k 2023-10-06T03:54:31.287 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 91 k 2023-10-06T03:54:31.287 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.5 M 2023-10-06T03:54:31.287 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 173 k 2023-10-06T03:54:31.287 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-06T03:54:31.287 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrading: 2023-10-06T03:54:31.288 INFO:teuthology.orchestra.run.smithi157.stdout: librados2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.4 M 2023-10-06T03:54:31.288 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.5 M 2023-10-06T03:54:31.288 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2023-10-06T03:54:31.288 INFO:teuthology.orchestra.run.smithi157.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-06T03:54:31.288 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 23 M 2023-10-06T03:54:31.288 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 23 k 2023-10-06T03:54:31.288 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 2.3 M 2023-10-06T03:54:31.288 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 238 k 2023-10-06T03:54:31.289 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.9 M 2023-10-06T03:54:31.289 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 17 M 2023-10-06T03:54:31.289 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 14 k 2023-10-06T03:54:31.289 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 24 k 2023-10-06T03:54:31.289 INFO:teuthology.orchestra.run.smithi157.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-06T03:54:31.289 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T03:54:31.289 INFO:teuthology.orchestra.run.smithi157.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-06T03:54:31.289 INFO:teuthology.orchestra.run.smithi157.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-06T03:54:31.289 INFO:teuthology.orchestra.run.smithi157.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-06T03:54:31.289 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 175 k 2023-10-06T03:54:31.289 INFO:teuthology.orchestra.run.smithi157.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-06T03:54:31.290 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-06T03:54:31.290 INFO:teuthology.orchestra.run.smithi157.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T03:54:31.290 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-06T03:54:31.290 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-06T03:54:31.290 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 506 k 2023-10-06T03:54:31.290 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-06T03:54:31.290 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 5.6 M 2023-10-06T03:54:31.290 INFO:teuthology.orchestra.run.smithi157.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-06T03:54:31.290 INFO:teuthology.orchestra.run.smithi157.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-06T03:54:31.290 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T03:54:31.290 INFO:teuthology.orchestra.run.smithi157.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-06T03:54:31.291 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-06T03:54:31.291 INFO:teuthology.orchestra.run.smithi157.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-06T03:54:31.291 INFO:teuthology.orchestra.run.smithi157.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-06T03:54:31.291 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-06T03:54:31.291 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-06T03:54:31.291 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-06T03:54:31.291 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-06T03:54:31.291 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T03:54:31.291 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-06T03:54:31.291 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T03:54:31.292 INFO:teuthology.orchestra.run.smithi157.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T03:54:31.292 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T03:54:31.292 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 45 k 2023-10-06T03:54:31.292 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 115 k 2023-10-06T03:54:31.292 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T03:54:31.292 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T03:54:31.292 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T03:54:31.292 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T03:54:31.292 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T03:54:31.292 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T03:54:31.292 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-06T03:54:31.293 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-06T03:54:31.293 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-06T03:54:31.293 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T03:54:31.293 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T03:54:31.293 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T03:54:31.293 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-06T03:54:31.293 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-06T03:54:31.293 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T03:54:31.294 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T03:54:31.294 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-06T03:54:31.294 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-06T03:54:31.294 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-06T03:54:31.294 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T03:54:31.294 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T03:54:31.294 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-06T03:54:31.294 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-06T03:54:31.294 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-06T03:54:31.294 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-06T03:54:31.294 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T03:54:31.295 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T03:54:31.295 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T03:54:31.295 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T03:54:31.295 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-06T03:54:31.295 INFO:teuthology.orchestra.run.smithi157.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T03:54:31.295 INFO:teuthology.orchestra.run.smithi157.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T03:54:31.295 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T03:54:31.295 INFO:teuthology.orchestra.run.smithi157.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T03:54:31.295 INFO:teuthology.orchestra.run.smithi157.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T03:54:31.295 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T03:54:31.295 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T03:54:31.296 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T03:54:31.296 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-06T03:54:31.296 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T03:54:31.296 INFO:teuthology.orchestra.run.smithi157.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-06T03:54:31.296 INFO:teuthology.orchestra.run.smithi157.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T03:54:31.296 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-06T03:54:31.296 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-06T03:54:31.296 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:54:31.296 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2023-10-06T03:54:31.296 INFO:teuthology.orchestra.run.smithi157.stdout:====================================================================================================================== 2023-10-06T03:54:31.296 INFO:teuthology.orchestra.run.smithi157.stdout:Install 118 Packages 2023-10-06T03:54:31.297 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrade 2 Packages 2023-10-06T03:54:31.297 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:54:31.297 INFO:teuthology.orchestra.run.smithi175.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T03:54:31.298 INFO:teuthology.orchestra.run.smithi175.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T03:54:31.298 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 201 M 2023-10-06T03:54:31.298 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2023-10-06T03:54:31.299 INFO:teuthology.orchestra.run.smithi175.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-06T03:54:31.299 INFO:teuthology.orchestra.run.smithi175.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-06T03:54:31.300 INFO:teuthology.orchestra.run.smithi175.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-06T03:54:31.301 INFO:teuthology.orchestra.run.smithi175.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T03:54:31.303 INFO:teuthology.orchestra.run.smithi175.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T03:54:31.305 INFO:teuthology.orchestra.run.smithi175.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-06T03:54:31.306 INFO:teuthology.orchestra.run.smithi175.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-06T03:54:31.797 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-10-06T03:54:31.800 INFO:teuthology.orchestra.run.smithi157.stdout:(1/120): ceph-17.2.6-1274.g68e53a4a.el8.x86_64. 42 kB/s | 6.4 kB 00:00 2023-10-06T03:54:31.808 INFO:teuthology.orchestra.run.smithi175.stdout:====================================================================================================================== 2023-10-06T03:54:31.808 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2023-10-06T03:54:31.808 INFO:teuthology.orchestra.run.smithi175.stdout:====================================================================================================================== 2023-10-06T03:54:31.808 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-10-06T03:54:31.808 INFO:teuthology.orchestra.run.smithi175.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-06T03:54:31.808 INFO:teuthology.orchestra.run.smithi175.stdout: ceph x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 6.4 k 2023-10-06T03:54:31.808 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 5.2 M 2023-10-06T03:54:31.809 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 877 k 2023-10-06T03:54:31.809 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 162 k 2023-10-06T03:54:31.809 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 1.6 M 2023-10-06T03:54:31.809 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 121 k 2023-10-06T03:54:31.809 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 1.7 M 2023-10-06T03:54:31.809 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 7.4 M 2023-10-06T03:54:31.809 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 47 k 2023-10-06T03:54:31.809 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 17 M 2023-10-06T03:54:31.809 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 52 M 2023-10-06T03:54:31.809 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 267 k 2023-10-06T03:54:31.809 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 80 k 2023-10-06T03:54:31.810 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-mirror x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 255 k 2023-10-06T03:54:31.810 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-top noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 18 k 2023-10-06T03:54:31.810 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-06T03:54:31.810 INFO:teuthology.orchestra.run.smithi175.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-06T03:54:31.810 INFO:teuthology.orchestra.run.smithi175.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-06T03:54:31.810 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 25 k 2023-10-06T03:54:31.810 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 797 k 2023-10-06T03:54:31.810 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 130 k 2023-10-06T03:54:31.810 INFO:teuthology.orchestra.run.smithi175.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-06T03:54:31.810 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-06T03:54:31.811 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-06T03:54:31.811 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 190 k 2023-10-06T03:54:31.811 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 371 k 2023-10-06T03:54:31.811 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 358 k 2023-10-06T03:54:31.811 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 107 k 2023-10-06T03:54:31.811 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 91 k 2023-10-06T03:54:31.811 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.5 M 2023-10-06T03:54:31.811 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 173 k 2023-10-06T03:54:31.811 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-06T03:54:31.811 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrading: 2023-10-06T03:54:31.812 INFO:teuthology.orchestra.run.smithi175.stdout: librados2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.4 M 2023-10-06T03:54:31.812 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.5 M 2023-10-06T03:54:31.812 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2023-10-06T03:54:31.812 INFO:teuthology.orchestra.run.smithi175.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-06T03:54:31.812 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 23 M 2023-10-06T03:54:31.812 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 23 k 2023-10-06T03:54:31.812 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 2.3 M 2023-10-06T03:54:31.812 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 238 k 2023-10-06T03:54:31.812 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.9 M 2023-10-06T03:54:31.812 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 17 M 2023-10-06T03:54:31.812 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 14 k 2023-10-06T03:54:31.813 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 24 k 2023-10-06T03:54:31.813 INFO:teuthology.orchestra.run.smithi175.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-06T03:54:31.813 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T03:54:31.813 INFO:teuthology.orchestra.run.smithi175.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-06T03:54:31.813 INFO:teuthology.orchestra.run.smithi175.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-06T03:54:31.813 INFO:teuthology.orchestra.run.smithi175.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-06T03:54:31.813 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 175 k 2023-10-06T03:54:31.813 INFO:teuthology.orchestra.run.smithi175.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-06T03:54:31.813 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-06T03:54:31.813 INFO:teuthology.orchestra.run.smithi175.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T03:54:31.814 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-06T03:54:31.814 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-06T03:54:31.814 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 506 k 2023-10-06T03:54:31.814 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-06T03:54:31.814 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 5.6 M 2023-10-06T03:54:31.814 INFO:teuthology.orchestra.run.smithi175.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-06T03:54:31.814 INFO:teuthology.orchestra.run.smithi175.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-06T03:54:31.814 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T03:54:31.814 INFO:teuthology.orchestra.run.smithi175.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-06T03:54:31.814 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-06T03:54:31.815 INFO:teuthology.orchestra.run.smithi175.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-06T03:54:31.815 INFO:teuthology.orchestra.run.smithi175.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-06T03:54:31.815 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-06T03:54:31.815 INFO:teuthology.orchestra.run.smithi175.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-06T03:54:31.815 INFO:teuthology.orchestra.run.smithi175.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-06T03:54:31.815 INFO:teuthology.orchestra.run.smithi175.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-06T03:54:31.815 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T03:54:31.815 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-06T03:54:31.815 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T03:54:31.815 INFO:teuthology.orchestra.run.smithi175.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T03:54:31.816 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T03:54:31.816 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 45 k 2023-10-06T03:54:31.816 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 115 k 2023-10-06T03:54:31.816 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T03:54:31.816 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T03:54:31.816 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T03:54:31.816 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T03:54:31.816 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T03:54:31.816 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T03:54:31.816 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-06T03:54:31.816 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-06T03:54:31.817 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-06T03:54:31.817 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T03:54:31.817 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T03:54:31.817 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T03:54:31.817 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-06T03:54:31.817 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-06T03:54:31.817 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T03:54:31.817 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T03:54:31.817 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-06T03:54:31.817 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-06T03:54:31.818 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-06T03:54:31.818 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T03:54:31.818 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T03:54:31.818 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-06T03:54:31.818 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-06T03:54:31.818 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-06T03:54:31.818 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-06T03:54:31.818 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T03:54:31.818 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T03:54:31.818 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T03:54:31.818 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T03:54:31.819 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-06T03:54:31.819 INFO:teuthology.orchestra.run.smithi175.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T03:54:31.819 INFO:teuthology.orchestra.run.smithi175.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T03:54:31.819 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T03:54:31.819 INFO:teuthology.orchestra.run.smithi175.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T03:54:31.819 INFO:teuthology.orchestra.run.smithi175.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T03:54:31.819 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T03:54:31.819 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T03:54:31.819 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T03:54:31.819 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-06T03:54:31.820 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T03:54:31.820 INFO:teuthology.orchestra.run.smithi175.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-06T03:54:31.820 INFO:teuthology.orchestra.run.smithi175.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T03:54:31.820 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-06T03:54:31.820 INFO:teuthology.orchestra.run.smithi175.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-06T03:54:31.820 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:54:31.820 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-10-06T03:54:31.820 INFO:teuthology.orchestra.run.smithi175.stdout:====================================================================================================================== 2023-10-06T03:54:31.820 INFO:teuthology.orchestra.run.smithi175.stdout:Install 118 Packages 2023-10-06T03:54:31.820 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrade 2 Packages 2023-10-06T03:54:31.820 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:54:31.821 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 201 M 2023-10-06T03:54:31.821 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-10-06T03:54:32.074 INFO:teuthology.orchestra.run.smithi157.stdout:(2/120): ceph-fuse-17.2.6-1274.g68e53a4a.el8.x8 3.1 MB/s | 877 kB 00:00 2023-10-06T03:54:32.134 INFO:teuthology.orchestra.run.smithi157.stdout:(3/120): ceph-immutable-object-cache-17.2.6-127 2.6 MB/s | 162 kB 00:00 2023-10-06T03:54:32.188 INFO:teuthology.orchestra.run.smithi175.stdout:(1/120): ceph-17.2.6-1274.g68e53a4a.el8.x86_64. 28 kB/s | 6.4 kB 00:00 2023-10-06T03:54:32.461 INFO:teuthology.orchestra.run.smithi157.stdout:(4/120): ceph-mds-17.2.6-1274.g68e53a4a.el8.x86 7.0 MB/s | 2.3 MB 00:00 2023-10-06T03:54:32.475 INFO:teuthology.orchestra.run.smithi157.stdout:(5/120): ceph-base-17.2.6-1274.g68e53a4a.el8.x8 6.2 MB/s | 5.2 MB 00:00 2023-10-06T03:54:32.652 INFO:teuthology.orchestra.run.smithi157.stdout:(6/120): ceph-mgr-17.2.6-1274.g68e53a4a.el8.x86 8.2 MB/s | 1.6 MB 00:00 2023-10-06T03:54:32.988 INFO:teuthology.orchestra.run.smithi157.stdout:(7/120): ceph-mon-17.2.6-1274.g68e53a4a.el8.x86 7.7 MB/s | 3.9 MB 00:00 2023-10-06T03:54:33.075 INFO:teuthology.orchestra.run.smithi175.stdout:(2/120): ceph-fuse-17.2.6-1274.g68e53a4a.el8.x8 989 kB/s | 877 kB 00:00 2023-10-06T03:54:33.241 INFO:teuthology.orchestra.run.smithi175.stdout:(3/120): ceph-immutable-object-cache-17.2.6-127 979 kB/s | 162 kB 00:00 2023-10-06T03:54:34.519 INFO:teuthology.orchestra.run.smithi175.stdout:(4/120): ceph-mds-17.2.6-1274.g68e53a4a.el8.x86 1.8 MB/s | 2.3 MB 00:01 2023-10-06T03:54:34.533 INFO:teuthology.orchestra.run.smithi157.stdout:(8/120): ceph-common-17.2.6-1274.g68e53a4a.el8. 8.1 MB/s | 23 MB 00:02 2023-10-06T03:54:34.593 INFO:teuthology.orchestra.run.smithi157.stdout:(9/120): ceph-selinux-17.2.6-1274.g68e53a4a.el8 408 kB/s | 24 kB 00:00 2023-10-06T03:54:34.723 INFO:teuthology.orchestra.run.smithi157.stdout:(10/120): ceph-osd-17.2.6-1274.g68e53a4a.el8.x8 8.1 MB/s | 17 MB 00:02 2023-10-06T03:54:34.798 INFO:teuthology.orchestra.run.smithi157.stdout:(11/120): cephfs-mirror-17.2.6-1274.g68e53a4a.e 3.4 MB/s | 255 kB 00:00 2023-10-06T03:54:34.865 INFO:teuthology.orchestra.run.smithi157.stdout:(12/120): libcephfs-devel-17.2.6-1274.g68e53a4a 369 kB/s | 25 kB 00:00 2023-10-06T03:54:34.874 INFO:teuthology.orchestra.run.smithi175.stdout:(5/120): ceph-mgr-17.2.6-1274.g68e53a4a.el8.x86 4.4 MB/s | 1.6 MB 00:00 2023-10-06T03:54:35.006 INFO:teuthology.orchestra.run.smithi157.stdout:(13/120): ceph-radosgw-17.2.6-1274.g68e53a4a.el 8.3 MB/s | 17 MB 00:02 2023-10-06T03:54:35.012 INFO:teuthology.orchestra.run.smithi157.stdout:(14/120): libcephfs2-17.2.6-1274.g68e53a4a.el8. 5.3 MB/s | 797 kB 00:00 2023-10-06T03:54:35.058 INFO:teuthology.orchestra.run.smithi157.stdout:(15/120): libcephsqlite-17.2.6-1274.g68e53a4a.e 3.3 MB/s | 175 kB 00:00 2023-10-06T03:54:35.068 INFO:teuthology.orchestra.run.smithi157.stdout:(16/120): librados-devel-17.2.6-1274.g68e53a4a. 2.3 MB/s | 130 kB 00:00 2023-10-06T03:54:35.118 INFO:teuthology.orchestra.run.smithi175.stdout:(6/120): ceph-base-17.2.6-1274.g68e53a4a.el8.x8 1.6 MB/s | 5.2 MB 00:03 2023-10-06T03:54:35.123 INFO:teuthology.orchestra.run.smithi157.stdout:(17/120): libradosstriper1-17.2.6-1274.g68e53a4 7.6 MB/s | 506 kB 00:00 2023-10-06T03:54:35.188 INFO:teuthology.orchestra.run.smithi157.stdout:(18/120): python3-ceph-argparse-17.2.6-1274.g68 693 kB/s | 45 kB 00:00 2023-10-06T03:54:35.255 INFO:teuthology.orchestra.run.smithi157.stdout:(19/120): python3-ceph-common-17.2.6-1274.g68e5 1.7 MB/s | 115 kB 00:00 2023-10-06T03:54:35.327 INFO:teuthology.orchestra.run.smithi157.stdout:(20/120): python3-cephfs-17.2.6-1274.g68e53a4a. 2.6 MB/s | 190 kB 00:00 2023-10-06T03:54:35.408 INFO:teuthology.orchestra.run.smithi157.stdout:(21/120): python3-rados-17.2.6-1274.g68e53a4a.e 4.5 MB/s | 371 kB 00:00 2023-10-06T03:54:35.488 INFO:teuthology.orchestra.run.smithi157.stdout:(22/120): python3-rbd-17.2.6-1274.g68e53a4a.el8 4.4 MB/s | 358 kB 00:00 2023-10-06T03:54:35.557 INFO:teuthology.orchestra.run.smithi175.stdout:(7/120): ceph-mon-17.2.6-1274.g68e53a4a.el8.x86 5.8 MB/s | 3.9 MB 00:00 2023-10-06T03:54:35.557 INFO:teuthology.orchestra.run.smithi157.stdout:(23/120): python3-rgw-17.2.6-1274.g68e53a4a.el8 1.5 MB/s | 107 kB 00:00 2023-10-06T03:54:35.631 INFO:teuthology.orchestra.run.smithi157.stdout:(24/120): rbd-fuse-17.2.6-1274.g68e53a4a.el8.x8 1.2 MB/s | 91 kB 00:00 2023-10-06T03:54:35.850 INFO:teuthology.orchestra.run.smithi157.stdout:(25/120): librgw2-17.2.6-1274.g68e53a4a.el8.x86 7.1 MB/s | 5.6 MB 00:00 2023-10-06T03:54:35.923 INFO:teuthology.orchestra.run.smithi157.stdout:(26/120): rbd-nbd-17.2.6-1274.g68e53a4a.el8.x86 2.3 MB/s | 173 kB 00:00 2023-10-06T03:54:35.990 INFO:teuthology.orchestra.run.smithi157.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1274.g 345 kB/s | 23 kB 00:00 2023-10-06T03:54:36.018 INFO:teuthology.orchestra.run.smithi157.stdout:(28/120): rbd-mirror-17.2.6-1274.g68e53a4a.el8. 8.9 MB/s | 3.5 MB 00:00 2023-10-06T03:54:36.045 INFO:teuthology.orchestra.run.smithi157.stdout:(29/120): ceph-mgr-cephadm-17.2.6-1274.g68e53a4 2.2 MB/s | 121 kB 00:00 2023-10-06T03:54:36.231 INFO:teuthology.orchestra.run.smithi157.stdout:(30/120): ceph-mgr-dashboard-17.2.6-1274.g68e53 8.0 MB/s | 1.7 MB 00:00 2023-10-06T03:54:36.295 INFO:teuthology.orchestra.run.smithi157.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1274.g68 3.7 MB/s | 238 kB 00:00 2023-10-06T03:54:36.352 INFO:teuthology.orchestra.run.smithi157.stdout:(32/120): ceph-mgr-rook-17.2.6-1274.g68e53a4a.e 840 kB/s | 47 kB 00:00 2023-10-06T03:54:36.413 INFO:teuthology.orchestra.run.smithi157.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1274.g6 239 kB/s | 14 kB 00:00 2023-10-06T03:54:36.474 INFO:teuthology.orchestra.run.smithi157.stdout:(34/120): ceph-volume-17.2.6-1274.g68e53a4a.el8 4.2 MB/s | 267 kB 00:00 2023-10-06T03:54:36.535 INFO:teuthology.orchestra.run.smithi157.stdout:(35/120): cephadm-17.2.6-1274.g68e53a4a.el8.noa 1.3 MB/s | 80 kB 00:00 2023-10-06T03:54:36.593 INFO:teuthology.orchestra.run.smithi157.stdout:(36/120): cephfs-top-17.2.6-1274.g68e53a4a.el8. 309 kB/s | 18 kB 00:00 2023-10-06T03:54:36.724 INFO:teuthology.orchestra.run.smithi157.stdout:(37/120): gperftools-libs-2.7-9.el8.x86_64.rpm 2.3 MB/s | 306 kB 00:00 2023-10-06T03:54:36.745 INFO:teuthology.orchestra.run.smithi157.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 2.7 MB/s | 59 kB 00:00 2023-10-06T03:54:36.768 INFO:teuthology.orchestra.run.smithi157.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 3.3 MB/s | 75 kB 00:00 2023-10-06T03:54:36.815 INFO:teuthology.orchestra.run.smithi157.stdout:(40/120): python3-asyncssh-2.7.0-2.el8.noarch.r 9.9 MB/s | 476 kB 00:00 2023-10-06T03:54:36.835 INFO:teuthology.orchestra.run.smithi157.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 2.2 MB/s | 44 kB 00:00 2023-10-06T03:54:36.858 INFO:teuthology.orchestra.run.smithi157.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 7.9 MB/s | 185 kB 00:00 2023-10-06T03:54:36.877 INFO:teuthology.orchestra.run.smithi157.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 1.7 MB/s | 33 kB 00:00 2023-10-06T03:54:36.897 INFO:teuthology.orchestra.run.smithi157.stdout:(44/120): python3-certifi-2018.10.15-7.el8.noar 849 kB/s | 16 kB 00:00 2023-10-06T03:54:36.935 INFO:teuthology.orchestra.run.smithi157.stdout:(45/120): python3-cheroot-8.5.2-1.el8.noarch.rp 4.4 MB/s | 173 kB 00:00 2023-10-06T03:54:36.963 INFO:teuthology.orchestra.run.smithi157.stdout:(46/120): python3-cherrypy-18.4.0-1.el8.noarch. 14 MB/s | 384 kB 00:00 2023-10-06T03:54:36.983 INFO:teuthology.orchestra.run.smithi157.stdout:(47/120): python3-google-auth-1.1.1-10.el8.noar 4.0 MB/s | 81 kB 00:00 2023-10-06T03:54:37.003 INFO:teuthology.orchestra.run.smithi157.stdout:(48/120): python3-jaraco-6.2-6.el8.noarch.rpm 580 kB/s | 11 kB 00:00 2023-10-06T03:54:37.022 INFO:teuthology.orchestra.run.smithi157.stdout:(49/120): python3-jaraco-functools-2.0-4.el8.no 907 kB/s | 18 kB 00:00 2023-10-06T03:54:37.075 INFO:teuthology.orchestra.run.smithi157.stdout:(50/120): python3-kubernetes-11.0.0-6.el8.noarc 22 MB/s | 1.2 MB 00:00 2023-10-06T03:54:37.106 INFO:teuthology.orchestra.run.smithi157.stdout:(51/120): ceph-mgr-diskprediction-local-17.2.6- 7.0 MB/s | 7.4 MB 00:01 2023-10-06T03:54:37.107 INFO:teuthology.orchestra.run.smithi157.stdout:(52/120): python3-logutils-0.3.5-11.el8.noarch. 1.5 MB/s | 49 kB 00:00 2023-10-06T03:54:37.127 INFO:teuthology.orchestra.run.smithi157.stdout:(53/120): python3-natsort-7.1.1-2.el8.noarch.rp 3.0 MB/s | 60 kB 00:00 2023-10-06T03:54:37.148 INFO:teuthology.orchestra.run.smithi157.stdout:(54/120): python3-pecan-1.3.2-9.el8.noarch.rpm 13 MB/s | 283 kB 00:00 2023-10-06T03:54:37.168 INFO:teuthology.orchestra.run.smithi157.stdout:(55/120): python3-portend-2.6-1.el8.noarch.rpm 816 kB/s | 16 kB 00:00 2023-10-06T03:54:37.188 INFO:teuthology.orchestra.run.smithi157.stdout:(56/120): python3-repoze-lru-0.7-6.el8.noarch.r 1.7 MB/s | 34 kB 00:00 2023-10-06T03:54:37.189 INFO:teuthology.orchestra.run.smithi157.stdout:(57/120): python3-more-itertools-7.2.0-3.el8.no 713 kB/s | 59 kB 00:00 2023-10-06T03:54:37.209 INFO:teuthology.orchestra.run.smithi157.stdout:(58/120): python3-routes-2.4.1-12.el8.noarch.rp 9.3 MB/s | 196 kB 00:00 2023-10-06T03:54:37.222 INFO:teuthology.orchestra.run.smithi157.stdout:(59/120): python3-rsa-4.9-2.el8.noarch.rpm 1.9 MB/s | 64 kB 00:00 2023-10-06T03:54:37.230 INFO:teuthology.orchestra.run.smithi157.stdout:(60/120): python3-simplegeneric-0.8.1-17.el8.no 974 kB/s | 19 kB 00:00 2023-10-06T03:54:37.245 INFO:teuthology.orchestra.run.smithi157.stdout:(61/120): python3-singledispatch-3.4.0.3-18.el8 1.0 MB/s | 24 kB 00:00 2023-10-06T03:54:37.247 INFO:teuthology.orchestra.run.smithi157.stdout:(62/120): python3-tempora-1.14.1-5.el8.noarch.r 1.5 MB/s | 29 kB 00:00 2023-10-06T03:54:37.269 INFO:teuthology.orchestra.run.smithi157.stdout:(63/120): python3-waitress-1.4.3-1.el8.noarch.r 12 MB/s | 255 kB 00:00 2023-10-06T03:54:37.270 INFO:teuthology.orchestra.run.smithi157.stdout:(64/120): python3-trustme-0.6.0-4.el8.noarch.rp 1.1 MB/s | 27 kB 00:00 2023-10-06T03:54:37.290 INFO:teuthology.orchestra.run.smithi157.stdout:(65/120): python3-webob-1.8.5-1.el8.1.noarch.rp 12 MB/s | 251 kB 00:00 2023-10-06T03:54:37.300 INFO:teuthology.orchestra.run.smithi157.stdout:(66/120): python3-websocket-client-0.56.0-5.el8 2.1 MB/s | 61 kB 00:00 2023-10-06T03:54:37.310 INFO:teuthology.orchestra.run.smithi157.stdout:(67/120): python3-webtest-2.0.33-1.el8.noarch.r 4.2 MB/s | 85 kB 00:00 2023-10-06T03:54:37.329 INFO:teuthology.orchestra.run.smithi157.stdout:(68/120): python3-zc-lockfile-2.0-2.el8.noarch. 796 kB/s | 23 kB 00:00 2023-10-06T03:54:37.333 INFO:teuthology.orchestra.run.smithi157.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 18 MB/s | 70 kB 00:00 2023-10-06T03:54:37.357 INFO:teuthology.orchestra.run.smithi157.stdout:(70/120): thrift-0.13.0-2.el8.x86_64.rpm 37 MB/s | 1.7 MB 00:00 2023-10-06T03:54:37.363 INFO:teuthology.orchestra.run.smithi157.stdout:(71/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 618 kB/s | 19 kB 00:00 2023-10-06T03:54:37.380 INFO:teuthology.orchestra.run.smithi157.stdout:(72/120): python3-oauthlib-2.1.0-1.el8.noarch.r 9.2 MB/s | 155 kB 00:00 2023-10-06T03:54:37.391 INFO:teuthology.orchestra.run.smithi157.stdout:(73/120): python3-jwt-1.6.1-2.el8.noarch.rpm 1.3 MB/s | 43 kB 00:00 2023-10-06T03:54:37.404 INFO:teuthology.orchestra.run.smithi157.stdout:(74/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 9.2 MB/s | 220 kB 00:00 2023-10-06T03:54:37.412 INFO:teuthology.orchestra.run.smithi157.stdout:(75/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 2.2 MB/s | 48 kB 00:00 2023-10-06T03:54:37.421 INFO:teuthology.orchestra.run.smithi157.stdout:(76/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 13 MB/s | 229 kB 00:00 2023-10-06T03:54:37.427 INFO:teuthology.orchestra.run.smithi157.stdout:(77/120): python3-requests-oauthlib-1.0.0-1.el8 2.8 MB/s | 43 kB 00:00 2023-10-06T03:54:37.441 INFO:teuthology.orchestra.run.smithi157.stdout:(78/120): libverto-devel-0.3.0-5.el8.x86_64.rpm 1.2 MB/s | 18 kB 00:00 2023-10-06T03:54:37.462 INFO:teuthology.orchestra.run.smithi157.stdout:(79/120): openssl-devel-1.1.1g-15.el8_3.x86_64. 57 MB/s | 2.3 MB 00:00 2023-10-06T03:54:37.467 INFO:teuthology.orchestra.run.smithi157.stdout:(80/120): pcre2-devel-10.32-2.el8.x86_64.rpm 23 MB/s | 605 kB 00:00 2023-10-06T03:54:37.476 INFO:teuthology.orchestra.run.smithi157.stdout:(81/120): fuse-2.9.7-12.el8.x86_64.rpm 6.2 MB/s | 83 kB 00:00 2023-10-06T03:54:37.481 INFO:teuthology.orchestra.run.smithi157.stdout:(82/120): elfutils-libelf-devel-0.182-3.el8.x86 4.2 MB/s | 59 kB 00:00 2023-10-06T03:54:37.490 INFO:teuthology.orchestra.run.smithi157.stdout:(83/120): libquadmath-8.4.1-1.el8.x86_64.rpm 11 MB/s | 169 kB 00:00 2023-10-06T03:54:37.495 INFO:teuthology.orchestra.run.smithi157.stdout:(84/120): libcom_err-devel-1.45.6-1.el8.x86_64. 2.8 MB/s | 38 kB 00:00 2023-10-06T03:54:37.504 INFO:teuthology.orchestra.run.smithi157.stdout:(85/120): libsepol-devel-2.9-2.el8.x86_64.rpm 6.3 MB/s | 86 kB 00:00 2023-10-06T03:54:37.508 INFO:teuthology.orchestra.run.smithi157.stdout:(86/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 4.4 MB/s | 58 kB 00:00 2023-10-06T03:54:37.517 INFO:teuthology.orchestra.run.smithi157.stdout:(87/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 3.5 MB/s | 47 kB 00:00 2023-10-06T03:54:37.527 INFO:teuthology.orchestra.run.smithi157.stdout:(88/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.rp 29 MB/s | 559 kB 00:00 2023-10-06T03:54:37.536 INFO:teuthology.orchestra.run.smithi157.stdout:(89/120): libgfortran-8.4.1-1.el8.x86_64.rpm 33 MB/s | 639 kB 00:00 2023-10-06T03:54:37.540 INFO:teuthology.orchestra.run.smithi157.stdout:(90/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rpm 5.3 MB/s | 70 kB 00:00 2023-10-06T03:54:37.551 INFO:teuthology.orchestra.run.smithi157.stdout:(91/120): libselinux-devel-2.9-5.el8.x86_64.rpm 14 MB/s | 200 kB 00:00 2023-10-06T03:54:37.553 INFO:teuthology.orchestra.run.smithi157.stdout:(92/120): libuuid-devel-2.32.1-27.el8.x86_64.rp 7.3 MB/s | 97 kB 00:00 2023-10-06T03:54:37.565 INFO:teuthology.orchestra.run.smithi157.stdout:(93/120): python3-prettytable-0.7.2-14.el8.noar 3.6 MB/s | 44 kB 00:00 2023-10-06T03:54:37.577 INFO:teuthology.orchestra.run.smithi157.stdout:(94/120): python3-jsonpatch-1.21-2.el8.noarch.r 2.2 MB/s | 27 kB 00:00 2023-10-06T03:54:37.592 INFO:teuthology.orchestra.run.smithi157.stdout:(95/120): jq-1.5-12.el8.x86_64.rpm 11 MB/s | 161 kB 00:00 2023-10-06T03:54:37.609 INFO:teuthology.orchestra.run.smithi157.stdout:(96/120): flex-2.6.1-9.el8.x86_64.rpm 19 MB/s | 320 kB 00:00 2023-10-06T03:54:37.628 INFO:teuthology.orchestra.run.smithi157.stdout:(97/120): python3-werkzeug-0.12.2-4.el8.noarch. 25 MB/s | 457 kB 00:00 2023-10-06T03:54:37.710 INFO:teuthology.orchestra.run.smithi157.stdout:(98/120): python3-babel-2.5.1-5.el8.noarch.rpm 58 MB/s | 4.8 MB 00:00 2023-10-06T03:54:37.829 INFO:teuthology.orchestra.run.smithi157.stdout:(99/120): python3-scipy-1.0.0-20.module+el8.1.0 52 MB/s | 15 MB 00:00 2023-10-06T03:54:37.830 INFO:teuthology.orchestra.run.smithi157.stdout:(100/120): python3-markupsafe-0.23-19.el8.x86_6 326 kB/s | 39 kB 00:00 2023-10-06T03:54:37.833 INFO:teuthology.orchestra.run.smithi175.stdout:(8/120): ceph-radosgw-17.2.6-1274.g68e53a4a.el8 7.4 MB/s | 17 MB 00:02 2023-10-06T03:54:37.846 INFO:teuthology.orchestra.run.smithi157.stdout:(101/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 16 MB/s | 259 kB 00:00 2023-10-06T03:54:37.852 INFO:teuthology.orchestra.run.smithi157.stdout:(102/120): bison-3.0.4-10.el8.x86_64.rpm 30 MB/s | 688 kB 00:00 2023-10-06T03:54:37.862 INFO:teuthology.orchestra.run.smithi157.stdout:(103/120): python3-numpy-f2py-1.14.3-9.el8.x86_ 14 MB/s | 225 kB 00:00 2023-10-06T03:54:37.867 INFO:teuthology.orchestra.run.smithi157.stdout:(104/120): python3-pyasn1-0.3.7-6.el8.noarch.rp 8.3 MB/s | 126 kB 00:00 2023-10-06T03:54:37.884 INFO:teuthology.orchestra.run.smithi175.stdout:(9/120): ceph-selinux-17.2.6-1274.g68e53a4a.el8 478 kB/s | 24 kB 00:00 2023-10-06T03:54:37.888 INFO:teuthology.orchestra.run.smithi157.stdout:(105/120): oniguruma-6.8.2-2.el8.x86_64.rpm 8.7 MB/s | 187 kB 00:00 2023-10-06T03:54:37.909 INFO:teuthology.orchestra.run.smithi157.stdout:(106/120): python3-jinja2-2.10.1-2.el8_0.noarch 25 MB/s | 538 kB 00:00 2023-10-06T03:54:37.926 INFO:teuthology.orchestra.run.smithi157.stdout:(107/120): python3-numpy-1.14.3-9.el8.x86_64.rp 58 MB/s | 3.7 MB 00:00 2023-10-06T03:54:37.930 INFO:teuthology.orchestra.run.smithi157.stdout:(108/120): socat-1.7.3.3-2.el8.x86_64.rpm 14 MB/s | 302 kB 00:00 2023-10-06T03:54:37.943 INFO:teuthology.orchestra.run.smithi157.stdout:(109/120): python3-mako-1.0.6-13.el8.noarch.rpm 9.3 MB/s | 157 kB 00:00 2023-10-06T03:54:37.950 INFO:teuthology.orchestra.run.smithi157.stdout:(110/120): python3-pyasn1-modules-0.3.7-6.el8.n 5.4 MB/s | 110 kB 00:00 2023-10-06T03:54:37.961 INFO:teuthology.orchestra.run.smithi157.stdout:(111/120): librdkafka-0.11.4-1.el8.x86_64.rpm 19 MB/s | 353 kB 00:00 2023-10-06T03:54:37.963 INFO:teuthology.orchestra.run.smithi157.stdout:(112/120): python3-jsonpointer-1.10-11.el8.noar 1.5 MB/s | 20 kB 00:00 2023-10-06T03:54:37.976 INFO:teuthology.orchestra.run.smithi157.stdout:(113/120): python3-pytz-2017.2-9.el8.noarch.rpm 4.2 MB/s | 54 kB 00:00 2023-10-06T03:54:37.982 INFO:teuthology.orchestra.run.smithi157.stdout:(114/120): libtool-2.4.6-25.el8.x86_64.rpm 34 MB/s | 709 kB 00:00 2023-10-06T03:54:38.053 INFO:teuthology.orchestra.run.smithi157.stdout:(115/120): openblas-threads-0.3.12-1.el8.x86_64 62 MB/s | 4.7 MB 00:00 2023-10-06T03:54:38.071 INFO:teuthology.orchestra.run.smithi157.stdout:(116/120): openblas-0.3.12-1.el8.x86_64.rpm 52 MB/s | 4.6 MB 00:00 2023-10-06T03:54:38.078 INFO:teuthology.orchestra.run.smithi157.stdout:(117/120): automake-1.16.1-7.el8.noarch.rpm 27 MB/s | 713 kB 00:00 2023-10-06T03:54:38.732 INFO:teuthology.orchestra.run.smithi157.stdout:(118/120): librbd1-17.2.6-1274.g68e53a4a.el8.x8 5.4 MB/s | 3.5 MB 00:00 2023-10-06T03:54:38.839 INFO:teuthology.orchestra.run.smithi157.stdout:(119/120): librados2-17.2.6-1274.g68e53a4a.el8. 4.4 MB/s | 3.4 MB 00:00 2023-10-06T03:54:40.250 INFO:teuthology.orchestra.run.smithi175.stdout:(10/120): ceph-osd-17.2.6-1274.g68e53a4a.el8.x8 3.3 MB/s | 17 MB 00:05 2023-10-06T03:54:40.348 INFO:teuthology.orchestra.run.smithi175.stdout:(11/120): cephfs-mirror-17.2.6-1274.g68e53a4a.e 2.6 MB/s | 255 kB 00:00 2023-10-06T03:54:40.385 INFO:teuthology.orchestra.run.smithi175.stdout:(12/120): libcephfs-devel-17.2.6-1274.g68e53a4a 685 kB/s | 25 kB 00:00 2023-10-06T03:54:40.531 INFO:teuthology.orchestra.run.smithi175.stdout:(13/120): libcephfs2-17.2.6-1274.g68e53a4a.el8. 5.3 MB/s | 797 kB 00:00 2023-10-06T03:54:40.569 INFO:teuthology.orchestra.run.smithi175.stdout:(14/120): libcephsqlite-17.2.6-1274.g68e53a4a.e 4.5 MB/s | 175 kB 00:00 2023-10-06T03:54:40.607 INFO:teuthology.orchestra.run.smithi175.stdout:(15/120): librados-devel-17.2.6-1274.g68e53a4a. 3.4 MB/s | 130 kB 00:00 2023-10-06T03:54:40.716 INFO:teuthology.orchestra.run.smithi175.stdout:(16/120): libradosstriper1-17.2.6-1274.g68e53a4 4.5 MB/s | 506 kB 00:00 2023-10-06T03:54:40.761 INFO:teuthology.orchestra.run.smithi157.stdout:(120/120): ceph-test-17.2.6-1274.g68e53a4a.el8. 8.4 MB/s | 52 MB 00:06 2023-10-06T03:54:40.766 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2023-10-06T03:54:40.766 INFO:teuthology.orchestra.run.smithi157.stdout:Total 21 MB/s | 201 MB 00:09 2023-10-06T03:54:40.820 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2023-10-06T03:54:40.935 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2023-10-06T03:54:40.935 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2023-10-06T03:54:41.006 INFO:teuthology.orchestra.run.smithi175.stdout:(17/120): ceph-common-17.2.6-1274.g68e53a4a.el8 2.6 MB/s | 23 MB 00:09 2023-10-06T03:54:41.042 INFO:teuthology.orchestra.run.smithi175.stdout:(18/120): python3-ceph-argparse-17.2.6-1274.g68 1.2 MB/s | 45 kB 00:00 2023-10-06T03:54:41.067 INFO:teuthology.orchestra.run.smithi175.stdout:(19/120): python3-ceph-common-17.2.6-1274.g68e5 4.5 MB/s | 115 kB 00:00 2023-10-06T03:54:41.109 INFO:teuthology.orchestra.run.smithi175.stdout:(20/120): python3-cephfs-17.2.6-1274.g68e53a4a. 4.4 MB/s | 190 kB 00:00 2023-10-06T03:54:41.160 INFO:teuthology.orchestra.run.smithi175.stdout:(21/120): python3-rados-17.2.6-1274.g68e53a4a.e 7.2 MB/s | 371 kB 00:00 2023-10-06T03:54:41.204 INFO:teuthology.orchestra.run.smithi175.stdout:(22/120): python3-rbd-17.2.6-1274.g68e53a4a.el8 7.8 MB/s | 358 kB 00:00 2023-10-06T03:54:41.227 INFO:teuthology.orchestra.run.smithi175.stdout:(23/120): python3-rgw-17.2.6-1274.g68e53a4a.el8 4.7 MB/s | 107 kB 00:00 2023-10-06T03:54:41.248 INFO:teuthology.orchestra.run.smithi175.stdout:(24/120): rbd-fuse-17.2.6-1274.g68e53a4a.el8.x8 4.1 MB/s | 91 kB 00:00 2023-10-06T03:54:41.660 INFO:teuthology.orchestra.run.smithi175.stdout:(25/120): rbd-mirror-17.2.6-1274.g68e53a4a.el8. 8.4 MB/s | 3.5 MB 00:00 2023-10-06T03:54:41.684 INFO:teuthology.orchestra.run.smithi175.stdout:(26/120): rbd-nbd-17.2.6-1274.g68e53a4a.el8.x86 7.3 MB/s | 173 kB 00:00 2023-10-06T03:54:41.707 INFO:teuthology.orchestra.run.smithi175.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1274.g 1.1 MB/s | 23 kB 00:00 2023-10-06T03:54:41.727 INFO:teuthology.orchestra.run.smithi175.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1274.g68e53a4 5.5 MB/s | 121 kB 00:00 2023-10-06T03:54:41.803 INFO:teuthology.orchestra.run.smithi175.stdout:(29/120): librgw2-17.2.6-1274.g68e53a4a.el8.x86 5.1 MB/s | 5.6 MB 00:01 2023-10-06T03:54:41.931 INFO:teuthology.orchestra.run.smithi175.stdout:(30/120): ceph-mgr-dashboard-17.2.6-1274.g68e53 8.3 MB/s | 1.7 MB 00:00 2023-10-06T03:54:41.973 INFO:teuthology.orchestra.run.smithi175.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1274.g68 5.6 MB/s | 238 kB 00:00 2023-10-06T03:54:41.994 INFO:teuthology.orchestra.run.smithi175.stdout:(32/120): ceph-mgr-rook-17.2.6-1274.g68e53a4a.e 2.2 MB/s | 47 kB 00:00 2023-10-06T03:54:42.016 INFO:teuthology.orchestra.run.smithi175.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1274.g6 693 kB/s | 14 kB 00:00 2023-10-06T03:54:42.058 INFO:teuthology.orchestra.run.smithi175.stdout:(34/120): ceph-volume-17.2.6-1274.g68e53a4a.el8 6.2 MB/s | 267 kB 00:00 2023-10-06T03:54:42.079 INFO:teuthology.orchestra.run.smithi175.stdout:(35/120): cephadm-17.2.6-1274.g68e53a4a.el8.noa 3.7 MB/s | 80 kB 00:00 2023-10-06T03:54:42.100 INFO:teuthology.orchestra.run.smithi175.stdout:(36/120): cephfs-top-17.2.6-1274.g68e53a4a.el8. 860 kB/s | 18 kB 00:00 2023-10-06T03:54:42.675 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2023-10-06T03:54:42.679 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2023-10-06T03:54:42.781 INFO:teuthology.orchestra.run.smithi175.stdout:(37/120): ceph-test-17.2.6-1274.g68e53a4a.el8.x 11 MB/s | 52 MB 00:04 2023-10-06T03:54:42.845 INFO:teuthology.orchestra.run.smithi175.stdout:(38/120): gperftools-libs-2.7-9.el8.x86_64.rpm 411 kB/s | 306 kB 00:00 2023-10-06T03:54:42.948 INFO:teuthology.orchestra.run.smithi175.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 731 kB/s | 75 kB 00:00 2023-10-06T03:54:43.063 INFO:teuthology.orchestra.run.smithi175.stdout:(40/120): liboath-2.6.2-3.el8.x86_64.rpm 209 kB/s | 59 kB 00:00 2023-10-06T03:54:43.148 INFO:teuthology.orchestra.run.smithi175.stdout:(41/120): ceph-mgr-diskprediction-local-17.2.6- 5.5 MB/s | 7.4 MB 00:01 2023-10-06T03:54:43.210 INFO:teuthology.orchestra.run.smithi175.stdout:(42/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 506 kB/s | 44 kB 00:00 2023-10-06T03:54:43.220 INFO:teuthology.orchestra.run.smithi175.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 468 kB/s | 33 kB 00:00 2023-10-06T03:54:43.282 INFO:teuthology.orchestra.run.smithi175.stdout:(44/120): python3-certifi-2018.10.15-7.el8.noar 262 kB/s | 16 kB 00:00 2023-10-06T03:54:43.308 INFO:teuthology.orchestra.run.smithi175.stdout:(45/120): python3-asyncssh-2.7.0-2.el8.noarch.r 1.3 MB/s | 476 kB 00:00 2023-10-06T03:54:43.422 INFO:teuthology.orchestra.run.smithi175.stdout:(46/120): python3-cheroot-8.5.2-1.el8.noarch.rp 1.2 MB/s | 173 kB 00:00 2023-10-06T03:54:43.491 INFO:teuthology.orchestra.run.smithi175.stdout:(47/120): python3-google-auth-1.1.1-10.el8.noar 1.2 MB/s | 81 kB 00:00 2023-10-06T03:54:43.515 INFO:teuthology.orchestra.run.smithi175.stdout:(48/120): python3-beautifulsoup4-4.6.3-2.el8.1. 505 kB/s | 185 kB 00:00 2023-10-06T03:54:43.546 INFO:teuthology.orchestra.run.smithi175.stdout:(49/120): python3-cherrypy-18.4.0-1.el8.noarch. 1.6 MB/s | 384 kB 00:00 2023-10-06T03:54:43.547 INFO:teuthology.orchestra.run.smithi175.stdout:(50/120): python3-jaraco-6.2-6.el8.noarch.rpm 198 kB/s | 11 kB 00:00 2023-10-06T03:54:43.573 INFO:teuthology.orchestra.run.smithi175.stdout:(51/120): python3-jaraco-functools-2.0-4.el8.no 305 kB/s | 18 kB 00:00 2023-10-06T03:54:43.610 INFO:teuthology.orchestra.run.smithi175.stdout:(52/120): python3-logutils-0.3.5-11.el8.noarch. 775 kB/s | 49 kB 00:00 2023-10-06T03:54:43.639 INFO:teuthology.orchestra.run.smithi175.stdout:(53/120): python3-more-itertools-7.2.0-3.el8.no 894 kB/s | 59 kB 00:00 2023-10-06T03:54:43.676 INFO:teuthology.orchestra.run.smithi175.stdout:(54/120): python3-natsort-7.1.1-2.el8.noarch.rp 929 kB/s | 60 kB 00:00 2023-10-06T03:54:43.732 INFO:teuthology.orchestra.run.smithi175.stdout:(55/120): python3-portend-2.6-1.el8.noarch.rpm 278 kB/s | 16 kB 00:00 2023-10-06T03:54:43.750 INFO:teuthology.orchestra.run.smithi175.stdout:(56/120): python3-pecan-1.3.2-9.el8.noarch.rpm 2.5 MB/s | 283 kB 00:00 2023-10-06T03:54:43.792 INFO:teuthology.orchestra.run.smithi175.stdout:(57/120): python3-repoze-lru-0.7-6.el8.noarch.r 561 kB/s | 34 kB 00:00 2023-10-06T03:54:43.819 INFO:teuthology.orchestra.run.smithi175.stdout:(58/120): python3-routes-2.4.1-12.el8.noarch.rp 2.8 MB/s | 196 kB 00:00 2023-10-06T03:54:43.857 INFO:teuthology.orchestra.run.smithi175.stdout:(59/120): python3-rsa-4.9-2.el8.noarch.rpm 992 kB/s | 64 kB 00:00 2023-10-06T03:54:43.874 INFO:teuthology.orchestra.run.smithi175.stdout:(60/120): python3-simplegeneric-0.8.1-17.el8.no 338 kB/s | 19 kB 00:00 2023-10-06T03:54:43.915 INFO:teuthology.orchestra.run.smithi175.stdout:(61/120): python3-singledispatch-3.4.0.3-18.el8 413 kB/s | 24 kB 00:00 2023-10-06T03:54:43.922 INFO:teuthology.orchestra.run.smithi175.stdout:(62/120): python3-kubernetes-11.0.0-6.el8.noarc 3.1 MB/s | 1.2 MB 00:00 2023-10-06T03:54:43.930 INFO:teuthology.orchestra.run.smithi175.stdout:(63/120): python3-tempora-1.14.1-5.el8.noarch.r 521 kB/s | 29 kB 00:00 2023-10-06T03:54:43.975 INFO:teuthology.orchestra.run.smithi175.stdout:(64/120): python3-trustme-0.6.0-4.el8.noarch.rp 467 kB/s | 27 kB 00:00 2023-10-06T03:54:44.003 INFO:teuthology.orchestra.run.smithi175.stdout:(65/120): python3-webob-1.8.5-1.el8.1.noarch.rp 3.4 MB/s | 251 kB 00:00 2023-10-06T03:54:44.028 INFO:teuthology.orchestra.run.smithi175.stdout:(66/120): python3-waitress-1.4.3-1.el8.noarch.r 2.4 MB/s | 255 kB 00:00 2023-10-06T03:54:44.040 INFO:teuthology.orchestra.run.smithi175.stdout:(67/120): python3-websocket-client-0.56.0-5.el8 943 kB/s | 61 kB 00:00 2023-10-06T03:54:44.065 INFO:teuthology.orchestra.run.smithi175.stdout:(68/120): python3-webtest-2.0.33-1.el8.noarch.r 1.3 MB/s | 85 kB 00:00 2023-10-06T03:54:44.070 INFO:teuthology.orchestra.run.smithi175.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 15 MB/s | 70 kB 00:00 2023-10-06T03:54:44.086 INFO:teuthology.orchestra.run.smithi175.stdout:(70/120): python3-zc-lockfile-2.0-2.el8.noarch. 391 kB/s | 23 kB 00:00 2023-10-06T03:54:44.119 INFO:teuthology.orchestra.run.smithi175.stdout:(71/120): python3-prettytable-0.7.2-14.el8.noar 1.4 MB/s | 44 kB 00:00 2023-10-06T03:54:44.135 INFO:teuthology.orchestra.run.smithi175.stdout:(72/120): python3-jsonpatch-1.21-2.el8.noarch.r 1.6 MB/s | 27 kB 00:00 2023-10-06T03:54:44.153 INFO:teuthology.orchestra.run.smithi175.stdout:(73/120): jq-1.5-12.el8.x86_64.rpm 8.9 MB/s | 161 kB 00:00 2023-10-06T03:54:44.173 INFO:teuthology.orchestra.run.smithi175.stdout:(74/120): flex-2.6.1-9.el8.x86_64.rpm 16 MB/s | 320 kB 00:00 2023-10-06T03:54:44.196 INFO:teuthology.orchestra.run.smithi175.stdout:(75/120): python3-werkzeug-0.12.2-4.el8.noarch. 19 MB/s | 457 kB 00:00 2023-10-06T03:54:44.342 INFO:teuthology.orchestra.run.smithi175.stdout:(76/120): python3-scipy-1.0.0-20.module+el8.1.0 53 MB/s | 15 MB 00:00 2023-10-06T03:54:44.349 INFO:teuthology.orchestra.run.smithi175.stdout:(77/120): thrift-0.13.0-2.el8.x86_64.rpm 5.6 MB/s | 1.7 MB 00:00 2023-10-06T03:54:44.363 INFO:teuthology.orchestra.run.smithi175.stdout:(78/120): python3-markupsafe-0.23-19.el8.x86_64 1.9 MB/s | 39 kB 00:00 2023-10-06T03:54:44.385 INFO:teuthology.orchestra.run.smithi175.stdout:(79/120): python3-babel-2.5.1-5.el8.noarch.rpm 25 MB/s | 4.8 MB 00:00 2023-10-06T03:54:44.389 INFO:teuthology.orchestra.run.smithi175.stdout:(80/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 10 MB/s | 259 kB 00:00 2023-10-06T03:54:44.394 INFO:teuthology.orchestra.run.smithi175.stdout:(81/120): bison-3.0.4-10.el8.x86_64.rpm 15 MB/s | 688 kB 00:00 2023-10-06T03:54:44.404 INFO:teuthology.orchestra.run.smithi175.stdout:(82/120): python3-numpy-f2py-1.14.3-9.el8.x86_6 12 MB/s | 225 kB 00:00 2023-10-06T03:54:44.409 INFO:teuthology.orchestra.run.smithi175.stdout:(83/120): python3-pyasn1-0.3.7-6.el8.noarch.rpm 6.2 MB/s | 126 kB 00:00 2023-10-06T03:54:44.423 INFO:teuthology.orchestra.run.smithi175.stdout:(84/120): oniguruma-6.8.2-2.el8.x86_64.rpm 9.8 MB/s | 187 kB 00:00 2023-10-06T03:54:44.436 INFO:teuthology.orchestra.run.smithi175.stdout:(85/120): python3-jinja2-2.10.1-2.el8_0.noarch. 20 MB/s | 538 kB 00:00 2023-10-06T03:54:44.447 INFO:teuthology.orchestra.run.smithi175.stdout:(86/120): socat-1.7.3.3-2.el8.x86_64.rpm 12 MB/s | 302 kB 00:00 2023-10-06T03:54:44.462 INFO:teuthology.orchestra.run.smithi175.stdout:(87/120): python3-numpy-1.14.3-9.el8.x86_64.rpm 55 MB/s | 3.7 MB 00:00 2023-10-06T03:54:44.464 INFO:teuthology.orchestra.run.smithi175.stdout:(88/120): python3-pyasn1-modules-0.3.7-6.el8.no 6.3 MB/s | 110 kB 00:00 2023-10-06T03:54:44.465 INFO:teuthology.orchestra.run.smithi175.stdout:(89/120): python3-mako-1.0.6-13.el8.noarch.rpm 5.2 MB/s | 157 kB 00:00 2023-10-06T03:54:44.481 INFO:teuthology.orchestra.run.smithi175.stdout:(90/120): python3-jsonpointer-1.10-11.el8.noarc 1.3 MB/s | 20 kB 00:00 2023-10-06T03:54:44.484 INFO:teuthology.orchestra.run.smithi175.stdout:(91/120): librdkafka-0.11.4-1.el8.x86_64.rpm 17 MB/s | 353 kB 00:00 2023-10-06T03:54:44.490 INFO:teuthology.orchestra.run.smithi175.stdout:(92/120): libtool-2.4.6-25.el8.x86_64.rpm 28 MB/s | 709 kB 00:00 2023-10-06T03:54:44.495 INFO:teuthology.orchestra.run.smithi175.stdout:(93/120): python3-pytz-2017.2-9.el8.noarch.rpm 3.6 MB/s | 54 kB 00:00 2023-10-06T03:54:44.526 INFO:teuthology.orchestra.run.smithi175.stdout:(94/120): automake-1.16.1-7.el8.noarch.rpm 23 MB/s | 713 kB 00:00 2023-10-06T03:54:44.543 INFO:teuthology.orchestra.run.smithi175.stdout:(95/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 1.1 MB/s | 19 kB 00:00 2023-10-06T03:54:44.562 INFO:teuthology.orchestra.run.smithi175.stdout:(96/120): openblas-threads-0.3.12-1.el8.x86_64. 60 MB/s | 4.7 MB 00:00 2023-10-06T03:54:44.563 INFO:teuthology.orchestra.run.smithi175.stdout:(97/120): python3-jwt-1.6.1-2.el8.noarch.rpm 2.1 MB/s | 43 kB 00:00 2023-10-06T03:54:44.582 INFO:teuthology.orchestra.run.smithi175.stdout:(98/120): openblas-0.3.12-1.el8.x86_64.rpm 50 MB/s | 4.6 MB 00:00 2023-10-06T03:54:44.584 INFO:teuthology.orchestra.run.smithi175.stdout:(99/120): python3-oauthlib-2.1.0-1.el8.noarch.r 7.0 MB/s | 155 kB 00:00 2023-10-06T03:54:44.585 INFO:teuthology.orchestra.run.smithi175.stdout:(100/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 9.7 MB/s | 220 kB 00:00 2023-10-06T03:54:44.603 INFO:teuthology.orchestra.run.smithi175.stdout:(101/120): keyutils-libs-devel-1.5.10-6.el8.x86 2.8 MB/s | 48 kB 00:00 2023-10-06T03:54:44.603 INFO:teuthology.orchestra.run.smithi175.stdout:(102/120): python3-requests-oauthlib-1.0.0-1.el 2.7 MB/s | 43 kB 00:00 2023-10-06T03:54:44.603 INFO:teuthology.orchestra.run.smithi175.stdout:(103/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 13 MB/s | 229 kB 00:00 2023-10-06T03:54:44.618 INFO:teuthology.orchestra.run.smithi175.stdout:(104/120): libverto-devel-0.3.0-5.el8.x86_64.rp 1.0 MB/s | 18 kB 00:00 2023-10-06T03:54:44.625 INFO:teuthology.orchestra.run.smithi175.stdout:(105/120): pcre2-devel-10.32-2.el8.x86_64.rpm 25 MB/s | 605 kB 00:00 2023-10-06T03:54:44.635 INFO:teuthology.orchestra.run.smithi175.stdout:(106/120): fuse-2.9.7-12.el8.x86_64.rpm 4.6 MB/s | 83 kB 00:00 2023-10-06T03:54:44.642 INFO:teuthology.orchestra.run.smithi175.stdout:(107/120): openssl-devel-1.1.1g-15.el8_3.x86_64 53 MB/s | 2.3 MB 00:00 2023-10-06T03:54:44.644 INFO:teuthology.orchestra.run.smithi175.stdout:(108/120): elfutils-libelf-devel-0.182-3.el8.x8 3.1 MB/s | 59 kB 00:00 2023-10-06T03:54:44.656 INFO:teuthology.orchestra.run.smithi175.stdout:(109/120): libquadmath-8.4.1-1.el8.x86_64.rpm 8.2 MB/s | 169 kB 00:00 2023-10-06T03:54:44.660 INFO:teuthology.orchestra.run.smithi175.stdout:(110/120): libcom_err-devel-1.45.6-1.el8.x86_64 2.1 MB/s | 38 kB 00:00 2023-10-06T03:54:44.662 INFO:teuthology.orchestra.run.smithi175.stdout:(111/120): libsepol-devel-2.9-2.el8.x86_64.rpm 4.6 MB/s | 86 kB 00:00 2023-10-06T03:54:44.676 INFO:teuthology.orchestra.run.smithi175.stdout:(112/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 2.8 MB/s | 58 kB 00:00 2023-10-06T03:54:44.684 INFO:teuthology.orchestra.run.smithi175.stdout:(113/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.r 26 MB/s | 559 kB 00:00 2023-10-06T03:54:44.690 INFO:teuthology.orchestra.run.smithi175.stdout:(114/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.6 MB/s | 47 kB 00:00 2023-10-06T03:54:44.747 INFO:teuthology.orchestra.run.smithi175.stdout:(115/120): libgfortran-8.4.1-1.el8.x86_64.rpm 27 MB/s | 639 kB 00:00 2023-10-06T03:54:44.747 INFO:teuthology.orchestra.run.smithi175.stdout:(116/120): libselinux-devel-2.9-5.el8.x86_64.rp 11 MB/s | 200 kB 00:00 2023-10-06T03:54:44.747 INFO:teuthology.orchestra.run.smithi175.stdout:(117/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rp 2.7 MB/s | 70 kB 00:00 2023-10-06T03:54:44.747 INFO:teuthology.orchestra.run.smithi175.stdout:(118/120): libuuid-devel-2.32.1-27.el8.x86_64.r 5.2 MB/s | 97 kB 00:00 2023-10-06T03:54:45.116 INFO:teuthology.orchestra.run.smithi175.stdout:(119/120): librados2-17.2.6-1274.g68e53a4a.el8. 8.4 MB/s | 3.4 MB 00:00 2023-10-06T03:54:45.376 INFO:teuthology.orchestra.run.smithi175.stdout:(120/120): librbd1-17.2.6-1274.g68e53a4a.el8.x8 5.3 MB/s | 3.5 MB 00:00 2023-10-06T03:54:45.386 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-10-06T03:54:45.386 INFO:teuthology.orchestra.run.smithi175.stdout:Total 15 MB/s | 201 MB 00:13 2023-10-06T03:54:45.431 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-10-06T03:54:45.537 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-10-06T03:54:45.538 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-10-06T03:54:47.061 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2023-10-06T03:54:47.248 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-10-06T03:54:47.252 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-10-06T03:54:47.328 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-06T03:54:47.492 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-06T03:54:47.616 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-06T03:54:47.817 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-06T03:54:48.178 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-06T03:54:48.416 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 4/122 2023-10-06T03:54:48.823 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 4/122 2023-10-06T03:54:49.025 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 5/122 2023-10-06T03:54:49.153 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 5/122 2023-10-06T03:54:49.341 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T03:54:49.439 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T03:54:49.646 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 7/122 2023-10-06T03:54:50.005 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/122 2023-10-06T03:54:50.669 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-06T03:54:50.833 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T03:54:50.931 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T03:54:51.236 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-06T03:54:51.398 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-06T03:54:51.582 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T03:54:51.673 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T03:54:51.815 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-10-06T03:54:51.816 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 14/122 2023-10-06T03:54:51.894 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 14/122 2023-10-06T03:54:52.023 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-06T03:54:52.070 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 15/122 2023-10-06T03:54:52.270 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 2/122 2023-10-06T03:54:52.331 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 16/122 2023-10-06T03:54:52.437 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-06T03:54:52.582 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-06T03:54:52.590 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/122 2023-10-06T03:54:52.745 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 18/122 2023-10-06T03:54:52.798 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-06T03:54:52.891 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-06T03:54:53.034 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-06T03:54:53.175 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-06T03:54:53.199 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-06T03:54:53.412 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 5/122 2023-10-06T03:54:53.737 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-06T03:54:53.798 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 5/122 2023-10-06T03:54:53.907 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-06T03:54:53.995 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T03:54:54.115 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T03:54:54.303 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 7/122 2023-10-06T03:54:54.384 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 7/122 2023-10-06T03:54:54.430 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-06T03:54:54.624 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-0.3.12-1.el8.x86_64 22/122 2023-10-06T03:54:54.765 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 8/122 2023-10-06T03:54:55.354 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 22/122 2023-10-06T03:54:55.463 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-06T03:54:55.611 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 23/122 2023-10-06T03:54:55.678 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T03:54:55.782 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T03:54:55.870 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 24/122 2023-10-06T03:54:56.057 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-06T03:54:56.119 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-06T03:54:56.218 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-06T03:54:56.469 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T03:54:56.575 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T03:54:56.728 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 14/122 2023-10-06T03:54:56.810 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 14/122 2023-10-06T03:54:57.224 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8. 26/122 2023-10-06T03:54:57.394 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el 27/122 2023-10-06T03:54:57.596 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 15/122 2023-10-06T03:54:57.688 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_6 28/122 2023-10-06T03:54:57.794 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-06T03:54:57.916 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 29/122 2023-10-06T03:54:57.920 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-06T03:54:58.119 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-06T03:54:58.200 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 30/122 2023-10-06T03:54:58.423 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-06T03:54:58.672 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-06T03:54:58.861 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-06T03:54:59.390 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-06T03:54:59.611 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-0.3.12-1.el8.x86_64 19/122 2023-10-06T03:55:00.331 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 19/122 2023-10-06T03:55:00.514 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-06T03:55:00.605 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 20/122 2023-10-06T03:55:00.817 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 21/122 2023-10-06T03:55:00.898 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 32/122 2023-10-06T03:55:00.981 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 22/122 2023-10-06T03:55:01.258 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/122 2023-10-06T03:55:01.569 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 24/122 2023-10-06T03:55:01.805 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-06T03:55:01.919 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/122 2023-10-06T03:55:01.983 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8. 26/122 2023-10-06T03:55:02.109 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el 27/122 2023-10-06T03:55:02.263 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/122 2023-10-06T03:55:02.393 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_6 28/122 2023-10-06T03:55:02.560 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/122 2023-10-06T03:55:02.804 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-06T03:55:03.026 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-06T03:55:03.232 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 38/122 2023-10-06T03:55:03.434 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 39/122 2023-10-06T03:55:03.443 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 29/122 2023-10-06T03:55:03.699 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_6 40/122 2023-10-06T03:55:03.754 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 30/122 2023-10-06T03:55:03.927 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : automake-1.16.1-7.el8.noarch 41/122 2023-10-06T03:55:04.067 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 31/122 2023-10-06T03:55:04.121 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 42/122 2023-10-06T03:55:04.332 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 43/122 2023-10-06T03:55:04.340 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 32/122 2023-10-06T03:55:04.534 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 44/122 2023-10-06T03:55:04.609 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 33/122 2023-10-06T03:55:04.722 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-06T03:55:04.788 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-06T03:55:04.834 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-06T03:55:04.947 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : jq-1.5-12.el8.x86_64 46/122 2023-10-06T03:55:05.106 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 47/122 2023-10-06T03:55:05.308 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 48/122 2023-10-06T03:55:05.477 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 49/122 2023-10-06T03:55:05.667 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 50/122 2023-10-06T03:55:05.841 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : fuse-2.9.7-12.el8.x86_64 51/122 2023-10-06T03:55:05.993 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 52/122 2023-10-06T03:55:06.188 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 53/122 2023-10-06T03:55:06.350 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 54/122 2023-10-06T03:55:06.690 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/122 2023-10-06T03:55:06.887 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-06T03:55:06.954 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/122 2023-10-06T03:55:07.189 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/122 2023-10-06T03:55:07.212 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 35/122 2023-10-06T03:55:07.465 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-06T03:55:07.476 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 58/122 2023-10-06T03:55:07.647 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-06T03:55:07.679 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-06T03:55:07.859 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-06T03:55:07.902 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 38/122 2023-10-06T03:55:08.110 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-06T03:55:08.124 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 39/122 2023-10-06T03:55:08.323 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_6 40/122 2023-10-06T03:55:08.346 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-06T03:55:08.551 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 41/122 2023-10-06T03:55:08.571 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-06T03:55:08.706 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 42/122 2023-10-06T03:55:08.741 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-06T03:55:08.907 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-06T03:55:08.911 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 43/122 2023-10-06T03:55:09.070 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : fuse-2.9.7-12.el8.x86_64 44/122 2023-10-06T03:55:09.131 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-06T03:55:09.230 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 45/122 2023-10-06T03:55:09.334 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-06T03:55:09.444 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 46/122 2023-10-06T03:55:09.566 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-06T03:55:09.638 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 47/122 2023-10-06T03:55:09.774 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-06T03:55:09.971 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 48/122 2023-10-06T03:55:09.979 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-06T03:55:10.221 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 49/122 2023-10-06T03:55:10.230 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-06T03:55:10.483 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 50/122 2023-10-06T03:55:10.493 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-06T03:55:10.711 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-06T03:55:10.857 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 51/122 2023-10-06T03:55:10.998 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-06T03:55:11.091 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : automake-1.16.1-7.el8.noarch 52/122 2023-10-06T03:55:11.245 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-06T03:55:11.324 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 53/122 2023-10-06T03:55:11.437 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-06T03:55:11.504 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 54/122 2023-10-06T03:55:11.686 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 55/122 2023-10-06T03:55:11.687 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-06T03:55:11.850 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-06T03:55:11.951 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-06T03:55:12.133 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : jq-1.5-12.el8.x86_64 57/122 2023-10-06T03:55:12.353 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 58/122 2023-10-06T03:55:12.500 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-06T03:55:12.687 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-06T03:55:12.845 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-06T03:55:12.929 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-06T03:55:13.145 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-06T03:55:13.149 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-06T03:55:13.349 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-06T03:55:13.364 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-06T03:55:13.521 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-06T03:55:13.677 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-06T03:55:13.715 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-06T03:55:13.901 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-06T03:55:13.967 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-06T03:55:14.121 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-06T03:55:14.313 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-06T03:55:14.369 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-06T03:55:14.546 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-06T03:55:14.552 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-06T03:55:14.746 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-06T03:55:14.767 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-06T03:55:14.962 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-06T03:55:15.003 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-06T03:55:15.223 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-06T03:55:15.448 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-06T03:55:15.756 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-06T03:55:16.024 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-06T03:55:16.258 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-06T03:55:16.506 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-06T03:55:17.652 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-06T03:55:17.946 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-06T03:55:18.200 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-06T03:55:18.605 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-06T03:55:18.896 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-06T03:55:19.210 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-06T03:55:19.417 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-06T03:55:19.649 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-06T03:55:19.849 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-06T03:55:20.168 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T03:55:20.378 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T03:55:20.963 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T03:55:21.224 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 88/122 2023-10-06T03:55:21.325 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 88/122 2023-10-06T03:55:21.461 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 89/122 2023-10-06T03:55:24.977 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T03:55:25.223 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T03:55:25.810 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T03:55:26.098 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 88/122 2023-10-06T03:55:26.186 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 88/122 2023-10-06T03:55:26.314 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 89/122 2023-10-06T03:55:39.060 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 89/122 2023-10-06T03:55:39.060 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /sys 2023-10-06T03:55:39.060 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /proc 2023-10-06T03:55:39.060 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /dev 2023-10-06T03:55:39.060 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /run 2023-10-06T03:55:39.060 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /mnt 2023-10-06T03:55:39.061 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /var/tmp 2023-10-06T03:55:39.061 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /home 2023-10-06T03:55:39.061 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /tmp 2023-10-06T03:55:39.061 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /dev 2023-10-06T03:55:39.061 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:55:39.417 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 90/122 2023-10-06T03:55:40.900 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 90/122 2023-10-06T03:55:41.079 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 91/122 2023-10-06T03:55:41.302 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 91/122 2023-10-06T03:55:41.433 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 92/122 2023-10-06T03:55:41.838 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 92/122 2023-10-06T03:55:41.983 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 93/122 2023-10-06T03:55:42.000 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 93/122 2023-10-06T03:55:44.076 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 89/122 2023-10-06T03:55:44.076 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /sys 2023-10-06T03:55:44.076 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /proc 2023-10-06T03:55:44.076 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /dev 2023-10-06T03:55:44.076 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /run 2023-10-06T03:55:44.076 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /mnt 2023-10-06T03:55:44.076 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /var/tmp 2023-10-06T03:55:44.077 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /home 2023-10-06T03:55:44.077 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /tmp 2023-10-06T03:55:44.077 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /dev 2023-10-06T03:55:44.077 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:55:44.477 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 90/122 2023-10-06T03:55:44.833 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 94/122 2023-10-06T03:55:45.006 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 94/122 2023-10-06T03:55:45.197 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.e 95/122 2023-10-06T03:55:45.397 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a. 96/122 2023-10-06T03:55:45.565 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 97/122 2023-10-06T03:55:45.784 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 97/122 2023-10-06T03:55:45.952 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 90/122 2023-10-06T03:55:45.994 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 98/122 2023-10-06T03:55:46.136 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 91/122 2023-10-06T03:55:46.355 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 91/122 2023-10-06T03:55:46.502 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 92/122 2023-10-06T03:55:46.899 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 92/122 2023-10-06T03:55:46.963 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 98/122 2023-10-06T03:55:47.094 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 93/122 2023-10-06T03:55:47.110 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 93/122 2023-10-06T03:55:47.123 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 99/122 2023-10-06T03:55:47.223 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 99/122 2023-10-06T03:55:47.609 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el 100/122 2023-10-06T03:55:47.786 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 101/122 2023-10-06T03:55:47.823 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 101/122 2023-10-06T03:55:47.998 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 102/122 2023-10-06T03:55:48.010 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 102/122 2023-10-06T03:55:49.606 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 103/122 2023-10-06T03:55:49.827 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 104/122 2023-10-06T03:55:49.913 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 94/122 2023-10-06T03:55:49.933 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 104/122 2023-10-06T03:55:50.131 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1274.g68e53 105/122 2023-10-06T03:55:50.132 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 94/122 2023-10-06T03:55:50.176 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1274.g68e53 105/122 2023-10-06T03:55:50.274 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.e 95/122 2023-10-06T03:55:50.337 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 106/122 2023-10-06T03:55:50.502 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a. 96/122 2023-10-06T03:55:50.659 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 97/122 2023-10-06T03:55:50.684 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 106/122 2023-10-06T03:55:50.864 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 107/122 2023-10-06T03:55:50.872 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 97/122 2023-10-06T03:55:51.051 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 98/122 2023-10-06T03:55:52.010 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 98/122 2023-10-06T03:55:52.184 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 99/122 2023-10-06T03:55:52.281 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 99/122 2023-10-06T03:55:52.647 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el 100/122 2023-10-06T03:55:52.851 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 101/122 2023-10-06T03:55:52.886 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 101/122 2023-10-06T03:55:53.075 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 102/122 2023-10-06T03:55:53.086 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 102/122 2023-10-06T03:55:54.663 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 103/122 2023-10-06T03:55:54.893 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 104/122 2023-10-06T03:55:55.019 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 104/122 2023-10-06T03:55:55.209 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1274.g68e53 105/122 2023-10-06T03:55:55.251 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1274.g68e53 105/122 2023-10-06T03:55:55.434 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 106/122 2023-10-06T03:55:55.778 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 106/122 2023-10-06T03:55:55.784 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 107/122 2023-10-06T03:55:55.942 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 107/122 2023-10-06T03:55:56.730 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 108/122 2023-10-06T03:55:57.101 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 109/122 2023-10-06T03:55:57.293 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 110/122 2023-10-06T03:55:57.525 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 111/122 2023-10-06T03:55:57.688 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libtool-2.4.6-25.el8.x86_64 112/122 2023-10-06T03:55:57.699 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 112/122 2023-10-06T03:55:57.901 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_ 113/122 2023-10-06T03:55:58.059 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-06T03:55:58.213 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 115/122 2023-10-06T03:55:58.413 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 116/122 2023-10-06T03:55:58.660 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 117/122 2023-10-06T03:55:58.854 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : bison-3.0.4-10.el8.x86_64 118/122 2023-10-06T03:55:58.920 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 118/122 2023-10-06T03:55:59.138 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flex-2.6.1-9.el8.x86_64 119/122 2023-10-06T03:55:59.147 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 119/122 2023-10-06T03:55:59.282 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 120/122 2023-10-06T03:55:59.282 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T03:55:59.491 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T03:55:59.491 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T03:56:00.313 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T03:56:00.313 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 1/122 2023-10-06T03:56:00.313 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 2/122 2023-10-06T03:56:00.313 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 3/122 2023-10-06T03:56:00.314 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 4/122 2023-10-06T03:56:00.314 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1274.g68e53 5/122 2023-10-06T03:56:00.314 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T03:56:00.314 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 7/122 2023-10-06T03:56:00.314 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 8/122 2023-10-06T03:56:00.314 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 9/122 2023-10-06T03:56:00.314 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T03:56:00.314 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 11/122 2023-10-06T03:56:00.314 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 12/122 2023-10-06T03:56:00.314 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T03:56:00.314 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_ 14/122 2023-10-06T03:56:00.315 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 15/122 2023-10-06T03:56:00.315 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 16/122 2023-10-06T03:56:00.315 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_6 17/122 2023-10-06T03:56:00.315 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 18/122 2023-10-06T03:56:00.315 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 19/122 2023-10-06T03:56:00.315 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el 20/122 2023-10-06T03:56:00.315 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8. 21/122 2023-10-06T03:56:00.315 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_6 22/122 2023-10-06T03:56:00.315 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 23/122 2023-10-06T03:56:00.315 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 24/122 2023-10-06T03:56:00.315 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 25/122 2023-10-06T03:56:00.315 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 26/122 2023-10-06T03:56:00.316 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 27/122 2023-10-06T03:56:00.316 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 28/122 2023-10-06T03:56:00.316 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a. 29/122 2023-10-06T03:56:00.316 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 30/122 2023-10-06T03:56:00.316 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 31/122 2023-10-06T03:56:00.316 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 32/122 2023-10-06T03:56:00.316 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el 33/122 2023-10-06T03:56:00.316 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 34/122 2023-10-06T03:56:00.316 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.e 35/122 2023-10-06T03:56:00.316 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 36/122 2023-10-06T03:56:00.316 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 37/122 2023-10-06T03:56:00.316 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 38/122 2023-10-06T03:56:00.317 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-06T03:56:00.317 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-06T03:56:00.317 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-06T03:56:00.317 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-06T03:56:00.317 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-06T03:56:00.317 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-06T03:56:00.317 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-06T03:56:00.317 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-06T03:56:00.317 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-06T03:56:00.317 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-06T03:56:00.317 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-06T03:56:00.318 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-06T03:56:00.318 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-06T03:56:00.318 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-06T03:56:00.318 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-06T03:56:00.318 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-06T03:56:00.318 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-06T03:56:00.318 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-06T03:56:00.318 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-06T03:56:00.318 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-06T03:56:00.318 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-06T03:56:00.318 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-06T03:56:00.319 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-06T03:56:00.319 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-06T03:56:00.319 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-06T03:56:00.319 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-06T03:56:00.319 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-06T03:56:00.319 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-06T03:56:00.319 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-06T03:56:00.319 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-06T03:56:00.319 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-06T03:56:00.319 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-06T03:56:00.319 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-06T03:56:00.319 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 72/122 2023-10-06T03:56:00.320 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/122 2023-10-06T03:56:00.320 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/122 2023-10-06T03:56:00.320 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/122 2023-10-06T03:56:00.320 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/122 2023-10-06T03:56:00.320 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 77/122 2023-10-06T03:56:00.320 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/122 2023-10-06T03:56:00.320 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 79/122 2023-10-06T03:56:00.320 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 80/122 2023-10-06T03:56:00.320 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 81/122 2023-10-06T03:56:00.320 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 82/122 2023-10-06T03:56:00.320 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 83/122 2023-10-06T03:56:00.320 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 84/122 2023-10-06T03:56:00.321 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 85/122 2023-10-06T03:56:00.321 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 86/122 2023-10-06T03:56:00.321 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 87/122 2023-10-06T03:56:00.321 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 88/122 2023-10-06T03:56:00.321 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 89/122 2023-10-06T03:56:00.321 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 90/122 2023-10-06T03:56:00.321 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 91/122 2023-10-06T03:56:00.321 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 92/122 2023-10-06T03:56:00.321 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 93/122 2023-10-06T03:56:00.321 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 94/122 2023-10-06T03:56:00.321 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 95/122 2023-10-06T03:56:00.322 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 96/122 2023-10-06T03:56:00.322 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : jq-1.5-12.el8.x86_64 97/122 2023-10-06T03:56:00.322 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flex-2.6.1-9.el8.x86_64 98/122 2023-10-06T03:56:00.322 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 99/122 2023-10-06T03:56:00.322 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 100/122 2023-10-06T03:56:00.322 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 101/122 2023-10-06T03:56:00.322 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : bison-3.0.4-10.el8.x86_64 102/122 2023-10-06T03:56:00.322 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 103/122 2023-10-06T03:56:00.322 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 104/122 2023-10-06T03:56:00.322 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/122 2023-10-06T03:56:00.322 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 106/122 2023-10-06T03:56:00.322 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/122 2023-10-06T03:56:00.323 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 108/122 2023-10-06T03:56:00.323 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 109/122 2023-10-06T03:56:00.323 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 110/122 2023-10-06T03:56:00.323 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 111/122 2023-10-06T03:56:00.323 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 112/122 2023-10-06T03:56:00.323 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 113/122 2023-10-06T03:56:00.323 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 114/122 2023-10-06T03:56:00.323 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 115/122 2023-10-06T03:56:00.323 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 116/122 2023-10-06T03:56:00.323 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 117/122 2023-10-06T03:56:00.323 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : automake-1.16.1-7.el8.noarch 118/122 2023-10-06T03:56:00.323 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 119/122 2023-10-06T03:56:00.324 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-06T03:56:00.324 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 121/122 2023-10-06T03:56:00.845 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 107/122 2023-10-06T03:56:01.191 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 108/122 2023-10-06T03:56:01.357 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libtool-2.4.6-25.el8.x86_64 109/122 2023-10-06T03:56:02.034 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 109/122 2023-10-06T03:56:02.397 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T03:56:02.397 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:40 ago on Fri 06 Oct 2023 03:54:22 AM UTC. 2023-10-06T03:56:02.415 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 110/122 2023-10-06T03:56:02.647 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 111/122 2023-10-06T03:56:02.794 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 112/122 2023-10-06T03:56:02.953 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_ 113/122 2023-10-06T03:56:03.120 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-06T03:56:03.266 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 115/122 2023-10-06T03:56:03.482 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 116/122 2023-10-06T03:56:03.633 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 117/122 2023-10-06T03:56:03.907 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 118/122 2023-10-06T03:56:04.106 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : bison-3.0.4-10.el8.x86_64 119/122 2023-10-06T03:56:04.183 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 119/122 2023-10-06T03:56:04.365 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flex-2.6.1-9.el8.x86_64 120/122 2023-10-06T03:56:04.374 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 120/122 2023-10-06T03:56:04.374 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T03:56:04.601 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T03:56:04.601 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T03:56:04.742 INFO:teuthology.orchestra.run.smithi157.stdout:Installed products updated. 2023-10-06T03:56:04.878 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:04.879 INFO:teuthology.orchestra.run.smithi157.stdout:Upgraded: 2023-10-06T03:56:04.879 INFO:teuthology.orchestra.run.smithi157.stdout: librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.879 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.879 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2023-10-06T03:56:04.879 INFO:teuthology.orchestra.run.smithi157.stdout: automake-1.16.1-7.el8.noarch 2023-10-06T03:56:04.879 INFO:teuthology.orchestra.run.smithi157.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T03:56:04.879 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.880 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.880 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.880 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.880 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T03:56:04.880 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.880 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.880 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.880 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T03:56:04.880 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T03:56:04.881 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T03:56:04.881 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T03:56:04.881 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T03:56:04.881 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.881 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.881 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T03:56:04.881 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.881 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.881 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.882 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T03:56:04.882 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T03:56:04.882 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.882 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T03:56:04.882 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-06T03:56:04.882 INFO:teuthology.orchestra.run.smithi157.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T03:56:04.882 INFO:teuthology.orchestra.run.smithi157.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-06T03:56:04.882 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T03:56:04.882 INFO:teuthology.orchestra.run.smithi157.stdout: jq-1.5-12.el8.x86_64 2023-10-06T03:56:04.883 INFO:teuthology.orchestra.run.smithi157.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-06T03:56:04.883 INFO:teuthology.orchestra.run.smithi157.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-06T03:56:04.883 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T03:56:04.883 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.883 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.883 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.883 INFO:teuthology.orchestra.run.smithi157.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-06T03:56:04.883 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-06T03:56:04.884 INFO:teuthology.orchestra.run.smithi157.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T03:56:04.884 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-06T03:56:04.884 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-06T03:56:04.884 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.884 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.884 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-06T03:56:04.884 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.884 INFO:teuthology.orchestra.run.smithi157.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-06T03:56:04.885 INFO:teuthology.orchestra.run.smithi157.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-06T03:56:04.885 INFO:teuthology.orchestra.run.smithi157.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T03:56:04.885 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T03:56:04.885 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-06T03:56:04.885 INFO:teuthology.orchestra.run.smithi157.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-06T03:56:04.885 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T03:56:04.885 INFO:teuthology.orchestra.run.smithi157.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T03:56:04.885 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-06T03:56:04.885 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-06T03:56:04.885 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-06T03:56:04.886 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-06T03:56:04.886 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-06T03:56:04.886 INFO:teuthology.orchestra.run.smithi157.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-06T03:56:04.886 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T03:56:04.886 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-06T03:56:04.886 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T03:56:04.886 INFO:teuthology.orchestra.run.smithi157.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T03:56:04.886 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T03:56:04.887 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.887 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.887 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.887 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T03:56:04.887 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T03:56:04.887 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T03:56:04.887 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T03:56:04.887 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T03:56:04.887 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T03:56:04.888 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-06T03:56:04.888 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T03:56:04.888 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T03:56:04.888 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T03:56:04.888 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T03:56:04.888 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T03:56:04.888 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-06T03:56:04.888 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T03:56:04.888 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T03:56:04.888 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T03:56:04.888 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-06T03:56:04.889 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-06T03:56:04.889 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T03:56:04.889 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T03:56:04.889 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T03:56:04.889 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T03:56:04.889 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T03:56:04.889 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T03:56:04.889 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-06T03:56:04.889 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.889 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.889 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T03:56:04.890 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T03:56:04.890 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.890 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T03:56:04.890 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T03:56:04.890 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-06T03:56:04.890 INFO:teuthology.orchestra.run.smithi157.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T03:56:04.890 INFO:teuthology.orchestra.run.smithi157.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T03:56:04.890 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T03:56:04.890 INFO:teuthology.orchestra.run.smithi157.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T03:56:04.890 INFO:teuthology.orchestra.run.smithi157.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T03:56:04.890 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T03:56:04.890 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T03:56:04.891 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T03:56:04.891 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T03:56:04.891 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T03:56:04.891 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.891 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.891 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:04.891 INFO:teuthology.orchestra.run.smithi157.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-06T03:56:04.891 INFO:teuthology.orchestra.run.smithi157.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T03:56:04.891 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-06T03:56:04.892 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T03:56:04.892 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-06T03:56:04.892 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:04.892 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-10-06T03:56:05.146 DEBUG:teuthology.parallel:result is None 2023-10-06T03:56:05.401 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T03:56:05.401 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 1/122 2023-10-06T03:56:05.401 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 2/122 2023-10-06T03:56:05.401 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 3/122 2023-10-06T03:56:05.401 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 4/122 2023-10-06T03:56:05.401 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1274.g68e53 5/122 2023-10-06T03:56:05.401 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T03:56:05.401 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 7/122 2023-10-06T03:56:05.401 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 8/122 2023-10-06T03:56:05.401 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 9/122 2023-10-06T03:56:05.402 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T03:56:05.402 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 11/122 2023-10-06T03:56:05.402 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 12/122 2023-10-06T03:56:05.402 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T03:56:05.402 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_ 14/122 2023-10-06T03:56:05.402 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 15/122 2023-10-06T03:56:05.402 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 16/122 2023-10-06T03:56:05.402 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_6 17/122 2023-10-06T03:56:05.403 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 18/122 2023-10-06T03:56:05.403 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 19/122 2023-10-06T03:56:05.403 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el 20/122 2023-10-06T03:56:05.403 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8. 21/122 2023-10-06T03:56:05.403 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_6 22/122 2023-10-06T03:56:05.403 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 23/122 2023-10-06T03:56:05.403 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 24/122 2023-10-06T03:56:05.403 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 25/122 2023-10-06T03:56:05.403 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 26/122 2023-10-06T03:56:05.403 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 27/122 2023-10-06T03:56:05.403 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 28/122 2023-10-06T03:56:05.403 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a. 29/122 2023-10-06T03:56:05.404 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 30/122 2023-10-06T03:56:05.411 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 31/122 2023-10-06T03:56:05.411 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 32/122 2023-10-06T03:56:05.411 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el 33/122 2023-10-06T03:56:05.411 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 34/122 2023-10-06T03:56:05.412 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.e 35/122 2023-10-06T03:56:05.412 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 36/122 2023-10-06T03:56:05.412 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 37/122 2023-10-06T03:56:05.412 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 38/122 2023-10-06T03:56:05.412 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-06T03:56:05.412 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-06T03:56:05.412 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-06T03:56:05.412 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-06T03:56:05.413 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-06T03:56:05.413 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-06T03:56:05.413 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-06T03:56:05.413 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-06T03:56:05.413 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-06T03:56:05.413 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-06T03:56:05.413 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-06T03:56:05.413 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-06T03:56:05.414 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-06T03:56:05.414 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-06T03:56:05.414 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-06T03:56:05.414 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-06T03:56:05.414 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-06T03:56:05.414 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-06T03:56:05.414 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-06T03:56:05.414 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-06T03:56:05.414 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-06T03:56:05.414 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-06T03:56:05.415 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-06T03:56:05.415 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-06T03:56:05.415 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-06T03:56:05.415 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-06T03:56:05.415 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-06T03:56:05.415 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-06T03:56:05.415 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-06T03:56:05.415 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-06T03:56:05.415 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-06T03:56:05.416 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-06T03:56:05.416 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-06T03:56:05.416 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 72/122 2023-10-06T03:56:05.416 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/122 2023-10-06T03:56:05.416 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 74/122 2023-10-06T03:56:05.416 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : jq-1.5-12.el8.x86_64 75/122 2023-10-06T03:56:05.416 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flex-2.6.1-9.el8.x86_64 76/122 2023-10-06T03:56:05.416 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/122 2023-10-06T03:56:05.416 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 78/122 2023-10-06T03:56:05.416 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 79/122 2023-10-06T03:56:05.416 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : bison-3.0.4-10.el8.x86_64 80/122 2023-10-06T03:56:05.416 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 81/122 2023-10-06T03:56:05.417 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 82/122 2023-10-06T03:56:05.417 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 83/122 2023-10-06T03:56:05.417 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 84/122 2023-10-06T03:56:05.417 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 85/122 2023-10-06T03:56:05.417 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 86/122 2023-10-06T03:56:05.417 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 87/122 2023-10-06T03:56:05.417 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 88/122 2023-10-06T03:56:05.417 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 89/122 2023-10-06T03:56:05.417 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 90/122 2023-10-06T03:56:05.418 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 91/122 2023-10-06T03:56:05.418 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 92/122 2023-10-06T03:56:05.418 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 93/122 2023-10-06T03:56:05.418 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 94/122 2023-10-06T03:56:05.418 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 95/122 2023-10-06T03:56:05.418 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : automake-1.16.1-7.el8.noarch 96/122 2023-10-06T03:56:05.418 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 97/122 2023-10-06T03:56:05.418 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 98/122 2023-10-06T03:56:05.418 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 99/122 2023-10-06T03:56:05.418 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 100/122 2023-10-06T03:56:05.419 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 101/122 2023-10-06T03:56:05.419 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 102/122 2023-10-06T03:56:05.419 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 103/122 2023-10-06T03:56:05.419 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 104/122 2023-10-06T03:56:05.419 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 105/122 2023-10-06T03:56:05.419 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 106/122 2023-10-06T03:56:05.419 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 107/122 2023-10-06T03:56:05.419 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 108/122 2023-10-06T03:56:05.419 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 109/122 2023-10-06T03:56:05.420 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 110/122 2023-10-06T03:56:05.420 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 111/122 2023-10-06T03:56:05.420 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 112/122 2023-10-06T03:56:05.420 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 113/122 2023-10-06T03:56:05.420 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 114/122 2023-10-06T03:56:05.420 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 115/122 2023-10-06T03:56:05.420 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 116/122 2023-10-06T03:56:05.420 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 117/122 2023-10-06T03:56:05.420 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 118/122 2023-10-06T03:56:05.420 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 119/122 2023-10-06T03:56:05.420 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-06T03:56:05.421 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 121/122 2023-10-06T03:56:07.797 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T03:56:07.797 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:45 ago on Fri 06 Oct 2023 03:54:22 AM UTC. 2023-10-06T03:56:11.452 INFO:teuthology.orchestra.run.smithi175.stdout:Installed products updated. 2023-10-06T03:56:11.582 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:11.582 INFO:teuthology.orchestra.run.smithi175.stdout:Upgraded: 2023-10-06T03:56:11.582 INFO:teuthology.orchestra.run.smithi175.stdout: librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.582 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.582 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-10-06T03:56:11.582 INFO:teuthology.orchestra.run.smithi175.stdout: automake-1.16.1-7.el8.noarch 2023-10-06T03:56:11.582 INFO:teuthology.orchestra.run.smithi175.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T03:56:11.582 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.583 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.583 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.583 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.583 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T03:56:11.583 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.583 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.583 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.583 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T03:56:11.583 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T03:56:11.583 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T03:56:11.583 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T03:56:11.583 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T03:56:11.584 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.584 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.584 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T03:56:11.584 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.584 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.584 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.584 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T03:56:11.584 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T03:56:11.584 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.584 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T03:56:11.584 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-06T03:56:11.584 INFO:teuthology.orchestra.run.smithi175.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T03:56:11.585 INFO:teuthology.orchestra.run.smithi175.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-06T03:56:11.585 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T03:56:11.585 INFO:teuthology.orchestra.run.smithi175.stdout: jq-1.5-12.el8.x86_64 2023-10-06T03:56:11.585 INFO:teuthology.orchestra.run.smithi175.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-06T03:56:11.585 INFO:teuthology.orchestra.run.smithi175.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-06T03:56:11.585 INFO:teuthology.orchestra.run.smithi175.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T03:56:11.585 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.585 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.585 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.585 INFO:teuthology.orchestra.run.smithi175.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-06T03:56:11.585 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-06T03:56:11.585 INFO:teuthology.orchestra.run.smithi175.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T03:56:11.586 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-06T03:56:11.586 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-06T03:56:11.586 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.586 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.586 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-06T03:56:11.586 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.586 INFO:teuthology.orchestra.run.smithi175.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-06T03:56:11.586 INFO:teuthology.orchestra.run.smithi175.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-06T03:56:11.586 INFO:teuthology.orchestra.run.smithi175.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T03:56:11.586 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T03:56:11.586 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-06T03:56:11.586 INFO:teuthology.orchestra.run.smithi175.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-06T03:56:11.587 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T03:56:11.587 INFO:teuthology.orchestra.run.smithi175.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T03:56:11.587 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-06T03:56:11.587 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-06T03:56:11.587 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-06T03:56:11.587 INFO:teuthology.orchestra.run.smithi175.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-06T03:56:11.587 INFO:teuthology.orchestra.run.smithi175.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-06T03:56:11.587 INFO:teuthology.orchestra.run.smithi175.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-06T03:56:11.587 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T03:56:11.587 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-06T03:56:11.587 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T03:56:11.587 INFO:teuthology.orchestra.run.smithi175.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T03:56:11.588 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T03:56:11.588 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.588 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.588 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.588 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T03:56:11.588 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T03:56:11.588 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T03:56:11.588 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T03:56:11.588 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T03:56:11.588 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T03:56:11.588 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-06T03:56:11.588 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T03:56:11.589 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T03:56:11.589 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T03:56:11.589 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T03:56:11.589 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T03:56:11.589 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-06T03:56:11.589 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T03:56:11.589 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T03:56:11.589 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T03:56:11.589 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-06T03:56:11.589 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-06T03:56:11.589 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T03:56:11.589 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T03:56:11.590 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T03:56:11.590 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T03:56:11.590 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T03:56:11.590 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T03:56:11.590 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-06T03:56:11.590 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.590 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.590 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T03:56:11.590 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T03:56:11.590 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.590 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T03:56:11.590 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T03:56:11.591 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-06T03:56:11.591 INFO:teuthology.orchestra.run.smithi175.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T03:56:11.591 INFO:teuthology.orchestra.run.smithi175.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T03:56:11.591 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T03:56:11.591 INFO:teuthology.orchestra.run.smithi175.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T03:56:11.591 INFO:teuthology.orchestra.run.smithi175.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T03:56:11.591 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T03:56:11.591 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T03:56:11.591 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T03:56:11.591 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T03:56:11.591 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T03:56:11.591 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.592 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.592 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T03:56:11.592 INFO:teuthology.orchestra.run.smithi175.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-06T03:56:11.592 INFO:teuthology.orchestra.run.smithi175.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T03:56:11.592 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-06T03:56:11.592 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T03:56:11.592 INFO:teuthology.orchestra.run.smithi175.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-06T03:56:11.593 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:11.593 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-10-06T03:56:11.780 DEBUG:teuthology.parallel:result is None 2023-10-06T03:56:11.781 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T03:56:11.940 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T03:56:11.964 INFO:teuthology.orchestra.run.smithi157.stdout:17.2.6-1274.g68e53a4a.el8 2023-10-06T03:56:11.965 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1274.g68e53a4a.el8 2023-10-06T03:56:11.965 INFO:teuthology.task.install:The correct ceph version 17.2.6-1274.g68e53a4a is installed. 2023-10-06T03:56:11.965 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T03:56:12.101 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T03:56:12.121 INFO:teuthology.orchestra.run.smithi175.stdout:17.2.6-1274.g68e53a4a.el8 2023-10-06T03:56:12.122 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1274.g68e53a4a.el8 2023-10-06T03:56:12.122 INFO:teuthology.task.install:The correct ceph version 17.2.6-1274.g68e53a4a is installed. 2023-10-06T03:56:12.122 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-06T03:56:12.123 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-06T03:56:12.123 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T03:56:12.163 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-10-06T03:56:12.163 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T03:56:12.199 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-06T03:56:12.199 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-06T03:56:12.199 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T03:56:12.229 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T03:56:12.295 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-10-06T03:56:12.295 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T03:56:12.318 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T03:56:12.382 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-06T03:56:12.382 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-06T03:56:12.382 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T03:56:12.407 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T03:56:12.471 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-10-06T03:56:12.472 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T03:56:12.495 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T03:56:12.557 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-06T03:56:12.558 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-06T03:56:12.558 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T03:56:12.583 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T03:56:12.647 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-10-06T03:56:12.647 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T03:56:12.670 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T03:56:12.733 INFO:teuthology.run_tasks:Running task ceph... 2023-10-06T03:56:12.798 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-06T03:56:12.799 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 777 /var/log/ceph 2023-10-06T03:56:12.801 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 777 /var/log/ceph 2023-10-06T03:56:12.823 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-06T03:56:12.823 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T03:56:12.865 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T03:56:12.887 INFO:tasks.ceph:Creating extra log directories... 2023-10-06T03:56:12.888 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T03:56:12.929 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T03:56:12.959 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-06T03:56:12.959 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-06T03:56:12.959 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-06T03:56:12.960 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-06T03:56:12.960 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T03:56:13.006 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T03:56:13.110 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-06T03:56:13.134 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-10-06T03:56:13.134 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T03:56:13.148 DEBUG:teuthology.orchestra.run.smithi175:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T03:56:13.257 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-06T03:56:13.301 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-06T03:56:13.301 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': False, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'Replacing daemon mds'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-06T03:56:13.302 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412234', 'branch': 'wip-yuri3-testing-2023-10-05-0943-quincy', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/snaps}}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7412234', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.4', 'overrides': {'admin_socket': {'branch': 'wip-yuri3-testing-2023-10-05-0943-quincy'}, 'ceph': {'cephfs': {'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': False, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'Replacing daemon mds'], 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri3-testing-2023-10-05-0943-quincy', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 4002, 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-yuri3-testing-2023-10-05-0943-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'targets': {'smithi157.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBqwONQQ9SRgP7wl/eGNxCKZEU6YhxyvvULbzoPkEvm/SMYB0MAzt8PvgJdbr+t7Aux/At50nJ+Dt9rZoU+QuSw=', 'smithi175.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMD1lY8k/JMCThp+vPfiNZh2sGw9juoPiQx7FaD+ZXMGbasjgL2nRuw9URi7/tDh0qjtjGWE635fMHv52/5FaFg='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}}, {'ceph': None}, {'kclient': None}, {'mds_thrash': None}, {'workunit': {'clients': {'all': ['fs/snaps']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-05_23:54:57', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-06T03:56:13.302 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T03:56:13.305 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T03:56:13.350 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T03:56:13.359 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T03:56:13.413 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-06T03:56:13.413 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T03:56:13.436 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T03:56:13.436 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_1 2023-10-06T03:56:13.491 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T03:56:13.491 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T03:56:13.491 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 218420 Links: 1 2023-10-06T03:56:13.491 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T03:56:13.491 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T03:56:13.491 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2023-10-06 03:55:59.898612770 +0000 2023-10-06T03:56:13.491 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2023-10-06 03:53:35.167613022 +0000 2023-10-06T03:56:13.491 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2023-10-06 03:53:35.167613022 +0000 2023-10-06T03:56:13.491 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2023-10-06T03:56:13.492 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T03:56:13.554 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-10-06T03:56:13.554 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-10-06T03:56:13.554 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000123869 s, 4.1 MB/s 2023-10-06T03:56:13.556 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T03:56:13.612 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_2 2023-10-06T03:56:13.667 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T03:56:13.667 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T03:56:13.667 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 219747 Links: 1 2023-10-06T03:56:13.667 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T03:56:13.667 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T03:56:13.667 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2023-10-06 03:55:59.898612770 +0000 2023-10-06T03:56:13.667 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2023-10-06 03:53:35.649603998 +0000 2023-10-06T03:56:13.668 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2023-10-06 03:53:35.649603998 +0000 2023-10-06T03:56:13.668 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2023-10-06T03:56:13.668 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T03:56:13.729 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-10-06T03:56:13.730 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-10-06T03:56:13.730 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.00010222 s, 5.0 MB/s 2023-10-06T03:56:13.731 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T03:56:13.786 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_3 2023-10-06T03:56:13.841 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T03:56:13.841 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T03:56:13.841 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 219906 Links: 1 2023-10-06T03:56:13.841 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T03:56:13.841 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T03:56:13.841 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2023-10-06 03:55:59.898612770 +0000 2023-10-06T03:56:13.841 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2023-10-06 03:53:36.125595087 +0000 2023-10-06T03:56:13.842 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2023-10-06 03:53:36.125595087 +0000 2023-10-06T03:56:13.842 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2023-10-06T03:56:13.842 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T03:56:13.904 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-10-06T03:56:13.904 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-10-06T03:56:13.904 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.0001088 s, 4.7 MB/s 2023-10-06T03:56:13.905 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T03:56:13.961 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_4 2023-10-06T03:56:14.016 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T03:56:14.016 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T03:56:14.016 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 216040 Links: 1 2023-10-06T03:56:14.016 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T03:56:14.016 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T03:56:14.017 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2023-10-06 03:55:59.898612770 +0000 2023-10-06T03:56:14.017 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2023-10-06 03:53:36.627585689 +0000 2023-10-06T03:56:14.017 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2023-10-06 03:53:36.627585689 +0000 2023-10-06T03:56:14.017 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2023-10-06T03:56:14.017 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T03:56:14.079 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-10-06T03:56:14.079 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-10-06T03:56:14.079 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000112045 s, 4.6 MB/s 2023-10-06T03:56:14.080 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T03:56:14.136 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T03:56:14.136 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-10-06T03:56:14.137 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T03:56:14.152 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T03:56:14.152 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_1 2023-10-06T03:56:14.206 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T03:56:14.207 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T03:56:14.207 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 224522 Links: 1 2023-10-06T03:56:14.207 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T03:56:14.207 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T03:56:14.207 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2023-10-06 03:56:05.042274876 +0000 2023-10-06T03:56:14.207 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2023-10-06 03:53:35.439000655 +0000 2023-10-06T03:56:14.207 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2023-10-06 03:53:35.439000655 +0000 2023-10-06T03:56:14.207 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: - 2023-10-06T03:56:14.208 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T03:56:14.270 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2023-10-06T03:56:14.270 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2023-10-06T03:56:14.270 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000238077 s, 2.2 MB/s 2023-10-06T03:56:14.271 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T03:56:14.327 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_2 2023-10-06T03:56:14.381 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T03:56:14.381 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T03:56:14.382 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 225286 Links: 1 2023-10-06T03:56:14.382 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T03:56:14.382 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T03:56:14.382 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2023-10-06 03:56:05.042274876 +0000 2023-10-06T03:56:14.382 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2023-10-06 03:53:35.936993172 +0000 2023-10-06T03:56:14.382 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2023-10-06 03:53:35.936993172 +0000 2023-10-06T03:56:14.382 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: - 2023-10-06T03:56:14.382 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T03:56:14.444 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2023-10-06T03:56:14.444 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2023-10-06T03:56:14.444 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000222196 s, 2.3 MB/s 2023-10-06T03:56:14.445 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T03:56:14.501 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_3 2023-10-06T03:56:14.556 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T03:56:14.556 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T03:56:14.556 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 225406 Links: 1 2023-10-06T03:56:14.557 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T03:56:14.557 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T03:56:14.557 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2023-10-06 03:56:05.042274876 +0000 2023-10-06T03:56:14.557 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2023-10-06 03:53:36.410986084 +0000 2023-10-06T03:56:14.557 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2023-10-06 03:53:36.410986084 +0000 2023-10-06T03:56:14.557 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: - 2023-10-06T03:56:14.557 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T03:56:14.619 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2023-10-06T03:56:14.619 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2023-10-06T03:56:14.619 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000219144 s, 2.3 MB/s 2023-10-06T03:56:14.620 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T03:56:14.676 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_4 2023-10-06T03:56:14.730 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T03:56:14.730 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T03:56:14.730 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 226341 Links: 1 2023-10-06T03:56:14.730 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T03:56:14.730 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T03:56:14.730 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2023-10-06 03:56:05.043274861 +0000 2023-10-06T03:56:14.730 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2023-10-06 03:53:36.926978369 +0000 2023-10-06T03:56:14.731 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2023-10-06 03:53:36.926978369 +0000 2023-10-06T03:56:14.733 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: - 2023-10-06T03:56:14.734 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T03:56:14.792 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2023-10-06T03:56:14.792 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2023-10-06T03:56:14.792 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000205397 s, 2.5 MB/s 2023-10-06T03:56:14.793 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T03:56:14.849 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T03:56:14.849 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi157.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi175.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T03:56:14.849 INFO:tasks.ceph:Generating config... 2023-10-06T03:56:14.850 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-06T03:56:14.850 INFO:tasks.ceph:[client] debug client = 20 2023-10-06T03:56:14.850 INFO:tasks.ceph:[client] debug ms = 1 2023-10-06T03:56:14.850 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-06T03:56:14.851 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-06T03:56:14.851 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-10-06T03:56:14.851 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-06T03:56:14.851 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-06T03:56:14.851 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2023-10-06T03:56:14.851 INFO:tasks.ceph:[mds] mds bal merge size = 5 2023-10-06T03:56:14.851 INFO:tasks.ceph:[mds] mds bal split bits = 3 2023-10-06T03:56:14.851 INFO:tasks.ceph:[mds] mds bal split size = 100 2023-10-06T03:56:14.851 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-06T03:56:14.851 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-06T03:56:14.851 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = False 2023-10-06T03:56:14.851 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-06T03:56:14.852 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-06T03:56:14.852 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2023-10-06T03:56:14.852 INFO:tasks.ceph:[mds] mon client directed command retry = 5 2023-10-06T03:56:14.852 INFO:tasks.ceph:[mds] ms inject delay max = 1 2023-10-06T03:56:14.852 INFO:tasks.ceph:[mds] ms inject delay probability = 0.005 2023-10-06T03:56:14.852 INFO:tasks.ceph:[mds] ms inject delay type = client mds osd 2023-10-06T03:56:14.852 INFO:tasks.ceph:[mds] ms inject socket failures = 2500 2023-10-06T03:56:14.852 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-06T03:56:14.852 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-06T03:56:14.852 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-06T03:56:14.852 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-06T03:56:14.852 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-06T03:56:14.853 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-06T03:56:14.853 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-06T03:56:14.853 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-06T03:56:14.853 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-06T03:56:14.853 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-06T03:56:14.853 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2023-10-06T03:56:14.853 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-06T03:56:14.853 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-10-06T03:56:14.853 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-10-06T03:56:14.853 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-06T03:56:14.853 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-06T03:56:14.853 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-10-06T03:56:14.854 INFO:tasks.ceph:[osd] mon client directed command retry = 5 2023-10-06T03:56:14.854 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-06T03:56:14.854 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-06T03:56:14.854 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-06T03:56:14.854 INFO:tasks.ceph:[osd] ms inject delay max = 1 2023-10-06T03:56:14.854 INFO:tasks.ceph:[osd] ms inject delay probability = 0.005 2023-10-06T03:56:14.854 INFO:tasks.ceph:[osd] ms inject delay type = client mds 2023-10-06T03:56:14.854 INFO:tasks.ceph:[osd] ms inject socket failures = 2500 2023-10-06T03:56:14.855 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-06T03:56:14.855 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2023-10-06T03:56:14.855 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-06T03:56:14.855 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-06T03:56:14.855 INFO:tasks.ceph:Setting up mon.a... 2023-10-06T03:56:14.855 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-06T03:56:14.901 INFO:teuthology.orchestra.run.smithi157.stdout:creating /etc/ceph/ceph.keyring 2023-10-06T03:56:14.903 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-06T03:56:14.943 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T03:56:15.007 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.157'), ('mon.b', '172.21.15.175'), ('mon.c', '[v2:172.21.15.175:3301,v1:172.21.15.175:6790]')] 2023-10-06T03:56:15.008 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.157,172.21.15.175,[v2:172.21.15.175:3301,v1:172.21.15.175:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': False, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-06T03:56:15.009 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-06T03:56:15.009 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T03:56:15.062 DEBUG:teuthology.orchestra.run.smithi157:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.157 --add b 172.21.15.175 --addv c '[v2:172.21.15.175:3301,v1:172.21.15.175:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:56:15.144 INFO:teuthology.orchestra.run.smithi157.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:56:15.144 INFO:teuthology.orchestra.run.smithi157.stdout:monmaptool: generated fsid 28664a7a-fe30-4efe-be5a-6abf9318b700 2023-10-06T03:56:15.144 INFO:teuthology.orchestra.run.smithi157.stdout:setting min_mon_release = octopus 2023-10-06T03:56:15.144 INFO:teuthology.orchestra.run.smithi157.stdout:epoch 0 2023-10-06T03:56:15.144 INFO:teuthology.orchestra.run.smithi157.stdout:fsid 28664a7a-fe30-4efe-be5a-6abf9318b700 2023-10-06T03:56:15.144 INFO:teuthology.orchestra.run.smithi157.stdout:last_changed 2023-10-06T03:56:15.143627+0000 2023-10-06T03:56:15.145 INFO:teuthology.orchestra.run.smithi157.stdout:created 2023-10-06T03:56:15.143627+0000 2023-10-06T03:56:15.145 INFO:teuthology.orchestra.run.smithi157.stdout:min_mon_release 15 (octopus) 2023-10-06T03:56:15.145 INFO:teuthology.orchestra.run.smithi157.stdout:election_strategy: 1 2023-10-06T03:56:15.145 INFO:teuthology.orchestra.run.smithi157.stdout:0: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.a 2023-10-06T03:56:15.145 INFO:teuthology.orchestra.run.smithi157.stdout:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2023-10-06T03:56:15.145 INFO:teuthology.orchestra.run.smithi157.stdout:2: [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] mon.c 2023-10-06T03:56:15.145 INFO:teuthology.orchestra.run.smithi157.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-06T03:56:15.146 DEBUG:teuthology.orchestra.run.smithi157:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T03:56:15.201 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 28664a7a-fe30-4efe-be5a-6abf9318b700... 2023-10-06T03:56:15.202 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T03:56:15.243 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T03:56:15.284 INFO:teuthology.orchestra.run.smithi157.stdout:[global] 2023-10-06T03:56:15.284 INFO:teuthology.orchestra.run.smithi157.stdout: chdir = "" 2023-10-06T03:56:15.284 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T03:56:15.284 INFO:teuthology.orchestra.run.smithi157.stdout: auth supported = cephx 2023-10-06T03:56:15.284 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:15.284 INFO:teuthology.orchestra.run.smithi157.stdout: filestore xattr use omap = true 2023-10-06T03:56:15.285 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:15.285 INFO:teuthology.orchestra.run.smithi157.stdout: mon clock drift allowed = 1.000 2023-10-06T03:56:15.285 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:15.285 INFO:teuthology.orchestra.run.smithi157.stdout: osd crush chooseleaf type = 0 2023-10-06T03:56:15.285 INFO:teuthology.orchestra.run.smithi157.stdout: auth debug = true 2023-10-06T03:56:15.285 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:15.285 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on old message = true 2023-10-06T03:56:15.285 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on bug = true 2023-10-06T03:56:15.285 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:15.285 INFO:teuthology.orchestra.run.smithi157.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T03:56:15.285 INFO:teuthology.orchestra.run.smithi157.stdout: mon pg warn max object skew = 0 2023-10-06T03:56:15.286 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:15.286 INFO:teuthology.orchestra.run.smithi157.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T03:56:15.286 INFO:teuthology.orchestra.run.smithi157.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T03:56:15.286 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:15.286 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default size = 2 2023-10-06T03:56:15.286 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:15.286 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow primary affinity = true 2023-10-06T03:56:15.286 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow pg remap = true 2023-10-06T03:56:15.286 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on legacy crush tunables = false 2023-10-06T03:56:15.286 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on crush straw calc version zero = false 2023-10-06T03:56:15.286 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on no sortbitwise = false 2023-10-06T03:56:15.287 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on osd down out interval zero = false 2023-10-06T03:56:15.287 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on too few osds = false 2023-10-06T03:56:15.287 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T03:56:15.287 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T03:56:15.287 INFO:teuthology.orchestra.run.smithi157.stdout: mon_allow_pool_size_one = true 2023-10-06T03:56:15.287 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:15.287 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T03:56:15.287 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:15.287 INFO:teuthology.orchestra.run.smithi157.stdout: osd default data pool replay window = 5 2023-10-06T03:56:15.287 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:15.287 INFO:teuthology.orchestra.run.smithi157.stdout: mon allow pool delete = true 2023-10-06T03:56:15.288 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:15.288 INFO:teuthology.orchestra.run.smithi157.stdout: mon cluster log file level = debug 2023-10-06T03:56:15.288 INFO:teuthology.orchestra.run.smithi157.stdout: debug asserts on shutdown = true 2023-10-06T03:56:15.288 INFO:teuthology.orchestra.run.smithi157.stdout: mon health detail to clog = false 2023-10-06T03:56:15.288 INFO:teuthology.orchestra.run.smithi157.stdout: mon host = "172.21.15.157,172.21.15.175,[v2:172.21.15.175:3301,v1:172.21.15.175:6790]" 2023-10-06T03:56:15.288 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on skipped message = False 2023-10-06T03:56:15.288 INFO:teuthology.orchestra.run.smithi157.stdout: fsid = 28664a7a-fe30-4efe-be5a-6abf9318b700 2023-10-06T03:56:15.288 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:15.288 INFO:teuthology.orchestra.run.smithi157.stdout:[osd] 2023-10-06T03:56:15.288 INFO:teuthology.orchestra.run.smithi157.stdout: osd journal size = 100 2023-10-06T03:56:15.288 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:15.288 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub load threshold = 5.0 2023-10-06T03:56:15.289 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub max interval = 600 2023-10-06T03:56:15.289 INFO:teuthology.orchestra.run.smithi157.stdout: osd mclock profile = high_recovery_ops 2023-10-06T03:56:15.289 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:15.289 INFO:teuthology.orchestra.run.smithi157.stdout: osd recover clone overlap = true 2023-10-06T03:56:15.289 INFO:teuthology.orchestra.run.smithi157.stdout: osd recovery max chunk = 1048576 2023-10-06T03:56:15.289 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:15.289 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug shutdown = true 2023-10-06T03:56:15.289 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug op order = true 2023-10-06T03:56:15.289 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug verify stray on activate = true 2023-10-06T03:56:15.289 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:15.289 INFO:teuthology.orchestra.run.smithi157.stdout: osd open classes on start = true 2023-10-06T03:56:15.290 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug pg log writeout = true 2023-10-06T03:56:15.290 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:15.290 INFO:teuthology.orchestra.run.smithi157.stdout: osd deep scrub update digest min age = 30 2023-10-06T03:56:15.290 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:15.290 INFO:teuthology.orchestra.run.smithi157.stdout: osd map max advance = 10 2023-10-06T03:56:15.290 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:15.290 INFO:teuthology.orchestra.run.smithi157.stdout: journal zero on create = true 2023-10-06T03:56:15.290 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:15.290 INFO:teuthology.orchestra.run.smithi157.stdout: filestore ondisk finisher threads = 3 2023-10-06T03:56:15.290 INFO:teuthology.orchestra.run.smithi157.stdout: filestore apply finisher threads = 3 2023-10-06T03:56:15.291 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:15.291 INFO:teuthology.orchestra.run.smithi157.stdout: bdev debug aio = true 2023-10-06T03:56:15.291 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug misdirected ops = true 2023-10-06T03:56:15.291 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore block size = 96636764160 2023-10-06T03:56:15.291 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore compression mode = aggressive 2023-10-06T03:56:15.291 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore fsck on mount = True 2023-10-06T03:56:15.291 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluefs = 20 2023-10-06T03:56:15.291 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluestore = 20 2023-10-06T03:56:15.291 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-10-06T03:56:15.291 INFO:teuthology.orchestra.run.smithi157.stdout: debug osd = 20 2023-10-06T03:56:15.291 INFO:teuthology.orchestra.run.smithi157.stdout: debug rocksdb = 10 2023-10-06T03:56:15.291 INFO:teuthology.orchestra.run.smithi157.stdout: mon client directed command retry = 5 2023-10-06T03:56:15.292 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T03:56:15.292 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd full ratio = 0.9 2023-10-06T03:56:15.292 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd nearfull ratio = 0.8 2023-10-06T03:56:15.292 INFO:teuthology.orchestra.run.smithi157.stdout: ms inject delay max = 1 2023-10-06T03:56:15.292 INFO:teuthology.orchestra.run.smithi157.stdout: ms inject delay probability = 0.005 2023-10-06T03:56:15.292 INFO:teuthology.orchestra.run.smithi157.stdout: ms inject delay type = client mds 2023-10-06T03:56:15.292 INFO:teuthology.orchestra.run.smithi157.stdout: ms inject socket failures = 2500 2023-10-06T03:56:15.292 INFO:teuthology.orchestra.run.smithi157.stdout: osd failsafe full ratio = 0.95 2023-10-06T03:56:15.292 INFO:teuthology.orchestra.run.smithi157.stdout: osd max markdown count = 1000 2023-10-06T03:56:15.292 INFO:teuthology.orchestra.run.smithi157.stdout: osd objectstore = bluestore 2023-10-06T03:56:15.292 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2023-10-06T03:56:15.292 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:15.293 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr] 2023-10-06T03:56:15.293 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-10-06T03:56:15.293 INFO:teuthology.orchestra.run.smithi157.stdout: debug mgr = 20 2023-10-06T03:56:15.293 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2023-10-06T03:56:15.293 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2023-10-06T03:56:15.293 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2023-10-06T03:56:15.293 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2023-10-06T03:56:15.293 INFO:teuthology.orchestra.run.smithi157.stdout: mgr/telemetry/nag = false 2023-10-06T03:56:15.293 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:15.293 INFO:teuthology.orchestra.run.smithi157.stdout:[mon] 2023-10-06T03:56:15.293 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-10-06T03:56:15.294 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2023-10-06T03:56:15.294 INFO:teuthology.orchestra.run.smithi157.stdout: debug paxos = 20 2023-10-06T03:56:15.294 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2023-10-06T03:56:15.294 INFO:teuthology.orchestra.run.smithi157.stdout: mon data avail warn = 5 2023-10-06T03:56:15.294 INFO:teuthology.orchestra.run.smithi157.stdout: mon mgr mkfs grace = 240 2023-10-06T03:56:15.294 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2023-10-06T03:56:15.294 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd reporter subtree level = osd 2023-10-06T03:56:15.294 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd prime pg temp = true 2023-10-06T03:56:15.294 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2023-10-06T03:56:15.294 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:15.294 INFO:teuthology.orchestra.run.smithi157.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T03:56:15.295 INFO:teuthology.orchestra.run.smithi157.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T03:56:15.295 INFO:teuthology.orchestra.run.smithi157.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T03:56:15.295 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:15.295 INFO:teuthology.orchestra.run.smithi157.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T03:56:15.295 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T03:56:15.295 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T03:56:15.295 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:15.295 INFO:teuthology.orchestra.run.smithi157.stdout: # 1m isn't quite enough 2023-10-06T03:56:15.295 INFO:teuthology.orchestra.run.smithi157.stdout: mon_down_mkfs_grace = 2m 2023-10-06T03:56:15.295 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:15.295 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_filestore_osds = false 2023-10-06T03:56:15.295 INFO:teuthology.orchestra.run.smithi157.stdout: mon op complaint time = 120 2023-10-06T03:56:15.296 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:15.296 INFO:teuthology.orchestra.run.smithi157.stdout:[client] 2023-10-06T03:56:15.296 INFO:teuthology.orchestra.run.smithi157.stdout: rgw cache enabled = true 2023-10-06T03:56:15.296 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable ops log = true 2023-10-06T03:56:15.296 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable usage log = true 2023-10-06T03:56:15.296 INFO:teuthology.orchestra.run.smithi157.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T03:56:15.296 INFO:teuthology.orchestra.run.smithi157.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T03:56:15.296 INFO:teuthology.orchestra.run.smithi157.stdout: client mount timeout = 600 2023-10-06T03:56:15.296 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2023-10-06T03:56:15.296 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-10-06T03:56:15.296 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2023-10-06T03:56:15.296 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2023-10-06T03:56:15.297 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.a] 2023-10-06T03:56:15.297 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.b] 2023-10-06T03:56:15.297 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.c] 2023-10-06T03:56:15.297 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2023-10-06T03:56:15.297 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2023-10-06T03:56:15.297 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.e] 2023-10-06T03:56:15.297 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2023-10-06T03:56:15.297 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2023-10-06T03:56:15.297 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.f] 2023-10-06T03:56:15.297 INFO:teuthology.orchestra.run.smithi157.stdout:[mds] 2023-10-06T03:56:15.297 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds = 20 2023-10-06T03:56:15.297 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-10-06T03:56:15.298 INFO:teuthology.orchestra.run.smithi157.stdout: mds bal fragment size max = 10000 2023-10-06T03:56:15.298 INFO:teuthology.orchestra.run.smithi157.stdout: mds bal merge size = 5 2023-10-06T03:56:15.298 INFO:teuthology.orchestra.run.smithi157.stdout: mds bal split bits = 3 2023-10-06T03:56:15.298 INFO:teuthology.orchestra.run.smithi157.stdout: mds bal split size = 100 2023-10-06T03:56:15.298 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug frag = True 2023-10-06T03:56:15.298 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug scatterstat = True 2023-10-06T03:56:15.298 INFO:teuthology.orchestra.run.smithi157.stdout: mds oft prefetch dirfrags = False 2023-10-06T03:56:15.298 INFO:teuthology.orchestra.run.smithi157.stdout: mds op complaint time = 180 2023-10-06T03:56:15.298 INFO:teuthology.orchestra.run.smithi157.stdout: mds verify scatter = True 2023-10-06T03:56:15.298 INFO:teuthology.orchestra.run.smithi157.stdout: mds_sleep_rank_change = 5000000.0 2023-10-06T03:56:15.298 INFO:teuthology.orchestra.run.smithi157.stdout: mon client directed command retry = 5 2023-10-06T03:56:15.299 INFO:teuthology.orchestra.run.smithi157.stdout: ms inject delay max = 1 2023-10-06T03:56:15.299 INFO:teuthology.orchestra.run.smithi157.stdout: ms inject delay probability = 0.005 2023-10-06T03:56:15.299 INFO:teuthology.orchestra.run.smithi157.stdout: ms inject delay type = client mds osd 2023-10-06T03:56:15.299 INFO:teuthology.orchestra.run.smithi157.stdout: ms inject socket failures = 2500 2023-10-06T03:56:15.299 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2023-10-06T03:56:15.299 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2023-10-06T03:56:15.299 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2023-10-06T03:56:15.300 INFO:teuthology.orchestra.run.smithi175.stdout:[global] 2023-10-06T03:56:15.301 INFO:teuthology.orchestra.run.smithi175.stdout: chdir = "" 2023-10-06T03:56:15.301 INFO:teuthology.orchestra.run.smithi175.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T03:56:15.301 INFO:teuthology.orchestra.run.smithi175.stdout: auth supported = cephx 2023-10-06T03:56:15.301 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:15.301 INFO:teuthology.orchestra.run.smithi175.stdout: filestore xattr use omap = true 2023-10-06T03:56:15.301 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:15.301 INFO:teuthology.orchestra.run.smithi175.stdout: mon clock drift allowed = 1.000 2023-10-06T03:56:15.301 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:15.301 INFO:teuthology.orchestra.run.smithi175.stdout: osd crush chooseleaf type = 0 2023-10-06T03:56:15.301 INFO:teuthology.orchestra.run.smithi175.stdout: auth debug = true 2023-10-06T03:56:15.301 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:15.302 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on old message = true 2023-10-06T03:56:15.302 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on bug = true 2023-10-06T03:56:15.302 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:15.302 INFO:teuthology.orchestra.run.smithi175.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T03:56:15.302 INFO:teuthology.orchestra.run.smithi175.stdout: mon pg warn max object skew = 0 2023-10-06T03:56:15.302 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:15.302 INFO:teuthology.orchestra.run.smithi175.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T03:56:15.302 INFO:teuthology.orchestra.run.smithi175.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T03:56:15.302 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:15.302 INFO:teuthology.orchestra.run.smithi175.stdout: osd pool default size = 2 2023-10-06T03:56:15.302 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:15.303 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd allow primary affinity = true 2023-10-06T03:56:15.303 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd allow pg remap = true 2023-10-06T03:56:15.303 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on legacy crush tunables = false 2023-10-06T03:56:15.303 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on crush straw calc version zero = false 2023-10-06T03:56:15.303 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on no sortbitwise = false 2023-10-06T03:56:15.303 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on osd down out interval zero = false 2023-10-06T03:56:15.303 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on too few osds = false 2023-10-06T03:56:15.303 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T03:56:15.303 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T03:56:15.303 INFO:teuthology.orchestra.run.smithi175.stdout: mon_allow_pool_size_one = true 2023-10-06T03:56:15.303 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:15.303 INFO:teuthology.orchestra.run.smithi175.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T03:56:15.304 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:15.304 INFO:teuthology.orchestra.run.smithi175.stdout: osd default data pool replay window = 5 2023-10-06T03:56:15.304 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:15.304 INFO:teuthology.orchestra.run.smithi175.stdout: mon allow pool delete = true 2023-10-06T03:56:15.304 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:15.304 INFO:teuthology.orchestra.run.smithi175.stdout: mon cluster log file level = debug 2023-10-06T03:56:15.304 INFO:teuthology.orchestra.run.smithi175.stdout: debug asserts on shutdown = true 2023-10-06T03:56:15.304 INFO:teuthology.orchestra.run.smithi175.stdout: mon health detail to clog = false 2023-10-06T03:56:15.304 INFO:teuthology.orchestra.run.smithi175.stdout: mon host = "172.21.15.157,172.21.15.175,[v2:172.21.15.175:3301,v1:172.21.15.175:6790]" 2023-10-06T03:56:15.304 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on skipped message = False 2023-10-06T03:56:15.304 INFO:teuthology.orchestra.run.smithi175.stdout: fsid = 28664a7a-fe30-4efe-be5a-6abf9318b700 2023-10-06T03:56:15.304 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:15.305 INFO:teuthology.orchestra.run.smithi175.stdout:[osd] 2023-10-06T03:56:15.305 INFO:teuthology.orchestra.run.smithi175.stdout: osd journal size = 100 2023-10-06T03:56:15.305 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:15.305 INFO:teuthology.orchestra.run.smithi175.stdout: osd scrub load threshold = 5.0 2023-10-06T03:56:15.305 INFO:teuthology.orchestra.run.smithi175.stdout: osd scrub max interval = 600 2023-10-06T03:56:15.305 INFO:teuthology.orchestra.run.smithi175.stdout: osd mclock profile = high_recovery_ops 2023-10-06T03:56:15.305 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:15.305 INFO:teuthology.orchestra.run.smithi175.stdout: osd recover clone overlap = true 2023-10-06T03:56:15.305 INFO:teuthology.orchestra.run.smithi175.stdout: osd recovery max chunk = 1048576 2023-10-06T03:56:15.305 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:15.305 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug shutdown = true 2023-10-06T03:56:15.305 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug op order = true 2023-10-06T03:56:15.306 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug verify stray on activate = true 2023-10-06T03:56:15.306 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:15.306 INFO:teuthology.orchestra.run.smithi175.stdout: osd open classes on start = true 2023-10-06T03:56:15.306 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug pg log writeout = true 2023-10-06T03:56:15.306 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:15.306 INFO:teuthology.orchestra.run.smithi175.stdout: osd deep scrub update digest min age = 30 2023-10-06T03:56:15.306 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:15.306 INFO:teuthology.orchestra.run.smithi175.stdout: osd map max advance = 10 2023-10-06T03:56:15.306 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:15.306 INFO:teuthology.orchestra.run.smithi175.stdout: journal zero on create = true 2023-10-06T03:56:15.306 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:15.307 INFO:teuthology.orchestra.run.smithi175.stdout: filestore ondisk finisher threads = 3 2023-10-06T03:56:15.307 INFO:teuthology.orchestra.run.smithi175.stdout: filestore apply finisher threads = 3 2023-10-06T03:56:15.307 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:15.307 INFO:teuthology.orchestra.run.smithi175.stdout: bdev debug aio = true 2023-10-06T03:56:15.307 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug misdirected ops = true 2023-10-06T03:56:15.307 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore block size = 96636764160 2023-10-06T03:56:15.307 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore compression mode = aggressive 2023-10-06T03:56:15.307 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore fsck on mount = True 2023-10-06T03:56:15.307 INFO:teuthology.orchestra.run.smithi175.stdout: debug bluefs = 20 2023-10-06T03:56:15.307 INFO:teuthology.orchestra.run.smithi175.stdout: debug bluestore = 20 2023-10-06T03:56:15.307 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2023-10-06T03:56:15.307 INFO:teuthology.orchestra.run.smithi175.stdout: debug osd = 20 2023-10-06T03:56:15.308 INFO:teuthology.orchestra.run.smithi175.stdout: debug rocksdb = 10 2023-10-06T03:56:15.308 INFO:teuthology.orchestra.run.smithi175.stdout: mon client directed command retry = 5 2023-10-06T03:56:15.308 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T03:56:15.308 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd full ratio = 0.9 2023-10-06T03:56:15.308 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd nearfull ratio = 0.8 2023-10-06T03:56:15.308 INFO:teuthology.orchestra.run.smithi175.stdout: ms inject delay max = 1 2023-10-06T03:56:15.308 INFO:teuthology.orchestra.run.smithi175.stdout: ms inject delay probability = 0.005 2023-10-06T03:56:15.308 INFO:teuthology.orchestra.run.smithi175.stdout: ms inject delay type = client mds 2023-10-06T03:56:15.308 INFO:teuthology.orchestra.run.smithi175.stdout: ms inject socket failures = 2500 2023-10-06T03:56:15.308 INFO:teuthology.orchestra.run.smithi175.stdout: osd failsafe full ratio = 0.95 2023-10-06T03:56:15.308 INFO:teuthology.orchestra.run.smithi175.stdout: osd max markdown count = 1000 2023-10-06T03:56:15.308 INFO:teuthology.orchestra.run.smithi175.stdout: osd objectstore = bluestore 2023-10-06T03:56:15.309 INFO:teuthology.orchestra.run.smithi175.stdout: osd op complaint time = 180 2023-10-06T03:56:15.309 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:15.309 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr] 2023-10-06T03:56:15.309 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2023-10-06T03:56:15.309 INFO:teuthology.orchestra.run.smithi175.stdout: debug mgr = 20 2023-10-06T03:56:15.309 INFO:teuthology.orchestra.run.smithi175.stdout: debug mon = 20 2023-10-06T03:56:15.309 INFO:teuthology.orchestra.run.smithi175.stdout: debug auth = 20 2023-10-06T03:56:15.309 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min pgs per osd = 4 2023-10-06T03:56:15.309 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min bytes per osd = 10 2023-10-06T03:56:15.309 INFO:teuthology.orchestra.run.smithi175.stdout: mgr/telemetry/nag = false 2023-10-06T03:56:15.309 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:15.309 INFO:teuthology.orchestra.run.smithi175.stdout:[mon] 2023-10-06T03:56:15.310 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2023-10-06T03:56:15.310 INFO:teuthology.orchestra.run.smithi175.stdout: debug mon = 20 2023-10-06T03:56:15.310 INFO:teuthology.orchestra.run.smithi175.stdout: debug paxos = 20 2023-10-06T03:56:15.310 INFO:teuthology.orchestra.run.smithi175.stdout: debug auth = 20 2023-10-06T03:56:15.310 INFO:teuthology.orchestra.run.smithi175.stdout: mon data avail warn = 5 2023-10-06T03:56:15.310 INFO:teuthology.orchestra.run.smithi175.stdout: mon mgr mkfs grace = 240 2023-10-06T03:56:15.310 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min pgs per osd = 4 2023-10-06T03:56:15.310 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd reporter subtree level = osd 2023-10-06T03:56:15.310 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd prime pg temp = true 2023-10-06T03:56:15.310 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min bytes per osd = 10 2023-10-06T03:56:15.310 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:15.311 INFO:teuthology.orchestra.run.smithi175.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T03:56:15.311 INFO:teuthology.orchestra.run.smithi175.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T03:56:15.311 INFO:teuthology.orchestra.run.smithi175.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T03:56:15.311 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:15.311 INFO:teuthology.orchestra.run.smithi175.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T03:56:15.311 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T03:56:15.311 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T03:56:15.311 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:15.311 INFO:teuthology.orchestra.run.smithi175.stdout: # 1m isn't quite enough 2023-10-06T03:56:15.311 INFO:teuthology.orchestra.run.smithi175.stdout: mon_down_mkfs_grace = 2m 2023-10-06T03:56:15.311 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:15.311 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_filestore_osds = false 2023-10-06T03:56:15.312 INFO:teuthology.orchestra.run.smithi175.stdout: mon op complaint time = 120 2023-10-06T03:56:15.312 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-10-06T03:56:15.312 INFO:teuthology.orchestra.run.smithi175.stdout:[client] 2023-10-06T03:56:15.312 INFO:teuthology.orchestra.run.smithi175.stdout: rgw cache enabled = true 2023-10-06T03:56:15.312 INFO:teuthology.orchestra.run.smithi175.stdout: rgw enable ops log = true 2023-10-06T03:56:15.312 INFO:teuthology.orchestra.run.smithi175.stdout: rgw enable usage log = true 2023-10-06T03:56:15.312 INFO:teuthology.orchestra.run.smithi175.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T03:56:15.312 INFO:teuthology.orchestra.run.smithi175.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T03:56:15.312 INFO:teuthology.orchestra.run.smithi175.stdout: client mount timeout = 600 2023-10-06T03:56:15.312 INFO:teuthology.orchestra.run.smithi175.stdout: debug client = 20 2023-10-06T03:56:15.312 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2023-10-06T03:56:15.312 INFO:teuthology.orchestra.run.smithi175.stdout: rados mon op timeout = 900 2023-10-06T03:56:15.313 INFO:teuthology.orchestra.run.smithi175.stdout: rados osd op timeout = 900 2023-10-06T03:56:15.313 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.a] 2023-10-06T03:56:15.313 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.b] 2023-10-06T03:56:15.313 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.c] 2023-10-06T03:56:15.313 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.a] 2023-10-06T03:56:15.313 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.c] 2023-10-06T03:56:15.313 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.e] 2023-10-06T03:56:15.313 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.b] 2023-10-06T03:56:15.313 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.d] 2023-10-06T03:56:15.313 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.f] 2023-10-06T03:56:15.313 INFO:teuthology.orchestra.run.smithi175.stdout:[mds] 2023-10-06T03:56:15.313 INFO:teuthology.orchestra.run.smithi175.stdout: debug mds = 20 2023-10-06T03:56:15.314 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2023-10-06T03:56:15.314 INFO:teuthology.orchestra.run.smithi175.stdout: mds bal fragment size max = 10000 2023-10-06T03:56:15.314 INFO:teuthology.orchestra.run.smithi175.stdout: mds bal merge size = 5 2023-10-06T03:56:15.314 INFO:teuthology.orchestra.run.smithi175.stdout: mds bal split bits = 3 2023-10-06T03:56:15.314 INFO:teuthology.orchestra.run.smithi175.stdout: mds bal split size = 100 2023-10-06T03:56:15.314 INFO:teuthology.orchestra.run.smithi175.stdout: mds debug frag = True 2023-10-06T03:56:15.314 INFO:teuthology.orchestra.run.smithi175.stdout: mds debug scatterstat = True 2023-10-06T03:56:15.314 INFO:teuthology.orchestra.run.smithi175.stdout: mds oft prefetch dirfrags = False 2023-10-06T03:56:15.314 INFO:teuthology.orchestra.run.smithi175.stdout: mds op complaint time = 180 2023-10-06T03:56:15.314 INFO:teuthology.orchestra.run.smithi175.stdout: mds verify scatter = True 2023-10-06T03:56:15.314 INFO:teuthology.orchestra.run.smithi175.stdout: mds_sleep_rank_change = 5000000.0 2023-10-06T03:56:15.314 INFO:teuthology.orchestra.run.smithi175.stdout: mon client directed command retry = 5 2023-10-06T03:56:15.315 INFO:teuthology.orchestra.run.smithi175.stdout: ms inject delay max = 1 2023-10-06T03:56:15.315 INFO:teuthology.orchestra.run.smithi175.stdout: ms inject delay probability = 0.005 2023-10-06T03:56:15.315 INFO:teuthology.orchestra.run.smithi175.stdout: ms inject delay type = client mds osd 2023-10-06T03:56:15.315 INFO:teuthology.orchestra.run.smithi175.stdout: ms inject socket failures = 2500 2023-10-06T03:56:15.315 INFO:teuthology.orchestra.run.smithi175.stdout: osd op complaint time = 180 2023-10-06T03:56:15.315 INFO:teuthology.orchestra.run.smithi175.stdout: rados mon op timeout = 900 2023-10-06T03:56:15.315 INFO:teuthology.orchestra.run.smithi175.stdout: rados osd op timeout = 900 2023-10-06T03:56:15.315 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-06T03:56:15.315 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-06T03:56:15.392 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-06T03:56:15.392 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-06T03:56:15.392 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-06T03:56:15.407 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-06T03:56:15.407 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-06T03:56:15.461 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi157.front.sepia.ceph.com 2023-10-06T03:56:15.462 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-06T03:56:15.462 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T03:56:15.462 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T03:56:15.535 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-06T03:56:15.535 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:56:15.589 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi175.front.sepia.ceph.com 2023-10-06T03:56:15.590 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-10-06T03:56:15.590 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T03:56:15.590 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T03:56:15.623 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-10-06T03:56:15.623 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:56:15.678 INFO:tasks.ceph:Setting up mon nodes... 2023-10-06T03:56:15.678 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-06T03:56:15.678 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T03:56:15.726 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T03:56:15.728 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T03:56:15.781 INFO:teuthology.orchestra.run.smithi175.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T03:56:15.783 INFO:tasks.ceph:Setting up mds nodes... 2023-10-06T03:56:15.783 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T03:56:15.831 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T03:56:15.834 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-06T03:56:15.857 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T03:56:15.945 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T03:56:15.948 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-06T03:56:15.972 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2023-10-06T03:56:16.060 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2023-10-06T03:56:16.062 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2023-10-06T03:56:16.086 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T03:56:16.134 INFO:teuthology.orchestra.run.smithi175.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T03:56:16.136 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-06T03:56:16.159 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T03:56:16.246 INFO:teuthology.orchestra.run.smithi175.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T03:56:16.248 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-06T03:56:16.311 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2023-10-06T03:56:16.397 INFO:teuthology.orchestra.run.smithi175.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2023-10-06T03:56:16.399 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2023-10-06T03:56:16.421 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-06T03:56:16.421 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-06T03:56:16.459 INFO:teuthology.orchestra.run.smithi157.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-06T03:56:16.471 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-06T03:56:16.472 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi157.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi175.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T03:56:16.472 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-06T03:56:16.536 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T03:56:16.536 INFO:tasks.ceph:role: osd.0 2023-10-06T03:56:16.536 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi157.front.sepia.ceph.com 2023-10-06T03:56:16.536 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T03:56:16.600 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T03:56:16.600 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T03:56:16.600 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T03:56:16.600 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 2023-10-06T03:56:16.600 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T03:56:16.600 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2023-10-06T03:56:16.600 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T03:56:16.601 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T03:56:16.601 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T03:56:16.601 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T03:56:16.623 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2023-10-06T03:56:16.625 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2023-10-06T03:56:16.625 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-06T03:56:16.703 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-06T03:56:16.772 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-06T03:56:16.837 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T03:56:16.837 INFO:tasks.ceph:role: osd.1 2023-10-06T03:56:16.837 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi157.front.sepia.ceph.com 2023-10-06T03:56:16.838 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T03:56:16.901 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T03:56:16.901 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T03:56:16.901 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T03:56:16.901 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 2023-10-06T03:56:16.901 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T03:56:16.901 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2023-10-06T03:56:16.901 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T03:56:16.902 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T03:56:16.902 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T03:56:16.902 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T03:56:16.923 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2023-10-06T03:56:16.925 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2023-10-06T03:56:16.925 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-06T03:56:17.002 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-06T03:56:17.069 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-06T03:56:17.134 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T03:56:17.134 INFO:tasks.ceph:role: osd.2 2023-10-06T03:56:17.134 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi157.front.sepia.ceph.com 2023-10-06T03:56:17.134 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T03:56:17.198 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T03:56:17.198 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T03:56:17.198 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T03:56:17.198 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 2023-10-06T03:56:17.198 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T03:56:17.198 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2023-10-06T03:56:17.198 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T03:56:17.199 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T03:56:17.199 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T03:56:17.199 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T03:56:17.220 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2023-10-06T03:56:17.222 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2023-10-06T03:56:17.222 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-06T03:56:17.299 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-06T03:56:17.367 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-06T03:56:17.432 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T03:56:17.432 INFO:tasks.ceph:role: osd.3 2023-10-06T03:56:17.432 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi157.front.sepia.ceph.com 2023-10-06T03:56:17.432 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T03:56:17.496 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T03:56:17.496 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T03:56:17.496 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T03:56:17.496 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 2023-10-06T03:56:17.496 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T03:56:17.496 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2023-10-06T03:56:17.496 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T03:56:17.496 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T03:56:17.497 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T03:56:17.497 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T03:56:17.519 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2023-10-06T03:56:17.520 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2023-10-06T03:56:17.520 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-06T03:56:17.597 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-06T03:56:17.665 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:56:17.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:17.758+0000 7f550a9f7540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-06T03:56:17.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:17.758+0000 7f550a9f7540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-06T03:56:17.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:17.758+0000 7f550a9f7540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T03:56:17.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:17.758+0000 7f550a9f7540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T03:56:17.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:17.758+0000 7f550a9f7540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T03:56:17.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:17.758+0000 7f550a9f7540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-06T03:56:18.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:18.279+0000 7f550a9f7540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:56:19.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:19.041+0000 7f550a9f7540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:56:19.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:19.804+0000 7f550a9f7540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:56:20.605 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-06T03:56:20.633 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:56:20.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:20.716+0000 7fb7d5112540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-06T03:56:20.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:20.716+0000 7fb7d5112540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-06T03:56:20.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:20.716+0000 7fb7d5112540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T03:56:20.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:20.716+0000 7fb7d5112540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T03:56:20.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:20.716+0000 7fb7d5112540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T03:56:20.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:20.717+0000 7fb7d5112540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-06T03:56:21.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:21.241+0000 7fb7d5112540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:56:22.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:22.002+0000 7fb7d5112540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:56:22.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:22.766+0000 7fb7d5112540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:56:23.564 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-06T03:56:23.588 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:56:23.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:23.668+0000 7f3a3f298540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-06T03:56:23.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:23.668+0000 7f3a3f298540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-06T03:56:23.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:23.668+0000 7f3a3f298540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T03:56:23.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:23.668+0000 7f3a3f298540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T03:56:23.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:23.668+0000 7f3a3f298540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T03:56:23.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:23.668+0000 7f3a3f298540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-06T03:56:24.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:24.190+0000 7f3a3f298540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:56:24.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:24.950+0000 7f3a3f298540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:56:25.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:25.713+0000 7f3a3f298540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:56:26.515 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-06T03:56:26.585 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:56:26.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:26.671+0000 7f716cf30540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-06T03:56:26.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:26.671+0000 7f716cf30540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-06T03:56:26.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:26.671+0000 7f716cf30540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T03:56:26.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:26.671+0000 7f716cf30540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T03:56:26.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:26.671+0000 7f716cf30540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T03:56:26.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:26.671+0000 7f716cf30540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-06T03:56:27.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:27.199+0000 7f716cf30540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:56:27.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:27.959+0000 7f716cf30540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:56:28.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:28.722+0000 7f716cf30540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:56:29.522 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-06T03:56:29.547 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-06T03:56:29.576 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T03:56:29.576 INFO:tasks.ceph:role: osd.4 2023-10-06T03:56:29.576 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi175.front.sepia.ceph.com 2023-10-06T03:56:29.576 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T03:56:29.651 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T03:56:29.652 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T03:56:29.652 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T03:56:29.652 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 2023-10-06T03:56:29.652 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T03:56:29.652 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2023-10-06T03:56:29.652 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T03:56:29.652 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T03:56:29.652 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T03:56:29.652 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T03:56:30.015 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2023-10-06T03:56:30.017 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2023-10-06T03:56:30.017 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-06T03:56:30.155 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-06T03:56:30.186 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-06T03:56:30.252 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T03:56:30.253 INFO:tasks.ceph:role: osd.5 2023-10-06T03:56:30.253 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi175.front.sepia.ceph.com 2023-10-06T03:56:30.253 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T03:56:30.325 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T03:56:30.325 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T03:56:30.325 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T03:56:30.325 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 2023-10-06T03:56:30.325 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T03:56:30.326 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2023-10-06T03:56:30.326 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T03:56:30.326 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T03:56:30.326 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T03:56:30.326 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T03:56:30.684 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2023-10-06T03:56:30.685 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2023-10-06T03:56:30.685 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-06T03:56:30.775 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-06T03:56:30.809 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-06T03:56:30.875 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T03:56:30.876 INFO:tasks.ceph:role: osd.6 2023-10-06T03:56:30.876 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi175.front.sepia.ceph.com 2023-10-06T03:56:30.876 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T03:56:30.949 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T03:56:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T03:56:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T03:56:30.949 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 2023-10-06T03:56:30.949 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T03:56:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2023-10-06T03:56:30.950 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T03:56:30.950 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T03:56:30.950 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T03:56:30.950 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T03:56:31.312 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2023-10-06T03:56:31.314 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2023-10-06T03:56:31.314 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-06T03:56:31.420 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-06T03:56:31.449 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-06T03:56:31.515 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T03:56:31.515 INFO:tasks.ceph:role: osd.7 2023-10-06T03:56:31.515 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi175.front.sepia.ceph.com 2023-10-06T03:56:31.515 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T03:56:31.588 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T03:56:31.588 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T03:56:31.588 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T03:56:31.588 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 2023-10-06T03:56:31.588 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T03:56:31.588 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2023-10-06T03:56:31.588 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T03:56:31.588 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T03:56:31.589 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T03:56:31.589 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T03:56:31.962 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2023-10-06T03:56:31.964 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2023-10-06T03:56:31.964 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-06T03:56:32.071 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-06T03:56:32.101 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:56:32.184 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:32.181+0000 7f9e74217540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-06T03:56:32.184 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:32.182+0000 7f9e74217540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-06T03:56:32.184 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:32.182+0000 7f9e74217540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T03:56:32.184 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:32.182+0000 7f9e74217540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T03:56:32.184 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:32.182+0000 7f9e74217540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T03:56:32.190 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:32.188+0000 7f9e74217540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-06T03:56:32.723 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:32.721+0000 7f9e74217540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:56:33.490 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:33.488+0000 7f9e74217540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:56:34.259 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:34.257+0000 7f9e74217540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:56:35.060 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-06T03:56:35.129 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:56:35.216 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:35.214+0000 7f1a34762540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-06T03:56:35.216 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:35.214+0000 7f1a34762540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-06T03:56:35.217 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:35.214+0000 7f1a34762540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T03:56:35.217 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:35.214+0000 7f1a34762540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T03:56:35.217 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:35.214+0000 7f1a34762540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T03:56:35.217 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:35.214+0000 7f1a34762540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-06T03:56:35.743 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:35.741+0000 7f1a34762540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:56:36.508 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:36.506+0000 7f1a34762540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:56:37.273 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:37.271+0000 7f1a34762540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:56:38.073 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-06T03:56:38.101 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:56:38.185 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:38.182+0000 7fd1f5fb1540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-06T03:56:38.185 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:38.183+0000 7fd1f5fb1540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-06T03:56:38.185 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:38.183+0000 7fd1f5fb1540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T03:56:38.186 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:38.183+0000 7fd1f5fb1540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T03:56:38.186 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:38.183+0000 7fd1f5fb1540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T03:56:38.186 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:38.183+0000 7fd1f5fb1540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-06T03:56:38.715 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:38.713+0000 7fd1f5fb1540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:56:39.480 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:39.478+0000 7fd1f5fb1540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:56:40.251 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:40.249+0000 7fd1f5fb1540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:56:41.046 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-06T03:56:41.116 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:56:41.202 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:41.199+0000 7f59f1c86540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-06T03:56:41.202 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:41.199+0000 7f59f1c86540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-06T03:56:41.202 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:41.199+0000 7f59f1c86540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T03:56:41.202 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:41.199+0000 7f59f1c86540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T03:56:41.203 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:41.199+0000 7f59f1c86540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T03:56:41.203 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:41.199+0000 7f59f1c86540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-06T03:56:41.729 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:41.727+0000 7f59f1c86540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:56:42.500 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:42.498+0000 7f59f1c86540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:56:43.267 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:43.265+0000 7f59f1c86540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:56:43.304 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:56:43.307 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:56:44.063 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-06T03:56:44.090 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-06T03:56:44.090 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-06T03:56:44.090 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-06T03:56:44.116 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-06T03:56:44.117 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-06T03:56:44.183 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-06T03:56:44.183 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-06T03:56:44.249 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-06T03:56:44.249 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2023-10-06T03:56:44.314 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-06T03:56:44.314 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-06T03:56:44.379 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-06T03:56:44.380 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-06T03:56:44.445 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-06T03:56:44.445 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-06T03:56:44.509 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-06T03:56:44.509 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-06T03:56:44.573 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-10-06T03:56:44.573 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-06T03:56:44.601 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-10-06T03:56:44.601 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-06T03:56:44.668 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-10-06T03:56:44.668 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-06T03:56:44.734 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-10-06T03:56:44.734 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2023-10-06T03:56:44.799 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-10-06T03:56:44.799 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-06T03:56:44.864 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-10-06T03:56:44.864 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-06T03:56:44.929 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-10-06T03:56:44.930 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-06T03:56:44.994 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-10-06T03:56:44.994 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-06T03:56:45.058 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-06T03:56:45.059 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-06T03:56:45.075 INFO:tasks.ceph:Adding keys to all mons... 2023-10-06T03:56:45.075 DEBUG:teuthology.orchestra.run.smithi157:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T03:56:45.117 DEBUG:teuthology.orchestra.run.smithi175:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T03:56:45.141 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr.y] 2023-10-06T03:56:45.141 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBfhR9lvExAKxAAvgQK7N+JaLhDMwRNvODlWw== 2023-10-06T03:56:45.141 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.a] 2023-10-06T03:56:45.141 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBfhR9lH7WOMRAAAheo4Wz41kJKOA3KZBJ03g== 2023-10-06T03:56:45.141 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.c] 2023-10-06T03:56:45.141 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBfhR9lvK5ZOBAAtTq3pxJ0nLKGrz6hSz/1Ug== 2023-10-06T03:56:45.141 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.e] 2023-10-06T03:56:45.141 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBghR9lTjGNAxAAI70MLk+YdCZhroJZq5XA+A== 2023-10-06T03:56:45.141 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.0] 2023-10-06T03:56:45.141 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBhhR9lrfY+LRAArQcqPb6QgOi46NJEiAtEgw== 2023-10-06T03:56:45.142 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.1] 2023-10-06T03:56:45.142 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBkhR9l9FXFKhAAgHX/92iugXLkqqu19/ls2A== 2023-10-06T03:56:45.142 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.2] 2023-10-06T03:56:45.142 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBnhR9ljj3lJxAA/HJf7EQX/qazujWJXhXgoQ== 2023-10-06T03:56:45.142 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.3] 2023-10-06T03:56:45.142 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBqhR9lbBUQKBAAXFnfMNjz0I6Y5OqlSst9/w== 2023-10-06T03:56:45.142 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr.x] 2023-10-06T03:56:45.142 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBfhR9lNCOHLhAAGYrkF02wS4PSgqLgncIExA== 2023-10-06T03:56:45.142 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.b] 2023-10-06T03:56:45.142 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBghR9l/ST2BxAA0K+vZvcQcViwtkLxdK3Knw== 2023-10-06T03:56:45.143 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.d] 2023-10-06T03:56:45.143 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBghR9lTL2iDhAAPbtfA7B0aNoGFSy1acm4hQ== 2023-10-06T03:56:45.143 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.f] 2023-10-06T03:56:45.143 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBghR9lFk6iFxAA8rZJEae+u8s0cFhN04RtEg== 2023-10-06T03:56:45.143 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.4] 2023-10-06T03:56:45.143 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBwhR9lR7bsChAA12gC9lg7oZswlRici5epQA== 2023-10-06T03:56:45.143 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.5] 2023-10-06T03:56:45.143 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBzhR9lUzDaDBAArSi0MFQ/ePASBew6M4r9wQ== 2023-10-06T03:56:45.143 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.6] 2023-10-06T03:56:45.144 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB2hR9lr3j8ChAAYZqG3nb1b6Ct6gt/IUQGdQ== 2023-10-06T03:56:45.144 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.7] 2023-10-06T03:56:45.144 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB5hR9lqJ30CxAAGFdNJMpFUoWuNJlpwvLkPQ== 2023-10-06T03:56:45.144 INFO:teuthology.orchestra.run.smithi175.stdout:[client.0] 2023-10-06T03:56:45.144 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBghR9lrzhYGxAAmjNS7YTMHGxrLYwYzdEs5A== 2023-10-06T03:56:45.146 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr.y] 2023-10-06T03:56:45.146 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBfhR9lvExAKxAAvgQK7N+JaLhDMwRNvODlWw== 2023-10-06T03:56:45.146 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2023-10-06T03:56:45.146 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBfhR9lH7WOMRAAAheo4Wz41kJKOA3KZBJ03g== 2023-10-06T03:56:45.146 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2023-10-06T03:56:45.146 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBfhR9lvK5ZOBAAtTq3pxJ0nLKGrz6hSz/1Ug== 2023-10-06T03:56:45.146 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.e] 2023-10-06T03:56:45.146 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBghR9lTjGNAxAAI70MLk+YdCZhroJZq5XA+A== 2023-10-06T03:56:45.147 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.0] 2023-10-06T03:56:45.147 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBhhR9lrfY+LRAArQcqPb6QgOi46NJEiAtEgw== 2023-10-06T03:56:45.147 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.1] 2023-10-06T03:56:45.147 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBkhR9l9FXFKhAAgHX/92iugXLkqqu19/ls2A== 2023-10-06T03:56:45.147 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.2] 2023-10-06T03:56:45.147 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBnhR9ljj3lJxAA/HJf7EQX/qazujWJXhXgoQ== 2023-10-06T03:56:45.147 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.3] 2023-10-06T03:56:45.147 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBqhR9lbBUQKBAAXFnfMNjz0I6Y5OqlSst9/w== 2023-10-06T03:56:45.147 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr.x] 2023-10-06T03:56:45.147 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBfhR9lNCOHLhAAGYrkF02wS4PSgqLgncIExA== 2023-10-06T03:56:45.148 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2023-10-06T03:56:45.148 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBghR9l/ST2BxAA0K+vZvcQcViwtkLxdK3Knw== 2023-10-06T03:56:45.148 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2023-10-06T03:56:45.148 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBghR9lTL2iDhAAPbtfA7B0aNoGFSy1acm4hQ== 2023-10-06T03:56:45.148 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.f] 2023-10-06T03:56:45.148 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBghR9lFk6iFxAA8rZJEae+u8s0cFhN04RtEg== 2023-10-06T03:56:45.148 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.4] 2023-10-06T03:56:45.148 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBwhR9lR7bsChAA12gC9lg7oZswlRici5epQA== 2023-10-06T03:56:45.149 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.5] 2023-10-06T03:56:45.149 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBzhR9lUzDaDBAArSi0MFQ/ePASBew6M4r9wQ== 2023-10-06T03:56:45.149 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.6] 2023-10-06T03:56:45.149 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQB2hR9lr3j8ChAAYZqG3nb1b6Ct6gt/IUQGdQ== 2023-10-06T03:56:45.149 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.7] 2023-10-06T03:56:45.149 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQB5hR9lqJ30CxAAGFdNJMpFUoWuNJlpwvLkPQ== 2023-10-06T03:56:45.149 INFO:teuthology.orchestra.run.smithi157.stdout:[client.0] 2023-10-06T03:56:45.149 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBghR9lrzhYGxAAmjNS7YTMHGxrLYwYzdEs5A== 2023-10-06T03:56:45.150 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T03:56:45.189 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T03:56:45.237 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T03:56:45.240 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T03:56:45.289 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T03:56:45.291 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T03:56:45.340 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T03:56:45.342 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T03:56:45.391 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:56:45.393 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:56:45.439 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:56:45.441 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:56:45.486 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:56:45.489 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:56:45.534 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:56:45.537 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:56:45.584 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T03:56:45.586 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T03:56:45.632 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T03:56:45.634 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T03:56:45.683 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T03:56:45.687 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T03:56:45.736 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T03:56:45.738 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T03:56:45.787 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:56:45.789 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:56:45.835 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:56:45.837 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:56:45.883 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:56:45.885 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:56:45.931 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:56:45.933 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:56:45.979 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T03:56:45.981 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T03:56:46.029 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-06T03:56:46.030 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-06T03:56:46.054 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T03:56:46.145 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-06T03:56:46.171 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-06T03:56:46.194 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T03:56:46.284 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-06T03:56:46.307 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-06T03:56:46.370 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T03:56:46.460 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-06T03:56:46.482 DEBUG:teuthology.orchestra.run.smithi157:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:56:46.485 DEBUG:teuthology.orchestra.run.smithi175:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:56:46.536 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-06T03:56:46.537 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-06T03:56:46.537 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-06T03:56:46.539 INFO:tasks.ceph.mon.a:Started 2023-10-06T03:56:46.539 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-06T03:56:46.539 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-06T03:56:46.579 INFO:tasks.ceph.mon.b:Started 2023-10-06T03:56:46.579 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-06T03:56:46.579 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-06T03:56:46.581 INFO:tasks.ceph.mon.c:Started 2023-10-06T03:56:46.581 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-06T03:56:46.581 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-06T03:56:46.581 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-06T03:56:46.583 INFO:tasks.ceph.mgr.y:Started 2023-10-06T03:56:46.584 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-06T03:56:46.584 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-06T03:56:46.585 INFO:tasks.ceph.mgr.x:Started 2023-10-06T03:56:46.585 INFO:tasks.ceph:Setting crush tunables to default 2023-10-06T03:56:46.586 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd crush tunables default 2023-10-06T03:56:46.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:46.701+0000 7f3b127c8700 1 Processor -- start 2023-10-06T03:56:46.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:46.702+0000 7f3b127c8700 1 -- start start 2023-10-06T03:56:46.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:46.702+0000 7f3b127c8700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3b0c06a780 0x7f3b0c154010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:46.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:46.702+0000 7f3b127c8700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b0c0c36d0 con 0x7f3b0c06e2e0 2023-10-06T03:56:46.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:46.702+0000 7f3b127c8700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b0c0c3e30 con 0x7f3b0c068df0 2023-10-06T03:56:46.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:46.702+0000 7f3b127c8700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b0c0c4110 con 0x7f3b0c06a780 2023-10-06T03:56:46.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:46.702+0000 7f3b0bfff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3b0c06a780 0x7f3b0c154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:46.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:46.702+0000 7f3b10d65700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f3b0c06e2e0 0x7f3b0c15e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:36988/0 (socket says 172.21.15.157:36988) 2023-10-06T03:56:46.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:46.703+0000 7f3b0bfff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3b0c06a780 0x7f3b0c154010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46072/0 (socket says 172.21.15.157:46072) 2023-10-06T03:56:46.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:46.703+0000 7f3b10d65700 1 -- 172.21.15.157:0/4076793705 learned_addr learned my addr 172.21.15.157:0/4076793705 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:56:46.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:46.703+0000 7f3b0bfff700 1 -- 172.21.15.157:0/4076793705 >> v2:172.21.15.157:3300/0 conn(0x7f3b0c06a780 msgr2=0x7f3b0c154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T03:56:46.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:46.703+0000 7f3b0bfff700 1 -- 172.21.15.157:0/4076793705 >> v2:172.21.15.157:3300/0 conn(0x7f3b0c06a780 msgr2=0x7f3b0c154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T03:56:46.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:46.703+0000 7f3b0bfff700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.157:3300/0 conn(0x7f3b0c06a780 0x7f3b0c154010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T03:56:46.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:46.703+0000 7f3b0bfff700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.157:3300/0 conn(0x7f3b0c06a780 0x7f3b0c154010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:56:46.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:46.904+0000 7f3b0bfff700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.157:3300/0 conn(0x7f3b0c06a780 0x7f3b0c154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:46.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:46.904+0000 7f3b0bfff700 1 -- 172.21.15.157:0/4076793705 >> v2:172.21.15.157:3300/0 conn(0x7f3b0c06a780 msgr2=0x7f3b0c154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T03:56:46.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:46.904+0000 7f3b0bfff700 1 -- 172.21.15.157:0/4076793705 >> v2:172.21.15.157:3300/0 conn(0x7f3b0c06a780 msgr2=0x7f3b0c154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T03:56:46.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:46.904+0000 7f3b0bfff700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.157:3300/0 conn(0x7f3b0c06a780 0x7f3b0c154010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T03:56:46.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:46.904+0000 7f3b0bfff700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.157:3300/0 conn(0x7f3b0c06a780 0x7f3b0c154010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:56:47.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:47.305+0000 7f3b0bfff700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.157:3300/0 conn(0x7f3b0c06a780 0x7f3b0c154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:47.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:47.305+0000 7f3b0bfff700 1 -- 172.21.15.157:0/4076793705 >> v2:172.21.15.157:3300/0 conn(0x7f3b0c06a780 msgr2=0x7f3b0c154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T03:56:47.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:47.305+0000 7f3b0bfff700 1 -- 172.21.15.157:0/4076793705 >> v2:172.21.15.157:3300/0 conn(0x7f3b0c06a780 msgr2=0x7f3b0c154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T03:56:47.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:47.305+0000 7f3b0bfff700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.157:3300/0 conn(0x7f3b0c06a780 0x7f3b0c154010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T03:56:47.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:47.305+0000 7f3b0bfff700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.157:3300/0 conn(0x7f3b0c06a780 0x7f3b0c154010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T03:56:48.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:48.106+0000 7f3b0bfff700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.157:3300/0 conn(0x7f3b0c06a780 0x7f3b0c154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:48.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:48.106+0000 7f3b0bfff700 1 -- 172.21.15.157:0/4076793705 >> v2:172.21.15.157:3300/0 conn(0x7f3b0c06a780 msgr2=0x7f3b0c154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T03:56:48.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:48.106+0000 7f3b0bfff700 1 -- 172.21.15.157:0/4076793705 >> v2:172.21.15.157:3300/0 conn(0x7f3b0c06a780 msgr2=0x7f3b0c154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T03:56:48.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:48.106+0000 7f3b0bfff700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.157:3300/0 conn(0x7f3b0c06a780 0x7f3b0c154010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T03:56:48.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:48.106+0000 7f3b0bfff700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.157:3300/0 conn(0x7f3b0c06a780 0x7f3b0c154010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T03:56:49.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:49.702+0000 7f3b09ffb700 1 -- 172.21.15.157:0/4076793705 >> v2:172.21.15.157:3300/0 conn(0x7f3b0c06a780 msgr2=0x7f3b0c154010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T03:56:49.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:49.702+0000 7f3b09ffb700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.157:3300/0 conn(0x7f3b0c06a780 0x7f3b0c154010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:49.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:49.702+0000 7f3b09ffb700 1 -- 172.21.15.157:0/4076793705 >> v1:172.21.15.175:6789/0 conn(0x7f3b0c068df0 legacy=0x7f3b0c150d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:49.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:49.702+0000 7f3b09ffb700 1 -- 172.21.15.157:0/4076793705 >> v1:172.21.15.157:6789/0 conn(0x7f3b0c06e2e0 legacy=0x7f3b0c15e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:49.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:49.702+0000 7f3b09ffb700 1 --2- 172.21.15.157:0/4076793705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3af8003780 0x7f3af8005ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:49.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:49.702+0000 7f3b09ffb700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.175:3300/0 conn(0x7f3af80062c0 0x7f3af8008750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:49.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:49.702+0000 7f3b09ffb700 1 -- 172.21.15.157:0/4076793705 --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b0c0c4110 con 0x7f3af80062c0 2023-10-06T03:56:49.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:49.702+0000 7f3b09ffb700 1 -- 172.21.15.157:0/4076793705 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af8008e00 con 0x7f3af8003780 2023-10-06T03:56:49.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:49.703+0000 7f3b0bfff700 1 --2- 172.21.15.157:0/4076793705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3af8003780 0x7f3af8005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:49.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:49.703+0000 7f3b0b7fe700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.175:3300/0 conn(0x7f3af80062c0 0x7f3af8008750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:49.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:49.703+0000 7f3b0bfff700 1 -- 172.21.15.157:0/4076793705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3af8003780 msgr2=0x7f3af8005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T03:56:49.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:49.703+0000 7f3b0bfff700 1 -- 172.21.15.157:0/4076793705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3af8003780 msgr2=0x7f3af8005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T03:56:49.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:49.703+0000 7f3b0bfff700 1 --2- 172.21.15.157:0/4076793705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3af8003780 0x7f3af8005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T03:56:49.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:49.703+0000 7f3b0bfff700 1 --2- 172.21.15.157:0/4076793705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3af8003780 0x7f3af8005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:56:49.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:49.703+0000 7f3b0b7fe700 1 -- 172.21.15.157:0/4076793705 >> v2:172.21.15.175:3300/0 conn(0x7f3af80062c0 msgr2=0x7f3af8008750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T03:56:49.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:49.703+0000 7f3b0b7fe700 1 -- 172.21.15.157:0/4076793705 >> v2:172.21.15.175:3300/0 conn(0x7f3af80062c0 msgr2=0x7f3af8008750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T03:56:49.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:49.703+0000 7f3b0b7fe700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.175:3300/0 conn(0x7f3af80062c0 0x7f3af8008750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T03:56:49.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:49.703+0000 7f3b0b7fe700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.175:3300/0 conn(0x7f3af80062c0 0x7f3af8008750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:56:49.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:49.904+0000 7f3b0bfff700 1 --2- 172.21.15.157:0/4076793705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3af8003780 0x7f3af8005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:49.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:49.904+0000 7f3b0b7fe700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.175:3300/0 conn(0x7f3af80062c0 0x7f3af8008750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:49.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:49.904+0000 7f3b0b7fe700 1 -- 172.21.15.157:0/4076793705 >> v2:172.21.15.175:3300/0 conn(0x7f3af80062c0 msgr2=0x7f3af8008750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T03:56:49.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:49.904+0000 7f3b0b7fe700 1 -- 172.21.15.157:0/4076793705 >> v2:172.21.15.175:3300/0 conn(0x7f3af80062c0 msgr2=0x7f3af8008750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T03:56:49.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:49.904+0000 7f3b0bfff700 1 -- 172.21.15.157:0/4076793705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3af8003780 msgr2=0x7f3af8005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T03:56:49.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:49.904+0000 7f3b0bfff700 1 -- 172.21.15.157:0/4076793705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3af8003780 msgr2=0x7f3af8005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T03:56:49.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:49.904+0000 7f3b0bfff700 1 --2- 172.21.15.157:0/4076793705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3af8003780 0x7f3af8005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T03:56:49.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:49.904+0000 7f3b0b7fe700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.175:3300/0 conn(0x7f3af80062c0 0x7f3af8008750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T03:56:49.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:49.904+0000 7f3b0bfff700 1 --2- 172.21.15.157:0/4076793705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3af8003780 0x7f3af8005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:56:49.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:49.904+0000 7f3b0b7fe700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.175:3300/0 conn(0x7f3af80062c0 0x7f3af8008750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:56:50.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:50.305+0000 7f3b0bfff700 1 --2- 172.21.15.157:0/4076793705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3af8003780 0x7f3af8005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:50.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:50.305+0000 7f3b0b7fe700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.175:3300/0 conn(0x7f3af80062c0 0x7f3af8008750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:50.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:50.305+0000 7f3b0bfff700 1 -- 172.21.15.157:0/4076793705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3af8003780 msgr2=0x7f3af8005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T03:56:50.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:50.305+0000 7f3b0bfff700 1 -- 172.21.15.157:0/4076793705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3af8003780 msgr2=0x7f3af8005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T03:56:50.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:50.305+0000 7f3b0bfff700 1 --2- 172.21.15.157:0/4076793705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3af8003780 0x7f3af8005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T03:56:50.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:50.305+0000 7f3b0b7fe700 1 -- 172.21.15.157:0/4076793705 >> v2:172.21.15.175:3300/0 conn(0x7f3af80062c0 msgr2=0x7f3af8008750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T03:56:50.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:50.305+0000 7f3b0b7fe700 1 -- 172.21.15.157:0/4076793705 >> v2:172.21.15.175:3300/0 conn(0x7f3af80062c0 msgr2=0x7f3af8008750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T03:56:50.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:50.305+0000 7f3b0bfff700 1 --2- 172.21.15.157:0/4076793705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3af8003780 0x7f3af8005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T03:56:50.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:50.305+0000 7f3b0b7fe700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.175:3300/0 conn(0x7f3af80062c0 0x7f3af8008750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T03:56:50.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:50.305+0000 7f3b0b7fe700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.175:3300/0 conn(0x7f3af80062c0 0x7f3af8008750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T03:56:51.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:51.106+0000 7f3b0bfff700 1 --2- 172.21.15.157:0/4076793705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3af8003780 0x7f3af8005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:51.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:51.106+0000 7f3b0b7fe700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.175:3300/0 conn(0x7f3af80062c0 0x7f3af8008750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:51.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:51.107+0000 7f3b0bfff700 1 -- 172.21.15.157:0/4076793705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3af8003780 msgr2=0x7f3af8005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T03:56:51.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:51.107+0000 7f3b0bfff700 1 -- 172.21.15.157:0/4076793705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3af8003780 msgr2=0x7f3af8005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T03:56:51.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:51.107+0000 7f3b0b7fe700 1 -- 172.21.15.157:0/4076793705 >> v2:172.21.15.175:3300/0 conn(0x7f3af80062c0 msgr2=0x7f3af8008750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T03:56:51.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:51.107+0000 7f3b0bfff700 1 --2- 172.21.15.157:0/4076793705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3af8003780 0x7f3af8005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T03:56:51.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:51.107+0000 7f3b0b7fe700 1 -- 172.21.15.157:0/4076793705 >> v2:172.21.15.175:3300/0 conn(0x7f3af80062c0 msgr2=0x7f3af8008750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T03:56:51.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:51.107+0000 7f3b0b7fe700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.175:3300/0 conn(0x7f3af80062c0 0x7f3af8008750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T03:56:51.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:51.107+0000 7f3b0bfff700 1 --2- 172.21.15.157:0/4076793705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3af8003780 0x7f3af8005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T03:56:51.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:51.107+0000 7f3b0b7fe700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.175:3300/0 conn(0x7f3af80062c0 0x7f3af8008750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T03:56:52.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.703+0000 7f3b09ffb700 1 -- 172.21.15.157:0/4076793705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3af8003780 msgr2=0x7f3af8005ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T03:56:52.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.703+0000 7f3b09ffb700 1 --2- 172.21.15.157:0/4076793705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3af8003780 0x7f3af8005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:52.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.703+0000 7f3b09ffb700 1 -- 172.21.15.157:0/4076793705 >> v2:172.21.15.175:3300/0 conn(0x7f3af80062c0 msgr2=0x7f3af8008750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T03:56:52.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.703+0000 7f3b09ffb700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.175:3300/0 conn(0x7f3af80062c0 0x7f3af8008750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:52.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.703+0000 7f3b10d65700 1 -- 172.21.15.157:0/4076793705 reap_dead start 2023-10-06T03:56:52.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.703+0000 7f3b09ffb700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.175:3300/0 conn(0x7f3af8003b90 0x7f3af8006740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:52.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.703+0000 7f3b09ffb700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.157:3300/0 conn(0x7f3af80087c0 0x7f3af800d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:52.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.703+0000 7f3b09ffb700 1 -- 172.21.15.157:0/4076793705 --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b0c0c43f0 con 0x7f3af8006c80 2023-10-06T03:56:52.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.703+0000 7f3b09ffb700 1 -- 172.21.15.157:0/4076793705 --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b0c0c4110 con 0x7f3af80087c0 2023-10-06T03:56:52.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.703+0000 7f3b09ffb700 1 -- 172.21.15.157:0/4076793705 --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3af8008e00 con 0x7f3af8003b90 2023-10-06T03:56:52.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.703+0000 7f3b10d65700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.157:3300/0 conn(0x7f3af80087c0 0x7f3af800d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:52.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.703+0000 7f3b0bfff700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.175:3300/0 conn(0x7f3af8003b90 0x7f3af8006740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:52.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.703+0000 7f3b10d65700 1 -- 172.21.15.157:0/4076793705 >> v2:172.21.15.175:3300/0 conn(0x7f3af8003b90 msgr2=0x7f3af8006740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:56:52.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.703+0000 7f3b10d65700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.175:3300/0 conn(0x7f3af8003b90 0x7f3af8006740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:52.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.703+0000 7f3b10d65700 1 -- 172.21.15.157:0/4076793705 >> v1:172.21.15.157:6789/0 conn(0x7f3af8006c80 legacy=0x7f3af80070a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:52.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.703+0000 7f3b10d65700 1 -- 172.21.15.157:0/4076793705 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af800e9d0 con 0x7f3af80087c0 2023-10-06T03:56:52.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.703+0000 7f3b0affd700 1 -- 172.21.15.157:0/4076793705 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1819547194 0 0) 0x7f3b0c0c43f0 con 0x7f3af8006c80 2023-10-06T03:56:52.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.703+0000 7f3b0bfff700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.175:3300/0 conn(0x7f3af8003b90 0x7f3af8006740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:56:52.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.703+0000 7f3b10d65700 1 --2- 172.21.15.157:0/4076793705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af80087c0 0x7f3af800d2b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c153d00 tx=0x7f3b04001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36675d6e1eb106e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:56:52.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.703+0000 7f3b0affd700 1 -- 172.21.15.157:0/4076793705 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b04002100 con 0x7f3af80087c0 2023-10-06T03:56:52.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.703+0000 7f3b0affd700 1 -- 172.21.15.157:0/4076793705 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0400cb40 con 0x7f3af80087c0 2023-10-06T03:56:52.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.703+0000 7f3b0affd700 1 -- 172.21.15.157:0/4076793705 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0400cec0 con 0x7f3af80087c0 2023-10-06T03:56:52.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.704+0000 7f3b127c8700 1 -- 172.21.15.157:0/4076793705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af80087c0 msgr2=0x7f3af800d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:52.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.704+0000 7f3b127c8700 1 --2- 172.21.15.157:0/4076793705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af80087c0 0x7f3af800d2b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c153d00 tx=0x7f3b04001a00 comp rx=0 tx=0).stop 2023-10-06T03:56:52.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.704+0000 7f3b127c8700 1 -- 172.21.15.157:0/4076793705 shutdown_connections 2023-10-06T03:56:52.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.704+0000 7f3b127c8700 1 --2- 172.21.15.157:0/4076793705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af80087c0 0x7f3af800d2b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:52.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.704+0000 7f3b127c8700 1 --2- 172.21.15.157:0/4076793705 >> v2:172.21.15.175:3300/0 conn(0x7f3af8003b90 0x7f3af8006740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:52.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.704+0000 7f3b127c8700 1 -- 172.21.15.157:0/4076793705 >> 172.21.15.157:0/4076793705 conn(0x7f3b0c06b510 msgr2=0x7f3b0c06b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:56:52.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.704+0000 7f3b127c8700 1 -- 172.21.15.157:0/4076793705 shutdown_connections 2023-10-06T03:56:52.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.704+0000 7f3b127c8700 1 -- 172.21.15.157:0/4076793705 wait complete. 2023-10-06T03:56:52.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.705+0000 7f3b127c8700 1 Processor -- start 2023-10-06T03:56:52.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.706+0000 7f3b127c8700 1 -- start start 2023-10-06T03:56:52.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.706+0000 7f3b127c8700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3af8003b90 0x7f3b0c151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:52.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.706+0000 7f3b127c8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af8006c80 0x7f3b0c167600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:52.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.706+0000 7f3b127c8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3af80087c0 0x7f3b0c14bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:52.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.706+0000 7f3b127c8700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c0c9830 con 0x7f3af8006c80 2023-10-06T03:56:52.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.706+0000 7f3b127c8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c156ee0 con 0x7f3af80087c0 2023-10-06T03:56:52.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.706+0000 7f3b127c8700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c156610 con 0x7f3af8003b90 2023-10-06T03:56:52.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.706+0000 7f3b0b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af8006c80 0x7f3b0c167600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:52.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.706+0000 7f3b0b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af8006c80 0x7f3b0c167600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46174/0 (socket says 172.21.15.157:46174) 2023-10-06T03:56:52.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.706+0000 7f3b0b7fe700 1 -- 172.21.15.157:0/2076159188 learned_addr learned my addr 172.21.15.157:0/2076159188 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:56:52.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.706+0000 7f3b0bfff700 1 --2- 172.21.15.157:0/2076159188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3af8003b90 0x7f3b0c151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:52.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.706+0000 7f3b10d65700 1 --2- 172.21.15.157:0/2076159188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3af80087c0 0x7f3b0c14bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:52.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.707+0000 7f3b0b7fe700 1 -- 172.21.15.157:0/2076159188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3af8003b90 msgr2=0x7f3b0c151e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:52.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.707+0000 7f3b0b7fe700 1 --2- 172.21.15.157:0/2076159188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3af8003b90 0x7f3b0c151e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:52.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.707+0000 7f3b0b7fe700 1 -- 172.21.15.157:0/2076159188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3af80087c0 msgr2=0x7f3b0c14bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:52.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.707+0000 7f3b0b7fe700 1 --2- 172.21.15.157:0/2076159188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3af80087c0 0x7f3b0c14bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:52.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.707+0000 7f3b0b7fe700 1 -- 172.21.15.157:0/2076159188 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0c0c36d0 con 0x7f3af8006c80 2023-10-06T03:56:52.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.707+0000 7f3b0bfff700 1 --2- 172.21.15.157:0/2076159188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3af8003b90 0x7f3b0c151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:56:52.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.707+0000 7f3b0b7fe700 1 --2- 172.21.15.157:0/2076159188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af8006c80 0x7f3b0c167600 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3b00013680 tx=0x7f3b00009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:56:52.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.707+0000 7f3af3fff700 1 -- 172.21.15.157:0/2076159188 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b00016730 con 0x7f3af8006c80 2023-10-06T03:56:52.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.707+0000 7f3af3fff700 1 -- 172.21.15.157:0/2076159188 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b00016890 con 0x7f3af8006c80 2023-10-06T03:56:52.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.707+0000 7f3b127c8700 1 -- 172.21.15.157:0/2076159188 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b0c1568f0 con 0x7f3af8006c80 2023-10-06T03:56:52.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.707+0000 7f3af3fff700 1 -- 172.21.15.157:0/2076159188 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b00016c50 con 0x7f3af8006c80 2023-10-06T03:56:52.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.707+0000 7f3b127c8700 1 -- 172.21.15.157:0/2076159188 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b0c153010 con 0x7f3af8006c80 2023-10-06T03:56:52.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.707+0000 7f3af3fff700 1 -- 172.21.15.157:0/2076159188 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f3b0001f450 con 0x7f3af8006c80 2023-10-06T03:56:52.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.707+0000 7f3af3fff700 1 -- 172.21.15.157:0/2076159188 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f3b0001e700 con 0x7f3af8006c80 2023-10-06T03:56:52.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.708+0000 7f3b127c8700 1 -- 172.21.15.157:0/2076159188 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b0c153230 con 0x7f3af8006c80 2023-10-06T03:56:52.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.710+0000 7f3af3fff700 1 -- 172.21.15.157:0/2076159188 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f3b0003e6d0 con 0x7f3af8006c80 2023-10-06T03:56:52.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.784+0000 7f3b127c8700 1 -- 172.21.15.157:0/2076159188 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f3b0c0c9ca0 con 0x7f3af8006c80 2023-10-06T03:56:52.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.787+0000 7f3af3fff700 1 -- 172.21.15.157:0/2076159188 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3b0002d070 con 0x7f3af8006c80 2023-10-06T03:56:52.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.788+0000 7f3b127c8700 1 -- 172.21.15.157:0/2076159188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af8006c80 msgr2=0x7f3b0c167600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:52.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.788+0000 7f3b127c8700 1 --2- 172.21.15.157:0/2076159188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af8006c80 0x7f3b0c167600 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3b00013680 tx=0x7f3b00009040 comp rx=0 tx=0).stop 2023-10-06T03:56:52.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.788+0000 7f3b127c8700 1 -- 172.21.15.157:0/2076159188 shutdown_connections 2023-10-06T03:56:52.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.788+0000 7f3b127c8700 1 --2- 172.21.15.157:0/2076159188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3af80087c0 0x7f3b0c14bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:52.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.788+0000 7f3b127c8700 1 --2- 172.21.15.157:0/2076159188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3af8003b90 0x7f3b0c151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:52.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.788+0000 7f3b127c8700 1 --2- 172.21.15.157:0/2076159188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3af8006c80 0x7f3b0c167600 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:52.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.788+0000 7f3b127c8700 1 -- 172.21.15.157:0/2076159188 >> 172.21.15.157:0/2076159188 conn(0x7f3b0c06b510 msgr2=0x7f3b0c0c7170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:56:52.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.788+0000 7f3b127c8700 1 -- 172.21.15.157:0/2076159188 shutdown_connections 2023-10-06T03:56:52.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:52.788+0000 7f3b127c8700 1 -- 172.21.15.157:0/2076159188 wait complete. 2023-10-06T03:56:52.791 INFO:teuthology.orchestra.run.smithi157.stderr:adjusted tunables profile to default 2023-10-06T03:56:52.801 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-06T03:56:52.802 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-06T03:56:52.802 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-06T03:56:52.826 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-06T03:56:52.826 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-06T03:56:52.892 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-06T03:56:52.892 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-06T03:56:52.957 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-06T03:56:52.957 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-06T03:56:53.023 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-10-06T03:56:53.024 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-06T03:56:53.049 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-10-06T03:56:53.049 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-06T03:56:53.063 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-10-06T03:56:53.061+0000 7fcdc1ec5180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:56:53.071 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-10-06T03:56:53.070+0000 7fe9d654c180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:56:53.087 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-10-06T03:56:53.087 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-06T03:56:53.151 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-10-06T03:56:53.152 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-06T03:56:53.202 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-10-06T03:56:53.200+0000 7fcdc1ec5180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:56:53.205 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-10-06T03:56:53.203+0000 7fe9d654c180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:56:53.216 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 3f677cac-e9f4-4608-acac-718b072ce984 0 2023-10-06T03:56:53.280 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-10-06T03:56:53.279+0000 7fe9d654c180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:56:53.281 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-10-06T03:56:53.280+0000 7fcdc1ec5180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:56:53.373 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.370+0000 7f2b00f76700 1 Processor -- start 2023-10-06T03:56:53.373 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.371+0000 7f2b00f76700 1 -- start start 2023-10-06T03:56:53.375 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.371+0000 7f2b00f76700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2afc0b1e40 0x7f2afc157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:53.375 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.371+0000 7f2b00f76700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2afc0b04d0 0x7f2afc06dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:53.375 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.371+0000 7f2b00f76700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2afc06d8d0 con 0x7f2afc06c2d0 2023-10-06T03:56:53.375 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.371+0000 7f2b00f76700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2afc0af8f0 con 0x7f2afc0b04d0 2023-10-06T03:56:53.375 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.371+0000 7f2b00f76700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc15a690 con 0x7f2afc0b1e40 2023-10-06T03:56:53.376 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.371+0000 7f2afb7fe700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2afc0b1e40 0x7f2afc157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:53.376 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.371+0000 7f2afaffd700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2afc0b04d0 0x7f2afc06dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:53.376 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.371+0000 7f2afbfff700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f2afc06c2d0 0x7f2afc1563b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:52756/0 (socket says 172.21.15.175:52756) 2023-10-06T03:56:53.376 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.371+0000 7f2afbfff700 1 -- 172.21.15.175:0/1760940014 learned_addr learned my addr 172.21.15.175:0/1760940014 (peer_addr_for_me v1:172.21.15.175:0/0) 2023-10-06T03:56:53.376 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.372+0000 7f2afaffd700 1 -- 172.21.15.175:0/1760940014 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2afc0b1e40 msgr2=0x7f2afc157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:56:53.376 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.372+0000 7f2afaffd700 1 --2- 172.21.15.175:0/1760940014 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2afc0b1e40 0x7f2afc157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:53.376 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.372+0000 7f2afaffd700 1 -- 172.21.15.175:0/1760940014 >> v1:172.21.15.175:6789/0 conn(0x7f2afc06c2d0 legacy=0x7f2afc1563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:53.376 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.372+0000 7f2afaffd700 1 -- 172.21.15.175:0/1760940014 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2afc06ccc0 con 0x7f2afc0b04d0 2023-10-06T03:56:53.376 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.372+0000 7f2afb7fe700 1 --2- 172.21.15.175:0/1760940014 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2afc0b1e40 0x7f2afc157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:56:53.376 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.372+0000 7f2afaffd700 1 --2- 172.21.15.175:0/1760940014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2afc0b04d0 0x7f2afc06dd60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2af00098e0 tx=0x7f2af00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9176b80fccae156 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:56:53.376 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.372+0000 7f2afa7fc700 1 -- 172.21.15.175:0/1760940014 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af000b640 con 0x7f2afc0b04d0 2023-10-06T03:56:53.377 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.372+0000 7f2afa7fc700 1 -- 172.21.15.175:0/1760940014 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af000b7a0 con 0x7f2afc0b04d0 2023-10-06T03:56:53.377 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.372+0000 7f2afa7fc700 1 -- 172.21.15.175:0/1760940014 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af000bb40 con 0x7f2afc0b04d0 2023-10-06T03:56:53.377 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.373+0000 7f2b00f76700 1 -- 172.21.15.175:0/1760940014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2afc0b04d0 msgr2=0x7f2afc06dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:53.377 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.373+0000 7f2b00f76700 1 --2- 172.21.15.175:0/1760940014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2afc0b04d0 0x7f2afc06dd60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2af00098e0 tx=0x7f2af00091a0 comp rx=0 tx=0).stop 2023-10-06T03:56:53.377 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.373+0000 7f2b00f76700 1 -- 172.21.15.175:0/1760940014 shutdown_connections 2023-10-06T03:56:53.377 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.373+0000 7f2b00f76700 1 --2- 172.21.15.175:0/1760940014 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2afc0b1e40 0x7f2afc157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:53.377 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.373+0000 7f2b00f76700 1 --2- 172.21.15.175:0/1760940014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2afc0b04d0 0x7f2afc06dd60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:53.377 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.373+0000 7f2b00f76700 1 -- 172.21.15.175:0/1760940014 >> 172.21.15.175:0/1760940014 conn(0x7f2afc0cc540 msgr2=0x7f2afc0cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:56:53.377 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.373+0000 7f2b00f76700 1 -- 172.21.15.175:0/1760940014 shutdown_connections 2023-10-06T03:56:53.377 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.373+0000 7f2b00f76700 1 -- 172.21.15.175:0/1760940014 wait complete. 2023-10-06T03:56:53.380 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.374+0000 7f2b00f76700 1 Processor -- start 2023-10-06T03:56:53.380 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.374+0000 7f2b00f76700 1 -- start start 2023-10-06T03:56:53.380 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.375+0000 7f2b00f76700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2afc06c2d0 0x7f2afc0cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:53.380 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.375+0000 7f2b00f76700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2afc0b04d0 0x7f2afc14dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:53.380 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.375+0000 7f2b00f76700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2afc0b1e40 0x7f2afc14e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:53.380 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.375+0000 7f2b00f76700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc153e80 con 0x7f2afc06c2d0 2023-10-06T03:56:53.380 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.375+0000 7f2b00f76700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc150050 con 0x7f2afc0b04d0 2023-10-06T03:56:53.380 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.375+0000 7f2b00f76700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc153be0 con 0x7f2afc0b1e40 2023-10-06T03:56:53.380 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.375+0000 7f2afaffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2afc0b04d0 0x7f2afc14dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:53.381 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.375+0000 7f2afbfff700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2afc0b1e40 0x7f2afc14e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:53.381 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.375+0000 7f2afaffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2afc0b04d0 0x7f2afc14dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:34410/0 (socket says 172.21.15.175:34410) 2023-10-06T03:56:53.381 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.375+0000 7f2afaffd700 1 -- 172.21.15.175:0/2862462088 learned_addr learned my addr 172.21.15.175:0/2862462088 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-10-06T03:56:53.381 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.375+0000 7f2afb7fe700 1 --2- 172.21.15.175:0/2862462088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2afc06c2d0 0x7f2afc0cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:53.381 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.375+0000 7f2afbfff700 1 -- 172.21.15.175:0/2862462088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2afc0b04d0 msgr2=0x7f2afc14dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:53.381 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.375+0000 7f2afbfff700 1 --2- 172.21.15.175:0/2862462088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2afc0b04d0 0x7f2afc14dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:53.381 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.375+0000 7f2afbfff700 1 -- 172.21.15.175:0/2862462088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2afc06c2d0 msgr2=0x7f2afc0cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:53.381 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.375+0000 7f2afbfff700 1 --2- 172.21.15.175:0/2862462088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2afc06c2d0 0x7f2afc0cb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:53.381 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.375+0000 7f2afbfff700 1 -- 172.21.15.175:0/2862462088 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aec009420 con 0x7f2afc0b1e40 2023-10-06T03:56:53.381 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.375+0000 7f2afbfff700 1 --2- 172.21.15.175:0/2862462088 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2afc0b1e40 0x7f2afc14e320 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2af400cb70 tx=0x7f2af4001900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:56:53.381 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.375+0000 7f2ae37fe700 1 -- 172.21.15.175:0/2862462088 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af400a870 con 0x7f2afc0b1e40 2023-10-06T03:56:53.381 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.375+0000 7f2ae37fe700 1 -- 172.21.15.175:0/2862462088 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af400a9d0 con 0x7f2afc0b1e40 2023-10-06T03:56:53.382 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.375+0000 7f2b00f76700 1 -- 172.21.15.175:0/2862462088 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af0009420 con 0x7f2afc0b1e40 2023-10-06T03:56:53.382 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.375+0000 7f2ae37fe700 1 -- 172.21.15.175:0/2862462088 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af400ad70 con 0x7f2afc0b1e40 2023-10-06T03:56:53.382 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.375+0000 7f2b00f76700 1 -- 172.21.15.175:0/2862462088 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2afc14fa00 con 0x7f2afc0b1e40 2023-10-06T03:56:53.382 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.375+0000 7f2ae37fe700 1 -- 172.21.15.175:0/2862462088 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f2af4007300 con 0x7f2afc0b1e40 2023-10-06T03:56:53.382 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.375+0000 7f2ae37fe700 1 -- 172.21.15.175:0/2862462088 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f2af401b750 con 0x7f2afc0b1e40 2023-10-06T03:56:53.382 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.376+0000 7f2b00f76700 1 -- 172.21.15.175:0/2862462088 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2afc088f90 con 0x7f2afc0b1e40 2023-10-06T03:56:53.382 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.378+0000 7f2ae37fe700 1 -- 172.21.15.175:0/2862462088 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f2af4007740 con 0x7f2afc0b1e40 2023-10-06T03:56:53.463 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.461+0000 7f2b00f76700 1 -- 172.21.15.175:0/2862462088 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3f677cac-e9f4-4608-acac-718b072ce984", "id": 0} v 0) v1 -- 0x7f2afc06cfa0 con 0x7f2afc0b1e40 2023-10-06T03:56:53.465 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-10-06T03:56:53.464+0000 7fe9d654c180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:56:53.466 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.464+0000 7f2ae37fe700 1 -- 172.21.15.175:0/2862462088 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3f677cac-e9f4-4608-acac-718b072ce984", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2af40078e0 con 0x7f2afc0b1e40 2023-10-06T03:56:53.466 INFO:teuthology.orchestra.run.smithi175.stdout:0 2023-10-06T03:56:53.467 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.465+0000 7f2b00f76700 1 -- 172.21.15.175:0/2862462088 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2afc0b1e40 msgr2=0x7f2afc14e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:53.467 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.465+0000 7f2b00f76700 1 --2- 172.21.15.175:0/2862462088 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2afc0b1e40 0x7f2afc14e320 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2af400cb70 tx=0x7f2af4001900 comp rx=0 tx=0).stop 2023-10-06T03:56:53.467 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.465+0000 7f2b00f76700 1 -- 172.21.15.175:0/2862462088 shutdown_connections 2023-10-06T03:56:53.468 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.465+0000 7f2b00f76700 1 --2- 172.21.15.175:0/2862462088 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2afc0b1e40 0x7f2afc14e320 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:53.468 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.465+0000 7f2b00f76700 1 --2- 172.21.15.175:0/2862462088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2afc06c2d0 0x7f2afc0cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:53.468 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.465+0000 7f2b00f76700 1 --2- 172.21.15.175:0/2862462088 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2afc0b04d0 0x7f2afc14dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:53.468 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.465+0000 7f2b00f76700 1 -- 172.21.15.175:0/2862462088 >> 172.21.15.175:0/2862462088 conn(0x7f2afc0cc540 msgr2=0x7f2afc156900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:56:53.468 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.465+0000 7f2b00f76700 1 -- 172.21.15.175:0/2862462088 shutdown_connections 2023-10-06T03:56:53.468 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.465+0000 7f2b00f76700 1 -- 172.21.15.175:0/2862462088 wait complete. 2023-10-06T03:56:53.472 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-10-06T03:56:53.470+0000 7fcdc1ec5180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:56:53.478 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 7d771892-bb91-4e9b-9610-051861c11247 1 2023-10-06T03:56:53.633 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.631+0000 7f21cdff2700 1 Processor -- start 2023-10-06T03:56:53.634 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.631+0000 7f21cdff2700 1 -- start start 2023-10-06T03:56:53.640 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.632+0000 7f21cdff2700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f21c806cbb0 0x7f21c806d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:53.641 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.632+0000 7f21cdff2700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21c80c9780 con 0x7f21c8150410 2023-10-06T03:56:53.641 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.632+0000 7f21cdff2700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21c806e0c0 con 0x7f21c806c1f0 2023-10-06T03:56:53.641 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.632+0000 7f21cdff2700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21c80c8c40 con 0x7f21c806cbb0 2023-10-06T03:56:53.641 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.632+0000 7f21c6ffd700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f21c806c1f0 0x7f21c814fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:52766/0 (socket says 172.21.15.175:52766) 2023-10-06T03:56:53.641 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.632+0000 7f21c6ffd700 1 -- 172.21.15.175:0/1257927602 learned_addr learned my addr 172.21.15.175:0/1257927602 (peer_addr_for_me v1:172.21.15.175:0/0) 2023-10-06T03:56:53.641 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.632+0000 7f21c77fe700 1 --2- 172.21.15.175:0/1257927602 >> v2:172.21.15.157:3300/0 conn(0x7f21c806cbb0 0x7f21c806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:53.641 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.632+0000 7f21c67fc700 1 -- 172.21.15.175:0/1257927602 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1228527321 0 0) 0x7f21c806e0c0 con 0x7f21c806c1f0 2023-10-06T03:56:53.641 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.632+0000 7f21c67fc700 1 -- 172.21.15.175:0/1257927602 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21b0003570 con 0x7f21c806c1f0 2023-10-06T03:56:53.642 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.632+0000 7f21c67fc700 1 -- 172.21.15.175:0/1257927602 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1034296508 0 0) 0x7f21c80c9780 con 0x7f21c8150410 2023-10-06T03:56:53.642 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.632+0000 7f21c67fc700 1 -- 172.21.15.175:0/1257927602 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21c806e0c0 con 0x7f21c8150410 2023-10-06T03:56:53.642 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.633+0000 7f21c77fe700 1 -- 172.21.15.175:0/1257927602 >> v1:172.21.15.175:6789/0 conn(0x7f21c806c1f0 legacy=0x7f21c814fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:53.642 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.633+0000 7f21c77fe700 1 -- 172.21.15.175:0/1257927602 >> v1:172.21.15.157:6789/0 conn(0x7f21c8150410 legacy=0x7f21c8155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:53.642 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.633+0000 7f21c77fe700 1 -- 172.21.15.175:0/1257927602 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c80c8e50 con 0x7f21c806cbb0 2023-10-06T03:56:53.642 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.633+0000 7f21c77fe700 1 --2- 172.21.15.175:0/1257927602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21c806cbb0 0x7f21c806d530 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f21b80098e0 tx=0x7f21b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb7bf68d213d761 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:56:53.642 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.633+0000 7f21c67fc700 1 -- 172.21.15.175:0/1257927602 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b8011630 con 0x7f21c806cbb0 2023-10-06T03:56:53.642 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.633+0000 7f21c67fc700 1 -- 172.21.15.175:0/1257927602 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b8011790 con 0x7f21c806cbb0 2023-10-06T03:56:53.642 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.633+0000 7f21c67fc700 1 -- 172.21.15.175:0/1257927602 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b8011b30 con 0x7f21c806cbb0 2023-10-06T03:56:53.643 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.633+0000 7f21cdff2700 1 -- 172.21.15.175:0/1257927602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21c806cbb0 msgr2=0x7f21c806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:53.643 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.633+0000 7f21cdff2700 1 --2- 172.21.15.175:0/1257927602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21c806cbb0 0x7f21c806d530 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f21b80098e0 tx=0x7f21b80091a0 comp rx=0 tx=0).stop 2023-10-06T03:56:53.643 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.633+0000 7f21cdff2700 1 -- 172.21.15.175:0/1257927602 shutdown_connections 2023-10-06T03:56:53.643 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.633+0000 7f21cdff2700 1 --2- 172.21.15.175:0/1257927602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21c806cbb0 0x7f21c806d530 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:53.643 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.633+0000 7f21cdff2700 1 -- 172.21.15.175:0/1257927602 >> 172.21.15.175:0/1257927602 conn(0x7f21c806f230 msgr2=0x7f21c806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:56:53.643 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.634+0000 7f21cdff2700 1 -- 172.21.15.175:0/1257927602 shutdown_connections 2023-10-06T03:56:53.643 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.634+0000 7f21cdff2700 1 -- 172.21.15.175:0/1257927602 wait complete. 2023-10-06T03:56:53.643 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.635+0000 7f21cdff2700 1 Processor -- start 2023-10-06T03:56:53.643 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.635+0000 7f21cdff2700 1 -- start start 2023-10-06T03:56:53.644 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.635+0000 7f21cdff2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21c806c1f0 0x7f21c80af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:53.644 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.635+0000 7f21cdff2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21c806cbb0 0x7f21c8157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:53.644 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.635+0000 7f21cdff2700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21c8150410 0x7f21c8158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:53.644 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.635+0000 7f21cdff2700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c816a8e0 con 0x7f21c806c1f0 2023-10-06T03:56:53.644 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.635+0000 7f21cdff2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c816a360 con 0x7f21c806cbb0 2023-10-06T03:56:53.644 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.635+0000 7f21cdff2700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c81579c0 con 0x7f21c8150410 2023-10-06T03:56:53.644 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.636+0000 7f21c6ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21c806cbb0 0x7f21c8157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:53.644 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.636+0000 7f21c6ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21c806cbb0 0x7f21c8157ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:34422/0 (socket says 172.21.15.175:34422) 2023-10-06T03:56:53.644 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.636+0000 7f21c6ffd700 1 -- 172.21.15.175:0/3689373201 learned_addr learned my addr 172.21.15.175:0/3689373201 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-10-06T03:56:53.644 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.636+0000 7f21c7fff700 1 --2- 172.21.15.175:0/3689373201 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21c8150410 0x7f21c8158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:53.644 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.636+0000 7f21c77fe700 1 --2- 172.21.15.175:0/3689373201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21c806c1f0 0x7f21c80af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:53.645 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.636+0000 7f21c6ffd700 1 -- 172.21.15.175:0/3689373201 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21c8150410 msgr2=0x7f21c8158420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:53.645 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.636+0000 7f21c6ffd700 1 --2- 172.21.15.175:0/3689373201 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21c8150410 0x7f21c8158420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:53.645 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.636+0000 7f21c6ffd700 1 -- 172.21.15.175:0/3689373201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21c806c1f0 msgr2=0x7f21c80af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:53.645 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.636+0000 7f21c6ffd700 1 --2- 172.21.15.175:0/3689373201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21c806c1f0 0x7f21c80af360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:53.645 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.636+0000 7f21c6ffd700 1 -- 172.21.15.175:0/3689373201 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b8009420 con 0x7f21c806cbb0 2023-10-06T03:56:53.645 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.636+0000 7f21c6ffd700 1 --2- 172.21.15.175:0/3689373201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21c806cbb0 0x7f21c8157ee0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f21bc00c790 tx=0x7f21bc007e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:56:53.645 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.636+0000 7f21af7fe700 1 -- 172.21.15.175:0/3689373201 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc005b30 con 0x7f21c806cbb0 2023-10-06T03:56:53.645 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.636+0000 7f21af7fe700 1 -- 172.21.15.175:0/3689373201 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21bc005c90 con 0x7f21c806cbb0 2023-10-06T03:56:53.645 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.636+0000 7f21cdff2700 1 -- 172.21.15.175:0/3689373201 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c816a600 con 0x7f21c806cbb0 2023-10-06T03:56:53.645 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.636+0000 7f21af7fe700 1 -- 172.21.15.175:0/3689373201 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc007300 con 0x7f21c806cbb0 2023-10-06T03:56:53.646 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.636+0000 7f21cdff2700 1 -- 172.21.15.175:0/3689373201 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c80c5a70 con 0x7f21c806cbb0 2023-10-06T03:56:53.646 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.636+0000 7f21af7fe700 1 -- 172.21.15.175:0/3689373201 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f21bc01c6c0 con 0x7f21c806cbb0 2023-10-06T03:56:53.646 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.636+0000 7f21af7fe700 1 -- 172.21.15.175:0/3689373201 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f21bc0127c0 con 0x7f21c806cbb0 2023-10-06T03:56:53.646 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.638+0000 7f21cdff2700 1 -- 172.21.15.175:0/3689373201 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c80c55f0 con 0x7f21c806cbb0 2023-10-06T03:56:53.646 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-10-06T03:56:53.638+0000 7fe9d654c180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:56:53.647 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.640+0000 7f21af7fe700 1 -- 172.21.15.175:0/3689373201 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f21c80c55f0 con 0x7f21c806cbb0 2023-10-06T03:56:53.652 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-10-06T03:56:53.650+0000 7fcdc1ec5180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:56:53.718 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-10-06T03:56:53.717+0000 7fe9d654c180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:56:53.724 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.722+0000 7f21cdff2700 1 -- 172.21.15.175:0/3689373201 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7d771892-bb91-4e9b-9610-051861c11247", "id": 1} v 0) v1 -- 0x7f21c8157c00 con 0x7f21c806cbb0 2023-10-06T03:56:53.727 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.725+0000 7f21af7fe700 1 -- 172.21.15.175:0/3689373201 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7d771892-bb91-4e9b-9610-051861c11247", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f21c8157c00 con 0x7f21c806cbb0 2023-10-06T03:56:53.727 INFO:teuthology.orchestra.run.smithi175.stdout:1 2023-10-06T03:56:53.728 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.726+0000 7f21cdff2700 1 -- 172.21.15.175:0/3689373201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21c806cbb0 msgr2=0x7f21c8157ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:53.728 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.726+0000 7f21cdff2700 1 --2- 172.21.15.175:0/3689373201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21c806cbb0 0x7f21c8157ee0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f21bc00c790 tx=0x7f21bc007e80 comp rx=0 tx=0).stop 2023-10-06T03:56:53.728 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.726+0000 7f21cdff2700 1 -- 172.21.15.175:0/3689373201 shutdown_connections 2023-10-06T03:56:53.728 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.726+0000 7f21cdff2700 1 --2- 172.21.15.175:0/3689373201 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21c8150410 0x7f21c8158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:53.728 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.726+0000 7f21cdff2700 1 --2- 172.21.15.175:0/3689373201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21c806c1f0 0x7f21c80af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:53.729 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.726+0000 7f21cdff2700 1 --2- 172.21.15.175:0/3689373201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21c806cbb0 0x7f21c8157ee0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:53.729 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.726+0000 7f21cdff2700 1 -- 172.21.15.175:0/3689373201 >> 172.21.15.175:0/3689373201 conn(0x7f21c806f230 msgr2=0x7f21c814d780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:56:53.729 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.726+0000 7f21cdff2700 1 -- 172.21.15.175:0/3689373201 shutdown_connections 2023-10-06T03:56:53.729 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.726+0000 7f21cdff2700 1 -- 172.21.15.175:0/3689373201 wait complete. 2023-10-06T03:56:53.730 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-10-06T03:56:53.729+0000 7fcdc1ec5180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:56:53.740 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 5c1f7d4e-d185-47cf-ab6c-453b4e3009c2 2 2023-10-06T03:56:53.855 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.852+0000 7fc4c6d42700 1 Processor -- start 2023-10-06T03:56:53.855 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.853+0000 7fc4c6d42700 1 -- start start 2023-10-06T03:56:53.855 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-10-06T03:56:53.854+0000 7fe9d654c180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:56:53.858 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.853+0000 7fc4c6d42700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4c00ca630 0x7fc4c00c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:53.858 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.853+0000 7fc4c6d42700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc4c006eed0 0x7fc4c00c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:53.858 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.853+0000 7fc4c6d42700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c00c68f0 con 0x7fc4c00688e0 2023-10-06T03:56:53.858 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.853+0000 7fc4c6d42700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4c006d9d0 con 0x7fc4c006eed0 2023-10-06T03:56:53.858 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.853+0000 7fc4c6d42700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c00c59e0 con 0x7fc4c00ca630 2023-10-06T03:56:53.858 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.854+0000 7fc4c4ade700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4c00ca630 0x7fc4c00c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:53.858 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.854+0000 7fc4c4ade700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4c00ca630 0x7fc4c00c6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:38168/0 (socket says 172.21.15.175:38168) 2023-10-06T03:56:53.858 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.854+0000 7fc4c4ade700 1 -- 172.21.15.175:0/2814621782 learned_addr learned my addr 172.21.15.175:0/2814621782 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-10-06T03:56:53.858 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.854+0000 7fc4bffff700 1 --2- 172.21.15.175:0/2814621782 >> v2:172.21.15.157:3300/0 conn(0x7fc4c006eed0 0x7fc4c00c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:53.859 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.854+0000 7fc4c4ade700 1 -- 172.21.15.175:0/2814621782 >> v2:172.21.15.157:3300/0 conn(0x7fc4c006eed0 msgr2=0x7fc4c00c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:56:53.859 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.854+0000 7fc4c4ade700 1 --2- 172.21.15.175:0/2814621782 >> v2:172.21.15.157:3300/0 conn(0x7fc4c006eed0 0x7fc4c00c72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:53.859 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.854+0000 7fc4c4ade700 1 -- 172.21.15.175:0/2814621782 >> v1:172.21.15.157:6789/0 conn(0x7fc4c00688e0 legacy=0x7fc4c0159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:53.859 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.854+0000 7fc4c4ade700 1 -- 172.21.15.175:0/2814621782 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c006e7d0 con 0x7fc4c00ca630 2023-10-06T03:56:53.859 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.854+0000 7fc4c4ade700 1 --2- 172.21.15.175:0/2814621782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4c00ca630 0x7fc4c00c6d80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40099b0 tx=0x7fc4b4009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f055a352cb5877a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:56:53.859 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.854+0000 7fc4bf7fe700 1 -- 172.21.15.175:0/2814621782 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4009ea0 con 0x7fc4c00ca630 2023-10-06T03:56:53.859 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.854+0000 7fc4bf7fe700 1 -- 172.21.15.175:0/2814621782 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b400b860 con 0x7fc4c00ca630 2023-10-06T03:56:53.859 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.854+0000 7fc4bf7fe700 1 -- 172.21.15.175:0/2814621782 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b400bc00 con 0x7fc4c00ca630 2023-10-06T03:56:53.859 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.855+0000 7fc4c6d42700 1 -- 172.21.15.175:0/2814621782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4c00ca630 msgr2=0x7fc4c00c6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:53.859 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.855+0000 7fc4c6d42700 1 --2- 172.21.15.175:0/2814621782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4c00ca630 0x7fc4c00c6d80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40099b0 tx=0x7fc4b4009250 comp rx=0 tx=0).stop 2023-10-06T03:56:53.859 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.855+0000 7fc4c6d42700 1 -- 172.21.15.175:0/2814621782 shutdown_connections 2023-10-06T03:56:53.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.855+0000 7fc4c6d42700 1 --2- 172.21.15.175:0/2814621782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4c00ca630 0x7fc4c00c6d80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:53.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.855+0000 7fc4c6d42700 1 --2- 172.21.15.175:0/2814621782 >> v2:172.21.15.157:3300/0 conn(0x7fc4c006eed0 0x7fc4c00c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:53.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.855+0000 7fc4c6d42700 1 -- 172.21.15.175:0/2814621782 >> 172.21.15.175:0/2814621782 conn(0x7fc4c00692b0 msgr2=0x7fc4c00696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:56:53.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.855+0000 7fc4c6d42700 1 -- 172.21.15.175:0/2814621782 shutdown_connections 2023-10-06T03:56:53.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.855+0000 7fc4c6d42700 1 -- 172.21.15.175:0/2814621782 wait complete. 2023-10-06T03:56:53.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.856+0000 7fc4c6d42700 1 Processor -- start 2023-10-06T03:56:53.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.856+0000 7fc4c6d42700 1 -- start start 2023-10-06T03:56:53.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.857+0000 7fc4c6d42700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4c00688e0 0x7fc4c00c0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:53.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.857+0000 7fc4c6d42700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4c006eed0 0x7fc4c00c0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:53.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.857+0000 7fc4c6d42700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c00ca630 0x7fc4c0150980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:53.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.857+0000 7fc4c6d42700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c01504f0 con 0x7fc4c00ca630 2023-10-06T03:56:53.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.857+0000 7fc4c6d42700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c0169160 con 0x7fc4c00688e0 2023-10-06T03:56:53.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.857+0000 7fc4c6d42700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c014f8e0 con 0x7fc4c006eed0 2023-10-06T03:56:53.864 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.857+0000 7fc4c4ade700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4c00688e0 0x7fc4c00c0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:53.864 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.857+0000 7fc4bffff700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4c006eed0 0x7fc4c00c0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:53.864 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.857+0000 7fc4c4ade700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4c00688e0 0x7fc4c00c0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:34432/0 (socket says 172.21.15.175:34432) 2023-10-06T03:56:53.864 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.857+0000 7fc4c4ade700 1 -- 172.21.15.175:0/1861426081 learned_addr learned my addr 172.21.15.175:0/1861426081 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-10-06T03:56:53.864 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.857+0000 7fc4bffff700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4c006eed0 0x7fc4c00c0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:38176/0 (socket says 172.21.15.175:38176) 2023-10-06T03:56:53.864 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.857+0000 7fc4c52df700 1 --2- 172.21.15.175:0/1861426081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c00ca630 0x7fc4c0150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:53.864 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.857+0000 7fc4bffff700 1 -- 172.21.15.175:0/1861426081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4c00688e0 msgr2=0x7fc4c00c0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:53.864 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.857+0000 7fc4bffff700 1 --2- 172.21.15.175:0/1861426081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4c00688e0 0x7fc4c00c0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:53.864 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.857+0000 7fc4bffff700 1 -- 172.21.15.175:0/1861426081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c00ca630 msgr2=0x7fc4c0150980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:53.864 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.857+0000 7fc4bffff700 1 --2- 172.21.15.175:0/1861426081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c00ca630 0x7fc4c0150980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:53.865 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.857+0000 7fc4bffff700 1 -- 172.21.15.175:0/1861426081 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b40094f0 con 0x7fc4c006eed0 2023-10-06T03:56:53.865 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.857+0000 7fc4c4ade700 1 --2- 172.21.15.175:0/1861426081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4c00688e0 0x7fc4c00c0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:56:53.865 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.857+0000 7fc4bffff700 1 --2- 172.21.15.175:0/1861426081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4c006eed0 0x7fc4c00c0dd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00ea50 tx=0x7fc4ac0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:56:53.865 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.857+0000 7fc4bcff9700 1 -- 172.21.15.175:0/1861426081 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac00ee90 con 0x7fc4c006eed0 2023-10-06T03:56:53.865 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.857+0000 7fc4bcff9700 1 -- 172.21.15.175:0/1861426081 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4ac00c7f0 con 0x7fc4c006eed0 2023-10-06T03:56:53.865 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.857+0000 7fc4c6d42700 1 -- 172.21.15.175:0/1861426081 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c014fbc0 con 0x7fc4c006eed0 2023-10-06T03:56:53.865 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.857+0000 7fc4bcff9700 1 -- 172.21.15.175:0/1861426081 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac00cb90 con 0x7fc4c006eed0 2023-10-06T03:56:53.865 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.857+0000 7fc4c6d42700 1 -- 172.21.15.175:0/1861426081 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c0150180 con 0x7fc4c006eed0 2023-10-06T03:56:53.865 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.858+0000 7fc4bcff9700 1 -- 172.21.15.175:0/1861426081 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fc4ac005c80 con 0x7fc4c006eed0 2023-10-06T03:56:53.865 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.858+0000 7fc4bcff9700 1 -- 172.21.15.175:0/1861426081 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7fc4ac013470 con 0x7fc4c006eed0 2023-10-06T03:56:53.865 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.858+0000 7fc4c6d42700 1 -- 172.21.15.175:0/1861426081 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c006a8f0 con 0x7fc4c006eed0 2023-10-06T03:56:53.866 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.860+0000 7fc4bcff9700 1 -- 172.21.15.175:0/1861426081 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fc4ac021060 con 0x7fc4c006eed0 2023-10-06T03:56:53.872 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-10-06T03:56:53.870+0000 7fcdc1ec5180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:56:53.945 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.943+0000 7fc4c6d42700 1 -- 172.21.15.175:0/1861426081 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5c1f7d4e-d185-47cf-ab6c-453b4e3009c2", "id": 2} v 0) v1 -- 0x7fc4c004bb40 con 0x7fc4c006eed0 2023-10-06T03:56:53.950 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.948+0000 7fc4bcff9700 1 -- 172.21.15.175:0/1861426081 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5c1f7d4e-d185-47cf-ab6c-453b4e3009c2", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc4ac017070 con 0x7fc4c006eed0 2023-10-06T03:56:53.950 INFO:teuthology.orchestra.run.smithi175.stdout:2 2023-10-06T03:56:53.951 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.949+0000 7fc4c6d42700 1 -- 172.21.15.175:0/1861426081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4c006eed0 msgr2=0x7fc4c00c0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:53.951 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.949+0000 7fc4c6d42700 1 --2- 172.21.15.175:0/1861426081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4c006eed0 0x7fc4c00c0dd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00ea50 tx=0x7fc4ac0091a0 comp rx=0 tx=0).stop 2023-10-06T03:56:53.952 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.949+0000 7fc4c6d42700 1 -- 172.21.15.175:0/1861426081 shutdown_connections 2023-10-06T03:56:53.952 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.949+0000 7fc4c6d42700 1 --2- 172.21.15.175:0/1861426081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4c00688e0 0x7fc4c00c0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:53.952 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.949+0000 7fc4c6d42700 1 --2- 172.21.15.175:0/1861426081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c00ca630 0x7fc4c0150980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:53.952 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.949+0000 7fc4c6d42700 1 --2- 172.21.15.175:0/1861426081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4c006eed0 0x7fc4c00c0dd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:53.952 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.949+0000 7fc4c6d42700 1 -- 172.21.15.175:0/1861426081 >> 172.21.15.175:0/1861426081 conn(0x7fc4c00692b0 msgr2=0x7fc4c0067930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:56:53.952 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.949+0000 7fc4c6d42700 1 -- 172.21.15.175:0/1861426081 shutdown_connections 2023-10-06T03:56:53.952 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:53.949+0000 7fc4c6d42700 1 -- 172.21.15.175:0/1861426081 wait complete. 2023-10-06T03:56:53.953 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-10-06T03:56:53.950+0000 7fe9d654c180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:56:53.963 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new f6a1e825-43c7-4afd-8daf-e51d45afc75e 3 2023-10-06T03:56:53.971 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-10-06T03:56:53.969+0000 7fcdc1ec5180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:56:54.078 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.076+0000 7f0047fb2700 1 Processor -- start 2023-10-06T03:56:54.078 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.076+0000 7f0047fb2700 1 -- start start 2023-10-06T03:56:54.081 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.077+0000 7f0047fb2700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f004006dc40 0x7f004014fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:54.081 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.077+0000 7f0047fb2700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f004006c1f0 0x7f0040150240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:54.081 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.077+0000 7f0047fb2700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00400c9780 con 0x7f004006cbb0 2023-10-06T03:56:54.081 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.077+0000 7f0047fb2700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00400c8c40 con 0x7f004006c1f0 2023-10-06T03:56:54.081 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.077+0000 7f0047fb2700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004006e0c0 con 0x7f004006dc40 2023-10-06T03:56:54.081 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.077+0000 7f004554d700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f004006dc40 0x7f004014fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:54.081 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.077+0000 7f0045d4e700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f004006cbb0 0x7f004006d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:52790/0 (socket says 172.21.15.175:52790) 2023-10-06T03:56:54.081 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.077+0000 7f0045d4e700 1 -- 172.21.15.175:0/1872365115 learned_addr learned my addr 172.21.15.175:0/1872365115 (peer_addr_for_me v1:172.21.15.175:0/0) 2023-10-06T03:56:54.081 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.077+0000 7f004654f700 1 --2- 172.21.15.175:0/1872365115 >> v2:172.21.15.157:3300/0 conn(0x7f004006c1f0 0x7f0040150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:54.081 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.077+0000 7f0044d4c700 1 -- 172.21.15.175:0/1872365115 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 576117080 0 0) 0x7f00400c9780 con 0x7f004006cbb0 2023-10-06T03:56:54.082 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.078+0000 7f0044d4c700 1 -- 172.21.15.175:0/1872365115 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f002c003570 con 0x7f004006cbb0 2023-10-06T03:56:54.082 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.078+0000 7f004554d700 1 -- 172.21.15.175:0/1872365115 >> v2:172.21.15.157:3300/0 conn(0x7f004006c1f0 msgr2=0x7f0040150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:56:54.082 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.078+0000 7f004554d700 1 --2- 172.21.15.175:0/1872365115 >> v2:172.21.15.157:3300/0 conn(0x7f004006c1f0 0x7f0040150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.082 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.078+0000 7f004554d700 1 -- 172.21.15.175:0/1872365115 >> v1:172.21.15.175:6789/0 conn(0x7f004006cbb0 legacy=0x7f004006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:54.082 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.078+0000 7f004554d700 1 -- 172.21.15.175:0/1872365115 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00400c8e50 con 0x7f004006dc40 2023-10-06T03:56:54.082 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.078+0000 7f004554d700 1 --2- 172.21.15.175:0/1872365115 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f004006dc40 0x7f004014fd00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f003c0098e0 tx=0x7f003c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb4a1b918f8b4074 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:56:54.082 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.078+0000 7f0044d4c700 1 -- 172.21.15.175:0/1872365115 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c011630 con 0x7f004006dc40 2023-10-06T03:56:54.082 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.078+0000 7f0044d4c700 1 -- 172.21.15.175:0/1872365115 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003c011790 con 0x7f004006dc40 2023-10-06T03:56:54.083 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.078+0000 7f0044d4c700 1 -- 172.21.15.175:0/1872365115 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c011b30 con 0x7f004006dc40 2023-10-06T03:56:54.083 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.078+0000 7f0047fb2700 1 -- 172.21.15.175:0/1872365115 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f004006dc40 msgr2=0x7f004014fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:54.083 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.078+0000 7f0047fb2700 1 --2- 172.21.15.175:0/1872365115 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f004006dc40 0x7f004014fd00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f003c0098e0 tx=0x7f003c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.083 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.078+0000 7f0047fb2700 1 -- 172.21.15.175:0/1872365115 shutdown_connections 2023-10-06T03:56:54.083 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.078+0000 7f0047fb2700 1 --2- 172.21.15.175:0/1872365115 >> v2:172.21.15.157:3300/0 conn(0x7f004006c1f0 0x7f0040150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.083 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.078+0000 7f0047fb2700 1 --2- 172.21.15.175:0/1872365115 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f004006dc40 0x7f004014fd00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.083 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.078+0000 7f0047fb2700 1 -- 172.21.15.175:0/1872365115 >> 172.21.15.175:0/1872365115 conn(0x7f004006f230 msgr2=0x7f004006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:56:54.083 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.079+0000 7f0047fb2700 1 -- 172.21.15.175:0/1872365115 shutdown_connections 2023-10-06T03:56:54.083 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.079+0000 7f0047fb2700 1 -- 172.21.15.175:0/1872365115 wait complete. 2023-10-06T03:56:54.085 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.080+0000 7f0047fb2700 1 Processor -- start 2023-10-06T03:56:54.085 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.080+0000 7f0047fb2700 1 -- start start 2023-10-06T03:56:54.085 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.080+0000 7f0047fb2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f004006c1f0 0x7f0040169a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:54.085 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.080+0000 7f0047fb2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f004006cbb0 0x7f0040169fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:54.086 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.080+0000 7f0047fb2700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0040156ac0 0x7f00400c26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:54.086 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.080+0000 7f0047fb2700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00400c53a0 con 0x7f004006c1f0 2023-10-06T03:56:54.086 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.080+0000 7f0047fb2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00400c2230 con 0x7f004006cbb0 2023-10-06T03:56:54.086 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.080+0000 7f0047fb2700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00400c1f50 con 0x7f0040156ac0 2023-10-06T03:56:54.086 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.080+0000 7f004554d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f004006cbb0 0x7f0040169fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:54.086 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.080+0000 7f004654f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0040156ac0 0x7f00400c26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:54.086 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.080+0000 7f004654f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0040156ac0 0x7f00400c26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:38190/0 (socket says 172.21.15.175:38190) 2023-10-06T03:56:54.086 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.080+0000 7f004654f700 1 -- 172.21.15.175:0/3596298311 learned_addr learned my addr 172.21.15.175:0/3596298311 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-10-06T03:56:54.086 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.080+0000 7f004554d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f004006cbb0 0x7f0040169fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:34446/0 (socket says 172.21.15.175:34446) 2023-10-06T03:56:54.086 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.080+0000 7f0045d4e700 1 --2- 172.21.15.175:0/3596298311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f004006c1f0 0x7f0040169a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:54.087 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.081+0000 7f004654f700 1 -- 172.21.15.175:0/3596298311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f004006cbb0 msgr2=0x7f0040169fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:54.087 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.081+0000 7f004654f700 1 --2- 172.21.15.175:0/3596298311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f004006cbb0 0x7f0040169fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.087 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.081+0000 7f004654f700 1 -- 172.21.15.175:0/3596298311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f004006c1f0 msgr2=0x7f0040169a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:54.087 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.081+0000 7f004654f700 1 --2- 172.21.15.175:0/3596298311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f004006c1f0 0x7f0040169a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.087 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.081+0000 7f004654f700 1 -- 172.21.15.175:0/3596298311 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003c009420 con 0x7f0040156ac0 2023-10-06T03:56:54.087 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.081+0000 7f0045d4e700 1 --2- 172.21.15.175:0/3596298311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f004006c1f0 0x7f0040169a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:56:54.087 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.081+0000 7f004554d700 1 --2- 172.21.15.175:0/3596298311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f004006cbb0 0x7f0040169fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:56:54.087 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.081+0000 7f004654f700 1 --2- 172.21.15.175:0/3596298311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0040156ac0 0x7f00400c26c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0038009ec0 tx=0x7f003800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:56:54.087 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.081+0000 7f0035ffb700 1 -- 172.21.15.175:0/3596298311 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003800cc20 con 0x7f0040156ac0 2023-10-06T03:56:54.087 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.081+0000 7f0035ffb700 1 -- 172.21.15.175:0/3596298311 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003800f040 con 0x7f0040156ac0 2023-10-06T03:56:54.087 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.081+0000 7f0047fb2700 1 -- 172.21.15.175:0/3596298311 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00400c4230 con 0x7f0040156ac0 2023-10-06T03:56:54.087 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.081+0000 7f0035ffb700 1 -- 172.21.15.175:0/3596298311 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0038014480 con 0x7f0040156ac0 2023-10-06T03:56:54.088 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.081+0000 7f0047fb2700 1 -- 172.21.15.175:0/3596298311 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00400c44d0 con 0x7f0040156ac0 2023-10-06T03:56:54.088 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.081+0000 7f0035ffb700 1 -- 172.21.15.175:0/3596298311 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f00380147f0 con 0x7f0040156ac0 2023-10-06T03:56:54.088 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.081+0000 7f0035ffb700 1 -- 172.21.15.175:0/3596298311 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f003801da80 con 0x7f0040156ac0 2023-10-06T03:56:54.088 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.082+0000 7f0047fb2700 1 -- 172.21.15.175:0/3596298311 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00400c23e0 con 0x7f0040156ac0 2023-10-06T03:56:54.088 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.084+0000 7f0035ffb700 1 -- 172.21.15.175:0/3596298311 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f00400c23e0 con 0x7f0040156ac0 2023-10-06T03:56:54.175 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.173+0000 7f0047fb2700 1 -- 172.21.15.175:0/3596298311 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f6a1e825-43c7-4afd-8daf-e51d45afc75e", "id": 3} v 0) v1 -- 0x7f00400c2090 con 0x7f0040156ac0 2023-10-06T03:56:54.181 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.179+0000 7f0035ffb700 1 -- 172.21.15.175:0/3596298311 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f6a1e825-43c7-4afd-8daf-e51d45afc75e", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f00400c2090 con 0x7f0040156ac0 2023-10-06T03:56:54.181 INFO:teuthology.orchestra.run.smithi175.stdout:3 2023-10-06T03:56:54.182 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.180+0000 7f0047fb2700 1 -- 172.21.15.175:0/3596298311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0040156ac0 msgr2=0x7f00400c26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:54.182 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.180+0000 7f0047fb2700 1 --2- 172.21.15.175:0/3596298311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0040156ac0 0x7f00400c26c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0038009ec0 tx=0x7f003800b040 comp rx=0 tx=0).stop 2023-10-06T03:56:54.182 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.180+0000 7f0047fb2700 1 -- 172.21.15.175:0/3596298311 shutdown_connections 2023-10-06T03:56:54.182 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.180+0000 7f0047fb2700 1 --2- 172.21.15.175:0/3596298311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0040156ac0 0x7f00400c26c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.182 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.180+0000 7f0047fb2700 1 --2- 172.21.15.175:0/3596298311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f004006c1f0 0x7f0040169a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.183 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.180+0000 7f0047fb2700 1 --2- 172.21.15.175:0/3596298311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f004006cbb0 0x7f0040169fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.183 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.180+0000 7f0047fb2700 1 -- 172.21.15.175:0/3596298311 >> 172.21.15.175:0/3596298311 conn(0x7f004006f230 msgr2=0x7f00400cbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:56:54.183 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.180+0000 7f0047fb2700 1 -- 172.21.15.175:0/3596298311 shutdown_connections 2023-10-06T03:56:54.183 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.180+0000 7f0047fb2700 1 -- 172.21.15.175:0/3596298311 wait complete. 2023-10-06T03:56:54.193 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 8ad6e3d8-9a31-4f85-9410-417b7ac5da97 4 2023-10-06T03:56:54.351 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.349+0000 7ffbb3626700 1 Processor -- start 2023-10-06T03:56:54.351 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.349+0000 7ffbb3626700 1 -- start start 2023-10-06T03:56:54.352 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.350+0000 7ffbb3626700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ffbac06dc40 0x7ffbac14fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:54.353 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.350+0000 7ffbb3626700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbac0c9780 con 0x7ffbac06cbb0 2023-10-06T03:56:54.353 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.350+0000 7ffbb3626700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbac06e0c0 con 0x7ffbac06c1f0 2023-10-06T03:56:54.353 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.350+0000 7ffbb3626700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbac0c8c40 con 0x7ffbac06dc40 2023-10-06T03:56:54.353 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.350+0000 7ffbb0bc1700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ffbac06dc40 0x7ffbac14fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:54.353 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.350+0000 7ffbb0bc1700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ffbac06dc40 0x7ffbac14fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:34452/0 (socket says 172.21.15.175:34452) 2023-10-06T03:56:54.353 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.350+0000 7ffbb0bc1700 1 -- 172.21.15.175:0/3781466704 learned_addr learned my addr 172.21.15.175:0/3781466704 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-10-06T03:56:54.353 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.350+0000 7ffbb1bc3700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7ffbac06c1f0 0x7ffbac155f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:52806/0 (socket says 172.21.15.175:52806) 2023-10-06T03:56:54.353 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.350+0000 7ffbb0bc1700 1 -- 172.21.15.175:0/3781466704 >> v1:172.21.15.175:6789/0 conn(0x7ffbac06c1f0 legacy=0x7ffbac155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:54.353 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.350+0000 7ffbb0bc1700 1 -- 172.21.15.175:0/3781466704 >> v1:172.21.15.157:6789/0 conn(0x7ffbac06cbb0 legacy=0x7ffbac06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:54.353 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.350+0000 7ffbb0bc1700 1 -- 172.21.15.175:0/3781466704 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbac0c8e50 con 0x7ffbac06dc40 2023-10-06T03:56:54.353 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.350+0000 7ffbb0bc1700 1 --2- 172.21.15.175:0/3781466704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbac06dc40 0x7ffbac14fd00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c0099b0 tx=0x7ffb9c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=911d2038d9eadbdf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:56:54.354 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.350+0000 7ffba3fff700 1 -- 172.21.15.175:0/3781466704 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c00b8f0 con 0x7ffbac06dc40 2023-10-06T03:56:54.354 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.350+0000 7ffba3fff700 1 -- 172.21.15.175:0/3781466704 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb9c00ba50 con 0x7ffbac06dc40 2023-10-06T03:56:54.354 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.350+0000 7ffba3fff700 1 -- 172.21.15.175:0/3781466704 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c00bdf0 con 0x7ffbac06dc40 2023-10-06T03:56:54.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.351+0000 7ffbb3626700 1 -- 172.21.15.175:0/3781466704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbac06dc40 msgr2=0x7ffbac14fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:54.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.351+0000 7ffbb3626700 1 --2- 172.21.15.175:0/3781466704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbac06dc40 0x7ffbac14fd00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c0099b0 tx=0x7ffb9c009250 comp rx=0 tx=0).stop 2023-10-06T03:56:54.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.351+0000 7ffbb3626700 1 -- 172.21.15.175:0/3781466704 shutdown_connections 2023-10-06T03:56:54.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.351+0000 7ffbb3626700 1 --2- 172.21.15.175:0/3781466704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbac06dc40 0x7ffbac14fd00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.351+0000 7ffbb3626700 1 -- 172.21.15.175:0/3781466704 >> 172.21.15.175:0/3781466704 conn(0x7ffbac06f230 msgr2=0x7ffbac06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:56:54.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.351+0000 7ffbb3626700 1 -- 172.21.15.175:0/3781466704 shutdown_connections 2023-10-06T03:56:54.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.351+0000 7ffbb3626700 1 -- 172.21.15.175:0/3781466704 wait complete. 2023-10-06T03:56:54.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.353+0000 7ffbb3626700 1 Processor -- start 2023-10-06T03:56:54.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.353+0000 7ffbb3626700 1 -- start start 2023-10-06T03:56:54.358 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.354+0000 7ffbb3626700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffbac06c1f0 0x7ffbac14b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:54.358 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.354+0000 7ffbb3626700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbac06cbb0 0x7ffbac167c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:54.358 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.354+0000 7ffbb3626700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbac0c6680 0x7ffbac1681c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:54.358 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.354+0000 7ffbb3626700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac16ad30 con 0x7ffbac06cbb0 2023-10-06T03:56:54.358 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.354+0000 7ffbb3626700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac154570 con 0x7ffbac0c6680 2023-10-06T03:56:54.358 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.354+0000 7ffbb3626700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac14ecc0 con 0x7ffbac06c1f0 2023-10-06T03:56:54.358 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.354+0000 7ffbb1bc3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbac0c6680 0x7ffbac1681c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:54.358 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.354+0000 7ffbb13c2700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffbac06c1f0 0x7ffbac14b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:54.358 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.354+0000 7ffbb1bc3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbac0c6680 0x7ffbac1681c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:34460/0 (socket says 172.21.15.175:34460) 2023-10-06T03:56:54.359 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.354+0000 7ffbb1bc3700 1 -- 172.21.15.175:0/383210895 learned_addr learned my addr 172.21.15.175:0/383210895 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-10-06T03:56:54.359 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.354+0000 7ffbb0bc1700 1 --2- 172.21.15.175:0/383210895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbac06cbb0 0x7ffbac167c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:54.359 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.355+0000 7ffbb1bc3700 1 -- 172.21.15.175:0/383210895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffbac06c1f0 msgr2=0x7ffbac14b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:54.359 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.355+0000 7ffbb1bc3700 1 --2- 172.21.15.175:0/383210895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffbac06c1f0 0x7ffbac14b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.359 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.355+0000 7ffbb1bc3700 1 -- 172.21.15.175:0/383210895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbac06cbb0 msgr2=0x7ffbac167c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:54.359 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.355+0000 7ffbb1bc3700 1 --2- 172.21.15.175:0/383210895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbac06cbb0 0x7ffbac167c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.359 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.355+0000 7ffbb1bc3700 1 -- 172.21.15.175:0/383210895 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9c0094f0 con 0x7ffbac0c6680 2023-10-06T03:56:54.359 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.355+0000 7ffbb13c2700 1 --2- 172.21.15.175:0/383210895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffbac06c1f0 0x7ffbac14b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:56:54.359 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.355+0000 7ffbb1bc3700 1 --2- 172.21.15.175:0/383210895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbac0c6680 0x7ffbac1681c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ffba400cd60 tx=0x7ffba4001b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:56:54.359 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.355+0000 7ffba17fa700 1 -- 172.21.15.175:0/383210895 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba4005df0 con 0x7ffbac0c6680 2023-10-06T03:56:54.359 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.355+0000 7ffba17fa700 1 -- 172.21.15.175:0/383210895 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba400a830 con 0x7ffbac0c6680 2023-10-06T03:56:54.359 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.355+0000 7ffbb3626700 1 -- 172.21.15.175:0/383210895 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbac0c4b90 con 0x7ffbac0c6680 2023-10-06T03:56:54.360 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.355+0000 7ffba17fa700 1 -- 172.21.15.175:0/383210895 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba400abf0 con 0x7ffbac0c6680 2023-10-06T03:56:54.360 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.355+0000 7ffbb3626700 1 -- 172.21.15.175:0/383210895 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbac1688c0 con 0x7ffbac0c6680 2023-10-06T03:56:54.360 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.355+0000 7ffba17fa700 1 -- 172.21.15.175:0/383210895 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7ffba40070f0 con 0x7ffbac0c6680 2023-10-06T03:56:54.360 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.355+0000 7ffba17fa700 1 -- 172.21.15.175:0/383210895 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7ffba401b910 con 0x7ffbac0c6680 2023-10-06T03:56:54.360 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.356+0000 7ffbb3626700 1 -- 172.21.15.175:0/383210895 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbac0c4960 con 0x7ffbac0c6680 2023-10-06T03:56:54.360 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.358+0000 7ffba17fa700 1 -- 172.21.15.175:0/383210895 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7ffbac0c4960 con 0x7ffbac0c6680 2023-10-06T03:56:54.449 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.447+0000 7ffbb3626700 1 -- 172.21.15.175:0/383210895 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8ad6e3d8-9a31-4f85-9410-417b7ac5da97", "id": 4} v 0) v1 -- 0x7ffbac0c4d20 con 0x7ffbac0c6680 2023-10-06T03:56:54.457 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.455+0000 7ffba17fa700 1 -- 172.21.15.175:0/383210895 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8ad6e3d8-9a31-4f85-9410-417b7ac5da97", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7ffba4007580 con 0x7ffbac0c6680 2023-10-06T03:56:54.457 INFO:teuthology.orchestra.run.smithi175.stdout:4 2023-10-06T03:56:54.458 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.456+0000 7ffbb3626700 1 -- 172.21.15.175:0/383210895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbac0c6680 msgr2=0x7ffbac1681c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:54.458 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.456+0000 7ffbb3626700 1 --2- 172.21.15.175:0/383210895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbac0c6680 0x7ffbac1681c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ffba400cd60 tx=0x7ffba4001b10 comp rx=0 tx=0).stop 2023-10-06T03:56:54.458 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.456+0000 7ffbb3626700 1 -- 172.21.15.175:0/383210895 shutdown_connections 2023-10-06T03:56:54.459 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.456+0000 7ffbb3626700 1 --2- 172.21.15.175:0/383210895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffbac0c6680 0x7ffbac1681c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.459 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.456+0000 7ffbb3626700 1 --2- 172.21.15.175:0/383210895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffbac06c1f0 0x7ffbac14b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.459 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.456+0000 7ffbb3626700 1 --2- 172.21.15.175:0/383210895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbac06cbb0 0x7ffbac167c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.459 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.456+0000 7ffbb3626700 1 -- 172.21.15.175:0/383210895 >> 172.21.15.175:0/383210895 conn(0x7ffbac06f230 msgr2=0x7ffbac0caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:56:54.459 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.456+0000 7ffbb3626700 1 -- 172.21.15.175:0/383210895 shutdown_connections 2023-10-06T03:56:54.459 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.456+0000 7ffbb3626700 1 -- 172.21.15.175:0/383210895 wait complete. 2023-10-06T03:56:54.463 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-10-06T03:56:54.462+0000 7fe9d654c180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:56:54.469 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new f68aca2a-f182-4c00-8e21-241d09c02ed3 5 2023-10-06T03:56:54.517 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-10-06T03:56:54.515+0000 7fcdc1ec5180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:56:54.541 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-10-06T03:56:54.539+0000 7fe9d654c180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:56:54.583 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.581+0000 7fe3cbc07700 1 Processor -- start 2023-10-06T03:56:54.583 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.581+0000 7fe3cbc07700 1 -- start start 2023-10-06T03:56:54.585 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.582+0000 7fe3cbc07700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3c406c810 0x7fe3c4069130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:54.585 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.582+0000 7fe3cbc07700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3c4155f20 con 0x7fe3c40c5fc0 2023-10-06T03:56:54.585 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.582+0000 7fe3cbc07700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3c406d4a0 con 0x7fe3c406c440 2023-10-06T03:56:54.585 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.582+0000 7fe3cbc07700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c406f310 con 0x7fe3c406c810 2023-10-06T03:56:54.585 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.582+0000 7fe3ca1a4700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3c406c810 0x7fe3c4069130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:54.585 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.582+0000 7fe3c99a3700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fe3c406c440 0x7fe3c4152e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:52814/0 (socket says 172.21.15.175:52814) 2023-10-06T03:56:54.586 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.582+0000 7fe3c99a3700 1 -- 172.21.15.175:0/2716089137 learned_addr learned my addr 172.21.15.175:0/2716089137 (peer_addr_for_me v1:172.21.15.175:0/0) 2023-10-06T03:56:54.586 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.582+0000 7fe3c89a1700 1 -- 172.21.15.175:0/2716089137 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3385857487 0 0) 0x7fe3c406d4a0 con 0x7fe3c406c440 2023-10-06T03:56:54.586 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.582+0000 7fe3ca1a4700 1 -- 172.21.15.175:0/2716089137 >> v1:172.21.15.175:6789/0 conn(0x7fe3c406c440 legacy=0x7fe3c4152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:54.586 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.582+0000 7fe3ca1a4700 1 -- 172.21.15.175:0/2716089137 >> v1:172.21.15.157:6789/0 conn(0x7fe3c40c5fc0 legacy=0x7fe3c415e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:54.586 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.582+0000 7fe3ca1a4700 1 -- 172.21.15.175:0/2716089137 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c406d780 con 0x7fe3c406c810 2023-10-06T03:56:54.586 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.582+0000 7fe3ca1a4700 1 --2- 172.21.15.175:0/2716089137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3c406c810 0x7fe3c4069130 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc009e90 tx=0x7fe3bc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e2688664ef137e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:56:54.586 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.582+0000 7fe3c89a1700 1 -- 172.21.15.175:0/2716089137 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc00e070 con 0x7fe3c406c810 2023-10-06T03:56:54.586 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.582+0000 7fe3c89a1700 1 -- 172.21.15.175:0/2716089137 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3bc00bb30 con 0x7fe3c406c810 2023-10-06T03:56:54.586 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.582+0000 7fe3c89a1700 1 -- 172.21.15.175:0/2716089137 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc012430 con 0x7fe3c406c810 2023-10-06T03:56:54.586 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.583+0000 7fe3cbc07700 1 -- 172.21.15.175:0/2716089137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3c406c810 msgr2=0x7fe3c4069130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:54.586 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.583+0000 7fe3cbc07700 1 --2- 172.21.15.175:0/2716089137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3c406c810 0x7fe3c4069130 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc009e90 tx=0x7fe3bc009780 comp rx=0 tx=0).stop 2023-10-06T03:56:54.587 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.583+0000 7fe3cbc07700 1 -- 172.21.15.175:0/2716089137 shutdown_connections 2023-10-06T03:56:54.587 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.583+0000 7fe3cbc07700 1 --2- 172.21.15.175:0/2716089137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3c406c810 0x7fe3c4069130 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.587 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.583+0000 7fe3cbc07700 1 -- 172.21.15.175:0/2716089137 >> 172.21.15.175:0/2716089137 conn(0x7fe3c4068370 msgr2=0x7fe3c4068770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:56:54.587 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.583+0000 7fe3cbc07700 1 -- 172.21.15.175:0/2716089137 shutdown_connections 2023-10-06T03:56:54.587 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.583+0000 7fe3cbc07700 1 -- 172.21.15.175:0/2716089137 wait complete. 2023-10-06T03:56:54.589 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.584+0000 7fe3cbc07700 1 Processor -- start 2023-10-06T03:56:54.589 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.584+0000 7fe3cbc07700 1 -- start start 2023-10-06T03:56:54.589 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.585+0000 7fe3cbc07700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3c406c440 0x7fe3c4063950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:54.589 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.585+0000 7fe3cbc07700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3c406c810 0x7fe3c4165e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:54.589 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.585+0000 7fe3cbc07700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3c40c5fc0 0x7fe3c414f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:54.589 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.585+0000 7fe3cbc07700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c41579c0 con 0x7fe3c406c440 2023-10-06T03:56:54.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.585+0000 7fe3cbc07700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c4157e50 con 0x7fe3c406c810 2023-10-06T03:56:54.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.585+0000 7fe3cbc07700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c4158d30 con 0x7fe3c40c5fc0 2023-10-06T03:56:54.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.585+0000 7fe3ca1a4700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3c40c5fc0 0x7fe3c414f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:54.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.585+0000 7fe3c91a2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3c406c810 0x7fe3c4165e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:54.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.585+0000 7fe3c91a2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3c406c810 0x7fe3c4165e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:34470/0 (socket says 172.21.15.175:34470) 2023-10-06T03:56:54.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.585+0000 7fe3ca1a4700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3c40c5fc0 0x7fe3c414f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:38214/0 (socket says 172.21.15.175:38214) 2023-10-06T03:56:54.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.585+0000 7fe3c91a2700 1 -- 172.21.15.175:0/3067620623 learned_addr learned my addr 172.21.15.175:0/3067620623 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-10-06T03:56:54.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.585+0000 7fe3c99a3700 1 --2- 172.21.15.175:0/3067620623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3c406c440 0x7fe3c4063950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:54.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.585+0000 7fe3ca1a4700 1 -- 172.21.15.175:0/3067620623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3c406c810 msgr2=0x7fe3c4165e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:54.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.585+0000 7fe3ca1a4700 1 --2- 172.21.15.175:0/3067620623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3c406c810 0x7fe3c4165e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.585+0000 7fe3ca1a4700 1 -- 172.21.15.175:0/3067620623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3c406c440 msgr2=0x7fe3c4063950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:54.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.585+0000 7fe3ca1a4700 1 --2- 172.21.15.175:0/3067620623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3c406c440 0x7fe3c4063950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.591 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.585+0000 7fe3ca1a4700 1 -- 172.21.15.175:0/3067620623 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3bc009a20 con 0x7fe3c40c5fc0 2023-10-06T03:56:54.591 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.585+0000 7fe3c91a2700 1 --2- 172.21.15.175:0/3067620623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3c406c810 0x7fe3c4165e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:56:54.591 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.585+0000 7fe3ca1a4700 1 --2- 172.21.15.175:0/3067620623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3c40c5fc0 0x7fe3c414f970 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc015000 tx=0x7fe3bc018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:56:54.591 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.585+0000 7fe3b9ffb700 1 -- 172.21.15.175:0/3067620623 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc027470 con 0x7fe3c40c5fc0 2023-10-06T03:56:54.591 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.585+0000 7fe3b9ffb700 1 -- 172.21.15.175:0/3067620623 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3bc0275f0 con 0x7fe3c40c5fc0 2023-10-06T03:56:54.591 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.585+0000 7fe3cbc07700 1 -- 172.21.15.175:0/3067620623 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3c4159000 con 0x7fe3c40c5fc0 2023-10-06T03:56:54.591 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.585+0000 7fe3b9ffb700 1 -- 172.21.15.175:0/3067620623 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc0279b0 con 0x7fe3c40c5fc0 2023-10-06T03:56:54.591 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.586+0000 7fe3cbc07700 1 -- 172.21.15.175:0/3067620623 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c4157370 con 0x7fe3c40c5fc0 2023-10-06T03:56:54.591 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.586+0000 7fe3b9ffb700 1 -- 172.21.15.175:0/3067620623 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fe3bc024070 con 0x7fe3c40c5fc0 2023-10-06T03:56:54.591 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.586+0000 7fe3b9ffb700 1 -- 172.21.15.175:0/3067620623 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7fe3bc027da0 con 0x7fe3c40c5fc0 2023-10-06T03:56:54.591 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.586+0000 7fe3cbc07700 1 -- 172.21.15.175:0/3067620623 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3c406a8f0 con 0x7fe3c40c5fc0 2023-10-06T03:56:54.592 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.588+0000 7fe3b9ffb700 1 -- 172.21.15.175:0/3067620623 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fe3bc02c020 con 0x7fe3c40c5fc0 2023-10-06T03:56:54.598 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-10-06T03:56:54.596+0000 7fcdc1ec5180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:56:54.619 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-10-06T03:56:54.618+0000 7fe9d654c180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:56:54.673 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.670+0000 7fe3cbc07700 1 -- 172.21.15.175:0/3067620623 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f68aca2a-f182-4c00-8e21-241d09c02ed3", "id": 5} v 0) v1 -- 0x7fe3c4157b30 con 0x7fe3c40c5fc0 2023-10-06T03:56:54.677 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-10-06T03:56:54.675+0000 7fcdc1ec5180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:56:54.680 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.678+0000 7fe3b9ffb700 1 -- 172.21.15.175:0/3067620623 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f68aca2a-f182-4c00-8e21-241d09c02ed3", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe3bc036020 con 0x7fe3c40c5fc0 2023-10-06T03:56:54.681 INFO:teuthology.orchestra.run.smithi175.stdout:5 2023-10-06T03:56:54.682 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.679+0000 7fe3cbc07700 1 -- 172.21.15.175:0/3067620623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3c40c5fc0 msgr2=0x7fe3c414f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:54.682 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.679+0000 7fe3cbc07700 1 --2- 172.21.15.175:0/3067620623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3c40c5fc0 0x7fe3c414f970 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc015000 tx=0x7fe3bc018000 comp rx=0 tx=0).stop 2023-10-06T03:56:54.682 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.680+0000 7fe3cbc07700 1 -- 172.21.15.175:0/3067620623 shutdown_connections 2023-10-06T03:56:54.682 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.680+0000 7fe3cbc07700 1 --2- 172.21.15.175:0/3067620623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3c40c5fc0 0x7fe3c414f970 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.682 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.680+0000 7fe3cbc07700 1 --2- 172.21.15.175:0/3067620623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3c406c440 0x7fe3c4063950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.682 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.680+0000 7fe3cbc07700 1 --2- 172.21.15.175:0/3067620623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3c406c810 0x7fe3c4165e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.682 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.680+0000 7fe3cbc07700 1 -- 172.21.15.175:0/3067620623 >> 172.21.15.175:0/3067620623 conn(0x7fe3c4068370 msgr2=0x7fe3c4152120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:56:54.682 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.680+0000 7fe3cbc07700 1 -- 172.21.15.175:0/3067620623 shutdown_connections 2023-10-06T03:56:54.683 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.680+0000 7fe3cbc07700 1 -- 172.21.15.175:0/3067620623 wait complete. 2023-10-06T03:56:54.693 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new d867cd89-3430-40ca-862f-d2bea3de9d85 6 2023-10-06T03:56:54.808 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.806+0000 7faaccfce700 1 Processor -- start 2023-10-06T03:56:54.808 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.806+0000 7faaccfce700 1 -- start start 2023-10-06T03:56:54.810 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.807+0000 7faaccfce700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faac8150500 0x7faac80c7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:54.810 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.807+0000 7faaccfce700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faac8157dd0 0x7faac80c1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:54.810 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.807+0000 7faaccfce700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac8156220 con 0x7faac8150130 2023-10-06T03:56:54.810 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.807+0000 7faaccfce700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac815a560 con 0x7faac8157dd0 2023-10-06T03:56:54.810 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.807+0000 7faaccfce700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac8156010 con 0x7faac8150500 2023-10-06T03:56:54.810 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.807+0000 7faac659c700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7faac8150130 0x7faac81576c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:52826/0 (socket says 172.21.15.175:52826) 2023-10-06T03:56:54.810 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.807+0000 7faac5d9b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faac8150500 0x7faac80c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:54.810 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.807+0000 7faac659c700 1 -- 172.21.15.175:0/1515854161 learned_addr learned my addr 172.21.15.175:0/1515854161 (peer_addr_for_me v1:172.21.15.175:0/0) 2023-10-06T03:56:54.810 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.807+0000 7faac6d9d700 1 --2- 172.21.15.175:0/1515854161 >> v2:172.21.15.157:3300/0 conn(0x7faac8157dd0 0x7faac80c1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:54.811 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.807+0000 7faac559a700 1 -- 172.21.15.175:0/1515854161 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 119111362 0 0) 0x7faac8156220 con 0x7faac8150130 2023-10-06T03:56:54.811 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.807+0000 7faac5d9b700 1 -- 172.21.15.175:0/1515854161 >> v2:172.21.15.157:3300/0 conn(0x7faac8157dd0 msgr2=0x7faac80c1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:56:54.811 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.807+0000 7faac5d9b700 1 --2- 172.21.15.175:0/1515854161 >> v2:172.21.15.157:3300/0 conn(0x7faac8157dd0 0x7faac80c1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.811 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.807+0000 7faac5d9b700 1 -- 172.21.15.175:0/1515854161 >> v1:172.21.15.175:6789/0 conn(0x7faac8150130 legacy=0x7faac81576c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:54.811 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.807+0000 7faac5d9b700 1 -- 172.21.15.175:0/1515854161 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac8156500 con 0x7faac8150500 2023-10-06T03:56:54.811 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.808+0000 7faac5d9b700 1 --2- 172.21.15.175:0/1515854161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faac8150500 0x7faac80c7080 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faabc0098e0 tx=0x7faabc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a49f20c303d1c34 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:56:54.811 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.808+0000 7faac559a700 1 -- 172.21.15.175:0/1515854161 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faabc00b6d0 con 0x7faac8150500 2023-10-06T03:56:54.811 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.808+0000 7faac559a700 1 -- 172.21.15.175:0/1515854161 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faabc00b830 con 0x7faac8150500 2023-10-06T03:56:54.813 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.808+0000 7faac559a700 1 -- 172.21.15.175:0/1515854161 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faabc00bbd0 con 0x7faac8150500 2023-10-06T03:56:54.816 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.808+0000 7faaccfce700 1 -- 172.21.15.175:0/1515854161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faac8150500 msgr2=0x7faac80c7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:54.816 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.808+0000 7faaccfce700 1 --2- 172.21.15.175:0/1515854161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faac8150500 0x7faac80c7080 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faabc0098e0 tx=0x7faabc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.816 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.808+0000 7faaccfce700 1 -- 172.21.15.175:0/1515854161 shutdown_connections 2023-10-06T03:56:54.816 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.808+0000 7faaccfce700 1 --2- 172.21.15.175:0/1515854161 >> v2:172.21.15.157:3300/0 conn(0x7faac8157dd0 0x7faac80c1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.816 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.808+0000 7faaccfce700 1 --2- 172.21.15.175:0/1515854161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faac8150500 0x7faac80c7080 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.816 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.809+0000 7faaccfce700 1 -- 172.21.15.175:0/1515854161 >> 172.21.15.175:0/1515854161 conn(0x7faac80631f0 msgr2=0x7faac806f200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:56:54.817 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.809+0000 7faaccfce700 1 -- 172.21.15.175:0/1515854161 shutdown_connections 2023-10-06T03:56:54.817 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.809+0000 7faaccfce700 1 -- 172.21.15.175:0/1515854161 wait complete. 2023-10-06T03:56:54.817 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.810+0000 7faaccfce700 1 Processor -- start 2023-10-06T03:56:54.817 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.810+0000 7faaccfce700 1 -- start start 2023-10-06T03:56:54.817 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.810+0000 7faaccfce700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faac8150130 0x7faac81486e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:54.817 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.810+0000 7faaccfce700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faac8150500 0x7faac8149800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:54.817 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.810+0000 7faaccfce700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faac8157dd0 0x7faac80c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:54.817 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.810+0000 7faaccfce700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac80c99e0 con 0x7faac8150130 2023-10-06T03:56:54.817 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.810+0000 7faaccfce700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac80c8150 con 0x7faac8150500 2023-10-06T03:56:54.817 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.810+0000 7faaccfce700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac8165cf0 con 0x7faac8157dd0 2023-10-06T03:56:54.817 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.810+0000 7faac6d9d700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faac8157dd0 0x7faac80c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:54.818 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.810+0000 7faac5d9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faac8150500 0x7faac8149800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:54.818 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.810+0000 7faac6d9d700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faac8157dd0 0x7faac80c85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:38226/0 (socket says 172.21.15.175:38226) 2023-10-06T03:56:54.818 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.810+0000 7faac6d9d700 1 -- 172.21.15.175:0/358701829 learned_addr learned my addr 172.21.15.175:0/358701829 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-10-06T03:56:54.818 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.810+0000 7faac659c700 1 --2- 172.21.15.175:0/358701829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faac8150130 0x7faac81486e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:54.818 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.811+0000 7faac5d9b700 1 -- 172.21.15.175:0/358701829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faac8157dd0 msgr2=0x7faac80c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:54.818 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.811+0000 7faac5d9b700 1 --2- 172.21.15.175:0/358701829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faac8157dd0 0x7faac80c85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.818 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.811+0000 7faac5d9b700 1 -- 172.21.15.175:0/358701829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faac8150130 msgr2=0x7faac81486e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:54.818 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.811+0000 7faac5d9b700 1 --2- 172.21.15.175:0/358701829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faac8150130 0x7faac81486e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.818 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.811+0000 7faac5d9b700 1 -- 172.21.15.175:0/358701829 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faabc009420 con 0x7faac8150500 2023-10-06T03:56:54.818 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.811+0000 7faac6d9d700 1 --2- 172.21.15.175:0/358701829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faac8157dd0 0x7faac80c85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:56:54.818 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.811+0000 7faac659c700 1 --2- 172.21.15.175:0/358701829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faac8150130 0x7faac81486e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:56:54.819 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.811+0000 7faac5d9b700 1 --2- 172.21.15.175:0/358701829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faac8150500 0x7faac8149800 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7faac808a020 tx=0x7faabc017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:56:54.819 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.811+0000 7faab67fc700 1 -- 172.21.15.175:0/358701829 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faabc000d10 con 0x7faac8150500 2023-10-06T03:56:54.819 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.811+0000 7faab67fc700 1 -- 172.21.15.175:0/358701829 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faabc00bbb0 con 0x7faac8150500 2023-10-06T03:56:54.819 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.811+0000 7faab67fc700 1 -- 172.21.15.175:0/358701829 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faabc011650 con 0x7faac8150500 2023-10-06T03:56:54.819 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.811+0000 7faaccfce700 1 -- 172.21.15.175:0/358701829 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac80c7b00 con 0x7faac8150500 2023-10-06T03:56:54.819 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.811+0000 7faaccfce700 1 -- 172.21.15.175:0/358701829 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac8148ca0 con 0x7faac8150500 2023-10-06T03:56:54.819 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.811+0000 7faab67fc700 1 -- 172.21.15.175:0/358701829 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7faabc011c40 con 0x7faac8150500 2023-10-06T03:56:54.819 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.811+0000 7faab67fc700 1 -- 172.21.15.175:0/358701829 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7faabc0193b0 con 0x7faac8150500 2023-10-06T03:56:54.819 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.812+0000 7faaccfce700 1 -- 172.21.15.175:0/358701829 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac80c7fa0 con 0x7faac8150500 2023-10-06T03:56:54.820 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.814+0000 7faab67fc700 1 -- 172.21.15.175:0/358701829 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7faabc02a060 con 0x7faac8150500 2023-10-06T03:56:54.901 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.899+0000 7faaccfce700 1 -- 172.21.15.175:0/358701829 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d867cd89-3430-40ca-862f-d2bea3de9d85", "id": 6} v 0) v1 -- 0x7faac806d550 con 0x7faac8150500 2023-10-06T03:56:54.907 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.905+0000 7faab67fc700 1 -- 172.21.15.175:0/358701829 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d867cd89-3430-40ca-862f-d2bea3de9d85", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7faabc015020 con 0x7faac8150500 2023-10-06T03:56:54.907 INFO:teuthology.orchestra.run.smithi175.stdout:6 2023-10-06T03:56:54.908 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.906+0000 7faaccfce700 1 -- 172.21.15.175:0/358701829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faac8150500 msgr2=0x7faac8149800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:54.908 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.906+0000 7faaccfce700 1 --2- 172.21.15.175:0/358701829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faac8150500 0x7faac8149800 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7faac808a020 tx=0x7faabc017000 comp rx=0 tx=0).stop 2023-10-06T03:56:54.909 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.906+0000 7faaccfce700 1 -- 172.21.15.175:0/358701829 shutdown_connections 2023-10-06T03:56:54.909 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.906+0000 7faaccfce700 1 --2- 172.21.15.175:0/358701829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faac8157dd0 0x7faac80c85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.909 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.906+0000 7faaccfce700 1 --2- 172.21.15.175:0/358701829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faac8150130 0x7faac81486e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.909 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.906+0000 7faaccfce700 1 --2- 172.21.15.175:0/358701829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faac8150500 0x7faac8149800 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:54.909 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.906+0000 7faaccfce700 1 -- 172.21.15.175:0/358701829 >> 172.21.15.175:0/358701829 conn(0x7faac80631f0 msgr2=0x7faac8158f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:56:54.909 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.906+0000 7faaccfce700 1 -- 172.21.15.175:0/358701829 shutdown_connections 2023-10-06T03:56:54.909 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:54.906+0000 7faaccfce700 1 -- 172.21.15.175:0/358701829 wait complete. 2023-10-06T03:56:54.921 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 930cd464-79f1-47cb-b4ff-188ccde74fff 7 2023-10-06T03:56:55.033 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.030+0000 7fada788f700 1 Processor -- start 2023-10-06T03:56:55.033 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.031+0000 7fada788f700 1 -- start start 2023-10-06T03:56:55.035 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.031+0000 7fada788f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fada006cbb0 0x7fada006d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:55.035 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.031+0000 7fada788f700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fada006c1f0 0x7fada006da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:55.035 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.031+0000 7fada788f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fada006cf80 0x7fada014fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:55.035 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.031+0000 7fada788f700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fada00c8c40 con 0x7fada006c1f0 2023-10-06T03:56:55.036 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.031+0000 7fada788f700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fada00c9780 con 0x7fada006cf80 2023-10-06T03:56:55.036 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.031+0000 7fada788f700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada006e0c0 con 0x7fada006cbb0 2023-10-06T03:56:55.036 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.032+0000 7fada562b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fada006cbb0 0x7fada006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:55.036 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.032+0000 7fada5e2c700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fada006cf80 0x7fada014fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:55.036 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.032+0000 7fada562b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fada006cbb0 0x7fada006d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:38228/0 (socket says 172.21.15.175:38228) 2023-10-06T03:56:55.036 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.032+0000 7fada562b700 1 -- 172.21.15.175:0/3017942088 learned_addr learned my addr 172.21.15.175:0/3017942088 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-10-06T03:56:55.036 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.032+0000 7fada4e2a700 1 --2- 172.21.15.175:0/3017942088 >> v2:172.21.15.157:3300/0 conn(0x7fada006c1f0 0x7fada006da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:55.036 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.032+0000 7fada562b700 1 -- 172.21.15.175:0/3017942088 >> v2:172.21.15.175:3300/0 conn(0x7fada006cf80 msgr2=0x7fada014fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:56:55.036 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.032+0000 7fada562b700 1 --2- 172.21.15.175:0/3017942088 >> v2:172.21.15.175:3300/0 conn(0x7fada006cf80 0x7fada014fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:55.036 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.032+0000 7fada562b700 1 -- 172.21.15.175:0/3017942088 >> v2:172.21.15.157:3300/0 conn(0x7fada006c1f0 msgr2=0x7fada006da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:56:55.036 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.032+0000 7fada562b700 1 --2- 172.21.15.175:0/3017942088 >> v2:172.21.15.157:3300/0 conn(0x7fada006c1f0 0x7fada006da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:55.037 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.032+0000 7fada562b700 1 -- 172.21.15.175:0/3017942088 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada00c8e50 con 0x7fada006cbb0 2023-10-06T03:56:55.037 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.032+0000 7fada4e2a700 1 --2- 172.21.15.175:0/3017942088 >> v2:172.21.15.157:3300/0 conn(0x7fada006c1f0 0x7fada006da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:56:55.037 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.032+0000 7fada562b700 1 --2- 172.21.15.175:0/3017942088 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fada006cbb0 0x7fada006d530 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fad940099b0 tx=0x7fad94009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc41c217515793b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:56:55.037 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.032+0000 7fad93fff700 1 -- 172.21.15.175:0/3017942088 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad94009ea0 con 0x7fada006cbb0 2023-10-06T03:56:55.037 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.032+0000 7fad93fff700 1 -- 172.21.15.175:0/3017942088 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad9400b8b0 con 0x7fada006cbb0 2023-10-06T03:56:55.037 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.032+0000 7fad93fff700 1 -- 172.21.15.175:0/3017942088 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9400bc50 con 0x7fada006cbb0 2023-10-06T03:56:55.037 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.033+0000 7fada788f700 1 -- 172.21.15.175:0/3017942088 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fada006cbb0 msgr2=0x7fada006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:55.037 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.033+0000 7fada788f700 1 --2- 172.21.15.175:0/3017942088 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fada006cbb0 0x7fada006d530 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fad940099b0 tx=0x7fad94009250 comp rx=0 tx=0).stop 2023-10-06T03:56:55.037 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.033+0000 7fada788f700 1 -- 172.21.15.175:0/3017942088 shutdown_connections 2023-10-06T03:56:55.037 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.033+0000 7fada788f700 1 --2- 172.21.15.175:0/3017942088 >> v2:172.21.15.175:3300/0 conn(0x7fada006cf80 0x7fada014fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:55.037 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.033+0000 7fada788f700 1 --2- 172.21.15.175:0/3017942088 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fada006cbb0 0x7fada006d530 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:55.038 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.033+0000 7fada788f700 1 --2- 172.21.15.175:0/3017942088 >> v2:172.21.15.157:3300/0 conn(0x7fada006c1f0 0x7fada006da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:55.038 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.033+0000 7fada788f700 1 -- 172.21.15.175:0/3017942088 >> 172.21.15.175:0/3017942088 conn(0x7fada006f230 msgr2=0x7fada006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:56:55.038 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.033+0000 7fada788f700 1 -- 172.21.15.175:0/3017942088 shutdown_connections 2023-10-06T03:56:55.038 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.033+0000 7fada788f700 1 -- 172.21.15.175:0/3017942088 wait complete. 2023-10-06T03:56:55.040 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.034+0000 7fada788f700 1 Processor -- start 2023-10-06T03:56:55.040 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.034+0000 7fada788f700 1 -- start start 2023-10-06T03:56:55.040 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.035+0000 7fada788f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fada006c1f0 0x7fada00c2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:55.041 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.035+0000 7fada788f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fada006cbb0 0x7fada0166520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:55.041 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.035+0000 7fada788f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fada006cf80 0x7fada0159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:55.041 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.035+0000 7fada788f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada0155700 con 0x7fada006cf80 2023-10-06T03:56:55.041 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.035+0000 7fada788f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada01559a0 con 0x7fada006c1f0 2023-10-06T03:56:55.041 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.035+0000 7fada788f700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada0155c80 con 0x7fada006cbb0 2023-10-06T03:56:55.041 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.035+0000 7fada4e2a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fada006cbb0 0x7fada0166520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:55.041 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.035+0000 7fada4e2a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fada006cbb0 0x7fada0166520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:38236/0 (socket says 172.21.15.175:38236) 2023-10-06T03:56:55.041 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.035+0000 7fada4e2a700 1 -- 172.21.15.175:0/2605194235 learned_addr learned my addr 172.21.15.175:0/2605194235 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-10-06T03:56:55.041 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.035+0000 7fada562b700 1 --2- 172.21.15.175:0/2605194235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fada006c1f0 0x7fada00c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:55.042 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.035+0000 7fada5e2c700 1 --2- 172.21.15.175:0/2605194235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fada006cf80 0x7fada0159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:55.042 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.035+0000 7fada4e2a700 1 -- 172.21.15.175:0/2605194235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fada006c1f0 msgr2=0x7fada00c2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:55.042 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.035+0000 7fada4e2a700 1 --2- 172.21.15.175:0/2605194235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fada006c1f0 0x7fada00c2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:55.042 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.035+0000 7fada4e2a700 1 -- 172.21.15.175:0/2605194235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fada006cf80 msgr2=0x7fada0159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:55.042 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.035+0000 7fada4e2a700 1 --2- 172.21.15.175:0/2605194235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fada006cf80 0x7fada0159120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:55.042 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.035+0000 7fada4e2a700 1 -- 172.21.15.175:0/2605194235 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad940094f0 con 0x7fada006cbb0 2023-10-06T03:56:55.042 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.035+0000 7fada562b700 1 --2- 172.21.15.175:0/2605194235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fada006c1f0 0x7fada00c2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:56:55.042 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.035+0000 7fada4e2a700 1 --2- 172.21.15.175:0/2605194235 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fada006cbb0 0x7fada0166520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fad9c00e8d0 tx=0x7fad9c008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:56:55.042 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.035+0000 7fad917fa700 1 -- 172.21.15.175:0/2605194235 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9c013410 con 0x7fada006cbb0 2023-10-06T03:56:55.042 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.035+0000 7fad917fa700 1 -- 172.21.15.175:0/2605194235 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad9c013570 con 0x7fada006cbb0 2023-10-06T03:56:55.042 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.035+0000 7fada788f700 1 -- 172.21.15.175:0/2605194235 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fada00bfcf0 con 0x7fada006cbb0 2023-10-06T03:56:55.043 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.035+0000 7fad917fa700 1 -- 172.21.15.175:0/2605194235 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9c013910 con 0x7fada006cbb0 2023-10-06T03:56:55.043 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.036+0000 7fada788f700 1 -- 172.21.15.175:0/2605194235 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada00c02b0 con 0x7fada006cbb0 2023-10-06T03:56:55.043 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.037+0000 7fad917fa700 1 -- 172.21.15.175:0/2605194235 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fad9c013d00 con 0x7fada006cbb0 2023-10-06T03:56:55.043 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.037+0000 7fada788f700 1 -- 172.21.15.175:0/2605194235 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fada00b0020 con 0x7fada006cbb0 2023-10-06T03:56:55.043 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.037+0000 7fad917fa700 1 -- 172.21.15.175:0/2605194235 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7fad9c01ba80 con 0x7fada006cbb0 2023-10-06T03:56:55.043 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.039+0000 7fad917fa700 1 -- 172.21.15.175:0/2605194235 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fad9c020060 con 0x7fada006cbb0 2023-10-06T03:56:55.141 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.139+0000 7fada788f700 1 -- 172.21.15.175:0/2605194235 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd new", "uuid": "930cd464-79f1-47cb-b4ff-188ccde74fff", "id": 7} v 0) v1 -- 0x7fada00c04d0 con 0x7fada006cbb0 2023-10-06T03:56:55.146 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.145+0000 7fad917fa700 1 -- 172.21.15.175:0/2605194235 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "930cd464-79f1-47cb-b4ff-188ccde74fff", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fad9c02a0b0 con 0x7fada006cbb0 2023-10-06T03:56:55.147 INFO:teuthology.orchestra.run.smithi175.stdout:7 2023-10-06T03:56:55.148 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.146+0000 7fada788f700 1 -- 172.21.15.175:0/2605194235 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fada006cbb0 msgr2=0x7fada0166520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:55.148 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.146+0000 7fada788f700 1 --2- 172.21.15.175:0/2605194235 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fada006cbb0 0x7fada0166520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fad9c00e8d0 tx=0x7fad9c008040 comp rx=0 tx=0).stop 2023-10-06T03:56:55.148 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.146+0000 7fada788f700 1 -- 172.21.15.175:0/2605194235 shutdown_connections 2023-10-06T03:56:55.148 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.146+0000 7fada788f700 1 --2- 172.21.15.175:0/2605194235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fada006c1f0 0x7fada00c2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:55.148 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.146+0000 7fada788f700 1 --2- 172.21.15.175:0/2605194235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fada006cf80 0x7fada0159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:55.148 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.146+0000 7fada788f700 1 --2- 172.21.15.175:0/2605194235 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fada006cbb0 0x7fada0166520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:55.149 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.146+0000 7fada788f700 1 -- 172.21.15.175:0/2605194235 >> 172.21.15.175:0/2605194235 conn(0x7fada006f230 msgr2=0x7fada006d530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:56:55.149 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.146+0000 7fada788f700 1 -- 172.21.15.175:0/2605194235 shutdown_connections 2023-10-06T03:56:55.149 INFO:teuthology.orchestra.run.smithi175.stderr:2023-10-06T03:56:55.146+0000 7fada788f700 1 -- 172.21.15.175:0/2605194235 wait complete. 2023-10-06T03:56:55.164 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-06T03:56:55.165 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-06T03:56:55.167 INFO:tasks.ceph.osd.0:Started 2023-10-06T03:56:55.167 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-06T03:56:55.167 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-06T03:56:55.169 INFO:tasks.ceph.osd.1:Started 2023-10-06T03:56:55.169 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-06T03:56:55.169 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-06T03:56:55.171 INFO:tasks.ceph.osd.2:Started 2023-10-06T03:56:55.171 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-06T03:56:55.172 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-06T03:56:55.174 INFO:tasks.ceph.osd.3:Started 2023-10-06T03:56:55.174 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-06T03:56:55.174 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-06T03:56:55.176 INFO:tasks.ceph.osd.4:Started 2023-10-06T03:56:55.176 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-06T03:56:55.176 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-06T03:56:55.178 INFO:tasks.ceph.osd.5:Started 2023-10-06T03:56:55.178 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-06T03:56:55.178 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-06T03:56:55.180 INFO:tasks.ceph.osd.6:Started 2023-10-06T03:56:55.180 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-06T03:56:55.180 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-06T03:56:55.182 INFO:tasks.ceph.osd.7:Started 2023-10-06T03:56:55.183 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:56:55.213 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-10-06T03:56:55.210+0000 7fe9d654c180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:56:55.316 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-10-06T03:56:55.314+0000 7fcdc1ec5180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:56:55.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.344+0000 7f96c0b85700 1 Processor -- start 2023-10-06T03:56:55.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.344+0000 7f96c0b85700 1 -- start start 2023-10-06T03:56:55.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.345+0000 7f96c0b85700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96bc0b04d0 0x7f96bc157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:55.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.345+0000 7f96c0b85700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f96bc06c2d0 0x7f96bc06dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:55.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.345+0000 7f96c0b85700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96bc06d8d0 con 0x7f96bc0b1e40 2023-10-06T03:56:55.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.345+0000 7f96c0b85700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96bc0af8f0 con 0x7f96bc06c2d0 2023-10-06T03:56:55.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.345+0000 7f96c0b85700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc15a690 con 0x7f96bc0b04d0 2023-10-06T03:56:55.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.345+0000 7f96bbfff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f96bc06c2d0 0x7f96bc06dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:55.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.345+0000 7f96bb7fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f96bc0b1e40 0x7f96bc1563b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41842/0 (socket says 172.21.15.157:41842) 2023-10-06T03:56:55.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.345+0000 7f96bb7fe700 1 -- 172.21.15.157:0/3555748022 learned_addr learned my addr 172.21.15.157:0/3555748022 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:56:55.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.345+0000 7f96b2ffd700 1 --2- 172.21.15.157:0/3555748022 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96bc0b04d0 0x7f96bc157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:55.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.346+0000 7f96baffd700 1 -- 172.21.15.157:0/3555748022 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 408775427 0 0) 0x7f96bc06d8d0 con 0x7f96bc0b1e40 2023-10-06T03:56:55.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.346+0000 7f96baffd700 1 -- 172.21.15.157:0/3555748022 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96a8003570 con 0x7f96bc0b1e40 2023-10-06T03:56:55.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.346+0000 7f96bbfff700 1 -- 172.21.15.157:0/3555748022 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96bc0b04d0 msgr2=0x7f96bc157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:56:55.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.346+0000 7f96bbfff700 1 --2- 172.21.15.157:0/3555748022 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96bc0b04d0 0x7f96bc157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:55.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.346+0000 7f96bbfff700 1 -- 172.21.15.157:0/3555748022 >> v1:172.21.15.157:6789/0 conn(0x7f96bc0b1e40 legacy=0x7f96bc1563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:55.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.346+0000 7f96bbfff700 1 -- 172.21.15.157:0/3555748022 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96bc06ccc0 con 0x7f96bc06c2d0 2023-10-06T03:56:55.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.346+0000 7f96bbfff700 1 --2- 172.21.15.157:0/3555748022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96bc06c2d0 0x7f96bc06dd60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009e60 tx=0x7f96b4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59c38554fff853a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:56:55.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.346+0000 7f96baffd700 1 -- 172.21.15.157:0/3555748022 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b400e070 con 0x7f96bc06c2d0 2023-10-06T03:56:55.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.346+0000 7f96baffd700 1 -- 172.21.15.157:0/3555748022 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b400bab0 con 0x7f96bc06c2d0 2023-10-06T03:56:55.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.346+0000 7f96baffd700 1 -- 172.21.15.157:0/3555748022 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b400be50 con 0x7f96bc06c2d0 2023-10-06T03:56:55.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.346+0000 7f96c0b85700 1 -- 172.21.15.157:0/3555748022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96bc06c2d0 msgr2=0x7f96bc06dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:55.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.347+0000 7f96c0b85700 1 --2- 172.21.15.157:0/3555748022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96bc06c2d0 0x7f96bc06dd60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009e60 tx=0x7f96b4009780 comp rx=0 tx=0).stop 2023-10-06T03:56:55.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.347+0000 7f96c0b85700 1 -- 172.21.15.157:0/3555748022 shutdown_connections 2023-10-06T03:56:55.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.347+0000 7f96c0b85700 1 --2- 172.21.15.157:0/3555748022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96bc06c2d0 0x7f96bc06dd60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:55.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.347+0000 7f96c0b85700 1 --2- 172.21.15.157:0/3555748022 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96bc0b04d0 0x7f96bc157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:55.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.347+0000 7f96c0b85700 1 -- 172.21.15.157:0/3555748022 >> 172.21.15.157:0/3555748022 conn(0x7f96bc0cc540 msgr2=0x7f96bc0cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:56:55.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.347+0000 7f96c0b85700 1 -- 172.21.15.157:0/3555748022 shutdown_connections 2023-10-06T03:56:55.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.347+0000 7f96c0b85700 1 -- 172.21.15.157:0/3555748022 wait complete. 2023-10-06T03:56:55.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.348+0000 7f96c0b85700 1 Processor -- start 2023-10-06T03:56:55.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.348+0000 7f96c0b85700 1 -- start start 2023-10-06T03:56:55.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.349+0000 7f96c0b85700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96bc06c2d0 0x7f96bc169090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:55.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.349+0000 7f96c0b85700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96bc0b04d0 0x7f96bc0c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:55.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.349+0000 7f96c0b85700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96bc0b1e40 0x7f96bc0c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:55.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.349+0000 7f96c0b85700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc0c82d0 con 0x7f96bc0b04d0 2023-10-06T03:56:55.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.349+0000 7f96c0b85700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc15d6c0 con 0x7f96bc06c2d0 2023-10-06T03:56:55.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.349+0000 7f96c0b85700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc169d60 con 0x7f96bc0b1e40 2023-10-06T03:56:55.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.349+0000 7f96b2ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96bc0b04d0 0x7f96bc0c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:55.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.349+0000 7f96b2ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96bc0b04d0 0x7f96bc0c8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52292/0 (socket says 172.21.15.157:52292) 2023-10-06T03:56:55.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.349+0000 7f96b2ffd700 1 -- 172.21.15.157:0/35268973 learned_addr learned my addr 172.21.15.157:0/35268973 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:56:55.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.349+0000 7f96bbfff700 1 --2- 172.21.15.157:0/35268973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96bc0b1e40 0x7f96bc0c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:55.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.349+0000 7f96bb7fe700 1 --2- 172.21.15.157:0/35268973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96bc06c2d0 0x7f96bc169090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:55.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.349+0000 7f96b2ffd700 1 -- 172.21.15.157:0/35268973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96bc0b1e40 msgr2=0x7f96bc0c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:55.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.349+0000 7f96b2ffd700 1 --2- 172.21.15.157:0/35268973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96bc0b1e40 0x7f96bc0c8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:55.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.349+0000 7f96b2ffd700 1 -- 172.21.15.157:0/35268973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96bc06c2d0 msgr2=0x7f96bc169090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:55.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.349+0000 7f96b2ffd700 1 --2- 172.21.15.157:0/35268973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96bc06c2d0 0x7f96bc169090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:55.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.349+0000 7f96b2ffd700 1 -- 172.21.15.157:0/35268973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b4009a20 con 0x7f96bc0b04d0 2023-10-06T03:56:55.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.349+0000 7f96b2ffd700 1 --2- 172.21.15.157:0/35268973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96bc0b04d0 0x7f96bc0c8760 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f96a400ecb0 tx=0x7f96a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:56:55.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.349+0000 7f96b3fff700 1 -- 172.21.15.157:0/35268973 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a400c850 con 0x7f96bc0b04d0 2023-10-06T03:56:55.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.349+0000 7f96b3fff700 1 -- 172.21.15.157:0/35268973 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a400c9b0 con 0x7f96bc0b04d0 2023-10-06T03:56:55.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.349+0000 7f96c0b85700 1 -- 172.21.15.157:0/35268973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96bc15d420 con 0x7f96bc0b04d0 2023-10-06T03:56:55.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.349+0000 7f96b3fff700 1 -- 172.21.15.157:0/35268973 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a400cd50 con 0x7f96bc0b04d0 2023-10-06T03:56:55.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.349+0000 7f96c0b85700 1 -- 172.21.15.157:0/35268973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96bc169a80 con 0x7f96bc0b04d0 2023-10-06T03:56:55.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.350+0000 7f96b3fff700 1 -- 172.21.15.157:0/35268973 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f96a4005c80 con 0x7f96bc0b04d0 2023-10-06T03:56:55.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.350+0000 7f96b3fff700 1 -- 172.21.15.157:0/35268973 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f96a4006700 con 0x7f96bc0b04d0 2023-10-06T03:56:55.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.350+0000 7f96c0b85700 1 -- 172.21.15.157:0/35268973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96bc088f90 con 0x7f96bc0b04d0 2023-10-06T03:56:55.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.352+0000 7f96b3fff700 1 -- 172.21.15.157:0/35268973 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f96a4021060 con 0x7f96bc0b04d0 2023-10-06T03:56:55.390 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-10-06T03:56:55.388+0000 7fe9d654c180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:56:55.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.421+0000 7f96c0b85700 1 -- 172.21.15.157:0/35268973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96bc0c8410 con 0x7f96bc0b04d0 2023-10-06T03:56:55.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.422+0000 7f96b3fff700 1 -- 172.21.15.157:0/35268973 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f96a4017070 con 0x7f96bc0b04d0 2023-10-06T03:56:55.424 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:56:55.424 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":10,"fsid":"28664a7a-fe30-4efe-be5a-6abf9318b700","created":"2023-10-06T03:56:51.887538+0000","modified":"2023-10-06T03:56:55.141505+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T03:56:55.141505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"3f677cac-e9f4-4608-acac-718b072ce984","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7d771892-bb91-4e9b-9610-051861c11247","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5c1f7d4e-d185-47cf-ab6c-453b4e3009c2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f6a1e825-43c7-4afd-8daf-e51d45afc75e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8ad6e3d8-9a31-4f85-9410-417b7ac5da97","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f68aca2a-f182-4c00-8e21-241d09c02ed3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d867cd89-3430-40ca-862f-d2bea3de9d85","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"930cd464-79f1-47cb-b4ff-188ccde74fff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:56:55.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.423+0000 7f96c0b85700 1 -- 172.21.15.157:0/35268973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96bc0b04d0 msgr2=0x7f96bc0c8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:55.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.423+0000 7f96c0b85700 1 --2- 172.21.15.157:0/35268973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96bc0b04d0 0x7f96bc0c8760 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f96a400ecb0 tx=0x7f96a40091a0 comp rx=0 tx=0).stop 2023-10-06T03:56:55.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.423+0000 7f96c0b85700 1 -- 172.21.15.157:0/35268973 shutdown_connections 2023-10-06T03:56:55.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.423+0000 7f96c0b85700 1 --2- 172.21.15.157:0/35268973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96bc06c2d0 0x7f96bc169090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:55.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.423+0000 7f96c0b85700 1 --2- 172.21.15.157:0/35268973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96bc0b1e40 0x7f96bc0c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:55.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.423+0000 7f96c0b85700 1 --2- 172.21.15.157:0/35268973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96bc0b04d0 0x7f96bc0c8760 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:55.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.423+0000 7f96c0b85700 1 -- 172.21.15.157:0/35268973 >> 172.21.15.157:0/35268973 conn(0x7f96bc0cc540 msgr2=0x7f96bc1573c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:56:55.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.423+0000 7f96c0b85700 1 -- 172.21.15.157:0/35268973 shutdown_connections 2023-10-06T03:56:55.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:56:55.423+0000 7f96c0b85700 1 -- 172.21.15.157:0/35268973 wait complete. 2023-10-06T03:56:55.436 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-06T03:56:55.437 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-06T03:56:55.451 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-10-06T03:56:55.449+0000 7fcdc1ec5180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:56:55.474 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-10-06T03:56:55.473+0000 7fe9d654c180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:56:55.533 INFO:tasks.ceph.osd.1.smithi157.stderr:2023-10-06T03:56:55.531+0000 7f3208a08540 -1 Falling back to public interface 2023-10-06T03:56:55.536 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-10-06T03:56:55.535+0000 7fcdc1ec5180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:56:55.746 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-10-06T03:56:55.744+0000 7fe9d654c180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:56:55.773 INFO:tasks.ceph.osd.4.smithi175.stderr:2023-10-06T03:56:55.770+0000 7eff49554540 -1 Falling back to public interface 2023-10-06T03:56:55.773 INFO:tasks.ceph.osd.6.smithi175.stderr:2023-10-06T03:56:55.770+0000 7f0497c48540 -1 Falling back to public interface 2023-10-06T03:56:55.774 INFO:tasks.ceph.osd.4.smithi175.stderr:2023-10-06T03:56:55.770+0000 7eff4397d700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6801/0: (98) Address already in use 2023-10-06T03:56:55.774 INFO:tasks.ceph.osd.6.smithi175.stderr:2023-10-06T03:56:55.770+0000 7f0492071700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6801/0: (98) Address already in use 2023-10-06T03:56:55.774 INFO:tasks.ceph.osd.6.smithi175.stderr:2023-10-06T03:56:55.771+0000 7f0492071700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6804/0: (98) Address already in use 2023-10-06T03:56:55.775 INFO:tasks.ceph.osd.0.smithi157.stderr:2023-10-06T03:56:55.771+0000 7f5b77935540 -1 Falling back to public interface 2023-10-06T03:56:55.777 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-10-06T03:56:55.775+0000 7fd05550d540 -1 Falling back to public interface 2023-10-06T03:56:55.784 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-10-06T03:56:55.782+0000 7fcdc1ec5180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:56:55.790 INFO:tasks.ceph.osd.3.smithi157.stderr:2023-10-06T03:56:55.788+0000 7f88ea1cd540 -1 Falling back to public interface 2023-10-06T03:56:55.793 INFO:tasks.ceph.osd.2.smithi157.stderr:2023-10-06T03:56:55.792+0000 7fbf64d2d540 -1 Falling back to public interface 2023-10-06T03:56:55.803 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-10-06T03:56:55.801+0000 7f4552c3c540 -1 Falling back to public interface 2023-10-06T03:56:55.826 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-10-06T03:56:55.824+0000 7fe9d654c180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:56:55.862 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-10-06T03:56:55.860+0000 7fcdc1ec5180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:56:56.242 DEBUG:teuthology.orchestra.run.smithi157:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T03:56:56.351 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.349+0000 7f323c85a700 1 Processor -- start 2023-10-06T03:56:56.351 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.349+0000 7f323c85a700 1 -- start start 2023-10-06T03:56:56.352 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.350+0000 7f323c85a700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f323806ad80 0x7f3238068550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:56.352 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.350+0000 7f323c85a700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f323806cd40 0x7f3238068a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:56.353 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.350+0000 7f323c85a700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3238153ed0 con 0x7f323806a9b0 2023-10-06T03:56:56.353 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.350+0000 7f323c85a700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32380c46e0 con 0x7f323806cd40 2023-10-06T03:56:56.353 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.350+0000 7f323c85a700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32381541b0 con 0x7f323806ad80 2023-10-06T03:56:56.353 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.350+0000 7f3236d9d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f323806cd40 0x7f3238068a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:56.353 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.350+0000 7f3236d9d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f323806cd40 0x7f3238068a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52296/0 (socket says 172.21.15.157:52296) 2023-10-06T03:56:56.353 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.350+0000 7f3236d9d700 1 -- 172.21.15.157:0/1464911524 learned_addr learned my addr 172.21.15.157:0/1464911524 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:56:56.353 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.350+0000 7f3235d9b700 1 --2- 172.21.15.157:0/1464911524 >> v2:172.21.15.175:3300/0 conn(0x7f323806ad80 0x7f3238068550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:56.353 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.351+0000 7f3236d9d700 1 -- 172.21.15.157:0/1464911524 >> v2:172.21.15.175:3300/0 conn(0x7f323806ad80 msgr2=0x7f3238068550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:56:56.353 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.351+0000 7f3236d9d700 1 --2- 172.21.15.157:0/1464911524 >> v2:172.21.15.175:3300/0 conn(0x7f323806ad80 0x7f3238068550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:56.353 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.351+0000 7f323559a700 1 -- 172.21.15.157:0/1464911524 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 194466512 0 0) 0x7f3238153ed0 con 0x7f323806a9b0 2023-10-06T03:56:56.353 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.351+0000 7f3236d9d700 1 -- 172.21.15.157:0/1464911524 >> v1:172.21.15.175:6789/0 conn(0x7f323806a9b0 legacy=0x7f32380c6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:56.354 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.351+0000 7f3236d9d700 1 -- 172.21.15.157:0/1464911524 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3238154490 con 0x7f323806cd40 2023-10-06T03:56:56.354 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.351+0000 7f3236d9d700 1 --2- 172.21.15.157:0/1464911524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f323806cd40 0x7f3238068a90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f323806c150 tx=0x7f3230009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5227a06408f266 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:56:56.354 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.351+0000 7f323559a700 1 -- 172.21.15.157:0/1464911524 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323000e070 con 0x7f323806cd40 2023-10-06T03:56:56.354 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.351+0000 7f323559a700 1 -- 172.21.15.157:0/1464911524 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323000ba20 con 0x7f323806cd40 2023-10-06T03:56:56.354 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.351+0000 7f323559a700 1 -- 172.21.15.157:0/1464911524 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32300092e0 con 0x7f323806cd40 2023-10-06T03:56:56.357 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.352+0000 7f323c85a700 1 -- 172.21.15.157:0/1464911524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f323806cd40 msgr2=0x7f3238068a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:56.357 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.352+0000 7f323c85a700 1 --2- 172.21.15.157:0/1464911524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f323806cd40 0x7f3238068a90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f323806c150 tx=0x7f3230009840 comp rx=0 tx=0).stop 2023-10-06T03:56:56.357 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.352+0000 7f323c85a700 1 -- 172.21.15.157:0/1464911524 shutdown_connections 2023-10-06T03:56:56.357 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.352+0000 7f323c85a700 1 --2- 172.21.15.157:0/1464911524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f323806cd40 0x7f3238068a90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:56.357 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.352+0000 7f323c85a700 1 --2- 172.21.15.157:0/1464911524 >> v2:172.21.15.175:3300/0 conn(0x7f323806ad80 0x7f3238068550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:56.357 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.352+0000 7f323c85a700 1 -- 172.21.15.157:0/1464911524 >> 172.21.15.157:0/1464911524 conn(0x7f3238063280 msgr2=0x7f323806b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:56:56.357 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.352+0000 7f323c85a700 1 -- 172.21.15.157:0/1464911524 shutdown_connections 2023-10-06T03:56:56.357 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.352+0000 7f323c85a700 1 -- 172.21.15.157:0/1464911524 wait complete. 2023-10-06T03:56:56.357 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.353+0000 7f323c85a700 1 Processor -- start 2023-10-06T03:56:56.357 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.353+0000 7f323c85a700 1 -- start start 2023-10-06T03:56:56.358 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.353+0000 7f323c85a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f323806a9b0 0x7f3238156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:56.358 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.353+0000 7f323c85a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f323806ad80 0x7f3238167450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:56.358 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.353+0000 7f323c85a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f323806cd40 0x7f3238167990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:56:56.358 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.353+0000 7f323c85a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3238155640 con 0x7f323806cd40 2023-10-06T03:56:56.358 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.353+0000 7f323c85a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32381489f0 con 0x7f323806a9b0 2023-10-06T03:56:56.358 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.353+0000 7f323c85a700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32380c3970 con 0x7f323806ad80 2023-10-06T03:56:56.358 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.354+0000 7f3236d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f323806cd40 0x7f3238167990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:56.358 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.354+0000 7f3236d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f323806cd40 0x7f3238167990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52302/0 (socket says 172.21.15.157:52302) 2023-10-06T03:56:56.358 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.354+0000 7f3236d9d700 1 -- 172.21.15.157:0/550546019 learned_addr learned my addr 172.21.15.157:0/550546019 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:56:56.358 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.354+0000 7f3235d9b700 1 --2- 172.21.15.157:0/550546019 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f323806ad80 0x7f3238167450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:56.358 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.354+0000 7f323659c700 1 --2- 172.21.15.157:0/550546019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f323806a9b0 0x7f3238156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:56:56.359 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.354+0000 7f3236d9d700 1 -- 172.21.15.157:0/550546019 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f323806ad80 msgr2=0x7f3238167450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:56.359 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.354+0000 7f3236d9d700 1 --2- 172.21.15.157:0/550546019 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f323806ad80 0x7f3238167450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:56.359 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.354+0000 7f3236d9d700 1 -- 172.21.15.157:0/550546019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f323806a9b0 msgr2=0x7f3238156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:56.359 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.354+0000 7f3236d9d700 1 --2- 172.21.15.157:0/550546019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f323806a9b0 0x7f3238156770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:56.359 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.354+0000 7f3236d9d700 1 -- 172.21.15.157:0/550546019 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3230009ae0 con 0x7f323806cd40 2023-10-06T03:56:56.359 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.354+0000 7f3236d9d700 1 --2- 172.21.15.157:0/550546019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f323806cd40 0x7f3238167990 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3230009ab0 tx=0x7f3230018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:56:56.359 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.354+0000 7f32267fc700 1 -- 172.21.15.157:0/550546019 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3230008ac0 con 0x7f323806cd40 2023-10-06T03:56:56.359 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.354+0000 7f32267fc700 1 -- 172.21.15.157:0/550546019 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3230008c40 con 0x7f323806cd40 2023-10-06T03:56:56.359 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.354+0000 7f32267fc700 1 -- 172.21.15.157:0/550546019 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3230026460 con 0x7f323806cd40 2023-10-06T03:56:56.359 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.354+0000 7f323c85a700 1 -- 172.21.15.157:0/550546019 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323814d5b0 con 0x7f323806cd40 2023-10-06T03:56:56.359 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.354+0000 7f323c85a700 1 -- 172.21.15.157:0/550546019 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3238155f70 con 0x7f323806cd40 2023-10-06T03:56:56.360 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.354+0000 7f32267fc700 1 -- 172.21.15.157:0/550546019 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f3230003040 con 0x7f323806cd40 2023-10-06T03:56:56.360 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.354+0000 7f32267fc700 1 -- 172.21.15.157:0/550546019 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f3230026cb0 con 0x7f323806cd40 2023-10-06T03:56:56.360 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.355+0000 7f323c85a700 1 -- 172.21.15.157:0/550546019 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f323805f230 con 0x7f323806cd40 2023-10-06T03:56:56.360 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.357+0000 7f32267fc700 1 -- 172.21.15.157:0/550546019 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f3230021070 con 0x7f323806cd40 2023-10-06T03:56:56.427 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.426+0000 7f323c85a700 1 -- 172.21.15.157:0/550546019 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32380c2a80 con 0x7f323806cd40 2023-10-06T03:56:56.428 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.427+0000 7f32267fc700 1 -- 172.21.15.157:0/550546019 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f3230026850 con 0x7f323806cd40 2023-10-06T03:56:56.428 INFO:teuthology.misc.health.smithi157.stdout: 2023-10-06T03:56:56.428 INFO:teuthology.misc.health.smithi157.stdout:{"epoch":10,"fsid":"28664a7a-fe30-4efe-be5a-6abf9318b700","created":"2023-10-06T03:56:51.887538+0000","modified":"2023-10-06T03:56:55.141505+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T03:56:55.141505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"3f677cac-e9f4-4608-acac-718b072ce984","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7d771892-bb91-4e9b-9610-051861c11247","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5c1f7d4e-d185-47cf-ab6c-453b4e3009c2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f6a1e825-43c7-4afd-8daf-e51d45afc75e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8ad6e3d8-9a31-4f85-9410-417b7ac5da97","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f68aca2a-f182-4c00-8e21-241d09c02ed3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d867cd89-3430-40ca-862f-d2bea3de9d85","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"930cd464-79f1-47cb-b4ff-188ccde74fff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:56:56.429 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.428+0000 7f323c85a700 1 -- 172.21.15.157:0/550546019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f323806cd40 msgr2=0x7f3238167990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:56:56.429 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.428+0000 7f323c85a700 1 --2- 172.21.15.157:0/550546019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f323806cd40 0x7f3238167990 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3230009ab0 tx=0x7f3230018000 comp rx=0 tx=0).stop 2023-10-06T03:56:56.430 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.428+0000 7f323c85a700 1 -- 172.21.15.157:0/550546019 shutdown_connections 2023-10-06T03:56:56.430 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.428+0000 7f323c85a700 1 --2- 172.21.15.157:0/550546019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f323806a9b0 0x7f3238156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:56.430 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.428+0000 7f323c85a700 1 --2- 172.21.15.157:0/550546019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f323806cd40 0x7f3238167990 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:56.430 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.428+0000 7f323c85a700 1 --2- 172.21.15.157:0/550546019 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f323806ad80 0x7f3238167450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:56:56.430 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.428+0000 7f323c85a700 1 -- 172.21.15.157:0/550546019 >> 172.21.15.157:0/550546019 conn(0x7f3238063280 msgr2=0x7f32380c6260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:56:56.430 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.428+0000 7f323c85a700 1 -- 172.21.15.157:0/550546019 shutdown_connections 2023-10-06T03:56:56.430 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:56:56.428+0000 7f323c85a700 1 -- 172.21.15.157:0/550546019 wait complete. 2023-10-06T03:56:56.439 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-06T03:56:56.669 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-10-06T03:56:56.668+0000 7fe9d654c180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:56:56.675 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-10-06T03:56:56.673+0000 7fcdc1ec5180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:56:56.743 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-10-06T03:56:56.741+0000 7fe9d654c180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:56:56.746 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-10-06T03:56:56.745+0000 7fcdc1ec5180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:56:56.818 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-10-06T03:56:56.816+0000 7fcdc1ec5180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:56:56.819 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-10-06T03:56:56.818+0000 7fe9d654c180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:56:56.952 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-10-06T03:56:56.950+0000 7fcdc1ec5180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:56:56.956 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-10-06T03:56:56.955+0000 7fe9d654c180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:56:57.034 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-10-06T03:56:57.032+0000 7fcdc1ec5180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:56:57.053 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-10-06T03:56:57.052+0000 7fe9d654c180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:56:57.184 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-10-06T03:56:57.182+0000 7fcdc1ec5180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:56:57.207 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-10-06T03:56:57.205+0000 7fe9d654c180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:56:57.259 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-10-06T03:56:57.257+0000 7fcdc1ec5180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:56:57.283 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-10-06T03:56:57.282+0000 7fe9d654c180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:56:57.332 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-10-06T03:56:57.329+0000 7fcdc1ec5180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:56:57.358 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-10-06T03:56:57.356+0000 7fe9d654c180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:56:57.405 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-10-06T03:56:57.403+0000 7fcdc1ec5180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:56:57.434 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-10-06T03:56:57.433+0000 7fe9d654c180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:56:58.431 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-10-06T03:56:58.428+0000 7fcd7b2c5700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-06T03:56:58.431 INFO:tasks.ceph.mgr.x.smithi175.stderr:2023-10-06T03:56:58.429+0000 7fcd7b2c5700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-06T03:56:59.046 INFO:tasks.ceph.osd.1.smithi157.stderr:2023-10-06T03:56:59.044+0000 7f3208a08540 -1 osd.1 0 log_to_monitors true 2023-10-06T03:56:59.259 INFO:tasks.ceph.osd.0.smithi157.stderr:2023-10-06T03:56:59.258+0000 7f5b77935540 -1 osd.0 0 log_to_monitors true 2023-10-06T03:56:59.267 INFO:tasks.ceph.osd.6.smithi175.stderr:2023-10-06T03:56:59.266+0000 7f0497c48540 -1 osd.6 0 log_to_monitors true 2023-10-06T03:56:59.301 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-10-06T03:56:59.300+0000 7fd05550d540 -1 osd.5 0 log_to_monitors true 2023-10-06T03:56:59.302 INFO:tasks.ceph.osd.3.smithi157.stderr:2023-10-06T03:56:59.300+0000 7f88ea1cd540 -1 osd.3 0 log_to_monitors true 2023-10-06T03:56:59.315 INFO:tasks.ceph.osd.2.smithi157.stderr:2023-10-06T03:56:59.314+0000 7fbf64d2d540 -1 osd.2 0 log_to_monitors true 2023-10-06T03:56:59.337 INFO:tasks.ceph.osd.4.smithi175.stderr:2023-10-06T03:56:59.335+0000 7eff49554540 -1 osd.4 0 log_to_monitors true 2023-10-06T03:56:59.356 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-10-06T03:56:59.355+0000 7f4552c3c540 -1 osd.7 0 log_to_monitors true 2023-10-06T03:57:01.292 INFO:tasks.ceph.osd.2.smithi157.stderr:2023-10-06T03:57:01.291+0000 7fbf5b94f700 -1 osd.2 0 waiting for initial osdmap 2023-10-06T03:57:01.296 INFO:tasks.ceph.osd.2.smithi157.stderr:2023-10-06T03:57:01.294+0000 7fbf54746700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T03:57:01.300 INFO:tasks.ceph.osd.1.smithi157.stderr:2023-10-06T03:57:01.297+0000 7f3200e2d700 -1 osd.1 0 waiting for initial osdmap 2023-10-06T03:57:01.302 INFO:tasks.ceph.osd.1.smithi157.stderr:2023-10-06T03:57:01.301+0000 7f31f8421700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T03:57:01.336 INFO:tasks.ceph.osd.3.smithi157.stderr:2023-10-06T03:57:01.333+0000 7f88e0def700 -1 osd.3 0 waiting for initial osdmap 2023-10-06T03:57:01.338 INFO:tasks.ceph.osd.3.smithi157.stderr:2023-10-06T03:57:01.337+0000 7f88d9be6700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T03:57:01.347 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-10-06T03:57:01.345+0000 7f454b061700 -1 osd.7 0 waiting for initial osdmap 2023-10-06T03:57:01.351 INFO:tasks.ceph.osd.7.smithi175.stderr:2023-10-06T03:57:01.349+0000 7f4542655700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T03:57:01.354 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-10-06T03:57:01.353+0000 7fd04c12f700 -1 osd.5 0 waiting for initial osdmap 2023-10-06T03:57:01.355 INFO:tasks.ceph.osd.4.smithi175.stderr:2023-10-06T03:57:01.352+0000 7eff40176700 -1 osd.4 0 waiting for initial osdmap 2023-10-06T03:57:01.359 INFO:tasks.ceph.osd.4.smithi175.stderr:2023-10-06T03:57:01.357+0000 7eff38f6d700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T03:57:01.361 INFO:tasks.ceph.osd.5.smithi175.stderr:2023-10-06T03:57:01.359+0000 7fd044f26700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T03:57:01.368 INFO:tasks.ceph.osd.6.smithi175.stderr:2023-10-06T03:57:01.366+0000 7f048e86a700 -1 osd.6 0 waiting for initial osdmap 2023-10-06T03:57:01.371 INFO:tasks.ceph.osd.6.smithi175.stderr:2023-10-06T03:57:01.369+0000 7f0487661700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T03:57:01.957 INFO:tasks.ceph.osd.0.smithi157.stderr:2023-10-06T03:57:01.955+0000 7f5b6e557700 -1 osd.0 0 waiting for initial osdmap 2023-10-06T03:57:01.961 INFO:tasks.ceph.osd.0.smithi157.stderr:2023-10-06T03:57:01.959+0000 7f5b6734e700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T03:57:03.246 DEBUG:teuthology.orchestra.run.smithi157:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T03:57:03.359 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.357+0000 7f0157c4a700 1 Processor -- start 2023-10-06T03:57:03.359 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.358+0000 7f0157c4a700 1 -- start start 2023-10-06T03:57:03.359 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.358+0000 7f0157c4a700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0150153250 0x7f015006d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:03.360 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.358+0000 7f0157c4a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0150153620 0x7f0150156460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:03.361 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.358+0000 7f0157c4a700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0150155fd0 con 0x7f015006f120 2023-10-06T03:57:03.361 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.358+0000 7f0157c4a700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f015006d980 con 0x7f0150153250 2023-10-06T03:57:03.361 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.358+0000 7f0157c4a700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01500ca650 con 0x7f0150153620 2023-10-06T03:57:03.362 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.358+0000 7f01559e6700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0150153250 0x7f015006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:03.362 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.359+0000 7f01551e5700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f015006f120 0x7f0150068030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44034/0 (socket says 172.21.15.157:44034) 2023-10-06T03:57:03.362 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.359+0000 7f01551e5700 1 -- 172.21.15.157:0/801214910 learned_addr learned my addr 172.21.15.157:0/801214910 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:03.362 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.359+0000 7f01561e7700 1 --2- 172.21.15.157:0/801214910 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0150153620 0x7f0150156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:03.362 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.359+0000 7f01559e6700 1 -- 172.21.15.157:0/801214910 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0150153620 msgr2=0x7f0150156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:03.362 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.359+0000 7f01559e6700 1 --2- 172.21.15.157:0/801214910 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0150153620 0x7f0150156460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:03.362 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.359+0000 7f01559e6700 1 -- 172.21.15.157:0/801214910 >> v1:172.21.15.157:6789/0 conn(0x7f015006f120 legacy=0x7f0150068030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:03.362 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.359+0000 7f01559e6700 1 -- 172.21.15.157:0/801214910 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015006dba0 con 0x7f0150153250 2023-10-06T03:57:03.362 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.359+0000 7f01561e7700 1 --2- 172.21.15.157:0/801214910 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0150153620 0x7f0150156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:57:03.362 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.359+0000 7f01549e4700 1 -- 172.21.15.157:0/801214910 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3417027015 0 0) 0x7f0150155fd0 con 0x7f015006f120 2023-10-06T03:57:03.362 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.359+0000 7f01559e6700 1 --2- 172.21.15.157:0/801214910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0150153250 0x7f015006d440 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f014c0098e0 tx=0x7f014c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b7c8709cb1daf9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:03.363 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.359+0000 7f01549e4700 1 -- 172.21.15.157:0/801214910 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014c009e80 con 0x7f0150153250 2023-10-06T03:57:03.363 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.359+0000 7f01549e4700 1 -- 172.21.15.157:0/801214910 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014c00b860 con 0x7f0150153250 2023-10-06T03:57:03.363 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.359+0000 7f01549e4700 1 -- 172.21.15.157:0/801214910 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014c00bbe0 con 0x7f0150153250 2023-10-06T03:57:03.363 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.360+0000 7f0157c4a700 1 -- 172.21.15.157:0/801214910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0150153250 msgr2=0x7f015006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:03.363 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.360+0000 7f0157c4a700 1 --2- 172.21.15.157:0/801214910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0150153250 0x7f015006d440 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f014c0098e0 tx=0x7f014c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:03.363 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.360+0000 7f0157c4a700 1 -- 172.21.15.157:0/801214910 shutdown_connections 2023-10-06T03:57:03.363 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.360+0000 7f0157c4a700 1 --2- 172.21.15.157:0/801214910 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0150153620 0x7f0150156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:03.363 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.360+0000 7f0157c4a700 1 --2- 172.21.15.157:0/801214910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0150153250 0x7f015006d440 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:03.363 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.360+0000 7f0157c4a700 1 -- 172.21.15.157:0/801214910 >> 172.21.15.157:0/801214910 conn(0x7f0150069490 msgr2=0x7f0150069890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:03.363 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.360+0000 7f0157c4a700 1 -- 172.21.15.157:0/801214910 shutdown_connections 2023-10-06T03:57:03.363 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.360+0000 7f0157c4a700 1 -- 172.21.15.157:0/801214910 wait complete. 2023-10-06T03:57:03.366 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.361+0000 7f0157c4a700 1 Processor -- start 2023-10-06T03:57:03.366 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.361+0000 7f0157c4a700 1 -- start start 2023-10-06T03:57:03.366 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.362+0000 7f0157c4a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f015006f120 0x7f01500ad570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:03.366 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.362+0000 7f0157c4a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0150153250 0x7f0150158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:03.366 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.362+0000 7f0157c4a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0150153620 0x7f01500c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:03.366 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.362+0000 7f0157c4a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0150150ef0 con 0x7f0150153620 2023-10-06T03:57:03.367 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.362+0000 7f0157c4a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01500c9ba0 con 0x7f015006f120 2023-10-06T03:57:03.367 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.362+0000 7f0157c4a700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0150166ee0 con 0x7f0150153250 2023-10-06T03:57:03.367 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.362+0000 7f01561e7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0150153620 0x7f01500c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:03.367 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.362+0000 7f01561e7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0150153620 0x7f01500c85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58366/0 (socket says 172.21.15.157:58366) 2023-10-06T03:57:03.367 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.362+0000 7f01561e7700 1 -- 172.21.15.157:0/1469700479 learned_addr learned my addr 172.21.15.157:0/1469700479 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:03.367 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.362+0000 7f01551e5700 1 --2- 172.21.15.157:0/1469700479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0150153250 0x7f0150158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:03.367 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.362+0000 7f01559e6700 1 --2- 172.21.15.157:0/1469700479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f015006f120 0x7f01500ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:03.367 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.362+0000 7f01561e7700 1 -- 172.21.15.157:0/1469700479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0150153250 msgr2=0x7f0150158910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:03.367 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.362+0000 7f01561e7700 1 --2- 172.21.15.157:0/1469700479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0150153250 0x7f0150158910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:03.367 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.362+0000 7f01561e7700 1 -- 172.21.15.157:0/1469700479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f015006f120 msgr2=0x7f01500ad570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:03.367 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.362+0000 7f01561e7700 1 --2- 172.21.15.157:0/1469700479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f015006f120 0x7f01500ad570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:03.367 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.362+0000 7f01561e7700 1 -- 172.21.15.157:0/1469700479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014c009420 con 0x7f0150153620 2023-10-06T03:57:03.368 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.362+0000 7f01551e5700 1 --2- 172.21.15.157:0/1469700479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0150153250 0x7f0150158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:03.368 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.362+0000 7f01559e6700 1 --2- 172.21.15.157:0/1469700479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f015006f120 0x7f01500ad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:03.368 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.362+0000 7f01561e7700 1 --2- 172.21.15.157:0/1469700479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0150153620 0x7f01500c85a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f014800cef0 tx=0x7f0148007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:03.368 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.362+0000 7f0145ffb700 1 -- 172.21.15.157:0/1469700479 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014800ae00 con 0x7f0150153620 2023-10-06T03:57:03.368 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.362+0000 7f0145ffb700 1 -- 172.21.15.157:0/1469700479 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0148016420 con 0x7f0150153620 2023-10-06T03:57:03.368 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.362+0000 7f0157c4a700 1 -- 172.21.15.157:0/1469700479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0150150c10 con 0x7f0150153620 2023-10-06T03:57:03.368 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.362+0000 7f0145ffb700 1 -- 172.21.15.157:0/1469700479 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01480167e0 con 0x7f0150153620 2023-10-06T03:57:03.368 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.362+0000 7f0157c4a700 1 -- 172.21.15.157:0/1469700479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0150166b70 con 0x7f0150153620 2023-10-06T03:57:03.368 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.363+0000 7f0157c4a700 1 -- 172.21.15.157:0/1469700479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f015005f230 con 0x7f0150153620 2023-10-06T03:57:03.368 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.364+0000 7f0145ffb700 1 -- 172.21.15.157:0/1469700479 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f0148016bd0 con 0x7f0150153620 2023-10-06T03:57:03.368 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.364+0000 7f0145ffb700 1 --2- 172.21.15.157:0/1469700479 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0134066ea0 0x7f0134069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:03.368 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.364+0000 7f0145ffb700 1 -- 172.21.15.157:0/1469700479 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4467+0+0 (secure 0 0 0) 0x7f0148012070 con 0x7f0150153620 2023-10-06T03:57:03.369 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.364+0000 7f01559e6700 1 --2- 172.21.15.157:0/1469700479 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0134066ea0 0x7f0134069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:03.369 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.365+0000 7f01559e6700 1 --2- 172.21.15.157:0/1469700479 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0134066ea0 0x7f0134069350 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f014c0091e0 tx=0x7f014c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:03.370 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.369+0000 7f0145ffb700 1 -- 172.21.15.157:0/1469700479 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f014804e5a0 con 0x7f0150153620 2023-10-06T03:57:03.566 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.565+0000 7f0157c4a700 1 -- 172.21.15.157:0/1469700479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01500ca8f0 con 0x7f0150153620 2023-10-06T03:57:03.567 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.566+0000 7f0145ffb700 1 -- 172.21.15.157:0/1469700479 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10628 (secure 0 0 0) 0x7f0148052100 con 0x7f0150153620 2023-10-06T03:57:03.567 INFO:teuthology.misc.health.smithi157.stdout: 2023-10-06T03:57:03.568 INFO:teuthology.misc.health.smithi157.stdout:{"epoch":14,"fsid":"28664a7a-fe30-4efe-be5a-6abf9318b700","created":"2023-10-06T03:56:51.887538+0000","modified":"2023-10-06T03:57:03.300884+0000","last_up_change":"2023-10-06T03:57:02.299440+0000","last_in_change":"2023-10-06T03:56:55.141505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"3f677cac-e9f4-4608-acac-718b072ce984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":90640},{"type":"v1","addr":"172.21.15.157:6809","nonce":90640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":90640},{"type":"v1","addr":"172.21.15.157:6811","nonce":90640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":90640},{"type":"v1","addr":"172.21.15.157:6815","nonce":90640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":90640},{"type":"v1","addr":"172.21.15.157:6813","nonce":90640}]},"public_addr":"172.21.15.157:6809/90640","cluster_addr":"172.21.15.157:6811/90640","heartbeat_back_addr":"172.21.15.157:6815/90640","heartbeat_front_addr":"172.21.15.157:6813/90640","state":["exists","up"]},{"osd":1,"uuid":"7d771892-bb91-4e9b-9610-051861c11247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":90654},{"type":"v1","addr":"172.21.15.157:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":90654},{"type":"v1","addr":"172.21.15.157:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":90654},{"type":"v1","addr":"172.21.15.157:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":90654},{"type":"v1","addr":"172.21.15.157:6805","nonce":90654}]},"public_addr":"172.21.15.157:6801/90654","cluster_addr":"172.21.15.157:6803/90654","heartbeat_back_addr":"172.21.15.157:6807/90654","heartbeat_front_addr":"172.21.15.157:6805/90654","state":["exists","up"]},{"osd":2,"uuid":"5c1f7d4e-d185-47cf-ab6c-453b4e3009c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":90657},{"type":"v1","addr":"172.21.15.157:6825","nonce":90657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":90657},{"type":"v1","addr":"172.21.15.157:6827","nonce":90657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":90657},{"type":"v1","addr":"172.21.15.157:6831","nonce":90657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":90657},{"type":"v1","addr":"172.21.15.157:6829","nonce":90657}]},"public_addr":"172.21.15.157:6825/90657","cluster_addr":"172.21.15.157:6827/90657","heartbeat_back_addr":"172.21.15.157:6831/90657","heartbeat_front_addr":"172.21.15.157:6829/90657","state":["exists","up"]},{"osd":3,"uuid":"f6a1e825-43c7-4afd-8daf-e51d45afc75e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":90641},{"type":"v1","addr":"172.21.15.157:6817","nonce":90641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":90641},{"type":"v1","addr":"172.21.15.157:6819","nonce":90641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":90641},{"type":"v1","addr":"172.21.15.157:6823","nonce":90641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":90641},{"type":"v1","addr":"172.21.15.157:6821","nonce":90641}]},"public_addr":"172.21.15.157:6817/90641","cluster_addr":"172.21.15.157:6819/90641","heartbeat_back_addr":"172.21.15.157:6823/90641","heartbeat_front_addr":"172.21.15.157:6821/90641","state":["exists","up"]},{"osd":4,"uuid":"8ad6e3d8-9a31-4f85-9410-417b7ac5da97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":91261},{"type":"v1","addr":"172.21.15.175:6803","nonce":91261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":91261},{"type":"v1","addr":"172.21.15.175:6806","nonce":91261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":91261},{"type":"v1","addr":"172.21.15.175:6814","nonce":91261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":91261},{"type":"v1","addr":"172.21.15.175:6810","nonce":91261}]},"public_addr":"172.21.15.175:6803/91261","cluster_addr":"172.21.15.175:6806/91261","heartbeat_back_addr":"172.21.15.175:6814/91261","heartbeat_front_addr":"172.21.15.175:6810/91261","state":["exists","up"]},{"osd":5,"uuid":"f68aca2a-f182-4c00-8e21-241d09c02ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":91262},{"type":"v1","addr":"172.21.15.175:6817","nonce":91262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":91262},{"type":"v1","addr":"172.21.15.175:6819","nonce":91262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":91262},{"type":"v1","addr":"172.21.15.175:6823","nonce":91262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":91262},{"type":"v1","addr":"172.21.15.175:6821","nonce":91262}]},"public_addr":"172.21.15.175:6817/91262","cluster_addr":"172.21.15.175:6819/91262","heartbeat_back_addr":"172.21.15.175:6823/91262","heartbeat_front_addr":"172.21.15.175:6821/91262","state":["exists","up"]},{"osd":6,"uuid":"d867cd89-3430-40ca-862f-d2bea3de9d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":91263},{"type":"v1","addr":"172.21.15.175:6801","nonce":91263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6805","nonce":91263},{"type":"v1","addr":"172.21.15.175:6807","nonce":91263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6813","nonce":91263},{"type":"v1","addr":"172.21.15.175:6815","nonce":91263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":91263},{"type":"v1","addr":"172.21.15.175:6811","nonce":91263}]},"public_addr":"172.21.15.175:6801/91263","cluster_addr":"172.21.15.175:6807/91263","heartbeat_back_addr":"172.21.15.175:6815/91263","heartbeat_front_addr":"172.21.15.175:6811/91263","state":["exists","up"]},{"osd":7,"uuid":"930cd464-79f1-47cb-b4ff-188ccde74fff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":91306},{"type":"v1","addr":"172.21.15.175:6825","nonce":91306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":91306},{"type":"v1","addr":"172.21.15.175:6827","nonce":91306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":91306},{"type":"v1","addr":"172.21.15.175:6831","nonce":91306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":91306},{"type":"v1","addr":"172.21.15.175:6829","nonce":91306}]},"public_addr":"172.21.15.175:6825/91306","cluster_addr":"172.21.15.175:6827/91306","heartbeat_back_addr":"172.21.15.175:6831/91306","heartbeat_front_addr":"172.21.15.175:6829/91306","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.290673+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.056390+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.292903+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.275861+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.332095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.327453+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.317585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.309330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:57:03.570 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.568+0000 7f0157c4a700 1 -- 172.21.15.157:0/1469700479 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0134066ea0 msgr2=0x7f0134069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:03.570 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.568+0000 7f0157c4a700 1 --2- 172.21.15.157:0/1469700479 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0134066ea0 0x7f0134069350 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f014c0091e0 tx=0x7f014c017000 comp rx=0 tx=0).stop 2023-10-06T03:57:03.570 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.568+0000 7f0157c4a700 1 -- 172.21.15.157:0/1469700479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0150153620 msgr2=0x7f01500c85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:03.570 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.568+0000 7f0157c4a700 1 --2- 172.21.15.157:0/1469700479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0150153620 0x7f01500c85a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f014800cef0 tx=0x7f0148007860 comp rx=0 tx=0).stop 2023-10-06T03:57:03.570 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.568+0000 7f0157c4a700 1 -- 172.21.15.157:0/1469700479 shutdown_connections 2023-10-06T03:57:03.570 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.568+0000 7f0157c4a700 1 --2- 172.21.15.157:0/1469700479 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0134066ea0 0x7f0134069350 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:03.570 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.568+0000 7f0157c4a700 1 --2- 172.21.15.157:0/1469700479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f015006f120 0x7f01500ad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:03.571 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.568+0000 7f0157c4a700 1 --2- 172.21.15.157:0/1469700479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0150153620 0x7f01500c85a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:03.571 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.568+0000 7f0157c4a700 1 --2- 172.21.15.157:0/1469700479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0150153250 0x7f0150158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:03.571 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.568+0000 7f0157c4a700 1 -- 172.21.15.157:0/1469700479 >> 172.21.15.157:0/1469700479 conn(0x7f0150069490 msgr2=0x7f01500c5070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:03.571 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.568+0000 7f0157c4a700 1 -- 172.21.15.157:0/1469700479 shutdown_connections 2023-10-06T03:57:03.571 INFO:teuthology.misc.health.smithi157.stderr:2023-10-06T03:57:03.568+0000 7f0157c4a700 1 -- 172.21.15.157:0/1469700479 wait complete. 2023-10-06T03:57:03.580 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-06T03:57:03.581 INFO:tasks.ceph:Creating RBD pool 2023-10-06T03:57:03.581 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd pool create rbd 8 2023-10-06T03:57:03.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.693+0000 7fb513963700 1 Processor -- start 2023-10-06T03:57:03.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.693+0000 7fb513963700 1 -- start start 2023-10-06T03:57:03.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.694+0000 7fb513963700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb50c14dcd0 0x7fb50c153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:03.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.694+0000 7fb513963700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb50c0c5c00 0x7fb50c0ca440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:03.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.694+0000 7fb513963700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb50c0c5770 con 0x7fb50c14e0a0 2023-10-06T03:57:03.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.694+0000 7fb513963700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb50c154c80 con 0x7fb50c14dcd0 2023-10-06T03:57:03.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.694+0000 7fb513963700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb50c154200 con 0x7fb50c0c5c00 2023-10-06T03:57:03.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.694+0000 7fb5116ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb50c14dcd0 0x7fb50c153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:03.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.694+0000 7fb510efe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb50c14e0a0 0x7fb50c0c18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44044/0 (socket says 172.21.15.157:44044) 2023-10-06T03:57:03.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.694+0000 7fb5116ff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb50c14dcd0 0x7fb50c153cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58372/0 (socket says 172.21.15.157:58372) 2023-10-06T03:57:03.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.694+0000 7fb510efe700 1 -- 172.21.15.157:0/3378353904 learned_addr learned my addr 172.21.15.157:0/3378353904 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:03.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.694+0000 7fb511f00700 1 --2- 172.21.15.157:0/3378353904 >> v2:172.21.15.175:3300/0 conn(0x7fb50c0c5c00 0x7fb50c0ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:03.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.695+0000 7fb5116ff700 1 -- 172.21.15.157:0/3378353904 >> v2:172.21.15.175:3300/0 conn(0x7fb50c0c5c00 msgr2=0x7fb50c0ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:03.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.695+0000 7fb5116ff700 1 --2- 172.21.15.157:0/3378353904 >> v2:172.21.15.175:3300/0 conn(0x7fb50c0c5c00 0x7fb50c0ca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:03.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.695+0000 7fb5116ff700 1 -- 172.21.15.157:0/3378353904 >> v1:172.21.15.157:6789/0 conn(0x7fb50c14e0a0 legacy=0x7fb50c0c18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:03.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.695+0000 7fb5116ff700 1 -- 172.21.15.157:0/3378353904 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50c1543e0 con 0x7fb50c14dcd0 2023-10-06T03:57:03.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.695+0000 7fb503fff700 1 -- 172.21.15.157:0/3378353904 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 814669835 0 0) 0x7fb50c0c5770 con 0x7fb50c14e0a0 2023-10-06T03:57:03.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.695+0000 7fb5116ff700 1 --2- 172.21.15.157:0/3378353904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb50c14dcd0 0x7fb50c153cc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb50c14e570 tx=0x7fb508009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=153cdbe8c82023ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:03.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.695+0000 7fb503fff700 1 -- 172.21.15.157:0/3378353904 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50800e070 con 0x7fb50c14dcd0 2023-10-06T03:57:03.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.695+0000 7fb503fff700 1 -- 172.21.15.157:0/3378353904 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50800bbf0 con 0x7fb50c14dcd0 2023-10-06T03:57:03.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.695+0000 7fb503fff700 1 -- 172.21.15.157:0/3378353904 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb508012430 con 0x7fb50c14dcd0 2023-10-06T03:57:03.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.695+0000 7fb513963700 1 -- 172.21.15.157:0/3378353904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb50c14dcd0 msgr2=0x7fb50c153cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:03.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.695+0000 7fb513963700 1 --2- 172.21.15.157:0/3378353904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb50c14dcd0 0x7fb50c153cc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb50c14e570 tx=0x7fb508009780 comp rx=0 tx=0).stop 2023-10-06T03:57:03.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.695+0000 7fb513963700 1 -- 172.21.15.157:0/3378353904 shutdown_connections 2023-10-06T03:57:03.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.695+0000 7fb513963700 1 --2- 172.21.15.157:0/3378353904 >> v2:172.21.15.175:3300/0 conn(0x7fb50c0c5c00 0x7fb50c0ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:03.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.695+0000 7fb513963700 1 --2- 172.21.15.157:0/3378353904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb50c14dcd0 0x7fb50c153cc0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:03.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.695+0000 7fb513963700 1 -- 172.21.15.157:0/3378353904 >> 172.21.15.157:0/3378353904 conn(0x7fb50c0692b0 msgr2=0x7fb50c0696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:03.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.695+0000 7fb513963700 1 -- 172.21.15.157:0/3378353904 shutdown_connections 2023-10-06T03:57:03.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.695+0000 7fb513963700 1 -- 172.21.15.157:0/3378353904 wait complete. 2023-10-06T03:57:03.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.697+0000 7fb513963700 1 Processor -- start 2023-10-06T03:57:03.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.697+0000 7fb513963700 1 -- start start 2023-10-06T03:57:03.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.697+0000 7fb513963700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb50c0c5c00 0x7fb50c148290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:03.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.697+0000 7fb513963700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb50c14dcd0 0x7fb50c0c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:03.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.697+0000 7fb513963700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb50c14e0a0 0x7fb50c0c9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:03.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.697+0000 7fb513963700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c0ca6a0 con 0x7fb50c14dcd0 2023-10-06T03:57:03.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.697+0000 7fb513963700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c166fb0 con 0x7fb50c0c5c00 2023-10-06T03:57:03.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.697+0000 7fb513963700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c167b30 con 0x7fb50c14e0a0 2023-10-06T03:57:03.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.697+0000 7fb510efe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb50c14dcd0 0x7fb50c0c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:03.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.698+0000 7fb510efe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb50c14dcd0 0x7fb50c0c91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58382/0 (socket says 172.21.15.157:58382) 2023-10-06T03:57:03.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.698+0000 7fb510efe700 1 -- 172.21.15.157:0/2518050945 learned_addr learned my addr 172.21.15.157:0/2518050945 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:03.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.698+0000 7fb511f00700 1 --2- 172.21.15.157:0/2518050945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb50c14e0a0 0x7fb50c0c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:03.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.698+0000 7fb5116ff700 1 --2- 172.21.15.157:0/2518050945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb50c0c5c00 0x7fb50c148290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:03.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.698+0000 7fb510efe700 1 -- 172.21.15.157:0/2518050945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb50c14e0a0 msgr2=0x7fb50c0c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:03.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.698+0000 7fb510efe700 1 --2- 172.21.15.157:0/2518050945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb50c14e0a0 0x7fb50c0c9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:03.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.698+0000 7fb510efe700 1 -- 172.21.15.157:0/2518050945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb50c0c5c00 msgr2=0x7fb50c148290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:03.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.698+0000 7fb510efe700 1 --2- 172.21.15.157:0/2518050945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb50c0c5c00 0x7fb50c148290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:03.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.698+0000 7fb510efe700 1 -- 172.21.15.157:0/2518050945 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb508009a20 con 0x7fb50c14dcd0 2023-10-06T03:57:03.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.698+0000 7fb510efe700 1 --2- 172.21.15.157:0/2518050945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb50c14dcd0 0x7fb50c0c91e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00c5e0 tx=0x7fb4fc001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:03.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.698+0000 7fb5017fa700 1 -- 172.21.15.157:0/2518050945 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc005b30 con 0x7fb50c14dcd0 2023-10-06T03:57:03.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.698+0000 7fb5017fa700 1 -- 172.21.15.157:0/2518050945 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4fc005c90 con 0x7fb50c14dcd0 2023-10-06T03:57:03.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.698+0000 7fb513963700 1 -- 172.21.15.157:0/2518050945 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb50c1516e0 con 0x7fb50c14dcd0 2023-10-06T03:57:03.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.698+0000 7fb5017fa700 1 -- 172.21.15.157:0/2518050945 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc007300 con 0x7fb50c14dcd0 2023-10-06T03:57:03.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.698+0000 7fb513963700 1 -- 172.21.15.157:0/2518050945 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb50c153c50 con 0x7fb50c14dcd0 2023-10-06T03:57:03.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.699+0000 7fb513963700 1 -- 172.21.15.157:0/2518050945 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb50c167ea0 con 0x7fb50c14dcd0 2023-10-06T03:57:03.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.699+0000 7fb5017fa700 1 -- 172.21.15.157:0/2518050945 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fb4fc01b3b0 con 0x7fb50c14dcd0 2023-10-06T03:57:03.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.700+0000 7fb5017fa700 1 --2- 172.21.15.157:0/2518050945 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb4f0066ea0 0x7fb4f0069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:03.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.700+0000 7fb5017fa700 1 -- 172.21.15.157:0/2518050945 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4467+0+0 (secure 0 0 0) 0x7fb4fc083200 con 0x7fb50c14dcd0 2023-10-06T03:57:03.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.700+0000 7fb5116ff700 1 --2- 172.21.15.157:0/2518050945 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb4f0066ea0 0x7fb4f0069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:03.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.700+0000 7fb5116ff700 1 --2- 172.21.15.157:0/2518050945 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb4f0066ea0 0x7fb4f0069350 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb5080099f0 tx=0x7fb508019000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:03.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.704+0000 7fb5017fa700 1 -- 172.21.15.157:0/2518050945 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4fc050a20 con 0x7fb50c14dcd0 2023-10-06T03:57:03.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:03.888+0000 7fb513963700 1 -- 172.21.15.157:0/2518050945 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fb50c153bf0 con 0x7fb50c14dcd0 2023-10-06T03:57:04.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.308+0000 7fb5017fa700 1 -- 172.21.15.157:0/2518050945 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v15) v1 ==== 109+0+0 (secure 0 0 0) 0x7fb4fc054040 con 0x7fb50c14dcd0 2023-10-06T03:57:04.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.311+0000 7fb513963700 1 -- 172.21.15.157:0/2518050945 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb4f0066ea0 msgr2=0x7fb4f0069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:04.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.311+0000 7fb513963700 1 --2- 172.21.15.157:0/2518050945 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb4f0066ea0 0x7fb4f0069350 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb5080099f0 tx=0x7fb508019000 comp rx=0 tx=0).stop 2023-10-06T03:57:04.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.311+0000 7fb513963700 1 -- 172.21.15.157:0/2518050945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb50c14dcd0 msgr2=0x7fb50c0c91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:04.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.311+0000 7fb513963700 1 --2- 172.21.15.157:0/2518050945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb50c14dcd0 0x7fb50c0c91e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00c5e0 tx=0x7fb4fc001070 comp rx=0 tx=0).stop 2023-10-06T03:57:04.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.311+0000 7fb513963700 1 -- 172.21.15.157:0/2518050945 shutdown_connections 2023-10-06T03:57:04.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.311+0000 7fb513963700 1 --2- 172.21.15.157:0/2518050945 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb4f0066ea0 0x7fb4f0069350 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:04.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.311+0000 7fb513963700 1 --2- 172.21.15.157:0/2518050945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb50c0c5c00 0x7fb50c148290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:04.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.311+0000 7fb513963700 1 --2- 172.21.15.157:0/2518050945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb50c14e0a0 0x7fb50c0c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:04.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.311+0000 7fb513963700 1 --2- 172.21.15.157:0/2518050945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb50c14dcd0 0x7fb50c0c91e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:04.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.311+0000 7fb513963700 1 -- 172.21.15.157:0/2518050945 >> 172.21.15.157:0/2518050945 conn(0x7fb50c0692b0 msgr2=0x7fb50c0c6060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:04.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.311+0000 7fb513963700 1 -- 172.21.15.157:0/2518050945 shutdown_connections 2023-10-06T03:57:04.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.311+0000 7fb513963700 1 -- 172.21.15.157:0/2518050945 wait complete. 2023-10-06T03:57:04.314 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'rbd' created 2023-10-06T03:57:04.325 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-10-06T03:57:04.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.439+0000 7f0413365700 1 Processor -- start 2023-10-06T03:57:04.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.439+0000 7f0413365700 1 -- start start 2023-10-06T03:57:04.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.440+0000 7f0413365700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f040c06b070 0x7f040c069c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:04.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.440+0000 7f0413365700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f040c0ad950 0x7f040c154780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:04.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.440+0000 7f0413365700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f040c155730 con 0x7f040c0ad580 2023-10-06T03:57:04.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.440+0000 7f0413365700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f040c0ae9a0 con 0x7f040c06b070 2023-10-06T03:57:04.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.440+0000 7f0413365700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f040c1579d0 con 0x7f040c0ad950 2023-10-06T03:57:04.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.440+0000 7f0410900700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f040c06b070 0x7f040c069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:04.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.440+0000 7f0410900700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f040c06b070 0x7f040c069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58386/0 (socket says 172.21.15.157:58386) 2023-10-06T03:57:04.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.440+0000 7f0410900700 1 -- 172.21.15.157:0/3139905261 learned_addr learned my addr 172.21.15.157:0/3139905261 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:04.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.440+0000 7f0411902700 1 --2- 172.21.15.157:0/3139905261 >> v2:172.21.15.175:3300/0 conn(0x7f040c0ad950 0x7f040c154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:04.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.440+0000 7f0410900700 1 -- 172.21.15.157:0/3139905261 >> v2:172.21.15.175:3300/0 conn(0x7f040c0ad950 msgr2=0x7f040c154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:04.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.440+0000 7f0410900700 1 --2- 172.21.15.157:0/3139905261 >> v2:172.21.15.175:3300/0 conn(0x7f040c0ad950 0x7f040c154780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:04.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.440+0000 7f0410900700 1 -- 172.21.15.157:0/3139905261 >> v1:172.21.15.175:6789/0 conn(0x7f040c0ad580 legacy=0x7f040c155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:04.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.440+0000 7f0410900700 1 -- 172.21.15.157:0/3139905261 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040c157cb0 con 0x7f040c06b070 2023-10-06T03:57:04.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.440+0000 7f0411902700 1 --2- 172.21.15.157:0/3139905261 >> v2:172.21.15.175:3300/0 conn(0x7f040c0ad950 0x7f040c154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:57:04.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.440+0000 7f0410900700 1 --2- 172.21.15.157:0/3139905261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f040c06b070 0x7f040c069c50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f040c0ad2c0 tx=0x7f03fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4558296028ddb3d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:04.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.441+0000 7f0403fff700 1 -- 172.21.15.157:0/3139905261 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc00b820 con 0x7f040c06b070 2023-10-06T03:57:04.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.441+0000 7f0403fff700 1 -- 172.21.15.157:0/3139905261 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03fc00b980 con 0x7f040c06b070 2023-10-06T03:57:04.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.441+0000 7f0403fff700 1 -- 172.21.15.157:0/3139905261 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc00bd20 con 0x7f040c06b070 2023-10-06T03:57:04.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.441+0000 7f0413365700 1 -- 172.21.15.157:0/3139905261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f040c06b070 msgr2=0x7f040c069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:04.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.441+0000 7f0413365700 1 --2- 172.21.15.157:0/3139905261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f040c06b070 0x7f040c069c50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f040c0ad2c0 tx=0x7f03fc009250 comp rx=0 tx=0).stop 2023-10-06T03:57:04.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.441+0000 7f0413365700 1 -- 172.21.15.157:0/3139905261 shutdown_connections 2023-10-06T03:57:04.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.441+0000 7f0413365700 1 --2- 172.21.15.157:0/3139905261 >> v2:172.21.15.175:3300/0 conn(0x7f040c0ad950 0x7f040c154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:04.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.441+0000 7f0413365700 1 --2- 172.21.15.157:0/3139905261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f040c06b070 0x7f040c069c50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:04.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.441+0000 7f0413365700 1 -- 172.21.15.157:0/3139905261 >> 172.21.15.157:0/3139905261 conn(0x7f040c063cc0 msgr2=0x7f040c069190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:04.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.441+0000 7f0413365700 1 -- 172.21.15.157:0/3139905261 shutdown_connections 2023-10-06T03:57:04.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.441+0000 7f0413365700 1 -- 172.21.15.157:0/3139905261 wait complete. 2023-10-06T03:57:04.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.442+0000 7f0413365700 1 Processor -- start 2023-10-06T03:57:04.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.442+0000 7f0413365700 1 -- start start 2023-10-06T03:57:04.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.443+0000 7f0413365700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f040c06b070 0x7f040c0c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:04.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.443+0000 7f0413365700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f040c0ad580 0x7f040c0c4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:04.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.443+0000 7f0413365700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f040c0ad950 0x7f040c167a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:04.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.443+0000 7f0413365700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040c168710 con 0x7f040c06b070 2023-10-06T03:57:04.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.443+0000 7f0413365700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040c0c1cf0 con 0x7f040c0ad580 2023-10-06T03:57:04.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.443+0000 7f0413365700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f040c0c3970 con 0x7f040c0ad950 2023-10-06T03:57:04.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.443+0000 7f0411101700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f040c06b070 0x7f040c0c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:04.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.443+0000 7f0411101700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f040c06b070 0x7f040c0c3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58398/0 (socket says 172.21.15.157:58398) 2023-10-06T03:57:04.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.443+0000 7f0411101700 1 -- 172.21.15.157:0/2433032268 learned_addr learned my addr 172.21.15.157:0/2433032268 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:04.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.443+0000 7f0411902700 1 --2- 172.21.15.157:0/2433032268 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f040c0ad950 0x7f040c167a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:04.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.443+0000 7f0410900700 1 --2- 172.21.15.157:0/2433032268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f040c0ad580 0x7f040c0c4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:04.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.443+0000 7f0411101700 1 -- 172.21.15.157:0/2433032268 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f040c0ad950 msgr2=0x7f040c167a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:04.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.443+0000 7f0411101700 1 --2- 172.21.15.157:0/2433032268 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f040c0ad950 0x7f040c167a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:04.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.443+0000 7f0411101700 1 -- 172.21.15.157:0/2433032268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f040c0ad580 msgr2=0x7f040c0c4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:04.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.443+0000 7f0411101700 1 --2- 172.21.15.157:0/2433032268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f040c0ad580 0x7f040c0c4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:04.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.443+0000 7f0411101700 1 -- 172.21.15.157:0/2433032268 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03fc0094f0 con 0x7f040c06b070 2023-10-06T03:57:04.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.444+0000 7f0411101700 1 --2- 172.21.15.157:0/2433032268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f040c06b070 0x7f040c0c3e00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f040800c600 tx=0x7f0408007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:04.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.444+0000 7f04017fa700 1 -- 172.21.15.157:0/2433032268 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0408011590 con 0x7f040c06b070 2023-10-06T03:57:04.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.444+0000 7f04017fa700 1 -- 172.21.15.157:0/2433032268 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0408011710 con 0x7f040c06b070 2023-10-06T03:57:04.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.444+0000 7f0413365700 1 -- 172.21.15.157:0/2433032268 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f040c1584b0 con 0x7f040c06b070 2023-10-06T03:57:04.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.444+0000 7f04017fa700 1 -- 172.21.15.157:0/2433032268 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0408011ad0 con 0x7f040c06b070 2023-10-06T03:57:04.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.444+0000 7f0413365700 1 -- 172.21.15.157:0/2433032268 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f040c0a6510 con 0x7f040c06b070 2023-10-06T03:57:04.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.445+0000 7f0413365700 1 -- 172.21.15.157:0/2433032268 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f040c066b20 con 0x7f040c06b070 2023-10-06T03:57:04.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.445+0000 7f04017fa700 1 -- 172.21.15.157:0/2433032268 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f040801c460 con 0x7f040c06b070 2023-10-06T03:57:04.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.446+0000 7f04017fa700 1 --2- 172.21.15.157:0/2433032268 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f03f0066ea0 0x7f03f0069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:04.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.446+0000 7f04017fa700 1 -- 172.21.15.157:0/2433032268 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5184+0+0 (secure 0 0 0) 0x7f0408083e70 con 0x7f040c06b070 2023-10-06T03:57:04.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.446+0000 7f0410900700 1 --2- 172.21.15.157:0/2433032268 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f03f0066ea0 0x7f03f0069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:04.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.446+0000 7f0410900700 1 --2- 172.21.15.157:0/2433032268 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f03f0066ea0 0x7f03f0069350 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f040c0ace30 tx=0x7f03fc016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:04.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.450+0000 7f04017fa700 1 -- 172.21.15.157:0/2433032268 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f04080513d0 con 0x7f040c06b070 2023-10-06T03:57:04.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:04.635+0000 7f0413365700 1 -- 172.21.15.157:0/2433032268 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f040c04bb40 con 0x7f040c06b070 2023-10-06T03:57:05.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.310+0000 7f04017fa700 1 -- 172.21.15.157:0/2433032268 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v16) v1 ==== 173+0+0 (secure 0 0 0) 0x7f04080549f0 con 0x7f040c06b070 2023-10-06T03:57:05.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.313+0000 7f0413365700 1 -- 172.21.15.157:0/2433032268 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f03f0066ea0 msgr2=0x7f03f0069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:05.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.313+0000 7f0413365700 1 --2- 172.21.15.157:0/2433032268 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f03f0066ea0 0x7f03f0069350 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f040c0ace30 tx=0x7f03fc016000 comp rx=0 tx=0).stop 2023-10-06T03:57:05.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.313+0000 7f0413365700 1 -- 172.21.15.157:0/2433032268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f040c06b070 msgr2=0x7f040c0c3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:05.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.313+0000 7f0413365700 1 --2- 172.21.15.157:0/2433032268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f040c06b070 0x7f040c0c3e00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f040800c600 tx=0x7f0408007f60 comp rx=0 tx=0).stop 2023-10-06T03:57:05.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.313+0000 7f0413365700 1 -- 172.21.15.157:0/2433032268 shutdown_connections 2023-10-06T03:57:05.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.313+0000 7f0413365700 1 --2- 172.21.15.157:0/2433032268 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f03f0066ea0 0x7f03f0069350 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:05.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.313+0000 7f0413365700 1 --2- 172.21.15.157:0/2433032268 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f040c0ad950 0x7f040c167a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:05.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.313+0000 7f0413365700 1 --2- 172.21.15.157:0/2433032268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f040c06b070 0x7f040c0c3e00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:05.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.313+0000 7f0413365700 1 --2- 172.21.15.157:0/2433032268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f040c0ad580 0x7f040c0c4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:05.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.313+0000 7f0413365700 1 -- 172.21.15.157:0/2433032268 >> 172.21.15.157:0/2433032268 conn(0x7f040c063cc0 msgr2=0x7f040c0c69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:05.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.313+0000 7f0413365700 1 -- 172.21.15.157:0/2433032268 shutdown_connections 2023-10-06T03:57:05.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.313+0000 7f0413365700 1 -- 172.21.15.157:0/2433032268 wait complete. 2023-10-06T03:57:05.319 INFO:teuthology.orchestra.run.smithi157.stderr:enabled application 'rbd' on pool 'rbd' 2023-10-06T03:57:05.329 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-06T03:57:05.329 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T03:57:05.329 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T03:57:05.331 INFO:tasks.ceph.mds.a:Started 2023-10-06T03:57:05.331 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T03:57:05.332 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T03:57:05.333 INFO:tasks.ceph.mds.c:Started 2023-10-06T03:57:05.334 INFO:tasks.ceph.mds.e:Restarting daemon 2023-10-06T03:57:05.334 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2023-10-06T03:57:05.335 INFO:tasks.ceph.mds.e:Started 2023-10-06T03:57:05.336 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T03:57:05.336 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T03:57:05.338 INFO:tasks.ceph.mds.b:Started 2023-10-06T03:57:05.338 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T03:57:05.338 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T03:57:05.340 INFO:tasks.ceph.mds.d:Started 2023-10-06T03:57:05.340 INFO:tasks.ceph.mds.f:Restarting daemon 2023-10-06T03:57:05.341 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2023-10-06T03:57:05.342 INFO:tasks.ceph.mds.f:Started 2023-10-06T03:57:05.343 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-06T03:57:05.343 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:57:05.407 INFO:tasks.ceph.mds.a.smithi157.stdout:starting mds.a at 2023-10-06T03:57:05.407 INFO:tasks.ceph.mds.c.smithi157.stdout:starting mds.c at 2023-10-06T03:57:05.411 INFO:tasks.ceph.mds.e.smithi157.stdout:starting mds.e at 2023-10-06T03:57:05.430 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2023-10-06T03:57:05.452 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2023-10-06T03:57:05.455 INFO:tasks.ceph.mds.f.smithi175.stdout:starting mds.f at 2023-10-06T03:57:05.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.468+0000 7f519145e700 1 Processor -- start 2023-10-06T03:57:05.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.468+0000 7f519145e700 1 -- start start 2023-10-06T03:57:05.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.469+0000 7f519145e700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f518c0b1d40 0x7f518c159df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:05.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.469+0000 7f519145e700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f518c0b2110 0x7f518c0af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:05.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.469+0000 7f519145e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f518c06c1f0 0x7f518c14fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:05.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.469+0000 7f519145e700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f518c06ced0 con 0x7f518c0b1d40 2023-10-06T03:57:05.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.469+0000 7f519145e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f518c06cc10 con 0x7f518c06c1f0 2023-10-06T03:57:05.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.469+0000 7f519145e700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518c06e570 con 0x7f518c0b2110 2023-10-06T03:57:05.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.469+0000 7f518affd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f518c0b1d40 0x7f518c159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:05.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.469+0000 7f518affd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f518c0b1d40 0x7f518c159df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58424/0 (socket says 172.21.15.157:58424) 2023-10-06T03:57:05.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.469+0000 7f518affd700 1 -- 172.21.15.157:0/3506326298 learned_addr learned my addr 172.21.15.157:0/3506326298 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:05.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.469+0000 7f518b7fe700 1 --2- 172.21.15.157:0/3506326298 >> v2:172.21.15.175:3300/0 conn(0x7f518c06c1f0 0x7f518c14fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:05.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.469+0000 7f518a7fc700 1 --2- 172.21.15.157:0/3506326298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f518c0b2110 0x7f518c0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:05.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.469+0000 7f518affd700 1 -- 172.21.15.157:0/3506326298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f518c0b2110 msgr2=0x7f518c0af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:05.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.469+0000 7f518affd700 1 --2- 172.21.15.157:0/3506326298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f518c0b2110 0x7f518c0af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:05.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.469+0000 7f518affd700 1 -- 172.21.15.157:0/3506326298 >> v2:172.21.15.175:3300/0 conn(0x7f518c06c1f0 msgr2=0x7f518c14fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:05.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.469+0000 7f518affd700 1 --2- 172.21.15.157:0/3506326298 >> v2:172.21.15.175:3300/0 conn(0x7f518c06c1f0 0x7f518c14fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:05.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.469+0000 7f518affd700 1 -- 172.21.15.157:0/3506326298 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c06d7b0 con 0x7f518c0b1d40 2023-10-06T03:57:05.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.469+0000 7f518a7fc700 1 --2- 172.21.15.157:0/3506326298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f518c0b2110 0x7f518c0af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:57:05.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.469+0000 7f518affd700 1 --2- 172.21.15.157:0/3506326298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518c0b1d40 0x7f518c159df0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f518c0b2700 tx=0x7f517c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f951e3f4f95615b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:05.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.469+0000 7f5189ffb700 1 -- 172.21.15.157:0/3506326298 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c00b8f0 con 0x7f518c0b1d40 2023-10-06T03:57:05.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.469+0000 7f5189ffb700 1 -- 172.21.15.157:0/3506326298 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517c00ba50 con 0x7f518c0b1d40 2023-10-06T03:57:05.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.469+0000 7f5189ffb700 1 -- 172.21.15.157:0/3506326298 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c00bdf0 con 0x7f518c0b1d40 2023-10-06T03:57:05.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.470+0000 7f519145e700 1 -- 172.21.15.157:0/3506326298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518c0b1d40 msgr2=0x7f518c159df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:05.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.470+0000 7f519145e700 1 --2- 172.21.15.157:0/3506326298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518c0b1d40 0x7f518c159df0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f518c0b2700 tx=0x7f517c009250 comp rx=0 tx=0).stop 2023-10-06T03:57:05.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.470+0000 7f519145e700 1 -- 172.21.15.157:0/3506326298 shutdown_connections 2023-10-06T03:57:05.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.470+0000 7f519145e700 1 --2- 172.21.15.157:0/3506326298 >> v2:172.21.15.175:3300/0 conn(0x7f518c06c1f0 0x7f518c14fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:05.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.470+0000 7f519145e700 1 --2- 172.21.15.157:0/3506326298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518c0b1d40 0x7f518c159df0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:05.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.470+0000 7f519145e700 1 --2- 172.21.15.157:0/3506326298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f518c0b2110 0x7f518c0af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:05.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.470+0000 7f519145e700 1 -- 172.21.15.157:0/3506326298 >> 172.21.15.157:0/3506326298 conn(0x7f518c0cc6e0 msgr2=0x7f518c0ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:05.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.470+0000 7f519145e700 1 -- 172.21.15.157:0/3506326298 shutdown_connections 2023-10-06T03:57:05.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.470+0000 7f519145e700 1 -- 172.21.15.157:0/3506326298 wait complete. 2023-10-06T03:57:05.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.471+0000 7f519145e700 1 Processor -- start 2023-10-06T03:57:05.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.471+0000 7f519145e700 1 -- start start 2023-10-06T03:57:05.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.472+0000 7f519145e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518c06c1f0 0x7f518c0bfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:05.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.472+0000 7f519145e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f518c0b1d40 0x7f518c153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:05.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.472+0000 7f519145e700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f518c0b2110 0x7f518c14c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:05.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.472+0000 7f519145e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518c0c0b00 con 0x7f518c06c1f0 2023-10-06T03:57:05.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.472+0000 7f519145e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518c166870 con 0x7f518c0b1d40 2023-10-06T03:57:05.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.472+0000 7f519145e700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518c0c0540 con 0x7f518c0b2110 2023-10-06T03:57:05.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.472+0000 7f518affd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518c06c1f0 0x7f518c0bfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:05.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.472+0000 7f518affd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518c06c1f0 0x7f518c0bfeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58434/0 (socket says 172.21.15.157:58434) 2023-10-06T03:57:05.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.472+0000 7f518affd700 1 -- 172.21.15.157:0/3934127416 learned_addr learned my addr 172.21.15.157:0/3934127416 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:05.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.472+0000 7f518a7fc700 1 --2- 172.21.15.157:0/3934127416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f518c0b1d40 0x7f518c153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:05.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.472+0000 7f518b7fe700 1 --2- 172.21.15.157:0/3934127416 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f518c0b2110 0x7f518c14c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:05.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.472+0000 7f518affd700 1 -- 172.21.15.157:0/3934127416 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f518c0b2110 msgr2=0x7f518c14c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:05.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.472+0000 7f518affd700 1 --2- 172.21.15.157:0/3934127416 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f518c0b2110 0x7f518c14c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:05.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.472+0000 7f518affd700 1 -- 172.21.15.157:0/3934127416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f518c0b1d40 msgr2=0x7f518c153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:05.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.472+0000 7f518affd700 1 --2- 172.21.15.157:0/3934127416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f518c0b1d40 0x7f518c153a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:05.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.472+0000 7f518affd700 1 -- 172.21.15.157:0/3934127416 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f517c0094f0 con 0x7f518c06c1f0 2023-10-06T03:57:05.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.472+0000 7f518affd700 1 --2- 172.21.15.157:0/3934127416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518c06c1f0 0x7f518c0bfeb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f518c08a240 tx=0x7f517c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:05.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.473+0000 7f5172ffd700 1 -- 172.21.15.157:0/3934127416 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c024440 con 0x7f518c06c1f0 2023-10-06T03:57:05.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.473+0000 7f5172ffd700 1 -- 172.21.15.157:0/3934127416 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517c0245a0 con 0x7f518c06c1f0 2023-10-06T03:57:05.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.473+0000 7f519145e700 1 -- 172.21.15.157:0/3934127416 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f518c0c0820 con 0x7f518c06c1f0 2023-10-06T03:57:05.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.473+0000 7f5172ffd700 1 -- 172.21.15.157:0/3934127416 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c024960 con 0x7f518c06c1f0 2023-10-06T03:57:05.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.473+0000 7f519145e700 1 -- 172.21.15.157:0/3934127416 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f518c166500 con 0x7f518c06c1f0 2023-10-06T03:57:05.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.473+0000 7f519145e700 1 -- 172.21.15.157:0/3934127416 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f518c166080 con 0x7f518c06c1f0 2023-10-06T03:57:05.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.474+0000 7f5172ffd700 1 -- 172.21.15.157:0/3934127416 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f517c024d50 con 0x7f518c06c1f0 2023-10-06T03:57:05.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.474+0000 7f5172ffd700 1 --2- 172.21.15.157:0/3934127416 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5178066ea0 0x7f5178069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:05.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.474+0000 7f5172ffd700 1 -- 172.21.15.157:0/3934127416 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f517c085120 con 0x7f518c06c1f0 2023-10-06T03:57:05.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.474+0000 7f518a7fc700 1 --2- 172.21.15.157:0/3934127416 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5178066ea0 0x7f5178069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:05.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.475+0000 7f518a7fc700 1 --2- 172.21.15.157:0/3934127416 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5178066ea0 0x7f5178069350 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f518c155510 tx=0x7f5180008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:05.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.478+0000 7f5172ffd700 1 -- 172.21.15.157:0/3934127416 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f518c166080 con 0x7f518c06c1f0 2023-10-06T03:57:05.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.665+0000 7f519145e700 1 -- 172.21.15.157:0/3934127416 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f518c166080 con 0x7f518c06c1f0 2023-10-06T03:57:05.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.666+0000 7f5172ffd700 1 -- 172.21.15.157:0/3934127416 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+13721 (secure 0 0 0) 0x7f518c166080 con 0x7f518c06c1f0 2023-10-06T03:57:05.667 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:05.668 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":16,"fsid":"28664a7a-fe30-4efe-be5a-6abf9318b700","created":"2023-10-06T03:56:51.887538+0000","modified":"2023-10-06T03:57:05.307858+0000","last_up_change":"2023-10-06T03:57:02.299440+0000","last_in_change":"2023-10-06T03:56:55.141505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:57:03.455121+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:57:03.905661+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"3f677cac-e9f4-4608-acac-718b072ce984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":90640},{"type":"v1","addr":"172.21.15.157:6809","nonce":90640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":90640},{"type":"v1","addr":"172.21.15.157:6811","nonce":90640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":90640},{"type":"v1","addr":"172.21.15.157:6815","nonce":90640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":90640},{"type":"v1","addr":"172.21.15.157:6813","nonce":90640}]},"public_addr":"172.21.15.157:6809/90640","cluster_addr":"172.21.15.157:6811/90640","heartbeat_back_addr":"172.21.15.157:6815/90640","heartbeat_front_addr":"172.21.15.157:6813/90640","state":["exists","up"]},{"osd":1,"uuid":"7d771892-bb91-4e9b-9610-051861c11247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":90654},{"type":"v1","addr":"172.21.15.157:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":90654},{"type":"v1","addr":"172.21.15.157:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":90654},{"type":"v1","addr":"172.21.15.157:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":90654},{"type":"v1","addr":"172.21.15.157:6805","nonce":90654}]},"public_addr":"172.21.15.157:6801/90654","cluster_addr":"172.21.15.157:6803/90654","heartbeat_back_addr":"172.21.15.157:6807/90654","heartbeat_front_addr":"172.21.15.157:6805/90654","state":["exists","up"]},{"osd":2,"uuid":"5c1f7d4e-d185-47cf-ab6c-453b4e3009c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":90657},{"type":"v1","addr":"172.21.15.157:6825","nonce":90657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":90657},{"type":"v1","addr":"172.21.15.157:6827","nonce":90657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":90657},{"type":"v1","addr":"172.21.15.157:6831","nonce":90657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":90657},{"type":"v1","addr":"172.21.15.157:6829","nonce":90657}]},"public_addr":"172.21.15.157:6825/90657","cluster_addr":"172.21.15.157:6827/90657","heartbeat_back_addr":"172.21.15.157:6831/90657","heartbeat_front_addr":"172.21.15.157:6829/90657","state":["exists","up"]},{"osd":3,"uuid":"f6a1e825-43c7-4afd-8daf-e51d45afc75e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":90641},{"type":"v1","addr":"172.21.15.157:6817","nonce":90641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":90641},{"type":"v1","addr":"172.21.15.157:6819","nonce":90641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":90641},{"type":"v1","addr":"172.21.15.157:6823","nonce":90641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":90641},{"type":"v1","addr":"172.21.15.157:6821","nonce":90641}]},"public_addr":"172.21.15.157:6817/90641","cluster_addr":"172.21.15.157:6819/90641","heartbeat_back_addr":"172.21.15.157:6823/90641","heartbeat_front_addr":"172.21.15.157:6821/90641","state":["exists","up"]},{"osd":4,"uuid":"8ad6e3d8-9a31-4f85-9410-417b7ac5da97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":91261},{"type":"v1","addr":"172.21.15.175:6803","nonce":91261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":91261},{"type":"v1","addr":"172.21.15.175:6806","nonce":91261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":91261},{"type":"v1","addr":"172.21.15.175:6814","nonce":91261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":91261},{"type":"v1","addr":"172.21.15.175:6810","nonce":91261}]},"public_addr":"172.21.15.175:6803/91261","cluster_addr":"172.21.15.175:6806/91261","heartbeat_back_addr":"172.21.15.175:6814/91261","heartbeat_front_addr":"172.21.15.175:6810/91261","state":["exists","up"]},{"osd":5,"uuid":"f68aca2a-f182-4c00-8e21-241d09c02ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":91262},{"type":"v1","addr":"172.21.15.175:6817","nonce":91262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":91262},{"type":"v1","addr":"172.21.15.175:6819","nonce":91262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":91262},{"type":"v1","addr":"172.21.15.175:6823","nonce":91262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":91262},{"type":"v1","addr":"172.21.15.175:6821","nonce":91262}]},"public_addr":"172.21.15.175:6817/91262","cluster_addr":"172.21.15.175:6819/91262","heartbeat_back_addr":"172.21.15.175:6823/91262","heartbeat_front_addr":"172.21.15.175:6821/91262","state":["exists","up"]},{"osd":6,"uuid":"d867cd89-3430-40ca-862f-d2bea3de9d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":91263},{"type":"v1","addr":"172.21.15.175:6801","nonce":91263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6805","nonce":91263},{"type":"v1","addr":"172.21.15.175:6807","nonce":91263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6813","nonce":91263},{"type":"v1","addr":"172.21.15.175:6815","nonce":91263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":91263},{"type":"v1","addr":"172.21.15.175:6811","nonce":91263}]},"public_addr":"172.21.15.175:6801/91263","cluster_addr":"172.21.15.175:6807/91263","heartbeat_back_addr":"172.21.15.175:6815/91263","heartbeat_front_addr":"172.21.15.175:6811/91263","state":["exists","up"]},{"osd":7,"uuid":"930cd464-79f1-47cb-b4ff-188ccde74fff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":91306},{"type":"v1","addr":"172.21.15.175:6825","nonce":91306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":91306},{"type":"v1","addr":"172.21.15.175:6827","nonce":91306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":91306},{"type":"v1","addr":"172.21.15.175:6831","nonce":91306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":91306},{"type":"v1","addr":"172.21.15.175:6829","nonce":91306}]},"public_addr":"172.21.15.175:6825/91306","cluster_addr":"172.21.15.175:6827/91306","heartbeat_back_addr":"172.21.15.175:6831/91306","heartbeat_front_addr":"172.21.15.175:6829/91306","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.290673+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.056390+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.292903+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.275861+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.332095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.327453+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.317585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.309330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:57:05.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.668+0000 7f519145e700 1 -- 172.21.15.157:0/3934127416 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5178066ea0 msgr2=0x7f5178069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:05.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.668+0000 7f519145e700 1 --2- 172.21.15.157:0/3934127416 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5178066ea0 0x7f5178069350 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f518c155510 tx=0x7f5180008040 comp rx=0 tx=0).stop 2023-10-06T03:57:05.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.668+0000 7f519145e700 1 -- 172.21.15.157:0/3934127416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518c06c1f0 msgr2=0x7f518c0bfeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:05.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.668+0000 7f519145e700 1 --2- 172.21.15.157:0/3934127416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518c06c1f0 0x7f518c0bfeb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f518c08a240 tx=0x7f517c017000 comp rx=0 tx=0).stop 2023-10-06T03:57:05.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.668+0000 7f519145e700 1 -- 172.21.15.157:0/3934127416 shutdown_connections 2023-10-06T03:57:05.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.668+0000 7f519145e700 1 --2- 172.21.15.157:0/3934127416 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5178066ea0 0x7f5178069350 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:05.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.668+0000 7f519145e700 1 --2- 172.21.15.157:0/3934127416 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f518c0b2110 0x7f518c14c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:05.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.668+0000 7f519145e700 1 --2- 172.21.15.157:0/3934127416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518c06c1f0 0x7f518c0bfeb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:05.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.668+0000 7f519145e700 1 --2- 172.21.15.157:0/3934127416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f518c0b1d40 0x7f518c153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:05.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.668+0000 7f519145e700 1 -- 172.21.15.157:0/3934127416 >> 172.21.15.157:0/3934127416 conn(0x7f518c0cc6e0 msgr2=0x7f518c156f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:05.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.668+0000 7f519145e700 1 -- 172.21.15.157:0/3934127416 shutdown_connections 2023-10-06T03:57:05.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.668+0000 7f519145e700 1 -- 172.21.15.157:0/3934127416 wait complete. 2023-10-06T03:57:05.682 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:57:03.455121+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:57:03.905661+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T03:57:05.682 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:57:05.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.804+0000 7fa96d370700 1 Processor -- start 2023-10-06T03:57:05.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.804+0000 7fa96d370700 1 -- start start 2023-10-06T03:57:05.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.805+0000 7fa96d370700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa96814ccf0 0x7fa9680c0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:05.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.805+0000 7fa96d370700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa9680631f0 0x7fa96806d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:05.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.805+0000 7fa96d370700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9680c3390 con 0x7fa96806d840 2023-10-06T03:57:05.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.805+0000 7fa96d370700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa968063800 con 0x7fa9680631f0 2023-10-06T03:57:05.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.805+0000 7fa96d370700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9681560f0 con 0x7fa96814ccf0 2023-10-06T03:57:05.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.805+0000 7fa9667fc700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa9680631f0 0x7fa96806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:05.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.805+0000 7fa9677fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa96806d840 0x7fa9680c3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44068/0 (socket says 172.21.15.157:44068) 2023-10-06T03:57:05.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.805+0000 7fa9677fe700 1 -- 172.21.15.157:0/2422373951 learned_addr learned my addr 172.21.15.157:0/2422373951 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:05.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.805+0000 7fa9667fc700 1 -- 172.21.15.157:0/2422373951 >> v2:172.21.15.175:3300/0 conn(0x7fa96814ccf0 msgr2=0x7fa9680c0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:05.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.805+0000 7fa966ffd700 1 --2- 172.21.15.157:0/2422373951 >> v2:172.21.15.175:3300/0 conn(0x7fa96814ccf0 0x7fa9680c0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:05.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.805+0000 7fa9667fc700 1 --2- 172.21.15.157:0/2422373951 >> v2:172.21.15.175:3300/0 conn(0x7fa96814ccf0 0x7fa9680c0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:05.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.805+0000 7fa965ffb700 1 -- 172.21.15.157:0/2422373951 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 519022864 0 0) 0x7fa9680c3390 con 0x7fa96806d840 2023-10-06T03:57:05.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.805+0000 7fa9667fc700 1 -- 172.21.15.157:0/2422373951 >> v1:172.21.15.157:6789/0 conn(0x7fa96806d840 legacy=0x7fa9680c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:05.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.805+0000 7fa9667fc700 1 -- 172.21.15.157:0/2422373951 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9681563d0 con 0x7fa9680631f0 2023-10-06T03:57:05.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.805+0000 7fa966ffd700 1 --2- 172.21.15.157:0/2422373951 >> v2:172.21.15.175:3300/0 conn(0x7fa96814ccf0 0x7fa9680c0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:57:05.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.806+0000 7fa9667fc700 1 --2- 172.21.15.157:0/2422373951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9680631f0 0x7fa96806d300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa9680c9650 tx=0x7fa9500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2995f626c24f8895 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:05.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.806+0000 7fa965ffb700 1 -- 172.21.15.157:0/2422373951 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95000b740 con 0x7fa9680631f0 2023-10-06T03:57:05.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.806+0000 7fa965ffb700 1 -- 172.21.15.157:0/2422373951 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95000b8a0 con 0x7fa9680631f0 2023-10-06T03:57:05.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.806+0000 7fa965ffb700 1 -- 172.21.15.157:0/2422373951 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95000bc40 con 0x7fa9680631f0 2023-10-06T03:57:05.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.806+0000 7fa96d370700 1 -- 172.21.15.157:0/2422373951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9680631f0 msgr2=0x7fa96806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:05.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.806+0000 7fa96d370700 1 --2- 172.21.15.157:0/2422373951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9680631f0 0x7fa96806d300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa9680c9650 tx=0x7fa9500091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:05.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.806+0000 7fa96d370700 1 -- 172.21.15.157:0/2422373951 shutdown_connections 2023-10-06T03:57:05.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.806+0000 7fa96d370700 1 --2- 172.21.15.157:0/2422373951 >> v2:172.21.15.175:3300/0 conn(0x7fa96814ccf0 0x7fa9680c0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:05.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.806+0000 7fa96d370700 1 --2- 172.21.15.157:0/2422373951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9680631f0 0x7fa96806d300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:05.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.806+0000 7fa96d370700 1 -- 172.21.15.157:0/2422373951 >> 172.21.15.157:0/2422373951 conn(0x7fa9680c8a20 msgr2=0x7fa9680c8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:05.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.806+0000 7fa96d370700 1 -- 172.21.15.157:0/2422373951 shutdown_connections 2023-10-06T03:57:05.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.806+0000 7fa96d370700 1 -- 172.21.15.157:0/2422373951 wait complete. 2023-10-06T03:57:05.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.807+0000 7fa96d370700 1 Processor -- start 2023-10-06T03:57:05.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.807+0000 7fa96d370700 1 -- start start 2023-10-06T03:57:05.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.808+0000 7fa96d370700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9680631f0 0x7fa96814f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:05.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.808+0000 7fa96d370700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa96806d840 0x7fa96814f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:05.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.808+0000 7fa96d370700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa96814ccf0 0x7fa968155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:05.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.808+0000 7fa96d370700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa968157730 con 0x7fa96814ccf0 2023-10-06T03:57:05.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.808+0000 7fa96d370700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa968164300 con 0x7fa96806d840 2023-10-06T03:57:05.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.808+0000 7fa96d370700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa968157980 con 0x7fa9680631f0 2023-10-06T03:57:05.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.808+0000 7fa9677fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa96814ccf0 0x7fa968155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:05.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.808+0000 7fa9677fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa96814ccf0 0x7fa968155420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58464/0 (socket says 172.21.15.157:58464) 2023-10-06T03:57:05.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.808+0000 7fa9677fe700 1 -- 172.21.15.157:0/2980988205 learned_addr learned my addr 172.21.15.157:0/2980988205 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:05.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.808+0000 7fa9667fc700 1 --2- 172.21.15.157:0/2980988205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa96806d840 0x7fa96814f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:05.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.808+0000 7fa966ffd700 1 --2- 172.21.15.157:0/2980988205 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9680631f0 0x7fa96814f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:05.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.809+0000 7fa9677fe700 1 -- 172.21.15.157:0/2980988205 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9680631f0 msgr2=0x7fa96814f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:05.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.809+0000 7fa9677fe700 1 --2- 172.21.15.157:0/2980988205 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9680631f0 0x7fa96814f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:05.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.809+0000 7fa9677fe700 1 -- 172.21.15.157:0/2980988205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa96806d840 msgr2=0x7fa96814f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:05.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.809+0000 7fa9677fe700 1 --2- 172.21.15.157:0/2980988205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa96806d840 0x7fa96814f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:05.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.809+0000 7fa9677fe700 1 -- 172.21.15.157:0/2980988205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa950009420 con 0x7fa96814ccf0 2023-10-06T03:57:05.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.809+0000 7fa9667fc700 1 --2- 172.21.15.157:0/2980988205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa96806d840 0x7fa96814f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:05.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.809+0000 7fa9677fe700 1 --2- 172.21.15.157:0/2980988205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa96814ccf0 0x7fa968155420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa95800c600 tx=0x7fa958007d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:05.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.809+0000 7fa95effd700 1 -- 172.21.15.157:0/2980988205 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa958005b80 con 0x7fa96814ccf0 2023-10-06T03:57:05.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.809+0000 7fa95effd700 1 -- 172.21.15.157:0/2980988205 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa958005ce0 con 0x7fa96814ccf0 2023-10-06T03:57:05.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.809+0000 7fa96d370700 1 -- 172.21.15.157:0/2980988205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa968157c60 con 0x7fa96814ccf0 2023-10-06T03:57:05.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.809+0000 7fa95effd700 1 -- 172.21.15.157:0/2980988205 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa958007320 con 0x7fa96814ccf0 2023-10-06T03:57:05.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.809+0000 7fa96d370700 1 -- 172.21.15.157:0/2980988205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa968158220 con 0x7fa96814ccf0 2023-10-06T03:57:05.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.810+0000 7fa96d370700 1 -- 172.21.15.157:0/2980988205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa96806b4c0 con 0x7fa96814ccf0 2023-10-06T03:57:05.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.810+0000 7fa95effd700 1 -- 172.21.15.157:0/2980988205 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fa9580076d0 con 0x7fa96814ccf0 2023-10-06T03:57:05.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.811+0000 7fa95effd700 1 --2- 172.21.15.157:0/2980988205 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa948066ea0 0x7fa948069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:05.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.811+0000 7fa95effd700 1 -- 172.21.15.157:0/2980988205 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fa9580837b0 con 0x7fa96814ccf0 2023-10-06T03:57:05.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.811+0000 7fa966ffd700 1 --2- 172.21.15.157:0/2980988205 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa948066ea0 0x7fa948069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:05.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.811+0000 7fa966ffd700 1 --2- 172.21.15.157:0/2980988205 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa948066ea0 0x7fa948069350 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa96000e4b0 tx=0x7fa960007670 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:05.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:05.815+0000 7fa95effd700 1 -- 172.21.15.157:0/2980988205 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa958050cf0 con 0x7fa96814ccf0 2023-10-06T03:57:06.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.015+0000 7fa96d370700 1 -- 172.21.15.157:0/2980988205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa968063f80 con 0x7fa96814ccf0 2023-10-06T03:57:06.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.015+0000 7fa95effd700 1 -- 172.21.15.157:0/2980988205 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa9580170a0 con 0x7fa96814ccf0 2023-10-06T03:57:06.017 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-10-06T03:57:06.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.017+0000 7fa96d370700 1 -- 172.21.15.157:0/2980988205 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa948066ea0 msgr2=0x7fa948069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:06.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.017+0000 7fa96d370700 1 --2- 172.21.15.157:0/2980988205 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa948066ea0 0x7fa948069350 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa96000e4b0 tx=0x7fa960007670 comp rx=0 tx=0).stop 2023-10-06T03:57:06.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.017+0000 7fa96d370700 1 -- 172.21.15.157:0/2980988205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa96814ccf0 msgr2=0x7fa968155420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:06.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.017+0000 7fa96d370700 1 --2- 172.21.15.157:0/2980988205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa96814ccf0 0x7fa968155420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa95800c600 tx=0x7fa958007d60 comp rx=0 tx=0).stop 2023-10-06T03:57:06.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.017+0000 7fa96d370700 1 -- 172.21.15.157:0/2980988205 shutdown_connections 2023-10-06T03:57:06.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.017+0000 7fa96d370700 1 --2- 172.21.15.157:0/2980988205 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa948066ea0 0x7fa948069350 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:06.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.017+0000 7fa96d370700 1 --2- 172.21.15.157:0/2980988205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa96814ccf0 0x7fa968155420 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:06.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.017+0000 7fa96d370700 1 --2- 172.21.15.157:0/2980988205 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9680631f0 0x7fa96814f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:06.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.017+0000 7fa96d370700 1 --2- 172.21.15.157:0/2980988205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa96806d840 0x7fa96814f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:06.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.017+0000 7fa96d370700 1 -- 172.21.15.157:0/2980988205 >> 172.21.15.157:0/2980988205 conn(0x7fa9680c8a20 msgr2=0x7fa9680c1400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:06.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.018+0000 7fa96d370700 1 -- 172.21.15.157:0/2980988205 shutdown_connections 2023-10-06T03:57:06.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.018+0000 7fa96d370700 1 -- 172.21.15.157:0/2980988205 wait complete. 2023-10-06T03:57:06.032 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:57:06.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.151+0000 7f0d7d2d1700 1 Processor -- start 2023-10-06T03:57:06.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.151+0000 7f0d7d2d1700 1 -- start start 2023-10-06T03:57:06.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.152+0000 7f0d7d2d1700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0d78150410 0x7f0d78155f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:06.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.152+0000 7f0d7d2d1700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d780c9780 con 0x7f0d7806dc40 2023-10-06T03:57:06.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.152+0000 7f0d7d2d1700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d7806e0c0 con 0x7f0d7806cbb0 2023-10-06T03:57:06.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.152+0000 7f0d7d2d1700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d780c8c40 con 0x7f0d78150410 2023-10-06T03:57:06.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.152+0000 7f0d777fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0d78150410 0x7f0d78155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:06.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.152+0000 7f0d6e7fc700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f0d7806dc40 0x7f0d7814fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44082/0 (socket says 172.21.15.157:44082) 2023-10-06T03:57:06.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.152+0000 7f0d6e7fc700 1 -- 172.21.15.157:0/2771466852 learned_addr learned my addr 172.21.15.157:0/2771466852 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:06.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.153+0000 7f0d777fe700 1 -- 172.21.15.157:0/2771466852 >> v1:172.21.15.175:6789/0 conn(0x7f0d7806cbb0 legacy=0x7f0d7806d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:06.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.153+0000 7f0d777fe700 1 -- 172.21.15.157:0/2771466852 >> v1:172.21.15.157:6789/0 conn(0x7f0d7806dc40 legacy=0x7f0d7814fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:06.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.153+0000 7f0d777fe700 1 -- 172.21.15.157:0/2771466852 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d780c8e50 con 0x7f0d78150410 2023-10-06T03:57:06.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.153+0000 7f0d777fe700 1 --2- 172.21.15.157:0/2771466852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d78150410 0x7f0d78155f70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009e60 tx=0x7f0d70009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951973e81fad9d8b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:06.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.153+0000 7f0d767fc700 1 -- 172.21.15.157:0/2771466852 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7000e070 con 0x7f0d78150410 2023-10-06T03:57:06.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.153+0000 7f0d767fc700 1 -- 172.21.15.157:0/2771466852 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7000bcc0 con 0x7f0d78150410 2023-10-06T03:57:06.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.153+0000 7f0d767fc700 1 -- 172.21.15.157:0/2771466852 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d70012680 con 0x7f0d78150410 2023-10-06T03:57:06.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.153+0000 7f0d7d2d1700 1 -- 172.21.15.157:0/2771466852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d78150410 msgr2=0x7f0d78155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:06.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.153+0000 7f0d7d2d1700 1 --2- 172.21.15.157:0/2771466852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d78150410 0x7f0d78155f70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009e60 tx=0x7f0d70009780 comp rx=0 tx=0).stop 2023-10-06T03:57:06.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.153+0000 7f0d7d2d1700 1 -- 172.21.15.157:0/2771466852 shutdown_connections 2023-10-06T03:57:06.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.153+0000 7f0d7d2d1700 1 --2- 172.21.15.157:0/2771466852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d78150410 0x7f0d78155f70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:06.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.153+0000 7f0d7d2d1700 1 -- 172.21.15.157:0/2771466852 >> 172.21.15.157:0/2771466852 conn(0x7f0d7806f230 msgr2=0x7f0d7806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:06.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.153+0000 7f0d7d2d1700 1 -- 172.21.15.157:0/2771466852 shutdown_connections 2023-10-06T03:57:06.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.153+0000 7f0d7d2d1700 1 -- 172.21.15.157:0/2771466852 wait complete. 2023-10-06T03:57:06.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.155+0000 7f0d7d2d1700 1 Processor -- start 2023-10-06T03:57:06.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.155+0000 7f0d7d2d1700 1 -- start start 2023-10-06T03:57:06.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.155+0000 7f0d7d2d1700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d7806cbb0 0x7f0d7814a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:06.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.155+0000 7f0d7d2d1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d78150410 0x7f0d78167ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:06.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.155+0000 7f0d7d2d1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d78168430 0x7f0d78169a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:06.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.155+0000 7f0d7d2d1700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d78167aa0 con 0x7f0d78168430 2023-10-06T03:57:06.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.155+0000 7f0d7d2d1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d781533e0 con 0x7f0d78150410 2023-10-06T03:57:06.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.155+0000 7f0d7d2d1700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d78152570 con 0x7f0d7806cbb0 2023-10-06T03:57:06.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.156+0000 7f0d777fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d78168430 0x7f0d78169a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:06.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.156+0000 7f0d777fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d78168430 0x7f0d78169a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58490/0 (socket says 172.21.15.157:58490) 2023-10-06T03:57:06.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.156+0000 7f0d777fe700 1 -- 172.21.15.157:0/985698532 learned_addr learned my addr 172.21.15.157:0/985698532 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:06.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.156+0000 7f0d6e7fc700 1 --2- 172.21.15.157:0/985698532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d78150410 0x7f0d78167ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:06.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.156+0000 7f0d76ffd700 1 --2- 172.21.15.157:0/985698532 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d7806cbb0 0x7f0d7814a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:06.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.156+0000 7f0d777fe700 1 -- 172.21.15.157:0/985698532 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d7806cbb0 msgr2=0x7f0d7814a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:06.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.156+0000 7f0d777fe700 1 --2- 172.21.15.157:0/985698532 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d7806cbb0 0x7f0d7814a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:06.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.156+0000 7f0d777fe700 1 -- 172.21.15.157:0/985698532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d78150410 msgr2=0x7f0d78167ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:06.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.156+0000 7f0d777fe700 1 --2- 172.21.15.157:0/985698532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d78150410 0x7f0d78167ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:06.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.156+0000 7f0d777fe700 1 -- 172.21.15.157:0/985698532 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d70009a20 con 0x7f0d78168430 2023-10-06T03:57:06.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.156+0000 7f0d777fe700 1 --2- 172.21.15.157:0/985698532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d78168430 0x7f0d78169a70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000b6c0 tx=0x7f0d70019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:06.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.156+0000 7f0d6f7fe700 1 -- 172.21.15.157:0/985698532 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7000bed0 con 0x7f0d78168430 2023-10-06T03:57:06.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.156+0000 7f0d6f7fe700 1 -- 172.21.15.157:0/985698532 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7001e630 con 0x7f0d78168430 2023-10-06T03:57:06.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.156+0000 7f0d6f7fe700 1 -- 172.21.15.157:0/985698532 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7001e9f0 con 0x7f0d78168430 2023-10-06T03:57:06.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.156+0000 7f0d7d2d1700 1 -- 172.21.15.157:0/985698532 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d780bfc70 con 0x7f0d78168430 2023-10-06T03:57:06.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.156+0000 7f0d7d2d1700 1 -- 172.21.15.157:0/985698532 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d7814e860 con 0x7f0d78168430 2023-10-06T03:57:06.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.157+0000 7f0d7d2d1700 1 -- 172.21.15.157:0/985698532 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d78167c10 con 0x7f0d78168430 2023-10-06T03:57:06.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.157+0000 7f0d6f7fe700 1 -- 172.21.15.157:0/985698532 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f0d70024070 con 0x7f0d78168430 2023-10-06T03:57:06.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.158+0000 7f0d6f7fe700 1 --2- 172.21.15.157:0/985698532 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0d58066f20 0x7f0d580693d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:06.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.158+0000 7f0d6f7fe700 1 -- 172.21.15.157:0/985698532 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f0d70086c00 con 0x7f0d78168430 2023-10-06T03:57:06.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.158+0000 7f0d76ffd700 1 --2- 172.21.15.157:0/985698532 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0d58066f20 0x7f0d580693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:06.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.158+0000 7f0d76ffd700 1 --2- 172.21.15.157:0/985698532 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0d58066f20 0x7f0d580693d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0d68000c00 tx=0x7f0d68007ea0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:06.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.162+0000 7f0d6f7fe700 1 -- 172.21.15.157:0/985698532 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d700540a0 con 0x7f0d78168430 2023-10-06T03:57:06.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.351+0000 7f0d7d2d1700 1 -- 172.21.15.157:0/985698532 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0d780c9130 con 0x7f0d78168430 2023-10-06T03:57:06.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.351+0000 7f0d6f7fe700 1 -- 172.21.15.157:0/985698532 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0d700576c0 con 0x7f0d78168430 2023-10-06T03:57:06.353 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-10-06T03:57:06.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.353+0000 7f0d7d2d1700 1 -- 172.21.15.157:0/985698532 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0d58066f20 msgr2=0x7f0d580693d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:06.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.353+0000 7f0d7d2d1700 1 --2- 172.21.15.157:0/985698532 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0d58066f20 0x7f0d580693d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0d68000c00 tx=0x7f0d68007ea0 comp rx=0 tx=0).stop 2023-10-06T03:57:06.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.353+0000 7f0d7d2d1700 1 -- 172.21.15.157:0/985698532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d78168430 msgr2=0x7f0d78169a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:06.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.353+0000 7f0d7d2d1700 1 --2- 172.21.15.157:0/985698532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d78168430 0x7f0d78169a70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000b6c0 tx=0x7f0d70019000 comp rx=0 tx=0).stop 2023-10-06T03:57:06.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.353+0000 7f0d7d2d1700 1 -- 172.21.15.157:0/985698532 shutdown_connections 2023-10-06T03:57:06.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.353+0000 7f0d7d2d1700 1 --2- 172.21.15.157:0/985698532 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0d58066f20 0x7f0d580693d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:06.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.353+0000 7f0d7d2d1700 1 --2- 172.21.15.157:0/985698532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d78168430 0x7f0d78169a70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:06.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.353+0000 7f0d7d2d1700 1 --2- 172.21.15.157:0/985698532 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d7806cbb0 0x7f0d7814a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:06.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.353+0000 7f0d7d2d1700 1 --2- 172.21.15.157:0/985698532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d78150410 0x7f0d78167ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:06.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.353+0000 7f0d7d2d1700 1 -- 172.21.15.157:0/985698532 >> 172.21.15.157:0/985698532 conn(0x7f0d7806f230 msgr2=0x7f0d7814b590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:06.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.353+0000 7f0d7d2d1700 1 -- 172.21.15.157:0/985698532 shutdown_connections 2023-10-06T03:57:06.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.353+0000 7f0d7d2d1700 1 -- 172.21.15.157:0/985698532 wait complete. 2023-10-06T03:57:06.366 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:57:06.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.482+0000 7fe50e244700 1 Processor -- start 2023-10-06T03:57:06.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.482+0000 7fe50e244700 1 -- start start 2023-10-06T03:57:06.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.483+0000 7fe50e244700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe50806a780 0x7fe508159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:06.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.483+0000 7fe50e244700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe5080bc750 0x7fe50814f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:06.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.483+0000 7fe50e244700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5081569b0 con 0x7fe50806ab50 2023-10-06T03:57:06.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.483+0000 7fe50e244700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe508155e90 con 0x7fe5080bc750 2023-10-06T03:57:06.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.483+0000 7fe50e244700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe50815a310 con 0x7fe50806a780 2023-10-06T03:57:06.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.483+0000 7fe507fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe5080bc750 0x7fe50814f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:06.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.483+0000 7fe506ffd700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe50806ab50 0x7fe50806c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44088/0 (socket says 172.21.15.157:44088) 2023-10-06T03:57:06.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.483+0000 7fe506ffd700 1 -- 172.21.15.157:0/3286941240 learned_addr learned my addr 172.21.15.157:0/3286941240 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:06.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.483+0000 7fe5077fe700 1 --2- 172.21.15.157:0/3286941240 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe50806a780 0x7fe508159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:06.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.484+0000 7fe507fff700 1 -- 172.21.15.157:0/3286941240 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe50806a780 msgr2=0x7fe508159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:06.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.484+0000 7fe507fff700 1 --2- 172.21.15.157:0/3286941240 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe50806a780 0x7fe508159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:06.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.484+0000 7fe507fff700 1 -- 172.21.15.157:0/3286941240 >> v1:172.21.15.157:6789/0 conn(0x7fe50806ab50 legacy=0x7fe50806c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:06.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.484+0000 7fe507fff700 1 -- 172.21.15.157:0/3286941240 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe50806ea70 con 0x7fe5080bc750 2023-10-06T03:57:06.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.484+0000 7fe507fff700 1 --2- 172.21.15.157:0/3286941240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5080bc750 0x7fe50814f850 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe500009910 tx=0x7fe5000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7167bba75d04b17d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:06.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.484+0000 7fe5067fc700 1 -- 172.21.15.157:0/3286941240 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50000b8b0 con 0x7fe5080bc750 2023-10-06T03:57:06.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.484+0000 7fe5067fc700 1 -- 172.21.15.157:0/3286941240 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50000ba10 con 0x7fe5080bc750 2023-10-06T03:57:06.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.484+0000 7fe5067fc700 1 -- 172.21.15.157:0/3286941240 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50000bdb0 con 0x7fe5080bc750 2023-10-06T03:57:06.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.484+0000 7fe50e244700 1 -- 172.21.15.157:0/3286941240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5080bc750 msgr2=0x7fe50814f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:06.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.484+0000 7fe50e244700 1 --2- 172.21.15.157:0/3286941240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5080bc750 0x7fe50814f850 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe500009910 tx=0x7fe5000091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:06.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.484+0000 7fe50e244700 1 -- 172.21.15.157:0/3286941240 shutdown_connections 2023-10-06T03:57:06.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.484+0000 7fe50e244700 1 --2- 172.21.15.157:0/3286941240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5080bc750 0x7fe50814f850 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:06.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.484+0000 7fe50e244700 1 --2- 172.21.15.157:0/3286941240 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe50806a780 0x7fe508159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:06.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.484+0000 7fe50e244700 1 -- 172.21.15.157:0/3286941240 >> 172.21.15.157:0/3286941240 conn(0x7fe50806b510 msgr2=0x7fe50806b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:06.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.485+0000 7fe50e244700 1 -- 172.21.15.157:0/3286941240 shutdown_connections 2023-10-06T03:57:06.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.485+0000 7fe50e244700 1 -- 172.21.15.157:0/3286941240 wait complete. 2023-10-06T03:57:06.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.486+0000 7fe50e244700 1 Processor -- start 2023-10-06T03:57:06.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.486+0000 7fe50e244700 1 -- start start 2023-10-06T03:57:06.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.486+0000 7fe50e244700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe50806a780 0x7fe508166900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:06.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.486+0000 7fe50e244700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe50806ab50 0x7fe50815bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:06.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.486+0000 7fe50e244700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5080bc750 0x7fe508153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:06.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.486+0000 7fe50e244700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe508166e40 con 0x7fe50806ab50 2023-10-06T03:57:06.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.486+0000 7fe50e244700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5080c8ba0 con 0x7fe5080bc750 2023-10-06T03:57:06.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.486+0000 7fe50e244700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe508155ad0 con 0x7fe50806a780 2023-10-06T03:57:06.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.487+0000 7fe506ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe50806ab50 0x7fe50815bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:06.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.487+0000 7fe506ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe50806ab50 0x7fe50815bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58502/0 (socket says 172.21.15.157:58502) 2023-10-06T03:57:06.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.487+0000 7fe506ffd700 1 -- 172.21.15.157:0/804626684 learned_addr learned my addr 172.21.15.157:0/804626684 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:06.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.487+0000 7fe507fff700 1 --2- 172.21.15.157:0/804626684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5080bc750 0x7fe508153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:06.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.487+0000 7fe5077fe700 1 --2- 172.21.15.157:0/804626684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe50806a780 0x7fe508166900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:06.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.487+0000 7fe506ffd700 1 -- 172.21.15.157:0/804626684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe50806a780 msgr2=0x7fe508166900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:06.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.487+0000 7fe506ffd700 1 --2- 172.21.15.157:0/804626684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe50806a780 0x7fe508166900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:06.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.487+0000 7fe506ffd700 1 -- 172.21.15.157:0/804626684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5080bc750 msgr2=0x7fe508153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:06.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.487+0000 7fe506ffd700 1 --2- 172.21.15.157:0/804626684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5080bc750 0x7fe508153af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:06.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.487+0000 7fe506ffd700 1 -- 172.21.15.157:0/804626684 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe500009420 con 0x7fe50806ab50 2023-10-06T03:57:06.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.487+0000 7fe507fff700 1 --2- 172.21.15.157:0/804626684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5080bc750 0x7fe508153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:06.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.487+0000 7fe506ffd700 1 --2- 172.21.15.157:0/804626684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe50806ab50 0x7fe50815bd50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00c7f0 tx=0x7fe4fc007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:06.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.487+0000 7fe4ef7fe700 1 -- 172.21.15.157:0/804626684 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc005b30 con 0x7fe50806ab50 2023-10-06T03:57:06.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.487+0000 7fe4ef7fe700 1 -- 172.21.15.157:0/804626684 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4fc005cb0 con 0x7fe50806ab50 2023-10-06T03:57:06.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.487+0000 7fe50e244700 1 -- 172.21.15.157:0/804626684 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5080c8df0 con 0x7fe50806ab50 2023-10-06T03:57:06.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.487+0000 7fe4ef7fe700 1 -- 172.21.15.157:0/804626684 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc004a80 con 0x7fe50806ab50 2023-10-06T03:57:06.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.487+0000 7fe50e244700 1 -- 172.21.15.157:0/804626684 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5080c93b0 con 0x7fe50806ab50 2023-10-06T03:57:06.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.488+0000 7fe50e244700 1 -- 172.21.15.157:0/804626684 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5080692b0 con 0x7fe50806ab50 2023-10-06T03:57:06.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.488+0000 7fe4ef7fe700 1 -- 172.21.15.157:0/804626684 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fe4fc0070f0 con 0x7fe50806ab50 2023-10-06T03:57:06.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.489+0000 7fe4ef7fe700 1 --2- 172.21.15.157:0/804626684 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe4f4066ea0 0x7fe4f4069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:06.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.489+0000 7fe4ef7fe700 1 -- 172.21.15.157:0/804626684 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fe4fc083630 con 0x7fe50806ab50 2023-10-06T03:57:06.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.489+0000 7fe5077fe700 1 --2- 172.21.15.157:0/804626684 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe4f4066ea0 0x7fe4f4069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:06.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.489+0000 7fe5077fe700 1 --2- 172.21.15.157:0/804626684 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe4f4066ea0 0x7fe4f4069350 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe4f80075a0 tx=0x7fe4f80097a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:06.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.493+0000 7fe4ef7fe700 1 -- 172.21.15.157:0/804626684 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe4fc050bf0 con 0x7fe50806ab50 2023-10-06T03:57:06.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.721+0000 7fe50e244700 1 -- 172.21.15.157:0/804626684 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe50804bb40 con 0x7fe50806ab50 2023-10-06T03:57:06.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.721+0000 7fe4ef7fe700 1 -- 172.21.15.157:0/804626684 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+5054 (secure 0 0 0) 0x7fe4fc018020 con 0x7fe50806ab50 2023-10-06T03:57:06.723 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:06.723 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4241,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/4177493830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4177493830},{"type":"v1","addr":"172.21.15.175:6839","nonce":4177493830}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-10-06T03:57:06.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.724+0000 7fe50e244700 1 -- 172.21.15.157:0/804626684 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe4f4066ea0 msgr2=0x7fe4f4069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:06.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.724+0000 7fe50e244700 1 --2- 172.21.15.157:0/804626684 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe4f4066ea0 0x7fe4f4069350 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe4f80075a0 tx=0x7fe4f80097a0 comp rx=0 tx=0).stop 2023-10-06T03:57:06.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.724+0000 7fe50e244700 1 -- 172.21.15.157:0/804626684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe50806ab50 msgr2=0x7fe50815bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:06.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.724+0000 7fe50e244700 1 --2- 172.21.15.157:0/804626684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe50806ab50 0x7fe50815bd50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00c7f0 tx=0x7fe4fc007f70 comp rx=0 tx=0).stop 2023-10-06T03:57:06.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.724+0000 7fe50e244700 1 -- 172.21.15.157:0/804626684 shutdown_connections 2023-10-06T03:57:06.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.724+0000 7fe50e244700 1 --2- 172.21.15.157:0/804626684 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe4f4066ea0 0x7fe4f4069350 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:06.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.724+0000 7fe50e244700 1 --2- 172.21.15.157:0/804626684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5080bc750 0x7fe508153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:06.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.724+0000 7fe50e244700 1 --2- 172.21.15.157:0/804626684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe50806a780 0x7fe508166900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:06.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.724+0000 7fe50e244700 1 --2- 172.21.15.157:0/804626684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe50806ab50 0x7fe50815bd50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:06.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.724+0000 7fe50e244700 1 -- 172.21.15.157:0/804626684 >> 172.21.15.157:0/804626684 conn(0x7fe50806b510 msgr2=0x7fe508158850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:06.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.724+0000 7fe50e244700 1 -- 172.21.15.157:0/804626684 shutdown_connections 2023-10-06T03:57:06.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.724+0000 7fe50e244700 1 -- 172.21.15.157:0/804626684 wait complete. 2023-10-06T03:57:06.727 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 3 2023-10-06T03:57:06.737 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:57:06.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.851+0000 7f1d78c5c700 1 Processor -- start 2023-10-06T03:57:06.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.851+0000 7f1d78c5c700 1 -- start start 2023-10-06T03:57:06.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.852+0000 7f1d78c5c700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d740c8790 0x7f1d74155b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:06.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.852+0000 7f1d78c5c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1d740cc050 0x7f1d740c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:06.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.852+0000 7f1d78c5c700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d740af480 con 0x7f1d740c8b60 2023-10-06T03:57:06.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.852+0000 7f1d78c5c700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d740af760 con 0x7f1d740cc050 2023-10-06T03:57:06.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.852+0000 7f1d78c5c700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d740afa40 con 0x7f1d740c8790 2023-10-06T03:57:06.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.852+0000 7f1d73fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1d740cc050 0x7f1d740c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:06.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.852+0000 7f1d73fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1d740cc050 0x7f1d740c75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58514/0 (socket says 172.21.15.157:58514) 2023-10-06T03:57:06.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.852+0000 7f1d73fff700 1 -- 172.21.15.157:0/2687232462 learned_addr learned my addr 172.21.15.157:0/2687232462 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:06.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.852+0000 7f1d737fe700 1 --2- 172.21.15.157:0/2687232462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d740c8790 0x7f1d74155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:06.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.852+0000 7f1d73fff700 1 -- 172.21.15.157:0/2687232462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d740c8790 msgr2=0x7f1d74155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:06.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.853+0000 7f1d73fff700 1 --2- 172.21.15.157:0/2687232462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d740c8790 0x7f1d74155b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:06.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.853+0000 7f1d73fff700 1 -- 172.21.15.157:0/2687232462 >> v1:172.21.15.175:6789/0 conn(0x7f1d740c8b60 legacy=0x7f1d740b1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:06.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.853+0000 7f1d73fff700 1 -- 172.21.15.157:0/2687232462 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d740afd20 con 0x7f1d740cc050 2023-10-06T03:57:06.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.853+0000 7f1d737fe700 1 --2- 172.21.15.157:0/2687232462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d740c8790 0x7f1d74155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:06.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.853+0000 7f1d73fff700 1 --2- 172.21.15.157:0/2687232462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d740cc050 0x7f1d740c75d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c009f20 tx=0x7f1d6c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95949be248bc5bf3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:06.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.853+0000 7f1d72ffd700 1 -- 172.21.15.157:0/2687232462 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c00e070 con 0x7f1d740cc050 2023-10-06T03:57:06.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.853+0000 7f1d72ffd700 1 -- 172.21.15.157:0/2687232462 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6c00ba80 con 0x7f1d740cc050 2023-10-06T03:57:06.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.853+0000 7f1d72ffd700 1 -- 172.21.15.157:0/2687232462 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c0092e0 con 0x7f1d740cc050 2023-10-06T03:57:06.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.853+0000 7f1d78c5c700 1 -- 172.21.15.157:0/2687232462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d740cc050 msgr2=0x7f1d740c75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:06.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.853+0000 7f1d78c5c700 1 --2- 172.21.15.157:0/2687232462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d740cc050 0x7f1d740c75d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c009f20 tx=0x7f1d6c009840 comp rx=0 tx=0).stop 2023-10-06T03:57:06.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.853+0000 7f1d78c5c700 1 -- 172.21.15.157:0/2687232462 shutdown_connections 2023-10-06T03:57:06.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.853+0000 7f1d78c5c700 1 --2- 172.21.15.157:0/2687232462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d740cc050 0x7f1d740c75d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:06.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.853+0000 7f1d78c5c700 1 --2- 172.21.15.157:0/2687232462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d740c8790 0x7f1d74155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:06.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.853+0000 7f1d78c5c700 1 -- 172.21.15.157:0/2687232462 >> 172.21.15.157:0/2687232462 conn(0x7f1d740cc540 msgr2=0x7f1d740cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:06.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.853+0000 7f1d78c5c700 1 -- 172.21.15.157:0/2687232462 shutdown_connections 2023-10-06T03:57:06.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.853+0000 7f1d78c5c700 1 -- 172.21.15.157:0/2687232462 wait complete. 2023-10-06T03:57:06.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.854+0000 7f1d78c5c700 1 Processor -- start 2023-10-06T03:57:06.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.855+0000 7f1d78c5c700 1 -- start start 2023-10-06T03:57:06.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.855+0000 7f1d78c5c700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d740c8790 0x7f1d74169400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:06.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.855+0000 7f1d78c5c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d740c8b60 0x7f1d74158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:06.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.855+0000 7f1d78c5c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d740cc050 0x7f1d7414e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:06.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.855+0000 7f1d78c5c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7416a060 con 0x7f1d740c8b60 2023-10-06T03:57:06.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.855+0000 7f1d78c5c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d74150a20 con 0x7f1d740cc050 2023-10-06T03:57:06.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.855+0000 7f1d78c5c700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d741527c0 con 0x7f1d740c8790 2023-10-06T03:57:06.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.855+0000 7f1d6bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d740c8b60 0x7f1d74158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:06.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.855+0000 7f1d6bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d740c8b60 0x7f1d74158210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58530/0 (socket says 172.21.15.157:58530) 2023-10-06T03:57:06.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.855+0000 7f1d6bfff700 1 -- 172.21.15.157:0/4237468705 learned_addr learned my addr 172.21.15.157:0/4237468705 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:06.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.856+0000 7f1d73fff700 1 --2- 172.21.15.157:0/4237468705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d740cc050 0x7f1d7414e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:06.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.856+0000 7f1d737fe700 1 --2- 172.21.15.157:0/4237468705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d740c8790 0x7f1d74169400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:06.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.856+0000 7f1d6bfff700 1 -- 172.21.15.157:0/4237468705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d740c8790 msgr2=0x7f1d74169400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:06.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.856+0000 7f1d6bfff700 1 --2- 172.21.15.157:0/4237468705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d740c8790 0x7f1d74169400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:06.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.856+0000 7f1d6bfff700 1 -- 172.21.15.157:0/4237468705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d740cc050 msgr2=0x7f1d7414e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:06.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.856+0000 7f1d6bfff700 1 --2- 172.21.15.157:0/4237468705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d740cc050 0x7f1d7414e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:06.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.856+0000 7f1d6bfff700 1 -- 172.21.15.157:0/4237468705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d6c009ae0 con 0x7f1d740c8b60 2023-10-06T03:57:06.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.856+0000 7f1d6bfff700 1 --2- 172.21.15.157:0/4237468705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d740c8b60 0x7f1d74158210 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00c590 tx=0x7f1d5c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:06.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.856+0000 7f1d6b7fe700 1 -- 172.21.15.157:0/4237468705 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c011590 con 0x7f1d740c8b60 2023-10-06T03:57:06.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.856+0000 7f1d6b7fe700 1 -- 172.21.15.157:0/4237468705 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d5c0116f0 con 0x7f1d740c8b60 2023-10-06T03:57:06.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.856+0000 7f1d78c5c700 1 -- 172.21.15.157:0/4237468705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d74169d80 con 0x7f1d740c8b60 2023-10-06T03:57:06.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.856+0000 7f1d6b7fe700 1 -- 172.21.15.157:0/4237468705 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c011ab0 con 0x7f1d740c8b60 2023-10-06T03:57:06.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.856+0000 7f1d78c5c700 1 -- 172.21.15.157:0/4237468705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d74152170 con 0x7f1d740c8b60 2023-10-06T03:57:06.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.857+0000 7f1d78c5c700 1 -- 172.21.15.157:0/4237468705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d740b1140 con 0x7f1d740c8b60 2023-10-06T03:57:06.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.857+0000 7f1d6b7fe700 1 -- 172.21.15.157:0/4237468705 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f1d5c01b3b0 con 0x7f1d740c8b60 2023-10-06T03:57:06.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.858+0000 7f1d6b7fe700 1 --2- 172.21.15.157:0/4237468705 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1d54066ea0 0x7f1d54069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:06.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.858+0000 7f1d6b7fe700 1 -- 172.21.15.157:0/4237468705 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f1d5c0837b0 con 0x7f1d740c8b60 2023-10-06T03:57:06.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.858+0000 7f1d737fe700 1 --2- 172.21.15.157:0/4237468705 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1d54066ea0 0x7f1d54069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:06.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.858+0000 7f1d737fe700 1 --2- 172.21.15.157:0/4237468705 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1d54066ea0 0x7f1d54069350 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1d64006f00 tx=0x7f1d64008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:06.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:06.862+0000 7f1d6b7fe700 1 -- 172.21.15.157:0/4237468705 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1d5c050cf0 con 0x7f1d740c8b60 2023-10-06T03:57:07.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.044+0000 7f1d78c5c700 1 -- 172.21.15.157:0/4237468705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d7404bb40 con 0x7f1d740c8b60 2023-10-06T03:57:07.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.045+0000 7f1d6b7fe700 1 -- 172.21.15.157:0/4237468705 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+13695 (secure 0 0 0) 0x7f1d5c017020 con 0x7f1d740c8b60 2023-10-06T03:57:07.046 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:07.046 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":17,"fsid":"28664a7a-fe30-4efe-be5a-6abf9318b700","created":"2023-10-06T03:56:51.887538+0000","modified":"2023-10-06T03:57:06.309108+0000","last_up_change":"2023-10-06T03:57:02.299440+0000","last_in_change":"2023-10-06T03:56:55.141505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:57:03.455121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:57:03.905661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"3f677cac-e9f4-4608-acac-718b072ce984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":90640},{"type":"v1","addr":"172.21.15.157:6809","nonce":90640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":90640},{"type":"v1","addr":"172.21.15.157:6811","nonce":90640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":90640},{"type":"v1","addr":"172.21.15.157:6815","nonce":90640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":90640},{"type":"v1","addr":"172.21.15.157:6813","nonce":90640}]},"public_addr":"172.21.15.157:6809/90640","cluster_addr":"172.21.15.157:6811/90640","heartbeat_back_addr":"172.21.15.157:6815/90640","heartbeat_front_addr":"172.21.15.157:6813/90640","state":["exists","up"]},{"osd":1,"uuid":"7d771892-bb91-4e9b-9610-051861c11247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":90654},{"type":"v1","addr":"172.21.15.157:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":90654},{"type":"v1","addr":"172.21.15.157:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":90654},{"type":"v1","addr":"172.21.15.157:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":90654},{"type":"v1","addr":"172.21.15.157:6805","nonce":90654}]},"public_addr":"172.21.15.157:6801/90654","cluster_addr":"172.21.15.157:6803/90654","heartbeat_back_addr":"172.21.15.157:6807/90654","heartbeat_front_addr":"172.21.15.157:6805/90654","state":["exists","up"]},{"osd":2,"uuid":"5c1f7d4e-d185-47cf-ab6c-453b4e3009c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":90657},{"type":"v1","addr":"172.21.15.157:6825","nonce":90657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":90657},{"type":"v1","addr":"172.21.15.157:6827","nonce":90657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":90657},{"type":"v1","addr":"172.21.15.157:6831","nonce":90657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":90657},{"type":"v1","addr":"172.21.15.157:6829","nonce":90657}]},"public_addr":"172.21.15.157:6825/90657","cluster_addr":"172.21.15.157:6827/90657","heartbeat_back_addr":"172.21.15.157:6831/90657","heartbeat_front_addr":"172.21.15.157:6829/90657","state":["exists","up"]},{"osd":3,"uuid":"f6a1e825-43c7-4afd-8daf-e51d45afc75e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":90641},{"type":"v1","addr":"172.21.15.157:6817","nonce":90641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":90641},{"type":"v1","addr":"172.21.15.157:6819","nonce":90641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":90641},{"type":"v1","addr":"172.21.15.157:6823","nonce":90641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":90641},{"type":"v1","addr":"172.21.15.157:6821","nonce":90641}]},"public_addr":"172.21.15.157:6817/90641","cluster_addr":"172.21.15.157:6819/90641","heartbeat_back_addr":"172.21.15.157:6823/90641","heartbeat_front_addr":"172.21.15.157:6821/90641","state":["exists","up"]},{"osd":4,"uuid":"8ad6e3d8-9a31-4f85-9410-417b7ac5da97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":91261},{"type":"v1","addr":"172.21.15.175:6803","nonce":91261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":91261},{"type":"v1","addr":"172.21.15.175:6806","nonce":91261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":91261},{"type":"v1","addr":"172.21.15.175:6814","nonce":91261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":91261},{"type":"v1","addr":"172.21.15.175:6810","nonce":91261}]},"public_addr":"172.21.15.175:6803/91261","cluster_addr":"172.21.15.175:6806/91261","heartbeat_back_addr":"172.21.15.175:6814/91261","heartbeat_front_addr":"172.21.15.175:6810/91261","state":["exists","up"]},{"osd":5,"uuid":"f68aca2a-f182-4c00-8e21-241d09c02ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":91262},{"type":"v1","addr":"172.21.15.175:6817","nonce":91262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":91262},{"type":"v1","addr":"172.21.15.175:6819","nonce":91262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":91262},{"type":"v1","addr":"172.21.15.175:6823","nonce":91262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":91262},{"type":"v1","addr":"172.21.15.175:6821","nonce":91262}]},"public_addr":"172.21.15.175:6817/91262","cluster_addr":"172.21.15.175:6819/91262","heartbeat_back_addr":"172.21.15.175:6823/91262","heartbeat_front_addr":"172.21.15.175:6821/91262","state":["exists","up"]},{"osd":6,"uuid":"d867cd89-3430-40ca-862f-d2bea3de9d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":91263},{"type":"v1","addr":"172.21.15.175:6801","nonce":91263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6805","nonce":91263},{"type":"v1","addr":"172.21.15.175:6807","nonce":91263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6813","nonce":91263},{"type":"v1","addr":"172.21.15.175:6815","nonce":91263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":91263},{"type":"v1","addr":"172.21.15.175:6811","nonce":91263}]},"public_addr":"172.21.15.175:6801/91263","cluster_addr":"172.21.15.175:6807/91263","heartbeat_back_addr":"172.21.15.175:6815/91263","heartbeat_front_addr":"172.21.15.175:6811/91263","state":["exists","up"]},{"osd":7,"uuid":"930cd464-79f1-47cb-b4ff-188ccde74fff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":91306},{"type":"v1","addr":"172.21.15.175:6825","nonce":91306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":91306},{"type":"v1","addr":"172.21.15.175:6827","nonce":91306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":91306},{"type":"v1","addr":"172.21.15.175:6831","nonce":91306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":91306},{"type":"v1","addr":"172.21.15.175:6829","nonce":91306}]},"public_addr":"172.21.15.175:6825/91306","cluster_addr":"172.21.15.175:6827/91306","heartbeat_back_addr":"172.21.15.175:6831/91306","heartbeat_front_addr":"172.21.15.175:6829/91306","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.290673+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.056390+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.292903+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.275861+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.332095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.327453+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.317585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.309330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:57:07.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.047+0000 7f1d78c5c700 1 -- 172.21.15.157:0/4237468705 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1d54066ea0 msgr2=0x7f1d54069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:07.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.047+0000 7f1d78c5c700 1 --2- 172.21.15.157:0/4237468705 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1d54066ea0 0x7f1d54069350 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1d64006f00 tx=0x7f1d64008040 comp rx=0 tx=0).stop 2023-10-06T03:57:07.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.047+0000 7f1d78c5c700 1 -- 172.21.15.157:0/4237468705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d740c8b60 msgr2=0x7f1d74158210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:07.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.047+0000 7f1d78c5c700 1 --2- 172.21.15.157:0/4237468705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d740c8b60 0x7f1d74158210 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00c590 tx=0x7f1d5c00c950 comp rx=0 tx=0).stop 2023-10-06T03:57:07.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.047+0000 7f1d78c5c700 1 -- 172.21.15.157:0/4237468705 shutdown_connections 2023-10-06T03:57:07.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.047+0000 7f1d78c5c700 1 --2- 172.21.15.157:0/4237468705 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1d54066ea0 0x7f1d54069350 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:07.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.047+0000 7f1d78c5c700 1 --2- 172.21.15.157:0/4237468705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d740cc050 0x7f1d7414e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:07.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.047+0000 7f1d78c5c700 1 --2- 172.21.15.157:0/4237468705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d740c8790 0x7f1d74169400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:07.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.047+0000 7f1d78c5c700 1 --2- 172.21.15.157:0/4237468705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d740c8b60 0x7f1d74158210 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:07.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.047+0000 7f1d78c5c700 1 -- 172.21.15.157:0/4237468705 >> 172.21.15.157:0/4237468705 conn(0x7f1d740cc540 msgr2=0x7f1d7406d920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:07.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.047+0000 7f1d78c5c700 1 -- 172.21.15.157:0/4237468705 shutdown_connections 2023-10-06T03:57:07.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.047+0000 7f1d78c5c700 1 -- 172.21.15.157:0/4237468705 wait complete. 2023-10-06T03:57:07.060 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:57:03.455121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:57:03.905661+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T03:57:07.061 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:57:07.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.176+0000 7f6f2d654700 1 Processor -- start 2023-10-06T03:57:07.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.176+0000 7f6f2d654700 1 -- start start 2023-10-06T03:57:07.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.177+0000 7f6f2d654700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6f28150410 0x7f6f28155f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:07.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.177+0000 7f6f2d654700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f280c9780 con 0x7f6f2806dc40 2023-10-06T03:57:07.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.177+0000 7f6f2d654700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f2806e0c0 con 0x7f6f2806cbb0 2023-10-06T03:57:07.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.177+0000 7f6f2d654700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f280c8c40 con 0x7f6f28150410 2023-10-06T03:57:07.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.177+0000 7f6f277fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6f28150410 0x7f6f28155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:07.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.177+0000 7f6f1ffff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f6f2806dc40 0x7f6f2814fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44098/0 (socket says 172.21.15.157:44098) 2023-10-06T03:57:07.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.177+0000 7f6f1ffff700 1 -- 172.21.15.157:0/1214641145 learned_addr learned my addr 172.21.15.157:0/1214641145 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:07.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.177+0000 7f6f277fe700 1 -- 172.21.15.157:0/1214641145 >> v1:172.21.15.175:6789/0 conn(0x7f6f2806cbb0 legacy=0x7f6f2806d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:07.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.177+0000 7f6f277fe700 1 -- 172.21.15.157:0/1214641145 >> v1:172.21.15.157:6789/0 conn(0x7f6f2806dc40 legacy=0x7f6f2814fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:07.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.178+0000 7f6f277fe700 1 -- 172.21.15.157:0/1214641145 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f280c8e50 con 0x7f6f28150410 2023-10-06T03:57:07.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.177+0000 7f6f267fc700 1 -- 172.21.15.157:0/1214641145 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3660427079 0 0) 0x7f6f280c9780 con 0x7f6f2806dc40 2023-10-06T03:57:07.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.178+0000 7f6f277fe700 1 --2- 172.21.15.157:0/1214641145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f28150410 0x7f6f28155f70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009e60 tx=0x7f6f20009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e82867ebcc52fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:07.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.178+0000 7f6f267fc700 1 -- 172.21.15.157:0/1214641145 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2000e070 con 0x7f6f28150410 2023-10-06T03:57:07.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.178+0000 7f6f267fc700 1 -- 172.21.15.157:0/1214641145 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2000bc80 con 0x7f6f28150410 2023-10-06T03:57:07.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.178+0000 7f6f267fc700 1 -- 172.21.15.157:0/1214641145 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f20012430 con 0x7f6f28150410 2023-10-06T03:57:07.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.178+0000 7f6f2d654700 1 -- 172.21.15.157:0/1214641145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f28150410 msgr2=0x7f6f28155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:07.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.178+0000 7f6f2d654700 1 --2- 172.21.15.157:0/1214641145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f28150410 0x7f6f28155f70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009e60 tx=0x7f6f20009780 comp rx=0 tx=0).stop 2023-10-06T03:57:07.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.178+0000 7f6f2d654700 1 -- 172.21.15.157:0/1214641145 shutdown_connections 2023-10-06T03:57:07.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.178+0000 7f6f2d654700 1 --2- 172.21.15.157:0/1214641145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f28150410 0x7f6f28155f70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:07.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.178+0000 7f6f2d654700 1 -- 172.21.15.157:0/1214641145 >> 172.21.15.157:0/1214641145 conn(0x7f6f2806f230 msgr2=0x7f6f2806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:07.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.178+0000 7f6f2d654700 1 -- 172.21.15.157:0/1214641145 shutdown_connections 2023-10-06T03:57:07.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.178+0000 7f6f2d654700 1 -- 172.21.15.157:0/1214641145 wait complete. 2023-10-06T03:57:07.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.179+0000 7f6f2d654700 1 Processor -- start 2023-10-06T03:57:07.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.180+0000 7f6f2d654700 1 -- start start 2023-10-06T03:57:07.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.180+0000 7f6f2d654700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f2806cbb0 0x7f6f28167570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:07.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.180+0000 7f6f2d654700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f28150410 0x7f6f28167ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:07.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.180+0000 7f6f2d654700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f28148930 0x7f6f28148db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:07.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.180+0000 7f6f2d654700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f280c4350 con 0x7f6f28150410 2023-10-06T03:57:07.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.180+0000 7f6f2d654700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f280c5430 con 0x7f6f2806cbb0 2023-10-06T03:57:07.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.180+0000 7f6f2d654700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f28158120 con 0x7f6f28148930 2023-10-06T03:57:07.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.180+0000 7f6f1ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f28150410 0x7f6f28167ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:07.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.180+0000 7f6f1ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f28150410 0x7f6f28167ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58558/0 (socket says 172.21.15.157:58558) 2023-10-06T03:57:07.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.180+0000 7f6f1ffff700 1 -- 172.21.15.157:0/446372577 learned_addr learned my addr 172.21.15.157:0/446372577 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:07.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.181+0000 7f6f26ffd700 1 --2- 172.21.15.157:0/446372577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f2806cbb0 0x7f6f28167570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:07.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.181+0000 7f6f277fe700 1 --2- 172.21.15.157:0/446372577 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f28148930 0x7f6f28148db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:07.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.181+0000 7f6f1ffff700 1 -- 172.21.15.157:0/446372577 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f28148930 msgr2=0x7f6f28148db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:07.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.181+0000 7f6f1ffff700 1 --2- 172.21.15.157:0/446372577 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f28148930 0x7f6f28148db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:07.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.181+0000 7f6f1ffff700 1 -- 172.21.15.157:0/446372577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f2806cbb0 msgr2=0x7f6f28167570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:07.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.181+0000 7f6f1ffff700 1 --2- 172.21.15.157:0/446372577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f2806cbb0 0x7f6f28167570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:07.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.181+0000 7f6f1ffff700 1 -- 172.21.15.157:0/446372577 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f20009a20 con 0x7f6f28150410 2023-10-06T03:57:07.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.181+0000 7f6f277fe700 1 --2- 172.21.15.157:0/446372577 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f28148930 0x7f6f28148db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:57:07.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.181+0000 7f6f26ffd700 1 --2- 172.21.15.157:0/446372577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f2806cbb0 0x7f6f28167570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:57:07.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.181+0000 7f6f1ffff700 1 --2- 172.21.15.157:0/446372577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f28150410 0x7f6f28167ab0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6f1000c810 tx=0x7f6f10007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:07.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.181+0000 7f6f1effd700 1 -- 172.21.15.157:0/446372577 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f10005b30 con 0x7f6f28150410 2023-10-06T03:57:07.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.181+0000 7f6f1effd700 1 -- 172.21.15.157:0/446372577 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f10005c90 con 0x7f6f28150410 2023-10-06T03:57:07.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.181+0000 7f6f2d654700 1 -- 172.21.15.157:0/446372577 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f280c6410 con 0x7f6f28150410 2023-10-06T03:57:07.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.181+0000 7f6f1effd700 1 -- 172.21.15.157:0/446372577 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f10007300 con 0x7f6f28150410 2023-10-06T03:57:07.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.181+0000 7f6f2d654700 1 -- 172.21.15.157:0/446372577 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f280c4820 con 0x7f6f28150410 2023-10-06T03:57:07.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.182+0000 7f6f2d654700 1 -- 172.21.15.157:0/446372577 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f28089ca0 con 0x7f6f28150410 2023-10-06T03:57:07.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.183+0000 7f6f1effd700 1 -- 172.21.15.157:0/446372577 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f6f1001b400 con 0x7f6f28150410 2023-10-06T03:57:07.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.183+0000 7f6f1effd700 1 --2- 172.21.15.157:0/446372577 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6f08066ed0 0x7f6f08069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:07.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.183+0000 7f6f1effd700 1 -- 172.21.15.157:0/446372577 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f6f10083df0 con 0x7f6f28150410 2023-10-06T03:57:07.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.183+0000 7f6f26ffd700 1 --2- 172.21.15.157:0/446372577 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6f08066ed0 0x7f6f08069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:07.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.184+0000 7f6f26ffd700 1 --2- 172.21.15.157:0/446372577 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6f08066ed0 0x7f6f08069380 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6f18002880 tx=0x7f6f18009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:07.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.187+0000 7f6f1effd700 1 -- 172.21.15.157:0/446372577 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f100513b0 con 0x7f6f28150410 2023-10-06T03:57:07.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.370+0000 7f6f2d654700 1 -- 172.21.15.157:0/446372577 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6f280c5570 con 0x7f6f28150410 2023-10-06T03:57:07.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.371+0000 7f6f1effd700 1 -- 172.21.15.157:0/446372577 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6f100549d0 con 0x7f6f28150410 2023-10-06T03:57:07.372 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-10-06T03:57:07.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.373+0000 7f6f2d654700 1 -- 172.21.15.157:0/446372577 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6f08066ed0 msgr2=0x7f6f08069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:07.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.373+0000 7f6f2d654700 1 --2- 172.21.15.157:0/446372577 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6f08066ed0 0x7f6f08069380 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6f18002880 tx=0x7f6f18009040 comp rx=0 tx=0).stop 2023-10-06T03:57:07.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.373+0000 7f6f2d654700 1 -- 172.21.15.157:0/446372577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f28150410 msgr2=0x7f6f28167ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:07.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.373+0000 7f6f2d654700 1 --2- 172.21.15.157:0/446372577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f28150410 0x7f6f28167ab0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6f1000c810 tx=0x7f6f10007df0 comp rx=0 tx=0).stop 2023-10-06T03:57:07.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.373+0000 7f6f2d654700 1 -- 172.21.15.157:0/446372577 shutdown_connections 2023-10-06T03:57:07.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.373+0000 7f6f2d654700 1 --2- 172.21.15.157:0/446372577 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6f08066ed0 0x7f6f08069380 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:07.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.373+0000 7f6f2d654700 1 --2- 172.21.15.157:0/446372577 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f2806cbb0 0x7f6f28167570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:07.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.373+0000 7f6f2d654700 1 --2- 172.21.15.157:0/446372577 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f28148930 0x7f6f28148db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:07.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.373+0000 7f6f2d654700 1 --2- 172.21.15.157:0/446372577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f28150410 0x7f6f28167ab0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:07.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.373+0000 7f6f2d654700 1 -- 172.21.15.157:0/446372577 >> 172.21.15.157:0/446372577 conn(0x7f6f2806f230 msgr2=0x7f6f2814ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:07.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.373+0000 7f6f2d654700 1 -- 172.21.15.157:0/446372577 shutdown_connections 2023-10-06T03:57:07.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.373+0000 7f6f2d654700 1 -- 172.21.15.157:0/446372577 wait complete. 2023-10-06T03:57:07.387 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:57:07.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.509+0000 7fc15bc4d700 1 Processor -- start 2023-10-06T03:57:07.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.509+0000 7fc15bc4d700 1 -- start start 2023-10-06T03:57:07.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.510+0000 7fc15bc4d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc1540c9460 0x7fc1541501c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:07.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.510+0000 7fc15bc4d700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc15406b710 con 0x7fc154157b20 2023-10-06T03:57:07.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.510+0000 7fc15bc4d700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc154156030 con 0x7fc15414fdf0 2023-10-06T03:57:07.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.510+0000 7fc15bc4d700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc154069ab0 con 0x7fc1540c9460 2023-10-06T03:57:07.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.510+0000 7fc1591e8700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc1540c9460 0x7fc1541501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:07.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.510+0000 7fc15a1ea700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc154157b20 0x7fc1540c8680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44100/0 (socket says 172.21.15.157:44100) 2023-10-06T03:57:07.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.510+0000 7fc1591e8700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc1540c9460 0x7fc1541501c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58564/0 (socket says 172.21.15.157:58564) 2023-10-06T03:57:07.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.510+0000 7fc15a1ea700 1 -- 172.21.15.157:0/604260300 learned_addr learned my addr 172.21.15.157:0/604260300 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:07.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.510+0000 7fc1591e8700 1 -- 172.21.15.157:0/604260300 >> v1:172.21.15.175:6789/0 conn(0x7fc15414fdf0 legacy=0x7fc154157410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:07.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.510+0000 7fc1591e8700 1 -- 172.21.15.157:0/604260300 >> v1:172.21.15.157:6789/0 conn(0x7fc154157b20 legacy=0x7fc1540c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:07.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.511+0000 7fc1591e8700 1 -- 172.21.15.157:0/604260300 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc154069d90 con 0x7fc1540c9460 2023-10-06T03:57:07.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.511+0000 7fc1589e7700 1 -- 172.21.15.157:0/604260300 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2368361056 0 0) 0x7fc15406b710 con 0x7fc154157b20 2023-10-06T03:57:07.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.511+0000 7fc1591e8700 1 --2- 172.21.15.157:0/604260300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1540c9460 0x7fc1541501c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc1440098e0 tx=0x7fc1440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b4071ba5afe805 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:07.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.511+0000 7fc1589e7700 1 -- 172.21.15.157:0/604260300 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc144009e80 con 0x7fc1540c9460 2023-10-06T03:57:07.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.511+0000 7fc1589e7700 1 -- 172.21.15.157:0/604260300 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14400b860 con 0x7fc1540c9460 2023-10-06T03:57:07.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.511+0000 7fc1589e7700 1 -- 172.21.15.157:0/604260300 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14400bbe0 con 0x7fc1540c9460 2023-10-06T03:57:07.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.511+0000 7fc15bc4d700 1 -- 172.21.15.157:0/604260300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1540c9460 msgr2=0x7fc1541501c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:07.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.511+0000 7fc15bc4d700 1 --2- 172.21.15.157:0/604260300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1540c9460 0x7fc1541501c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc1440098e0 tx=0x7fc1440091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:07.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.511+0000 7fc15bc4d700 1 -- 172.21.15.157:0/604260300 shutdown_connections 2023-10-06T03:57:07.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.511+0000 7fc15bc4d700 1 --2- 172.21.15.157:0/604260300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1540c9460 0x7fc1541501c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:07.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.511+0000 7fc15bc4d700 1 -- 172.21.15.157:0/604260300 >> 172.21.15.157:0/604260300 conn(0x7fc154067530 msgr2=0x7fc154067930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:07.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.511+0000 7fc15bc4d700 1 -- 172.21.15.157:0/604260300 shutdown_connections 2023-10-06T03:57:07.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.511+0000 7fc15bc4d700 1 -- 172.21.15.157:0/604260300 wait complete. 2023-10-06T03:57:07.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.512+0000 7fc15bc4d700 1 Processor -- start 2023-10-06T03:57:07.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.513+0000 7fc15bc4d700 1 -- start start 2023-10-06T03:57:07.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.513+0000 7fc15bc4d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1540c9460 0x7fc154159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:07.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.513+0000 7fc15bc4d700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc15414fdf0 0x7fc15414ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:07.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.513+0000 7fc15bc4d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc154157b20 0x7fc15414a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:07.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.513+0000 7fc15bc4d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc154159a90 con 0x7fc1540c9460 2023-10-06T03:57:07.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.513+0000 7fc15bc4d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15414d8b0 con 0x7fc154157b20 2023-10-06T03:57:07.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.513+0000 7fc15bc4d700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc15414db90 con 0x7fc15414fdf0 2023-10-06T03:57:07.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.513+0000 7fc1599e9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1540c9460 0x7fc154159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:07.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.513+0000 7fc1599e9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1540c9460 0x7fc154159f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58576/0 (socket says 172.21.15.157:58576) 2023-10-06T03:57:07.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.513+0000 7fc1599e9700 1 -- 172.21.15.157:0/1869855973 learned_addr learned my addr 172.21.15.157:0/1869855973 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:07.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.513+0000 7fc1591e8700 1 --2- 172.21.15.157:0/1869855973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc15414fdf0 0x7fc15414ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:07.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.513+0000 7fc15a1ea700 1 --2- 172.21.15.157:0/1869855973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc154157b20 0x7fc15414a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:07.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.514+0000 7fc1599e9700 1 -- 172.21.15.157:0/1869855973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc15414fdf0 msgr2=0x7fc15414ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:07.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.514+0000 7fc1599e9700 1 --2- 172.21.15.157:0/1869855973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc15414fdf0 0x7fc15414ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:07.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.514+0000 7fc1599e9700 1 -- 172.21.15.157:0/1869855973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc154157b20 msgr2=0x7fc15414a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:07.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.514+0000 7fc1599e9700 1 --2- 172.21.15.157:0/1869855973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc154157b20 0x7fc15414a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:07.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.514+0000 7fc1599e9700 1 -- 172.21.15.157:0/1869855973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc144009420 con 0x7fc1540c9460 2023-10-06T03:57:07.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.514+0000 7fc1599e9700 1 --2- 172.21.15.157:0/1869855973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1540c9460 0x7fc154159f20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc15000c590 tx=0x7fc15000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:07.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.514+0000 7fc149ffb700 1 -- 172.21.15.157:0/1869855973 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc150011590 con 0x7fc1540c9460 2023-10-06T03:57:07.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.514+0000 7fc149ffb700 1 -- 172.21.15.157:0/1869855973 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc150011710 con 0x7fc1540c9460 2023-10-06T03:57:07.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.514+0000 7fc15bc4d700 1 -- 172.21.15.157:0/1869855973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc15414de70 con 0x7fc1540c9460 2023-10-06T03:57:07.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.514+0000 7fc149ffb700 1 -- 172.21.15.157:0/1869855973 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc150011ad0 con 0x7fc1540c9460 2023-10-06T03:57:07.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.514+0000 7fc15bc4d700 1 -- 172.21.15.157:0/1869855973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc154159720 con 0x7fc1540c9460 2023-10-06T03:57:07.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.515+0000 7fc15bc4d700 1 -- 172.21.15.157:0/1869855973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc15406d390 con 0x7fc1540c9460 2023-10-06T03:57:07.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.515+0000 7fc149ffb700 1 -- 172.21.15.157:0/1869855973 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fc15001b3b0 con 0x7fc1540c9460 2023-10-06T03:57:07.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.516+0000 7fc149ffb700 1 --2- 172.21.15.157:0/1869855973 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc138066ea0 0x7fc138069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:07.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.516+0000 7fc149ffb700 1 -- 172.21.15.157:0/1869855973 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fc150083780 con 0x7fc1540c9460 2023-10-06T03:57:07.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.516+0000 7fc1591e8700 1 --2- 172.21.15.157:0/1869855973 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc138066ea0 0x7fc138069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:07.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.516+0000 7fc1591e8700 1 --2- 172.21.15.157:0/1869855973 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc138066ea0 0x7fc138069350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc1440093f0 tx=0x7fc144017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:07.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.520+0000 7fc149ffb700 1 -- 172.21.15.157:0/1869855973 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc150050cc0 con 0x7fc1540c9460 2023-10-06T03:57:07.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.704+0000 7fc15bc4d700 1 -- 172.21.15.157:0/1869855973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc154159d70 con 0x7fc1540c9460 2023-10-06T03:57:07.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.704+0000 7fc149ffb700 1 -- 172.21.15.157:0/1869855973 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc1500170a0 con 0x7fc1540c9460 2023-10-06T03:57:07.706 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-10-06T03:57:07.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.706+0000 7fc15bc4d700 1 -- 172.21.15.157:0/1869855973 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc138066ea0 msgr2=0x7fc138069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:07.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.706+0000 7fc15bc4d700 1 --2- 172.21.15.157:0/1869855973 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc138066ea0 0x7fc138069350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc1440093f0 tx=0x7fc144017000 comp rx=0 tx=0).stop 2023-10-06T03:57:07.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.706+0000 7fc15bc4d700 1 -- 172.21.15.157:0/1869855973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1540c9460 msgr2=0x7fc154159f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:07.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.706+0000 7fc15bc4d700 1 --2- 172.21.15.157:0/1869855973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1540c9460 0x7fc154159f20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc15000c590 tx=0x7fc15000c8a0 comp rx=0 tx=0).stop 2023-10-06T03:57:07.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.706+0000 7fc15bc4d700 1 -- 172.21.15.157:0/1869855973 shutdown_connections 2023-10-06T03:57:07.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.706+0000 7fc15bc4d700 1 --2- 172.21.15.157:0/1869855973 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc138066ea0 0x7fc138069350 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:07.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.706+0000 7fc15bc4d700 1 --2- 172.21.15.157:0/1869855973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc154157b20 0x7fc15414a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:07.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.706+0000 7fc15bc4d700 1 --2- 172.21.15.157:0/1869855973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1540c9460 0x7fc154159f20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:07.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.706+0000 7fc15bc4d700 1 --2- 172.21.15.157:0/1869855973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc15414fdf0 0x7fc15414ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:07.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.706+0000 7fc15bc4d700 1 -- 172.21.15.157:0/1869855973 >> 172.21.15.157:0/1869855973 conn(0x7fc154067530 msgr2=0x7fc15415d450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:07.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.706+0000 7fc15bc4d700 1 -- 172.21.15.157:0/1869855973 shutdown_connections 2023-10-06T03:57:07.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.706+0000 7fc15bc4d700 1 -- 172.21.15.157:0/1869855973 wait complete. 2023-10-06T03:57:07.720 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T03:57:07.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.832+0000 7f2fe6e71700 1 Processor -- start 2023-10-06T03:57:07.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.832+0000 7f2fe6e71700 1 -- start start 2023-10-06T03:57:07.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.833+0000 7f2fe6e71700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2fe014dcd0 0x7f2fe0153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:07.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.833+0000 7f2fe6e71700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fe014e0a0 0x7f2fe00c18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:07.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.833+0000 7f2fe6e71700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe00c5770 con 0x7f2fe00c5c00 2023-10-06T03:57:07.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.833+0000 7f2fe6e71700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe0154c80 con 0x7f2fe014dcd0 2023-10-06T03:57:07.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.833+0000 7f2fe6e71700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe0154200 con 0x7f2fe014e0a0 2023-10-06T03:57:07.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.833+0000 7f2fd7fff700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fe014e0a0 0x7f2fe00c18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:07.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.833+0000 7f2fe4c0d700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2fe014dcd0 0x7f2fe0153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:07.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.833+0000 7f2fe4c0d700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2fe014dcd0 0x7f2fe0153cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:50252/0 (socket says 172.21.15.157:50252) 2023-10-06T03:57:07.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.833+0000 7f2fe540e700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f2fe00c5c00 0x7f2fe00ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.157:46698/0 (socket says 172.21.15.157:46698) 2023-10-06T03:57:07.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.833+0000 7f2fd7fff700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fe014e0a0 0x7f2fe00c18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:57570/0 (socket says 172.21.15.157:57570) 2023-10-06T03:57:07.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.833+0000 7f2fe4c0d700 1 -- 172.21.15.157:0/2124250932 learned_addr learned my addr 172.21.15.157:0/2124250932 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:07.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.834+0000 7f2fd7fff700 1 -- 172.21.15.157:0/2124250932 >> v2:172.21.15.175:3300/0 conn(0x7f2fe014dcd0 msgr2=0x7f2fe0153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:07.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.834+0000 7f2fd7fff700 1 --2- 172.21.15.157:0/2124250932 >> v2:172.21.15.175:3300/0 conn(0x7f2fe014dcd0 0x7f2fe0153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:07.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.834+0000 7f2fd7fff700 1 -- 172.21.15.157:0/2124250932 >> v1:172.21.15.175:6789/0 conn(0x7f2fe00c5c00 legacy=0x7f2fe00ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:07.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.834+0000 7f2fd7fff700 1 -- 172.21.15.157:0/2124250932 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe01543e0 con 0x7f2fe014e0a0 2023-10-06T03:57:07.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.834+0000 7f2fd77fe700 1 -- 172.21.15.157:0/2124250932 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1082806598 0 0) 0x7f2fe00c5770 con 0x7f2fe00c5c00 2023-10-06T03:57:07.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.834+0000 7f2fe4c0d700 1 --2- 172.21.15.157:0/2124250932 >> v2:172.21.15.175:3300/0 conn(0x7f2fe014dcd0 0x7f2fe0153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:57:07.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.834+0000 7f2fd7fff700 1 --2- 172.21.15.157:0/2124250932 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fe014e0a0 0x7f2fe00c18c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2fd00098e0 tx=0x7f2fd00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cf8cbe4b0428529 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:07.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.834+0000 7f2fd77fe700 1 -- 172.21.15.157:0/2124250932 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd000b640 con 0x7f2fe014e0a0 2023-10-06T03:57:07.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.834+0000 7f2fd77fe700 1 -- 172.21.15.157:0/2124250932 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd000b7a0 con 0x7f2fe014e0a0 2023-10-06T03:57:07.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.834+0000 7f2fd77fe700 1 -- 172.21.15.157:0/2124250932 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd000bb40 con 0x7f2fe014e0a0 2023-10-06T03:57:07.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.835+0000 7f2fe6e71700 1 -- 172.21.15.157:0/2124250932 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fe014e0a0 msgr2=0x7f2fe00c18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:07.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.835+0000 7f2fe6e71700 1 --2- 172.21.15.157:0/2124250932 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fe014e0a0 0x7f2fe00c18c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2fd00098e0 tx=0x7f2fd00091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:07.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.835+0000 7f2fe6e71700 1 -- 172.21.15.157:0/2124250932 shutdown_connections 2023-10-06T03:57:07.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.835+0000 7f2fe6e71700 1 --2- 172.21.15.157:0/2124250932 >> v2:172.21.15.175:3300/0 conn(0x7f2fe014dcd0 0x7f2fe0153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:07.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.835+0000 7f2fe6e71700 1 --2- 172.21.15.157:0/2124250932 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fe014e0a0 0x7f2fe00c18c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:07.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.835+0000 7f2fe6e71700 1 -- 172.21.15.157:0/2124250932 >> 172.21.15.157:0/2124250932 conn(0x7f2fe00692b0 msgr2=0x7f2fe00696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:07.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.835+0000 7f2fe6e71700 1 -- 172.21.15.157:0/2124250932 shutdown_connections 2023-10-06T03:57:07.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.835+0000 7f2fe6e71700 1 -- 172.21.15.157:0/2124250932 wait complete. 2023-10-06T03:57:07.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.836+0000 7f2fe6e71700 1 Processor -- start 2023-10-06T03:57:07.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.836+0000 7f2fe6e71700 1 -- start start 2023-10-06T03:57:07.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.837+0000 7f2fe6e71700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fe00c5c00 0x7f2fe014e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:07.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.837+0000 7f2fe6e71700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fe014dcd0 0x7f2fe0163c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:07.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.837+0000 7f2fe6e71700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fe014e0a0 0x7f2fe0152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:07.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.837+0000 7f2fe6e71700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe0156fa0 con 0x7f2fe014e0a0 2023-10-06T03:57:07.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.837+0000 7f2fe6e71700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe01564d0 con 0x7f2fe014dcd0 2023-10-06T03:57:07.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.837+0000 7f2fe6e71700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe014fc70 con 0x7f2fe00c5c00 2023-10-06T03:57:07.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.837+0000 7f2fe540e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fe014e0a0 0x7f2fe0152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:07.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.837+0000 7f2fe540e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fe014e0a0 0x7f2fe0152290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58592/0 (socket says 172.21.15.157:58592) 2023-10-06T03:57:07.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.837+0000 7f2fe540e700 1 -- 172.21.15.157:0/1173237454 learned_addr learned my addr 172.21.15.157:0/1173237454 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:07.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.837+0000 7f2fe4c0d700 1 --2- 172.21.15.157:0/1173237454 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fe00c5c00 0x7f2fe014e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:07.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.837+0000 7f2fd7fff700 1 --2- 172.21.15.157:0/1173237454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fe014dcd0 0x7f2fe0163c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:07.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.837+0000 7f2fe540e700 1 -- 172.21.15.157:0/1173237454 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fe00c5c00 msgr2=0x7f2fe014e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:07.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.837+0000 7f2fe540e700 1 --2- 172.21.15.157:0/1173237454 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fe00c5c00 0x7f2fe014e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:07.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.837+0000 7f2fe540e700 1 -- 172.21.15.157:0/1173237454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fe014dcd0 msgr2=0x7f2fe0163c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:07.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.837+0000 7f2fe540e700 1 --2- 172.21.15.157:0/1173237454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fe014dcd0 0x7f2fe0163c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:07.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.837+0000 7f2fe540e700 1 -- 172.21.15.157:0/1173237454 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fdc009ae0 con 0x7f2fe014e0a0 2023-10-06T03:57:07.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.837+0000 7f2fd7fff700 1 --2- 172.21.15.157:0/1173237454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fe014dcd0 0x7f2fe0163c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:07.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.837+0000 7f2fe540e700 1 --2- 172.21.15.157:0/1173237454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fe014e0a0 0x7f2fe0152290 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800c5e0 tx=0x7f2fd8007d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:07.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.837+0000 7f2fd4ff9700 1 -- 172.21.15.157:0/1173237454 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8005a70 con 0x7f2fe014e0a0 2023-10-06T03:57:07.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.837+0000 7f2fd4ff9700 1 -- 172.21.15.157:0/1173237454 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd8005bd0 con 0x7f2fe014e0a0 2023-10-06T03:57:07.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.837+0000 7f2fd4ff9700 1 -- 172.21.15.157:0/1173237454 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd80070d0 con 0x7f2fe014e0a0 2023-10-06T03:57:07.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.837+0000 7f2fe6e71700 1 -- 172.21.15.157:0/1173237454 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd0009420 con 0x7f2fe014e0a0 2023-10-06T03:57:07.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.837+0000 7f2fe6e71700 1 -- 172.21.15.157:0/1173237454 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe0156700 con 0x7f2fe014e0a0 2023-10-06T03:57:07.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.838+0000 7f2fe6e71700 1 -- 172.21.15.157:0/1173237454 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe006a8f0 con 0x7f2fe014e0a0 2023-10-06T03:57:07.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.839+0000 7f2fd4ff9700 1 -- 172.21.15.157:0/1173237454 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f2fd8007440 con 0x7f2fe014e0a0 2023-10-06T03:57:07.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.839+0000 7f2fd4ff9700 1 --2- 172.21.15.157:0/1173237454 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2fcc066ed0 0x7f2fcc069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:07.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.839+0000 7f2fd4ff9700 1 -- 172.21.15.157:0/1173237454 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f2fd8083720 con 0x7f2fe014e0a0 2023-10-06T03:57:07.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.839+0000 7f2fe4c0d700 1 --2- 172.21.15.157:0/1173237454 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2fcc066ed0 0x7f2fcc069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:07.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.840+0000 7f2fe4c0d700 1 --2- 172.21.15.157:0/1173237454 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2fcc066ed0 0x7f2fcc069380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0072d0 tx=0x7f2fdc00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:07.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:07.843+0000 7f2fd4ff9700 1 -- 172.21.15.157:0/1173237454 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2fd8050c60 con 0x7f2fe014e0a0 2023-10-06T03:57:08.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.030+0000 7f2fe6e71700 1 -- 172.21.15.157:0/1173237454 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2fe004bb40 con 0x7f2fe014e0a0 2023-10-06T03:57:08.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.031+0000 7f2fd4ff9700 1 -- 172.21.15.157:0/1173237454 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+126 (secure 0 0 0) 0x7f2fd8017020 con 0x7f2fe014e0a0 2023-10-06T03:57:08.032 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:08.032 INFO:teuthology.orchestra.run.smithi157.stdout:[ 2023-10-06T03:57:08.032 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-10-06T03:57:08.033 INFO:teuthology.orchestra.run.smithi157.stdout: "poolnum": 1, 2023-10-06T03:57:08.033 INFO:teuthology.orchestra.run.smithi157.stdout: "poolname": ".mgr" 2023-10-06T03:57:08.033 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-10-06T03:57:08.033 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-10-06T03:57:08.033 INFO:teuthology.orchestra.run.smithi157.stdout: "poolnum": 2, 2023-10-06T03:57:08.033 INFO:teuthology.orchestra.run.smithi157.stdout: "poolname": "rbd" 2023-10-06T03:57:08.033 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-10-06T03:57:08.033 INFO:teuthology.orchestra.run.smithi157.stdout:] 2023-10-06T03:57:08.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.032+0000 7f2fe6e71700 1 -- 172.21.15.157:0/1173237454 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2fcc066ed0 msgr2=0x7f2fcc069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:08.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.032+0000 7f2fe6e71700 1 --2- 172.21.15.157:0/1173237454 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2fcc066ed0 0x7f2fcc069380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0072d0 tx=0x7f2fdc00b040 comp rx=0 tx=0).stop 2023-10-06T03:57:08.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.032+0000 7f2fe6e71700 1 -- 172.21.15.157:0/1173237454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fe014e0a0 msgr2=0x7f2fe0152290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:08.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.032+0000 7f2fe6e71700 1 --2- 172.21.15.157:0/1173237454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fe014e0a0 0x7f2fe0152290 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800c5e0 tx=0x7f2fd8007d60 comp rx=0 tx=0).stop 2023-10-06T03:57:08.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.032+0000 7f2fe6e71700 1 -- 172.21.15.157:0/1173237454 shutdown_connections 2023-10-06T03:57:08.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.032+0000 7f2fe6e71700 1 --2- 172.21.15.157:0/1173237454 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2fcc066ed0 0x7f2fcc069380 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:08.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.032+0000 7f2fe6e71700 1 --2- 172.21.15.157:0/1173237454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fe014e0a0 0x7f2fe0152290 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:08.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.032+0000 7f2fe6e71700 1 --2- 172.21.15.157:0/1173237454 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fe00c5c00 0x7f2fe014e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:08.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.032+0000 7f2fe6e71700 1 --2- 172.21.15.157:0/1173237454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fe014dcd0 0x7f2fe0163c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:08.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.032+0000 7f2fe6e71700 1 -- 172.21.15.157:0/1173237454 >> 172.21.15.157:0/1173237454 conn(0x7f2fe00692b0 msgr2=0x7f2fe00c1840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:08.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.033+0000 7f2fe6e71700 1 -- 172.21.15.157:0/1173237454 shutdown_connections 2023-10-06T03:57:08.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.033+0000 7f2fe6e71700 1 -- 172.21.15.157:0/1173237454 wait complete. 2023-10-06T03:57:08.046 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T03:57:08.046 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T03:57:08.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.163+0000 7f6b62087700 1 Processor -- start 2023-10-06T03:57:08.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.163+0000 7f6b62087700 1 -- start start 2023-10-06T03:57:08.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.164+0000 7f6b62087700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6b5c06ab50 0x7f6b5c159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:08.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.164+0000 7f6b62087700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5c1569b0 con 0x7f6b5c06a780 2023-10-06T03:57:08.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.164+0000 7f6b62087700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5c06ea70 con 0x7f6b5c0bc750 2023-10-06T03:57:08.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.164+0000 7f6b62087700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b5c155e90 con 0x7f6b5c06ab50 2023-10-06T03:57:08.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.164+0000 7f6b5b7fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f6b5c06a780 0x7f6b5c06c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44114/0 (socket says 172.21.15.157:44114) 2023-10-06T03:57:08.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.164+0000 7f6b5b7fe700 1 -- 172.21.15.157:0/3646552496 learned_addr learned my addr 172.21.15.157:0/3646552496 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:08.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.165+0000 7f6b5a7fc700 1 -- 172.21.15.157:0/3646552496 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1204633547 0 0) 0x7f6b5c1569b0 con 0x7f6b5c06a780 2023-10-06T03:57:08.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.165+0000 7f6b5affd700 1 --2- 172.21.15.157:0/3646552496 >> v2:172.21.15.175:3300/0 conn(0x7f6b5c06ab50 0x7f6b5c159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:08.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.165+0000 7f6b5a7fc700 1 -- 172.21.15.157:0/3646552496 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b44003570 con 0x7f6b5c06a780 2023-10-06T03:57:08.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.165+0000 7f6b5a7fc700 1 -- 172.21.15.157:0/3646552496 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2347160561 0 0) 0x7f6b44003570 con 0x7f6b5c06a780 2023-10-06T03:57:08.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.165+0000 7f6b5a7fc700 1 -- 172.21.15.157:0/3646552496 >> v2:172.21.15.175:3300/0 conn(0x7f6b5c06ab50 msgr2=0x7f6b5c159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:08.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.165+0000 7f6b5a7fc700 1 --2- 172.21.15.157:0/3646552496 >> v2:172.21.15.175:3300/0 conn(0x7f6b5c06ab50 0x7f6b5c159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:08.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.165+0000 7f6b5a7fc700 1 -- 172.21.15.157:0/3646552496 >> v1:172.21.15.175:6789/0 conn(0x7f6b5c0bc750 legacy=0x7f6b5c14f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:08.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.165+0000 7f6b5a7fc700 1 -- 172.21.15.157:0/3646552496 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5c068e00 con 0x7f6b5c06a780 2023-10-06T03:57:08.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.165+0000 7f6b5a7fc700 1 -- 172.21.15.157:0/3646552496 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f6b4c0017f0 con 0x7f6b5c06a780 2023-10-06T03:57:08.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.165+0000 7f6b5a7fc700 1 -- 172.21.15.157:0/3646552496 >> v1:172.21.15.157:6789/0 conn(0x7f6b5c06a780 legacy=0x7f6b5c06c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:08.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.165+0000 7f6b5a7fc700 1 --2- 172.21.15.157:0/3646552496 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b44003a60 0x7f6b44003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:08.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.165+0000 7f6b5a7fc700 1 --2- 172.21.15.157:0/3646552496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b44004810 0x7f6b44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:08.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.165+0000 7f6b5a7fc700 1 --2- 172.21.15.157:0/3646552496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b440052e0 0x7f6b4400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:08.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.165+0000 7f6b5a7fc700 1 -- 172.21.15.157:0/3646552496 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4400c160 con 0x7f6b440052e0 2023-10-06T03:57:08.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.165+0000 7f6b5a7fc700 1 -- 172.21.15.157:0/3646552496 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4400c2d0 con 0x7f6b44004810 2023-10-06T03:57:08.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.165+0000 7f6b5a7fc700 1 -- 172.21.15.157:0/3646552496 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4400c490 con 0x7f6b44003a60 2023-10-06T03:57:08.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.165+0000 7f6b5bfff700 1 --2- 172.21.15.157:0/3646552496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b440052e0 0x7f6b4400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:08.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.165+0000 7f6b5affd700 1 --2- 172.21.15.157:0/3646552496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b44004810 0x7f6b44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:08.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.165+0000 7f6b5b7fe700 1 --2- 172.21.15.157:0/3646552496 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b44003a60 0x7f6b44003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:08.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.165+0000 7f6b5bfff700 1 -- 172.21.15.157:0/3646552496 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b44003a60 msgr2=0x7f6b44003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:08.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.165+0000 7f6b5bfff700 1 --2- 172.21.15.157:0/3646552496 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b44003a60 0x7f6b44003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:08.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.165+0000 7f6b5bfff700 1 -- 172.21.15.157:0/3646552496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b44004810 msgr2=0x7f6b44004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:08.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.165+0000 7f6b5bfff700 1 --2- 172.21.15.157:0/3646552496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b44004810 0x7f6b44004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:08.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.165+0000 7f6b5bfff700 1 -- 172.21.15.157:0/3646552496 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b44003570 con 0x7f6b440052e0 2023-10-06T03:57:08.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.165+0000 7f6b5bfff700 1 -- 172.21.15.157:0/3646552496 reap_dead start 2023-10-06T03:57:08.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.166+0000 7f6b5bfff700 1 --2- 172.21.15.157:0/3646552496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b440052e0 0x7f6b4400ba60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6b540018b0 tx=0x7f6b54007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743c9a7b1c935807 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:08.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.166+0000 7f6b5a7fc700 1 -- 172.21.15.157:0/3646552496 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b54012420 con 0x7f6b440052e0 2023-10-06T03:57:08.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.166+0000 7f6b5a7fc700 1 -- 172.21.15.157:0/3646552496 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b54012580 con 0x7f6b440052e0 2023-10-06T03:57:08.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.166+0000 7f6b5a7fc700 1 -- 172.21.15.157:0/3646552496 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b54012920 con 0x7f6b440052e0 2023-10-06T03:57:08.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.166+0000 7f6b62087700 1 -- 172.21.15.157:0/3646552496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b440052e0 msgr2=0x7f6b4400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:08.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.166+0000 7f6b62087700 1 --2- 172.21.15.157:0/3646552496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b440052e0 0x7f6b4400ba60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6b540018b0 tx=0x7f6b54007d30 comp rx=0 tx=0).stop 2023-10-06T03:57:08.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.166+0000 7f6b62087700 1 -- 172.21.15.157:0/3646552496 shutdown_connections 2023-10-06T03:57:08.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.166+0000 7f6b62087700 1 --2- 172.21.15.157:0/3646552496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b440052e0 0x7f6b4400ba60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:08.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.166+0000 7f6b62087700 1 -- 172.21.15.157:0/3646552496 >> 172.21.15.157:0/3646552496 conn(0x7f6b5c06b510 msgr2=0x7f6b5c06b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:08.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.166+0000 7f6b62087700 1 -- 172.21.15.157:0/3646552496 shutdown_connections 2023-10-06T03:57:08.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.166+0000 7f6b62087700 1 -- 172.21.15.157:0/3646552496 wait complete. 2023-10-06T03:57:08.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.167+0000 7f6b62087700 1 Processor -- start 2023-10-06T03:57:08.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.167+0000 7f6b62087700 1 -- start start 2023-10-06T03:57:08.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.168+0000 7f6b62087700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b440052e0 0x7f6b5c166910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:08.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.168+0000 7f6b62087700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b5c0b92e0 0x7f6b5c0c1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:08.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.168+0000 7f6b62087700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b5c15b240 0x7f6b5c169360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:08.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.168+0000 7f6b62087700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5c0b97f0 con 0x7f6b440052e0 2023-10-06T03:57:08.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.168+0000 7f6b62087700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5c166480 con 0x7f6b5c15b240 2023-10-06T03:57:08.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.168+0000 7f6b62087700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5c0c39d0 con 0x7f6b5c0b92e0 2023-10-06T03:57:08.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.168+0000 7f6b5b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b440052e0 0x7f6b5c166910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:08.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.168+0000 7f6b5b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b440052e0 0x7f6b5c166910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58608/0 (socket says 172.21.15.157:58608) 2023-10-06T03:57:08.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.168+0000 7f6b5b7fe700 1 -- 172.21.15.157:0/2806981850 learned_addr learned my addr 172.21.15.157:0/2806981850 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:08.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.168+0000 7f6b5affd700 1 --2- 172.21.15.157:0/2806981850 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b5c0b92e0 0x7f6b5c0c1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:08.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.168+0000 7f6b5bfff700 1 --2- 172.21.15.157:0/2806981850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b5c15b240 0x7f6b5c169360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:08.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.169+0000 7f6b5b7fe700 1 -- 172.21.15.157:0/2806981850 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b5c0b92e0 msgr2=0x7f6b5c0c1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:08.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.169+0000 7f6b5b7fe700 1 --2- 172.21.15.157:0/2806981850 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b5c0b92e0 0x7f6b5c0c1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:08.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.169+0000 7f6b5b7fe700 1 -- 172.21.15.157:0/2806981850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b5c15b240 msgr2=0x7f6b5c169360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:08.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.169+0000 7f6b5b7fe700 1 --2- 172.21.15.157:0/2806981850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b5c15b240 0x7f6b5c169360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:08.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.169+0000 7f6b5b7fe700 1 -- 172.21.15.157:0/2806981850 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5c0b9a40 con 0x7f6b440052e0 2023-10-06T03:57:08.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.169+0000 7f6b5b7fe700 1 --2- 172.21.15.157:0/2806981850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b440052e0 0x7f6b5c166910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c007a90 tx=0x7f6b4c0021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:08.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.169+0000 7f6b437fe700 1 -- 172.21.15.157:0/2806981850 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c00acc0 con 0x7f6b440052e0 2023-10-06T03:57:08.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.169+0000 7f6b437fe700 1 -- 172.21.15.157:0/2806981850 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4c00ae20 con 0x7f6b440052e0 2023-10-06T03:57:08.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.169+0000 7f6b62087700 1 -- 172.21.15.157:0/2806981850 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b5c166110 con 0x7f6b440052e0 2023-10-06T03:57:08.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.169+0000 7f6b437fe700 1 -- 172.21.15.157:0/2806981850 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c013640 con 0x7f6b440052e0 2023-10-06T03:57:08.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.169+0000 7f6b62087700 1 -- 172.21.15.157:0/2806981850 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b5c0bc130 con 0x7f6b440052e0 2023-10-06T03:57:08.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.170+0000 7f6b62087700 1 -- 172.21.15.157:0/2806981850 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b5c1551f0 con 0x7f6b440052e0 2023-10-06T03:57:08.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.170+0000 7f6b437fe700 1 -- 172.21.15.157:0/2806981850 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f6b4c0139f0 con 0x7f6b440052e0 2023-10-06T03:57:08.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.170+0000 7f6b437fe700 1 --2- 172.21.15.157:0/2806981850 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6b48066ed0 0x7f6b48069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:08.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.170+0000 7f6b437fe700 1 -- 172.21.15.157:0/2806981850 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f6b4c084bb0 con 0x7f6b440052e0 2023-10-06T03:57:08.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.171+0000 7f6b5affd700 1 --2- 172.21.15.157:0/2806981850 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6b48066ed0 0x7f6b48069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:08.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.171+0000 7f6b5affd700 1 --2- 172.21.15.157:0/2806981850 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6b48066ed0 0x7f6b48069380 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000b0b0 tx=0x7f6b5000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:08.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.175+0000 7f6b437fe700 1 -- 172.21.15.157:0/2806981850 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6b4c0520f0 con 0x7f6b440052e0 2023-10-06T03:57:08.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.326+0000 7f6b437fe700 1 -- 172.21.15.157:0/2806981850 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6b4c011d90 con 0x7f6b440052e0 2023-10-06T03:57:08.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:08.359+0000 7f6b62087700 1 -- 172.21.15.157:0/2806981850 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6b5c166760 con 0x7f6b440052e0 2023-10-06T03:57:09.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.327+0000 7f6b437fe700 1 -- 172.21.15.157:0/2806981850 <== mon.0 v2:172.21.15.157:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6b4c055b80 con 0x7f6b440052e0 2023-10-06T03:57:09.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.332+0000 7f6b62087700 1 -- 172.21.15.157:0/2806981850 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6b48066ed0 msgr2=0x7f6b48069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:09.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.332+0000 7f6b62087700 1 --2- 172.21.15.157:0/2806981850 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6b48066ed0 0x7f6b48069380 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000b0b0 tx=0x7f6b5000b040 comp rx=0 tx=0).stop 2023-10-06T03:57:09.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.332+0000 7f6b62087700 1 -- 172.21.15.157:0/2806981850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b440052e0 msgr2=0x7f6b5c166910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:09.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.332+0000 7f6b62087700 1 --2- 172.21.15.157:0/2806981850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b440052e0 0x7f6b5c166910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c007a90 tx=0x7f6b4c0021d0 comp rx=0 tx=0).stop 2023-10-06T03:57:09.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.332+0000 7f6b62087700 1 -- 172.21.15.157:0/2806981850 shutdown_connections 2023-10-06T03:57:09.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.332+0000 7f6b62087700 1 --2- 172.21.15.157:0/2806981850 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6b48066ed0 0x7f6b48069380 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:09.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.332+0000 7f6b62087700 1 --2- 172.21.15.157:0/2806981850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b5c15b240 0x7f6b5c169360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:09.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.332+0000 7f6b62087700 1 --2- 172.21.15.157:0/2806981850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b440052e0 0x7f6b5c166910 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:09.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.332+0000 7f6b62087700 1 --2- 172.21.15.157:0/2806981850 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b5c0b92e0 0x7f6b5c0c1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:09.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.332+0000 7f6b62087700 1 -- 172.21.15.157:0/2806981850 >> 172.21.15.157:0/2806981850 conn(0x7f6b5c06b510 msgr2=0x7f6b5c153090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:09.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.333+0000 7f6b62087700 1 -- 172.21.15.157:0/2806981850 shutdown_connections 2023-10-06T03:57:09.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.333+0000 7f6b62087700 1 -- 172.21.15.157:0/2806981850 wait complete. 2023-10-06T03:57:09.375 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' created 2023-10-06T03:57:09.375 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T03:57:09.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.498+0000 7fbdfccb1700 1 Processor -- start 2023-10-06T03:57:09.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.498+0000 7fbdfccb1700 1 -- start start 2023-10-06T03:57:09.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.499+0000 7fbdfccb1700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbdf80c6290 0x7fbdf80c0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:09.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.499+0000 7fbdfccb1700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdf80cb1c0 con 0x7fbdf8067530 2023-10-06T03:57:09.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.499+0000 7fbdfccb1700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdf81577f0 con 0x7fbdf8157c80 2023-10-06T03:57:09.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.499+0000 7fbdfccb1700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdf80bf610 con 0x7fbdf80c6290 2023-10-06T03:57:09.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.499+0000 7fbdf7fff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fbdf8067530 0x7fbdf8063900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44126/0 (socket says 172.21.15.157:44126) 2023-10-06T03:57:09.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.499+0000 7fbdf7fff700 1 -- 172.21.15.157:0/842696646 learned_addr learned my addr 172.21.15.157:0/842696646 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:09.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.499+0000 7fbdf77fe700 1 --2- 172.21.15.157:0/842696646 >> v2:172.21.15.175:3300/0 conn(0x7fbdf80c6290 0x7fbdf80c0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:09.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.499+0000 7fbdf67fc700 1 -- 172.21.15.157:0/842696646 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2967995514 0 0) 0x7fbdf80cb1c0 con 0x7fbdf8067530 2023-10-06T03:57:09.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.499+0000 7fbdf67fc700 1 -- 172.21.15.157:0/842696646 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbde0003570 con 0x7fbdf8067530 2023-10-06T03:57:09.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.499+0000 7fbdf67fc700 1 -- 172.21.15.157:0/842696646 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 277762408 0 0) 0x7fbde0003570 con 0x7fbdf8067530 2023-10-06T03:57:09.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.499+0000 7fbdf67fc700 1 -- 172.21.15.157:0/842696646 >> v2:172.21.15.175:3300/0 conn(0x7fbdf80c6290 msgr2=0x7fbdf80c0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:09.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.499+0000 7fbdf67fc700 1 --2- 172.21.15.157:0/842696646 >> v2:172.21.15.175:3300/0 conn(0x7fbdf80c6290 0x7fbdf80c0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:09.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.499+0000 7fbdf67fc700 1 -- 172.21.15.157:0/842696646 >> v1:172.21.15.175:6789/0 conn(0x7fbdf8157c80 legacy=0x7fbdf80631f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:09.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.499+0000 7fbdf67fc700 1 -- 172.21.15.157:0/842696646 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf8064010 con 0x7fbdf8067530 2023-10-06T03:57:09.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.499+0000 7fbdf67fc700 1 -- 172.21.15.157:0/842696646 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7fbdf00017f0 con 0x7fbdf8067530 2023-10-06T03:57:09.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.499+0000 7fbdf67fc700 1 -- 172.21.15.157:0/842696646 >> v1:172.21.15.157:6789/0 conn(0x7fbdf8067530 legacy=0x7fbdf8063900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:09.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.499+0000 7fbdf67fc700 1 --2- 172.21.15.157:0/842696646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbde0003a60 0x7fbde0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:09.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.500+0000 7fbdf67fc700 1 --2- 172.21.15.157:0/842696646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbde0004810 0x7fbde0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:09.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.500+0000 7fbdf67fc700 1 --2- 172.21.15.157:0/842696646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbde00051c0 0x7fbde000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:09.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.500+0000 7fbdf67fc700 1 -- 172.21.15.157:0/842696646 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf80bf610 con 0x7fbde00051c0 2023-10-06T03:57:09.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.500+0000 7fbdf67fc700 1 -- 172.21.15.157:0/842696646 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde000c0d0 con 0x7fbde0004810 2023-10-06T03:57:09.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.500+0000 7fbdf67fc700 1 -- 172.21.15.157:0/842696646 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde000c290 con 0x7fbde0003a60 2023-10-06T03:57:09.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.500+0000 7fbdf7fff700 1 --2- 172.21.15.157:0/842696646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbde00051c0 0x7fbde000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:09.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.500+0000 7fbdf77fe700 1 --2- 172.21.15.157:0/842696646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbde0003a60 0x7fbde0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:09.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.500+0000 7fbdf6ffd700 1 --2- 172.21.15.157:0/842696646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbde0004810 0x7fbde0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:09.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.500+0000 7fbdf7fff700 1 -- 172.21.15.157:0/842696646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbde0003a60 msgr2=0x7fbde0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:09.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.500+0000 7fbdf7fff700 1 --2- 172.21.15.157:0/842696646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbde0003a60 0x7fbde0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:09.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.500+0000 7fbdf7fff700 1 -- 172.21.15.157:0/842696646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbde0004810 msgr2=0x7fbde0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:09.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.500+0000 7fbdf7fff700 1 --2- 172.21.15.157:0/842696646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbde0004810 0x7fbde0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:09.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.500+0000 7fbdf7fff700 1 -- 172.21.15.157:0/842696646 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde0003570 con 0x7fbde00051c0 2023-10-06T03:57:09.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.500+0000 7fbdf7fff700 1 -- 172.21.15.157:0/842696646 reap_dead start 2023-10-06T03:57:09.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.500+0000 7fbdf7fff700 1 --2- 172.21.15.157:0/842696646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbde00051c0 0x7fbde000ba60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbdf000a930 tx=0x7fbdf00021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b38da9f5650245d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:09.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.500+0000 7fbdf67fc700 1 -- 172.21.15.157:0/842696646 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf00070d0 con 0x7fbde00051c0 2023-10-06T03:57:09.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.500+0000 7fbdf67fc700 1 -- 172.21.15.157:0/842696646 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf0007230 con 0x7fbde00051c0 2023-10-06T03:57:09.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.500+0000 7fbdf67fc700 1 -- 172.21.15.157:0/842696646 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf00075d0 con 0x7fbde00051c0 2023-10-06T03:57:09.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.501+0000 7fbdfccb1700 1 -- 172.21.15.157:0/842696646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbde00051c0 msgr2=0x7fbde000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:09.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.501+0000 7fbdfccb1700 1 --2- 172.21.15.157:0/842696646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbde00051c0 0x7fbde000ba60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbdf000a930 tx=0x7fbdf00021d0 comp rx=0 tx=0).stop 2023-10-06T03:57:09.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.501+0000 7fbdfccb1700 1 -- 172.21.15.157:0/842696646 shutdown_connections 2023-10-06T03:57:09.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.501+0000 7fbdfccb1700 1 --2- 172.21.15.157:0/842696646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbde00051c0 0x7fbde000ba60 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:09.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.501+0000 7fbdfccb1700 1 -- 172.21.15.157:0/842696646 >> 172.21.15.157:0/842696646 conn(0x7fbdf806c1f0 msgr2=0x7fbdf806c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:09.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.501+0000 7fbdfccb1700 1 -- 172.21.15.157:0/842696646 shutdown_connections 2023-10-06T03:57:09.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.501+0000 7fbdfccb1700 1 -- 172.21.15.157:0/842696646 wait complete. 2023-10-06T03:57:09.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.502+0000 7fbdfccb1700 1 Processor -- start 2023-10-06T03:57:09.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.502+0000 7fbdfccb1700 1 -- start start 2023-10-06T03:57:09.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.503+0000 7fbdfccb1700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbde00051c0 0x7fbdf80af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:09.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.503+0000 7fbdfccb1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdf80c1980 0x7fbdf8164410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:09.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.503+0000 7fbdfccb1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdf80c2910 0x7fbdf80c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:09.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.503+0000 7fbdfccb1700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf80c2d90 con 0x7fbdf80c2910 2023-10-06T03:57:09.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.503+0000 7fbdfccb1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf80af770 con 0x7fbdf80c1980 2023-10-06T03:57:09.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.503+0000 7fbdfccb1700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf80ae6e0 con 0x7fbde00051c0 2023-10-06T03:57:09.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.503+0000 7fbdf7fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdf80c2910 0x7fbdf80c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:09.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.503+0000 7fbdf7fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdf80c2910 0x7fbdf80c6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58636/0 (socket says 172.21.15.157:58636) 2023-10-06T03:57:09.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.503+0000 7fbdf7fff700 1 -- 172.21.15.157:0/1912623391 learned_addr learned my addr 172.21.15.157:0/1912623391 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:09.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.503+0000 7fbdf77fe700 1 --2- 172.21.15.157:0/1912623391 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbde00051c0 0x7fbdf80af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:09.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.503+0000 7fbdf6ffd700 1 --2- 172.21.15.157:0/1912623391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdf80c1980 0x7fbdf8164410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:09.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.503+0000 7fbdf7fff700 1 -- 172.21.15.157:0/1912623391 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbde00051c0 msgr2=0x7fbdf80af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:09.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.503+0000 7fbdf7fff700 1 --2- 172.21.15.157:0/1912623391 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbde00051c0 0x7fbdf80af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:09.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.503+0000 7fbdf7fff700 1 -- 172.21.15.157:0/1912623391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdf80c1980 msgr2=0x7fbdf8164410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:09.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.503+0000 7fbdf7fff700 1 --2- 172.21.15.157:0/1912623391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdf80c1980 0x7fbdf8164410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:09.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.503+0000 7fbdf7fff700 1 -- 172.21.15.157:0/1912623391 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf80af9c0 con 0x7fbdf80c2910 2023-10-06T03:57:09.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.503+0000 7fbdf7fff700 1 --2- 172.21.15.157:0/1912623391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdf80c2910 0x7fbdf80c6f90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0000f80 tx=0x7fbdf0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:09.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.503+0000 7fbddf7fe700 1 -- 172.21.15.157:0/1912623391 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf00075b0 con 0x7fbdf80c2910 2023-10-06T03:57:09.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.503+0000 7fbddf7fe700 1 -- 172.21.15.157:0/1912623391 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf001d420 con 0x7fbdf80c2910 2023-10-06T03:57:09.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.503+0000 7fbdfccb1700 1 -- 172.21.15.157:0/1912623391 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf80ae950 con 0x7fbdf80c2910 2023-10-06T03:57:09.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.503+0000 7fbddf7fe700 1 -- 172.21.15.157:0/1912623391 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf001d820 con 0x7fbdf80c2910 2023-10-06T03:57:09.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.503+0000 7fbdfccb1700 1 -- 172.21.15.157:0/1912623391 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf80afca0 con 0x7fbdf80c2910 2023-10-06T03:57:09.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.504+0000 7fbdfccb1700 1 -- 172.21.15.157:0/1912623391 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf806a490 con 0x7fbdf80c2910 2023-10-06T03:57:09.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.505+0000 7fbddf7fe700 1 -- 172.21.15.157:0/1912623391 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbdf001dc50 con 0x7fbdf80c2910 2023-10-06T03:57:09.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.505+0000 7fbddf7fe700 1 --2- 172.21.15.157:0/1912623391 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbde4066f30 0x7fbde40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:09.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.505+0000 7fbddf7fe700 1 -- 172.21.15.157:0/1912623391 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5576+0+0 (secure 0 0 0) 0x7fbdf0086ad0 con 0x7fbdf80c2910 2023-10-06T03:57:09.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.505+0000 7fbdf77fe700 1 --2- 172.21.15.157:0/1912623391 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbde4066f30 0x7fbde40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:09.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.506+0000 7fbdf77fe700 1 --2- 172.21.15.157:0/1912623391 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbde4066f30 0x7fbde40693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbde8006f00 tx=0x7fbde80091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:09.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.510+0000 7fbddf7fe700 1 -- 172.21.15.157:0/1912623391 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbdf0053dc0 con 0x7fbdf80c2910 2023-10-06T03:57:09.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:09.691+0000 7fbdfccb1700 1 -- 172.21.15.157:0/1912623391 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbdf804bb40 con 0x7fbdf80c2910 2023-10-06T03:57:10.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.329+0000 7fbddf7fe700 1 -- 172.21.15.157:0/1912623391 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbdf00573e0 con 0x7fbdf80c2910 2023-10-06T03:57:10.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.333+0000 7fbdfccb1700 1 -- 172.21.15.157:0/1912623391 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbde4066f30 msgr2=0x7fbde40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:10.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.333+0000 7fbdfccb1700 1 --2- 172.21.15.157:0/1912623391 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbde4066f30 0x7fbde40693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbde8006f00 tx=0x7fbde80091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:10.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.333+0000 7fbdfccb1700 1 -- 172.21.15.157:0/1912623391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdf80c2910 msgr2=0x7fbdf80c6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:10.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.333+0000 7fbdfccb1700 1 --2- 172.21.15.157:0/1912623391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdf80c2910 0x7fbdf80c6f90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0000f80 tx=0x7fbdf0016000 comp rx=0 tx=0).stop 2023-10-06T03:57:10.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.334+0000 7fbdfccb1700 1 -- 172.21.15.157:0/1912623391 shutdown_connections 2023-10-06T03:57:10.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.334+0000 7fbdfccb1700 1 --2- 172.21.15.157:0/1912623391 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbde4066f30 0x7fbde40693e0 secure :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbde8006f00 tx=0x7fbde80091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:10.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.334+0000 7fbdfccb1700 1 --2- 172.21.15.157:0/1912623391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdf80c2910 0x7fbdf80c6f90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:10.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.334+0000 7fbdfccb1700 1 --2- 172.21.15.157:0/1912623391 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbde00051c0 0x7fbdf80af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:10.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.334+0000 7fbdfccb1700 1 --2- 172.21.15.157:0/1912623391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdf80c1980 0x7fbdf8164410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:10.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.334+0000 7fbdfccb1700 1 -- 172.21.15.157:0/1912623391 >> 172.21.15.157:0/1912623391 conn(0x7fbdf806c1f0 msgr2=0x7fbdf8154cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:10.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.335+0000 7fbdfccb1700 1 -- 172.21.15.157:0/1912623391 shutdown_connections 2023-10-06T03:57:10.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.335+0000 7fbdfccb1700 1 -- 172.21.15.157:0/1912623391 wait complete. 2023-10-06T03:57:10.351 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' created 2023-10-06T03:57:10.363 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T03:57:10.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.480+0000 7f74a4534700 1 Processor -- start 2023-10-06T03:57:10.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.480+0000 7f74a4534700 1 -- start start 2023-10-06T03:57:10.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.481+0000 7f74a4534700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f749c068920 0x7f749c156840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:10.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.481+0000 7f74a4534700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f749c0c52e0 0x7f749c156d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:10.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.481+0000 7f74a4534700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f749c06d530 con 0x7f749c0ca630 2023-10-06T03:57:10.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.481+0000 7f74a4534700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f749c06d9c0 con 0x7f749c0c52e0 2023-10-06T03:57:10.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.481+0000 7f74a4534700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f749c06f0c0 con 0x7f749c068920 2023-10-06T03:57:10.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.481+0000 7f74a2ad1700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f749c0c52e0 0x7f749c156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:10.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.481+0000 7f74a22d0700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f749c0ca630 0x7f749c0c6ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44128/0 (socket says 172.21.15.157:44128) 2023-10-06T03:57:10.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.481+0000 7f74a22d0700 1 -- 172.21.15.157:0/4020493762 learned_addr learned my addr 172.21.15.157:0/4020493762 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:10.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.481+0000 7f74a1acf700 1 --2- 172.21.15.157:0/4020493762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f749c068920 0x7f749c156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:10.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.482+0000 7f74a2ad1700 1 -- 172.21.15.157:0/4020493762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f749c068920 msgr2=0x7f749c156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:10.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.482+0000 7f74a2ad1700 1 --2- 172.21.15.157:0/4020493762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f749c068920 0x7f749c156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:10.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.482+0000 7f74a2ad1700 1 -- 172.21.15.157:0/4020493762 >> v1:172.21.15.157:6789/0 conn(0x7f749c0ca630 legacy=0x7f749c0c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:10.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.482+0000 7f74a12ce700 1 -- 172.21.15.157:0/4020493762 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1810335804 0 0) 0x7f749c06d530 con 0x7f749c0ca630 2023-10-06T03:57:10.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.482+0000 7f74a2ad1700 1 -- 172.21.15.157:0/4020493762 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749c06f3a0 con 0x7f749c0c52e0 2023-10-06T03:57:10.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.482+0000 7f74a2ad1700 1 --2- 172.21.15.157:0/4020493762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f749c0c52e0 0x7f749c156d80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f74940098e0 tx=0x7f74940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0178e55d4a7969e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:10.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.482+0000 7f74a12ce700 1 -- 172.21.15.157:0/4020493762 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7494009e80 con 0x7f749c0c52e0 2023-10-06T03:57:10.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.482+0000 7f74a12ce700 1 -- 172.21.15.157:0/4020493762 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749400b860 con 0x7f749c0c52e0 2023-10-06T03:57:10.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.482+0000 7f74a12ce700 1 -- 172.21.15.157:0/4020493762 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749400bbe0 con 0x7f749c0c52e0 2023-10-06T03:57:10.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.482+0000 7f74a4534700 1 -- 172.21.15.157:0/4020493762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f749c0c52e0 msgr2=0x7f749c156d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:10.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.482+0000 7f74a4534700 1 --2- 172.21.15.157:0/4020493762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f749c0c52e0 0x7f749c156d80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f74940098e0 tx=0x7f74940091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:10.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.483+0000 7f74a4534700 1 -- 172.21.15.157:0/4020493762 shutdown_connections 2023-10-06T03:57:10.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.483+0000 7f74a4534700 1 --2- 172.21.15.157:0/4020493762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f749c0c52e0 0x7f749c156d80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:10.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.483+0000 7f74a4534700 1 --2- 172.21.15.157:0/4020493762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f749c068920 0x7f749c156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:10.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.483+0000 7f74a4534700 1 -- 172.21.15.157:0/4020493762 >> 172.21.15.157:0/4020493762 conn(0x7f749c0692b0 msgr2=0x7f749c0696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:10.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.483+0000 7f74a4534700 1 -- 172.21.15.157:0/4020493762 shutdown_connections 2023-10-06T03:57:10.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.483+0000 7f74a4534700 1 -- 172.21.15.157:0/4020493762 wait complete. 2023-10-06T03:57:10.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.484+0000 7f74a4534700 1 Processor -- start 2023-10-06T03:57:10.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.484+0000 7f74a4534700 1 -- start start 2023-10-06T03:57:10.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.484+0000 7f74a4534700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f749c068920 0x7f749c1517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:10.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.484+0000 7f74a4534700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f749c0c52e0 0x7f749c0bf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:10.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.484+0000 7f74a4534700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f749c0ca630 0x7f749c0bf800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:10.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.484+0000 7f74a4534700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749c167dc0 con 0x7f749c0ca630 2023-10-06T03:57:10.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.484+0000 7f74a4534700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749c155b30 con 0x7f749c0c52e0 2023-10-06T03:57:10.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.484+0000 7f74a4534700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f749c0c75a0 con 0x7f749c068920 2023-10-06T03:57:10.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.484+0000 7f74a2ad1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f749c0ca630 0x7f749c0bf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:10.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.484+0000 7f74a2ad1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f749c0ca630 0x7f749c0bf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58648/0 (socket says 172.21.15.157:58648) 2023-10-06T03:57:10.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.484+0000 7f74a2ad1700 1 -- 172.21.15.157:0/182646513 learned_addr learned my addr 172.21.15.157:0/182646513 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:10.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.485+0000 7f74a22d0700 1 --2- 172.21.15.157:0/182646513 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f749c068920 0x7f749c1517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:10.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.485+0000 7f74a1acf700 1 --2- 172.21.15.157:0/182646513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f749c0c52e0 0x7f749c0bf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:10.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.485+0000 7f74a2ad1700 1 -- 172.21.15.157:0/182646513 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f749c068920 msgr2=0x7f749c1517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:10.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.485+0000 7f74a2ad1700 1 --2- 172.21.15.157:0/182646513 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f749c068920 0x7f749c1517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:10.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.485+0000 7f74a2ad1700 1 -- 172.21.15.157:0/182646513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f749c0c52e0 msgr2=0x7f749c0bf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:10.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.485+0000 7f74a2ad1700 1 --2- 172.21.15.157:0/182646513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f749c0c52e0 0x7f749c0bf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:10.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.485+0000 7f74a2ad1700 1 -- 172.21.15.157:0/182646513 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7494009420 con 0x7f749c0ca630 2023-10-06T03:57:10.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.485+0000 7f74a2ad1700 1 --2- 172.21.15.157:0/182646513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f749c0ca630 0x7f749c0bf800 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f749400b3c0 tx=0x7f7494017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:10.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.485+0000 7f74927fc700 1 -- 172.21.15.157:0/182646513 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7494011630 con 0x7f749c0ca630 2023-10-06T03:57:10.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.485+0000 7f74927fc700 1 -- 172.21.15.157:0/182646513 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74940117b0 con 0x7f749c0ca630 2023-10-06T03:57:10.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.485+0000 7f74927fc700 1 -- 172.21.15.157:0/182646513 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7494011b70 con 0x7f749c0ca630 2023-10-06T03:57:10.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.485+0000 7f74a4534700 1 -- 172.21.15.157:0/182646513 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f749c0c64a0 con 0x7f749c0ca630 2023-10-06T03:57:10.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.485+0000 7f74a4534700 1 -- 172.21.15.157:0/182646513 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f749c0bea70 con 0x7f749c0ca630 2023-10-06T03:57:10.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.486+0000 7f74a4534700 1 -- 172.21.15.157:0/182646513 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f749c167f30 con 0x7f749c0ca630 2023-10-06T03:57:10.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.486+0000 7f74927fc700 1 -- 172.21.15.157:0/182646513 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f74940253f0 con 0x7f749c0ca630 2023-10-06T03:57:10.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.487+0000 7f74927fc700 1 --2- 172.21.15.157:0/182646513 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7480066f30 0x7f74800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:10.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.487+0000 7f74927fc700 1 -- 172.21.15.157:0/182646513 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f7494086290 con 0x7f749c0ca630 2023-10-06T03:57:10.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.487+0000 7f74a22d0700 1 --2- 172.21.15.157:0/182646513 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7480066f30 0x7f74800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:10.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.487+0000 7f74a22d0700 1 --2- 172.21.15.157:0/182646513 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7480066f30 0x7f74800693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7498002230 tx=0x7f7498009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:10.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.491+0000 7f74927fc700 1 -- 172.21.15.157:0/182646513 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7494053400 con 0x7f749c0ca630 2023-10-06T03:57:10.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:10.717+0000 7f74a4534700 1 -- 172.21.15.157:0/182646513 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f749c04bb40 con 0x7f749c0ca630 2023-10-06T03:57:11.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.366+0000 7f74927fc700 1 -- 172.21.15.157:0/182646513 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f7494056a20 con 0x7f749c0ca630 2023-10-06T03:57:11.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.376+0000 7f74a4534700 1 -- 172.21.15.157:0/182646513 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7480066f30 msgr2=0x7f74800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:11.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.376+0000 7f74a4534700 1 --2- 172.21.15.157:0/182646513 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7480066f30 0x7f74800693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7498002230 tx=0x7f7498009040 comp rx=0 tx=0).stop 2023-10-06T03:57:11.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.376+0000 7f74a4534700 1 -- 172.21.15.157:0/182646513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f749c0ca630 msgr2=0x7f749c0bf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:11.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.376+0000 7f74a4534700 1 --2- 172.21.15.157:0/182646513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f749c0ca630 0x7f749c0bf800 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f749400b3c0 tx=0x7f7494017000 comp rx=0 tx=0).stop 2023-10-06T03:57:11.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.377+0000 7f74a4534700 1 -- 172.21.15.157:0/182646513 shutdown_connections 2023-10-06T03:57:11.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.377+0000 7f74a4534700 1 --2- 172.21.15.157:0/182646513 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7480066f30 0x7f74800693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:11.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.377+0000 7f74a4534700 1 --2- 172.21.15.157:0/182646513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f749c0ca630 0x7f749c0bf800 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:11.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.377+0000 7f74a4534700 1 --2- 172.21.15.157:0/182646513 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f749c068920 0x7f749c1517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:11.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.377+0000 7f74a4534700 1 --2- 172.21.15.157:0/182646513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f749c0c52e0 0x7f749c0bf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:11.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.377+0000 7f74a4534700 1 -- 172.21.15.157:0/182646513 >> 172.21.15.157:0/182646513 conn(0x7f749c0692b0 msgr2=0x7f749c0c4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:11.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.378+0000 7f74a4534700 1 -- 172.21.15.157:0/182646513 shutdown_connections 2023-10-06T03:57:11.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.378+0000 7f74a4534700 1 -- 172.21.15.157:0/182646513 wait complete. 2023-10-06T03:57:11.380 INFO:teuthology.orchestra.run.smithi157.stderr:new fs with metadata pool 3 and data pool 4 2023-10-06T03:57:11.391 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:57:11.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.505+0000 7f8163014700 1 Processor -- start 2023-10-06T03:57:11.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.505+0000 7f8163014700 1 -- start start 2023-10-06T03:57:11.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.506+0000 7f8163014700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f815c067da0 0x7f815c0c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:11.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.506+0000 7f8163014700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f815c068170 0x7f815c153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:11.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.506+0000 7f8163014700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f815c0c1020 con 0x7f815c0c4cb0 2023-10-06T03:57:11.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.506+0000 7f8163014700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f815c157b00 con 0x7f815c067da0 2023-10-06T03:57:11.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.506+0000 7f8163014700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f815c0c0cf0 con 0x7f815c068170 2023-10-06T03:57:11.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.506+0000 7f8162012700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f815c067da0 0x7f815c0c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:11.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.506+0000 7f8162012700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f815c067da0 0x7f815c0c7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58660/0 (socket says 172.21.15.157:58660) 2023-10-06T03:57:11.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.506+0000 7f8162012700 1 -- 172.21.15.157:0/741056379 learned_addr learned my addr 172.21.15.157:0/741056379 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:11.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.506+0000 7f8161811700 1 --2- 172.21.15.157:0/741056379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f815c068170 0x7f815c153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:11.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.506+0000 7f8162012700 1 -- 172.21.15.157:0/741056379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f815c068170 msgr2=0x7f815c153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:11.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.506+0000 7f8162012700 1 --2- 172.21.15.157:0/741056379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f815c068170 0x7f815c153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:11.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.506+0000 7f8162012700 1 -- 172.21.15.157:0/741056379 >> v1:172.21.15.175:6789/0 conn(0x7f815c0c4cb0 legacy=0x7f815c150bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:11.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.506+0000 7f8162012700 1 -- 172.21.15.157:0/741056379 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f815c157da0 con 0x7f815c067da0 2023-10-06T03:57:11.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.506+0000 7f8162012700 1 --2- 172.21.15.157:0/741056379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f815c067da0 0x7f815c0c7930 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f81580099b0 tx=0x7f8158009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bff19cc21f3862c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:11.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.506+0000 7f8161010700 1 -- 172.21.15.157:0/741056379 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815800b860 con 0x7f815c067da0 2023-10-06T03:57:11.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.507+0000 7f8161010700 1 -- 172.21.15.157:0/741056379 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f815800b9c0 con 0x7f815c067da0 2023-10-06T03:57:11.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.507+0000 7f8161010700 1 -- 172.21.15.157:0/741056379 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815800bd60 con 0x7f815c067da0 2023-10-06T03:57:11.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.507+0000 7f8163014700 1 -- 172.21.15.157:0/741056379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f815c067da0 msgr2=0x7f815c0c7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:11.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.507+0000 7f8163014700 1 --2- 172.21.15.157:0/741056379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f815c067da0 0x7f815c0c7930 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f81580099b0 tx=0x7f8158009250 comp rx=0 tx=0).stop 2023-10-06T03:57:11.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.507+0000 7f8163014700 1 -- 172.21.15.157:0/741056379 shutdown_connections 2023-10-06T03:57:11.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.507+0000 7f8163014700 1 --2- 172.21.15.157:0/741056379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f815c067da0 0x7f815c0c7930 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:11.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.507+0000 7f8163014700 1 --2- 172.21.15.157:0/741056379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f815c068170 0x7f815c153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:11.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.507+0000 7f8163014700 1 -- 172.21.15.157:0/741056379 >> 172.21.15.157:0/741056379 conn(0x7f815c06c1f0 msgr2=0x7f815c06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:11.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.507+0000 7f8163014700 1 -- 172.21.15.157:0/741056379 shutdown_connections 2023-10-06T03:57:11.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.507+0000 7f8163014700 1 -- 172.21.15.157:0/741056379 wait complete. 2023-10-06T03:57:11.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.508+0000 7f8163014700 1 Processor -- start 2023-10-06T03:57:11.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.508+0000 7f8163014700 1 -- start start 2023-10-06T03:57:11.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.509+0000 7f8163014700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f815c067da0 0x7f815c14c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:11.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.509+0000 7f8163014700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f815c068170 0x7f815c0c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:11.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.509+0000 7f8163014700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f815c0c4cb0 0x7f815c0c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:11.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.509+0000 7f8163014700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815c0c6dc0 con 0x7f815c0c4cb0 2023-10-06T03:57:11.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.509+0000 7f8163014700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815c14a8a0 con 0x7f815c067da0 2023-10-06T03:57:11.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.509+0000 7f8163014700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f815c14ab60 con 0x7f815c068170 2023-10-06T03:57:11.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.509+0000 7f8162813700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f815c0c4cb0 0x7f815c0c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:11.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.509+0000 7f8162813700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f815c0c4cb0 0x7f815c0c9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58664/0 (socket says 172.21.15.157:58664) 2023-10-06T03:57:11.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.509+0000 7f8162813700 1 -- 172.21.15.157:0/2713075724 learned_addr learned my addr 172.21.15.157:0/2713075724 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:11.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.509+0000 7f8162012700 1 --2- 172.21.15.157:0/2713075724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f815c067da0 0x7f815c14c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:11.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.509+0000 7f8161811700 1 --2- 172.21.15.157:0/2713075724 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f815c068170 0x7f815c0c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:11.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.509+0000 7f8162813700 1 -- 172.21.15.157:0/2713075724 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f815c068170 msgr2=0x7f815c0c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:11.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.509+0000 7f8162813700 1 --2- 172.21.15.157:0/2713075724 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f815c068170 0x7f815c0c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:11.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.509+0000 7f8162813700 1 -- 172.21.15.157:0/2713075724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f815c067da0 msgr2=0x7f815c14c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:11.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.509+0000 7f8162813700 1 --2- 172.21.15.157:0/2713075724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f815c067da0 0x7f815c14c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:11.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.509+0000 7f8162813700 1 -- 172.21.15.157:0/2713075724 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81580094f0 con 0x7f815c0c4cb0 2023-10-06T03:57:11.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.509+0000 7f8162813700 1 --2- 172.21.15.157:0/2713075724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f815c0c4cb0 0x7f815c0c9490 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f815400c840 tx=0x7f8154001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:11.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.510+0000 7f81527fc700 1 -- 172.21.15.157:0/2713075724 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8154012630 con 0x7f815c0c4cb0 2023-10-06T03:57:11.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.510+0000 7f81527fc700 1 -- 172.21.15.157:0/2713075724 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81540127b0 con 0x7f815c0c4cb0 2023-10-06T03:57:11.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.510+0000 7f8163014700 1 -- 172.21.15.157:0/2713075724 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f815c167e70 con 0x7f815c0c4cb0 2023-10-06T03:57:11.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.510+0000 7f81527fc700 1 -- 172.21.15.157:0/2713075724 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8154012b70 con 0x7f815c0c4cb0 2023-10-06T03:57:11.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.510+0000 7f8163014700 1 -- 172.21.15.157:0/2713075724 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f815c14d150 con 0x7f815c0c4cb0 2023-10-06T03:57:11.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.511+0000 7f8163014700 1 -- 172.21.15.157:0/2713075724 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f815c14d760 con 0x7f815c0c4cb0 2023-10-06T03:57:11.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.511+0000 7f81527fc700 1 -- 172.21.15.157:0/2713075724 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f815401a3a0 con 0x7f815c0c4cb0 2023-10-06T03:57:11.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.511+0000 7f81527fc700 1 --2- 172.21.15.157:0/2713075724 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8140066f60 0x7f8140069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:11.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.511+0000 7f81527fc700 1 -- 172.21.15.157:0/2713075724 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8154084770 con 0x7f815c0c4cb0 2023-10-06T03:57:11.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.511+0000 7f8162012700 1 --2- 172.21.15.157:0/2713075724 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8140066f60 0x7f8140069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:11.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.512+0000 7f8162012700 1 --2- 172.21.15.157:0/2713075724 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8140066f60 0x7f8140069410 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f81580099b0 tx=0x7f8158017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:11.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.515+0000 7f81527fc700 1 -- 172.21.15.157:0/2713075724 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8154051880 con 0x7f815c0c4cb0 2023-10-06T03:57:11.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.703+0000 7f8163014700 1 -- 172.21.15.157:0/2713075724 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f815c0bf2d0 con 0x7f815c0c4cb0 2023-10-06T03:57:11.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.704+0000 7f81527fc700 1 -- 172.21.15.157:0/2713075724 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16927 (secure 0 0 0) 0x7f8154054ea0 con 0x7f815c0c4cb0 2023-10-06T03:57:11.706 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:11.706 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":20,"fsid":"28664a7a-fe30-4efe-be5a-6abf9318b700","created":"2023-10-06T03:56:51.887538+0000","modified":"2023-10-06T03:57:10.718465+0000","last_up_change":"2023-10-06T03:57:02.299440+0000","last_in_change":"2023-10-06T03:56:55.141505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:57:03.455121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:57:03.905661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:57:08.377186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:57:09.709582+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f677cac-e9f4-4608-acac-718b072ce984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":90640},{"type":"v1","addr":"172.21.15.157:6809","nonce":90640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":90640},{"type":"v1","addr":"172.21.15.157:6811","nonce":90640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":90640},{"type":"v1","addr":"172.21.15.157:6815","nonce":90640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":90640},{"type":"v1","addr":"172.21.15.157:6813","nonce":90640}]},"public_addr":"172.21.15.157:6809/90640","cluster_addr":"172.21.15.157:6811/90640","heartbeat_back_addr":"172.21.15.157:6815/90640","heartbeat_front_addr":"172.21.15.157:6813/90640","state":["exists","up"]},{"osd":1,"uuid":"7d771892-bb91-4e9b-9610-051861c11247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":90654},{"type":"v1","addr":"172.21.15.157:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":90654},{"type":"v1","addr":"172.21.15.157:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":90654},{"type":"v1","addr":"172.21.15.157:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":90654},{"type":"v1","addr":"172.21.15.157:6805","nonce":90654}]},"public_addr":"172.21.15.157:6801/90654","cluster_addr":"172.21.15.157:6803/90654","heartbeat_back_addr":"172.21.15.157:6807/90654","heartbeat_front_addr":"172.21.15.157:6805/90654","state":["exists","up"]},{"osd":2,"uuid":"5c1f7d4e-d185-47cf-ab6c-453b4e3009c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":90657},{"type":"v1","addr":"172.21.15.157:6825","nonce":90657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":90657},{"type":"v1","addr":"172.21.15.157:6827","nonce":90657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":90657},{"type":"v1","addr":"172.21.15.157:6831","nonce":90657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":90657},{"type":"v1","addr":"172.21.15.157:6829","nonce":90657}]},"public_addr":"172.21.15.157:6825/90657","cluster_addr":"172.21.15.157:6827/90657","heartbeat_back_addr":"172.21.15.157:6831/90657","heartbeat_front_addr":"172.21.15.157:6829/90657","state":["exists","up"]},{"osd":3,"uuid":"f6a1e825-43c7-4afd-8daf-e51d45afc75e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":90641},{"type":"v1","addr":"172.21.15.157:6817","nonce":90641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":90641},{"type":"v1","addr":"172.21.15.157:6819","nonce":90641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":90641},{"type":"v1","addr":"172.21.15.157:6823","nonce":90641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":90641},{"type":"v1","addr":"172.21.15.157:6821","nonce":90641}]},"public_addr":"172.21.15.157:6817/90641","cluster_addr":"172.21.15.157:6819/90641","heartbeat_back_addr":"172.21.15.157:6823/90641","heartbeat_front_addr":"172.21.15.157:6821/90641","state":["exists","up"]},{"osd":4,"uuid":"8ad6e3d8-9a31-4f85-9410-417b7ac5da97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":91261},{"type":"v1","addr":"172.21.15.175:6803","nonce":91261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":91261},{"type":"v1","addr":"172.21.15.175:6806","nonce":91261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":91261},{"type":"v1","addr":"172.21.15.175:6814","nonce":91261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":91261},{"type":"v1","addr":"172.21.15.175:6810","nonce":91261}]},"public_addr":"172.21.15.175:6803/91261","cluster_addr":"172.21.15.175:6806/91261","heartbeat_back_addr":"172.21.15.175:6814/91261","heartbeat_front_addr":"172.21.15.175:6810/91261","state":["exists","up"]},{"osd":5,"uuid":"f68aca2a-f182-4c00-8e21-241d09c02ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":91262},{"type":"v1","addr":"172.21.15.175:6817","nonce":91262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":91262},{"type":"v1","addr":"172.21.15.175:6819","nonce":91262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":91262},{"type":"v1","addr":"172.21.15.175:6823","nonce":91262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":91262},{"type":"v1","addr":"172.21.15.175:6821","nonce":91262}]},"public_addr":"172.21.15.175:6817/91262","cluster_addr":"172.21.15.175:6819/91262","heartbeat_back_addr":"172.21.15.175:6823/91262","heartbeat_front_addr":"172.21.15.175:6821/91262","state":["exists","up"]},{"osd":6,"uuid":"d867cd89-3430-40ca-862f-d2bea3de9d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":91263},{"type":"v1","addr":"172.21.15.175:6801","nonce":91263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6805","nonce":91263},{"type":"v1","addr":"172.21.15.175:6807","nonce":91263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6813","nonce":91263},{"type":"v1","addr":"172.21.15.175:6815","nonce":91263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":91263},{"type":"v1","addr":"172.21.15.175:6811","nonce":91263}]},"public_addr":"172.21.15.175:6801/91263","cluster_addr":"172.21.15.175:6807/91263","heartbeat_back_addr":"172.21.15.175:6815/91263","heartbeat_front_addr":"172.21.15.175:6811/91263","state":["exists","up"]},{"osd":7,"uuid":"930cd464-79f1-47cb-b4ff-188ccde74fff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":91306},{"type":"v1","addr":"172.21.15.175:6825","nonce":91306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":91306},{"type":"v1","addr":"172.21.15.175:6827","nonce":91306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":91306},{"type":"v1","addr":"172.21.15.175:6831","nonce":91306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":91306},{"type":"v1","addr":"172.21.15.175:6829","nonce":91306}]},"public_addr":"172.21.15.175:6825/91306","cluster_addr":"172.21.15.175:6827/91306","heartbeat_back_addr":"172.21.15.175:6831/91306","heartbeat_front_addr":"172.21.15.175:6829/91306","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.290673+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.056390+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.292903+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.275861+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.332095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.327453+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.317585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.309330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:57:11.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.706+0000 7f8163014700 1 -- 172.21.15.157:0/2713075724 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8140066f60 msgr2=0x7f8140069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:11.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.706+0000 7f8163014700 1 --2- 172.21.15.157:0/2713075724 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8140066f60 0x7f8140069410 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f81580099b0 tx=0x7f8158017000 comp rx=0 tx=0).stop 2023-10-06T03:57:11.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.706+0000 7f8163014700 1 -- 172.21.15.157:0/2713075724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f815c0c4cb0 msgr2=0x7f815c0c9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:11.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.706+0000 7f8163014700 1 --2- 172.21.15.157:0/2713075724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f815c0c4cb0 0x7f815c0c9490 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f815400c840 tx=0x7f8154001670 comp rx=0 tx=0).stop 2023-10-06T03:57:11.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.706+0000 7f8163014700 1 -- 172.21.15.157:0/2713075724 shutdown_connections 2023-10-06T03:57:11.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.706+0000 7f8163014700 1 --2- 172.21.15.157:0/2713075724 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8140066f60 0x7f8140069410 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:11.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.706+0000 7f8163014700 1 --2- 172.21.15.157:0/2713075724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f815c067da0 0x7f815c14c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:11.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.706+0000 7f8163014700 1 --2- 172.21.15.157:0/2713075724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f815c0c4cb0 0x7f815c0c9490 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:11.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.706+0000 7f8163014700 1 --2- 172.21.15.157:0/2713075724 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f815c068170 0x7f815c0c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:11.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.706+0000 7f8163014700 1 -- 172.21.15.157:0/2713075724 >> 172.21.15.157:0/2713075724 conn(0x7f815c06c1f0 msgr2=0x7f815c0631f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:11.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.706+0000 7f8163014700 1 -- 172.21.15.157:0/2713075724 shutdown_connections 2023-10-06T03:57:11.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.706+0000 7f8163014700 1 -- 172.21.15.157:0/2713075724 wait complete. 2023-10-06T03:57:11.720 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:57:11.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.832+0000 7f96a3b08700 1 Processor -- start 2023-10-06T03:57:11.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.833+0000 7f96a3b08700 1 -- start start 2023-10-06T03:57:11.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.833+0000 7f96a3b08700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f969c0ca630 0x7f969c0c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:11.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.833+0000 7f96a3b08700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f969c0c68f0 con 0x7f969c0688e0 2023-10-06T03:57:11.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.833+0000 7f96a3b08700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f969c06d9d0 con 0x7f969c06eed0 2023-10-06T03:57:11.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.833+0000 7f96a3b08700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969c0c59e0 con 0x7f969c0ca630 2023-10-06T03:57:11.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.833+0000 7f96a20a5700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f969c0688e0 0x7f969c159e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44130/0 (socket says 172.21.15.157:44130) 2023-10-06T03:57:11.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.833+0000 7f96a20a5700 1 -- 172.21.15.157:0/3635181563 learned_addr learned my addr 172.21.15.157:0/3635181563 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:11.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.834+0000 7f96a18a4700 1 --2- 172.21.15.157:0/3635181563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f969c0ca630 0x7f969c0c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:11.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.834+0000 7f96a08a2700 1 -- 172.21.15.157:0/3635181563 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2814482014 0 0) 0x7f969c0c68f0 con 0x7f969c0688e0 2023-10-06T03:57:11.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.834+0000 7f96a08a2700 1 -- 172.21.15.157:0/3635181563 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9688003570 con 0x7f969c0688e0 2023-10-06T03:57:11.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.834+0000 7f96a08a2700 1 -- 172.21.15.157:0/3635181563 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3201370500 0 0) 0x7f9688003570 con 0x7f969c0688e0 2023-10-06T03:57:11.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.834+0000 7f96a08a2700 1 -- 172.21.15.157:0/3635181563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f969c0ca630 msgr2=0x7f969c0c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:11.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.834+0000 7f96a08a2700 1 --2- 172.21.15.157:0/3635181563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f969c0ca630 0x7f969c0c6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:11.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.834+0000 7f96a08a2700 1 -- 172.21.15.157:0/3635181563 >> v1:172.21.15.175:6789/0 conn(0x7f969c06eed0 legacy=0x7f969c159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:11.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.834+0000 7f96a08a2700 1 -- 172.21.15.157:0/3635181563 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969c15a540 con 0x7f969c0688e0 2023-10-06T03:57:11.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.834+0000 7f96a08a2700 1 -- 172.21.15.157:0/3635181563 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f96940017f0 con 0x7f969c0688e0 2023-10-06T03:57:11.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.834+0000 7f96a08a2700 1 -- 172.21.15.157:0/3635181563 >> v1:172.21.15.157:6789/0 conn(0x7f969c0688e0 legacy=0x7f969c159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:11.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.834+0000 7f96a08a2700 1 --2- 172.21.15.157:0/3635181563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9688003a90 0x7f9688003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:11.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.834+0000 7f96a08a2700 1 --2- 172.21.15.157:0/3635181563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f969c0ca630 0x7f9688004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:11.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.834+0000 7f96a08a2700 1 --2- 172.21.15.157:0/3635181563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9688004d50 0x7f96880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:11.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.834+0000 7f96a08a2700 1 -- 172.21.15.157:0/3635181563 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c0c59e0 con 0x7f9688003a90 2023-10-06T03:57:11.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.834+0000 7f96a08a2700 1 -- 172.21.15.157:0/3635181563 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968800bbf0 con 0x7f9688004d50 2023-10-06T03:57:11.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.834+0000 7f96a08a2700 1 -- 172.21.15.157:0/3635181563 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f968800bd60 con 0x7f969c0ca630 2023-10-06T03:57:11.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.834+0000 7f96a18a4700 1 --2- 172.21.15.157:0/3635181563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9688003a90 0x7f9688003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:11.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.834+0000 7f96a20a5700 1 --2- 172.21.15.157:0/3635181563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9688004d50 0x7f96880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:11.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.834+0000 7f96a10a3700 1 --2- 172.21.15.157:0/3635181563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f969c0ca630 0x7f9688004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:11.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.834+0000 7f96a18a4700 1 -- 172.21.15.157:0/3635181563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f969c0ca630 msgr2=0x7f9688004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:11.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.834+0000 7f96a18a4700 1 --2- 172.21.15.157:0/3635181563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f969c0ca630 0x7f9688004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:11.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.834+0000 7f96a18a4700 1 -- 172.21.15.157:0/3635181563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9688004d50 msgr2=0x7f96880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:11.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.834+0000 7f96a18a4700 1 --2- 172.21.15.157:0/3635181563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9688004d50 0x7f96880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:11.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.834+0000 7f96a18a4700 1 -- 172.21.15.157:0/3635181563 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9688003570 con 0x7f9688003a90 2023-10-06T03:57:11.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.835+0000 7f96a18a4700 1 --2- 172.21.15.157:0/3635181563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9688003a90 0x7f9688003ec0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9698009c00 tx=0x7f96980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18de4bb470ff9ad6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:11.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.835+0000 7f96a08a2700 1 -- 172.21.15.157:0/3635181563 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969800e710 con 0x7f9688003a90 2023-10-06T03:57:11.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.835+0000 7f96a08a2700 1 -- 172.21.15.157:0/3635181563 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969800e870 con 0x7f9688003a90 2023-10-06T03:57:11.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.835+0000 7f96a08a2700 1 -- 172.21.15.157:0/3635181563 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969800ec30 con 0x7f9688003a90 2023-10-06T03:57:11.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.835+0000 7f96a3b08700 1 -- 172.21.15.157:0/3635181563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9688003a90 msgr2=0x7f9688003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:11.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.835+0000 7f96a3b08700 1 --2- 172.21.15.157:0/3635181563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9688003a90 0x7f9688003ec0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9698009c00 tx=0x7f96980091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:11.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.835+0000 7f96a20a5700 1 -- 172.21.15.157:0/3635181563 reap_dead start 2023-10-06T03:57:11.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.835+0000 7f96a3b08700 1 -- 172.21.15.157:0/3635181563 shutdown_connections 2023-10-06T03:57:11.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.835+0000 7f96a3b08700 1 -- 172.21.15.157:0/3635181563 >> 172.21.15.157:0/3635181563 conn(0x7f969c0692b0 msgr2=0x7f969c0696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:11.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.835+0000 7f96a3b08700 1 -- 172.21.15.157:0/3635181563 shutdown_connections 2023-10-06T03:57:11.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.835+0000 7f96a3b08700 1 -- 172.21.15.157:0/3635181563 wait complete. 2023-10-06T03:57:11.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.836+0000 7f96a3b08700 1 Processor -- start 2023-10-06T03:57:11.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.836+0000 7f96a3b08700 1 -- start start 2023-10-06T03:57:11.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.837+0000 7f96a3b08700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9688003a90 0x7f969c169650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:11.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.837+0000 7f96a3b08700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f969c169b90 0x7f969c150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:11.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.837+0000 7f96a3b08700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f969c151280 0x7f969c167490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:11.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.837+0000 7f96a3b08700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c0c9430 con 0x7f969c151280 2023-10-06T03:57:11.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.837+0000 7f96a3b08700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c15da20 con 0x7f969c169b90 2023-10-06T03:57:11.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.837+0000 7f96a3b08700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969c15c860 con 0x7f9688003a90 2023-10-06T03:57:11.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.837+0000 7f96a20a5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f969c151280 0x7f969c167490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:11.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.837+0000 7f96a20a5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f969c151280 0x7f969c167490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58684/0 (socket says 172.21.15.157:58684) 2023-10-06T03:57:11.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.837+0000 7f96a20a5700 1 -- 172.21.15.157:0/1223174405 learned_addr learned my addr 172.21.15.157:0/1223174405 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:11.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.837+0000 7f96a18a4700 1 --2- 172.21.15.157:0/1223174405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9688003a90 0x7f969c169650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:11.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.837+0000 7f96a10a3700 1 --2- 172.21.15.157:0/1223174405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f969c169b90 0x7f969c150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:11.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.838+0000 7f96a20a5700 1 -- 172.21.15.157:0/1223174405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9688003a90 msgr2=0x7f969c169650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:11.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.838+0000 7f96a20a5700 1 --2- 172.21.15.157:0/1223174405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9688003a90 0x7f969c169650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:11.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.838+0000 7f96a20a5700 1 -- 172.21.15.157:0/1223174405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f969c169b90 msgr2=0x7f969c150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:11.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.838+0000 7f96a20a5700 1 --2- 172.21.15.157:0/1223174405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f969c169b90 0x7f969c150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:11.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.838+0000 7f96a20a5700 1 -- 172.21.15.157:0/1223174405 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969c16a160 con 0x7f969c151280 2023-10-06T03:57:11.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.838+0000 7f96a20a5700 1 --2- 172.21.15.157:0/1223174405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f969c151280 0x7f969c167490 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9694001350 tx=0x7f96940021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:11.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.838+0000 7f9691ffb700 1 -- 172.21.15.157:0/1223174405 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9694001dd0 con 0x7f969c151280 2023-10-06T03:57:11.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.838+0000 7f9691ffb700 1 -- 172.21.15.157:0/1223174405 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969400ad40 con 0x7f969c151280 2023-10-06T03:57:11.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.838+0000 7f9691ffb700 1 -- 172.21.15.157:0/1223174405 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9694013640 con 0x7f969c151280 2023-10-06T03:57:11.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.838+0000 7f96a3b08700 1 -- 172.21.15.157:0/1223174405 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f969c15d6b0 con 0x7f969c151280 2023-10-06T03:57:11.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.838+0000 7f96a3b08700 1 -- 172.21.15.157:0/1223174405 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f969c0c3db0 con 0x7f969c151280 2023-10-06T03:57:11.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.839+0000 7f96a3b08700 1 -- 172.21.15.157:0/1223174405 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f969c15dbd0 con 0x7f969c151280 2023-10-06T03:57:11.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.839+0000 7f9691ffb700 1 -- 172.21.15.157:0/1223174405 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f96940139f0 con 0x7f969c151280 2023-10-06T03:57:11.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.839+0000 7f9691ffb700 1 --2- 172.21.15.157:0/1223174405 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9680066f60 0x7f9680069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:11.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.840+0000 7f9691ffb700 1 -- 172.21.15.157:0/1223174405 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f96940850d0 con 0x7f969c151280 2023-10-06T03:57:11.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.840+0000 7f96a18a4700 1 --2- 172.21.15.157:0/1223174405 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9680066f60 0x7f9680069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:11.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.840+0000 7f96a18a4700 1 --2- 172.21.15.157:0/1223174405 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9680066f60 0x7f9680069410 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f96980091e0 tx=0x7f969801a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:11.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:11.844+0000 7f9691ffb700 1 -- 172.21.15.157:0/1223174405 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f969c15dbd0 con 0x7f969c151280 2023-10-06T03:57:12.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.029+0000 7f96a3b08700 1 -- 172.21.15.157:0/1223174405 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f969c15db90 con 0x7f969c151280 2023-10-06T03:57:12.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.030+0000 7f9691ffb700 1 -- 172.21.15.157:0/1223174405 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16927 (secure 0 0 0) 0x7f96940522a0 con 0x7f969c151280 2023-10-06T03:57:12.032 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:12.032 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":20,"fsid":"28664a7a-fe30-4efe-be5a-6abf9318b700","created":"2023-10-06T03:56:51.887538+0000","modified":"2023-10-06T03:57:10.718465+0000","last_up_change":"2023-10-06T03:57:02.299440+0000","last_in_change":"2023-10-06T03:56:55.141505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:57:03.455121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:57:03.905661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:57:08.377186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:57:09.709582+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f677cac-e9f4-4608-acac-718b072ce984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":90640},{"type":"v1","addr":"172.21.15.157:6809","nonce":90640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":90640},{"type":"v1","addr":"172.21.15.157:6811","nonce":90640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":90640},{"type":"v1","addr":"172.21.15.157:6815","nonce":90640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":90640},{"type":"v1","addr":"172.21.15.157:6813","nonce":90640}]},"public_addr":"172.21.15.157:6809/90640","cluster_addr":"172.21.15.157:6811/90640","heartbeat_back_addr":"172.21.15.157:6815/90640","heartbeat_front_addr":"172.21.15.157:6813/90640","state":["exists","up"]},{"osd":1,"uuid":"7d771892-bb91-4e9b-9610-051861c11247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":90654},{"type":"v1","addr":"172.21.15.157:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":90654},{"type":"v1","addr":"172.21.15.157:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":90654},{"type":"v1","addr":"172.21.15.157:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":90654},{"type":"v1","addr":"172.21.15.157:6805","nonce":90654}]},"public_addr":"172.21.15.157:6801/90654","cluster_addr":"172.21.15.157:6803/90654","heartbeat_back_addr":"172.21.15.157:6807/90654","heartbeat_front_addr":"172.21.15.157:6805/90654","state":["exists","up"]},{"osd":2,"uuid":"5c1f7d4e-d185-47cf-ab6c-453b4e3009c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":90657},{"type":"v1","addr":"172.21.15.157:6825","nonce":90657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":90657},{"type":"v1","addr":"172.21.15.157:6827","nonce":90657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":90657},{"type":"v1","addr":"172.21.15.157:6831","nonce":90657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":90657},{"type":"v1","addr":"172.21.15.157:6829","nonce":90657}]},"public_addr":"172.21.15.157:6825/90657","cluster_addr":"172.21.15.157:6827/90657","heartbeat_back_addr":"172.21.15.157:6831/90657","heartbeat_front_addr":"172.21.15.157:6829/90657","state":["exists","up"]},{"osd":3,"uuid":"f6a1e825-43c7-4afd-8daf-e51d45afc75e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":90641},{"type":"v1","addr":"172.21.15.157:6817","nonce":90641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":90641},{"type":"v1","addr":"172.21.15.157:6819","nonce":90641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":90641},{"type":"v1","addr":"172.21.15.157:6823","nonce":90641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":90641},{"type":"v1","addr":"172.21.15.157:6821","nonce":90641}]},"public_addr":"172.21.15.157:6817/90641","cluster_addr":"172.21.15.157:6819/90641","heartbeat_back_addr":"172.21.15.157:6823/90641","heartbeat_front_addr":"172.21.15.157:6821/90641","state":["exists","up"]},{"osd":4,"uuid":"8ad6e3d8-9a31-4f85-9410-417b7ac5da97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":91261},{"type":"v1","addr":"172.21.15.175:6803","nonce":91261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":91261},{"type":"v1","addr":"172.21.15.175:6806","nonce":91261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":91261},{"type":"v1","addr":"172.21.15.175:6814","nonce":91261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":91261},{"type":"v1","addr":"172.21.15.175:6810","nonce":91261}]},"public_addr":"172.21.15.175:6803/91261","cluster_addr":"172.21.15.175:6806/91261","heartbeat_back_addr":"172.21.15.175:6814/91261","heartbeat_front_addr":"172.21.15.175:6810/91261","state":["exists","up"]},{"osd":5,"uuid":"f68aca2a-f182-4c00-8e21-241d09c02ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":91262},{"type":"v1","addr":"172.21.15.175:6817","nonce":91262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":91262},{"type":"v1","addr":"172.21.15.175:6819","nonce":91262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":91262},{"type":"v1","addr":"172.21.15.175:6823","nonce":91262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":91262},{"type":"v1","addr":"172.21.15.175:6821","nonce":91262}]},"public_addr":"172.21.15.175:6817/91262","cluster_addr":"172.21.15.175:6819/91262","heartbeat_back_addr":"172.21.15.175:6823/91262","heartbeat_front_addr":"172.21.15.175:6821/91262","state":["exists","up"]},{"osd":6,"uuid":"d867cd89-3430-40ca-862f-d2bea3de9d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":91263},{"type":"v1","addr":"172.21.15.175:6801","nonce":91263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6805","nonce":91263},{"type":"v1","addr":"172.21.15.175:6807","nonce":91263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6813","nonce":91263},{"type":"v1","addr":"172.21.15.175:6815","nonce":91263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":91263},{"type":"v1","addr":"172.21.15.175:6811","nonce":91263}]},"public_addr":"172.21.15.175:6801/91263","cluster_addr":"172.21.15.175:6807/91263","heartbeat_back_addr":"172.21.15.175:6815/91263","heartbeat_front_addr":"172.21.15.175:6811/91263","state":["exists","up"]},{"osd":7,"uuid":"930cd464-79f1-47cb-b4ff-188ccde74fff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":91306},{"type":"v1","addr":"172.21.15.175:6825","nonce":91306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":91306},{"type":"v1","addr":"172.21.15.175:6827","nonce":91306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":91306},{"type":"v1","addr":"172.21.15.175:6831","nonce":91306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":91306},{"type":"v1","addr":"172.21.15.175:6829","nonce":91306}]},"public_addr":"172.21.15.175:6825/91306","cluster_addr":"172.21.15.175:6827/91306","heartbeat_back_addr":"172.21.15.175:6831/91306","heartbeat_front_addr":"172.21.15.175:6829/91306","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.290673+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.056390+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.292903+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.275861+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.332095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.327453+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.317585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.309330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:57:12.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.032+0000 7f96a3b08700 1 -- 172.21.15.157:0/1223174405 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9680066f60 msgr2=0x7f9680069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:12.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.032+0000 7f96a3b08700 1 --2- 172.21.15.157:0/1223174405 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9680066f60 0x7f9680069410 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f96980091e0 tx=0x7f969801a000 comp rx=0 tx=0).stop 2023-10-06T03:57:12.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.032+0000 7f96a3b08700 1 -- 172.21.15.157:0/1223174405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f969c151280 msgr2=0x7f969c167490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:12.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.032+0000 7f96a3b08700 1 --2- 172.21.15.157:0/1223174405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f969c151280 0x7f969c167490 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9694001350 tx=0x7f96940021d0 comp rx=0 tx=0).stop 2023-10-06T03:57:12.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.032+0000 7f96a3b08700 1 -- 172.21.15.157:0/1223174405 shutdown_connections 2023-10-06T03:57:12.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.032+0000 7f96a3b08700 1 --2- 172.21.15.157:0/1223174405 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9680066f60 0x7f9680069410 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:12.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.032+0000 7f96a3b08700 1 --2- 172.21.15.157:0/1223174405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f969c151280 0x7f969c167490 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:12.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.032+0000 7f96a3b08700 1 --2- 172.21.15.157:0/1223174405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9688003a90 0x7f969c169650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:12.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.032+0000 7f96a3b08700 1 --2- 172.21.15.157:0/1223174405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f969c169b90 0x7f969c150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:12.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.032+0000 7f96a3b08700 1 -- 172.21.15.157:0/1223174405 >> 172.21.15.157:0/1223174405 conn(0x7f969c0692b0 msgr2=0x7f969c154940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:12.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.032+0000 7f96a3b08700 1 -- 172.21.15.157:0/1223174405 shutdown_connections 2023-10-06T03:57:12.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.032+0000 7f96a3b08700 1 -- 172.21.15.157:0/1223174405 wait complete. 2023-10-06T03:57:12.045 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T03:57:12.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.161+0000 7f861e8a4700 1 Processor -- start 2023-10-06T03:57:12.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.161+0000 7f861e8a4700 1 -- start start 2023-10-06T03:57:12.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.162+0000 7f861e8a4700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f86180b1d20 0x7f86180af370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:12.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.162+0000 7f861e8a4700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f86180af8b0 0x7f8618155db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:12.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.162+0000 7f861e8a4700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f861806e590 con 0x7f861806c260 2023-10-06T03:57:12.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.162+0000 7f861e8a4700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8618156d20 con 0x7f86180b1d20 2023-10-06T03:57:12.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.162+0000 7f861e8a4700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8618156a40 con 0x7f86180af8b0 2023-10-06T03:57:12.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.162+0000 7f8617fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f86180b1d20 0x7f86180af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:12.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.162+0000 7f861ce41700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f861806c260 0x7f861814fa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44144/0 (socket says 172.21.15.157:44144) 2023-10-06T03:57:12.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.162+0000 7f861ce41700 1 -- 172.21.15.157:0/907263223 learned_addr learned my addr 172.21.15.157:0/907263223 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:12.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.162+0000 7f86177fe700 1 --2- 172.21.15.157:0/907263223 >> v2:172.21.15.175:3300/0 conn(0x7f86180af8b0 0x7f8618155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:12.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.163+0000 7f8617fff700 1 -- 172.21.15.157:0/907263223 >> v2:172.21.15.175:3300/0 conn(0x7f86180af8b0 msgr2=0x7f8618155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:12.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.163+0000 7f8617fff700 1 --2- 172.21.15.157:0/907263223 >> v2:172.21.15.175:3300/0 conn(0x7f86180af8b0 0x7f8618155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:12.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.163+0000 7f8617fff700 1 -- 172.21.15.157:0/907263223 >> v1:172.21.15.157:6789/0 conn(0x7f861806c260 legacy=0x7f861814fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:12.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.163+0000 7f8617fff700 1 -- 172.21.15.157:0/907263223 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f861806d7d0 con 0x7f86180b1d20 2023-10-06T03:57:12.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.163+0000 7f8616ffd700 1 -- 172.21.15.157:0/907263223 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 518466321 0 0) 0x7f861806e590 con 0x7f861806c260 2023-10-06T03:57:12.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.163+0000 7f8617fff700 1 --2- 172.21.15.157:0/907263223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86180b1d20 0x7f86180af370 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f861806d190 tx=0x7f86080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9524fb2609376f83 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:12.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.163+0000 7f8616ffd700 1 -- 172.21.15.157:0/907263223 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8608011630 con 0x7f86180b1d20 2023-10-06T03:57:12.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.163+0000 7f8616ffd700 1 -- 172.21.15.157:0/907263223 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8608011790 con 0x7f86180b1d20 2023-10-06T03:57:12.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.163+0000 7f8616ffd700 1 -- 172.21.15.157:0/907263223 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8608011b30 con 0x7f86180b1d20 2023-10-06T03:57:12.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.163+0000 7f861e8a4700 1 -- 172.21.15.157:0/907263223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86180b1d20 msgr2=0x7f86180af370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:12.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.163+0000 7f861e8a4700 1 --2- 172.21.15.157:0/907263223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86180b1d20 0x7f86180af370 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f861806d190 tx=0x7f86080091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:12.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.163+0000 7f861e8a4700 1 -- 172.21.15.157:0/907263223 shutdown_connections 2023-10-06T03:57:12.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.163+0000 7f861e8a4700 1 --2- 172.21.15.157:0/907263223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86180b1d20 0x7f86180af370 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:12.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.163+0000 7f861e8a4700 1 --2- 172.21.15.157:0/907263223 >> v2:172.21.15.175:3300/0 conn(0x7f86180af8b0 0x7f8618155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:12.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.163+0000 7f861e8a4700 1 -- 172.21.15.157:0/907263223 >> 172.21.15.157:0/907263223 conn(0x7f86180cc6e0 msgr2=0x7f86180ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:12.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.164+0000 7f861e8a4700 1 -- 172.21.15.157:0/907263223 shutdown_connections 2023-10-06T03:57:12.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.164+0000 7f861e8a4700 1 -- 172.21.15.157:0/907263223 wait complete. 2023-10-06T03:57:12.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.165+0000 7f861e8a4700 1 Processor -- start 2023-10-06T03:57:12.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.165+0000 7f861e8a4700 1 -- start start 2023-10-06T03:57:12.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.165+0000 7f861e8a4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f861806c260 0x7f8618157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:12.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.165+0000 7f861e8a4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86180af8b0 0x7f8618152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:12.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.165+0000 7f861e8a4700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86180b1d20 0x7f8618152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:12.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.165+0000 7f861e8a4700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86181515c0 con 0x7f861806c260 2023-10-06T03:57:12.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.165+0000 7f861e8a4700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8618151b80 con 0x7f86180af8b0 2023-10-06T03:57:12.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.165+0000 7f861e8a4700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8618157e70 con 0x7f86180b1d20 2023-10-06T03:57:12.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.166+0000 7f8617fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f861806c260 0x7f8618157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:12.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.166+0000 7f8617fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f861806c260 0x7f8618157580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58708/0 (socket says 172.21.15.157:58708) 2023-10-06T03:57:12.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.166+0000 7f8617fff700 1 -- 172.21.15.157:0/900673189 learned_addr learned my addr 172.21.15.157:0/900673189 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:12.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.166+0000 7f861ce41700 1 --2- 172.21.15.157:0/900673189 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86180b1d20 0x7f8618152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:12.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.166+0000 7f86177fe700 1 --2- 172.21.15.157:0/900673189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86180af8b0 0x7f8618152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:12.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.166+0000 7f8617fff700 1 -- 172.21.15.157:0/900673189 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86180b1d20 msgr2=0x7f8618152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:12.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.166+0000 7f8617fff700 1 --2- 172.21.15.157:0/900673189 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86180b1d20 0x7f8618152550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:12.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.166+0000 7f8617fff700 1 -- 172.21.15.157:0/900673189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86180af8b0 msgr2=0x7f8618152010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:12.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.166+0000 7f8617fff700 1 --2- 172.21.15.157:0/900673189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86180af8b0 0x7f8618152010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:12.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.166+0000 7f8617fff700 1 -- 172.21.15.157:0/900673189 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8608009420 con 0x7f861806c260 2023-10-06T03:57:12.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.166+0000 7f86177fe700 1 --2- 172.21.15.157:0/900673189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86180af8b0 0x7f8618152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:12.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.166+0000 7f8617fff700 1 --2- 172.21.15.157:0/900673189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f861806c260 0x7f8618157580 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f861808a240 tx=0x7f8608017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:12.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.166+0000 7f85fffff700 1 -- 172.21.15.157:0/900673189 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8608011b10 con 0x7f861806c260 2023-10-06T03:57:12.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.166+0000 7f85fffff700 1 -- 172.21.15.157:0/900673189 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86080253f0 con 0x7f861806c260 2023-10-06T03:57:12.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.166+0000 7f861e8a4700 1 -- 172.21.15.157:0/900673189 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86181694d0 con 0x7f861806c260 2023-10-06T03:57:12.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.166+0000 7f85fffff700 1 -- 172.21.15.157:0/900673189 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8608025790 con 0x7f861806c260 2023-10-06T03:57:12.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.166+0000 7f861e8a4700 1 -- 172.21.15.157:0/900673189 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f861814c070 con 0x7f861806c260 2023-10-06T03:57:12.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.167+0000 7f861e8a4700 1 -- 172.21.15.157:0/900673189 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86180896c0 con 0x7f861806c260 2023-10-06T03:57:12.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.168+0000 7f85fffff700 1 -- 172.21.15.157:0/900673189 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8608025b80 con 0x7f861806c260 2023-10-06T03:57:12.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.168+0000 7f85fffff700 1 --2- 172.21.15.157:0/900673189 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8604066f60 0x7f8604069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:12.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.168+0000 7f85fffff700 1 -- 172.21.15.157:0/900673189 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8608085f20 con 0x7f861806c260 2023-10-06T03:57:12.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.168+0000 7f86177fe700 1 --2- 172.21.15.157:0/900673189 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8604066f60 0x7f8604069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:12.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.169+0000 7f86177fe700 1 --2- 172.21.15.157:0/900673189 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8604066f60 0x7f8604069410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8618150220 tx=0x7f860c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:12.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.172+0000 7f85fffff700 1 -- 172.21.15.157:0/900673189 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8608053030 con 0x7f861806c260 2023-10-06T03:57:12.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:12.401+0000 7f861e8a4700 1 -- 172.21.15.157:0/900673189 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f861806dab0 con 0x7f861806c260 2023-10-06T03:57:13.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.368+0000 7f85fffff700 1 -- 172.21.15.157:0/900673189 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7f86080150b0 con 0x7f861806c260 2023-10-06T03:57:13.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.370+0000 7f861e8a4700 1 -- 172.21.15.157:0/900673189 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8604066f60 msgr2=0x7f8604069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:13.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.370+0000 7f861e8a4700 1 --2- 172.21.15.157:0/900673189 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8604066f60 0x7f8604069410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8618150220 tx=0x7f860c008040 comp rx=0 tx=0).stop 2023-10-06T03:57:13.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.370+0000 7f861e8a4700 1 -- 172.21.15.157:0/900673189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f861806c260 msgr2=0x7f8618157580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:13.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.370+0000 7f861e8a4700 1 --2- 172.21.15.157:0/900673189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f861806c260 0x7f8618157580 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f861808a240 tx=0x7f8608017000 comp rx=0 tx=0).stop 2023-10-06T03:57:13.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.370+0000 7f861e8a4700 1 -- 172.21.15.157:0/900673189 shutdown_connections 2023-10-06T03:57:13.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.370+0000 7f861e8a4700 1 --2- 172.21.15.157:0/900673189 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8604066f60 0x7f8604069410 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:13.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.370+0000 7f861e8a4700 1 --2- 172.21.15.157:0/900673189 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86180b1d20 0x7f8618152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:13.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.370+0000 7f861e8a4700 1 --2- 172.21.15.157:0/900673189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f861806c260 0x7f8618157580 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:13.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.370+0000 7f861e8a4700 1 --2- 172.21.15.157:0/900673189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86180af8b0 0x7f8618152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:13.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.370+0000 7f861e8a4700 1 -- 172.21.15.157:0/900673189 >> 172.21.15.157:0/900673189 conn(0x7f86180cc6e0 msgr2=0x7f86180cbf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:13.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.370+0000 7f861e8a4700 1 -- 172.21.15.157:0/900673189 shutdown_connections 2023-10-06T03:57:13.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.370+0000 7f861e8a4700 1 -- 172.21.15.157:0/900673189 wait complete. 2023-10-06T03:57:13.385 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2023-10-06T03:57:13.411 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:57:13.429 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:57:13.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.506+0000 7f774fa29700 1 Processor -- start 2023-10-06T03:57:13.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.506+0000 7f774fa29700 1 -- start start 2023-10-06T03:57:13.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.507+0000 7f774fa29700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f774814dcd0 0x7f7748153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:13.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.507+0000 7f774fa29700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f77480c5c00 0x7f77480ca440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:13.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.507+0000 7f774fa29700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77480c5770 con 0x7f774814e0a0 2023-10-06T03:57:13.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.507+0000 7f774fa29700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7748154c80 con 0x7f77480c5c00 2023-10-06T03:57:13.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.507+0000 7f774fa29700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7748154200 con 0x7f774814dcd0 2023-10-06T03:57:13.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.507+0000 7f774cfc4700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f774814e0a0 0x7f77480c18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39906/0 (socket says 172.21.15.157:39906) 2023-10-06T03:57:13.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.507+0000 7f774cfc4700 1 -- 172.21.15.157:0/4052650466 learned_addr learned my addr 172.21.15.157:0/4052650466 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:13.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.507+0000 7f774d7c5700 1 --2- 172.21.15.157:0/4052650466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f774814dcd0 0x7f7748153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:13.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.508+0000 7f773ffff700 1 -- 172.21.15.157:0/4052650466 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4258588285 0 0) 0x7f77480c5770 con 0x7f774814e0a0 2023-10-06T03:57:13.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.508+0000 7f774dfc6700 1 --2- 172.21.15.157:0/4052650466 >> v2:172.21.15.175:3300/0 conn(0x7f77480c5c00 0x7f77480ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:13.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.508+0000 7f773ffff700 1 -- 172.21.15.157:0/4052650466 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7734003570 con 0x7f774814e0a0 2023-10-06T03:57:13.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.508+0000 7f773ffff700 1 -- 172.21.15.157:0/4052650466 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2601787204 0 0) 0x7f7734003570 con 0x7f774814e0a0 2023-10-06T03:57:13.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.508+0000 7f773ffff700 1 -- 172.21.15.157:0/4052650466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f774814dcd0 msgr2=0x7f7748153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:13.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.508+0000 7f773ffff700 1 --2- 172.21.15.157:0/4052650466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f774814dcd0 0x7f7748153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:13.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.508+0000 7f773ffff700 1 -- 172.21.15.157:0/4052650466 >> v2:172.21.15.175:3300/0 conn(0x7f77480c5c00 msgr2=0x7f77480ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:13.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.508+0000 7f773ffff700 1 --2- 172.21.15.157:0/4052650466 >> v2:172.21.15.175:3300/0 conn(0x7f77480c5c00 0x7f77480ca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:13.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.508+0000 7f773ffff700 1 -- 172.21.15.157:0/4052650466 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77481543e0 con 0x7f774814e0a0 2023-10-06T03:57:13.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.508+0000 7f773ffff700 1 -- 172.21.15.157:0/4052650466 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f7738001470 con 0x7f774814e0a0 2023-10-06T03:57:13.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.508+0000 7f774dfc6700 1 --2- 172.21.15.157:0/4052650466 >> v2:172.21.15.175:3300/0 conn(0x7f77480c5c00 0x7f77480ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:13.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.508+0000 7f773ffff700 1 -- 172.21.15.157:0/4052650466 >> v1:172.21.15.157:6789/0 conn(0x7f774814e0a0 legacy=0x7f77480c18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:13.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.508+0000 7f773ffff700 1 --2- 172.21.15.157:0/4052650466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7734003a60 0x7f7734003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:13.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.508+0000 7f773ffff700 1 --2- 172.21.15.157:0/4052650466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f774814dcd0 0x7f7734004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:13.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.508+0000 7f773ffff700 1 --2- 172.21.15.157:0/4052650466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7734004d50 0x7f77340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:13.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.508+0000 7f773ffff700 1 -- 172.21.15.157:0/4052650466 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7748154c80 con 0x7f7734004d50 2023-10-06T03:57:13.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.508+0000 7f773ffff700 1 -- 172.21.15.157:0/4052650466 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7748154200 con 0x7f7734003a60 2023-10-06T03:57:13.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.508+0000 7f773ffff700 1 -- 172.21.15.157:0/4052650466 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f773400bae0 con 0x7f774814dcd0 2023-10-06T03:57:13.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.508+0000 7f774dfc6700 1 --2- 172.21.15.157:0/4052650466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7734004d50 0x7f77340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:13.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.508+0000 7f774d7c5700 1 --2- 172.21.15.157:0/4052650466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7734003a60 0x7f7734003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:13.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.508+0000 7f774cfc4700 1 --2- 172.21.15.157:0/4052650466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f774814dcd0 0x7f7734004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:13.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.508+0000 7f774dfc6700 1 -- 172.21.15.157:0/4052650466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f774814dcd0 msgr2=0x7f7734004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:13.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.508+0000 7f774dfc6700 1 --2- 172.21.15.157:0/4052650466 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f774814dcd0 0x7f7734004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:13.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.508+0000 7f774dfc6700 1 -- 172.21.15.157:0/4052650466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7734003a60 msgr2=0x7f7734003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:13.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.508+0000 7f774dfc6700 1 --2- 172.21.15.157:0/4052650466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7734003a60 0x7f7734003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:13.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.508+0000 7f774dfc6700 1 -- 172.21.15.157:0/4052650466 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7734003570 con 0x7f7734004d50 2023-10-06T03:57:13.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.508+0000 7f774dfc6700 1 --2- 172.21.15.157:0/4052650466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7734004d50 0x7f77340051c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7740009ba0 tx=0x7f7740008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0ab6944a898424 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:13.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.509+0000 7f773ffff700 1 -- 172.21.15.157:0/4052650466 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774000c070 con 0x7f7734004d50 2023-10-06T03:57:13.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.509+0000 7f773ffff700 1 -- 172.21.15.157:0/4052650466 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f774000e5e0 con 0x7f7734004d50 2023-10-06T03:57:13.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.509+0000 7f773ffff700 1 -- 172.21.15.157:0/4052650466 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7740009510 con 0x7f7734004d50 2023-10-06T03:57:13.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.509+0000 7f774fa29700 1 -- 172.21.15.157:0/4052650466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7734004d50 msgr2=0x7f77340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:13.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.509+0000 7f774fa29700 1 --2- 172.21.15.157:0/4052650466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7734004d50 0x7f77340051c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7740009ba0 tx=0x7f7740008040 comp rx=0 tx=0).stop 2023-10-06T03:57:13.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.509+0000 7f774dfc6700 1 -- 172.21.15.157:0/4052650466 reap_dead start 2023-10-06T03:57:13.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.509+0000 7f774fa29700 1 -- 172.21.15.157:0/4052650466 shutdown_connections 2023-10-06T03:57:13.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.509+0000 7f774fa29700 1 -- 172.21.15.157:0/4052650466 >> 172.21.15.157:0/4052650466 conn(0x7f77480692b0 msgr2=0x7f77480696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:13.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.509+0000 7f774fa29700 1 -- 172.21.15.157:0/4052650466 shutdown_connections 2023-10-06T03:57:13.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.509+0000 7f774fa29700 1 -- 172.21.15.157:0/4052650466 wait complete. 2023-10-06T03:57:13.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.510+0000 7f774fa29700 1 Processor -- start 2023-10-06T03:57:13.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.510+0000 7f774fa29700 1 -- start start 2023-10-06T03:57:13.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.511+0000 7f774fa29700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7748149570 0x7f77481649a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:13.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.511+0000 7f774fa29700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7748149940 0x7f77480ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:13.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.511+0000 7f774fa29700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7748150230 0x7f77481506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:13.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.511+0000 7f774fa29700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7748164ee0 con 0x7f7748150230 2023-10-06T03:57:13.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.511+0000 7f774fa29700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77480cae00 con 0x7f7748149940 2023-10-06T03:57:13.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.511+0000 7f774fa29700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77480c9440 con 0x7f7748149570 2023-10-06T03:57:13.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.511+0000 7f774dfc6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7748150230 0x7f77481506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:13.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.511+0000 7f774dfc6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7748150230 0x7f77481506d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34182/0 (socket says 172.21.15.157:34182) 2023-10-06T03:57:13.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.511+0000 7f774dfc6700 1 -- 172.21.15.157:0/3900999176 learned_addr learned my addr 172.21.15.157:0/3900999176 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:13.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.511+0000 7f774d7c5700 1 --2- 172.21.15.157:0/3900999176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7748149570 0x7f77481649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:13.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.511+0000 7f774cfc4700 1 --2- 172.21.15.157:0/3900999176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7748149940 0x7f77480ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:13.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.512+0000 7f774dfc6700 1 -- 172.21.15.157:0/3900999176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7748149570 msgr2=0x7f77481649a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:13.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.512+0000 7f774dfc6700 1 --2- 172.21.15.157:0/3900999176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7748149570 0x7f77481649a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:13.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.512+0000 7f774dfc6700 1 -- 172.21.15.157:0/3900999176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7748149940 msgr2=0x7f77480ca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:13.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.512+0000 7f774dfc6700 1 --2- 172.21.15.157:0/3900999176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7748149940 0x7f77480ca8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:13.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.512+0000 7f774dfc6700 1 -- 172.21.15.157:0/3900999176 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77480cb050 con 0x7f7748150230 2023-10-06T03:57:13.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.512+0000 7f774dfc6700 1 --2- 172.21.15.157:0/3900999176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7748150230 0x7f77481506d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7740002000 tx=0x7f774001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:13.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.512+0000 7f773d7fa700 1 -- 172.21.15.157:0/3900999176 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7740009510 con 0x7f7748150230 2023-10-06T03:57:13.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.512+0000 7f773d7fa700 1 -- 172.21.15.157:0/3900999176 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7740013ad0 con 0x7f7748150230 2023-10-06T03:57:13.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.512+0000 7f774fa29700 1 -- 172.21.15.157:0/3900999176 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77480c96e0 con 0x7f7748150230 2023-10-06T03:57:13.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.512+0000 7f773d7fa700 1 -- 172.21.15.157:0/3900999176 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7740013e90 con 0x7f7748150230 2023-10-06T03:57:13.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.512+0000 7f774fa29700 1 -- 172.21.15.157:0/3900999176 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77480c9ca0 con 0x7f7748150230 2023-10-06T03:57:13.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.513+0000 7f774fa29700 1 -- 172.21.15.157:0/3900999176 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f774806a8f0 con 0x7f7748150230 2023-10-06T03:57:13.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.513+0000 7f773d7fa700 1 -- 172.21.15.157:0/3900999176 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f774001d650 con 0x7f7748150230 2023-10-06T03:57:13.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.513+0000 7f773d7fa700 1 --2- 172.21.15.157:0/3900999176 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f772c066f60 0x7f772c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:13.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.514+0000 7f773d7fa700 1 -- 172.21.15.157:0/3900999176 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7740000e40 con 0x7f7748150230 2023-10-06T03:57:13.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.514+0000 7f774d7c5700 1 --2- 172.21.15.157:0/3900999176 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f772c066f60 0x7f772c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:13.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.514+0000 7f774d7c5700 1 --2- 172.21.15.157:0/3900999176 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f772c066f60 0x7f772c069410 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f77440094d0 tx=0x7f774400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:13.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.518+0000 7f773d7fa700 1 -- 172.21.15.157:0/3900999176 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7740052ef0 con 0x7f7748150230 2023-10-06T03:57:13.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:13.742+0000 7f774fa29700 1 -- 172.21.15.157:0/3900999176 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f77480bd990 con 0x7f7748150230 2023-10-06T03:57:14.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.374+0000 7f773d7fa700 1 -- 172.21.15.157:0/3900999176 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7f7740056510 con 0x7f7748150230 2023-10-06T03:57:14.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.376+0000 7f774fa29700 1 -- 172.21.15.157:0/3900999176 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f772c066f60 msgr2=0x7f772c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:14.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.376+0000 7f774fa29700 1 --2- 172.21.15.157:0/3900999176 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f772c066f60 0x7f772c069410 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f77440094d0 tx=0x7f774400b040 comp rx=0 tx=0).stop 2023-10-06T03:57:14.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.376+0000 7f774fa29700 1 -- 172.21.15.157:0/3900999176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7748150230 msgr2=0x7f77481506d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:14.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.376+0000 7f774fa29700 1 --2- 172.21.15.157:0/3900999176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7748150230 0x7f77481506d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7740002000 tx=0x7f774001a000 comp rx=0 tx=0).stop 2023-10-06T03:57:14.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.377+0000 7f774fa29700 1 -- 172.21.15.157:0/3900999176 shutdown_connections 2023-10-06T03:57:14.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.377+0000 7f774fa29700 1 --2- 172.21.15.157:0/3900999176 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f772c066f60 0x7f772c069410 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:14.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.377+0000 7f774fa29700 1 --2- 172.21.15.157:0/3900999176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7748150230 0x7f77481506d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:14.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.377+0000 7f774fa29700 1 --2- 172.21.15.157:0/3900999176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7748149570 0x7f77481649a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:14.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.377+0000 7f774fa29700 1 --2- 172.21.15.157:0/3900999176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7748149940 0x7f77480ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:14.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.377+0000 7f774fa29700 1 -- 172.21.15.157:0/3900999176 >> 172.21.15.157:0/3900999176 conn(0x7f77480692b0 msgr2=0x7f774814aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:14.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.377+0000 7f774fa29700 1 -- 172.21.15.157:0/3900999176 shutdown_connections 2023-10-06T03:57:14.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.377+0000 7f774fa29700 1 -- 172.21.15.157:0/3900999176 wait complete. 2023-10-06T03:57:14.390 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T03:57:14.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.547+0000 7fdfa420c700 1 Processor -- start 2023-10-06T03:57:14.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.547+0000 7fdfa420c700 1 -- start start 2023-10-06T03:57:14.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.548+0000 7fdfa420c700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdf9c14dcd0 0x7fdf9c153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:14.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.548+0000 7fdfa420c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdf9c0c5c00 0x7fdf9c0ca440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:14.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.548+0000 7fdfa420c700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf9c0c5770 con 0x7fdf9c14e0a0 2023-10-06T03:57:14.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.548+0000 7fdfa420c700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf9c154c80 con 0x7fdf9c0c5c00 2023-10-06T03:57:14.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.548+0000 7fdfa420c700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf9c154200 con 0x7fdf9c14dcd0 2023-10-06T03:57:14.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.548+0000 7fdfa17a7700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fdf9c14e0a0 0x7fdf9c0c18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39918/0 (socket says 172.21.15.157:39918) 2023-10-06T03:57:14.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.548+0000 7fdfa27a9700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdf9c0c5c00 0x7fdf9c0ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:14.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.548+0000 7fdfa17a7700 1 -- 172.21.15.157:0/2709567411 learned_addr learned my addr 172.21.15.157:0/2709567411 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:14.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.548+0000 7fdfa1fa8700 1 --2- 172.21.15.157:0/2709567411 >> v2:172.21.15.175:3300/0 conn(0x7fdf9c14dcd0 0x7fdf9c153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:14.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.548+0000 7fdfa27a9700 1 -- 172.21.15.157:0/2709567411 >> v2:172.21.15.175:3300/0 conn(0x7fdf9c14dcd0 msgr2=0x7fdf9c153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:14.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.548+0000 7fdfa27a9700 1 --2- 172.21.15.157:0/2709567411 >> v2:172.21.15.175:3300/0 conn(0x7fdf9c14dcd0 0x7fdf9c153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:14.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.548+0000 7fdfa27a9700 1 -- 172.21.15.157:0/2709567411 >> v1:172.21.15.157:6789/0 conn(0x7fdf9c14e0a0 legacy=0x7fdf9c0c18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:14.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.548+0000 7fdfa27a9700 1 -- 172.21.15.157:0/2709567411 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9c1543e0 con 0x7fdf9c0c5c00 2023-10-06T03:57:14.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.548+0000 7fdfa0fa6700 1 -- 172.21.15.157:0/2709567411 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2378642028 0 0) 0x7fdf9c0c5770 con 0x7fdf9c14e0a0 2023-10-06T03:57:14.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.548+0000 7fdfa27a9700 1 --2- 172.21.15.157:0/2709567411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf9c0c5c00 0x7fdf9c0ca440 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c06cd20 tx=0x7fdf94009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ffdf89895a244d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:14.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.548+0000 7fdfa0fa6700 1 -- 172.21.15.157:0/2709567411 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9400e070 con 0x7fdf9c0c5c00 2023-10-06T03:57:14.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.548+0000 7fdfa0fa6700 1 -- 172.21.15.157:0/2709567411 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9400bcc0 con 0x7fdf9c0c5c00 2023-10-06T03:57:14.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.548+0000 7fdfa0fa6700 1 -- 172.21.15.157:0/2709567411 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94012680 con 0x7fdf9c0c5c00 2023-10-06T03:57:14.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.549+0000 7fdfa420c700 1 -- 172.21.15.157:0/2709567411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf9c0c5c00 msgr2=0x7fdf9c0ca440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:14.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.549+0000 7fdfa420c700 1 --2- 172.21.15.157:0/2709567411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf9c0c5c00 0x7fdf9c0ca440 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c06cd20 tx=0x7fdf94009780 comp rx=0 tx=0).stop 2023-10-06T03:57:14.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.549+0000 7fdfa420c700 1 -- 172.21.15.157:0/2709567411 shutdown_connections 2023-10-06T03:57:14.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.549+0000 7fdfa420c700 1 --2- 172.21.15.157:0/2709567411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf9c0c5c00 0x7fdf9c0ca440 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:14.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.549+0000 7fdfa420c700 1 --2- 172.21.15.157:0/2709567411 >> v2:172.21.15.175:3300/0 conn(0x7fdf9c14dcd0 0x7fdf9c153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:14.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.549+0000 7fdfa420c700 1 -- 172.21.15.157:0/2709567411 >> 172.21.15.157:0/2709567411 conn(0x7fdf9c0692b0 msgr2=0x7fdf9c0696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:14.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.549+0000 7fdfa420c700 1 -- 172.21.15.157:0/2709567411 shutdown_connections 2023-10-06T03:57:14.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.549+0000 7fdfa420c700 1 -- 172.21.15.157:0/2709567411 wait complete. 2023-10-06T03:57:14.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.550+0000 7fdfa420c700 1 Processor -- start 2023-10-06T03:57:14.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.550+0000 7fdfa420c700 1 -- start start 2023-10-06T03:57:14.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.551+0000 7fdfa420c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf9c0c5c00 0x7fdf9c0cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:14.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.551+0000 7fdfa420c700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf9c14dcd0 0x7fdf9c1520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:14.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.551+0000 7fdfa420c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf9c14e0a0 0x7fdf9c152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:14.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.551+0000 7fdfa420c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c0c8810 con 0x7fdf9c0c5c00 2023-10-06T03:57:14.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.551+0000 7fdfa420c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c1690c0 con 0x7fdf9c14e0a0 2023-10-06T03:57:14.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.551+0000 7fdfa420c700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c0cb210 con 0x7fdf9c14dcd0 2023-10-06T03:57:14.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.551+0000 7fdfa1fa8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf9c0c5c00 0x7fdf9c0cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:14.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.551+0000 7fdfa1fa8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf9c0c5c00 0x7fdf9c0cacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34192/0 (socket says 172.21.15.157:34192) 2023-10-06T03:57:14.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.551+0000 7fdfa1fa8700 1 -- 172.21.15.157:0/3238688521 learned_addr learned my addr 172.21.15.157:0/3238688521 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:14.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.551+0000 7fdfa27a9700 1 --2- 172.21.15.157:0/3238688521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf9c14e0a0 0x7fdf9c152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:14.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.551+0000 7fdfa17a7700 1 --2- 172.21.15.157:0/3238688521 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf9c14dcd0 0x7fdf9c1520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:14.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.551+0000 7fdfa1fa8700 1 -- 172.21.15.157:0/3238688521 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf9c14dcd0 msgr2=0x7fdf9c1520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:14.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.551+0000 7fdfa1fa8700 1 --2- 172.21.15.157:0/3238688521 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf9c14dcd0 0x7fdf9c1520d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:14.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.551+0000 7fdfa1fa8700 1 -- 172.21.15.157:0/3238688521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf9c14e0a0 msgr2=0x7fdf9c152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:14.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.551+0000 7fdfa1fa8700 1 --2- 172.21.15.157:0/3238688521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf9c14e0a0 0x7fdf9c152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:14.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.551+0000 7fdfa1fa8700 1 -- 172.21.15.157:0/3238688521 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf94009a20 con 0x7fdf9c0c5c00 2023-10-06T03:57:14.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.551+0000 7fdfa27a9700 1 --2- 172.21.15.157:0/3238688521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf9c14e0a0 0x7fdf9c152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:57:14.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.551+0000 7fdfa17a7700 1 --2- 172.21.15.157:0/3238688521 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf9c14dcd0 0x7fdf9c1520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:57:14.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.551+0000 7fdfa1fa8700 1 --2- 172.21.15.157:0/3238688521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf9c0c5c00 0x7fdf9c0cacd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdf9800d890 tx=0x7fdf98008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:14.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.551+0000 7fdf91ffb700 1 -- 172.21.15.157:0/3238688521 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9800b600 con 0x7fdf9c0c5c00 2023-10-06T03:57:14.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.551+0000 7fdf91ffb700 1 -- 172.21.15.157:0/3238688521 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9800b760 con 0x7fdf9c0c5c00 2023-10-06T03:57:14.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.551+0000 7fdfa420c700 1 -- 172.21.15.157:0/3238688521 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf9c169360 con 0x7fdf9c0c5c00 2023-10-06T03:57:14.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.552+0000 7fdf91ffb700 1 -- 172.21.15.157:0/3238688521 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9800bae0 con 0x7fdf9c0c5c00 2023-10-06T03:57:14.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.552+0000 7fdfa420c700 1 -- 172.21.15.157:0/3238688521 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf9c169920 con 0x7fdf9c0c5c00 2023-10-06T03:57:14.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.553+0000 7fdfa420c700 1 -- 172.21.15.157:0/3238688521 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf9c06a8f0 con 0x7fdf9c0c5c00 2023-10-06T03:57:14.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.553+0000 7fdf91ffb700 1 -- 172.21.15.157:0/3238688521 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdf9801e460 con 0x7fdf9c0c5c00 2023-10-06T03:57:14.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.553+0000 7fdf91ffb700 1 --2- 172.21.15.157:0/3238688521 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdf80066f30 0x7fdf800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:14.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.553+0000 7fdf91ffb700 1 -- 172.21.15.157:0/3238688521 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdf98084d50 con 0x7fdf9c0c5c00 2023-10-06T03:57:14.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.553+0000 7fdfa17a7700 1 --2- 172.21.15.157:0/3238688521 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdf80066f30 0x7fdf800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:14.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.554+0000 7fdfa17a7700 1 --2- 172.21.15.157:0/3238688521 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdf80066f30 0x7fdf800693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c14af70 tx=0x7fdf8c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:14.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.557+0000 7fdf91ffb700 1 -- 172.21.15.157:0/3238688521 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdf98088030 con 0x7fdf9c0c5c00 2023-10-06T03:57:14.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:14.782+0000 7fdfa420c700 1 -- 172.21.15.157:0/3238688521 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fdf9c04bb40 con 0x7fdf9c0c5c00 2023-10-06T03:57:15.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.381+0000 7fdf91ffb700 1 -- 172.21.15.157:0/3238688521 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdf980558f0 con 0x7fdf9c0c5c00 2023-10-06T03:57:15.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.383+0000 7fdfa420c700 1 -- 172.21.15.157:0/3238688521 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdf80066f30 msgr2=0x7fdf800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:15.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.383+0000 7fdfa420c700 1 --2- 172.21.15.157:0/3238688521 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdf80066f30 0x7fdf800693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c14af70 tx=0x7fdf8c009040 comp rx=0 tx=0).stop 2023-10-06T03:57:15.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.383+0000 7fdfa420c700 1 -- 172.21.15.157:0/3238688521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf9c0c5c00 msgr2=0x7fdf9c0cacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:15.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.383+0000 7fdfa420c700 1 --2- 172.21.15.157:0/3238688521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf9c0c5c00 0x7fdf9c0cacd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdf9800d890 tx=0x7fdf98008040 comp rx=0 tx=0).stop 2023-10-06T03:57:15.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.383+0000 7fdfa420c700 1 -- 172.21.15.157:0/3238688521 shutdown_connections 2023-10-06T03:57:15.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.383+0000 7fdfa420c700 1 --2- 172.21.15.157:0/3238688521 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdf80066f30 0x7fdf800693e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:15.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.383+0000 7fdfa420c700 1 --2- 172.21.15.157:0/3238688521 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf9c14e0a0 0x7fdf9c152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:15.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.383+0000 7fdfa420c700 1 --2- 172.21.15.157:0/3238688521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf9c0c5c00 0x7fdf9c0cacd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:15.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.383+0000 7fdfa420c700 1 --2- 172.21.15.157:0/3238688521 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf9c14dcd0 0x7fdf9c1520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:15.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.383+0000 7fdfa420c700 1 -- 172.21.15.157:0/3238688521 >> 172.21.15.157:0/3238688521 conn(0x7fdf9c0692b0 msgr2=0x7fdf9c14a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:15.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.383+0000 7fdfa420c700 1 -- 172.21.15.157:0/3238688521 shutdown_connections 2023-10-06T03:57:15.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.383+0000 7fdfa420c700 1 -- 172.21.15.157:0/3238688521 wait complete. 2023-10-06T03:57:15.398 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-10-06T03:57:15.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.516+0000 7f0214b34700 1 Processor -- start 2023-10-06T03:57:15.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.516+0000 7f0214b34700 1 -- start start 2023-10-06T03:57:15.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.517+0000 7f0214b34700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02100ca450 0x7f02100c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:15.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.517+0000 7f0214b34700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0210068600 0x7f02100c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:15.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.517+0000 7f0214b34700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02100c6410 con 0x7f02100c5250 2023-10-06T03:57:15.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.517+0000 7f0214b34700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f021006d300 con 0x7f0210068600 2023-10-06T03:57:15.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.517+0000 7f0214b34700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02100675a0 con 0x7f02100ca450 2023-10-06T03:57:15.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.517+0000 7f020effd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0210068600 0x7f02100c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:15.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.517+0000 7f020effd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0210068600 0x7f02100c6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34206/0 (socket says 172.21.15.157:34206) 2023-10-06T03:57:15.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.517+0000 7f020effd700 1 -- 172.21.15.157:0/1708011693 learned_addr learned my addr 172.21.15.157:0/1708011693 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:15.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.517+0000 7f020f7fe700 1 --2- 172.21.15.157:0/1708011693 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02100ca450 0x7f02100c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:15.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.517+0000 7f020effd700 1 -- 172.21.15.157:0/1708011693 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02100ca450 msgr2=0x7f02100c68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:15.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.517+0000 7f020effd700 1 --2- 172.21.15.157:0/1708011693 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02100ca450 0x7f02100c68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:15.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.517+0000 7f020effd700 1 -- 172.21.15.157:0/1708011693 >> v1:172.21.15.175:6789/0 conn(0x7f02100c5250 legacy=0x7f0210156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:15.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.517+0000 7f020effd700 1 -- 172.21.15.157:0/1708011693 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02100c60c0 con 0x7f0210068600 2023-10-06T03:57:15.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.517+0000 7f020effd700 1 --2- 172.21.15.157:0/1708011693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0210068600 0x7f02100c6de0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f02040099b0 tx=0x7f0204009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90dc88da39c97d26 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:15.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.518+0000 7f020e7fc700 1 -- 172.21.15.157:0/1708011693 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020400b860 con 0x7f0210068600 2023-10-06T03:57:15.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.518+0000 7f020e7fc700 1 -- 172.21.15.157:0/1708011693 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f020400b9c0 con 0x7f0210068600 2023-10-06T03:57:15.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.518+0000 7f020e7fc700 1 -- 172.21.15.157:0/1708011693 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020400bd60 con 0x7f0210068600 2023-10-06T03:57:15.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.518+0000 7f0214b34700 1 -- 172.21.15.157:0/1708011693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0210068600 msgr2=0x7f02100c6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:15.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.518+0000 7f0214b34700 1 --2- 172.21.15.157:0/1708011693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0210068600 0x7f02100c6de0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f02040099b0 tx=0x7f0204009250 comp rx=0 tx=0).stop 2023-10-06T03:57:15.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.518+0000 7f0214b34700 1 -- 172.21.15.157:0/1708011693 shutdown_connections 2023-10-06T03:57:15.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.518+0000 7f0214b34700 1 --2- 172.21.15.157:0/1708011693 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02100ca450 0x7f02100c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:15.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.518+0000 7f0214b34700 1 --2- 172.21.15.157:0/1708011693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0210068600 0x7f02100c6de0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:15.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.518+0000 7f0214b34700 1 -- 172.21.15.157:0/1708011693 >> 172.21.15.157:0/1708011693 conn(0x7f0210069100 msgr2=0x7f0210069500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:15.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.518+0000 7f0214b34700 1 -- 172.21.15.157:0/1708011693 shutdown_connections 2023-10-06T03:57:15.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.518+0000 7f0214b34700 1 -- 172.21.15.157:0/1708011693 wait complete. 2023-10-06T03:57:15.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.519+0000 7f0214b34700 1 Processor -- start 2023-10-06T03:57:15.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.519+0000 7f0214b34700 1 -- start start 2023-10-06T03:57:15.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.520+0000 7f0214b34700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0210068600 0x7f02100c1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:15.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.520+0000 7f0214b34700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02100c5250 0x7f02101693d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:15.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.520+0000 7f0214b34700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02100ca450 0x7f0210169910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:15.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.520+0000 7f0214b34700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0210169e50 con 0x7f02100ca450 2023-10-06T03:57:15.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.520+0000 7f0214b34700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02100c4100 con 0x7f02100c5250 2023-10-06T03:57:15.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.520+0000 7f0214b34700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f021015ad00 con 0x7f0210068600 2023-10-06T03:57:15.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.520+0000 7f020f7fe700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0210068600 0x7f02100c1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:15.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.520+0000 7f020ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02100ca450 0x7f0210169910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:15.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.520+0000 7f020f7fe700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0210068600 0x7f02100c1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:49904/0 (socket says 172.21.15.157:49904) 2023-10-06T03:57:15.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.520+0000 7f020ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02100ca450 0x7f0210169910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34210/0 (socket says 172.21.15.157:34210) 2023-10-06T03:57:15.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.520+0000 7f020f7fe700 1 -- 172.21.15.157:0/720122602 learned_addr learned my addr 172.21.15.157:0/720122602 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:15.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.520+0000 7f020effd700 1 --2- 172.21.15.157:0/720122602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02100c5250 0x7f02101693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:15.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.521+0000 7f020ffff700 1 -- 172.21.15.157:0/720122602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0210068600 msgr2=0x7f02100c1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:15.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.521+0000 7f020ffff700 1 --2- 172.21.15.157:0/720122602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0210068600 0x7f02100c1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:15.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.521+0000 7f020ffff700 1 -- 172.21.15.157:0/720122602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02100c5250 msgr2=0x7f02101693d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:15.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.521+0000 7f020ffff700 1 --2- 172.21.15.157:0/720122602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02100c5250 0x7f02101693d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:15.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.521+0000 7f020ffff700 1 -- 172.21.15.157:0/720122602 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02040094f0 con 0x7f02100ca450 2023-10-06T03:57:15.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.521+0000 7f020ffff700 1 --2- 172.21.15.157:0/720122602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02100ca450 0x7f0210169910 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f020800c4e0 tx=0x7f0208007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:15.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.521+0000 7f01f77fe700 1 -- 172.21.15.157:0/720122602 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0208011590 con 0x7f02100ca450 2023-10-06T03:57:15.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.521+0000 7f01f77fe700 1 -- 172.21.15.157:0/720122602 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02080116f0 con 0x7f02100ca450 2023-10-06T03:57:15.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.521+0000 7f0214b34700 1 -- 172.21.15.157:0/720122602 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02100c4330 con 0x7f02100ca450 2023-10-06T03:57:15.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.521+0000 7f01f77fe700 1 -- 172.21.15.157:0/720122602 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0208011ab0 con 0x7f02100ca450 2023-10-06T03:57:15.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.521+0000 7f0214b34700 1 -- 172.21.15.157:0/720122602 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f021015a920 con 0x7f02100ca450 2023-10-06T03:57:15.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.522+0000 7f0214b34700 1 -- 172.21.15.157:0/720122602 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f021015aeb0 con 0x7f02100ca450 2023-10-06T03:57:15.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.522+0000 7f01f77fe700 1 -- 172.21.15.157:0/720122602 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f020801b3b0 con 0x7f02100ca450 2023-10-06T03:57:15.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.523+0000 7f01f77fe700 1 --2- 172.21.15.157:0/720122602 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f01fc066f30 0x7f01fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:15.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.523+0000 7f01f77fe700 1 -- 172.21.15.157:0/720122602 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0208083a40 con 0x7f02100ca450 2023-10-06T03:57:15.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.523+0000 7f020f7fe700 1 --2- 172.21.15.157:0/720122602 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f01fc066f30 0x7f01fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:15.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.523+0000 7f020f7fe700 1 --2- 172.21.15.157:0/720122602 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f01fc066f30 0x7f01fc0693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0200009fd0 tx=0x7f0200009880 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:15.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.527+0000 7f01f77fe700 1 -- 172.21.15.157:0/720122602 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f021015aeb0 con 0x7f02100ca450 2023-10-06T03:57:15.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:15.753+0000 7f0214b34700 1 -- 172.21.15.157:0/720122602 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f021015aeb0 con 0x7f02100ca450 2023-10-06T03:57:16.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.385+0000 7f01f77fe700 1 -- 172.21.15.157:0/720122602 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v8) v1 ==== 115+0+0 (secure 0 0 0) 0x7f021015aeb0 con 0x7f02100ca450 2023-10-06T03:57:16.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.387+0000 7f0214b34700 1 -- 172.21.15.157:0/720122602 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f01fc066f30 msgr2=0x7f01fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:16.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.387+0000 7f0214b34700 1 --2- 172.21.15.157:0/720122602 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f01fc066f30 0x7f01fc0693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0200009fd0 tx=0x7f0200009880 comp rx=0 tx=0).stop 2023-10-06T03:57:16.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.387+0000 7f0214b34700 1 -- 172.21.15.157:0/720122602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02100ca450 msgr2=0x7f0210169910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:16.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.387+0000 7f0214b34700 1 --2- 172.21.15.157:0/720122602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02100ca450 0x7f0210169910 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f020800c4e0 tx=0x7f0208007e60 comp rx=0 tx=0).stop 2023-10-06T03:57:16.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.387+0000 7f0214b34700 1 -- 172.21.15.157:0/720122602 shutdown_connections 2023-10-06T03:57:16.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.387+0000 7f0214b34700 1 --2- 172.21.15.157:0/720122602 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f01fc066f30 0x7f01fc0693e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:16.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.387+0000 7f0214b34700 1 --2- 172.21.15.157:0/720122602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02100ca450 0x7f0210169910 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:16.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.387+0000 7f0214b34700 1 --2- 172.21.15.157:0/720122602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0210068600 0x7f02100c1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:16.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.387+0000 7f0214b34700 1 --2- 172.21.15.157:0/720122602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02100c5250 0x7f02101693d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:16.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.387+0000 7f0214b34700 1 -- 172.21.15.157:0/720122602 >> 172.21.15.157:0/720122602 conn(0x7f0210069100 msgr2=0x7f02100c9490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:16.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.387+0000 7f0214b34700 1 -- 172.21.15.157:0/720122602 shutdown_connections 2023-10-06T03:57:16.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.387+0000 7f0214b34700 1 -- 172.21.15.157:0/720122602 wait complete. 2023-10-06T03:57:16.401 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:57:16.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.515+0000 7f6b2db4e700 1 Processor -- start 2023-10-06T03:57:16.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.515+0000 7f6b2db4e700 1 -- start start 2023-10-06T03:57:16.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.516+0000 7f6b2db4e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6b2806cbb0 0x7f6b2806d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:16.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.516+0000 7f6b2db4e700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6b2806c1f0 0x7f6b2806da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:16.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.516+0000 7f6b2db4e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b280c9780 con 0x7f6b2806cf80 2023-10-06T03:57:16.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.516+0000 7f6b2db4e700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b280c8c40 con 0x7f6b2806c1f0 2023-10-06T03:57:16.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.516+0000 7f6b2db4e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b2806e0c0 con 0x7f6b2806cbb0 2023-10-06T03:57:16.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.516+0000 7f6b26ffd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6b2806c1f0 0x7f6b2806da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:16.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.516+0000 7f6b26ffd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6b2806c1f0 0x7f6b2806da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34226/0 (socket says 172.21.15.157:34226) 2023-10-06T03:57:16.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.516+0000 7f6b26ffd700 1 -- 172.21.15.157:0/3455745213 learned_addr learned my addr 172.21.15.157:0/3455745213 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:16.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.516+0000 7f6b26ffd700 1 -- 172.21.15.157:0/3455745213 >> v2:172.21.15.175:3300/0 conn(0x7f6b2806cbb0 msgr2=0x7f6b2806d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:16.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.516+0000 7f6b277fe700 1 --2- 172.21.15.157:0/3455745213 >> v2:172.21.15.175:3300/0 conn(0x7f6b2806cbb0 0x7f6b2806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:16.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.516+0000 7f6b26ffd700 1 --2- 172.21.15.157:0/3455745213 >> v2:172.21.15.175:3300/0 conn(0x7f6b2806cbb0 0x7f6b2806d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:16.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.516+0000 7f6b26ffd700 1 -- 172.21.15.157:0/3455745213 >> v1:172.21.15.175:6789/0 conn(0x7f6b2806cf80 legacy=0x7f6b2814fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:16.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.516+0000 7f6b26ffd700 1 -- 172.21.15.157:0/3455745213 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b280c8e50 con 0x7f6b2806c1f0 2023-10-06T03:57:16.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.516+0000 7f6b26ffd700 1 --2- 172.21.15.157:0/3455745213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b2806c1f0 0x7f6b2806da70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6b2806fd70 tx=0x7f6b1c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d14272fd7e1956a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:16.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.516+0000 7f6b267fc700 1 -- 172.21.15.157:0/3455745213 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c00b8b0 con 0x7f6b2806c1f0 2023-10-06T03:57:16.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.516+0000 7f6b267fc700 1 -- 172.21.15.157:0/3455745213 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1c00ba10 con 0x7f6b2806c1f0 2023-10-06T03:57:16.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.516+0000 7f6b267fc700 1 -- 172.21.15.157:0/3455745213 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c00bdb0 con 0x7f6b2806c1f0 2023-10-06T03:57:16.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.517+0000 7f6b2db4e700 1 -- 172.21.15.157:0/3455745213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b2806c1f0 msgr2=0x7f6b2806da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:16.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.517+0000 7f6b2db4e700 1 --2- 172.21.15.157:0/3455745213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b2806c1f0 0x7f6b2806da70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6b2806fd70 tx=0x7f6b1c009250 comp rx=0 tx=0).stop 2023-10-06T03:57:16.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.517+0000 7f6b2db4e700 1 -- 172.21.15.157:0/3455745213 shutdown_connections 2023-10-06T03:57:16.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.517+0000 7f6b2db4e700 1 --2- 172.21.15.157:0/3455745213 >> v2:172.21.15.175:3300/0 conn(0x7f6b2806cbb0 0x7f6b2806d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:16.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.517+0000 7f6b2db4e700 1 --2- 172.21.15.157:0/3455745213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b2806c1f0 0x7f6b2806da70 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:16.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.517+0000 7f6b2db4e700 1 -- 172.21.15.157:0/3455745213 >> 172.21.15.157:0/3455745213 conn(0x7f6b2806f230 msgr2=0x7f6b2806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:16.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.517+0000 7f6b2db4e700 1 -- 172.21.15.157:0/3455745213 shutdown_connections 2023-10-06T03:57:16.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.517+0000 7f6b2db4e700 1 -- 172.21.15.157:0/3455745213 wait complete. 2023-10-06T03:57:16.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.518+0000 7f6b2db4e700 1 Processor -- start 2023-10-06T03:57:16.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.518+0000 7f6b2db4e700 1 -- start start 2023-10-06T03:57:16.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.519+0000 7f6b2db4e700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b2806c1f0 0x7f6b280c40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:16.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.519+0000 7f6b2db4e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b2806cbb0 0x7f6b280cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:16.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.519+0000 7f6b2db4e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b2806cf80 0x7f6b28151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:16.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.519+0000 7f6b2db4e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b28169900 con 0x7f6b2806cbb0 2023-10-06T03:57:16.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.519+0000 7f6b2db4e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b28169ba0 con 0x7f6b2806cf80 2023-10-06T03:57:16.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.519+0000 7f6b2db4e700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b28169e40 con 0x7f6b2806c1f0 2023-10-06T03:57:16.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.519+0000 7f6b26ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b2806cbb0 0x7f6b280cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:16.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.519+0000 7f6b26ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b2806cbb0 0x7f6b280cbc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34238/0 (socket says 172.21.15.157:34238) 2023-10-06T03:57:16.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.519+0000 7f6b26ffd700 1 -- 172.21.15.157:0/1677507259 learned_addr learned my addr 172.21.15.157:0/1677507259 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:16.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.519+0000 7f6b26ffd700 1 -- 172.21.15.157:0/1677507259 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b2806c1f0 msgr2=0x7f6b280c40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:16.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.519+0000 7f6b26ffd700 1 --2- 172.21.15.157:0/1677507259 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b2806c1f0 0x7f6b280c40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:16.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.519+0000 7f6b26ffd700 1 -- 172.21.15.157:0/1677507259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b2806cf80 msgr2=0x7f6b28151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:16.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.519+0000 7f6b26ffd700 1 --2- 172.21.15.157:0/1677507259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b2806cf80 0x7f6b28151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:16.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.519+0000 7f6b26ffd700 1 -- 172.21.15.157:0/1677507259 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1c0094f0 con 0x7f6b2806cbb0 2023-10-06T03:57:16.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.519+0000 7f6b26ffd700 1 --2- 172.21.15.157:0/1677507259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b2806cbb0 0x7f6b280cbc80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c003080 tx=0x7f6b1c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:16.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.519+0000 7f6b0f7fe700 1 -- 172.21.15.157:0/1677507259 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c011e20 con 0x7f6b2806cbb0 2023-10-06T03:57:16.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.519+0000 7f6b0f7fe700 1 -- 172.21.15.157:0/1677507259 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1c01b680 con 0x7f6b2806cbb0 2023-10-06T03:57:16.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.519+0000 7f6b2db4e700 1 -- 172.21.15.157:0/1677507259 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b28151540 con 0x7f6b2806cbb0 2023-10-06T03:57:16.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.519+0000 7f6b0f7fe700 1 -- 172.21.15.157:0/1677507259 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c01ba40 con 0x7f6b2806cbb0 2023-10-06T03:57:16.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.520+0000 7f6b2db4e700 1 -- 172.21.15.157:0/1677507259 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2816a3c0 con 0x7f6b2806cbb0 2023-10-06T03:57:16.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.520+0000 7f6b2db4e700 1 -- 172.21.15.157:0/1677507259 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b28151760 con 0x7f6b2806cbb0 2023-10-06T03:57:16.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.521+0000 7f6b0f7fe700 1 -- 172.21.15.157:0/1677507259 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6b1c01c450 con 0x7f6b2806cbb0 2023-10-06T03:57:16.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.521+0000 7f6b0f7fe700 1 --2- 172.21.15.157:0/1677507259 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6b14066f30 0x7f6b140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:16.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.521+0000 7f6b0f7fe700 1 -- 172.21.15.157:0/1677507259 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6b1c0863b0 con 0x7f6b2806cbb0 2023-10-06T03:57:16.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.521+0000 7f6b277fe700 1 --2- 172.21.15.157:0/1677507259 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6b14066f30 0x7f6b140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:16.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.522+0000 7f6b277fe700 1 --2- 172.21.15.157:0/1677507259 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6b14066f30 0x7f6b140693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6b18007970 tx=0x7f6b18008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:16.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.525+0000 7f6b0f7fe700 1 -- 172.21.15.157:0/1677507259 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6b28151760 con 0x7f6b2806cbb0 2023-10-06T03:57:16.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.751+0000 7f6b2db4e700 1 -- 172.21.15.157:0/1677507259 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b28151730 con 0x7f6b2806cbb0 2023-10-06T03:57:16.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.752+0000 7f6b0f7fe700 1 -- 172.21.15.157:0/1677507259 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6138 (secure 0 0 0) 0x7f6b1c0534c0 con 0x7f6b2806cbb0 2023-10-06T03:57:16.753 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:16.753 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4241,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/4177493830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4177493830},{"type":"v1","addr":"172.21.15.175:6839","nonce":4177493830}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T03:57:16.386174+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1],"up":{"mds_0":4296,"mds_1":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:creating","state_seq":1,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:57:16.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.754+0000 7f6b2db4e700 1 -- 172.21.15.157:0/1677507259 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6b14066f30 msgr2=0x7f6b140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:16.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.754+0000 7f6b2db4e700 1 --2- 172.21.15.157:0/1677507259 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6b14066f30 0x7f6b140693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6b18007970 tx=0x7f6b18008040 comp rx=0 tx=0).stop 2023-10-06T03:57:16.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.754+0000 7f6b2db4e700 1 -- 172.21.15.157:0/1677507259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b2806cbb0 msgr2=0x7f6b280cbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:16.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.754+0000 7f6b2db4e700 1 --2- 172.21.15.157:0/1677507259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b2806cbb0 0x7f6b280cbc80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c003080 tx=0x7f6b1c016000 comp rx=0 tx=0).stop 2023-10-06T03:57:16.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.754+0000 7f6b2db4e700 1 -- 172.21.15.157:0/1677507259 shutdown_connections 2023-10-06T03:57:16.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.754+0000 7f6b2db4e700 1 --2- 172.21.15.157:0/1677507259 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6b14066f30 0x7f6b140693e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:16.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.754+0000 7f6b2db4e700 1 --2- 172.21.15.157:0/1677507259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b2806cf80 0x7f6b28151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:16.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.754+0000 7f6b2db4e700 1 --2- 172.21.15.157:0/1677507259 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b2806c1f0 0x7f6b280c40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:16.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.754+0000 7f6b2db4e700 1 --2- 172.21.15.157:0/1677507259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b2806cbb0 0x7f6b280cbc80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:16.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.754+0000 7f6b2db4e700 1 -- 172.21.15.157:0/1677507259 >> 172.21.15.157:0/1677507259 conn(0x7f6b2806f230 msgr2=0x7f6b280c1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:16.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.754+0000 7f6b2db4e700 1 -- 172.21.15.157:0/1677507259 shutdown_connections 2023-10-06T03:57:16.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.754+0000 7f6b2db4e700 1 -- 172.21.15.157:0/1677507259 wait complete. 2023-10-06T03:57:16.757 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 9 2023-10-06T03:57:16.767 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:57:16.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.880+0000 7f2187db1700 1 Processor -- start 2023-10-06T03:57:16.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.880+0000 7f2187db1700 1 -- start start 2023-10-06T03:57:16.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.881+0000 7f2187db1700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2180150410 0x7f2180155f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:16.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.881+0000 7f2187db1700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21800c9780 con 0x7f218006cbb0 2023-10-06T03:57:16.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.881+0000 7f2187db1700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f218006e0c0 con 0x7f218006dc40 2023-10-06T03:57:16.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.881+0000 7f2187db1700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21800c8c40 con 0x7f2180150410 2023-10-06T03:57:16.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.881+0000 7f2185b4d700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f218006cbb0 0x7f218006d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39924/0 (socket says 172.21.15.157:39924) 2023-10-06T03:57:16.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.881+0000 7f2185b4d700 1 -- 172.21.15.157:0/1469418197 learned_addr learned my addr 172.21.15.157:0/1469418197 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:16.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.881+0000 7f218634e700 1 --2- 172.21.15.157:0/1469418197 >> v2:172.21.15.157:3300/0 conn(0x7f2180150410 0x7f2180155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:16.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.881+0000 7f218634e700 1 -- 172.21.15.157:0/1469418197 >> v1:172.21.15.175:6789/0 conn(0x7f218006dc40 legacy=0x7f218014fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:16.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.881+0000 7f218634e700 1 -- 172.21.15.157:0/1469418197 >> v1:172.21.15.157:6789/0 conn(0x7f218006cbb0 legacy=0x7f218006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:16.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.881+0000 7f218634e700 1 -- 172.21.15.157:0/1469418197 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21800c8e50 con 0x7f2180150410 2023-10-06T03:57:16.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.881+0000 7f2184b4b700 1 -- 172.21.15.157:0/1469418197 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1860347021 0 0) 0x7f21800c9780 con 0x7f218006cbb0 2023-10-06T03:57:16.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.881+0000 7f218634e700 1 --2- 172.21.15.157:0/1469418197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2180150410 0x7f2180155f70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2170009e60 tx=0x7f2170009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46540e08577b750c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:16.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.881+0000 7f2184b4b700 1 -- 172.21.15.157:0/1469418197 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217000e070 con 0x7f2180150410 2023-10-06T03:57:16.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.882+0000 7f2184b4b700 1 -- 172.21.15.157:0/1469418197 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217000bc80 con 0x7f2180150410 2023-10-06T03:57:16.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.882+0000 7f2184b4b700 1 -- 172.21.15.157:0/1469418197 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2170012430 con 0x7f2180150410 2023-10-06T03:57:16.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.882+0000 7f2187db1700 1 -- 172.21.15.157:0/1469418197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2180150410 msgr2=0x7f2180155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:16.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.882+0000 7f2187db1700 1 --2- 172.21.15.157:0/1469418197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2180150410 0x7f2180155f70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2170009e60 tx=0x7f2170009780 comp rx=0 tx=0).stop 2023-10-06T03:57:16.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.882+0000 7f2187db1700 1 -- 172.21.15.157:0/1469418197 shutdown_connections 2023-10-06T03:57:16.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.882+0000 7f2187db1700 1 --2- 172.21.15.157:0/1469418197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2180150410 0x7f2180155f70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:16.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.882+0000 7f2187db1700 1 -- 172.21.15.157:0/1469418197 >> 172.21.15.157:0/1469418197 conn(0x7f218006f230 msgr2=0x7f218006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:16.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.882+0000 7f2187db1700 1 -- 172.21.15.157:0/1469418197 shutdown_connections 2023-10-06T03:57:16.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.882+0000 7f2187db1700 1 -- 172.21.15.157:0/1469418197 wait complete. 2023-10-06T03:57:16.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.883+0000 7f2187db1700 1 Processor -- start 2023-10-06T03:57:16.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.883+0000 7f2187db1700 1 -- start start 2023-10-06T03:57:16.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.884+0000 7f2187db1700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f218006cbb0 0x7f21800c4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:16.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.884+0000 7f2187db1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2180150410 0x7f21801489f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:16.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.884+0000 7f2187db1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21800cbff0 0x7f2180148f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:16.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.884+0000 7f2187db1700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2180166030 con 0x7f21800cbff0 2023-10-06T03:57:16.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.884+0000 7f2187db1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2180148560 con 0x7f2180150410 2023-10-06T03:57:16.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.884+0000 7f2187db1700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2180166240 con 0x7f218006cbb0 2023-10-06T03:57:16.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.884+0000 7f218634e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21800cbff0 0x7f2180148f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:16.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.884+0000 7f218634e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21800cbff0 0x7f2180148f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34242/0 (socket says 172.21.15.157:34242) 2023-10-06T03:57:16.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.884+0000 7f218634e700 1 -- 172.21.15.157:0/3066229735 learned_addr learned my addr 172.21.15.157:0/3066229735 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:16.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.884+0000 7f2185b4d700 1 --2- 172.21.15.157:0/3066229735 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f218006cbb0 0x7f21800c4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:16.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.884+0000 7f218534c700 1 --2- 172.21.15.157:0/3066229735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2180150410 0x7f21801489f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:16.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.884+0000 7f218634e700 1 -- 172.21.15.157:0/3066229735 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f218006cbb0 msgr2=0x7f21800c4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:16.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.884+0000 7f218634e700 1 --2- 172.21.15.157:0/3066229735 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f218006cbb0 0x7f21800c4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:16.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.884+0000 7f218634e700 1 -- 172.21.15.157:0/3066229735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2180150410 msgr2=0x7f21801489f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:16.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.884+0000 7f218634e700 1 --2- 172.21.15.157:0/3066229735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2180150410 0x7f21801489f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:16.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.884+0000 7f218634e700 1 -- 172.21.15.157:0/3066229735 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2170009a20 con 0x7f21800cbff0 2023-10-06T03:57:16.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.884+0000 7f218634e700 1 --2- 172.21.15.157:0/3066229735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21800cbff0 0x7f2180148f30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2170009d30 tx=0x7f2170018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:16.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.885+0000 7f217dffb700 1 -- 172.21.15.157:0/3066229735 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217001e640 con 0x7f21800cbff0 2023-10-06T03:57:16.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.885+0000 7f217dffb700 1 -- 172.21.15.157:0/3066229735 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217001e7c0 con 0x7f21800cbff0 2023-10-06T03:57:16.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.885+0000 7f2187db1700 1 -- 172.21.15.157:0/3066229735 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2180166520 con 0x7f21800cbff0 2023-10-06T03:57:16.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.885+0000 7f217dffb700 1 -- 172.21.15.157:0/3066229735 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217001eb80 con 0x7f21800cbff0 2023-10-06T03:57:16.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.885+0000 7f2187db1700 1 -- 172.21.15.157:0/3066229735 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2180166ae0 con 0x7f21800cbff0 2023-10-06T03:57:16.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.886+0000 7f2187db1700 1 -- 172.21.15.157:0/3066229735 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2180089ca0 con 0x7f21800cbff0 2023-10-06T03:57:16.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.886+0000 7f217dffb700 1 -- 172.21.15.157:0/3066229735 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2170024070 con 0x7f21800cbff0 2023-10-06T03:57:16.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.886+0000 7f217dffb700 1 --2- 172.21.15.157:0/3066229735 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f216c066f30 0x7f216c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:16.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.887+0000 7f217dffb700 1 -- 172.21.15.157:0/3066229735 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2170087130 con 0x7f21800cbff0 2023-10-06T03:57:16.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.887+0000 7f2185b4d700 1 --2- 172.21.15.157:0/3066229735 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f216c066f30 0x7f216c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:16.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.887+0000 7f2185b4d700 1 --2- 172.21.15.157:0/3066229735 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f216c066f30 0x7f216c0693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2178007fd0 tx=0x7f2178007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:16.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:16.891+0000 7f217dffb700 1 -- 172.21.15.157:0/3066229735 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f21700541a0 con 0x7f21800cbff0 2023-10-06T03:57:17.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.077+0000 7f2187db1700 1 -- 172.21.15.157:0/3066229735 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21801486a0 con 0x7f21800cbff0 2023-10-06T03:57:17.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.078+0000 7f217dffb700 1 -- 172.21.15.157:0/3066229735 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f21700577c0 con 0x7f21800cbff0 2023-10-06T03:57:17.079 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:17.079 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":21,"fsid":"28664a7a-fe30-4efe-be5a-6abf9318b700","created":"2023-10-06T03:56:51.887538+0000","modified":"2023-10-06T03:57:12.362537+0000","last_up_change":"2023-10-06T03:57:02.299440+0000","last_in_change":"2023-10-06T03:56:55.141505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:57:03.455121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:57:03.905661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:57:08.377186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:57:09.709582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f677cac-e9f4-4608-acac-718b072ce984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":90640},{"type":"v1","addr":"172.21.15.157:6809","nonce":90640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":90640},{"type":"v1","addr":"172.21.15.157:6811","nonce":90640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":90640},{"type":"v1","addr":"172.21.15.157:6815","nonce":90640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":90640},{"type":"v1","addr":"172.21.15.157:6813","nonce":90640}]},"public_addr":"172.21.15.157:6809/90640","cluster_addr":"172.21.15.157:6811/90640","heartbeat_back_addr":"172.21.15.157:6815/90640","heartbeat_front_addr":"172.21.15.157:6813/90640","state":["exists","up"]},{"osd":1,"uuid":"7d771892-bb91-4e9b-9610-051861c11247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":90654},{"type":"v1","addr":"172.21.15.157:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":90654},{"type":"v1","addr":"172.21.15.157:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":90654},{"type":"v1","addr":"172.21.15.157:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":90654},{"type":"v1","addr":"172.21.15.157:6805","nonce":90654}]},"public_addr":"172.21.15.157:6801/90654","cluster_addr":"172.21.15.157:6803/90654","heartbeat_back_addr":"172.21.15.157:6807/90654","heartbeat_front_addr":"172.21.15.157:6805/90654","state":["exists","up"]},{"osd":2,"uuid":"5c1f7d4e-d185-47cf-ab6c-453b4e3009c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":90657},{"type":"v1","addr":"172.21.15.157:6825","nonce":90657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":90657},{"type":"v1","addr":"172.21.15.157:6827","nonce":90657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":90657},{"type":"v1","addr":"172.21.15.157:6831","nonce":90657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":90657},{"type":"v1","addr":"172.21.15.157:6829","nonce":90657}]},"public_addr":"172.21.15.157:6825/90657","cluster_addr":"172.21.15.157:6827/90657","heartbeat_back_addr":"172.21.15.157:6831/90657","heartbeat_front_addr":"172.21.15.157:6829/90657","state":["exists","up"]},{"osd":3,"uuid":"f6a1e825-43c7-4afd-8daf-e51d45afc75e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":90641},{"type":"v1","addr":"172.21.15.157:6817","nonce":90641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":90641},{"type":"v1","addr":"172.21.15.157:6819","nonce":90641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":90641},{"type":"v1","addr":"172.21.15.157:6823","nonce":90641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":90641},{"type":"v1","addr":"172.21.15.157:6821","nonce":90641}]},"public_addr":"172.21.15.157:6817/90641","cluster_addr":"172.21.15.157:6819/90641","heartbeat_back_addr":"172.21.15.157:6823/90641","heartbeat_front_addr":"172.21.15.157:6821/90641","state":["exists","up"]},{"osd":4,"uuid":"8ad6e3d8-9a31-4f85-9410-417b7ac5da97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":91261},{"type":"v1","addr":"172.21.15.175:6803","nonce":91261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":91261},{"type":"v1","addr":"172.21.15.175:6806","nonce":91261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":91261},{"type":"v1","addr":"172.21.15.175:6814","nonce":91261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":91261},{"type":"v1","addr":"172.21.15.175:6810","nonce":91261}]},"public_addr":"172.21.15.175:6803/91261","cluster_addr":"172.21.15.175:6806/91261","heartbeat_back_addr":"172.21.15.175:6814/91261","heartbeat_front_addr":"172.21.15.175:6810/91261","state":["exists","up"]},{"osd":5,"uuid":"f68aca2a-f182-4c00-8e21-241d09c02ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":91262},{"type":"v1","addr":"172.21.15.175:6817","nonce":91262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":91262},{"type":"v1","addr":"172.21.15.175:6819","nonce":91262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":91262},{"type":"v1","addr":"172.21.15.175:6823","nonce":91262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":91262},{"type":"v1","addr":"172.21.15.175:6821","nonce":91262}]},"public_addr":"172.21.15.175:6817/91262","cluster_addr":"172.21.15.175:6819/91262","heartbeat_back_addr":"172.21.15.175:6823/91262","heartbeat_front_addr":"172.21.15.175:6821/91262","state":["exists","up"]},{"osd":6,"uuid":"d867cd89-3430-40ca-862f-d2bea3de9d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":91263},{"type":"v1","addr":"172.21.15.175:6801","nonce":91263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6805","nonce":91263},{"type":"v1","addr":"172.21.15.175:6807","nonce":91263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6813","nonce":91263},{"type":"v1","addr":"172.21.15.175:6815","nonce":91263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":91263},{"type":"v1","addr":"172.21.15.175:6811","nonce":91263}]},"public_addr":"172.21.15.175:6801/91263","cluster_addr":"172.21.15.175:6807/91263","heartbeat_back_addr":"172.21.15.175:6815/91263","heartbeat_front_addr":"172.21.15.175:6811/91263","state":["exists","up"]},{"osd":7,"uuid":"930cd464-79f1-47cb-b4ff-188ccde74fff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":91306},{"type":"v1","addr":"172.21.15.175:6825","nonce":91306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":91306},{"type":"v1","addr":"172.21.15.175:6827","nonce":91306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":91306},{"type":"v1","addr":"172.21.15.175:6831","nonce":91306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":91306},{"type":"v1","addr":"172.21.15.175:6829","nonce":91306}]},"public_addr":"172.21.15.175:6825/91306","cluster_addr":"172.21.15.175:6827/91306","heartbeat_back_addr":"172.21.15.175:6831/91306","heartbeat_front_addr":"172.21.15.175:6829/91306","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.290673+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.056390+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.292903+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.275861+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.332095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.327453+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.317585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.309330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:57:17.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.080+0000 7f2187db1700 1 -- 172.21.15.157:0/3066229735 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f216c066f30 msgr2=0x7f216c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:17.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.080+0000 7f2187db1700 1 --2- 172.21.15.157:0/3066229735 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f216c066f30 0x7f216c0693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2178007fd0 tx=0x7f2178007d30 comp rx=0 tx=0).stop 2023-10-06T03:57:17.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.080+0000 7f2187db1700 1 -- 172.21.15.157:0/3066229735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21800cbff0 msgr2=0x7f2180148f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:17.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.080+0000 7f2187db1700 1 --2- 172.21.15.157:0/3066229735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21800cbff0 0x7f2180148f30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2170009d30 tx=0x7f2170018000 comp rx=0 tx=0).stop 2023-10-06T03:57:17.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.080+0000 7f2187db1700 1 -- 172.21.15.157:0/3066229735 shutdown_connections 2023-10-06T03:57:17.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.080+0000 7f2187db1700 1 --2- 172.21.15.157:0/3066229735 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f216c066f30 0x7f216c0693e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.080+0000 7f2187db1700 1 --2- 172.21.15.157:0/3066229735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21800cbff0 0x7f2180148f30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.080+0000 7f2187db1700 1 --2- 172.21.15.157:0/3066229735 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f218006cbb0 0x7f21800c4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.080+0000 7f2187db1700 1 --2- 172.21.15.157:0/3066229735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2180150410 0x7f21801489f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.080+0000 7f2187db1700 1 -- 172.21.15.157:0/3066229735 >> 172.21.15.157:0/3066229735 conn(0x7f218006f230 msgr2=0x7f21800c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:17.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.080+0000 7f2187db1700 1 -- 172.21.15.157:0/3066229735 shutdown_connections 2023-10-06T03:57:17.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.080+0000 7f2187db1700 1 -- 172.21.15.157:0/3066229735 wait complete. 2023-10-06T03:57:17.093 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T03:57:17.093 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:57:17.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.206+0000 7f71090d6700 1 Processor -- start 2023-10-06T03:57:17.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.206+0000 7f71090d6700 1 -- start start 2023-10-06T03:57:17.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.207+0000 7f71090d6700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f71040b1e40 0x7f7104157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:17.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.207+0000 7f71090d6700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f710406d8d0 con 0x7f710406c2d0 2023-10-06T03:57:17.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.207+0000 7f71090d6700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71040af8f0 con 0x7f71040b04d0 2023-10-06T03:57:17.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.207+0000 7f71090d6700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f710415a690 con 0x7f71040b1e40 2023-10-06T03:57:17.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.207+0000 7f71088d5700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f710406c2d0 0x7f7104156ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39928/0 (socket says 172.21.15.157:39928) 2023-10-06T03:57:17.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.207+0000 7f71088d5700 1 -- 172.21.15.157:0/1913578515 learned_addr learned my addr 172.21.15.157:0/1913578515 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:17.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.207+0000 7f7103fff700 1 --2- 172.21.15.157:0/1913578515 >> v2:172.21.15.175:3300/0 conn(0x7f71040b1e40 0x7f7104157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:17.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.207+0000 7f7102ffd700 1 -- 172.21.15.157:0/1913578515 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4081358547 0 0) 0x7f710406d8d0 con 0x7f710406c2d0 2023-10-06T03:57:17.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.207+0000 7f7102ffd700 1 -- 172.21.15.157:0/1913578515 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70ec003570 con 0x7f710406c2d0 2023-10-06T03:57:17.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.207+0000 7f7103fff700 1 -- 172.21.15.157:0/1913578515 >> v1:172.21.15.175:6789/0 conn(0x7f71040b04d0 legacy=0x7f71041563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:17.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.207+0000 7f7103fff700 1 -- 172.21.15.157:0/1913578515 >> v1:172.21.15.157:6789/0 conn(0x7f710406c2d0 legacy=0x7f7104156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:17.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.207+0000 7f7103fff700 1 -- 172.21.15.157:0/1913578515 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71041571d0 con 0x7f71040b1e40 2023-10-06T03:57:17.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.208+0000 7f7103fff700 1 --2- 172.21.15.157:0/1913578515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71040b1e40 0x7f7104157750 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f70f40098e0 tx=0x7f70f40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=782e276a5b84a7ce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:17.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.208+0000 7f7102ffd700 1 -- 172.21.15.157:0/1913578515 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f400b640 con 0x7f71040b1e40 2023-10-06T03:57:17.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.208+0000 7f7102ffd700 1 -- 172.21.15.157:0/1913578515 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f400b7a0 con 0x7f71040b1e40 2023-10-06T03:57:17.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.208+0000 7f7102ffd700 1 -- 172.21.15.157:0/1913578515 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f400bb20 con 0x7f71040b1e40 2023-10-06T03:57:17.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.208+0000 7f71090d6700 1 -- 172.21.15.157:0/1913578515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71040b1e40 msgr2=0x7f7104157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:17.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.208+0000 7f71090d6700 1 --2- 172.21.15.157:0/1913578515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71040b1e40 0x7f7104157750 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f70f40098e0 tx=0x7f70f40091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.208+0000 7f71090d6700 1 -- 172.21.15.157:0/1913578515 shutdown_connections 2023-10-06T03:57:17.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.208+0000 7f71090d6700 1 --2- 172.21.15.157:0/1913578515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71040b1e40 0x7f7104157750 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.208+0000 7f71090d6700 1 -- 172.21.15.157:0/1913578515 >> 172.21.15.157:0/1913578515 conn(0x7f71040cc540 msgr2=0x7f71040cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:17.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.208+0000 7f71090d6700 1 -- 172.21.15.157:0/1913578515 shutdown_connections 2023-10-06T03:57:17.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.208+0000 7f71090d6700 1 -- 172.21.15.157:0/1913578515 wait complete. 2023-10-06T03:57:17.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.209+0000 7f71090d6700 1 Processor -- start 2023-10-06T03:57:17.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.210+0000 7f71090d6700 1 -- start start 2023-10-06T03:57:17.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.210+0000 7f71090d6700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f710406c2d0 0x7f7104154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:17.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.210+0000 7f71090d6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71040b04d0 0x7f7104166470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:17.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.210+0000 7f71090d6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71040b1e40 0x7f710414ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:17.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.210+0000 7f71090d6700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71040c6ff0 con 0x7f71040b1e40 2023-10-06T03:57:17.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.210+0000 7f71090d6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7104165fe0 con 0x7f71040b04d0 2023-10-06T03:57:17.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.210+0000 7f71090d6700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f710415a180 con 0x7f710406c2d0 2023-10-06T03:57:17.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.210+0000 7f71088d5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71040b1e40 0x7f710414ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:17.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.210+0000 7f71088d5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71040b1e40 0x7f710414ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34244/0 (socket says 172.21.15.157:34244) 2023-10-06T03:57:17.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.210+0000 7f71088d5700 1 -- 172.21.15.157:0/1460630012 learned_addr learned my addr 172.21.15.157:0/1460630012 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:17.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.210+0000 7f71037fe700 1 --2- 172.21.15.157:0/1460630012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71040b04d0 0x7f7104166470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:17.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.210+0000 7f7103fff700 1 --2- 172.21.15.157:0/1460630012 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f710406c2d0 0x7f7104154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:17.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.211+0000 7f71088d5700 1 -- 172.21.15.157:0/1460630012 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f710406c2d0 msgr2=0x7f7104154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:17.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.211+0000 7f71088d5700 1 --2- 172.21.15.157:0/1460630012 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f710406c2d0 0x7f7104154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.211+0000 7f71088d5700 1 -- 172.21.15.157:0/1460630012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71040b04d0 msgr2=0x7f7104166470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:17.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.211+0000 7f71088d5700 1 --2- 172.21.15.157:0/1460630012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71040b04d0 0x7f7104166470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.211+0000 7f71088d5700 1 -- 172.21.15.157:0/1460630012 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f4009420 con 0x7f71040b1e40 2023-10-06T03:57:17.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.211+0000 7f71088d5700 1 --2- 172.21.15.157:0/1460630012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71040b1e40 0x7f710414ecf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00cca0 tx=0x7f70fc001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:17.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.211+0000 7f70ebfff700 1 -- 172.21.15.157:0/1460630012 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc00a8d0 con 0x7f71040b1e40 2023-10-06T03:57:17.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.211+0000 7f70ebfff700 1 -- 172.21.15.157:0/1460630012 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70fc00aa30 con 0x7f71040b1e40 2023-10-06T03:57:17.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.211+0000 7f71090d6700 1 -- 172.21.15.157:0/1460630012 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7104165c90 con 0x7f71040b1e40 2023-10-06T03:57:17.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.211+0000 7f70ebfff700 1 -- 172.21.15.157:0/1460630012 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc00adb0 con 0x7f71040b1e40 2023-10-06T03:57:17.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.211+0000 7f71090d6700 1 -- 172.21.15.157:0/1460630012 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7104169cc0 con 0x7f71040b1e40 2023-10-06T03:57:17.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.212+0000 7f71090d6700 1 -- 172.21.15.157:0/1460630012 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7104088f90 con 0x7f71040b1e40 2023-10-06T03:57:17.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.212+0000 7f70ebfff700 1 -- 172.21.15.157:0/1460630012 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f70fc007300 con 0x7f71040b1e40 2023-10-06T03:57:17.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.213+0000 7f70ebfff700 1 --2- 172.21.15.157:0/1460630012 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f70f0066f60 0x7f70f0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:17.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.213+0000 7f70ebfff700 1 -- 172.21.15.157:0/1460630012 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f70fc084a10 con 0x7f71040b1e40 2023-10-06T03:57:17.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.213+0000 7f7103fff700 1 --2- 172.21.15.157:0/1460630012 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f70f0066f60 0x7f70f0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:17.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.213+0000 7f7103fff700 1 --2- 172.21.15.157:0/1460630012 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f70f0066f60 0x7f70f0069410 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f70f40093f0 tx=0x7f70f4017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:17.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.217+0000 7f70ebfff700 1 -- 172.21.15.157:0/1460630012 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70fc051ba0 con 0x7f71040b1e40 2023-10-06T03:57:17.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.399+0000 7f71090d6700 1 -- 172.21.15.157:0/1460630012 --> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7104165e20 con 0x7f70f0066f60 2023-10-06T03:57:17.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.408+0000 7f70ebfff700 1 -- 172.21.15.157:0/1460630012 <== mgr.4107 v2:172.21.15.175:6832/90661 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286697 (secure 0 0 0) 0x7f7104165e20 con 0x7f70f0066f60 2023-10-06T03:57:17.413 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:17.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.411+0000 7f71090d6700 1 -- 172.21.15.157:0/1460630012 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f70f0066f60 msgr2=0x7f70f0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:17.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.411+0000 7f71090d6700 1 --2- 172.21.15.157:0/1460630012 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f70f0066f60 0x7f70f0069410 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f70f40093f0 tx=0x7f70f4017000 comp rx=0 tx=0).stop 2023-10-06T03:57:17.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.411+0000 7f71090d6700 1 -- 172.21.15.157:0/1460630012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71040b1e40 msgr2=0x7f710414ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:17.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.411+0000 7f71090d6700 1 --2- 172.21.15.157:0/1460630012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71040b1e40 0x7f710414ecf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00cca0 tx=0x7f70fc001ae0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.411+0000 7f71090d6700 1 -- 172.21.15.157:0/1460630012 shutdown_connections 2023-10-06T03:57:17.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.411+0000 7f71090d6700 1 --2- 172.21.15.157:0/1460630012 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f70f0066f60 0x7f70f0069410 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.411+0000 7f71090d6700 1 --2- 172.21.15.157:0/1460630012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71040b1e40 0x7f710414ecf0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.411+0000 7f71090d6700 1 --2- 172.21.15.157:0/1460630012 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f710406c2d0 0x7f7104154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.411+0000 7f71090d6700 1 --2- 172.21.15.157:0/1460630012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71040b04d0 0x7f7104166470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.411+0000 7f71090d6700 1 -- 172.21.15.157:0/1460630012 >> 172.21.15.157:0/1460630012 conn(0x7f71040cc540 msgr2=0x7f71040c4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:17.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.411+0000 7f71090d6700 1 -- 172.21.15.157:0/1460630012 shutdown_connections 2023-10-06T03:57:17.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.411+0000 7f71090d6700 1 -- 172.21.15.157:0/1460630012 wait complete. 2023-10-06T03:57:17.438 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-10-06T03:57:17.440 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-06T03:57:15.426266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":84,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2380028,"kb_used_data":1724,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752594692,"statfs":{"total":773094113280,"available":770656964608,"internally_reserved":0,"allocated":1765376,"data_stored":1757335,"data_compressed":64858,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":14,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001403"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372019+0000","last_change":"2023-10-06T03:57:11.372019+0000","last_active":"2023-10-06T03:57:11.372019+0000","last_peered":"2023-10-06T03:57:11.372019+0000","last_clean":"2023-10-06T03:57:11.372019+0000","last_became_active":"2023-10-06T03:57:11.371756+0000","last_became_peered":"2023-10-06T03:57:11.371756+0000","last_unstale":"2023-10-06T03:57:11.372019+0000","last_undegraded":"2023-10-06T03:57:11.372019+0000","last_fullsized":"2023-10-06T03:57:11.372019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:50.086528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.370371+0000","last_change":"2023-10-06T03:57:11.370371+0000","last_active":"2023-10-06T03:57:11.370371+0000","last_peered":"2023-10-06T03:57:11.370371+0000","last_clean":"2023-10-06T03:57:11.370371+0000","last_became_active":"2023-10-06T03:57:11.370070+0000","last_became_peered":"2023-10-06T03:57:11.370070+0000","last_unstale":"2023-10-06T03:57:11.370371+0000","last_undegraded":"2023-10-06T03:57:11.370371+0000","last_fullsized":"2023-10-06T03:57:11.370371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:00.917607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371725+0000","last_change":"2023-10-06T03:57:11.366298+0000","last_active":"2023-10-06T03:57:11.371725+0000","last_peered":"2023-10-06T03:57:11.371725+0000","last_clean":"2023-10-06T03:57:11.371725+0000","last_became_active":"2023-10-06T03:57:11.366012+0000","last_became_peered":"2023-10-06T03:57:11.366012+0000","last_unstale":"2023-10-06T03:57:11.371725+0000","last_undegraded":"2023-10-06T03:57:11.371725+0000","last_fullsized":"2023-10-06T03:57:11.371725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:45.802287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.365933+0000","last_change":"2023-10-06T03:57:11.365933+0000","last_active":"2023-10-06T03:57:11.365933+0000","last_peered":"2023-10-06T03:57:11.365933+0000","last_clean":"2023-10-06T03:57:11.365933+0000","last_became_active":"2023-10-06T03:57:11.365673+0000","last_became_peered":"2023-10-06T03:57:11.365673+0000","last_unstale":"2023-10-06T03:57:11.365933+0000","last_undegraded":"2023-10-06T03:57:11.365933+0000","last_fullsized":"2023-10-06T03:57:11.365933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:40.953382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.361569+0000","last_change":"2023-10-06T03:57:11.361569+0000","last_active":"2023-10-06T03:57:11.361569+0000","last_peered":"2023-10-06T03:57:11.361569+0000","last_clean":"2023-10-06T03:57:11.361569+0000","last_became_active":"2023-10-06T03:57:11.361302+0000","last_became_peered":"2023-10-06T03:57:11.361302+0000","last_unstale":"2023-10-06T03:57:11.361569+0000","last_undegraded":"2023-10-06T03:57:11.361569+0000","last_fullsized":"2023-10-06T03:57:11.361569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:57.552493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372859+0000","last_change":"2023-10-06T03:57:11.369953+0000","last_active":"2023-10-06T03:57:11.372859+0000","last_peered":"2023-10-06T03:57:11.372859+0000","last_clean":"2023-10-06T03:57:11.372859+0000","last_became_active":"2023-10-06T03:57:11.368775+0000","last_became_peered":"2023-10-06T03:57:11.368775+0000","last_unstale":"2023-10-06T03:57:11.372859+0000","last_undegraded":"2023-10-06T03:57:11.372859+0000","last_fullsized":"2023-10-06T03:57:11.372859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:22.368117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.380867+0000","last_change":"2023-10-06T03:57:11.378048+0000","last_active":"2023-10-06T03:57:11.380867+0000","last_peered":"2023-10-06T03:57:11.380867+0000","last_clean":"2023-10-06T03:57:11.380867+0000","last_became_active":"2023-10-06T03:57:11.377786+0000","last_became_peered":"2023-10-06T03:57:11.377786+0000","last_unstale":"2023-10-06T03:57:11.380867+0000","last_undegraded":"2023-10-06T03:57:11.380867+0000","last_fullsized":"2023-10-06T03:57:11.380867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:12:54.218593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.374003+0000","last_change":"2023-10-06T03:57:11.374003+0000","last_active":"2023-10-06T03:57:11.374003+0000","last_peered":"2023-10-06T03:57:11.374003+0000","last_clean":"2023-10-06T03:57:11.374003+0000","last_became_active":"2023-10-06T03:57:11.373765+0000","last_became_peered":"2023-10-06T03:57:11.373765+0000","last_unstale":"2023-10-06T03:57:11.374003+0000","last_undegraded":"2023-10-06T03:57:11.374003+0000","last_fullsized":"2023-10-06T03:57:11.374003+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:05.838007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373794+0000","last_change":"2023-10-06T03:57:11.373794+0000","last_active":"2023-10-06T03:57:11.373794+0000","last_peered":"2023-10-06T03:57:11.373794+0000","last_clean":"2023-10-06T03:57:11.373794+0000","last_became_active":"2023-10-06T03:57:11.368004+0000","last_became_peered":"2023-10-06T03:57:11.368004+0000","last_unstale":"2023-10-06T03:57:11.373794+0000","last_undegraded":"2023-10-06T03:57:11.373794+0000","last_fullsized":"2023-10-06T03:57:11.373794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:16.760049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.379370+0000","last_change":"2023-10-06T03:57:11.379370+0000","last_active":"2023-10-06T03:57:11.379370+0000","last_peered":"2023-10-06T03:57:11.379370+0000","last_clean":"2023-10-06T03:57:11.379370+0000","last_became_active":"2023-10-06T03:57:11.379107+0000","last_became_peered":"2023-10-06T03:57:11.379107+0000","last_unstale":"2023-10-06T03:57:11.379370+0000","last_undegraded":"2023-10-06T03:57:11.379370+0000","last_fullsized":"2023-10-06T03:57:11.379370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:18.373103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371742+0000","last_change":"2023-10-06T03:57:11.369172+0000","last_active":"2023-10-06T03:57:11.371742+0000","last_peered":"2023-10-06T03:57:11.371742+0000","last_clean":"2023-10-06T03:57:11.371742+0000","last_became_active":"2023-10-06T03:57:11.368256+0000","last_became_peered":"2023-10-06T03:57:11.368256+0000","last_unstale":"2023-10-06T03:57:11.371742+0000","last_undegraded":"2023-10-06T03:57:11.371742+0000","last_fullsized":"2023-10-06T03:57:11.371742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:50.736367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.368685+0000","last_change":"2023-10-06T03:57:11.368685+0000","last_active":"2023-10-06T03:57:11.368685+0000","last_peered":"2023-10-06T03:57:11.368685+0000","last_clean":"2023-10-06T03:57:11.368685+0000","last_became_active":"2023-10-06T03:57:11.368425+0000","last_became_peered":"2023-10-06T03:57:11.368425+0000","last_unstale":"2023-10-06T03:57:11.368685+0000","last_undegraded":"2023-10-06T03:57:11.368685+0000","last_fullsized":"2023-10-06T03:57:11.368685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:06.004423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367646+0000","last_change":"2023-10-06T03:57:11.367646+0000","last_active":"2023-10-06T03:57:11.367646+0000","last_peered":"2023-10-06T03:57:11.367646+0000","last_clean":"2023-10-06T03:57:11.367646+0000","last_became_active":"2023-10-06T03:57:11.367079+0000","last_became_peered":"2023-10-06T03:57:11.367079+0000","last_unstale":"2023-10-06T03:57:11.367646+0000","last_undegraded":"2023-10-06T03:57:11.367646+0000","last_fullsized":"2023-10-06T03:57:11.367646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:09:54.331206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373481+0000","last_change":"2023-10-06T03:57:11.373481+0000","last_active":"2023-10-06T03:57:11.373481+0000","last_peered":"2023-10-06T03:57:11.373481+0000","last_clean":"2023-10-06T03:57:11.373481+0000","last_became_active":"2023-10-06T03:57:11.373216+0000","last_became_peered":"2023-10-06T03:57:11.373216+0000","last_unstale":"2023-10-06T03:57:11.373481+0000","last_undegraded":"2023-10-06T03:57:11.373481+0000","last_fullsized":"2023-10-06T03:57:11.373481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:56.083203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.379250+0000","last_change":"2023-10-06T03:57:11.376012+0000","last_active":"2023-10-06T03:57:11.379250+0000","last_peered":"2023-10-06T03:57:11.379250+0000","last_clean":"2023-10-06T03:57:11.379250+0000","last_became_active":"2023-10-06T03:57:11.375689+0000","last_became_peered":"2023-10-06T03:57:11.375689+0000","last_unstale":"2023-10-06T03:57:11.379250+0000","last_undegraded":"2023-10-06T03:57:11.379250+0000","last_fullsized":"2023-10-06T03:57:11.379250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:42.953739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372989+0000","last_change":"2023-10-06T03:57:11.366808+0000","last_active":"2023-10-06T03:57:11.372989+0000","last_peered":"2023-10-06T03:57:11.372989+0000","last_clean":"2023-10-06T03:57:11.372989+0000","last_became_active":"2023-10-06T03:57:11.366543+0000","last_became_peered":"2023-10-06T03:57:11.366543+0000","last_unstale":"2023-10-06T03:57:11.372989+0000","last_undegraded":"2023-10-06T03:57:11.372989+0000","last_fullsized":"2023-10-06T03:57:11.372989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:16.212967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367276+0000","last_change":"2023-10-06T03:57:11.367276+0000","last_active":"2023-10-06T03:57:11.367276+0000","last_peered":"2023-10-06T03:57:11.367276+0000","last_clean":"2023-10-06T03:57:11.367276+0000","last_became_active":"2023-10-06T03:57:11.366767+0000","last_became_peered":"2023-10-06T03:57:11.366767+0000","last_unstale":"2023-10-06T03:57:11.367276+0000","last_undegraded":"2023-10-06T03:57:11.367276+0000","last_fullsized":"2023-10-06T03:57:11.367276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:43.113575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367129+0000","last_change":"2023-10-06T03:57:11.362667+0000","last_active":"2023-10-06T03:57:11.367129+0000","last_peered":"2023-10-06T03:57:11.367129+0000","last_clean":"2023-10-06T03:57:11.367129+0000","last_became_active":"2023-10-06T03:57:11.362377+0000","last_became_peered":"2023-10-06T03:57:11.362377+0000","last_unstale":"2023-10-06T03:57:11.367129+0000","last_undegraded":"2023-10-06T03:57:11.367129+0000","last_fullsized":"2023-10-06T03:57:11.367129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:40.600383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.364126+0000","last_change":"2023-10-06T03:57:11.364126+0000","last_active":"2023-10-06T03:57:11.364126+0000","last_peered":"2023-10-06T03:57:11.364126+0000","last_clean":"2023-10-06T03:57:11.364126+0000","last_became_active":"2023-10-06T03:57:11.363792+0000","last_became_peered":"2023-10-06T03:57:11.363792+0000","last_unstale":"2023-10-06T03:57:11.364126+0000","last_undegraded":"2023-10-06T03:57:11.364126+0000","last_fullsized":"2023-10-06T03:57:11.364126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:49.185450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372624+0000","last_change":"2023-10-06T03:57:11.364974+0000","last_active":"2023-10-06T03:57:11.372624+0000","last_peered":"2023-10-06T03:57:11.372624+0000","last_clean":"2023-10-06T03:57:11.372624+0000","last_became_active":"2023-10-06T03:57:11.364726+0000","last_became_peered":"2023-10-06T03:57:11.364726+0000","last_unstale":"2023-10-06T03:57:11.372624+0000","last_undegraded":"2023-10-06T03:57:11.372624+0000","last_fullsized":"2023-10-06T03:57:11.372624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:56.866733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.379227+0000","last_change":"2023-10-06T03:57:11.375987+0000","last_active":"2023-10-06T03:57:11.379227+0000","last_peered":"2023-10-06T03:57:11.379227+0000","last_clean":"2023-10-06T03:57:11.379227+0000","last_became_active":"2023-10-06T03:57:11.375668+0000","last_became_peered":"2023-10-06T03:57:11.375668+0000","last_unstale":"2023-10-06T03:57:11.379227+0000","last_undegraded":"2023-10-06T03:57:11.379227+0000","last_fullsized":"2023-10-06T03:57:11.379227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:53.810875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.365414+0000","last_change":"2023-10-06T03:57:11.365414+0000","last_active":"2023-10-06T03:57:11.365414+0000","last_peered":"2023-10-06T03:57:11.365414+0000","last_clean":"2023-10-06T03:57:11.365414+0000","last_became_active":"2023-10-06T03:57:11.365149+0000","last_became_peered":"2023-10-06T03:57:11.365149+0000","last_unstale":"2023-10-06T03:57:11.365414+0000","last_undegraded":"2023-10-06T03:57:11.365414+0000","last_fullsized":"2023-10-06T03:57:11.365414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:20.911710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.362366+0000","last_change":"2023-10-06T03:57:11.362366+0000","last_active":"2023-10-06T03:57:11.362366+0000","last_peered":"2023-10-06T03:57:11.362366+0000","last_clean":"2023-10-06T03:57:11.362366+0000","last_became_active":"2023-10-06T03:57:11.362128+0000","last_became_peered":"2023-10-06T03:57:11.362128+0000","last_unstale":"2023-10-06T03:57:11.362366+0000","last_undegraded":"2023-10-06T03:57:11.362366+0000","last_fullsized":"2023-10-06T03:57:11.362366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:53.744674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.379282+0000","last_change":"2023-10-06T03:57:11.375969+0000","last_active":"2023-10-06T03:57:11.379282+0000","last_peered":"2023-10-06T03:57:11.379282+0000","last_clean":"2023-10-06T03:57:11.379282+0000","last_became_active":"2023-10-06T03:57:11.375702+0000","last_became_peered":"2023-10-06T03:57:11.375702+0000","last_unstale":"2023-10-06T03:57:11.379282+0000","last_undegraded":"2023-10-06T03:57:11.379282+0000","last_fullsized":"2023-10-06T03:57:11.379282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:31.204231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377763+0000","last_change":"2023-10-06T03:57:11.375099+0000","last_active":"2023-10-06T03:57:11.377763+0000","last_peered":"2023-10-06T03:57:11.377763+0000","last_clean":"2023-10-06T03:57:11.377763+0000","last_became_active":"2023-10-06T03:57:11.374842+0000","last_became_peered":"2023-10-06T03:57:11.374842+0000","last_unstale":"2023-10-06T03:57:11.377763+0000","last_undegraded":"2023-10-06T03:57:11.377763+0000","last_fullsized":"2023-10-06T03:57:11.377763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:54.438974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.369108+0000","last_change":"2023-10-06T03:57:11.365900+0000","last_active":"2023-10-06T03:57:11.369108+0000","last_peered":"2023-10-06T03:57:11.369108+0000","last_clean":"2023-10-06T03:57:11.369108+0000","last_became_active":"2023-10-06T03:57:11.361349+0000","last_became_peered":"2023-10-06T03:57:11.361349+0000","last_unstale":"2023-10-06T03:57:11.369108+0000","last_undegraded":"2023-10-06T03:57:11.369108+0000","last_fullsized":"2023-10-06T03:57:11.369108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:30.646781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.376082+0000","last_change":"2023-10-06T03:57:11.376082+0000","last_active":"2023-10-06T03:57:11.376082+0000","last_peered":"2023-10-06T03:57:11.376082+0000","last_clean":"2023-10-06T03:57:11.376082+0000","last_became_active":"2023-10-06T03:57:11.375846+0000","last_became_peered":"2023-10-06T03:57:11.375846+0000","last_unstale":"2023-10-06T03:57:11.376082+0000","last_undegraded":"2023-10-06T03:57:11.376082+0000","last_fullsized":"2023-10-06T03:57:11.376082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:28.080614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.356815+0000","last_change":"2023-10-06T03:57:10.348317+0000","last_active":"2023-10-06T03:57:11.356815+0000","last_peered":"2023-10-06T03:57:11.356815+0000","last_clean":"2023-10-06T03:57:11.356815+0000","last_became_active":"2023-10-06T03:57:10.348078+0000","last_became_peered":"2023-10-06T03:57:10.348078+0000","last_unstale":"2023-10-06T03:57:11.356815+0000","last_undegraded":"2023-10-06T03:57:11.356815+0000","last_fullsized":"2023-10-06T03:57:11.356815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:55.058324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.370506+0000","last_change":"2023-10-06T03:57:11.370506+0000","last_active":"2023-10-06T03:57:11.370506+0000","last_peered":"2023-10-06T03:57:11.370506+0000","last_clean":"2023-10-06T03:57:11.370506+0000","last_became_active":"2023-10-06T03:57:11.370245+0000","last_became_peered":"2023-10-06T03:57:11.370245+0000","last_unstale":"2023-10-06T03:57:11.370506+0000","last_undegraded":"2023-10-06T03:57:11.370506+0000","last_fullsized":"2023-10-06T03:57:11.370506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:47:18.189772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367542+0000","last_change":"2023-10-06T03:57:10.345401+0000","last_active":"2023-10-06T03:57:11.367542+0000","last_peered":"2023-10-06T03:57:11.367542+0000","last_clean":"2023-10-06T03:57:11.367542+0000","last_became_active":"2023-10-06T03:57:10.345150+0000","last_became_peered":"2023-10-06T03:57:10.345150+0000","last_unstale":"2023-10-06T03:57:11.367542+0000","last_undegraded":"2023-10-06T03:57:11.367542+0000","last_fullsized":"2023-10-06T03:57:11.367542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:25.758193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.376046+0000","last_change":"2023-10-06T03:57:11.376046+0000","last_active":"2023-10-06T03:57:11.376046+0000","last_peered":"2023-10-06T03:57:11.376046+0000","last_clean":"2023-10-06T03:57:11.376046+0000","last_became_active":"2023-10-06T03:57:11.375790+0000","last_became_peered":"2023-10-06T03:57:11.375790+0000","last_unstale":"2023-10-06T03:57:11.376046+0000","last_undegraded":"2023-10-06T03:57:11.376046+0000","last_fullsized":"2023-10-06T03:57:11.376046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:33.775431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.351291+0000","last_change":"2023-10-06T03:57:10.347562+0000","last_active":"2023-10-06T03:57:10.351291+0000","last_peered":"2023-10-06T03:57:10.351291+0000","last_clean":"2023-10-06T03:57:10.351291+0000","last_became_active":"2023-10-06T03:57:10.347294+0000","last_became_peered":"2023-10-06T03:57:10.347294+0000","last_unstale":"2023-10-06T03:57:10.351291+0000","last_undegraded":"2023-10-06T03:57:10.351291+0000","last_fullsized":"2023-10-06T03:57:10.351291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:25.090992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372022+0000","last_change":"2023-10-06T03:57:11.372022+0000","last_active":"2023-10-06T03:57:11.372022+0000","last_peered":"2023-10-06T03:57:11.372022+0000","last_clean":"2023-10-06T03:57:11.372022+0000","last_became_active":"2023-10-06T03:57:11.371758+0000","last_became_peered":"2023-10-06T03:57:11.371758+0000","last_unstale":"2023-10-06T03:57:11.372022+0000","last_undegraded":"2023-10-06T03:57:11.372022+0000","last_fullsized":"2023-10-06T03:57:11.372022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:10.922157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.348539+0000","last_change":"2023-10-06T03:57:10.347406+0000","last_active":"2023-10-06T03:57:10.348539+0000","last_peered":"2023-10-06T03:57:10.348539+0000","last_clean":"2023-10-06T03:57:10.348539+0000","last_became_active":"2023-10-06T03:57:10.347151+0000","last_became_peered":"2023-10-06T03:57:10.347151+0000","last_unstale":"2023-10-06T03:57:10.348539+0000","last_undegraded":"2023-10-06T03:57:10.348539+0000","last_fullsized":"2023-10-06T03:57:10.348539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:20.427620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.370240+0000","last_change":"2023-10-06T03:57:11.370240+0000","last_active":"2023-10-06T03:57:11.370240+0000","last_peered":"2023-10-06T03:57:11.370240+0000","last_clean":"2023-10-06T03:57:11.370240+0000","last_became_active":"2023-10-06T03:57:11.369938+0000","last_became_peered":"2023-10-06T03:57:11.369938+0000","last_unstale":"2023-10-06T03:57:11.370240+0000","last_undegraded":"2023-10-06T03:57:11.370240+0000","last_fullsized":"2023-10-06T03:57:11.370240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:07.105832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.350424+0000","last_change":"2023-10-06T03:57:10.346054+0000","last_active":"2023-10-06T03:57:10.350424+0000","last_peered":"2023-10-06T03:57:10.350424+0000","last_clean":"2023-10-06T03:57:10.350424+0000","last_became_active":"2023-10-06T03:57:10.345721+0000","last_became_peered":"2023-10-06T03:57:10.345721+0000","last_unstale":"2023-10-06T03:57:10.350424+0000","last_undegraded":"2023-10-06T03:57:10.350424+0000","last_fullsized":"2023-10-06T03:57:10.350424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:54.481304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.376525+0000","last_change":"2023-10-06T03:57:11.376525+0000","last_active":"2023-10-06T03:57:11.376525+0000","last_peered":"2023-10-06T03:57:11.376525+0000","last_clean":"2023-10-06T03:57:11.376525+0000","last_became_active":"2023-10-06T03:57:11.376291+0000","last_became_peered":"2023-10-06T03:57:11.376291+0000","last_unstale":"2023-10-06T03:57:11.376525+0000","last_undegraded":"2023-10-06T03:57:11.376525+0000","last_fullsized":"2023-10-06T03:57:11.376525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:31.993426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.366706+0000","last_change":"2023-10-06T03:57:10.340531+0000","last_active":"2023-10-06T03:57:11.366706+0000","last_peered":"2023-10-06T03:57:11.366706+0000","last_clean":"2023-10-06T03:57:11.366706+0000","last_became_active":"2023-10-06T03:57:10.340232+0000","last_became_peered":"2023-10-06T03:57:10.340232+0000","last_unstale":"2023-10-06T03:57:11.366706+0000","last_undegraded":"2023-10-06T03:57:11.366706+0000","last_fullsized":"2023-10-06T03:57:11.366706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:43:26.332055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377803+0000","last_change":"2023-10-06T03:57:11.375070+0000","last_active":"2023-10-06T03:57:11.377803+0000","last_peered":"2023-10-06T03:57:11.377803+0000","last_clean":"2023-10-06T03:57:11.377803+0000","last_became_active":"2023-10-06T03:57:11.374892+0000","last_became_peered":"2023-10-06T03:57:11.374892+0000","last_unstale":"2023-10-06T03:57:11.377803+0000","last_undegraded":"2023-10-06T03:57:11.377803+0000","last_fullsized":"2023-10-06T03:57:11.377803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:51.198745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.356848+0000","last_change":"2023-10-06T03:57:10.346758+0000","last_active":"2023-10-06T03:57:11.356848+0000","last_peered":"2023-10-06T03:57:11.356848+0000","last_clean":"2023-10-06T03:57:11.356848+0000","last_became_active":"2023-10-06T03:57:10.346430+0000","last_became_peered":"2023-10-06T03:57:10.346430+0000","last_unstale":"2023-10-06T03:57:11.356848+0000","last_undegraded":"2023-10-06T03:57:11.356848+0000","last_fullsized":"2023-10-06T03:57:11.356848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:10.809242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371658+0000","last_change":"2023-10-06T03:57:11.367808+0000","last_active":"2023-10-06T03:57:11.371658+0000","last_peered":"2023-10-06T03:57:11.371658+0000","last_clean":"2023-10-06T03:57:11.371658+0000","last_became_active":"2023-10-06T03:57:11.367565+0000","last_became_peered":"2023-10-06T03:57:11.367565+0000","last_unstale":"2023-10-06T03:57:11.371658+0000","last_undegraded":"2023-10-06T03:57:11.371658+0000","last_fullsized":"2023-10-06T03:57:11.371658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:14.409116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.365509+0000","last_change":"2023-10-06T03:57:10.351062+0000","last_active":"2023-10-06T03:57:11.365509+0000","last_peered":"2023-10-06T03:57:11.365509+0000","last_clean":"2023-10-06T03:57:11.365509+0000","last_became_active":"2023-10-06T03:57:10.350795+0000","last_became_peered":"2023-10-06T03:57:10.350795+0000","last_unstale":"2023-10-06T03:57:11.365509+0000","last_undegraded":"2023-10-06T03:57:11.365509+0000","last_fullsized":"2023-10-06T03:57:11.365509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:54:45.714518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367489+0000","last_change":"2023-10-06T03:57:11.367489+0000","last_active":"2023-10-06T03:57:11.367489+0000","last_peered":"2023-10-06T03:57:11.367489+0000","last_clean":"2023-10-06T03:57:11.367489+0000","last_became_active":"2023-10-06T03:57:11.367254+0000","last_became_peered":"2023-10-06T03:57:11.367254+0000","last_unstale":"2023-10-06T03:57:11.367489+0000","last_undegraded":"2023-10-06T03:57:11.367489+0000","last_fullsized":"2023-10-06T03:57:11.367489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:04.706485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.344944+0000","last_change":"2023-10-06T03:57:10.338125+0000","last_active":"2023-10-06T03:57:10.344944+0000","last_peered":"2023-10-06T03:57:10.344944+0000","last_clean":"2023-10-06T03:57:10.344944+0000","last_became_active":"2023-10-06T03:57:10.337798+0000","last_became_peered":"2023-10-06T03:57:10.337798+0000","last_unstale":"2023-10-06T03:57:10.344944+0000","last_undegraded":"2023-10-06T03:57:10.344944+0000","last_fullsized":"2023-10-06T03:57:10.344944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:00.482657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.376461+0000","last_change":"2023-10-06T03:57:11.376461+0000","last_active":"2023-10-06T03:57:11.376461+0000","last_peered":"2023-10-06T03:57:11.376461+0000","last_clean":"2023-10-06T03:57:11.376461+0000","last_became_active":"2023-10-06T03:57:11.376223+0000","last_became_peered":"2023-10-06T03:57:11.376223+0000","last_unstale":"2023-10-06T03:57:11.376461+0000","last_undegraded":"2023-10-06T03:57:11.376461+0000","last_fullsized":"2023-10-06T03:57:11.376461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:04:37.408871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.364778+0000","last_change":"2023-10-06T03:57:10.350511+0000","last_active":"2023-10-06T03:57:11.364778+0000","last_peered":"2023-10-06T03:57:11.364778+0000","last_clean":"2023-10-06T03:57:11.364778+0000","last_became_active":"2023-10-06T03:57:10.350219+0000","last_became_peered":"2023-10-06T03:57:10.350219+0000","last_unstale":"2023-10-06T03:57:11.364778+0000","last_undegraded":"2023-10-06T03:57:11.364778+0000","last_fullsized":"2023-10-06T03:57:11.364778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:56.043506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.362700+0000","last_change":"2023-10-06T03:57:11.362700+0000","last_active":"2023-10-06T03:57:11.362700+0000","last_peered":"2023-10-06T03:57:11.362700+0000","last_clean":"2023-10-06T03:57:11.362700+0000","last_became_active":"2023-10-06T03:57:11.362443+0000","last_became_peered":"2023-10-06T03:57:11.362443+0000","last_unstale":"2023-10-06T03:57:11.362700+0000","last_undegraded":"2023-10-06T03:57:11.362700+0000","last_fullsized":"2023-10-06T03:57:11.362700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:21:09.143085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.349221+0000","last_change":"2023-10-06T03:57:10.346259+0000","last_active":"2023-10-06T03:57:10.349221+0000","last_peered":"2023-10-06T03:57:10.349221+0000","last_clean":"2023-10-06T03:57:10.349221+0000","last_became_active":"2023-10-06T03:57:10.345424+0000","last_became_peered":"2023-10-06T03:57:10.345424+0000","last_unstale":"2023-10-06T03:57:10.349221+0000","last_undegraded":"2023-10-06T03:57:10.349221+0000","last_fullsized":"2023-10-06T03:57:10.349221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:19.278206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.376053+0000","last_change":"2023-10-06T03:57:11.376053+0000","last_active":"2023-10-06T03:57:11.376053+0000","last_peered":"2023-10-06T03:57:11.376053+0000","last_clean":"2023-10-06T03:57:11.376053+0000","last_became_active":"2023-10-06T03:57:11.375824+0000","last_became_peered":"2023-10-06T03:57:11.375824+0000","last_unstale":"2023-10-06T03:57:11.376053+0000","last_undegraded":"2023-10-06T03:57:11.376053+0000","last_fullsized":"2023-10-06T03:57:11.376053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:24:26.156866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.366961+0000","last_change":"2023-10-06T03:57:10.350408+0000","last_active":"2023-10-06T03:57:11.366961+0000","last_peered":"2023-10-06T03:57:11.366961+0000","last_clean":"2023-10-06T03:57:11.366961+0000","last_became_active":"2023-10-06T03:57:10.350135+0000","last_became_peered":"2023-10-06T03:57:10.350135+0000","last_unstale":"2023-10-06T03:57:11.366961+0000","last_undegraded":"2023-10-06T03:57:11.366961+0000","last_fullsized":"2023-10-06T03:57:11.366961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:43.522111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.376524+0000","last_change":"2023-10-06T03:57:11.376524+0000","last_active":"2023-10-06T03:57:11.376524+0000","last_peered":"2023-10-06T03:57:11.376524+0000","last_clean":"2023-10-06T03:57:11.376524+0000","last_became_active":"2023-10-06T03:57:11.376236+0000","last_became_peered":"2023-10-06T03:57:11.376236+0000","last_unstale":"2023-10-06T03:57:11.376524+0000","last_undegraded":"2023-10-06T03:57:11.376524+0000","last_fullsized":"2023-10-06T03:57:11.376524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:33.854818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367008+0000","last_change":"2023-10-06T03:57:10.345799+0000","last_active":"2023-10-06T03:57:11.367008+0000","last_peered":"2023-10-06T03:57:11.367008+0000","last_clean":"2023-10-06T03:57:11.367008+0000","last_became_active":"2023-10-06T03:57:10.345510+0000","last_became_peered":"2023-10-06T03:57:10.345510+0000","last_unstale":"2023-10-06T03:57:11.367008+0000","last_undegraded":"2023-10-06T03:57:11.367008+0000","last_fullsized":"2023-10-06T03:57:11.367008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:09.567314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372383+0000","last_change":"2023-10-06T03:57:11.368662+0000","last_active":"2023-10-06T03:57:11.372383+0000","last_peered":"2023-10-06T03:57:11.372383+0000","last_clean":"2023-10-06T03:57:11.372383+0000","last_became_active":"2023-10-06T03:57:11.368375+0000","last_became_peered":"2023-10-06T03:57:11.368375+0000","last_unstale":"2023-10-06T03:57:11.372383+0000","last_undegraded":"2023-10-06T03:57:11.372383+0000","last_fullsized":"2023-10-06T03:57:11.372383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:27.125523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.366893+0000","last_change":"2023-10-06T03:57:10.346945+0000","last_active":"2023-10-06T03:57:11.366893+0000","last_peered":"2023-10-06T03:57:11.366893+0000","last_clean":"2023-10-06T03:57:11.366893+0000","last_became_active":"2023-10-06T03:57:10.346689+0000","last_became_peered":"2023-10-06T03:57:10.346689+0000","last_unstale":"2023-10-06T03:57:11.366893+0000","last_undegraded":"2023-10-06T03:57:11.366893+0000","last_fullsized":"2023-10-06T03:57:11.366893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:43.420455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372308+0000","last_change":"2023-10-06T03:57:11.372308+0000","last_active":"2023-10-06T03:57:11.372308+0000","last_peered":"2023-10-06T03:57:11.372308+0000","last_clean":"2023-10-06T03:57:11.372308+0000","last_became_active":"2023-10-06T03:57:11.371795+0000","last_became_peered":"2023-10-06T03:57:11.371795+0000","last_unstale":"2023-10-06T03:57:11.372308+0000","last_undegraded":"2023-10-06T03:57:11.372308+0000","last_fullsized":"2023-10-06T03:57:11.372308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:00.324478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.351416+0000","last_change":"2023-10-06T03:57:10.348592+0000","last_active":"2023-10-06T03:57:10.351416+0000","last_peered":"2023-10-06T03:57:10.351416+0000","last_clean":"2023-10-06T03:57:10.351416+0000","last_became_active":"2023-10-06T03:57:10.348291+0000","last_became_peered":"2023-10-06T03:57:10.348291+0000","last_unstale":"2023-10-06T03:57:10.351416+0000","last_undegraded":"2023-10-06T03:57:10.351416+0000","last_fullsized":"2023-10-06T03:57:10.351416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:00.660348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.366510+0000","last_change":"2023-10-06T03:57:11.366510+0000","last_active":"2023-10-06T03:57:11.366510+0000","last_peered":"2023-10-06T03:57:11.366510+0000","last_clean":"2023-10-06T03:57:11.366510+0000","last_became_active":"2023-10-06T03:57:11.365077+0000","last_became_peered":"2023-10-06T03:57:11.365077+0000","last_unstale":"2023-10-06T03:57:11.366510+0000","last_undegraded":"2023-10-06T03:57:11.366510+0000","last_fullsized":"2023-10-06T03:57:11.366510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:25.199391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.351559+0000","last_change":"2023-10-06T03:57:10.348418+0000","last_active":"2023-10-06T03:57:10.351559+0000","last_peered":"2023-10-06T03:57:10.351559+0000","last_clean":"2023-10-06T03:57:10.351559+0000","last_became_active":"2023-10-06T03:57:10.348055+0000","last_became_peered":"2023-10-06T03:57:10.348055+0000","last_unstale":"2023-10-06T03:57:10.351559+0000","last_undegraded":"2023-10-06T03:57:10.351559+0000","last_fullsized":"2023-10-06T03:57:10.351559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:26.102022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373629+0000","last_change":"2023-10-06T03:57:11.370493+0000","last_active":"2023-10-06T03:57:11.373629+0000","last_peered":"2023-10-06T03:57:11.373629+0000","last_clean":"2023-10-06T03:57:11.373629+0000","last_became_active":"2023-10-06T03:57:11.370073+0000","last_became_peered":"2023-10-06T03:57:11.370073+0000","last_unstale":"2023-10-06T03:57:11.373629+0000","last_undegraded":"2023-10-06T03:57:11.373629+0000","last_fullsized":"2023-10-06T03:57:11.373629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:58.417951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.354826+0000","last_change":"2023-10-06T03:57:10.352966+0000","last_active":"2023-10-06T03:57:10.354826+0000","last_peered":"2023-10-06T03:57:10.354826+0000","last_clean":"2023-10-06T03:57:10.354826+0000","last_became_active":"2023-10-06T03:57:10.352653+0000","last_became_peered":"2023-10-06T03:57:10.352653+0000","last_unstale":"2023-10-06T03:57:10.354826+0000","last_undegraded":"2023-10-06T03:57:10.354826+0000","last_fullsized":"2023-10-06T03:57:10.354826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:32.092557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371704+0000","last_change":"2023-10-06T03:57:11.364052+0000","last_active":"2023-10-06T03:57:11.371704+0000","last_peered":"2023-10-06T03:57:11.371704+0000","last_clean":"2023-10-06T03:57:11.371704+0000","last_became_active":"2023-10-06T03:57:11.363830+0000","last_became_peered":"2023-10-06T03:57:11.363830+0000","last_unstale":"2023-10-06T03:57:11.371704+0000","last_undegraded":"2023-10-06T03:57:11.371704+0000","last_fullsized":"2023-10-06T03:57:11.371704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:04.084019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.357124+0000","last_change":"2023-10-06T03:57:10.349421+0000","last_active":"2023-10-06T03:57:11.357124+0000","last_peered":"2023-10-06T03:57:11.357124+0000","last_clean":"2023-10-06T03:57:11.357124+0000","last_became_active":"2023-10-06T03:57:10.349114+0000","last_became_peered":"2023-10-06T03:57:10.349114+0000","last_unstale":"2023-10-06T03:57:11.357124+0000","last_undegraded":"2023-10-06T03:57:11.357124+0000","last_fullsized":"2023-10-06T03:57:11.357124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:08.209791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372296+0000","last_change":"2023-10-06T03:57:11.367516+0000","last_active":"2023-10-06T03:57:11.372296+0000","last_peered":"2023-10-06T03:57:11.372296+0000","last_clean":"2023-10-06T03:57:11.372296+0000","last_became_active":"2023-10-06T03:57:11.367257+0000","last_became_peered":"2023-10-06T03:57:11.367257+0000","last_unstale":"2023-10-06T03:57:11.372296+0000","last_undegraded":"2023-10-06T03:57:11.372296+0000","last_fullsized":"2023-10-06T03:57:11.372296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:43.628819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.351309+0000","last_change":"2023-10-06T03:57:10.347649+0000","last_active":"2023-10-06T03:57:10.351309+0000","last_peered":"2023-10-06T03:57:10.351309+0000","last_clean":"2023-10-06T03:57:10.351309+0000","last_became_active":"2023-10-06T03:57:10.347433+0000","last_became_peered":"2023-10-06T03:57:10.347433+0000","last_unstale":"2023-10-06T03:57:10.351309+0000","last_undegraded":"2023-10-06T03:57:10.351309+0000","last_fullsized":"2023-10-06T03:57:10.351309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:55.951494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372561+0000","last_change":"2023-10-06T03:57:11.370092+0000","last_active":"2023-10-06T03:57:11.372561+0000","last_peered":"2023-10-06T03:57:11.372561+0000","last_clean":"2023-10-06T03:57:11.372561+0000","last_became_active":"2023-10-06T03:57:11.369825+0000","last_became_peered":"2023-10-06T03:57:11.369825+0000","last_unstale":"2023-10-06T03:57:11.372561+0000","last_undegraded":"2023-10-06T03:57:11.372561+0000","last_fullsized":"2023-10-06T03:57:11.372561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:43.415929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367709+0000","last_change":"2023-10-06T03:57:05.663312+0000","last_active":"2023-10-06T03:57:11.367709+0000","last_peered":"2023-10-06T03:57:11.367709+0000","last_clean":"2023-10-06T03:57:11.367709+0000","last_became_active":"2023-10-06T03:57:05.663046+0000","last_became_peered":"2023-10-06T03:57:05.663046+0000","last_unstale":"2023-10-06T03:57:11.367709+0000","last_undegraded":"2023-10-06T03:57:11.367709+0000","last_fullsized":"2023-10-06T03:57:11.367709+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:31.174359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.351387+0000","last_change":"2023-10-06T03:57:10.348930+0000","last_active":"2023-10-06T03:57:10.351387+0000","last_peered":"2023-10-06T03:57:10.351387+0000","last_clean":"2023-10-06T03:57:10.351387+0000","last_became_active":"2023-10-06T03:57:10.348608+0000","last_became_peered":"2023-10-06T03:57:10.348608+0000","last_unstale":"2023-10-06T03:57:10.351387+0000","last_undegraded":"2023-10-06T03:57:10.351387+0000","last_fullsized":"2023-10-06T03:57:10.351387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:00:09.283718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377785+0000","last_change":"2023-10-06T03:57:11.371108+0000","last_active":"2023-10-06T03:57:11.377785+0000","last_peered":"2023-10-06T03:57:11.377785+0000","last_clean":"2023-10-06T03:57:11.377785+0000","last_became_active":"2023-10-06T03:57:11.370847+0000","last_became_peered":"2023-10-06T03:57:11.370847+0000","last_unstale":"2023-10-06T03:57:11.377785+0000","last_undegraded":"2023-10-06T03:57:11.377785+0000","last_fullsized":"2023-10-06T03:57:11.377785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:38.111171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.345339+0000","last_change":"2023-10-06T03:57:05.318834+0000","last_active":"2023-10-06T03:57:10.345339+0000","last_peered":"2023-10-06T03:57:10.345339+0000","last_clean":"2023-10-06T03:57:10.345339+0000","last_became_active":"2023-10-06T03:57:05.318572+0000","last_became_peered":"2023-10-06T03:57:05.318572+0000","last_unstale":"2023-10-06T03:57:10.345339+0000","last_undegraded":"2023-10-06T03:57:10.345339+0000","last_fullsized":"2023-10-06T03:57:10.345339+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:21.951095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.358229+0000","last_change":"2023-10-06T03:57:10.348556+0000","last_active":"2023-10-06T03:57:11.358229+0000","last_peered":"2023-10-06T03:57:11.358229+0000","last_clean":"2023-10-06T03:57:11.358229+0000","last_became_active":"2023-10-06T03:57:10.348271+0000","last_became_peered":"2023-10-06T03:57:10.348271+0000","last_unstale":"2023-10-06T03:57:11.358229+0000","last_undegraded":"2023-10-06T03:57:11.358229+0000","last_fullsized":"2023-10-06T03:57:11.358229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:58.470989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.368920+0000","last_change":"2023-10-06T03:57:11.365237+0000","last_active":"2023-10-06T03:57:11.368920+0000","last_peered":"2023-10-06T03:57:11.368920+0000","last_clean":"2023-10-06T03:57:11.368920+0000","last_became_active":"2023-10-06T03:57:11.365006+0000","last_became_peered":"2023-10-06T03:57:11.365006+0000","last_unstale":"2023-10-06T03:57:11.368920+0000","last_undegraded":"2023-10-06T03:57:11.368920+0000","last_fullsized":"2023-10-06T03:57:11.368920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:33.005015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.342459+0000","last_change":"2023-10-06T03:57:05.314695+0000","last_active":"2023-10-06T03:57:10.342459+0000","last_peered":"2023-10-06T03:57:10.342459+0000","last_clean":"2023-10-06T03:57:10.342459+0000","last_became_active":"2023-10-06T03:57:05.314408+0000","last_became_peered":"2023-10-06T03:57:05.314408+0000","last_unstale":"2023-10-06T03:57:10.342459+0000","last_undegraded":"2023-10-06T03:57:10.342459+0000","last_fullsized":"2023-10-06T03:57:10.342459+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:19.042145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367206+0000","last_change":"2023-10-06T03:57:10.346078+0000","last_active":"2023-10-06T03:57:11.367206+0000","last_peered":"2023-10-06T03:57:11.367206+0000","last_clean":"2023-10-06T03:57:11.367206+0000","last_became_active":"2023-10-06T03:57:10.345769+0000","last_became_peered":"2023-10-06T03:57:10.345769+0000","last_unstale":"2023-10-06T03:57:11.367206+0000","last_undegraded":"2023-10-06T03:57:11.367206+0000","last_fullsized":"2023-10-06T03:57:11.367206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:16.630681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.376583+0000","last_change":"2023-10-06T03:57:11.370372+0000","last_active":"2023-10-06T03:57:11.376583+0000","last_peered":"2023-10-06T03:57:11.376583+0000","last_clean":"2023-10-06T03:57:11.376583+0000","last_became_active":"2023-10-06T03:57:11.370076+0000","last_became_peered":"2023-10-06T03:57:11.370076+0000","last_unstale":"2023-10-06T03:57:11.376583+0000","last_undegraded":"2023-10-06T03:57:11.376583+0000","last_fullsized":"2023-10-06T03:57:11.376583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:18:59.517155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.345962+0000","last_change":"2023-10-06T03:57:05.318778+0000","last_active":"2023-10-06T03:57:10.345962+0000","last_peered":"2023-10-06T03:57:10.345962+0000","last_clean":"2023-10-06T03:57:10.345962+0000","last_became_active":"2023-10-06T03:57:05.318456+0000","last_became_peered":"2023-10-06T03:57:05.318456+0000","last_unstale":"2023-10-06T03:57:10.345962+0000","last_undegraded":"2023-10-06T03:57:10.345962+0000","last_fullsized":"2023-10-06T03:57:10.345962+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:39.769241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.362382+0000","last_change":"2023-10-06T03:57:05.664245+0000","last_active":"2023-10-06T03:57:11.362382+0000","last_peered":"2023-10-06T03:57:11.362382+0000","last_clean":"2023-10-06T03:57:11.362382+0000","last_became_active":"2023-10-06T03:57:05.663426+0000","last_became_peered":"2023-10-06T03:57:05.663426+0000","last_unstale":"2023-10-06T03:57:11.362382+0000","last_undegraded":"2023-10-06T03:57:11.362382+0000","last_fullsized":"2023-10-06T03:57:11.362382+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:29.322599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.351611+0000","last_change":"2023-10-06T03:57:10.351059+0000","last_active":"2023-10-06T03:57:10.351611+0000","last_peered":"2023-10-06T03:57:10.351611+0000","last_clean":"2023-10-06T03:57:10.351611+0000","last_became_active":"2023-10-06T03:57:10.348879+0000","last_became_peered":"2023-10-06T03:57:10.348879+0000","last_unstale":"2023-10-06T03:57:10.351611+0000","last_undegraded":"2023-10-06T03:57:10.351611+0000","last_fullsized":"2023-10-06T03:57:10.351611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:35.764148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373786+0000","last_change":"2023-10-06T03:57:11.373786+0000","last_active":"2023-10-06T03:57:11.373786+0000","last_peered":"2023-10-06T03:57:11.373786+0000","last_clean":"2023-10-06T03:57:11.373786+0000","last_became_active":"2023-10-06T03:57:11.373452+0000","last_became_peered":"2023-10-06T03:57:11.373452+0000","last_unstale":"2023-10-06T03:57:11.373786+0000","last_undegraded":"2023-10-06T03:57:11.373786+0000","last_fullsized":"2023-10-06T03:57:11.373786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:20.524065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.364987+0000","last_change":"2023-10-06T03:57:05.318790+0000","last_active":"2023-10-06T03:57:11.364987+0000","last_peered":"2023-10-06T03:57:11.364987+0000","last_clean":"2023-10-06T03:57:11.364987+0000","last_became_active":"2023-10-06T03:57:05.318531+0000","last_became_peered":"2023-10-06T03:57:05.318531+0000","last_unstale":"2023-10-06T03:57:11.364987+0000","last_undegraded":"2023-10-06T03:57:11.364987+0000","last_fullsized":"2023-10-06T03:57:11.364987+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:12.876998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.346053+0000","last_change":"2023-10-06T03:57:10.343328+0000","last_active":"2023-10-06T03:57:10.346053+0000","last_peered":"2023-10-06T03:57:10.346053+0000","last_clean":"2023-10-06T03:57:10.346053+0000","last_became_active":"2023-10-06T03:57:10.343051+0000","last_became_peered":"2023-10-06T03:57:10.343051+0000","last_unstale":"2023-10-06T03:57:10.346053+0000","last_undegraded":"2023-10-06T03:57:10.346053+0000","last_fullsized":"2023-10-06T03:57:10.346053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:27.722475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373581+0000","last_change":"2023-10-06T03:57:11.373581+0000","last_active":"2023-10-06T03:57:11.373581+0000","last_peered":"2023-10-06T03:57:11.373581+0000","last_clean":"2023-10-06T03:57:11.373581+0000","last_became_active":"2023-10-06T03:57:11.373264+0000","last_became_peered":"2023-10-06T03:57:11.373264+0000","last_unstale":"2023-10-06T03:57:11.373581+0000","last_undegraded":"2023-10-06T03:57:11.373581+0000","last_fullsized":"2023-10-06T03:57:11.373581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:15.950245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.366791+0000","last_change":"2023-10-06T03:57:05.663770+0000","last_active":"2023-10-06T03:57:11.366791+0000","last_peered":"2023-10-06T03:57:11.366791+0000","last_clean":"2023-10-06T03:57:11.366791+0000","last_became_active":"2023-10-06T03:57:05.663501+0000","last_became_peered":"2023-10-06T03:57:05.663501+0000","last_unstale":"2023-10-06T03:57:11.366791+0000","last_undegraded":"2023-10-06T03:57:11.366791+0000","last_fullsized":"2023-10-06T03:57:11.366791+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:09.806915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367831+0000","last_change":"2023-10-06T03:57:10.346585+0000","last_active":"2023-10-06T03:57:11.367831+0000","last_peered":"2023-10-06T03:57:11.367831+0000","last_clean":"2023-10-06T03:57:11.367831+0000","last_became_active":"2023-10-06T03:57:10.345743+0000","last_became_peered":"2023-10-06T03:57:10.345743+0000","last_unstale":"2023-10-06T03:57:11.367831+0000","last_undegraded":"2023-10-06T03:57:11.367831+0000","last_fullsized":"2023-10-06T03:57:11.367831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:54.589702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373844+0000","last_change":"2023-10-06T03:57:11.373844+0000","last_active":"2023-10-06T03:57:11.373844+0000","last_peered":"2023-10-06T03:57:11.373844+0000","last_clean":"2023-10-06T03:57:11.373844+0000","last_became_active":"2023-10-06T03:57:11.373543+0000","last_became_peered":"2023-10-06T03:57:11.373543+0000","last_unstale":"2023-10-06T03:57:11.373844+0000","last_undegraded":"2023-10-06T03:57:11.373844+0000","last_fullsized":"2023-10-06T03:57:11.373844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:15.152354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.348501+0000","last_change":"2023-10-06T03:57:10.344618+0000","last_active":"2023-10-06T03:57:10.348501+0000","last_peered":"2023-10-06T03:57:10.348501+0000","last_clean":"2023-10-06T03:57:10.348501+0000","last_became_active":"2023-10-06T03:57:10.344324+0000","last_became_peered":"2023-10-06T03:57:10.344324+0000","last_unstale":"2023-10-06T03:57:10.348501+0000","last_undegraded":"2023-10-06T03:57:10.348501+0000","last_fullsized":"2023-10-06T03:57:10.348501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:06.062331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.368350+0000","last_change":"2023-10-06T03:57:05.317973+0000","last_active":"2023-10-06T03:57:11.368350+0000","last_peered":"2023-10-06T03:57:11.368350+0000","last_clean":"2023-10-06T03:57:11.368350+0000","last_became_active":"2023-10-06T03:57:05.317715+0000","last_became_peered":"2023-10-06T03:57:05.317715+0000","last_unstale":"2023-10-06T03:57:11.368350+0000","last_undegraded":"2023-10-06T03:57:11.368350+0000","last_fullsized":"2023-10-06T03:57:11.368350+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:37:51.858909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.369706+0000","last_change":"2023-10-06T03:57:11.369706+0000","last_active":"2023-10-06T03:57:11.369706+0000","last_peered":"2023-10-06T03:57:11.369706+0000","last_clean":"2023-10-06T03:57:11.369706+0000","last_became_active":"2023-10-06T03:57:11.369445+0000","last_became_peered":"2023-10-06T03:57:11.369445+0000","last_unstale":"2023-10-06T03:57:11.369706+0000","last_undegraded":"2023-10-06T03:57:11.369706+0000","last_fullsized":"2023-10-06T03:57:11.369706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:52.999421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.339408+0000","last_change":"2023-10-06T03:57:05.320847+0000","last_active":"2023-10-06T03:57:10.339408+0000","last_peered":"2023-10-06T03:57:10.339408+0000","last_clean":"2023-10-06T03:57:10.339408+0000","last_became_active":"2023-10-06T03:57:05.320583+0000","last_became_peered":"2023-10-06T03:57:05.320583+0000","last_unstale":"2023-10-06T03:57:10.339408+0000","last_undegraded":"2023-10-06T03:57:10.339408+0000","last_fullsized":"2023-10-06T03:57:10.339408+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:12.772449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.345588+0000","last_change":"2023-10-06T03:57:10.341132+0000","last_active":"2023-10-06T03:57:10.345588+0000","last_peered":"2023-10-06T03:57:10.345588+0000","last_clean":"2023-10-06T03:57:10.345588+0000","last_became_active":"2023-10-06T03:57:10.339424+0000","last_became_peered":"2023-10-06T03:57:10.339424+0000","last_unstale":"2023-10-06T03:57:10.345588+0000","last_undegraded":"2023-10-06T03:57:10.345588+0000","last_fullsized":"2023-10-06T03:57:10.345588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:39.675711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.374372+0000","last_change":"2023-10-06T03:57:11.372497+0000","last_active":"2023-10-06T03:57:11.374372+0000","last_peered":"2023-10-06T03:57:11.374372+0000","last_clean":"2023-10-06T03:57:11.374372+0000","last_became_active":"2023-10-06T03:57:11.372215+0000","last_became_peered":"2023-10-06T03:57:11.372215+0000","last_unstale":"2023-10-06T03:57:11.374372+0000","last_undegraded":"2023-10-06T03:57:11.374372+0000","last_fullsized":"2023-10-06T03:57:11.374372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:23.559480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.346078+0000","last_change":"2023-10-06T03:57:10.344051+0000","last_active":"2023-10-06T03:57:10.346078+0000","last_peered":"2023-10-06T03:57:10.346078+0000","last_clean":"2023-10-06T03:57:10.346078+0000","last_became_active":"2023-10-06T03:57:10.343865+0000","last_became_peered":"2023-10-06T03:57:10.343865+0000","last_unstale":"2023-10-06T03:57:10.346078+0000","last_undegraded":"2023-10-06T03:57:10.346078+0000","last_fullsized":"2023-10-06T03:57:10.346078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:32.522470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.369927+0000","last_change":"2023-10-06T03:57:11.369927+0000","last_active":"2023-10-06T03:57:11.369927+0000","last_peered":"2023-10-06T03:57:11.369927+0000","last_clean":"2023-10-06T03:57:11.369927+0000","last_became_active":"2023-10-06T03:57:11.369681+0000","last_became_peered":"2023-10-06T03:57:11.369681+0000","last_unstale":"2023-10-06T03:57:11.369927+0000","last_undegraded":"2023-10-06T03:57:11.369927+0000","last_fullsized":"2023-10-06T03:57:11.369927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:53.858179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.351660+0000","last_change":"2023-10-06T03:57:10.349666+0000","last_active":"2023-10-06T03:57:10.351660+0000","last_peered":"2023-10-06T03:57:10.351660+0000","last_clean":"2023-10-06T03:57:10.351660+0000","last_became_active":"2023-10-06T03:57:10.349320+0000","last_became_peered":"2023-10-06T03:57:10.349320+0000","last_unstale":"2023-10-06T03:57:10.351660+0000","last_undegraded":"2023-10-06T03:57:10.351660+0000","last_fullsized":"2023-10-06T03:57:10.351660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:45:49.709519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373619+0000","last_change":"2023-10-06T03:57:11.370442+0000","last_active":"2023-10-06T03:57:11.373619+0000","last_peered":"2023-10-06T03:57:11.373619+0000","last_clean":"2023-10-06T03:57:11.373619+0000","last_became_active":"2023-10-06T03:57:11.370229+0000","last_became_peered":"2023-10-06T03:57:11.370229+0000","last_unstale":"2023-10-06T03:57:11.373619+0000","last_undegraded":"2023-10-06T03:57:11.373619+0000","last_fullsized":"2023-10-06T03:57:11.373619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:26.174331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.351687+0000","last_change":"2023-10-06T03:57:10.350263+0000","last_active":"2023-10-06T03:57:10.351687+0000","last_peered":"2023-10-06T03:57:10.351687+0000","last_clean":"2023-10-06T03:57:10.351687+0000","last_became_active":"2023-10-06T03:57:10.350083+0000","last_became_peered":"2023-10-06T03:57:10.350083+0000","last_unstale":"2023-10-06T03:57:10.351687+0000","last_undegraded":"2023-10-06T03:57:10.351687+0000","last_fullsized":"2023-10-06T03:57:10.351687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:05.433032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372850+0000","last_change":"2023-10-06T03:57:11.369397+0000","last_active":"2023-10-06T03:57:11.372850+0000","last_peered":"2023-10-06T03:57:11.372850+0000","last_clean":"2023-10-06T03:57:11.372850+0000","last_became_active":"2023-10-06T03:57:11.369123+0000","last_became_peered":"2023-10-06T03:57:11.369123+0000","last_unstale":"2023-10-06T03:57:11.372850+0000","last_undegraded":"2023-10-06T03:57:11.372850+0000","last_fullsized":"2023-10-06T03:57:11.372850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:28.139036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.351215+0000","last_change":"2023-10-06T03:57:10.347049+0000","last_active":"2023-10-06T03:57:10.351215+0000","last_peered":"2023-10-06T03:57:10.351215+0000","last_clean":"2023-10-06T03:57:10.351215+0000","last_became_active":"2023-10-06T03:57:10.344583+0000","last_became_peered":"2023-10-06T03:57:10.344583+0000","last_unstale":"2023-10-06T03:57:10.351215+0000","last_undegraded":"2023-10-06T03:57:10.351215+0000","last_fullsized":"2023-10-06T03:57:10.351215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:32.280257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.366741+0000","last_change":"2023-10-06T03:57:11.366741+0000","last_active":"2023-10-06T03:57:11.366741+0000","last_peered":"2023-10-06T03:57:11.366741+0000","last_clean":"2023-10-06T03:57:11.366741+0000","last_became_active":"2023-10-06T03:57:11.366500+0000","last_became_peered":"2023-10-06T03:57:11.366500+0000","last_unstale":"2023-10-06T03:57:11.366741+0000","last_undegraded":"2023-10-06T03:57:11.366741+0000","last_fullsized":"2023-10-06T03:57:11.366741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:24.714603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.350305+0000","last_change":"2023-10-06T03:57:10.348170+0000","last_active":"2023-10-06T03:57:10.350305+0000","last_peered":"2023-10-06T03:57:10.350305+0000","last_clean":"2023-10-06T03:57:10.350305+0000","last_became_active":"2023-10-06T03:57:10.347897+0000","last_became_peered":"2023-10-06T03:57:10.347897+0000","last_unstale":"2023-10-06T03:57:10.350305+0000","last_undegraded":"2023-10-06T03:57:10.350305+0000","last_fullsized":"2023-10-06T03:57:10.350305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:40.756246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372969+0000","last_change":"2023-10-06T03:57:11.364368+0000","last_active":"2023-10-06T03:57:11.372969+0000","last_peered":"2023-10-06T03:57:11.372969+0000","last_clean":"2023-10-06T03:57:11.372969+0000","last_became_active":"2023-10-06T03:57:11.364150+0000","last_became_peered":"2023-10-06T03:57:11.364150+0000","last_unstale":"2023-10-06T03:57:11.372969+0000","last_undegraded":"2023-10-06T03:57:11.372969+0000","last_fullsized":"2023-10-06T03:57:11.372969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:11.038080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.366721+0000","last_change":"2023-10-06T03:57:11.366721+0000","last_active":"2023-10-06T03:57:11.366721+0000","last_peered":"2023-10-06T03:57:11.366721+0000","last_clean":"2023-10-06T03:57:11.366721+0000","last_became_active":"2023-10-06T03:57:11.366524+0000","last_became_peered":"2023-10-06T03:57:11.366524+0000","last_unstale":"2023-10-06T03:57:11.366721+0000","last_undegraded":"2023-10-06T03:57:11.366721+0000","last_fullsized":"2023-10-06T03:57:11.366721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:33.044858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371109+0000","last_change":"2023-10-06T03:57:11.371109+0000","last_active":"2023-10-06T03:57:11.371109+0000","last_peered":"2023-10-06T03:57:11.371109+0000","last_clean":"2023-10-06T03:57:11.371109+0000","last_became_active":"2023-10-06T03:57:11.370848+0000","last_became_peered":"2023-10-06T03:57:11.370848+0000","last_unstale":"2023-10-06T03:57:11.371109+0000","last_undegraded":"2023-10-06T03:57:11.371109+0000","last_fullsized":"2023-10-06T03:57:11.371109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:15.919090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372839+0000","last_change":"2023-10-06T03:57:11.364188+0000","last_active":"2023-10-06T03:57:11.372839+0000","last_peered":"2023-10-06T03:57:11.372839+0000","last_clean":"2023-10-06T03:57:11.372839+0000","last_became_active":"2023-10-06T03:57:11.363875+0000","last_became_peered":"2023-10-06T03:57:11.363875+0000","last_unstale":"2023-10-06T03:57:11.372839+0000","last_undegraded":"2023-10-06T03:57:11.372839+0000","last_fullsized":"2023-10-06T03:57:11.372839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:29.236344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377420+0000","last_change":"2023-10-06T03:57:11.377420+0000","last_active":"2023-10-06T03:57:11.377420+0000","last_peered":"2023-10-06T03:57:11.377420+0000","last_clean":"2023-10-06T03:57:11.377420+0000","last_became_active":"2023-10-06T03:57:11.377126+0000","last_became_peered":"2023-10-06T03:57:11.377126+0000","last_unstale":"2023-10-06T03:57:11.377420+0000","last_undegraded":"2023-10-06T03:57:11.377420+0000","last_fullsized":"2023-10-06T03:57:11.377420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:05.041737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377733+0000","last_change":"2023-10-06T03:57:11.375777+0000","last_active":"2023-10-06T03:57:11.377733+0000","last_peered":"2023-10-06T03:57:11.377733+0000","last_clean":"2023-10-06T03:57:11.377733+0000","last_became_active":"2023-10-06T03:57:11.375539+0000","last_became_peered":"2023-10-06T03:57:11.375539+0000","last_unstale":"2023-10-06T03:57:11.377733+0000","last_undegraded":"2023-10-06T03:57:11.377733+0000","last_fullsized":"2023-10-06T03:57:11.377733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:42.707488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64858,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297552,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074288,"statfs":{"total":96636764160,"available":96332070912,"internally_reserved":0,"allocated":270336,"data_stored":662562,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":72194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":72194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297548,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074292,"statfs":{"total":96636764160,"available":96332075008,"internally_reserved":0,"allocated":266240,"data_stored":661609,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":72194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":72194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":72194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":72194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:57:17.448 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:57:17.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.597+0000 7f26237e6700 1 Processor -- start 2023-10-06T03:57:17.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.597+0000 7f26237e6700 1 -- start start 2023-10-06T03:57:17.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.598+0000 7f26237e6700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f261c0ca630 0x7f261c0c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:17.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.598+0000 7f26237e6700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f261c06d530 con 0x7f261c156f50 2023-10-06T03:57:17.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.598+0000 7f26237e6700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f261c06d9c0 con 0x7f261c068920 2023-10-06T03:57:17.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.598+0000 7f26237e6700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f261c06f0c0 con 0x7f261c0ca630 2023-10-06T03:57:17.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.598+0000 7f2621d83700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f261c156f50 0x7f261c1608c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39934/0 (socket says 172.21.15.157:39934) 2023-10-06T03:57:17.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.598+0000 7f2621d83700 1 -- 172.21.15.157:0/1951285972 learned_addr learned my addr 172.21.15.157:0/1951285972 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:17.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.598+0000 7f2621582700 1 --2- 172.21.15.157:0/1951285972 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f261c0ca630 0x7f261c0c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:17.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.598+0000 7f2613fff700 1 -- 172.21.15.157:0/1951285972 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 52839573 0 0) 0x7f261c06d530 con 0x7f261c156f50 2023-10-06T03:57:17.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.598+0000 7f2613fff700 1 -- 172.21.15.157:0/1951285972 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2608003570 con 0x7f261c156f50 2023-10-06T03:57:17.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.599+0000 7f2613fff700 1 -- 172.21.15.157:0/1951285972 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3147740021 0 0) 0x7f2608003570 con 0x7f261c156f50 2023-10-06T03:57:17.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.599+0000 7f2613fff700 1 -- 172.21.15.157:0/1951285972 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f261c0ca630 msgr2=0x7f261c0c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:17.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.599+0000 7f2613fff700 1 --2- 172.21.15.157:0/1951285972 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f261c0ca630 0x7f261c0c6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.599+0000 7f2613fff700 1 -- 172.21.15.157:0/1951285972 >> v1:172.21.15.175:6789/0 conn(0x7f261c068920 legacy=0x7f261c156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:17.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.599+0000 7f2613fff700 1 -- 172.21.15.157:0/1951285972 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261c06f3a0 con 0x7f261c156f50 2023-10-06T03:57:17.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.599+0000 7f2613fff700 1 -- 172.21.15.157:0/1951285972 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f26140017f0 con 0x7f261c156f50 2023-10-06T03:57:17.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.599+0000 7f2613fff700 1 -- 172.21.15.157:0/1951285972 >> v1:172.21.15.157:6789/0 conn(0x7f261c156f50 legacy=0x7f261c1608c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:17.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.599+0000 7f2613fff700 1 --2- 172.21.15.157:0/1951285972 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f261c0ca630 0x7f2608003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:17.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.599+0000 7f2613fff700 1 --2- 172.21.15.157:0/1951285972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2608003fd0 0x7f2608004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:17.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.599+0000 7f2613fff700 1 --2- 172.21.15.157:0/1951285972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2608004d50 0x7f26080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:17.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.599+0000 7f2613fff700 1 -- 172.21.15.157:0/1951285972 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261c06f0c0 con 0x7f2608004d50 2023-10-06T03:57:17.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.599+0000 7f2613fff700 1 -- 172.21.15.157:0/1951285972 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260800bba0 con 0x7f2608003fd0 2023-10-06T03:57:17.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.599+0000 7f2613fff700 1 -- 172.21.15.157:0/1951285972 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f260800bd60 con 0x7f261c0ca630 2023-10-06T03:57:17.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.599+0000 7f2621d83700 1 --2- 172.21.15.157:0/1951285972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2608004d50 0x7f26080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:17.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.599+0000 7f2621582700 1 --2- 172.21.15.157:0/1951285972 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f261c0ca630 0x7f2608003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:17.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.599+0000 7f2620d81700 1 --2- 172.21.15.157:0/1951285972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2608003fd0 0x7f2608004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:17.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.599+0000 7f2621d83700 1 -- 172.21.15.157:0/1951285972 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f261c0ca630 msgr2=0x7f2608003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:17.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.599+0000 7f2621d83700 1 --2- 172.21.15.157:0/1951285972 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f261c0ca630 0x7f2608003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.599+0000 7f2621d83700 1 -- 172.21.15.157:0/1951285972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2608003fd0 msgr2=0x7f2608004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:17.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.599+0000 7f2621d83700 1 --2- 172.21.15.157:0/1951285972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2608003fd0 0x7f2608004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.599+0000 7f2621d83700 1 -- 172.21.15.157:0/1951285972 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2608003570 con 0x7f2608004d50 2023-10-06T03:57:17.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.599+0000 7f2621d83700 1 --2- 172.21.15.157:0/1951285972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2608004d50 0x7f26080051c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f261400a850 tx=0x7f2614002120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc11d31ffc67c75 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:17.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.599+0000 7f2613fff700 1 -- 172.21.15.157:0/1951285972 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26140070f0 con 0x7f2608004d50 2023-10-06T03:57:17.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.599+0000 7f2613fff700 1 -- 172.21.15.157:0/1951285972 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2614007250 con 0x7f2608004d50 2023-10-06T03:57:17.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.599+0000 7f2613fff700 1 -- 172.21.15.157:0/1951285972 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614007610 con 0x7f2608004d50 2023-10-06T03:57:17.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.600+0000 7f26237e6700 1 -- 172.21.15.157:0/1951285972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2608004d50 msgr2=0x7f26080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:17.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.600+0000 7f26237e6700 1 --2- 172.21.15.157:0/1951285972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2608004d50 0x7f26080051c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f261400a850 tx=0x7f2614002120 comp rx=0 tx=0).stop 2023-10-06T03:57:17.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.600+0000 7f2621d83700 1 -- 172.21.15.157:0/1951285972 reap_dead start 2023-10-06T03:57:17.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.600+0000 7f26237e6700 1 -- 172.21.15.157:0/1951285972 shutdown_connections 2023-10-06T03:57:17.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.600+0000 7f26237e6700 1 -- 172.21.15.157:0/1951285972 >> 172.21.15.157:0/1951285972 conn(0x7f261c0692b0 msgr2=0x7f261c0696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:17.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.600+0000 7f26237e6700 1 -- 172.21.15.157:0/1951285972 shutdown_connections 2023-10-06T03:57:17.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.600+0000 7f26237e6700 1 -- 172.21.15.157:0/1951285972 wait complete. 2023-10-06T03:57:17.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.601+0000 7f26237e6700 1 Processor -- start 2023-10-06T03:57:17.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.601+0000 7f26237e6700 1 -- start start 2023-10-06T03:57:17.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.602+0000 7f26237e6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2608004d50 0x7f261c169b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:17.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.602+0000 7f26237e6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f261c0c4f90 0x7f261c0aaad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:17.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.602+0000 7f26237e6700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f261c151780 0x7f261c151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:17.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.602+0000 7f26237e6700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261c0ab8f0 con 0x7f2608004d50 2023-10-06T03:57:17.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.602+0000 7f26237e6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261c1696e0 con 0x7f261c0c4f90 2023-10-06T03:57:17.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.602+0000 7f26237e6700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f261c1693b0 con 0x7f261c151780 2023-10-06T03:57:17.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.602+0000 7f2621582700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2608004d50 0x7f261c169b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:17.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.602+0000 7f2621582700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2608004d50 0x7f261c169b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34258/0 (socket says 172.21.15.157:34258) 2023-10-06T03:57:17.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.602+0000 7f2621582700 1 -- 172.21.15.157:0/646161983 learned_addr learned my addr 172.21.15.157:0/646161983 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:17.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.602+0000 7f2621d83700 1 --2- 172.21.15.157:0/646161983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f261c151780 0x7f261c151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:17.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.602+0000 7f2620d81700 1 --2- 172.21.15.157:0/646161983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f261c0c4f90 0x7f261c0aaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:17.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.602+0000 7f2621582700 1 -- 172.21.15.157:0/646161983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f261c151780 msgr2=0x7f261c151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:17.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.602+0000 7f2621582700 1 --2- 172.21.15.157:0/646161983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f261c151780 0x7f261c151c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.602+0000 7f2621582700 1 -- 172.21.15.157:0/646161983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f261c0c4f90 msgr2=0x7f261c0aaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:17.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.602+0000 7f2621582700 1 --2- 172.21.15.157:0/646161983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f261c0c4f90 0x7f261c0aaad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.602+0000 7f2621582700 1 -- 172.21.15.157:0/646161983 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261c15aee0 con 0x7f2608004d50 2023-10-06T03:57:17.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.602+0000 7f2621582700 1 --2- 172.21.15.157:0/646161983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2608004d50 0x7f261c169b70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2618011b10 tx=0x7f26180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:17.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.603+0000 7f26117fa700 1 -- 172.21.15.157:0/646161983 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261800f940 con 0x7f2608004d50 2023-10-06T03:57:17.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.603+0000 7f26117fa700 1 -- 172.21.15.157:0/646161983 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261800faa0 con 0x7f2608004d50 2023-10-06T03:57:17.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.603+0000 7f26237e6700 1 -- 172.21.15.157:0/646161983 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f261c15a470 con 0x7f2608004d50 2023-10-06T03:57:17.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.603+0000 7f26117fa700 1 -- 172.21.15.157:0/646161983 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261800fe20 con 0x7f2608004d50 2023-10-06T03:57:17.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.603+0000 7f26237e6700 1 -- 172.21.15.157:0/646161983 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f261c152160 con 0x7f2608004d50 2023-10-06T03:57:17.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.604+0000 7f26237e6700 1 -- 172.21.15.157:0/646161983 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f261c06a8f0 con 0x7f2608004d50 2023-10-06T03:57:17.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.604+0000 7f26117fa700 1 -- 172.21.15.157:0/646161983 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2618014650 con 0x7f2608004d50 2023-10-06T03:57:17.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.604+0000 7f26117fa700 1 --2- 172.21.15.157:0/646161983 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2600066f30 0x7f26000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:17.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.604+0000 7f26117fa700 1 -- 172.21.15.157:0/646161983 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2618018030 con 0x7f2608004d50 2023-10-06T03:57:17.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.604+0000 7f2620d81700 1 --2- 172.21.15.157:0/646161983 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2600066f30 0x7f26000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:17.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.605+0000 7f2620d81700 1 --2- 172.21.15.157:0/646161983 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2600066f30 0x7f26000693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f261c154500 tx=0x7f260c007d10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:17.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.609+0000 7f26117fa700 1 -- 172.21.15.157:0/646161983 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2618052130 con 0x7f2608004d50 2023-10-06T03:57:17.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.787+0000 7f26237e6700 1 -- 172.21.15.157:0/646161983 --> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f261c169820 con 0x7f2600066f30 2023-10-06T03:57:17.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.795+0000 7f26117fa700 1 -- 172.21.15.157:0/646161983 <== mgr.4107 v2:172.21.15.175:6832/90661 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286697 (secure 0 0 0) 0x7f260c003d70 con 0x7f2600066f30 2023-10-06T03:57:17.797 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:17.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.798+0000 7f26237e6700 1 -- 172.21.15.157:0/646161983 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2600066f30 msgr2=0x7f26000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:17.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.798+0000 7f26237e6700 1 --2- 172.21.15.157:0/646161983 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2600066f30 0x7f26000693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f261c154500 tx=0x7f260c007d10 comp rx=0 tx=0).stop 2023-10-06T03:57:17.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.798+0000 7f26237e6700 1 -- 172.21.15.157:0/646161983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2608004d50 msgr2=0x7f261c169b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:17.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.798+0000 7f26237e6700 1 --2- 172.21.15.157:0/646161983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2608004d50 0x7f261c169b70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2618011b10 tx=0x7f26180091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.798+0000 7f26237e6700 1 -- 172.21.15.157:0/646161983 shutdown_connections 2023-10-06T03:57:17.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.798+0000 7f26237e6700 1 --2- 172.21.15.157:0/646161983 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2600066f30 0x7f26000693e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.798+0000 7f26237e6700 1 --2- 172.21.15.157:0/646161983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f261c151780 0x7f261c151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.798+0000 7f26237e6700 1 --2- 172.21.15.157:0/646161983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2608004d50 0x7f261c169b70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.798+0000 7f26237e6700 1 --2- 172.21.15.157:0/646161983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f261c0c4f90 0x7f261c0aaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.798+0000 7f26237e6700 1 -- 172.21.15.157:0/646161983 >> 172.21.15.157:0/646161983 conn(0x7f261c0692b0 msgr2=0x7f261c1554a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:17.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.798+0000 7f26237e6700 1 -- 172.21.15.157:0/646161983 shutdown_connections 2023-10-06T03:57:17.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.798+0000 7f26237e6700 1 -- 172.21.15.157:0/646161983 wait complete. 2023-10-06T03:57:17.812 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-10-06T03:57:17.813 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-06T03:57:17.426847+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":84,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2380028,"kb_used_data":1724,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752594692,"statfs":{"total":773094113280,"available":770656964608,"internally_reserved":0,"allocated":1765376,"data_stored":1757335,"data_compressed":64858,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":14,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.098114"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372019+0000","last_change":"2023-10-06T03:57:11.372019+0000","last_active":"2023-10-06T03:57:11.372019+0000","last_peered":"2023-10-06T03:57:11.372019+0000","last_clean":"2023-10-06T03:57:11.372019+0000","last_became_active":"2023-10-06T03:57:11.371756+0000","last_became_peered":"2023-10-06T03:57:11.371756+0000","last_unstale":"2023-10-06T03:57:11.372019+0000","last_undegraded":"2023-10-06T03:57:11.372019+0000","last_fullsized":"2023-10-06T03:57:11.372019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:50.086528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.370371+0000","last_change":"2023-10-06T03:57:11.370371+0000","last_active":"2023-10-06T03:57:11.370371+0000","last_peered":"2023-10-06T03:57:11.370371+0000","last_clean":"2023-10-06T03:57:11.370371+0000","last_became_active":"2023-10-06T03:57:11.370070+0000","last_became_peered":"2023-10-06T03:57:11.370070+0000","last_unstale":"2023-10-06T03:57:11.370371+0000","last_undegraded":"2023-10-06T03:57:11.370371+0000","last_fullsized":"2023-10-06T03:57:11.370371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:00.917607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371725+0000","last_change":"2023-10-06T03:57:11.366298+0000","last_active":"2023-10-06T03:57:11.371725+0000","last_peered":"2023-10-06T03:57:11.371725+0000","last_clean":"2023-10-06T03:57:11.371725+0000","last_became_active":"2023-10-06T03:57:11.366012+0000","last_became_peered":"2023-10-06T03:57:11.366012+0000","last_unstale":"2023-10-06T03:57:11.371725+0000","last_undegraded":"2023-10-06T03:57:11.371725+0000","last_fullsized":"2023-10-06T03:57:11.371725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:45.802287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.365933+0000","last_change":"2023-10-06T03:57:11.365933+0000","last_active":"2023-10-06T03:57:11.365933+0000","last_peered":"2023-10-06T03:57:11.365933+0000","last_clean":"2023-10-06T03:57:11.365933+0000","last_became_active":"2023-10-06T03:57:11.365673+0000","last_became_peered":"2023-10-06T03:57:11.365673+0000","last_unstale":"2023-10-06T03:57:11.365933+0000","last_undegraded":"2023-10-06T03:57:11.365933+0000","last_fullsized":"2023-10-06T03:57:11.365933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:40.953382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.361569+0000","last_change":"2023-10-06T03:57:11.361569+0000","last_active":"2023-10-06T03:57:11.361569+0000","last_peered":"2023-10-06T03:57:11.361569+0000","last_clean":"2023-10-06T03:57:11.361569+0000","last_became_active":"2023-10-06T03:57:11.361302+0000","last_became_peered":"2023-10-06T03:57:11.361302+0000","last_unstale":"2023-10-06T03:57:11.361569+0000","last_undegraded":"2023-10-06T03:57:11.361569+0000","last_fullsized":"2023-10-06T03:57:11.361569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:57.552493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372859+0000","last_change":"2023-10-06T03:57:11.369953+0000","last_active":"2023-10-06T03:57:11.372859+0000","last_peered":"2023-10-06T03:57:11.372859+0000","last_clean":"2023-10-06T03:57:11.372859+0000","last_became_active":"2023-10-06T03:57:11.368775+0000","last_became_peered":"2023-10-06T03:57:11.368775+0000","last_unstale":"2023-10-06T03:57:11.372859+0000","last_undegraded":"2023-10-06T03:57:11.372859+0000","last_fullsized":"2023-10-06T03:57:11.372859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:22.368117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.380867+0000","last_change":"2023-10-06T03:57:11.378048+0000","last_active":"2023-10-06T03:57:11.380867+0000","last_peered":"2023-10-06T03:57:11.380867+0000","last_clean":"2023-10-06T03:57:11.380867+0000","last_became_active":"2023-10-06T03:57:11.377786+0000","last_became_peered":"2023-10-06T03:57:11.377786+0000","last_unstale":"2023-10-06T03:57:11.380867+0000","last_undegraded":"2023-10-06T03:57:11.380867+0000","last_fullsized":"2023-10-06T03:57:11.380867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:12:54.218593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.374003+0000","last_change":"2023-10-06T03:57:11.374003+0000","last_active":"2023-10-06T03:57:11.374003+0000","last_peered":"2023-10-06T03:57:11.374003+0000","last_clean":"2023-10-06T03:57:11.374003+0000","last_became_active":"2023-10-06T03:57:11.373765+0000","last_became_peered":"2023-10-06T03:57:11.373765+0000","last_unstale":"2023-10-06T03:57:11.374003+0000","last_undegraded":"2023-10-06T03:57:11.374003+0000","last_fullsized":"2023-10-06T03:57:11.374003+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:05.838007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373794+0000","last_change":"2023-10-06T03:57:11.373794+0000","last_active":"2023-10-06T03:57:11.373794+0000","last_peered":"2023-10-06T03:57:11.373794+0000","last_clean":"2023-10-06T03:57:11.373794+0000","last_became_active":"2023-10-06T03:57:11.368004+0000","last_became_peered":"2023-10-06T03:57:11.368004+0000","last_unstale":"2023-10-06T03:57:11.373794+0000","last_undegraded":"2023-10-06T03:57:11.373794+0000","last_fullsized":"2023-10-06T03:57:11.373794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:16.760049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.379370+0000","last_change":"2023-10-06T03:57:11.379370+0000","last_active":"2023-10-06T03:57:11.379370+0000","last_peered":"2023-10-06T03:57:11.379370+0000","last_clean":"2023-10-06T03:57:11.379370+0000","last_became_active":"2023-10-06T03:57:11.379107+0000","last_became_peered":"2023-10-06T03:57:11.379107+0000","last_unstale":"2023-10-06T03:57:11.379370+0000","last_undegraded":"2023-10-06T03:57:11.379370+0000","last_fullsized":"2023-10-06T03:57:11.379370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:18.373103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371742+0000","last_change":"2023-10-06T03:57:11.369172+0000","last_active":"2023-10-06T03:57:11.371742+0000","last_peered":"2023-10-06T03:57:11.371742+0000","last_clean":"2023-10-06T03:57:11.371742+0000","last_became_active":"2023-10-06T03:57:11.368256+0000","last_became_peered":"2023-10-06T03:57:11.368256+0000","last_unstale":"2023-10-06T03:57:11.371742+0000","last_undegraded":"2023-10-06T03:57:11.371742+0000","last_fullsized":"2023-10-06T03:57:11.371742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:50.736367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.368685+0000","last_change":"2023-10-06T03:57:11.368685+0000","last_active":"2023-10-06T03:57:11.368685+0000","last_peered":"2023-10-06T03:57:11.368685+0000","last_clean":"2023-10-06T03:57:11.368685+0000","last_became_active":"2023-10-06T03:57:11.368425+0000","last_became_peered":"2023-10-06T03:57:11.368425+0000","last_unstale":"2023-10-06T03:57:11.368685+0000","last_undegraded":"2023-10-06T03:57:11.368685+0000","last_fullsized":"2023-10-06T03:57:11.368685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:06.004423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367646+0000","last_change":"2023-10-06T03:57:11.367646+0000","last_active":"2023-10-06T03:57:11.367646+0000","last_peered":"2023-10-06T03:57:11.367646+0000","last_clean":"2023-10-06T03:57:11.367646+0000","last_became_active":"2023-10-06T03:57:11.367079+0000","last_became_peered":"2023-10-06T03:57:11.367079+0000","last_unstale":"2023-10-06T03:57:11.367646+0000","last_undegraded":"2023-10-06T03:57:11.367646+0000","last_fullsized":"2023-10-06T03:57:11.367646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:09:54.331206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373481+0000","last_change":"2023-10-06T03:57:11.373481+0000","last_active":"2023-10-06T03:57:11.373481+0000","last_peered":"2023-10-06T03:57:11.373481+0000","last_clean":"2023-10-06T03:57:11.373481+0000","last_became_active":"2023-10-06T03:57:11.373216+0000","last_became_peered":"2023-10-06T03:57:11.373216+0000","last_unstale":"2023-10-06T03:57:11.373481+0000","last_undegraded":"2023-10-06T03:57:11.373481+0000","last_fullsized":"2023-10-06T03:57:11.373481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:56.083203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.379250+0000","last_change":"2023-10-06T03:57:11.376012+0000","last_active":"2023-10-06T03:57:11.379250+0000","last_peered":"2023-10-06T03:57:11.379250+0000","last_clean":"2023-10-06T03:57:11.379250+0000","last_became_active":"2023-10-06T03:57:11.375689+0000","last_became_peered":"2023-10-06T03:57:11.375689+0000","last_unstale":"2023-10-06T03:57:11.379250+0000","last_undegraded":"2023-10-06T03:57:11.379250+0000","last_fullsized":"2023-10-06T03:57:11.379250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:42.953739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372989+0000","last_change":"2023-10-06T03:57:11.366808+0000","last_active":"2023-10-06T03:57:11.372989+0000","last_peered":"2023-10-06T03:57:11.372989+0000","last_clean":"2023-10-06T03:57:11.372989+0000","last_became_active":"2023-10-06T03:57:11.366543+0000","last_became_peered":"2023-10-06T03:57:11.366543+0000","last_unstale":"2023-10-06T03:57:11.372989+0000","last_undegraded":"2023-10-06T03:57:11.372989+0000","last_fullsized":"2023-10-06T03:57:11.372989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:16.212967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367276+0000","last_change":"2023-10-06T03:57:11.367276+0000","last_active":"2023-10-06T03:57:11.367276+0000","last_peered":"2023-10-06T03:57:11.367276+0000","last_clean":"2023-10-06T03:57:11.367276+0000","last_became_active":"2023-10-06T03:57:11.366767+0000","last_became_peered":"2023-10-06T03:57:11.366767+0000","last_unstale":"2023-10-06T03:57:11.367276+0000","last_undegraded":"2023-10-06T03:57:11.367276+0000","last_fullsized":"2023-10-06T03:57:11.367276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:43.113575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367129+0000","last_change":"2023-10-06T03:57:11.362667+0000","last_active":"2023-10-06T03:57:11.367129+0000","last_peered":"2023-10-06T03:57:11.367129+0000","last_clean":"2023-10-06T03:57:11.367129+0000","last_became_active":"2023-10-06T03:57:11.362377+0000","last_became_peered":"2023-10-06T03:57:11.362377+0000","last_unstale":"2023-10-06T03:57:11.367129+0000","last_undegraded":"2023-10-06T03:57:11.367129+0000","last_fullsized":"2023-10-06T03:57:11.367129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:40.600383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.364126+0000","last_change":"2023-10-06T03:57:11.364126+0000","last_active":"2023-10-06T03:57:11.364126+0000","last_peered":"2023-10-06T03:57:11.364126+0000","last_clean":"2023-10-06T03:57:11.364126+0000","last_became_active":"2023-10-06T03:57:11.363792+0000","last_became_peered":"2023-10-06T03:57:11.363792+0000","last_unstale":"2023-10-06T03:57:11.364126+0000","last_undegraded":"2023-10-06T03:57:11.364126+0000","last_fullsized":"2023-10-06T03:57:11.364126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:49.185450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372624+0000","last_change":"2023-10-06T03:57:11.364974+0000","last_active":"2023-10-06T03:57:11.372624+0000","last_peered":"2023-10-06T03:57:11.372624+0000","last_clean":"2023-10-06T03:57:11.372624+0000","last_became_active":"2023-10-06T03:57:11.364726+0000","last_became_peered":"2023-10-06T03:57:11.364726+0000","last_unstale":"2023-10-06T03:57:11.372624+0000","last_undegraded":"2023-10-06T03:57:11.372624+0000","last_fullsized":"2023-10-06T03:57:11.372624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:56.866733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.379227+0000","last_change":"2023-10-06T03:57:11.375987+0000","last_active":"2023-10-06T03:57:11.379227+0000","last_peered":"2023-10-06T03:57:11.379227+0000","last_clean":"2023-10-06T03:57:11.379227+0000","last_became_active":"2023-10-06T03:57:11.375668+0000","last_became_peered":"2023-10-06T03:57:11.375668+0000","last_unstale":"2023-10-06T03:57:11.379227+0000","last_undegraded":"2023-10-06T03:57:11.379227+0000","last_fullsized":"2023-10-06T03:57:11.379227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:53.810875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.365414+0000","last_change":"2023-10-06T03:57:11.365414+0000","last_active":"2023-10-06T03:57:11.365414+0000","last_peered":"2023-10-06T03:57:11.365414+0000","last_clean":"2023-10-06T03:57:11.365414+0000","last_became_active":"2023-10-06T03:57:11.365149+0000","last_became_peered":"2023-10-06T03:57:11.365149+0000","last_unstale":"2023-10-06T03:57:11.365414+0000","last_undegraded":"2023-10-06T03:57:11.365414+0000","last_fullsized":"2023-10-06T03:57:11.365414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:20.911710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.362366+0000","last_change":"2023-10-06T03:57:11.362366+0000","last_active":"2023-10-06T03:57:11.362366+0000","last_peered":"2023-10-06T03:57:11.362366+0000","last_clean":"2023-10-06T03:57:11.362366+0000","last_became_active":"2023-10-06T03:57:11.362128+0000","last_became_peered":"2023-10-06T03:57:11.362128+0000","last_unstale":"2023-10-06T03:57:11.362366+0000","last_undegraded":"2023-10-06T03:57:11.362366+0000","last_fullsized":"2023-10-06T03:57:11.362366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:53.744674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.379282+0000","last_change":"2023-10-06T03:57:11.375969+0000","last_active":"2023-10-06T03:57:11.379282+0000","last_peered":"2023-10-06T03:57:11.379282+0000","last_clean":"2023-10-06T03:57:11.379282+0000","last_became_active":"2023-10-06T03:57:11.375702+0000","last_became_peered":"2023-10-06T03:57:11.375702+0000","last_unstale":"2023-10-06T03:57:11.379282+0000","last_undegraded":"2023-10-06T03:57:11.379282+0000","last_fullsized":"2023-10-06T03:57:11.379282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:31.204231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377763+0000","last_change":"2023-10-06T03:57:11.375099+0000","last_active":"2023-10-06T03:57:11.377763+0000","last_peered":"2023-10-06T03:57:11.377763+0000","last_clean":"2023-10-06T03:57:11.377763+0000","last_became_active":"2023-10-06T03:57:11.374842+0000","last_became_peered":"2023-10-06T03:57:11.374842+0000","last_unstale":"2023-10-06T03:57:11.377763+0000","last_undegraded":"2023-10-06T03:57:11.377763+0000","last_fullsized":"2023-10-06T03:57:11.377763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:54.438974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.369108+0000","last_change":"2023-10-06T03:57:11.365900+0000","last_active":"2023-10-06T03:57:11.369108+0000","last_peered":"2023-10-06T03:57:11.369108+0000","last_clean":"2023-10-06T03:57:11.369108+0000","last_became_active":"2023-10-06T03:57:11.361349+0000","last_became_peered":"2023-10-06T03:57:11.361349+0000","last_unstale":"2023-10-06T03:57:11.369108+0000","last_undegraded":"2023-10-06T03:57:11.369108+0000","last_fullsized":"2023-10-06T03:57:11.369108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:30.646781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.376082+0000","last_change":"2023-10-06T03:57:11.376082+0000","last_active":"2023-10-06T03:57:11.376082+0000","last_peered":"2023-10-06T03:57:11.376082+0000","last_clean":"2023-10-06T03:57:11.376082+0000","last_became_active":"2023-10-06T03:57:11.375846+0000","last_became_peered":"2023-10-06T03:57:11.375846+0000","last_unstale":"2023-10-06T03:57:11.376082+0000","last_undegraded":"2023-10-06T03:57:11.376082+0000","last_fullsized":"2023-10-06T03:57:11.376082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:28.080614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.356815+0000","last_change":"2023-10-06T03:57:10.348317+0000","last_active":"2023-10-06T03:57:11.356815+0000","last_peered":"2023-10-06T03:57:11.356815+0000","last_clean":"2023-10-06T03:57:11.356815+0000","last_became_active":"2023-10-06T03:57:10.348078+0000","last_became_peered":"2023-10-06T03:57:10.348078+0000","last_unstale":"2023-10-06T03:57:11.356815+0000","last_undegraded":"2023-10-06T03:57:11.356815+0000","last_fullsized":"2023-10-06T03:57:11.356815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:55.058324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.370506+0000","last_change":"2023-10-06T03:57:11.370506+0000","last_active":"2023-10-06T03:57:11.370506+0000","last_peered":"2023-10-06T03:57:11.370506+0000","last_clean":"2023-10-06T03:57:11.370506+0000","last_became_active":"2023-10-06T03:57:11.370245+0000","last_became_peered":"2023-10-06T03:57:11.370245+0000","last_unstale":"2023-10-06T03:57:11.370506+0000","last_undegraded":"2023-10-06T03:57:11.370506+0000","last_fullsized":"2023-10-06T03:57:11.370506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:47:18.189772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367542+0000","last_change":"2023-10-06T03:57:10.345401+0000","last_active":"2023-10-06T03:57:11.367542+0000","last_peered":"2023-10-06T03:57:11.367542+0000","last_clean":"2023-10-06T03:57:11.367542+0000","last_became_active":"2023-10-06T03:57:10.345150+0000","last_became_peered":"2023-10-06T03:57:10.345150+0000","last_unstale":"2023-10-06T03:57:11.367542+0000","last_undegraded":"2023-10-06T03:57:11.367542+0000","last_fullsized":"2023-10-06T03:57:11.367542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:25.758193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.376046+0000","last_change":"2023-10-06T03:57:11.376046+0000","last_active":"2023-10-06T03:57:11.376046+0000","last_peered":"2023-10-06T03:57:11.376046+0000","last_clean":"2023-10-06T03:57:11.376046+0000","last_became_active":"2023-10-06T03:57:11.375790+0000","last_became_peered":"2023-10-06T03:57:11.375790+0000","last_unstale":"2023-10-06T03:57:11.376046+0000","last_undegraded":"2023-10-06T03:57:11.376046+0000","last_fullsized":"2023-10-06T03:57:11.376046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:33.775431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.351291+0000","last_change":"2023-10-06T03:57:10.347562+0000","last_active":"2023-10-06T03:57:10.351291+0000","last_peered":"2023-10-06T03:57:10.351291+0000","last_clean":"2023-10-06T03:57:10.351291+0000","last_became_active":"2023-10-06T03:57:10.347294+0000","last_became_peered":"2023-10-06T03:57:10.347294+0000","last_unstale":"2023-10-06T03:57:10.351291+0000","last_undegraded":"2023-10-06T03:57:10.351291+0000","last_fullsized":"2023-10-06T03:57:10.351291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:25.090992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372022+0000","last_change":"2023-10-06T03:57:11.372022+0000","last_active":"2023-10-06T03:57:11.372022+0000","last_peered":"2023-10-06T03:57:11.372022+0000","last_clean":"2023-10-06T03:57:11.372022+0000","last_became_active":"2023-10-06T03:57:11.371758+0000","last_became_peered":"2023-10-06T03:57:11.371758+0000","last_unstale":"2023-10-06T03:57:11.372022+0000","last_undegraded":"2023-10-06T03:57:11.372022+0000","last_fullsized":"2023-10-06T03:57:11.372022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:10.922157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.348539+0000","last_change":"2023-10-06T03:57:10.347406+0000","last_active":"2023-10-06T03:57:10.348539+0000","last_peered":"2023-10-06T03:57:10.348539+0000","last_clean":"2023-10-06T03:57:10.348539+0000","last_became_active":"2023-10-06T03:57:10.347151+0000","last_became_peered":"2023-10-06T03:57:10.347151+0000","last_unstale":"2023-10-06T03:57:10.348539+0000","last_undegraded":"2023-10-06T03:57:10.348539+0000","last_fullsized":"2023-10-06T03:57:10.348539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:20.427620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.370240+0000","last_change":"2023-10-06T03:57:11.370240+0000","last_active":"2023-10-06T03:57:11.370240+0000","last_peered":"2023-10-06T03:57:11.370240+0000","last_clean":"2023-10-06T03:57:11.370240+0000","last_became_active":"2023-10-06T03:57:11.369938+0000","last_became_peered":"2023-10-06T03:57:11.369938+0000","last_unstale":"2023-10-06T03:57:11.370240+0000","last_undegraded":"2023-10-06T03:57:11.370240+0000","last_fullsized":"2023-10-06T03:57:11.370240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:07.105832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.350424+0000","last_change":"2023-10-06T03:57:10.346054+0000","last_active":"2023-10-06T03:57:10.350424+0000","last_peered":"2023-10-06T03:57:10.350424+0000","last_clean":"2023-10-06T03:57:10.350424+0000","last_became_active":"2023-10-06T03:57:10.345721+0000","last_became_peered":"2023-10-06T03:57:10.345721+0000","last_unstale":"2023-10-06T03:57:10.350424+0000","last_undegraded":"2023-10-06T03:57:10.350424+0000","last_fullsized":"2023-10-06T03:57:10.350424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:54.481304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.376525+0000","last_change":"2023-10-06T03:57:11.376525+0000","last_active":"2023-10-06T03:57:11.376525+0000","last_peered":"2023-10-06T03:57:11.376525+0000","last_clean":"2023-10-06T03:57:11.376525+0000","last_became_active":"2023-10-06T03:57:11.376291+0000","last_became_peered":"2023-10-06T03:57:11.376291+0000","last_unstale":"2023-10-06T03:57:11.376525+0000","last_undegraded":"2023-10-06T03:57:11.376525+0000","last_fullsized":"2023-10-06T03:57:11.376525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:31.993426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.366706+0000","last_change":"2023-10-06T03:57:10.340531+0000","last_active":"2023-10-06T03:57:11.366706+0000","last_peered":"2023-10-06T03:57:11.366706+0000","last_clean":"2023-10-06T03:57:11.366706+0000","last_became_active":"2023-10-06T03:57:10.340232+0000","last_became_peered":"2023-10-06T03:57:10.340232+0000","last_unstale":"2023-10-06T03:57:11.366706+0000","last_undegraded":"2023-10-06T03:57:11.366706+0000","last_fullsized":"2023-10-06T03:57:11.366706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:43:26.332055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377803+0000","last_change":"2023-10-06T03:57:11.375070+0000","last_active":"2023-10-06T03:57:11.377803+0000","last_peered":"2023-10-06T03:57:11.377803+0000","last_clean":"2023-10-06T03:57:11.377803+0000","last_became_active":"2023-10-06T03:57:11.374892+0000","last_became_peered":"2023-10-06T03:57:11.374892+0000","last_unstale":"2023-10-06T03:57:11.377803+0000","last_undegraded":"2023-10-06T03:57:11.377803+0000","last_fullsized":"2023-10-06T03:57:11.377803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:51.198745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.356848+0000","last_change":"2023-10-06T03:57:10.346758+0000","last_active":"2023-10-06T03:57:11.356848+0000","last_peered":"2023-10-06T03:57:11.356848+0000","last_clean":"2023-10-06T03:57:11.356848+0000","last_became_active":"2023-10-06T03:57:10.346430+0000","last_became_peered":"2023-10-06T03:57:10.346430+0000","last_unstale":"2023-10-06T03:57:11.356848+0000","last_undegraded":"2023-10-06T03:57:11.356848+0000","last_fullsized":"2023-10-06T03:57:11.356848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:10.809242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371658+0000","last_change":"2023-10-06T03:57:11.367808+0000","last_active":"2023-10-06T03:57:11.371658+0000","last_peered":"2023-10-06T03:57:11.371658+0000","last_clean":"2023-10-06T03:57:11.371658+0000","last_became_active":"2023-10-06T03:57:11.367565+0000","last_became_peered":"2023-10-06T03:57:11.367565+0000","last_unstale":"2023-10-06T03:57:11.371658+0000","last_undegraded":"2023-10-06T03:57:11.371658+0000","last_fullsized":"2023-10-06T03:57:11.371658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:14.409116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.365509+0000","last_change":"2023-10-06T03:57:10.351062+0000","last_active":"2023-10-06T03:57:11.365509+0000","last_peered":"2023-10-06T03:57:11.365509+0000","last_clean":"2023-10-06T03:57:11.365509+0000","last_became_active":"2023-10-06T03:57:10.350795+0000","last_became_peered":"2023-10-06T03:57:10.350795+0000","last_unstale":"2023-10-06T03:57:11.365509+0000","last_undegraded":"2023-10-06T03:57:11.365509+0000","last_fullsized":"2023-10-06T03:57:11.365509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:54:45.714518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367489+0000","last_change":"2023-10-06T03:57:11.367489+0000","last_active":"2023-10-06T03:57:11.367489+0000","last_peered":"2023-10-06T03:57:11.367489+0000","last_clean":"2023-10-06T03:57:11.367489+0000","last_became_active":"2023-10-06T03:57:11.367254+0000","last_became_peered":"2023-10-06T03:57:11.367254+0000","last_unstale":"2023-10-06T03:57:11.367489+0000","last_undegraded":"2023-10-06T03:57:11.367489+0000","last_fullsized":"2023-10-06T03:57:11.367489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:04.706485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.344944+0000","last_change":"2023-10-06T03:57:10.338125+0000","last_active":"2023-10-06T03:57:10.344944+0000","last_peered":"2023-10-06T03:57:10.344944+0000","last_clean":"2023-10-06T03:57:10.344944+0000","last_became_active":"2023-10-06T03:57:10.337798+0000","last_became_peered":"2023-10-06T03:57:10.337798+0000","last_unstale":"2023-10-06T03:57:10.344944+0000","last_undegraded":"2023-10-06T03:57:10.344944+0000","last_fullsized":"2023-10-06T03:57:10.344944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:00.482657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.376461+0000","last_change":"2023-10-06T03:57:11.376461+0000","last_active":"2023-10-06T03:57:11.376461+0000","last_peered":"2023-10-06T03:57:11.376461+0000","last_clean":"2023-10-06T03:57:11.376461+0000","last_became_active":"2023-10-06T03:57:11.376223+0000","last_became_peered":"2023-10-06T03:57:11.376223+0000","last_unstale":"2023-10-06T03:57:11.376461+0000","last_undegraded":"2023-10-06T03:57:11.376461+0000","last_fullsized":"2023-10-06T03:57:11.376461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:04:37.408871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.364778+0000","last_change":"2023-10-06T03:57:10.350511+0000","last_active":"2023-10-06T03:57:11.364778+0000","last_peered":"2023-10-06T03:57:11.364778+0000","last_clean":"2023-10-06T03:57:11.364778+0000","last_became_active":"2023-10-06T03:57:10.350219+0000","last_became_peered":"2023-10-06T03:57:10.350219+0000","last_unstale":"2023-10-06T03:57:11.364778+0000","last_undegraded":"2023-10-06T03:57:11.364778+0000","last_fullsized":"2023-10-06T03:57:11.364778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:56.043506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.362700+0000","last_change":"2023-10-06T03:57:11.362700+0000","last_active":"2023-10-06T03:57:11.362700+0000","last_peered":"2023-10-06T03:57:11.362700+0000","last_clean":"2023-10-06T03:57:11.362700+0000","last_became_active":"2023-10-06T03:57:11.362443+0000","last_became_peered":"2023-10-06T03:57:11.362443+0000","last_unstale":"2023-10-06T03:57:11.362700+0000","last_undegraded":"2023-10-06T03:57:11.362700+0000","last_fullsized":"2023-10-06T03:57:11.362700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:21:09.143085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.349221+0000","last_change":"2023-10-06T03:57:10.346259+0000","last_active":"2023-10-06T03:57:10.349221+0000","last_peered":"2023-10-06T03:57:10.349221+0000","last_clean":"2023-10-06T03:57:10.349221+0000","last_became_active":"2023-10-06T03:57:10.345424+0000","last_became_peered":"2023-10-06T03:57:10.345424+0000","last_unstale":"2023-10-06T03:57:10.349221+0000","last_undegraded":"2023-10-06T03:57:10.349221+0000","last_fullsized":"2023-10-06T03:57:10.349221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:19.278206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.376053+0000","last_change":"2023-10-06T03:57:11.376053+0000","last_active":"2023-10-06T03:57:11.376053+0000","last_peered":"2023-10-06T03:57:11.376053+0000","last_clean":"2023-10-06T03:57:11.376053+0000","last_became_active":"2023-10-06T03:57:11.375824+0000","last_became_peered":"2023-10-06T03:57:11.375824+0000","last_unstale":"2023-10-06T03:57:11.376053+0000","last_undegraded":"2023-10-06T03:57:11.376053+0000","last_fullsized":"2023-10-06T03:57:11.376053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:24:26.156866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.366961+0000","last_change":"2023-10-06T03:57:10.350408+0000","last_active":"2023-10-06T03:57:11.366961+0000","last_peered":"2023-10-06T03:57:11.366961+0000","last_clean":"2023-10-06T03:57:11.366961+0000","last_became_active":"2023-10-06T03:57:10.350135+0000","last_became_peered":"2023-10-06T03:57:10.350135+0000","last_unstale":"2023-10-06T03:57:11.366961+0000","last_undegraded":"2023-10-06T03:57:11.366961+0000","last_fullsized":"2023-10-06T03:57:11.366961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:43.522111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.376524+0000","last_change":"2023-10-06T03:57:11.376524+0000","last_active":"2023-10-06T03:57:11.376524+0000","last_peered":"2023-10-06T03:57:11.376524+0000","last_clean":"2023-10-06T03:57:11.376524+0000","last_became_active":"2023-10-06T03:57:11.376236+0000","last_became_peered":"2023-10-06T03:57:11.376236+0000","last_unstale":"2023-10-06T03:57:11.376524+0000","last_undegraded":"2023-10-06T03:57:11.376524+0000","last_fullsized":"2023-10-06T03:57:11.376524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:33.854818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367008+0000","last_change":"2023-10-06T03:57:10.345799+0000","last_active":"2023-10-06T03:57:11.367008+0000","last_peered":"2023-10-06T03:57:11.367008+0000","last_clean":"2023-10-06T03:57:11.367008+0000","last_became_active":"2023-10-06T03:57:10.345510+0000","last_became_peered":"2023-10-06T03:57:10.345510+0000","last_unstale":"2023-10-06T03:57:11.367008+0000","last_undegraded":"2023-10-06T03:57:11.367008+0000","last_fullsized":"2023-10-06T03:57:11.367008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:09.567314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372383+0000","last_change":"2023-10-06T03:57:11.368662+0000","last_active":"2023-10-06T03:57:11.372383+0000","last_peered":"2023-10-06T03:57:11.372383+0000","last_clean":"2023-10-06T03:57:11.372383+0000","last_became_active":"2023-10-06T03:57:11.368375+0000","last_became_peered":"2023-10-06T03:57:11.368375+0000","last_unstale":"2023-10-06T03:57:11.372383+0000","last_undegraded":"2023-10-06T03:57:11.372383+0000","last_fullsized":"2023-10-06T03:57:11.372383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:27.125523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.366893+0000","last_change":"2023-10-06T03:57:10.346945+0000","last_active":"2023-10-06T03:57:11.366893+0000","last_peered":"2023-10-06T03:57:11.366893+0000","last_clean":"2023-10-06T03:57:11.366893+0000","last_became_active":"2023-10-06T03:57:10.346689+0000","last_became_peered":"2023-10-06T03:57:10.346689+0000","last_unstale":"2023-10-06T03:57:11.366893+0000","last_undegraded":"2023-10-06T03:57:11.366893+0000","last_fullsized":"2023-10-06T03:57:11.366893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:43.420455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372308+0000","last_change":"2023-10-06T03:57:11.372308+0000","last_active":"2023-10-06T03:57:11.372308+0000","last_peered":"2023-10-06T03:57:11.372308+0000","last_clean":"2023-10-06T03:57:11.372308+0000","last_became_active":"2023-10-06T03:57:11.371795+0000","last_became_peered":"2023-10-06T03:57:11.371795+0000","last_unstale":"2023-10-06T03:57:11.372308+0000","last_undegraded":"2023-10-06T03:57:11.372308+0000","last_fullsized":"2023-10-06T03:57:11.372308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:00.324478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.351416+0000","last_change":"2023-10-06T03:57:10.348592+0000","last_active":"2023-10-06T03:57:10.351416+0000","last_peered":"2023-10-06T03:57:10.351416+0000","last_clean":"2023-10-06T03:57:10.351416+0000","last_became_active":"2023-10-06T03:57:10.348291+0000","last_became_peered":"2023-10-06T03:57:10.348291+0000","last_unstale":"2023-10-06T03:57:10.351416+0000","last_undegraded":"2023-10-06T03:57:10.351416+0000","last_fullsized":"2023-10-06T03:57:10.351416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:00.660348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.366510+0000","last_change":"2023-10-06T03:57:11.366510+0000","last_active":"2023-10-06T03:57:11.366510+0000","last_peered":"2023-10-06T03:57:11.366510+0000","last_clean":"2023-10-06T03:57:11.366510+0000","last_became_active":"2023-10-06T03:57:11.365077+0000","last_became_peered":"2023-10-06T03:57:11.365077+0000","last_unstale":"2023-10-06T03:57:11.366510+0000","last_undegraded":"2023-10-06T03:57:11.366510+0000","last_fullsized":"2023-10-06T03:57:11.366510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:25.199391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.351559+0000","last_change":"2023-10-06T03:57:10.348418+0000","last_active":"2023-10-06T03:57:10.351559+0000","last_peered":"2023-10-06T03:57:10.351559+0000","last_clean":"2023-10-06T03:57:10.351559+0000","last_became_active":"2023-10-06T03:57:10.348055+0000","last_became_peered":"2023-10-06T03:57:10.348055+0000","last_unstale":"2023-10-06T03:57:10.351559+0000","last_undegraded":"2023-10-06T03:57:10.351559+0000","last_fullsized":"2023-10-06T03:57:10.351559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:26.102022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373629+0000","last_change":"2023-10-06T03:57:11.370493+0000","last_active":"2023-10-06T03:57:11.373629+0000","last_peered":"2023-10-06T03:57:11.373629+0000","last_clean":"2023-10-06T03:57:11.373629+0000","last_became_active":"2023-10-06T03:57:11.370073+0000","last_became_peered":"2023-10-06T03:57:11.370073+0000","last_unstale":"2023-10-06T03:57:11.373629+0000","last_undegraded":"2023-10-06T03:57:11.373629+0000","last_fullsized":"2023-10-06T03:57:11.373629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:58.417951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.354826+0000","last_change":"2023-10-06T03:57:10.352966+0000","last_active":"2023-10-06T03:57:10.354826+0000","last_peered":"2023-10-06T03:57:10.354826+0000","last_clean":"2023-10-06T03:57:10.354826+0000","last_became_active":"2023-10-06T03:57:10.352653+0000","last_became_peered":"2023-10-06T03:57:10.352653+0000","last_unstale":"2023-10-06T03:57:10.354826+0000","last_undegraded":"2023-10-06T03:57:10.354826+0000","last_fullsized":"2023-10-06T03:57:10.354826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:32.092557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371704+0000","last_change":"2023-10-06T03:57:11.364052+0000","last_active":"2023-10-06T03:57:11.371704+0000","last_peered":"2023-10-06T03:57:11.371704+0000","last_clean":"2023-10-06T03:57:11.371704+0000","last_became_active":"2023-10-06T03:57:11.363830+0000","last_became_peered":"2023-10-06T03:57:11.363830+0000","last_unstale":"2023-10-06T03:57:11.371704+0000","last_undegraded":"2023-10-06T03:57:11.371704+0000","last_fullsized":"2023-10-06T03:57:11.371704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:04.084019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.357124+0000","last_change":"2023-10-06T03:57:10.349421+0000","last_active":"2023-10-06T03:57:11.357124+0000","last_peered":"2023-10-06T03:57:11.357124+0000","last_clean":"2023-10-06T03:57:11.357124+0000","last_became_active":"2023-10-06T03:57:10.349114+0000","last_became_peered":"2023-10-06T03:57:10.349114+0000","last_unstale":"2023-10-06T03:57:11.357124+0000","last_undegraded":"2023-10-06T03:57:11.357124+0000","last_fullsized":"2023-10-06T03:57:11.357124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:08.209791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372296+0000","last_change":"2023-10-06T03:57:11.367516+0000","last_active":"2023-10-06T03:57:11.372296+0000","last_peered":"2023-10-06T03:57:11.372296+0000","last_clean":"2023-10-06T03:57:11.372296+0000","last_became_active":"2023-10-06T03:57:11.367257+0000","last_became_peered":"2023-10-06T03:57:11.367257+0000","last_unstale":"2023-10-06T03:57:11.372296+0000","last_undegraded":"2023-10-06T03:57:11.372296+0000","last_fullsized":"2023-10-06T03:57:11.372296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:43.628819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.351309+0000","last_change":"2023-10-06T03:57:10.347649+0000","last_active":"2023-10-06T03:57:10.351309+0000","last_peered":"2023-10-06T03:57:10.351309+0000","last_clean":"2023-10-06T03:57:10.351309+0000","last_became_active":"2023-10-06T03:57:10.347433+0000","last_became_peered":"2023-10-06T03:57:10.347433+0000","last_unstale":"2023-10-06T03:57:10.351309+0000","last_undegraded":"2023-10-06T03:57:10.351309+0000","last_fullsized":"2023-10-06T03:57:10.351309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:55.951494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372561+0000","last_change":"2023-10-06T03:57:11.370092+0000","last_active":"2023-10-06T03:57:11.372561+0000","last_peered":"2023-10-06T03:57:11.372561+0000","last_clean":"2023-10-06T03:57:11.372561+0000","last_became_active":"2023-10-06T03:57:11.369825+0000","last_became_peered":"2023-10-06T03:57:11.369825+0000","last_unstale":"2023-10-06T03:57:11.372561+0000","last_undegraded":"2023-10-06T03:57:11.372561+0000","last_fullsized":"2023-10-06T03:57:11.372561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:43.415929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367709+0000","last_change":"2023-10-06T03:57:05.663312+0000","last_active":"2023-10-06T03:57:11.367709+0000","last_peered":"2023-10-06T03:57:11.367709+0000","last_clean":"2023-10-06T03:57:11.367709+0000","last_became_active":"2023-10-06T03:57:05.663046+0000","last_became_peered":"2023-10-06T03:57:05.663046+0000","last_unstale":"2023-10-06T03:57:11.367709+0000","last_undegraded":"2023-10-06T03:57:11.367709+0000","last_fullsized":"2023-10-06T03:57:11.367709+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:31.174359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.351387+0000","last_change":"2023-10-06T03:57:10.348930+0000","last_active":"2023-10-06T03:57:10.351387+0000","last_peered":"2023-10-06T03:57:10.351387+0000","last_clean":"2023-10-06T03:57:10.351387+0000","last_became_active":"2023-10-06T03:57:10.348608+0000","last_became_peered":"2023-10-06T03:57:10.348608+0000","last_unstale":"2023-10-06T03:57:10.351387+0000","last_undegraded":"2023-10-06T03:57:10.351387+0000","last_fullsized":"2023-10-06T03:57:10.351387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:00:09.283718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377785+0000","last_change":"2023-10-06T03:57:11.371108+0000","last_active":"2023-10-06T03:57:11.377785+0000","last_peered":"2023-10-06T03:57:11.377785+0000","last_clean":"2023-10-06T03:57:11.377785+0000","last_became_active":"2023-10-06T03:57:11.370847+0000","last_became_peered":"2023-10-06T03:57:11.370847+0000","last_unstale":"2023-10-06T03:57:11.377785+0000","last_undegraded":"2023-10-06T03:57:11.377785+0000","last_fullsized":"2023-10-06T03:57:11.377785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:38.111171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.345339+0000","last_change":"2023-10-06T03:57:05.318834+0000","last_active":"2023-10-06T03:57:10.345339+0000","last_peered":"2023-10-06T03:57:10.345339+0000","last_clean":"2023-10-06T03:57:10.345339+0000","last_became_active":"2023-10-06T03:57:05.318572+0000","last_became_peered":"2023-10-06T03:57:05.318572+0000","last_unstale":"2023-10-06T03:57:10.345339+0000","last_undegraded":"2023-10-06T03:57:10.345339+0000","last_fullsized":"2023-10-06T03:57:10.345339+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:21.951095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.358229+0000","last_change":"2023-10-06T03:57:10.348556+0000","last_active":"2023-10-06T03:57:11.358229+0000","last_peered":"2023-10-06T03:57:11.358229+0000","last_clean":"2023-10-06T03:57:11.358229+0000","last_became_active":"2023-10-06T03:57:10.348271+0000","last_became_peered":"2023-10-06T03:57:10.348271+0000","last_unstale":"2023-10-06T03:57:11.358229+0000","last_undegraded":"2023-10-06T03:57:11.358229+0000","last_fullsized":"2023-10-06T03:57:11.358229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:58.470989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.368920+0000","last_change":"2023-10-06T03:57:11.365237+0000","last_active":"2023-10-06T03:57:11.368920+0000","last_peered":"2023-10-06T03:57:11.368920+0000","last_clean":"2023-10-06T03:57:11.368920+0000","last_became_active":"2023-10-06T03:57:11.365006+0000","last_became_peered":"2023-10-06T03:57:11.365006+0000","last_unstale":"2023-10-06T03:57:11.368920+0000","last_undegraded":"2023-10-06T03:57:11.368920+0000","last_fullsized":"2023-10-06T03:57:11.368920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:33.005015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.342459+0000","last_change":"2023-10-06T03:57:05.314695+0000","last_active":"2023-10-06T03:57:10.342459+0000","last_peered":"2023-10-06T03:57:10.342459+0000","last_clean":"2023-10-06T03:57:10.342459+0000","last_became_active":"2023-10-06T03:57:05.314408+0000","last_became_peered":"2023-10-06T03:57:05.314408+0000","last_unstale":"2023-10-06T03:57:10.342459+0000","last_undegraded":"2023-10-06T03:57:10.342459+0000","last_fullsized":"2023-10-06T03:57:10.342459+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:19.042145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367206+0000","last_change":"2023-10-06T03:57:10.346078+0000","last_active":"2023-10-06T03:57:11.367206+0000","last_peered":"2023-10-06T03:57:11.367206+0000","last_clean":"2023-10-06T03:57:11.367206+0000","last_became_active":"2023-10-06T03:57:10.345769+0000","last_became_peered":"2023-10-06T03:57:10.345769+0000","last_unstale":"2023-10-06T03:57:11.367206+0000","last_undegraded":"2023-10-06T03:57:11.367206+0000","last_fullsized":"2023-10-06T03:57:11.367206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:16.630681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.376583+0000","last_change":"2023-10-06T03:57:11.370372+0000","last_active":"2023-10-06T03:57:11.376583+0000","last_peered":"2023-10-06T03:57:11.376583+0000","last_clean":"2023-10-06T03:57:11.376583+0000","last_became_active":"2023-10-06T03:57:11.370076+0000","last_became_peered":"2023-10-06T03:57:11.370076+0000","last_unstale":"2023-10-06T03:57:11.376583+0000","last_undegraded":"2023-10-06T03:57:11.376583+0000","last_fullsized":"2023-10-06T03:57:11.376583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:18:59.517155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.345962+0000","last_change":"2023-10-06T03:57:05.318778+0000","last_active":"2023-10-06T03:57:10.345962+0000","last_peered":"2023-10-06T03:57:10.345962+0000","last_clean":"2023-10-06T03:57:10.345962+0000","last_became_active":"2023-10-06T03:57:05.318456+0000","last_became_peered":"2023-10-06T03:57:05.318456+0000","last_unstale":"2023-10-06T03:57:10.345962+0000","last_undegraded":"2023-10-06T03:57:10.345962+0000","last_fullsized":"2023-10-06T03:57:10.345962+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:39.769241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.362382+0000","last_change":"2023-10-06T03:57:05.664245+0000","last_active":"2023-10-06T03:57:11.362382+0000","last_peered":"2023-10-06T03:57:11.362382+0000","last_clean":"2023-10-06T03:57:11.362382+0000","last_became_active":"2023-10-06T03:57:05.663426+0000","last_became_peered":"2023-10-06T03:57:05.663426+0000","last_unstale":"2023-10-06T03:57:11.362382+0000","last_undegraded":"2023-10-06T03:57:11.362382+0000","last_fullsized":"2023-10-06T03:57:11.362382+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:29.322599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.351611+0000","last_change":"2023-10-06T03:57:10.351059+0000","last_active":"2023-10-06T03:57:10.351611+0000","last_peered":"2023-10-06T03:57:10.351611+0000","last_clean":"2023-10-06T03:57:10.351611+0000","last_became_active":"2023-10-06T03:57:10.348879+0000","last_became_peered":"2023-10-06T03:57:10.348879+0000","last_unstale":"2023-10-06T03:57:10.351611+0000","last_undegraded":"2023-10-06T03:57:10.351611+0000","last_fullsized":"2023-10-06T03:57:10.351611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:35.764148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373786+0000","last_change":"2023-10-06T03:57:11.373786+0000","last_active":"2023-10-06T03:57:11.373786+0000","last_peered":"2023-10-06T03:57:11.373786+0000","last_clean":"2023-10-06T03:57:11.373786+0000","last_became_active":"2023-10-06T03:57:11.373452+0000","last_became_peered":"2023-10-06T03:57:11.373452+0000","last_unstale":"2023-10-06T03:57:11.373786+0000","last_undegraded":"2023-10-06T03:57:11.373786+0000","last_fullsized":"2023-10-06T03:57:11.373786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:20.524065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.364987+0000","last_change":"2023-10-06T03:57:05.318790+0000","last_active":"2023-10-06T03:57:11.364987+0000","last_peered":"2023-10-06T03:57:11.364987+0000","last_clean":"2023-10-06T03:57:11.364987+0000","last_became_active":"2023-10-06T03:57:05.318531+0000","last_became_peered":"2023-10-06T03:57:05.318531+0000","last_unstale":"2023-10-06T03:57:11.364987+0000","last_undegraded":"2023-10-06T03:57:11.364987+0000","last_fullsized":"2023-10-06T03:57:11.364987+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:12.876998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.346053+0000","last_change":"2023-10-06T03:57:10.343328+0000","last_active":"2023-10-06T03:57:10.346053+0000","last_peered":"2023-10-06T03:57:10.346053+0000","last_clean":"2023-10-06T03:57:10.346053+0000","last_became_active":"2023-10-06T03:57:10.343051+0000","last_became_peered":"2023-10-06T03:57:10.343051+0000","last_unstale":"2023-10-06T03:57:10.346053+0000","last_undegraded":"2023-10-06T03:57:10.346053+0000","last_fullsized":"2023-10-06T03:57:10.346053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:27.722475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373581+0000","last_change":"2023-10-06T03:57:11.373581+0000","last_active":"2023-10-06T03:57:11.373581+0000","last_peered":"2023-10-06T03:57:11.373581+0000","last_clean":"2023-10-06T03:57:11.373581+0000","last_became_active":"2023-10-06T03:57:11.373264+0000","last_became_peered":"2023-10-06T03:57:11.373264+0000","last_unstale":"2023-10-06T03:57:11.373581+0000","last_undegraded":"2023-10-06T03:57:11.373581+0000","last_fullsized":"2023-10-06T03:57:11.373581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:15.950245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.366791+0000","last_change":"2023-10-06T03:57:05.663770+0000","last_active":"2023-10-06T03:57:11.366791+0000","last_peered":"2023-10-06T03:57:11.366791+0000","last_clean":"2023-10-06T03:57:11.366791+0000","last_became_active":"2023-10-06T03:57:05.663501+0000","last_became_peered":"2023-10-06T03:57:05.663501+0000","last_unstale":"2023-10-06T03:57:11.366791+0000","last_undegraded":"2023-10-06T03:57:11.366791+0000","last_fullsized":"2023-10-06T03:57:11.366791+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:09.806915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367831+0000","last_change":"2023-10-06T03:57:10.346585+0000","last_active":"2023-10-06T03:57:11.367831+0000","last_peered":"2023-10-06T03:57:11.367831+0000","last_clean":"2023-10-06T03:57:11.367831+0000","last_became_active":"2023-10-06T03:57:10.345743+0000","last_became_peered":"2023-10-06T03:57:10.345743+0000","last_unstale":"2023-10-06T03:57:11.367831+0000","last_undegraded":"2023-10-06T03:57:11.367831+0000","last_fullsized":"2023-10-06T03:57:11.367831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:54.589702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373844+0000","last_change":"2023-10-06T03:57:11.373844+0000","last_active":"2023-10-06T03:57:11.373844+0000","last_peered":"2023-10-06T03:57:11.373844+0000","last_clean":"2023-10-06T03:57:11.373844+0000","last_became_active":"2023-10-06T03:57:11.373543+0000","last_became_peered":"2023-10-06T03:57:11.373543+0000","last_unstale":"2023-10-06T03:57:11.373844+0000","last_undegraded":"2023-10-06T03:57:11.373844+0000","last_fullsized":"2023-10-06T03:57:11.373844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:15.152354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.348501+0000","last_change":"2023-10-06T03:57:10.344618+0000","last_active":"2023-10-06T03:57:10.348501+0000","last_peered":"2023-10-06T03:57:10.348501+0000","last_clean":"2023-10-06T03:57:10.348501+0000","last_became_active":"2023-10-06T03:57:10.344324+0000","last_became_peered":"2023-10-06T03:57:10.344324+0000","last_unstale":"2023-10-06T03:57:10.348501+0000","last_undegraded":"2023-10-06T03:57:10.348501+0000","last_fullsized":"2023-10-06T03:57:10.348501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:06.062331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.368350+0000","last_change":"2023-10-06T03:57:05.317973+0000","last_active":"2023-10-06T03:57:11.368350+0000","last_peered":"2023-10-06T03:57:11.368350+0000","last_clean":"2023-10-06T03:57:11.368350+0000","last_became_active":"2023-10-06T03:57:05.317715+0000","last_became_peered":"2023-10-06T03:57:05.317715+0000","last_unstale":"2023-10-06T03:57:11.368350+0000","last_undegraded":"2023-10-06T03:57:11.368350+0000","last_fullsized":"2023-10-06T03:57:11.368350+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:37:51.858909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.369706+0000","last_change":"2023-10-06T03:57:11.369706+0000","last_active":"2023-10-06T03:57:11.369706+0000","last_peered":"2023-10-06T03:57:11.369706+0000","last_clean":"2023-10-06T03:57:11.369706+0000","last_became_active":"2023-10-06T03:57:11.369445+0000","last_became_peered":"2023-10-06T03:57:11.369445+0000","last_unstale":"2023-10-06T03:57:11.369706+0000","last_undegraded":"2023-10-06T03:57:11.369706+0000","last_fullsized":"2023-10-06T03:57:11.369706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:52.999421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.339408+0000","last_change":"2023-10-06T03:57:05.320847+0000","last_active":"2023-10-06T03:57:10.339408+0000","last_peered":"2023-10-06T03:57:10.339408+0000","last_clean":"2023-10-06T03:57:10.339408+0000","last_became_active":"2023-10-06T03:57:05.320583+0000","last_became_peered":"2023-10-06T03:57:05.320583+0000","last_unstale":"2023-10-06T03:57:10.339408+0000","last_undegraded":"2023-10-06T03:57:10.339408+0000","last_fullsized":"2023-10-06T03:57:10.339408+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:12.772449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.345588+0000","last_change":"2023-10-06T03:57:10.341132+0000","last_active":"2023-10-06T03:57:10.345588+0000","last_peered":"2023-10-06T03:57:10.345588+0000","last_clean":"2023-10-06T03:57:10.345588+0000","last_became_active":"2023-10-06T03:57:10.339424+0000","last_became_peered":"2023-10-06T03:57:10.339424+0000","last_unstale":"2023-10-06T03:57:10.345588+0000","last_undegraded":"2023-10-06T03:57:10.345588+0000","last_fullsized":"2023-10-06T03:57:10.345588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:39.675711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.374372+0000","last_change":"2023-10-06T03:57:11.372497+0000","last_active":"2023-10-06T03:57:11.374372+0000","last_peered":"2023-10-06T03:57:11.374372+0000","last_clean":"2023-10-06T03:57:11.374372+0000","last_became_active":"2023-10-06T03:57:11.372215+0000","last_became_peered":"2023-10-06T03:57:11.372215+0000","last_unstale":"2023-10-06T03:57:11.374372+0000","last_undegraded":"2023-10-06T03:57:11.374372+0000","last_fullsized":"2023-10-06T03:57:11.374372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:23.559480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.346078+0000","last_change":"2023-10-06T03:57:10.344051+0000","last_active":"2023-10-06T03:57:10.346078+0000","last_peered":"2023-10-06T03:57:10.346078+0000","last_clean":"2023-10-06T03:57:10.346078+0000","last_became_active":"2023-10-06T03:57:10.343865+0000","last_became_peered":"2023-10-06T03:57:10.343865+0000","last_unstale":"2023-10-06T03:57:10.346078+0000","last_undegraded":"2023-10-06T03:57:10.346078+0000","last_fullsized":"2023-10-06T03:57:10.346078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:32.522470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.369927+0000","last_change":"2023-10-06T03:57:11.369927+0000","last_active":"2023-10-06T03:57:11.369927+0000","last_peered":"2023-10-06T03:57:11.369927+0000","last_clean":"2023-10-06T03:57:11.369927+0000","last_became_active":"2023-10-06T03:57:11.369681+0000","last_became_peered":"2023-10-06T03:57:11.369681+0000","last_unstale":"2023-10-06T03:57:11.369927+0000","last_undegraded":"2023-10-06T03:57:11.369927+0000","last_fullsized":"2023-10-06T03:57:11.369927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:53.858179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.351660+0000","last_change":"2023-10-06T03:57:10.349666+0000","last_active":"2023-10-06T03:57:10.351660+0000","last_peered":"2023-10-06T03:57:10.351660+0000","last_clean":"2023-10-06T03:57:10.351660+0000","last_became_active":"2023-10-06T03:57:10.349320+0000","last_became_peered":"2023-10-06T03:57:10.349320+0000","last_unstale":"2023-10-06T03:57:10.351660+0000","last_undegraded":"2023-10-06T03:57:10.351660+0000","last_fullsized":"2023-10-06T03:57:10.351660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:45:49.709519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373619+0000","last_change":"2023-10-06T03:57:11.370442+0000","last_active":"2023-10-06T03:57:11.373619+0000","last_peered":"2023-10-06T03:57:11.373619+0000","last_clean":"2023-10-06T03:57:11.373619+0000","last_became_active":"2023-10-06T03:57:11.370229+0000","last_became_peered":"2023-10-06T03:57:11.370229+0000","last_unstale":"2023-10-06T03:57:11.373619+0000","last_undegraded":"2023-10-06T03:57:11.373619+0000","last_fullsized":"2023-10-06T03:57:11.373619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:26.174331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.351687+0000","last_change":"2023-10-06T03:57:10.350263+0000","last_active":"2023-10-06T03:57:10.351687+0000","last_peered":"2023-10-06T03:57:10.351687+0000","last_clean":"2023-10-06T03:57:10.351687+0000","last_became_active":"2023-10-06T03:57:10.350083+0000","last_became_peered":"2023-10-06T03:57:10.350083+0000","last_unstale":"2023-10-06T03:57:10.351687+0000","last_undegraded":"2023-10-06T03:57:10.351687+0000","last_fullsized":"2023-10-06T03:57:10.351687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:05.433032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372850+0000","last_change":"2023-10-06T03:57:11.369397+0000","last_active":"2023-10-06T03:57:11.372850+0000","last_peered":"2023-10-06T03:57:11.372850+0000","last_clean":"2023-10-06T03:57:11.372850+0000","last_became_active":"2023-10-06T03:57:11.369123+0000","last_became_peered":"2023-10-06T03:57:11.369123+0000","last_unstale":"2023-10-06T03:57:11.372850+0000","last_undegraded":"2023-10-06T03:57:11.372850+0000","last_fullsized":"2023-10-06T03:57:11.372850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:28.139036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.351215+0000","last_change":"2023-10-06T03:57:10.347049+0000","last_active":"2023-10-06T03:57:10.351215+0000","last_peered":"2023-10-06T03:57:10.351215+0000","last_clean":"2023-10-06T03:57:10.351215+0000","last_became_active":"2023-10-06T03:57:10.344583+0000","last_became_peered":"2023-10-06T03:57:10.344583+0000","last_unstale":"2023-10-06T03:57:10.351215+0000","last_undegraded":"2023-10-06T03:57:10.351215+0000","last_fullsized":"2023-10-06T03:57:10.351215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:32.280257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.366741+0000","last_change":"2023-10-06T03:57:11.366741+0000","last_active":"2023-10-06T03:57:11.366741+0000","last_peered":"2023-10-06T03:57:11.366741+0000","last_clean":"2023-10-06T03:57:11.366741+0000","last_became_active":"2023-10-06T03:57:11.366500+0000","last_became_peered":"2023-10-06T03:57:11.366500+0000","last_unstale":"2023-10-06T03:57:11.366741+0000","last_undegraded":"2023-10-06T03:57:11.366741+0000","last_fullsized":"2023-10-06T03:57:11.366741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:24.714603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-06T03:57:10.350305+0000","last_change":"2023-10-06T03:57:10.348170+0000","last_active":"2023-10-06T03:57:10.350305+0000","last_peered":"2023-10-06T03:57:10.350305+0000","last_clean":"2023-10-06T03:57:10.350305+0000","last_became_active":"2023-10-06T03:57:10.347897+0000","last_became_peered":"2023-10-06T03:57:10.347897+0000","last_unstale":"2023-10-06T03:57:10.350305+0000","last_undegraded":"2023-10-06T03:57:10.350305+0000","last_fullsized":"2023-10-06T03:57:10.350305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:40.756246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372969+0000","last_change":"2023-10-06T03:57:11.364368+0000","last_active":"2023-10-06T03:57:11.372969+0000","last_peered":"2023-10-06T03:57:11.372969+0000","last_clean":"2023-10-06T03:57:11.372969+0000","last_became_active":"2023-10-06T03:57:11.364150+0000","last_became_peered":"2023-10-06T03:57:11.364150+0000","last_unstale":"2023-10-06T03:57:11.372969+0000","last_undegraded":"2023-10-06T03:57:11.372969+0000","last_fullsized":"2023-10-06T03:57:11.372969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:11.038080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.366721+0000","last_change":"2023-10-06T03:57:11.366721+0000","last_active":"2023-10-06T03:57:11.366721+0000","last_peered":"2023-10-06T03:57:11.366721+0000","last_clean":"2023-10-06T03:57:11.366721+0000","last_became_active":"2023-10-06T03:57:11.366524+0000","last_became_peered":"2023-10-06T03:57:11.366524+0000","last_unstale":"2023-10-06T03:57:11.366721+0000","last_undegraded":"2023-10-06T03:57:11.366721+0000","last_fullsized":"2023-10-06T03:57:11.366721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:33.044858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371109+0000","last_change":"2023-10-06T03:57:11.371109+0000","last_active":"2023-10-06T03:57:11.371109+0000","last_peered":"2023-10-06T03:57:11.371109+0000","last_clean":"2023-10-06T03:57:11.371109+0000","last_became_active":"2023-10-06T03:57:11.370848+0000","last_became_peered":"2023-10-06T03:57:11.370848+0000","last_unstale":"2023-10-06T03:57:11.371109+0000","last_undegraded":"2023-10-06T03:57:11.371109+0000","last_fullsized":"2023-10-06T03:57:11.371109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:15.919090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372839+0000","last_change":"2023-10-06T03:57:11.364188+0000","last_active":"2023-10-06T03:57:11.372839+0000","last_peered":"2023-10-06T03:57:11.372839+0000","last_clean":"2023-10-06T03:57:11.372839+0000","last_became_active":"2023-10-06T03:57:11.363875+0000","last_became_peered":"2023-10-06T03:57:11.363875+0000","last_unstale":"2023-10-06T03:57:11.372839+0000","last_undegraded":"2023-10-06T03:57:11.372839+0000","last_fullsized":"2023-10-06T03:57:11.372839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:29.236344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377420+0000","last_change":"2023-10-06T03:57:11.377420+0000","last_active":"2023-10-06T03:57:11.377420+0000","last_peered":"2023-10-06T03:57:11.377420+0000","last_clean":"2023-10-06T03:57:11.377420+0000","last_became_active":"2023-10-06T03:57:11.377126+0000","last_became_peered":"2023-10-06T03:57:11.377126+0000","last_unstale":"2023-10-06T03:57:11.377420+0000","last_undegraded":"2023-10-06T03:57:11.377420+0000","last_fullsized":"2023-10-06T03:57:11.377420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:05.041737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377733+0000","last_change":"2023-10-06T03:57:11.375777+0000","last_active":"2023-10-06T03:57:11.377733+0000","last_peered":"2023-10-06T03:57:11.377733+0000","last_clean":"2023-10-06T03:57:11.377733+0000","last_became_active":"2023-10-06T03:57:11.375539+0000","last_became_peered":"2023-10-06T03:57:11.375539+0000","last_unstale":"2023-10-06T03:57:11.377733+0000","last_undegraded":"2023-10-06T03:57:11.377733+0000","last_fullsized":"2023-10-06T03:57:11.377733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:42.707488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64858,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297552,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074288,"statfs":{"total":96636764160,"available":96332070912,"internally_reserved":0,"allocated":270336,"data_stored":662562,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":72194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":72194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297548,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074292,"statfs":{"total":96636764160,"available":96332075008,"internally_reserved":0,"allocated":266240,"data_stored":661609,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":72194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":72194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":72194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":72194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:57:17.821 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T03:57:17.822 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-06T03:57:17.822 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-06T03:57:17.822 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-06T03:57:17.823 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-06T03:57:17.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.970+0000 7ff64ffe8700 1 Processor -- start 2023-10-06T03:57:17.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.970+0000 7ff64ffe8700 1 -- start start 2023-10-06T03:57:17.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.971+0000 7ff64ffe8700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff64806cbb0 0x7ff64806d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:17.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.971+0000 7ff64ffe8700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff64806c1f0 0x7ff64806da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:17.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.971+0000 7ff64ffe8700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff64806cf80 0x7ff64814fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:17.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.971+0000 7ff64ffe8700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6480c8c40 con 0x7ff64806cf80 2023-10-06T03:57:17.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.971+0000 7ff64ffe8700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6480c9780 con 0x7ff64806cbb0 2023-10-06T03:57:17.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.971+0000 7ff64ffe8700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff64806e0c0 con 0x7ff64806c1f0 2023-10-06T03:57:17.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.971+0000 7ff64e585700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff64806cf80 0x7ff64814fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:17.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.971+0000 7ff64e585700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff64806cf80 0x7ff64814fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34270/0 (socket says 172.21.15.157:34270) 2023-10-06T03:57:17.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.971+0000 7ff64e585700 1 -- 172.21.15.157:0/2076247345 learned_addr learned my addr 172.21.15.157:0/2076247345 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:17.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.971+0000 7ff64d583700 1 --2- 172.21.15.157:0/2076247345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff64806c1f0 0x7ff64806da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:17.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.971+0000 7ff64dd84700 1 --2- 172.21.15.157:0/2076247345 >> v2:172.21.15.175:3300/0 conn(0x7ff64806cbb0 0x7ff64806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:17.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.972+0000 7ff64e585700 1 -- 172.21.15.157:0/2076247345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff64806c1f0 msgr2=0x7ff64806da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:17.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.972+0000 7ff64e585700 1 --2- 172.21.15.157:0/2076247345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff64806c1f0 0x7ff64806da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.972+0000 7ff64e585700 1 -- 172.21.15.157:0/2076247345 >> v2:172.21.15.175:3300/0 conn(0x7ff64806cbb0 msgr2=0x7ff64806d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:17.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.972+0000 7ff64e585700 1 --2- 172.21.15.157:0/2076247345 >> v2:172.21.15.175:3300/0 conn(0x7ff64806cbb0 0x7ff64806d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.972+0000 7ff64e585700 1 -- 172.21.15.157:0/2076247345 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6480c8e50 con 0x7ff64806cf80 2023-10-06T03:57:17.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.972+0000 7ff64e585700 1 --2- 172.21.15.157:0/2076247345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff64806cf80 0x7ff64814fd00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff6400099b0 tx=0x7ff640009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1669d34321b2b302 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:17.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.972+0000 7ff64cd82700 1 -- 172.21.15.157:0/2076247345 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64000b8b0 con 0x7ff64806cf80 2023-10-06T03:57:17.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.972+0000 7ff64cd82700 1 -- 172.21.15.157:0/2076247345 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff64000ba10 con 0x7ff64806cf80 2023-10-06T03:57:17.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.972+0000 7ff64cd82700 1 -- 172.21.15.157:0/2076247345 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64000bdb0 con 0x7ff64806cf80 2023-10-06T03:57:17.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.972+0000 7ff64ffe8700 1 -- 172.21.15.157:0/2076247345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff64806cf80 msgr2=0x7ff64814fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:17.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.972+0000 7ff64ffe8700 1 --2- 172.21.15.157:0/2076247345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff64806cf80 0x7ff64814fd00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff6400099b0 tx=0x7ff640009250 comp rx=0 tx=0).stop 2023-10-06T03:57:17.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.972+0000 7ff64ffe8700 1 -- 172.21.15.157:0/2076247345 shutdown_connections 2023-10-06T03:57:17.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.972+0000 7ff64ffe8700 1 --2- 172.21.15.157:0/2076247345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff64806cf80 0x7ff64814fd00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.972+0000 7ff64ffe8700 1 --2- 172.21.15.157:0/2076247345 >> v2:172.21.15.175:3300/0 conn(0x7ff64806cbb0 0x7ff64806d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.972+0000 7ff64ffe8700 1 --2- 172.21.15.157:0/2076247345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff64806c1f0 0x7ff64806da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.972+0000 7ff64ffe8700 1 -- 172.21.15.157:0/2076247345 >> 172.21.15.157:0/2076247345 conn(0x7ff64806f230 msgr2=0x7ff64806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:17.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.972+0000 7ff64ffe8700 1 -- 172.21.15.157:0/2076247345 shutdown_connections 2023-10-06T03:57:17.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.973+0000 7ff64ffe8700 1 -- 172.21.15.157:0/2076247345 wait complete. 2023-10-06T03:57:17.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.973+0000 7ff64ffe8700 1 Processor -- start 2023-10-06T03:57:17.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.974+0000 7ff64ffe8700 1 -- start start 2023-10-06T03:57:17.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.974+0000 7ff64ffe8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff64806c1f0 0x7ff64814a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:17.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.974+0000 7ff64ffe8700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff64806cbb0 0x7ff64814ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:17.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.974+0000 7ff64ffe8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff64806cf80 0x7ff6481504a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:17.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.974+0000 7ff64ffe8700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64814fad0 con 0x7ff64806c1f0 2023-10-06T03:57:17.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.974+0000 7ff64ffe8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff648144b00 con 0x7ff64806cf80 2023-10-06T03:57:17.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.974+0000 7ff64ffe8700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff648144820 con 0x7ff64806cbb0 2023-10-06T03:57:17.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.974+0000 7ff64dd84700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff64806c1f0 0x7ff64814a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:17.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.974+0000 7ff64dd84700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff64806c1f0 0x7ff64814a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34284/0 (socket says 172.21.15.157:34284) 2023-10-06T03:57:17.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.974+0000 7ff64dd84700 1 -- 172.21.15.157:0/3358872027 learned_addr learned my addr 172.21.15.157:0/3358872027 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:17.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.974+0000 7ff64d583700 1 --2- 172.21.15.157:0/3358872027 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff64806cbb0 0x7ff64814ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:17.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.974+0000 7ff64e585700 1 --2- 172.21.15.157:0/3358872027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff64806cf80 0x7ff6481504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:17.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.975+0000 7ff64dd84700 1 -- 172.21.15.157:0/3358872027 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff64806cbb0 msgr2=0x7ff64814ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:17.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.975+0000 7ff64dd84700 1 --2- 172.21.15.157:0/3358872027 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff64806cbb0 0x7ff64814ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.975+0000 7ff64dd84700 1 -- 172.21.15.157:0/3358872027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff64806cf80 msgr2=0x7ff6481504a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:17.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.975+0000 7ff64dd84700 1 --2- 172.21.15.157:0/3358872027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff64806cf80 0x7ff6481504a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:17.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.975+0000 7ff64dd84700 1 -- 172.21.15.157:0/3358872027 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6400094f0 con 0x7ff64806c1f0 2023-10-06T03:57:17.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.975+0000 7ff64e585700 1 --2- 172.21.15.157:0/3358872027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff64806cf80 0x7ff6481504a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:17.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.975+0000 7ff64dd84700 1 --2- 172.21.15.157:0/3358872027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff64806c1f0 0x7ff64814a5c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff64400e190 tx=0x7ff644009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:17.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.975+0000 7ff63dffb700 1 -- 172.21.15.157:0/3358872027 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64400ed20 con 0x7ff64806c1f0 2023-10-06T03:57:17.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.975+0000 7ff63dffb700 1 -- 172.21.15.157:0/3358872027 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff64400ee80 con 0x7ff64806c1f0 2023-10-06T03:57:17.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.975+0000 7ff63dffb700 1 -- 172.21.15.157:0/3358872027 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff644014670 con 0x7ff64806c1f0 2023-10-06T03:57:17.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.975+0000 7ff64ffe8700 1 -- 172.21.15.157:0/3358872027 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6481521a0 con 0x7ff64806c1f0 2023-10-06T03:57:17.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.975+0000 7ff64ffe8700 1 -- 172.21.15.157:0/3358872027 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff648152760 con 0x7ff64806c1f0 2023-10-06T03:57:17.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.976+0000 7ff64ffe8700 1 -- 172.21.15.157:0/3358872027 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff648089ca0 con 0x7ff64806c1f0 2023-10-06T03:57:17.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.976+0000 7ff63dffb700 1 -- 172.21.15.157:0/3358872027 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff644014a20 con 0x7ff64806c1f0 2023-10-06T03:57:17.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.977+0000 7ff63dffb700 1 --2- 172.21.15.157:0/3358872027 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff62c066f60 0x7ff62c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:17.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.977+0000 7ff63dffb700 1 -- 172.21.15.157:0/3358872027 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff644084d80 con 0x7ff64806c1f0 2023-10-06T03:57:17.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.977+0000 7ff64d583700 1 --2- 172.21.15.157:0/3358872027 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff62c066f60 0x7ff62c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:17.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.977+0000 7ff64d583700 1 --2- 172.21.15.157:0/3358872027 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff62c066f60 0x7ff62c069410 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff648151e00 tx=0x7ff638008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:17.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:17.981+0000 7ff63dffb700 1 -- 172.21.15.157:0/3358872027 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff644051e90 con 0x7ff64806c1f0 2023-10-06T03:57:18.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.217+0000 7ff64ffe8700 1 -- 172.21.15.157:0/3358872027 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7ff6480c9130 con 0x7ff64806c1f0 2023-10-06T03:57:18.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.220+0000 7ff63dffb700 1 -- 172.21.15.157:0/3358872027 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+163705 (secure 0 0 0) 0x7ff6440554b0 con 0x7ff64806c1f0 2023-10-06T03:57:18.222 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:18.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.222+0000 7ff64ffe8700 1 -- 172.21.15.157:0/3358872027 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff62c066f60 msgr2=0x7ff62c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:18.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.222+0000 7ff64ffe8700 1 --2- 172.21.15.157:0/3358872027 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff62c066f60 0x7ff62c069410 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff648151e00 tx=0x7ff638008040 comp rx=0 tx=0).stop 2023-10-06T03:57:18.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.222+0000 7ff64ffe8700 1 -- 172.21.15.157:0/3358872027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff64806c1f0 msgr2=0x7ff64814a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:18.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.222+0000 7ff64ffe8700 1 --2- 172.21.15.157:0/3358872027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff64806c1f0 0x7ff64814a5c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff64400e190 tx=0x7ff644009920 comp rx=0 tx=0).stop 2023-10-06T03:57:18.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.222+0000 7ff64ffe8700 1 -- 172.21.15.157:0/3358872027 shutdown_connections 2023-10-06T03:57:18.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.222+0000 7ff64ffe8700 1 --2- 172.21.15.157:0/3358872027 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff62c066f60 0x7ff62c069410 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:18.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.222+0000 7ff64ffe8700 1 --2- 172.21.15.157:0/3358872027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff64806cf80 0x7ff6481504a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:18.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.222+0000 7ff64ffe8700 1 --2- 172.21.15.157:0/3358872027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff64806c1f0 0x7ff64814a5c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:18.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.222+0000 7ff64ffe8700 1 --2- 172.21.15.157:0/3358872027 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff64806cbb0 0x7ff64814ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:18.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.222+0000 7ff64ffe8700 1 -- 172.21.15.157:0/3358872027 >> 172.21.15.157:0/3358872027 conn(0x7ff64806f230 msgr2=0x7ff64814e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:18.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.222+0000 7ff64ffe8700 1 -- 172.21.15.157:0/3358872027 shutdown_connections 2023-10-06T03:57:18.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.222+0000 7ff64ffe8700 1 -- 172.21.15.157:0/3358872027 wait complete. 2023-10-06T03:57:18.235 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":5,"active_gid":4107,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":90661},{"type":"v1","addr":"172.21.15.175:6833","nonce":90661}]},"active_addr":"172.21.15.175:6833/90661","active_change":"2023-10-06T03:56:57.418480+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":1303823500}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":2096479977}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":558723734}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":2594589126}]}]} 2023-10-06T03:57:18.239 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-06T03:57:18.240 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-06T03:57:18.240 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:57:18.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.354+0000 7f0e2c4e9700 1 Processor -- start 2023-10-06T03:57:18.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.354+0000 7f0e2c4e9700 1 -- start start 2023-10-06T03:57:18.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.355+0000 7f0e2c4e9700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e2406dc40 0x7f0e2414fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:18.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.355+0000 7f0e2c4e9700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e240c9780 con 0x7f0e2406c1f0 2023-10-06T03:57:18.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.355+0000 7f0e2c4e9700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e2406e0c0 con 0x7f0e2406cbb0 2023-10-06T03:57:18.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.355+0000 7f0e2c4e9700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e240c8c40 con 0x7f0e2406dc40 2023-10-06T03:57:18.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.355+0000 7f0e2aa86700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f0e2406c1f0 0x7f0e24155f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39944/0 (socket says 172.21.15.157:39944) 2023-10-06T03:57:18.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.355+0000 7f0e2aa86700 1 -- 172.21.15.157:0/1536864525 learned_addr learned my addr 172.21.15.157:0/1536864525 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:18.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.355+0000 7f0e29a84700 1 --2- 172.21.15.157:0/1536864525 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e2406dc40 0x7f0e2414fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:18.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.356+0000 7f0e29283700 1 -- 172.21.15.157:0/1536864525 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 653689997 0 0) 0x7f0e240c9780 con 0x7f0e2406c1f0 2023-10-06T03:57:18.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.356+0000 7f0e29283700 1 -- 172.21.15.157:0/1536864525 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e10003570 con 0x7f0e2406c1f0 2023-10-06T03:57:18.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.356+0000 7f0e29283700 1 -- 172.21.15.157:0/1536864525 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3632961632 0 0) 0x7f0e2406e0c0 con 0x7f0e2406cbb0 2023-10-06T03:57:18.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.356+0000 7f0e29283700 1 -- 172.21.15.157:0/1536864525 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e240c9780 con 0x7f0e2406cbb0 2023-10-06T03:57:18.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.356+0000 7f0e29a84700 1 -- 172.21.15.157:0/1536864525 >> v1:172.21.15.175:6789/0 conn(0x7f0e2406cbb0 legacy=0x7f0e2406d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:18.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.356+0000 7f0e29a84700 1 -- 172.21.15.157:0/1536864525 >> v1:172.21.15.157:6789/0 conn(0x7f0e2406c1f0 legacy=0x7f0e24155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:18.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.356+0000 7f0e29a84700 1 -- 172.21.15.157:0/1536864525 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e240c8e50 con 0x7f0e2406dc40 2023-10-06T03:57:18.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.356+0000 7f0e29a84700 1 --2- 172.21.15.157:0/1536864525 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e2406dc40 0x7f0e2414fd00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009910 tx=0x7f0e140091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7e64593de2d508c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:18.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.356+0000 7f0e29283700 1 -- 172.21.15.157:0/1536864525 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1400b690 con 0x7f0e2406dc40 2023-10-06T03:57:18.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.356+0000 7f0e29283700 1 -- 172.21.15.157:0/1536864525 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1400b7f0 con 0x7f0e2406dc40 2023-10-06T03:57:18.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.356+0000 7f0e29283700 1 -- 172.21.15.157:0/1536864525 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1400bb70 con 0x7f0e2406dc40 2023-10-06T03:57:18.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.357+0000 7f0e2c4e9700 1 -- 172.21.15.157:0/1536864525 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e2406dc40 msgr2=0x7f0e2414fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:18.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.357+0000 7f0e2c4e9700 1 --2- 172.21.15.157:0/1536864525 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e2406dc40 0x7f0e2414fd00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009910 tx=0x7f0e140091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:18.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.357+0000 7f0e2c4e9700 1 -- 172.21.15.157:0/1536864525 shutdown_connections 2023-10-06T03:57:18.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.357+0000 7f0e2c4e9700 1 --2- 172.21.15.157:0/1536864525 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e2406dc40 0x7f0e2414fd00 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:18.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.357+0000 7f0e2c4e9700 1 -- 172.21.15.157:0/1536864525 >> 172.21.15.157:0/1536864525 conn(0x7f0e2406f230 msgr2=0x7f0e2406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:18.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.357+0000 7f0e2c4e9700 1 -- 172.21.15.157:0/1536864525 shutdown_connections 2023-10-06T03:57:18.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.357+0000 7f0e2c4e9700 1 -- 172.21.15.157:0/1536864525 wait complete. 2023-10-06T03:57:18.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.358+0000 7f0e2c4e9700 1 Processor -- start 2023-10-06T03:57:18.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.358+0000 7f0e2c4e9700 1 -- start start 2023-10-06T03:57:18.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.358+0000 7f0e2c4e9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e2406c1f0 0x7f0e24158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:18.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.358+0000 7f0e2c4e9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e2406cbb0 0x7f0e241575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:18.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.358+0000 7f0e2c4e9700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e24166850 0x7f0e24159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:18.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.358+0000 7f0e2c4e9700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e240c3310 con 0x7f0e2406cbb0 2023-10-06T03:57:18.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.358+0000 7f0e2c4e9700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e24157140 con 0x7f0e2406c1f0 2023-10-06T03:57:18.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.358+0000 7f0e2c4e9700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e24157b10 con 0x7f0e24166850 2023-10-06T03:57:18.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.359+0000 7f0e29a84700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e2406cbb0 0x7f0e241575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:18.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.359+0000 7f0e29a84700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e2406cbb0 0x7f0e241575d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34288/0 (socket says 172.21.15.157:34288) 2023-10-06T03:57:18.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.359+0000 7f0e29a84700 1 -- 172.21.15.157:0/2825928991 learned_addr learned my addr 172.21.15.157:0/2825928991 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:18.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.359+0000 7f0e2a285700 1 --2- 172.21.15.157:0/2825928991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e2406c1f0 0x7f0e24158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:18.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.359+0000 7f0e2aa86700 1 --2- 172.21.15.157:0/2825928991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e24166850 0x7f0e24159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:18.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.359+0000 7f0e29a84700 1 -- 172.21.15.157:0/2825928991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e24166850 msgr2=0x7f0e24159410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:18.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.359+0000 7f0e29a84700 1 --2- 172.21.15.157:0/2825928991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e24166850 0x7f0e24159410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:18.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.359+0000 7f0e29a84700 1 -- 172.21.15.157:0/2825928991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e2406c1f0 msgr2=0x7f0e24158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:18.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.359+0000 7f0e29a84700 1 --2- 172.21.15.157:0/2825928991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e2406c1f0 0x7f0e24158640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:18.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.359+0000 7f0e29a84700 1 -- 172.21.15.157:0/2825928991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e14009420 con 0x7f0e2406cbb0 2023-10-06T03:57:18.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.359+0000 7f0e29a84700 1 --2- 172.21.15.157:0/2825928991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e2406cbb0 0x7f0e241575d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0e240c3960 tx=0x7f0e14017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:18.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.359+0000 7f0e1a7fc700 1 -- 172.21.15.157:0/2825928991 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14000d10 con 0x7f0e2406cbb0 2023-10-06T03:57:18.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.359+0000 7f0e1a7fc700 1 -- 172.21.15.157:0/2825928991 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1400b960 con 0x7f0e2406cbb0 2023-10-06T03:57:18.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.359+0000 7f0e2c4e9700 1 -- 172.21.15.157:0/2825928991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e240c3030 con 0x7f0e2406cbb0 2023-10-06T03:57:18.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.359+0000 7f0e1a7fc700 1 -- 172.21.15.157:0/2825928991 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14025650 con 0x7f0e2406cbb0 2023-10-06T03:57:18.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.359+0000 7f0e2c4e9700 1 -- 172.21.15.157:0/2825928991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e241535e0 con 0x7f0e2406cbb0 2023-10-06T03:57:18.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.360+0000 7f0e2c4e9700 1 -- 172.21.15.157:0/2825928991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e241537a0 con 0x7f0e2406cbb0 2023-10-06T03:57:18.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.360+0000 7f0e1a7fc700 1 -- 172.21.15.157:0/2825928991 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0e14023070 con 0x7f0e2406cbb0 2023-10-06T03:57:18.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.361+0000 7f0e1a7fc700 1 --2- 172.21.15.157:0/2825928991 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0e08066f60 0x7f0e08069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:18.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.361+0000 7f0e1a7fc700 1 -- 172.21.15.157:0/2825928991 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0e14085930 con 0x7f0e2406cbb0 2023-10-06T03:57:18.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.361+0000 7f0e2a285700 1 --2- 172.21.15.157:0/2825928991 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0e08066f60 0x7f0e08069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:18.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.361+0000 7f0e2a285700 1 --2- 172.21.15.157:0/2825928991 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0e08066f60 0x7f0e08069410 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0e20000f80 tx=0x7f0e20002720 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:18.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.365+0000 7f0e1a7fc700 1 -- 172.21.15.157:0/2825928991 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e14052b00 con 0x7f0e2406cbb0 2023-10-06T03:57:18.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.551+0000 7f0e2c4e9700 1 -- 172.21.15.157:0/2825928991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e24157280 con 0x7f0e2406cbb0 2023-10-06T03:57:18.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.552+0000 7f0e1a7fc700 1 -- 172.21.15.157:0/2825928991 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f0e14015090 con 0x7f0e2406cbb0 2023-10-06T03:57:18.554 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:18.554 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":21,"fsid":"28664a7a-fe30-4efe-be5a-6abf9318b700","created":"2023-10-06T03:56:51.887538+0000","modified":"2023-10-06T03:57:12.362537+0000","last_up_change":"2023-10-06T03:57:02.299440+0000","last_in_change":"2023-10-06T03:56:55.141505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:57:03.455121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:57:03.905661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:57:08.377186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:57:09.709582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f677cac-e9f4-4608-acac-718b072ce984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":90640},{"type":"v1","addr":"172.21.15.157:6809","nonce":90640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":90640},{"type":"v1","addr":"172.21.15.157:6811","nonce":90640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":90640},{"type":"v1","addr":"172.21.15.157:6815","nonce":90640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":90640},{"type":"v1","addr":"172.21.15.157:6813","nonce":90640}]},"public_addr":"172.21.15.157:6809/90640","cluster_addr":"172.21.15.157:6811/90640","heartbeat_back_addr":"172.21.15.157:6815/90640","heartbeat_front_addr":"172.21.15.157:6813/90640","state":["exists","up"]},{"osd":1,"uuid":"7d771892-bb91-4e9b-9610-051861c11247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":90654},{"type":"v1","addr":"172.21.15.157:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":90654},{"type":"v1","addr":"172.21.15.157:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":90654},{"type":"v1","addr":"172.21.15.157:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":90654},{"type":"v1","addr":"172.21.15.157:6805","nonce":90654}]},"public_addr":"172.21.15.157:6801/90654","cluster_addr":"172.21.15.157:6803/90654","heartbeat_back_addr":"172.21.15.157:6807/90654","heartbeat_front_addr":"172.21.15.157:6805/90654","state":["exists","up"]},{"osd":2,"uuid":"5c1f7d4e-d185-47cf-ab6c-453b4e3009c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":90657},{"type":"v1","addr":"172.21.15.157:6825","nonce":90657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":90657},{"type":"v1","addr":"172.21.15.157:6827","nonce":90657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":90657},{"type":"v1","addr":"172.21.15.157:6831","nonce":90657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":90657},{"type":"v1","addr":"172.21.15.157:6829","nonce":90657}]},"public_addr":"172.21.15.157:6825/90657","cluster_addr":"172.21.15.157:6827/90657","heartbeat_back_addr":"172.21.15.157:6831/90657","heartbeat_front_addr":"172.21.15.157:6829/90657","state":["exists","up"]},{"osd":3,"uuid":"f6a1e825-43c7-4afd-8daf-e51d45afc75e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":90641},{"type":"v1","addr":"172.21.15.157:6817","nonce":90641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":90641},{"type":"v1","addr":"172.21.15.157:6819","nonce":90641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":90641},{"type":"v1","addr":"172.21.15.157:6823","nonce":90641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":90641},{"type":"v1","addr":"172.21.15.157:6821","nonce":90641}]},"public_addr":"172.21.15.157:6817/90641","cluster_addr":"172.21.15.157:6819/90641","heartbeat_back_addr":"172.21.15.157:6823/90641","heartbeat_front_addr":"172.21.15.157:6821/90641","state":["exists","up"]},{"osd":4,"uuid":"8ad6e3d8-9a31-4f85-9410-417b7ac5da97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":91261},{"type":"v1","addr":"172.21.15.175:6803","nonce":91261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":91261},{"type":"v1","addr":"172.21.15.175:6806","nonce":91261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":91261},{"type":"v1","addr":"172.21.15.175:6814","nonce":91261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":91261},{"type":"v1","addr":"172.21.15.175:6810","nonce":91261}]},"public_addr":"172.21.15.175:6803/91261","cluster_addr":"172.21.15.175:6806/91261","heartbeat_back_addr":"172.21.15.175:6814/91261","heartbeat_front_addr":"172.21.15.175:6810/91261","state":["exists","up"]},{"osd":5,"uuid":"f68aca2a-f182-4c00-8e21-241d09c02ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":91262},{"type":"v1","addr":"172.21.15.175:6817","nonce":91262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":91262},{"type":"v1","addr":"172.21.15.175:6819","nonce":91262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":91262},{"type":"v1","addr":"172.21.15.175:6823","nonce":91262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":91262},{"type":"v1","addr":"172.21.15.175:6821","nonce":91262}]},"public_addr":"172.21.15.175:6817/91262","cluster_addr":"172.21.15.175:6819/91262","heartbeat_back_addr":"172.21.15.175:6823/91262","heartbeat_front_addr":"172.21.15.175:6821/91262","state":["exists","up"]},{"osd":6,"uuid":"d867cd89-3430-40ca-862f-d2bea3de9d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":91263},{"type":"v1","addr":"172.21.15.175:6801","nonce":91263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6805","nonce":91263},{"type":"v1","addr":"172.21.15.175:6807","nonce":91263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6813","nonce":91263},{"type":"v1","addr":"172.21.15.175:6815","nonce":91263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":91263},{"type":"v1","addr":"172.21.15.175:6811","nonce":91263}]},"public_addr":"172.21.15.175:6801/91263","cluster_addr":"172.21.15.175:6807/91263","heartbeat_back_addr":"172.21.15.175:6815/91263","heartbeat_front_addr":"172.21.15.175:6811/91263","state":["exists","up"]},{"osd":7,"uuid":"930cd464-79f1-47cb-b4ff-188ccde74fff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":91306},{"type":"v1","addr":"172.21.15.175:6825","nonce":91306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":91306},{"type":"v1","addr":"172.21.15.175:6827","nonce":91306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":91306},{"type":"v1","addr":"172.21.15.175:6831","nonce":91306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":91306},{"type":"v1","addr":"172.21.15.175:6829","nonce":91306}]},"public_addr":"172.21.15.175:6825/91306","cluster_addr":"172.21.15.175:6827/91306","heartbeat_back_addr":"172.21.15.175:6831/91306","heartbeat_front_addr":"172.21.15.175:6829/91306","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.290673+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.056390+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.292903+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.275861+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.332095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.327453+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.317585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.309330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:57:18.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.555+0000 7f0e2c4e9700 1 -- 172.21.15.157:0/2825928991 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0e08066f60 msgr2=0x7f0e08069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:18.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.555+0000 7f0e2c4e9700 1 --2- 172.21.15.157:0/2825928991 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0e08066f60 0x7f0e08069410 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0e20000f80 tx=0x7f0e20002720 comp rx=0 tx=0).stop 2023-10-06T03:57:18.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.555+0000 7f0e2c4e9700 1 -- 172.21.15.157:0/2825928991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e2406cbb0 msgr2=0x7f0e241575d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:18.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.555+0000 7f0e2c4e9700 1 --2- 172.21.15.157:0/2825928991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e2406cbb0 0x7f0e241575d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0e240c3960 tx=0x7f0e14017000 comp rx=0 tx=0).stop 2023-10-06T03:57:18.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.555+0000 7f0e2c4e9700 1 -- 172.21.15.157:0/2825928991 shutdown_connections 2023-10-06T03:57:18.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.555+0000 7f0e2c4e9700 1 --2- 172.21.15.157:0/2825928991 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0e08066f60 0x7f0e08069410 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:18.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.555+0000 7f0e2c4e9700 1 --2- 172.21.15.157:0/2825928991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e2406c1f0 0x7f0e24158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:18.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.555+0000 7f0e2c4e9700 1 --2- 172.21.15.157:0/2825928991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e24166850 0x7f0e24159410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:18.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.555+0000 7f0e2c4e9700 1 --2- 172.21.15.157:0/2825928991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e2406cbb0 0x7f0e241575d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:18.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.555+0000 7f0e2c4e9700 1 -- 172.21.15.157:0/2825928991 >> 172.21.15.157:0/2825928991 conn(0x7f0e2406f230 msgr2=0x7f0e2414fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:18.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.555+0000 7f0e2c4e9700 1 -- 172.21.15.157:0/2825928991 shutdown_connections 2023-10-06T03:57:18.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.555+0000 7f0e2c4e9700 1 -- 172.21.15.157:0/2825928991 wait complete. 2023-10-06T03:57:18.568 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-06T03:57:18.568 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:57:18.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.681+0000 7f4c9696c700 1 Processor -- start 2023-10-06T03:57:18.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.682+0000 7f4c9696c700 1 -- start start 2023-10-06T03:57:18.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.682+0000 7f4c9696c700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c9006cbb0 0x7f4c9006d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:18.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.682+0000 7f4c9696c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4c9006c1f0 0x7f4c9006da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:18.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.682+0000 7f4c9696c700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c900c9780 con 0x7f4c9006cf80 2023-10-06T03:57:18.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.682+0000 7f4c9696c700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c900c8c40 con 0x7f4c9006c1f0 2023-10-06T03:57:18.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.682+0000 7f4c9696c700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9006e0c0 con 0x7f4c9006cbb0 2023-10-06T03:57:18.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.682+0000 7f4c8f7fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4c9006c1f0 0x7f4c9006da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:18.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.682+0000 7f4c94f09700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f4c9006cf80 0x7f4c9014fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39958/0 (socket says 172.21.15.157:39958) 2023-10-06T03:57:18.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.682+0000 7f4c8f7fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4c9006c1f0 0x7f4c9006da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34298/0 (socket says 172.21.15.157:34298) 2023-10-06T03:57:18.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.682+0000 7f4c94f09700 1 -- 172.21.15.157:0/3961196780 learned_addr learned my addr 172.21.15.157:0/3961196780 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:18.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.683+0000 7f4c8ffff700 1 --2- 172.21.15.157:0/3961196780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c9006cbb0 0x7f4c9006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:18.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.683+0000 7f4c8f7fe700 1 -- 172.21.15.157:0/3961196780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c9006cbb0 msgr2=0x7f4c9006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:18.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.683+0000 7f4c8f7fe700 1 --2- 172.21.15.157:0/3961196780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c9006cbb0 0x7f4c9006d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:18.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.683+0000 7f4c8f7fe700 1 -- 172.21.15.157:0/3961196780 >> v1:172.21.15.157:6789/0 conn(0x7f4c9006cf80 legacy=0x7f4c9014fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:18.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.683+0000 7f4c8f7fe700 1 -- 172.21.15.157:0/3961196780 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c900c8e50 con 0x7f4c9006c1f0 2023-10-06T03:57:18.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.683+0000 7f4c8effd700 1 -- 172.21.15.157:0/3961196780 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213657333 0 0) 0x7f4c900c9780 con 0x7f4c9006cf80 2023-10-06T03:57:18.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.683+0000 7f4c8f7fe700 1 --2- 172.21.15.157:0/3961196780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c9006c1f0 0x7f4c9006da70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009910 tx=0x7f4c840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcbaa5af10bf8203 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:18.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.683+0000 7f4c8effd700 1 -- 172.21.15.157:0/3961196780 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8400b8b0 con 0x7f4c9006c1f0 2023-10-06T03:57:18.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.683+0000 7f4c8effd700 1 -- 172.21.15.157:0/3961196780 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8400ba10 con 0x7f4c9006c1f0 2023-10-06T03:57:18.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.683+0000 7f4c8effd700 1 -- 172.21.15.157:0/3961196780 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8400bdb0 con 0x7f4c9006c1f0 2023-10-06T03:57:18.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.683+0000 7f4c9696c700 1 -- 172.21.15.157:0/3961196780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c9006c1f0 msgr2=0x7f4c9006da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:18.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.683+0000 7f4c9696c700 1 --2- 172.21.15.157:0/3961196780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c9006c1f0 0x7f4c9006da70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009910 tx=0x7f4c840091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:18.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.683+0000 7f4c9696c700 1 -- 172.21.15.157:0/3961196780 shutdown_connections 2023-10-06T03:57:18.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.683+0000 7f4c9696c700 1 --2- 172.21.15.157:0/3961196780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c9006cbb0 0x7f4c9006d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:18.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.683+0000 7f4c9696c700 1 --2- 172.21.15.157:0/3961196780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c9006c1f0 0x7f4c9006da70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:18.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.683+0000 7f4c9696c700 1 -- 172.21.15.157:0/3961196780 >> 172.21.15.157:0/3961196780 conn(0x7f4c9006f230 msgr2=0x7f4c9006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:18.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.684+0000 7f4c9696c700 1 -- 172.21.15.157:0/3961196780 shutdown_connections 2023-10-06T03:57:18.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.684+0000 7f4c9696c700 1 -- 172.21.15.157:0/3961196780 wait complete. 2023-10-06T03:57:18.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.685+0000 7f4c9696c700 1 Processor -- start 2023-10-06T03:57:18.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.685+0000 7f4c9696c700 1 -- start start 2023-10-06T03:57:18.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.685+0000 7f4c9696c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c9006c1f0 0x7f4c90169cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:18.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.685+0000 7f4c9696c700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c9006cbb0 0x7f4c900c55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:18.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.685+0000 7f4c9696c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c9006cf80 0x7f4c900c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:18.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.685+0000 7f4c9696c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c90153bb0 con 0x7f4c9006c1f0 2023-10-06T03:57:18.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.685+0000 7f4c9696c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c90154170 con 0x7f4c9006cf80 2023-10-06T03:57:18.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.685+0000 7f4c9696c700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c90154450 con 0x7f4c9006cbb0 2023-10-06T03:57:18.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.685+0000 7f4c8ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c9006c1f0 0x7f4c90169cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:18.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.686+0000 7f4c8ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c9006c1f0 0x7f4c90169cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34308/0 (socket says 172.21.15.157:34308) 2023-10-06T03:57:18.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.686+0000 7f4c8ffff700 1 -- 172.21.15.157:0/2867044629 learned_addr learned my addr 172.21.15.157:0/2867044629 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:18.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.686+0000 7f4c8f7fe700 1 --2- 172.21.15.157:0/2867044629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c9006cbb0 0x7f4c900c55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:18.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.686+0000 7f4c8ffff700 1 -- 172.21.15.157:0/2867044629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c9006cbb0 msgr2=0x7f4c900c55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:18.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.686+0000 7f4c8ffff700 1 --2- 172.21.15.157:0/2867044629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c9006cbb0 0x7f4c900c55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:18.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.686+0000 7f4c8ffff700 1 -- 172.21.15.157:0/2867044629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c9006cf80 msgr2=0x7f4c900c5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:18.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.686+0000 7f4c8ffff700 1 --2- 172.21.15.157:0/2867044629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c9006cf80 0x7f4c900c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:18.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.686+0000 7f4c8ffff700 1 -- 172.21.15.157:0/2867044629 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c84009420 con 0x7f4c9006c1f0 2023-10-06T03:57:18.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.686+0000 7f4c8ffff700 1 --2- 172.21.15.157:0/2867044629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c9006c1f0 0x7f4c90169cd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000d890 tx=0x7f4c80008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:18.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.686+0000 7f4c77fff700 1 -- 172.21.15.157:0/2867044629 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8000b600 con 0x7f4c9006c1f0 2023-10-06T03:57:18.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.686+0000 7f4c77fff700 1 -- 172.21.15.157:0/2867044629 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8000b760 con 0x7f4c9006c1f0 2023-10-06T03:57:18.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.686+0000 7f4c9696c700 1 -- 172.21.15.157:0/2867044629 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c901572c0 con 0x7f4c9006c1f0 2023-10-06T03:57:18.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.686+0000 7f4c77fff700 1 -- 172.21.15.157:0/2867044629 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8000bae0 con 0x7f4c9006c1f0 2023-10-06T03:57:18.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.686+0000 7f4c9696c700 1 -- 172.21.15.157:0/2867044629 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c90153e00 con 0x7f4c9006c1f0 2023-10-06T03:57:18.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.687+0000 7f4c9696c700 1 -- 172.21.15.157:0/2867044629 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c9016a9e0 con 0x7f4c9006c1f0 2023-10-06T03:57:18.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.687+0000 7f4c77fff700 1 -- 172.21.15.157:0/2867044629 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4c8001e460 con 0x7f4c9006c1f0 2023-10-06T03:57:18.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.688+0000 7f4c77fff700 1 --2- 172.21.15.157:0/2867044629 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4c7c066f60 0x7f4c7c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:18.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.688+0000 7f4c77fff700 1 -- 172.21.15.157:0/2867044629 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4c80084d40 con 0x7f4c9006c1f0 2023-10-06T03:57:18.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.688+0000 7f4c8f7fe700 1 --2- 172.21.15.157:0/2867044629 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4c7c066f60 0x7f4c7c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:18.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.689+0000 7f4c8f7fe700 1 --2- 172.21.15.157:0/2867044629 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4c7c066f60 0x7f4c7c069410 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4c90153370 tx=0x7f4c84017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:18.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.692+0000 7f4c77fff700 1 -- 172.21.15.157:0/2867044629 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c80051e50 con 0x7f4c9006c1f0 2023-10-06T03:57:18.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.878+0000 7f4c9696c700 1 -- 172.21.15.157:0/2867044629 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c900c9130 con 0x7f4c9006c1f0 2023-10-06T03:57:18.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.879+0000 7f4c77fff700 1 -- 172.21.15.157:0/2867044629 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f4c80055470 con 0x7f4c9006c1f0 2023-10-06T03:57:18.880 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:18.880 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":21,"fsid":"28664a7a-fe30-4efe-be5a-6abf9318b700","created":"2023-10-06T03:56:51.887538+0000","modified":"2023-10-06T03:57:12.362537+0000","last_up_change":"2023-10-06T03:57:02.299440+0000","last_in_change":"2023-10-06T03:56:55.141505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:57:03.455121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:57:03.905661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:57:08.377186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:57:09.709582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f677cac-e9f4-4608-acac-718b072ce984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":90640},{"type":"v1","addr":"172.21.15.157:6809","nonce":90640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":90640},{"type":"v1","addr":"172.21.15.157:6811","nonce":90640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":90640},{"type":"v1","addr":"172.21.15.157:6815","nonce":90640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":90640},{"type":"v1","addr":"172.21.15.157:6813","nonce":90640}]},"public_addr":"172.21.15.157:6809/90640","cluster_addr":"172.21.15.157:6811/90640","heartbeat_back_addr":"172.21.15.157:6815/90640","heartbeat_front_addr":"172.21.15.157:6813/90640","state":["exists","up"]},{"osd":1,"uuid":"7d771892-bb91-4e9b-9610-051861c11247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":90654},{"type":"v1","addr":"172.21.15.157:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":90654},{"type":"v1","addr":"172.21.15.157:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":90654},{"type":"v1","addr":"172.21.15.157:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":90654},{"type":"v1","addr":"172.21.15.157:6805","nonce":90654}]},"public_addr":"172.21.15.157:6801/90654","cluster_addr":"172.21.15.157:6803/90654","heartbeat_back_addr":"172.21.15.157:6807/90654","heartbeat_front_addr":"172.21.15.157:6805/90654","state":["exists","up"]},{"osd":2,"uuid":"5c1f7d4e-d185-47cf-ab6c-453b4e3009c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":90657},{"type":"v1","addr":"172.21.15.157:6825","nonce":90657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":90657},{"type":"v1","addr":"172.21.15.157:6827","nonce":90657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":90657},{"type":"v1","addr":"172.21.15.157:6831","nonce":90657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":90657},{"type":"v1","addr":"172.21.15.157:6829","nonce":90657}]},"public_addr":"172.21.15.157:6825/90657","cluster_addr":"172.21.15.157:6827/90657","heartbeat_back_addr":"172.21.15.157:6831/90657","heartbeat_front_addr":"172.21.15.157:6829/90657","state":["exists","up"]},{"osd":3,"uuid":"f6a1e825-43c7-4afd-8daf-e51d45afc75e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":90641},{"type":"v1","addr":"172.21.15.157:6817","nonce":90641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":90641},{"type":"v1","addr":"172.21.15.157:6819","nonce":90641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":90641},{"type":"v1","addr":"172.21.15.157:6823","nonce":90641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":90641},{"type":"v1","addr":"172.21.15.157:6821","nonce":90641}]},"public_addr":"172.21.15.157:6817/90641","cluster_addr":"172.21.15.157:6819/90641","heartbeat_back_addr":"172.21.15.157:6823/90641","heartbeat_front_addr":"172.21.15.157:6821/90641","state":["exists","up"]},{"osd":4,"uuid":"8ad6e3d8-9a31-4f85-9410-417b7ac5da97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":91261},{"type":"v1","addr":"172.21.15.175:6803","nonce":91261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":91261},{"type":"v1","addr":"172.21.15.175:6806","nonce":91261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":91261},{"type":"v1","addr":"172.21.15.175:6814","nonce":91261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":91261},{"type":"v1","addr":"172.21.15.175:6810","nonce":91261}]},"public_addr":"172.21.15.175:6803/91261","cluster_addr":"172.21.15.175:6806/91261","heartbeat_back_addr":"172.21.15.175:6814/91261","heartbeat_front_addr":"172.21.15.175:6810/91261","state":["exists","up"]},{"osd":5,"uuid":"f68aca2a-f182-4c00-8e21-241d09c02ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":91262},{"type":"v1","addr":"172.21.15.175:6817","nonce":91262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":91262},{"type":"v1","addr":"172.21.15.175:6819","nonce":91262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":91262},{"type":"v1","addr":"172.21.15.175:6823","nonce":91262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":91262},{"type":"v1","addr":"172.21.15.175:6821","nonce":91262}]},"public_addr":"172.21.15.175:6817/91262","cluster_addr":"172.21.15.175:6819/91262","heartbeat_back_addr":"172.21.15.175:6823/91262","heartbeat_front_addr":"172.21.15.175:6821/91262","state":["exists","up"]},{"osd":6,"uuid":"d867cd89-3430-40ca-862f-d2bea3de9d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":91263},{"type":"v1","addr":"172.21.15.175:6801","nonce":91263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6805","nonce":91263},{"type":"v1","addr":"172.21.15.175:6807","nonce":91263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6813","nonce":91263},{"type":"v1","addr":"172.21.15.175:6815","nonce":91263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":91263},{"type":"v1","addr":"172.21.15.175:6811","nonce":91263}]},"public_addr":"172.21.15.175:6801/91263","cluster_addr":"172.21.15.175:6807/91263","heartbeat_back_addr":"172.21.15.175:6815/91263","heartbeat_front_addr":"172.21.15.175:6811/91263","state":["exists","up"]},{"osd":7,"uuid":"930cd464-79f1-47cb-b4ff-188ccde74fff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":91306},{"type":"v1","addr":"172.21.15.175:6825","nonce":91306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":91306},{"type":"v1","addr":"172.21.15.175:6827","nonce":91306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":91306},{"type":"v1","addr":"172.21.15.175:6831","nonce":91306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":91306},{"type":"v1","addr":"172.21.15.175:6829","nonce":91306}]},"public_addr":"172.21.15.175:6825/91306","cluster_addr":"172.21.15.175:6827/91306","heartbeat_back_addr":"172.21.15.175:6831/91306","heartbeat_front_addr":"172.21.15.175:6829/91306","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.290673+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.056390+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.292903+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.275861+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.332095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.327453+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.317585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.309330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:57:18.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.881+0000 7f4c9696c700 1 -- 172.21.15.157:0/2867044629 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4c7c066f60 msgr2=0x7f4c7c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:18.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.881+0000 7f4c9696c700 1 --2- 172.21.15.157:0/2867044629 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4c7c066f60 0x7f4c7c069410 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4c90153370 tx=0x7f4c84017000 comp rx=0 tx=0).stop 2023-10-06T03:57:18.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.881+0000 7f4c9696c700 1 -- 172.21.15.157:0/2867044629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c9006c1f0 msgr2=0x7f4c90169cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:18.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.881+0000 7f4c9696c700 1 --2- 172.21.15.157:0/2867044629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c9006c1f0 0x7f4c90169cd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000d890 tx=0x7f4c80008040 comp rx=0 tx=0).stop 2023-10-06T03:57:18.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.881+0000 7f4c9696c700 1 -- 172.21.15.157:0/2867044629 shutdown_connections 2023-10-06T03:57:18.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.881+0000 7f4c9696c700 1 --2- 172.21.15.157:0/2867044629 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4c7c066f60 0x7f4c7c069410 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:18.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.881+0000 7f4c9696c700 1 --2- 172.21.15.157:0/2867044629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c9006cf80 0x7f4c900c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:18.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.881+0000 7f4c9696c700 1 --2- 172.21.15.157:0/2867044629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c9006c1f0 0x7f4c90169cd0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:18.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.881+0000 7f4c9696c700 1 --2- 172.21.15.157:0/2867044629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c9006cbb0 0x7f4c900c55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:18.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.881+0000 7f4c9696c700 1 -- 172.21.15.157:0/2867044629 >> 172.21.15.157:0/2867044629 conn(0x7f4c9006f230 msgr2=0x7f4c90150e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:18.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.881+0000 7f4c9696c700 1 -- 172.21.15.157:0/2867044629 shutdown_connections 2023-10-06T03:57:18.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:18.881+0000 7f4c9696c700 1 -- 172.21.15.157:0/2867044629 wait complete. 2023-10-06T03:57:18.895 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-06T03:57:18.895 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-06T03:57:18.896 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-06T03:57:18.896 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-06T03:57:18.896 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-06T03:57:18.897 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-06T03:57:18.897 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-06T03:57:18.898 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-06T03:57:19.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.087+0000 7f192906e700 1 Processor -- start 2023-10-06T03:57:19.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.087+0000 7f192906e700 1 -- start start 2023-10-06T03:57:19.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.088+0000 7f192906e700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19240adca0 0x7f192415b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.088+0000 7f192906e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f19240ae070 0x7f192406a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.088+0000 7f192906e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1924069c00 con 0x7f192406c1f0 2023-10-06T03:57:19.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.088+0000 7f192906e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f192406cc70 con 0x7f19240ae070 2023-10-06T03:57:19.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.088+0000 7f192906e700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f192415b7b0 con 0x7f19240adca0 2023-10-06T03:57:19.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.089+0000 7f192359e700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f192406c1f0 0x7f19240c2c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.157:38264/0 (socket says 172.21.15.157:38264) 2023-10-06T03:57:19.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.089+0000 7f192359e700 1 -- 172.21.15.157:0/409550175 learned_addr learned my addr 172.21.15.157:0/409550175 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:19.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.089+0000 7f192259c700 1 --2- 172.21.15.157:0/409550175 >> v2:172.21.15.175:3300/0 conn(0x7f19240ae070 0x7f192406a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.089+0000 7f1922d9d700 1 --2- 172.21.15.157:0/409550175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19240adca0 0x7f192415b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.089+0000 7fd877654700 1 Processor -- start 2023-10-06T03:57:19.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.090+0000 7fd877654700 1 -- start start 2023-10-06T03:57:19.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.091+0000 7fd877654700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd87006c440 0x7fd87006e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.091+0000 7fd877654700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd87006c810 0x7fd870069410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.091+0000 7fd877654700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd870069950 0x7fd8700c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.091+0000 7fd877654700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd87014cb50 con 0x7fd87006c440 2023-10-06T03:57:19.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.091+0000 7fd877654700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8700c3a80 con 0x7fd870069950 2023-10-06T03:57:19.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.091+0000 7fd877654700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8700c37a0 con 0x7fd87006c810 2023-10-06T03:57:19.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.091+0000 7fd874bef700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd87006c810 0x7fd870069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.091+0000 7fd874bef700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd87006c810 0x7fd870069410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:50050/0 (socket says 172.21.15.157:50050) 2023-10-06T03:57:19.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.089+0000 7f1921d9b700 1 -- 172.21.15.157:0/409550175 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2882741543 0 0) 0x7f1924069c00 con 0x7f192406c1f0 2023-10-06T03:57:19.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.089+0000 7f1921d9b700 1 -- 172.21.15.157:0/409550175 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f190c003570 con 0x7f192406c1f0 2023-10-06T03:57:19.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.089+0000 7f1922d9d700 1 -- 172.21.15.157:0/409550175 >> v2:172.21.15.175:3300/0 conn(0x7f19240ae070 msgr2=0x7f192406a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:19.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.089+0000 7f1922d9d700 1 --2- 172.21.15.157:0/409550175 >> v2:172.21.15.175:3300/0 conn(0x7f19240ae070 0x7f192406a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.089+0000 7f1922d9d700 1 -- 172.21.15.157:0/409550175 >> v1:172.21.15.175:6789/0 conn(0x7f192406c1f0 legacy=0x7f19240c2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.090+0000 7f1922d9d700 1 -- 172.21.15.157:0/409550175 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192406c730 con 0x7f19240adca0 2023-10-06T03:57:19.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.090+0000 7f192259c700 1 --2- 172.21.15.157:0/409550175 >> v2:172.21.15.175:3300/0 conn(0x7f19240ae070 0x7f192406a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:57:19.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.090+0000 7f1922d9d700 1 --2- 172.21.15.157:0/409550175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19240adca0 0x7f192415b270 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f19140098e0 tx=0x7f19140091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d162fb12ad1a721d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.090+0000 7f1921d9b700 1 -- 172.21.15.157:0/409550175 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1914011630 con 0x7f19240adca0 2023-10-06T03:57:19.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.090+0000 7f1921d9b700 1 -- 172.21.15.157:0/409550175 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1914011790 con 0x7f19240adca0 2023-10-06T03:57:19.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.090+0000 7f1921d9b700 1 -- 172.21.15.157:0/409550175 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1914011b10 con 0x7f19240adca0 2023-10-06T03:57:19.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.091+0000 7f192906e700 1 -- 172.21.15.157:0/409550175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19240adca0 msgr2=0x7f192415b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.091+0000 7f192906e700 1 --2- 172.21.15.157:0/409550175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19240adca0 0x7f192415b270 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f19140098e0 tx=0x7f19140091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.091+0000 7f192906e700 1 -- 172.21.15.157:0/409550175 shutdown_connections 2023-10-06T03:57:19.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.091+0000 7f192906e700 1 --2- 172.21.15.157:0/409550175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19240adca0 0x7f192415b270 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.091+0000 7f192906e700 1 --2- 172.21.15.157:0/409550175 >> v2:172.21.15.175:3300/0 conn(0x7f19240ae070 0x7f192406a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.091+0000 7f192906e700 1 -- 172.21.15.157:0/409550175 >> 172.21.15.157:0/409550175 conn(0x7f19240c82e0 msgr2=0x7f19240c86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.091+0000 7fd874bef700 1 -- 172.21.15.157:0/4198253539 learned_addr learned my addr 172.21.15.157:0/4198253539 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:19.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.091+0000 7f192906e700 1 -- 172.21.15.157:0/409550175 shutdown_connections 2023-10-06T03:57:19.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.091+0000 7f192906e700 1 -- 172.21.15.157:0/409550175 wait complete. 2023-10-06T03:57:19.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.091+0000 7fd874bef700 1 -- 172.21.15.157:0/4198253539 >> v2:172.21.15.175:3300/0 conn(0x7fd870069950 msgr2=0x7fd8700c2c50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T03:57:19.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.091+0000 7fd874bef700 1 --2- 172.21.15.157:0/4198253539 >> v2:172.21.15.175:3300/0 conn(0x7fd870069950 0x7fd8700c2c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.091+0000 7fd874bef700 1 -- 172.21.15.157:0/4198253539 >> v2:172.21.15.157:3300/0 conn(0x7fd87006c440 msgr2=0x7fd87006e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:19.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.092+0000 7fd874bef700 1 --2- 172.21.15.157:0/4198253539 >> v2:172.21.15.157:3300/0 conn(0x7fd87006c440 0x7fd87006e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.092+0000 7fd874bef700 1 -- 172.21.15.157:0/4198253539 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd870155b60 con 0x7fd87006c810 2023-10-06T03:57:19.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.092+0000 7fd874bef700 1 --2- 172.21.15.157:0/4198253539 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd87006c810 0x7fd870069410 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd86800d300 tx=0x7fd8680076e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54f440728b969656 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.093+0000 7fd86ffff700 1 -- 172.21.15.157:0/4198253539 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd868010070 con 0x7fd87006c810 2023-10-06T03:57:19.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.093+0000 7fd86ffff700 1 -- 172.21.15.157:0/4198253539 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86800de00 con 0x7fd87006c810 2023-10-06T03:57:19.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.093+0000 7fd86ffff700 1 -- 172.21.15.157:0/4198253539 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd868007da0 con 0x7fd87006c810 2023-10-06T03:57:19.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.093+0000 7f192906e700 1 Processor -- start 2023-10-06T03:57:19.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.094+0000 7fd877654700 1 -- 172.21.15.157:0/4198253539 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd87006c810 msgr2=0x7fd870069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.094+0000 7fd877654700 1 --2- 172.21.15.157:0/4198253539 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd87006c810 0x7fd870069410 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd86800d300 tx=0x7fd8680076e0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.093+0000 7f192906e700 1 -- start start 2023-10-06T03:57:19.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.094+0000 7f192906e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f192406c1f0 0x7f19241374f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.094+0000 7f192906e700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19240adca0 0x7f1924159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.094+0000 7f192906e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19240ae070 0x7f1924159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.094+0000 7f192906e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1924157720 con 0x7f192406c1f0 2023-10-06T03:57:19.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.094+0000 7f192906e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1924154e10 con 0x7f19240ae070 2023-10-06T03:57:19.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.094+0000 7f192906e700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1924131a40 con 0x7f19240adca0 2023-10-06T03:57:19.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.095+0000 7f1922d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f192406c1f0 0x7f19241374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.095+0000 7f1922d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f192406c1f0 0x7f19241374f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34334/0 (socket says 172.21.15.157:34334) 2023-10-06T03:57:19.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.095+0000 7f1922d9d700 1 -- 172.21.15.157:0/912029013 learned_addr learned my addr 172.21.15.157:0/912029013 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:19.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.095+0000 7f1922d9d700 1 -- 172.21.15.157:0/912029013 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19240adca0 msgr2=0x7f1924159a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:57:19.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.095+0000 7f1922d9d700 1 --2- 172.21.15.157:0/912029013 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19240adca0 0x7f1924159a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.095+0000 7f1922d9d700 1 -- 172.21.15.157:0/912029013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19240ae070 msgr2=0x7f1924159f70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T03:57:19.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.095+0000 7f1922d9d700 1 --2- 172.21.15.157:0/912029013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19240ae070 0x7f1924159f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.095+0000 7f1922d9d700 1 -- 172.21.15.157:0/912029013 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1918009420 con 0x7f192406c1f0 2023-10-06T03:57:19.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.095+0000 7fd877654700 1 -- 172.21.15.157:0/4198253539 shutdown_connections 2023-10-06T03:57:19.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.095+0000 7fd877654700 1 --2- 172.21.15.157:0/4198253539 >> v2:172.21.15.175:3300/0 conn(0x7fd870069950 0x7fd8700c2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.095+0000 7fd877654700 1 --2- 172.21.15.157:0/4198253539 >> v2:172.21.15.157:3300/0 conn(0x7fd87006c440 0x7fd87006e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.095+0000 7fd877654700 1 --2- 172.21.15.157:0/4198253539 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd87006c810 0x7fd870069410 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.095+0000 7fd877654700 1 -- 172.21.15.157:0/4198253539 >> 172.21.15.157:0/4198253539 conn(0x7fd8700c82e0 msgr2=0x7fd8700c86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.095+0000 7f1922d9d700 1 --2- 172.21.15.157:0/912029013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f192406c1f0 0x7f19241374f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1914009ed0 tx=0x7f1914017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.095+0000 7f53985b7700 1 Processor -- start 2023-10-06T03:57:19.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.095+0000 7f53985b7700 1 -- start start 2023-10-06T03:57:19.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.096+0000 7f190affd700 1 -- 172.21.15.157:0/912029013 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1914011900 con 0x7f192406c1f0 2023-10-06T03:57:19.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.096+0000 7f190affd700 1 -- 172.21.15.157:0/912029013 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1914011a80 con 0x7f192406c1f0 2023-10-06T03:57:19.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.096+0000 7f190affd700 1 -- 172.21.15.157:0/912029013 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1914025670 con 0x7f192406c1f0 2023-10-06T03:57:19.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.096+0000 7f192906e700 1 -- 172.21.15.157:0/912029013 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1914009420 con 0x7f192406c1f0 2023-10-06T03:57:19.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.096+0000 7f192906e700 1 -- 172.21.15.157:0/912029013 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19241584a0 con 0x7f192406c1f0 2023-10-06T03:57:19.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.097+0000 7f192906e700 1 -- 172.21.15.157:0/912029013 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f19240716b0 con 0x7f192406c1f0 2023-10-06T03:57:19.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.097+0000 7fd877654700 1 -- 172.21.15.157:0/4198253539 shutdown_connections 2023-10-06T03:57:19.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.097+0000 7f53985b7700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f53900ae070 0x7f539015b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.097+0000 7f53985b7700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f539006c1f0 0x7f53900c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.097+0000 7f53985b7700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5390069c00 con 0x7f53900adca0 2023-10-06T03:57:19.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.097+0000 7f53985b7700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f539006cc70 con 0x7f539006c1f0 2023-10-06T03:57:19.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.097+0000 7f53985b7700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f539015b7b0 con 0x7f53900ae070 2023-10-06T03:57:19.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.097+0000 7f5396353700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f53900adca0 0x7f539006a090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39966/0 (socket says 172.21.15.157:39966) 2023-10-06T03:57:19.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.097+0000 7f5396353700 1 -- 172.21.15.157:0/1520532365 learned_addr learned my addr 172.21.15.157:0/1520532365 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:19.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.097+0000 7fd877654700 1 -- 172.21.15.157:0/4198253539 wait complete. 2023-10-06T03:57:19.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.097+0000 7f5395b52700 1 --2- 172.21.15.157:0/1520532365 >> v2:172.21.15.175:3300/0 conn(0x7f53900ae070 0x7f539015b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.098+0000 7f5395b52700 1 -- 172.21.15.157:0/1520532365 >> v2:172.21.15.157:3300/0 conn(0x7f539006c1f0 msgr2=0x7f53900c2c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T03:57:19.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.098+0000 7f5395b52700 1 --2- 172.21.15.157:0/1520532365 >> v2:172.21.15.157:3300/0 conn(0x7f539006c1f0 0x7f53900c2c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.098+0000 7f5395b52700 1 -- 172.21.15.157:0/1520532365 >> v1:172.21.15.157:6789/0 conn(0x7f53900adca0 legacy=0x7f539006a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:19.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.098+0000 7f5395b52700 1 -- 172.21.15.157:0/1520532365 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f539006a7a0 con 0x7f53900ae070 2023-10-06T03:57:19.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.098+0000 7f190affd700 1 -- 172.21.15.157:0/912029013 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1914023070 con 0x7f192406c1f0 2023-10-06T03:57:19.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.098+0000 7f5395b52700 1 --2- 172.21.15.157:0/1520532365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53900ae070 0x7f539015b270 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f539006fdd0 tx=0x7f538c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c523dd3856ae167 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.096+0000 7ff51550c700 1 Processor -- start 2023-10-06T03:57:19.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.096+0000 7ff51550c700 1 -- start start 2023-10-06T03:57:19.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.097+0000 7ff51550c700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff5101546f0 0x7ff51006a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.097+0000 7ff51550c700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff51015b200 con 0x7ff5100bf8f0 2023-10-06T03:57:19.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.097+0000 7ff51550c700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff51015a310 con 0x7ff5100bf520 2023-10-06T03:57:19.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.097+0000 7ff51550c700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff51006b480 con 0x7ff5101546f0 2023-10-06T03:57:19.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.097+0000 7ff50f7fe700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff5101546f0 0x7ff51006a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.098+0000 7ff50f7fe700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff5101546f0 0x7ff51006a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:35050/0 (socket says 172.21.15.157:35050) 2023-10-06T03:57:19.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.098+0000 7ff50f7fe700 1 -- 172.21.15.157:0/321383680 learned_addr learned my addr 172.21.15.157:0/321383680 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:19.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.098+0000 7ff50f7fe700 1 -- 172.21.15.157:0/321383680 >> v1:172.21.15.175:6789/0 conn(0x7ff5100bf520 legacy=0x7ff5100631f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T03:57:19.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.098+0000 7ff50f7fe700 1 -- 172.21.15.157:0/321383680 >> v1:172.21.15.157:6789/0 conn(0x7ff5100bf8f0 legacy=0x7ff510153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:19.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.098+0000 7ff50f7fe700 1 -- 172.21.15.157:0/321383680 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51015a5f0 con 0x7ff5101546f0 2023-10-06T03:57:19.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.098+0000 7ff50f7fe700 1 --2- 172.21.15.157:0/321383680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5101546f0 0x7ff51006a6a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff5040099b0 tx=0x7ff504009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3eaba48ff2fe96e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.098+0000 7ff50dffb700 1 -- 172.21.15.157:0/321383680 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50400b7e0 con 0x7ff5101546f0 2023-10-06T03:57:19.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.098+0000 7ff50dffb700 1 -- 172.21.15.157:0/321383680 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50400b940 con 0x7ff5101546f0 2023-10-06T03:57:19.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.098+0000 7ff50dffb700 1 -- 172.21.15.157:0/321383680 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50400bce0 con 0x7ff5101546f0 2023-10-06T03:57:19.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.099+0000 7f6f1ab13700 1 Processor -- start 2023-10-06T03:57:19.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.099+0000 7f190affd700 1 --2- 172.21.15.157:0/912029013 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1910067000 0x7f19100694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.099+0000 7f190affd700 1 -- 172.21.15.157:0/912029013 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f19140861d0 con 0x7f192406c1f0 2023-10-06T03:57:19.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.099+0000 7fd877654700 1 Processor -- start 2023-10-06T03:57:19.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.099+0000 7fd877654700 1 -- start start 2023-10-06T03:57:19.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.099+0000 7f192259c700 1 --2- 172.21.15.157:0/912029013 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1910067000 0x7f19100694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.099+0000 7f190affd700 1 --2- 172.21.15.157:0/912029013 >> [v2:172.21.15.175:6800/91261,v1:172.21.15.175:6803/91261] conn(0x7f191006f8b0 0x7f1910071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.099+0000 7f190affd700 1 -- 172.21.15.157:0/912029013 --> [v2:172.21.15.175:6800/91261,v1:172.21.15.175:6803/91261] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1910072370 con 0x7f191006f8b0 2023-10-06T03:57:19.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.099+0000 7f190affd700 1 -- 172.21.15.157:0/912029013 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19140865a0 con 0x7f192406c1f0 2023-10-06T03:57:19.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.099+0000 7f192359e700 1 --2- 172.21.15.157:0/912029013 >> [v2:172.21.15.175:6800/91261,v1:172.21.15.175:6803/91261] conn(0x7f191006f8b0 0x7f1910071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.099+0000 7ff51550c700 1 -- 172.21.15.157:0/321383680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5101546f0 msgr2=0x7ff51006a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7ff51550c700 1 --2- 172.21.15.157:0/321383680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5101546f0 0x7ff51006a6a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff5040099b0 tx=0x7ff504009250 comp rx=0 tx=0).stop 2023-10-06T03:57:19.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7f192259c700 1 --2- 172.21.15.157:0/912029013 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1910067000 0x7f19100694b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f19240c4290 tx=0x7f19180091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.098+0000 7f5395351700 1 -- 172.21.15.157:0/1520532365 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c00b700 con 0x7f53900ae070 2023-10-06T03:57:19.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.099+0000 7f5395351700 1 -- 172.21.15.157:0/1520532365 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f538c00b880 con 0x7f53900ae070 2023-10-06T03:57:19.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.099+0000 7f5395351700 1 -- 172.21.15.157:0/1520532365 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c00bc40 con 0x7f53900ae070 2023-10-06T03:57:19.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.099+0000 7f53985b7700 1 -- 172.21.15.157:0/1520532365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53900ae070 msgr2=0x7f539015b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.099+0000 7f53985b7700 1 --2- 172.21.15.157:0/1520532365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53900ae070 0x7f539015b270 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f539006fdd0 tx=0x7f538c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7ff51550c700 1 -- 172.21.15.157:0/321383680 shutdown_connections 2023-10-06T03:57:19.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7ff51550c700 1 --2- 172.21.15.157:0/321383680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5101546f0 0x7ff51006a6a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7ff51550c700 1 -- 172.21.15.157:0/321383680 >> 172.21.15.157:0/321383680 conn(0x7ff5100c8b20 msgr2=0x7ff5100c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7f53985b7700 1 -- 172.21.15.157:0/1520532365 shutdown_connections 2023-10-06T03:57:19.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7f53985b7700 1 --2- 172.21.15.157:0/1520532365 >> v2:172.21.15.157:3300/0 conn(0x7f539006c1f0 0x7f53900c2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7f53985b7700 1 --2- 172.21.15.157:0/1520532365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53900ae070 0x7f539015b270 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7f53985b7700 1 -- 172.21.15.157:0/1520532365 >> 172.21.15.157:0/1520532365 conn(0x7f53900c82e0 msgr2=0x7f53900c86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7f192359e700 1 --2- 172.21.15.157:0/912029013 >> [v2:172.21.15.175:6800/91261,v1:172.21.15.175:6803/91261] conn(0x7f191006f8b0 0x7f1910071cc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7ff3597b3700 1 Processor -- start 2023-10-06T03:57:19.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.099+0000 7f6f1ab13700 1 -- start start 2023-10-06T03:57:19.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7f6f1ab13700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f1415afa0 0x7f6f1406e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7f6f1ab13700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f1406d180 con 0x7f6f140b1e80 2023-10-06T03:57:19.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7f6f1ab13700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f1406d460 con 0x7f6f140b2250 2023-10-06T03:57:19.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7f6f1ab13700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1406cf70 con 0x7f6f1415afa0 2023-10-06T03:57:19.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7f6f13fff700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f1415afa0 0x7f6f1406e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7f6f188af700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f6f140b1e80 0x7f6f1406c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39982/0 (socket says 172.21.15.157:39982) 2023-10-06T03:57:19.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7f6f188af700 1 -- 172.21.15.157:0/59279779 learned_addr learned my addr 172.21.15.157:0/59279779 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:19.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7ff3597b3700 1 -- start start 2023-10-06T03:57:19.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7fd877654700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd870069950 0x7fd87014e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7fd877654700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd87006c440 0x7fd8701731c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7fd877654700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd87006c810 0x7fd870173700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7fd877654700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8700c8a70 con 0x7fd87006c810 2023-10-06T03:57:19.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7fd877654700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd870152320 con 0x7fd87006c440 2023-10-06T03:57:19.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7fd877654700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd870153100 con 0x7fd870069950 2023-10-06T03:57:19.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7ff51550c700 1 -- 172.21.15.157:0/321383680 shutdown_connections 2023-10-06T03:57:19.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7ff51550c700 1 -- 172.21.15.157:0/321383680 wait complete. 2023-10-06T03:57:19.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7f6f13fff700 1 -- 172.21.15.157:0/59279779 >> v1:172.21.15.175:6789/0 conn(0x7f6f140b2250 legacy=0x7f6f14157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7f6f13fff700 1 -- 172.21.15.157:0/59279779 >> v1:172.21.15.157:6789/0 conn(0x7f6f140b1e80 legacy=0x7f6f1406c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7f6f13fff700 1 -- 172.21.15.157:0/59279779 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1406d740 con 0x7f6f1415afa0 2023-10-06T03:57:19.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.100+0000 7fd874bef700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd87006c440 0x7fd8701731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.101+0000 7fd8753f0700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd870069950 0x7fd87014e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.101+0000 7fd874bef700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd87006c440 0x7fd8701731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:35054/0 (socket says 172.21.15.157:35054) 2023-10-06T03:57:19.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.101+0000 7fd874bef700 1 -- 172.21.15.157:0/4182634397 learned_addr learned my addr 172.21.15.157:0/4182634397 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:19.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.101+0000 7fd8753f0700 1 --2- 172.21.15.157:0/4182634397 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd870069950 0x7fd87014e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:50070/0 (socket says 172.21.15.157:50070) 2023-10-06T03:57:19.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.101+0000 7f32bd4b2700 1 Processor -- start 2023-10-06T03:57:19.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.101+0000 7fd875bf1700 1 --2- 172.21.15.157:0/4182634397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd87006c810 0x7fd870173700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.101+0000 7f32bd4b2700 1 -- start start 2023-10-06T03:57:19.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.101+0000 7f190affd700 1 -- 172.21.15.157:0/912029013 <== osd.4 v2:172.21.15.175:6800/91261 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f1910072370 con 0x7f191006f8b0 2023-10-06T03:57:19.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.101+0000 7f6f13fff700 1 --2- 172.21.15.157:0/59279779 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f1415afa0 0x7f6f1406e220 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009910 tx=0x7f6f040091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=218a34306aa84b93 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.101+0000 7fd874bef700 1 -- 172.21.15.157:0/4182634397 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd870069950 msgr2=0x7fd87014e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.101+0000 7fd874bef700 1 --2- 172.21.15.157:0/4182634397 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd870069950 0x7fd87014e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.101+0000 7fd874bef700 1 -- 172.21.15.157:0/4182634397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd87006c810 msgr2=0x7fd870173700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.101+0000 7fd874bef700 1 --2- 172.21.15.157:0/4182634397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd87006c810 0x7fd870173700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.101+0000 7fd874bef700 1 -- 172.21.15.157:0/4182634397 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd868007980 con 0x7fd87006c440 2023-10-06T03:57:19.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.101+0000 7f6f137fe700 1 -- 172.21.15.157:0/59279779 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0400b8f0 con 0x7f6f1415afa0 2023-10-06T03:57:19.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.101+0000 7f6f137fe700 1 -- 172.21.15.157:0/59279779 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0400ba50 con 0x7f6f1415afa0 2023-10-06T03:57:19.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.101+0000 7f6f137fe700 1 -- 172.21.15.157:0/59279779 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0400bdf0 con 0x7f6f1415afa0 2023-10-06T03:57:19.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.101+0000 7f53985b7700 1 -- 172.21.15.157:0/1520532365 shutdown_connections 2023-10-06T03:57:19.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.101+0000 7ff3597b3700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff35406c440 0x7ff35406e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.101+0000 7ff3597b3700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff35406c810 0x7ff354069410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.101+0000 7ff3597b3700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff35414cb50 con 0x7ff354069950 2023-10-06T03:57:19.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.101+0000 7ff3597b3700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3540c3a80 con 0x7ff35406c440 2023-10-06T03:57:19.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.101+0000 7ff3597b3700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3540c37a0 con 0x7ff35406c810 2023-10-06T03:57:19.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.102+0000 7ff352ffd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff35406c440 0x7ff35406e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.102+0000 7ff352ffd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff35406c440 0x7ff35406e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34344/0 (socket says 172.21.15.157:34344) 2023-10-06T03:57:19.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.102+0000 7ff352ffd700 1 -- 172.21.15.157:0/4086063271 learned_addr learned my addr 172.21.15.157:0/4086063271 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:19.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.102+0000 7ff3527fc700 1 --2- 172.21.15.157:0/4086063271 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff35406c810 0x7ff354069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.102+0000 7ff51550c700 1 Processor -- start 2023-10-06T03:57:19.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.102+0000 7f32bd4b2700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32b80c4840 0x7f32b81500e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.102+0000 7f32bd4b2700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b806e730 con 0x7f32b806aa40 2023-10-06T03:57:19.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.102+0000 7f32bd4b2700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b815a6c0 con 0x7f32b80c4470 2023-10-06T03:57:19.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.102+0000 7f32bd4b2700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b8159ab0 con 0x7f32b80c4840 2023-10-06T03:57:19.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.102+0000 7fd874bef700 1 --2- 172.21.15.157:0/4182634397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd87006c440 0x7fd8701731c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd8700c38e0 tx=0x7fd86801b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.102+0000 7f32b6ffd700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f32b80c4470 0x7f32b806df10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.157:38288/0 (socket says 172.21.15.157:38288) 2023-10-06T03:57:19.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.102+0000 7f32b6ffd700 1 -- 172.21.15.157:0/1018256112 learned_addr learned my addr 172.21.15.157:0/1018256112 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:19.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.102+0000 7f32ae7fc700 1 --2- 172.21.15.157:0/1018256112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32b80c4840 0x7f32b81500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.102+0000 7ff352ffd700 1 -- 172.21.15.157:0/4086063271 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff35406c810 msgr2=0x7ff354069410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:19.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.102+0000 7ff352ffd700 1 --2- 172.21.15.157:0/4086063271 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff35406c810 0x7ff354069410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.102+0000 7ff352ffd700 1 -- 172.21.15.157:0/4086063271 >> v1:172.21.15.157:6789/0 conn(0x7ff354069950 legacy=0x7ff3540c2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.102+0000 7ff352ffd700 1 -- 172.21.15.157:0/4086063271 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff354155b60 con 0x7ff35406c440 2023-10-06T03:57:19.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.103+0000 7f6f1ab13700 1 -- 172.21.15.157:0/59279779 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f1415afa0 msgr2=0x7f6f1406e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.103+0000 7f6f1ab13700 1 --2- 172.21.15.157:0/59279779 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f1415afa0 0x7f6f1406e220 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009910 tx=0x7f6f040091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.102+0000 7ff51550c700 1 -- start start 2023-10-06T03:57:19.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.103+0000 7f32b67fc700 1 -- 172.21.15.157:0/1018256112 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4073905520 0 0) 0x7f32b815a6c0 con 0x7f32b80c4470 2023-10-06T03:57:19.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.103+0000 7f32b67fc700 1 -- 172.21.15.157:0/1018256112 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32a4003570 con 0x7f32b80c4470 2023-10-06T03:57:19.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.102+0000 7fd86d7fa700 1 -- 172.21.15.157:0/4182634397 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd868020440 con 0x7fd87006c440 2023-10-06T03:57:19.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.103+0000 7fd86d7fa700 1 -- 172.21.15.157:0/4182634397 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8680205a0 con 0x7fd87006c440 2023-10-06T03:57:19.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.103+0000 7fd86d7fa700 1 -- 172.21.15.157:0/4182634397 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd868020960 con 0x7fd87006c440 2023-10-06T03:57:19.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.103+0000 7ff352ffd700 1 --2- 172.21.15.157:0/4086063271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff35406c440 0x7ff35406e2a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff3440074b0 tx=0x7ff34400ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64833298311483 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.101+0000 7f53985b7700 1 -- 172.21.15.157:0/1520532365 wait complete. 2023-10-06T03:57:19.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.103+0000 7f32b67fc700 1 -- 172.21.15.157:0/1018256112 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 236262942 0 0) 0x7f32a4003570 con 0x7f32b80c4470 2023-10-06T03:57:19.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.103+0000 7f53985b7700 1 Processor -- start 2023-10-06T03:57:19.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.103+0000 7ff51550c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5100bf520 0x7ff510140d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.103+0000 7ff51550c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5100bf8f0 0x7ff510141850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.103+0000 7ff51550c700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5101546f0 0x7ff510156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.103+0000 7ff51550c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51006cd00 con 0x7ff5100bf8f0 2023-10-06T03:57:19.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.103+0000 7ff51550c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff510157640 con 0x7ff5100bf520 2023-10-06T03:57:19.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.103+0000 7ff51550c700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5100c4d10 con 0x7ff5101546f0 2023-10-06T03:57:19.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.103+0000 7f32b67fc700 1 -- 172.21.15.157:0/1018256112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32b80c4840 msgr2=0x7f32b81500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:19.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.103+0000 7f32b67fc700 1 --2- 172.21.15.157:0/1018256112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32b80c4840 0x7f32b81500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.103+0000 7f32b67fc700 1 -- 172.21.15.157:0/1018256112 >> v1:172.21.15.157:6789/0 conn(0x7f32b806aa40 legacy=0x7f32b81564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.103+0000 7f32b67fc700 1 -- 172.21.15.157:0/1018256112 --> v1:172.21.15.175:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b8159d90 con 0x7f32b80c4470 2023-10-06T03:57:19.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.103+0000 7f32b67fc700 1 -- 172.21.15.157:0/1018256112 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f32a0001470 con 0x7f32b80c4470 2023-10-06T03:57:19.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.103+0000 7f32b67fc700 1 -- 172.21.15.157:0/1018256112 >> v1:172.21.15.175:6789/0 conn(0x7f32b80c4470 legacy=0x7f32b806df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.103+0000 7f32b67fc700 1 --2- 172.21.15.157:0/1018256112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32a4003a60 0x7f32a4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.103+0000 7f32b67fc700 1 --2- 172.21.15.157:0/1018256112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32a4004810 0x7f32a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.103+0000 7f32b67fc700 1 --2- 172.21.15.157:0/1018256112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32a40051c0 0x7f32a4014270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.103+0000 7f32b67fc700 1 -- 172.21.15.157:0/1018256112 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b8159ab0 con 0x7f32a40051c0 2023-10-06T03:57:19.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.103+0000 7f32b67fc700 1 -- 172.21.15.157:0/1018256112 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a4014860 con 0x7f32a4004810 2023-10-06T03:57:19.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.103+0000 7f32b67fc700 1 -- 172.21.15.157:0/1018256112 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a4014a20 con 0x7f32a4003a60 2023-10-06T03:57:19.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7ff50e7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5100bf8f0 0x7ff510141850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7ff50e7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5100bf8f0 0x7ff510141850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34364/0 (socket says 172.21.15.157:34364) 2023-10-06T03:57:19.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7f32b77fe700 1 --2- 172.21.15.157:0/1018256112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32a40051c0 0x7f32a4014270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7ff50e7fc700 1 -- 172.21.15.157:0/3261109047 learned_addr learned my addr 172.21.15.157:0/3261109047 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:19.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7ff50f7fe700 1 --2- 172.21.15.157:0/3261109047 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5101546f0 0x7ff510156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.103+0000 7f53985b7700 1 -- start start 2023-10-06T03:57:19.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7ff50effd700 1 --2- 172.21.15.157:0/3261109047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5100bf520 0x7ff510140d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7f32b6ffd700 1 --2- 172.21.15.157:0/1018256112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32a4003a60 0x7f32a4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7ff351ffb700 1 -- 172.21.15.157:0/4086063271 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34400f070 con 0x7ff35406c440 2023-10-06T03:57:19.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7ff351ffb700 1 -- 172.21.15.157:0/4086063271 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff344008bf0 con 0x7ff35406c440 2023-10-06T03:57:19.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7ff351ffb700 1 -- 172.21.15.157:0/4086063271 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34400a0d0 con 0x7ff35406c440 2023-10-06T03:57:19.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7ff50e7fc700 1 -- 172.21.15.157:0/3261109047 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5101546f0 msgr2=0x7ff510156a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7ff50e7fc700 1 --2- 172.21.15.157:0/3261109047 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5101546f0 0x7ff510156a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7ff50e7fc700 1 -- 172.21.15.157:0/3261109047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5100bf520 msgr2=0x7ff510140d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7ff50e7fc700 1 --2- 172.21.15.157:0/3261109047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5100bf520 0x7ff510140d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7ff50e7fc700 1 -- 172.21.15.157:0/3261109047 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5040094f0 con 0x7ff5100bf8f0 2023-10-06T03:57:19.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7f32b77fe700 1 -- 172.21.15.157:0/1018256112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32a4003a60 msgr2=0x7f32a4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:19.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7f32b77fe700 1 --2- 172.21.15.157:0/1018256112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32a4003a60 0x7f32a4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7f32b77fe700 1 -- 172.21.15.157:0/1018256112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32a4004810 msgr2=0x7f32a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:19.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7f32b77fe700 1 --2- 172.21.15.157:0/1018256112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32a4004810 0x7f32a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7f32b77fe700 1 -- 172.21.15.157:0/1018256112 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a4003570 con 0x7f32a40051c0 2023-10-06T03:57:19.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7ff50e7fc700 1 --2- 172.21.15.157:0/3261109047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5100bf8f0 0x7ff510141850 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff50800bbb0 tx=0x7ff508009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7f32b77fe700 1 --2- 172.21.15.157:0/1018256112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32a40051c0 0x7f32a4014270 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f32b00018a0 tx=0x7f32b0007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892e0740350630c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7ff4f6ffd700 1 -- 172.21.15.157:0/3261109047 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5080092e0 con 0x7ff5100bf8f0 2023-10-06T03:57:19.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.105+0000 7ff4f6ffd700 1 -- 172.21.15.157:0/3261109047 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50800c6f0 con 0x7ff5100bf8f0 2023-10-06T03:57:19.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.105+0000 7ff4f6ffd700 1 -- 172.21.15.157:0/3261109047 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50800ca90 con 0x7ff5100bf8f0 2023-10-06T03:57:19.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.105+0000 7ff51550c700 1 -- 172.21.15.157:0/3261109047 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff51006ca60 con 0x7ff5100bf8f0 2023-10-06T03:57:19.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.105+0000 7f32b67fc700 1 -- 172.21.15.157:0/1018256112 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b0012400 con 0x7f32a40051c0 2023-10-06T03:57:19.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.105+0000 7f32b67fc700 1 -- 172.21.15.157:0/1018256112 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b0012560 con 0x7f32a40051c0 2023-10-06T03:57:19.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.105+0000 7f32b67fc700 1 -- 172.21.15.157:0/1018256112 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b0012900 con 0x7f32a40051c0 2023-10-06T03:57:19.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.105+0000 7ff51550c700 1 -- 172.21.15.157:0/3261109047 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff51006d3c0 con 0x7ff5100bf8f0 2023-10-06T03:57:19.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.105+0000 7ff3597b3700 1 -- 172.21.15.157:0/4086063271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff35406c440 msgr2=0x7ff35406e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.105+0000 7ff3597b3700 1 --2- 172.21.15.157:0/4086063271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff35406c440 0x7ff35406e2a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff3440074b0 tx=0x7ff34400ac80 comp rx=0 tx=0).stop 2023-10-06T03:57:19.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7f53985b7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f539006c1f0 0x7f5390159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7f53985b7700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53900adca0 0x7f53901566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7f53985b7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53900ae070 0x7f5390157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7f53985b7700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539015d6a0 con 0x7f53900ae070 2023-10-06T03:57:19.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7f53985b7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5390155db0 con 0x7f539006c1f0 2023-10-06T03:57:19.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7f53985b7700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f539006e240 con 0x7f53900adca0 2023-10-06T03:57:19.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.104+0000 7f5396353700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f539006c1f0 0x7f5390159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.105+0000 7f5396353700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f539006c1f0 0x7f5390159b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:35092/0 (socket says 172.21.15.157:35092) 2023-10-06T03:57:19.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.105+0000 7f5396353700 1 -- 172.21.15.157:0/64787291 learned_addr learned my addr 172.21.15.157:0/64787291 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:19.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.105+0000 7f5395b52700 1 --2- 172.21.15.157:0/64787291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53900adca0 0x7f53901566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.105+0000 7f5396b54700 1 --2- 172.21.15.157:0/64787291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53900ae070 0x7f5390157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.105+0000 7f5395b52700 1 -- 172.21.15.157:0/64787291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f539006c1f0 msgr2=0x7f5390159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.105+0000 7f5395b52700 1 --2- 172.21.15.157:0/64787291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f539006c1f0 0x7f5390159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.105+0000 7f5395b52700 1 -- 172.21.15.157:0/64787291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53900ae070 msgr2=0x7f5390157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.105+0000 7f5395b52700 1 --2- 172.21.15.157:0/64787291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53900ae070 0x7f5390157dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.105+0000 7f5395b52700 1 -- 172.21.15.157:0/64787291 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538c009420 con 0x7f53900adca0 2023-10-06T03:57:19.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.105+0000 7ff3597b3700 1 -- 172.21.15.157:0/4086063271 shutdown_connections 2023-10-06T03:57:19.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.105+0000 7ff3597b3700 1 --2- 172.21.15.157:0/4086063271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff35406c440 0x7ff35406e2a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.105+0000 7ff3597b3700 1 --2- 172.21.15.157:0/4086063271 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff35406c810 0x7ff354069410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.105+0000 7ff3597b3700 1 -- 172.21.15.157:0/4086063271 >> 172.21.15.157:0/4086063271 conn(0x7ff3540c82e0 msgr2=0x7ff3540c86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.105+0000 7ff3597b3700 1 -- 172.21.15.157:0/4086063271 shutdown_connections 2023-10-06T03:57:19.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.105+0000 7ff3597b3700 1 -- 172.21.15.157:0/4086063271 wait complete. 2023-10-06T03:57:19.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.105+0000 7f6f1ab13700 1 -- 172.21.15.157:0/59279779 shutdown_connections 2023-10-06T03:57:19.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.106+0000 7f6f1ab13700 1 --2- 172.21.15.157:0/59279779 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f1415afa0 0x7f6f1406e220 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.106+0000 7f6f1ab13700 1 -- 172.21.15.157:0/59279779 >> 172.21.15.157:0/59279779 conn(0x7f6f140cc6e0 msgr2=0x7f6f140ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.106+0000 7fd877654700 1 -- 172.21.15.157:0/4182634397 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd870152580 con 0x7fd87006c440 2023-10-06T03:57:19.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.106+0000 7fd877654700 1 -- 172.21.15.157:0/4182634397 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd870152b40 con 0x7fd87006c440 2023-10-06T03:57:19.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.106+0000 7ff51550c700 1 -- 172.21.15.157:0/3261109047 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff5100642a0 con 0x7ff5100bf8f0 2023-10-06T03:57:19.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.106+0000 7f32bd4b2700 1 -- 172.21.15.157:0/1018256112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32a40051c0 msgr2=0x7f32a4014270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.106+0000 7f32bd4b2700 1 --2- 172.21.15.157:0/1018256112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32a40051c0 0x7f32a4014270 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f32b00018a0 tx=0x7f32b0007cf0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.106+0000 7f32b77fe700 1 -- 172.21.15.157:0/1018256112 reap_dead start 2023-10-06T03:57:19.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.106+0000 7f5395b52700 1 --2- 172.21.15.157:0/64787291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53900adca0 0x7f53901566f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f538c0093f0 tx=0x7f538c016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.106+0000 7f6f1ab13700 1 -- 172.21.15.157:0/59279779 shutdown_connections 2023-10-06T03:57:19.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.106+0000 7f32bd4b2700 1 -- 172.21.15.157:0/1018256112 shutdown_connections 2023-10-06T03:57:19.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.106+0000 7f32bd4b2700 1 -- 172.21.15.157:0/1018256112 >> 172.21.15.157:0/1018256112 conn(0x7f32b80c82e0 msgr2=0x7f32b80c86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.107+0000 7fd877654700 1 -- 172.21.15.157:0/4182634397 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd870172e00 con 0x7fd87006c440 2023-10-06T03:57:19.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.106+0000 7f6f1ab13700 1 -- 172.21.15.157:0/59279779 wait complete. 2023-10-06T03:57:19.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.106+0000 7f53867fc700 1 -- 172.21.15.157:0/64787291 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c00bac0 con 0x7f53900adca0 2023-10-06T03:57:19.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.106+0000 7f53867fc700 1 -- 172.21.15.157:0/64787291 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f538c020070 con 0x7f53900adca0 2023-10-06T03:57:19.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.106+0000 7f53867fc700 1 -- 172.21.15.157:0/64787291 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c01b6a0 con 0x7f53900adca0 2023-10-06T03:57:19.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.106+0000 7f53985b7700 1 -- 172.21.15.157:0/64787291 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f539015a440 con 0x7f53900adca0 2023-10-06T03:57:19.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.106+0000 7f53985b7700 1 -- 172.21.15.157:0/64787291 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5390155480 con 0x7f53900adca0 2023-10-06T03:57:19.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.107+0000 7ff3597b3700 1 Processor -- start 2023-10-06T03:57:19.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.107+0000 7ff3597b3700 1 -- start start 2023-10-06T03:57:19.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.107+0000 7ff4f6ffd700 1 -- 172.21.15.157:0/3261109047 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff508012070 con 0x7ff5100bf8f0 2023-10-06T03:57:19.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.108+0000 7ff4f6ffd700 1 --2- 172.21.15.157:0/3261109047 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff4fc066f30 0x7ff4fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.108+0000 7ff4f6ffd700 1 -- 172.21.15.157:0/3261109047 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff5080844f0 con 0x7ff5100bf8f0 2023-10-06T03:57:19.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.108+0000 7f6f1ab13700 1 Processor -- start 2023-10-06T03:57:19.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.108+0000 7ff4f6ffd700 1 --2- 172.21.15.157:0/3261109047 >> [v2:172.21.15.157:6824/90657,v1:172.21.15.157:6825/90657] conn(0x7ff4fc06f7e0 0x7ff4fc071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.108+0000 7ff4f6ffd700 1 -- 172.21.15.157:0/3261109047 --> [v2:172.21.15.157:6824/90657,v1:172.21.15.157:6825/90657] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff4fc0722a0 con 0x7ff4fc06f7e0 2023-10-06T03:57:19.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.108+0000 7ff4f6ffd700 1 -- 172.21.15.157:0/3261109047 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5080848c0 con 0x7ff5100bf8f0 2023-10-06T03:57:19.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.108+0000 7ff3597b3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff354069950 0x7ff35414d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.108+0000 7ff3597b3700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff35406c440 0x7ff3540abd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.108+0000 7ff50f7fe700 1 --2- 172.21.15.157:0/3261109047 >> [v2:172.21.15.157:6824/90657,v1:172.21.15.157:6825/90657] conn(0x7ff4fc06f7e0 0x7ff4fc071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.108+0000 7ff3597b3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff35406c810 0x7ff3540ac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.108+0000 7ff3597b3700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35414d330 con 0x7ff35406c810 2023-10-06T03:57:19.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.108+0000 7ff3597b3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff354151b30 con 0x7ff354069950 2023-10-06T03:57:19.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.108+0000 7ff3597b3700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff35414cd00 con 0x7ff35406c440 2023-10-06T03:57:19.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.108+0000 7f6f1ab13700 1 -- start start 2023-10-06T03:57:19.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.109+0000 7ff50f7fe700 1 --2- 172.21.15.157:0/3261109047 >> [v2:172.21.15.157:6824/90657,v1:172.21.15.157:6825/90657] conn(0x7ff4fc06f7e0 0x7ff4fc071bf0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.109+0000 7fd86d7fa700 1 -- 172.21.15.157:0/4182634397 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd868003040 con 0x7fd87006c440 2023-10-06T03:57:19.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.108+0000 7f53985b7700 1 -- 172.21.15.157:0/64787291 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f53900716b0 con 0x7f53900adca0 2023-10-06T03:57:19.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.109+0000 7ff50effd700 1 --2- 172.21.15.157:0/3261109047 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff4fc066f30 0x7ff4fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.109+0000 7f6f1ab13700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f140b1e80 0x7f6f1406b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.109+0000 7f32bd4b2700 1 -- 172.21.15.157:0/1018256112 shutdown_connections 2023-10-06T03:57:19.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.109+0000 7f32bd4b2700 1 -- 172.21.15.157:0/1018256112 wait complete. 2023-10-06T03:57:19.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.109+0000 7f6f1ab13700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f140b2250 0x7f6f1416dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.109+0000 7f6f1ab13700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f1415afa0 0x7f6f140694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.109+0000 7f6f1ab13700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1416ec20 con 0x7f6f140b1e80 2023-10-06T03:57:19.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.109+0000 7f6f1ab13700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f140af3e0 con 0x7f6f1415afa0 2023-10-06T03:57:19.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.109+0000 7f6f1ab13700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f14067bc0 con 0x7f6f140b2250 2023-10-06T03:57:19.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.109+0000 7fd86d7fa700 1 --2- 172.21.15.157:0/4182634397 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd85c067030 0x7fd85c0694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.109+0000 7f6f190b0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f1415afa0 0x7f6f140694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.109+0000 7fd86d7fa700 1 -- 172.21.15.157:0/4182634397 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd868086030 con 0x7fd87006c440 2023-10-06T03:57:19.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.109+0000 7f6f190b0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f1415afa0 0x7f6f140694e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:35104/0 (socket says 172.21.15.157:35104) 2023-10-06T03:57:19.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.109+0000 7f6f190b0700 1 -- 172.21.15.157:0/2146913966 learned_addr learned my addr 172.21.15.157:0/2146913966 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:19.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.110+0000 7fd86d7fa700 1 --2- 172.21.15.157:0/4182634397 >> [v2:172.21.15.157:6808/90640,v1:172.21.15.157:6809/90640] conn(0x7fd85c06f8e0 0x7fd85c071cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.110+0000 7fd86d7fa700 1 -- 172.21.15.157:0/4182634397 --> [v2:172.21.15.157:6808/90640,v1:172.21.15.157:6809/90640] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd85c0723a0 con 0x7fd85c06f8e0 2023-10-06T03:57:19.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.110+0000 7fd86d7fa700 1 -- 172.21.15.157:0/4182634397 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd868088610 con 0x7fd87006c440 2023-10-06T03:57:19.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.110+0000 7fd875bf1700 1 --2- 172.21.15.157:0/4182634397 >> [v2:172.21.15.157:6808/90640,v1:172.21.15.157:6809/90640] conn(0x7fd85c06f8e0 0x7fd85c071cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.110+0000 7fd875bf1700 1 --2- 172.21.15.157:0/4182634397 >> [v2:172.21.15.157:6808/90640,v1:172.21.15.157:6809/90640] conn(0x7fd85c06f8e0 0x7fd85c071cf0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.110+0000 7f6f190b0700 1 -- 172.21.15.157:0/2146913966 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f140b2250 msgr2=0x7f6f1416dfc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T03:57:19.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.110+0000 7f6f190b0700 1 --2- 172.21.15.157:0/2146913966 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f140b2250 0x7f6f1416dfc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.110+0000 7f6f190b0700 1 -- 172.21.15.157:0/2146913966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f140b1e80 msgr2=0x7f6f1406b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.111+0000 7f6f190b0700 1 --2- 172.21.15.157:0/2146913966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f140b1e80 0x7f6f1406b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.111+0000 7f6f190b0700 1 -- 172.21.15.157:0/2146913966 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f04009420 con 0x7f6f1415afa0 2023-10-06T03:57:19.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.108+0000 7ff3527fc700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff35406c440 0x7ff3540abd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.108+0000 7ff3527fc700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff35406c440 0x7ff3540abd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:50102/0 (socket says 172.21.15.157:50102) 2023-10-06T03:57:19.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.108+0000 7ff3527fc700 1 -- 172.21.15.157:0/651892751 learned_addr learned my addr 172.21.15.157:0/651892751 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:19.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.109+0000 7ff352ffd700 1 --2- 172.21.15.157:0/651892751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff354069950 0x7ff35414d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.109+0000 7ff352ffd700 1 -- 172.21.15.157:0/651892751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff35406c440 msgr2=0x7ff3540abd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.109+0000 7ff352ffd700 1 --2- 172.21.15.157:0/651892751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff35406c440 0x7ff3540abd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.109+0000 7ff352ffd700 1 -- 172.21.15.157:0/651892751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff35406c810 msgr2=0x7ff3540ac2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.109+0000 7ff352ffd700 1 --2- 172.21.15.157:0/651892751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff35406c810 0x7ff3540ac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.109+0000 7ff352ffd700 1 -- 172.21.15.157:0/651892751 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff344007040 con 0x7ff354069950 2023-10-06T03:57:19.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.110+0000 7ff352ffd700 1 --2- 172.21.15.157:0/651892751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff354069950 0x7ff35414d9f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff344007480 tx=0x7ff344019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.110+0000 7ff33affd700 1 -- 172.21.15.157:0/651892751 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3440078d0 con 0x7ff354069950 2023-10-06T03:57:19.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.110+0000 7ff33affd700 1 -- 172.21.15.157:0/651892751 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff344007a30 con 0x7ff354069950 2023-10-06T03:57:19.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.110+0000 7ff33affd700 1 -- 172.21.15.157:0/651892751 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff344007df0 con 0x7ff354069950 2023-10-06T03:57:19.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.110+0000 7fd8753f0700 1 --2- 172.21.15.157:0/4182634397 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd85c067030 0x7fd85c0694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.111+0000 7f6f190b0700 1 --2- 172.21.15.157:0/2146913966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f1415afa0 0x7f6f140694e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6f0800c560 tx=0x7f6f08007ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.111+0000 7ff3597b3700 1 -- 172.21.15.157:0/651892751 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff354173ba0 con 0x7ff354069950 2023-10-06T03:57:19.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.111+0000 7ff3597b3700 1 -- 172.21.15.157:0/651892751 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff35414cf50 con 0x7ff354069950 2023-10-06T03:57:19.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.111+0000 7ff4f6ffd700 1 -- 172.21.15.157:0/3261109047 <== osd.2 v2:172.21.15.157:6824/90657 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7ff4fc0722a0 con 0x7ff4fc06f7e0 2023-10-06T03:57:19.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.111+0000 7f32bd4b2700 1 Processor -- start 2023-10-06T03:57:19.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.111+0000 7f6f10ff9700 1 -- 172.21.15.157:0/2146913966 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f08011590 con 0x7f6f1415afa0 2023-10-06T03:57:19.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.111+0000 7f32bd4b2700 1 -- start start 2023-10-06T03:57:19.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.111+0000 7f6f10ff9700 1 -- 172.21.15.157:0/2146913966 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f08011710 con 0x7f6f1415afa0 2023-10-06T03:57:19.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.111+0000 7f6f10ff9700 1 -- 172.21.15.157:0/2146913966 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f08011ad0 con 0x7f6f1415afa0 2023-10-06T03:57:19.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.111+0000 7f6f1ab13700 1 -- 172.21.15.157:0/2146913966 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f140af680 con 0x7f6f1415afa0 2023-10-06T03:57:19.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.112+0000 7ff50effd700 1 --2- 172.21.15.157:0/3261109047 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff4fc066f30 0x7ff4fc0693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff500006fd0 tx=0x7ff5000090f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.112+0000 7f6f1ab13700 1 -- 172.21.15.157:0/2146913966 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f14067e60 con 0x7f6f1415afa0 2023-10-06T03:57:19.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.111+0000 7f53867fc700 1 -- 172.21.15.157:0/64787291 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f538c022030 con 0x7f53900adca0 2023-10-06T03:57:19.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.111+0000 7f53867fc700 1 --2- 172.21.15.157:0/64787291 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5374067080 0x7f5374069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.112+0000 7f53867fc700 1 -- 172.21.15.157:0/64787291 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f538c0863b0 con 0x7f53900adca0 2023-10-06T03:57:19.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.112+0000 7f53867fc700 1 --2- 172.21.15.157:0/64787291 >> [v2:172.21.15.157:6800/90654,v1:172.21.15.157:6801/90654] conn(0x7f537406f930 0x7f5374071d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.112+0000 7f53867fc700 1 -- 172.21.15.157:0/64787291 --> [v2:172.21.15.157:6800/90654,v1:172.21.15.157:6801/90654] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f53740723f0 con 0x7f537406f930 2023-10-06T03:57:19.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.112+0000 7f53867fc700 1 -- 172.21.15.157:0/64787291 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f538c088950 con 0x7f53900adca0 2023-10-06T03:57:19.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.112+0000 7f5396353700 1 --2- 172.21.15.157:0/64787291 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5374067080 0x7f5374069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.112+0000 7ff3597b3700 1 -- 172.21.15.157:0/651892751 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff3540716b0 con 0x7ff354069950 2023-10-06T03:57:19.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.112+0000 7f5396353700 1 --2- 172.21.15.157:0/64787291 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5374067080 0x7f5374069530 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5388006f40 tx=0x7f5388005480 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.112+0000 7f32bd4b2700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32a40051c0 0x7f32b8146eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.112+0000 7f32bd4b2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32b80bd130 0x7f32b80bec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.112+0000 7f32bd4b2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b8158aa0 0x7f32b81449f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.112+0000 7f32bd4b2700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b80c80f0 con 0x7f32b8158aa0 2023-10-06T03:57:19.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.112+0000 7f32bd4b2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b80be7a0 con 0x7f32b80bd130 2023-10-06T03:57:19.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.112+0000 7f32bd4b2700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b80c7e30 con 0x7f32a40051c0 2023-10-06T03:57:19.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.113+0000 7f32b77fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b8158aa0 0x7f32b81449f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.113+0000 7f32b77fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b8158aa0 0x7f32b81449f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34396/0 (socket says 172.21.15.157:34396) 2023-10-06T03:57:19.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.113+0000 7f32b77fe700 1 -- 172.21.15.157:0/2161114439 learned_addr learned my addr 172.21.15.157:0/2161114439 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:19.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.113+0000 7f5396b54700 1 --2- 172.21.15.157:0/64787291 >> [v2:172.21.15.157:6800/90654,v1:172.21.15.157:6801/90654] conn(0x7f537406f930 0x7f5374071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.113+0000 7f32ae7fc700 1 --2- 172.21.15.157:0/2161114439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32b80bd130 0x7f32b80bec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.113+0000 7f32b77fe700 1 -- 172.21.15.157:0/2161114439 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32a40051c0 msgr2=0x7f32b8146eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.113+0000 7f32b77fe700 1 --2- 172.21.15.157:0/2161114439 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32a40051c0 0x7f32b8146eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.113+0000 7f32b77fe700 1 -- 172.21.15.157:0/2161114439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32b80bd130 msgr2=0x7f32b80bec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.113+0000 7f32b77fe700 1 --2- 172.21.15.157:0/2161114439 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32b80bd130 0x7f32b80bec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.113+0000 7f32b77fe700 1 -- 172.21.15.157:0/2161114439 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b80bd750 con 0x7f32b8158aa0 2023-10-06T03:57:19.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.113+0000 7f32b77fe700 1 --2- 172.21.15.157:0/2161114439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b8158aa0 0x7f32b81449f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f32b0011000 tx=0x7f32b0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.113+0000 7f6f10ff9700 1 -- 172.21.15.157:0/2146913966 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6f08012420 con 0x7f6f1415afa0 2023-10-06T03:57:19.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.112+0000 7fd8753f0700 1 --2- 172.21.15.157:0/4182634397 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd85c067030 0x7fd85c0694e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd864006fd0 tx=0x7fd864008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.113+0000 7fd86d7fa700 1 -- 172.21.15.157:0/4182634397 <== osd.0 v2:172.21.15.157:6808/90640 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fd85c0723a0 con 0x7fd85c06f8e0 2023-10-06T03:57:19.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.113+0000 7f6f1ab13700 1 -- 172.21.15.157:0/2146913966 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6f1406cbf0 con 0x7f6f1415afa0 2023-10-06T03:57:19.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.114+0000 7f6f10ff9700 1 --2- 172.21.15.157:0/2146913966 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6f00067000 0x7f6f000694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.114+0000 7f6f10ff9700 1 -- 172.21.15.157:0/2146913966 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6f08083c00 con 0x7f6f1415afa0 2023-10-06T03:57:19.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.114+0000 7f6f188af700 1 --2- 172.21.15.157:0/2146913966 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6f00067000 0x7f6f000694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.114+0000 7f6f10ff9700 1 --2- 172.21.15.157:0/2146913966 >> [v2:172.21.15.157:6816/90641,v1:172.21.15.157:6817/90641] conn(0x7f6f0006f8b0 0x7f6f00071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.114+0000 7f6f10ff9700 1 -- 172.21.15.157:0/2146913966 --> [v2:172.21.15.157:6816/90641,v1:172.21.15.157:6817/90641] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6f00072370 con 0x7f6f0006f8b0 2023-10-06T03:57:19.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.114+0000 7f6f13fff700 1 --2- 172.21.15.157:0/2146913966 >> [v2:172.21.15.157:6816/90641,v1:172.21.15.157:6817/90641] conn(0x7f6f0006f8b0 0x7f6f00071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.114+0000 7f6f10ff9700 1 -- 172.21.15.157:0/2146913966 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f08086240 con 0x7f6f1415afa0 2023-10-06T03:57:19.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.114+0000 7ff33affd700 1 -- 172.21.15.157:0/651892751 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff344008250 con 0x7ff354069950 2023-10-06T03:57:19.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.114+0000 7ff33affd700 1 --2- 172.21.15.157:0/651892751 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff340066fb0 0x7ff340069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.115+0000 7ff33affd700 1 -- 172.21.15.157:0/651892751 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff3440855a0 con 0x7ff354069950 2023-10-06T03:57:19.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.115+0000 7ff3527fc700 1 --2- 172.21.15.157:0/651892751 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff340066fb0 0x7ff340069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.115+0000 7ff33affd700 1 --2- 172.21.15.157:0/651892751 >> [v2:172.21.15.175:6824/91306,v1:172.21.15.175:6825/91306] conn(0x7ff34006f7e0 0x7ff340071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.115+0000 7ff33affd700 1 -- 172.21.15.157:0/651892751 --> [v2:172.21.15.175:6824/91306,v1:172.21.15.175:6825/91306] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff3400722a0 con 0x7ff34006f7e0 2023-10-06T03:57:19.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.115+0000 7f6f13fff700 1 --2- 172.21.15.157:0/2146913966 >> [v2:172.21.15.157:6816/90641,v1:172.21.15.157:6817/90641] conn(0x7f6f0006f8b0 0x7f6f00071cc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.115+0000 7ff33affd700 1 -- 172.21.15.157:0/651892751 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff344052610 con 0x7ff354069950 2023-10-06T03:57:19.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.115+0000 7f6f188af700 1 --2- 172.21.15.157:0/2146913966 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6f00067000 0x7f6f000694b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c0020e0 tx=0x7f6f0c001690 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.115+0000 7ff3537fe700 1 --2- 172.21.15.157:0/651892751 >> [v2:172.21.15.175:6824/91306,v1:172.21.15.175:6825/91306] conn(0x7ff34006f7e0 0x7ff340071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.116+0000 7ff3537fe700 1 --2- 172.21.15.157:0/651892751 >> [v2:172.21.15.175:6824/91306,v1:172.21.15.175:6825/91306] conn(0x7ff34006f7e0 0x7ff340071bf0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.116+0000 7f6f10ff9700 1 -- 172.21.15.157:0/2146913966 <== osd.3 v2:172.21.15.157:6816/90641 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f6f00072370 con 0x7f6f0006f8b0 2023-10-06T03:57:19.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.116+0000 7ff3527fc700 1 --2- 172.21.15.157:0/651892751 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff340066fb0 0x7ff340069460 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff354173ad0 tx=0x7ff3480091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.116+0000 7ff33affd700 1 -- 172.21.15.157:0/651892751 <== osd.7 v2:172.21.15.175:6824/91306 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7ff3400722a0 con 0x7ff34006f7e0 2023-10-06T03:57:19.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.114+0000 7f32af7fe700 1 -- 172.21.15.157:0/2161114439 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b0012400 con 0x7f32b8158aa0 2023-10-06T03:57:19.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.114+0000 7f32af7fe700 1 -- 172.21.15.157:0/2161114439 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b00128e0 con 0x7f32b8158aa0 2023-10-06T03:57:19.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.114+0000 7f32af7fe700 1 -- 172.21.15.157:0/2161114439 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b001e640 con 0x7f32b8158aa0 2023-10-06T03:57:19.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.115+0000 7f32bd4b2700 1 -- 172.21.15.157:0/2161114439 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b80bda30 con 0x7f32b8158aa0 2023-10-06T03:57:19.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.115+0000 7f32bd4b2700 1 -- 172.21.15.157:0/2161114439 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b80bdd10 con 0x7f32b8158aa0 2023-10-06T03:57:19.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.116+0000 7f32bd4b2700 1 -- 172.21.15.157:0/2161114439 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f32b80716b0 con 0x7f32b8158aa0 2023-10-06T03:57:19.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.117+0000 7f5396b54700 1 --2- 172.21.15.157:0/64787291 >> [v2:172.21.15.157:6800/90654,v1:172.21.15.157:6801/90654] conn(0x7f537406f930 0x7f5374071d40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.117+0000 7f32af7fe700 1 -- 172.21.15.157:0/2161114439 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f32b001e9b0 con 0x7f32b8158aa0 2023-10-06T03:57:19.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.118+0000 7f32af7fe700 1 --2- 172.21.15.157:0/2161114439 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3298067000 0x7f32980694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.118+0000 7f32af7fe700 1 -- 172.21.15.157:0/2161114439 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f32b00867f0 con 0x7f32b8158aa0 2023-10-06T03:57:19.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.118+0000 7f32b6ffd700 1 --2- 172.21.15.157:0/2161114439 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3298067000 0x7f32980694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.118+0000 7f32af7fe700 1 --2- 172.21.15.157:0/2161114439 >> [v2:172.21.15.175:6802/91263,v1:172.21.15.175:6801/91263] conn(0x7f329806f8b0 0x7f3298071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.118+0000 7f32af7fe700 1 -- 172.21.15.157:0/2161114439 --> [v2:172.21.15.175:6802/91263,v1:172.21.15.175:6801/91263] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3298072370 con 0x7f329806f8b0 2023-10-06T03:57:19.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.118+0000 7f32af7fe700 1 -- 172.21.15.157:0/2161114439 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32b0086bc0 con 0x7f32b8158aa0 2023-10-06T03:57:19.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.118+0000 7f32ae7fc700 1 --2- 172.21.15.157:0/2161114439 >> [v2:172.21.15.175:6802/91263,v1:172.21.15.175:6801/91263] conn(0x7f329806f8b0 0x7f3298071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.118+0000 7f32b6ffd700 1 --2- 172.21.15.157:0/2161114439 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3298067000 0x7f32980694b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f32a00010e0 tx=0x7f32a0001da0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.118+0000 7f32ae7fc700 1 --2- 172.21.15.157:0/2161114439 >> [v2:172.21.15.175:6802/91263,v1:172.21.15.175:6801/91263] conn(0x7f329806f8b0 0x7f3298071cc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.119+0000 7f32af7fe700 1 -- 172.21.15.157:0/2161114439 <== osd.6 v2:172.21.15.175:6802/91263 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f3298072370 con 0x7f329806f8b0 2023-10-06T03:57:19.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.118+0000 7f53867fc700 1 -- 172.21.15.157:0/64787291 <== osd.1 v2:172.21.15.157:6800/90654 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f53740723f0 con 0x7f537406f930 2023-10-06T03:57:19.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.122+0000 7fade52b7700 1 Processor -- start 2023-10-06T03:57:19.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.122+0000 7fade52b7700 1 -- start start 2023-10-06T03:57:19.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.123+0000 7fade52b7700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fade006ab50 0x7fade006d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.123+0000 7fade52b7700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade00c6160 con 0x7fade006ddf0 2023-10-06T03:57:19.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.123+0000 7fade52b7700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade0069040 con 0x7fade006a780 2023-10-06T03:57:19.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.123+0000 7fade52b7700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade00c5c00 con 0x7fade006ab50 2023-10-06T03:57:19.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.123+0000 7faddf7fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fade006ddf0 0x7fade006e230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40012/0 (socket says 172.21.15.157:40012) 2023-10-06T03:57:19.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.123+0000 7faddf7fe700 1 -- 172.21.15.157:0/1358319931 learned_addr learned my addr 172.21.15.157:0/1358319931 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:19.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.124+0000 7fadddffb700 1 -- 172.21.15.157:0/1358319931 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 384642071 0 0) 0x7fade00c6160 con 0x7fade006ddf0 2023-10-06T03:57:19.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.124+0000 7fadddffb700 1 -- 172.21.15.157:0/1358319931 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadc8003570 con 0x7fade006ddf0 2023-10-06T03:57:19.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.124+0000 7fadddffb700 1 -- 172.21.15.157:0/1358319931 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 613477912 0 0) 0x7fadc8003570 con 0x7fade006ddf0 2023-10-06T03:57:19.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.124+0000 7fadddffb700 1 -- 172.21.15.157:0/1358319931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fade006ab50 msgr2=0x7fade006d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:19.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.124+0000 7fadddffb700 1 --2- 172.21.15.157:0/1358319931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fade006ab50 0x7fade006d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.124+0000 7fadddffb700 1 -- 172.21.15.157:0/1358319931 >> v1:172.21.15.175:6789/0 conn(0x7fade006a780 legacy=0x7fade00c65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:19.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.124+0000 7fadddffb700 1 -- 172.21.15.157:0/1358319931 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade0153930 con 0x7fade006ddf0 2023-10-06T03:57:19.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.124+0000 7fadddffb700 1 -- 172.21.15.157:0/1358319931 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7fadd80017f0 con 0x7fade006ddf0 2023-10-06T03:57:19.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.124+0000 7fadddffb700 1 -- 172.21.15.157:0/1358319931 >> v1:172.21.15.157:6789/0 conn(0x7fade006ddf0 legacy=0x7fade006e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.124+0000 7fadddffb700 1 --2- 172.21.15.157:0/1358319931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fadc8003b10 0x7fadc8003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.124+0000 7fadddffb700 1 --2- 172.21.15.157:0/1358319931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fadc8004810 0x7fadc8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.124+0000 7fadddffb700 1 --2- 172.21.15.157:0/1358319931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadc80053c0 0x7fadc800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.124+0000 7fadddffb700 1 -- 172.21.15.157:0/1358319931 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc800c200 con 0x7fadc80053c0 2023-10-06T03:57:19.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.124+0000 7fadddffb700 1 -- 172.21.15.157:0/1358319931 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc800c370 con 0x7fadc8004810 2023-10-06T03:57:19.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.124+0000 7fadddffb700 1 -- 172.21.15.157:0/1358319931 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc800c530 con 0x7fadc8003b10 2023-10-06T03:57:19.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.124+0000 7faddf7fe700 1 --2- 172.21.15.157:0/1358319931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadc80053c0 0x7fadc800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.125+0000 7faddf7fe700 1 -- 172.21.15.157:0/1358319931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fadc8003b10 msgr2=0x7fadc8003f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T03:57:19.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.125+0000 7faddf7fe700 1 --2- 172.21.15.157:0/1358319931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fadc8003b10 0x7fadc8003f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.125+0000 7faddf7fe700 1 -- 172.21.15.157:0/1358319931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fadc8004810 msgr2=0x7fadc8004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T03:57:19.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.125+0000 7faddf7fe700 1 --2- 172.21.15.157:0/1358319931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fadc8004810 0x7fadc8004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.125+0000 7faddf7fe700 1 -- 172.21.15.157:0/1358319931 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade0153930 con 0x7fadc80053c0 2023-10-06T03:57:19.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.125+0000 7faddf7fe700 1 --2- 172.21.15.157:0/1358319931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadc80053c0 0x7fadc800ba60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fadd800a440 tx=0x7fadd800ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b642a290c898560 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.125+0000 7fadddffb700 1 -- 172.21.15.157:0/1358319931 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd8013770 con 0x7fadc80053c0 2023-10-06T03:57:19.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.125+0000 7fadddffb700 1 -- 172.21.15.157:0/1358319931 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd8013910 con 0x7fadc80053c0 2023-10-06T03:57:19.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.125+0000 7fadddffb700 1 -- 172.21.15.157:0/1358319931 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd8013cd0 con 0x7fadc80053c0 2023-10-06T03:57:19.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.126+0000 7fade52b7700 1 -- 172.21.15.157:0/1358319931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadc80053c0 msgr2=0x7fadc800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.126+0000 7fade52b7700 1 --2- 172.21.15.157:0/1358319931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadc80053c0 0x7fadc800ba60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fadd800a440 tx=0x7fadd800ac10 comp rx=0 tx=0).stop 2023-10-06T03:57:19.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.126+0000 7faddf7fe700 1 -- 172.21.15.157:0/1358319931 reap_dead start 2023-10-06T03:57:19.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.126+0000 7fade52b7700 1 -- 172.21.15.157:0/1358319931 shutdown_connections 2023-10-06T03:57:19.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.126+0000 7fade52b7700 1 -- 172.21.15.157:0/1358319931 >> 172.21.15.157:0/1358319931 conn(0x7fade006b510 msgr2=0x7fade006b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.127+0000 7f192906e700 1 -- 172.21.15.157:0/912029013 --> [v2:172.21.15.175:6800/91261,v1:172.21.15.175:6803/91261] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1924068190 con 0x7f191006f8b0 2023-10-06T03:57:19.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.129+0000 7f190affd700 1 -- 172.21.15.157:0/912029013 <== osd.4 v2:172.21.15.175:6800/91261 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1924068190 con 0x7f191006f8b0 2023-10-06T03:57:19.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.129+0000 7f192906e700 1 -- 172.21.15.157:0/912029013 >> [v2:172.21.15.175:6800/91261,v1:172.21.15.175:6803/91261] conn(0x7f191006f8b0 msgr2=0x7f1910071cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.129+0000 7f192906e700 1 --2- 172.21.15.157:0/912029013 >> [v2:172.21.15.175:6800/91261,v1:172.21.15.175:6803/91261] conn(0x7f191006f8b0 0x7f1910071cc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.129+0000 7f192906e700 1 -- 172.21.15.157:0/912029013 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1910067000 msgr2=0x7f19100694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.129+0000 7f192906e700 1 --2- 172.21.15.157:0/912029013 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1910067000 0x7f19100694b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f19240c4290 tx=0x7f19180091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.129+0000 7f192906e700 1 -- 172.21.15.157:0/912029013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f192406c1f0 msgr2=0x7f19241374f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.129+0000 7f192906e700 1 --2- 172.21.15.157:0/912029013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f192406c1f0 0x7f19241374f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1914009ed0 tx=0x7f1914017000 comp rx=0 tx=0).stop 2023-10-06T03:57:19.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.130+0000 7f192359e700 1 -- 172.21.15.157:0/912029013 reap_dead start 2023-10-06T03:57:19.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.130+0000 7f192906e700 1 -- 172.21.15.157:0/912029013 shutdown_connections 2023-10-06T03:57:19.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.130+0000 7f192906e700 1 -- 172.21.15.157:0/912029013 >> 172.21.15.157:0/912029013 conn(0x7f19240c82e0 msgr2=0x7f19240c7560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.130+0000 7f192906e700 1 -- 172.21.15.157:0/912029013 shutdown_connections 2023-10-06T03:57:19.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.130+0000 7f192906e700 1 -- 172.21.15.157:0/912029013 wait complete. 2023-10-06T03:57:19.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.133+0000 7ff51550c700 1 -- 172.21.15.157:0/3261109047 --> [v2:172.21.15.157:6824/90657,v1:172.21.15.157:6825/90657] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff51006d220 con 0x7ff4fc06f7e0 2023-10-06T03:57:19.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.133+0000 7ff4f6ffd700 1 -- 172.21.15.157:0/3261109047 <== osd.2 v2:172.21.15.157:6824/90657 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff51006d220 con 0x7ff4fc06f7e0 2023-10-06T03:57:19.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.134+0000 7ff51550c700 1 -- 172.21.15.157:0/3261109047 >> [v2:172.21.15.157:6824/90657,v1:172.21.15.157:6825/90657] conn(0x7ff4fc06f7e0 msgr2=0x7ff4fc071bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.134+0000 7ff51550c700 1 --2- 172.21.15.157:0/3261109047 >> [v2:172.21.15.157:6824/90657,v1:172.21.15.157:6825/90657] conn(0x7ff4fc06f7e0 0x7ff4fc071bf0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.136+0000 7ff51550c700 1 -- 172.21.15.157:0/3261109047 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff4fc066f30 msgr2=0x7ff4fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.136+0000 7ff51550c700 1 --2- 172.21.15.157:0/3261109047 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff4fc066f30 0x7ff4fc0693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff500006fd0 tx=0x7ff5000090f0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.136+0000 7ff51550c700 1 -- 172.21.15.157:0/3261109047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5100bf8f0 msgr2=0x7ff510141850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.136+0000 7ff51550c700 1 --2- 172.21.15.157:0/3261109047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5100bf8f0 0x7ff510141850 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff50800bbb0 tx=0x7ff508009840 comp rx=0 tx=0).stop 2023-10-06T03:57:19.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.136+0000 7ff50f7fe700 1 -- 172.21.15.157:0/3261109047 reap_dead start 2023-10-06T03:57:19.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.136+0000 7ff51550c700 1 -- 172.21.15.157:0/3261109047 shutdown_connections 2023-10-06T03:57:19.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.136+0000 7ff51550c700 1 -- 172.21.15.157:0/3261109047 >> 172.21.15.157:0/3261109047 conn(0x7ff5100c8b20 msgr2=0x7ff5100c4600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.181 INFO:teuthology.orchestra.run.smithi157.stdout:55834574855 2023-10-06T03:57:19.182 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T03:57:19.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.142+0000 7fade52b7700 1 -- 172.21.15.157:0/1358319931 shutdown_connections 2023-10-06T03:57:19.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.142+0000 7fade52b7700 1 -- 172.21.15.157:0/1358319931 wait complete. 2023-10-06T03:57:19.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.143+0000 7fade52b7700 1 Processor -- start 2023-10-06T03:57:19.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.143+0000 7fade52b7700 1 -- start start 2023-10-06T03:57:19.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.143+0000 7ff51550c700 1 -- 172.21.15.157:0/3261109047 shutdown_connections 2023-10-06T03:57:19.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.143+0000 7ff51550c700 1 -- 172.21.15.157:0/3261109047 wait complete. 2023-10-06T03:57:19.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.144+0000 7fade52b7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fadc80053c0 0x7fade00bd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.144+0000 7fade52b7700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fade00c20a0 0x7fade00c14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.144+0000 7fade52b7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade0161c50 0x7fade014d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.144+0000 7fade52b7700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade00bdd30 con 0x7fade0161c50 2023-10-06T03:57:19.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.144+0000 7fade52b7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade0161540 con 0x7fadc80053c0 2023-10-06T03:57:19.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.144+0000 7fade52b7700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade01617c0 con 0x7fade00c20a0 2023-10-06T03:57:19.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.144+0000 7faddf7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade0161c50 0x7fade014d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.144+0000 7faddf7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade0161c50 0x7fade014d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34412/0 (socket says 172.21.15.157:34412) 2023-10-06T03:57:19.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.144+0000 7faddf7fe700 1 -- 172.21.15.157:0/3282892166 learned_addr learned my addr 172.21.15.157:0/3282892166 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:19.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.145+0000 7faddeffd700 1 --2- 172.21.15.157:0/3282892166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fadc80053c0 0x7fade00bd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.145+0000 7fadde7fc700 1 --2- 172.21.15.157:0/3282892166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fade00c20a0 0x7fade00c14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.145+0000 7faddf7fe700 1 -- 172.21.15.157:0/3282892166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fade00c20a0 msgr2=0x7fade00c14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.145+0000 7faddf7fe700 1 --2- 172.21.15.157:0/3282892166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fade00c20a0 0x7fade00c14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.145+0000 7faddf7fe700 1 -- 172.21.15.157:0/3282892166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fadc80053c0 msgr2=0x7fade00bd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.145+0000 7faddf7fe700 1 --2- 172.21.15.157:0/3282892166 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fadc80053c0 0x7fade00bd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.145+0000 7faddf7fe700 1 -- 172.21.15.157:0/3282892166 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade0155a10 con 0x7fade0161c50 2023-10-06T03:57:19.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.145+0000 7faddf7fe700 1 --2- 172.21.15.157:0/3282892166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade0161c50 0x7fade014d510 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fadd800a410 tx=0x7fadd8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.145+0000 7fadc6ffd700 1 -- 172.21.15.157:0/3282892166 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd8023070 con 0x7fade0161c50 2023-10-06T03:57:19.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.145+0000 7fadc6ffd700 1 -- 172.21.15.157:0/3282892166 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd8007320 con 0x7fade0161c50 2023-10-06T03:57:19.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.145+0000 7fade52b7700 1 -- 172.21.15.157:0/3282892166 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade006cb60 con 0x7fade0161c50 2023-10-06T03:57:19.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.145+0000 7fadc6ffd700 1 -- 172.21.15.157:0/3282892166 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd80076c0 con 0x7fade0161c50 2023-10-06T03:57:19.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.146+0000 7fade52b7700 1 -- 172.21.15.157:0/3282892166 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade00bf590 con 0x7fade0161c50 2023-10-06T03:57:19.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.147+0000 7ff3597b3700 1 -- 172.21.15.157:0/651892751 --> [v2:172.21.15.175:6824/91306,v1:172.21.15.175:6825/91306] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff354068190 con 0x7ff34006f7e0 2023-10-06T03:57:19.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.147+0000 7ff33affd700 1 -- 172.21.15.157:0/651892751 <== osd.7 v2:172.21.15.175:6824/91306 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff354068190 con 0x7ff34006f7e0 2023-10-06T03:57:19.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.147+0000 7ff3597b3700 1 -- 172.21.15.157:0/651892751 >> [v2:172.21.15.175:6824/91306,v1:172.21.15.175:6825/91306] conn(0x7ff34006f7e0 msgr2=0x7ff340071bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.147+0000 7ff3597b3700 1 --2- 172.21.15.157:0/651892751 >> [v2:172.21.15.175:6824/91306,v1:172.21.15.175:6825/91306] conn(0x7ff34006f7e0 0x7ff340071bf0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.148+0000 7fadc6ffd700 1 -- 172.21.15.157:0/3282892166 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fadd801c420 con 0x7fade0161c50 2023-10-06T03:57:19.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.148+0000 7ff3597b3700 1 -- 172.21.15.157:0/651892751 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff340066fb0 msgr2=0x7ff340069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.148+0000 7ff3597b3700 1 --2- 172.21.15.157:0/651892751 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff340066fb0 0x7ff340069460 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff354173ad0 tx=0x7ff3480091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.148+0000 7ff3597b3700 1 -- 172.21.15.157:0/651892751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff354069950 msgr2=0x7ff35414d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.148+0000 7ff3597b3700 1 --2- 172.21.15.157:0/651892751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff354069950 0x7ff35414d9f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff344007480 tx=0x7ff344019000 comp rx=0 tx=0).stop 2023-10-06T03:57:19.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.148+0000 7ff3537fe700 1 -- 172.21.15.157:0/651892751 reap_dead start 2023-10-06T03:57:19.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.148+0000 7ff3597b3700 1 -- 172.21.15.157:0/651892751 shutdown_connections 2023-10-06T03:57:19.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.148+0000 7ff3597b3700 1 -- 172.21.15.157:0/651892751 >> 172.21.15.157:0/651892751 conn(0x7ff3540c82e0 msgr2=0x7ff3540c6360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.148+0000 7fadc6ffd700 1 --2- 172.21.15.157:0/3282892166 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fadcc066f60 0x7fadcc069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.148+0000 7fadc6ffd700 1 -- 172.21.15.157:0/3282892166 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fadd8089800 con 0x7fade0161c50 2023-10-06T03:57:19.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.149+0000 7ff3597b3700 1 -- 172.21.15.157:0/651892751 shutdown_connections 2023-10-06T03:57:19.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.148+0000 7faddeffd700 1 --2- 172.21.15.157:0/3282892166 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fadcc066f60 0x7fadcc069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.149+0000 7ff3597b3700 1 -- 172.21.15.157:0/651892751 wait complete. 2023-10-06T03:57:19.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.149+0000 7f53985b7700 1 -- 172.21.15.157:0/64787291 --> [v2:172.21.15.157:6800/90654,v1:172.21.15.157:6801/90654] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5390068190 con 0x7f537406f930 2023-10-06T03:57:19.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.149+0000 7faddeffd700 1 --2- 172.21.15.157:0/3282892166 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fadcc066f60 0x7fadcc069410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fade00ca3d0 tx=0x7fadd4007ea0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.149+0000 7f53867fc700 1 -- 172.21.15.157:0/64787291 <== osd.1 v2:172.21.15.157:6800/90654 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5390068190 con 0x7f537406f930 2023-10-06T03:57:19.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.149+0000 7f53985b7700 1 -- 172.21.15.157:0/64787291 >> [v2:172.21.15.157:6800/90654,v1:172.21.15.157:6801/90654] conn(0x7f537406f930 msgr2=0x7f5374071d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.149+0000 7f53985b7700 1 --2- 172.21.15.157:0/64787291 >> [v2:172.21.15.157:6800/90654,v1:172.21.15.157:6801/90654] conn(0x7f537406f930 0x7f5374071d40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.150+0000 7f53985b7700 1 -- 172.21.15.157:0/64787291 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5374067080 msgr2=0x7f5374069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.150+0000 7f53985b7700 1 --2- 172.21.15.157:0/64787291 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5374067080 0x7f5374069530 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5388006f40 tx=0x7f5388005480 comp rx=0 tx=0).stop 2023-10-06T03:57:19.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.150+0000 7f53985b7700 1 -- 172.21.15.157:0/64787291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53900adca0 msgr2=0x7f53901566f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.150+0000 7f53985b7700 1 --2- 172.21.15.157:0/64787291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53900adca0 0x7f53901566f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f538c0093f0 tx=0x7f538c016000 comp rx=0 tx=0).stop 2023-10-06T03:57:19.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.150+0000 7f5396b54700 1 -- 172.21.15.157:0/64787291 reap_dead start 2023-10-06T03:57:19.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.150+0000 7f53985b7700 1 -- 172.21.15.157:0/64787291 shutdown_connections 2023-10-06T03:57:19.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.150+0000 7f53985b7700 1 -- 172.21.15.157:0/64787291 >> 172.21.15.157:0/64787291 conn(0x7f53900c82e0 msgr2=0x7f53900c2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.150+0000 7f53985b7700 1 -- 172.21.15.157:0/64787291 shutdown_connections 2023-10-06T03:57:19.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.150+0000 7fade52b7700 1 --2- 172.21.15.157:0/3282892166 >> [v2:172.21.15.175:6816/91262,v1:172.21.15.175:6817/91262] conn(0x7fade01620d0 0x7fade014da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.150+0000 7fade52b7700 1 -- 172.21.15.157:0/3282892166 --> [v2:172.21.15.175:6816/91262,v1:172.21.15.175:6817/91262] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fade00bfd80 con 0x7fade01620d0 2023-10-06T03:57:19.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.151+0000 7f53985b7700 1 -- 172.21.15.157:0/64787291 wait complete. 2023-10-06T03:57:19.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.151+0000 7fadde7fc700 1 --2- 172.21.15.157:0/3282892166 >> [v2:172.21.15.175:6816/91262,v1:172.21.15.175:6817/91262] conn(0x7fade01620d0 0x7fade014da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.151+0000 7fadde7fc700 1 --2- 172.21.15.157:0/3282892166 >> [v2:172.21.15.175:6816/91262,v1:172.21.15.175:6817/91262] conn(0x7fade01620d0 0x7fade014da50 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.151+0000 7f32bd4b2700 1 -- 172.21.15.157:0/2161114439 --> [v2:172.21.15.175:6802/91263,v1:172.21.15.175:6801/91263] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f32b804bb40 con 0x7f329806f8b0 2023-10-06T03:57:19.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.152+0000 7fadc6ffd700 1 -- 172.21.15.157:0/3282892166 <== osd.5 v2:172.21.15.175:6816/91262 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fade00bfd80 con 0x7fade01620d0 2023-10-06T03:57:19.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.152+0000 7f32af7fe700 1 -- 172.21.15.157:0/2161114439 <== osd.6 v2:172.21.15.175:6802/91263 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f32b804bb40 con 0x7f329806f8b0 2023-10-06T03:57:19.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.152+0000 7f32bd4b2700 1 -- 172.21.15.157:0/2161114439 >> [v2:172.21.15.175:6802/91263,v1:172.21.15.175:6801/91263] conn(0x7f329806f8b0 msgr2=0x7f3298071cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.152+0000 7f32bd4b2700 1 --2- 172.21.15.157:0/2161114439 >> [v2:172.21.15.175:6802/91263,v1:172.21.15.175:6801/91263] conn(0x7f329806f8b0 0x7f3298071cc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.154+0000 7f6f1ab13700 1 -- 172.21.15.157:0/2146913966 --> [v2:172.21.15.157:6816/90641,v1:172.21.15.157:6817/90641] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6f140896c0 con 0x7f6f0006f8b0 2023-10-06T03:57:19.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.154+0000 7f6f10ff9700 1 -- 172.21.15.157:0/2146913966 <== osd.3 v2:172.21.15.157:6816/90641 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6f140896c0 con 0x7f6f0006f8b0 2023-10-06T03:57:19.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.154+0000 7fd877654700 1 -- 172.21.15.157:0/4182634397 --> [v2:172.21.15.157:6808/90640,v1:172.21.15.157:6809/90640] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd8700716b0 con 0x7fd85c06f8e0 2023-10-06T03:57:19.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.155+0000 7fd86d7fa700 1 -- 172.21.15.157:0/4182634397 <== osd.0 v2:172.21.15.157:6808/90640 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd8700716b0 con 0x7fd85c06f8e0 2023-10-06T03:57:19.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.155+0000 7f32bd4b2700 1 -- 172.21.15.157:0/2161114439 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3298067000 msgr2=0x7f32980694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.155+0000 7f32bd4b2700 1 --2- 172.21.15.157:0/2161114439 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3298067000 0x7f32980694b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f32a00010e0 tx=0x7f32a0001da0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.155+0000 7f32bd4b2700 1 -- 172.21.15.157:0/2161114439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b8158aa0 msgr2=0x7f32b81449f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.155+0000 7f32bd4b2700 1 --2- 172.21.15.157:0/2161114439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b8158aa0 0x7f32b81449f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f32b0011000 tx=0x7f32b0015000 comp rx=0 tx=0).stop 2023-10-06T03:57:19.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.155+0000 7f32b77fe700 1 -- 172.21.15.157:0/2161114439 reap_dead start 2023-10-06T03:57:19.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.155+0000 7f6f1ab13700 1 -- 172.21.15.157:0/2146913966 >> [v2:172.21.15.157:6816/90641,v1:172.21.15.157:6817/90641] conn(0x7f6f0006f8b0 msgr2=0x7f6f00071cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.155+0000 7f6f1ab13700 1 --2- 172.21.15.157:0/2146913966 >> [v2:172.21.15.157:6816/90641,v1:172.21.15.157:6817/90641] conn(0x7f6f0006f8b0 0x7f6f00071cc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.155+0000 7f6f1ab13700 1 -- 172.21.15.157:0/2146913966 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6f00067000 msgr2=0x7f6f000694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.155+0000 7f6f1ab13700 1 --2- 172.21.15.157:0/2146913966 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6f00067000 0x7f6f000694b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c0020e0 tx=0x7f6f0c001690 comp rx=0 tx=0).stop 2023-10-06T03:57:19.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.155+0000 7f6f1ab13700 1 -- 172.21.15.157:0/2146913966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f1415afa0 msgr2=0x7f6f140694e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.155+0000 7f6f1ab13700 1 --2- 172.21.15.157:0/2146913966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f1415afa0 0x7f6f140694e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6f0800c560 tx=0x7f6f08007ef0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.155+0000 7f32bd4b2700 1 -- 172.21.15.157:0/2161114439 shutdown_connections 2023-10-06T03:57:19.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.155+0000 7f32bd4b2700 1 -- 172.21.15.157:0/2161114439 >> 172.21.15.157:0/2161114439 conn(0x7f32b80c82e0 msgr2=0x7f32b806f160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.155+0000 7f32bd4b2700 1 -- 172.21.15.157:0/2161114439 shutdown_connections 2023-10-06T03:57:19.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.155+0000 7f32bd4b2700 1 -- 172.21.15.157:0/2161114439 wait complete. 2023-10-06T03:57:19.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.155+0000 7fd877654700 1 -- 172.21.15.157:0/4182634397 >> [v2:172.21.15.157:6808/90640,v1:172.21.15.157:6809/90640] conn(0x7fd85c06f8e0 msgr2=0x7fd85c071cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.155+0000 7fd877654700 1 --2- 172.21.15.157:0/4182634397 >> [v2:172.21.15.157:6808/90640,v1:172.21.15.157:6809/90640] conn(0x7fd85c06f8e0 0x7fd85c071cf0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.156+0000 7fd877654700 1 -- 172.21.15.157:0/4182634397 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd85c067030 msgr2=0x7fd85c0694e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.156+0000 7fd877654700 1 --2- 172.21.15.157:0/4182634397 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd85c067030 0x7fd85c0694e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd864006fd0 tx=0x7fd864008040 comp rx=0 tx=0).stop 2023-10-06T03:57:19.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.156+0000 7fd877654700 1 -- 172.21.15.157:0/4182634397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd87006c440 msgr2=0x7fd8701731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.156+0000 7fd877654700 1 --2- 172.21.15.157:0/4182634397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd87006c440 0x7fd8701731c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd8700c38e0 tx=0x7fd86801b000 comp rx=0 tx=0).stop 2023-10-06T03:57:19.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.156+0000 7fd875bf1700 1 -- 172.21.15.157:0/4182634397 reap_dead start 2023-10-06T03:57:19.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.156+0000 7f6f190b0700 1 -- 172.21.15.157:0/2146913966 reap_dead start 2023-10-06T03:57:19.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.156+0000 7fd877654700 1 -- 172.21.15.157:0/4182634397 shutdown_connections 2023-10-06T03:57:19.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.156+0000 7fd877654700 1 -- 172.21.15.157:0/4182634397 >> 172.21.15.157:0/4182634397 conn(0x7fd8700c82e0 msgr2=0x7fd8700c2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.161+0000 7f6f1ab13700 1 -- 172.21.15.157:0/2146913966 shutdown_connections 2023-10-06T03:57:19.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.161+0000 7f6f1ab13700 1 -- 172.21.15.157:0/2146913966 >> 172.21.15.157:0/2146913966 conn(0x7f6f140cc6e0 msgr2=0x7f6f1406e120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.161+0000 7fd877654700 1 -- 172.21.15.157:0/4182634397 shutdown_connections 2023-10-06T03:57:19.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.161+0000 7f6f1ab13700 1 -- 172.21.15.157:0/2146913966 shutdown_connections 2023-10-06T03:57:19.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.161+0000 7f6f1ab13700 1 -- 172.21.15.157:0/2146913966 wait complete. 2023-10-06T03:57:19.202 INFO:teuthology.orchestra.run.smithi157.stdout:55834574854 2023-10-06T03:57:19.202 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T03:57:19.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.171+0000 7fd877654700 1 -- 172.21.15.157:0/4182634397 wait complete. 2023-10-06T03:57:19.203 INFO:teuthology.orchestra.run.smithi157.stdout:55834574854 2023-10-06T03:57:19.203 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T03:57:19.204 INFO:teuthology.orchestra.run.smithi157.stdout:55834574854 2023-10-06T03:57:19.205 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T03:57:19.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.176+0000 7fade52b7700 1 -- 172.21.15.157:0/3282892166 --> [v2:172.21.15.175:6816/91262,v1:172.21.15.175:6817/91262] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fade014df90 con 0x7fade01620d0 2023-10-06T03:57:19.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.176+0000 7fadc6ffd700 1 -- 172.21.15.157:0/3282892166 <== osd.5 v2:172.21.15.175:6816/91262 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fade014df90 con 0x7fade01620d0 2023-10-06T03:57:19.206 INFO:teuthology.orchestra.run.smithi157.stdout:55834574854 2023-10-06T03:57:19.206 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T03:57:19.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.177+0000 7fade52b7700 1 -- 172.21.15.157:0/3282892166 >> [v2:172.21.15.175:6816/91262,v1:172.21.15.175:6817/91262] conn(0x7fade01620d0 msgr2=0x7fade014da50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.177+0000 7fade52b7700 1 --2- 172.21.15.157:0/3282892166 >> [v2:172.21.15.175:6816/91262,v1:172.21.15.175:6817/91262] conn(0x7fade01620d0 0x7fade014da50 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.177+0000 7fade52b7700 1 -- 172.21.15.157:0/3282892166 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fadcc066f60 msgr2=0x7fadcc069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.177+0000 7fade52b7700 1 --2- 172.21.15.157:0/3282892166 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fadcc066f60 0x7fadcc069410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fade00ca3d0 tx=0x7fadd4007ea0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.177+0000 7fade52b7700 1 -- 172.21.15.157:0/3282892166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade0161c50 msgr2=0x7fade014d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.177+0000 7fade52b7700 1 --2- 172.21.15.157:0/3282892166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade0161c50 0x7fade014d510 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fadd800a410 tx=0x7fadd8019000 comp rx=0 tx=0).stop 2023-10-06T03:57:19.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.177+0000 7faddf7fe700 1 -- 172.21.15.157:0/3282892166 reap_dead start 2023-10-06T03:57:19.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.177+0000 7fade52b7700 1 -- 172.21.15.157:0/3282892166 shutdown_connections 2023-10-06T03:57:19.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.177+0000 7fade52b7700 1 -- 172.21.15.157:0/3282892166 >> 172.21.15.157:0/3282892166 conn(0x7fade006b510 msgr2=0x7fade006b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.177+0000 7fade52b7700 1 -- 172.21.15.157:0/3282892166 shutdown_connections 2023-10-06T03:57:19.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.177+0000 7fade52b7700 1 -- 172.21.15.157:0/3282892166 wait complete. 2023-10-06T03:57:19.209 INFO:teuthology.orchestra.run.smithi157.stdout:55834574854 2023-10-06T03:57:19.209 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T03:57:19.210 INFO:teuthology.orchestra.run.smithi157.stdout:55834574854 2023-10-06T03:57:19.210 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T03:57:19.211 INFO:teuthology.orchestra.run.smithi157.stdout:55834574854 2023-10-06T03:57:19.211 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T03:57:19.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.435+0000 7f574a17a700 1 Processor -- start 2023-10-06T03:57:19.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.435+0000 7f574a17a700 1 -- start start 2023-10-06T03:57:19.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.436+0000 7f574a17a700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f57440be500 0x7f57440c5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.436+0000 7f574a17a700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f574406d460 0x7f57440be8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.436+0000 7f574a17a700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574406f680 con 0x7f5744069410 2023-10-06T03:57:19.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.436+0000 7f574a17a700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57440bee10 con 0x7f57440be500 2023-10-06T03:57:19.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.436+0000 7f574a17a700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f574406d900 con 0x7f574406d460 2023-10-06T03:57:19.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.436+0000 7f5743fff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f5744069410 0x7f574415ded0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40020/0 (socket says 172.21.15.157:40020) 2023-10-06T03:57:19.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.437+0000 7f5743fff700 1 -- 172.21.15.157:0/2841798407 learned_addr learned my addr 172.21.15.157:0/2841798407 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:19.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.437+0000 7f5742ffd700 1 --2- 172.21.15.157:0/2841798407 >> v2:172.21.15.175:3300/0 conn(0x7f574406d460 0x7f57440be8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.437+0000 7f57437fe700 1 --2- 172.21.15.157:0/2841798407 >> v2:172.21.15.157:3300/0 conn(0x7f57440be500 0x7f57440c5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.437+0000 7f57437fe700 1 -- 172.21.15.157:0/2841798407 >> v2:172.21.15.175:3300/0 conn(0x7f574406d460 msgr2=0x7f57440be8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:19.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.437+0000 7f57437fe700 1 --2- 172.21.15.157:0/2841798407 >> v2:172.21.15.175:3300/0 conn(0x7f574406d460 0x7f57440be8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.437+0000 7f57437fe700 1 -- 172.21.15.157:0/2841798407 >> v1:172.21.15.157:6789/0 conn(0x7f5744069410 legacy=0x7f574415ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.437+0000 7f57437fe700 1 -- 172.21.15.157:0/2841798407 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574415ec90 con 0x7f57440be500 2023-10-06T03:57:19.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.437+0000 7f57437fe700 1 --2- 172.21.15.157:0/2841798407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57440be500 0x7f57440c5460 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5734009910 tx=0x7f57340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70cbc7a462129eff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.438+0000 7f57427fc700 1 -- 172.21.15.157:0/2841798407 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573400b7e0 con 0x7f57440be500 2023-10-06T03:57:19.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.438+0000 7f57427fc700 1 -- 172.21.15.157:0/2841798407 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f573400b940 con 0x7f57440be500 2023-10-06T03:57:19.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.438+0000 7f57427fc700 1 -- 172.21.15.157:0/2841798407 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573400bd00 con 0x7f57440be500 2023-10-06T03:57:19.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.439+0000 7f574a17a700 1 -- 172.21.15.157:0/2841798407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57440be500 msgr2=0x7f57440c5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.439+0000 7f574a17a700 1 --2- 172.21.15.157:0/2841798407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57440be500 0x7f57440c5460 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5734009910 tx=0x7f57340091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.439+0000 7f574a17a700 1 -- 172.21.15.157:0/2841798407 shutdown_connections 2023-10-06T03:57:19.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.439+0000 7f574a17a700 1 --2- 172.21.15.157:0/2841798407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57440be500 0x7f57440c5460 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.439+0000 7f574a17a700 1 --2- 172.21.15.157:0/2841798407 >> v2:172.21.15.175:3300/0 conn(0x7f574406d460 0x7f57440be8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.439+0000 7f574a17a700 1 -- 172.21.15.157:0/2841798407 >> 172.21.15.157:0/2841798407 conn(0x7f57440c8d70 msgr2=0x7f57440c4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.441+0000 7f574a17a700 1 -- 172.21.15.157:0/2841798407 shutdown_connections 2023-10-06T03:57:19.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.442+0000 7f574a17a700 1 -- 172.21.15.157:0/2841798407 wait complete. 2023-10-06T03:57:19.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.443+0000 7f113468a700 1 Processor -- start 2023-10-06T03:57:19.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.443+0000 7faefd704700 1 Processor -- start 2023-10-06T03:57:19.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.443+0000 7faefd704700 1 -- start start 2023-10-06T03:57:19.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.443+0000 7f396bd58700 1 Processor -- start 2023-10-06T03:57:19.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.443+0000 7f54b8ea2700 1 Processor -- start 2023-10-06T03:57:19.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.443+0000 7f574a17a700 1 Processor -- start 2023-10-06T03:57:19.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.443+0000 7f396bd58700 1 -- start start 2023-10-06T03:57:19.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.443+0000 7f54b8ea2700 1 -- start start 2023-10-06T03:57:19.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.444+0000 7faefd704700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faef806a530 0x7faef806ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.444+0000 7faefd704700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faef806e8c0 0x7faef8158270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.444+0000 7faefd704700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7faef8068ba0 0x7faef815acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.444+0000 7faefd704700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faef806c9e0 con 0x7faef806e8c0 2023-10-06T03:57:19.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.444+0000 7faefd704700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faef80c3e80 con 0x7faef8068ba0 2023-10-06T03:57:19.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.444+0000 7faefd704700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef80c4160 con 0x7faef806a530 2023-10-06T03:57:19.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.443+0000 7f113468a700 1 -- start start 2023-10-06T03:57:19.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.444+0000 7f113468a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f112c06c440 0x7f112c06e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.444+0000 7f113468a700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f112c06c810 0x7f112c069410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.444+0000 7f113468a700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f112c14cb50 con 0x7f112c069950 2023-10-06T03:57:19.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.444+0000 7f113468a700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f112c0c3a80 con 0x7f112c06c810 2023-10-06T03:57:19.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.444+0000 7f113468a700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f112c0c37a0 con 0x7f112c06c440 2023-10-06T03:57:19.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.444+0000 7faef67fc700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faef806e8c0 0x7faef8158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7faef77fe700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7faef8068ba0 0x7faef815acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7faef67fc700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faef806e8c0 0x7faef8158270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34434/0 (socket says 172.21.15.157:34434) 2023-10-06T03:57:19.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7faef67fc700 1 -- 172.21.15.157:0/887854863 learned_addr learned my addr 172.21.15.157:0/887854863 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:19.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7faef6ffd700 1 --2- 172.21.15.157:0/887854863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faef806a530 0x7faef806ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.444+0000 7f1132c27700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f112c069950 0x7f112c0c2c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40036/0 (socket says 172.21.15.157:40036) 2023-10-06T03:57:19.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.444+0000 7f1132c27700 1 -- 172.21.15.157:0/1567426394 learned_addr learned my addr 172.21.15.157:0/1567426394 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:19.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f1131c25700 1 --2- 172.21.15.157:0/1567426394 >> v2:172.21.15.175:3300/0 conn(0x7f112c06c810 0x7f112c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f1132426700 1 --2- 172.21.15.157:0/1567426394 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f112c06c440 0x7f112c06e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.443+0000 7f574a17a700 1 -- start start 2023-10-06T03:57:19.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f574a17a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5744069410 0x7f574415aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f574a17a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f574406d460 0x7f574414baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f574a17a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57440be500 0x7f574414bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f574a17a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574415d0a0 con 0x7f57440be500 2023-10-06T03:57:19.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f574a17a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574415dbe0 con 0x7f574406d460 2023-10-06T03:57:19.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f574a17a700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574415bac0 con 0x7f5744069410 2023-10-06T03:57:19.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f54b8ea2700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f54b415e530 0x7f54b4069410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f54b8ea2700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f54b4069950 0x7f54b40ad930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f54b8ea2700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54b406fc90 con 0x7f54b415e160 2023-10-06T03:57:19.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f54b8ea2700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b4155130 con 0x7f54b4069950 2023-10-06T03:57:19.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f54b8ea2700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b406f800 con 0x7f54b415e530 2023-10-06T03:57:19.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f5742ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f574406d460 0x7f574414baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f5742ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f574406d460 0x7f574414baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:35146/0 (socket says 172.21.15.157:35146) 2023-10-06T03:57:19.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f5742ffd700 1 -- 172.21.15.157:0/1289536747 learned_addr learned my addr 172.21.15.157:0/1289536747 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:19.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f396bd58700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f396415e160 0x7f39640c5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f396bd58700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f396415e9a0 0x7f39640ad930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f396bd58700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f396406fc90 con 0x7f396415e530 2023-10-06T03:57:19.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f396bd58700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3964155130 con 0x7f396415e9a0 2023-10-06T03:57:19.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f396bd58700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f396406f800 con 0x7f396415e160 2023-10-06T03:57:19.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f54b1d9b700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f54b415e530 0x7f54b4069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f54b1d9b700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f54b415e530 0x7f54b4069410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:35148/0 (socket says 172.21.15.157:35148) 2023-10-06T03:57:19.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f54b1d9b700 1 -- 172.21.15.157:0/2965664163 learned_addr learned my addr 172.21.15.157:0/2965664163 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:19.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f57437fe700 1 --2- 172.21.15.157:0/1289536747 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5744069410 0x7f574415aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f5743fff700 1 --2- 172.21.15.157:0/1289536747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57440be500 0x7f574414bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f1131c25700 1 -- 172.21.15.157:0/1567426394 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f112c06c440 msgr2=0x7f112c06e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:19.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f1131c25700 1 --2- 172.21.15.157:0/1567426394 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f112c06c440 0x7f112c06e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f1131c25700 1 -- 172.21.15.157:0/1567426394 >> v1:172.21.15.157:6789/0 conn(0x7f112c069950 legacy=0x7f112c0c2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f1131c25700 1 -- 172.21.15.157:0/1567426394 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f112c155b60 con 0x7f112c06c810 2023-10-06T03:57:19.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f3969af4700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f396415e160 0x7f39640c5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f3969af4700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f396415e160 0x7f39640c5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:50180/0 (socket says 172.21.15.157:50180) 2023-10-06T03:57:19.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f3969af4700 1 -- 172.21.15.157:0/3946496057 learned_addr learned my addr 172.21.15.157:0/3946496057 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:19.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f396a2f5700 1 --2- 172.21.15.157:0/3946496057 >> v2:172.21.15.157:3300/0 conn(0x7f396415e9a0 0x7f39640ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.446+0000 7f54b2d9d700 1 --2- 172.21.15.157:0/2965664163 >> v2:172.21.15.157:3300/0 conn(0x7f54b4069950 0x7f54b40ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.446+0000 7f1131c25700 1 --2- 172.21.15.157:0/1567426394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f112c06c810 0x7f112c069410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f11280098e0 tx=0x7f11280091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba891def95f743e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.446+0000 7f3969af4700 1 -- 172.21.15.157:0/3946496057 >> v2:172.21.15.157:3300/0 conn(0x7f396415e9a0 msgr2=0x7f39640ad930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:19.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.446+0000 7f3969af4700 1 --2- 172.21.15.157:0/3946496057 >> v2:172.21.15.157:3300/0 conn(0x7f396415e9a0 0x7f39640ad930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.446+0000 7f3969af4700 1 -- 172.21.15.157:0/3946496057 >> v1:172.21.15.157:6789/0 conn(0x7f396415e530 legacy=0x7f3964069410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:19.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.446+0000 7f3969af4700 1 -- 172.21.15.157:0/3946496057 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f396406e950 con 0x7f396415e160 2023-10-06T03:57:19.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.446+0000 7f54b159a700 1 -- 172.21.15.157:0/2965664163 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3817961057 0 0) 0x7f54a4001110 con 0x7f54b415e160 2023-10-06T03:57:19.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.446+0000 7f54b159a700 1 -- 172.21.15.157:0/2965664163 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f549c003570 con 0x7f54b415e160 2023-10-06T03:57:19.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.446+0000 7f1131424700 1 -- 172.21.15.157:0/1567426394 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1128009e80 con 0x7f112c06c810 2023-10-06T03:57:19.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.446+0000 7f1131424700 1 -- 172.21.15.157:0/1567426394 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112800b8f0 con 0x7f112c06c810 2023-10-06T03:57:19.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.446+0000 7f1131424700 1 -- 172.21.15.157:0/1567426394 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112800bcb0 con 0x7f112c06c810 2023-10-06T03:57:19.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f5742ffd700 1 -- 172.21.15.157:0/1289536747 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5744069410 msgr2=0x7f574415aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f5742ffd700 1 --2- 172.21.15.157:0/1289536747 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5744069410 0x7f574415aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f5742ffd700 1 -- 172.21.15.157:0/1289536747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57440be500 msgr2=0x7f574414bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7f5742ffd700 1 --2- 172.21.15.157:0/1289536747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57440be500 0x7f574414bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.446+0000 7f5742ffd700 1 -- 172.21.15.157:0/1289536747 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5734009420 con 0x7f574406d460 2023-10-06T03:57:19.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.446+0000 7f5742ffd700 1 --2- 172.21.15.157:0/1289536747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f574406d460 0x7f574414baa0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f573800c880 tx=0x7f5738008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.446+0000 7f3969af4700 1 --2- 172.21.15.157:0/3946496057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f396415e160 0x7f39640c5460 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f395c009f50 tx=0x7f395c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc372ce723f03fac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.446+0000 7f572b7fe700 1 -- 172.21.15.157:0/1289536747 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5738011600 con 0x7f574406d460 2023-10-06T03:57:19.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.446+0000 7f54b159a700 1 -- 172.21.15.157:0/2965664163 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1062579448 0 0) 0x7f549c003570 con 0x7f54b415e160 2023-10-06T03:57:19.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.446+0000 7f3968af2700 1 -- 172.21.15.157:0/3946496057 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c00e070 con 0x7f396415e160 2023-10-06T03:57:19.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.446+0000 7f3968af2700 1 -- 172.21.15.157:0/3946496057 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395c00ba30 con 0x7f396415e160 2023-10-06T03:57:19.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.446+0000 7f3968af2700 1 -- 172.21.15.157:0/3946496057 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c0092e0 con 0x7f396415e160 2023-10-06T03:57:19.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.446+0000 7f54b2d9d700 1 -- 172.21.15.157:0/2965664163 >> v2:172.21.15.175:3300/0 conn(0x7f54b415e530 msgr2=0x7f54b4069410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:19.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.446+0000 7f54b2d9d700 1 --2- 172.21.15.157:0/2965664163 >> v2:172.21.15.175:3300/0 conn(0x7f54b415e530 0x7f54b4069410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.446+0000 7f54b2d9d700 1 -- 172.21.15.157:0/2965664163 >> v1:172.21.15.157:6789/0 conn(0x7f54b415e160 legacy=0x7f54b40c5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.446+0000 7f54b2d9d700 1 -- 172.21.15.157:0/2965664163 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b406e950 con 0x7f54b4069950 2023-10-06T03:57:19.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.446+0000 7f572b7fe700 1 -- 172.21.15.157:0/1289536747 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5738011780 con 0x7f574406d460 2023-10-06T03:57:19.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.446+0000 7f572b7fe700 1 -- 172.21.15.157:0/1289536747 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5738011b40 con 0x7f574406d460 2023-10-06T03:57:19.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.446+0000 7f574a17a700 1 -- 172.21.15.157:0/1289536747 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574414b4a0 con 0x7f574406d460 2023-10-06T03:57:19.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.446+0000 7f574a17a700 1 -- 172.21.15.157:0/1289536747 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574415d590 con 0x7f574406d460 2023-10-06T03:57:19.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.446+0000 7f54b2d9d700 1 --2- 172.21.15.157:0/2965664163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54b4069950 0x7f54b40ad930 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f54ac009e60 tx=0x7f54ac009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33910f5622e773 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7faef77fe700 1 -- 172.21.15.157:0/887854863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faef806a530 msgr2=0x7faef806ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:19.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7faef77fe700 1 --2- 172.21.15.157:0/887854863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faef806a530 0x7faef806ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7faef77fe700 1 -- 172.21.15.157:0/887854863 >> v2:172.21.15.157:3300/0 conn(0x7faef806e8c0 msgr2=0x7faef8158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:19.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7faef77fe700 1 --2- 172.21.15.157:0/887854863 >> v2:172.21.15.157:3300/0 conn(0x7faef806e8c0 0x7faef8158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.445+0000 7faef77fe700 1 -- 172.21.15.157:0/887854863 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef806c390 con 0x7faef8068ba0 2023-10-06T03:57:19.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.447+0000 7f54b159a700 1 -- 172.21.15.157:0/2965664163 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ac00e070 con 0x7f54b4069950 2023-10-06T03:57:19.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.447+0000 7f54b159a700 1 -- 172.21.15.157:0/2965664163 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54ac00b850 con 0x7f54b4069950 2023-10-06T03:57:19.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.447+0000 7f54b159a700 1 -- 172.21.15.157:0/2965664163 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a4001600 con 0x7f54b4069950 2023-10-06T03:57:19.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.447+0000 7f396bd58700 1 -- 172.21.15.157:0/3946496057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f396415e160 msgr2=0x7f39640c5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.447+0000 7f396bd58700 1 --2- 172.21.15.157:0/3946496057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f396415e160 0x7f39640c5460 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f395c009f50 tx=0x7f395c009840 comp rx=0 tx=0).stop 2023-10-06T03:57:19.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.447+0000 7faef77fe700 1 --2- 172.21.15.157:0/887854863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faef8068ba0 0x7faef815acc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faef806de90 tx=0x7faef0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15a39a8bf960a851 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.447+0000 7fe439461700 1 Processor -- start 2023-10-06T03:57:19.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.447+0000 7fe439461700 1 -- start start 2023-10-06T03:57:19.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.448+0000 7f396bd58700 1 -- 172.21.15.157:0/3946496057 shutdown_connections 2023-10-06T03:57:19.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.448+0000 7f396bd58700 1 --2- 172.21.15.157:0/3946496057 >> v2:172.21.15.157:3300/0 conn(0x7f396415e9a0 0x7f39640ad930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.448+0000 7f396bd58700 1 --2- 172.21.15.157:0/3946496057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f396415e160 0x7f39640c5460 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.448+0000 7f396bd58700 1 -- 172.21.15.157:0/3946496057 >> 172.21.15.157:0/3946496057 conn(0x7f39640c8d70 msgr2=0x7f39640c4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.448+0000 7f574a17a700 1 -- 172.21.15.157:0/1289536747 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5744071d00 con 0x7f574406d460 2023-10-06T03:57:19.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.447+0000 7faef5ffb700 1 -- 172.21.15.157:0/887854863 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef000e070 con 0x7faef8068ba0 2023-10-06T03:57:19.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.447+0000 7faef5ffb700 1 -- 172.21.15.157:0/887854863 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef000bb60 con 0x7faef8068ba0 2023-10-06T03:57:19.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.448+0000 7f396bd58700 1 -- 172.21.15.157:0/3946496057 shutdown_connections 2023-10-06T03:57:19.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.448+0000 7f54b8ea2700 1 -- 172.21.15.157:0/2965664163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54b4069950 msgr2=0x7f54b40ad930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.448+0000 7f54b8ea2700 1 --2- 172.21.15.157:0/2965664163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54b4069950 0x7f54b40ad930 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f54ac009e60 tx=0x7f54ac009780 comp rx=0 tx=0).stop 2023-10-06T03:57:19.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.448+0000 7f54b8ea2700 1 -- 172.21.15.157:0/2965664163 shutdown_connections 2023-10-06T03:57:19.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.448+0000 7f54b8ea2700 1 --2- 172.21.15.157:0/2965664163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54b4069950 0x7f54b40ad930 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.448+0000 7f54b8ea2700 1 --2- 172.21.15.157:0/2965664163 >> v2:172.21.15.175:3300/0 conn(0x7f54b415e530 0x7f54b4069410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.448+0000 7f54b8ea2700 1 -- 172.21.15.157:0/2965664163 >> 172.21.15.157:0/2965664163 conn(0x7f54b40c8d70 msgr2=0x7f54b40c4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.448+0000 7f113468a700 1 -- 172.21.15.157:0/1567426394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f112c06c810 msgr2=0x7f112c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.448+0000 7f113468a700 1 --2- 172.21.15.157:0/1567426394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f112c06c810 0x7f112c069410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f11280098e0 tx=0x7f11280091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.448+0000 7f54b8ea2700 1 -- 172.21.15.157:0/2965664163 shutdown_connections 2023-10-06T03:57:19.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.448+0000 7f54b8ea2700 1 -- 172.21.15.157:0/2965664163 wait complete. 2023-10-06T03:57:19.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.448+0000 7f572b7fe700 1 -- 172.21.15.157:0/1289536747 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5738018360 con 0x7f574406d460 2023-10-06T03:57:19.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.448+0000 7f396bd58700 1 -- 172.21.15.157:0/3946496057 wait complete. 2023-10-06T03:57:19.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.448+0000 7f572b7fe700 1 --2- 172.21.15.157:0/1289536747 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5730067030 0x7f57300694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.449+0000 7f572b7fe700 1 -- 172.21.15.157:0/1289536747 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f57380842d0 con 0x7f574406d460 2023-10-06T03:57:19.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.449+0000 7faef5ffb700 1 -- 172.21.15.157:0/887854863 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef0012430 con 0x7faef8068ba0 2023-10-06T03:57:19.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.448+0000 7f248b693700 1 Processor -- start 2023-10-06T03:57:19.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.448+0000 7f248b693700 1 -- start start 2023-10-06T03:57:19.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.449+0000 7fe439461700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe43415e160 0x7fe4340c5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.449+0000 7fe439461700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe43406fc90 con 0x7fe43415e530 2023-10-06T03:57:19.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.449+0000 7fe439461700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe43406f800 con 0x7fe43415e9a0 2023-10-06T03:57:19.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.449+0000 7fe439461700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe434155130 con 0x7fe43415e160 2023-10-06T03:57:19.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.449+0000 7f57437fe700 1 --2- 172.21.15.157:0/1289536747 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5730067030 0x7f57300694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.449+0000 7fe4327fc700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe43415e530 0x7fe434069410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40070/0 (socket says 172.21.15.157:40070) 2023-10-06T03:57:19.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.449+0000 7fe4327fc700 1 -- 172.21.15.157:0/1990164867 learned_addr learned my addr 172.21.15.157:0/1990164867 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:19.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.449+0000 7fe432ffd700 1 --2- 172.21.15.157:0/1990164867 >> v2:172.21.15.157:3300/0 conn(0x7fe43415e160 0x7fe4340c5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.449+0000 7f113468a700 1 -- 172.21.15.157:0/1567426394 shutdown_connections 2023-10-06T03:57:19.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.449+0000 7f113468a700 1 --2- 172.21.15.157:0/1567426394 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f112c06c440 0x7f112c06e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.449+0000 7f113468a700 1 --2- 172.21.15.157:0/1567426394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f112c06c810 0x7f112c069410 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.449+0000 7f113468a700 1 -- 172.21.15.157:0/1567426394 >> 172.21.15.157:0/1567426394 conn(0x7f112c0c82e0 msgr2=0x7f112c0c86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.449+0000 7fe432ffd700 1 -- 172.21.15.157:0/1990164867 >> v1:172.21.15.175:6789/0 conn(0x7fe43415e9a0 legacy=0x7fe4340ad930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.449+0000 7fe432ffd700 1 -- 172.21.15.157:0/1990164867 >> v1:172.21.15.157:6789/0 conn(0x7fe43415e530 legacy=0x7fe434069410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.449+0000 7fe432ffd700 1 -- 172.21.15.157:0/1990164867 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe43406e950 con 0x7fe43415e160 2023-10-06T03:57:19.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.449+0000 7fe431ffb700 1 -- 172.21.15.157:0/1990164867 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2029064109 0 0) 0x7fe42c001490 con 0x7fe43415e530 2023-10-06T03:57:19.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.450+0000 7f396bd58700 1 Processor -- start 2023-10-06T03:57:19.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.450+0000 7fe432ffd700 1 --2- 172.21.15.157:0/1990164867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe43415e160 0x7fe4340c5460 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe4280098e0 tx=0x7fe4280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3a0756248ad5eb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.450+0000 7f54b8ea2700 1 Processor -- start 2023-10-06T03:57:19.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.450+0000 7f57437fe700 1 --2- 172.21.15.157:0/1289536747 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5730067030 0x7f57300694e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5734009910 tx=0x7f5734017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.450+0000 7f54b8ea2700 1 -- start start 2023-10-06T03:57:19.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.450+0000 7fe431ffb700 1 -- 172.21.15.157:0/1990164867 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe428009e80 con 0x7fe43415e160 2023-10-06T03:57:19.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.450+0000 7fe431ffb700 1 -- 172.21.15.157:0/1990164867 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe42800b860 con 0x7fe43415e160 2023-10-06T03:57:19.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.450+0000 7fe431ffb700 1 -- 172.21.15.157:0/1990164867 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42800bbe0 con 0x7fe43415e160 2023-10-06T03:57:19.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.450+0000 7faefd704700 1 -- 172.21.15.157:0/887854863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faef8068ba0 msgr2=0x7faef815acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.450+0000 7faefd704700 1 --2- 172.21.15.157:0/887854863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faef8068ba0 0x7faef815acc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faef806de90 tx=0x7faef0009780 comp rx=0 tx=0).stop 2023-10-06T03:57:19.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.449+0000 7f248b693700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f248406c440 0x7f248415aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.449+0000 7f248b693700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f24840bd1d0 0x7f248406df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.449+0000 7f248b693700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24840bd840 con 0x7f248406c810 2023-10-06T03:57:19.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.449+0000 7f248b693700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24840bec20 con 0x7f24840bd1d0 2023-10-06T03:57:19.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.449+0000 7f248b693700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24840bd630 con 0x7f248406c440 2023-10-06T03:57:19.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.449+0000 7f248942f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f248406c440 0x7f248415aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.449+0000 7f248942f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f248406c440 0x7f248415aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:50188/0 (socket says 172.21.15.157:50188) 2023-10-06T03:57:19.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.449+0000 7f248942f700 1 -- 172.21.15.157:0/2663563514 learned_addr learned my addr 172.21.15.157:0/2663563514 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:19.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.450+0000 7f248942f700 1 -- 172.21.15.157:0/2663563514 >> v2:172.21.15.175:3300/0 conn(0x7f24840bd1d0 msgr2=0x7f248406df10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T03:57:19.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.450+0000 7f248942f700 1 --2- 172.21.15.157:0/2663563514 >> v2:172.21.15.175:3300/0 conn(0x7f24840bd1d0 0x7f248406df10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.450+0000 7f248942f700 1 -- 172.21.15.157:0/2663563514 >> v1:172.21.15.157:6789/0 conn(0x7f248406c810 legacy=0x7f24840c2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:19.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.450+0000 7f248942f700 1 -- 172.21.15.157:0/2663563514 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24840bdb20 con 0x7f248406c440 2023-10-06T03:57:19.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.450+0000 7f396bd58700 1 -- start start 2023-10-06T03:57:19.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f113468a700 1 -- 172.21.15.157:0/1567426394 shutdown_connections 2023-10-06T03:57:19.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f113468a700 1 -- 172.21.15.157:0/1567426394 wait complete. 2023-10-06T03:57:19.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7faefd704700 1 -- 172.21.15.157:0/887854863 shutdown_connections 2023-10-06T03:57:19.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7faefd704700 1 --2- 172.21.15.157:0/887854863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faef8068ba0 0x7faef815acc0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7faefd704700 1 --2- 172.21.15.157:0/887854863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faef806a530 0x7faef806ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7faefd704700 1 --2- 172.21.15.157:0/887854863 >> v2:172.21.15.157:3300/0 conn(0x7faef806e8c0 0x7faef8158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7faefd704700 1 -- 172.21.15.157:0/887854863 >> 172.21.15.157:0/887854863 conn(0x7faef806b510 msgr2=0x7faef806b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f248942f700 1 --2- 172.21.15.157:0/2663563514 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f248406c440 0x7f248415aef0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2478009f20 tx=0x7f2478009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be3c2251c2f4b207 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7fe439461700 1 -- 172.21.15.157:0/1990164867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe43415e160 msgr2=0x7fe4340c5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7fe439461700 1 --2- 172.21.15.157:0/1990164867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe43415e160 0x7fe4340c5460 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe4280098e0 tx=0x7fe4280091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f396bd58700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f396415e160 0x7f3964158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f396bd58700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f396415e530 0x7f396416c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f396bd58700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f396415e9a0 0x7f39640c15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f396bd58700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396416acd0 con 0x7f396415e530 2023-10-06T03:57:19.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f396bd58700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396416bca0 con 0x7f396415e160 2023-10-06T03:57:19.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f396bd58700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f396416efa0 con 0x7f396415e9a0 2023-10-06T03:57:19.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f54b8ea2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54b4069950 0x7f54b416e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f54b8ea2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54b415e160 0x7f54b415ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f54b8ea2700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54b415e530 0x7f54b4150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f54b8ea2700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b416da00 con 0x7f54b415e160 2023-10-06T03:57:19.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f54b8ea2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b416e920 con 0x7f54b4069950 2023-10-06T03:57:19.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f54b8ea2700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b40bd880 con 0x7f54b415e530 2023-10-06T03:57:19.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f39692f3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f396415e530 0x7f396416c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f39692f3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f396415e530 0x7f396416c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34468/0 (socket says 172.21.15.157:34468) 2023-10-06T03:57:19.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f39692f3700 1 -- 172.21.15.157:0/1387426133 learned_addr learned my addr 172.21.15.157:0/1387426133 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:19.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f3969af4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f396415e160 0x7f3964158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f396a2f5700 1 --2- 172.21.15.157:0/1387426133 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f396415e9a0 0x7f39640c15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f54b259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54b4069950 0x7f54b416e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7faefd704700 1 -- 172.21.15.157:0/887854863 shutdown_connections 2023-10-06T03:57:19.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7fe439461700 1 -- 172.21.15.157:0/1990164867 shutdown_connections 2023-10-06T03:57:19.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7fe439461700 1 --2- 172.21.15.157:0/1990164867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe43415e160 0x7fe4340c5460 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7fe439461700 1 -- 172.21.15.157:0/1990164867 >> 172.21.15.157:0/1990164867 conn(0x7fe4340c8d70 msgr2=0x7fe4340c4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7faefd704700 1 -- 172.21.15.157:0/887854863 wait complete. 2023-10-06T03:57:19.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f54b259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54b4069950 0x7f54b416e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:35180/0 (socket says 172.21.15.157:35180) 2023-10-06T03:57:19.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f54b259c700 1 -- 172.21.15.157:0/1776836632 learned_addr learned my addr 172.21.15.157:0/1776836632 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:19.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f54b2d9d700 1 --2- 172.21.15.157:0/1776836632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54b415e530 0x7f54b4150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f54b1d9b700 1 --2- 172.21.15.157:0/1776836632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54b415e160 0x7f54b415ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f2483fff700 1 -- 172.21.15.157:0/2663563514 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247800e070 con 0x7f248406c440 2023-10-06T03:57:19.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f2483fff700 1 -- 172.21.15.157:0/2663563514 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247800ba80 con 0x7f248406c440 2023-10-06T03:57:19.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f2483fff700 1 -- 172.21.15.157:0/2663563514 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24780092e0 con 0x7f248406c440 2023-10-06T03:57:19.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f3969af4700 1 -- 172.21.15.157:0/1387426133 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f396415e9a0 msgr2=0x7f39640c15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f3969af4700 1 --2- 172.21.15.157:0/1387426133 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f396415e9a0 0x7f39640c15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f3969af4700 1 -- 172.21.15.157:0/1387426133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f396415e530 msgr2=0x7f396416c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f3969af4700 1 --2- 172.21.15.157:0/1387426133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f396415e530 0x7f396416c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.451+0000 7f3969af4700 1 -- 172.21.15.157:0/1387426133 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f395c009ae0 con 0x7f396415e160 2023-10-06T03:57:19.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.452+0000 7f54b1d9b700 1 -- 172.21.15.157:0/1776836632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54b415e530 msgr2=0x7f54b4150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.452+0000 7f54b1d9b700 1 --2- 172.21.15.157:0/1776836632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54b415e530 0x7f54b4150900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.452+0000 7f54b1d9b700 1 -- 172.21.15.157:0/1776836632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54b4069950 msgr2=0x7f54b416e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.452+0000 7f54b1d9b700 1 --2- 172.21.15.157:0/1776836632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54b4069950 0x7f54b416e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.452+0000 7f54b1d9b700 1 -- 172.21.15.157:0/1776836632 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ac009a20 con 0x7f54b415e160 2023-10-06T03:57:19.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.452+0000 7fe439461700 1 -- 172.21.15.157:0/1990164867 shutdown_connections 2023-10-06T03:57:19.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.452+0000 7f3969af4700 1 --2- 172.21.15.157:0/1387426133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f396415e160 0x7f3964158750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f395c014000 tx=0x7f395c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.452+0000 7f54b2d9d700 1 --2- 172.21.15.157:0/1776836632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54b415e530 0x7f54b4150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:19.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.452+0000 7fe439461700 1 -- 172.21.15.157:0/1990164867 wait complete. 2023-10-06T03:57:19.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.452+0000 7f113468a700 1 Processor -- start 2023-10-06T03:57:19.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.452+0000 7f113468a700 1 -- start start 2023-10-06T03:57:19.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.452+0000 7f248b693700 1 -- 172.21.15.157:0/2663563514 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f248406c440 msgr2=0x7f248415aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.452+0000 7f248b693700 1 --2- 172.21.15.157:0/2663563514 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f248406c440 0x7f248415aef0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2478009f20 tx=0x7f2478009840 comp rx=0 tx=0).stop 2023-10-06T03:57:19.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.453+0000 7f248b693700 1 -- 172.21.15.157:0/2663563514 shutdown_connections 2023-10-06T03:57:19.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.453+0000 7f248b693700 1 --2- 172.21.15.157:0/2663563514 >> v2:172.21.15.175:3300/0 conn(0x7f24840bd1d0 0x7f248406df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.453+0000 7f248b693700 1 --2- 172.21.15.157:0/2663563514 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f248406c440 0x7f248415aef0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.453+0000 7f248b693700 1 -- 172.21.15.157:0/2663563514 >> 172.21.15.157:0/2663563514 conn(0x7f24840c4cd0 msgr2=0x7f24840c50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.453+0000 7f54b1d9b700 1 --2- 172.21.15.157:0/1776836632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54b415e160 0x7f54b415ff80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f54a800d8c0 tx=0x7f54a8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.453+0000 7f248b693700 1 -- 172.21.15.157:0/2663563514 shutdown_connections 2023-10-06T03:57:19.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.453+0000 7faefd704700 1 Processor -- start 2023-10-06T03:57:19.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.453+0000 7f248b693700 1 -- 172.21.15.157:0/2663563514 wait complete. 2023-10-06T03:57:19.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.453+0000 7f54a27fc700 1 -- 172.21.15.157:0/1776836632 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a8014620 con 0x7f54b415e160 2023-10-06T03:57:19.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.453+0000 7f54a27fc700 1 -- 172.21.15.157:0/1776836632 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a8014780 con 0x7f54b415e160 2023-10-06T03:57:19.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.453+0000 7f54b8ea2700 1 -- 172.21.15.157:0/1776836632 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b416df50 con 0x7f54b415e160 2023-10-06T03:57:19.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.453+0000 7f54b8ea2700 1 -- 172.21.15.157:0/1776836632 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b40bdab0 con 0x7f54b415e160 2023-10-06T03:57:19.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.453+0000 7f54a27fc700 1 -- 172.21.15.157:0/1776836632 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a8014b40 con 0x7f54b415e160 2023-10-06T03:57:19.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.453+0000 7fe439461700 1 Processor -- start 2023-10-06T03:57:19.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.453+0000 7faefd704700 1 -- start start 2023-10-06T03:57:19.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.454+0000 7f572b7fe700 1 -- 172.21.15.157:0/1289536747 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5738054ef0 con 0x7f574406d460 2023-10-06T03:57:19.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.454+0000 7faefd704700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faef8068ba0 0x7faef8151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.454+0000 7faefd704700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faef806a530 0x7faef814f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.454+0000 7f3959ffb700 1 -- 172.21.15.157:0/1387426133 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c0094e0 con 0x7f396415e160 2023-10-06T03:57:19.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.454+0000 7f3959ffb700 1 -- 172.21.15.157:0/1387426133 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395c0128d0 con 0x7f396415e160 2023-10-06T03:57:19.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.454+0000 7f3959ffb700 1 -- 172.21.15.157:0/1387426133 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c012c70 con 0x7f396415e160 2023-10-06T03:57:19.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.454+0000 7f396bd58700 1 -- 172.21.15.157:0/1387426133 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f396416aee0 con 0x7f396415e160 2023-10-06T03:57:19.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.454+0000 7f396bd58700 1 -- 172.21.15.157:0/1387426133 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f396416b4a0 con 0x7f396415e160 2023-10-06T03:57:19.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.454+0000 7faefd704700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faef814fb70 0x7faef814e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.454+0000 7faefd704700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef816ab30 con 0x7faef814fb70 2023-10-06T03:57:19.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.454+0000 7faefd704700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef80bc790 con 0x7faef8068ba0 2023-10-06T03:57:19.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.454+0000 7faefd704700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef816a6a0 con 0x7faef806a530 2023-10-06T03:57:19.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.454+0000 7faef67fc700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faef806a530 0x7faef814f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.455+0000 7faef67fc700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faef806a530 0x7faef814f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:50220/0 (socket says 172.21.15.157:50220) 2023-10-06T03:57:19.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.455+0000 7faef67fc700 1 -- 172.21.15.157:0/1042522249 learned_addr learned my addr 172.21.15.157:0/1042522249 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:19.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.455+0000 7faef77fe700 1 --2- 172.21.15.157:0/1042522249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faef814fb70 0x7faef814e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.455+0000 7faef77fe700 1 -- 172.21.15.157:0/1042522249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faef806a530 msgr2=0x7faef814f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.455+0000 7faef77fe700 1 --2- 172.21.15.157:0/1042522249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faef806a530 0x7faef814f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.455+0000 7faef77fe700 1 -- 172.21.15.157:0/1042522249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faef8068ba0 msgr2=0x7faef8151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.455+0000 7faef77fe700 1 --2- 172.21.15.157:0/1042522249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faef8068ba0 0x7faef8151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.455+0000 7faef77fe700 1 -- 172.21.15.157:0/1042522249 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef0009a20 con 0x7faef814fb70 2023-10-06T03:57:19.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.457+0000 7faef77fe700 1 --2- 172.21.15.157:0/1042522249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faef814fb70 0x7faef814e750 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7faef8067c80 tx=0x7faef0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.456+0000 7f3959ffb700 1 -- 172.21.15.157:0/1387426133 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f395c01d630 con 0x7f396415e160 2023-10-06T03:57:19.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.456+0000 7f54b8ea2700 1 -- 172.21.15.157:0/1776836632 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54980052b0 con 0x7f54b415e160 2023-10-06T03:57:19.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.454+0000 7f248b693700 1 Processor -- start 2023-10-06T03:57:19.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.455+0000 7f248b693700 1 -- start start 2023-10-06T03:57:19.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.456+0000 7f248b693700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f248406c440 0x7f248415cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.456+0000 7f248b693700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f248406c810 0x7f2484152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.456+0000 7f248b693700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24840bd1d0 0x7f2484153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.456+0000 7f248b693700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248416a490 con 0x7f248406c810 2023-10-06T03:57:19.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.456+0000 7f248b693700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24840ab3c0 con 0x7f248406c440 2023-10-06T03:57:19.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.456+0000 7f248b693700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f248416a680 con 0x7f24840bd1d0 2023-10-06T03:57:19.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.457+0000 7f2488c2e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f248406c810 0x7f2484152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.457+0000 7f2488c2e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f248406c810 0x7f2484152e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34506/0 (socket says 172.21.15.157:34506) 2023-10-06T03:57:19.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.457+0000 7f2488c2e700 1 -- 172.21.15.157:0/3970321600 learned_addr learned my addr 172.21.15.157:0/3970321600 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:19.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.457+0000 7f2489c30700 1 --2- 172.21.15.157:0/3970321600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24840bd1d0 0x7f2484153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.457+0000 7f2489c30700 1 -- 172.21.15.157:0/3970321600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f248406c440 msgr2=0x7f248415cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.457+0000 7f2489c30700 1 --2- 172.21.15.157:0/3970321600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f248406c440 0x7f248415cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.457+0000 7f2489c30700 1 -- 172.21.15.157:0/3970321600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f248406c810 msgr2=0x7f2484152e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.457+0000 7f2489c30700 1 --2- 172.21.15.157:0/3970321600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f248406c810 0x7f2484152e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.457+0000 7f2489c30700 1 -- 172.21.15.157:0/3970321600 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2478009ae0 con 0x7f24840bd1d0 2023-10-06T03:57:19.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.457+0000 7f2489c30700 1 --2- 172.21.15.157:0/3970321600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24840bd1d0 0x7f2484153370 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f247c00b460 tx=0x7f247c006f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.453+0000 7f113468a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f112c069950 0x7f112c154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.453+0000 7f113468a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f112c06c440 0x7f112c154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.453+0000 7f113468a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f112c06c810 0x7f112c13b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.453+0000 7f113468a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112c0ac160 con 0x7f112c06c810 2023-10-06T03:57:19.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.453+0000 7f113468a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112c153d10 con 0x7f112c069950 2023-10-06T03:57:19.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.453+0000 7f113468a700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f112c0ad5a0 con 0x7f112c06c440 2023-10-06T03:57:19.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.453+0000 7f1132c27700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f112c06c810 0x7f112c13b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.453+0000 7f1132c27700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f112c06c810 0x7f112c13b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34488/0 (socket says 172.21.15.157:34488) 2023-10-06T03:57:19.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.453+0000 7f1132c27700 1 -- 172.21.15.157:0/3602065573 learned_addr learned my addr 172.21.15.157:0/3602065573 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:19.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.454+0000 7f1132c27700 1 -- 172.21.15.157:0/3602065573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f112c06c440 msgr2=0x7f112c154a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.454+0000 7f1132c27700 1 --2- 172.21.15.157:0/3602065573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f112c06c440 0x7f112c154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.454+0000 7f1132c27700 1 -- 172.21.15.157:0/3602065573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f112c069950 msgr2=0x7f112c154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.454+0000 7f1132c27700 1 --2- 172.21.15.157:0/3602065573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f112c069950 0x7f112c154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.454+0000 7f1132c27700 1 -- 172.21.15.157:0/3602065573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1128009420 con 0x7f112c06c810 2023-10-06T03:57:19.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.455+0000 7f1132c27700 1 --2- 172.21.15.157:0/3602065573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f112c06c810 0x7f112c13b910 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f112400cbc0 tx=0x7f1124001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.455+0000 7f11227fc700 1 -- 172.21.15.157:0/3602065573 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112400a8e0 con 0x7f112c06c810 2023-10-06T03:57:19.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.455+0000 7f11227fc700 1 -- 172.21.15.157:0/3602065573 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112400aa40 con 0x7f112c06c810 2023-10-06T03:57:19.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.455+0000 7f11227fc700 1 -- 172.21.15.157:0/3602065573 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112400ae00 con 0x7f112c06c810 2023-10-06T03:57:19.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.455+0000 7f113468a700 1 -- 172.21.15.157:0/3602065573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f112c14c130 con 0x7f112c06c810 2023-10-06T03:57:19.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.455+0000 7f113468a700 1 -- 172.21.15.157:0/3602065573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f112c1539a0 con 0x7f112c06c810 2023-10-06T03:57:19.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.456+0000 7f113468a700 1 -- 172.21.15.157:0/3602065573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f112c153800 con 0x7f112c06c810 2023-10-06T03:57:19.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.457+0000 7f11227fc700 1 -- 172.21.15.157:0/3602065573 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1124007300 con 0x7f112c06c810 2023-10-06T03:57:19.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.457+0000 7f11227fc700 1 --2- 172.21.15.157:0/3602065573 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1110066f30 0x7f11100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.454+0000 7fe439461700 1 -- start start 2023-10-06T03:57:19.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.455+0000 7fe439461700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe43415e160 0x7fe434156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.455+0000 7fe439461700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe43415e530 0x7fe4341569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.455+0000 7fe439461700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe43415e9a0 0x7fe43416aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.455+0000 7fe439461700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe434155ff0 con 0x7fe43415e530 2023-10-06T03:57:19.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.455+0000 7fe439461700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe434157c90 con 0x7fe43415e160 2023-10-06T03:57:19.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.455+0000 7fe439461700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe43406d460 con 0x7fe43415e9a0 2023-10-06T03:57:19.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.456+0000 7fe432ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe43415e160 0x7fe434156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.456+0000 7fe432ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe43415e160 0x7fe434156480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:35200/0 (socket says 172.21.15.157:35200) 2023-10-06T03:57:19.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.456+0000 7fe432ffd700 1 -- 172.21.15.157:0/3758789071 learned_addr learned my addr 172.21.15.157:0/3758789071 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:19.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.457+0000 7fe4337fe700 1 --2- 172.21.15.157:0/3758789071 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe43415e9a0 0x7fe43416aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.457+0000 7fe4337fe700 1 -- 172.21.15.157:0/3758789071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe43415e160 msgr2=0x7fe434156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.457+0000 7fe4337fe700 1 --2- 172.21.15.157:0/3758789071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe43415e160 0x7fe434156480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.457+0000 7fe4337fe700 1 -- 172.21.15.157:0/3758789071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe43415e530 msgr2=0x7fe4341569c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:57:19.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.457+0000 7fe4337fe700 1 --2- 172.21.15.157:0/3758789071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe43415e530 0x7fe4341569c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.457+0000 7fe4337fe700 1 -- 172.21.15.157:0/3758789071 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe428009420 con 0x7fe43415e9a0 2023-10-06T03:57:19.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.458+0000 7f24817fa700 1 -- 172.21.15.157:0/3970321600 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c00f510 con 0x7f24840bd1d0 2023-10-06T03:57:19.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.458+0000 7f11227fc700 1 -- 172.21.15.157:0/3602065573 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1124084a20 con 0x7f112c06c810 2023-10-06T03:57:19.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.458+0000 7fe4337fe700 1 --2- 172.21.15.157:0/3758789071 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe43415e9a0 0x7fe43416aa60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe42400c4b0 tx=0x7fe424007e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.458+0000 7f24817fa700 1 -- 172.21.15.157:0/3970321600 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247c00f670 con 0x7f24840bd1d0 2023-10-06T03:57:19.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.458+0000 7f248b693700 1 -- 172.21.15.157:0/3970321600 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f248416a940 con 0x7f24840bd1d0 2023-10-06T03:57:19.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.457+0000 7f3959ffb700 1 --2- 172.21.15.157:0/1387426133 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3948066fb0 0x7f3948069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.457+0000 7f3959ffb700 1 -- 172.21.15.157:0/1387426133 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f395c085b10 con 0x7f396415e160 2023-10-06T03:57:19.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.457+0000 7f396bd58700 1 -- 172.21.15.157:0/1387426133 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39640716b0 con 0x7f396415e160 2023-10-06T03:57:19.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.458+0000 7f24817fa700 1 -- 172.21.15.157:0/3970321600 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c00fa30 con 0x7f24840bd1d0 2023-10-06T03:57:19.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.458+0000 7f248b693700 1 -- 172.21.15.157:0/3970321600 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f248416af00 con 0x7f24840bd1d0 2023-10-06T03:57:19.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.458+0000 7fe41affd700 1 -- 172.21.15.157:0/3758789071 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe424011590 con 0x7fe43415e9a0 2023-10-06T03:57:19.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.458+0000 7fe41affd700 1 -- 172.21.15.157:0/3758789071 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4240116f0 con 0x7fe43415e9a0 2023-10-06T03:57:19.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.458+0000 7fe41affd700 1 -- 172.21.15.157:0/3758789071 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe424011ab0 con 0x7fe43415e9a0 2023-10-06T03:57:19.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.458+0000 7fe439461700 1 -- 172.21.15.157:0/3758789071 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe43416b5e0 con 0x7fe43415e9a0 2023-10-06T03:57:19.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.458+0000 7fe439461700 1 -- 172.21.15.157:0/3758789071 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe43415b280 con 0x7fe43415e9a0 2023-10-06T03:57:19.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.459+0000 7f1132426700 1 --2- 172.21.15.157:0/3602065573 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1110066f30 0x7f11100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.459+0000 7f248b693700 1 -- 172.21.15.157:0/3970321600 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2484068750 con 0x7f24840bd1d0 2023-10-06T03:57:19.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.459+0000 7f24817fa700 1 -- 172.21.15.157:0/3970321600 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f247c01a400 con 0x7f24840bd1d0 2023-10-06T03:57:19.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.459+0000 7f54a27fc700 1 -- 172.21.15.157:0/1776836632 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f54a801d400 con 0x7f54b415e160 2023-10-06T03:57:19.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.460+0000 7fe41affd700 1 -- 172.21.15.157:0/3758789071 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe42401b3b0 con 0x7fe43415e9a0 2023-10-06T03:57:19.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.460+0000 7f54a27fc700 1 --2- 172.21.15.157:0/1776836632 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5494066fb0 0x7f5494069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.460+0000 7f24817fa700 1 --2- 172.21.15.157:0/3970321600 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2468067030 0x7f24680694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.460+0000 7f54a27fc700 1 -- 172.21.15.157:0/1776836632 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f54a8084dc0 con 0x7f54b415e160 2023-10-06T03:57:19.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.460+0000 7f24817fa700 1 -- 172.21.15.157:0/3970321600 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f247c083f30 con 0x7f24840bd1d0 2023-10-06T03:57:19.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.460+0000 7f248942f700 1 --2- 172.21.15.157:0/3970321600 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2468067030 0x7f24680694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.460+0000 7f54b259c700 1 --2- 172.21.15.157:0/1776836632 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5494066fb0 0x7f5494069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.460+0000 7fe41affd700 1 --2- 172.21.15.157:0/3758789071 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe420067000 0x7fe4200694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.461+0000 7fe41affd700 1 -- 172.21.15.157:0/3758789071 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe424083af0 con 0x7fe43415e9a0 2023-10-06T03:57:19.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.461+0000 7f54b259c700 1 --2- 172.21.15.157:0/1776836632 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5494066fb0 0x7f5494069460 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f54a4007ec0 tx=0x7f54a4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.461+0000 7f248942f700 1 --2- 172.21.15.157:0/3970321600 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2468067030 0x7f24680694e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2478009f20 tx=0x7f2478018000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.462+0000 7f39692f3700 1 --2- 172.21.15.157:0/1387426133 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3948066fb0 0x7f3948069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.462+0000 7fe432ffd700 1 --2- 172.21.15.157:0/3758789071 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe420067000 0x7fe4200694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.462+0000 7fe439461700 1 -- 172.21.15.157:0/3758789071 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4140052b0 con 0x7fe43415e9a0 2023-10-06T03:57:19.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.462+0000 7f1132426700 1 --2- 172.21.15.157:0/3602065573 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1110066f30 0x7f11100693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f111c009a60 tx=0x7f111c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.462+0000 7f39692f3700 1 --2- 172.21.15.157:0/1387426133 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3948066fb0 0x7f3948069460 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f39640be480 tx=0x7f3954007f60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.462+0000 7fe432ffd700 1 --2- 172.21.15.157:0/3758789071 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe420067000 0x7fe4200694b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe428009fd0 tx=0x7fe428017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.463+0000 7f3959ffb700 1 -- 172.21.15.157:0/1387426133 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f395c052b80 con 0x7f396415e160 2023-10-06T03:57:19.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.464+0000 7f54a27fc700 1 -- 172.21.15.157:0/1776836632 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f54a8051e30 con 0x7f54b415e160 2023-10-06T03:57:19.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.465+0000 7fe1fd618700 1 Processor -- start 2023-10-06T03:57:19.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.465+0000 7fe1fd618700 1 -- start start 2023-10-06T03:57:19.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.465+0000 7f24817fa700 1 -- 172.21.15.157:0/3970321600 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f247c0511c0 con 0x7f24840bd1d0 2023-10-06T03:57:19.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.465+0000 7fe1fd618700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe1f8155f60 0x7fe1f806cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.466+0000 7fe1fd618700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1f80c1480 con 0x7fe1f8156330 2023-10-06T03:57:19.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.466+0000 7fe1fd618700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1f80c1720 con 0x7fe1f80c8830 2023-10-06T03:57:19.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.466+0000 7fe1fd618700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1f80c1a00 con 0x7fe1f8155f60 2023-10-06T03:57:19.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.466+0000 7fe1f67fc700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe1f8156330 0x7fe1f80c8120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40078/0 (socket says 172.21.15.157:40078) 2023-10-06T03:57:19.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.466+0000 7fe1f67fc700 1 -- 172.21.15.157:0/2299613309 learned_addr learned my addr 172.21.15.157:0/2299613309 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:19.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.466+0000 7fe1f6ffd700 1 --2- 172.21.15.157:0/2299613309 >> v2:172.21.15.157:3300/0 conn(0x7fe1f8155f60 0x7fe1f806cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.466+0000 7fe1f6ffd700 1 -- 172.21.15.157:0/2299613309 >> v1:172.21.15.175:6789/0 conn(0x7fe1f80c8830 legacy=0x7fe1f814f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:19.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.466+0000 7fe1f6ffd700 1 -- 172.21.15.157:0/2299613309 >> v1:172.21.15.157:6789/0 conn(0x7fe1f8156330 legacy=0x7fe1f80c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.466+0000 7fe1f6ffd700 1 -- 172.21.15.157:0/2299613309 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f80c1ce0 con 0x7fe1f8155f60 2023-10-06T03:57:19.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.466+0000 7fe1f6ffd700 1 --2- 172.21.15.157:0/2299613309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1f8155f60 0x7fe1f806cc00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe1e80098e0 tx=0x7fe1e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6754c138bd617611 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.470+0000 7fe41affd700 1 -- 172.21.15.157:0/3758789071 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe424017020 con 0x7fe43415e9a0 2023-10-06T03:57:19.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.471+0000 7fe1f5ffb700 1 -- 172.21.15.157:0/2299613309 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8009e80 con 0x7fe1f8155f60 2023-10-06T03:57:19.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.471+0000 7fe1f5ffb700 1 -- 172.21.15.157:0/2299613309 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e800b820 con 0x7fe1f8155f60 2023-10-06T03:57:19.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.471+0000 7fe1f5ffb700 1 -- 172.21.15.157:0/2299613309 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e800bba0 con 0x7fe1f8155f60 2023-10-06T03:57:19.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.471+0000 7f11227fc700 1 -- 172.21.15.157:0/3602065573 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f112c153800 con 0x7f112c06c810 2023-10-06T03:57:19.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.472+0000 7fe1fd618700 1 -- 172.21.15.157:0/2299613309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1f8155f60 msgr2=0x7fe1f806cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.472+0000 7fe1fd618700 1 --2- 172.21.15.157:0/2299613309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1f8155f60 0x7fe1f806cc00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe1e80098e0 tx=0x7fe1e80091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.472+0000 7fe1fd618700 1 -- 172.21.15.157:0/2299613309 shutdown_connections 2023-10-06T03:57:19.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.472+0000 7fe1fd618700 1 --2- 172.21.15.157:0/2299613309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1f8155f60 0x7fe1f806cc00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.472+0000 7fe1fd618700 1 -- 172.21.15.157:0/2299613309 >> 172.21.15.157:0/2299613309 conn(0x7fe1f806dc70 msgr2=0x7fe1f806e070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.472+0000 7fe1fd618700 1 -- 172.21.15.157:0/2299613309 shutdown_connections 2023-10-06T03:57:19.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.472+0000 7fe1fd618700 1 -- 172.21.15.157:0/2299613309 wait complete. 2023-10-06T03:57:19.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.474+0000 7faedeffd700 1 -- 172.21.15.157:0/1042522249 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef0027470 con 0x7faef814fb70 2023-10-06T03:57:19.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.474+0000 7faedeffd700 1 -- 172.21.15.157:0/1042522249 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef0027610 con 0x7faef814fb70 2023-10-06T03:57:19.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.474+0000 7faedeffd700 1 -- 172.21.15.157:0/1042522249 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef00279d0 con 0x7faef814fb70 2023-10-06T03:57:19.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.474+0000 7faefd704700 1 -- 172.21.15.157:0/1042522249 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef80bc4b0 con 0x7faef814fb70 2023-10-06T03:57:19.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.474+0000 7faefd704700 1 -- 172.21.15.157:0/1042522249 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef816a050 con 0x7faef814fb70 2023-10-06T03:57:19.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.474+0000 7fe1fd618700 1 Processor -- start 2023-10-06T03:57:19.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.475+0000 7fe1fd618700 1 -- start start 2023-10-06T03:57:19.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.476+0000 7faedeffd700 1 -- 172.21.15.157:0/1042522249 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faef0024070 con 0x7faef814fb70 2023-10-06T03:57:19.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.476+0000 7faefd704700 1 -- 172.21.15.157:0/1042522249 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faef80692b0 con 0x7faef814fb70 2023-10-06T03:57:19.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.477+0000 7faedeffd700 1 --2- 172.21.15.157:0/1042522249 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faee4067000 0x7faee40694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.477+0000 7faedeffd700 1 -- 172.21.15.157:0/1042522249 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7faef0086d00 con 0x7faef814fb70 2023-10-06T03:57:19.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.476+0000 7fe1fd618700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1f80c8830 0x7fe1f80cb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.476+0000 7fe1fd618700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1f8155f60 0x7fe1f80cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.476+0000 7fe1fd618700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1f8156330 0x7fe1f80cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.476+0000 7fe1fd618700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f816a7d0 con 0x7fe1f80c8830 2023-10-06T03:57:19.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.476+0000 7fe1fd618700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f80cad80 con 0x7fe1f8155f60 2023-10-06T03:57:19.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.476+0000 7fe1fd618700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f816a280 con 0x7fe1f8156330 2023-10-06T03:57:19.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.477+0000 7fe1f77fe700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1f8156330 0x7fe1f80cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.477+0000 7fe1f77fe700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1f8156330 0x7fe1f80cc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:50236/0 (socket says 172.21.15.157:50236) 2023-10-06T03:57:19.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.477+0000 7fe1f77fe700 1 -- 172.21.15.157:0/2108172346 learned_addr learned my addr 172.21.15.157:0/2108172346 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:19.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.478+0000 7fe1f77fe700 1 -- 172.21.15.157:0/2108172346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1f8155f60 msgr2=0x7fe1f80cb750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:57:19.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.478+0000 7fe1f77fe700 1 --2- 172.21.15.157:0/2108172346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1f8155f60 0x7fe1f80cb750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.478+0000 7fe1f77fe700 1 -- 172.21.15.157:0/2108172346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1f80c8830 msgr2=0x7fe1f80cb210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:57:19.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.478+0000 7fe1f77fe700 1 --2- 172.21.15.157:0/2108172346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1f80c8830 0x7fe1f80cb210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.478+0000 7fe1f77fe700 1 -- 172.21.15.157:0/2108172346 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e8009420 con 0x7fe1f8156330 2023-10-06T03:57:19.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.478+0000 7fe1f77fe700 1 --2- 172.21.15.157:0/2108172346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1f8156330 0x7fe1f80cc400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000c8c0 tx=0x7fe1f00026a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.485+0000 7fe1deffd700 1 -- 172.21.15.157:0/2108172346 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f000ce60 con 0x7fe1f8156330 2023-10-06T03:57:19.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.485+0000 7fe1deffd700 1 -- 172.21.15.157:0/2108172346 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f000a7a0 con 0x7fe1f8156330 2023-10-06T03:57:19.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.485+0000 7fe1deffd700 1 -- 172.21.15.157:0/2108172346 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f000ab40 con 0x7fe1f8156330 2023-10-06T03:57:19.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.485+0000 7fe1fd618700 1 -- 172.21.15.157:0/2108172346 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f8154ba0 con 0x7fe1f8156330 2023-10-06T03:57:19.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.485+0000 7fe1fd618700 1 -- 172.21.15.157:0/2108172346 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f80ca450 con 0x7fe1f8156330 2023-10-06T03:57:19.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.487+0000 7fe1fd618700 1 -- 172.21.15.157:0/2108172346 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1f806bd00 con 0x7fe1f8156330 2023-10-06T03:57:19.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.489+0000 7faef6ffd700 1 --2- 172.21.15.157:0/1042522249 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faee4067000 0x7faee40694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.489+0000 7faedeffd700 1 -- 172.21.15.157:0/1042522249 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faef0053e90 con 0x7faef814fb70 2023-10-06T03:57:19.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.491+0000 7fe1deffd700 1 -- 172.21.15.157:0/2108172346 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe1f0012420 con 0x7fe1f8156330 2023-10-06T03:57:19.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.491+0000 7fe1deffd700 1 --2- 172.21.15.157:0/2108172346 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe1e4067080 0x7fe1e4069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:19.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.491+0000 7fe1deffd700 1 -- 172.21.15.157:0/2108172346 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe1f0084fc0 con 0x7fe1f8156330 2023-10-06T03:57:19.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.493+0000 7fe1f6ffd700 1 --2- 172.21.15.157:0/2108172346 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe1e4067080 0x7fe1e4069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:19.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.494+0000 7fe1deffd700 1 -- 172.21.15.157:0/2108172346 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe1f0052130 con 0x7fe1f8156330 2023-10-06T03:57:19.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.525+0000 7faef6ffd700 1 --2- 172.21.15.157:0/1042522249 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faee4067000 0x7faee40694b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faeec009a60 tx=0x7faeec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.548+0000 7fe1f6ffd700 1 --2- 172.21.15.157:0/2108172346 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe1e4067080 0x7fe1e4069530 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800c010 tx=0x7fe1e8016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:19.845 INFO:teuthology.orchestra.run.smithi157.stdout:55834574852 2023-10-06T03:57:19.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.842+0000 7f574a17a700 1 -- 172.21.15.157:0/1289536747 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f57440716b0 con 0x7f574406d460 2023-10-06T03:57:19.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.843+0000 7f572b7fe700 1 -- 172.21.15.157:0/1289536747 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5738054a80 con 0x7f574406d460 2023-10-06T03:57:19.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.843+0000 7fe439461700 1 -- 172.21.15.157:0/3758789071 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fe414005830 con 0x7fe43415e9a0 2023-10-06T03:57:19.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.844+0000 7fe41affd700 1 -- 172.21.15.157:0/3758789071 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe424050cc0 con 0x7fe43415e9a0 2023-10-06T03:57:19.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.847+0000 7f54b8ea2700 1 -- 172.21.15.157:0/1776836632 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5498005560 con 0x7f54b415e160 2023-10-06T03:57:19.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.847+0000 7f574a17a700 1 -- 172.21.15.157:0/1289536747 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5730067030 msgr2=0x7f57300694e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.847+0000 7f574a17a700 1 --2- 172.21.15.157:0/1289536747 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5730067030 0x7f57300694e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5734009910 tx=0x7f5734017000 comp rx=0 tx=0).stop 2023-10-06T03:57:19.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.847+0000 7f574a17a700 1 -- 172.21.15.157:0/1289536747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f574406d460 msgr2=0x7f574414baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.847+0000 7f574a17a700 1 --2- 172.21.15.157:0/1289536747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f574406d460 0x7f574414baa0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f573800c880 tx=0x7f5738008040 comp rx=0 tx=0).stop 2023-10-06T03:57:19.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.847+0000 7f574a17a700 1 -- 172.21.15.157:0/1289536747 shutdown_connections 2023-10-06T03:57:19.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.847+0000 7f574a17a700 1 --2- 172.21.15.157:0/1289536747 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5730067030 0x7f57300694e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.847+0000 7f574a17a700 1 --2- 172.21.15.157:0/1289536747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57440be500 0x7f574414bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.847+0000 7f574a17a700 1 --2- 172.21.15.157:0/1289536747 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5744069410 0x7f574415aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.847+0000 7f574a17a700 1 --2- 172.21.15.157:0/1289536747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f574406d460 0x7f574414baa0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.847+0000 7f574a17a700 1 -- 172.21.15.157:0/1289536747 >> 172.21.15.157:0/1289536747 conn(0x7f57440c8d70 msgr2=0x7f57440c4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.847+0000 7f574a17a700 1 -- 172.21.15.157:0/1289536747 shutdown_connections 2023-10-06T03:57:19.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.847+0000 7f574a17a700 1 -- 172.21.15.157:0/1289536747 wait complete. 2023-10-06T03:57:19.851 INFO:teuthology.orchestra.run.smithi157.stdout:55834574852 2023-10-06T03:57:19.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.847+0000 7fe439461700 1 -- 172.21.15.157:0/3758789071 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe420067000 msgr2=0x7fe4200694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.847+0000 7fe439461700 1 --2- 172.21.15.157:0/3758789071 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe420067000 0x7fe4200694b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe428009fd0 tx=0x7fe428017000 comp rx=0 tx=0).stop 2023-10-06T03:57:19.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.847+0000 7fe439461700 1 -- 172.21.15.157:0/3758789071 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe43415e9a0 msgr2=0x7fe43416aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.847+0000 7fe439461700 1 --2- 172.21.15.157:0/3758789071 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe43415e9a0 0x7fe43416aa60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe42400c4b0 tx=0x7fe424007e60 comp rx=0 tx=0).stop 2023-10-06T03:57:19.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.847+0000 7fe439461700 1 -- 172.21.15.157:0/3758789071 shutdown_connections 2023-10-06T03:57:19.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.847+0000 7fe439461700 1 --2- 172.21.15.157:0/3758789071 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe420067000 0x7fe4200694b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.847+0000 7fe439461700 1 --2- 172.21.15.157:0/3758789071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe43415e160 0x7fe434156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.847+0000 7fe439461700 1 --2- 172.21.15.157:0/3758789071 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe43415e9a0 0x7fe43416aa60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.847+0000 7fe439461700 1 --2- 172.21.15.157:0/3758789071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe43415e530 0x7fe4341569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.847+0000 7fe439461700 1 -- 172.21.15.157:0/3758789071 >> 172.21.15.157:0/3758789071 conn(0x7fe4340c8d70 msgr2=0x7fe43415ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.848+0000 7fe439461700 1 -- 172.21.15.157:0/3758789071 shutdown_connections 2023-10-06T03:57:19.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.848+0000 7fe439461700 1 -- 172.21.15.157:0/3758789071 wait complete. 2023-10-06T03:57:19.852 INFO:teuthology.orchestra.run.smithi157.stdout:55834574852 2023-10-06T03:57:19.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.848+0000 7f54a27fc700 1 -- 172.21.15.157:0/1776836632 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f54a8055450 con 0x7f54b415e160 2023-10-06T03:57:19.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.857+0000 7f248b693700 1 -- 172.21.15.157:0/3970321600 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f24840ab530 con 0x7f24840bd1d0 2023-10-06T03:57:19.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.857+0000 7f396bd58700 1 -- 172.21.15.157:0/1387426133 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f396406fc70 con 0x7f396415e160 2023-10-06T03:57:19.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.857+0000 7f24817fa700 1 -- 172.21.15.157:0/3970321600 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f247c0547e0 con 0x7f24840bd1d0 2023-10-06T03:57:19.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.857+0000 7f3959ffb700 1 -- 172.21.15.157:0/1387426133 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f395c0561a0 con 0x7f396415e160 2023-10-06T03:57:19.859 INFO:teuthology.orchestra.run.smithi157.stdout:55834574852 2023-10-06T03:57:19.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.857+0000 7f54b8ea2700 1 -- 172.21.15.157:0/1776836632 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5494066fb0 msgr2=0x7f5494069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.857+0000 7f54b8ea2700 1 --2- 172.21.15.157:0/1776836632 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5494066fb0 0x7f5494069460 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f54a4007ec0 tx=0x7f54a4009040 comp rx=0 tx=0).stop 2023-10-06T03:57:19.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.857+0000 7f54b8ea2700 1 -- 172.21.15.157:0/1776836632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54b415e160 msgr2=0x7f54b415ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.858+0000 7f54b8ea2700 1 --2- 172.21.15.157:0/1776836632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54b415e160 0x7f54b415ff80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f54a800d8c0 tx=0x7f54a8008040 comp rx=0 tx=0).stop 2023-10-06T03:57:19.863 INFO:teuthology.orchestra.run.smithi157.stdout:55834574852 2023-10-06T03:57:19.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.858+0000 7f54b8ea2700 1 -- 172.21.15.157:0/1776836632 shutdown_connections 2023-10-06T03:57:19.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.858+0000 7f54b8ea2700 1 --2- 172.21.15.157:0/1776836632 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5494066fb0 0x7f5494069460 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f54a4007ec0 tx=0x7f54a4009040 comp rx=0 tx=0).stop 2023-10-06T03:57:19.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.858+0000 7f54b8ea2700 1 --2- 172.21.15.157:0/1776836632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54b4069950 0x7f54b416e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.858+0000 7f54b8ea2700 1 --2- 172.21.15.157:0/1776836632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54b415e530 0x7f54b4150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.858+0000 7f54b8ea2700 1 --2- 172.21.15.157:0/1776836632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54b415e160 0x7f54b415ff80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.858+0000 7f54b8ea2700 1 -- 172.21.15.157:0/1776836632 >> 172.21.15.157:0/1776836632 conn(0x7f54b40c8d70 msgr2=0x7f54b415d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.862+0000 7f248b693700 1 -- 172.21.15.157:0/3970321600 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2468067030 msgr2=0x7f24680694e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.862+0000 7f248b693700 1 --2- 172.21.15.157:0/3970321600 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2468067030 0x7f24680694e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2478009f20 tx=0x7f2478018000 comp rx=0 tx=0).stop 2023-10-06T03:57:19.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.862+0000 7f248b693700 1 -- 172.21.15.157:0/3970321600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24840bd1d0 msgr2=0x7f2484153370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.862+0000 7f248b693700 1 --2- 172.21.15.157:0/3970321600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24840bd1d0 0x7f2484153370 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f247c00b460 tx=0x7f247c006f40 comp rx=0 tx=0).stop 2023-10-06T03:57:19.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.862+0000 7f54b8ea2700 1 -- 172.21.15.157:0/1776836632 shutdown_connections 2023-10-06T03:57:19.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.862+0000 7f248b693700 1 -- 172.21.15.157:0/3970321600 shutdown_connections 2023-10-06T03:57:19.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.862+0000 7f248b693700 1 --2- 172.21.15.157:0/3970321600 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2468067030 0x7f24680694e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.862+0000 7f248b693700 1 --2- 172.21.15.157:0/3970321600 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f248406c440 0x7f248415cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.862+0000 7f248b693700 1 --2- 172.21.15.157:0/3970321600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24840bd1d0 0x7f2484153370 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.862+0000 7f248b693700 1 --2- 172.21.15.157:0/3970321600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f248406c810 0x7f2484152e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.862+0000 7f248b693700 1 -- 172.21.15.157:0/3970321600 >> 172.21.15.157:0/3970321600 conn(0x7f24840c4cd0 msgr2=0x7f2484159420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.862+0000 7f396bd58700 1 -- 172.21.15.157:0/1387426133 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3948066fb0 msgr2=0x7f3948069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.862+0000 7f396bd58700 1 --2- 172.21.15.157:0/1387426133 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3948066fb0 0x7f3948069460 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f39640be480 tx=0x7f3954007f60 comp rx=0 tx=0).stop 2023-10-06T03:57:19.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.862+0000 7f396bd58700 1 -- 172.21.15.157:0/1387426133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f396415e160 msgr2=0x7f3964158750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.862+0000 7f396bd58700 1 --2- 172.21.15.157:0/1387426133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f396415e160 0x7f3964158750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f395c014000 tx=0x7f395c017000 comp rx=0 tx=0).stop 2023-10-06T03:57:19.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.862+0000 7f248b693700 1 -- 172.21.15.157:0/3970321600 shutdown_connections 2023-10-06T03:57:19.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.862+0000 7f54b8ea2700 1 -- 172.21.15.157:0/1776836632 wait complete. 2023-10-06T03:57:19.868 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.6 2023-10-06T03:57:19.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.863+0000 7f248b693700 1 -- 172.21.15.157:0/3970321600 wait complete. 2023-10-06T03:57:19.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.866+0000 7f113468a700 1 -- 172.21.15.157:0/3602065573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f112c153800 con 0x7f112c06c810 2023-10-06T03:57:19.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.867+0000 7f11227fc700 1 -- 172.21.15.157:0/3602065573 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f112c153800 con 0x7f112c06c810 2023-10-06T03:57:19.869 INFO:teuthology.orchestra.run.smithi157.stdout:55834574852 2023-10-06T03:57:19.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.867+0000 7f396bd58700 1 -- 172.21.15.157:0/1387426133 shutdown_connections 2023-10-06T03:57:19.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.867+0000 7f396bd58700 1 --2- 172.21.15.157:0/1387426133 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3948066fb0 0x7f3948069460 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.867+0000 7f396bd58700 1 --2- 172.21.15.157:0/1387426133 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f396415e160 0x7f3964158750 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.867+0000 7f396bd58700 1 --2- 172.21.15.157:0/1387426133 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f396415e9a0 0x7f39640c15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.867+0000 7f396bd58700 1 --2- 172.21.15.157:0/1387426133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f396415e530 0x7f396416c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.867+0000 7f396bd58700 1 -- 172.21.15.157:0/1387426133 >> 172.21.15.157:0/1387426133 conn(0x7f39640c8d70 msgr2=0x7f39640c2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.867+0000 7f396bd58700 1 -- 172.21.15.157:0/1387426133 shutdown_connections 2023-10-06T03:57:19.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.867+0000 7f396bd58700 1 -- 172.21.15.157:0/1387426133 wait complete. 2023-10-06T03:57:19.870 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.5 2023-10-06T03:57:19.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.870+0000 7f113468a700 1 -- 172.21.15.157:0/3602065573 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1110066f30 msgr2=0x7f11100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.870+0000 7f113468a700 1 --2- 172.21.15.157:0/3602065573 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1110066f30 0x7f11100693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f111c009a60 tx=0x7f111c008040 comp rx=0 tx=0).stop 2023-10-06T03:57:19.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.870+0000 7f113468a700 1 -- 172.21.15.157:0/3602065573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f112c06c810 msgr2=0x7f112c13b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.870+0000 7f113468a700 1 --2- 172.21.15.157:0/3602065573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f112c06c810 0x7f112c13b910 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f112400cbc0 tx=0x7f1124001970 comp rx=0 tx=0).stop 2023-10-06T03:57:19.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.871+0000 7f113468a700 1 -- 172.21.15.157:0/3602065573 shutdown_connections 2023-10-06T03:57:19.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.871+0000 7f113468a700 1 --2- 172.21.15.157:0/3602065573 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1110066f30 0x7f11100693e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.871+0000 7f113468a700 1 --2- 172.21.15.157:0/3602065573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f112c069950 0x7f112c154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.871+0000 7f113468a700 1 --2- 172.21.15.157:0/3602065573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f112c06c810 0x7f112c13b910 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.871+0000 7f113468a700 1 --2- 172.21.15.157:0/3602065573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f112c06c440 0x7f112c154a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.871+0000 7f113468a700 1 -- 172.21.15.157:0/3602065573 >> 172.21.15.157:0/3602065573 conn(0x7f112c0c82e0 msgr2=0x7f112c06ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.871+0000 7f113468a700 1 -- 172.21.15.157:0/3602065573 shutdown_connections 2023-10-06T03:57:19.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.871+0000 7f113468a700 1 -- 172.21.15.157:0/3602065573 wait complete. 2023-10-06T03:57:19.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.872+0000 7faefd704700 1 -- 172.21.15.157:0/1042522249 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7faef816a850 con 0x7faef814fb70 2023-10-06T03:57:19.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.872+0000 7faedeffd700 1 -- 172.21.15.157:0/1042522249 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faef816a850 con 0x7faef814fb70 2023-10-06T03:57:19.876 INFO:teuthology.orchestra.run.smithi157.stdout:55834574853 2023-10-06T03:57:19.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.875+0000 7faefd704700 1 -- 172.21.15.157:0/1042522249 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faee4067000 msgr2=0x7faee40694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.875+0000 7faefd704700 1 --2- 172.21.15.157:0/1042522249 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faee4067000 0x7faee40694b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faeec009a60 tx=0x7faeec008040 comp rx=0 tx=0).stop 2023-10-06T03:57:19.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.875+0000 7faefd704700 1 -- 172.21.15.157:0/1042522249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faef814fb70 msgr2=0x7faef814e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.875+0000 7faefd704700 1 --2- 172.21.15.157:0/1042522249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faef814fb70 0x7faef814e750 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7faef8067c80 tx=0x7faef0019000 comp rx=0 tx=0).stop 2023-10-06T03:57:19.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.875+0000 7faefd704700 1 -- 172.21.15.157:0/1042522249 shutdown_connections 2023-10-06T03:57:19.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.875+0000 7faefd704700 1 --2- 172.21.15.157:0/1042522249 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faee4067000 0x7faee40694b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.875+0000 7faefd704700 1 --2- 172.21.15.157:0/1042522249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faef8068ba0 0x7faef8151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.875+0000 7faefd704700 1 --2- 172.21.15.157:0/1042522249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faef814fb70 0x7faef814e750 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.875+0000 7faefd704700 1 --2- 172.21.15.157:0/1042522249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faef806a530 0x7faef814f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.875+0000 7faefd704700 1 -- 172.21.15.157:0/1042522249 >> 172.21.15.157:0/1042522249 conn(0x7faef806b510 msgr2=0x7faef8070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.881+0000 7faefd704700 1 -- 172.21.15.157:0/1042522249 shutdown_connections 2023-10-06T03:57:19.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.881+0000 7faefd704700 1 -- 172.21.15.157:0/1042522249 wait complete. 2023-10-06T03:57:19.888 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.7 2023-10-06T03:57:19.889 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.2 2023-10-06T03:57:19.890 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.1 2023-10-06T03:57:19.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.888+0000 7fe1fd618700 1 -- 172.21.15.157:0/2108172346 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe1f80c1fc0 con 0x7fe1f8156330 2023-10-06T03:57:19.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.889+0000 7fe1deffd700 1 -- 172.21.15.157:0/2108172346 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe1f0016020 con 0x7fe1f8156330 2023-10-06T03:57:19.890 INFO:teuthology.orchestra.run.smithi157.stdout:55834574852 2023-10-06T03:57:19.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.892+0000 7fe1fd618700 1 -- 172.21.15.157:0/2108172346 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe1e4067080 msgr2=0x7fe1e4069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.892+0000 7fe1fd618700 1 --2- 172.21.15.157:0/2108172346 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe1e4067080 0x7fe1e4069530 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800c010 tx=0x7fe1e8016000 comp rx=0 tx=0).stop 2023-10-06T03:57:19.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.892+0000 7fe1fd618700 1 -- 172.21.15.157:0/2108172346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1f8156330 msgr2=0x7fe1f80cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:19.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.892+0000 7fe1fd618700 1 --2- 172.21.15.157:0/2108172346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1f8156330 0x7fe1f80cc400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000c8c0 tx=0x7fe1f00026a0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.892+0000 7fe1fd618700 1 -- 172.21.15.157:0/2108172346 shutdown_connections 2023-10-06T03:57:19.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.892+0000 7fe1fd618700 1 --2- 172.21.15.157:0/2108172346 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe1e4067080 0x7fe1e4069530 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.892+0000 7fe1fd618700 1 --2- 172.21.15.157:0/2108172346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1f8156330 0x7fe1f80cc400 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.892+0000 7fe1fd618700 1 --2- 172.21.15.157:0/2108172346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1f80c8830 0x7fe1f80cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.892+0000 7fe1fd618700 1 --2- 172.21.15.157:0/2108172346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1f8155f60 0x7fe1f80cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:19.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.892+0000 7fe1fd618700 1 -- 172.21.15.157:0/2108172346 >> 172.21.15.157:0/2108172346 conn(0x7fe1f806dc70 msgr2=0x7fe1f8156e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:19.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.892+0000 7fe1fd618700 1 -- 172.21.15.157:0/2108172346 shutdown_connections 2023-10-06T03:57:19.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:19.892+0000 7fe1fd618700 1 -- 172.21.15.157:0/2108172346 wait complete. 2023-10-06T03:57:19.896 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.0 2023-10-06T03:57:19.900 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.4 2023-10-06T03:57:19.907 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.3 2023-10-06T03:57:20.868 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T03:57:20.871 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T03:57:20.889 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T03:57:20.890 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T03:57:20.891 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T03:57:20.897 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T03:57:20.901 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T03:57:20.907 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T03:57:21.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.045+0000 7f05ebfd5700 1 Processor -- start 2023-10-06T03:57:21.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.045+0000 7f05ebfd5700 1 -- start start 2023-10-06T03:57:21.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.046+0000 7f05ebfd5700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05e406c440 0x7f05e406e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.046+0000 7f05ebfd5700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f05e406c810 0x7f05e4069410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.046+0000 7f05ebfd5700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f05e4069950 0x7f05e40c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.046+0000 7f05ebfd5700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e414cb50 con 0x7f05e4069950 2023-10-06T03:57:21.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.046+0000 7f05ebfd5700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e40c3a80 con 0x7f05e406c810 2023-10-06T03:57:21.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.046+0000 7f05ebfd5700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e40c37a0 con 0x7f05e406c440 2023-10-06T03:57:21.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.047+0000 7f05e9d71700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05e406c440 0x7f05e406e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.047+0000 7f05e9570700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f05e406c810 0x7f05e4069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.047+0000 7f05e9d71700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05e406c440 0x7f05e406e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:50250/0 (socket says 172.21.15.157:50250) 2023-10-06T03:57:21.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.047+0000 7f05e9d71700 1 -- 172.21.15.157:0/4147506617 learned_addr learned my addr 172.21.15.157:0/4147506617 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:21.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.048+0000 7f05ea572700 1 --2- 172.21.15.157:0/4147506617 >> v2:172.21.15.157:3300/0 conn(0x7f05e4069950 0x7f05e40c2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.048+0000 7f05ea572700 1 -- 172.21.15.157:0/4147506617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05e406c440 msgr2=0x7f05e406e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:21.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.048+0000 7f05ea572700 1 --2- 172.21.15.157:0/4147506617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05e406c440 0x7f05e406e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.048+0000 7f05ea572700 1 -- 172.21.15.157:0/4147506617 >> v2:172.21.15.175:3300/0 conn(0x7f05e406c810 msgr2=0x7f05e4069410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:21.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.048+0000 7f05ea572700 1 --2- 172.21.15.157:0/4147506617 >> v2:172.21.15.175:3300/0 conn(0x7f05e406c810 0x7f05e4069410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.048+0000 7f05ea572700 1 -- 172.21.15.157:0/4147506617 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e4155b60 con 0x7f05e4069950 2023-10-06T03:57:21.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.048+0000 7f05ea572700 1 --2- 172.21.15.157:0/4147506617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05e4069950 0x7f05e40c2c50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f05e4068030 tx=0x7f05e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed367dc3f41f6c1d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:21.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.049+0000 7f05e8d6f700 1 -- 172.21.15.157:0/4147506617 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e00108a0 con 0x7f05e4069950 2023-10-06T03:57:21.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.049+0000 7f05e8d6f700 1 -- 172.21.15.157:0/4147506617 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e0010a00 con 0x7f05e4069950 2023-10-06T03:57:21.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.049+0000 7f05e8d6f700 1 -- 172.21.15.157:0/4147506617 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e0010da0 con 0x7f05e4069950 2023-10-06T03:57:21.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.050+0000 7f05ebfd5700 1 -- 172.21.15.157:0/4147506617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05e4069950 msgr2=0x7f05e40c2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.050+0000 7f05ebfd5700 1 --2- 172.21.15.157:0/4147506617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05e4069950 0x7f05e40c2c50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f05e4068030 tx=0x7f05e00091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.050+0000 7f05ebfd5700 1 -- 172.21.15.157:0/4147506617 shutdown_connections 2023-10-06T03:57:21.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.050+0000 7f05ebfd5700 1 --2- 172.21.15.157:0/4147506617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05e4069950 0x7f05e40c2c50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.050+0000 7f05ebfd5700 1 --2- 172.21.15.157:0/4147506617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05e406c440 0x7f05e406e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.050+0000 7f05ebfd5700 1 --2- 172.21.15.157:0/4147506617 >> v2:172.21.15.175:3300/0 conn(0x7f05e406c810 0x7f05e4069410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.050+0000 7f05ebfd5700 1 -- 172.21.15.157:0/4147506617 >> 172.21.15.157:0/4147506617 conn(0x7f05e40c82e0 msgr2=0x7f05e40c86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:21.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.050+0000 7f05ebfd5700 1 -- 172.21.15.157:0/4147506617 shutdown_connections 2023-10-06T03:57:21.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.051+0000 7f05ebfd5700 1 -- 172.21.15.157:0/4147506617 wait complete. 2023-10-06T03:57:21.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.052+0000 7f05ebfd5700 1 Processor -- start 2023-10-06T03:57:21.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.052+0000 7f05ebfd5700 1 -- start start 2023-10-06T03:57:21.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.053+0000 7f05ebfd5700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05e4069950 0x7f05e414d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.053+0000 7f05ebfd5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05e406c440 0x7f05e40c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.053+0000 7f05ebfd5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05e406c810 0x7f05e4145cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.053+0000 7f05ebfd5700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e416d9e0 con 0x7f05e406c440 2023-10-06T03:57:21.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.053+0000 7f05ebfd5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e416e280 con 0x7f05e406c810 2023-10-06T03:57:21.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.053+0000 7f05ebfd5700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e4154160 con 0x7f05e4069950 2023-10-06T03:57:21.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.054+0000 7f05e9570700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05e406c440 0x7f05e40c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.054+0000 7f05e9570700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05e406c440 0x7f05e40c6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34534/0 (socket says 172.21.15.157:34534) 2023-10-06T03:57:21.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.054+0000 7f05e9570700 1 -- 172.21.15.157:0/1335511473 learned_addr learned my addr 172.21.15.157:0/1335511473 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:21.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.054+0000 7f05ea572700 1 --2- 172.21.15.157:0/1335511473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05e406c810 0x7f05e4145cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.055+0000 7f05e9570700 1 -- 172.21.15.157:0/1335511473 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05e4069950 msgr2=0x7f05e414d820 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T03:57:21.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.055+0000 7f05e9570700 1 --2- 172.21.15.157:0/1335511473 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05e4069950 0x7f05e414d820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.055+0000 7f05e9570700 1 -- 172.21.15.157:0/1335511473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05e406c810 msgr2=0x7f05e4145cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.055+0000 7f05e9570700 1 --2- 172.21.15.157:0/1335511473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05e406c810 0x7f05e4145cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.055+0000 7f05e9570700 1 -- 172.21.15.157:0/1335511473 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e0009420 con 0x7f05e406c440 2023-10-06T03:57:21.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.055+0000 7f05e9570700 1 --2- 172.21.15.157:0/1335511473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05e406c440 0x7f05e40c6d70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f05d400ecb0 tx=0x7f05d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:21.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.056+0000 7f05d9ffb700 1 -- 172.21.15.157:0/1335511473 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d400c920 con 0x7f05e406c440 2023-10-06T03:57:21.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.056+0000 7f05d9ffb700 1 -- 172.21.15.157:0/1335511473 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d400ca80 con 0x7f05e406c440 2023-10-06T03:57:21.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.056+0000 7f05d9ffb700 1 -- 172.21.15.157:0/1335511473 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d400ce40 con 0x7f05e406c440 2023-10-06T03:57:21.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.056+0000 7f05ebfd5700 1 -- 172.21.15.157:0/1335511473 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e4154440 con 0x7f05e406c440 2023-10-06T03:57:21.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.056+0000 7f05ebfd5700 1 -- 172.21.15.157:0/1335511473 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e416df10 con 0x7f05e406c440 2023-10-06T03:57:21.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.060+0000 7f05d9ffb700 1 -- 172.21.15.157:0/1335511473 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f05d4013680 con 0x7f05e406c440 2023-10-06T03:57:21.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.060+0000 7f05d9ffb700 1 --2- 172.21.15.157:0/1335511473 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f05d00671e0 0x7f05d0069690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.061+0000 7f05d9ffb700 1 -- 172.21.15.157:0/1335511473 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f05d40849e0 con 0x7f05e406c440 2023-10-06T03:57:21.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.061+0000 7f05ebfd5700 1 -- 172.21.15.157:0/1335511473 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05e40c4770 con 0x7f05e406c440 2023-10-06T03:57:21.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.070+0000 7f05e9d71700 1 --2- 172.21.15.157:0/1335511473 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f05d00671e0 0x7f05d0069690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.070+0000 7f05d9ffb700 1 -- 172.21.15.157:0/1335511473 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05d4051af0 con 0x7f05e406c440 2023-10-06T03:57:21.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.073+0000 7f05e9d71700 1 --2- 172.21.15.157:0/1335511473 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f05d00671e0 0x7f05d0069690 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f05dc0072c0 tx=0x7f05dc0076f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:21.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.106+0000 7ffa3d58c700 1 Processor -- start 2023-10-06T03:57:21.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.106+0000 7ffa3d58c700 1 -- start start 2023-10-06T03:57:21.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.107+0000 7ffa3d58c700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa3806c810 0x7ffa3806e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.107+0000 7ffa3d58c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa380c2c50 0x7ffa380c30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.107+0000 7ffa3d58c700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa3814cb50 con 0x7ffa3806c440 2023-10-06T03:57:21.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.107+0000 7ffa3d58c700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa380c3a80 con 0x7ffa380c2c50 2023-10-06T03:57:21.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.107+0000 7ffa3d58c700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa380c37a0 con 0x7ffa3806c810 2023-10-06T03:57:21.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.107+0000 7ffa377fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa380c2c50 0x7ffa380c30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.107+0000 7ffa377fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa380c2c50 0x7ffa380c30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34540/0 (socket says 172.21.15.157:34540) 2023-10-06T03:57:21.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.107+0000 7ffa377fe700 1 -- 172.21.15.157:0/3740651182 learned_addr learned my addr 172.21.15.157:0/3740651182 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:21.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.108+0000 7ffa367fc700 1 --2- 172.21.15.157:0/3740651182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa3806c810 0x7ffa3806e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.108+0000 7ffa377fe700 1 -- 172.21.15.157:0/3740651182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa3806c810 msgr2=0x7ffa3806e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:21.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.108+0000 7ffa377fe700 1 --2- 172.21.15.157:0/3740651182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa3806c810 0x7ffa3806e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.108+0000 7ffa377fe700 1 -- 172.21.15.157:0/3740651182 >> v1:172.21.15.175:6789/0 conn(0x7ffa3806c440 legacy=0x7ffa38069410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.108+0000 7ffa377fe700 1 -- 172.21.15.157:0/3740651182 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa38155b60 con 0x7ffa380c2c50 2023-10-06T03:57:21.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.108+0000 7ffa377fe700 1 --2- 172.21.15.157:0/3740651182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa380c2c50 0x7ffa380c30f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ffa30009d40 tx=0x7ffa30009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f125a4e7f84f9c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:21.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.108+0000 7ffa35ffb700 1 -- 172.21.15.157:0/3740651182 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3000dc20 con 0x7ffa380c2c50 2023-10-06T03:57:21.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.108+0000 7ffa35ffb700 1 -- 172.21.15.157:0/3740651182 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3000dd80 con 0x7ffa380c2c50 2023-10-06T03:57:21.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.108+0000 7ffa35ffb700 1 -- 172.21.15.157:0/3740651182 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3000f670 con 0x7ffa380c2c50 2023-10-06T03:57:21.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.109+0000 7ffa3d58c700 1 -- 172.21.15.157:0/3740651182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa380c2c50 msgr2=0x7ffa380c30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.109+0000 7ffa3d58c700 1 --2- 172.21.15.157:0/3740651182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa380c2c50 0x7ffa380c30f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ffa30009d40 tx=0x7ffa30009f90 comp rx=0 tx=0).stop 2023-10-06T03:57:21.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.110+0000 7ffa3d58c700 1 -- 172.21.15.157:0/3740651182 shutdown_connections 2023-10-06T03:57:21.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.110+0000 7ffa3d58c700 1 --2- 172.21.15.157:0/3740651182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa380c2c50 0x7ffa380c30f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.110+0000 7ffa3d58c700 1 --2- 172.21.15.157:0/3740651182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa3806c810 0x7ffa3806e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.110+0000 7ffa3d58c700 1 -- 172.21.15.157:0/3740651182 >> 172.21.15.157:0/3740651182 conn(0x7ffa380c82e0 msgr2=0x7ffa380c86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:21.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.110+0000 7ffa3d58c700 1 -- 172.21.15.157:0/3740651182 shutdown_connections 2023-10-06T03:57:21.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.110+0000 7ffa3d58c700 1 -- 172.21.15.157:0/3740651182 wait complete. 2023-10-06T03:57:21.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.112+0000 7ffa3d58c700 1 Processor -- start 2023-10-06T03:57:21.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.112+0000 7f1a0dc0e700 1 Processor -- start 2023-10-06T03:57:21.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.112+0000 7f1a0dc0e700 1 -- start start 2023-10-06T03:57:21.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.110+0000 7fde4ae83700 1 Processor -- start 2023-10-06T03:57:21.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.111+0000 7fde4ae83700 1 -- start start 2023-10-06T03:57:21.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.112+0000 7fde4ae83700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde4406ab50 0x7fde4406d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.112+0000 7fde4ae83700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fde4406ddf0 0x7fde4406e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.112+0000 7fde4ae83700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde440c6160 con 0x7fde4406a780 2023-10-06T03:57:21.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.112+0000 7fde4ae83700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde440c5c00 con 0x7fde4406ddf0 2023-10-06T03:57:21.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.112+0000 7fde4ae83700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde44069040 con 0x7fde4406ab50 2023-10-06T03:57:21.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.112+0000 7fde3bfff700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde4406ab50 0x7fde4406d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.113+0000 7fde3bfff700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde4406ab50 0x7fde4406d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:50270/0 (socket says 172.21.15.157:50270) 2023-10-06T03:57:21.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.113+0000 7fde3bfff700 1 -- 172.21.15.157:0/536276618 learned_addr learned my addr 172.21.15.157:0/536276618 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:21.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.113+0000 7fde3bfff700 1 -- 172.21.15.157:0/536276618 >> v2:172.21.15.157:3300/0 conn(0x7fde4406ddf0 msgr2=0x7fde4406e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:21.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.113+0000 7fde3bfff700 1 --2- 172.21.15.157:0/536276618 >> v2:172.21.15.157:3300/0 conn(0x7fde4406ddf0 0x7fde4406e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.113+0000 7fde3bfff700 1 -- 172.21.15.157:0/536276618 >> v1:172.21.15.175:6789/0 conn(0x7fde4406a780 legacy=0x7fde440c65f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T03:57:21.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.113+0000 7fde3bfff700 1 -- 172.21.15.157:0/536276618 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde44153930 con 0x7fde4406ab50 2023-10-06T03:57:21.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.112+0000 7ffa3d58c700 1 -- start start 2023-10-06T03:57:21.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.113+0000 7ffa3d58c700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa3806c440 0x7ffa3814bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.113+0000 7ffa3d58c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa3806c810 0x7ffa381445e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.113+0000 7ffa3d58c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa380c2c50 0x7ffa38154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.113+0000 7ffa3d58c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3816ac60 con 0x7ffa380c2c50 2023-10-06T03:57:21.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.113+0000 7ffa3d58c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3816af40 con 0x7ffa3806c810 2023-10-06T03:57:21.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.113+0000 7ffa3d58c700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3816b220 con 0x7ffa3806c440 2023-10-06T03:57:21.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.113+0000 7ffa36ffd700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa3806c440 0x7ffa3814bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.113+0000 7ffa367fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa3806c810 0x7ffa381445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.113+0000 7ffa367fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa3806c810 0x7ffa381445e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:35240/0 (socket says 172.21.15.157:35240) 2023-10-06T03:57:21.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.113+0000 7ffa367fc700 1 -- 172.21.15.157:0/703356541 learned_addr learned my addr 172.21.15.157:0/703356541 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:21.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.113+0000 7fde3bfff700 1 --2- 172.21.15.157:0/536276618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde4406ab50 0x7fde4406d8b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fde400099b0 tx=0x7fde40009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f2b5b02f21ca28e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:21.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.114+0000 7fde3b7fe700 1 -- 172.21.15.157:0/536276618 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4000b820 con 0x7fde4406ab50 2023-10-06T03:57:21.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.114+0000 7fde3b7fe700 1 -- 172.21.15.157:0/536276618 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4000b980 con 0x7fde4406ab50 2023-10-06T03:57:21.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.114+0000 7fde3b7fe700 1 -- 172.21.15.157:0/536276618 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4000bd20 con 0x7fde4406ab50 2023-10-06T03:57:21.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.115+0000 7fde4ae83700 1 -- 172.21.15.157:0/536276618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde4406ab50 msgr2=0x7fde4406d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.115+0000 7fde4ae83700 1 --2- 172.21.15.157:0/536276618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde4406ab50 0x7fde4406d8b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fde400099b0 tx=0x7fde40009250 comp rx=0 tx=0).stop 2023-10-06T03:57:21.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.114+0000 7ffa367fc700 1 -- 172.21.15.157:0/703356541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa3806c440 msgr2=0x7ffa3814bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.114+0000 7ffa367fc700 1 --2- 172.21.15.157:0/703356541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa3806c440 0x7ffa3814bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.114+0000 7ffa367fc700 1 -- 172.21.15.157:0/703356541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa380c2c50 msgr2=0x7ffa38154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.114+0000 7ffa367fc700 1 --2- 172.21.15.157:0/703356541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa380c2c50 0x7ffa38154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.114+0000 7ffa367fc700 1 -- 172.21.15.157:0/703356541 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa30009850 con 0x7ffa3806c810 2023-10-06T03:57:21.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.115+0000 7fde4ae83700 1 -- 172.21.15.157:0/536276618 shutdown_connections 2023-10-06T03:57:21.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.115+0000 7fde4ae83700 1 --2- 172.21.15.157:0/536276618 >> v2:172.21.15.157:3300/0 conn(0x7fde4406ddf0 0x7fde4406e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.115+0000 7fde4ae83700 1 --2- 172.21.15.157:0/536276618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde4406ab50 0x7fde4406d8b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.115+0000 7fde4ae83700 1 -- 172.21.15.157:0/536276618 >> 172.21.15.157:0/536276618 conn(0x7fde4406b510 msgr2=0x7fde4406b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:21.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.115+0000 7fde4ae83700 1 -- 172.21.15.157:0/536276618 shutdown_connections 2023-10-06T03:57:21.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.115+0000 7fde4ae83700 1 -- 172.21.15.157:0/536276618 wait complete. 2023-10-06T03:57:21.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.116+0000 7ffa367fc700 1 --2- 172.21.15.157:0/703356541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa3806c810 0x7ffa381445e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c00e930 tx=0x7ffa2c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:21.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.116+0000 7ffa1effd700 1 -- 172.21.15.157:0/703356541 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c013680 con 0x7ffa3806c810 2023-10-06T03:57:21.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.116+0000 7ffa1effd700 1 -- 172.21.15.157:0/703356541 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa2c0137e0 con 0x7ffa3806c810 2023-10-06T03:57:21.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.117+0000 7fde4ae83700 1 Processor -- start 2023-10-06T03:57:21.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.117+0000 7fbf735c3700 1 Processor -- start 2023-10-06T03:57:21.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.117+0000 7ffa1effd700 1 -- 172.21.15.157:0/703356541 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c013ba0 con 0x7ffa3806c810 2023-10-06T03:57:21.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.117+0000 7ffa3d58c700 1 -- 172.21.15.157:0/703356541 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa3816b500 con 0x7ffa3806c810 2023-10-06T03:57:21.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.117+0000 7ffa3d58c700 1 -- 172.21.15.157:0/703356541 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa3814c270 con 0x7ffa3806c810 2023-10-06T03:57:21.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.118+0000 7ffa3d58c700 1 -- 172.21.15.157:0/703356541 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa380716b0 con 0x7ffa3806c810 2023-10-06T03:57:21.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.118+0000 7f35d4b04700 1 Processor -- start 2023-10-06T03:57:21.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.117+0000 7fde4ae83700 1 -- start start 2023-10-06T03:57:21.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.118+0000 7f35d4b04700 1 -- start start 2023-10-06T03:57:21.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.117+0000 7fbf735c3700 1 -- start start 2023-10-06T03:57:21.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.118+0000 7fbf735c3700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbf6c06a780 0x7fbf6c06ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.118+0000 7fbf735c3700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbf6c06ab50 0x7fbf6c0c2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.118+0000 7fbf735c3700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf6c06c9e0 con 0x7fbf6c0c3d70 2023-10-06T03:57:21.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.118+0000 7fde4ae83700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde4406a780 0x7fde4416ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.118+0000 7fde4ae83700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde4406ab50 0x7fde440be740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.118+0000 7fde4ae83700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde4406ddf0 0x7fde440bec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.118+0000 7fde4ae83700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde440be2b0 con 0x7fde4406a780 2023-10-06T03:57:21.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.118+0000 7fde4ae83700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde440b5890 con 0x7fde4406ab50 2023-10-06T03:57:21.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.118+0000 7fde4ae83700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde44151080 con 0x7fde4406ddf0 2023-10-06T03:57:21.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.118+0000 7fbf735c3700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf6c15b340 con 0x7fbf6c06ab50 2023-10-06T03:57:21.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.118+0000 7fbf735c3700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf6c15adc0 con 0x7fbf6c06a780 2023-10-06T03:57:21.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.118+0000 7fbf7135f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbf6c06a780 0x7fbf6c06ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.118+0000 7fbf7135f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbf6c06a780 0x7fbf6c06ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:35256/0 (socket says 172.21.15.157:35256) 2023-10-06T03:57:21.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.118+0000 7fbf7135f700 1 -- 172.21.15.157:0/406492970 learned_addr learned my addr 172.21.15.157:0/406492970 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:21.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.118+0000 7fde3bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde4406ab50 0x7fde440be740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.118+0000 7fde49420700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde4406ddf0 0x7fde440bec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.118+0000 7fde3bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde4406ab50 0x7fde440be740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:35258/0 (socket says 172.21.15.157:35258) 2023-10-06T03:57:21.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.118+0000 7fde3bfff700 1 -- 172.21.15.157:0/2043386849 learned_addr learned my addr 172.21.15.157:0/2043386849 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:21.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.118+0000 7fbf71b60700 1 --2- 172.21.15.157:0/406492970 >> v2:172.21.15.157:3300/0 conn(0x7fbf6c06ab50 0x7fbf6c0c2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.114+0000 7f1a0dc0e700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1a0814ccf0 0x7f1a080c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.114+0000 7f1a0dc0e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1a080c5db0 0x7f1a08156c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.114+0000 7f1a0dc0e700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a080c37e0 con 0x7f1a08153990 2023-10-06T03:57:21.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.114+0000 7f1a0dc0e700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a08152ca0 con 0x7f1a0814ccf0 2023-10-06T03:57:21.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.114+0000 7f1a0dc0e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a0806e4d0 con 0x7f1a080c5db0 2023-10-06T03:57:21.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.114+0000 7f1a07fff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f1a08153990 0x7f1a080c3d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40092/0 (socket says 172.21.15.157:40092) 2023-10-06T03:57:21.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.114+0000 7f1a07fff700 1 -- 172.21.15.157:0/3663775757 learned_addr learned my addr 172.21.15.157:0/3663775757 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:21.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.116+0000 7f1a067fc700 1 -- 172.21.15.157:0/3663775757 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3850148222 0 0) 0x7f1a00001490 con 0x7f1a08153990 2023-10-06T03:57:21.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.116+0000 7f1a067fc700 1 -- 172.21.15.157:0/3663775757 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19ec003570 con 0x7f1a08153990 2023-10-06T03:57:21.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.116+0000 7f1a067fc700 1 -- 172.21.15.157:0/3663775757 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2665265524 0 0) 0x7f19ec003570 con 0x7f1a08153990 2023-10-06T03:57:21.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.116+0000 7f1a067fc700 1 -- 172.21.15.157:0/3663775757 >> v2:172.21.15.175:3300/0 conn(0x7f1a080c5db0 msgr2=0x7f1a08156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:21.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.116+0000 7f1a067fc700 1 --2- 172.21.15.157:0/3663775757 >> v2:172.21.15.175:3300/0 conn(0x7f1a080c5db0 0x7f1a08156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.116+0000 7f1a067fc700 1 -- 172.21.15.157:0/3663775757 >> v2:172.21.15.157:3300/0 conn(0x7f1a0814ccf0 msgr2=0x7f1a080c65d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T03:57:21.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.116+0000 7f1a067fc700 1 --2- 172.21.15.157:0/3663775757 >> v2:172.21.15.157:3300/0 conn(0x7f1a0814ccf0 0x7f1a080c65d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.116+0000 7f1a067fc700 1 -- 172.21.15.157:0/3663775757 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a08153500 con 0x7f1a08153990 2023-10-06T03:57:21.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.116+0000 7f1a067fc700 1 -- 172.21.15.157:0/3663775757 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f1a00001980 con 0x7f1a08153990 2023-10-06T03:57:21.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.116+0000 7f1a067fc700 1 -- 172.21.15.157:0/3663775757 >> v1:172.21.15.157:6789/0 conn(0x7f1a08153990 legacy=0x7f1a080c3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.116+0000 7f1a067fc700 1 --2- 172.21.15.157:0/3663775757 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19ec003b30 0x7f19ec003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.116+0000 7f1a067fc700 1 --2- 172.21.15.157:0/3663775757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19ec004810 0x7f19ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.116+0000 7f1a067fc700 1 --2- 172.21.15.157:0/3663775757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19ec005410 0x7f19ec00bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.116+0000 7f1a067fc700 1 -- 172.21.15.157:0/3663775757 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a08152ca0 con 0x7f19ec005410 2023-10-06T03:57:21.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.116+0000 7f1a067fc700 1 -- 172.21.15.157:0/3663775757 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0806e4d0 con 0x7f19ec004810 2023-10-06T03:57:21.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.116+0000 7f1a067fc700 1 -- 172.21.15.157:0/3663775757 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec00c3c0 con 0x7f19ec003b30 2023-10-06T03:57:21.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.116+0000 7f1a07fff700 1 --2- 172.21.15.157:0/3663775757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19ec005410 0x7f19ec00bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.117+0000 7f1a07fff700 1 -- 172.21.15.157:0/3663775757 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19ec003b30 msgr2=0x7f19ec003f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T03:57:21.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.117+0000 7f1a07fff700 1 --2- 172.21.15.157:0/3663775757 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19ec003b30 0x7f19ec003f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.117+0000 7f1a07fff700 1 -- 172.21.15.157:0/3663775757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19ec004810 msgr2=0x7f19ec004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T03:57:21.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.117+0000 7f1a07fff700 1 --2- 172.21.15.157:0/3663775757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19ec004810 0x7f19ec004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.117+0000 7f1a07fff700 1 -- 172.21.15.157:0/3663775757 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ec003570 con 0x7f19ec005410 2023-10-06T03:57:21.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.119+0000 7fbf7135f700 1 -- 172.21.15.157:0/406492970 >> v2:172.21.15.157:3300/0 conn(0x7fbf6c06ab50 msgr2=0x7fbf6c0c2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:21.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.119+0000 7fbf7135f700 1 --2- 172.21.15.157:0/406492970 >> v2:172.21.15.157:3300/0 conn(0x7fbf6c06ab50 0x7fbf6c0c2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.119+0000 7fbf7135f700 1 -- 172.21.15.157:0/406492970 >> v1:172.21.15.157:6789/0 conn(0x7fbf6c0c3d70 legacy=0x7fbf6c0c2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:21.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.119+0000 7fbf7135f700 1 -- 172.21.15.157:0/406492970 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf6c068350 con 0x7fbf6c06a780 2023-10-06T03:57:21.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.117+0000 7f1a07fff700 1 -- 172.21.15.157:0/3663775757 reap_dead start 2023-10-06T03:57:21.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.117+0000 7f1a07fff700 1 --2- 172.21.15.157:0/3663775757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19ec005410 0x7f19ec00bb00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1a0000aaa0 tx=0x7f1a08153470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2946f5d93e7cde5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:21.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.119+0000 7f1a067fc700 1 -- 172.21.15.157:0/3663775757 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0000aee0 con 0x7f19ec005410 2023-10-06T03:57:21.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.119+0000 7fbf7135f700 1 --2- 172.21.15.157:0/406492970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf6c06a780 0x7fbf6c06ce70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c067d20 tx=0x7fbf68009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f48ed6b0c92a6f28 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:21.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.119+0000 7fde49420700 1 -- 172.21.15.157:0/2043386849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde4406ab50 msgr2=0x7fde440be740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.119+0000 7fde49420700 1 --2- 172.21.15.157:0/2043386849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde4406ab50 0x7fde440be740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.119+0000 7fde49420700 1 -- 172.21.15.157:0/2043386849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde4406a780 msgr2=0x7fde4416ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.119+0000 7fde49420700 1 --2- 172.21.15.157:0/2043386849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde4406a780 0x7fde4416ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.119+0000 7fde49420700 1 -- 172.21.15.157:0/2043386849 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde400094f0 con 0x7fde4406ddf0 2023-10-06T03:57:21.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.119+0000 7fde49420700 1 --2- 172.21.15.157:0/2043386849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde4406ddf0 0x7fde440bec80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00bad0 tx=0x7fde3c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:21.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.119+0000 7fde38ff9700 1 -- 172.21.15.157:0/2043386849 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c00c840 con 0x7fde4406ddf0 2023-10-06T03:57:21.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.119+0000 7fde38ff9700 1 -- 172.21.15.157:0/2043386849 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3c00c9c0 con 0x7fde4406ddf0 2023-10-06T03:57:21.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.119+0000 7fde38ff9700 1 -- 172.21.15.157:0/2043386849 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c00cd80 con 0x7fde4406ddf0 2023-10-06T03:57:21.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.119+0000 7fde4ae83700 1 -- 172.21.15.157:0/2043386849 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde4416e730 con 0x7fde4406ddf0 2023-10-06T03:57:21.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.119+0000 7fde4ae83700 1 -- 172.21.15.157:0/2043386849 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde440bdf40 con 0x7fde4406ddf0 2023-10-06T03:57:21.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.120+0000 7ffa1effd700 1 -- 172.21.15.157:0/703356541 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffa2c00c460 con 0x7ffa3806c810 2023-10-06T03:57:21.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.119+0000 7f1a067fc700 1 -- 172.21.15.157:0/3663775757 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0000c550 con 0x7f19ec005410 2023-10-06T03:57:21.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.119+0000 7f1a067fc700 1 -- 172.21.15.157:0/3663775757 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0000c910 con 0x7f19ec005410 2023-10-06T03:57:21.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.120+0000 7f1a0dc0e700 1 -- 172.21.15.157:0/3663775757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19ec005410 msgr2=0x7f19ec00bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.120+0000 7f1a0dc0e700 1 --2- 172.21.15.157:0/3663775757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19ec005410 0x7f19ec00bb00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1a0000aaa0 tx=0x7f1a08153470 comp rx=0 tx=0).stop 2023-10-06T03:57:21.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.120+0000 7f1a0dc0e700 1 -- 172.21.15.157:0/3663775757 shutdown_connections 2023-10-06T03:57:21.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.120+0000 7f1a0dc0e700 1 --2- 172.21.15.157:0/3663775757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19ec005410 0x7f19ec00bb00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.120+0000 7f1a0dc0e700 1 -- 172.21.15.157:0/3663775757 >> 172.21.15.157:0/3663775757 conn(0x7f1a0806b6a0 msgr2=0x7f1a080631f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:21.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.119+0000 7f35d4b04700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f35d00adbf0 0x7f35d006e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.119+0000 7f35d4b04700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f35d006e030 0x7f35d00694f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.119+0000 7f35d4b04700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d006d060 con 0x7f35d01531a0 2023-10-06T03:57:21.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.119+0000 7f35d4b04700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d0156bc0 con 0x7f35d006e030 2023-10-06T03:57:21.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.119+0000 7f35d4b04700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d006c500 con 0x7f35d00adbf0 2023-10-06T03:57:21.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.119+0000 7f35ced9d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f35d006e030 0x7f35d00694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.120+0000 7f35ced9d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f35d006e030 0x7f35d00694f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34582/0 (socket says 172.21.15.157:34582) 2023-10-06T03:57:21.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.120+0000 7f35ced9d700 1 -- 172.21.15.157:0/1596948224 learned_addr learned my addr 172.21.15.157:0/1596948224 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:21.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.120+0000 7f35ced9d700 1 -- 172.21.15.157:0/1596948224 >> v2:172.21.15.175:3300/0 conn(0x7f35d00adbf0 msgr2=0x7f35d006e590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T03:57:21.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.120+0000 7f35ced9d700 1 --2- 172.21.15.157:0/1596948224 >> v2:172.21.15.175:3300/0 conn(0x7f35d00adbf0 0x7f35d006e590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.120+0000 7f35ced9d700 1 -- 172.21.15.157:0/1596948224 >> v1:172.21.15.175:6789/0 conn(0x7f35d01531a0 legacy=0x7f35d0157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.120+0000 7f35ced9d700 1 -- 172.21.15.157:0/1596948224 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d006c7c0 con 0x7f35d006e030 2023-10-06T03:57:21.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.120+0000 7f1a0dc0e700 1 -- 172.21.15.157:0/3663775757 shutdown_connections 2023-10-06T03:57:21.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.120+0000 7f35ced9d700 1 --2- 172.21.15.157:0/1596948224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35d006e030 0x7f35d00694f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f35d0068c40 tx=0x7f35c80143c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2bd8dbbaa54af3b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:21.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.120+0000 7f1a0dc0e700 1 -- 172.21.15.157:0/3663775757 wait complete. 2023-10-06T03:57:21.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.121+0000 7f35cd59a700 1 -- 172.21.15.157:0/1596948224 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c8013bd0 con 0x7f35d006e030 2023-10-06T03:57:21.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.121+0000 7ffa1effd700 1 --2- 172.21.15.157:0/703356541 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ffa24066f60 0x7ffa24069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.121+0000 7ffa1effd700 1 -- 172.21.15.157:0/703356541 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ffa2c084f50 con 0x7ffa3806c810 2023-10-06T03:57:21.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.121+0000 7f35cd59a700 1 -- 172.21.15.157:0/1596948224 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c8013d30 con 0x7f35d006e030 2023-10-06T03:57:21.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.121+0000 7f35cd59a700 1 -- 172.21.15.157:0/1596948224 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c80140f0 con 0x7f35d006e030 2023-10-06T03:57:21.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.121+0000 7fde38ff9700 1 -- 172.21.15.157:0/2043386849 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fde3c014620 con 0x7fde4406ddf0 2023-10-06T03:57:21.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.122+0000 7fde38ff9700 1 --2- 172.21.15.157:0/2043386849 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fde30066f60 0x7fde30069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.122+0000 7f4f1788f700 1 Processor -- start 2023-10-06T03:57:21.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.122+0000 7fde38ff9700 1 -- 172.21.15.157:0/2043386849 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fde3c084540 con 0x7fde4406ddf0 2023-10-06T03:57:21.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.122+0000 7f35d4b04700 1 -- 172.21.15.157:0/1596948224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35d006e030 msgr2=0x7f35d00694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.122+0000 7f35d4b04700 1 --2- 172.21.15.157:0/1596948224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35d006e030 0x7f35d00694f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f35d0068c40 tx=0x7f35c80143c0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.122+0000 7f4f1788f700 1 -- start start 2023-10-06T03:57:21.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.122+0000 7f35d4b04700 1 -- 172.21.15.157:0/1596948224 shutdown_connections 2023-10-06T03:57:21.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.122+0000 7f35d4b04700 1 --2- 172.21.15.157:0/1596948224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35d006e030 0x7f35d00694f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.122+0000 7f35d4b04700 1 --2- 172.21.15.157:0/1596948224 >> v2:172.21.15.175:3300/0 conn(0x7f35d00adbf0 0x7f35d006e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.122+0000 7f35d4b04700 1 -- 172.21.15.157:0/1596948224 >> 172.21.15.157:0/1596948224 conn(0x7f35d00c82e0 msgr2=0x7f35d00c86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:21.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.122+0000 7f35d4b04700 1 -- 172.21.15.157:0/1596948224 shutdown_connections 2023-10-06T03:57:21.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.122+0000 7fde48c1f700 1 --2- 172.21.15.157:0/2043386849 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fde30066f60 0x7fde30069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.122+0000 7f35d4b04700 1 -- 172.21.15.157:0/1596948224 wait complete. 2023-10-06T03:57:21.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.122+0000 7f1a0dc0e700 1 Processor -- start 2023-10-06T03:57:21.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.122+0000 7ffa36ffd700 1 --2- 172.21.15.157:0/703356541 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ffa24066f60 0x7ffa24069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.122+0000 7f1a0dc0e700 1 -- start start 2023-10-06T03:57:21.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.123+0000 7ffa36ffd700 1 --2- 172.21.15.157:0/703356541 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ffa24066f60 0x7ffa24069410 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffa28000c00 tx=0x7ffa28007ea0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:21.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.122+0000 7fde4ae83700 1 -- 172.21.15.157:0/2043386849 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4404bb40 con 0x7fde4406ddf0 2023-10-06T03:57:21.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.123+0000 7fde48c1f700 1 --2- 172.21.15.157:0/2043386849 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fde30066f60 0x7fde30069410 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fde340097c0 tx=0x7fde34006ce0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:21.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.123+0000 7f4f1788f700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4f100c4840 0x7f4f101500e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.123+0000 7f4f1788f700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f1006e730 con 0x7f4f1006aa40 2023-10-06T03:57:21.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.123+0000 7f4f1788f700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f1015a6c0 con 0x7f4f100c4470 2023-10-06T03:57:21.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.123+0000 7f4f1788f700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f10159ab0 con 0x7f4f100c4840 2023-10-06T03:57:21.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.123+0000 7f4f14e2a700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4f100c4840 0x7f4f101500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.123+0000 7f35d4b04700 1 Processor -- start 2023-10-06T03:57:21.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.123+0000 7f4f14e2a700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4f100c4840 0x7f4f101500e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34586/0 (socket says 172.21.15.157:34586) 2023-10-06T03:57:21.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.123+0000 7f4f14e2a700 1 -- 172.21.15.157:0/4292839805 learned_addr learned my addr 172.21.15.157:0/4292839805 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:21.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.123+0000 7f1a0dc0e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19ec005410 0x7f1a08159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.123+0000 7f1a0dc0e700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a081346b0 0x7f1a081584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.123+0000 7f1a0dc0e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a08158a30 0x7f1a08165ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.123+0000 7f1a0dc0e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a08133940 con 0x7f1a08158a30 2023-10-06T03:57:21.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.123+0000 7f1a0dc0e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a08157d80 con 0x7f19ec005410 2023-10-06T03:57:21.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.123+0000 7f1a0dc0e700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a08158ed0 con 0x7f1a081346b0 2023-10-06T03:57:21.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.123+0000 7f35d4b04700 1 -- start start 2023-10-06T03:57:21.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.123+0000 7f1a07fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a08158a30 0x7f1a08165ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.123+0000 7f1a077fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19ec005410 0x7f1a08159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.123+0000 7f1a07fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a08158a30 0x7f1a08165ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34594/0 (socket says 172.21.15.157:34594) 2023-10-06T03:57:21.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.123+0000 7f1a07fff700 1 -- 172.21.15.157:0/2734807942 learned_addr learned my addr 172.21.15.157:0/2734807942 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:21.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.124+0000 7f4f14e2a700 1 -- 172.21.15.157:0/4292839805 >> v1:172.21.15.175:6789/0 conn(0x7f4f100c4470 legacy=0x7f4f1006df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:21.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.124+0000 7f4f14e2a700 1 -- 172.21.15.157:0/4292839805 >> v1:172.21.15.157:6789/0 conn(0x7f4f1006aa40 legacy=0x7f4f101564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:21.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.124+0000 7f4f14e2a700 1 -- 172.21.15.157:0/4292839805 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f10159d90 con 0x7f4f100c4840 2023-10-06T03:57:21.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.124+0000 7f1a07fff700 1 -- 172.21.15.157:0/2734807942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a081346b0 msgr2=0x7f1a081584f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:57:21.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.124+0000 7f1a07fff700 1 --2- 172.21.15.157:0/2734807942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a081346b0 0x7f1a081584f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.124+0000 7f1a07fff700 1 -- 172.21.15.157:0/2734807942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19ec005410 msgr2=0x7f1a08159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.124+0000 7f1a07fff700 1 --2- 172.21.15.157:0/2734807942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19ec005410 0x7f1a08159a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.124+0000 7f1a07fff700 1 -- 172.21.15.157:0/2734807942 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a08133680 con 0x7f1a08158a30 2023-10-06T03:57:21.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.124+0000 7fbf63fff700 1 -- 172.21.15.157:0/406492970 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf68009e70 con 0x7fbf6c06a780 2023-10-06T03:57:21.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.124+0000 7fbf63fff700 1 -- 172.21.15.157:0/406492970 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf6800b8b0 con 0x7fbf6c06a780 2023-10-06T03:57:21.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.124+0000 7fbf63fff700 1 -- 172.21.15.157:0/406492970 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6800bc30 con 0x7fbf6c06a780 2023-10-06T03:57:21.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.124+0000 7f4f14e2a700 1 --2- 172.21.15.157:0/4292839805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f100c4840 0x7f4f101500e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4efc0099b0 tx=0x7f4efc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7383648a206f6c9f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:21.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.124+0000 7f4f0ffff700 1 -- 172.21.15.157:0/4292839805 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4efc00b820 con 0x7f4f100c4840 2023-10-06T03:57:21.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.124+0000 7f4f0ffff700 1 -- 172.21.15.157:0/4292839805 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4efc00b980 con 0x7f4f100c4840 2023-10-06T03:57:21.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.124+0000 7f4f0ffff700 1 -- 172.21.15.157:0/4292839805 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4efc00bd20 con 0x7f4f100c4840 2023-10-06T03:57:21.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.124+0000 7ffa1effd700 1 -- 172.21.15.157:0/703356541 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa2c052060 con 0x7ffa3806c810 2023-10-06T03:57:21.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.124+0000 7f1a07fff700 1 --2- 172.21.15.157:0/2734807942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a08158a30 0x7f1a08165ca0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1a0000a520 tx=0x7f1a00038000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:21.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.125+0000 7f19ff7fe700 1 -- 172.21.15.157:0/2734807942 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a00004bb0 con 0x7f1a08158a30 2023-10-06T03:57:21.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.125+0000 7f19ff7fe700 1 -- 172.21.15.157:0/2734807942 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a00004d30 con 0x7f1a08158a30 2023-10-06T03:57:21.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.125+0000 7f19ff7fe700 1 -- 172.21.15.157:0/2734807942 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a00004ea0 con 0x7f1a08158a30 2023-10-06T03:57:21.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.125+0000 7f1a0dc0e700 1 -- 172.21.15.157:0/2734807942 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a08163240 con 0x7f1a08158a30 2023-10-06T03:57:21.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.125+0000 7f1a0dc0e700 1 -- 172.21.15.157:0/2734807942 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a08157fd0 con 0x7f1a08158a30 2023-10-06T03:57:21.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.126+0000 7f1a0dc0e700 1 -- 172.21.15.157:0/2734807942 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a081581c0 con 0x7f1a08158a30 2023-10-06T03:57:21.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.128+0000 7f19ff7fe700 1 -- 172.21.15.157:0/2734807942 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1a00040910 con 0x7f1a08158a30 2023-10-06T03:57:21.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.128+0000 7f19ff7fe700 1 --2- 172.21.15.157:0/2734807942 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f19f0067080 0x7f19f0069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.128+0000 7f19ff7fe700 1 -- 172.21.15.157:0/2734807942 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1a000a8990 con 0x7f1a08158a30 2023-10-06T03:57:21.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.128+0000 7f1a077fe700 1 --2- 172.21.15.157:0/2734807942 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f19f0067080 0x7f19f0069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.125+0000 7fbf735c3700 1 -- 172.21.15.157:0/406492970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf6c06a780 msgr2=0x7fbf6c06ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.125+0000 7fbf735c3700 1 --2- 172.21.15.157:0/406492970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf6c06a780 0x7fbf6c06ce70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c067d20 tx=0x7fbf68009250 comp rx=0 tx=0).stop 2023-10-06T03:57:21.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.129+0000 7fbf735c3700 1 -- 172.21.15.157:0/406492970 shutdown_connections 2023-10-06T03:57:21.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.129+0000 7fbf735c3700 1 --2- 172.21.15.157:0/406492970 >> v2:172.21.15.157:3300/0 conn(0x7fbf6c06ab50 0x7fbf6c0c2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.129+0000 7fbf735c3700 1 --2- 172.21.15.157:0/406492970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf6c06a780 0x7fbf6c06ce70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.130+0000 7fbf735c3700 1 -- 172.21.15.157:0/406492970 >> 172.21.15.157:0/406492970 conn(0x7fbf6c06b510 msgr2=0x7fbf6c06b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:21.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.125+0000 7f4f1788f700 1 -- 172.21.15.157:0/4292839805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f100c4840 msgr2=0x7f4f101500e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.125+0000 7f4f1788f700 1 --2- 172.21.15.157:0/4292839805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f100c4840 0x7f4f101500e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4efc0099b0 tx=0x7f4efc009250 comp rx=0 tx=0).stop 2023-10-06T03:57:21.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.126+0000 7f4f1788f700 1 -- 172.21.15.157:0/4292839805 shutdown_connections 2023-10-06T03:57:21.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.126+0000 7f4f1788f700 1 --2- 172.21.15.157:0/4292839805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f100c4840 0x7f4f101500e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.126+0000 7f4f1788f700 1 -- 172.21.15.157:0/4292839805 >> 172.21.15.157:0/4292839805 conn(0x7f4f100c82e0 msgr2=0x7f4f100c86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:21.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.126+0000 7f4f1788f700 1 -- 172.21.15.157:0/4292839805 shutdown_connections 2023-10-06T03:57:21.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.126+0000 7f4f1788f700 1 -- 172.21.15.157:0/4292839805 wait complete. 2023-10-06T03:57:21.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.128+0000 7f4f1788f700 1 Processor -- start 2023-10-06T03:57:21.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.128+0000 7f4f1788f700 1 -- start start 2023-10-06T03:57:21.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.129+0000 7f4f1788f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f1006aa40 0x7f4f10143e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.129+0000 7f4f1788f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f100c4470 0x7f4f100c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.129+0000 7f4f1788f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f100c4840 0x7f4f100c7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.129+0000 7f4f1788f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f10155480 con 0x7f4f100c4840 2023-10-06T03:57:21.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.129+0000 7f4f1788f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f100c6c60 con 0x7f4f100c4470 2023-10-06T03:57:21.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.129+0000 7f4f1788f700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f10154c20 con 0x7f4f1006aa40 2023-10-06T03:57:21.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.129+0000 7f4f1562b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f1006aa40 0x7f4f10143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.129+0000 7f4f1562b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f1006aa40 0x7f4f10143e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:50294/0 (socket says 172.21.15.157:50294) 2023-10-06T03:57:21.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.129+0000 7f4f1562b700 1 -- 172.21.15.157:0/3649544010 learned_addr learned my addr 172.21.15.157:0/3649544010 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:21.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.129+0000 7f4f15e2c700 1 --2- 172.21.15.157:0/3649544010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f100c4840 0x7f4f100c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.129+0000 7f4f14e2a700 1 --2- 172.21.15.157:0/3649544010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f100c4470 0x7f4f100c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.129+0000 7f4f1562b700 1 -- 172.21.15.157:0/3649544010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f100c4470 msgr2=0x7f4f100c70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.129+0000 7f4f1562b700 1 --2- 172.21.15.157:0/3649544010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f100c4470 0x7f4f100c70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.129+0000 7f4f1562b700 1 -- 172.21.15.157:0/3649544010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f100c4840 msgr2=0x7f4f100c7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.129+0000 7f4f1562b700 1 --2- 172.21.15.157:0/3649544010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f100c4840 0x7f4f100c7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.129+0000 7f4f1562b700 1 -- 172.21.15.157:0/3649544010 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4efc0094f0 con 0x7f4f1006aa40 2023-10-06T03:57:21.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.124+0000 7f35d4b04700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35d006e030 0x7f35d0165e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.124+0000 7f35d4b04700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35d00adbf0 0x7f35d0151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.124+0000 7f35d4b04700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35d01531a0 0x7f35d01522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.124+0000 7f35d4b04700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d01663b0 con 0x7f35d006e030 2023-10-06T03:57:21.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.124+0000 7f35d4b04700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d00c3600 con 0x7f35d00adbf0 2023-10-06T03:57:21.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.124+0000 7f35d4b04700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d0162ae0 con 0x7f35d01531a0 2023-10-06T03:57:21.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.125+0000 7f35ce59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35d006e030 0x7f35d0165e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.125+0000 7f35ce59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35d006e030 0x7f35d0165e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34610/0 (socket says 172.21.15.157:34610) 2023-10-06T03:57:21.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.125+0000 7f35ce59c700 1 -- 172.21.15.157:0/2212375302 learned_addr learned my addr 172.21.15.157:0/2212375302 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:21.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.125+0000 7f35ce59c700 1 -- 172.21.15.157:0/2212375302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35d01531a0 msgr2=0x7f35d01522d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:57:21.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.125+0000 7f35ce59c700 1 --2- 172.21.15.157:0/2212375302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35d01531a0 0x7f35d01522d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.125+0000 7f35ce59c700 1 -- 172.21.15.157:0/2212375302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35d00adbf0 msgr2=0x7f35d0151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.125+0000 7f35cdd9b700 1 --2- 172.21.15.157:0/2212375302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35d00adbf0 0x7f35d0151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.125+0000 7f35ce59c700 1 --2- 172.21.15.157:0/2212375302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35d00adbf0 0x7f35d0151d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.125+0000 7f35ce59c700 1 -- 172.21.15.157:0/2212375302 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c8014600 con 0x7f35d006e030 2023-10-06T03:57:21.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.126+0000 7f35ce59c700 1 --2- 172.21.15.157:0/2212375302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35d006e030 0x7f35d0165e70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f35c000b5e0 tx=0x7f35c00090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:21.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.126+0000 7f35be7fc700 1 -- 172.21.15.157:0/2212375302 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c000f510 con 0x7f35d006e030 2023-10-06T03:57:21.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.126+0000 7f35be7fc700 1 -- 172.21.15.157:0/2212375302 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c000f690 con 0x7f35d006e030 2023-10-06T03:57:21.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.126+0000 7f35be7fc700 1 -- 172.21.15.157:0/2212375302 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c000fa50 con 0x7f35d006e030 2023-10-06T03:57:21.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.126+0000 7f35d4b04700 1 -- 172.21.15.157:0/2212375302 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d0161f30 con 0x7f35d006e030 2023-10-06T03:57:21.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.126+0000 7f35d4b04700 1 -- 172.21.15.157:0/2212375302 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d01621b0 con 0x7f35d006e030 2023-10-06T03:57:21.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.128+0000 7f35be7fc700 1 -- 172.21.15.157:0/2212375302 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f35c001a400 con 0x7f35d006e030 2023-10-06T03:57:21.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.129+0000 7f35d4b04700 1 -- 172.21.15.157:0/2212375302 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d00716b0 con 0x7f35d006e030 2023-10-06T03:57:21.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.129+0000 7f35be7fc700 1 --2- 172.21.15.157:0/2212375302 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f35b0066f30 0x7f35b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.129+0000 7f35be7fc700 1 -- 172.21.15.157:0/2212375302 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f35c0084280 con 0x7f35d006e030 2023-10-06T03:57:21.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.129+0000 7f35cdd9b700 1 --2- 172.21.15.157:0/2212375302 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f35b0066f30 0x7f35b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.129+0000 7f4f15e2c700 1 --2- 172.21.15.157:0/3649544010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f100c4840 0x7f4f100c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:21.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.130+0000 7fbf735c3700 1 -- 172.21.15.157:0/406492970 shutdown_connections 2023-10-06T03:57:21.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.130+0000 7fbf735c3700 1 -- 172.21.15.157:0/406492970 wait complete. 2023-10-06T03:57:21.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.130+0000 7fde38ff9700 1 -- 172.21.15.157:0/2043386849 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde3c051650 con 0x7fde4406ddf0 2023-10-06T03:57:21.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.130+0000 7f35cdd9b700 1 --2- 172.21.15.157:0/2212375302 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f35b0066f30 0x7f35b00693e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f35d0144a00 tx=0x7f35c4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:21.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.130+0000 7f1a077fe700 1 --2- 172.21.15.157:0/2734807942 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f19f0067080 0x7f19f0069530 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1a08061120 tx=0x7f19f8006f90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:21.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.130+0000 7f4f1562b700 1 --2- 172.21.15.157:0/3649544010 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f1006aa40 0x7f4f10143e80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400bbb0 tx=0x7f4f04009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:21.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.130+0000 7fa510431700 1 Processor -- start 2023-10-06T03:57:21.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.130+0000 7f4f0d7fa700 1 -- 172.21.15.157:0/3649544010 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f040092e0 con 0x7f4f1006aa40 2023-10-06T03:57:21.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.130+0000 7f4f0d7fa700 1 -- 172.21.15.157:0/3649544010 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0400c6f0 con 0x7f4f1006aa40 2023-10-06T03:57:21.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.130+0000 7f4f0d7fa700 1 -- 172.21.15.157:0/3649544010 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0400ca90 con 0x7f4f1006aa40 2023-10-06T03:57:21.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.130+0000 7fa510431700 1 -- start start 2023-10-06T03:57:21.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.131+0000 7fa510431700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa5080bf8f0 0x7fa50806a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.131+0000 7fa510431700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa508153fe0 0x7fa508154450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.131+0000 7fa510431700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa50815b200 con 0x7fa5080bf520 2023-10-06T03:57:21.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.131+0000 7fa510431700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa50806b480 con 0x7fa5080bf8f0 2023-10-06T03:57:21.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.131+0000 7fa510431700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa508154990 con 0x7fa508153fe0 2023-10-06T03:57:21.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.131+0000 7f4f1788f700 1 -- 172.21.15.157:0/3649544010 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f100c25d0 con 0x7f4f1006aa40 2023-10-06T03:57:21.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.131+0000 7f4f1788f700 1 -- 172.21.15.157:0/3649544010 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f101551a0 con 0x7f4f1006aa40 2023-10-06T03:57:21.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.131+0000 7fa50d9cc700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa5080bf8f0 0x7fa50806a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.132+0000 7fa50e9ce700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa508153fe0 0x7fa508154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.132+0000 7fa50d9cc700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa5080bf8f0 0x7fa50806a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:35298/0 (socket says 172.21.15.157:35298) 2023-10-06T03:57:21.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.132+0000 7fa50d9cc700 1 -- 172.21.15.157:0/3077941738 learned_addr learned my addr 172.21.15.157:0/3077941738 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:21.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.132+0000 7fa50e9ce700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa508153fe0 0x7fa508154450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:50302/0 (socket says 172.21.15.157:50302) 2023-10-06T03:57:21.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.132+0000 7fa50d9cc700 1 -- 172.21.15.157:0/3077941738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa508153fe0 msgr2=0x7fa508154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:21.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.132+0000 7fa50d9cc700 1 --2- 172.21.15.157:0/3077941738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa508153fe0 0x7fa508154450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.132+0000 7fa50d9cc700 1 -- 172.21.15.157:0/3077941738 >> v1:172.21.15.175:6789/0 conn(0x7fa5080bf520 legacy=0x7fa5080631f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:21.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.132+0000 7fa50d9cc700 1 -- 172.21.15.157:0/3077941738 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50815a310 con 0x7fa5080bf8f0 2023-10-06T03:57:21.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.132+0000 7fa50d9cc700 1 --2- 172.21.15.157:0/3077941738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5080bf8f0 0x7fa50806a6a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa5040099b0 tx=0x7fa504009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed95f324e8c7fd89 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:21.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.132+0000 7fa50d1cb700 1 -- 172.21.15.157:0/3077941738 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa504009ea0 con 0x7fa5080bf8f0 2023-10-06T03:57:21.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.132+0000 7fa50d1cb700 1 -- 172.21.15.157:0/3077941738 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50400b820 con 0x7fa5080bf8f0 2023-10-06T03:57:21.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.132+0000 7fa50d1cb700 1 -- 172.21.15.157:0/3077941738 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50400bbc0 con 0x7fa5080bf8f0 2023-10-06T03:57:21.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.132+0000 7fbf735c3700 1 Processor -- start 2023-10-06T03:57:21.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.133+0000 7f4f1788f700 1 -- 172.21.15.157:0/3649544010 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f100716b0 con 0x7f4f1006aa40 2023-10-06T03:57:21.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.133+0000 7fa510431700 1 -- 172.21.15.157:0/3077941738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5080bf8f0 msgr2=0x7fa50806a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.133+0000 7fa510431700 1 --2- 172.21.15.157:0/3077941738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5080bf8f0 0x7fa50806a6a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa5040099b0 tx=0x7fa504009250 comp rx=0 tx=0).stop 2023-10-06T03:57:21.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.133+0000 7f4f0d7fa700 1 -- 172.21.15.157:0/3649544010 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4f04012070 con 0x7f4f1006aa40 2023-10-06T03:57:21.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.133+0000 7fa510431700 1 -- 172.21.15.157:0/3077941738 shutdown_connections 2023-10-06T03:57:21.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.133+0000 7fa510431700 1 --2- 172.21.15.157:0/3077941738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa508153fe0 0x7fa508154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.133+0000 7fa510431700 1 --2- 172.21.15.157:0/3077941738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5080bf8f0 0x7fa50806a6a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.133+0000 7fa510431700 1 -- 172.21.15.157:0/3077941738 >> 172.21.15.157:0/3077941738 conn(0x7fa5080c8b20 msgr2=0x7fa5080c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:21.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.134+0000 7f4f0d7fa700 1 --2- 172.21.15.157:0/3649544010 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4ef4067000 0x7f4ef40694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.134+0000 7f4f0d7fa700 1 -- 172.21.15.157:0/3649544010 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4f040842a0 con 0x7f4f1006aa40 2023-10-06T03:57:21.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.134+0000 7f4f14e2a700 1 --2- 172.21.15.157:0/3649544010 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4ef4067000 0x7f4ef40694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.133+0000 7fbf735c3700 1 -- start start 2023-10-06T03:57:21.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.134+0000 7fbf735c3700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf6c06a780 0x7fbf6c0c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.134+0000 7fbf735c3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf6c06ab50 0x7fbf6c0c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.134+0000 7fbf735c3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf6c0c3d70 0x7fbf6c1346a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.134+0000 7fbf735c3700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c0be7c0 con 0x7fbf6c06ab50 2023-10-06T03:57:21.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.134+0000 7fbf735c3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c0bfeb0 con 0x7fbf6c0c3d70 2023-10-06T03:57:21.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.134+0000 7fbf735c3700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c0bf060 con 0x7fbf6c06a780 2023-10-06T03:57:21.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.134+0000 7fbf70b5e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf6c06ab50 0x7fbf6c0c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.134+0000 7fbf70b5e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf6c06ab50 0x7fbf6c0c9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34626/0 (socket says 172.21.15.157:34626) 2023-10-06T03:57:21.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.134+0000 7fbf70b5e700 1 -- 172.21.15.157:0/3889652641 learned_addr learned my addr 172.21.15.157:0/3889652641 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:21.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.134+0000 7fbf71b60700 1 --2- 172.21.15.157:0/3889652641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf6c0c3d70 0x7fbf6c1346a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.135+0000 7fa510431700 1 -- 172.21.15.157:0/3077941738 shutdown_connections 2023-10-06T03:57:21.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.135+0000 7fbf70b5e700 1 -- 172.21.15.157:0/3889652641 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf6c06a780 msgr2=0x7fbf6c0c9320 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T03:57:21.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.135+0000 7fbf70b5e700 1 --2- 172.21.15.157:0/3889652641 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf6c06a780 0x7fbf6c0c9320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.135+0000 7fbf70b5e700 1 -- 172.21.15.157:0/3889652641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf6c0c3d70 msgr2=0x7fbf6c1346a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.135+0000 7fbf70b5e700 1 --2- 172.21.15.157:0/3889652641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf6c0c3d70 0x7fbf6c1346a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.135+0000 7fbf70b5e700 1 -- 172.21.15.157:0/3889652641 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf680094f0 con 0x7fbf6c06ab50 2023-10-06T03:57:21.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.135+0000 7fbf71b60700 1 --2- 172.21.15.157:0/3889652641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf6c0c3d70 0x7fbf6c1346a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:21.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.135+0000 7fa510431700 1 -- 172.21.15.157:0/3077941738 wait complete. 2023-10-06T03:57:21.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.135+0000 7fbf70b5e700 1 --2- 172.21.15.157:0/3889652641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf6c06ab50 0x7fbf6c0c9860 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00b5e0 tx=0x7fbf5c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:21.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.136+0000 7f19ff7fe700 1 -- 172.21.15.157:0/2734807942 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a00075a80 con 0x7f1a08158a30 2023-10-06T03:57:21.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.136+0000 7fa510431700 1 Processor -- start 2023-10-06T03:57:21.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.137+0000 7fa510431700 1 -- start start 2023-10-06T03:57:21.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.138+0000 7f4f14e2a700 1 --2- 172.21.15.157:0/3649544010 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4ef4067000 0x7f4ef40694b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4efc0099b0 tx=0x7f4efc016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:21.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.138+0000 7fa510431700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5080bf8f0 0x7fa5080bfe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.138+0000 7fa510431700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa508153fe0 0x7fa5080c0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.138+0000 7fa510431700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa50814ad80 0x7fa50814e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.138+0000 7fa510431700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50814bf20 con 0x7fa50814ad80 2023-10-06T03:57:21.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.138+0000 7fa510431700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5080c2410 con 0x7fa5080bf8f0 2023-10-06T03:57:21.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.138+0000 7fa510431700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5080c29d0 con 0x7fa508153fe0 2023-10-06T03:57:21.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.138+0000 7fa50e9ce700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa50814ad80 0x7fa50814e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.138+0000 7fa50e9ce700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa50814ad80 0x7fa50814e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34630/0 (socket says 172.21.15.157:34630) 2023-10-06T03:57:21.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.138+0000 7fa50e9ce700 1 -- 172.21.15.157:0/2373825451 learned_addr learned my addr 172.21.15.157:0/2373825451 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:21.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.138+0000 7fa50e1cd700 1 --2- 172.21.15.157:0/2373825451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5080bf8f0 0x7fa5080bfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.138+0000 7fa50e9ce700 1 -- 172.21.15.157:0/2373825451 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa508153fe0 msgr2=0x7fa5080c0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.138+0000 7fa50e9ce700 1 --2- 172.21.15.157:0/2373825451 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa508153fe0 0x7fa5080c0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.138+0000 7fa50e9ce700 1 -- 172.21.15.157:0/2373825451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5080bf8f0 msgr2=0x7fa5080bfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.138+0000 7fa50e9ce700 1 --2- 172.21.15.157:0/2373825451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5080bf8f0 0x7fa5080bfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.138+0000 7fa50e9ce700 1 -- 172.21.15.157:0/2373825451 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5040094f0 con 0x7fa50814ad80 2023-10-06T03:57:21.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.139+0000 7fa50e9ce700 1 --2- 172.21.15.157:0/2373825451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa50814ad80 0x7fa50814e990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa500009df0 tx=0x7fa50000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:21.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.139+0000 7f4f0d7fa700 1 -- 172.21.15.157:0/3649544010 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4f04051430 con 0x7f4f1006aa40 2023-10-06T03:57:21.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.145+0000 7fa4fe7fc700 1 -- 172.21.15.157:0/2373825451 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50000cb50 con 0x7fa50814ad80 2023-10-06T03:57:21.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.145+0000 7f35be7fc700 1 -- 172.21.15.157:0/2212375302 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f35c0054e10 con 0x7f35d006e030 2023-10-06T03:57:21.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.145+0000 7fa4fe7fc700 1 -- 172.21.15.157:0/2373825451 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50000f040 con 0x7fa50814ad80 2023-10-06T03:57:21.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.145+0000 7fa510431700 1 -- 172.21.15.157:0/2373825451 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5080c2c90 con 0x7fa50814ad80 2023-10-06T03:57:21.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.145+0000 7fa510431700 1 -- 172.21.15.157:0/2373825451 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa50814cab0 con 0x7fa50814ad80 2023-10-06T03:57:21.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.145+0000 7fa4fe7fc700 1 -- 172.21.15.157:0/2373825451 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa500014480 con 0x7fa50814ad80 2023-10-06T03:57:21.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.148+0000 7fbf617fa700 1 -- 172.21.15.157:0/3889652641 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c00f510 con 0x7fbf6c06ab50 2023-10-06T03:57:21.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.148+0000 7fbf617fa700 1 -- 172.21.15.157:0/3889652641 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5c00f670 con 0x7fbf6c06ab50 2023-10-06T03:57:21.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.148+0000 7fbf617fa700 1 -- 172.21.15.157:0/3889652641 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c00fa10 con 0x7fbf6c06ab50 2023-10-06T03:57:21.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.149+0000 7fbf735c3700 1 -- 172.21.15.157:0/3889652641 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf6c0bf300 con 0x7fbf6c06ab50 2023-10-06T03:57:21.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.149+0000 7fbf735c3700 1 -- 172.21.15.157:0/3889652641 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf6c0bf860 con 0x7fbf6c06ab50 2023-10-06T03:57:21.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.149+0000 7fa4fe7fc700 1 -- 172.21.15.157:0/2373825451 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa5000147f0 con 0x7fa50814ad80 2023-10-06T03:57:21.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.149+0000 7fa4fe7fc700 1 --2- 172.21.15.157:0/2373825451 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa4ec066fb0 0x7fa4ec069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.149+0000 7fa4fe7fc700 1 -- 172.21.15.157:0/2373825451 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa500084fa0 con 0x7fa50814ad80 2023-10-06T03:57:21.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.149+0000 7fa50e1cd700 1 --2- 172.21.15.157:0/2373825451 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa4ec066fb0 0x7fa4ec069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.150+0000 7fa50e1cd700 1 --2- 172.21.15.157:0/2373825451 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa4ec066fb0 0x7fa4ec069460 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa50806c010 tx=0x7fa4f8007ea0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:21.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.153+0000 7fa510431700 1 -- 172.21.15.157:0/2373825451 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4f00052b0 con 0x7fa50814ad80 2023-10-06T03:57:21.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.152+0000 7fbf617fa700 1 -- 172.21.15.157:0/3889652641 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbf5c01a400 con 0x7fbf6c06ab50 2023-10-06T03:57:21.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.152+0000 7fbf617fa700 1 --2- 172.21.15.157:0/3889652641 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbf50067000 0x7fbf500694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.152+0000 7fbf617fa700 1 -- 172.21.15.157:0/3889652641 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbf5c084100 con 0x7fbf6c06ab50 2023-10-06T03:57:21.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.152+0000 7fbf7135f700 1 --2- 172.21.15.157:0/3889652641 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbf50067000 0x7fbf500694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.153+0000 7fbf7135f700 1 --2- 172.21.15.157:0/3889652641 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbf50067000 0x7fbf500694b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c067d20 tx=0x7fbf68016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:21.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.161+0000 7fbf735c3700 1 -- 172.21.15.157:0/3889652641 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf6c0692b0 con 0x7fbf6c06ab50 2023-10-06T03:57:21.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.167+0000 7fa4fe7fc700 1 -- 172.21.15.157:0/2373825451 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa500052010 con 0x7fa50814ad80 2023-10-06T03:57:21.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.170+0000 7fbf617fa700 1 -- 172.21.15.157:0/3889652641 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbf5c051370 con 0x7fbf6c06ab50 2023-10-06T03:57:21.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.458+0000 7f05ebfd5700 1 -- 172.21.15.157:0/1335511473 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f05e40716b0 con 0x7f05e406c440 2023-10-06T03:57:21.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.460+0000 7f05d9ffb700 1 -- 172.21.15.157:0/1335511473 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f05d4017090 con 0x7f05e406c440 2023-10-06T03:57:21.464 INFO:teuthology.orchestra.run.smithi157.stdout:55834574854 2023-10-06T03:57:21.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.463+0000 7f05ebfd5700 1 -- 172.21.15.157:0/1335511473 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f05d00671e0 msgr2=0x7f05d0069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.463+0000 7f05ebfd5700 1 --2- 172.21.15.157:0/1335511473 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f05d00671e0 0x7f05d0069690 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f05dc0072c0 tx=0x7f05dc0076f0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.463+0000 7f05ebfd5700 1 -- 172.21.15.157:0/1335511473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05e406c440 msgr2=0x7f05e40c6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.463+0000 7f05ebfd5700 1 --2- 172.21.15.157:0/1335511473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05e406c440 0x7f05e40c6d70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f05d400ecb0 tx=0x7f05d40091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.464+0000 7f05ebfd5700 1 -- 172.21.15.157:0/1335511473 shutdown_connections 2023-10-06T03:57:21.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.464+0000 7f05ebfd5700 1 --2- 172.21.15.157:0/1335511473 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f05d00671e0 0x7f05d0069690 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.464+0000 7f05ebfd5700 1 --2- 172.21.15.157:0/1335511473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05e406c810 0x7f05e4145cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.464+0000 7f05ebfd5700 1 --2- 172.21.15.157:0/1335511473 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05e4069950 0x7f05e414d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.464+0000 7f05ebfd5700 1 --2- 172.21.15.157:0/1335511473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05e406c440 0x7f05e40c6d70 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.464+0000 7f05ebfd5700 1 -- 172.21.15.157:0/1335511473 >> 172.21.15.157:0/1335511473 conn(0x7f05e40c82e0 msgr2=0x7f05e406ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:21.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.465+0000 7f05ebfd5700 1 -- 172.21.15.157:0/1335511473 shutdown_connections 2023-10-06T03:57:21.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.465+0000 7f05ebfd5700 1 -- 172.21.15.157:0/1335511473 wait complete. 2023-10-06T03:57:21.494 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2023-10-06T03:57:21.494 DEBUG:teuthology.parallel:result is None 2023-10-06T03:57:21.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.515+0000 7f1a0dc0e700 1 -- 172.21.15.157:0/2734807942 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f1a081581c0 con 0x7f1a08158a30 2023-10-06T03:57:21.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.518+0000 7f19ff7fe700 1 -- 172.21.15.157:0/2734807942 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1a000790a0 con 0x7f1a08158a30 2023-10-06T03:57:21.520 INFO:teuthology.orchestra.run.smithi157.stdout:55834574854 2023-10-06T03:57:21.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.520+0000 7f1a0dc0e700 1 -- 172.21.15.157:0/2734807942 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f19f0067080 msgr2=0x7f19f0069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.520+0000 7f1a0dc0e700 1 --2- 172.21.15.157:0/2734807942 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f19f0067080 0x7f19f0069530 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1a08061120 tx=0x7f19f8006f90 comp rx=0 tx=0).stop 2023-10-06T03:57:21.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.520+0000 7f1a0dc0e700 1 -- 172.21.15.157:0/2734807942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a08158a30 msgr2=0x7f1a08165ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.520+0000 7f1a0dc0e700 1 --2- 172.21.15.157:0/2734807942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a08158a30 0x7f1a08165ca0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1a0000a520 tx=0x7f1a00038000 comp rx=0 tx=0).stop 2023-10-06T03:57:21.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.521+0000 7f1a0dc0e700 1 -- 172.21.15.157:0/2734807942 shutdown_connections 2023-10-06T03:57:21.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.521+0000 7f1a0dc0e700 1 --2- 172.21.15.157:0/2734807942 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f19f0067080 0x7f19f0069530 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.521+0000 7f1a0dc0e700 1 --2- 172.21.15.157:0/2734807942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19ec005410 0x7f1a08159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.521+0000 7f1a0dc0e700 1 --2- 172.21.15.157:0/2734807942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1a08158a30 0x7f1a08165ca0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.521+0000 7f1a0dc0e700 1 --2- 172.21.15.157:0/2734807942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a081346b0 0x7f1a081584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.521+0000 7f1a0dc0e700 1 -- 172.21.15.157:0/2734807942 >> 172.21.15.157:0/2734807942 conn(0x7f1a0806b6a0 msgr2=0x7f1a080c53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:21.523 INFO:teuthology.orchestra.run.smithi157.stdout:55834574854 2023-10-06T03:57:21.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.522+0000 7f1a0dc0e700 1 -- 172.21.15.157:0/2734807942 shutdown_connections 2023-10-06T03:57:21.524 INFO:teuthology.orchestra.run.smithi157.stdout:55834574854 2023-10-06T03:57:21.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.523+0000 7ffa3d58c700 1 -- 172.21.15.157:0/703356541 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ffa3804bb40 con 0x7ffa3806c810 2023-10-06T03:57:21.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.522+0000 7f4f1788f700 1 -- 172.21.15.157:0/3649544010 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4f100adb50 con 0x7f4f1006aa40 2023-10-06T03:57:21.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.522+0000 7f4f0d7fa700 1 -- 172.21.15.157:0/3649544010 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4f04054a50 con 0x7f4f1006aa40 2023-10-06T03:57:21.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.521+0000 7fde4ae83700 1 -- 172.21.15.157:0/2043386849 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fde440c2670 con 0x7fde4406ddf0 2023-10-06T03:57:21.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.521+0000 7fde38ff9700 1 -- 172.21.15.157:0/2043386849 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fde3c054c70 con 0x7fde4406ddf0 2023-10-06T03:57:21.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.524+0000 7fde4ae83700 1 -- 172.21.15.157:0/2043386849 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fde30066f60 msgr2=0x7fde30069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.524+0000 7fde4ae83700 1 --2- 172.21.15.157:0/2043386849 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fde30066f60 0x7fde30069410 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fde340097c0 tx=0x7fde34006ce0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.524+0000 7fde4ae83700 1 -- 172.21.15.157:0/2043386849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde4406ddf0 msgr2=0x7fde440bec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.524+0000 7fde4ae83700 1 --2- 172.21.15.157:0/2043386849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde4406ddf0 0x7fde440bec80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00bad0 tx=0x7fde3c009780 comp rx=0 tx=0).stop 2023-10-06T03:57:21.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.525+0000 7f4f1788f700 1 -- 172.21.15.157:0/3649544010 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4ef4067000 msgr2=0x7f4ef40694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.525+0000 7f4f1788f700 1 --2- 172.21.15.157:0/3649544010 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4ef4067000 0x7f4ef40694b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4efc0099b0 tx=0x7f4efc016000 comp rx=0 tx=0).stop 2023-10-06T03:57:21.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.525+0000 7f4f1788f700 1 -- 172.21.15.157:0/3649544010 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f1006aa40 msgr2=0x7f4f10143e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.525+0000 7f4f1788f700 1 --2- 172.21.15.157:0/3649544010 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f1006aa40 0x7f4f10143e80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400bbb0 tx=0x7f4f04009840 comp rx=0 tx=0).stop 2023-10-06T03:57:21.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.525+0000 7f4f1788f700 1 -- 172.21.15.157:0/3649544010 shutdown_connections 2023-10-06T03:57:21.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.525+0000 7f4f1788f700 1 --2- 172.21.15.157:0/3649544010 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4ef4067000 0x7f4ef40694b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.525+0000 7f4f1788f700 1 --2- 172.21.15.157:0/3649544010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f100c4840 0x7f4f100c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.525+0000 7f4f1788f700 1 --2- 172.21.15.157:0/3649544010 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f1006aa40 0x7f4f10143e80 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400bbb0 tx=0x7f4f04009840 comp rx=0 tx=0).stop 2023-10-06T03:57:21.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.525+0000 7f4f1788f700 1 --2- 172.21.15.157:0/3649544010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f100c4470 0x7f4f100c70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.525+0000 7f4f1788f700 1 -- 172.21.15.157:0/3649544010 >> 172.21.15.157:0/3649544010 conn(0x7f4f100c82e0 msgr2=0x7f4f1006c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:21.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.526+0000 7fde4ae83700 1 -- 172.21.15.157:0/2043386849 shutdown_connections 2023-10-06T03:57:21.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.526+0000 7fde4ae83700 1 --2- 172.21.15.157:0/2043386849 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fde30066f60 0x7fde30069410 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.526+0000 7fde4ae83700 1 --2- 172.21.15.157:0/2043386849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde4406ddf0 0x7fde440bec80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.526+0000 7fde4ae83700 1 --2- 172.21.15.157:0/2043386849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde4406a780 0x7fde4416ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.526+0000 7fde4ae83700 1 --2- 172.21.15.157:0/2043386849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde4406ab50 0x7fde440be740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.526+0000 7fde4ae83700 1 -- 172.21.15.157:0/2043386849 >> 172.21.15.157:0/2043386849 conn(0x7fde4406b510 msgr2=0x7fde44155c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:21.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.526+0000 7fde4ae83700 1 -- 172.21.15.157:0/2043386849 shutdown_connections 2023-10-06T03:57:21.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.526+0000 7fde4ae83700 1 -- 172.21.15.157:0/2043386849 wait complete. 2023-10-06T03:57:21.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.526+0000 7ffa1effd700 1 -- 172.21.15.157:0/703356541 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffa2c055680 con 0x7ffa3806c810 2023-10-06T03:57:21.530 INFO:teuthology.orchestra.run.smithi157.stdout:55834574854 2023-10-06T03:57:21.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.527+0000 7f1a0dc0e700 1 -- 172.21.15.157:0/2734807942 wait complete. 2023-10-06T03:57:21.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.528+0000 7f4f1788f700 1 -- 172.21.15.157:0/3649544010 shutdown_connections 2023-10-06T03:57:21.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.528+0000 7f4f1788f700 1 -- 172.21.15.157:0/3649544010 wait complete. 2023-10-06T03:57:21.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.530+0000 7ffa3d58c700 1 -- 172.21.15.157:0/703356541 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ffa24066f60 msgr2=0x7ffa24069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.530+0000 7ffa3d58c700 1 --2- 172.21.15.157:0/703356541 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ffa24066f60 0x7ffa24069410 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffa28000c00 tx=0x7ffa28007ea0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.530+0000 7ffa3d58c700 1 -- 172.21.15.157:0/703356541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa3806c810 msgr2=0x7ffa381445e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.530+0000 7ffa3d58c700 1 --2- 172.21.15.157:0/703356541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa3806c810 0x7ffa381445e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c00e930 tx=0x7ffa2c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.530+0000 7ffa3d58c700 1 -- 172.21.15.157:0/703356541 shutdown_connections 2023-10-06T03:57:21.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.530+0000 7ffa3d58c700 1 --2- 172.21.15.157:0/703356541 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ffa24066f60 0x7ffa24069410 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffa28000c00 tx=0x7ffa28007ea0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.530+0000 7ffa3d58c700 1 --2- 172.21.15.157:0/703356541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa380c2c50 0x7ffa38154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.530+0000 7ffa3d58c700 1 --2- 172.21.15.157:0/703356541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa3806c440 0x7ffa3814bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.530+0000 7ffa3d58c700 1 --2- 172.21.15.157:0/703356541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa3806c810 0x7ffa381445e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.530+0000 7ffa3d58c700 1 -- 172.21.15.157:0/703356541 >> 172.21.15.157:0/703356541 conn(0x7ffa380c82e0 msgr2=0x7ffa3806a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:21.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.531+0000 7ffa3d58c700 1 -- 172.21.15.157:0/703356541 shutdown_connections 2023-10-06T03:57:21.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.531+0000 7ffa3d58c700 1 -- 172.21.15.157:0/703356541 wait complete. 2023-10-06T03:57:21.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.533+0000 7f35d4b04700 1 -- 172.21.15.157:0/2212375302 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f35d006caa0 con 0x7f35d006e030 2023-10-06T03:57:21.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.537+0000 7f35be7fc700 1 -- 172.21.15.157:0/2212375302 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f35c00549a0 con 0x7f35d006e030 2023-10-06T03:57:21.540 INFO:teuthology.orchestra.run.smithi157.stdout:55834574854 2023-10-06T03:57:21.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.542+0000 7f35d4b04700 1 -- 172.21.15.157:0/2212375302 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f35b0066f30 msgr2=0x7f35b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.542+0000 7f35d4b04700 1 --2- 172.21.15.157:0/2212375302 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f35b0066f30 0x7f35b00693e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f35d0144a00 tx=0x7f35c4009040 comp rx=0 tx=0).stop 2023-10-06T03:57:21.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.542+0000 7f35d4b04700 1 -- 172.21.15.157:0/2212375302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35d006e030 msgr2=0x7f35d0165e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.542+0000 7f35d4b04700 1 --2- 172.21.15.157:0/2212375302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35d006e030 0x7f35d0165e70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f35c000b5e0 tx=0x7f35c00090f0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.542+0000 7f35d4b04700 1 -- 172.21.15.157:0/2212375302 shutdown_connections 2023-10-06T03:57:21.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.542+0000 7f35d4b04700 1 --2- 172.21.15.157:0/2212375302 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f35b0066f30 0x7f35b00693e0 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f35d0144a00 tx=0x7f35c4009040 comp rx=0 tx=0).stop 2023-10-06T03:57:21.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.542+0000 7f35d4b04700 1 --2- 172.21.15.157:0/2212375302 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35d01531a0 0x7f35d01522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.542+0000 7f35d4b04700 1 --2- 172.21.15.157:0/2212375302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35d006e030 0x7f35d0165e70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.542+0000 7f35d4b04700 1 --2- 172.21.15.157:0/2212375302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35d00adbf0 0x7f35d0151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.542+0000 7f35d4b04700 1 -- 172.21.15.157:0/2212375302 >> 172.21.15.157:0/2212375302 conn(0x7f35d00c82e0 msgr2=0x7f35d006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:21.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.544+0000 7f35d4b04700 1 -- 172.21.15.157:0/2212375302 shutdown_connections 2023-10-06T03:57:21.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.548+0000 7f35d4b04700 1 -- 172.21.15.157:0/2212375302 wait complete. 2023-10-06T03:57:21.551 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2023-10-06T03:57:21.551 DEBUG:teuthology.parallel:result is None 2023-10-06T03:57:21.553 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2023-10-06T03:57:21.553 DEBUG:teuthology.parallel:result is None 2023-10-06T03:57:21.554 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2023-10-06T03:57:21.554 DEBUG:teuthology.parallel:result is None 2023-10-06T03:57:21.556 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2023-10-06T03:57:21.556 DEBUG:teuthology.parallel:result is None 2023-10-06T03:57:21.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.556+0000 7fbf735c3700 1 -- 172.21.15.157:0/3889652641 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fbf6c15b060 con 0x7fbf6c06ab50 2023-10-06T03:57:21.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.557+0000 7fbf617fa700 1 -- 172.21.15.157:0/3889652641 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbf5c054990 con 0x7fbf6c06ab50 2023-10-06T03:57:21.558 INFO:teuthology.orchestra.run.smithi157.stdout:55834574854 2023-10-06T03:57:21.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.559+0000 7fbf735c3700 1 -- 172.21.15.157:0/3889652641 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbf50067000 msgr2=0x7fbf500694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.559+0000 7fbf735c3700 1 --2- 172.21.15.157:0/3889652641 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbf50067000 0x7fbf500694b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c067d20 tx=0x7fbf68016000 comp rx=0 tx=0).stop 2023-10-06T03:57:21.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.559+0000 7fbf735c3700 1 -- 172.21.15.157:0/3889652641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf6c06ab50 msgr2=0x7fbf6c0c9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.559+0000 7fbf735c3700 1 --2- 172.21.15.157:0/3889652641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf6c06ab50 0x7fbf6c0c9860 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00b5e0 tx=0x7fbf5c0090f0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.559+0000 7fbf735c3700 1 -- 172.21.15.157:0/3889652641 shutdown_connections 2023-10-06T03:57:21.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.559+0000 7fbf735c3700 1 --2- 172.21.15.157:0/3889652641 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbf50067000 0x7fbf500694b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.559+0000 7fbf735c3700 1 --2- 172.21.15.157:0/3889652641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf6c0c3d70 0x7fbf6c1346a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.559+0000 7fbf735c3700 1 --2- 172.21.15.157:0/3889652641 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf6c06a780 0x7fbf6c0c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.559+0000 7fbf735c3700 1 --2- 172.21.15.157:0/3889652641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf6c06ab50 0x7fbf6c0c9860 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.559+0000 7fbf735c3700 1 -- 172.21.15.157:0/3889652641 >> 172.21.15.157:0/3889652641 conn(0x7fbf6c06b510 msgr2=0x7fbf6c0c6430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:21.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.559+0000 7fbf735c3700 1 -- 172.21.15.157:0/3889652641 shutdown_connections 2023-10-06T03:57:21.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.559+0000 7fbf735c3700 1 -- 172.21.15.157:0/3889652641 wait complete. 2023-10-06T03:57:21.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.560+0000 7fa510431700 1 -- 172.21.15.157:0/2373825451 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fa4f0005560 con 0x7fa50814ad80 2023-10-06T03:57:21.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.561+0000 7fa4fe7fc700 1 -- 172.21.15.157:0/2373825451 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa500018030 con 0x7fa50814ad80 2023-10-06T03:57:21.563 INFO:teuthology.orchestra.run.smithi157.stdout:55834574855 2023-10-06T03:57:21.564 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2023-10-06T03:57:21.564 DEBUG:teuthology.parallel:result is None 2023-10-06T03:57:21.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.563+0000 7fa510431700 1 -- 172.21.15.157:0/2373825451 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa4ec066fb0 msgr2=0x7fa4ec069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.563+0000 7fa510431700 1 --2- 172.21.15.157:0/2373825451 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa4ec066fb0 0x7fa4ec069460 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa50806c010 tx=0x7fa4f8007ea0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.563+0000 7fa510431700 1 -- 172.21.15.157:0/2373825451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa50814ad80 msgr2=0x7fa50814e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.563+0000 7fa510431700 1 --2- 172.21.15.157:0/2373825451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa50814ad80 0x7fa50814e990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa500009df0 tx=0x7fa50000b040 comp rx=0 tx=0).stop 2023-10-06T03:57:21.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.563+0000 7fa510431700 1 -- 172.21.15.157:0/2373825451 shutdown_connections 2023-10-06T03:57:21.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.563+0000 7fa510431700 1 --2- 172.21.15.157:0/2373825451 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa4ec066fb0 0x7fa4ec069460 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.563+0000 7fa510431700 1 --2- 172.21.15.157:0/2373825451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5080bf8f0 0x7fa5080bfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.563+0000 7fa510431700 1 --2- 172.21.15.157:0/2373825451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa50814ad80 0x7fa50814e990 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.563+0000 7fa510431700 1 --2- 172.21.15.157:0/2373825451 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa508153fe0 0x7fa5080c0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.563+0000 7fa510431700 1 -- 172.21.15.157:0/2373825451 >> 172.21.15.157:0/2373825451 conn(0x7fa5080c8b20 msgr2=0x7fa5080c3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:21.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.563+0000 7fa510431700 1 -- 172.21.15.157:0/2373825451 shutdown_connections 2023-10-06T03:57:21.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.563+0000 7fa510431700 1 -- 172.21.15.157:0/2373825451 wait complete. 2023-10-06T03:57:21.574 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2023-10-06T03:57:21.574 DEBUG:teuthology.parallel:result is None 2023-10-06T03:57:21.578 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2023-10-06T03:57:21.578 DEBUG:teuthology.parallel:result is None 2023-10-06T03:57:21.578 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T03:57:21.578 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:57:21.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.732+0000 7fd46772e700 1 Processor -- start 2023-10-06T03:57:21.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.732+0000 7fd46772e700 1 -- start start 2023-10-06T03:57:21.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.733+0000 7fd46772e700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4600b1d40 0x7fd460159df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.733+0000 7fd46772e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd4600b2110 0x7fd4600af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.733+0000 7fd46772e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd46006ced0 con 0x7fd46006c1f0 2023-10-06T03:57:21.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.733+0000 7fd46772e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd46006cc10 con 0x7fd4600b2110 2023-10-06T03:57:21.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.733+0000 7fd46772e700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46006e570 con 0x7fd4600b1d40 2023-10-06T03:57:21.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.733+0000 7fd464cc9700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd4600b2110 0x7fd4600af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.733+0000 7fd4654ca700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4600b1d40 0x7fd460159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.733+0000 7fd464cc9700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd4600b2110 0x7fd4600af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:35320/0 (socket says 172.21.15.157:35320) 2023-10-06T03:57:21.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.733+0000 7fd464cc9700 1 -- 172.21.15.157:0/2805696167 learned_addr learned my addr 172.21.15.157:0/2805696167 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:21.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.733+0000 7fd465ccb700 1 --1- 172.21.15.157:0/2805696167 >> v1:172.21.15.175:6789/0 conn(0x7fd46006c1f0 0x7fd46014fbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.157:38386/0 (socket says 172.21.15.157:38386) 2023-10-06T03:57:21.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.733+0000 7fd4654ca700 1 -- 172.21.15.157:0/2805696167 >> v2:172.21.15.175:3300/0 conn(0x7fd4600b2110 msgr2=0x7fd4600af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:21.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.733+0000 7fd4654ca700 1 --2- 172.21.15.157:0/2805696167 >> v2:172.21.15.175:3300/0 conn(0x7fd4600b2110 0x7fd4600af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.733+0000 7fd4654ca700 1 -- 172.21.15.157:0/2805696167 >> v1:172.21.15.175:6789/0 conn(0x7fd46006c1f0 legacy=0x7fd46014fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.733+0000 7fd4654ca700 1 -- 172.21.15.157:0/2805696167 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46006d7b0 con 0x7fd4600b1d40 2023-10-06T03:57:21.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.734+0000 7fd4654ca700 1 --2- 172.21.15.157:0/2805696167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4600b1d40 0x7fd460159df0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0098e0 tx=0x7fd45c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc236ae741f4cfb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:21.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.734+0000 7fd457fff700 1 -- 172.21.15.157:0/2805696167 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c00b6d0 con 0x7fd4600b1d40 2023-10-06T03:57:21.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.734+0000 7fd457fff700 1 -- 172.21.15.157:0/2805696167 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45c00b830 con 0x7fd4600b1d40 2023-10-06T03:57:21.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.734+0000 7fd457fff700 1 -- 172.21.15.157:0/2805696167 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c00bbd0 con 0x7fd4600b1d40 2023-10-06T03:57:21.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.734+0000 7fd46772e700 1 -- 172.21.15.157:0/2805696167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4600b1d40 msgr2=0x7fd460159df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.734+0000 7fd46772e700 1 --2- 172.21.15.157:0/2805696167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4600b1d40 0x7fd460159df0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0098e0 tx=0x7fd45c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.734+0000 7fd46772e700 1 -- 172.21.15.157:0/2805696167 shutdown_connections 2023-10-06T03:57:21.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.734+0000 7fd46772e700 1 --2- 172.21.15.157:0/2805696167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4600b1d40 0x7fd460159df0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.734+0000 7fd46772e700 1 --2- 172.21.15.157:0/2805696167 >> v2:172.21.15.175:3300/0 conn(0x7fd4600b2110 0x7fd4600af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.734+0000 7fd46772e700 1 -- 172.21.15.157:0/2805696167 >> 172.21.15.157:0/2805696167 conn(0x7fd4600cc6e0 msgr2=0x7fd4600ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:21.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.734+0000 7fd46772e700 1 -- 172.21.15.157:0/2805696167 shutdown_connections 2023-10-06T03:57:21.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.734+0000 7fd46772e700 1 -- 172.21.15.157:0/2805696167 wait complete. 2023-10-06T03:57:21.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.735+0000 7fd46772e700 1 Processor -- start 2023-10-06T03:57:21.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.735+0000 7fd46772e700 1 -- start start 2023-10-06T03:57:21.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.736+0000 7fd46772e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd46006c1f0 0x7fd460152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.736+0000 7fd46772e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4600b1d40 0x7fd4601512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.736+0000 7fd46772e700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4600b2110 0x7fd460151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.736+0000 7fd46772e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4601538e0 con 0x7fd4600b1d40 2023-10-06T03:57:21.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.736+0000 7fd46772e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4600c64d0 con 0x7fd46006c1f0 2023-10-06T03:57:21.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.736+0000 7fd46772e700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4600c9a80 con 0x7fd4600b2110 2023-10-06T03:57:21.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.736+0000 7fd464cc9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4600b1d40 0x7fd4601512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.736+0000 7fd464cc9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4600b1d40 0x7fd4601512f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34642/0 (socket says 172.21.15.157:34642) 2023-10-06T03:57:21.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.736+0000 7fd464cc9700 1 -- 172.21.15.157:0/4205132687 learned_addr learned my addr 172.21.15.157:0/4205132687 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:21.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.736+0000 7fd465ccb700 1 --2- 172.21.15.157:0/4205132687 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4600b2110 0x7fd460151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.736+0000 7fd4654ca700 1 --2- 172.21.15.157:0/4205132687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd46006c1f0 0x7fd460152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.737+0000 7fd464cc9700 1 -- 172.21.15.157:0/4205132687 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4600b2110 msgr2=0x7fd460151830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.737+0000 7fd464cc9700 1 --2- 172.21.15.157:0/4205132687 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4600b2110 0x7fd460151830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.737+0000 7fd464cc9700 1 -- 172.21.15.157:0/4205132687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd46006c1f0 msgr2=0x7fd460152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.737+0000 7fd464cc9700 1 --2- 172.21.15.157:0/4205132687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd46006c1f0 0x7fd460152860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.737+0000 7fd464cc9700 1 -- 172.21.15.157:0/4205132687 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd45c009420 con 0x7fd4600b1d40 2023-10-06T03:57:21.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.737+0000 7fd4654ca700 1 --2- 172.21.15.157:0/4205132687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd46006c1f0 0x7fd460152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:21.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.737+0000 7fd464cc9700 1 --2- 172.21.15.157:0/4205132687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4600b1d40 0x7fd4601512f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd45000e980 tx=0x7fd450009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:21.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.737+0000 7fd4557fa700 1 -- 172.21.15.157:0/4205132687 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45000ee70 con 0x7fd4600b1d40 2023-10-06T03:57:21.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.737+0000 7fd4557fa700 1 -- 172.21.15.157:0/4205132687 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45000c850 con 0x7fd4600b1d40 2023-10-06T03:57:21.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.737+0000 7fd4557fa700 1 -- 172.21.15.157:0/4205132687 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45000cbd0 con 0x7fd4600b1d40 2023-10-06T03:57:21.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.737+0000 7fd46772e700 1 -- 172.21.15.157:0/4205132687 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4600c9d20 con 0x7fd4600b1d40 2023-10-06T03:57:21.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.737+0000 7fd46772e700 1 -- 172.21.15.157:0/4205132687 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd460150dd0 con 0x7fd4600b1d40 2023-10-06T03:57:21.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.738+0000 7fd46772e700 1 -- 172.21.15.157:0/4205132687 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd460150ff0 con 0x7fd4600b1d40 2023-10-06T03:57:21.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.738+0000 7fd4557fa700 1 -- 172.21.15.157:0/4205132687 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd450013470 con 0x7fd4600b1d40 2023-10-06T03:57:21.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.739+0000 7fd4557fa700 1 --2- 172.21.15.157:0/4205132687 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd444066f60 0x7fd444069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:21.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.739+0000 7fd4557fa700 1 -- 172.21.15.157:0/4205132687 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd4500848d0 con 0x7fd4600b1d40 2023-10-06T03:57:21.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.739+0000 7fd4654ca700 1 --2- 172.21.15.157:0/4205132687 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd444066f60 0x7fd444069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:21.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.739+0000 7fd4654ca700 1 --2- 172.21.15.157:0/4205132687 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd444066f60 0x7fd444069410 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0093f0 tx=0x7fd45c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:21.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.743+0000 7fd4557fa700 1 -- 172.21.15.157:0/4205132687 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd450051940 con 0x7fd4600b1d40 2023-10-06T03:57:21.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.922+0000 7fd46772e700 1 -- 172.21.15.157:0/4205132687 --> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd46006da90 con 0x7fd444066f60 2023-10-06T03:57:21.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.931+0000 7fd4557fa700 1 -- 172.21.15.157:0/4205132687 <== mgr.4107 v2:172.21.15.175:6832/90661 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286705 (secure 0 0 0) 0x7fd46006da90 con 0x7fd444066f60 2023-10-06T03:57:21.936 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:21.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.934+0000 7fd46772e700 1 -- 172.21.15.157:0/4205132687 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd444066f60 msgr2=0x7fd444069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.934+0000 7fd46772e700 1 --2- 172.21.15.157:0/4205132687 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd444066f60 0x7fd444069410 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0093f0 tx=0x7fd45c017000 comp rx=0 tx=0).stop 2023-10-06T03:57:21.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.934+0000 7fd46772e700 1 -- 172.21.15.157:0/4205132687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4600b1d40 msgr2=0x7fd4601512f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:21.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.935+0000 7fd46772e700 1 --2- 172.21.15.157:0/4205132687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4600b1d40 0x7fd4601512f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd45000e980 tx=0x7fd450009250 comp rx=0 tx=0).stop 2023-10-06T03:57:21.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.935+0000 7fd46772e700 1 -- 172.21.15.157:0/4205132687 shutdown_connections 2023-10-06T03:57:21.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.935+0000 7fd46772e700 1 --2- 172.21.15.157:0/4205132687 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd444066f60 0x7fd444069410 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.935+0000 7fd46772e700 1 --2- 172.21.15.157:0/4205132687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd46006c1f0 0x7fd460152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.935+0000 7fd46772e700 1 --2- 172.21.15.157:0/4205132687 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4600b2110 0x7fd460151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.935+0000 7fd46772e700 1 --2- 172.21.15.157:0/4205132687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4600b1d40 0x7fd4601512f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:21.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.935+0000 7fd46772e700 1 -- 172.21.15.157:0/4205132687 >> 172.21.15.157:0/4205132687 conn(0x7fd4600cc6e0 msgr2=0x7fd46014f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:21.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.935+0000 7fd46772e700 1 -- 172.21.15.157:0/4205132687 shutdown_connections 2023-10-06T03:57:21.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:21.935+0000 7fd46772e700 1 -- 172.21.15.157:0/4205132687 wait complete. 2023-10-06T03:57:21.941 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-10-06T03:57:21.948 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-06T03:57:21.427975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2380188,"kb_used_data":1884,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752594532,"statfs":{"total":773094113280,"available":770656800768,"internally_reserved":0,"allocated":1929216,"data_stored":1815075,"data_compressed":64858,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":58,"num_write_kb":275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002727"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378564+0000","last_change":"2023-10-06T03:57:11.372019+0000","last_active":"2023-10-06T03:57:11.378564+0000","last_peered":"2023-10-06T03:57:11.378564+0000","last_clean":"2023-10-06T03:57:11.378564+0000","last_became_active":"2023-10-06T03:57:11.371756+0000","last_became_peered":"2023-10-06T03:57:11.371756+0000","last_unstale":"2023-10-06T03:57:11.378564+0000","last_undegraded":"2023-10-06T03:57:11.378564+0000","last_fullsized":"2023-10-06T03:57:11.378564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:50.086528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372505+0000","last_change":"2023-10-06T03:57:11.370371+0000","last_active":"2023-10-06T03:57:11.372505+0000","last_peered":"2023-10-06T03:57:11.372505+0000","last_clean":"2023-10-06T03:57:11.372505+0000","last_became_active":"2023-10-06T03:57:11.370070+0000","last_became_peered":"2023-10-06T03:57:11.370070+0000","last_unstale":"2023-10-06T03:57:11.372505+0000","last_undegraded":"2023-10-06T03:57:11.372505+0000","last_fullsized":"2023-10-06T03:57:11.372505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:00.917607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371725+0000","last_change":"2023-10-06T03:57:11.366298+0000","last_active":"2023-10-06T03:57:11.371725+0000","last_peered":"2023-10-06T03:57:11.371725+0000","last_clean":"2023-10-06T03:57:11.371725+0000","last_became_active":"2023-10-06T03:57:11.366012+0000","last_became_peered":"2023-10-06T03:57:11.366012+0000","last_unstale":"2023-10-06T03:57:11.371725+0000","last_undegraded":"2023-10-06T03:57:11.371725+0000","last_fullsized":"2023-10-06T03:57:11.371725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:45.802287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.374334+0000","last_change":"2023-10-06T03:57:11.365933+0000","last_active":"2023-10-06T03:57:11.374334+0000","last_peered":"2023-10-06T03:57:11.374334+0000","last_clean":"2023-10-06T03:57:11.374334+0000","last_became_active":"2023-10-06T03:57:11.365673+0000","last_became_peered":"2023-10-06T03:57:11.365673+0000","last_unstale":"2023-10-06T03:57:11.374334+0000","last_undegraded":"2023-10-06T03:57:11.374334+0000","last_fullsized":"2023-10-06T03:57:11.374334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:40.953382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.365753+0000","last_change":"2023-10-06T03:57:11.361569+0000","last_active":"2023-10-06T03:57:11.365753+0000","last_peered":"2023-10-06T03:57:11.365753+0000","last_clean":"2023-10-06T03:57:11.365753+0000","last_became_active":"2023-10-06T03:57:11.361302+0000","last_became_peered":"2023-10-06T03:57:11.361302+0000","last_unstale":"2023-10-06T03:57:11.365753+0000","last_undegraded":"2023-10-06T03:57:11.365753+0000","last_fullsized":"2023-10-06T03:57:11.365753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:57.552493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372859+0000","last_change":"2023-10-06T03:57:11.369953+0000","last_active":"2023-10-06T03:57:11.372859+0000","last_peered":"2023-10-06T03:57:11.372859+0000","last_clean":"2023-10-06T03:57:11.372859+0000","last_became_active":"2023-10-06T03:57:11.368775+0000","last_became_peered":"2023-10-06T03:57:11.368775+0000","last_unstale":"2023-10-06T03:57:11.372859+0000","last_undegraded":"2023-10-06T03:57:11.372859+0000","last_fullsized":"2023-10-06T03:57:11.372859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:22.368117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.380867+0000","last_change":"2023-10-06T03:57:11.378048+0000","last_active":"2023-10-06T03:57:11.380867+0000","last_peered":"2023-10-06T03:57:11.380867+0000","last_clean":"2023-10-06T03:57:11.380867+0000","last_became_active":"2023-10-06T03:57:11.377786+0000","last_became_peered":"2023-10-06T03:57:11.377786+0000","last_unstale":"2023-10-06T03:57:11.380867+0000","last_undegraded":"2023-10-06T03:57:11.380867+0000","last_fullsized":"2023-10-06T03:57:11.380867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:12:54.218593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377264+0000","last_change":"2023-10-06T03:57:11.374003+0000","last_active":"2023-10-06T03:57:11.377264+0000","last_peered":"2023-10-06T03:57:11.377264+0000","last_clean":"2023-10-06T03:57:11.377264+0000","last_became_active":"2023-10-06T03:57:11.373765+0000","last_became_peered":"2023-10-06T03:57:11.373765+0000","last_unstale":"2023-10-06T03:57:11.377264+0000","last_undegraded":"2023-10-06T03:57:11.377264+0000","last_fullsized":"2023-10-06T03:57:11.377264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:05.838007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378921+0000","last_change":"2023-10-06T03:57:11.373794+0000","last_active":"2023-10-06T03:57:11.378921+0000","last_peered":"2023-10-06T03:57:11.378921+0000","last_clean":"2023-10-06T03:57:11.378921+0000","last_became_active":"2023-10-06T03:57:11.368004+0000","last_became_peered":"2023-10-06T03:57:11.368004+0000","last_unstale":"2023-10-06T03:57:11.378921+0000","last_undegraded":"2023-10-06T03:57:11.378921+0000","last_fullsized":"2023-10-06T03:57:11.378921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:16.760049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.382630+0000","last_change":"2023-10-06T03:57:11.379370+0000","last_active":"2023-10-06T03:57:11.382630+0000","last_peered":"2023-10-06T03:57:11.382630+0000","last_clean":"2023-10-06T03:57:11.382630+0000","last_became_active":"2023-10-06T03:57:11.379107+0000","last_became_peered":"2023-10-06T03:57:11.379107+0000","last_unstale":"2023-10-06T03:57:11.382630+0000","last_undegraded":"2023-10-06T03:57:11.382630+0000","last_fullsized":"2023-10-06T03:57:11.382630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:18.373103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371742+0000","last_change":"2023-10-06T03:57:11.369172+0000","last_active":"2023-10-06T03:57:11.371742+0000","last_peered":"2023-10-06T03:57:11.371742+0000","last_clean":"2023-10-06T03:57:11.371742+0000","last_became_active":"2023-10-06T03:57:11.368256+0000","last_became_peered":"2023-10-06T03:57:11.368256+0000","last_unstale":"2023-10-06T03:57:11.371742+0000","last_undegraded":"2023-10-06T03:57:11.371742+0000","last_fullsized":"2023-10-06T03:57:11.371742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:50.736367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373531+0000","last_change":"2023-10-06T03:57:11.368685+0000","last_active":"2023-10-06T03:57:11.373531+0000","last_peered":"2023-10-06T03:57:11.373531+0000","last_clean":"2023-10-06T03:57:11.373531+0000","last_became_active":"2023-10-06T03:57:11.368425+0000","last_became_peered":"2023-10-06T03:57:11.368425+0000","last_unstale":"2023-10-06T03:57:11.373531+0000","last_undegraded":"2023-10-06T03:57:11.373531+0000","last_fullsized":"2023-10-06T03:57:11.373531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:06.004423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372891+0000","last_change":"2023-10-06T03:57:11.367646+0000","last_active":"2023-10-06T03:57:11.372891+0000","last_peered":"2023-10-06T03:57:11.372891+0000","last_clean":"2023-10-06T03:57:11.372891+0000","last_became_active":"2023-10-06T03:57:11.367079+0000","last_became_peered":"2023-10-06T03:57:11.367079+0000","last_unstale":"2023-10-06T03:57:11.372891+0000","last_undegraded":"2023-10-06T03:57:11.372891+0000","last_fullsized":"2023-10-06T03:57:11.372891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:09:54.331206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377012+0000","last_change":"2023-10-06T03:57:11.373481+0000","last_active":"2023-10-06T03:57:11.377012+0000","last_peered":"2023-10-06T03:57:11.377012+0000","last_clean":"2023-10-06T03:57:11.377012+0000","last_became_active":"2023-10-06T03:57:11.373216+0000","last_became_peered":"2023-10-06T03:57:11.373216+0000","last_unstale":"2023-10-06T03:57:11.377012+0000","last_undegraded":"2023-10-06T03:57:11.377012+0000","last_fullsized":"2023-10-06T03:57:11.377012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:56.083203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.379250+0000","last_change":"2023-10-06T03:57:11.376012+0000","last_active":"2023-10-06T03:57:11.379250+0000","last_peered":"2023-10-06T03:57:11.379250+0000","last_clean":"2023-10-06T03:57:11.379250+0000","last_became_active":"2023-10-06T03:57:11.375689+0000","last_became_peered":"2023-10-06T03:57:11.375689+0000","last_unstale":"2023-10-06T03:57:11.379250+0000","last_undegraded":"2023-10-06T03:57:11.379250+0000","last_fullsized":"2023-10-06T03:57:11.379250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:42.953739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372989+0000","last_change":"2023-10-06T03:57:11.366808+0000","last_active":"2023-10-06T03:57:11.372989+0000","last_peered":"2023-10-06T03:57:11.372989+0000","last_clean":"2023-10-06T03:57:11.372989+0000","last_became_active":"2023-10-06T03:57:11.366543+0000","last_became_peered":"2023-10-06T03:57:11.366543+0000","last_unstale":"2023-10-06T03:57:11.372989+0000","last_undegraded":"2023-10-06T03:57:11.372989+0000","last_fullsized":"2023-10-06T03:57:11.372989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:16.212967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378753+0000","last_change":"2023-10-06T03:57:11.367276+0000","last_active":"2023-10-06T03:57:11.378753+0000","last_peered":"2023-10-06T03:57:11.378753+0000","last_clean":"2023-10-06T03:57:11.378753+0000","last_became_active":"2023-10-06T03:57:11.366767+0000","last_became_peered":"2023-10-06T03:57:11.366767+0000","last_unstale":"2023-10-06T03:57:11.378753+0000","last_undegraded":"2023-10-06T03:57:11.378753+0000","last_fullsized":"2023-10-06T03:57:11.378753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:43.113575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367129+0000","last_change":"2023-10-06T03:57:11.362667+0000","last_active":"2023-10-06T03:57:11.367129+0000","last_peered":"2023-10-06T03:57:11.367129+0000","last_clean":"2023-10-06T03:57:11.367129+0000","last_became_active":"2023-10-06T03:57:11.362377+0000","last_became_peered":"2023-10-06T03:57:11.362377+0000","last_unstale":"2023-10-06T03:57:11.367129+0000","last_undegraded":"2023-10-06T03:57:11.367129+0000","last_fullsized":"2023-10-06T03:57:11.367129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:40.600383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.365567+0000","last_change":"2023-10-06T03:57:11.364126+0000","last_active":"2023-10-06T03:57:11.365567+0000","last_peered":"2023-10-06T03:57:11.365567+0000","last_clean":"2023-10-06T03:57:11.365567+0000","last_became_active":"2023-10-06T03:57:11.363792+0000","last_became_peered":"2023-10-06T03:57:11.363792+0000","last_unstale":"2023-10-06T03:57:11.365567+0000","last_undegraded":"2023-10-06T03:57:11.365567+0000","last_fullsized":"2023-10-06T03:57:11.365567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:49.185450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372624+0000","last_change":"2023-10-06T03:57:11.364974+0000","last_active":"2023-10-06T03:57:11.372624+0000","last_peered":"2023-10-06T03:57:11.372624+0000","last_clean":"2023-10-06T03:57:11.372624+0000","last_became_active":"2023-10-06T03:57:11.364726+0000","last_became_peered":"2023-10-06T03:57:11.364726+0000","last_unstale":"2023-10-06T03:57:11.372624+0000","last_undegraded":"2023-10-06T03:57:11.372624+0000","last_fullsized":"2023-10-06T03:57:11.372624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:56.866733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.379227+0000","last_change":"2023-10-06T03:57:11.375987+0000","last_active":"2023-10-06T03:57:11.379227+0000","last_peered":"2023-10-06T03:57:11.379227+0000","last_clean":"2023-10-06T03:57:11.379227+0000","last_became_active":"2023-10-06T03:57:11.375668+0000","last_became_peered":"2023-10-06T03:57:11.375668+0000","last_unstale":"2023-10-06T03:57:11.379227+0000","last_undegraded":"2023-10-06T03:57:11.379227+0000","last_fullsized":"2023-10-06T03:57:11.379227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:53.810875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.366478+0000","last_change":"2023-10-06T03:57:11.365414+0000","last_active":"2023-10-06T03:57:11.366478+0000","last_peered":"2023-10-06T03:57:11.366478+0000","last_clean":"2023-10-06T03:57:11.366478+0000","last_became_active":"2023-10-06T03:57:11.365149+0000","last_became_peered":"2023-10-06T03:57:11.365149+0000","last_unstale":"2023-10-06T03:57:11.366478+0000","last_undegraded":"2023-10-06T03:57:11.366478+0000","last_fullsized":"2023-10-06T03:57:11.366478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:20.911710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.370638+0000","last_change":"2023-10-06T03:57:11.362366+0000","last_active":"2023-10-06T03:57:11.370638+0000","last_peered":"2023-10-06T03:57:11.370638+0000","last_clean":"2023-10-06T03:57:11.370638+0000","last_became_active":"2023-10-06T03:57:11.362128+0000","last_became_peered":"2023-10-06T03:57:11.362128+0000","last_unstale":"2023-10-06T03:57:11.370638+0000","last_undegraded":"2023-10-06T03:57:11.370638+0000","last_fullsized":"2023-10-06T03:57:11.370638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:53.744674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.379282+0000","last_change":"2023-10-06T03:57:11.375969+0000","last_active":"2023-10-06T03:57:11.379282+0000","last_peered":"2023-10-06T03:57:11.379282+0000","last_clean":"2023-10-06T03:57:11.379282+0000","last_became_active":"2023-10-06T03:57:11.375702+0000","last_became_peered":"2023-10-06T03:57:11.375702+0000","last_unstale":"2023-10-06T03:57:11.379282+0000","last_undegraded":"2023-10-06T03:57:11.379282+0000","last_fullsized":"2023-10-06T03:57:11.379282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:31.204231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377763+0000","last_change":"2023-10-06T03:57:11.375099+0000","last_active":"2023-10-06T03:57:11.377763+0000","last_peered":"2023-10-06T03:57:11.377763+0000","last_clean":"2023-10-06T03:57:11.377763+0000","last_became_active":"2023-10-06T03:57:11.374842+0000","last_became_peered":"2023-10-06T03:57:11.374842+0000","last_unstale":"2023-10-06T03:57:11.377763+0000","last_undegraded":"2023-10-06T03:57:11.377763+0000","last_fullsized":"2023-10-06T03:57:11.377763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:54.438974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.369108+0000","last_change":"2023-10-06T03:57:11.365900+0000","last_active":"2023-10-06T03:57:11.369108+0000","last_peered":"2023-10-06T03:57:11.369108+0000","last_clean":"2023-10-06T03:57:11.369108+0000","last_became_active":"2023-10-06T03:57:11.361349+0000","last_became_peered":"2023-10-06T03:57:11.361349+0000","last_unstale":"2023-10-06T03:57:11.369108+0000","last_undegraded":"2023-10-06T03:57:11.369108+0000","last_fullsized":"2023-10-06T03:57:11.369108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:30.646781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378945+0000","last_change":"2023-10-06T03:57:11.376082+0000","last_active":"2023-10-06T03:57:11.378945+0000","last_peered":"2023-10-06T03:57:11.378945+0000","last_clean":"2023-10-06T03:57:11.378945+0000","last_became_active":"2023-10-06T03:57:11.375846+0000","last_became_peered":"2023-10-06T03:57:11.375846+0000","last_unstale":"2023-10-06T03:57:11.378945+0000","last_undegraded":"2023-10-06T03:57:11.378945+0000","last_fullsized":"2023-10-06T03:57:11.378945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:28.080614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:12.369461+0000","last_change":"2023-10-06T03:57:10.348317+0000","last_active":"2023-10-06T03:57:12.369461+0000","last_peered":"2023-10-06T03:57:12.369461+0000","last_clean":"2023-10-06T03:57:12.369461+0000","last_became_active":"2023-10-06T03:57:10.348078+0000","last_became_peered":"2023-10-06T03:57:10.348078+0000","last_unstale":"2023-10-06T03:57:12.369461+0000","last_undegraded":"2023-10-06T03:57:12.369461+0000","last_fullsized":"2023-10-06T03:57:12.369461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:55.058324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372610+0000","last_change":"2023-10-06T03:57:11.370506+0000","last_active":"2023-10-06T03:57:11.372610+0000","last_peered":"2023-10-06T03:57:11.372610+0000","last_clean":"2023-10-06T03:57:11.372610+0000","last_became_active":"2023-10-06T03:57:11.370245+0000","last_became_peered":"2023-10-06T03:57:11.370245+0000","last_unstale":"2023-10-06T03:57:11.372610+0000","last_undegraded":"2023-10-06T03:57:11.372610+0000","last_fullsized":"2023-10-06T03:57:11.372610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:47:18.189772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367542+0000","last_change":"2023-10-06T03:57:10.345401+0000","last_active":"2023-10-06T03:57:11.367542+0000","last_peered":"2023-10-06T03:57:11.367542+0000","last_clean":"2023-10-06T03:57:11.367542+0000","last_became_active":"2023-10-06T03:57:10.345150+0000","last_became_peered":"2023-10-06T03:57:10.345150+0000","last_unstale":"2023-10-06T03:57:11.367542+0000","last_undegraded":"2023-10-06T03:57:11.367542+0000","last_fullsized":"2023-10-06T03:57:11.367542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:25.758193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.379135+0000","last_change":"2023-10-06T03:57:11.376046+0000","last_active":"2023-10-06T03:57:11.379135+0000","last_peered":"2023-10-06T03:57:11.379135+0000","last_clean":"2023-10-06T03:57:11.379135+0000","last_became_active":"2023-10-06T03:57:11.375790+0000","last_became_peered":"2023-10-06T03:57:11.375790+0000","last_unstale":"2023-10-06T03:57:11.379135+0000","last_undegraded":"2023-10-06T03:57:11.379135+0000","last_fullsized":"2023-10-06T03:57:11.379135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:33.775431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.359916+0000","last_change":"2023-10-06T03:57:10.347562+0000","last_active":"2023-10-06T03:57:11.359916+0000","last_peered":"2023-10-06T03:57:11.359916+0000","last_clean":"2023-10-06T03:57:11.359916+0000","last_became_active":"2023-10-06T03:57:10.347294+0000","last_became_peered":"2023-10-06T03:57:10.347294+0000","last_unstale":"2023-10-06T03:57:11.359916+0000","last_undegraded":"2023-10-06T03:57:11.359916+0000","last_fullsized":"2023-10-06T03:57:11.359916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:25.090992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.375571+0000","last_change":"2023-10-06T03:57:11.372022+0000","last_active":"2023-10-06T03:57:11.375571+0000","last_peered":"2023-10-06T03:57:11.375571+0000","last_clean":"2023-10-06T03:57:11.375571+0000","last_became_active":"2023-10-06T03:57:11.371758+0000","last_became_peered":"2023-10-06T03:57:11.371758+0000","last_unstale":"2023-10-06T03:57:11.375571+0000","last_undegraded":"2023-10-06T03:57:11.375571+0000","last_fullsized":"2023-10-06T03:57:11.375571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:10.922157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.354126+0000","last_change":"2023-10-06T03:57:10.347406+0000","last_active":"2023-10-06T03:57:11.354126+0000","last_peered":"2023-10-06T03:57:11.354126+0000","last_clean":"2023-10-06T03:57:11.354126+0000","last_became_active":"2023-10-06T03:57:10.347151+0000","last_became_peered":"2023-10-06T03:57:10.347151+0000","last_unstale":"2023-10-06T03:57:11.354126+0000","last_undegraded":"2023-10-06T03:57:11.354126+0000","last_fullsized":"2023-10-06T03:57:11.354126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:20.427620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372580+0000","last_change":"2023-10-06T03:57:11.370240+0000","last_active":"2023-10-06T03:57:11.372580+0000","last_peered":"2023-10-06T03:57:11.372580+0000","last_clean":"2023-10-06T03:57:11.372580+0000","last_became_active":"2023-10-06T03:57:11.369938+0000","last_became_peered":"2023-10-06T03:57:11.369938+0000","last_unstale":"2023-10-06T03:57:11.372580+0000","last_undegraded":"2023-10-06T03:57:11.372580+0000","last_fullsized":"2023-10-06T03:57:11.372580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:07.105832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.364676+0000","last_change":"2023-10-06T03:57:10.346054+0000","last_active":"2023-10-06T03:57:11.364676+0000","last_peered":"2023-10-06T03:57:11.364676+0000","last_clean":"2023-10-06T03:57:11.364676+0000","last_became_active":"2023-10-06T03:57:10.345721+0000","last_became_peered":"2023-10-06T03:57:10.345721+0000","last_unstale":"2023-10-06T03:57:11.364676+0000","last_undegraded":"2023-10-06T03:57:11.364676+0000","last_fullsized":"2023-10-06T03:57:11.364676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:54.481304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378702+0000","last_change":"2023-10-06T03:57:11.376525+0000","last_active":"2023-10-06T03:57:11.378702+0000","last_peered":"2023-10-06T03:57:11.378702+0000","last_clean":"2023-10-06T03:57:11.378702+0000","last_became_active":"2023-10-06T03:57:11.376291+0000","last_became_peered":"2023-10-06T03:57:11.376291+0000","last_unstale":"2023-10-06T03:57:11.378702+0000","last_undegraded":"2023-10-06T03:57:11.378702+0000","last_fullsized":"2023-10-06T03:57:11.378702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:31.993426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:16.371903+0000","last_change":"2023-10-06T03:57:10.340531+0000","last_active":"2023-10-06T03:57:16.371903+0000","last_peered":"2023-10-06T03:57:16.371903+0000","last_clean":"2023-10-06T03:57:16.371903+0000","last_became_active":"2023-10-06T03:57:10.340232+0000","last_became_peered":"2023-10-06T03:57:10.340232+0000","last_unstale":"2023-10-06T03:57:16.371903+0000","last_undegraded":"2023-10-06T03:57:16.371903+0000","last_fullsized":"2023-10-06T03:57:16.371903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:43:26.332055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377803+0000","last_change":"2023-10-06T03:57:11.375070+0000","last_active":"2023-10-06T03:57:11.377803+0000","last_peered":"2023-10-06T03:57:11.377803+0000","last_clean":"2023-10-06T03:57:11.377803+0000","last_became_active":"2023-10-06T03:57:11.374892+0000","last_became_peered":"2023-10-06T03:57:11.374892+0000","last_unstale":"2023-10-06T03:57:11.377803+0000","last_undegraded":"2023-10-06T03:57:11.377803+0000","last_fullsized":"2023-10-06T03:57:11.377803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:51.198745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"21'6","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:16.375229+0000","last_change":"2023-10-06T03:57:10.346758+0000","last_active":"2023-10-06T03:57:16.375229+0000","last_peered":"2023-10-06T03:57:16.375229+0000","last_clean":"2023-10-06T03:57:16.375229+0000","last_became_active":"2023-10-06T03:57:10.346430+0000","last_became_peered":"2023-10-06T03:57:10.346430+0000","last_unstale":"2023-10-06T03:57:16.375229+0000","last_undegraded":"2023-10-06T03:57:16.375229+0000","last_fullsized":"2023-10-06T03:57:16.375229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:10.809242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371658+0000","last_change":"2023-10-06T03:57:11.367808+0000","last_active":"2023-10-06T03:57:11.371658+0000","last_peered":"2023-10-06T03:57:11.371658+0000","last_clean":"2023-10-06T03:57:11.371658+0000","last_became_active":"2023-10-06T03:57:11.367565+0000","last_became_peered":"2023-10-06T03:57:11.367565+0000","last_unstale":"2023-10-06T03:57:11.371658+0000","last_undegraded":"2023-10-06T03:57:11.371658+0000","last_fullsized":"2023-10-06T03:57:11.371658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:14.409116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:16.374798+0000","last_change":"2023-10-06T03:57:10.351062+0000","last_active":"2023-10-06T03:57:16.374798+0000","last_peered":"2023-10-06T03:57:16.374798+0000","last_clean":"2023-10-06T03:57:16.374798+0000","last_became_active":"2023-10-06T03:57:10.350795+0000","last_became_peered":"2023-10-06T03:57:10.350795+0000","last_unstale":"2023-10-06T03:57:16.374798+0000","last_undegraded":"2023-10-06T03:57:16.374798+0000","last_fullsized":"2023-10-06T03:57:16.374798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:54:45.714518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.370549+0000","last_change":"2023-10-06T03:57:11.367489+0000","last_active":"2023-10-06T03:57:11.370549+0000","last_peered":"2023-10-06T03:57:11.370549+0000","last_clean":"2023-10-06T03:57:11.370549+0000","last_became_active":"2023-10-06T03:57:11.367254+0000","last_became_peered":"2023-10-06T03:57:11.367254+0000","last_unstale":"2023-10-06T03:57:11.370549+0000","last_undegraded":"2023-10-06T03:57:11.370549+0000","last_fullsized":"2023-10-06T03:57:11.370549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:04.706485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.352684+0000","last_change":"2023-10-06T03:57:10.338125+0000","last_active":"2023-10-06T03:57:11.352684+0000","last_peered":"2023-10-06T03:57:11.352684+0000","last_clean":"2023-10-06T03:57:11.352684+0000","last_became_active":"2023-10-06T03:57:10.337798+0000","last_became_peered":"2023-10-06T03:57:10.337798+0000","last_unstale":"2023-10-06T03:57:11.352684+0000","last_undegraded":"2023-10-06T03:57:11.352684+0000","last_fullsized":"2023-10-06T03:57:11.352684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:00.482657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378957+0000","last_change":"2023-10-06T03:57:11.376461+0000","last_active":"2023-10-06T03:57:11.378957+0000","last_peered":"2023-10-06T03:57:11.378957+0000","last_clean":"2023-10-06T03:57:11.378957+0000","last_became_active":"2023-10-06T03:57:11.376223+0000","last_became_peered":"2023-10-06T03:57:11.376223+0000","last_unstale":"2023-10-06T03:57:11.378957+0000","last_undegraded":"2023-10-06T03:57:11.378957+0000","last_fullsized":"2023-10-06T03:57:11.378957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:04:37.408871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.364778+0000","last_change":"2023-10-06T03:57:10.350511+0000","last_active":"2023-10-06T03:57:11.364778+0000","last_peered":"2023-10-06T03:57:11.364778+0000","last_clean":"2023-10-06T03:57:11.364778+0000","last_became_active":"2023-10-06T03:57:10.350219+0000","last_became_peered":"2023-10-06T03:57:10.350219+0000","last_unstale":"2023-10-06T03:57:11.364778+0000","last_undegraded":"2023-10-06T03:57:11.364778+0000","last_fullsized":"2023-10-06T03:57:11.364778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:56.043506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.364939+0000","last_change":"2023-10-06T03:57:11.362700+0000","last_active":"2023-10-06T03:57:11.364939+0000","last_peered":"2023-10-06T03:57:11.364939+0000","last_clean":"2023-10-06T03:57:11.364939+0000","last_became_active":"2023-10-06T03:57:11.362443+0000","last_became_peered":"2023-10-06T03:57:11.362443+0000","last_unstale":"2023-10-06T03:57:11.364939+0000","last_undegraded":"2023-10-06T03:57:11.364939+0000","last_fullsized":"2023-10-06T03:57:11.364939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:21:09.143085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"21'8","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:16.372505+0000","last_change":"2023-10-06T03:57:10.346259+0000","last_active":"2023-10-06T03:57:16.372505+0000","last_peered":"2023-10-06T03:57:16.372505+0000","last_clean":"2023-10-06T03:57:16.372505+0000","last_became_active":"2023-10-06T03:57:10.345424+0000","last_became_peered":"2023-10-06T03:57:10.345424+0000","last_unstale":"2023-10-06T03:57:16.372505+0000","last_undegraded":"2023-10-06T03:57:16.372505+0000","last_fullsized":"2023-10-06T03:57:16.372505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:19.278206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378006+0000","last_change":"2023-10-06T03:57:11.376053+0000","last_active":"2023-10-06T03:57:11.378006+0000","last_peered":"2023-10-06T03:57:11.378006+0000","last_clean":"2023-10-06T03:57:11.378006+0000","last_became_active":"2023-10-06T03:57:11.375824+0000","last_became_peered":"2023-10-06T03:57:11.375824+0000","last_unstale":"2023-10-06T03:57:11.378006+0000","last_undegraded":"2023-10-06T03:57:11.378006+0000","last_fullsized":"2023-10-06T03:57:11.378006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:24:26.156866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:12.369811+0000","last_change":"2023-10-06T03:57:10.350408+0000","last_active":"2023-10-06T03:57:12.369811+0000","last_peered":"2023-10-06T03:57:12.369811+0000","last_clean":"2023-10-06T03:57:12.369811+0000","last_became_active":"2023-10-06T03:57:10.350135+0000","last_became_peered":"2023-10-06T03:57:10.350135+0000","last_unstale":"2023-10-06T03:57:12.369811+0000","last_undegraded":"2023-10-06T03:57:12.369811+0000","last_fullsized":"2023-10-06T03:57:12.369811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:43.522111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378954+0000","last_change":"2023-10-06T03:57:11.376524+0000","last_active":"2023-10-06T03:57:11.378954+0000","last_peered":"2023-10-06T03:57:11.378954+0000","last_clean":"2023-10-06T03:57:11.378954+0000","last_became_active":"2023-10-06T03:57:11.376236+0000","last_became_peered":"2023-10-06T03:57:11.376236+0000","last_unstale":"2023-10-06T03:57:11.378954+0000","last_undegraded":"2023-10-06T03:57:11.378954+0000","last_fullsized":"2023-10-06T03:57:11.378954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:33.854818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:16.374755+0000","last_change":"2023-10-06T03:57:10.345799+0000","last_active":"2023-10-06T03:57:16.374755+0000","last_peered":"2023-10-06T03:57:16.374755+0000","last_clean":"2023-10-06T03:57:16.374755+0000","last_became_active":"2023-10-06T03:57:10.345510+0000","last_became_peered":"2023-10-06T03:57:10.345510+0000","last_unstale":"2023-10-06T03:57:16.374755+0000","last_undegraded":"2023-10-06T03:57:16.374755+0000","last_fullsized":"2023-10-06T03:57:16.374755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:09.567314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372383+0000","last_change":"2023-10-06T03:57:11.368662+0000","last_active":"2023-10-06T03:57:11.372383+0000","last_peered":"2023-10-06T03:57:11.372383+0000","last_clean":"2023-10-06T03:57:11.372383+0000","last_became_active":"2023-10-06T03:57:11.368375+0000","last_became_peered":"2023-10-06T03:57:11.368375+0000","last_unstale":"2023-10-06T03:57:11.372383+0000","last_undegraded":"2023-10-06T03:57:11.372383+0000","last_fullsized":"2023-10-06T03:57:11.372383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:27.125523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:12.369814+0000","last_change":"2023-10-06T03:57:10.346945+0000","last_active":"2023-10-06T03:57:12.369814+0000","last_peered":"2023-10-06T03:57:12.369814+0000","last_clean":"2023-10-06T03:57:12.369814+0000","last_became_active":"2023-10-06T03:57:10.346689+0000","last_became_peered":"2023-10-06T03:57:10.346689+0000","last_unstale":"2023-10-06T03:57:12.369814+0000","last_undegraded":"2023-10-06T03:57:12.369814+0000","last_fullsized":"2023-10-06T03:57:12.369814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:43.420455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.375632+0000","last_change":"2023-10-06T03:57:11.372308+0000","last_active":"2023-10-06T03:57:11.375632+0000","last_peered":"2023-10-06T03:57:11.375632+0000","last_clean":"2023-10-06T03:57:11.375632+0000","last_became_active":"2023-10-06T03:57:11.371795+0000","last_became_peered":"2023-10-06T03:57:11.371795+0000","last_unstale":"2023-10-06T03:57:11.375632+0000","last_undegraded":"2023-10-06T03:57:11.375632+0000","last_fullsized":"2023-10-06T03:57:11.375632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:00.324478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.361016+0000","last_change":"2023-10-06T03:57:10.348592+0000","last_active":"2023-10-06T03:57:11.361016+0000","last_peered":"2023-10-06T03:57:11.361016+0000","last_clean":"2023-10-06T03:57:11.361016+0000","last_became_active":"2023-10-06T03:57:10.348291+0000","last_became_peered":"2023-10-06T03:57:10.348291+0000","last_unstale":"2023-10-06T03:57:11.361016+0000","last_undegraded":"2023-10-06T03:57:11.361016+0000","last_fullsized":"2023-10-06T03:57:11.361016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:00.660348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378672+0000","last_change":"2023-10-06T03:57:11.366510+0000","last_active":"2023-10-06T03:57:11.378672+0000","last_peered":"2023-10-06T03:57:11.378672+0000","last_clean":"2023-10-06T03:57:11.378672+0000","last_became_active":"2023-10-06T03:57:11.365077+0000","last_became_peered":"2023-10-06T03:57:11.365077+0000","last_unstale":"2023-10-06T03:57:11.378672+0000","last_undegraded":"2023-10-06T03:57:11.378672+0000","last_fullsized":"2023-10-06T03:57:11.378672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:25.199391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:16.368347+0000","last_change":"2023-10-06T03:57:10.348418+0000","last_active":"2023-10-06T03:57:16.368347+0000","last_peered":"2023-10-06T03:57:16.368347+0000","last_clean":"2023-10-06T03:57:16.368347+0000","last_became_active":"2023-10-06T03:57:10.348055+0000","last_became_peered":"2023-10-06T03:57:10.348055+0000","last_unstale":"2023-10-06T03:57:16.368347+0000","last_undegraded":"2023-10-06T03:57:16.368347+0000","last_fullsized":"2023-10-06T03:57:16.368347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:26.102022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373629+0000","last_change":"2023-10-06T03:57:11.370493+0000","last_active":"2023-10-06T03:57:11.373629+0000","last_peered":"2023-10-06T03:57:11.373629+0000","last_clean":"2023-10-06T03:57:11.373629+0000","last_became_active":"2023-10-06T03:57:11.370073+0000","last_became_peered":"2023-10-06T03:57:11.370073+0000","last_unstale":"2023-10-06T03:57:11.373629+0000","last_undegraded":"2023-10-06T03:57:11.373629+0000","last_fullsized":"2023-10-06T03:57:11.373629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:58.417951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.362176+0000","last_change":"2023-10-06T03:57:10.352966+0000","last_active":"2023-10-06T03:57:11.362176+0000","last_peered":"2023-10-06T03:57:11.362176+0000","last_clean":"2023-10-06T03:57:11.362176+0000","last_became_active":"2023-10-06T03:57:10.352653+0000","last_became_peered":"2023-10-06T03:57:10.352653+0000","last_unstale":"2023-10-06T03:57:11.362176+0000","last_undegraded":"2023-10-06T03:57:11.362176+0000","last_fullsized":"2023-10-06T03:57:11.362176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:32.092557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371704+0000","last_change":"2023-10-06T03:57:11.364052+0000","last_active":"2023-10-06T03:57:11.371704+0000","last_peered":"2023-10-06T03:57:11.371704+0000","last_clean":"2023-10-06T03:57:11.371704+0000","last_became_active":"2023-10-06T03:57:11.363830+0000","last_became_peered":"2023-10-06T03:57:11.363830+0000","last_unstale":"2023-10-06T03:57:11.371704+0000","last_undegraded":"2023-10-06T03:57:11.371704+0000","last_fullsized":"2023-10-06T03:57:11.371704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:04.084019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.357124+0000","last_change":"2023-10-06T03:57:10.349421+0000","last_active":"2023-10-06T03:57:11.357124+0000","last_peered":"2023-10-06T03:57:11.357124+0000","last_clean":"2023-10-06T03:57:11.357124+0000","last_became_active":"2023-10-06T03:57:10.349114+0000","last_became_peered":"2023-10-06T03:57:10.349114+0000","last_unstale":"2023-10-06T03:57:11.357124+0000","last_undegraded":"2023-10-06T03:57:11.357124+0000","last_fullsized":"2023-10-06T03:57:11.357124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:08.209791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372296+0000","last_change":"2023-10-06T03:57:11.367516+0000","last_active":"2023-10-06T03:57:11.372296+0000","last_peered":"2023-10-06T03:57:11.372296+0000","last_clean":"2023-10-06T03:57:11.372296+0000","last_became_active":"2023-10-06T03:57:11.367257+0000","last_became_peered":"2023-10-06T03:57:11.367257+0000","last_unstale":"2023-10-06T03:57:11.372296+0000","last_undegraded":"2023-10-06T03:57:11.372296+0000","last_fullsized":"2023-10-06T03:57:11.372296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:43.628819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.359638+0000","last_change":"2023-10-06T03:57:10.347649+0000","last_active":"2023-10-06T03:57:11.359638+0000","last_peered":"2023-10-06T03:57:11.359638+0000","last_clean":"2023-10-06T03:57:11.359638+0000","last_became_active":"2023-10-06T03:57:10.347433+0000","last_became_peered":"2023-10-06T03:57:10.347433+0000","last_unstale":"2023-10-06T03:57:11.359638+0000","last_undegraded":"2023-10-06T03:57:11.359638+0000","last_fullsized":"2023-10-06T03:57:11.359638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:55.951494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372561+0000","last_change":"2023-10-06T03:57:11.370092+0000","last_active":"2023-10-06T03:57:11.372561+0000","last_peered":"2023-10-06T03:57:11.372561+0000","last_clean":"2023-10-06T03:57:11.372561+0000","last_became_active":"2023-10-06T03:57:11.369825+0000","last_became_peered":"2023-10-06T03:57:11.369825+0000","last_unstale":"2023-10-06T03:57:11.372561+0000","last_undegraded":"2023-10-06T03:57:11.372561+0000","last_fullsized":"2023-10-06T03:57:11.372561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:43.415929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367709+0000","last_change":"2023-10-06T03:57:05.663312+0000","last_active":"2023-10-06T03:57:11.367709+0000","last_peered":"2023-10-06T03:57:11.367709+0000","last_clean":"2023-10-06T03:57:11.367709+0000","last_became_active":"2023-10-06T03:57:05.663046+0000","last_became_peered":"2023-10-06T03:57:05.663046+0000","last_unstale":"2023-10-06T03:57:11.367709+0000","last_undegraded":"2023-10-06T03:57:11.367709+0000","last_fullsized":"2023-10-06T03:57:11.367709+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:31.174359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.363783+0000","last_change":"2023-10-06T03:57:10.348930+0000","last_active":"2023-10-06T03:57:11.363783+0000","last_peered":"2023-10-06T03:57:11.363783+0000","last_clean":"2023-10-06T03:57:11.363783+0000","last_became_active":"2023-10-06T03:57:10.348608+0000","last_became_peered":"2023-10-06T03:57:10.348608+0000","last_unstale":"2023-10-06T03:57:11.363783+0000","last_undegraded":"2023-10-06T03:57:11.363783+0000","last_fullsized":"2023-10-06T03:57:11.363783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:00:09.283718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377785+0000","last_change":"2023-10-06T03:57:11.371108+0000","last_active":"2023-10-06T03:57:11.377785+0000","last_peered":"2023-10-06T03:57:11.377785+0000","last_clean":"2023-10-06T03:57:11.377785+0000","last_became_active":"2023-10-06T03:57:11.370847+0000","last_became_peered":"2023-10-06T03:57:11.370847+0000","last_unstale":"2023-10-06T03:57:11.377785+0000","last_undegraded":"2023-10-06T03:57:11.377785+0000","last_fullsized":"2023-10-06T03:57:11.377785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:38.111171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.362274+0000","last_change":"2023-10-06T03:57:05.318834+0000","last_active":"2023-10-06T03:57:11.362274+0000","last_peered":"2023-10-06T03:57:11.362274+0000","last_clean":"2023-10-06T03:57:11.362274+0000","last_became_active":"2023-10-06T03:57:05.318572+0000","last_became_peered":"2023-10-06T03:57:05.318572+0000","last_unstale":"2023-10-06T03:57:11.362274+0000","last_undegraded":"2023-10-06T03:57:11.362274+0000","last_fullsized":"2023-10-06T03:57:11.362274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:21.951095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.358229+0000","last_change":"2023-10-06T03:57:10.348556+0000","last_active":"2023-10-06T03:57:11.358229+0000","last_peered":"2023-10-06T03:57:11.358229+0000","last_clean":"2023-10-06T03:57:11.358229+0000","last_became_active":"2023-10-06T03:57:10.348271+0000","last_became_peered":"2023-10-06T03:57:10.348271+0000","last_unstale":"2023-10-06T03:57:11.358229+0000","last_undegraded":"2023-10-06T03:57:11.358229+0000","last_fullsized":"2023-10-06T03:57:11.358229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:58.470989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.368920+0000","last_change":"2023-10-06T03:57:11.365237+0000","last_active":"2023-10-06T03:57:11.368920+0000","last_peered":"2023-10-06T03:57:11.368920+0000","last_clean":"2023-10-06T03:57:11.368920+0000","last_became_active":"2023-10-06T03:57:11.365006+0000","last_became_peered":"2023-10-06T03:57:11.365006+0000","last_unstale":"2023-10-06T03:57:11.368920+0000","last_undegraded":"2023-10-06T03:57:11.368920+0000","last_fullsized":"2023-10-06T03:57:11.368920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:33.005015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.355768+0000","last_change":"2023-10-06T03:57:05.314695+0000","last_active":"2023-10-06T03:57:11.355768+0000","last_peered":"2023-10-06T03:57:11.355768+0000","last_clean":"2023-10-06T03:57:11.355768+0000","last_became_active":"2023-10-06T03:57:05.314408+0000","last_became_peered":"2023-10-06T03:57:05.314408+0000","last_unstale":"2023-10-06T03:57:11.355768+0000","last_undegraded":"2023-10-06T03:57:11.355768+0000","last_fullsized":"2023-10-06T03:57:11.355768+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:19.042145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367206+0000","last_change":"2023-10-06T03:57:10.346078+0000","last_active":"2023-10-06T03:57:11.367206+0000","last_peered":"2023-10-06T03:57:11.367206+0000","last_clean":"2023-10-06T03:57:11.367206+0000","last_became_active":"2023-10-06T03:57:10.345769+0000","last_became_peered":"2023-10-06T03:57:10.345769+0000","last_unstale":"2023-10-06T03:57:11.367206+0000","last_undegraded":"2023-10-06T03:57:11.367206+0000","last_fullsized":"2023-10-06T03:57:11.367206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:16.630681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.376583+0000","last_change":"2023-10-06T03:57:11.370372+0000","last_active":"2023-10-06T03:57:11.376583+0000","last_peered":"2023-10-06T03:57:11.376583+0000","last_clean":"2023-10-06T03:57:11.376583+0000","last_became_active":"2023-10-06T03:57:11.370076+0000","last_became_peered":"2023-10-06T03:57:11.370076+0000","last_unstale":"2023-10-06T03:57:11.376583+0000","last_undegraded":"2023-10-06T03:57:11.376583+0000","last_fullsized":"2023-10-06T03:57:11.376583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:18:59.517155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.362848+0000","last_change":"2023-10-06T03:57:05.318778+0000","last_active":"2023-10-06T03:57:11.362848+0000","last_peered":"2023-10-06T03:57:11.362848+0000","last_clean":"2023-10-06T03:57:11.362848+0000","last_became_active":"2023-10-06T03:57:05.318456+0000","last_became_peered":"2023-10-06T03:57:05.318456+0000","last_unstale":"2023-10-06T03:57:11.362848+0000","last_undegraded":"2023-10-06T03:57:11.362848+0000","last_fullsized":"2023-10-06T03:57:11.362848+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:39.769241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.362382+0000","last_change":"2023-10-06T03:57:05.664245+0000","last_active":"2023-10-06T03:57:11.362382+0000","last_peered":"2023-10-06T03:57:11.362382+0000","last_clean":"2023-10-06T03:57:11.362382+0000","last_became_active":"2023-10-06T03:57:05.663426+0000","last_became_peered":"2023-10-06T03:57:05.663426+0000","last_unstale":"2023-10-06T03:57:11.362382+0000","last_undegraded":"2023-10-06T03:57:11.362382+0000","last_fullsized":"2023-10-06T03:57:11.362382+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:29.322599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.359971+0000","last_change":"2023-10-06T03:57:10.351059+0000","last_active":"2023-10-06T03:57:11.359971+0000","last_peered":"2023-10-06T03:57:11.359971+0000","last_clean":"2023-10-06T03:57:11.359971+0000","last_became_active":"2023-10-06T03:57:10.348879+0000","last_became_peered":"2023-10-06T03:57:10.348879+0000","last_unstale":"2023-10-06T03:57:11.359971+0000","last_undegraded":"2023-10-06T03:57:11.359971+0000","last_fullsized":"2023-10-06T03:57:11.359971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:35.764148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.375601+0000","last_change":"2023-10-06T03:57:11.373786+0000","last_active":"2023-10-06T03:57:11.375601+0000","last_peered":"2023-10-06T03:57:11.375601+0000","last_clean":"2023-10-06T03:57:11.375601+0000","last_became_active":"2023-10-06T03:57:11.373452+0000","last_became_peered":"2023-10-06T03:57:11.373452+0000","last_unstale":"2023-10-06T03:57:11.375601+0000","last_undegraded":"2023-10-06T03:57:11.375601+0000","last_fullsized":"2023-10-06T03:57:11.375601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:20.524065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.364987+0000","last_change":"2023-10-06T03:57:05.318790+0000","last_active":"2023-10-06T03:57:11.364987+0000","last_peered":"2023-10-06T03:57:11.364987+0000","last_clean":"2023-10-06T03:57:11.364987+0000","last_became_active":"2023-10-06T03:57:05.318531+0000","last_became_peered":"2023-10-06T03:57:05.318531+0000","last_unstale":"2023-10-06T03:57:11.364987+0000","last_undegraded":"2023-10-06T03:57:11.364987+0000","last_fullsized":"2023-10-06T03:57:11.364987+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:12.876998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.352596+0000","last_change":"2023-10-06T03:57:10.343328+0000","last_active":"2023-10-06T03:57:11.352596+0000","last_peered":"2023-10-06T03:57:11.352596+0000","last_clean":"2023-10-06T03:57:11.352596+0000","last_became_active":"2023-10-06T03:57:10.343051+0000","last_became_peered":"2023-10-06T03:57:10.343051+0000","last_unstale":"2023-10-06T03:57:11.352596+0000","last_undegraded":"2023-10-06T03:57:11.352596+0000","last_fullsized":"2023-10-06T03:57:11.352596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:27.722475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378907+0000","last_change":"2023-10-06T03:57:11.373581+0000","last_active":"2023-10-06T03:57:11.378907+0000","last_peered":"2023-10-06T03:57:11.378907+0000","last_clean":"2023-10-06T03:57:11.378907+0000","last_became_active":"2023-10-06T03:57:11.373264+0000","last_became_peered":"2023-10-06T03:57:11.373264+0000","last_unstale":"2023-10-06T03:57:11.378907+0000","last_undegraded":"2023-10-06T03:57:11.378907+0000","last_fullsized":"2023-10-06T03:57:11.378907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:15.950245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.366791+0000","last_change":"2023-10-06T03:57:05.663770+0000","last_active":"2023-10-06T03:57:11.366791+0000","last_peered":"2023-10-06T03:57:11.366791+0000","last_clean":"2023-10-06T03:57:11.366791+0000","last_became_active":"2023-10-06T03:57:05.663501+0000","last_became_peered":"2023-10-06T03:57:05.663501+0000","last_unstale":"2023-10-06T03:57:11.366791+0000","last_undegraded":"2023-10-06T03:57:11.366791+0000","last_fullsized":"2023-10-06T03:57:11.366791+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:09.806915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367831+0000","last_change":"2023-10-06T03:57:10.346585+0000","last_active":"2023-10-06T03:57:11.367831+0000","last_peered":"2023-10-06T03:57:11.367831+0000","last_clean":"2023-10-06T03:57:11.367831+0000","last_became_active":"2023-10-06T03:57:10.345743+0000","last_became_peered":"2023-10-06T03:57:10.345743+0000","last_unstale":"2023-10-06T03:57:11.367831+0000","last_undegraded":"2023-10-06T03:57:11.367831+0000","last_fullsized":"2023-10-06T03:57:11.367831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:54.589702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378948+0000","last_change":"2023-10-06T03:57:11.373844+0000","last_active":"2023-10-06T03:57:11.378948+0000","last_peered":"2023-10-06T03:57:11.378948+0000","last_clean":"2023-10-06T03:57:11.378948+0000","last_became_active":"2023-10-06T03:57:11.373543+0000","last_became_peered":"2023-10-06T03:57:11.373543+0000","last_unstale":"2023-10-06T03:57:11.378948+0000","last_undegraded":"2023-10-06T03:57:11.378948+0000","last_fullsized":"2023-10-06T03:57:11.378948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:15.152354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:16.372867+0000","last_change":"2023-10-06T03:57:10.344618+0000","last_active":"2023-10-06T03:57:16.372867+0000","last_peered":"2023-10-06T03:57:16.372867+0000","last_clean":"2023-10-06T03:57:16.372867+0000","last_became_active":"2023-10-06T03:57:10.344324+0000","last_became_peered":"2023-10-06T03:57:10.344324+0000","last_unstale":"2023-10-06T03:57:16.372867+0000","last_undegraded":"2023-10-06T03:57:16.372867+0000","last_fullsized":"2023-10-06T03:57:16.372867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:06.062331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.368350+0000","last_change":"2023-10-06T03:57:05.317973+0000","last_active":"2023-10-06T03:57:11.368350+0000","last_peered":"2023-10-06T03:57:11.368350+0000","last_clean":"2023-10-06T03:57:11.368350+0000","last_became_active":"2023-10-06T03:57:05.317715+0000","last_became_peered":"2023-10-06T03:57:05.317715+0000","last_unstale":"2023-10-06T03:57:11.368350+0000","last_undegraded":"2023-10-06T03:57:11.368350+0000","last_fullsized":"2023-10-06T03:57:11.368350+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:37:51.858909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372084+0000","last_change":"2023-10-06T03:57:11.369706+0000","last_active":"2023-10-06T03:57:11.372084+0000","last_peered":"2023-10-06T03:57:11.372084+0000","last_clean":"2023-10-06T03:57:11.372084+0000","last_became_active":"2023-10-06T03:57:11.369445+0000","last_became_peered":"2023-10-06T03:57:11.369445+0000","last_unstale":"2023-10-06T03:57:11.372084+0000","last_undegraded":"2023-10-06T03:57:11.372084+0000","last_fullsized":"2023-10-06T03:57:11.372084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:52.999421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.362125+0000","last_change":"2023-10-06T03:57:05.320847+0000","last_active":"2023-10-06T03:57:11.362125+0000","last_peered":"2023-10-06T03:57:11.362125+0000","last_clean":"2023-10-06T03:57:11.362125+0000","last_became_active":"2023-10-06T03:57:05.320583+0000","last_became_peered":"2023-10-06T03:57:05.320583+0000","last_unstale":"2023-10-06T03:57:11.362125+0000","last_undegraded":"2023-10-06T03:57:11.362125+0000","last_fullsized":"2023-10-06T03:57:11.362125+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:12.772449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.353105+0000","last_change":"2023-10-06T03:57:10.341132+0000","last_active":"2023-10-06T03:57:11.353105+0000","last_peered":"2023-10-06T03:57:11.353105+0000","last_clean":"2023-10-06T03:57:11.353105+0000","last_became_active":"2023-10-06T03:57:10.339424+0000","last_became_peered":"2023-10-06T03:57:10.339424+0000","last_unstale":"2023-10-06T03:57:11.353105+0000","last_undegraded":"2023-10-06T03:57:11.353105+0000","last_fullsized":"2023-10-06T03:57:11.353105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:39.675711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.374372+0000","last_change":"2023-10-06T03:57:11.372497+0000","last_active":"2023-10-06T03:57:11.374372+0000","last_peered":"2023-10-06T03:57:11.374372+0000","last_clean":"2023-10-06T03:57:11.374372+0000","last_became_active":"2023-10-06T03:57:11.372215+0000","last_became_peered":"2023-10-06T03:57:11.372215+0000","last_unstale":"2023-10-06T03:57:11.374372+0000","last_undegraded":"2023-10-06T03:57:11.374372+0000","last_fullsized":"2023-10-06T03:57:11.374372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:23.559480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:13.050807+0000","last_change":"2023-10-06T03:57:10.344051+0000","last_active":"2023-10-06T03:57:13.050807+0000","last_peered":"2023-10-06T03:57:13.050807+0000","last_clean":"2023-10-06T03:57:13.050807+0000","last_became_active":"2023-10-06T03:57:10.343865+0000","last_became_peered":"2023-10-06T03:57:10.343865+0000","last_unstale":"2023-10-06T03:57:13.050807+0000","last_undegraded":"2023-10-06T03:57:13.050807+0000","last_fullsized":"2023-10-06T03:57:13.050807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:32.522470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.376665+0000","last_change":"2023-10-06T03:57:11.369927+0000","last_active":"2023-10-06T03:57:11.376665+0000","last_peered":"2023-10-06T03:57:11.376665+0000","last_clean":"2023-10-06T03:57:11.376665+0000","last_became_active":"2023-10-06T03:57:11.369681+0000","last_became_peered":"2023-10-06T03:57:11.369681+0000","last_unstale":"2023-10-06T03:57:11.376665+0000","last_undegraded":"2023-10-06T03:57:11.376665+0000","last_fullsized":"2023-10-06T03:57:11.376665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:53.858179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:16.373082+0000","last_change":"2023-10-06T03:57:10.349666+0000","last_active":"2023-10-06T03:57:16.373082+0000","last_peered":"2023-10-06T03:57:16.373082+0000","last_clean":"2023-10-06T03:57:16.373082+0000","last_became_active":"2023-10-06T03:57:10.349320+0000","last_became_peered":"2023-10-06T03:57:10.349320+0000","last_unstale":"2023-10-06T03:57:16.373082+0000","last_undegraded":"2023-10-06T03:57:16.373082+0000","last_fullsized":"2023-10-06T03:57:16.373082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:45:49.709519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373619+0000","last_change":"2023-10-06T03:57:11.370442+0000","last_active":"2023-10-06T03:57:11.373619+0000","last_peered":"2023-10-06T03:57:11.373619+0000","last_clean":"2023-10-06T03:57:11.373619+0000","last_became_active":"2023-10-06T03:57:11.370229+0000","last_became_peered":"2023-10-06T03:57:11.370229+0000","last_unstale":"2023-10-06T03:57:11.373619+0000","last_undegraded":"2023-10-06T03:57:11.373619+0000","last_fullsized":"2023-10-06T03:57:11.373619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:26.174331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:16.372960+0000","last_change":"2023-10-06T03:57:10.350263+0000","last_active":"2023-10-06T03:57:16.372960+0000","last_peered":"2023-10-06T03:57:16.372960+0000","last_clean":"2023-10-06T03:57:16.372960+0000","last_became_active":"2023-10-06T03:57:10.350083+0000","last_became_peered":"2023-10-06T03:57:10.350083+0000","last_unstale":"2023-10-06T03:57:16.372960+0000","last_undegraded":"2023-10-06T03:57:16.372960+0000","last_fullsized":"2023-10-06T03:57:16.372960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:05.433032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372850+0000","last_change":"2023-10-06T03:57:11.369397+0000","last_active":"2023-10-06T03:57:11.372850+0000","last_peered":"2023-10-06T03:57:11.372850+0000","last_clean":"2023-10-06T03:57:11.372850+0000","last_became_active":"2023-10-06T03:57:11.369123+0000","last_became_peered":"2023-10-06T03:57:11.369123+0000","last_unstale":"2023-10-06T03:57:11.372850+0000","last_undegraded":"2023-10-06T03:57:11.372850+0000","last_fullsized":"2023-10-06T03:57:11.372850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:28.139036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:13.050204+0000","last_change":"2023-10-06T03:57:10.347049+0000","last_active":"2023-10-06T03:57:13.050204+0000","last_peered":"2023-10-06T03:57:13.050204+0000","last_clean":"2023-10-06T03:57:13.050204+0000","last_became_active":"2023-10-06T03:57:10.344583+0000","last_became_peered":"2023-10-06T03:57:10.344583+0000","last_unstale":"2023-10-06T03:57:13.050204+0000","last_undegraded":"2023-10-06T03:57:13.050204+0000","last_fullsized":"2023-10-06T03:57:13.050204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:32.280257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.370657+0000","last_change":"2023-10-06T03:57:11.366741+0000","last_active":"2023-10-06T03:57:11.370657+0000","last_peered":"2023-10-06T03:57:11.370657+0000","last_clean":"2023-10-06T03:57:11.370657+0000","last_became_active":"2023-10-06T03:57:11.366500+0000","last_became_peered":"2023-10-06T03:57:11.366500+0000","last_unstale":"2023-10-06T03:57:11.370657+0000","last_undegraded":"2023-10-06T03:57:11.370657+0000","last_fullsized":"2023-10-06T03:57:11.370657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:24.714603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:16.372880+0000","last_change":"2023-10-06T03:57:10.348170+0000","last_active":"2023-10-06T03:57:16.372880+0000","last_peered":"2023-10-06T03:57:16.372880+0000","last_clean":"2023-10-06T03:57:16.372880+0000","last_became_active":"2023-10-06T03:57:10.347897+0000","last_became_peered":"2023-10-06T03:57:10.347897+0000","last_unstale":"2023-10-06T03:57:16.372880+0000","last_undegraded":"2023-10-06T03:57:16.372880+0000","last_fullsized":"2023-10-06T03:57:16.372880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:40.756246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372969+0000","last_change":"2023-10-06T03:57:11.364368+0000","last_active":"2023-10-06T03:57:11.372969+0000","last_peered":"2023-10-06T03:57:11.372969+0000","last_clean":"2023-10-06T03:57:11.372969+0000","last_became_active":"2023-10-06T03:57:11.364150+0000","last_became_peered":"2023-10-06T03:57:11.364150+0000","last_unstale":"2023-10-06T03:57:11.372969+0000","last_undegraded":"2023-10-06T03:57:11.372969+0000","last_fullsized":"2023-10-06T03:57:11.372969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:11.038080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.374372+0000","last_change":"2023-10-06T03:57:11.366721+0000","last_active":"2023-10-06T03:57:11.374372+0000","last_peered":"2023-10-06T03:57:11.374372+0000","last_clean":"2023-10-06T03:57:11.374372+0000","last_became_active":"2023-10-06T03:57:11.366524+0000","last_became_peered":"2023-10-06T03:57:11.366524+0000","last_unstale":"2023-10-06T03:57:11.374372+0000","last_undegraded":"2023-10-06T03:57:11.374372+0000","last_fullsized":"2023-10-06T03:57:11.374372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:33.044858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373497+0000","last_change":"2023-10-06T03:57:11.371109+0000","last_active":"2023-10-06T03:57:11.373497+0000","last_peered":"2023-10-06T03:57:11.373497+0000","last_clean":"2023-10-06T03:57:11.373497+0000","last_became_active":"2023-10-06T03:57:11.370848+0000","last_became_peered":"2023-10-06T03:57:11.370848+0000","last_unstale":"2023-10-06T03:57:11.373497+0000","last_undegraded":"2023-10-06T03:57:11.373497+0000","last_fullsized":"2023-10-06T03:57:11.373497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:15.919090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372839+0000","last_change":"2023-10-06T03:57:11.364188+0000","last_active":"2023-10-06T03:57:11.372839+0000","last_peered":"2023-10-06T03:57:11.372839+0000","last_clean":"2023-10-06T03:57:11.372839+0000","last_became_active":"2023-10-06T03:57:11.363875+0000","last_became_peered":"2023-10-06T03:57:11.363875+0000","last_unstale":"2023-10-06T03:57:11.372839+0000","last_undegraded":"2023-10-06T03:57:11.372839+0000","last_fullsized":"2023-10-06T03:57:11.372839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:29.236344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378301+0000","last_change":"2023-10-06T03:57:11.377420+0000","last_active":"2023-10-06T03:57:11.378301+0000","last_peered":"2023-10-06T03:57:11.378301+0000","last_clean":"2023-10-06T03:57:11.378301+0000","last_became_active":"2023-10-06T03:57:11.377126+0000","last_became_peered":"2023-10-06T03:57:11.377126+0000","last_unstale":"2023-10-06T03:57:11.378301+0000","last_undegraded":"2023-10-06T03:57:11.378301+0000","last_fullsized":"2023-10-06T03:57:11.378301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:05.041737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377733+0000","last_change":"2023-10-06T03:57:11.375777+0000","last_active":"2023-10-06T03:57:11.377733+0000","last_peered":"2023-10-06T03:57:11.377733+0000","last_clean":"2023-10-06T03:57:11.377733+0000","last_became_active":"2023-10-06T03:57:11.375539+0000","last_became_peered":"2023-10-06T03:57:11.375539+0000","last_unstale":"2023-10-06T03:57:11.377733+0000","last_undegraded":"2023-10-06T03:57:11.377733+0000","last_fullsized":"2023-10-06T03:57:11.377733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:42.707488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64858,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":311296,"data_stored":670955,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":78835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":78835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.219}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297564,"kb_used_data":276,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074276,"statfs":{"total":96636764160,"available":96332058624,"internally_reserved":0,"allocated":282624,"data_stored":668804,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.254}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":78835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297504,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074336,"statfs":{"total":96636764160,"available":96332120064,"internally_reserved":0,"allocated":221184,"data_stored":78925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":237568,"data_stored":80427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":79459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:57:21.954 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:57:22.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.101+0000 7fdd62846700 1 Processor -- start 2023-10-06T03:57:22.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.101+0000 7fdd62846700 1 -- start start 2023-10-06T03:57:22.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.102+0000 7fdd62846700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd5c0ca630 0x7fdd5c153e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:22.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.102+0000 7fdd62846700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdd5c0c3e50 0x7fdd5c0c42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:22.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.102+0000 7fdd62846700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd5c0c39c0 con 0x7fdd5c0caa00 2023-10-06T03:57:22.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.102+0000 7fdd62846700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd5c154340 con 0x7fdd5c0c3e50 2023-10-06T03:57:22.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.102+0000 7fdd62846700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c068b80 con 0x7fdd5c0ca630 2023-10-06T03:57:22.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.102+0000 7fdd60de3700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdd5c0c3e50 0x7fdd5c0c42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:22.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.102+0000 7fdd60de3700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdd5c0c3e50 0x7fdd5c0c42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34654/0 (socket says 172.21.15.157:34654) 2023-10-06T03:57:22.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.102+0000 7fdd60de3700 1 -- 172.21.15.157:0/2007213911 learned_addr learned my addr 172.21.15.157:0/2007213911 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:22.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.102+0000 7fdd5bfff700 1 --2- 172.21.15.157:0/2007213911 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd5c0ca630 0x7fdd5c153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:22.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.103+0000 7fdd60de3700 1 -- 172.21.15.157:0/2007213911 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd5c0ca630 msgr2=0x7fdd5c153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:22.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.103+0000 7fdd5affd700 1 -- 172.21.15.157:0/2007213911 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2895853620 0 0) 0x7fdd5c0c39c0 con 0x7fdd5c0caa00 2023-10-06T03:57:22.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.103+0000 7fdd60de3700 1 --2- 172.21.15.157:0/2007213911 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd5c0ca630 0x7fdd5c153e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:22.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.103+0000 7fdd60de3700 1 -- 172.21.15.157:0/2007213911 >> v1:172.21.15.175:6789/0 conn(0x7fdd5c0caa00 legacy=0x7fdd5c0c18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:22.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.103+0000 7fdd60de3700 1 -- 172.21.15.157:0/2007213911 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5c0c47f0 con 0x7fdd5c0c3e50 2023-10-06T03:57:22.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.103+0000 7fdd60de3700 1 --2- 172.21.15.157:0/2007213911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd5c0c3e50 0x7fdd5c0c42b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdd54009f20 tx=0x7fdd54009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5523dd33f6c84f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:22.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.103+0000 7fdd5affd700 1 -- 172.21.15.157:0/2007213911 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5400e070 con 0x7fdd5c0c3e50 2023-10-06T03:57:22.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.103+0000 7fdd5affd700 1 -- 172.21.15.157:0/2007213911 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5400ba60 con 0x7fdd5c0c3e50 2023-10-06T03:57:22.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.103+0000 7fdd5affd700 1 -- 172.21.15.157:0/2007213911 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd540092e0 con 0x7fdd5c0c3e50 2023-10-06T03:57:22.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.104+0000 7fdd62846700 1 -- 172.21.15.157:0/2007213911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd5c0c3e50 msgr2=0x7fdd5c0c42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:22.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.104+0000 7fdd62846700 1 --2- 172.21.15.157:0/2007213911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd5c0c3e50 0x7fdd5c0c42b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdd54009f20 tx=0x7fdd54009840 comp rx=0 tx=0).stop 2023-10-06T03:57:22.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.104+0000 7fdd62846700 1 -- 172.21.15.157:0/2007213911 shutdown_connections 2023-10-06T03:57:22.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.104+0000 7fdd62846700 1 --2- 172.21.15.157:0/2007213911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd5c0c3e50 0x7fdd5c0c42b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:22.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.104+0000 7fdd62846700 1 --2- 172.21.15.157:0/2007213911 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd5c0ca630 0x7fdd5c153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:22.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.104+0000 7fdd62846700 1 -- 172.21.15.157:0/2007213911 >> 172.21.15.157:0/2007213911 conn(0x7fdd5c0692b0 msgr2=0x7fdd5c0696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:22.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.104+0000 7fdd62846700 1 -- 172.21.15.157:0/2007213911 shutdown_connections 2023-10-06T03:57:22.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.104+0000 7fdd62846700 1 -- 172.21.15.157:0/2007213911 wait complete. 2023-10-06T03:57:22.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.105+0000 7fdd62846700 1 Processor -- start 2023-10-06T03:57:22.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.105+0000 7fdd62846700 1 -- start start 2023-10-06T03:57:22.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.106+0000 7fdd62846700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd5c0c3e50 0x7fdd5c145760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:22.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.106+0000 7fdd62846700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd5c0ca630 0x7fdd5c06e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:22.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.106+0000 7fdd62846700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd5c0caa00 0x7fdd5c146a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:22.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.106+0000 7fdd62846700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c145fe0 con 0x7fdd5c0ca630 2023-10-06T03:57:22.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.106+0000 7fdd62846700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c1465a0 con 0x7fdd5c0c3e50 2023-10-06T03:57:22.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.106+0000 7fdd62846700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c1430c0 con 0x7fdd5c0caa00 2023-10-06T03:57:22.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.106+0000 7fdd5b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd5c0ca630 0x7fdd5c06e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:22.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.106+0000 7fdd5b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd5c0ca630 0x7fdd5c06e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34656/0 (socket says 172.21.15.157:34656) 2023-10-06T03:57:22.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.106+0000 7fdd5b7fe700 1 -- 172.21.15.157:0/766723050 learned_addr learned my addr 172.21.15.157:0/766723050 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:22.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.106+0000 7fdd5bfff700 1 --2- 172.21.15.157:0/766723050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd5c0c3e50 0x7fdd5c145760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:22.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.106+0000 7fdd60de3700 1 --2- 172.21.15.157:0/766723050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd5c0caa00 0x7fdd5c146a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:22.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.106+0000 7fdd5b7fe700 1 -- 172.21.15.157:0/766723050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd5c0caa00 msgr2=0x7fdd5c146a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:22.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.106+0000 7fdd5b7fe700 1 --2- 172.21.15.157:0/766723050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd5c0caa00 0x7fdd5c146a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:22.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.106+0000 7fdd5b7fe700 1 -- 172.21.15.157:0/766723050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd5c0c3e50 msgr2=0x7fdd5c145760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:22.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.106+0000 7fdd5b7fe700 1 --2- 172.21.15.157:0/766723050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd5c0c3e50 0x7fdd5c145760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:22.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.106+0000 7fdd5b7fe700 1 -- 172.21.15.157:0/766723050 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd54009ae0 con 0x7fdd5c0ca630 2023-10-06T03:57:22.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.106+0000 7fdd5b7fe700 1 --2- 172.21.15.157:0/766723050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd5c0ca630 0x7fdd5c06e690 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000c590 tx=0x7fdd5000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:22.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.107+0000 7fdd43fff700 1 -- 172.21.15.157:0/766723050 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50011590 con 0x7fdd5c0ca630 2023-10-06T03:57:22.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.107+0000 7fdd43fff700 1 -- 172.21.15.157:0/766723050 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd50011710 con 0x7fdd5c0ca630 2023-10-06T03:57:22.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.107+0000 7fdd62846700 1 -- 172.21.15.157:0/766723050 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5c143d40 con 0x7fdd5c0ca630 2023-10-06T03:57:22.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.107+0000 7fdd43fff700 1 -- 172.21.15.157:0/766723050 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50011ad0 con 0x7fdd5c0ca630 2023-10-06T03:57:22.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.107+0000 7fdd62846700 1 -- 172.21.15.157:0/766723050 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5c146230 con 0x7fdd5c0ca630 2023-10-06T03:57:22.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.108+0000 7fdd62846700 1 -- 172.21.15.157:0/766723050 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd5c06a8f0 con 0x7fdd5c0ca630 2023-10-06T03:57:22.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.108+0000 7fdd43fff700 1 -- 172.21.15.157:0/766723050 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdd5001b3b0 con 0x7fdd5c0ca630 2023-10-06T03:57:22.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.108+0000 7fdd43fff700 1 --2- 172.21.15.157:0/766723050 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdd48066f30 0x7fdd480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:22.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.109+0000 7fdd43fff700 1 -- 172.21.15.157:0/766723050 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdd50083b40 con 0x7fdd5c0ca630 2023-10-06T03:57:22.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.109+0000 7fdd5bfff700 1 --2- 172.21.15.157:0/766723050 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdd48066f30 0x7fdd480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:22.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.109+0000 7fdd5bfff700 1 --2- 172.21.15.157:0/766723050 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdd48066f30 0x7fdd480693e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009300 tx=0x7fdd4c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:22.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.113+0000 7fdd43fff700 1 -- 172.21.15.157:0/766723050 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd50050c50 con 0x7fdd5c0ca630 2023-10-06T03:57:22.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.294+0000 7fdd62846700 1 -- 172.21.15.157:0/766723050 --> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdd5c1466e0 con 0x7fdd48066f30 2023-10-06T03:57:22.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.303+0000 7fdd43fff700 1 -- 172.21.15.157:0/766723050 <== mgr.4107 v2:172.21.15.175:6832/90661 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286705 (secure 0 0 0) 0x7fdd5c1466e0 con 0x7fdd48066f30 2023-10-06T03:57:22.307 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:22.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.305+0000 7fdd62846700 1 -- 172.21.15.157:0/766723050 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdd48066f30 msgr2=0x7fdd480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:22.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.305+0000 7fdd62846700 1 --2- 172.21.15.157:0/766723050 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdd48066f30 0x7fdd480693e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009300 tx=0x7fdd4c008040 comp rx=0 tx=0).stop 2023-10-06T03:57:22.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.305+0000 7fdd62846700 1 -- 172.21.15.157:0/766723050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd5c0ca630 msgr2=0x7fdd5c06e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:22.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.305+0000 7fdd62846700 1 --2- 172.21.15.157:0/766723050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd5c0ca630 0x7fdd5c06e690 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000c590 tx=0x7fdd5000c8a0 comp rx=0 tx=0).stop 2023-10-06T03:57:22.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.305+0000 7fdd62846700 1 -- 172.21.15.157:0/766723050 shutdown_connections 2023-10-06T03:57:22.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.305+0000 7fdd62846700 1 --2- 172.21.15.157:0/766723050 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdd48066f30 0x7fdd480693e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:22.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.305+0000 7fdd62846700 1 --2- 172.21.15.157:0/766723050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd5c0c3e50 0x7fdd5c145760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:22.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.305+0000 7fdd62846700 1 --2- 172.21.15.157:0/766723050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd5c0caa00 0x7fdd5c146a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:22.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.305+0000 7fdd62846700 1 --2- 172.21.15.157:0/766723050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd5c0ca630 0x7fdd5c06e690 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:22.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.305+0000 7fdd62846700 1 -- 172.21.15.157:0/766723050 >> 172.21.15.157:0/766723050 conn(0x7fdd5c0692b0 msgr2=0x7fdd5c14a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:22.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.305+0000 7fdd62846700 1 -- 172.21.15.157:0/766723050 shutdown_connections 2023-10-06T03:57:22.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.305+0000 7fdd62846700 1 -- 172.21.15.157:0/766723050 wait complete. 2023-10-06T03:57:22.310 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-10-06T03:57:22.319 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-06T03:57:21.427975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2380188,"kb_used_data":1884,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752594532,"statfs":{"total":773094113280,"available":770656800768,"internally_reserved":0,"allocated":1929216,"data_stored":1815075,"data_compressed":64858,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":58,"num_write_kb":275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002727"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378564+0000","last_change":"2023-10-06T03:57:11.372019+0000","last_active":"2023-10-06T03:57:11.378564+0000","last_peered":"2023-10-06T03:57:11.378564+0000","last_clean":"2023-10-06T03:57:11.378564+0000","last_became_active":"2023-10-06T03:57:11.371756+0000","last_became_peered":"2023-10-06T03:57:11.371756+0000","last_unstale":"2023-10-06T03:57:11.378564+0000","last_undegraded":"2023-10-06T03:57:11.378564+0000","last_fullsized":"2023-10-06T03:57:11.378564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:50.086528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372505+0000","last_change":"2023-10-06T03:57:11.370371+0000","last_active":"2023-10-06T03:57:11.372505+0000","last_peered":"2023-10-06T03:57:11.372505+0000","last_clean":"2023-10-06T03:57:11.372505+0000","last_became_active":"2023-10-06T03:57:11.370070+0000","last_became_peered":"2023-10-06T03:57:11.370070+0000","last_unstale":"2023-10-06T03:57:11.372505+0000","last_undegraded":"2023-10-06T03:57:11.372505+0000","last_fullsized":"2023-10-06T03:57:11.372505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:00.917607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371725+0000","last_change":"2023-10-06T03:57:11.366298+0000","last_active":"2023-10-06T03:57:11.371725+0000","last_peered":"2023-10-06T03:57:11.371725+0000","last_clean":"2023-10-06T03:57:11.371725+0000","last_became_active":"2023-10-06T03:57:11.366012+0000","last_became_peered":"2023-10-06T03:57:11.366012+0000","last_unstale":"2023-10-06T03:57:11.371725+0000","last_undegraded":"2023-10-06T03:57:11.371725+0000","last_fullsized":"2023-10-06T03:57:11.371725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:45.802287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.374334+0000","last_change":"2023-10-06T03:57:11.365933+0000","last_active":"2023-10-06T03:57:11.374334+0000","last_peered":"2023-10-06T03:57:11.374334+0000","last_clean":"2023-10-06T03:57:11.374334+0000","last_became_active":"2023-10-06T03:57:11.365673+0000","last_became_peered":"2023-10-06T03:57:11.365673+0000","last_unstale":"2023-10-06T03:57:11.374334+0000","last_undegraded":"2023-10-06T03:57:11.374334+0000","last_fullsized":"2023-10-06T03:57:11.374334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:40.953382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.365753+0000","last_change":"2023-10-06T03:57:11.361569+0000","last_active":"2023-10-06T03:57:11.365753+0000","last_peered":"2023-10-06T03:57:11.365753+0000","last_clean":"2023-10-06T03:57:11.365753+0000","last_became_active":"2023-10-06T03:57:11.361302+0000","last_became_peered":"2023-10-06T03:57:11.361302+0000","last_unstale":"2023-10-06T03:57:11.365753+0000","last_undegraded":"2023-10-06T03:57:11.365753+0000","last_fullsized":"2023-10-06T03:57:11.365753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:57.552493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372859+0000","last_change":"2023-10-06T03:57:11.369953+0000","last_active":"2023-10-06T03:57:11.372859+0000","last_peered":"2023-10-06T03:57:11.372859+0000","last_clean":"2023-10-06T03:57:11.372859+0000","last_became_active":"2023-10-06T03:57:11.368775+0000","last_became_peered":"2023-10-06T03:57:11.368775+0000","last_unstale":"2023-10-06T03:57:11.372859+0000","last_undegraded":"2023-10-06T03:57:11.372859+0000","last_fullsized":"2023-10-06T03:57:11.372859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:22.368117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.380867+0000","last_change":"2023-10-06T03:57:11.378048+0000","last_active":"2023-10-06T03:57:11.380867+0000","last_peered":"2023-10-06T03:57:11.380867+0000","last_clean":"2023-10-06T03:57:11.380867+0000","last_became_active":"2023-10-06T03:57:11.377786+0000","last_became_peered":"2023-10-06T03:57:11.377786+0000","last_unstale":"2023-10-06T03:57:11.380867+0000","last_undegraded":"2023-10-06T03:57:11.380867+0000","last_fullsized":"2023-10-06T03:57:11.380867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:12:54.218593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377264+0000","last_change":"2023-10-06T03:57:11.374003+0000","last_active":"2023-10-06T03:57:11.377264+0000","last_peered":"2023-10-06T03:57:11.377264+0000","last_clean":"2023-10-06T03:57:11.377264+0000","last_became_active":"2023-10-06T03:57:11.373765+0000","last_became_peered":"2023-10-06T03:57:11.373765+0000","last_unstale":"2023-10-06T03:57:11.377264+0000","last_undegraded":"2023-10-06T03:57:11.377264+0000","last_fullsized":"2023-10-06T03:57:11.377264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:05.838007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378921+0000","last_change":"2023-10-06T03:57:11.373794+0000","last_active":"2023-10-06T03:57:11.378921+0000","last_peered":"2023-10-06T03:57:11.378921+0000","last_clean":"2023-10-06T03:57:11.378921+0000","last_became_active":"2023-10-06T03:57:11.368004+0000","last_became_peered":"2023-10-06T03:57:11.368004+0000","last_unstale":"2023-10-06T03:57:11.378921+0000","last_undegraded":"2023-10-06T03:57:11.378921+0000","last_fullsized":"2023-10-06T03:57:11.378921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:16.760049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.382630+0000","last_change":"2023-10-06T03:57:11.379370+0000","last_active":"2023-10-06T03:57:11.382630+0000","last_peered":"2023-10-06T03:57:11.382630+0000","last_clean":"2023-10-06T03:57:11.382630+0000","last_became_active":"2023-10-06T03:57:11.379107+0000","last_became_peered":"2023-10-06T03:57:11.379107+0000","last_unstale":"2023-10-06T03:57:11.382630+0000","last_undegraded":"2023-10-06T03:57:11.382630+0000","last_fullsized":"2023-10-06T03:57:11.382630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:18.373103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371742+0000","last_change":"2023-10-06T03:57:11.369172+0000","last_active":"2023-10-06T03:57:11.371742+0000","last_peered":"2023-10-06T03:57:11.371742+0000","last_clean":"2023-10-06T03:57:11.371742+0000","last_became_active":"2023-10-06T03:57:11.368256+0000","last_became_peered":"2023-10-06T03:57:11.368256+0000","last_unstale":"2023-10-06T03:57:11.371742+0000","last_undegraded":"2023-10-06T03:57:11.371742+0000","last_fullsized":"2023-10-06T03:57:11.371742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:50.736367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373531+0000","last_change":"2023-10-06T03:57:11.368685+0000","last_active":"2023-10-06T03:57:11.373531+0000","last_peered":"2023-10-06T03:57:11.373531+0000","last_clean":"2023-10-06T03:57:11.373531+0000","last_became_active":"2023-10-06T03:57:11.368425+0000","last_became_peered":"2023-10-06T03:57:11.368425+0000","last_unstale":"2023-10-06T03:57:11.373531+0000","last_undegraded":"2023-10-06T03:57:11.373531+0000","last_fullsized":"2023-10-06T03:57:11.373531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:06.004423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372891+0000","last_change":"2023-10-06T03:57:11.367646+0000","last_active":"2023-10-06T03:57:11.372891+0000","last_peered":"2023-10-06T03:57:11.372891+0000","last_clean":"2023-10-06T03:57:11.372891+0000","last_became_active":"2023-10-06T03:57:11.367079+0000","last_became_peered":"2023-10-06T03:57:11.367079+0000","last_unstale":"2023-10-06T03:57:11.372891+0000","last_undegraded":"2023-10-06T03:57:11.372891+0000","last_fullsized":"2023-10-06T03:57:11.372891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:09:54.331206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377012+0000","last_change":"2023-10-06T03:57:11.373481+0000","last_active":"2023-10-06T03:57:11.377012+0000","last_peered":"2023-10-06T03:57:11.377012+0000","last_clean":"2023-10-06T03:57:11.377012+0000","last_became_active":"2023-10-06T03:57:11.373216+0000","last_became_peered":"2023-10-06T03:57:11.373216+0000","last_unstale":"2023-10-06T03:57:11.377012+0000","last_undegraded":"2023-10-06T03:57:11.377012+0000","last_fullsized":"2023-10-06T03:57:11.377012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:56.083203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.379250+0000","last_change":"2023-10-06T03:57:11.376012+0000","last_active":"2023-10-06T03:57:11.379250+0000","last_peered":"2023-10-06T03:57:11.379250+0000","last_clean":"2023-10-06T03:57:11.379250+0000","last_became_active":"2023-10-06T03:57:11.375689+0000","last_became_peered":"2023-10-06T03:57:11.375689+0000","last_unstale":"2023-10-06T03:57:11.379250+0000","last_undegraded":"2023-10-06T03:57:11.379250+0000","last_fullsized":"2023-10-06T03:57:11.379250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:42.953739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372989+0000","last_change":"2023-10-06T03:57:11.366808+0000","last_active":"2023-10-06T03:57:11.372989+0000","last_peered":"2023-10-06T03:57:11.372989+0000","last_clean":"2023-10-06T03:57:11.372989+0000","last_became_active":"2023-10-06T03:57:11.366543+0000","last_became_peered":"2023-10-06T03:57:11.366543+0000","last_unstale":"2023-10-06T03:57:11.372989+0000","last_undegraded":"2023-10-06T03:57:11.372989+0000","last_fullsized":"2023-10-06T03:57:11.372989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:16.212967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378753+0000","last_change":"2023-10-06T03:57:11.367276+0000","last_active":"2023-10-06T03:57:11.378753+0000","last_peered":"2023-10-06T03:57:11.378753+0000","last_clean":"2023-10-06T03:57:11.378753+0000","last_became_active":"2023-10-06T03:57:11.366767+0000","last_became_peered":"2023-10-06T03:57:11.366767+0000","last_unstale":"2023-10-06T03:57:11.378753+0000","last_undegraded":"2023-10-06T03:57:11.378753+0000","last_fullsized":"2023-10-06T03:57:11.378753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:43.113575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367129+0000","last_change":"2023-10-06T03:57:11.362667+0000","last_active":"2023-10-06T03:57:11.367129+0000","last_peered":"2023-10-06T03:57:11.367129+0000","last_clean":"2023-10-06T03:57:11.367129+0000","last_became_active":"2023-10-06T03:57:11.362377+0000","last_became_peered":"2023-10-06T03:57:11.362377+0000","last_unstale":"2023-10-06T03:57:11.367129+0000","last_undegraded":"2023-10-06T03:57:11.367129+0000","last_fullsized":"2023-10-06T03:57:11.367129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:40.600383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.365567+0000","last_change":"2023-10-06T03:57:11.364126+0000","last_active":"2023-10-06T03:57:11.365567+0000","last_peered":"2023-10-06T03:57:11.365567+0000","last_clean":"2023-10-06T03:57:11.365567+0000","last_became_active":"2023-10-06T03:57:11.363792+0000","last_became_peered":"2023-10-06T03:57:11.363792+0000","last_unstale":"2023-10-06T03:57:11.365567+0000","last_undegraded":"2023-10-06T03:57:11.365567+0000","last_fullsized":"2023-10-06T03:57:11.365567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:49.185450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372624+0000","last_change":"2023-10-06T03:57:11.364974+0000","last_active":"2023-10-06T03:57:11.372624+0000","last_peered":"2023-10-06T03:57:11.372624+0000","last_clean":"2023-10-06T03:57:11.372624+0000","last_became_active":"2023-10-06T03:57:11.364726+0000","last_became_peered":"2023-10-06T03:57:11.364726+0000","last_unstale":"2023-10-06T03:57:11.372624+0000","last_undegraded":"2023-10-06T03:57:11.372624+0000","last_fullsized":"2023-10-06T03:57:11.372624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:56.866733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.379227+0000","last_change":"2023-10-06T03:57:11.375987+0000","last_active":"2023-10-06T03:57:11.379227+0000","last_peered":"2023-10-06T03:57:11.379227+0000","last_clean":"2023-10-06T03:57:11.379227+0000","last_became_active":"2023-10-06T03:57:11.375668+0000","last_became_peered":"2023-10-06T03:57:11.375668+0000","last_unstale":"2023-10-06T03:57:11.379227+0000","last_undegraded":"2023-10-06T03:57:11.379227+0000","last_fullsized":"2023-10-06T03:57:11.379227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:53.810875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.366478+0000","last_change":"2023-10-06T03:57:11.365414+0000","last_active":"2023-10-06T03:57:11.366478+0000","last_peered":"2023-10-06T03:57:11.366478+0000","last_clean":"2023-10-06T03:57:11.366478+0000","last_became_active":"2023-10-06T03:57:11.365149+0000","last_became_peered":"2023-10-06T03:57:11.365149+0000","last_unstale":"2023-10-06T03:57:11.366478+0000","last_undegraded":"2023-10-06T03:57:11.366478+0000","last_fullsized":"2023-10-06T03:57:11.366478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:20.911710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.370638+0000","last_change":"2023-10-06T03:57:11.362366+0000","last_active":"2023-10-06T03:57:11.370638+0000","last_peered":"2023-10-06T03:57:11.370638+0000","last_clean":"2023-10-06T03:57:11.370638+0000","last_became_active":"2023-10-06T03:57:11.362128+0000","last_became_peered":"2023-10-06T03:57:11.362128+0000","last_unstale":"2023-10-06T03:57:11.370638+0000","last_undegraded":"2023-10-06T03:57:11.370638+0000","last_fullsized":"2023-10-06T03:57:11.370638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:53.744674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.379282+0000","last_change":"2023-10-06T03:57:11.375969+0000","last_active":"2023-10-06T03:57:11.379282+0000","last_peered":"2023-10-06T03:57:11.379282+0000","last_clean":"2023-10-06T03:57:11.379282+0000","last_became_active":"2023-10-06T03:57:11.375702+0000","last_became_peered":"2023-10-06T03:57:11.375702+0000","last_unstale":"2023-10-06T03:57:11.379282+0000","last_undegraded":"2023-10-06T03:57:11.379282+0000","last_fullsized":"2023-10-06T03:57:11.379282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:31.204231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377763+0000","last_change":"2023-10-06T03:57:11.375099+0000","last_active":"2023-10-06T03:57:11.377763+0000","last_peered":"2023-10-06T03:57:11.377763+0000","last_clean":"2023-10-06T03:57:11.377763+0000","last_became_active":"2023-10-06T03:57:11.374842+0000","last_became_peered":"2023-10-06T03:57:11.374842+0000","last_unstale":"2023-10-06T03:57:11.377763+0000","last_undegraded":"2023-10-06T03:57:11.377763+0000","last_fullsized":"2023-10-06T03:57:11.377763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:54.438974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.369108+0000","last_change":"2023-10-06T03:57:11.365900+0000","last_active":"2023-10-06T03:57:11.369108+0000","last_peered":"2023-10-06T03:57:11.369108+0000","last_clean":"2023-10-06T03:57:11.369108+0000","last_became_active":"2023-10-06T03:57:11.361349+0000","last_became_peered":"2023-10-06T03:57:11.361349+0000","last_unstale":"2023-10-06T03:57:11.369108+0000","last_undegraded":"2023-10-06T03:57:11.369108+0000","last_fullsized":"2023-10-06T03:57:11.369108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:30.646781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378945+0000","last_change":"2023-10-06T03:57:11.376082+0000","last_active":"2023-10-06T03:57:11.378945+0000","last_peered":"2023-10-06T03:57:11.378945+0000","last_clean":"2023-10-06T03:57:11.378945+0000","last_became_active":"2023-10-06T03:57:11.375846+0000","last_became_peered":"2023-10-06T03:57:11.375846+0000","last_unstale":"2023-10-06T03:57:11.378945+0000","last_undegraded":"2023-10-06T03:57:11.378945+0000","last_fullsized":"2023-10-06T03:57:11.378945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:28.080614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:12.369461+0000","last_change":"2023-10-06T03:57:10.348317+0000","last_active":"2023-10-06T03:57:12.369461+0000","last_peered":"2023-10-06T03:57:12.369461+0000","last_clean":"2023-10-06T03:57:12.369461+0000","last_became_active":"2023-10-06T03:57:10.348078+0000","last_became_peered":"2023-10-06T03:57:10.348078+0000","last_unstale":"2023-10-06T03:57:12.369461+0000","last_undegraded":"2023-10-06T03:57:12.369461+0000","last_fullsized":"2023-10-06T03:57:12.369461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:55.058324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372610+0000","last_change":"2023-10-06T03:57:11.370506+0000","last_active":"2023-10-06T03:57:11.372610+0000","last_peered":"2023-10-06T03:57:11.372610+0000","last_clean":"2023-10-06T03:57:11.372610+0000","last_became_active":"2023-10-06T03:57:11.370245+0000","last_became_peered":"2023-10-06T03:57:11.370245+0000","last_unstale":"2023-10-06T03:57:11.372610+0000","last_undegraded":"2023-10-06T03:57:11.372610+0000","last_fullsized":"2023-10-06T03:57:11.372610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:47:18.189772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367542+0000","last_change":"2023-10-06T03:57:10.345401+0000","last_active":"2023-10-06T03:57:11.367542+0000","last_peered":"2023-10-06T03:57:11.367542+0000","last_clean":"2023-10-06T03:57:11.367542+0000","last_became_active":"2023-10-06T03:57:10.345150+0000","last_became_peered":"2023-10-06T03:57:10.345150+0000","last_unstale":"2023-10-06T03:57:11.367542+0000","last_undegraded":"2023-10-06T03:57:11.367542+0000","last_fullsized":"2023-10-06T03:57:11.367542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:25.758193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.379135+0000","last_change":"2023-10-06T03:57:11.376046+0000","last_active":"2023-10-06T03:57:11.379135+0000","last_peered":"2023-10-06T03:57:11.379135+0000","last_clean":"2023-10-06T03:57:11.379135+0000","last_became_active":"2023-10-06T03:57:11.375790+0000","last_became_peered":"2023-10-06T03:57:11.375790+0000","last_unstale":"2023-10-06T03:57:11.379135+0000","last_undegraded":"2023-10-06T03:57:11.379135+0000","last_fullsized":"2023-10-06T03:57:11.379135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:33.775431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.359916+0000","last_change":"2023-10-06T03:57:10.347562+0000","last_active":"2023-10-06T03:57:11.359916+0000","last_peered":"2023-10-06T03:57:11.359916+0000","last_clean":"2023-10-06T03:57:11.359916+0000","last_became_active":"2023-10-06T03:57:10.347294+0000","last_became_peered":"2023-10-06T03:57:10.347294+0000","last_unstale":"2023-10-06T03:57:11.359916+0000","last_undegraded":"2023-10-06T03:57:11.359916+0000","last_fullsized":"2023-10-06T03:57:11.359916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:25.090992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.375571+0000","last_change":"2023-10-06T03:57:11.372022+0000","last_active":"2023-10-06T03:57:11.375571+0000","last_peered":"2023-10-06T03:57:11.375571+0000","last_clean":"2023-10-06T03:57:11.375571+0000","last_became_active":"2023-10-06T03:57:11.371758+0000","last_became_peered":"2023-10-06T03:57:11.371758+0000","last_unstale":"2023-10-06T03:57:11.375571+0000","last_undegraded":"2023-10-06T03:57:11.375571+0000","last_fullsized":"2023-10-06T03:57:11.375571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:10.922157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.354126+0000","last_change":"2023-10-06T03:57:10.347406+0000","last_active":"2023-10-06T03:57:11.354126+0000","last_peered":"2023-10-06T03:57:11.354126+0000","last_clean":"2023-10-06T03:57:11.354126+0000","last_became_active":"2023-10-06T03:57:10.347151+0000","last_became_peered":"2023-10-06T03:57:10.347151+0000","last_unstale":"2023-10-06T03:57:11.354126+0000","last_undegraded":"2023-10-06T03:57:11.354126+0000","last_fullsized":"2023-10-06T03:57:11.354126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:20.427620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372580+0000","last_change":"2023-10-06T03:57:11.370240+0000","last_active":"2023-10-06T03:57:11.372580+0000","last_peered":"2023-10-06T03:57:11.372580+0000","last_clean":"2023-10-06T03:57:11.372580+0000","last_became_active":"2023-10-06T03:57:11.369938+0000","last_became_peered":"2023-10-06T03:57:11.369938+0000","last_unstale":"2023-10-06T03:57:11.372580+0000","last_undegraded":"2023-10-06T03:57:11.372580+0000","last_fullsized":"2023-10-06T03:57:11.372580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:07.105832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.364676+0000","last_change":"2023-10-06T03:57:10.346054+0000","last_active":"2023-10-06T03:57:11.364676+0000","last_peered":"2023-10-06T03:57:11.364676+0000","last_clean":"2023-10-06T03:57:11.364676+0000","last_became_active":"2023-10-06T03:57:10.345721+0000","last_became_peered":"2023-10-06T03:57:10.345721+0000","last_unstale":"2023-10-06T03:57:11.364676+0000","last_undegraded":"2023-10-06T03:57:11.364676+0000","last_fullsized":"2023-10-06T03:57:11.364676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:54.481304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378702+0000","last_change":"2023-10-06T03:57:11.376525+0000","last_active":"2023-10-06T03:57:11.378702+0000","last_peered":"2023-10-06T03:57:11.378702+0000","last_clean":"2023-10-06T03:57:11.378702+0000","last_became_active":"2023-10-06T03:57:11.376291+0000","last_became_peered":"2023-10-06T03:57:11.376291+0000","last_unstale":"2023-10-06T03:57:11.378702+0000","last_undegraded":"2023-10-06T03:57:11.378702+0000","last_fullsized":"2023-10-06T03:57:11.378702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:31.993426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:16.371903+0000","last_change":"2023-10-06T03:57:10.340531+0000","last_active":"2023-10-06T03:57:16.371903+0000","last_peered":"2023-10-06T03:57:16.371903+0000","last_clean":"2023-10-06T03:57:16.371903+0000","last_became_active":"2023-10-06T03:57:10.340232+0000","last_became_peered":"2023-10-06T03:57:10.340232+0000","last_unstale":"2023-10-06T03:57:16.371903+0000","last_undegraded":"2023-10-06T03:57:16.371903+0000","last_fullsized":"2023-10-06T03:57:16.371903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:43:26.332055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377803+0000","last_change":"2023-10-06T03:57:11.375070+0000","last_active":"2023-10-06T03:57:11.377803+0000","last_peered":"2023-10-06T03:57:11.377803+0000","last_clean":"2023-10-06T03:57:11.377803+0000","last_became_active":"2023-10-06T03:57:11.374892+0000","last_became_peered":"2023-10-06T03:57:11.374892+0000","last_unstale":"2023-10-06T03:57:11.377803+0000","last_undegraded":"2023-10-06T03:57:11.377803+0000","last_fullsized":"2023-10-06T03:57:11.377803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:51.198745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"21'6","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:16.375229+0000","last_change":"2023-10-06T03:57:10.346758+0000","last_active":"2023-10-06T03:57:16.375229+0000","last_peered":"2023-10-06T03:57:16.375229+0000","last_clean":"2023-10-06T03:57:16.375229+0000","last_became_active":"2023-10-06T03:57:10.346430+0000","last_became_peered":"2023-10-06T03:57:10.346430+0000","last_unstale":"2023-10-06T03:57:16.375229+0000","last_undegraded":"2023-10-06T03:57:16.375229+0000","last_fullsized":"2023-10-06T03:57:16.375229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:10.809242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371658+0000","last_change":"2023-10-06T03:57:11.367808+0000","last_active":"2023-10-06T03:57:11.371658+0000","last_peered":"2023-10-06T03:57:11.371658+0000","last_clean":"2023-10-06T03:57:11.371658+0000","last_became_active":"2023-10-06T03:57:11.367565+0000","last_became_peered":"2023-10-06T03:57:11.367565+0000","last_unstale":"2023-10-06T03:57:11.371658+0000","last_undegraded":"2023-10-06T03:57:11.371658+0000","last_fullsized":"2023-10-06T03:57:11.371658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:14.409116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:16.374798+0000","last_change":"2023-10-06T03:57:10.351062+0000","last_active":"2023-10-06T03:57:16.374798+0000","last_peered":"2023-10-06T03:57:16.374798+0000","last_clean":"2023-10-06T03:57:16.374798+0000","last_became_active":"2023-10-06T03:57:10.350795+0000","last_became_peered":"2023-10-06T03:57:10.350795+0000","last_unstale":"2023-10-06T03:57:16.374798+0000","last_undegraded":"2023-10-06T03:57:16.374798+0000","last_fullsized":"2023-10-06T03:57:16.374798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:54:45.714518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.370549+0000","last_change":"2023-10-06T03:57:11.367489+0000","last_active":"2023-10-06T03:57:11.370549+0000","last_peered":"2023-10-06T03:57:11.370549+0000","last_clean":"2023-10-06T03:57:11.370549+0000","last_became_active":"2023-10-06T03:57:11.367254+0000","last_became_peered":"2023-10-06T03:57:11.367254+0000","last_unstale":"2023-10-06T03:57:11.370549+0000","last_undegraded":"2023-10-06T03:57:11.370549+0000","last_fullsized":"2023-10-06T03:57:11.370549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:04.706485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.352684+0000","last_change":"2023-10-06T03:57:10.338125+0000","last_active":"2023-10-06T03:57:11.352684+0000","last_peered":"2023-10-06T03:57:11.352684+0000","last_clean":"2023-10-06T03:57:11.352684+0000","last_became_active":"2023-10-06T03:57:10.337798+0000","last_became_peered":"2023-10-06T03:57:10.337798+0000","last_unstale":"2023-10-06T03:57:11.352684+0000","last_undegraded":"2023-10-06T03:57:11.352684+0000","last_fullsized":"2023-10-06T03:57:11.352684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:00.482657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378957+0000","last_change":"2023-10-06T03:57:11.376461+0000","last_active":"2023-10-06T03:57:11.378957+0000","last_peered":"2023-10-06T03:57:11.378957+0000","last_clean":"2023-10-06T03:57:11.378957+0000","last_became_active":"2023-10-06T03:57:11.376223+0000","last_became_peered":"2023-10-06T03:57:11.376223+0000","last_unstale":"2023-10-06T03:57:11.378957+0000","last_undegraded":"2023-10-06T03:57:11.378957+0000","last_fullsized":"2023-10-06T03:57:11.378957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:04:37.408871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.364778+0000","last_change":"2023-10-06T03:57:10.350511+0000","last_active":"2023-10-06T03:57:11.364778+0000","last_peered":"2023-10-06T03:57:11.364778+0000","last_clean":"2023-10-06T03:57:11.364778+0000","last_became_active":"2023-10-06T03:57:10.350219+0000","last_became_peered":"2023-10-06T03:57:10.350219+0000","last_unstale":"2023-10-06T03:57:11.364778+0000","last_undegraded":"2023-10-06T03:57:11.364778+0000","last_fullsized":"2023-10-06T03:57:11.364778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:56.043506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.364939+0000","last_change":"2023-10-06T03:57:11.362700+0000","last_active":"2023-10-06T03:57:11.364939+0000","last_peered":"2023-10-06T03:57:11.364939+0000","last_clean":"2023-10-06T03:57:11.364939+0000","last_became_active":"2023-10-06T03:57:11.362443+0000","last_became_peered":"2023-10-06T03:57:11.362443+0000","last_unstale":"2023-10-06T03:57:11.364939+0000","last_undegraded":"2023-10-06T03:57:11.364939+0000","last_fullsized":"2023-10-06T03:57:11.364939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:21:09.143085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"21'8","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:16.372505+0000","last_change":"2023-10-06T03:57:10.346259+0000","last_active":"2023-10-06T03:57:16.372505+0000","last_peered":"2023-10-06T03:57:16.372505+0000","last_clean":"2023-10-06T03:57:16.372505+0000","last_became_active":"2023-10-06T03:57:10.345424+0000","last_became_peered":"2023-10-06T03:57:10.345424+0000","last_unstale":"2023-10-06T03:57:16.372505+0000","last_undegraded":"2023-10-06T03:57:16.372505+0000","last_fullsized":"2023-10-06T03:57:16.372505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:19.278206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378006+0000","last_change":"2023-10-06T03:57:11.376053+0000","last_active":"2023-10-06T03:57:11.378006+0000","last_peered":"2023-10-06T03:57:11.378006+0000","last_clean":"2023-10-06T03:57:11.378006+0000","last_became_active":"2023-10-06T03:57:11.375824+0000","last_became_peered":"2023-10-06T03:57:11.375824+0000","last_unstale":"2023-10-06T03:57:11.378006+0000","last_undegraded":"2023-10-06T03:57:11.378006+0000","last_fullsized":"2023-10-06T03:57:11.378006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:24:26.156866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:12.369811+0000","last_change":"2023-10-06T03:57:10.350408+0000","last_active":"2023-10-06T03:57:12.369811+0000","last_peered":"2023-10-06T03:57:12.369811+0000","last_clean":"2023-10-06T03:57:12.369811+0000","last_became_active":"2023-10-06T03:57:10.350135+0000","last_became_peered":"2023-10-06T03:57:10.350135+0000","last_unstale":"2023-10-06T03:57:12.369811+0000","last_undegraded":"2023-10-06T03:57:12.369811+0000","last_fullsized":"2023-10-06T03:57:12.369811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:43.522111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378954+0000","last_change":"2023-10-06T03:57:11.376524+0000","last_active":"2023-10-06T03:57:11.378954+0000","last_peered":"2023-10-06T03:57:11.378954+0000","last_clean":"2023-10-06T03:57:11.378954+0000","last_became_active":"2023-10-06T03:57:11.376236+0000","last_became_peered":"2023-10-06T03:57:11.376236+0000","last_unstale":"2023-10-06T03:57:11.378954+0000","last_undegraded":"2023-10-06T03:57:11.378954+0000","last_fullsized":"2023-10-06T03:57:11.378954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:33.854818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:16.374755+0000","last_change":"2023-10-06T03:57:10.345799+0000","last_active":"2023-10-06T03:57:16.374755+0000","last_peered":"2023-10-06T03:57:16.374755+0000","last_clean":"2023-10-06T03:57:16.374755+0000","last_became_active":"2023-10-06T03:57:10.345510+0000","last_became_peered":"2023-10-06T03:57:10.345510+0000","last_unstale":"2023-10-06T03:57:16.374755+0000","last_undegraded":"2023-10-06T03:57:16.374755+0000","last_fullsized":"2023-10-06T03:57:16.374755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:09.567314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372383+0000","last_change":"2023-10-06T03:57:11.368662+0000","last_active":"2023-10-06T03:57:11.372383+0000","last_peered":"2023-10-06T03:57:11.372383+0000","last_clean":"2023-10-06T03:57:11.372383+0000","last_became_active":"2023-10-06T03:57:11.368375+0000","last_became_peered":"2023-10-06T03:57:11.368375+0000","last_unstale":"2023-10-06T03:57:11.372383+0000","last_undegraded":"2023-10-06T03:57:11.372383+0000","last_fullsized":"2023-10-06T03:57:11.372383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:27.125523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:12.369814+0000","last_change":"2023-10-06T03:57:10.346945+0000","last_active":"2023-10-06T03:57:12.369814+0000","last_peered":"2023-10-06T03:57:12.369814+0000","last_clean":"2023-10-06T03:57:12.369814+0000","last_became_active":"2023-10-06T03:57:10.346689+0000","last_became_peered":"2023-10-06T03:57:10.346689+0000","last_unstale":"2023-10-06T03:57:12.369814+0000","last_undegraded":"2023-10-06T03:57:12.369814+0000","last_fullsized":"2023-10-06T03:57:12.369814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:43.420455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.375632+0000","last_change":"2023-10-06T03:57:11.372308+0000","last_active":"2023-10-06T03:57:11.375632+0000","last_peered":"2023-10-06T03:57:11.375632+0000","last_clean":"2023-10-06T03:57:11.375632+0000","last_became_active":"2023-10-06T03:57:11.371795+0000","last_became_peered":"2023-10-06T03:57:11.371795+0000","last_unstale":"2023-10-06T03:57:11.375632+0000","last_undegraded":"2023-10-06T03:57:11.375632+0000","last_fullsized":"2023-10-06T03:57:11.375632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:00.324478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.361016+0000","last_change":"2023-10-06T03:57:10.348592+0000","last_active":"2023-10-06T03:57:11.361016+0000","last_peered":"2023-10-06T03:57:11.361016+0000","last_clean":"2023-10-06T03:57:11.361016+0000","last_became_active":"2023-10-06T03:57:10.348291+0000","last_became_peered":"2023-10-06T03:57:10.348291+0000","last_unstale":"2023-10-06T03:57:11.361016+0000","last_undegraded":"2023-10-06T03:57:11.361016+0000","last_fullsized":"2023-10-06T03:57:11.361016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:00.660348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378672+0000","last_change":"2023-10-06T03:57:11.366510+0000","last_active":"2023-10-06T03:57:11.378672+0000","last_peered":"2023-10-06T03:57:11.378672+0000","last_clean":"2023-10-06T03:57:11.378672+0000","last_became_active":"2023-10-06T03:57:11.365077+0000","last_became_peered":"2023-10-06T03:57:11.365077+0000","last_unstale":"2023-10-06T03:57:11.378672+0000","last_undegraded":"2023-10-06T03:57:11.378672+0000","last_fullsized":"2023-10-06T03:57:11.378672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:25.199391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:16.368347+0000","last_change":"2023-10-06T03:57:10.348418+0000","last_active":"2023-10-06T03:57:16.368347+0000","last_peered":"2023-10-06T03:57:16.368347+0000","last_clean":"2023-10-06T03:57:16.368347+0000","last_became_active":"2023-10-06T03:57:10.348055+0000","last_became_peered":"2023-10-06T03:57:10.348055+0000","last_unstale":"2023-10-06T03:57:16.368347+0000","last_undegraded":"2023-10-06T03:57:16.368347+0000","last_fullsized":"2023-10-06T03:57:16.368347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:26.102022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373629+0000","last_change":"2023-10-06T03:57:11.370493+0000","last_active":"2023-10-06T03:57:11.373629+0000","last_peered":"2023-10-06T03:57:11.373629+0000","last_clean":"2023-10-06T03:57:11.373629+0000","last_became_active":"2023-10-06T03:57:11.370073+0000","last_became_peered":"2023-10-06T03:57:11.370073+0000","last_unstale":"2023-10-06T03:57:11.373629+0000","last_undegraded":"2023-10-06T03:57:11.373629+0000","last_fullsized":"2023-10-06T03:57:11.373629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:58.417951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.362176+0000","last_change":"2023-10-06T03:57:10.352966+0000","last_active":"2023-10-06T03:57:11.362176+0000","last_peered":"2023-10-06T03:57:11.362176+0000","last_clean":"2023-10-06T03:57:11.362176+0000","last_became_active":"2023-10-06T03:57:10.352653+0000","last_became_peered":"2023-10-06T03:57:10.352653+0000","last_unstale":"2023-10-06T03:57:11.362176+0000","last_undegraded":"2023-10-06T03:57:11.362176+0000","last_fullsized":"2023-10-06T03:57:11.362176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:32.092557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371704+0000","last_change":"2023-10-06T03:57:11.364052+0000","last_active":"2023-10-06T03:57:11.371704+0000","last_peered":"2023-10-06T03:57:11.371704+0000","last_clean":"2023-10-06T03:57:11.371704+0000","last_became_active":"2023-10-06T03:57:11.363830+0000","last_became_peered":"2023-10-06T03:57:11.363830+0000","last_unstale":"2023-10-06T03:57:11.371704+0000","last_undegraded":"2023-10-06T03:57:11.371704+0000","last_fullsized":"2023-10-06T03:57:11.371704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:04.084019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.357124+0000","last_change":"2023-10-06T03:57:10.349421+0000","last_active":"2023-10-06T03:57:11.357124+0000","last_peered":"2023-10-06T03:57:11.357124+0000","last_clean":"2023-10-06T03:57:11.357124+0000","last_became_active":"2023-10-06T03:57:10.349114+0000","last_became_peered":"2023-10-06T03:57:10.349114+0000","last_unstale":"2023-10-06T03:57:11.357124+0000","last_undegraded":"2023-10-06T03:57:11.357124+0000","last_fullsized":"2023-10-06T03:57:11.357124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:08.209791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372296+0000","last_change":"2023-10-06T03:57:11.367516+0000","last_active":"2023-10-06T03:57:11.372296+0000","last_peered":"2023-10-06T03:57:11.372296+0000","last_clean":"2023-10-06T03:57:11.372296+0000","last_became_active":"2023-10-06T03:57:11.367257+0000","last_became_peered":"2023-10-06T03:57:11.367257+0000","last_unstale":"2023-10-06T03:57:11.372296+0000","last_undegraded":"2023-10-06T03:57:11.372296+0000","last_fullsized":"2023-10-06T03:57:11.372296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:43.628819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.359638+0000","last_change":"2023-10-06T03:57:10.347649+0000","last_active":"2023-10-06T03:57:11.359638+0000","last_peered":"2023-10-06T03:57:11.359638+0000","last_clean":"2023-10-06T03:57:11.359638+0000","last_became_active":"2023-10-06T03:57:10.347433+0000","last_became_peered":"2023-10-06T03:57:10.347433+0000","last_unstale":"2023-10-06T03:57:11.359638+0000","last_undegraded":"2023-10-06T03:57:11.359638+0000","last_fullsized":"2023-10-06T03:57:11.359638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:55.951494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372561+0000","last_change":"2023-10-06T03:57:11.370092+0000","last_active":"2023-10-06T03:57:11.372561+0000","last_peered":"2023-10-06T03:57:11.372561+0000","last_clean":"2023-10-06T03:57:11.372561+0000","last_became_active":"2023-10-06T03:57:11.369825+0000","last_became_peered":"2023-10-06T03:57:11.369825+0000","last_unstale":"2023-10-06T03:57:11.372561+0000","last_undegraded":"2023-10-06T03:57:11.372561+0000","last_fullsized":"2023-10-06T03:57:11.372561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:43.415929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367709+0000","last_change":"2023-10-06T03:57:05.663312+0000","last_active":"2023-10-06T03:57:11.367709+0000","last_peered":"2023-10-06T03:57:11.367709+0000","last_clean":"2023-10-06T03:57:11.367709+0000","last_became_active":"2023-10-06T03:57:05.663046+0000","last_became_peered":"2023-10-06T03:57:05.663046+0000","last_unstale":"2023-10-06T03:57:11.367709+0000","last_undegraded":"2023-10-06T03:57:11.367709+0000","last_fullsized":"2023-10-06T03:57:11.367709+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:31.174359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.363783+0000","last_change":"2023-10-06T03:57:10.348930+0000","last_active":"2023-10-06T03:57:11.363783+0000","last_peered":"2023-10-06T03:57:11.363783+0000","last_clean":"2023-10-06T03:57:11.363783+0000","last_became_active":"2023-10-06T03:57:10.348608+0000","last_became_peered":"2023-10-06T03:57:10.348608+0000","last_unstale":"2023-10-06T03:57:11.363783+0000","last_undegraded":"2023-10-06T03:57:11.363783+0000","last_fullsized":"2023-10-06T03:57:11.363783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:00:09.283718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377785+0000","last_change":"2023-10-06T03:57:11.371108+0000","last_active":"2023-10-06T03:57:11.377785+0000","last_peered":"2023-10-06T03:57:11.377785+0000","last_clean":"2023-10-06T03:57:11.377785+0000","last_became_active":"2023-10-06T03:57:11.370847+0000","last_became_peered":"2023-10-06T03:57:11.370847+0000","last_unstale":"2023-10-06T03:57:11.377785+0000","last_undegraded":"2023-10-06T03:57:11.377785+0000","last_fullsized":"2023-10-06T03:57:11.377785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:38.111171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.362274+0000","last_change":"2023-10-06T03:57:05.318834+0000","last_active":"2023-10-06T03:57:11.362274+0000","last_peered":"2023-10-06T03:57:11.362274+0000","last_clean":"2023-10-06T03:57:11.362274+0000","last_became_active":"2023-10-06T03:57:05.318572+0000","last_became_peered":"2023-10-06T03:57:05.318572+0000","last_unstale":"2023-10-06T03:57:11.362274+0000","last_undegraded":"2023-10-06T03:57:11.362274+0000","last_fullsized":"2023-10-06T03:57:11.362274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:21.951095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.358229+0000","last_change":"2023-10-06T03:57:10.348556+0000","last_active":"2023-10-06T03:57:11.358229+0000","last_peered":"2023-10-06T03:57:11.358229+0000","last_clean":"2023-10-06T03:57:11.358229+0000","last_became_active":"2023-10-06T03:57:10.348271+0000","last_became_peered":"2023-10-06T03:57:10.348271+0000","last_unstale":"2023-10-06T03:57:11.358229+0000","last_undegraded":"2023-10-06T03:57:11.358229+0000","last_fullsized":"2023-10-06T03:57:11.358229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:58.470989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.368920+0000","last_change":"2023-10-06T03:57:11.365237+0000","last_active":"2023-10-06T03:57:11.368920+0000","last_peered":"2023-10-06T03:57:11.368920+0000","last_clean":"2023-10-06T03:57:11.368920+0000","last_became_active":"2023-10-06T03:57:11.365006+0000","last_became_peered":"2023-10-06T03:57:11.365006+0000","last_unstale":"2023-10-06T03:57:11.368920+0000","last_undegraded":"2023-10-06T03:57:11.368920+0000","last_fullsized":"2023-10-06T03:57:11.368920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:33.005015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.355768+0000","last_change":"2023-10-06T03:57:05.314695+0000","last_active":"2023-10-06T03:57:11.355768+0000","last_peered":"2023-10-06T03:57:11.355768+0000","last_clean":"2023-10-06T03:57:11.355768+0000","last_became_active":"2023-10-06T03:57:05.314408+0000","last_became_peered":"2023-10-06T03:57:05.314408+0000","last_unstale":"2023-10-06T03:57:11.355768+0000","last_undegraded":"2023-10-06T03:57:11.355768+0000","last_fullsized":"2023-10-06T03:57:11.355768+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:19.042145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367206+0000","last_change":"2023-10-06T03:57:10.346078+0000","last_active":"2023-10-06T03:57:11.367206+0000","last_peered":"2023-10-06T03:57:11.367206+0000","last_clean":"2023-10-06T03:57:11.367206+0000","last_became_active":"2023-10-06T03:57:10.345769+0000","last_became_peered":"2023-10-06T03:57:10.345769+0000","last_unstale":"2023-10-06T03:57:11.367206+0000","last_undegraded":"2023-10-06T03:57:11.367206+0000","last_fullsized":"2023-10-06T03:57:11.367206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:16.630681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.376583+0000","last_change":"2023-10-06T03:57:11.370372+0000","last_active":"2023-10-06T03:57:11.376583+0000","last_peered":"2023-10-06T03:57:11.376583+0000","last_clean":"2023-10-06T03:57:11.376583+0000","last_became_active":"2023-10-06T03:57:11.370076+0000","last_became_peered":"2023-10-06T03:57:11.370076+0000","last_unstale":"2023-10-06T03:57:11.376583+0000","last_undegraded":"2023-10-06T03:57:11.376583+0000","last_fullsized":"2023-10-06T03:57:11.376583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:18:59.517155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.362848+0000","last_change":"2023-10-06T03:57:05.318778+0000","last_active":"2023-10-06T03:57:11.362848+0000","last_peered":"2023-10-06T03:57:11.362848+0000","last_clean":"2023-10-06T03:57:11.362848+0000","last_became_active":"2023-10-06T03:57:05.318456+0000","last_became_peered":"2023-10-06T03:57:05.318456+0000","last_unstale":"2023-10-06T03:57:11.362848+0000","last_undegraded":"2023-10-06T03:57:11.362848+0000","last_fullsized":"2023-10-06T03:57:11.362848+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:39.769241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.362382+0000","last_change":"2023-10-06T03:57:05.664245+0000","last_active":"2023-10-06T03:57:11.362382+0000","last_peered":"2023-10-06T03:57:11.362382+0000","last_clean":"2023-10-06T03:57:11.362382+0000","last_became_active":"2023-10-06T03:57:05.663426+0000","last_became_peered":"2023-10-06T03:57:05.663426+0000","last_unstale":"2023-10-06T03:57:11.362382+0000","last_undegraded":"2023-10-06T03:57:11.362382+0000","last_fullsized":"2023-10-06T03:57:11.362382+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:29.322599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.359971+0000","last_change":"2023-10-06T03:57:10.351059+0000","last_active":"2023-10-06T03:57:11.359971+0000","last_peered":"2023-10-06T03:57:11.359971+0000","last_clean":"2023-10-06T03:57:11.359971+0000","last_became_active":"2023-10-06T03:57:10.348879+0000","last_became_peered":"2023-10-06T03:57:10.348879+0000","last_unstale":"2023-10-06T03:57:11.359971+0000","last_undegraded":"2023-10-06T03:57:11.359971+0000","last_fullsized":"2023-10-06T03:57:11.359971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:35.764148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.375601+0000","last_change":"2023-10-06T03:57:11.373786+0000","last_active":"2023-10-06T03:57:11.375601+0000","last_peered":"2023-10-06T03:57:11.375601+0000","last_clean":"2023-10-06T03:57:11.375601+0000","last_became_active":"2023-10-06T03:57:11.373452+0000","last_became_peered":"2023-10-06T03:57:11.373452+0000","last_unstale":"2023-10-06T03:57:11.375601+0000","last_undegraded":"2023-10-06T03:57:11.375601+0000","last_fullsized":"2023-10-06T03:57:11.375601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:20.524065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.364987+0000","last_change":"2023-10-06T03:57:05.318790+0000","last_active":"2023-10-06T03:57:11.364987+0000","last_peered":"2023-10-06T03:57:11.364987+0000","last_clean":"2023-10-06T03:57:11.364987+0000","last_became_active":"2023-10-06T03:57:05.318531+0000","last_became_peered":"2023-10-06T03:57:05.318531+0000","last_unstale":"2023-10-06T03:57:11.364987+0000","last_undegraded":"2023-10-06T03:57:11.364987+0000","last_fullsized":"2023-10-06T03:57:11.364987+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:12.876998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.352596+0000","last_change":"2023-10-06T03:57:10.343328+0000","last_active":"2023-10-06T03:57:11.352596+0000","last_peered":"2023-10-06T03:57:11.352596+0000","last_clean":"2023-10-06T03:57:11.352596+0000","last_became_active":"2023-10-06T03:57:10.343051+0000","last_became_peered":"2023-10-06T03:57:10.343051+0000","last_unstale":"2023-10-06T03:57:11.352596+0000","last_undegraded":"2023-10-06T03:57:11.352596+0000","last_fullsized":"2023-10-06T03:57:11.352596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:27.722475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378907+0000","last_change":"2023-10-06T03:57:11.373581+0000","last_active":"2023-10-06T03:57:11.378907+0000","last_peered":"2023-10-06T03:57:11.378907+0000","last_clean":"2023-10-06T03:57:11.378907+0000","last_became_active":"2023-10-06T03:57:11.373264+0000","last_became_peered":"2023-10-06T03:57:11.373264+0000","last_unstale":"2023-10-06T03:57:11.378907+0000","last_undegraded":"2023-10-06T03:57:11.378907+0000","last_fullsized":"2023-10-06T03:57:11.378907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:15.950245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.366791+0000","last_change":"2023-10-06T03:57:05.663770+0000","last_active":"2023-10-06T03:57:11.366791+0000","last_peered":"2023-10-06T03:57:11.366791+0000","last_clean":"2023-10-06T03:57:11.366791+0000","last_became_active":"2023-10-06T03:57:05.663501+0000","last_became_peered":"2023-10-06T03:57:05.663501+0000","last_unstale":"2023-10-06T03:57:11.366791+0000","last_undegraded":"2023-10-06T03:57:11.366791+0000","last_fullsized":"2023-10-06T03:57:11.366791+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:09.806915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367831+0000","last_change":"2023-10-06T03:57:10.346585+0000","last_active":"2023-10-06T03:57:11.367831+0000","last_peered":"2023-10-06T03:57:11.367831+0000","last_clean":"2023-10-06T03:57:11.367831+0000","last_became_active":"2023-10-06T03:57:10.345743+0000","last_became_peered":"2023-10-06T03:57:10.345743+0000","last_unstale":"2023-10-06T03:57:11.367831+0000","last_undegraded":"2023-10-06T03:57:11.367831+0000","last_fullsized":"2023-10-06T03:57:11.367831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:54.589702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378948+0000","last_change":"2023-10-06T03:57:11.373844+0000","last_active":"2023-10-06T03:57:11.378948+0000","last_peered":"2023-10-06T03:57:11.378948+0000","last_clean":"2023-10-06T03:57:11.378948+0000","last_became_active":"2023-10-06T03:57:11.373543+0000","last_became_peered":"2023-10-06T03:57:11.373543+0000","last_unstale":"2023-10-06T03:57:11.378948+0000","last_undegraded":"2023-10-06T03:57:11.378948+0000","last_fullsized":"2023-10-06T03:57:11.378948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:15.152354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:16.372867+0000","last_change":"2023-10-06T03:57:10.344618+0000","last_active":"2023-10-06T03:57:16.372867+0000","last_peered":"2023-10-06T03:57:16.372867+0000","last_clean":"2023-10-06T03:57:16.372867+0000","last_became_active":"2023-10-06T03:57:10.344324+0000","last_became_peered":"2023-10-06T03:57:10.344324+0000","last_unstale":"2023-10-06T03:57:16.372867+0000","last_undegraded":"2023-10-06T03:57:16.372867+0000","last_fullsized":"2023-10-06T03:57:16.372867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:06.062331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.368350+0000","last_change":"2023-10-06T03:57:05.317973+0000","last_active":"2023-10-06T03:57:11.368350+0000","last_peered":"2023-10-06T03:57:11.368350+0000","last_clean":"2023-10-06T03:57:11.368350+0000","last_became_active":"2023-10-06T03:57:05.317715+0000","last_became_peered":"2023-10-06T03:57:05.317715+0000","last_unstale":"2023-10-06T03:57:11.368350+0000","last_undegraded":"2023-10-06T03:57:11.368350+0000","last_fullsized":"2023-10-06T03:57:11.368350+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:37:51.858909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372084+0000","last_change":"2023-10-06T03:57:11.369706+0000","last_active":"2023-10-06T03:57:11.372084+0000","last_peered":"2023-10-06T03:57:11.372084+0000","last_clean":"2023-10-06T03:57:11.372084+0000","last_became_active":"2023-10-06T03:57:11.369445+0000","last_became_peered":"2023-10-06T03:57:11.369445+0000","last_unstale":"2023-10-06T03:57:11.372084+0000","last_undegraded":"2023-10-06T03:57:11.372084+0000","last_fullsized":"2023-10-06T03:57:11.372084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:52.999421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.362125+0000","last_change":"2023-10-06T03:57:05.320847+0000","last_active":"2023-10-06T03:57:11.362125+0000","last_peered":"2023-10-06T03:57:11.362125+0000","last_clean":"2023-10-06T03:57:11.362125+0000","last_became_active":"2023-10-06T03:57:05.320583+0000","last_became_peered":"2023-10-06T03:57:05.320583+0000","last_unstale":"2023-10-06T03:57:11.362125+0000","last_undegraded":"2023-10-06T03:57:11.362125+0000","last_fullsized":"2023-10-06T03:57:11.362125+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:12.772449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.353105+0000","last_change":"2023-10-06T03:57:10.341132+0000","last_active":"2023-10-06T03:57:11.353105+0000","last_peered":"2023-10-06T03:57:11.353105+0000","last_clean":"2023-10-06T03:57:11.353105+0000","last_became_active":"2023-10-06T03:57:10.339424+0000","last_became_peered":"2023-10-06T03:57:10.339424+0000","last_unstale":"2023-10-06T03:57:11.353105+0000","last_undegraded":"2023-10-06T03:57:11.353105+0000","last_fullsized":"2023-10-06T03:57:11.353105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:39.675711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.374372+0000","last_change":"2023-10-06T03:57:11.372497+0000","last_active":"2023-10-06T03:57:11.374372+0000","last_peered":"2023-10-06T03:57:11.374372+0000","last_clean":"2023-10-06T03:57:11.374372+0000","last_became_active":"2023-10-06T03:57:11.372215+0000","last_became_peered":"2023-10-06T03:57:11.372215+0000","last_unstale":"2023-10-06T03:57:11.374372+0000","last_undegraded":"2023-10-06T03:57:11.374372+0000","last_fullsized":"2023-10-06T03:57:11.374372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:23.559480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:13.050807+0000","last_change":"2023-10-06T03:57:10.344051+0000","last_active":"2023-10-06T03:57:13.050807+0000","last_peered":"2023-10-06T03:57:13.050807+0000","last_clean":"2023-10-06T03:57:13.050807+0000","last_became_active":"2023-10-06T03:57:10.343865+0000","last_became_peered":"2023-10-06T03:57:10.343865+0000","last_unstale":"2023-10-06T03:57:13.050807+0000","last_undegraded":"2023-10-06T03:57:13.050807+0000","last_fullsized":"2023-10-06T03:57:13.050807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:32.522470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.376665+0000","last_change":"2023-10-06T03:57:11.369927+0000","last_active":"2023-10-06T03:57:11.376665+0000","last_peered":"2023-10-06T03:57:11.376665+0000","last_clean":"2023-10-06T03:57:11.376665+0000","last_became_active":"2023-10-06T03:57:11.369681+0000","last_became_peered":"2023-10-06T03:57:11.369681+0000","last_unstale":"2023-10-06T03:57:11.376665+0000","last_undegraded":"2023-10-06T03:57:11.376665+0000","last_fullsized":"2023-10-06T03:57:11.376665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:53.858179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:16.373082+0000","last_change":"2023-10-06T03:57:10.349666+0000","last_active":"2023-10-06T03:57:16.373082+0000","last_peered":"2023-10-06T03:57:16.373082+0000","last_clean":"2023-10-06T03:57:16.373082+0000","last_became_active":"2023-10-06T03:57:10.349320+0000","last_became_peered":"2023-10-06T03:57:10.349320+0000","last_unstale":"2023-10-06T03:57:16.373082+0000","last_undegraded":"2023-10-06T03:57:16.373082+0000","last_fullsized":"2023-10-06T03:57:16.373082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:45:49.709519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373619+0000","last_change":"2023-10-06T03:57:11.370442+0000","last_active":"2023-10-06T03:57:11.373619+0000","last_peered":"2023-10-06T03:57:11.373619+0000","last_clean":"2023-10-06T03:57:11.373619+0000","last_became_active":"2023-10-06T03:57:11.370229+0000","last_became_peered":"2023-10-06T03:57:11.370229+0000","last_unstale":"2023-10-06T03:57:11.373619+0000","last_undegraded":"2023-10-06T03:57:11.373619+0000","last_fullsized":"2023-10-06T03:57:11.373619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:26.174331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:16.372960+0000","last_change":"2023-10-06T03:57:10.350263+0000","last_active":"2023-10-06T03:57:16.372960+0000","last_peered":"2023-10-06T03:57:16.372960+0000","last_clean":"2023-10-06T03:57:16.372960+0000","last_became_active":"2023-10-06T03:57:10.350083+0000","last_became_peered":"2023-10-06T03:57:10.350083+0000","last_unstale":"2023-10-06T03:57:16.372960+0000","last_undegraded":"2023-10-06T03:57:16.372960+0000","last_fullsized":"2023-10-06T03:57:16.372960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:05.433032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372850+0000","last_change":"2023-10-06T03:57:11.369397+0000","last_active":"2023-10-06T03:57:11.372850+0000","last_peered":"2023-10-06T03:57:11.372850+0000","last_clean":"2023-10-06T03:57:11.372850+0000","last_became_active":"2023-10-06T03:57:11.369123+0000","last_became_peered":"2023-10-06T03:57:11.369123+0000","last_unstale":"2023-10-06T03:57:11.372850+0000","last_undegraded":"2023-10-06T03:57:11.372850+0000","last_fullsized":"2023-10-06T03:57:11.372850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:28.139036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:13.050204+0000","last_change":"2023-10-06T03:57:10.347049+0000","last_active":"2023-10-06T03:57:13.050204+0000","last_peered":"2023-10-06T03:57:13.050204+0000","last_clean":"2023-10-06T03:57:13.050204+0000","last_became_active":"2023-10-06T03:57:10.344583+0000","last_became_peered":"2023-10-06T03:57:10.344583+0000","last_unstale":"2023-10-06T03:57:13.050204+0000","last_undegraded":"2023-10-06T03:57:13.050204+0000","last_fullsized":"2023-10-06T03:57:13.050204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:32.280257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.370657+0000","last_change":"2023-10-06T03:57:11.366741+0000","last_active":"2023-10-06T03:57:11.370657+0000","last_peered":"2023-10-06T03:57:11.370657+0000","last_clean":"2023-10-06T03:57:11.370657+0000","last_became_active":"2023-10-06T03:57:11.366500+0000","last_became_peered":"2023-10-06T03:57:11.366500+0000","last_unstale":"2023-10-06T03:57:11.370657+0000","last_undegraded":"2023-10-06T03:57:11.370657+0000","last_fullsized":"2023-10-06T03:57:11.370657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:24.714603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:16.372880+0000","last_change":"2023-10-06T03:57:10.348170+0000","last_active":"2023-10-06T03:57:16.372880+0000","last_peered":"2023-10-06T03:57:16.372880+0000","last_clean":"2023-10-06T03:57:16.372880+0000","last_became_active":"2023-10-06T03:57:10.347897+0000","last_became_peered":"2023-10-06T03:57:10.347897+0000","last_unstale":"2023-10-06T03:57:16.372880+0000","last_undegraded":"2023-10-06T03:57:16.372880+0000","last_fullsized":"2023-10-06T03:57:16.372880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:40.756246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372969+0000","last_change":"2023-10-06T03:57:11.364368+0000","last_active":"2023-10-06T03:57:11.372969+0000","last_peered":"2023-10-06T03:57:11.372969+0000","last_clean":"2023-10-06T03:57:11.372969+0000","last_became_active":"2023-10-06T03:57:11.364150+0000","last_became_peered":"2023-10-06T03:57:11.364150+0000","last_unstale":"2023-10-06T03:57:11.372969+0000","last_undegraded":"2023-10-06T03:57:11.372969+0000","last_fullsized":"2023-10-06T03:57:11.372969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:11.038080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.374372+0000","last_change":"2023-10-06T03:57:11.366721+0000","last_active":"2023-10-06T03:57:11.374372+0000","last_peered":"2023-10-06T03:57:11.374372+0000","last_clean":"2023-10-06T03:57:11.374372+0000","last_became_active":"2023-10-06T03:57:11.366524+0000","last_became_peered":"2023-10-06T03:57:11.366524+0000","last_unstale":"2023-10-06T03:57:11.374372+0000","last_undegraded":"2023-10-06T03:57:11.374372+0000","last_fullsized":"2023-10-06T03:57:11.374372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:33.044858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373497+0000","last_change":"2023-10-06T03:57:11.371109+0000","last_active":"2023-10-06T03:57:11.373497+0000","last_peered":"2023-10-06T03:57:11.373497+0000","last_clean":"2023-10-06T03:57:11.373497+0000","last_became_active":"2023-10-06T03:57:11.370848+0000","last_became_peered":"2023-10-06T03:57:11.370848+0000","last_unstale":"2023-10-06T03:57:11.373497+0000","last_undegraded":"2023-10-06T03:57:11.373497+0000","last_fullsized":"2023-10-06T03:57:11.373497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:15.919090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372839+0000","last_change":"2023-10-06T03:57:11.364188+0000","last_active":"2023-10-06T03:57:11.372839+0000","last_peered":"2023-10-06T03:57:11.372839+0000","last_clean":"2023-10-06T03:57:11.372839+0000","last_became_active":"2023-10-06T03:57:11.363875+0000","last_became_peered":"2023-10-06T03:57:11.363875+0000","last_unstale":"2023-10-06T03:57:11.372839+0000","last_undegraded":"2023-10-06T03:57:11.372839+0000","last_fullsized":"2023-10-06T03:57:11.372839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:29.236344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378301+0000","last_change":"2023-10-06T03:57:11.377420+0000","last_active":"2023-10-06T03:57:11.378301+0000","last_peered":"2023-10-06T03:57:11.378301+0000","last_clean":"2023-10-06T03:57:11.378301+0000","last_became_active":"2023-10-06T03:57:11.377126+0000","last_became_peered":"2023-10-06T03:57:11.377126+0000","last_unstale":"2023-10-06T03:57:11.378301+0000","last_undegraded":"2023-10-06T03:57:11.378301+0000","last_fullsized":"2023-10-06T03:57:11.378301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:05.041737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377733+0000","last_change":"2023-10-06T03:57:11.375777+0000","last_active":"2023-10-06T03:57:11.377733+0000","last_peered":"2023-10-06T03:57:11.377733+0000","last_clean":"2023-10-06T03:57:11.377733+0000","last_became_active":"2023-10-06T03:57:11.375539+0000","last_became_peered":"2023-10-06T03:57:11.375539+0000","last_unstale":"2023-10-06T03:57:11.377733+0000","last_undegraded":"2023-10-06T03:57:11.377733+0000","last_fullsized":"2023-10-06T03:57:11.377733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:42.707488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64858,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":311296,"data_stored":670955,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":78835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":78835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.219}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297564,"kb_used_data":276,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074276,"statfs":{"total":96636764160,"available":96332058624,"internally_reserved":0,"allocated":282624,"data_stored":668804,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.254}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":78835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297504,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074336,"statfs":{"total":96636764160,"available":96332120064,"internally_reserved":0,"allocated":221184,"data_stored":78925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":237568,"data_stored":80427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":79459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:57:22.326 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T03:57:22.326 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-06T03:57:22.326 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-06T03:57:22.326 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T03:57:22.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.473+0000 7ff14ddd6700 1 Processor -- start 2023-10-06T03:57:22.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.473+0000 7ff14ddd6700 1 -- start start 2023-10-06T03:57:22.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.473+0000 7ff14ddd6700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff14806cbb0 0x7ff14806d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:22.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.473+0000 7ff14ddd6700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff14806c1f0 0x7ff14806da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:22.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.473+0000 7ff14ddd6700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff14806cf80 0x7ff14814fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:22.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.473+0000 7ff14ddd6700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1480c8c40 con 0x7ff14806cbb0 2023-10-06T03:57:22.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.473+0000 7ff14ddd6700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1480c9780 con 0x7ff14806c1f0 2023-10-06T03:57:22.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.473+0000 7ff14ddd6700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff14806e0c0 con 0x7ff14806cf80 2023-10-06T03:57:22.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.474+0000 7ff1477fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff14806cbb0 0x7ff14806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:22.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.474+0000 7ff1477fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff14806cbb0 0x7ff14806d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34658/0 (socket says 172.21.15.157:34658) 2023-10-06T03:57:22.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.474+0000 7ff1477fe700 1 -- 172.21.15.157:0/3552945956 learned_addr learned my addr 172.21.15.157:0/3552945956 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:22.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.474+0000 7ff146ffd700 1 --2- 172.21.15.157:0/3552945956 >> v2:172.21.15.175:3300/0 conn(0x7ff14806c1f0 0x7ff14806da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:22.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.474+0000 7ff1477fe700 1 -- 172.21.15.157:0/3552945956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff14806cf80 msgr2=0x7ff14814fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:22.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.474+0000 7ff147fff700 1 --2- 172.21.15.157:0/3552945956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff14806cf80 0x7ff14814fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:22.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.474+0000 7ff1477fe700 1 --2- 172.21.15.157:0/3552945956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff14806cf80 0x7ff14814fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:22.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.474+0000 7ff1477fe700 1 -- 172.21.15.157:0/3552945956 >> v2:172.21.15.175:3300/0 conn(0x7ff14806c1f0 msgr2=0x7ff14806da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:22.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.474+0000 7ff1477fe700 1 --2- 172.21.15.157:0/3552945956 >> v2:172.21.15.175:3300/0 conn(0x7ff14806c1f0 0x7ff14806da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:22.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.474+0000 7ff1477fe700 1 -- 172.21.15.157:0/3552945956 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1480c8e50 con 0x7ff14806cbb0 2023-10-06T03:57:22.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.474+0000 7ff146ffd700 1 --2- 172.21.15.157:0/3552945956 >> v2:172.21.15.175:3300/0 conn(0x7ff14806c1f0 0x7ff14806da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:57:22.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.474+0000 7ff147fff700 1 --2- 172.21.15.157:0/3552945956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff14806cf80 0x7ff14814fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:57:22.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.474+0000 7ff1477fe700 1 --2- 172.21.15.157:0/3552945956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff14806cbb0 0x7ff14806d530 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff148157070 tx=0x7ff140009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138c114653546795 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:22.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.474+0000 7ff1467fc700 1 -- 172.21.15.157:0/3552945956 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14000b8f0 con 0x7ff14806cbb0 2023-10-06T03:57:22.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.474+0000 7ff1467fc700 1 -- 172.21.15.157:0/3552945956 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff14000ba50 con 0x7ff14806cbb0 2023-10-06T03:57:22.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.474+0000 7ff1467fc700 1 -- 172.21.15.157:0/3552945956 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14000bdf0 con 0x7ff14806cbb0 2023-10-06T03:57:22.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.475+0000 7ff14ddd6700 1 -- 172.21.15.157:0/3552945956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff14806cbb0 msgr2=0x7ff14806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:22.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.475+0000 7ff14ddd6700 1 --2- 172.21.15.157:0/3552945956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff14806cbb0 0x7ff14806d530 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff148157070 tx=0x7ff140009250 comp rx=0 tx=0).stop 2023-10-06T03:57:22.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.475+0000 7ff14ddd6700 1 -- 172.21.15.157:0/3552945956 shutdown_connections 2023-10-06T03:57:22.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.475+0000 7ff14ddd6700 1 --2- 172.21.15.157:0/3552945956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff14806cf80 0x7ff14814fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:22.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.475+0000 7ff14ddd6700 1 --2- 172.21.15.157:0/3552945956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff14806cbb0 0x7ff14806d530 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:22.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.475+0000 7ff14ddd6700 1 --2- 172.21.15.157:0/3552945956 >> v2:172.21.15.175:3300/0 conn(0x7ff14806c1f0 0x7ff14806da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:22.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.475+0000 7ff14ddd6700 1 -- 172.21.15.157:0/3552945956 >> 172.21.15.157:0/3552945956 conn(0x7ff14806f230 msgr2=0x7ff14806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:22.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.475+0000 7ff14ddd6700 1 -- 172.21.15.157:0/3552945956 shutdown_connections 2023-10-06T03:57:22.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.475+0000 7ff14ddd6700 1 -- 172.21.15.157:0/3552945956 wait complete. 2023-10-06T03:57:22.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.476+0000 7ff14ddd6700 1 Processor -- start 2023-10-06T03:57:22.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.476+0000 7ff14ddd6700 1 -- start start 2023-10-06T03:57:22.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.477+0000 7ff14ddd6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff14806c1f0 0x7ff148158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:22.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.477+0000 7ff14ddd6700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff14806cbb0 0x7ff148169700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:22.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.477+0000 7ff14ddd6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff14806cf80 0x7ff148152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:22.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.477+0000 7ff14ddd6700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff148151000 con 0x7ff14806c1f0 2023-10-06T03:57:22.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.477+0000 7ff14ddd6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1481517f0 con 0x7ff14806cf80 2023-10-06T03:57:22.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.477+0000 7ff14ddd6700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff148166b40 con 0x7ff14806cbb0 2023-10-06T03:57:22.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.477+0000 7ff1477fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff14806c1f0 0x7ff148158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:22.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.477+0000 7ff1477fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff14806c1f0 0x7ff148158ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34674/0 (socket says 172.21.15.157:34674) 2023-10-06T03:57:22.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.477+0000 7ff1477fe700 1 -- 172.21.15.157:0/1198520039 learned_addr learned my addr 172.21.15.157:0/1198520039 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:22.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.477+0000 7ff1477fe700 1 -- 172.21.15.157:0/1198520039 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff14806cbb0 msgr2=0x7ff148169700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:22.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.477+0000 7ff146ffd700 1 --2- 172.21.15.157:0/1198520039 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff14806cbb0 0x7ff148169700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:22.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.477+0000 7ff147fff700 1 --2- 172.21.15.157:0/1198520039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff14806cf80 0x7ff148152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:22.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.477+0000 7ff1477fe700 1 --2- 172.21.15.157:0/1198520039 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff14806cbb0 0x7ff148169700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:22.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.477+0000 7ff1477fe700 1 -- 172.21.15.157:0/1198520039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff14806cf80 msgr2=0x7ff148152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:22.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.477+0000 7ff1477fe700 1 --2- 172.21.15.157:0/1198520039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff14806cf80 0x7ff148152d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:22.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.477+0000 7ff1477fe700 1 -- 172.21.15.157:0/1198520039 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1400094f0 con 0x7ff14806c1f0 2023-10-06T03:57:22.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.477+0000 7ff146ffd700 1 --2- 172.21.15.157:0/1198520039 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff14806cbb0 0x7ff148169700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:57:22.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.477+0000 7ff147fff700 1 --2- 172.21.15.157:0/1198520039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff14806cf80 0x7ff148152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:57:22.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.477+0000 7ff1477fe700 1 --2- 172.21.15.157:0/1198520039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff14806c1f0 0x7ff148158ec0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff1480c3960 tx=0x7ff140017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:22.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.477+0000 7ff12f7fe700 1 -- 172.21.15.157:0/1198520039 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff140011e60 con 0x7ff14806c1f0 2023-10-06T03:57:22.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.478+0000 7ff12f7fe700 1 -- 172.21.15.157:0/1198520039 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff14001a630 con 0x7ff14806c1f0 2023-10-06T03:57:22.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.478+0000 7ff12f7fe700 1 -- 172.21.15.157:0/1198520039 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14001a9f0 con 0x7ff14806c1f0 2023-10-06T03:57:22.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.478+0000 7ff14ddd6700 1 -- 172.21.15.157:0/1198520039 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff148166fd0 con 0x7ff14806c1f0 2023-10-06T03:57:22.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.478+0000 7ff14ddd6700 1 -- 172.21.15.157:0/1198520039 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff148150350 con 0x7ff14806c1f0 2023-10-06T03:57:22.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.478+0000 7ff14ddd6700 1 -- 172.21.15.157:0/1198520039 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff148089ca0 con 0x7ff14806c1f0 2023-10-06T03:57:22.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.479+0000 7ff12f7fe700 1 -- 172.21.15.157:0/1198520039 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff1400233e0 con 0x7ff14806c1f0 2023-10-06T03:57:22.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.479+0000 7ff12f7fe700 1 --2- 172.21.15.157:0/1198520039 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff134066f60 0x7ff134069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:22.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.479+0000 7ff12f7fe700 1 -- 172.21.15.157:0/1198520039 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff1400859d0 con 0x7ff14806c1f0 2023-10-06T03:57:22.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.479+0000 7ff146ffd700 1 --2- 172.21.15.157:0/1198520039 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff134066f60 0x7ff134069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:22.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.480+0000 7ff146ffd700 1 --2- 172.21.15.157:0/1198520039 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff134066f60 0x7ff134069410 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff14815a560 tx=0x7ff13c007600 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:22.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.483+0000 7ff12f7fe700 1 -- 172.21.15.157:0/1198520039 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff140052ae0 con 0x7ff14806c1f0 2023-10-06T03:57:22.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.712+0000 7ff14ddd6700 1 -- 172.21.15.157:0/1198520039 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff1480c9130 con 0x7ff14806c1f0 2023-10-06T03:57:22.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.712+0000 7ff12f7fe700 1 -- 172.21.15.157:0/1198520039 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+196 (secure 0 0 0) 0x7ff1400150e0 con 0x7ff14806c1f0 2023-10-06T03:57:22.715 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:22.715 INFO:teuthology.orchestra.run.smithi157.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-06T03:57:22.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.714+0000 7ff14ddd6700 1 -- 172.21.15.157:0/1198520039 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff134066f60 msgr2=0x7ff134069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:22.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.714+0000 7ff14ddd6700 1 --2- 172.21.15.157:0/1198520039 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff134066f60 0x7ff134069410 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff14815a560 tx=0x7ff13c007600 comp rx=0 tx=0).stop 2023-10-06T03:57:22.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.714+0000 7ff14ddd6700 1 -- 172.21.15.157:0/1198520039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff14806c1f0 msgr2=0x7ff148158ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:22.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.714+0000 7ff14ddd6700 1 --2- 172.21.15.157:0/1198520039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff14806c1f0 0x7ff148158ec0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff1480c3960 tx=0x7ff140017000 comp rx=0 tx=0).stop 2023-10-06T03:57:22.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.714+0000 7ff14ddd6700 1 -- 172.21.15.157:0/1198520039 shutdown_connections 2023-10-06T03:57:22.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.714+0000 7ff14ddd6700 1 --2- 172.21.15.157:0/1198520039 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff134066f60 0x7ff134069410 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:22.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.714+0000 7ff14ddd6700 1 --2- 172.21.15.157:0/1198520039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff14806cf80 0x7ff148152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:22.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.714+0000 7ff14ddd6700 1 --2- 172.21.15.157:0/1198520039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff14806c1f0 0x7ff148158ec0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:22.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.714+0000 7ff14ddd6700 1 --2- 172.21.15.157:0/1198520039 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff14806cbb0 0x7ff148169700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:22.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.714+0000 7ff14ddd6700 1 -- 172.21.15.157:0/1198520039 >> 172.21.15.157:0/1198520039 conn(0x7ff14806f230 msgr2=0x7ff14806d440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:22.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.714+0000 7ff14ddd6700 1 -- 172.21.15.157:0/1198520039 shutdown_connections 2023-10-06T03:57:22.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:22.714+0000 7ff14ddd6700 1 -- 172.21.15.157:0/1198520039 wait complete. 2023-10-06T03:57:25.728 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T03:57:25.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.852+0000 7f081dff7700 1 Processor -- start 2023-10-06T03:57:25.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.852+0000 7f081dff7700 1 -- start start 2023-10-06T03:57:25.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.853+0000 7f081dff7700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f081806cbb0 0x7f081806d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:25.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.853+0000 7f081dff7700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0818150410 0x7f081806da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:25.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.853+0000 7f081dff7700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08180c9780 con 0x7f081806c1f0 2023-10-06T03:57:25.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.853+0000 7f081dff7700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08180c8c40 con 0x7f0818150410 2023-10-06T03:57:25.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.853+0000 7f081dff7700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081806e0c0 con 0x7f081806cbb0 2023-10-06T03:57:25.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.853+0000 7f0816ffd700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f081806c1f0 0x7f081814fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:56648/0 (socket says 172.21.15.157:56648) 2023-10-06T03:57:25.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.853+0000 7f0816ffd700 1 -- 172.21.15.157:0/390178815 learned_addr learned my addr 172.21.15.157:0/390178815 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:25.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.854+0000 7f08177fe700 1 --2- 172.21.15.157:0/390178815 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f081806cbb0 0x7f081806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:25.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.854+0000 7f0817fff700 1 --2- 172.21.15.157:0/390178815 >> v2:172.21.15.175:3300/0 conn(0x7f0818150410 0x7f081806da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:25.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.854+0000 7f08167fc700 1 -- 172.21.15.157:0/390178815 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4180416789 0 0) 0x7f08180c9780 con 0x7f081806c1f0 2023-10-06T03:57:25.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.854+0000 7f08167fc700 1 -- 172.21.15.157:0/390178815 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0800003570 con 0x7f081806c1f0 2023-10-06T03:57:25.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.854+0000 7f08167fc700 1 -- 172.21.15.157:0/390178815 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1673794118 0 0) 0x7f0800003570 con 0x7f081806c1f0 2023-10-06T03:57:25.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.854+0000 7f08167fc700 1 -- 172.21.15.157:0/390178815 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f081806cbb0 msgr2=0x7f081806d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:25.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.854+0000 7f08167fc700 1 --2- 172.21.15.157:0/390178815 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f081806cbb0 0x7f081806d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:25.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.854+0000 7f08167fc700 1 -- 172.21.15.157:0/390178815 >> v2:172.21.15.175:3300/0 conn(0x7f0818150410 msgr2=0x7f081806da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:25.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.854+0000 7f08167fc700 1 --2- 172.21.15.157:0/390178815 >> v2:172.21.15.175:3300/0 conn(0x7f0818150410 0x7f081806da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:25.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.854+0000 7f08167fc700 1 -- 172.21.15.157:0/390178815 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08180c8e50 con 0x7f081806c1f0 2023-10-06T03:57:25.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.854+0000 7f08167fc700 1 -- 172.21.15.157:0/390178815 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f080c001470 con 0x7f081806c1f0 2023-10-06T03:57:25.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.854+0000 7f08167fc700 1 -- 172.21.15.157:0/390178815 >> v1:172.21.15.157:6789/0 conn(0x7f081806c1f0 legacy=0x7f081814fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:25.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.854+0000 7f08167fc700 1 --2- 172.21.15.157:0/390178815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0800003a60 0x7f0800003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:25.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.854+0000 7f08167fc700 1 --2- 172.21.15.157:0/390178815 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f081806cbb0 0x7f0800004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:25.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.854+0000 7f08167fc700 1 --2- 172.21.15.157:0/390178815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0800004d50 0x7f08000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:25.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.854+0000 7f08167fc700 1 -- 172.21.15.157:0/390178815 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08180c8c40 con 0x7f0800003a60 2023-10-06T03:57:25.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.854+0000 7f08167fc700 1 -- 172.21.15.157:0/390178815 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081806e0c0 con 0x7f0800004d50 2023-10-06T03:57:25.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.854+0000 7f08167fc700 1 -- 172.21.15.157:0/390178815 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f080000ba60 con 0x7f081806cbb0 2023-10-06T03:57:25.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.854+0000 7f08177fe700 1 --2- 172.21.15.157:0/390178815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0800003a60 0x7f0800003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:25.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.854+0000 7f0816ffd700 1 --2- 172.21.15.157:0/390178815 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f081806cbb0 0x7f0800004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:25.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.854+0000 7f0817fff700 1 --2- 172.21.15.157:0/390178815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0800004d50 0x7f08000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:25.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.854+0000 7f08177fe700 1 -- 172.21.15.157:0/390178815 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f081806cbb0 msgr2=0x7f0800004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:25.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.854+0000 7f08177fe700 1 --2- 172.21.15.157:0/390178815 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f081806cbb0 0x7f0800004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:25.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.854+0000 7f08177fe700 1 -- 172.21.15.157:0/390178815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0800004d50 msgr2=0x7f08000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:25.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.854+0000 7f08177fe700 1 --2- 172.21.15.157:0/390178815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0800004d50 0x7f08000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:25.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.854+0000 7f08177fe700 1 -- 172.21.15.157:0/390178815 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0800003570 con 0x7f0800003a60 2023-10-06T03:57:25.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.855+0000 7f08177fe700 1 --2- 172.21.15.157:0/390178815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0800003a60 0x7f0800003e70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f080800e210 tx=0x7f080800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51897efcede3044 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:25.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.855+0000 7f08167fc700 1 -- 172.21.15.157:0/390178815 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080800ea60 con 0x7f0800003a60 2023-10-06T03:57:25.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.855+0000 7f08167fc700 1 -- 172.21.15.157:0/390178815 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080800f040 con 0x7f0800003a60 2023-10-06T03:57:25.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.855+0000 7f08167fc700 1 -- 172.21.15.157:0/390178815 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080800ee10 con 0x7f0800003a60 2023-10-06T03:57:25.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.855+0000 7f081dff7700 1 -- 172.21.15.157:0/390178815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0800003a60 msgr2=0x7f0800003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:25.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.855+0000 7f081dff7700 1 --2- 172.21.15.157:0/390178815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0800003a60 0x7f0800003e70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f080800e210 tx=0x7f080800b040 comp rx=0 tx=0).stop 2023-10-06T03:57:25.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.855+0000 7f0817fff700 1 -- 172.21.15.157:0/390178815 reap_dead start 2023-10-06T03:57:25.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.855+0000 7f081dff7700 1 -- 172.21.15.157:0/390178815 shutdown_connections 2023-10-06T03:57:25.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.855+0000 7f081dff7700 1 -- 172.21.15.157:0/390178815 >> 172.21.15.157:0/390178815 conn(0x7f081806f230 msgr2=0x7f081806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:25.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.855+0000 7f081dff7700 1 -- 172.21.15.157:0/390178815 shutdown_connections 2023-10-06T03:57:25.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.855+0000 7f081dff7700 1 -- 172.21.15.157:0/390178815 wait complete. 2023-10-06T03:57:25.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.856+0000 7f081dff7700 1 Processor -- start 2023-10-06T03:57:25.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.857+0000 7f081dff7700 1 -- start start 2023-10-06T03:57:25.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.857+0000 7f081dff7700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0800003a60 0x7f0818154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:25.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.857+0000 7f081dff7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0818154850 0x7f08180ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:25.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.857+0000 7f081dff7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081814e830 0x7f08180c4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:25.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.857+0000 7f081dff7700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0818154080 con 0x7f081814e830 2023-10-06T03:57:25.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.857+0000 7f081dff7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0818155880 con 0x7f0818154850 2023-10-06T03:57:25.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.857+0000 7f081dff7700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0818155580 con 0x7f0800003a60 2023-10-06T03:57:25.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.857+0000 7f0817fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081814e830 0x7f08180c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:25.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.857+0000 7f08177fe700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0800003a60 0x7f0818154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:25.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.857+0000 7f0817fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081814e830 0x7f08180c4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58688/0 (socket says 172.21.15.157:58688) 2023-10-06T03:57:25.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.857+0000 7f0816ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0818154850 0x7f08180ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:25.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.857+0000 7f0817fff700 1 -- 172.21.15.157:0/3004076384 learned_addr learned my addr 172.21.15.157:0/3004076384 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:25.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.858+0000 7f0817fff700 1 -- 172.21.15.157:0/3004076384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0800003a60 msgr2=0x7f0818154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:25.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.858+0000 7f0817fff700 1 --2- 172.21.15.157:0/3004076384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0800003a60 0x7f0818154d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:25.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.858+0000 7f0817fff700 1 -- 172.21.15.157:0/3004076384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0818154850 msgr2=0x7f08180ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:25.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.858+0000 7f0817fff700 1 --2- 172.21.15.157:0/3004076384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0818154850 0x7f08180ca8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:25.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.858+0000 7f0817fff700 1 -- 172.21.15.157:0/3004076384 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0818153de0 con 0x7f081814e830 2023-10-06T03:57:25.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.858+0000 7f08177fe700 1 --2- 172.21.15.157:0/3004076384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0800003a60 0x7f0818154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:57:25.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.858+0000 7f0817fff700 1 --2- 172.21.15.157:0/3004076384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081814e830 0x7f08180c4960 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f081808a730 tx=0x7f08100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:25.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.858+0000 7f07ff7fe700 1 -- 172.21.15.157:0/3004076384 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0810013750 con 0x7f081814e830 2023-10-06T03:57:25.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.858+0000 7f07ff7fe700 1 -- 172.21.15.157:0/3004076384 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08100138b0 con 0x7f081814e830 2023-10-06T03:57:25.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.858+0000 7f081dff7700 1 -- 172.21.15.157:0/3004076384 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08180c6840 con 0x7f081814e830 2023-10-06T03:57:25.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.858+0000 7f07ff7fe700 1 -- 172.21.15.157:0/3004076384 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0810013c50 con 0x7f081814e830 2023-10-06T03:57:25.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.858+0000 7f081dff7700 1 -- 172.21.15.157:0/3004076384 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08180cd390 con 0x7f081814e830 2023-10-06T03:57:25.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.859+0000 7f081dff7700 1 -- 172.21.15.157:0/3004076384 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08180cd550 con 0x7f081814e830 2023-10-06T03:57:25.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.859+0000 7f07ff7fe700 1 -- 172.21.15.157:0/3004076384 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0810014650 con 0x7f081814e830 2023-10-06T03:57:25.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.860+0000 7f07ff7fe700 1 --2- 172.21.15.157:0/3004076384 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0804066f60 0x7f0804069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:25.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.860+0000 7f07ff7fe700 1 -- 172.21.15.157:0/3004076384 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0810084af0 con 0x7f081814e830 2023-10-06T03:57:25.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.860+0000 7f08177fe700 1 --2- 172.21.15.157:0/3004076384 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0804066f60 0x7f0804069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:25.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.860+0000 7f08177fe700 1 --2- 172.21.15.157:0/3004076384 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0804066f60 0x7f0804069410 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f080800afd0 tx=0x7f080801a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:25.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:25.864+0000 7f07ff7fe700 1 -- 172.21.15.157:0/3004076384 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0810051c80 con 0x7f081814e830 2023-10-06T03:57:26.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:26.090+0000 7f081dff7700 1 -- 172.21.15.157:0/3004076384 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f08181559c0 con 0x7f081814e830 2023-10-06T03:57:26.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:26.091+0000 7f07ff7fe700 1 -- 172.21.15.157:0/3004076384 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+196 (secure 0 0 0) 0x7f08100220e0 con 0x7f081814e830 2023-10-06T03:57:26.093 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:26.094 INFO:teuthology.orchestra.run.smithi157.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-06T03:57:26.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:26.092+0000 7f081dff7700 1 -- 172.21.15.157:0/3004076384 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0804066f60 msgr2=0x7f0804069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:26.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:26.092+0000 7f081dff7700 1 --2- 172.21.15.157:0/3004076384 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0804066f60 0x7f0804069410 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f080800afd0 tx=0x7f080801a000 comp rx=0 tx=0).stop 2023-10-06T03:57:26.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:26.093+0000 7f081dff7700 1 -- 172.21.15.157:0/3004076384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081814e830 msgr2=0x7f08180c4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:26.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:26.093+0000 7f081dff7700 1 --2- 172.21.15.157:0/3004076384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081814e830 0x7f08180c4960 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f081808a730 tx=0x7f08100091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:26.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:26.093+0000 7f081dff7700 1 -- 172.21.15.157:0/3004076384 shutdown_connections 2023-10-06T03:57:26.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:26.093+0000 7f081dff7700 1 --2- 172.21.15.157:0/3004076384 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0804066f60 0x7f0804069410 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:26.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:26.093+0000 7f081dff7700 1 --2- 172.21.15.157:0/3004076384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081814e830 0x7f08180c4960 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:26.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:26.093+0000 7f081dff7700 1 --2- 172.21.15.157:0/3004076384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0800003a60 0x7f0818154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:26.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:26.093+0000 7f081dff7700 1 --2- 172.21.15.157:0/3004076384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0818154850 0x7f08180ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:26.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:26.093+0000 7f081dff7700 1 -- 172.21.15.157:0/3004076384 >> 172.21.15.157:0/3004076384 conn(0x7f081806f230 msgr2=0x7f0818150900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:26.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:26.093+0000 7f081dff7700 1 -- 172.21.15.157:0/3004076384 shutdown_connections 2023-10-06T03:57:26.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:26.093+0000 7f081dff7700 1 -- 172.21.15.157:0/3004076384 wait complete. 2023-10-06T03:57:29.108 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T03:57:29.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.233+0000 7f6e03460700 1 Processor -- start 2023-10-06T03:57:29.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.233+0000 7f6e03460700 1 -- start start 2023-10-06T03:57:29.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.233+0000 7f6e03460700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6dfc150410 0x7f6dfc155f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:29.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.234+0000 7f6e03460700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dfc0c9780 con 0x7f6dfc06dc40 2023-10-06T03:57:29.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.234+0000 7f6e03460700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dfc06e0c0 con 0x7f6dfc06cbb0 2023-10-06T03:57:29.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.234+0000 7f6e03460700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dfc0c8c40 con 0x7f6dfc150410 2023-10-06T03:57:29.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.234+0000 7f6e019fd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6dfc150410 0x7f6dfc155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:29.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.234+0000 7f6e009fb700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f6dfc06dc40 0x7f6dfc14fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:56658/0 (socket says 172.21.15.157:56658) 2023-10-06T03:57:29.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.234+0000 7f6e009fb700 1 -- 172.21.15.157:0/3253605991 learned_addr learned my addr 172.21.15.157:0/3253605991 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:29.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.234+0000 7f6e019fd700 1 -- 172.21.15.157:0/3253605991 >> v1:172.21.15.175:6789/0 conn(0x7f6dfc06cbb0 legacy=0x7f6dfc06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:29.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.234+0000 7f6e019fd700 1 -- 172.21.15.157:0/3253605991 >> v1:172.21.15.157:6789/0 conn(0x7f6dfc06dc40 legacy=0x7f6dfc14fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:29.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.234+0000 7f6e019fd700 1 -- 172.21.15.157:0/3253605991 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dfc0c8e50 con 0x7f6dfc150410 2023-10-06T03:57:29.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.234+0000 7f6df3fff700 1 -- 172.21.15.157:0/3253605991 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3024993170 0 0) 0x7f6dfc0c9780 con 0x7f6dfc06dc40 2023-10-06T03:57:29.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.234+0000 7f6e019fd700 1 --2- 172.21.15.157:0/3253605991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dfc150410 0x7f6dfc155f70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6df4009e60 tx=0x7f6df4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d48d8cd7e019ca8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:29.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.234+0000 7f6df3fff700 1 -- 172.21.15.157:0/3253605991 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df400e070 con 0x7f6dfc150410 2023-10-06T03:57:29.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.234+0000 7f6df3fff700 1 -- 172.21.15.157:0/3253605991 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df400bc80 con 0x7f6dfc150410 2023-10-06T03:57:29.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.234+0000 7f6df3fff700 1 -- 172.21.15.157:0/3253605991 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df4012430 con 0x7f6dfc150410 2023-10-06T03:57:29.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.235+0000 7f6e03460700 1 -- 172.21.15.157:0/3253605991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dfc150410 msgr2=0x7f6dfc155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:29.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.235+0000 7f6e03460700 1 --2- 172.21.15.157:0/3253605991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dfc150410 0x7f6dfc155f70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6df4009e60 tx=0x7f6df4009780 comp rx=0 tx=0).stop 2023-10-06T03:57:29.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.235+0000 7f6e03460700 1 -- 172.21.15.157:0/3253605991 shutdown_connections 2023-10-06T03:57:29.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.235+0000 7f6e03460700 1 --2- 172.21.15.157:0/3253605991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dfc150410 0x7f6dfc155f70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:29.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.235+0000 7f6e03460700 1 -- 172.21.15.157:0/3253605991 >> 172.21.15.157:0/3253605991 conn(0x7f6dfc06f230 msgr2=0x7f6dfc06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:29.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.235+0000 7f6e03460700 1 -- 172.21.15.157:0/3253605991 shutdown_connections 2023-10-06T03:57:29.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.235+0000 7f6e03460700 1 -- 172.21.15.157:0/3253605991 wait complete. 2023-10-06T03:57:29.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.236+0000 7f6e03460700 1 Processor -- start 2023-10-06T03:57:29.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.236+0000 7f6e03460700 1 -- start start 2023-10-06T03:57:29.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.237+0000 7f6e03460700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6dfc06cbb0 0x7f6dfc151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:29.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.237+0000 7f6e03460700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dfc150410 0x7f6dfc151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:29.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.237+0000 7f6e03460700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dfc153400 0x7f6dfc169840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:29.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.237+0000 7f6e03460700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc0c3df0 con 0x7f6dfc153400 2023-10-06T03:57:29.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.237+0000 7f6e03460700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc16a520 con 0x7f6dfc150410 2023-10-06T03:57:29.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.237+0000 7f6e03460700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc0c5450 con 0x7f6dfc06cbb0 2023-10-06T03:57:29.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.237+0000 7f6e019fd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dfc153400 0x7f6dfc169840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:29.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.237+0000 7f6e019fd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dfc153400 0x7f6dfc169840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58702/0 (socket says 172.21.15.157:58702) 2023-10-06T03:57:29.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.237+0000 7f6e019fd700 1 -- 172.21.15.157:0/3466113208 learned_addr learned my addr 172.21.15.157:0/3466113208 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:29.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.237+0000 7f6e009fb700 1 --2- 172.21.15.157:0/3466113208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dfc150410 0x7f6dfc151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:29.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.237+0000 7f6e011fc700 1 --2- 172.21.15.157:0/3466113208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6dfc06cbb0 0x7f6dfc151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:29.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.237+0000 7f6e019fd700 1 -- 172.21.15.157:0/3466113208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6dfc06cbb0 msgr2=0x7f6dfc151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:29.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.237+0000 7f6e019fd700 1 --2- 172.21.15.157:0/3466113208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6dfc06cbb0 0x7f6dfc151820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:29.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.237+0000 7f6e019fd700 1 -- 172.21.15.157:0/3466113208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dfc150410 msgr2=0x7f6dfc151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:29.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.237+0000 7f6e019fd700 1 --2- 172.21.15.157:0/3466113208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dfc150410 0x7f6dfc151d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:29.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.237+0000 7f6e019fd700 1 -- 172.21.15.157:0/3466113208 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df4009a20 con 0x7f6dfc153400 2023-10-06T03:57:29.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.237+0000 7f6e019fd700 1 --2- 172.21.15.157:0/3466113208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dfc153400 0x7f6dfc169840 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6df4009d30 tx=0x7f6df4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:29.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.238+0000 7f6df17fa700 1 -- 172.21.15.157:0/3466113208 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df401e710 con 0x7f6dfc153400 2023-10-06T03:57:29.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.238+0000 7f6df17fa700 1 -- 172.21.15.157:0/3466113208 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df401e890 con 0x7f6dfc153400 2023-10-06T03:57:29.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.238+0000 7f6df17fa700 1 -- 172.21.15.157:0/3466113208 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df401ec50 con 0x7f6dfc153400 2023-10-06T03:57:29.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.238+0000 7f6e03460700 1 -- 172.21.15.157:0/3466113208 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dfc0c5730 con 0x7f6dfc153400 2023-10-06T03:57:29.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.238+0000 7f6e03460700 1 -- 172.21.15.157:0/3466113208 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dfc0c5cf0 con 0x7f6dfc153400 2023-10-06T03:57:29.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.239+0000 7f6e03460700 1 -- 172.21.15.157:0/3466113208 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dfc06cf80 con 0x7f6dfc153400 2023-10-06T03:57:29.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.239+0000 7f6df17fa700 1 -- 172.21.15.157:0/3466113208 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6df4024070 con 0x7f6dfc153400 2023-10-06T03:57:29.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.239+0000 7f6df17fa700 1 --2- 172.21.15.157:0/3466113208 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6de0066f60 0x7f6de0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:29.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.239+0000 7f6df17fa700 1 -- 172.21.15.157:0/3466113208 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6df4087180 con 0x7f6dfc153400 2023-10-06T03:57:29.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.239+0000 7f6e011fc700 1 --2- 172.21.15.157:0/3466113208 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6de0066f60 0x7f6de0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:29.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.240+0000 7f6e011fc700 1 --2- 172.21.15.157:0/3466113208 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6de0066f60 0x7f6de0069410 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6df8003870 tx=0x7f6df8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:29.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.244+0000 7f6df17fa700 1 -- 172.21.15.157:0/3466113208 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6df4054340 con 0x7f6dfc153400 2023-10-06T03:57:29.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.471+0000 7f6e03460700 1 -- 172.21.15.157:0/3466113208 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6dfc0c9130 con 0x7f6dfc153400 2023-10-06T03:57:29.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.472+0000 7f6df17fa700 1 -- 172.21.15.157:0/3466113208 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+196 (secure 0 0 0) 0x7f6df4057960 con 0x7f6dfc153400 2023-10-06T03:57:29.474 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:29.475 INFO:teuthology.orchestra.run.smithi157.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-06T03:57:29.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.474+0000 7f6e03460700 1 -- 172.21.15.157:0/3466113208 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6de0066f60 msgr2=0x7f6de0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:29.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.474+0000 7f6e03460700 1 --2- 172.21.15.157:0/3466113208 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6de0066f60 0x7f6de0069410 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6df8003870 tx=0x7f6df8009040 comp rx=0 tx=0).stop 2023-10-06T03:57:29.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.474+0000 7f6e03460700 1 -- 172.21.15.157:0/3466113208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dfc153400 msgr2=0x7f6dfc169840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:29.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.474+0000 7f6e03460700 1 --2- 172.21.15.157:0/3466113208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dfc153400 0x7f6dfc169840 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6df4009d30 tx=0x7f6df4018000 comp rx=0 tx=0).stop 2023-10-06T03:57:29.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.474+0000 7f6e03460700 1 -- 172.21.15.157:0/3466113208 shutdown_connections 2023-10-06T03:57:29.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.474+0000 7f6e03460700 1 --2- 172.21.15.157:0/3466113208 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6de0066f60 0x7f6de0069410 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:29.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.474+0000 7f6e03460700 1 --2- 172.21.15.157:0/3466113208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dfc153400 0x7f6dfc169840 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:29.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.474+0000 7f6e03460700 1 --2- 172.21.15.157:0/3466113208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6dfc06cbb0 0x7f6dfc151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:29.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.474+0000 7f6e03460700 1 --2- 172.21.15.157:0/3466113208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6dfc150410 0x7f6dfc151d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:29.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.474+0000 7f6e03460700 1 -- 172.21.15.157:0/3466113208 >> 172.21.15.157:0/3466113208 conn(0x7f6dfc06f230 msgr2=0x7f6dfc0af240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:29.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.474+0000 7f6e03460700 1 -- 172.21.15.157:0/3466113208 shutdown_connections 2023-10-06T03:57:29.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:29.474+0000 7f6e03460700 1 -- 172.21.15.157:0/3466113208 wait complete. 2023-10-06T03:57:32.489 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T03:57:32.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.618+0000 7f3c75935700 1 Processor -- start 2023-10-06T03:57:32.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.618+0000 7f3c75935700 1 -- start start 2023-10-06T03:57:32.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.619+0000 7f3c75935700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c700b1e40 0x7f3c70157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:32.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.619+0000 7f3c75935700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3c7006c2d0 0x7f3c7006dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:32.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.619+0000 7f3c75935700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c7006d8d0 con 0x7f3c700b04d0 2023-10-06T03:57:32.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.619+0000 7f3c75935700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c700af8f0 con 0x7f3c7006c2d0 2023-10-06T03:57:32.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.619+0000 7f3c75935700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7015a690 con 0x7f3c700b1e40 2023-10-06T03:57:32.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.619+0000 7f3c6ffff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f3c700b04d0 0x7f3c701563b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:56672/0 (socket says 172.21.15.157:56672) 2023-10-06T03:57:32.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.619+0000 7f3c6ffff700 1 -- 172.21.15.157:0/3596605704 learned_addr learned my addr 172.21.15.157:0/3596605704 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:32.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.619+0000 7f3c75134700 1 --2- 172.21.15.157:0/3596605704 >> v2:172.21.15.175:3300/0 conn(0x7f3c7006c2d0 0x7f3c7006dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:32.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.619+0000 7f3c74933700 1 --2- 172.21.15.157:0/3596605704 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c700b1e40 0x7f3c70157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:32.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.619+0000 7f3c6f7fe700 1 -- 172.21.15.157:0/3596605704 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3795983263 0 0) 0x7f3c7006d8d0 con 0x7f3c700b04d0 2023-10-06T03:57:32.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.619+0000 7f3c6f7fe700 1 -- 172.21.15.157:0/3596605704 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c58003570 con 0x7f3c700b04d0 2023-10-06T03:57:32.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.620+0000 7f3c6f7fe700 1 -- 172.21.15.157:0/3596605704 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3093824366 0 0) 0x7f3c58003570 con 0x7f3c700b04d0 2023-10-06T03:57:32.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.620+0000 7f3c6f7fe700 1 -- 172.21.15.157:0/3596605704 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c700b1e40 msgr2=0x7f3c70157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:32.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.620+0000 7f3c6f7fe700 1 --2- 172.21.15.157:0/3596605704 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c700b1e40 0x7f3c70157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:32.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.620+0000 7f3c6f7fe700 1 -- 172.21.15.157:0/3596605704 >> v2:172.21.15.175:3300/0 conn(0x7f3c7006c2d0 msgr2=0x7f3c7006dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:32.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.620+0000 7f3c6f7fe700 1 --2- 172.21.15.157:0/3596605704 >> v2:172.21.15.175:3300/0 conn(0x7f3c7006c2d0 0x7f3c7006dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:32.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.620+0000 7f3c6f7fe700 1 -- 172.21.15.157:0/3596605704 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7006ccc0 con 0x7f3c700b04d0 2023-10-06T03:57:32.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.620+0000 7f3c6f7fe700 1 -- 172.21.15.157:0/3596605704 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f3c64001470 con 0x7f3c700b04d0 2023-10-06T03:57:32.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.620+0000 7f3c6f7fe700 1 -- 172.21.15.157:0/3596605704 >> v1:172.21.15.157:6789/0 conn(0x7f3c700b04d0 legacy=0x7f3c701563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:32.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.620+0000 7f3c6f7fe700 1 --2- 172.21.15.157:0/3596605704 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c700b1e40 0x7f3c58003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:32.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.620+0000 7f3c6f7fe700 1 --2- 172.21.15.157:0/3596605704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c58003fe0 0x7f3c58004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:32.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.620+0000 7f3c6f7fe700 1 --2- 172.21.15.157:0/3596605704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c58004d50 0x7f3c580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:32.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.620+0000 7f3c6f7fe700 1 -- 172.21.15.157:0/3596605704 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c700af8f0 con 0x7f3c58003fe0 2023-10-06T03:57:32.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.620+0000 7f3c6f7fe700 1 -- 172.21.15.157:0/3596605704 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7015a690 con 0x7f3c58004d50 2023-10-06T03:57:32.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.620+0000 7f3c6f7fe700 1 -- 172.21.15.157:0/3596605704 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5800bb40 con 0x7f3c700b1e40 2023-10-06T03:57:32.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.620+0000 7f3c6ffff700 1 --2- 172.21.15.157:0/3596605704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c58003fe0 0x7f3c58004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:32.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.620+0000 7f3c74933700 1 --2- 172.21.15.157:0/3596605704 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c700b1e40 0x7f3c58003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:32.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.620+0000 7f3c75134700 1 --2- 172.21.15.157:0/3596605704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c58004d50 0x7f3c580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:32.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.620+0000 7f3c6ffff700 1 -- 172.21.15.157:0/3596605704 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c700b1e40 msgr2=0x7f3c58003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:32.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.620+0000 7f3c6ffff700 1 --2- 172.21.15.157:0/3596605704 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c700b1e40 0x7f3c58003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:32.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.620+0000 7f3c6ffff700 1 -- 172.21.15.157:0/3596605704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c58004d50 msgr2=0x7f3c580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:32.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.620+0000 7f3c6ffff700 1 --2- 172.21.15.157:0/3596605704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c58004d50 0x7f3c580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:32.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.620+0000 7f3c6ffff700 1 -- 172.21.15.157:0/3596605704 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c58003570 con 0x7f3c58003fe0 2023-10-06T03:57:32.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.620+0000 7f3c6ffff700 1 --2- 172.21.15.157:0/3596605704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c58003fe0 0x7f3c58004810 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400c390 tx=0x7f3c64001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5246bf53ff413e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:32.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.621+0000 7f3c6f7fe700 1 -- 172.21.15.157:0/3596605704 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6400cc60 con 0x7f3c58003fe0 2023-10-06T03:57:32.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.621+0000 7f3c6f7fe700 1 -- 172.21.15.157:0/3596605704 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6400cde0 con 0x7f3c58003fe0 2023-10-06T03:57:32.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.621+0000 7f3c6f7fe700 1 -- 172.21.15.157:0/3596605704 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c64005b70 con 0x7f3c58003fe0 2023-10-06T03:57:32.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.621+0000 7f3c75935700 1 -- 172.21.15.157:0/3596605704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c58003fe0 msgr2=0x7f3c58004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:32.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.621+0000 7f3c75935700 1 --2- 172.21.15.157:0/3596605704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c58003fe0 0x7f3c58004810 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400c390 tx=0x7f3c64001e90 comp rx=0 tx=0).stop 2023-10-06T03:57:32.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.621+0000 7f3c75134700 1 -- 172.21.15.157:0/3596605704 reap_dead start 2023-10-06T03:57:32.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.621+0000 7f3c75935700 1 -- 172.21.15.157:0/3596605704 shutdown_connections 2023-10-06T03:57:32.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.621+0000 7f3c75935700 1 -- 172.21.15.157:0/3596605704 >> 172.21.15.157:0/3596605704 conn(0x7f3c700cc540 msgr2=0x7f3c700cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:32.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.621+0000 7f3c75935700 1 -- 172.21.15.157:0/3596605704 shutdown_connections 2023-10-06T03:57:32.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.621+0000 7f3c75935700 1 -- 172.21.15.157:0/3596605704 wait complete. 2023-10-06T03:57:32.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.622+0000 7f3c75935700 1 Processor -- start 2023-10-06T03:57:32.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.622+0000 7f3c75935700 1 -- start start 2023-10-06T03:57:32.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.623+0000 7f3c75935700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c70165d50 0x7f3c70166120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:32.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.623+0000 7f3c75935700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c7006dd90 0x7f3c70150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:32.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.623+0000 7f3c75935700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c70150d80 0x7f3c7014d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:32.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.623+0000 7f3c75935700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7014f000 con 0x7f3c70165d50 2023-10-06T03:57:32.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.623+0000 7f3c75935700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7014f2e0 con 0x7f3c7006dd90 2023-10-06T03:57:32.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.623+0000 7f3c75935700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c700b23a0 con 0x7f3c70150d80 2023-10-06T03:57:32.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.623+0000 7f3c74933700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c70165d50 0x7f3c70166120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:32.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.623+0000 7f3c74933700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c70165d50 0x7f3c70166120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58716/0 (socket says 172.21.15.157:58716) 2023-10-06T03:57:32.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.623+0000 7f3c74933700 1 -- 172.21.15.157:0/3628665725 learned_addr learned my addr 172.21.15.157:0/3628665725 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:32.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.623+0000 7f3c6ffff700 1 --2- 172.21.15.157:0/3628665725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c7006dd90 0x7f3c70150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:32.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.623+0000 7f3c75134700 1 --2- 172.21.15.157:0/3628665725 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c70150d80 0x7f3c7014d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:32.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.623+0000 7f3c74933700 1 -- 172.21.15.157:0/3628665725 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c70150d80 msgr2=0x7f3c7014d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:32.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.623+0000 7f3c74933700 1 --2- 172.21.15.157:0/3628665725 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c70150d80 0x7f3c7014d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:32.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.623+0000 7f3c74933700 1 -- 172.21.15.157:0/3628665725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c7006dd90 msgr2=0x7f3c70150840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:32.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.623+0000 7f3c74933700 1 --2- 172.21.15.157:0/3628665725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c7006dd90 0x7f3c70150840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:32.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.623+0000 7f3c74933700 1 -- 172.21.15.157:0/3628665725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7008a1c0 con 0x7f3c70165d50 2023-10-06T03:57:32.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.624+0000 7f3c74933700 1 --2- 172.21.15.157:0/3628665725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c70165d50 0x7f3c70166120 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3c60011d70 tx=0x7f3c600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:32.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.624+0000 7f3c6cff9700 1 -- 172.21.15.157:0/3628665725 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6000f990 con 0x7f3c70165d50 2023-10-06T03:57:32.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.624+0000 7f3c6cff9700 1 -- 172.21.15.157:0/3628665725 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6000faf0 con 0x7f3c70165d50 2023-10-06T03:57:32.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.624+0000 7f3c75935700 1 -- 172.21.15.157:0/3628665725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c70166660 con 0x7f3c70165d50 2023-10-06T03:57:32.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.624+0000 7f3c6cff9700 1 -- 172.21.15.157:0/3628665725 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6000feb0 con 0x7f3c70165d50 2023-10-06T03:57:32.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.624+0000 7f3c75935700 1 -- 172.21.15.157:0/3628665725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c7015a410 con 0x7f3c70165d50 2023-10-06T03:57:32.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.625+0000 7f3c75935700 1 -- 172.21.15.157:0/3628665725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c70088f90 con 0x7f3c70165d50 2023-10-06T03:57:32.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.625+0000 7f3c6cff9700 1 -- 172.21.15.157:0/3628665725 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3c60014650 con 0x7f3c70165d50 2023-10-06T03:57:32.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.626+0000 7f3c6cff9700 1 --2- 172.21.15.157:0/3628665725 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c5c066f60 0x7f3c5c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:32.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.626+0000 7f3c6cff9700 1 -- 172.21.15.157:0/3628665725 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3c60018030 con 0x7f3c70165d50 2023-10-06T03:57:32.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.626+0000 7f3c6ffff700 1 --2- 172.21.15.157:0/3628665725 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c5c066f60 0x7f3c5c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:32.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.626+0000 7f3c6ffff700 1 --2- 172.21.15.157:0/3628665725 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c5c066f60 0x7f3c5c069410 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3c7004cc00 tx=0x7f3c64015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:32.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.630+0000 7f3c6cff9700 1 -- 172.21.15.157:0/3628665725 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c60052410 con 0x7f3c70165d50 2023-10-06T03:57:32.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.858+0000 7f3c75935700 1 -- 172.21.15.157:0/3628665725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3c7006cfa0 con 0x7f3c70165d50 2023-10-06T03:57:32.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.859+0000 7f3c6cff9700 1 -- 172.21.15.157:0/3628665725 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f3c60055a30 con 0x7f3c70165d50 2023-10-06T03:57:32.862 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:32.862 INFO:teuthology.orchestra.run.smithi157.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T03:57:32.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.861+0000 7f3c75935700 1 -- 172.21.15.157:0/3628665725 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c5c066f60 msgr2=0x7f3c5c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:32.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.861+0000 7f3c75935700 1 --2- 172.21.15.157:0/3628665725 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c5c066f60 0x7f3c5c069410 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3c7004cc00 tx=0x7f3c64015000 comp rx=0 tx=0).stop 2023-10-06T03:57:32.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.861+0000 7f3c75935700 1 -- 172.21.15.157:0/3628665725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c70165d50 msgr2=0x7f3c70166120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:32.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.861+0000 7f3c75935700 1 --2- 172.21.15.157:0/3628665725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c70165d50 0x7f3c70166120 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3c60011d70 tx=0x7f3c600091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:32.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.861+0000 7f3c75935700 1 -- 172.21.15.157:0/3628665725 shutdown_connections 2023-10-06T03:57:32.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.861+0000 7f3c75935700 1 --2- 172.21.15.157:0/3628665725 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c5c066f60 0x7f3c5c069410 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:32.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.861+0000 7f3c75935700 1 --2- 172.21.15.157:0/3628665725 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c70150d80 0x7f3c7014d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:32.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.861+0000 7f3c75935700 1 --2- 172.21.15.157:0/3628665725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c70165d50 0x7f3c70166120 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:32.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.861+0000 7f3c75935700 1 --2- 172.21.15.157:0/3628665725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c7006dd90 0x7f3c70150840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:32.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.861+0000 7f3c75935700 1 -- 172.21.15.157:0/3628665725 >> 172.21.15.157:0/3628665725 conn(0x7f3c700cc540 msgr2=0x7f3c70157440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:32.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.861+0000 7f3c75935700 1 -- 172.21.15.157:0/3628665725 shutdown_connections 2023-10-06T03:57:32.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.861+0000 7f3c75935700 1 -- 172.21.15.157:0/3628665725 wait complete. 2023-10-06T03:57:32.875 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-06T03:57:32.876 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:57:32.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.986+0000 7f8b80b1f700 1 Processor -- start 2023-10-06T03:57:32.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.986+0000 7f8b80b1f700 1 -- start start 2023-10-06T03:57:32.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.987+0000 7f8b80b1f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b7c06cbb0 0x7f8b7c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:32.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.987+0000 7f8b80b1f700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8b7c150410 0x7f8b7c06da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:32.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.987+0000 7f8b80b1f700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b7c0c9780 con 0x7f8b7c06c1f0 2023-10-06T03:57:32.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.987+0000 7f8b80b1f700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b7c0c8c40 con 0x7f8b7c150410 2023-10-06T03:57:32.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.987+0000 7f8b80b1f700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c06e0c0 con 0x7f8b7c06cbb0 2023-10-06T03:57:32.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.987+0000 7f8b7ad9d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8b7c150410 0x7f8b7c06da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:32.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.987+0000 7f8b7ad9d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8b7c150410 0x7f8b7c06da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58722/0 (socket says 172.21.15.157:58722) 2023-10-06T03:57:32.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.987+0000 7f8b7ad9d700 1 -- 172.21.15.157:0/1438785686 learned_addr learned my addr 172.21.15.157:0/1438785686 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:32.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.987+0000 7f8b7a59c700 1 --2- 172.21.15.157:0/1438785686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b7c06cbb0 0x7f8b7c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:32.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.988+0000 7f8b7ad9d700 1 -- 172.21.15.157:0/1438785686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b7c06cbb0 msgr2=0x7f8b7c06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:32.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.988+0000 7f8b7ad9d700 1 --2- 172.21.15.157:0/1438785686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b7c06cbb0 0x7f8b7c06d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:32.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.988+0000 7f8b7ad9d700 1 -- 172.21.15.157:0/1438785686 >> v1:172.21.15.175:6789/0 conn(0x7f8b7c06c1f0 legacy=0x7f8b7c14fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:32.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.988+0000 7f8b7ad9d700 1 -- 172.21.15.157:0/1438785686 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7c0c8e50 con 0x7f8b7c150410 2023-10-06T03:57:32.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.988+0000 7f8b7ad9d700 1 --2- 172.21.15.157:0/1438785686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b7c150410 0x7f8b7c06da70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8b740099b0 tx=0x7f8b74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6bf38fe8d76812 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:32.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.988+0000 7f8b7959a700 1 -- 172.21.15.157:0/1438785686 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b74009ea0 con 0x7f8b7c150410 2023-10-06T03:57:32.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.988+0000 7f8b7959a700 1 -- 172.21.15.157:0/1438785686 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7400b8f0 con 0x7f8b7c150410 2023-10-06T03:57:32.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.988+0000 7f8b7959a700 1 -- 172.21.15.157:0/1438785686 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7400bc90 con 0x7f8b7c150410 2023-10-06T03:57:32.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.988+0000 7f8b80b1f700 1 -- 172.21.15.157:0/1438785686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b7c150410 msgr2=0x7f8b7c06da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:32.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.988+0000 7f8b80b1f700 1 --2- 172.21.15.157:0/1438785686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b7c150410 0x7f8b7c06da70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8b740099b0 tx=0x7f8b74009250 comp rx=0 tx=0).stop 2023-10-06T03:57:32.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.988+0000 7f8b80b1f700 1 -- 172.21.15.157:0/1438785686 shutdown_connections 2023-10-06T03:57:32.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.988+0000 7f8b80b1f700 1 --2- 172.21.15.157:0/1438785686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b7c150410 0x7f8b7c06da70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:32.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.988+0000 7f8b80b1f700 1 --2- 172.21.15.157:0/1438785686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b7c06cbb0 0x7f8b7c06d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:32.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.988+0000 7f8b80b1f700 1 -- 172.21.15.157:0/1438785686 >> 172.21.15.157:0/1438785686 conn(0x7f8b7c06f230 msgr2=0x7f8b7c06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:32.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.989+0000 7f8b80b1f700 1 -- 172.21.15.157:0/1438785686 shutdown_connections 2023-10-06T03:57:32.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.989+0000 7f8b80b1f700 1 -- 172.21.15.157:0/1438785686 wait complete. 2023-10-06T03:57:32.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.990+0000 7f8b80b1f700 1 Processor -- start 2023-10-06T03:57:32.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.990+0000 7f8b80b1f700 1 -- start start 2023-10-06T03:57:32.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.990+0000 7f8b80b1f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b7c06c1f0 0x7f8b7c0c3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:32.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.990+0000 7f8b80b1f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b7c06cbb0 0x7f8b7c1489c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:32.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.990+0000 7f8b80b1f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b7c150410 0x7f8b7c148f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:32.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.990+0000 7f8b80b1f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c1541e0 con 0x7f8b7c150410 2023-10-06T03:57:32.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.990+0000 7f8b80b1f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c1536e0 con 0x7f8b7c06c1f0 2023-10-06T03:57:32.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.990+0000 7f8b80b1f700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c155980 con 0x7f8b7c06cbb0 2023-10-06T03:57:32.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.990+0000 7f8b7ad9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b7c150410 0x7f8b7c148f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:32.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.990+0000 7f8b7ad9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b7c150410 0x7f8b7c148f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58730/0 (socket says 172.21.15.157:58730) 2023-10-06T03:57:32.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.990+0000 7f8b7ad9d700 1 -- 172.21.15.157:0/2804613603 learned_addr learned my addr 172.21.15.157:0/2804613603 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:32.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.991+0000 7f8b7a59c700 1 --2- 172.21.15.157:0/2804613603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b7c06c1f0 0x7f8b7c0c3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:32.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.991+0000 7f8b79d9b700 1 --2- 172.21.15.157:0/2804613603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b7c06cbb0 0x7f8b7c1489c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:32.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.991+0000 7f8b7ad9d700 1 -- 172.21.15.157:0/2804613603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b7c06cbb0 msgr2=0x7f8b7c1489c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:32.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.991+0000 7f8b7ad9d700 1 --2- 172.21.15.157:0/2804613603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b7c06cbb0 0x7f8b7c1489c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:32.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.991+0000 7f8b7ad9d700 1 -- 172.21.15.157:0/2804613603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b7c06c1f0 msgr2=0x7f8b7c0c3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:32.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.991+0000 7f8b7ad9d700 1 --2- 172.21.15.157:0/2804613603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b7c06c1f0 0x7f8b7c0c3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:32.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.991+0000 7f8b7ad9d700 1 -- 172.21.15.157:0/2804613603 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b740094f0 con 0x7f8b7c150410 2023-10-06T03:57:32.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.991+0000 7f8b79d9b700 1 --2- 172.21.15.157:0/2804613603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b7c06cbb0 0x7f8b7c1489c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:32.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.991+0000 7f8b7ad9d700 1 --2- 172.21.15.157:0/2804613603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b7c150410 0x7f8b7c148f00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c0c3960 tx=0x7f8b74017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:32.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.991+0000 7f8b727fc700 1 -- 172.21.15.157:0/2804613603 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7400bc30 con 0x7f8b7c150410 2023-10-06T03:57:32.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.991+0000 7f8b727fc700 1 -- 172.21.15.157:0/2804613603 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b74011630 con 0x7f8b7c150410 2023-10-06T03:57:32.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.991+0000 7f8b727fc700 1 -- 172.21.15.157:0/2804613603 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b740119f0 con 0x7f8b7c150410 2023-10-06T03:57:32.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.991+0000 7f8b80b1f700 1 -- 172.21.15.157:0/2804613603 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b7c165be0 con 0x7f8b7c150410 2023-10-06T03:57:32.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.991+0000 7f8b80b1f700 1 -- 172.21.15.157:0/2804613603 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b7c154d70 con 0x7f8b7c150410 2023-10-06T03:57:32.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.992+0000 7f8b80b1f700 1 -- 172.21.15.157:0/2804613603 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b7c089ca0 con 0x7f8b7c150410 2023-10-06T03:57:32.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.992+0000 7f8b727fc700 1 -- 172.21.15.157:0/2804613603 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8b740253f0 con 0x7f8b7c150410 2023-10-06T03:57:32.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.993+0000 7f8b727fc700 1 --2- 172.21.15.157:0/2804613603 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8b5c066f60 0x7f8b5c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:32.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.993+0000 7f8b727fc700 1 -- 172.21.15.157:0/2804613603 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8b74086050 con 0x7f8b7c150410 2023-10-06T03:57:32.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.993+0000 7f8b7a59c700 1 --2- 172.21.15.157:0/2804613603 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8b5c066f60 0x7f8b5c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:32.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.993+0000 7f8b7a59c700 1 --2- 172.21.15.157:0/2804613603 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8b5c066f60 0x7f8b5c069410 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c00f570 tx=0x7f8b6c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:32.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:32.997+0000 7f8b727fc700 1 -- 172.21.15.157:0/2804613603 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b74053160 con 0x7f8b7c150410 2023-10-06T03:57:33.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.182+0000 7f8b80b1f700 1 -- 172.21.15.157:0/2804613603 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b7c155ac0 con 0x7f8b7c150410 2023-10-06T03:57:33.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.183+0000 7f8b727fc700 1 -- 172.21.15.157:0/2804613603 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f8b74006d70 con 0x7f8b7c150410 2023-10-06T03:57:33.185 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:33.186 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":21,"fsid":"28664a7a-fe30-4efe-be5a-6abf9318b700","created":"2023-10-06T03:56:51.887538+0000","modified":"2023-10-06T03:57:12.362537+0000","last_up_change":"2023-10-06T03:57:02.299440+0000","last_in_change":"2023-10-06T03:56:55.141505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:57:03.455121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:57:03.905661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:57:08.377186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:57:09.709582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f677cac-e9f4-4608-acac-718b072ce984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":90640},{"type":"v1","addr":"172.21.15.157:6809","nonce":90640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":90640},{"type":"v1","addr":"172.21.15.157:6811","nonce":90640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":90640},{"type":"v1","addr":"172.21.15.157:6815","nonce":90640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":90640},{"type":"v1","addr":"172.21.15.157:6813","nonce":90640}]},"public_addr":"172.21.15.157:6809/90640","cluster_addr":"172.21.15.157:6811/90640","heartbeat_back_addr":"172.21.15.157:6815/90640","heartbeat_front_addr":"172.21.15.157:6813/90640","state":["exists","up"]},{"osd":1,"uuid":"7d771892-bb91-4e9b-9610-051861c11247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":90654},{"type":"v1","addr":"172.21.15.157:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":90654},{"type":"v1","addr":"172.21.15.157:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":90654},{"type":"v1","addr":"172.21.15.157:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":90654},{"type":"v1","addr":"172.21.15.157:6805","nonce":90654}]},"public_addr":"172.21.15.157:6801/90654","cluster_addr":"172.21.15.157:6803/90654","heartbeat_back_addr":"172.21.15.157:6807/90654","heartbeat_front_addr":"172.21.15.157:6805/90654","state":["exists","up"]},{"osd":2,"uuid":"5c1f7d4e-d185-47cf-ab6c-453b4e3009c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":90657},{"type":"v1","addr":"172.21.15.157:6825","nonce":90657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":90657},{"type":"v1","addr":"172.21.15.157:6827","nonce":90657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":90657},{"type":"v1","addr":"172.21.15.157:6831","nonce":90657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":90657},{"type":"v1","addr":"172.21.15.157:6829","nonce":90657}]},"public_addr":"172.21.15.157:6825/90657","cluster_addr":"172.21.15.157:6827/90657","heartbeat_back_addr":"172.21.15.157:6831/90657","heartbeat_front_addr":"172.21.15.157:6829/90657","state":["exists","up"]},{"osd":3,"uuid":"f6a1e825-43c7-4afd-8daf-e51d45afc75e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":90641},{"type":"v1","addr":"172.21.15.157:6817","nonce":90641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":90641},{"type":"v1","addr":"172.21.15.157:6819","nonce":90641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":90641},{"type":"v1","addr":"172.21.15.157:6823","nonce":90641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":90641},{"type":"v1","addr":"172.21.15.157:6821","nonce":90641}]},"public_addr":"172.21.15.157:6817/90641","cluster_addr":"172.21.15.157:6819/90641","heartbeat_back_addr":"172.21.15.157:6823/90641","heartbeat_front_addr":"172.21.15.157:6821/90641","state":["exists","up"]},{"osd":4,"uuid":"8ad6e3d8-9a31-4f85-9410-417b7ac5da97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":91261},{"type":"v1","addr":"172.21.15.175:6803","nonce":91261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":91261},{"type":"v1","addr":"172.21.15.175:6806","nonce":91261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":91261},{"type":"v1","addr":"172.21.15.175:6814","nonce":91261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":91261},{"type":"v1","addr":"172.21.15.175:6810","nonce":91261}]},"public_addr":"172.21.15.175:6803/91261","cluster_addr":"172.21.15.175:6806/91261","heartbeat_back_addr":"172.21.15.175:6814/91261","heartbeat_front_addr":"172.21.15.175:6810/91261","state":["exists","up"]},{"osd":5,"uuid":"f68aca2a-f182-4c00-8e21-241d09c02ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":91262},{"type":"v1","addr":"172.21.15.175:6817","nonce":91262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":91262},{"type":"v1","addr":"172.21.15.175:6819","nonce":91262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":91262},{"type":"v1","addr":"172.21.15.175:6823","nonce":91262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":91262},{"type":"v1","addr":"172.21.15.175:6821","nonce":91262}]},"public_addr":"172.21.15.175:6817/91262","cluster_addr":"172.21.15.175:6819/91262","heartbeat_back_addr":"172.21.15.175:6823/91262","heartbeat_front_addr":"172.21.15.175:6821/91262","state":["exists","up"]},{"osd":6,"uuid":"d867cd89-3430-40ca-862f-d2bea3de9d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":91263},{"type":"v1","addr":"172.21.15.175:6801","nonce":91263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6805","nonce":91263},{"type":"v1","addr":"172.21.15.175:6807","nonce":91263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6813","nonce":91263},{"type":"v1","addr":"172.21.15.175:6815","nonce":91263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":91263},{"type":"v1","addr":"172.21.15.175:6811","nonce":91263}]},"public_addr":"172.21.15.175:6801/91263","cluster_addr":"172.21.15.175:6807/91263","heartbeat_back_addr":"172.21.15.175:6815/91263","heartbeat_front_addr":"172.21.15.175:6811/91263","state":["exists","up"]},{"osd":7,"uuid":"930cd464-79f1-47cb-b4ff-188ccde74fff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":91306},{"type":"v1","addr":"172.21.15.175:6825","nonce":91306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":91306},{"type":"v1","addr":"172.21.15.175:6827","nonce":91306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":91306},{"type":"v1","addr":"172.21.15.175:6831","nonce":91306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":91306},{"type":"v1","addr":"172.21.15.175:6829","nonce":91306}]},"public_addr":"172.21.15.175:6825/91306","cluster_addr":"172.21.15.175:6827/91306","heartbeat_back_addr":"172.21.15.175:6831/91306","heartbeat_front_addr":"172.21.15.175:6829/91306","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.290673+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.056390+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.292903+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.275861+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.332095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.327453+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.317585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.309330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:57:33.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.185+0000 7f8b80b1f700 1 -- 172.21.15.157:0/2804613603 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8b5c066f60 msgr2=0x7f8b5c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:33.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.185+0000 7f8b80b1f700 1 --2- 172.21.15.157:0/2804613603 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8b5c066f60 0x7f8b5c069410 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c00f570 tx=0x7f8b6c00b040 comp rx=0 tx=0).stop 2023-10-06T03:57:33.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.185+0000 7f8b80b1f700 1 -- 172.21.15.157:0/2804613603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b7c150410 msgr2=0x7f8b7c148f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:33.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.185+0000 7f8b80b1f700 1 --2- 172.21.15.157:0/2804613603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b7c150410 0x7f8b7c148f00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c0c3960 tx=0x7f8b74017000 comp rx=0 tx=0).stop 2023-10-06T03:57:33.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.185+0000 7f8b80b1f700 1 -- 172.21.15.157:0/2804613603 shutdown_connections 2023-10-06T03:57:33.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.185+0000 7f8b80b1f700 1 --2- 172.21.15.157:0/2804613603 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8b5c066f60 0x7f8b5c069410 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:33.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.185+0000 7f8b80b1f700 1 --2- 172.21.15.157:0/2804613603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b7c06c1f0 0x7f8b7c0c3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:33.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.185+0000 7f8b80b1f700 1 --2- 172.21.15.157:0/2804613603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b7c150410 0x7f8b7c148f00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:33.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.185+0000 7f8b80b1f700 1 --2- 172.21.15.157:0/2804613603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b7c06cbb0 0x7f8b7c1489c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:33.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.185+0000 7f8b80b1f700 1 -- 172.21.15.157:0/2804613603 >> 172.21.15.157:0/2804613603 conn(0x7f8b7c06f230 msgr2=0x7f8b7c150850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:33.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.185+0000 7f8b80b1f700 1 -- 172.21.15.157:0/2804613603 shutdown_connections 2023-10-06T03:57:33.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.185+0000 7f8b80b1f700 1 -- 172.21.15.157:0/2804613603 wait complete. 2023-10-06T03:57:33.200 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:57:03.455121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:57:03.905661+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:57:08.377186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:57:09.709582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:57:33.200 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:57:33.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.315+0000 7feb91ec5700 1 Processor -- start 2023-10-06T03:57:33.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.315+0000 7feb91ec5700 1 -- start start 2023-10-06T03:57:33.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.316+0000 7feb91ec5700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7feb8c06dc40 0x7feb8c14fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:33.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.316+0000 7feb91ec5700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feb8c06c1f0 0x7feb8c150240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:33.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.316+0000 7feb91ec5700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb8c0c9780 con 0x7feb8c06cbb0 2023-10-06T03:57:33.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.316+0000 7feb91ec5700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb8c0c8c40 con 0x7feb8c06c1f0 2023-10-06T03:57:33.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.316+0000 7feb91ec5700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb8c06e0c0 con 0x7feb8c06dc40 2023-10-06T03:57:33.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.316+0000 7feb8bfff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feb8c06c1f0 0x7feb8c150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:33.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.316+0000 7feb8bfff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feb8c06c1f0 0x7feb8c150240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35510/0 (socket says 172.21.15.157:35510) 2023-10-06T03:57:33.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.316+0000 7feb8bfff700 1 -- 172.21.15.157:0/2935239361 learned_addr learned my addr 172.21.15.157:0/2935239361 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:33.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.316+0000 7feb8affd700 1 --2- 172.21.15.157:0/2935239361 >> v2:172.21.15.175:3300/0 conn(0x7feb8c06dc40 0x7feb8c14fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:33.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.316+0000 7feb8bfff700 1 -- 172.21.15.157:0/2935239361 >> v2:172.21.15.175:3300/0 conn(0x7feb8c06dc40 msgr2=0x7feb8c14fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:33.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.316+0000 7feb8bfff700 1 --2- 172.21.15.157:0/2935239361 >> v2:172.21.15.175:3300/0 conn(0x7feb8c06dc40 0x7feb8c14fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:33.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.316+0000 7feb8bfff700 1 -- 172.21.15.157:0/2935239361 >> v1:172.21.15.175:6789/0 conn(0x7feb8c06cbb0 legacy=0x7feb8c06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:33.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.316+0000 7feb8bfff700 1 -- 172.21.15.157:0/2935239361 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb8c0c8e50 con 0x7feb8c06c1f0 2023-10-06T03:57:33.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.316+0000 7feb8affd700 1 --2- 172.21.15.157:0/2935239361 >> v2:172.21.15.175:3300/0 conn(0x7feb8c06dc40 0x7feb8c14fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:33.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.316+0000 7feb8bfff700 1 --2- 172.21.15.157:0/2935239361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb8c06c1f0 0x7feb8c150240 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7feb8c157070 tx=0x7feb84009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a15956f55b70634 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:33.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.317+0000 7feb8a7fc700 1 -- 172.21.15.157:0/2935239361 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8400b8f0 con 0x7feb8c06c1f0 2023-10-06T03:57:33.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.317+0000 7feb8a7fc700 1 -- 172.21.15.157:0/2935239361 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb8400ba50 con 0x7feb8c06c1f0 2023-10-06T03:57:33.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.317+0000 7feb8a7fc700 1 -- 172.21.15.157:0/2935239361 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8400bdf0 con 0x7feb8c06c1f0 2023-10-06T03:57:33.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.317+0000 7feb91ec5700 1 -- 172.21.15.157:0/2935239361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb8c06c1f0 msgr2=0x7feb8c150240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:33.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.317+0000 7feb91ec5700 1 --2- 172.21.15.157:0/2935239361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb8c06c1f0 0x7feb8c150240 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7feb8c157070 tx=0x7feb84009250 comp rx=0 tx=0).stop 2023-10-06T03:57:33.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.317+0000 7feb91ec5700 1 -- 172.21.15.157:0/2935239361 shutdown_connections 2023-10-06T03:57:33.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.317+0000 7feb91ec5700 1 --2- 172.21.15.157:0/2935239361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb8c06c1f0 0x7feb8c150240 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:33.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.317+0000 7feb91ec5700 1 --2- 172.21.15.157:0/2935239361 >> v2:172.21.15.175:3300/0 conn(0x7feb8c06dc40 0x7feb8c14fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:33.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.317+0000 7feb91ec5700 1 -- 172.21.15.157:0/2935239361 >> 172.21.15.157:0/2935239361 conn(0x7feb8c06f230 msgr2=0x7feb8c06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:33.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.317+0000 7feb91ec5700 1 -- 172.21.15.157:0/2935239361 shutdown_connections 2023-10-06T03:57:33.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.317+0000 7feb91ec5700 1 -- 172.21.15.157:0/2935239361 wait complete. 2023-10-06T03:57:33.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.318+0000 7feb91ec5700 1 Processor -- start 2023-10-06T03:57:33.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.318+0000 7feb91ec5700 1 -- start start 2023-10-06T03:57:33.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.319+0000 7feb91ec5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb8c06c1f0 0x7feb8c167300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:33.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.319+0000 7feb91ec5700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb8c06cbb0 0x7feb8c14a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:33.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.319+0000 7feb91ec5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb8c144330 0x7feb8c147c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:33.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.319+0000 7feb91ec5700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c14f370 con 0x7feb8c144330 2023-10-06T03:57:33.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.319+0000 7feb91ec5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c143610 con 0x7feb8c06c1f0 2023-10-06T03:57:33.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.319+0000 7feb91ec5700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c144830 con 0x7feb8c06cbb0 2023-10-06T03:57:33.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.319+0000 7feb8bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb8c144330 0x7feb8c147c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:33.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.319+0000 7feb8bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb8c144330 0x7feb8c147c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35522/0 (socket says 172.21.15.157:35522) 2023-10-06T03:57:33.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.319+0000 7feb8bfff700 1 -- 172.21.15.157:0/1943750080 learned_addr learned my addr 172.21.15.157:0/1943750080 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:33.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.319+0000 7feb8affd700 1 --2- 172.21.15.157:0/1943750080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb8c06cbb0 0x7feb8c14a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:33.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.319+0000 7feb8b7fe700 1 --2- 172.21.15.157:0/1943750080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb8c06c1f0 0x7feb8c167300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:33.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.319+0000 7feb8bfff700 1 -- 172.21.15.157:0/1943750080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb8c06cbb0 msgr2=0x7feb8c14a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:33.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.319+0000 7feb8bfff700 1 --2- 172.21.15.157:0/1943750080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb8c06cbb0 0x7feb8c14a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:33.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.319+0000 7feb8bfff700 1 -- 172.21.15.157:0/1943750080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb8c06c1f0 msgr2=0x7feb8c167300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:33.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.319+0000 7feb8bfff700 1 --2- 172.21.15.157:0/1943750080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb8c06c1f0 0x7feb8c167300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:33.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.319+0000 7feb8bfff700 1 -- 172.21.15.157:0/1943750080 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb840094f0 con 0x7feb8c144330 2023-10-06T03:57:33.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.319+0000 7feb8b7fe700 1 --2- 172.21.15.157:0/1943750080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb8c06c1f0 0x7feb8c167300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:33.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.319+0000 7feb8affd700 1 --2- 172.21.15.157:0/1943750080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb8c06cbb0 0x7feb8c14a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:33.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.320+0000 7feb8bfff700 1 --2- 172.21.15.157:0/1943750080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb8c144330 0x7feb8c147c00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feb84006f50 tx=0x7feb84017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:33.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.320+0000 7feb737fe700 1 -- 172.21.15.157:0/1943750080 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8401a710 con 0x7feb8c144330 2023-10-06T03:57:33.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.320+0000 7feb737fe700 1 -- 172.21.15.157:0/1943750080 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb8401f070 con 0x7feb8c144330 2023-10-06T03:57:33.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.320+0000 7feb737fe700 1 -- 172.21.15.157:0/1943750080 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8401aa80 con 0x7feb8c144330 2023-10-06T03:57:33.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.320+0000 7feb91ec5700 1 -- 172.21.15.157:0/1943750080 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb8c142a00 con 0x7feb8c144330 2023-10-06T03:57:33.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.320+0000 7feb91ec5700 1 -- 172.21.15.157:0/1943750080 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb8c142fc0 con 0x7feb8c144330 2023-10-06T03:57:33.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.321+0000 7feb91ec5700 1 -- 172.21.15.157:0/1943750080 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb8c06d1f0 con 0x7feb8c144330 2023-10-06T03:57:33.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.321+0000 7feb737fe700 1 -- 172.21.15.157:0/1943750080 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feb8401b430 con 0x7feb8c144330 2023-10-06T03:57:33.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.321+0000 7feb737fe700 1 --2- 172.21.15.157:0/1943750080 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7feb78066f30 0x7feb780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:33.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.321+0000 7feb737fe700 1 -- 172.21.15.157:0/1943750080 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7feb84021070 con 0x7feb8c144330 2023-10-06T03:57:33.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.322+0000 7feb8b7fe700 1 --2- 172.21.15.157:0/1943750080 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7feb78066f30 0x7feb780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:33.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.322+0000 7feb8b7fe700 1 --2- 172.21.15.157:0/1943750080 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7feb78066f30 0x7feb780693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7feb7c007fd0 tx=0x7feb7c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:33.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.326+0000 7feb737fe700 1 -- 172.21.15.157:0/1943750080 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feb84053430 con 0x7feb8c144330 2023-10-06T03:57:33.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.509+0000 7feb91ec5700 1 -- 172.21.15.157:0/1943750080 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feb8c143780 con 0x7feb8c144330 2023-10-06T03:57:33.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.509+0000 7feb737fe700 1 -- 172.21.15.157:0/1943750080 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7feb84056a50 con 0x7feb8c144330 2023-10-06T03:57:33.512 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-10-06T03:57:33.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.511+0000 7feb91ec5700 1 -- 172.21.15.157:0/1943750080 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7feb78066f30 msgr2=0x7feb780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:33.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.511+0000 7feb91ec5700 1 --2- 172.21.15.157:0/1943750080 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7feb78066f30 0x7feb780693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7feb7c007fd0 tx=0x7feb7c009040 comp rx=0 tx=0).stop 2023-10-06T03:57:33.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.511+0000 7feb91ec5700 1 -- 172.21.15.157:0/1943750080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb8c144330 msgr2=0x7feb8c147c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:33.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.511+0000 7feb91ec5700 1 --2- 172.21.15.157:0/1943750080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb8c144330 0x7feb8c147c00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feb84006f50 tx=0x7feb84017000 comp rx=0 tx=0).stop 2023-10-06T03:57:33.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.512+0000 7feb91ec5700 1 -- 172.21.15.157:0/1943750080 shutdown_connections 2023-10-06T03:57:33.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.512+0000 7feb91ec5700 1 --2- 172.21.15.157:0/1943750080 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7feb78066f30 0x7feb780693e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:33.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.512+0000 7feb91ec5700 1 --2- 172.21.15.157:0/1943750080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb8c06c1f0 0x7feb8c167300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:33.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.512+0000 7feb91ec5700 1 --2- 172.21.15.157:0/1943750080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb8c144330 0x7feb8c147c00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:33.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.512+0000 7feb91ec5700 1 --2- 172.21.15.157:0/1943750080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb8c06cbb0 0x7feb8c14a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:33.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.512+0000 7feb91ec5700 1 -- 172.21.15.157:0/1943750080 >> 172.21.15.157:0/1943750080 conn(0x7feb8c06f230 msgr2=0x7feb8c0b1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:33.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.512+0000 7feb91ec5700 1 -- 172.21.15.157:0/1943750080 shutdown_connections 2023-10-06T03:57:33.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.512+0000 7feb91ec5700 1 -- 172.21.15.157:0/1943750080 wait complete. 2023-10-06T03:57:33.526 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:57:33.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.642+0000 7fd49ffff700 1 Processor -- start 2023-10-06T03:57:33.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.642+0000 7fd49ffff700 1 -- start start 2023-10-06T03:57:33.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.642+0000 7fd49ffff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd4a00c8730 0x7fd4a00b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:33.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.642+0000 7fd49ffff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4a00afc20 con 0x7fd4a00c8b00 2023-10-06T03:57:33.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.643+0000 7fd49ffff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4a00cc300 con 0x7fd4a00c80b0 2023-10-06T03:57:33.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.643+0000 7fd49ffff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4a00af030 con 0x7fd4a00c8730 2023-10-06T03:57:33.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.643+0000 7fd49effd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd4a00c8730 0x7fd4a00b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:33.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.643+0000 7fd49f7fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd4a00c8b00 0x7fd4a0156750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39624/0 (socket says 172.21.15.157:39624) 2023-10-06T03:57:33.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.643+0000 7fd49effd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd4a00c8730 0x7fd4a00b1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35524/0 (socket says 172.21.15.157:35524) 2023-10-06T03:57:33.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.643+0000 7fd49f7fe700 1 -- 172.21.15.157:0/4290155194 learned_addr learned my addr 172.21.15.157:0/4290155194 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:33.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.643+0000 7fd49effd700 1 -- 172.21.15.157:0/4290155194 >> v1:172.21.15.175:6789/0 conn(0x7fd4a00c80b0 legacy=0x7fd4a00b00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:33.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.643+0000 7fd49effd700 1 -- 172.21.15.157:0/4290155194 >> v1:172.21.15.157:6789/0 conn(0x7fd4a00c8b00 legacy=0x7fd4a0156750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:33.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.643+0000 7fd49effd700 1 -- 172.21.15.157:0/4290155194 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a00b2320 con 0x7fd4a00c8730 2023-10-06T03:57:33.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.643+0000 7fd49effd700 1 --2- 172.21.15.157:0/4290155194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4a00c8730 0x7fd4a00b1ce0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd4940098e0 tx=0x7fd4940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b549b4431ea5395 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:33.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.643+0000 7fd49dffb700 1 -- 172.21.15.157:0/4290155194 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd494009e80 con 0x7fd4a00c8730 2023-10-06T03:57:33.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.643+0000 7fd49dffb700 1 -- 172.21.15.157:0/4290155194 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49400b860 con 0x7fd4a00c8730 2023-10-06T03:57:33.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.643+0000 7fd49dffb700 1 -- 172.21.15.157:0/4290155194 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49400bbe0 con 0x7fd4a00c8730 2023-10-06T03:57:33.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.644+0000 7fd49ffff700 1 -- 172.21.15.157:0/4290155194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4a00c8730 msgr2=0x7fd4a00b1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:33.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.644+0000 7fd49ffff700 1 --2- 172.21.15.157:0/4290155194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4a00c8730 0x7fd4a00b1ce0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd4940098e0 tx=0x7fd4940091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:33.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.644+0000 7fd49ffff700 1 -- 172.21.15.157:0/4290155194 shutdown_connections 2023-10-06T03:57:33.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.644+0000 7fd49ffff700 1 --2- 172.21.15.157:0/4290155194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4a00c8730 0x7fd4a00b1ce0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:33.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.644+0000 7fd49ffff700 1 -- 172.21.15.157:0/4290155194 >> 172.21.15.157:0/4290155194 conn(0x7fd4a00cc510 msgr2=0x7fd4a00cc910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:33.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.644+0000 7fd49ffff700 1 -- 172.21.15.157:0/4290155194 shutdown_connections 2023-10-06T03:57:33.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.644+0000 7fd49ffff700 1 -- 172.21.15.157:0/4290155194 wait complete. 2023-10-06T03:57:33.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.645+0000 7fd49ffff700 1 Processor -- start 2023-10-06T03:57:33.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.645+0000 7fd49ffff700 1 -- start start 2023-10-06T03:57:33.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.646+0000 7fd49ffff700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4a00c8730 0x7fd4a016a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:33.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.646+0000 7fd49ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4a00c8b00 0x7fd4a01533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:33.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.646+0000 7fd49ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4a015ea80 0x7fd4a00c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:33.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.646+0000 7fd49ffff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a015a5b0 con 0x7fd4a00c8b00 2023-10-06T03:57:33.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.646+0000 7fd49ffff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a015ef20 con 0x7fd4a015ea80 2023-10-06T03:57:33.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.646+0000 7fd49ffff700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a0153f20 con 0x7fd4a00c8730 2023-10-06T03:57:33.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.646+0000 7fd49e7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4a00c8b00 0x7fd4a01533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:33.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.646+0000 7fd49e7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4a00c8b00 0x7fd4a01533f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35538/0 (socket says 172.21.15.157:35538) 2023-10-06T03:57:33.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.646+0000 7fd49e7fc700 1 -- 172.21.15.157:0/2877258777 learned_addr learned my addr 172.21.15.157:0/2877258777 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:33.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.646+0000 7fd49effd700 1 --2- 172.21.15.157:0/2877258777 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4a00c8730 0x7fd4a016a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:33.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.646+0000 7fd49f7fe700 1 --2- 172.21.15.157:0/2877258777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4a015ea80 0x7fd4a00c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:33.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.646+0000 7fd49e7fc700 1 -- 172.21.15.157:0/2877258777 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4a00c8730 msgr2=0x7fd4a016a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:33.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.646+0000 7fd49e7fc700 1 --2- 172.21.15.157:0/2877258777 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4a00c8730 0x7fd4a016a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:33.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.646+0000 7fd49e7fc700 1 -- 172.21.15.157:0/2877258777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4a015ea80 msgr2=0x7fd4a00c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:33.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.646+0000 7fd49e7fc700 1 --2- 172.21.15.157:0/2877258777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4a015ea80 0x7fd4a00c4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:33.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.646+0000 7fd49e7fc700 1 -- 172.21.15.157:0/2877258777 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd494009420 con 0x7fd4a00c8b00 2023-10-06T03:57:33.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.646+0000 7fd49e7fc700 1 --2- 172.21.15.157:0/2877258777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4a00c8b00 0x7fd4a01533f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd49000c510 tx=0x7fd49000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:33.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.646+0000 7fd486ffd700 1 -- 172.21.15.157:0/2877258777 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd490011590 con 0x7fd4a00c8b00 2023-10-06T03:57:33.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.646+0000 7fd486ffd700 1 -- 172.21.15.157:0/2877258777 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd490011710 con 0x7fd4a00c8b00 2023-10-06T03:57:33.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.647+0000 7fd486ffd700 1 -- 172.21.15.157:0/2877258777 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd490011ad0 con 0x7fd4a00c8b00 2023-10-06T03:57:33.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.647+0000 7fd49ffff700 1 -- 172.21.15.157:0/2877258777 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4a0152f60 con 0x7fd4a00c8b00 2023-10-06T03:57:33.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.647+0000 7fd49ffff700 1 -- 172.21.15.157:0/2877258777 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a0154450 con 0x7fd4a00c8b00 2023-10-06T03:57:33.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.647+0000 7fd49ffff700 1 -- 172.21.15.157:0/2877258777 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4a00716d0 con 0x7fd4a00c8b00 2023-10-06T03:57:33.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.648+0000 7fd486ffd700 1 -- 172.21.15.157:0/2877258777 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd49001b3b0 con 0x7fd4a00c8b00 2023-10-06T03:57:33.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.648+0000 7fd486ffd700 1 --2- 172.21.15.157:0/2877258777 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd48c066f30 0x7fd48c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:33.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.648+0000 7fd486ffd700 1 -- 172.21.15.157:0/2877258777 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd490083b40 con 0x7fd4a00c8b00 2023-10-06T03:57:33.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.648+0000 7fd49effd700 1 --2- 172.21.15.157:0/2877258777 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd48c066f30 0x7fd48c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:33.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.649+0000 7fd49effd700 1 --2- 172.21.15.157:0/2877258777 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd48c066f30 0x7fd48c0693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd4940093f0 tx=0x7fd494017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:33.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.652+0000 7fd486ffd700 1 -- 172.21.15.157:0/2877258777 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd490050c50 con 0x7fd4a00c8b00 2023-10-06T03:57:33.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.833+0000 7fd49ffff700 1 -- 172.21.15.157:0/2877258777 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd4a00893a0 con 0x7fd4a00c8b00 2023-10-06T03:57:33.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.833+0000 7fd486ffd700 1 -- 172.21.15.157:0/2877258777 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd490017070 con 0x7fd4a00c8b00 2023-10-06T03:57:33.836 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-10-06T03:57:33.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.835+0000 7fd49ffff700 1 -- 172.21.15.157:0/2877258777 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd48c066f30 msgr2=0x7fd48c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:33.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.835+0000 7fd49ffff700 1 --2- 172.21.15.157:0/2877258777 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd48c066f30 0x7fd48c0693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd4940093f0 tx=0x7fd494017000 comp rx=0 tx=0).stop 2023-10-06T03:57:33.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.835+0000 7fd49ffff700 1 -- 172.21.15.157:0/2877258777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4a00c8b00 msgr2=0x7fd4a01533f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:33.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.835+0000 7fd49ffff700 1 --2- 172.21.15.157:0/2877258777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4a00c8b00 0x7fd4a01533f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd49000c510 tx=0x7fd49000c8d0 comp rx=0 tx=0).stop 2023-10-06T03:57:33.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.835+0000 7fd49ffff700 1 -- 172.21.15.157:0/2877258777 shutdown_connections 2023-10-06T03:57:33.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.835+0000 7fd49ffff700 1 --2- 172.21.15.157:0/2877258777 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd48c066f30 0x7fd48c0693e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:33.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.835+0000 7fd49ffff700 1 --2- 172.21.15.157:0/2877258777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4a015ea80 0x7fd4a00c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:33.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.835+0000 7fd49ffff700 1 --2- 172.21.15.157:0/2877258777 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4a00c8730 0x7fd4a016a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:33.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.835+0000 7fd49ffff700 1 --2- 172.21.15.157:0/2877258777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4a00c8b00 0x7fd4a01533f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:33.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.835+0000 7fd49ffff700 1 -- 172.21.15.157:0/2877258777 >> 172.21.15.157:0/2877258777 conn(0x7fd4a00cc510 msgr2=0x7fd4a0070940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:33.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.835+0000 7fd49ffff700 1 -- 172.21.15.157:0/2877258777 shutdown_connections 2023-10-06T03:57:33.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.835+0000 7fd49ffff700 1 -- 172.21.15.157:0/2877258777 wait complete. 2023-10-06T03:57:33.849 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:57:33.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.963+0000 7f834beec700 1 Processor -- start 2023-10-06T03:57:33.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.964+0000 7f834beec700 1 -- start start 2023-10-06T03:57:33.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.964+0000 7f834beec700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f834406dc40 0x7f834414fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:33.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.964+0000 7f834beec700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f834406c1f0 0x7f8344150240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:33.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.964+0000 7f834beec700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83440c9780 con 0x7f834406cbb0 2023-10-06T03:57:33.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.964+0000 7f834beec700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83440c8c40 con 0x7f834406dc40 2023-10-06T03:57:33.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.964+0000 7f834beec700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f834406e0c0 con 0x7f834406c1f0 2023-10-06T03:57:33.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.964+0000 7f8349c88700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f834406cbb0 0x7f834406d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39628/0 (socket says 172.21.15.157:39628) 2023-10-06T03:57:33.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.964+0000 7f8349c88700 1 -- 172.21.15.157:0/1273782801 learned_addr learned my addr 172.21.15.157:0/1273782801 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:33.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f834a489700 1 --2- 172.21.15.157:0/1273782801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f834406c1f0 0x7f8344150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:33.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f8349487700 1 --2- 172.21.15.157:0/1273782801 >> v2:172.21.15.175:3300/0 conn(0x7f834406dc40 0x7f834414fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:33.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f8348c86700 1 -- 172.21.15.157:0/1273782801 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2002700121 0 0) 0x7f83440c9780 con 0x7f834406cbb0 2023-10-06T03:57:33.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f8348c86700 1 -- 172.21.15.157:0/1273782801 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f832c003570 con 0x7f834406cbb0 2023-10-06T03:57:33.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f8348c86700 1 -- 172.21.15.157:0/1273782801 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 775402301 0 0) 0x7f832c003570 con 0x7f834406cbb0 2023-10-06T03:57:33.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f8348c86700 1 -- 172.21.15.157:0/1273782801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f834406c1f0 msgr2=0x7f8344150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:33.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f8348c86700 1 --2- 172.21.15.157:0/1273782801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f834406c1f0 0x7f8344150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:33.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f8348c86700 1 -- 172.21.15.157:0/1273782801 >> v2:172.21.15.175:3300/0 conn(0x7f834406dc40 msgr2=0x7f834414fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:33.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f8348c86700 1 --2- 172.21.15.157:0/1273782801 >> v2:172.21.15.175:3300/0 conn(0x7f834406dc40 0x7f834414fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:33.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f8348c86700 1 -- 172.21.15.157:0/1273782801 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83440c8e50 con 0x7f834406cbb0 2023-10-06T03:57:33.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f8348c86700 1 -- 172.21.15.157:0/1273782801 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f83340017f0 con 0x7f834406cbb0 2023-10-06T03:57:33.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f8349487700 1 --2- 172.21.15.157:0/1273782801 >> v2:172.21.15.175:3300/0 conn(0x7f834406dc40 0x7f834414fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:57:33.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f8348c86700 1 -- 172.21.15.157:0/1273782801 >> v1:172.21.15.157:6789/0 conn(0x7f834406cbb0 legacy=0x7f834406d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:33.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f8348c86700 1 --2- 172.21.15.157:0/1273782801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f832c003a60 0x7f832c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:33.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f8348c86700 1 --2- 172.21.15.157:0/1273782801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f834406c1f0 0x7f832c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:33.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f8348c86700 1 --2- 172.21.15.157:0/1273782801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f832c004d50 0x7f832c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:33.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f8348c86700 1 -- 172.21.15.157:0/1273782801 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83440c8c40 con 0x7f832c003a60 2023-10-06T03:57:33.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f8348c86700 1 -- 172.21.15.157:0/1273782801 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834406e0c0 con 0x7f832c004d50 2023-10-06T03:57:33.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f8348c86700 1 -- 172.21.15.157:0/1273782801 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832c00ba60 con 0x7f834406c1f0 2023-10-06T03:57:33.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f8349c88700 1 --2- 172.21.15.157:0/1273782801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f832c003a60 0x7f832c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:33.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f8349487700 1 --2- 172.21.15.157:0/1273782801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f834406c1f0 0x7f832c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:33.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f834a489700 1 --2- 172.21.15.157:0/1273782801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f832c004d50 0x7f832c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:33.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f8349c88700 1 -- 172.21.15.157:0/1273782801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f834406c1f0 msgr2=0x7f832c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:33.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f8349c88700 1 --2- 172.21.15.157:0/1273782801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f834406c1f0 0x7f832c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:33.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f8349c88700 1 -- 172.21.15.157:0/1273782801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f832c004d50 msgr2=0x7f832c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:33.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f8349c88700 1 --2- 172.21.15.157:0/1273782801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f832c004d50 0x7f832c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:33.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f8349c88700 1 -- 172.21.15.157:0/1273782801 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c003570 con 0x7f832c003a60 2023-10-06T03:57:33.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f8349487700 1 --2- 172.21.15.157:0/1273782801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f834406c1f0 0x7f832c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:33.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f834a489700 1 --2- 172.21.15.157:0/1273782801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f832c004d50 0x7f832c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:33.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.965+0000 7f8349c88700 1 --2- 172.21.15.157:0/1273782801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f832c003a60 0x7f832c003e70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f833400a970 tx=0x7f83440b1020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f626c596fe686cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:33.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.966+0000 7f8348c86700 1 -- 172.21.15.157:0/1273782801 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83340070d0 con 0x7f832c003a60 2023-10-06T03:57:33.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.966+0000 7f8348c86700 1 -- 172.21.15.157:0/1273782801 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8334007230 con 0x7f832c003a60 2023-10-06T03:57:33.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.966+0000 7f8348c86700 1 -- 172.21.15.157:0/1273782801 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83340075f0 con 0x7f832c003a60 2023-10-06T03:57:33.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.966+0000 7f834beec700 1 -- 172.21.15.157:0/1273782801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f832c003a60 msgr2=0x7f832c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:33.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.966+0000 7f834beec700 1 --2- 172.21.15.157:0/1273782801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f832c003a60 0x7f832c003e70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f833400a970 tx=0x7f83440b1020 comp rx=0 tx=0).stop 2023-10-06T03:57:33.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.966+0000 7f834a489700 1 -- 172.21.15.157:0/1273782801 reap_dead start 2023-10-06T03:57:33.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.966+0000 7f834beec700 1 -- 172.21.15.157:0/1273782801 shutdown_connections 2023-10-06T03:57:33.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.966+0000 7f834beec700 1 -- 172.21.15.157:0/1273782801 >> 172.21.15.157:0/1273782801 conn(0x7f834406f230 msgr2=0x7f834406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:33.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.966+0000 7f834beec700 1 -- 172.21.15.157:0/1273782801 shutdown_connections 2023-10-06T03:57:33.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.966+0000 7f834beec700 1 -- 172.21.15.157:0/1273782801 wait complete. 2023-10-06T03:57:33.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.967+0000 7f834beec700 1 Processor -- start 2023-10-06T03:57:33.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.967+0000 7f834beec700 1 -- start start 2023-10-06T03:57:33.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.968+0000 7f834beec700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f832c003a60 0x7f8344146790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:33.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.968+0000 7f834beec700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8344148550 0x7f8344167170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:33.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.968+0000 7f834beec700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8344152020 0x7f83440cce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:33.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.968+0000 7f834beec700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8344147d40 con 0x7f8344152020 2023-10-06T03:57:33.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.968+0000 7f834beec700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8344146300 con 0x7f8344148550 2023-10-06T03:57:33.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.968+0000 7f834beec700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8344152490 con 0x7f832c003a60 2023-10-06T03:57:33.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.968+0000 7f834a489700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8344152020 0x7f83440cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:33.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.968+0000 7f834a489700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8344152020 0x7f83440cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35556/0 (socket says 172.21.15.157:35556) 2023-10-06T03:57:33.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.968+0000 7f834a489700 1 -- 172.21.15.157:0/3198206440 learned_addr learned my addr 172.21.15.157:0/3198206440 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:33.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.968+0000 7f8349487700 1 --2- 172.21.15.157:0/3198206440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8344148550 0x7f8344167170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:33.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.968+0000 7f8349c88700 1 --2- 172.21.15.157:0/3198206440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f832c003a60 0x7f8344146790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:33.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.968+0000 7f834a489700 1 -- 172.21.15.157:0/3198206440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f832c003a60 msgr2=0x7f8344146790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:33.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.968+0000 7f834a489700 1 --2- 172.21.15.157:0/3198206440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f832c003a60 0x7f8344146790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:33.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.968+0000 7f834a489700 1 -- 172.21.15.157:0/3198206440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8344148550 msgr2=0x7f8344167170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:33.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.968+0000 7f834a489700 1 --2- 172.21.15.157:0/3198206440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8344148550 0x7f8344167170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:33.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.968+0000 7f834a489700 1 -- 172.21.15.157:0/3198206440 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8344147f90 con 0x7f8344152020 2023-10-06T03:57:33.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.969+0000 7f834a489700 1 --2- 172.21.15.157:0/3198206440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8344152020 0x7f83440cce70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f83440cc630 tx=0x7f833c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:33.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.969+0000 7f8341ffb700 1 -- 172.21.15.157:0/3198206440 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c015670 con 0x7f8344152020 2023-10-06T03:57:33.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.969+0000 7f8341ffb700 1 -- 172.21.15.157:0/3198206440 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833c0157d0 con 0x7f8344152020 2023-10-06T03:57:33.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.969+0000 7f834beec700 1 -- 172.21.15.157:0/3198206440 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8344145cb0 con 0x7f8344152020 2023-10-06T03:57:33.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.969+0000 7f8341ffb700 1 -- 172.21.15.157:0/3198206440 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c015b90 con 0x7f8344152020 2023-10-06T03:57:33.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.969+0000 7f834beec700 1 -- 172.21.15.157:0/3198206440 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83440c3d30 con 0x7f8344152020 2023-10-06T03:57:33.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.970+0000 7f834beec700 1 -- 172.21.15.157:0/3198206440 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8344146150 con 0x7f8344152020 2023-10-06T03:57:33.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.970+0000 7f8341ffb700 1 -- 172.21.15.157:0/3198206440 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f833c01e400 con 0x7f8344152020 2023-10-06T03:57:33.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.971+0000 7f8341ffb700 1 --2- 172.21.15.157:0/3198206440 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8330066f30 0x7f83300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:33.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.971+0000 7f8341ffb700 1 -- 172.21.15.157:0/3198206440 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f833c084a20 con 0x7f8344152020 2023-10-06T03:57:33.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.971+0000 7f8349c88700 1 --2- 172.21.15.157:0/3198206440 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8330066f30 0x7f83300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:33.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.971+0000 7f8349c88700 1 --2- 172.21.15.157:0/3198206440 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8330066f30 0x7f83300693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f833400afd0 tx=0x7f8334016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:33.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:33.975+0000 7f8341ffb700 1 -- 172.21.15.157:0/3198206440 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f833c051b30 con 0x7f8344152020 2023-10-06T03:57:34.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.157+0000 7f834beec700 1 -- 172.21.15.157:0/3198206440 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8344146440 con 0x7f8344152020 2023-10-06T03:57:34.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.157+0000 7f8341ffb700 1 -- 172.21.15.157:0/3198206440 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f833c055150 con 0x7f8344152020 2023-10-06T03:57:34.160 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-10-06T03:57:34.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.159+0000 7f834beec700 1 -- 172.21.15.157:0/3198206440 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8330066f30 msgr2=0x7f83300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:34.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.159+0000 7f834beec700 1 --2- 172.21.15.157:0/3198206440 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8330066f30 0x7f83300693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f833400afd0 tx=0x7f8334016000 comp rx=0 tx=0).stop 2023-10-06T03:57:34.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.159+0000 7f834beec700 1 -- 172.21.15.157:0/3198206440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8344152020 msgr2=0x7f83440cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:34.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.159+0000 7f834beec700 1 --2- 172.21.15.157:0/3198206440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8344152020 0x7f83440cce70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f83440cc630 tx=0x7f833c008040 comp rx=0 tx=0).stop 2023-10-06T03:57:34.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.159+0000 7f834beec700 1 -- 172.21.15.157:0/3198206440 shutdown_connections 2023-10-06T03:57:34.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.159+0000 7f834beec700 1 --2- 172.21.15.157:0/3198206440 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8330066f30 0x7f83300693e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.159+0000 7f834beec700 1 --2- 172.21.15.157:0/3198206440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8344152020 0x7f83440cce70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.159+0000 7f834beec700 1 --2- 172.21.15.157:0/3198206440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f832c003a60 0x7f8344146790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.159+0000 7f834beec700 1 --2- 172.21.15.157:0/3198206440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8344148550 0x7f8344167170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.159+0000 7f834beec700 1 -- 172.21.15.157:0/3198206440 >> 172.21.15.157:0/3198206440 conn(0x7f834406f230 msgr2=0x7f834414dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:34.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.159+0000 7f834beec700 1 -- 172.21.15.157:0/3198206440 shutdown_connections 2023-10-06T03:57:34.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.159+0000 7f834beec700 1 -- 172.21.15.157:0/3198206440 wait complete. 2023-10-06T03:57:34.173 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:57:34.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.289+0000 7fc09b0fb700 1 Processor -- start 2023-10-06T03:57:34.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.289+0000 7fc09b0fb700 1 -- start start 2023-10-06T03:57:34.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.290+0000 7fc09b0fb700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc0940ca450 0x7fc0940c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:34.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.290+0000 7fc09b0fb700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc094068600 0x7fc0940c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:34.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.290+0000 7fc09b0fb700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0940c6410 con 0x7fc0940c5250 2023-10-06T03:57:34.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.290+0000 7fc09b0fb700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc09406d300 con 0x7fc0940ca450 2023-10-06T03:57:34.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.290+0000 7fc09b0fb700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0940675a0 con 0x7fc094068600 2023-10-06T03:57:34.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.290+0000 7fc09a8fa700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc0940c5250 0x7fc094156580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39644/0 (socket says 172.21.15.157:39644) 2023-10-06T03:57:34.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.290+0000 7fc09a8fa700 1 -- 172.21.15.157:0/1558456122 learned_addr learned my addr 172.21.15.157:0/1558456122 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:34.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.290+0000 7fc09a0f9700 1 --2- 172.21.15.157:0/1558456122 >> v2:172.21.15.175:3300/0 conn(0x7fc0940ca450 0x7fc0940c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:34.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.290+0000 7fc0998f8700 1 --2- 172.21.15.157:0/1558456122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc094068600 0x7fc0940c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:34.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.290+0000 7fc0990f7700 1 -- 172.21.15.157:0/1558456122 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3910033420 0 0) 0x7fc0940c6410 con 0x7fc0940c5250 2023-10-06T03:57:34.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.290+0000 7fc0990f7700 1 -- 172.21.15.157:0/1558456122 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc080003570 con 0x7fc0940c5250 2023-10-06T03:57:34.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.291+0000 7fc09a0f9700 1 -- 172.21.15.157:0/1558456122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc094068600 msgr2=0x7fc0940c6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:34.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.291+0000 7fc09a0f9700 1 --2- 172.21.15.157:0/1558456122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc094068600 0x7fc0940c6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.291+0000 7fc09a0f9700 1 -- 172.21.15.157:0/1558456122 >> v1:172.21.15.157:6789/0 conn(0x7fc0940c5250 legacy=0x7fc094156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:34.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.291+0000 7fc09a0f9700 1 -- 172.21.15.157:0/1558456122 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0940c60c0 con 0x7fc0940ca450 2023-10-06T03:57:34.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.291+0000 7fc09a0f9700 1 --2- 172.21.15.157:0/1558456122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0940ca450 0x7fc0940c68a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc0900098e0 tx=0x7fc0900091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=314ad45c8a4d6e19 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:34.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.291+0000 7fc0990f7700 1 -- 172.21.15.157:0/1558456122 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc090011630 con 0x7fc0940ca450 2023-10-06T03:57:34.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.291+0000 7fc0990f7700 1 -- 172.21.15.157:0/1558456122 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc090011790 con 0x7fc0940ca450 2023-10-06T03:57:34.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.291+0000 7fc0990f7700 1 -- 172.21.15.157:0/1558456122 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc090011b30 con 0x7fc0940ca450 2023-10-06T03:57:34.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.291+0000 7fc09b0fb700 1 -- 172.21.15.157:0/1558456122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0940ca450 msgr2=0x7fc0940c68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:34.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.291+0000 7fc09b0fb700 1 --2- 172.21.15.157:0/1558456122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0940ca450 0x7fc0940c68a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc0900098e0 tx=0x7fc0900091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.291+0000 7fc09b0fb700 1 -- 172.21.15.157:0/1558456122 shutdown_connections 2023-10-06T03:57:34.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.291+0000 7fc09b0fb700 1 --2- 172.21.15.157:0/1558456122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc0940ca450 0x7fc0940c68a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.291+0000 7fc09b0fb700 1 --2- 172.21.15.157:0/1558456122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc094068600 0x7fc0940c6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.291+0000 7fc09b0fb700 1 -- 172.21.15.157:0/1558456122 >> 172.21.15.157:0/1558456122 conn(0x7fc094069100 msgr2=0x7fc094069500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:34.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.291+0000 7fc09b0fb700 1 -- 172.21.15.157:0/1558456122 shutdown_connections 2023-10-06T03:57:34.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.291+0000 7fc09b0fb700 1 -- 172.21.15.157:0/1558456122 wait complete. 2023-10-06T03:57:34.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.293+0000 7fc09b0fb700 1 Processor -- start 2023-10-06T03:57:34.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.293+0000 7fc09b0fb700 1 -- start start 2023-10-06T03:57:34.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.293+0000 7fc09b0fb700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc094068600 0x7fc0940c3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:34.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.293+0000 7fc09b0fb700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0940c5250 0x7fc0940c2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:34.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.293+0000 7fc09b0fb700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0940ca450 0x7fc094154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:34.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.293+0000 7fc09b0fb700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc094165d00 con 0x7fc0940c5250 2023-10-06T03:57:34.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.293+0000 7fc09b0fb700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0940c3f70 con 0x7fc094068600 2023-10-06T03:57:34.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.293+0000 7fc09b0fb700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0940c15e0 con 0x7fc0940ca450 2023-10-06T03:57:34.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.293+0000 7fc0998f8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0940c5250 0x7fc0940c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:34.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.293+0000 7fc0998f8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0940c5250 0x7fc0940c2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35562/0 (socket says 172.21.15.157:35562) 2023-10-06T03:57:34.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.293+0000 7fc0998f8700 1 -- 172.21.15.157:0/1887581157 learned_addr learned my addr 172.21.15.157:0/1887581157 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:34.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.293+0000 7fc09a0f9700 1 --2- 172.21.15.157:0/1887581157 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc094068600 0x7fc0940c3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:34.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.293+0000 7fc09a8fa700 1 --2- 172.21.15.157:0/1887581157 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0940ca450 0x7fc094154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:34.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.294+0000 7fc0998f8700 1 -- 172.21.15.157:0/1887581157 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0940ca450 msgr2=0x7fc094154000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:34.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.294+0000 7fc0998f8700 1 --2- 172.21.15.157:0/1887581157 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0940ca450 0x7fc094154000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.294+0000 7fc0998f8700 1 -- 172.21.15.157:0/1887581157 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc094068600 msgr2=0x7fc0940c3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:34.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.294+0000 7fc0998f8700 1 --2- 172.21.15.157:0/1887581157 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc094068600 0x7fc0940c3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.294+0000 7fc0998f8700 1 -- 172.21.15.157:0/1887581157 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc090009420 con 0x7fc0940c5250 2023-10-06T03:57:34.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.294+0000 7fc0998f8700 1 --2- 172.21.15.157:0/1887581157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0940c5250 0x7fc0940c2920 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc08400ec50 tx=0x7fc0840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:34.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.294+0000 7fc08a7fc700 1 -- 172.21.15.157:0/1887581157 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08400c850 con 0x7fc0940c5250 2023-10-06T03:57:34.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.294+0000 7fc08a7fc700 1 -- 172.21.15.157:0/1887581157 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08400c9b0 con 0x7fc0940c5250 2023-10-06T03:57:34.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.294+0000 7fc08a7fc700 1 -- 172.21.15.157:0/1887581157 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08400cd70 con 0x7fc0940c5250 2023-10-06T03:57:34.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.294+0000 7fc09b0fb700 1 -- 172.21.15.157:0/1887581157 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0941661d0 con 0x7fc0940c5250 2023-10-06T03:57:34.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.294+0000 7fc09b0fb700 1 -- 172.21.15.157:0/1887581157 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc094166790 con 0x7fc0940c5250 2023-10-06T03:57:34.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.295+0000 7fc09b0fb700 1 -- 172.21.15.157:0/1887581157 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0940cb130 con 0x7fc0940c5250 2023-10-06T03:57:34.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.295+0000 7fc08a7fc700 1 -- 172.21.15.157:0/1887581157 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc084013680 con 0x7fc0940c5250 2023-10-06T03:57:34.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.295+0000 7fc08a7fc700 1 --2- 172.21.15.157:0/1887581157 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc078066f30 0x7fc0780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:34.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.296+0000 7fc08a7fc700 1 -- 172.21.15.157:0/1887581157 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc0840848a0 con 0x7fc0940c5250 2023-10-06T03:57:34.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.296+0000 7fc09a0f9700 1 --2- 172.21.15.157:0/1887581157 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc078066f30 0x7fc0780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:34.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.296+0000 7fc09a0f9700 1 --2- 172.21.15.157:0/1887581157 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc078066f30 0x7fc0780693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc0900091e0 tx=0x7fc090017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:34.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.300+0000 7fc08a7fc700 1 -- 172.21.15.157:0/1887581157 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc0840519b0 con 0x7fc0940c5250 2023-10-06T03:57:34.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.484+0000 7fc09b0fb700 1 -- 172.21.15.157:0/1887581157 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc0940c1720 con 0x7fc0940c5250 2023-10-06T03:57:34.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.485+0000 7fc08a7fc700 1 -- 172.21.15.157:0/1887581157 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc084054fd0 con 0x7fc0940c5250 2023-10-06T03:57:34.487 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-10-06T03:57:34.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.487+0000 7fc09b0fb700 1 -- 172.21.15.157:0/1887581157 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc078066f30 msgr2=0x7fc0780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:34.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.487+0000 7fc09b0fb700 1 --2- 172.21.15.157:0/1887581157 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc078066f30 0x7fc0780693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc0900091e0 tx=0x7fc090017000 comp rx=0 tx=0).stop 2023-10-06T03:57:34.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.487+0000 7fc09b0fb700 1 -- 172.21.15.157:0/1887581157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0940c5250 msgr2=0x7fc0940c2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:34.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.487+0000 7fc09b0fb700 1 --2- 172.21.15.157:0/1887581157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0940c5250 0x7fc0940c2920 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc08400ec50 tx=0x7fc0840091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.487+0000 7fc09b0fb700 1 -- 172.21.15.157:0/1887581157 shutdown_connections 2023-10-06T03:57:34.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.487+0000 7fc09b0fb700 1 --2- 172.21.15.157:0/1887581157 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc078066f30 0x7fc0780693e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.487+0000 7fc09b0fb700 1 --2- 172.21.15.157:0/1887581157 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc094068600 0x7fc0940c3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.487+0000 7fc09b0fb700 1 --2- 172.21.15.157:0/1887581157 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc0940ca450 0x7fc094154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.487+0000 7fc09b0fb700 1 --2- 172.21.15.157:0/1887581157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0940c5250 0x7fc0940c2920 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.487+0000 7fc09b0fb700 1 -- 172.21.15.157:0/1887581157 >> 172.21.15.157:0/1887581157 conn(0x7fc094069100 msgr2=0x7fc0940c99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:34.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.487+0000 7fc09b0fb700 1 -- 172.21.15.157:0/1887581157 shutdown_connections 2023-10-06T03:57:34.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.487+0000 7fc09b0fb700 1 -- 172.21.15.157:0/1887581157 wait complete. 2023-10-06T03:57:34.501 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:57:34.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.613+0000 7f534eabd700 1 Processor -- start 2023-10-06T03:57:34.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.613+0000 7f534eabd700 1 -- start start 2023-10-06T03:57:34.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.614+0000 7f534eabd700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5348159df0 0x7f534814fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:34.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.614+0000 7f534eabd700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f534806ced0 con 0x7f53480b2110 2023-10-06T03:57:34.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.614+0000 7f534eabd700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53480afb10 con 0x7f53480b1d40 2023-10-06T03:57:34.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.614+0000 7f534eabd700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f534815a270 con 0x7f5348159df0 2023-10-06T03:57:34.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.614+0000 7f5347fff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f53480b2110 0x7f534806c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39660/0 (socket says 172.21.15.157:39660) 2023-10-06T03:57:34.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.614+0000 7f5347fff700 1 -- 172.21.15.157:0/3657655845 learned_addr learned my addr 172.21.15.157:0/3657655845 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:34.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.614+0000 7f53477fe700 1 -- 172.21.15.157:0/3657655845 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 212889739 0 0) 0x7f534806ced0 con 0x7f53480b2110 2023-10-06T03:57:34.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.614+0000 7f53477fe700 1 -- 172.21.15.157:0/3657655845 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5330003570 con 0x7f53480b2110 2023-10-06T03:57:34.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.614+0000 7f534d05a700 1 --2- 172.21.15.157:0/3657655845 >> v2:172.21.15.175:3300/0 conn(0x7f5348159df0 0x7f534814fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:34.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.615+0000 7f53477fe700 1 -- 172.21.15.157:0/3657655845 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1792740272 0 0) 0x7f5330003570 con 0x7f53480b2110 2023-10-06T03:57:34.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.615+0000 7f53477fe700 1 -- 172.21.15.157:0/3657655845 >> v2:172.21.15.175:3300/0 conn(0x7f5348159df0 msgr2=0x7f534814fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:34.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.615+0000 7f53477fe700 1 --2- 172.21.15.157:0/3657655845 >> v2:172.21.15.175:3300/0 conn(0x7f5348159df0 0x7f534814fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.615+0000 7f53477fe700 1 -- 172.21.15.157:0/3657655845 >> v1:172.21.15.175:6789/0 conn(0x7f53480b1d40 legacy=0x7f53480af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:34.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.615+0000 7f53477fe700 1 -- 172.21.15.157:0/3657655845 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534806cc10 con 0x7f53480b2110 2023-10-06T03:57:34.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.615+0000 7f53477fe700 1 -- 172.21.15.157:0/3657655845 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f533c001470 con 0x7f53480b2110 2023-10-06T03:57:34.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.615+0000 7f53477fe700 1 -- 172.21.15.157:0/3657655845 >> v1:172.21.15.157:6789/0 conn(0x7f53480b2110 legacy=0x7f534806c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:34.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.615+0000 7f53477fe700 1 --2- 172.21.15.157:0/3657655845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5330003a60 0x7f5330003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:34.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.615+0000 7f53477fe700 1 --2- 172.21.15.157:0/3657655845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5330004810 0x7f5330004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:34.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.615+0000 7f53477fe700 1 --2- 172.21.15.157:0/3657655845 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53300051c0 0x7f533000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:34.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.615+0000 7f53477fe700 1 -- 172.21.15.157:0/3657655845 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534815a270 con 0x7f5330003a60 2023-10-06T03:57:34.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.615+0000 7f53477fe700 1 -- 172.21.15.157:0/3657655845 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533000c0d0 con 0x7f5330004810 2023-10-06T03:57:34.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.615+0000 7f53477fe700 1 -- 172.21.15.157:0/3657655845 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f533000c290 con 0x7f53300051c0 2023-10-06T03:57:34.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.615+0000 7f534c859700 1 --2- 172.21.15.157:0/3657655845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5330003a60 0x7f5330003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:34.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.615+0000 7f5347fff700 1 --2- 172.21.15.157:0/3657655845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5330004810 0x7f5330004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:34.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.615+0000 7f534d05a700 1 --2- 172.21.15.157:0/3657655845 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53300051c0 0x7f533000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:34.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.615+0000 7f534c859700 1 -- 172.21.15.157:0/3657655845 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53300051c0 msgr2=0x7f533000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:34.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.615+0000 7f534c859700 1 --2- 172.21.15.157:0/3657655845 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53300051c0 0x7f533000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.615+0000 7f534c859700 1 -- 172.21.15.157:0/3657655845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5330004810 msgr2=0x7f5330004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:34.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.615+0000 7f534c859700 1 --2- 172.21.15.157:0/3657655845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5330004810 0x7f5330004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.615+0000 7f534c859700 1 -- 172.21.15.157:0/3657655845 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5330003570 con 0x7f5330003a60 2023-10-06T03:57:34.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.615+0000 7f534d05a700 1 -- 172.21.15.157:0/3657655845 reap_dead start 2023-10-06T03:57:34.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.615+0000 7f534c859700 1 --2- 172.21.15.157:0/3657655845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5330003a60 0x7f5330003e70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5338001fc0 tx=0x7f5338001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf155ce24ab825e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:34.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.615+0000 7f53477fe700 1 -- 172.21.15.157:0/3657655845 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5338001950 con 0x7f5330003a60 2023-10-06T03:57:34.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.615+0000 7f53477fe700 1 -- 172.21.15.157:0/3657655845 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533800c8b0 con 0x7f5330003a60 2023-10-06T03:57:34.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.616+0000 7f53477fe700 1 -- 172.21.15.157:0/3657655845 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533800cc50 con 0x7f5330003a60 2023-10-06T03:57:34.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.616+0000 7f534eabd700 1 -- 172.21.15.157:0/3657655845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5330003a60 msgr2=0x7f5330003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:34.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.616+0000 7f534eabd700 1 --2- 172.21.15.157:0/3657655845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5330003a60 0x7f5330003e70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5338001fc0 tx=0x7f5338001640 comp rx=0 tx=0).stop 2023-10-06T03:57:34.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.616+0000 7f534eabd700 1 -- 172.21.15.157:0/3657655845 shutdown_connections 2023-10-06T03:57:34.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.616+0000 7f534eabd700 1 --2- 172.21.15.157:0/3657655845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5330003a60 0x7f5330003e70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.616+0000 7f534eabd700 1 -- 172.21.15.157:0/3657655845 >> 172.21.15.157:0/3657655845 conn(0x7f53480cc6e0 msgr2=0x7f53480ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:34.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.616+0000 7f534eabd700 1 -- 172.21.15.157:0/3657655845 shutdown_connections 2023-10-06T03:57:34.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.616+0000 7f534eabd700 1 -- 172.21.15.157:0/3657655845 wait complete. 2023-10-06T03:57:34.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.617+0000 7f534eabd700 1 Processor -- start 2023-10-06T03:57:34.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.617+0000 7f534eabd700 1 -- start start 2023-10-06T03:57:34.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.618+0000 7f534eabd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5330003a60 0x7f534814c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:34.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.618+0000 7f534eabd700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53480cb1b0 0x7f53481589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:34.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.618+0000 7f534eabd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5348158ee0 0x7f5348169410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:34.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.618+0000 7f534eabd700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53481500c0 con 0x7f5348158ee0 2023-10-06T03:57:34.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.618+0000 7f534eabd700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5348150360 con 0x7f5330003a60 2023-10-06T03:57:34.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.618+0000 7f534eabd700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53481665b0 con 0x7f53480cb1b0 2023-10-06T03:57:34.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.618+0000 7f534d05a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5348158ee0 0x7f5348169410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:34.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.618+0000 7f534c859700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5330003a60 0x7f534814c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:34.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.618+0000 7f534d05a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5348158ee0 0x7f5348169410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35580/0 (socket says 172.21.15.157:35580) 2023-10-06T03:57:34.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.618+0000 7f534d05a700 1 -- 172.21.15.157:0/1748168673 learned_addr learned my addr 172.21.15.157:0/1748168673 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:34.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.618+0000 7f5347fff700 1 --2- 172.21.15.157:0/1748168673 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53480cb1b0 0x7f53481589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:34.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.618+0000 7f534d05a700 1 -- 172.21.15.157:0/1748168673 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53480cb1b0 msgr2=0x7f53481589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:34.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.618+0000 7f534d05a700 1 --2- 172.21.15.157:0/1748168673 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53480cb1b0 0x7f53481589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.618+0000 7f534d05a700 1 -- 172.21.15.157:0/1748168673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5330003a60 msgr2=0x7f534814c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:34.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.618+0000 7f534d05a700 1 --2- 172.21.15.157:0/1748168673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5330003a60 0x7f534814c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.618+0000 7f534d05a700 1 -- 172.21.15.157:0/1748168673 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5348166890 con 0x7f5348158ee0 2023-10-06T03:57:34.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.618+0000 7f534c859700 1 --2- 172.21.15.157:0/1748168673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5330003a60 0x7f534814c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:57:34.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.618+0000 7f534d05a700 1 --2- 172.21.15.157:0/1748168673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5348158ee0 0x7f5348169410 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f534806d6e0 tx=0x7f53400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:34.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.619+0000 7f5344ff9700 1 -- 172.21.15.157:0/1748168673 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5340013750 con 0x7f5348158ee0 2023-10-06T03:57:34.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.619+0000 7f5344ff9700 1 -- 172.21.15.157:0/1748168673 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53400138b0 con 0x7f5348158ee0 2023-10-06T03:57:34.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.619+0000 7f534eabd700 1 -- 172.21.15.157:0/1748168673 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53480c39a0 con 0x7f5348158ee0 2023-10-06T03:57:34.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.619+0000 7f5344ff9700 1 -- 172.21.15.157:0/1748168673 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5340013c50 con 0x7f5348158ee0 2023-10-06T03:57:34.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.619+0000 7f534eabd700 1 -- 172.21.15.157:0/1748168673 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53480c4c60 con 0x7f5348158ee0 2023-10-06T03:57:34.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.620+0000 7f534eabd700 1 -- 172.21.15.157:0/1748168673 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53480c3e40 con 0x7f5348158ee0 2023-10-06T03:57:34.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.620+0000 7f5344ff9700 1 -- 172.21.15.157:0/1748168673 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5340014650 con 0x7f5348158ee0 2023-10-06T03:57:34.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.620+0000 7f5344ff9700 1 --2- 172.21.15.157:0/1748168673 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5334066f30 0x7f53340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:34.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.621+0000 7f534c859700 1 --2- 172.21.15.157:0/1748168673 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5334066f30 0x7f53340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:34.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.621+0000 7f5344ff9700 1 -- 172.21.15.157:0/1748168673 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5340084c00 con 0x7f5348158ee0 2023-10-06T03:57:34.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.621+0000 7f534c859700 1 --2- 172.21.15.157:0/1748168673 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5334066f30 0x7f53340693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5338001390 tx=0x7f5338015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:34.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.626+0000 7f5344ff9700 1 -- 172.21.15.157:0/1748168673 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5340051cf0 con 0x7f5348158ee0 2023-10-06T03:57:34.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.849+0000 7f534eabd700 1 -- 172.21.15.157:0/1748168673 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5348150c00 con 0x7f5348158ee0 2023-10-06T03:57:34.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.850+0000 7f5344ff9700 1 -- 172.21.15.157:0/1748168673 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6185 (secure 0 0 0) 0x7f5340022090 con 0x7f5348158ee0 2023-10-06T03:57:34.852 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:34.852 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T03:57:32.634356+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"f","rank":4,"incarnation":15,"state":"up:creating","state_seq":1,"addr":"172.21.15.175:6839/4177493830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4177493830},{"type":"v1","addr":"172.21.15.175:6839","nonce":4177493830}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:57:34.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.851+0000 7f534eabd700 1 -- 172.21.15.157:0/1748168673 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5334066f30 msgr2=0x7f53340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:34.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.851+0000 7f534eabd700 1 --2- 172.21.15.157:0/1748168673 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5334066f30 0x7f53340693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5338001390 tx=0x7f5338015000 comp rx=0 tx=0).stop 2023-10-06T03:57:34.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.851+0000 7f534eabd700 1 -- 172.21.15.157:0/1748168673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5348158ee0 msgr2=0x7f5348169410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:34.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.851+0000 7f534eabd700 1 --2- 172.21.15.157:0/1748168673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5348158ee0 0x7f5348169410 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f534806d6e0 tx=0x7f53400091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.852+0000 7f534eabd700 1 -- 172.21.15.157:0/1748168673 shutdown_connections 2023-10-06T03:57:34.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.852+0000 7f534eabd700 1 --2- 172.21.15.157:0/1748168673 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5334066f30 0x7f53340693e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.852+0000 7f534eabd700 1 --2- 172.21.15.157:0/1748168673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5330003a60 0x7f534814c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.852+0000 7f534eabd700 1 --2- 172.21.15.157:0/1748168673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5348158ee0 0x7f5348169410 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.852+0000 7f534eabd700 1 --2- 172.21.15.157:0/1748168673 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53480cb1b0 0x7f53481589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.852+0000 7f534eabd700 1 -- 172.21.15.157:0/1748168673 >> 172.21.15.157:0/1748168673 conn(0x7f53480cc6e0 msgr2=0x7f5348157fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:34.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.852+0000 7f534eabd700 1 -- 172.21.15.157:0/1748168673 shutdown_connections 2023-10-06T03:57:34.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.852+0000 7f534eabd700 1 -- 172.21.15.157:0/1748168673 wait complete. 2023-10-06T03:57:34.856 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 15 2023-10-06T03:57:34.866 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:57:34.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.979+0000 7fb83176d700 1 Processor -- start 2023-10-06T03:57:34.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.979+0000 7fb83176d700 1 -- start start 2023-10-06T03:57:34.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.980+0000 7fb83176d700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb82c0b1e40 0x7fb82c157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:34.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.980+0000 7fb83176d700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb82c06d8d0 con 0x7fb82c06c2d0 2023-10-06T03:57:34.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.980+0000 7fb83176d700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb82c0af8f0 con 0x7fb82c0b04d0 2023-10-06T03:57:34.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.980+0000 7fb83176d700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c15a690 con 0x7fb82c0b1e40 2023-10-06T03:57:34.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.980+0000 7fb830f6c700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb82c06c2d0 0x7fb82c156ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39672/0 (socket says 172.21.15.157:39672) 2023-10-06T03:57:34.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.980+0000 7fb830f6c700 1 -- 172.21.15.157:0/2391844191 learned_addr learned my addr 172.21.15.157:0/2391844191 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:34.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.980+0000 7fb82bfff700 1 --2- 172.21.15.157:0/2391844191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb82c0b1e40 0x7fb82c157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:34.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.980+0000 7fb82affd700 1 -- 172.21.15.157:0/2391844191 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2377184617 0 0) 0x7fb82c06d8d0 con 0x7fb82c06c2d0 2023-10-06T03:57:34.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.980+0000 7fb82affd700 1 -- 172.21.15.157:0/2391844191 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb814003570 con 0x7fb82c06c2d0 2023-10-06T03:57:34.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.980+0000 7fb82affd700 1 -- 172.21.15.157:0/2391844191 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2490199411 0 0) 0x7fb82c0af8f0 con 0x7fb82c0b04d0 2023-10-06T03:57:34.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.980+0000 7fb82affd700 1 -- 172.21.15.157:0/2391844191 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb82c06d8d0 con 0x7fb82c0b04d0 2023-10-06T03:57:34.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.980+0000 7fb82bfff700 1 -- 172.21.15.157:0/2391844191 >> v1:172.21.15.175:6789/0 conn(0x7fb82c0b04d0 legacy=0x7fb82c1563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:34.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.980+0000 7fb82bfff700 1 -- 172.21.15.157:0/2391844191 >> v1:172.21.15.157:6789/0 conn(0x7fb82c06c2d0 legacy=0x7fb82c156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:34.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.980+0000 7fb82bfff700 1 -- 172.21.15.157:0/2391844191 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82c1571d0 con 0x7fb82c0b1e40 2023-10-06T03:57:34.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.980+0000 7fb82bfff700 1 --2- 172.21.15.157:0/2391844191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb82c0b1e40 0x7fb82c157750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009910 tx=0x7fb81c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56fb79e654138ff3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:34.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.980+0000 7fb82affd700 1 -- 172.21.15.157:0/2391844191 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c011630 con 0x7fb82c0b1e40 2023-10-06T03:57:34.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.980+0000 7fb82affd700 1 -- 172.21.15.157:0/2391844191 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81c011790 con 0x7fb82c0b1e40 2023-10-06T03:57:34.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.980+0000 7fb82affd700 1 -- 172.21.15.157:0/2391844191 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c011b30 con 0x7fb82c0b1e40 2023-10-06T03:57:34.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.980+0000 7fb83176d700 1 -- 172.21.15.157:0/2391844191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb82c0b1e40 msgr2=0x7fb82c157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:34.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.980+0000 7fb83176d700 1 --2- 172.21.15.157:0/2391844191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb82c0b1e40 0x7fb82c157750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009910 tx=0x7fb81c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.980+0000 7fb83176d700 1 -- 172.21.15.157:0/2391844191 shutdown_connections 2023-10-06T03:57:34.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.980+0000 7fb83176d700 1 --2- 172.21.15.157:0/2391844191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb82c0b1e40 0x7fb82c157750 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.980+0000 7fb83176d700 1 -- 172.21.15.157:0/2391844191 >> 172.21.15.157:0/2391844191 conn(0x7fb82c0cc540 msgr2=0x7fb82c0cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:34.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.980+0000 7fb83176d700 1 -- 172.21.15.157:0/2391844191 shutdown_connections 2023-10-06T03:57:34.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.980+0000 7fb83176d700 1 -- 172.21.15.157:0/2391844191 wait complete. 2023-10-06T03:57:34.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.981+0000 7fb83176d700 1 Processor -- start 2023-10-06T03:57:34.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.982+0000 7fb83176d700 1 -- start start 2023-10-06T03:57:34.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.982+0000 7fb83176d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb82c06c2d0 0x7fb82c0c8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:34.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.982+0000 7fb83176d700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb82c0b04d0 0x7fb82c154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:34.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.982+0000 7fb83176d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb82c0b1e40 0x7fb82c169330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:34.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.982+0000 7fb83176d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c1512c0 con 0x7fb82c06c2d0 2023-10-06T03:57:34.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.982+0000 7fb83176d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c0c6b80 con 0x7fb82c0b1e40 2023-10-06T03:57:34.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.982+0000 7fb83176d700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c0c6de0 con 0x7fb82c0b04d0 2023-10-06T03:57:34.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.982+0000 7fb82bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb82c06c2d0 0x7fb82c0c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:34.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.982+0000 7fb82bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb82c06c2d0 0x7fb82c0c8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35592/0 (socket says 172.21.15.157:35592) 2023-10-06T03:57:34.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.982+0000 7fb82bfff700 1 -- 172.21.15.157:0/2677469227 learned_addr learned my addr 172.21.15.157:0/2677469227 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:34.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.982+0000 7fb82b7fe700 1 --2- 172.21.15.157:0/2677469227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb82c0b04d0 0x7fb82c154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:34.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.983+0000 7fb830f6c700 1 --2- 172.21.15.157:0/2677469227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb82c0b1e40 0x7fb82c169330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:34.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.983+0000 7fb82bfff700 1 -- 172.21.15.157:0/2677469227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb82c0b04d0 msgr2=0x7fb82c154fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:34.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.983+0000 7fb82bfff700 1 --2- 172.21.15.157:0/2677469227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb82c0b04d0 0x7fb82c154fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.983+0000 7fb82bfff700 1 -- 172.21.15.157:0/2677469227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb82c0b1e40 msgr2=0x7fb82c169330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:34.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.983+0000 7fb82bfff700 1 --2- 172.21.15.157:0/2677469227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb82c0b1e40 0x7fb82c169330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:34.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.983+0000 7fb82bfff700 1 -- 172.21.15.157:0/2677469227 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c009420 con 0x7fb82c06c2d0 2023-10-06T03:57:34.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.983+0000 7fb82b7fe700 1 --2- 172.21.15.157:0/2677469227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb82c0b04d0 0x7fb82c154fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:34.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.983+0000 7fb82bfff700 1 --2- 172.21.15.157:0/2677469227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb82c06c2d0 0x7fb82c0c8320 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009f00 tx=0x7fb81c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:34.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.983+0000 7fb813fff700 1 -- 172.21.15.157:0/2677469227 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c020070 con 0x7fb82c06c2d0 2023-10-06T03:57:34.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.983+0000 7fb813fff700 1 -- 172.21.15.157:0/2677469227 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81c011900 con 0x7fb82c06c2d0 2023-10-06T03:57:34.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.983+0000 7fb83176d700 1 -- 172.21.15.157:0/2677469227 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb82c156730 con 0x7fb82c06c2d0 2023-10-06T03:57:34.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.983+0000 7fb813fff700 1 -- 172.21.15.157:0/2677469227 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c024410 con 0x7fb82c06c2d0 2023-10-06T03:57:34.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.983+0000 7fb83176d700 1 -- 172.21.15.157:0/2677469227 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb82c151510 con 0x7fb82c06c2d0 2023-10-06T03:57:34.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.984+0000 7fb83176d700 1 -- 172.21.15.157:0/2677469227 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb82c088f90 con 0x7fb82c06c2d0 2023-10-06T03:57:34.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.984+0000 7fb813fff700 1 -- 172.21.15.157:0/2677469227 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb81c0193b0 con 0x7fb82c06c2d0 2023-10-06T03:57:34.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.985+0000 7fb813fff700 1 --2- 172.21.15.157:0/2677469227 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb818066f30 0x7fb8180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:34.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.985+0000 7fb813fff700 1 -- 172.21.15.157:0/2677469227 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb81c085980 con 0x7fb82c06c2d0 2023-10-06T03:57:34.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.985+0000 7fb82b7fe700 1 --2- 172.21.15.157:0/2677469227 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb818066f30 0x7fb8180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:34.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.985+0000 7fb82b7fe700 1 --2- 172.21.15.157:0/2677469227 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb818066f30 0x7fb8180693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb820003990 tx=0x7fb820009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:34.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:34.990+0000 7fb813fff700 1 -- 172.21.15.157:0/2677469227 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb81c052a90 con 0x7fb82c06c2d0 2023-10-06T03:57:35.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.173+0000 7fb83176d700 1 -- 172.21.15.157:0/2677469227 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb82c06ccc0 con 0x7fb82c06c2d0 2023-10-06T03:57:35.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.174+0000 7fb813fff700 1 -- 172.21.15.157:0/2677469227 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16914 (secure 0 0 0) 0x7fb81c029020 con 0x7fb82c06c2d0 2023-10-06T03:57:35.177 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:35.177 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":21,"fsid":"28664a7a-fe30-4efe-be5a-6abf9318b700","created":"2023-10-06T03:56:51.887538+0000","modified":"2023-10-06T03:57:12.362537+0000","last_up_change":"2023-10-06T03:57:02.299440+0000","last_in_change":"2023-10-06T03:56:55.141505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:57:03.455121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:57:03.905661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:57:08.377186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:57:09.709582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f677cac-e9f4-4608-acac-718b072ce984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":90640},{"type":"v1","addr":"172.21.15.157:6809","nonce":90640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":90640},{"type":"v1","addr":"172.21.15.157:6811","nonce":90640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":90640},{"type":"v1","addr":"172.21.15.157:6815","nonce":90640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":90640},{"type":"v1","addr":"172.21.15.157:6813","nonce":90640}]},"public_addr":"172.21.15.157:6809/90640","cluster_addr":"172.21.15.157:6811/90640","heartbeat_back_addr":"172.21.15.157:6815/90640","heartbeat_front_addr":"172.21.15.157:6813/90640","state":["exists","up"]},{"osd":1,"uuid":"7d771892-bb91-4e9b-9610-051861c11247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":90654},{"type":"v1","addr":"172.21.15.157:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":90654},{"type":"v1","addr":"172.21.15.157:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":90654},{"type":"v1","addr":"172.21.15.157:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":90654},{"type":"v1","addr":"172.21.15.157:6805","nonce":90654}]},"public_addr":"172.21.15.157:6801/90654","cluster_addr":"172.21.15.157:6803/90654","heartbeat_back_addr":"172.21.15.157:6807/90654","heartbeat_front_addr":"172.21.15.157:6805/90654","state":["exists","up"]},{"osd":2,"uuid":"5c1f7d4e-d185-47cf-ab6c-453b4e3009c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":90657},{"type":"v1","addr":"172.21.15.157:6825","nonce":90657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":90657},{"type":"v1","addr":"172.21.15.157:6827","nonce":90657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":90657},{"type":"v1","addr":"172.21.15.157:6831","nonce":90657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":90657},{"type":"v1","addr":"172.21.15.157:6829","nonce":90657}]},"public_addr":"172.21.15.157:6825/90657","cluster_addr":"172.21.15.157:6827/90657","heartbeat_back_addr":"172.21.15.157:6831/90657","heartbeat_front_addr":"172.21.15.157:6829/90657","state":["exists","up"]},{"osd":3,"uuid":"f6a1e825-43c7-4afd-8daf-e51d45afc75e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":90641},{"type":"v1","addr":"172.21.15.157:6817","nonce":90641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":90641},{"type":"v1","addr":"172.21.15.157:6819","nonce":90641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":90641},{"type":"v1","addr":"172.21.15.157:6823","nonce":90641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":90641},{"type":"v1","addr":"172.21.15.157:6821","nonce":90641}]},"public_addr":"172.21.15.157:6817/90641","cluster_addr":"172.21.15.157:6819/90641","heartbeat_back_addr":"172.21.15.157:6823/90641","heartbeat_front_addr":"172.21.15.157:6821/90641","state":["exists","up"]},{"osd":4,"uuid":"8ad6e3d8-9a31-4f85-9410-417b7ac5da97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":91261},{"type":"v1","addr":"172.21.15.175:6803","nonce":91261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":91261},{"type":"v1","addr":"172.21.15.175:6806","nonce":91261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":91261},{"type":"v1","addr":"172.21.15.175:6814","nonce":91261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":91261},{"type":"v1","addr":"172.21.15.175:6810","nonce":91261}]},"public_addr":"172.21.15.175:6803/91261","cluster_addr":"172.21.15.175:6806/91261","heartbeat_back_addr":"172.21.15.175:6814/91261","heartbeat_front_addr":"172.21.15.175:6810/91261","state":["exists","up"]},{"osd":5,"uuid":"f68aca2a-f182-4c00-8e21-241d09c02ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":91262},{"type":"v1","addr":"172.21.15.175:6817","nonce":91262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":91262},{"type":"v1","addr":"172.21.15.175:6819","nonce":91262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":91262},{"type":"v1","addr":"172.21.15.175:6823","nonce":91262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":91262},{"type":"v1","addr":"172.21.15.175:6821","nonce":91262}]},"public_addr":"172.21.15.175:6817/91262","cluster_addr":"172.21.15.175:6819/91262","heartbeat_back_addr":"172.21.15.175:6823/91262","heartbeat_front_addr":"172.21.15.175:6821/91262","state":["exists","up"]},{"osd":6,"uuid":"d867cd89-3430-40ca-862f-d2bea3de9d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":91263},{"type":"v1","addr":"172.21.15.175:6801","nonce":91263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6805","nonce":91263},{"type":"v1","addr":"172.21.15.175:6807","nonce":91263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6813","nonce":91263},{"type":"v1","addr":"172.21.15.175:6815","nonce":91263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":91263},{"type":"v1","addr":"172.21.15.175:6811","nonce":91263}]},"public_addr":"172.21.15.175:6801/91263","cluster_addr":"172.21.15.175:6807/91263","heartbeat_back_addr":"172.21.15.175:6815/91263","heartbeat_front_addr":"172.21.15.175:6811/91263","state":["exists","up"]},{"osd":7,"uuid":"930cd464-79f1-47cb-b4ff-188ccde74fff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":91306},{"type":"v1","addr":"172.21.15.175:6825","nonce":91306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":91306},{"type":"v1","addr":"172.21.15.175:6827","nonce":91306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":91306},{"type":"v1","addr":"172.21.15.175:6831","nonce":91306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":91306},{"type":"v1","addr":"172.21.15.175:6829","nonce":91306}]},"public_addr":"172.21.15.175:6825/91306","cluster_addr":"172.21.15.175:6827/91306","heartbeat_back_addr":"172.21.15.175:6831/91306","heartbeat_front_addr":"172.21.15.175:6829/91306","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.290673+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.056390+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.292903+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.275861+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.332095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.327453+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.317585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.309330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:57:35.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.176+0000 7fb83176d700 1 -- 172.21.15.157:0/2677469227 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb818066f30 msgr2=0x7fb8180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:35.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.176+0000 7fb83176d700 1 --2- 172.21.15.157:0/2677469227 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb818066f30 0x7fb8180693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb820003990 tx=0x7fb820009040 comp rx=0 tx=0).stop 2023-10-06T03:57:35.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.176+0000 7fb83176d700 1 -- 172.21.15.157:0/2677469227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb82c06c2d0 msgr2=0x7fb82c0c8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:35.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.176+0000 7fb83176d700 1 --2- 172.21.15.157:0/2677469227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb82c06c2d0 0x7fb82c0c8320 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009f00 tx=0x7fb81c017000 comp rx=0 tx=0).stop 2023-10-06T03:57:35.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.176+0000 7fb83176d700 1 -- 172.21.15.157:0/2677469227 shutdown_connections 2023-10-06T03:57:35.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.176+0000 7fb83176d700 1 --2- 172.21.15.157:0/2677469227 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb818066f30 0x7fb8180693e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.176+0000 7fb83176d700 1 --2- 172.21.15.157:0/2677469227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb82c0b1e40 0x7fb82c169330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.176+0000 7fb83176d700 1 --2- 172.21.15.157:0/2677469227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb82c06c2d0 0x7fb82c0c8320 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.176+0000 7fb83176d700 1 --2- 172.21.15.157:0/2677469227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb82c0b04d0 0x7fb82c154fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.176+0000 7fb83176d700 1 -- 172.21.15.157:0/2677469227 >> 172.21.15.157:0/2677469227 conn(0x7fb82c0cc540 msgr2=0x7fb82c157440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:35.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.176+0000 7fb83176d700 1 -- 172.21.15.157:0/2677469227 shutdown_connections 2023-10-06T03:57:35.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.176+0000 7fb83176d700 1 -- 172.21.15.157:0/2677469227 wait complete. 2023-10-06T03:57:35.190 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:57:03.455121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:57:03.905661+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:57:08.377186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:57:09.709582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:57:35.191 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:57:35.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.306+0000 7f28b964f700 1 Processor -- start 2023-10-06T03:57:35.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.306+0000 7f28b964f700 1 -- start start 2023-10-06T03:57:35.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.307+0000 7f28b964f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28b406cbb0 0x7f28b406d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:35.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.307+0000 7f28b964f700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f28b406c1f0 0x7f28b406da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:35.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.307+0000 7f28b964f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f28b406cf80 0x7f28b414fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:35.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.307+0000 7f28b964f700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28b40c8c40 con 0x7f28b406c1f0 2023-10-06T03:57:35.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.307+0000 7f28b964f700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28b40c9780 con 0x7f28b406cf80 2023-10-06T03:57:35.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.307+0000 7f28b964f700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b406e0c0 con 0x7f28b406cbb0 2023-10-06T03:57:35.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.307+0000 7f28b27fc700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f28b406c1f0 0x7f28b406da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:35.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.307+0000 7f28b27fc700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f28b406c1f0 0x7f28b406da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35606/0 (socket says 172.21.15.157:35606) 2023-10-06T03:57:35.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.307+0000 7f28b27fc700 1 -- 172.21.15.157:0/105981080 learned_addr learned my addr 172.21.15.157:0/105981080 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:35.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.307+0000 7f28b37fe700 1 --2- 172.21.15.157:0/105981080 >> v2:172.21.15.175:3300/0 conn(0x7f28b406cf80 0x7f28b414fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:35.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.307+0000 7f28b2ffd700 1 --2- 172.21.15.157:0/105981080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28b406cbb0 0x7f28b406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:35.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.307+0000 7f28b27fc700 1 -- 172.21.15.157:0/105981080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28b406cbb0 msgr2=0x7f28b406d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:35.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.307+0000 7f28b27fc700 1 --2- 172.21.15.157:0/105981080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28b406cbb0 0x7f28b406d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.307+0000 7f28b27fc700 1 -- 172.21.15.157:0/105981080 >> v2:172.21.15.175:3300/0 conn(0x7f28b406cf80 msgr2=0x7f28b414fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:35.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.307+0000 7f28b27fc700 1 --2- 172.21.15.157:0/105981080 >> v2:172.21.15.175:3300/0 conn(0x7f28b406cf80 0x7f28b414fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.307+0000 7f28b27fc700 1 -- 172.21.15.157:0/105981080 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b40c8e50 con 0x7f28b406c1f0 2023-10-06T03:57:35.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.308+0000 7f28b27fc700 1 --2- 172.21.15.157:0/105981080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28b406c1f0 0x7f28b406da70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f28a80099b0 tx=0x7f28a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13bde2355fa519de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:35.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.308+0000 7f28b1ffb700 1 -- 172.21.15.157:0/105981080 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a800b8b0 con 0x7f28b406c1f0 2023-10-06T03:57:35.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.308+0000 7f28b1ffb700 1 -- 172.21.15.157:0/105981080 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28a800ba10 con 0x7f28b406c1f0 2023-10-06T03:57:35.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.308+0000 7f28b1ffb700 1 -- 172.21.15.157:0/105981080 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a800bdb0 con 0x7f28b406c1f0 2023-10-06T03:57:35.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.308+0000 7f28b964f700 1 -- 172.21.15.157:0/105981080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28b406c1f0 msgr2=0x7f28b406da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:35.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.308+0000 7f28b964f700 1 --2- 172.21.15.157:0/105981080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28b406c1f0 0x7f28b406da70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f28a80099b0 tx=0x7f28a8009250 comp rx=0 tx=0).stop 2023-10-06T03:57:35.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.308+0000 7f28b964f700 1 -- 172.21.15.157:0/105981080 shutdown_connections 2023-10-06T03:57:35.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.308+0000 7f28b964f700 1 --2- 172.21.15.157:0/105981080 >> v2:172.21.15.175:3300/0 conn(0x7f28b406cf80 0x7f28b414fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.308+0000 7f28b964f700 1 --2- 172.21.15.157:0/105981080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28b406cbb0 0x7f28b406d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.308+0000 7f28b964f700 1 --2- 172.21.15.157:0/105981080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28b406c1f0 0x7f28b406da70 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.308+0000 7f28b964f700 1 -- 172.21.15.157:0/105981080 >> 172.21.15.157:0/105981080 conn(0x7f28b406f230 msgr2=0x7f28b406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:35.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.308+0000 7f28b964f700 1 -- 172.21.15.157:0/105981080 shutdown_connections 2023-10-06T03:57:35.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.308+0000 7f28b964f700 1 -- 172.21.15.157:0/105981080 wait complete. 2023-10-06T03:57:35.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.310+0000 7f28b964f700 1 Processor -- start 2023-10-06T03:57:35.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.310+0000 7f28b964f700 1 -- start start 2023-10-06T03:57:35.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.310+0000 7f28b964f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28b406c1f0 0x7f28b40af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:35.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.310+0000 7f28b964f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28b406cbb0 0x7f28b4158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:35.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.310+0000 7f28b964f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28b406cf80 0x7f28b4158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:35.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.310+0000 7f28b964f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b40c1b00 con 0x7f28b406cf80 2023-10-06T03:57:35.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.310+0000 7f28b964f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b4158c50 con 0x7f28b406c1f0 2023-10-06T03:57:35.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.310+0000 7f28b964f700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b4158f10 con 0x7f28b406cbb0 2023-10-06T03:57:35.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.310+0000 7f28b37fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28b406cf80 0x7f28b4158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:35.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.310+0000 7f28b2ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28b406c1f0 0x7f28b40af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:35.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.310+0000 7f28b37fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28b406cf80 0x7f28b4158650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35614/0 (socket says 172.21.15.157:35614) 2023-10-06T03:57:35.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.310+0000 7f28b37fe700 1 -- 172.21.15.157:0/643235118 learned_addr learned my addr 172.21.15.157:0/643235118 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:35.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.310+0000 7f28b27fc700 1 --2- 172.21.15.157:0/643235118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28b406cbb0 0x7f28b4158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:35.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.311+0000 7f28b37fe700 1 -- 172.21.15.157:0/643235118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28b406cbb0 msgr2=0x7f28b4158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:35.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.311+0000 7f28b37fe700 1 --2- 172.21.15.157:0/643235118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28b406cbb0 0x7f28b4158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.311+0000 7f28b37fe700 1 -- 172.21.15.157:0/643235118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28b406c1f0 msgr2=0x7f28b40af3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:35.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.311+0000 7f28b37fe700 1 --2- 172.21.15.157:0/643235118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28b406c1f0 0x7f28b40af3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.311+0000 7f28b37fe700 1 -- 172.21.15.157:0/643235118 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a80094f0 con 0x7f28b406cf80 2023-10-06T03:57:35.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.311+0000 7f28b37fe700 1 --2- 172.21.15.157:0/643235118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28b406cf80 0x7f28b4158650 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f28ac00d860 tx=0x7f28ac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:35.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.311+0000 7f289affd700 1 -- 172.21.15.157:0/643235118 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac0125f0 con 0x7f28b406cf80 2023-10-06T03:57:35.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.311+0000 7f289affd700 1 -- 172.21.15.157:0/643235118 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28ac012770 con 0x7f28b406cf80 2023-10-06T03:57:35.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.311+0000 7f28b964f700 1 -- 172.21.15.157:0/643235118 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28b40c11d0 con 0x7f28b406cf80 2023-10-06T03:57:35.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.311+0000 7f289affd700 1 -- 172.21.15.157:0/643235118 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac012b30 con 0x7f28b406cf80 2023-10-06T03:57:35.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.311+0000 7f28b964f700 1 -- 172.21.15.157:0/643235118 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b40c1790 con 0x7f28b406cf80 2023-10-06T03:57:35.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.312+0000 7f28b964f700 1 -- 172.21.15.157:0/643235118 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28b40c1c70 con 0x7f28b406cf80 2023-10-06T03:57:35.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.312+0000 7f289affd700 1 -- 172.21.15.157:0/643235118 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f28ac01e460 con 0x7f28b406cf80 2023-10-06T03:57:35.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.313+0000 7f289affd700 1 --2- 172.21.15.157:0/643235118 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f28a0066f30 0x7f28a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:35.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.313+0000 7f289affd700 1 -- 172.21.15.157:0/643235118 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f28ac084c30 con 0x7f28b406cf80 2023-10-06T03:57:35.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.313+0000 7f28b2ffd700 1 --2- 172.21.15.157:0/643235118 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f28a0066f30 0x7f28a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:35.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.313+0000 7f28b2ffd700 1 --2- 172.21.15.157:0/643235118 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f28a0066f30 0x7f28a00693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f28a4009fd0 tx=0x7f28a4009880 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:35.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.318+0000 7f289affd700 1 -- 172.21.15.157:0/643235118 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28ac051d40 con 0x7f28b406cf80 2023-10-06T03:57:35.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.503+0000 7f28b964f700 1 -- 172.21.15.157:0/643235118 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f28b40c9130 con 0x7f28b406cf80 2023-10-06T03:57:35.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.505+0000 7f289affd700 1 -- 172.21.15.157:0/643235118 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f28ac055360 con 0x7f28b406cf80 2023-10-06T03:57:35.507 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-10-06T03:57:35.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.507+0000 7f28b964f700 1 -- 172.21.15.157:0/643235118 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f28a0066f30 msgr2=0x7f28a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:35.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.507+0000 7f28b964f700 1 --2- 172.21.15.157:0/643235118 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f28a0066f30 0x7f28a00693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f28a4009fd0 tx=0x7f28a4009880 comp rx=0 tx=0).stop 2023-10-06T03:57:35.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.507+0000 7f28b964f700 1 -- 172.21.15.157:0/643235118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28b406cf80 msgr2=0x7f28b4158650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:35.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.507+0000 7f28b964f700 1 --2- 172.21.15.157:0/643235118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28b406cf80 0x7f28b4158650 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f28ac00d860 tx=0x7f28ac008040 comp rx=0 tx=0).stop 2023-10-06T03:57:35.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.507+0000 7f28b964f700 1 -- 172.21.15.157:0/643235118 shutdown_connections 2023-10-06T03:57:35.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.507+0000 7f28b964f700 1 --2- 172.21.15.157:0/643235118 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f28a0066f30 0x7f28a00693e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.507+0000 7f28b964f700 1 --2- 172.21.15.157:0/643235118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28b406c1f0 0x7f28b40af3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.507+0000 7f28b964f700 1 --2- 172.21.15.157:0/643235118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28b406cf80 0x7f28b4158650 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.507+0000 7f28b964f700 1 --2- 172.21.15.157:0/643235118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28b406cbb0 0x7f28b4158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.507+0000 7f28b964f700 1 -- 172.21.15.157:0/643235118 >> 172.21.15.157:0/643235118 conn(0x7f28b406f230 msgr2=0x7f28b414ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:35.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.507+0000 7f28b964f700 1 -- 172.21.15.157:0/643235118 shutdown_connections 2023-10-06T03:57:35.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.507+0000 7f28b964f700 1 -- 172.21.15.157:0/643235118 wait complete. 2023-10-06T03:57:35.520 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:57:35.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.634+0000 7f5d5a2ab700 1 Processor -- start 2023-10-06T03:57:35.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.634+0000 7f5d5a2ab700 1 -- start start 2023-10-06T03:57:35.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.635+0000 7f5d5a2ab700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d540c81b0 0x7f5d54156a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:35.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.635+0000 7f5d5a2ab700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5d540c97d0 0x7f5d540c4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:35.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.635+0000 7f5d5a2ab700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5d54157490 0x7f5d54158c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:35.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.635+0000 7f5d5a2ab700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d541507c0 con 0x7f5d540c97d0 2023-10-06T03:57:35.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.635+0000 7f5d5a2ab700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d540c60e0 con 0x7f5d54157490 2023-10-06T03:57:35.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.635+0000 7f5d5a2ab700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540c40b0 con 0x7f5d540c81b0 2023-10-06T03:57:35.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.635+0000 7f5d537fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5d540c97d0 0x7f5d540c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:35.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.635+0000 7f5d537fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5d540c97d0 0x7f5d540c4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35618/0 (socket says 172.21.15.157:35618) 2023-10-06T03:57:35.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.635+0000 7f5d537fe700 1 -- 172.21.15.157:0/2005192876 learned_addr learned my addr 172.21.15.157:0/2005192876 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:35.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.635+0000 7f5d53fff700 1 --2- 172.21.15.157:0/2005192876 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d540c81b0 0x7f5d54156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:35.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.635+0000 7f5d58848700 1 --2- 172.21.15.157:0/2005192876 >> v2:172.21.15.175:3300/0 conn(0x7f5d54157490 0x7f5d54158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:35.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.636+0000 7f5d537fe700 1 -- 172.21.15.157:0/2005192876 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d540c81b0 msgr2=0x7f5d54156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:35.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.636+0000 7f5d537fe700 1 --2- 172.21.15.157:0/2005192876 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d540c81b0 0x7f5d54156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.636+0000 7f5d537fe700 1 -- 172.21.15.157:0/2005192876 >> v2:172.21.15.175:3300/0 conn(0x7f5d54157490 msgr2=0x7f5d54158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:35.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.636+0000 7f5d537fe700 1 --2- 172.21.15.157:0/2005192876 >> v2:172.21.15.175:3300/0 conn(0x7f5d54157490 0x7f5d54158c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.636+0000 7f5d537fe700 1 -- 172.21.15.157:0/2005192876 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d540c4a80 con 0x7f5d540c97d0 2023-10-06T03:57:35.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.636+0000 7f5d53fff700 1 --2- 172.21.15.157:0/2005192876 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d540c81b0 0x7f5d54156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:35.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.636+0000 7f5d537fe700 1 --2- 172.21.15.157:0/2005192876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d540c97d0 0x7f5d540c4540 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5d540c3580 tx=0x7f5d48009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6955519c9b6e6107 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:35.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.636+0000 7f5d52ffd700 1 -- 172.21.15.157:0/2005192876 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d48009e70 con 0x7f5d540c97d0 2023-10-06T03:57:35.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.636+0000 7f5d52ffd700 1 -- 172.21.15.157:0/2005192876 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4800b8f0 con 0x7f5d540c97d0 2023-10-06T03:57:35.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.636+0000 7f5d52ffd700 1 -- 172.21.15.157:0/2005192876 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4800bc90 con 0x7f5d540c97d0 2023-10-06T03:57:35.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.636+0000 7f5d5a2ab700 1 -- 172.21.15.157:0/2005192876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d540c97d0 msgr2=0x7f5d540c4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:35.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.636+0000 7f5d5a2ab700 1 --2- 172.21.15.157:0/2005192876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d540c97d0 0x7f5d540c4540 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5d540c3580 tx=0x7f5d48009250 comp rx=0 tx=0).stop 2023-10-06T03:57:35.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.636+0000 7f5d5a2ab700 1 -- 172.21.15.157:0/2005192876 shutdown_connections 2023-10-06T03:57:35.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.636+0000 7f5d5a2ab700 1 --2- 172.21.15.157:0/2005192876 >> v2:172.21.15.175:3300/0 conn(0x7f5d54157490 0x7f5d54158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.636+0000 7f5d5a2ab700 1 --2- 172.21.15.157:0/2005192876 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d540c81b0 0x7f5d54156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.636+0000 7f5d5a2ab700 1 --2- 172.21.15.157:0/2005192876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d540c97d0 0x7f5d540c4540 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.636+0000 7f5d5a2ab700 1 -- 172.21.15.157:0/2005192876 >> 172.21.15.157:0/2005192876 conn(0x7f5d540cd0b0 msgr2=0x7f5d540c9050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:35.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.636+0000 7f5d5a2ab700 1 -- 172.21.15.157:0/2005192876 shutdown_connections 2023-10-06T03:57:35.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.636+0000 7f5d5a2ab700 1 -- 172.21.15.157:0/2005192876 wait complete. 2023-10-06T03:57:35.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.637+0000 7f5d5a2ab700 1 Processor -- start 2023-10-06T03:57:35.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.638+0000 7f5d5a2ab700 1 -- start start 2023-10-06T03:57:35.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.638+0000 7f5d5a2ab700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d540c81b0 0x7f5d54169590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:35.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.638+0000 7f5d5a2ab700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d540c97d0 0x7f5d541672b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:35.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.638+0000 7f5d5a2ab700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d54157490 0x7f5d5415aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:35.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.638+0000 7f5d5a2ab700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540a7f30 con 0x7f5d540c97d0 2023-10-06T03:57:35.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.638+0000 7f5d5a2ab700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540c2e40 con 0x7f5d54157490 2023-10-06T03:57:35.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.638+0000 7f5d5a2ab700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540c32d0 con 0x7f5d540c81b0 2023-10-06T03:57:35.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.638+0000 7f5d537fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d540c97d0 0x7f5d541672b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:35.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.638+0000 7f5d537fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d540c97d0 0x7f5d541672b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35622/0 (socket says 172.21.15.157:35622) 2023-10-06T03:57:35.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.638+0000 7f5d537fe700 1 -- 172.21.15.157:0/353567695 learned_addr learned my addr 172.21.15.157:0/353567695 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:35.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.639+0000 7f5d53fff700 1 --2- 172.21.15.157:0/353567695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d540c81b0 0x7f5d54169590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:35.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.639+0000 7f5d58848700 1 --2- 172.21.15.157:0/353567695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d54157490 0x7f5d5415aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:35.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.639+0000 7f5d537fe700 1 -- 172.21.15.157:0/353567695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d540c81b0 msgr2=0x7f5d54169590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:35.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.639+0000 7f5d537fe700 1 --2- 172.21.15.157:0/353567695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d540c81b0 0x7f5d54169590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.639+0000 7f5d537fe700 1 -- 172.21.15.157:0/353567695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d54157490 msgr2=0x7f5d5415aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:35.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.639+0000 7f5d537fe700 1 --2- 172.21.15.157:0/353567695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d54157490 0x7f5d5415aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.639+0000 7f5d537fe700 1 -- 172.21.15.157:0/353567695 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d480094f0 con 0x7f5d540c97d0 2023-10-06T03:57:35.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.639+0000 7f5d58848700 1 --2- 172.21.15.157:0/353567695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d54157490 0x7f5d5415aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:35.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.639+0000 7f5d53fff700 1 --2- 172.21.15.157:0/353567695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d540c81b0 0x7f5d54169590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:35.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.639+0000 7f5d537fe700 1 --2- 172.21.15.157:0/353567695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d540c97d0 0x7f5d541672b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5d48009290 tx=0x7f5d48017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:35.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.639+0000 7f5d3bfff700 1 -- 172.21.15.157:0/353567695 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d48026480 con 0x7f5d540c97d0 2023-10-06T03:57:35.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.639+0000 7f5d3bfff700 1 -- 172.21.15.157:0/353567695 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4801f070 con 0x7f5d540c97d0 2023-10-06T03:57:35.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.639+0000 7f5d5a2ab700 1 -- 172.21.15.157:0/353567695 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d5406c3a0 con 0x7f5d540c97d0 2023-10-06T03:57:35.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.639+0000 7f5d3bfff700 1 -- 172.21.15.157:0/353567695 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d48026850 con 0x7f5d540c97d0 2023-10-06T03:57:35.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.639+0000 7f5d5a2ab700 1 -- 172.21.15.157:0/353567695 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d54169ef0 con 0x7f5d540c97d0 2023-10-06T03:57:35.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.640+0000 7f5d5a2ab700 1 -- 172.21.15.157:0/353567695 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d540c2ff0 con 0x7f5d540c97d0 2023-10-06T03:57:35.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.641+0000 7f5d3bfff700 1 -- 172.21.15.157:0/353567695 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5d4801a430 con 0x7f5d540c97d0 2023-10-06T03:57:35.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.641+0000 7f5d3bfff700 1 --2- 172.21.15.157:0/353567695 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5d40066f30 0x7f5d400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:35.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.641+0000 7f5d3bfff700 1 -- 172.21.15.157:0/353567695 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5d48021070 con 0x7f5d540c97d0 2023-10-06T03:57:35.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.641+0000 7f5d53fff700 1 --2- 172.21.15.157:0/353567695 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5d40066f30 0x7f5d400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:35.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.642+0000 7f5d53fff700 1 --2- 172.21.15.157:0/353567695 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5d40066f30 0x7f5d400693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5d44006f00 tx=0x7f5d44008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:35.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.647+0000 7f5d3bfff700 1 -- 172.21.15.157:0/353567695 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5d48053590 con 0x7f5d540c97d0 2023-10-06T03:57:35.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.831+0000 7f5d5a2ab700 1 -- 172.21.15.157:0/353567695 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5d540c2ff0 con 0x7f5d540c97d0 2023-10-06T03:57:35.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.831+0000 7f5d3bfff700 1 -- 172.21.15.157:0/353567695 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5d540c2ff0 con 0x7f5d540c97d0 2023-10-06T03:57:35.834 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-10-06T03:57:35.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.833+0000 7f5d5a2ab700 1 -- 172.21.15.157:0/353567695 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5d40066f30 msgr2=0x7f5d400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:35.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.833+0000 7f5d5a2ab700 1 --2- 172.21.15.157:0/353567695 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5d40066f30 0x7f5d400693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5d44006f00 tx=0x7f5d44008040 comp rx=0 tx=0).stop 2023-10-06T03:57:35.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.833+0000 7f5d5a2ab700 1 -- 172.21.15.157:0/353567695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d540c97d0 msgr2=0x7f5d541672b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:35.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.833+0000 7f5d5a2ab700 1 --2- 172.21.15.157:0/353567695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d540c97d0 0x7f5d541672b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5d48009290 tx=0x7f5d48017000 comp rx=0 tx=0).stop 2023-10-06T03:57:35.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.833+0000 7f5d5a2ab700 1 -- 172.21.15.157:0/353567695 shutdown_connections 2023-10-06T03:57:35.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.833+0000 7f5d5a2ab700 1 --2- 172.21.15.157:0/353567695 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5d40066f30 0x7f5d400693e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.833+0000 7f5d5a2ab700 1 --2- 172.21.15.157:0/353567695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d54157490 0x7f5d5415aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.833+0000 7f5d5a2ab700 1 --2- 172.21.15.157:0/353567695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d540c81b0 0x7f5d54169590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.833+0000 7f5d5a2ab700 1 --2- 172.21.15.157:0/353567695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d540c97d0 0x7f5d541672b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.833+0000 7f5d5a2ab700 1 -- 172.21.15.157:0/353567695 >> 172.21.15.157:0/353567695 conn(0x7f5d540cd0b0 msgr2=0x7f5d5406db30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:35.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.833+0000 7f5d5a2ab700 1 -- 172.21.15.157:0/353567695 shutdown_connections 2023-10-06T03:57:35.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.833+0000 7f5d5a2ab700 1 -- 172.21.15.157:0/353567695 wait complete. 2023-10-06T03:57:35.848 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:57:35.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.964+0000 7f32c3697700 1 Processor -- start 2023-10-06T03:57:35.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.964+0000 7f32c3697700 1 -- start start 2023-10-06T03:57:35.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.965+0000 7f32c3697700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f32bc0ca630 0x7f32bc0c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:35.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.965+0000 7f32c3697700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32bc068920 0x7f32bc156840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:35.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.965+0000 7f32c3697700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32bc06d530 con 0x7f32bc0c52e0 2023-10-06T03:57:35.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.965+0000 7f32c3697700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32bc06d9c0 con 0x7f32bc0ca630 2023-10-06T03:57:35.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.965+0000 7f32c3697700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc06f0c0 con 0x7f32bc068920 2023-10-06T03:57:35.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.965+0000 7f32c1433700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f32bc0ca630 0x7f32bc0c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:35.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.965+0000 7f32c1433700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f32bc0ca630 0x7f32bc0c6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35626/0 (socket says 172.21.15.157:35626) 2023-10-06T03:57:35.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.965+0000 7f32c1433700 1 -- 172.21.15.157:0/1995723091 learned_addr learned my addr 172.21.15.157:0/1995723091 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:35.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.965+0000 7f32c0c32700 1 --2- 172.21.15.157:0/1995723091 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32bc068920 0x7f32bc156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:35.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.966+0000 7f32c1433700 1 -- 172.21.15.157:0/1995723091 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32bc068920 msgr2=0x7f32bc156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:35.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.966+0000 7f32c1433700 1 --2- 172.21.15.157:0/1995723091 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32bc068920 0x7f32bc156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.966+0000 7f32c1433700 1 -- 172.21.15.157:0/1995723091 >> v1:172.21.15.175:6789/0 conn(0x7f32bc0c52e0 legacy=0x7f32bc159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:35.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.966+0000 7f32c1433700 1 -- 172.21.15.157:0/1995723091 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32bc06f3a0 con 0x7f32bc0ca630 2023-10-06T03:57:35.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.966+0000 7f32c1433700 1 --2- 172.21.15.157:0/1995723091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32bc0ca630 0x7f32bc0c6ef0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f32b80099b0 tx=0x7f32b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf061dd5590bb159 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:35.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.966+0000 7f32b3fff700 1 -- 172.21.15.157:0/1995723091 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b800b860 con 0x7f32bc0ca630 2023-10-06T03:57:35.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.966+0000 7f32b3fff700 1 -- 172.21.15.157:0/1995723091 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b800b9c0 con 0x7f32bc0ca630 2023-10-06T03:57:35.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.966+0000 7f32b3fff700 1 -- 172.21.15.157:0/1995723091 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b800bd60 con 0x7f32bc0ca630 2023-10-06T03:57:35.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.966+0000 7f32c3697700 1 -- 172.21.15.157:0/1995723091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32bc0ca630 msgr2=0x7f32bc0c6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:35.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.966+0000 7f32c3697700 1 --2- 172.21.15.157:0/1995723091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32bc0ca630 0x7f32bc0c6ef0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f32b80099b0 tx=0x7f32b8009250 comp rx=0 tx=0).stop 2023-10-06T03:57:35.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.966+0000 7f32c3697700 1 -- 172.21.15.157:0/1995723091 shutdown_connections 2023-10-06T03:57:35.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.966+0000 7f32c3697700 1 --2- 172.21.15.157:0/1995723091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32bc0ca630 0x7f32bc0c6ef0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.966+0000 7f32c3697700 1 --2- 172.21.15.157:0/1995723091 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32bc068920 0x7f32bc156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.966+0000 7f32c3697700 1 -- 172.21.15.157:0/1995723091 >> 172.21.15.157:0/1995723091 conn(0x7f32bc0692b0 msgr2=0x7f32bc0696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:35.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.966+0000 7f32c3697700 1 -- 172.21.15.157:0/1995723091 shutdown_connections 2023-10-06T03:57:35.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.967+0000 7f32c3697700 1 -- 172.21.15.157:0/1995723091 wait complete. 2023-10-06T03:57:35.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.968+0000 7f32c3697700 1 Processor -- start 2023-10-06T03:57:35.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.968+0000 7f32c3697700 1 -- start start 2023-10-06T03:57:35.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.968+0000 7f32c3697700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32bc068920 0x7f32bc0becb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:35.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.968+0000 7f32c3697700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32bc0c52e0 0x7f32bc152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:35.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.968+0000 7f32c3697700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32bc0ca630 0x7f32bc14eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:35.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.968+0000 7f32c3697700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc152880 con 0x7f32bc0c52e0 2023-10-06T03:57:35.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.968+0000 7f32c3697700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc0be170 con 0x7f32bc068920 2023-10-06T03:57:35.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.968+0000 7f32c3697700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc069aa0 con 0x7f32bc0ca630 2023-10-06T03:57:35.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.969+0000 7f32c0c32700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32bc0c52e0 0x7f32bc152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:35.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.969+0000 7f32c0c32700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32bc0c52e0 0x7f32bc152340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35630/0 (socket says 172.21.15.157:35630) 2023-10-06T03:57:35.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.969+0000 7f32c0c32700 1 -- 172.21.15.157:0/2367331295 learned_addr learned my addr 172.21.15.157:0/2367331295 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:35.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.969+0000 7f32c1433700 1 --2- 172.21.15.157:0/2367331295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32bc068920 0x7f32bc0becb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:35.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.969+0000 7f32c1c34700 1 --2- 172.21.15.157:0/2367331295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32bc0ca630 0x7f32bc14eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:35.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.969+0000 7f32c0c32700 1 -- 172.21.15.157:0/2367331295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32bc0ca630 msgr2=0x7f32bc14eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:35.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.969+0000 7f32c0c32700 1 --2- 172.21.15.157:0/2367331295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32bc0ca630 0x7f32bc14eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.969+0000 7f32c0c32700 1 -- 172.21.15.157:0/2367331295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32bc068920 msgr2=0x7f32bc0becb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:35.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.969+0000 7f32c0c32700 1 --2- 172.21.15.157:0/2367331295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32bc068920 0x7f32bc0becb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:35.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.969+0000 7f32c0c32700 1 -- 172.21.15.157:0/2367331295 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b80094f0 con 0x7f32bc0c52e0 2023-10-06T03:57:35.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.969+0000 7f32c0c32700 1 --2- 172.21.15.157:0/2367331295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32bc0c52e0 0x7f32bc152340 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00d890 tx=0x7f32ac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:35.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.969+0000 7f32b17fa700 1 -- 172.21.15.157:0/2367331295 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac00b690 con 0x7f32bc0c52e0 2023-10-06T03:57:35.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.969+0000 7f32b17fa700 1 -- 172.21.15.157:0/2367331295 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ac00b7f0 con 0x7f32bc0c52e0 2023-10-06T03:57:35.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.969+0000 7f32c3697700 1 -- 172.21.15.157:0/2367331295 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32bc0c0a40 con 0x7f32bc0c52e0 2023-10-06T03:57:35.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.969+0000 7f32b17fa700 1 -- 172.21.15.157:0/2367331295 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac00bb70 con 0x7f32bc0c52e0 2023-10-06T03:57:35.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.969+0000 7f32c3697700 1 -- 172.21.15.157:0/2367331295 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32bc0be380 con 0x7f32bc0c52e0 2023-10-06T03:57:35.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.970+0000 7f32c3697700 1 -- 172.21.15.157:0/2367331295 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32bc06a8f0 con 0x7f32bc0c52e0 2023-10-06T03:57:35.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.970+0000 7f32b17fa700 1 -- 172.21.15.157:0/2367331295 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f32ac01e460 con 0x7f32bc0c52e0 2023-10-06T03:57:35.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.971+0000 7f32b17fa700 1 --2- 172.21.15.157:0/2367331295 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f32a0066f60 0x7f32a0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:35.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.971+0000 7f32b17fa700 1 -- 172.21.15.157:0/2367331295 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f32ac084de0 con 0x7f32bc0c52e0 2023-10-06T03:57:35.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.971+0000 7f32c1433700 1 --2- 172.21.15.157:0/2367331295 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f32a0066f60 0x7f32a0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:35.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.971+0000 7f32c1433700 1 --2- 172.21.15.157:0/2367331295 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f32a0066f60 0x7f32a0069410 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f32b80099b0 tx=0x7f32b8017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:35.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:35.975+0000 7f32b17fa700 1 -- 172.21.15.157:0/2367331295 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32ac051ef0 con 0x7f32bc0c52e0 2023-10-06T03:57:36.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.156+0000 7f32c3697700 1 -- 172.21.15.157:0/2367331295 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f32bc0c6510 con 0x7f32bc0c52e0 2023-10-06T03:57:36.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.156+0000 7f32b17fa700 1 -- 172.21.15.157:0/2367331295 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f32ac055510 con 0x7f32bc0c52e0 2023-10-06T03:57:36.158 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-10-06T03:57:36.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.158+0000 7f32c3697700 1 -- 172.21.15.157:0/2367331295 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f32a0066f60 msgr2=0x7f32a0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:36.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.158+0000 7f32c3697700 1 --2- 172.21.15.157:0/2367331295 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f32a0066f60 0x7f32a0069410 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f32b80099b0 tx=0x7f32b8017000 comp rx=0 tx=0).stop 2023-10-06T03:57:36.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.158+0000 7f32c3697700 1 -- 172.21.15.157:0/2367331295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32bc0c52e0 msgr2=0x7f32bc152340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:36.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.158+0000 7f32c3697700 1 --2- 172.21.15.157:0/2367331295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32bc0c52e0 0x7f32bc152340 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00d890 tx=0x7f32ac008040 comp rx=0 tx=0).stop 2023-10-06T03:57:36.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.158+0000 7f32c3697700 1 -- 172.21.15.157:0/2367331295 shutdown_connections 2023-10-06T03:57:36.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.158+0000 7f32c3697700 1 --2- 172.21.15.157:0/2367331295 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f32a0066f60 0x7f32a0069410 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.158+0000 7f32c3697700 1 --2- 172.21.15.157:0/2367331295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32bc068920 0x7f32bc0becb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.158+0000 7f32c3697700 1 --2- 172.21.15.157:0/2367331295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32bc0ca630 0x7f32bc14eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.158+0000 7f32c3697700 1 --2- 172.21.15.157:0/2367331295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32bc0c52e0 0x7f32bc152340 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.158+0000 7f32c3697700 1 -- 172.21.15.157:0/2367331295 >> 172.21.15.157:0/2367331295 conn(0x7f32bc0692b0 msgr2=0x7f32bc14fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:36.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.158+0000 7f32c3697700 1 -- 172.21.15.157:0/2367331295 shutdown_connections 2023-10-06T03:57:36.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.158+0000 7f32c3697700 1 -- 172.21.15.157:0/2367331295 wait complete. 2023-10-06T03:57:36.172 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:57:36.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.289+0000 7f8ab37c6700 1 Processor -- start 2023-10-06T03:57:36.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.289+0000 7f8ab37c6700 1 -- start start 2023-10-06T03:57:36.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.289+0000 7f8ab37c6700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8aac06cbb0 0x7f8aac06d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:36.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.289+0000 7f8ab37c6700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8aac150410 0x7f8aac06da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:36.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.289+0000 7f8ab37c6700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aac0c9780 con 0x7f8aac06c1f0 2023-10-06T03:57:36.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.289+0000 7f8ab37c6700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aac0c8c40 con 0x7f8aac150410 2023-10-06T03:57:36.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.289+0000 7f8ab37c6700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aac06e0c0 con 0x7f8aac06cbb0 2023-10-06T03:57:36.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.290+0000 7f8ab1d63700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8aac150410 0x7f8aac06da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:36.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.290+0000 7f8ab0d61700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f8aac06c1f0 0x7f8aac14fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39680/0 (socket says 172.21.15.157:39680) 2023-10-06T03:57:36.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.290+0000 7f8ab0d61700 1 -- 172.21.15.157:0/3145032689 learned_addr learned my addr 172.21.15.157:0/3145032689 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:36.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.290+0000 7f8ab1562700 1 --2- 172.21.15.157:0/3145032689 >> v2:172.21.15.175:3300/0 conn(0x7f8aac06cbb0 0x7f8aac06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:36.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.290+0000 7f8ab1d63700 1 -- 172.21.15.157:0/3145032689 >> v2:172.21.15.175:3300/0 conn(0x7f8aac06cbb0 msgr2=0x7f8aac06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:36.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.290+0000 7f8ab1d63700 1 --2- 172.21.15.157:0/3145032689 >> v2:172.21.15.175:3300/0 conn(0x7f8aac06cbb0 0x7f8aac06d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.290+0000 7f8ab1d63700 1 -- 172.21.15.157:0/3145032689 >> v1:172.21.15.157:6789/0 conn(0x7f8aac06c1f0 legacy=0x7f8aac14fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:36.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.290+0000 7f8ab1d63700 1 -- 172.21.15.157:0/3145032689 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aac0c8e50 con 0x7f8aac150410 2023-10-06T03:57:36.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.290+0000 7f8aa3fff700 1 -- 172.21.15.157:0/3145032689 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 58024454 0 0) 0x7f8aac0c9780 con 0x7f8aac06c1f0 2023-10-06T03:57:36.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.290+0000 7f8ab1d63700 1 --2- 172.21.15.157:0/3145032689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aac150410 0x7f8aac06da70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8aac06fd70 tx=0x7f8aa40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=620b0fc400458778 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:36.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.290+0000 7f8aa3fff700 1 -- 172.21.15.157:0/3145032689 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa400b740 con 0x7f8aac150410 2023-10-06T03:57:36.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.290+0000 7f8aa3fff700 1 -- 172.21.15.157:0/3145032689 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa400b8a0 con 0x7f8aac150410 2023-10-06T03:57:36.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.290+0000 7f8aa3fff700 1 -- 172.21.15.157:0/3145032689 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa400bc40 con 0x7f8aac150410 2023-10-06T03:57:36.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.291+0000 7f8ab37c6700 1 -- 172.21.15.157:0/3145032689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aac150410 msgr2=0x7f8aac06da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:36.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.291+0000 7f8ab37c6700 1 --2- 172.21.15.157:0/3145032689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aac150410 0x7f8aac06da70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8aac06fd70 tx=0x7f8aa40091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.291+0000 7f8ab37c6700 1 -- 172.21.15.157:0/3145032689 shutdown_connections 2023-10-06T03:57:36.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.291+0000 7f8ab37c6700 1 --2- 172.21.15.157:0/3145032689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aac150410 0x7f8aac06da70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.291+0000 7f8ab37c6700 1 --2- 172.21.15.157:0/3145032689 >> v2:172.21.15.175:3300/0 conn(0x7f8aac06cbb0 0x7f8aac06d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.291+0000 7f8ab37c6700 1 -- 172.21.15.157:0/3145032689 >> 172.21.15.157:0/3145032689 conn(0x7f8aac06f230 msgr2=0x7f8aac06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:36.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.291+0000 7f8ab37c6700 1 -- 172.21.15.157:0/3145032689 shutdown_connections 2023-10-06T03:57:36.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.291+0000 7f8ab37c6700 1 -- 172.21.15.157:0/3145032689 wait complete. 2023-10-06T03:57:36.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.292+0000 7f8ab37c6700 1 Processor -- start 2023-10-06T03:57:36.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.292+0000 7f8ab37c6700 1 -- start start 2023-10-06T03:57:36.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.293+0000 7f8ab37c6700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8aac06c1f0 0x7f8aac166f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:36.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.293+0000 7f8ab37c6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aac06cbb0 0x7f8aac1699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:36.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.293+0000 7f8ab37c6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8aac150410 0x7f8aac06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:36.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.293+0000 7f8ab37c6700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac166b10 con 0x7f8aac06cbb0 2023-10-06T03:57:36.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.293+0000 7f8ab37c6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac1541e0 con 0x7f8aac150410 2023-10-06T03:57:36.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.293+0000 7f8ab37c6700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac154480 con 0x7f8aac06c1f0 2023-10-06T03:57:36.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.293+0000 7f8ab0d61700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aac06cbb0 0x7f8aac1699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:36.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.293+0000 7f8ab0d61700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aac06cbb0 0x7f8aac1699c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35638/0 (socket says 172.21.15.157:35638) 2023-10-06T03:57:36.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.293+0000 7f8ab0d61700 1 -- 172.21.15.157:0/1615175711 learned_addr learned my addr 172.21.15.157:0/1615175711 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:36.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.293+0000 7f8ab1d63700 1 --2- 172.21.15.157:0/1615175711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8aac150410 0x7f8aac06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:36.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.293+0000 7f8ab1562700 1 --2- 172.21.15.157:0/1615175711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8aac06c1f0 0x7f8aac166f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:36.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.293+0000 7f8ab0d61700 1 -- 172.21.15.157:0/1615175711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8aac06c1f0 msgr2=0x7f8aac166f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:36.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.293+0000 7f8ab0d61700 1 --2- 172.21.15.157:0/1615175711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8aac06c1f0 0x7f8aac166f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.293+0000 7f8ab0d61700 1 -- 172.21.15.157:0/1615175711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8aac150410 msgr2=0x7f8aac06d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:36.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.293+0000 7f8ab0d61700 1 --2- 172.21.15.157:0/1615175711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8aac150410 0x7f8aac06d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.293+0000 7f8ab0d61700 1 -- 172.21.15.157:0/1615175711 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa4009420 con 0x7f8aac06cbb0 2023-10-06T03:57:36.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.293+0000 7f8ab1562700 1 --2- 172.21.15.157:0/1615175711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8aac06c1f0 0x7f8aac166f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:57:36.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.293+0000 7f8ab1d63700 1 --2- 172.21.15.157:0/1615175711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8aac150410 0x7f8aac06d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:57:36.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.293+0000 7f8ab0d61700 1 --2- 172.21.15.157:0/1615175711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aac06cbb0 0x7f8aac1699c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00c7e0 tx=0x7f8a9c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:36.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.293+0000 7f8aa17fa700 1 -- 172.21.15.157:0/1615175711 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c005b30 con 0x7f8aac06cbb0 2023-10-06T03:57:36.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.293+0000 7f8aa17fa700 1 -- 172.21.15.157:0/1615175711 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a9c005c90 con 0x7f8aac06cbb0 2023-10-06T03:57:36.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.293+0000 7f8ab37c6700 1 -- 172.21.15.157:0/1615175711 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aac154720 con 0x7f8aac06cbb0 2023-10-06T03:57:36.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.293+0000 7f8aa17fa700 1 -- 172.21.15.157:0/1615175711 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c007300 con 0x7f8aac06cbb0 2023-10-06T03:57:36.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.293+0000 7f8ab37c6700 1 -- 172.21.15.157:0/1615175711 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aac154cc0 con 0x7f8aac06cbb0 2023-10-06T03:57:36.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.294+0000 7f8ab37c6700 1 -- 172.21.15.157:0/1615175711 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aac166c50 con 0x7f8aac06cbb0 2023-10-06T03:57:36.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.295+0000 7f8aa17fa700 1 -- 172.21.15.157:0/1615175711 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8a9c01b400 con 0x7f8aac06cbb0 2023-10-06T03:57:36.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.295+0000 7f8aa17fa700 1 --2- 172.21.15.157:0/1615175711 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8a90066f60 0x7f8a90069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:36.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.295+0000 7f8aa17fa700 1 -- 172.21.15.157:0/1615175711 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8a9c0841e0 con 0x7f8aac06cbb0 2023-10-06T03:57:36.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.295+0000 7f8ab1562700 1 --2- 172.21.15.157:0/1615175711 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8a90066f60 0x7f8a90069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:36.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.296+0000 7f8ab1562700 1 --2- 172.21.15.157:0/1615175711 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8a90066f60 0x7f8a90069410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8007540 tx=0x7f8aa800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:36.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.299+0000 7f8aa17fa700 1 -- 172.21.15.157:0/1615175711 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8aac166c50 con 0x7f8aac06cbb0 2023-10-06T03:57:36.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.482+0000 7f8ab37c6700 1 -- 172.21.15.157:0/1615175711 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8aac166c50 con 0x7f8aac06cbb0 2023-10-06T03:57:36.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.482+0000 7f8aa17fa700 1 -- 172.21.15.157:0/1615175711 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8aac166c50 con 0x7f8aac06cbb0 2023-10-06T03:57:36.484 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-10-06T03:57:36.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.484+0000 7f8ab37c6700 1 -- 172.21.15.157:0/1615175711 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8a90066f60 msgr2=0x7f8a90069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:36.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.484+0000 7f8ab37c6700 1 --2- 172.21.15.157:0/1615175711 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8a90066f60 0x7f8a90069410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8007540 tx=0x7f8aa800b040 comp rx=0 tx=0).stop 2023-10-06T03:57:36.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.484+0000 7f8ab37c6700 1 -- 172.21.15.157:0/1615175711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aac06cbb0 msgr2=0x7f8aac1699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:36.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.484+0000 7f8ab37c6700 1 --2- 172.21.15.157:0/1615175711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aac06cbb0 0x7f8aac1699c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00c7e0 tx=0x7f8a9c007df0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.484+0000 7f8ab37c6700 1 -- 172.21.15.157:0/1615175711 shutdown_connections 2023-10-06T03:57:36.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.484+0000 7f8ab37c6700 1 --2- 172.21.15.157:0/1615175711 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8a90066f60 0x7f8a90069410 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.484+0000 7f8ab37c6700 1 --2- 172.21.15.157:0/1615175711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8aac150410 0x7f8aac06d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.484+0000 7f8ab37c6700 1 --2- 172.21.15.157:0/1615175711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8aac06c1f0 0x7f8aac166f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.484+0000 7f8ab37c6700 1 --2- 172.21.15.157:0/1615175711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8aac06cbb0 0x7f8aac1699c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.484+0000 7f8ab37c6700 1 -- 172.21.15.157:0/1615175711 >> 172.21.15.157:0/1615175711 conn(0x7f8aac06f230 msgr2=0x7f8aac14df40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:36.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.484+0000 7f8ab37c6700 1 -- 172.21.15.157:0/1615175711 shutdown_connections 2023-10-06T03:57:36.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.484+0000 7f8ab37c6700 1 -- 172.21.15.157:0/1615175711 wait complete. 2023-10-06T03:57:36.497 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:57:36.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.611+0000 7fd411ccb700 1 Processor -- start 2023-10-06T03:57:36.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.611+0000 7fd411ccb700 1 -- start start 2023-10-06T03:57:36.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.612+0000 7fd411ccb700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd40c0c8b60 0x7fd40c155b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:36.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.612+0000 7fd411ccb700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd40c0cc050 0x7fd40c0c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:36.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.612+0000 7fd411ccb700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd40c0af480 con 0x7fd40c0c8790 2023-10-06T03:57:36.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.612+0000 7fd411ccb700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd40c0af760 con 0x7fd40c0c8b60 2023-10-06T03:57:36.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.612+0000 7fd411ccb700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd40c0afa40 con 0x7fd40c0cc050 2023-10-06T03:57:36.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.612+0000 7fd40bfff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd40c0c8b60 0x7fd40c155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:36.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.612+0000 7fd410cc9700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd40c0c8790 0x7fd40c0b1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39686/0 (socket says 172.21.15.157:39686) 2023-10-06T03:57:36.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.612+0000 7fd40bfff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd40c0c8b60 0x7fd40c155b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35646/0 (socket says 172.21.15.157:35646) 2023-10-06T03:57:36.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.612+0000 7fd410cc9700 1 -- 172.21.15.157:0/1191181704 learned_addr learned my addr 172.21.15.157:0/1191181704 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:36.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.612+0000 7fd4114ca700 1 --2- 172.21.15.157:0/1191181704 >> v2:172.21.15.175:3300/0 conn(0x7fd40c0cc050 0x7fd40c0c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:36.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.612+0000 7fd40bfff700 1 -- 172.21.15.157:0/1191181704 >> v2:172.21.15.175:3300/0 conn(0x7fd40c0cc050 msgr2=0x7fd40c0c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:36.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.612+0000 7fd40bfff700 1 --2- 172.21.15.157:0/1191181704 >> v2:172.21.15.175:3300/0 conn(0x7fd40c0cc050 0x7fd40c0c75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.612+0000 7fd40bfff700 1 -- 172.21.15.157:0/1191181704 >> v1:172.21.15.157:6789/0 conn(0x7fd40c0c8790 legacy=0x7fd40c0b1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:36.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.612+0000 7fd40bfff700 1 -- 172.21.15.157:0/1191181704 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c0afd20 con 0x7fd40c0c8b60 2023-10-06T03:57:36.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.612+0000 7fd40bfff700 1 --2- 172.21.15.157:0/1191181704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd40c0c8b60 0x7fd40c155b60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd40c156c40 tx=0x7fd3f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d54581e039aba0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:36.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.612+0000 7fd40b7fe700 1 -- 172.21.15.157:0/1191181704 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f800b6b0 con 0x7fd40c0c8b60 2023-10-06T03:57:36.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.613+0000 7fd40b7fe700 1 -- 172.21.15.157:0/1191181704 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f800b810 con 0x7fd40c0c8b60 2023-10-06T03:57:36.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.613+0000 7fd40b7fe700 1 -- 172.21.15.157:0/1191181704 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f800bbb0 con 0x7fd40c0c8b60 2023-10-06T03:57:36.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.613+0000 7fd411ccb700 1 -- 172.21.15.157:0/1191181704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd40c0c8b60 msgr2=0x7fd40c155b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:36.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.613+0000 7fd411ccb700 1 --2- 172.21.15.157:0/1191181704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd40c0c8b60 0x7fd40c155b60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd40c156c40 tx=0x7fd3f80091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.613+0000 7fd411ccb700 1 -- 172.21.15.157:0/1191181704 shutdown_connections 2023-10-06T03:57:36.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.613+0000 7fd411ccb700 1 --2- 172.21.15.157:0/1191181704 >> v2:172.21.15.175:3300/0 conn(0x7fd40c0cc050 0x7fd40c0c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.613+0000 7fd411ccb700 1 --2- 172.21.15.157:0/1191181704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd40c0c8b60 0x7fd40c155b60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.613+0000 7fd411ccb700 1 -- 172.21.15.157:0/1191181704 >> 172.21.15.157:0/1191181704 conn(0x7fd40c0cc540 msgr2=0x7fd40c0cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:36.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.613+0000 7fd411ccb700 1 -- 172.21.15.157:0/1191181704 shutdown_connections 2023-10-06T03:57:36.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.613+0000 7fd411ccb700 1 -- 172.21.15.157:0/1191181704 wait complete. 2023-10-06T03:57:36.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.614+0000 7fd411ccb700 1 Processor -- start 2023-10-06T03:57:36.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.614+0000 7fd411ccb700 1 -- start start 2023-10-06T03:57:36.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.615+0000 7fd411ccb700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd40c0c8790 0x7fd40c169f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:36.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.615+0000 7fd411ccb700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd40c0c8b60 0x7fd40c0c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:36.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.615+0000 7fd411ccb700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd40c0cc050 0x7fd40c0a86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:36.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.615+0000 7fd411ccb700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c0c32e0 con 0x7fd40c0c8b60 2023-10-06T03:57:36.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.615+0000 7fd411ccb700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c0c2d20 con 0x7fd40c0cc050 2023-10-06T03:57:36.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.615+0000 7fd411ccb700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c0c3000 con 0x7fd40c0c8790 2023-10-06T03:57:36.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.615+0000 7fd40bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd40c0c8b60 0x7fd40c0c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:36.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.615+0000 7fd40bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd40c0c8b60 0x7fd40c0c5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35660/0 (socket says 172.21.15.157:35660) 2023-10-06T03:57:36.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.615+0000 7fd40bfff700 1 -- 172.21.15.157:0/3941836531 learned_addr learned my addr 172.21.15.157:0/3941836531 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:36.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.615+0000 7fd4114ca700 1 --2- 172.21.15.157:0/3941836531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd40c0cc050 0x7fd40c0a86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:36.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.615+0000 7fd410cc9700 1 --2- 172.21.15.157:0/3941836531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd40c0c8790 0x7fd40c169f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:36.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.615+0000 7fd40bfff700 1 -- 172.21.15.157:0/3941836531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd40c0c8790 msgr2=0x7fd40c169f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:36.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.615+0000 7fd40bfff700 1 --2- 172.21.15.157:0/3941836531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd40c0c8790 0x7fd40c169f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.615+0000 7fd40bfff700 1 -- 172.21.15.157:0/3941836531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd40c0cc050 msgr2=0x7fd40c0a86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:36.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.615+0000 7fd40bfff700 1 --2- 172.21.15.157:0/3941836531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd40c0cc050 0x7fd40c0a86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.615+0000 7fd40bfff700 1 -- 172.21.15.157:0/3941836531 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f8009420 con 0x7fd40c0c8b60 2023-10-06T03:57:36.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.615+0000 7fd410cc9700 1 --2- 172.21.15.157:0/3941836531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd40c0c8790 0x7fd40c169f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:57:36.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.616+0000 7fd40bfff700 1 --2- 172.21.15.157:0/3941836531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd40c0c8b60 0x7fd40c0c5950 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd3f80097b0 tx=0x7fd3f8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:36.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.616+0000 7fd408ff9700 1 -- 172.21.15.157:0/3941836531 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f800bb90 con 0x7fd40c0c8b60 2023-10-06T03:57:36.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.616+0000 7fd408ff9700 1 -- 172.21.15.157:0/3941836531 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f8011420 con 0x7fd40c0c8b60 2023-10-06T03:57:36.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.616+0000 7fd411ccb700 1 -- 172.21.15.157:0/3941836531 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40c0a7bc0 con 0x7fd40c0c8b60 2023-10-06T03:57:36.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.616+0000 7fd408ff9700 1 -- 172.21.15.157:0/3941836531 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f80117c0 con 0x7fd40c0c8b60 2023-10-06T03:57:36.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.616+0000 7fd411ccb700 1 -- 172.21.15.157:0/3941836531 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40c0a8180 con 0x7fd40c0c8b60 2023-10-06T03:57:36.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.617+0000 7fd411ccb700 1 -- 172.21.15.157:0/3941836531 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd40c0b1140 con 0x7fd40c0c8b60 2023-10-06T03:57:36.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.617+0000 7fd408ff9700 1 -- 172.21.15.157:0/3941836531 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd3f8011bf0 con 0x7fd40c0c8b60 2023-10-06T03:57:36.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.618+0000 7fd408ff9700 1 --2- 172.21.15.157:0/3941836531 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd3f0066f30 0x7fd3f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:36.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.618+0000 7fd408ff9700 1 -- 172.21.15.157:0/3941836531 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd3f8086160 con 0x7fd40c0c8b60 2023-10-06T03:57:36.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.618+0000 7fd410cc9700 1 --2- 172.21.15.157:0/3941836531 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd3f0066f30 0x7fd3f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:36.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.618+0000 7fd410cc9700 1 --2- 172.21.15.157:0/3941836531 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd3f0066f30 0x7fd3f00693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd400001780 tx=0x7fd400009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:36.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.622+0000 7fd408ff9700 1 -- 172.21.15.157:0/3941836531 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd3f8053270 con 0x7fd40c0c8b60 2023-10-06T03:57:36.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.851+0000 7fd411ccb700 1 -- 172.21.15.157:0/3941836531 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd40c04bb40 con 0x7fd40c0c8b60 2023-10-06T03:57:36.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.851+0000 7fd408ff9700 1 -- 172.21.15.157:0/3941836531 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6185 (secure 0 0 0) 0x7fd3f8056890 con 0x7fd40c0c8b60 2023-10-06T03:57:36.853 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:36.853 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T03:57:32.634356+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"f","rank":4,"incarnation":15,"state":"up:creating","state_seq":1,"addr":"172.21.15.175:6839/4177493830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4177493830},{"type":"v1","addr":"172.21.15.175:6839","nonce":4177493830}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:57:36.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.854+0000 7fd411ccb700 1 -- 172.21.15.157:0/3941836531 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd3f0066f30 msgr2=0x7fd3f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:36.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.854+0000 7fd411ccb700 1 --2- 172.21.15.157:0/3941836531 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd3f0066f30 0x7fd3f00693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd400001780 tx=0x7fd400009040 comp rx=0 tx=0).stop 2023-10-06T03:57:36.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.854+0000 7fd411ccb700 1 -- 172.21.15.157:0/3941836531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd40c0c8b60 msgr2=0x7fd40c0c5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:36.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.854+0000 7fd411ccb700 1 --2- 172.21.15.157:0/3941836531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd40c0c8b60 0x7fd40c0c5950 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd3f80097b0 tx=0x7fd3f8017000 comp rx=0 tx=0).stop 2023-10-06T03:57:36.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.854+0000 7fd411ccb700 1 -- 172.21.15.157:0/3941836531 shutdown_connections 2023-10-06T03:57:36.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.854+0000 7fd411ccb700 1 --2- 172.21.15.157:0/3941836531 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd3f0066f30 0x7fd3f00693e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.854+0000 7fd411ccb700 1 --2- 172.21.15.157:0/3941836531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd40c0cc050 0x7fd40c0a86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.854+0000 7fd411ccb700 1 --2- 172.21.15.157:0/3941836531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd40c0c8790 0x7fd40c169f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.854+0000 7fd411ccb700 1 --2- 172.21.15.157:0/3941836531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd40c0c8b60 0x7fd40c0c5950 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.854+0000 7fd411ccb700 1 -- 172.21.15.157:0/3941836531 >> 172.21.15.157:0/3941836531 conn(0x7fd40c0cc540 msgr2=0x7fd40c155430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:36.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.854+0000 7fd411ccb700 1 -- 172.21.15.157:0/3941836531 shutdown_connections 2023-10-06T03:57:36.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.854+0000 7fd411ccb700 1 -- 172.21.15.157:0/3941836531 wait complete. 2023-10-06T03:57:36.857 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 15 2023-10-06T03:57:36.867 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:57:36.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.980+0000 7f729ffff700 1 Processor -- start 2023-10-06T03:57:36.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.980+0000 7f729ffff700 1 -- start start 2023-10-06T03:57:36.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.981+0000 7f729ffff700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72a0067d40 0x7f72a0152e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:36.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.981+0000 7f729ffff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f72a00b0030 0x7f72a0153390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:36.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.981+0000 7f729ffff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72a006d3a0 con 0x7f72a00c5ca0 2023-10-06T03:57:36.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.981+0000 7f729ffff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72a006ed70 con 0x7f72a00b0030 2023-10-06T03:57:36.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.981+0000 7f729ffff700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a00c7930 con 0x7f72a0067d40 2023-10-06T03:57:36.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.981+0000 7f729f7fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f72a00b0030 0x7f72a0153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:36.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.981+0000 7f729e7fc700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f72a00c5ca0 0x7f72a0159600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39700/0 (socket says 172.21.15.157:39700) 2023-10-06T03:57:36.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.981+0000 7f729e7fc700 1 -- 172.21.15.157:0/4090684291 learned_addr learned my addr 172.21.15.157:0/4090684291 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:36.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.981+0000 7f729effd700 1 --2- 172.21.15.157:0/4090684291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72a0067d40 0x7f72a0152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:36.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.981+0000 7f729f7fe700 1 -- 172.21.15.157:0/4090684291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72a0067d40 msgr2=0x7f72a0152e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:36.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.981+0000 7f729f7fe700 1 --2- 172.21.15.157:0/4090684291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72a0067d40 0x7f72a0152e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.981+0000 7f729f7fe700 1 -- 172.21.15.157:0/4090684291 >> v1:172.21.15.157:6789/0 conn(0x7f72a00c5ca0 legacy=0x7f72a0159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:36.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.981+0000 7f729f7fe700 1 -- 172.21.15.157:0/4090684291 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a006d610 con 0x7f72a00b0030 2023-10-06T03:57:36.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.981+0000 7f729dffb700 1 -- 172.21.15.157:0/4090684291 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1086754126 0 0) 0x7f72a006d3a0 con 0x7f72a00c5ca0 2023-10-06T03:57:36.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.982+0000 7f729f7fe700 1 --2- 172.21.15.157:0/4090684291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a00b0030 0x7f72a0153390 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f72980098e0 tx=0x7f72980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431f54b7ef69aefd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:36.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.982+0000 7f729dffb700 1 -- 172.21.15.157:0/4090684291 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7298009e80 con 0x7f72a00b0030 2023-10-06T03:57:36.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.982+0000 7f729dffb700 1 -- 172.21.15.157:0/4090684291 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729800b860 con 0x7f72a00b0030 2023-10-06T03:57:36.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.982+0000 7f729dffb700 1 -- 172.21.15.157:0/4090684291 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729800bbe0 con 0x7f72a00b0030 2023-10-06T03:57:36.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.982+0000 7f729ffff700 1 -- 172.21.15.157:0/4090684291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a00b0030 msgr2=0x7f72a0153390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:36.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.982+0000 7f729ffff700 1 --2- 172.21.15.157:0/4090684291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a00b0030 0x7f72a0153390 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f72980098e0 tx=0x7f72980091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.982+0000 7f729ffff700 1 -- 172.21.15.157:0/4090684291 shutdown_connections 2023-10-06T03:57:36.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.982+0000 7f729ffff700 1 --2- 172.21.15.157:0/4090684291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a00b0030 0x7f72a0153390 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.982+0000 7f729ffff700 1 --2- 172.21.15.157:0/4090684291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72a0067d40 0x7f72a0152e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.982+0000 7f729ffff700 1 -- 172.21.15.157:0/4090684291 >> 172.21.15.157:0/4090684291 conn(0x7f72a006c1f0 msgr2=0x7f72a006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:36.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.982+0000 7f729ffff700 1 -- 172.21.15.157:0/4090684291 shutdown_connections 2023-10-06T03:57:36.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.982+0000 7f729ffff700 1 -- 172.21.15.157:0/4090684291 wait complete. 2023-10-06T03:57:36.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.983+0000 7f729ffff700 1 Processor -- start 2023-10-06T03:57:36.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.984+0000 7f729ffff700 1 -- start start 2023-10-06T03:57:36.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.984+0000 7f729ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a0067d40 0x7f72a00c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:36.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.984+0000 7f729ffff700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72a00b0030 0x7f72a00c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:36.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.984+0000 7f729ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72a00c5ca0 0x7f72a0169600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:36.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.984+0000 7f729ffff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a014ecd0 con 0x7f72a0067d40 2023-10-06T03:57:36.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.984+0000 7f729ffff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a014ef30 con 0x7f72a00c5ca0 2023-10-06T03:57:36.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.984+0000 7f729ffff700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a014f210 con 0x7f72a00b0030 2023-10-06T03:57:36.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.984+0000 7f729effd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a0067d40 0x7f72a00c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:36.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.984+0000 7f729effd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a0067d40 0x7f72a00c8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35676/0 (socket says 172.21.15.157:35676) 2023-10-06T03:57:36.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.984+0000 7f729effd700 1 -- 172.21.15.157:0/3456441303 learned_addr learned my addr 172.21.15.157:0/3456441303 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:36.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.984+0000 7f729e7fc700 1 --2- 172.21.15.157:0/3456441303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72a00b0030 0x7f72a00c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:36.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.984+0000 7f729f7fe700 1 --2- 172.21.15.157:0/3456441303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72a00c5ca0 0x7f72a0169600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:36.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.985+0000 7f729effd700 1 -- 172.21.15.157:0/3456441303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72a00b0030 msgr2=0x7f72a00c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:36.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.985+0000 7f729effd700 1 --2- 172.21.15.157:0/3456441303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72a00b0030 0x7f72a00c91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.985+0000 7f729effd700 1 -- 172.21.15.157:0/3456441303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72a00c5ca0 msgr2=0x7f72a0169600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:36.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.985+0000 7f729effd700 1 --2- 172.21.15.157:0/3456441303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72a00c5ca0 0x7f72a0169600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:36.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.985+0000 7f729effd700 1 -- 172.21.15.157:0/3456441303 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7298009420 con 0x7f72a0067d40 2023-10-06T03:57:36.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.985+0000 7f729effd700 1 --2- 172.21.15.157:0/3456441303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a0067d40 0x7f72a00c8ca0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7290009fa0 tx=0x7f7290009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:36.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.985+0000 7f7286ffd700 1 -- 172.21.15.157:0/3456441303 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729000ebc0 con 0x7f72a0067d40 2023-10-06T03:57:36.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.985+0000 7f7286ffd700 1 -- 172.21.15.157:0/3456441303 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729000ed20 con 0x7f72a0067d40 2023-10-06T03:57:36.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.985+0000 7f729ffff700 1 -- 172.21.15.157:0/3456441303 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a00c81c0 con 0x7f72a0067d40 2023-10-06T03:57:36.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.985+0000 7f7286ffd700 1 -- 172.21.15.157:0/3456441303 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7290014670 con 0x7f72a0067d40 2023-10-06T03:57:36.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.985+0000 7f729ffff700 1 -- 172.21.15.157:0/3456441303 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a00c8780 con 0x7f72a0067d40 2023-10-06T03:57:36.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.986+0000 7f729ffff700 1 -- 172.21.15.157:0/3456441303 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a00c8940 con 0x7f72a0067d40 2023-10-06T03:57:36.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.986+0000 7f7286ffd700 1 -- 172.21.15.157:0/3456441303 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7290014a20 con 0x7f72a0067d40 2023-10-06T03:57:36.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.987+0000 7f7286ffd700 1 --2- 172.21.15.157:0/3456441303 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f728c066f30 0x7f728c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:36.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.987+0000 7f7286ffd700 1 -- 172.21.15.157:0/3456441303 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7290084c00 con 0x7f72a0067d40 2023-10-06T03:57:36.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.987+0000 7f729e7fc700 1 --2- 172.21.15.157:0/3456441303 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f728c066f30 0x7f728c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:36.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.987+0000 7f729e7fc700 1 --2- 172.21.15.157:0/3456441303 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f728c066f30 0x7f728c0693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7294007fd0 tx=0x7f7294007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:36.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:36.991+0000 7f7286ffd700 1 -- 172.21.15.157:0/3456441303 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f729008b080 con 0x7f72a0067d40 2023-10-06T03:57:37.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.175+0000 7f729ffff700 1 -- 172.21.15.157:0/3456441303 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72a00c8af0 con 0x7f72a0067d40 2023-10-06T03:57:37.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.176+0000 7f7286ffd700 1 -- 172.21.15.157:0/3456441303 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f7290051d10 con 0x7f72a0067d40 2023-10-06T03:57:37.178 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:37.178 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":21,"fsid":"28664a7a-fe30-4efe-be5a-6abf9318b700","created":"2023-10-06T03:56:51.887538+0000","modified":"2023-10-06T03:57:12.362537+0000","last_up_change":"2023-10-06T03:57:02.299440+0000","last_in_change":"2023-10-06T03:56:55.141505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:57:03.455121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:57:03.905661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:57:08.377186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:57:09.709582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f677cac-e9f4-4608-acac-718b072ce984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":90640},{"type":"v1","addr":"172.21.15.157:6809","nonce":90640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":90640},{"type":"v1","addr":"172.21.15.157:6811","nonce":90640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":90640},{"type":"v1","addr":"172.21.15.157:6815","nonce":90640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":90640},{"type":"v1","addr":"172.21.15.157:6813","nonce":90640}]},"public_addr":"172.21.15.157:6809/90640","cluster_addr":"172.21.15.157:6811/90640","heartbeat_back_addr":"172.21.15.157:6815/90640","heartbeat_front_addr":"172.21.15.157:6813/90640","state":["exists","up"]},{"osd":1,"uuid":"7d771892-bb91-4e9b-9610-051861c11247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":90654},{"type":"v1","addr":"172.21.15.157:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":90654},{"type":"v1","addr":"172.21.15.157:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":90654},{"type":"v1","addr":"172.21.15.157:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":90654},{"type":"v1","addr":"172.21.15.157:6805","nonce":90654}]},"public_addr":"172.21.15.157:6801/90654","cluster_addr":"172.21.15.157:6803/90654","heartbeat_back_addr":"172.21.15.157:6807/90654","heartbeat_front_addr":"172.21.15.157:6805/90654","state":["exists","up"]},{"osd":2,"uuid":"5c1f7d4e-d185-47cf-ab6c-453b4e3009c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":90657},{"type":"v1","addr":"172.21.15.157:6825","nonce":90657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":90657},{"type":"v1","addr":"172.21.15.157:6827","nonce":90657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":90657},{"type":"v1","addr":"172.21.15.157:6831","nonce":90657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":90657},{"type":"v1","addr":"172.21.15.157:6829","nonce":90657}]},"public_addr":"172.21.15.157:6825/90657","cluster_addr":"172.21.15.157:6827/90657","heartbeat_back_addr":"172.21.15.157:6831/90657","heartbeat_front_addr":"172.21.15.157:6829/90657","state":["exists","up"]},{"osd":3,"uuid":"f6a1e825-43c7-4afd-8daf-e51d45afc75e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":90641},{"type":"v1","addr":"172.21.15.157:6817","nonce":90641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":90641},{"type":"v1","addr":"172.21.15.157:6819","nonce":90641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":90641},{"type":"v1","addr":"172.21.15.157:6823","nonce":90641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":90641},{"type":"v1","addr":"172.21.15.157:6821","nonce":90641}]},"public_addr":"172.21.15.157:6817/90641","cluster_addr":"172.21.15.157:6819/90641","heartbeat_back_addr":"172.21.15.157:6823/90641","heartbeat_front_addr":"172.21.15.157:6821/90641","state":["exists","up"]},{"osd":4,"uuid":"8ad6e3d8-9a31-4f85-9410-417b7ac5da97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":91261},{"type":"v1","addr":"172.21.15.175:6803","nonce":91261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":91261},{"type":"v1","addr":"172.21.15.175:6806","nonce":91261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":91261},{"type":"v1","addr":"172.21.15.175:6814","nonce":91261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":91261},{"type":"v1","addr":"172.21.15.175:6810","nonce":91261}]},"public_addr":"172.21.15.175:6803/91261","cluster_addr":"172.21.15.175:6806/91261","heartbeat_back_addr":"172.21.15.175:6814/91261","heartbeat_front_addr":"172.21.15.175:6810/91261","state":["exists","up"]},{"osd":5,"uuid":"f68aca2a-f182-4c00-8e21-241d09c02ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":91262},{"type":"v1","addr":"172.21.15.175:6817","nonce":91262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":91262},{"type":"v1","addr":"172.21.15.175:6819","nonce":91262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":91262},{"type":"v1","addr":"172.21.15.175:6823","nonce":91262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":91262},{"type":"v1","addr":"172.21.15.175:6821","nonce":91262}]},"public_addr":"172.21.15.175:6817/91262","cluster_addr":"172.21.15.175:6819/91262","heartbeat_back_addr":"172.21.15.175:6823/91262","heartbeat_front_addr":"172.21.15.175:6821/91262","state":["exists","up"]},{"osd":6,"uuid":"d867cd89-3430-40ca-862f-d2bea3de9d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":91263},{"type":"v1","addr":"172.21.15.175:6801","nonce":91263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6805","nonce":91263},{"type":"v1","addr":"172.21.15.175:6807","nonce":91263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6813","nonce":91263},{"type":"v1","addr":"172.21.15.175:6815","nonce":91263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":91263},{"type":"v1","addr":"172.21.15.175:6811","nonce":91263}]},"public_addr":"172.21.15.175:6801/91263","cluster_addr":"172.21.15.175:6807/91263","heartbeat_back_addr":"172.21.15.175:6815/91263","heartbeat_front_addr":"172.21.15.175:6811/91263","state":["exists","up"]},{"osd":7,"uuid":"930cd464-79f1-47cb-b4ff-188ccde74fff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":91306},{"type":"v1","addr":"172.21.15.175:6825","nonce":91306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":91306},{"type":"v1","addr":"172.21.15.175:6827","nonce":91306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":91306},{"type":"v1","addr":"172.21.15.175:6831","nonce":91306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":91306},{"type":"v1","addr":"172.21.15.175:6829","nonce":91306}]},"public_addr":"172.21.15.175:6825/91306","cluster_addr":"172.21.15.175:6827/91306","heartbeat_back_addr":"172.21.15.175:6831/91306","heartbeat_front_addr":"172.21.15.175:6829/91306","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.290673+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.056390+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.292903+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.275861+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.332095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.327453+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.317585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.309330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:57:37.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.178+0000 7f729ffff700 1 -- 172.21.15.157:0/3456441303 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f728c066f30 msgr2=0x7f728c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:37.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.178+0000 7f729ffff700 1 --2- 172.21.15.157:0/3456441303 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f728c066f30 0x7f728c0693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7294007fd0 tx=0x7f7294007d30 comp rx=0 tx=0).stop 2023-10-06T03:57:37.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.178+0000 7f729ffff700 1 -- 172.21.15.157:0/3456441303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a0067d40 msgr2=0x7f72a00c8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:37.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.178+0000 7f729ffff700 1 --2- 172.21.15.157:0/3456441303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a0067d40 0x7f72a00c8ca0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7290009fa0 tx=0x7f7290009880 comp rx=0 tx=0).stop 2023-10-06T03:57:37.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.178+0000 7f729ffff700 1 -- 172.21.15.157:0/3456441303 shutdown_connections 2023-10-06T03:57:37.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.178+0000 7f729ffff700 1 --2- 172.21.15.157:0/3456441303 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f728c066f30 0x7f728c0693e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:37.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.178+0000 7f729ffff700 1 --2- 172.21.15.157:0/3456441303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72a00c5ca0 0x7f72a0169600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:37.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.178+0000 7f729ffff700 1 --2- 172.21.15.157:0/3456441303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a0067d40 0x7f72a00c8ca0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:37.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.178+0000 7f729ffff700 1 --2- 172.21.15.157:0/3456441303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72a00b0030 0x7f72a00c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:37.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.178+0000 7f729ffff700 1 -- 172.21.15.157:0/3456441303 >> 172.21.15.157:0/3456441303 conn(0x7f72a006c1f0 msgr2=0x7f72a0152c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:37.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.178+0000 7f729ffff700 1 -- 172.21.15.157:0/3456441303 shutdown_connections 2023-10-06T03:57:37.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.178+0000 7f729ffff700 1 -- 172.21.15.157:0/3456441303 wait complete. 2023-10-06T03:57:37.192 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:57:37.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.305+0000 7ff3c896c700 1 Processor -- start 2023-10-06T03:57:37.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.305+0000 7ff3c896c700 1 -- start start 2023-10-06T03:57:37.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.305+0000 7ff3c896c700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff3c4150410 0x7ff3c4155f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:37.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.305+0000 7ff3c896c700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c40c9780 con 0x7ff3c406cbb0 2023-10-06T03:57:37.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.305+0000 7ff3c896c700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c406e0c0 con 0x7ff3c406dc40 2023-10-06T03:57:37.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.306+0000 7ff3c896c700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3c40c8c40 con 0x7ff3c4150410 2023-10-06T03:57:37.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.306+0000 7ff3c259c700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff3c406cbb0 0x7ff3c406d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39714/0 (socket says 172.21.15.157:39714) 2023-10-06T03:57:37.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.306+0000 7ff3c259c700 1 -- 172.21.15.157:0/242465026 learned_addr learned my addr 172.21.15.157:0/242465026 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:37.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.306+0000 7ff3c2d9d700 1 --2- 172.21.15.157:0/242465026 >> v2:172.21.15.175:3300/0 conn(0x7ff3c4150410 0x7ff3c4155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:37.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.306+0000 7ff3c159a700 1 -- 172.21.15.157:0/242465026 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2689206856 0 0) 0x7ff3c40c9780 con 0x7ff3c406cbb0 2023-10-06T03:57:37.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.306+0000 7ff3c159a700 1 -- 172.21.15.157:0/242465026 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3ac003570 con 0x7ff3c406cbb0 2023-10-06T03:57:37.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.306+0000 7ff3c159a700 1 -- 172.21.15.157:0/242465026 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 646743946 0 0) 0x7ff3c406e0c0 con 0x7ff3c406dc40 2023-10-06T03:57:37.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.306+0000 7ff3c159a700 1 -- 172.21.15.157:0/242465026 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3c40c9780 con 0x7ff3c406dc40 2023-10-06T03:57:37.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.306+0000 7ff3c159a700 1 -- 172.21.15.157:0/242465026 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2086567231 0 0) 0x7ff3ac003570 con 0x7ff3c406cbb0 2023-10-06T03:57:37.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.306+0000 7ff3c2d9d700 1 -- 172.21.15.157:0/242465026 >> v1:172.21.15.175:6789/0 conn(0x7ff3c406dc40 legacy=0x7ff3c414fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:37.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.306+0000 7ff3c2d9d700 1 -- 172.21.15.157:0/242465026 >> v1:172.21.15.157:6789/0 conn(0x7ff3c406cbb0 legacy=0x7ff3c406d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:37.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.306+0000 7ff3c2d9d700 1 -- 172.21.15.157:0/242465026 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c40c8e50 con 0x7ff3c4150410 2023-10-06T03:57:37.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.306+0000 7ff3c2d9d700 1 --2- 172.21.15.157:0/242465026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3c4150410 0x7ff3c4155f70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009e60 tx=0x7ff3bc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7903bee665c6e681 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:37.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.306+0000 7ff3c159a700 1 -- 172.21.15.157:0/242465026 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc00e070 con 0x7ff3c4150410 2023-10-06T03:57:37.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.307+0000 7ff3c159a700 1 -- 172.21.15.157:0/242465026 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3bc00b830 con 0x7ff3c4150410 2023-10-06T03:57:37.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.307+0000 7ff3c159a700 1 -- 172.21.15.157:0/242465026 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4001470 con 0x7ff3c4150410 2023-10-06T03:57:37.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.307+0000 7ff3c896c700 1 -- 172.21.15.157:0/242465026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3c4150410 msgr2=0x7ff3c4155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:37.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.307+0000 7ff3c896c700 1 --2- 172.21.15.157:0/242465026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3c4150410 0x7ff3c4155f70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009e60 tx=0x7ff3bc009780 comp rx=0 tx=0).stop 2023-10-06T03:57:37.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.307+0000 7ff3c896c700 1 -- 172.21.15.157:0/242465026 shutdown_connections 2023-10-06T03:57:37.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.307+0000 7ff3c896c700 1 --2- 172.21.15.157:0/242465026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3c4150410 0x7ff3c4155f70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:37.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.307+0000 7ff3c896c700 1 -- 172.21.15.157:0/242465026 >> 172.21.15.157:0/242465026 conn(0x7ff3c406f230 msgr2=0x7ff3c406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:37.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.307+0000 7ff3c896c700 1 -- 172.21.15.157:0/242465026 shutdown_connections 2023-10-06T03:57:37.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.307+0000 7ff3c896c700 1 -- 172.21.15.157:0/242465026 wait complete. 2023-10-06T03:57:37.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.308+0000 7ff3c896c700 1 Processor -- start 2023-10-06T03:57:37.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.308+0000 7ff3c896c700 1 -- start start 2023-10-06T03:57:37.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.309+0000 7ff3c896c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3c406cbb0 0x7ff3c4169e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:37.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.309+0000 7ff3c896c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3c4150410 0x7ff3c41531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:37.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.309+0000 7ff3c896c700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3c414ab80 0x7ff3c40c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:37.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.309+0000 7ff3c896c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c41527b0 con 0x7ff3c4150410 2023-10-06T03:57:37.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.309+0000 7ff3c896c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c416a3a0 con 0x7ff3c406cbb0 2023-10-06T03:57:37.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.309+0000 7ff3c896c700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c41518c0 con 0x7ff3c414ab80 2023-10-06T03:57:37.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.309+0000 7ff3c1d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3c4150410 0x7ff3c41531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:37.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.309+0000 7ff3c1d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3c4150410 0x7ff3c41531f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35680/0 (socket says 172.21.15.157:35680) 2023-10-06T03:57:37.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.309+0000 7ff3c1d9b700 1 -- 172.21.15.157:0/1102265999 learned_addr learned my addr 172.21.15.157:0/1102265999 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:37.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.309+0000 7ff3c259c700 1 --2- 172.21.15.157:0/1102265999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3c406cbb0 0x7ff3c4169e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:37.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.309+0000 7ff3c2d9d700 1 --2- 172.21.15.157:0/1102265999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3c414ab80 0x7ff3c40c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:37.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.309+0000 7ff3c1d9b700 1 -- 172.21.15.157:0/1102265999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3c414ab80 msgr2=0x7ff3c40c60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:37.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.309+0000 7ff3c1d9b700 1 --2- 172.21.15.157:0/1102265999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3c414ab80 0x7ff3c40c60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:37.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.309+0000 7ff3c1d9b700 1 -- 172.21.15.157:0/1102265999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3c406cbb0 msgr2=0x7ff3c4169e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:37.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.309+0000 7ff3c1d9b700 1 --2- 172.21.15.157:0/1102265999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3c406cbb0 0x7ff3c4169e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:37.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.309+0000 7ff3c1d9b700 1 -- 172.21.15.157:0/1102265999 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3bc009a20 con 0x7ff3c4150410 2023-10-06T03:57:37.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.309+0000 7ff3c1d9b700 1 --2- 172.21.15.157:0/1102265999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3c4150410 0x7ff3c41531f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff3b800c7c0 tx=0x7ff3b8007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:37.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.310+0000 7ff3b27fc700 1 -- 172.21.15.157:0/1102265999 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8010510 con 0x7ff3c4150410 2023-10-06T03:57:37.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.310+0000 7ff3b27fc700 1 -- 172.21.15.157:0/1102265999 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b8010670 con 0x7ff3c4150410 2023-10-06T03:57:37.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.310+0000 7ff3c896c700 1 -- 172.21.15.157:0/1102265999 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c4151ba0 con 0x7ff3c4150410 2023-10-06T03:57:37.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.310+0000 7ff3b27fc700 1 -- 172.21.15.157:0/1102265999 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8010a10 con 0x7ff3c4150410 2023-10-06T03:57:37.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.310+0000 7ff3c896c700 1 -- 172.21.15.157:0/1102265999 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c4152160 con 0x7ff3c4150410 2023-10-06T03:57:37.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.311+0000 7ff3c896c700 1 -- 172.21.15.157:0/1102265999 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c4089ca0 con 0x7ff3c4150410 2023-10-06T03:57:37.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.311+0000 7ff3b27fc700 1 -- 172.21.15.157:0/1102265999 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff3b801b3b0 con 0x7ff3c4150410 2023-10-06T03:57:37.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.311+0000 7ff3b27fc700 1 --2- 172.21.15.157:0/1102265999 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff3a4066f30 0x7ff3a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:37.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.311+0000 7ff3b27fc700 1 -- 172.21.15.157:0/1102265999 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff3b8083ec0 con 0x7ff3c4150410 2023-10-06T03:57:37.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.312+0000 7ff3c259c700 1 --2- 172.21.15.157:0/1102265999 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff3a4066f30 0x7ff3a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:37.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.312+0000 7ff3c259c700 1 --2- 172.21.15.157:0/1102265999 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff3a4066f30 0x7ff3a40693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40011e0 tx=0x7ff3b4000bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:37.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.316+0000 7ff3b27fc700 1 -- 172.21.15.157:0/1102265999 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff3b8050fd0 con 0x7ff3c4150410 2023-10-06T03:57:37.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.541+0000 7ff3c896c700 1 -- 172.21.15.157:0/1102265999 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3c40c9130 con 0x7ff3c4150410 2023-10-06T03:57:37.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.542+0000 7ff3b27fc700 1 -- 172.21.15.157:0/1102265999 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6185 (secure 0 0 0) 0x7ff3b8017020 con 0x7ff3c4150410 2023-10-06T03:57:37.543 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:37.543 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T03:57:32.634356+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"f","rank":4,"incarnation":15,"state":"up:creating","state_seq":1,"addr":"172.21.15.175:6839/4177493830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4177493830},{"type":"v1","addr":"172.21.15.175:6839","nonce":4177493830}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:57:37.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.543+0000 7ff3c896c700 1 -- 172.21.15.157:0/1102265999 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff3a4066f30 msgr2=0x7ff3a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:37.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.543+0000 7ff3c896c700 1 --2- 172.21.15.157:0/1102265999 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff3a4066f30 0x7ff3a40693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40011e0 tx=0x7ff3b4000bc0 comp rx=0 tx=0).stop 2023-10-06T03:57:37.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.543+0000 7ff3c896c700 1 -- 172.21.15.157:0/1102265999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3c4150410 msgr2=0x7ff3c41531f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:37.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.544+0000 7ff3c896c700 1 --2- 172.21.15.157:0/1102265999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3c4150410 0x7ff3c41531f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff3b800c7c0 tx=0x7ff3b8007e10 comp rx=0 tx=0).stop 2023-10-06T03:57:37.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.544+0000 7ff3c896c700 1 -- 172.21.15.157:0/1102265999 shutdown_connections 2023-10-06T03:57:37.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.544+0000 7ff3c896c700 1 --2- 172.21.15.157:0/1102265999 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff3a4066f30 0x7ff3a40693e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:37.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.544+0000 7ff3c896c700 1 --2- 172.21.15.157:0/1102265999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3c406cbb0 0x7ff3c4169e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:37.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.544+0000 7ff3c896c700 1 --2- 172.21.15.157:0/1102265999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3c414ab80 0x7ff3c40c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:37.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.544+0000 7ff3c896c700 1 --2- 172.21.15.157:0/1102265999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3c4150410 0x7ff3c41531f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:37.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.544+0000 7ff3c896c700 1 -- 172.21.15.157:0/1102265999 >> 172.21.15.157:0/1102265999 conn(0x7ff3c406f230 msgr2=0x7ff3c40c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:37.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.544+0000 7ff3c896c700 1 -- 172.21.15.157:0/1102265999 shutdown_connections 2023-10-06T03:57:37.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:37.544+0000 7ff3c896c700 1 -- 172.21.15.157:0/1102265999 wait complete. 2023-10-06T03:57:37.547 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 15 2023-10-06T03:57:38.160 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T03:57:10.718782+0000', 'modified': '2023-10-06T03:57:32.634356+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 4296, 'mds_1': 4287, 'mds_2': 4279, 'mds_3': 4270, 'mds_4': 4241}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4241': {'gid': 4241, 'name': 'f', 'rank': 4, 'incarnation': 15, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.175:6839/4177493830', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6838', 'nonce': 4177493830}, {'type': 'v1', 'addr': '172.21.15.175:6839', 'nonce': 4177493830}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_4270': {'gid': 4270, 'name': 'b', 'rank': 3, 'incarnation': 13, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.175:6835/1596042861', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6834', 'nonce': 1596042861}, {'type': 'v1', 'addr': '172.21.15.175:6835', 'nonce': 1596042861}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_4279': {'gid': 4279, 'name': 'd', 'rank': 2, 'incarnation': 11, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.175:6837/1002404870', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6836', 'nonce': 1002404870}, {'type': 'v1', 'addr': '172.21.15.175:6837', 'nonce': 1002404870}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_4287': {'gid': 4287, 'name': 'c', 'rank': 1, 'incarnation': 9, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.157:6834/1179753185', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6832', 'nonce': 1179753185}, {'type': 'v1', 'addr': '172.21.15.157:6834', 'nonce': 1179753185}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_4296': {'gid': 4296, 'name': 'e', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.157:6837/4188652695', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6836', 'nonce': 4188652695}, {'type': 'v1', 'addr': '172.21.15.157:6837', 'nonce': 4188652695}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T03:57:38.160 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4241:up:creating 2023-10-06T03:57:39.162 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:57:39.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.285+0000 7fd50c32f700 1 Processor -- start 2023-10-06T03:57:39.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.285+0000 7fd50c32f700 1 -- start start 2023-10-06T03:57:39.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.286+0000 7fd50c32f700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd5040c2cb0 0x7fd50406c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:39.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.286+0000 7fd50c32f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5040bc850 0x7fd5041607c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:39.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.286+0000 7fd50c32f700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd504156760 con 0x7fd504068bc0 2023-10-06T03:57:39.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.286+0000 7fd50c32f700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd50415a3c0 con 0x7fd5040c2cb0 2023-10-06T03:57:39.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.286+0000 7fd50c32f700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd504155f00 con 0x7fd5040bc850 2023-10-06T03:57:39.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.286+0000 7fd50a0cb700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd5040c2cb0 0x7fd50406c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:39.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.287+0000 7fd5098ca700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd504068bc0 0x7fd50406a530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39724/0 (socket says 172.21.15.157:39724) 2023-10-06T03:57:39.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.287+0000 7fd5098ca700 1 -- 172.21.15.157:0/3908390803 learned_addr learned my addr 172.21.15.157:0/3908390803 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:39.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.287+0000 7fd50a8cc700 1 --2- 172.21.15.157:0/3908390803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5040bc850 0x7fd5041607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:39.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.287+0000 7fd50a0cb700 1 -- 172.21.15.157:0/3908390803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5040bc850 msgr2=0x7fd5041607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:39.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.287+0000 7fd50a0cb700 1 --2- 172.21.15.157:0/3908390803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5040bc850 0x7fd5041607c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:39.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.287+0000 7fd50a0cb700 1 -- 172.21.15.157:0/3908390803 >> v1:172.21.15.157:6789/0 conn(0x7fd504068bc0 legacy=0x7fd50406a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:39.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.287+0000 7fd5090c9700 1 -- 172.21.15.157:0/3908390803 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3979266936 0 0) 0x7fd504156760 con 0x7fd504068bc0 2023-10-06T03:57:39.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.287+0000 7fd50a0cb700 1 -- 172.21.15.157:0/3908390803 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd504159e40 con 0x7fd5040c2cb0 2023-10-06T03:57:39.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.287+0000 7fd50a8cc700 1 --2- 172.21.15.157:0/3908390803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5040bc850 0x7fd5041607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:57:39.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.287+0000 7fd50a0cb700 1 --2- 172.21.15.157:0/3908390803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5040c2cb0 0x7fd50406c1f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd500009e60 tx=0x7fd500009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7fa3f4d1cbe7358 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:39.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.287+0000 7fd5090c9700 1 -- 172.21.15.157:0/3908390803 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50000e070 con 0x7fd5040c2cb0 2023-10-06T03:57:39.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.287+0000 7fd5090c9700 1 -- 172.21.15.157:0/3908390803 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50000bc30 con 0x7fd5040c2cb0 2023-10-06T03:57:39.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.287+0000 7fd5090c9700 1 -- 172.21.15.157:0/3908390803 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd500012430 con 0x7fd5040c2cb0 2023-10-06T03:57:39.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.288+0000 7fd50c32f700 1 -- 172.21.15.157:0/3908390803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5040c2cb0 msgr2=0x7fd50406c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:39.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.288+0000 7fd50c32f700 1 --2- 172.21.15.157:0/3908390803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5040c2cb0 0x7fd50406c1f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd500009e60 tx=0x7fd500009780 comp rx=0 tx=0).stop 2023-10-06T03:57:39.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.288+0000 7fd50c32f700 1 -- 172.21.15.157:0/3908390803 shutdown_connections 2023-10-06T03:57:39.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.288+0000 7fd50c32f700 1 --2- 172.21.15.157:0/3908390803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5040bc850 0x7fd5041607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:39.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.288+0000 7fd50c32f700 1 --2- 172.21.15.157:0/3908390803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5040c2cb0 0x7fd50406c1f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:39.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.288+0000 7fd50c32f700 1 -- 172.21.15.157:0/3908390803 >> 172.21.15.157:0/3908390803 conn(0x7fd50406b510 msgr2=0x7fd50406b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:39.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.288+0000 7fd50c32f700 1 -- 172.21.15.157:0/3908390803 shutdown_connections 2023-10-06T03:57:39.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.288+0000 7fd50c32f700 1 -- 172.21.15.157:0/3908390803 wait complete. 2023-10-06T03:57:39.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.289+0000 7fd50c32f700 1 Processor -- start 2023-10-06T03:57:39.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.289+0000 7fd50c32f700 1 -- start start 2023-10-06T03:57:39.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.289+0000 7fd50c32f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd504068bc0 0x7fd5040ba630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:39.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.289+0000 7fd50c32f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5040bc850 0x7fd504158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:39.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.289+0000 7fd50c32f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5040c2cb0 0x7fd504169740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:39.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.290+0000 7fd50c32f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5040b9940 con 0x7fd5040bc850 2023-10-06T03:57:39.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.290+0000 7fd50c32f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5040ba460 con 0x7fd504068bc0 2023-10-06T03:57:39.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.290+0000 7fd50c32f700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd504157bc0 con 0x7fd5040c2cb0 2023-10-06T03:57:39.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.290+0000 7fd5098ca700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5040bc850 0x7fd504158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:39.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.290+0000 7fd5098ca700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5040bc850 0x7fd504158050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35702/0 (socket says 172.21.15.157:35702) 2023-10-06T03:57:39.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.290+0000 7fd5098ca700 1 -- 172.21.15.157:0/1354718174 learned_addr learned my addr 172.21.15.157:0/1354718174 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:39.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.290+0000 7fd50a0cb700 1 --2- 172.21.15.157:0/1354718174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd504068bc0 0x7fd5040ba630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:39.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.290+0000 7fd50a8cc700 1 --2- 172.21.15.157:0/1354718174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5040c2cb0 0x7fd504169740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:39.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.290+0000 7fd5098ca700 1 -- 172.21.15.157:0/1354718174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5040c2cb0 msgr2=0x7fd504169740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:39.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.290+0000 7fd5098ca700 1 --2- 172.21.15.157:0/1354718174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5040c2cb0 0x7fd504169740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:39.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.290+0000 7fd5098ca700 1 -- 172.21.15.157:0/1354718174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd504068bc0 msgr2=0x7fd5040ba630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:39.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.290+0000 7fd5098ca700 1 --2- 172.21.15.157:0/1354718174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd504068bc0 0x7fd5040ba630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:39.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.290+0000 7fd5098ca700 1 -- 172.21.15.157:0/1354718174 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd500009a20 con 0x7fd5040bc850 2023-10-06T03:57:39.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.290+0000 7fd50a8cc700 1 --2- 172.21.15.157:0/1354718174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5040c2cb0 0x7fd504169740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:57:39.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.290+0000 7fd5098ca700 1 --2- 172.21.15.157:0/1354718174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5040bc850 0x7fd504158050 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd4f400c7e0 tx=0x7fd4f4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:39.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.290+0000 7fd4fa7fc700 1 -- 172.21.15.157:0/1354718174 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4005b30 con 0x7fd5040bc850 2023-10-06T03:57:39.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.290+0000 7fd4fa7fc700 1 -- 172.21.15.157:0/1354718174 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f4005c90 con 0x7fd5040bc850 2023-10-06T03:57:39.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.290+0000 7fd50c32f700 1 -- 172.21.15.157:0/1354718174 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd50416a1e0 con 0x7fd5040bc850 2023-10-06T03:57:39.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.290+0000 7fd4fa7fc700 1 -- 172.21.15.157:0/1354718174 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4007300 con 0x7fd5040bc850 2023-10-06T03:57:39.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.290+0000 7fd50c32f700 1 -- 172.21.15.157:0/1354718174 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd504157850 con 0x7fd5040bc850 2023-10-06T03:57:39.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.291+0000 7fd50c32f700 1 -- 172.21.15.157:0/1354718174 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd504157d70 con 0x7fd5040bc850 2023-10-06T03:57:39.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.292+0000 7fd4fa7fc700 1 -- 172.21.15.157:0/1354718174 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd4f40076b0 con 0x7fd5040bc850 2023-10-06T03:57:39.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.292+0000 7fd4fa7fc700 1 --2- 172.21.15.157:0/1354718174 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd4e8066f30 0x7fd4e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:39.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.292+0000 7fd4fa7fc700 1 -- 172.21.15.157:0/1354718174 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd4f40841a0 con 0x7fd5040bc850 2023-10-06T03:57:39.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.292+0000 7fd50a0cb700 1 --2- 172.21.15.157:0/1354718174 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd4e8066f30 0x7fd4e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:39.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.293+0000 7fd50a0cb700 1 --2- 172.21.15.157:0/1354718174 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd4e8066f30 0x7fd4e80693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd500009e60 tx=0x7fd500019000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:39.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.296+0000 7fd4fa7fc700 1 -- 172.21.15.157:0/1354718174 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd504157d70 con 0x7fd5040bc850 2023-10-06T03:57:39.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.522+0000 7fd50c32f700 1 -- 172.21.15.157:0/1354718174 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd504157d70 con 0x7fd5040bc850 2023-10-06T03:57:39.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.522+0000 7fd4fa7fc700 1 -- 172.21.15.157:0/1354718174 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+6184 (secure 0 0 0) 0x7fd504157d70 con 0x7fd5040bc850 2023-10-06T03:57:39.524 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:39.524 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T03:57:37.649946+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"f","rank":4,"incarnation":15,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/4177493830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4177493830},{"type":"v1","addr":"172.21.15.175:6839","nonce":4177493830}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:57:39.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.524+0000 7fd50c32f700 1 -- 172.21.15.157:0/1354718174 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd4e8066f30 msgr2=0x7fd4e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:39.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.524+0000 7fd50c32f700 1 --2- 172.21.15.157:0/1354718174 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd4e8066f30 0x7fd4e80693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd500009e60 tx=0x7fd500019000 comp rx=0 tx=0).stop 2023-10-06T03:57:39.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.524+0000 7fd50c32f700 1 -- 172.21.15.157:0/1354718174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5040bc850 msgr2=0x7fd504158050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:39.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.524+0000 7fd50c32f700 1 --2- 172.21.15.157:0/1354718174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5040bc850 0x7fd504158050 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd4f400c7e0 tx=0x7fd4f4007df0 comp rx=0 tx=0).stop 2023-10-06T03:57:39.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.524+0000 7fd50c32f700 1 -- 172.21.15.157:0/1354718174 shutdown_connections 2023-10-06T03:57:39.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.524+0000 7fd50c32f700 1 --2- 172.21.15.157:0/1354718174 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd4e8066f30 0x7fd4e80693e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:39.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.524+0000 7fd50c32f700 1 --2- 172.21.15.157:0/1354718174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd504068bc0 0x7fd5040ba630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:39.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.524+0000 7fd50c32f700 1 --2- 172.21.15.157:0/1354718174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5040c2cb0 0x7fd504169740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:39.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.524+0000 7fd50c32f700 1 --2- 172.21.15.157:0/1354718174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5040bc850 0x7fd504158050 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:39.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.524+0000 7fd50c32f700 1 -- 172.21.15.157:0/1354718174 >> 172.21.15.157:0/1354718174 conn(0x7fd50406b510 msgr2=0x7fd504153090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:39.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.524+0000 7fd50c32f700 1 -- 172.21.15.157:0/1354718174 shutdown_connections 2023-10-06T03:57:39.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:39.524+0000 7fd50c32f700 1 -- 172.21.15.157:0/1354718174 wait complete. 2023-10-06T03:57:39.528 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 16 2023-10-06T03:57:40.141 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T03:57:10.718782+0000', 'modified': '2023-10-06T03:57:37.649946+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 4296, 'mds_1': 4287, 'mds_2': 4279, 'mds_3': 4270, 'mds_4': 4241}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4241': {'gid': 4241, 'name': 'f', 'rank': 4, 'incarnation': 15, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.175:6839/4177493830', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6838', 'nonce': 4177493830}, {'type': 'v1', 'addr': '172.21.15.175:6839', 'nonce': 4177493830}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_4270': {'gid': 4270, 'name': 'b', 'rank': 3, 'incarnation': 13, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.175:6835/1596042861', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6834', 'nonce': 1596042861}, {'type': 'v1', 'addr': '172.21.15.175:6835', 'nonce': 1596042861}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_4279': {'gid': 4279, 'name': 'd', 'rank': 2, 'incarnation': 11, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.175:6837/1002404870', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6836', 'nonce': 1002404870}, {'type': 'v1', 'addr': '172.21.15.175:6837', 'nonce': 1002404870}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_4287': {'gid': 4287, 'name': 'c', 'rank': 1, 'incarnation': 9, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.157:6834/1179753185', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6832', 'nonce': 1179753185}, {'type': 'v1', 'addr': '172.21.15.157:6834', 'nonce': 1179753185}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_4296': {'gid': 4296, 'name': 'e', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.157:6837/4188652695', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6836', 'nonce': 4188652695}, {'type': 'v1', 'addr': '172.21.15.157:6837', 'nonce': 4188652695}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T03:57:40.142 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-10-06T03:57:40.142 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-10-06T03:57:40.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.259+0000 7fa0aabf8700 1 Processor -- start 2023-10-06T03:57:40.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.259+0000 7fa0aabf8700 1 -- start start 2023-10-06T03:57:40.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.260+0000 7fa0aabf8700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa0a40c8b60 0x7fa0a4155b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:40.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.260+0000 7fa0aabf8700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0a40af480 con 0x7fa0a40cc050 2023-10-06T03:57:40.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.260+0000 7fa0aabf8700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0a40af760 con 0x7fa0a40c8790 2023-10-06T03:57:40.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.260+0000 7fa0aabf8700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0a40afa40 con 0x7fa0a40c8b60 2023-10-06T03:57:40.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.260+0000 7fa0aa3f7700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa0a40cc050 0x7fa0a40c75d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39736/0 (socket says 172.21.15.157:39736) 2023-10-06T03:57:40.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.260+0000 7fa0aa3f7700 1 -- 172.21.15.157:0/1024750014 learned_addr learned my addr 172.21.15.157:0/1024750014 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:40.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.260+0000 7fa0a93f5700 1 --2- 172.21.15.157:0/1024750014 >> v2:172.21.15.175:3300/0 conn(0x7fa0a40c8b60 0x7fa0a4155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:40.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.260+0000 7fa0a8bf4700 1 -- 172.21.15.157:0/1024750014 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4192910279 0 0) 0x7fa0a40af480 con 0x7fa0a40cc050 2023-10-06T03:57:40.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.260+0000 7fa0a8bf4700 1 -- 172.21.15.157:0/1024750014 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa090003570 con 0x7fa0a40cc050 2023-10-06T03:57:40.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.261+0000 7fa0a8bf4700 1 -- 172.21.15.157:0/1024750014 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3262760907 0 0) 0x7fa090003570 con 0x7fa0a40cc050 2023-10-06T03:57:40.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.261+0000 7fa0a8bf4700 1 -- 172.21.15.157:0/1024750014 >> v2:172.21.15.175:3300/0 conn(0x7fa0a40c8b60 msgr2=0x7fa0a4155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:40.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.261+0000 7fa0a8bf4700 1 --2- 172.21.15.157:0/1024750014 >> v2:172.21.15.175:3300/0 conn(0x7fa0a40c8b60 0x7fa0a4155b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:40.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.261+0000 7fa0a8bf4700 1 -- 172.21.15.157:0/1024750014 >> v1:172.21.15.175:6789/0 conn(0x7fa0a40c8790 legacy=0x7fa0a40b1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:40.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.261+0000 7fa0a8bf4700 1 -- 172.21.15.157:0/1024750014 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a40afd20 con 0x7fa0a40cc050 2023-10-06T03:57:40.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.261+0000 7fa0a8bf4700 1 -- 172.21.15.157:0/1024750014 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7fa09c001470 con 0x7fa0a40cc050 2023-10-06T03:57:40.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.261+0000 7fa0a8bf4700 1 -- 172.21.15.157:0/1024750014 >> v1:172.21.15.157:6789/0 conn(0x7fa0a40cc050 legacy=0x7fa0a40c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:40.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.261+0000 7fa0a8bf4700 1 --2- 172.21.15.157:0/1024750014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa090003a60 0x7fa090003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:40.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.261+0000 7fa0a8bf4700 1 --2- 172.21.15.157:0/1024750014 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa090004810 0x7fa090004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:40.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.261+0000 7fa0a8bf4700 1 --2- 172.21.15.157:0/1024750014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0900051c0 0x7fa09000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:40.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.261+0000 7fa0a8bf4700 1 -- 172.21.15.157:0/1024750014 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a40afa40 con 0x7fa0900051c0 2023-10-06T03:57:40.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.261+0000 7fa0a8bf4700 1 -- 172.21.15.157:0/1024750014 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09000c0d0 con 0x7fa090003a60 2023-10-06T03:57:40.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.261+0000 7fa0a8bf4700 1 -- 172.21.15.157:0/1024750014 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa09000c290 con 0x7fa090004810 2023-10-06T03:57:40.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.261+0000 7fa0aa3f7700 1 --2- 172.21.15.157:0/1024750014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0900051c0 0x7fa09000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:40.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.261+0000 7fa0a9bf6700 1 --2- 172.21.15.157:0/1024750014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa090003a60 0x7fa090003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:40.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.261+0000 7fa0a93f5700 1 --2- 172.21.15.157:0/1024750014 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa090004810 0x7fa090004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:40.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.261+0000 7fa0aa3f7700 1 -- 172.21.15.157:0/1024750014 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa090004810 msgr2=0x7fa090004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:40.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.261+0000 7fa0aa3f7700 1 --2- 172.21.15.157:0/1024750014 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa090004810 0x7fa090004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:40.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.261+0000 7fa0aa3f7700 1 -- 172.21.15.157:0/1024750014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa090003a60 msgr2=0x7fa090003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:40.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.261+0000 7fa0aa3f7700 1 --2- 172.21.15.157:0/1024750014 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa090003a60 0x7fa090003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:40.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.261+0000 7fa0aa3f7700 1 -- 172.21.15.157:0/1024750014 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa090003570 con 0x7fa0900051c0 2023-10-06T03:57:40.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.261+0000 7fa0aa3f7700 1 -- 172.21.15.157:0/1024750014 reap_dead start 2023-10-06T03:57:40.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.261+0000 7fa0aa3f7700 1 --2- 172.21.15.157:0/1024750014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0900051c0 0x7fa09000ba60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00c3c0 tx=0x7fa09c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f3e49f7c7f6467 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:40.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.261+0000 7fa0a8bf4700 1 -- 172.21.15.157:0/1024750014 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c00cc90 con 0x7fa0900051c0 2023-10-06T03:57:40.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.261+0000 7fa0a8bf4700 1 -- 172.21.15.157:0/1024750014 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09c00cdf0 con 0x7fa0900051c0 2023-10-06T03:57:40.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.261+0000 7fa0a8bf4700 1 -- 172.21.15.157:0/1024750014 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c005b50 con 0x7fa0900051c0 2023-10-06T03:57:40.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.262+0000 7fa0aabf8700 1 -- 172.21.15.157:0/1024750014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0900051c0 msgr2=0x7fa09000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:40.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.262+0000 7fa0aabf8700 1 --2- 172.21.15.157:0/1024750014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0900051c0 0x7fa09000ba60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00c3c0 tx=0x7fa09c001e50 comp rx=0 tx=0).stop 2023-10-06T03:57:40.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.262+0000 7fa0aabf8700 1 -- 172.21.15.157:0/1024750014 shutdown_connections 2023-10-06T03:57:40.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.262+0000 7fa0aabf8700 1 --2- 172.21.15.157:0/1024750014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0900051c0 0x7fa09000ba60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:40.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.262+0000 7fa0aabf8700 1 -- 172.21.15.157:0/1024750014 >> 172.21.15.157:0/1024750014 conn(0x7fa0a40cc540 msgr2=0x7fa0a40cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:40.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.262+0000 7fa0aabf8700 1 -- 172.21.15.157:0/1024750014 shutdown_connections 2023-10-06T03:57:40.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.262+0000 7fa0aabf8700 1 -- 172.21.15.157:0/1024750014 wait complete. 2023-10-06T03:57:40.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.263+0000 7fa0aabf8700 1 Processor -- start 2023-10-06T03:57:40.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.263+0000 7fa0aabf8700 1 -- start start 2023-10-06T03:57:40.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.264+0000 7fa0aabf8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0900051c0 0x7fa0a40c3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:40.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.264+0000 7fa0aabf8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0a414ba00 0x7fa0a414c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:40.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.264+0000 7fa0aabf8700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0a4150ad0 0x7fa0a414ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:40.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.264+0000 7fa0aabf8700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a4150120 con 0x7fa0900051c0 2023-10-06T03:57:40.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.264+0000 7fa0aabf8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a4152fe0 con 0x7fa0a414ba00 2023-10-06T03:57:40.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.264+0000 7fa0aabf8700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a41523d0 con 0x7fa0a4150ad0 2023-10-06T03:57:40.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.264+0000 7fa0a9bf6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0900051c0 0x7fa0a40c3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:40.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.264+0000 7fa0a9bf6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0900051c0 0x7fa0a40c3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35726/0 (socket says 172.21.15.157:35726) 2023-10-06T03:57:40.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.264+0000 7fa0a9bf6700 1 -- 172.21.15.157:0/1593925695 learned_addr learned my addr 172.21.15.157:0/1593925695 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:40.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.264+0000 7fa0a93f5700 1 --2- 172.21.15.157:0/1593925695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0a414ba00 0x7fa0a414c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:40.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.264+0000 7fa0aa3f7700 1 --2- 172.21.15.157:0/1593925695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0a4150ad0 0x7fa0a414ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:40.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.264+0000 7fa0a9bf6700 1 -- 172.21.15.157:0/1593925695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0a4150ad0 msgr2=0x7fa0a414ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:40.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.264+0000 7fa0a9bf6700 1 --2- 172.21.15.157:0/1593925695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0a4150ad0 0x7fa0a414ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:40.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.264+0000 7fa0a9bf6700 1 -- 172.21.15.157:0/1593925695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0a414ba00 msgr2=0x7fa0a414c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:40.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.264+0000 7fa0a9bf6700 1 --2- 172.21.15.157:0/1593925695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0a414ba00 0x7fa0a414c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:40.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.264+0000 7fa0a9bf6700 1 -- 172.21.15.157:0/1593925695 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a41526b0 con 0x7fa0900051c0 2023-10-06T03:57:40.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.264+0000 7fa0aa3f7700 1 --2- 172.21.15.157:0/1593925695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0a4150ad0 0x7fa0a414ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:40.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.264+0000 7fa0a9bf6700 1 --2- 172.21.15.157:0/1593925695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0900051c0 0x7fa0a40c3d30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000ebc0 tx=0x7fa0a0001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:40.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.264+0000 7fa099ffb700 1 -- 172.21.15.157:0/1593925695 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0006ba0 con 0x7fa0900051c0 2023-10-06T03:57:40.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.264+0000 7fa099ffb700 1 -- 172.21.15.157:0/1593925695 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a0006d00 con 0x7fa0900051c0 2023-10-06T03:57:40.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.265+0000 7fa0aabf8700 1 -- 172.21.15.157:0/1593925695 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0a4152990 con 0x7fa0900051c0 2023-10-06T03:57:40.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.265+0000 7fa099ffb700 1 -- 172.21.15.157:0/1593925695 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a000ca90 con 0x7fa0900051c0 2023-10-06T03:57:40.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.265+0000 7fa0aabf8700 1 -- 172.21.15.157:0/1593925695 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a40c2dd0 con 0x7fa0900051c0 2023-10-06T03:57:40.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.266+0000 7fa099ffb700 1 -- 172.21.15.157:0/1593925695 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa0a0015430 con 0x7fa0900051c0 2023-10-06T03:57:40.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.266+0000 7fa099ffb700 1 --2- 172.21.15.157:0/1593925695 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa088066f30 0x7fa0880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:40.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.266+0000 7fa099ffb700 1 -- 172.21.15.157:0/1593925695 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa0a0084b90 con 0x7fa0900051c0 2023-10-06T03:57:40.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.266+0000 7fa0a93f5700 1 --2- 172.21.15.157:0/1593925695 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa088066f30 0x7fa0880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:40.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.267+0000 7fa0a93f5700 1 --2- 172.21.15.157:0/1593925695 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa088066f30 0x7fa0880693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa0a40b0d50 tx=0x7fa0940091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:40.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.268+0000 7fa0b5152b80 1 Processor -- start 2023-10-06T03:57:40.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.268+0000 7fa0b5152b80 1 -- start start 2023-10-06T03:57:40.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.269+0000 7fa0b5152b80 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf3759a1e0 0x55bf37599490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:40.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.269+0000 7fa0b5152b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf3759a5b0 0x55bf37555170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:40.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.269+0000 7fa0b5152b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf375556b0 0x55bf375824f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:40.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.269+0000 7fa0b5152b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf374d8c80 con 0x55bf3759a5b0 2023-10-06T03:57:40.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.269+0000 7fa0b5152b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf374afa70 con 0x55bf375556b0 2023-10-06T03:57:40.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.269+0000 7fa0b5152b80 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf374ad270 con 0x55bf3759a1e0 2023-10-06T03:57:40.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.269+0000 7fa0a93f5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf3759a5b0 0x55bf37555170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:40.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.269+0000 7fa0a93f5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf3759a5b0 0x55bf37555170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35742/0 (socket says 172.21.15.157:35742) 2023-10-06T03:57:40.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.269+0000 7fa0a93f5700 1 -- 172.21.15.157:0/2328980552 learned_addr learned my addr 172.21.15.157:0/2328980552 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:40.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.269+0000 7fa0a9bf6700 1 --2- 172.21.15.157:0/2328980552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf3759a1e0 0x55bf37599490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:40.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.269+0000 7fa0aa3f7700 1 --2- 172.21.15.157:0/2328980552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf375556b0 0x55bf375824f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:40.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.269+0000 7fa0a93f5700 1 -- 172.21.15.157:0/2328980552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf3759a1e0 msgr2=0x55bf37599490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:40.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.269+0000 7fa0a93f5700 1 --2- 172.21.15.157:0/2328980552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf3759a1e0 0x55bf37599490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:40.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.269+0000 7fa0a93f5700 1 -- 172.21.15.157:0/2328980552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf375556b0 msgr2=0x55bf375824f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:40.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.269+0000 7fa0a93f5700 1 --2- 172.21.15.157:0/2328980552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf375556b0 0x55bf375824f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:40.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.269+0000 7fa0a93f5700 1 -- 172.21.15.157:0/2328980552 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf37343270 con 0x55bf3759a5b0 2023-10-06T03:57:40.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.269+0000 7fa0aa3f7700 1 --2- 172.21.15.157:0/2328980552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf375556b0 0x55bf375824f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:40.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.270+0000 7fa0a93f5700 1 --2- 172.21.15.157:0/2328980552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf3759a5b0 0x55bf37555170 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa094014d10 tx=0x7fa09401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef8e44510c2507c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:40.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.270+0000 7fa08effd700 1 -- 172.21.15.157:0/2328980552 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094015b70 con 0x55bf3759a5b0 2023-10-06T03:57:40.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.270+0000 7fa08effd700 1 -- 172.21.15.157:0/2328980552 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa094019070 con 0x55bf3759a5b0 2023-10-06T03:57:40.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.270+0000 7fa08effd700 1 -- 172.21.15.157:0/2328980552 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094022430 con 0x55bf3759a5b0 2023-10-06T03:57:40.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.270+0000 7fa0b5152b80 1 -- 172.21.15.157:0/2328980552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf3759a5b0 msgr2=0x55bf37555170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:40.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.270+0000 7fa0b5152b80 1 --2- 172.21.15.157:0/2328980552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf3759a5b0 0x55bf37555170 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa094014d10 tx=0x7fa09401c000 comp rx=0 tx=0).stop 2023-10-06T03:57:40.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.270+0000 7fa0b5152b80 1 -- 172.21.15.157:0/2328980552 shutdown_connections 2023-10-06T03:57:40.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.270+0000 7fa0b5152b80 1 --2- 172.21.15.157:0/2328980552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf375556b0 0x55bf375824f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:40.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.270+0000 7fa0b5152b80 1 --2- 172.21.15.157:0/2328980552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf3759a1e0 0x55bf37599490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:40.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.270+0000 7fa0b5152b80 1 --2- 172.21.15.157:0/2328980552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf3759a5b0 0x55bf37555170 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:40.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.270+0000 7fa0b5152b80 1 -- 172.21.15.157:0/2328980552 >> 172.21.15.157:0/2328980552 conn(0x55bf3755a9b0 msgr2=0x55bf3755ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:40.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.270+0000 7fa0b5152b80 1 -- 172.21.15.157:0/2328980552 shutdown_connections 2023-10-06T03:57:40.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.270+0000 7fa0b5152b80 1 -- 172.21.15.157:0/2328980552 wait complete. 2023-10-06T03:57:40.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.271+0000 7fa0b5152b80 1 -- start start 2023-10-06T03:57:40.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.271+0000 7fa0b5152b80 1 Processor -- start 2023-10-06T03:57:40.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.272+0000 7fa0b5152b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf375556b0 0x55bf375826a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:40.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.272+0000 7fa0b5152b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf3759a1e0 0x55bf375adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:40.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.272+0000 7fa0b5152b80 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf3759a5b0 0x55bf37580a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:40.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.272+0000 7fa0b5152b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf374a9060 con 0x55bf375556b0 2023-10-06T03:57:40.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.272+0000 7fa0b5152b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf37454890 con 0x55bf3759a1e0 2023-10-06T03:57:40.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.272+0000 7fa0b5152b80 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf3746fea0 con 0x55bf3759a5b0 2023-10-06T03:57:40.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.272+0000 7fa0b5152b80 10 client.0 _ll_register_callbacks cb 0x55bf37345f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:57:40.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.272+0000 7fa0a9bf6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf375556b0 0x55bf375826a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:40.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.272+0000 7fa0a9bf6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf375556b0 0x55bf375826a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35752/0 (socket says 172.21.15.157:35752) 2023-10-06T03:57:40.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.272+0000 7fa0a9bf6700 1 -- 172.21.15.157:0/3964887052 learned_addr learned my addr 172.21.15.157:0/3964887052 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:40.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.272+0000 7fa0a93f5700 1 --2- 172.21.15.157:0/3964887052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf3759a1e0 0x55bf375adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:40.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.272+0000 7fa0aa3f7700 1 --2- 172.21.15.157:0/3964887052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf3759a5b0 0x55bf37580a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:40.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.272+0000 7fa0a9bf6700 1 -- 172.21.15.157:0/3964887052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf3759a5b0 msgr2=0x55bf37580a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:40.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.272+0000 7fa0a9bf6700 1 --2- 172.21.15.157:0/3964887052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf3759a5b0 0x55bf37580a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:40.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.272+0000 7fa0a9bf6700 1 -- 172.21.15.157:0/3964887052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf3759a1e0 msgr2=0x55bf375adfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:40.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.272+0000 7fa0a9bf6700 1 --2- 172.21.15.157:0/3964887052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf3759a1e0 0x55bf375adfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:40.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.272+0000 7fa0a9bf6700 1 -- 172.21.15.157:0/3964887052 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0940148c0 con 0x55bf375556b0 2023-10-06T03:57:40.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.272+0000 7fa0aa3f7700 1 --2- 172.21.15.157:0/3964887052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf3759a5b0 0x55bf37580a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:40.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.272+0000 7fa0a9bf6700 1 --2- 172.21.15.157:0/3964887052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf375556b0 0x55bf375826a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0001750 tx=0x7fa0a0051dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db77db6245ec4395 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:40.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.272+0000 7fa083fff700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-10-06T03:57:40.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.272+0000 7fa083fff700 1 -- 172.21.15.157:0/3964887052 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a001c300 con 0x55bf375556b0 2023-10-06T03:57:40.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.272+0000 7fa083fff700 1 -- 172.21.15.157:0/3964887052 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa0a001c460 con 0x55bf375556b0 2023-10-06T03:57:40.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.272+0000 7fa0b5152b80 1 -- 172.21.15.157:0/3964887052 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bf3754d690 con 0x55bf375556b0 2023-10-06T03:57:40.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.272+0000 7fa083fff700 1 -- 172.21.15.157:0/3964887052 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0a0055dd0 con 0x55bf375556b0 2023-10-06T03:57:40.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.273+0000 7fa0b5152b80 10 client.4761 fetch_fsmap learned FSMap version 16 2023-10-06T03:57:40.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.273+0000 7fa0b5152b80 1 -- 172.21.15.157:0/3964887052 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55bf372e5fe0 con 0x55bf375556b0 2023-10-06T03:57:40.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.273+0000 7fa083fff700 1 -- 172.21.15.157:0/3964887052 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bf3754d690 con 0x55bf375556b0 2023-10-06T03:57:40.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.273+0000 7fa083fff700 1 -- 172.21.15.157:0/3964887052 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 16) v1 ==== 4238+0+0 (secure 0 0 0) 0x7fa0a0054e50 con 0x55bf375556b0 2023-10-06T03:57:40.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.273+0000 7fa0b5152b80 10 client.4761 fetch_fsmap finished waiting for FSMap version 16 2023-10-06T03:57:40.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.273+0000 7fa0b5152b80 10 client.4761 resolve_mds: resolved name 'f' to daemon mds.f 2023-10-06T03:57:40.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.273+0000 7fa0b5152b80 20 client.4761 populate_metadata read hostname 'smithi157' 2023-10-06T03:57:40.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.273+0000 7fa0b5152b80 1 --2- 172.21.15.157:0/3964887052 >> [v2:172.21.15.175:6838/4177493830,v1:172.21.15.175:6839/4177493830] conn(0x55bf375ae520 0x55bf37558440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:40.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.273+0000 7fa0b5152b80 4 client.4761 mds_command: new command op to 4241 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T03:57:40.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.273+0000 7fa0b5152b80 1 -- 172.21.15.157:0/3964887052 --> [v2:172.21.15.175:6838/4177493830,v1:172.21.15.175:6839/4177493830] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf3752a5c0 con 0x55bf375ae520 2023-10-06T03:57:40.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.273+0000 7fa0a93f5700 1 --2- 172.21.15.157:0/3964887052 >> [v2:172.21.15.175:6838/4177493830,v1:172.21.15.175:6839/4177493830] conn(0x55bf375ae520 0x55bf37558440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:40.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.273+0000 7fa083fff700 10 client.4761 ms_handle_connect on v2:172.21.15.175:6838/4177493830 2023-10-06T03:57:40.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:40.273+0000 7fa0a93f5700 1 --2- 172.21.15.157:0/3964887052 >> [v2:172.21.15.175:6838/4177493830,v1:172.21.15.175:6839/4177493830] conn(0x55bf375ae520 0x55bf37558440 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c247314a16d07529 server_cookie=a6b825aeec8629ca in_seq=0 out_seq=0 2023-10-06T03:57:42.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.654+0000 7fa083fff700 1 -- 172.21.15.157:0/3964887052 <== mds.4 v2:172.21.15.175:6838/4177493830 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55bf3752a5c0 con 0x55bf375ae520 2023-10-06T03:57:42.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.654+0000 7fa083fff700 10 client.4761 handle_command_reply: tid=0 2023-10-06T03:57:42.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.655+0000 7fa0b5152b80 1 client.4761 shutdown 2023-10-06T03:57:42.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.655+0000 7fa0b5152b80 1 -- 172.21.15.157:0/3964887052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf375556b0 msgr2=0x55bf375826a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:42.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.655+0000 7fa0b5152b80 1 --2- 172.21.15.157:0/3964887052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf375556b0 0x55bf375826a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0001750 tx=0x7fa0a0051dd0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.655+0000 7fa0b5152b80 1 -- 172.21.15.157:0/3964887052 shutdown_connections 2023-10-06T03:57:42.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.655+0000 7fa0b5152b80 1 --2- 172.21.15.157:0/3964887052 >> [v2:172.21.15.175:6838/4177493830,v1:172.21.15.175:6839/4177493830] conn(0x55bf375ae520 0x55bf37558440 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.655+0000 7fa0b5152b80 1 --2- 172.21.15.157:0/3964887052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf3759a5b0 0x55bf37580a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.655+0000 7fa0b5152b80 1 --2- 172.21.15.157:0/3964887052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf375556b0 0x55bf375826a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.655+0000 7fa0b5152b80 1 --2- 172.21.15.157:0/3964887052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf3759a1e0 0x55bf375adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.655+0000 7fa083fff700 0 client.4761 ms_handle_reset on v2:172.21.15.175:6838/4177493830 2023-10-06T03:57:42.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.655+0000 7fa0b5152b80 1 -- 172.21.15.157:0/3964887052 >> 172.21.15.157:0/3964887052 conn(0x55bf3755a9b0 msgr2=0x55bf37555b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:42.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.655+0000 7fa0b5152b80 1 -- 172.21.15.157:0/3964887052 shutdown_connections 2023-10-06T03:57:42.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.655+0000 7fa0b5152b80 1 -- 172.21.15.157:0/3964887052 wait complete. 2023-10-06T03:57:42.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.655+0000 7fa0b5152b80 20 client.4761 trim_cache size 0 max 16384 2023-10-06T03:57:42.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.674+0000 7fa0b5152b80 1 Processor -- start 2023-10-06T03:57:42.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.674+0000 7fa0b5152b80 1 -- start start 2023-10-06T03:57:42.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.674+0000 7fa0b5152b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf375556b0 0x55bf37598e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.674+0000 7fa0b5152b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf3759a1e0 0x55bf37555a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.674+0000 7fa0b5152b80 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf3759a5b0 0x55bf37555fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.674+0000 7fa0b5152b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf3732be30 con 0x55bf375556b0 2023-10-06T03:57:42.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.674+0000 7fa0b5152b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf375805c0 con 0x55bf3759a1e0 2023-10-06T03:57:42.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.674+0000 7fa0b5152b80 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf375ae380 con 0x55bf3759a5b0 2023-10-06T03:57:42.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.675+0000 7fa0a9bf6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf375556b0 0x55bf37598e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.675+0000 7fa0a9bf6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf375556b0 0x55bf37598e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35754/0 (socket says 172.21.15.157:35754) 2023-10-06T03:57:42.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.675+0000 7fa0a9bf6700 1 -- 172.21.15.157:0/700319327 learned_addr learned my addr 172.21.15.157:0/700319327 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:42.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.675+0000 7fa0aa3f7700 1 --2- 172.21.15.157:0/700319327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf3759a5b0 0x55bf37555fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.675+0000 7fa0a93f5700 1 --2- 172.21.15.157:0/700319327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf3759a1e0 0x55bf37555a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.675+0000 7fa0a9bf6700 1 -- 172.21.15.157:0/700319327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf3759a5b0 msgr2=0x55bf37555fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:42.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.675+0000 7fa0a9bf6700 1 --2- 172.21.15.157:0/700319327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf3759a5b0 0x55bf37555fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.675+0000 7fa0a9bf6700 1 -- 172.21.15.157:0/700319327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf3759a1e0 msgr2=0x55bf37555a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:42.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.675+0000 7fa0a9bf6700 1 --2- 172.21.15.157:0/700319327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf3759a1e0 0x55bf37555a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.675+0000 7fa0a9bf6700 1 -- 172.21.15.157:0/700319327 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf37581ca0 con 0x55bf375556b0 2023-10-06T03:57:42.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.675+0000 7fa0aa3f7700 1 --2- 172.21.15.157:0/700319327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf3759a5b0 0x55bf37555fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:42.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.675+0000 7fa0a9bf6700 1 --2- 172.21.15.157:0/700319327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf375556b0 0x55bf37598e70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0055600 tx=0x7fa0a006e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf484951e884a13 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:42.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.675+0000 7fa08f7fe700 1 -- 172.21.15.157:0/700319327 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a00552c0 con 0x55bf375556b0 2023-10-06T03:57:42.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.675+0000 7fa08f7fe700 1 -- 172.21.15.157:0/700319327 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a0055dd0 con 0x55bf375556b0 2023-10-06T03:57:42.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.675+0000 7fa08f7fe700 1 -- 172.21.15.157:0/700319327 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a000cbf0 con 0x55bf375556b0 2023-10-06T03:57:42.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.676+0000 7fa0b5152b80 1 -- 172.21.15.157:0/700319327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf375556b0 msgr2=0x55bf37598e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:42.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.676+0000 7fa0b5152b80 1 --2- 172.21.15.157:0/700319327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf375556b0 0x55bf37598e70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0055600 tx=0x7fa0a006e000 comp rx=0 tx=0).stop 2023-10-06T03:57:42.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.676+0000 7fa0b5152b80 1 -- 172.21.15.157:0/700319327 shutdown_connections 2023-10-06T03:57:42.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.676+0000 7fa0b5152b80 1 --2- 172.21.15.157:0/700319327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf3759a5b0 0x55bf37555fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.676+0000 7fa0b5152b80 1 --2- 172.21.15.157:0/700319327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf375556b0 0x55bf37598e70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.676+0000 7fa0b5152b80 1 --2- 172.21.15.157:0/700319327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf3759a1e0 0x55bf37555a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.676+0000 7fa0b5152b80 1 -- 172.21.15.157:0/700319327 >> 172.21.15.157:0/700319327 conn(0x55bf3755a9b0 msgr2=0x55bf3755b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:42.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.676+0000 7fa0b5152b80 1 -- 172.21.15.157:0/700319327 shutdown_connections 2023-10-06T03:57:42.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.676+0000 7fa0b5152b80 1 -- 172.21.15.157:0/700319327 wait complete. 2023-10-06T03:57:42.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.677+0000 7fa0b5152b80 1 -- start start 2023-10-06T03:57:42.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.677+0000 7fa0b5152b80 1 Processor -- start 2023-10-06T03:57:42.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.678+0000 7fa0b5152b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf375556b0 0x55bf375fad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.678+0000 7fa0b5152b80 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf3759a1e0 0x55bf3755aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.678+0000 7fa0b5152b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf3759a5b0 0x55bf3755b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.678+0000 7fa0b5152b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf375fb250 con 0x55bf375556b0 2023-10-06T03:57:42.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.678+0000 7fa0b5152b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf375587a0 con 0x55bf3759a5b0 2023-10-06T03:57:42.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.678+0000 7fa0b5152b80 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf37581720 con 0x55bf3759a1e0 2023-10-06T03:57:42.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.678+0000 7fa0b5152b80 10 client.0 _ll_register_callbacks cb 0x55bf37345f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:57:42.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.678+0000 7fa0a9bf6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf375556b0 0x55bf375fad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.678+0000 7fa0a9bf6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf375556b0 0x55bf375fad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35770/0 (socket says 172.21.15.157:35770) 2023-10-06T03:57:42.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.678+0000 7fa0a9bf6700 1 -- 172.21.15.157:0/2493637861 learned_addr learned my addr 172.21.15.157:0/2493637861 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:42.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.678+0000 7fa0a93f5700 1 --2- 172.21.15.157:0/2493637861 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf3759a1e0 0x55bf3755aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.678+0000 7fa0aa3f7700 1 --2- 172.21.15.157:0/2493637861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf3759a5b0 0x55bf3755b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.679+0000 7fa0a9bf6700 1 -- 172.21.15.157:0/2493637861 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf3759a1e0 msgr2=0x55bf3755aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:42.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.679+0000 7fa0a9bf6700 1 --2- 172.21.15.157:0/2493637861 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf3759a1e0 0x55bf3755aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.679+0000 7fa0a9bf6700 1 -- 172.21.15.157:0/2493637861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf3759a5b0 msgr2=0x55bf3755b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:42.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.679+0000 7fa0a9bf6700 1 --2- 172.21.15.157:0/2493637861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf3759a5b0 0x55bf3755b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.679+0000 7fa0a9bf6700 1 -- 172.21.15.157:0/2493637861 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0940148c0 con 0x55bf375556b0 2023-10-06T03:57:42.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.679+0000 7fa0a9bf6700 1 --2- 172.21.15.157:0/2493637861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf375556b0 0x55bf375fad10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa0a00558f0 tx=0x7fa0a0082000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db032fc370604a27 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:42.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.679+0000 7fa083fff700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-10-06T03:57:42.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.679+0000 7fa083fff700 1 -- 172.21.15.157:0/2493637861 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a00552c0 con 0x55bf375556b0 2023-10-06T03:57:42.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.679+0000 7fa083fff700 1 -- 172.21.15.157:0/2493637861 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa0a000cbf0 con 0x55bf375556b0 2023-10-06T03:57:42.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.679+0000 7fa0b5152b80 1 -- 172.21.15.157:0/2493637861 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa094019070 con 0x55bf375556b0 2023-10-06T03:57:42.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.679+0000 7fa083fff700 1 -- 172.21.15.157:0/2493637861 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0a005c680 con 0x55bf375556b0 2023-10-06T03:57:42.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.679+0000 7fa0b5152b80 10 client.4767 fetch_fsmap learned FSMap version 16 2023-10-06T03:57:42.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.679+0000 7fa0b5152b80 1 -- 172.21.15.157:0/2493637861 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55bf37477780 con 0x55bf375556b0 2023-10-06T03:57:42.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.679+0000 7fa083fff700 1 -- 172.21.15.157:0/2493637861 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a0076080 con 0x55bf375556b0 2023-10-06T03:57:42.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.679+0000 7fa083fff700 1 -- 172.21.15.157:0/2493637861 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 16) v1 ==== 4238+0+0 (secure 0 0 0) 0x7fa0a00712c0 con 0x55bf375556b0 2023-10-06T03:57:42.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.679+0000 7fa0b5152b80 10 client.4767 fetch_fsmap finished waiting for FSMap version 16 2023-10-06T03:57:42.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.679+0000 7fa0b5152b80 10 client.4767 resolve_mds: resolved name 'f' to daemon mds.f 2023-10-06T03:57:42.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.679+0000 7fa0b5152b80 20 client.4767 populate_metadata read hostname 'smithi157' 2023-10-06T03:57:42.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.679+0000 7fa0b5152b80 1 --2- 172.21.15.157:0/2493637861 >> [v2:172.21.15.175:6838/4177493830,v1:172.21.15.175:6839/4177493830] conn(0x55bf375580a0 0x55bf37600a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.679+0000 7fa0b5152b80 4 client.4767 mds_command: new command op to 4241 tid=0[{"prefix": "status"}] 2023-10-06T03:57:42.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.679+0000 7fa0b5152b80 1 -- 172.21.15.157:0/2493637861 --> [v2:172.21.15.175:6838/4177493830,v1:172.21.15.175:6839/4177493830] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bf3744be00 con 0x55bf375580a0 2023-10-06T03:57:42.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.680+0000 7fa0a93f5700 1 --2- 172.21.15.157:0/2493637861 >> [v2:172.21.15.175:6838/4177493830,v1:172.21.15.175:6839/4177493830] conn(0x55bf375580a0 0x55bf37600a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.680+0000 7fa083fff700 10 client.4767 ms_handle_connect on v2:172.21.15.175:6838/4177493830 2023-10-06T03:57:42.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.680+0000 7fa0a93f5700 1 --2- 172.21.15.157:0/2493637861 >> [v2:172.21.15.175:6838/4177493830,v1:172.21.15.175:6839/4177493830] conn(0x55bf375580a0 0x55bf37600a50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9dffc5e38e354d85 server_cookie=7d11e2264de6db2a in_seq=0 out_seq=0 2023-10-06T03:57:42.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.680+0000 7fa083fff700 1 -- 172.21.15.157:0/2493637861 <== mds.4 v2:172.21.15.175:6838/4177493830 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55bf3744be00 con 0x55bf375580a0 2023-10-06T03:57:42.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.680+0000 7fa083fff700 10 client.4767 handle_command_reply: tid=0 2023-10-06T03:57:42.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.680+0000 7fa0b5152b80 1 client.4767 shutdown 2023-10-06T03:57:42.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.680+0000 7fa0b5152b80 1 -- 172.21.15.157:0/2493637861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf375556b0 msgr2=0x55bf375fad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:42.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.680+0000 7fa0b5152b80 1 --2- 172.21.15.157:0/2493637861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf375556b0 0x55bf375fad10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa0a00558f0 tx=0x7fa0a0082000 comp rx=0 tx=0).stop 2023-10-06T03:57:42.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.680+0000 7fa0b5152b80 1 -- 172.21.15.157:0/2493637861 shutdown_connections 2023-10-06T03:57:42.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.680+0000 7fa0b5152b80 1 --2- 172.21.15.157:0/2493637861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55bf3759a5b0 0x55bf3755b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.680+0000 7fa0b5152b80 1 --2- 172.21.15.157:0/2493637861 >> [v2:172.21.15.175:6838/4177493830,v1:172.21.15.175:6839/4177493830] conn(0x55bf375580a0 0x55bf37600a50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.680+0000 7fa0b5152b80 1 --2- 172.21.15.157:0/2493637861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bf375556b0 0x55bf375fad10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.680+0000 7fa0b5152b80 1 --2- 172.21.15.157:0/2493637861 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55bf3759a1e0 0x55bf3755aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.680+0000 7fa083fff700 0 client.4767 ms_handle_reset on v2:172.21.15.175:6838/4177493830 2023-10-06T03:57:42.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.680+0000 7fa0b5152b80 1 -- 172.21.15.157:0/2493637861 >> 172.21.15.157:0/2493637861 conn(0x55bf3755a9b0 msgr2=0x55bf37555a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:42.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.681+0000 7fa0b5152b80 1 -- 172.21.15.157:0/2493637861 shutdown_connections 2023-10-06T03:57:42.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.681+0000 7fa0b5152b80 1 -- 172.21.15.157:0/2493637861 wait complete. 2023-10-06T03:57:42.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.681+0000 7fa0b5152b80 20 client.4767 trim_cache size 0 max 16384 2023-10-06T03:57:42.689 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-10-06T03:57:42.689 INFO:teuthology.orchestra.run.smithi157.stdout: "cluster_fsid": "28664a7a-fe30-4efe-be5a-6abf9318b700", 2023-10-06T03:57:42.689 INFO:teuthology.orchestra.run.smithi157.stdout: "whoami": 4, 2023-10-06T03:57:42.689 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 4241, 2023-10-06T03:57:42.690 INFO:teuthology.orchestra.run.smithi157.stdout: "want_state": "up:active", 2023-10-06T03:57:42.690 INFO:teuthology.orchestra.run.smithi157.stdout: "state": "up:active", 2023-10-06T03:57:42.690 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs", 2023-10-06T03:57:42.690 INFO:teuthology.orchestra.run.smithi157.stdout: "rank_uptime": 10.044917158000001, 2023-10-06T03:57:42.690 INFO:teuthology.orchestra.run.smithi157.stdout: "mdsmap_epoch": 16, 2023-10-06T03:57:42.690 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch": 21, 2023-10-06T03:57:42.690 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch_barrier": 21, 2023-10-06T03:57:42.690 INFO:teuthology.orchestra.run.smithi157.stdout: "uptime": 37.224566404999997 2023-10-06T03:57:42.690 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-10-06T03:57:42.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.681+0000 7fa0aabf8700 1 -- 172.21.15.157:0/1593925695 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa088066f30 msgr2=0x7fa0880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:42.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.681+0000 7fa0aabf8700 1 --2- 172.21.15.157:0/1593925695 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa088066f30 0x7fa0880693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa0a40b0d50 tx=0x7fa0940091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.681+0000 7fa0aabf8700 1 -- 172.21.15.157:0/1593925695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0900051c0 msgr2=0x7fa0a40c3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:42.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.681+0000 7fa0aabf8700 1 --2- 172.21.15.157:0/1593925695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0900051c0 0x7fa0a40c3d30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000ebc0 tx=0x7fa0a0001640 comp rx=0 tx=0).stop 2023-10-06T03:57:42.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.681+0000 7fa0aabf8700 1 -- 172.21.15.157:0/1593925695 shutdown_connections 2023-10-06T03:57:42.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.681+0000 7fa0aabf8700 1 --2- 172.21.15.157:0/1593925695 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa088066f30 0x7fa0880693e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.681+0000 7fa0aabf8700 1 --2- 172.21.15.157:0/1593925695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0a4150ad0 0x7fa0a414ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.681+0000 7fa0aabf8700 1 --2- 172.21.15.157:0/1593925695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0900051c0 0x7fa0a40c3d30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.681+0000 7fa0aabf8700 1 --2- 172.21.15.157:0/1593925695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0a414ba00 0x7fa0a414c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.681+0000 7fa0aabf8700 1 -- 172.21.15.157:0/1593925695 >> 172.21.15.157:0/1593925695 conn(0x7fa0a40cc540 msgr2=0x7fa0a406d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:42.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.681+0000 7fa0aabf8700 1 -- 172.21.15.157:0/1593925695 shutdown_connections 2023-10-06T03:57:42.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.681+0000 7fa0aabf8700 1 -- 172.21.15.157:0/1593925695 wait complete. 2023-10-06T03:57:42.697 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T03:57:42.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.817+0000 7f86a6234700 1 Processor -- start 2023-10-06T03:57:42.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.817+0000 7f86a6234700 1 -- start start 2023-10-06T03:57:42.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.818+0000 7f86a6234700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f86a0150410 0x7f86a0155f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.818+0000 7f86a6234700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86a00c9780 con 0x7f86a006dc40 2023-10-06T03:57:42.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.818+0000 7f86a6234700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86a006e0c0 con 0x7f86a006cbb0 2023-10-06T03:57:42.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.818+0000 7f86a6234700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86a00c8c40 con 0x7f86a0150410 2023-10-06T03:57:42.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.818+0000 7f869effd700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f86a006dc40 0x7f86a014fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39746/0 (socket says 172.21.15.157:39746) 2023-10-06T03:57:42.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.818+0000 7f869effd700 1 -- 172.21.15.157:0/327698121 learned_addr learned my addr 172.21.15.157:0/327698121 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:42.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.818+0000 7f869ffff700 1 --2- 172.21.15.157:0/327698121 >> v2:172.21.15.175:3300/0 conn(0x7f86a0150410 0x7f86a0155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.818+0000 7f869e7fc700 1 -- 172.21.15.157:0/327698121 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4228861770 0 0) 0x7f86a00c9780 con 0x7f86a006dc40 2023-10-06T03:57:42.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.818+0000 7f869e7fc700 1 -- 172.21.15.157:0/327698121 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8688003570 con 0x7f86a006dc40 2023-10-06T03:57:42.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.819+0000 7f869ffff700 1 -- 172.21.15.157:0/327698121 >> v1:172.21.15.175:6789/0 conn(0x7f86a006cbb0 legacy=0x7f86a006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:42.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.819+0000 7f869e7fc700 1 -- 172.21.15.157:0/327698121 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2393696573 0 0) 0x7f8688003570 con 0x7f86a006dc40 2023-10-06T03:57:42.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.819+0000 7f869ffff700 1 -- 172.21.15.157:0/327698121 >> v1:172.21.15.157:6789/0 conn(0x7f86a006dc40 legacy=0x7f86a014fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:42.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.819+0000 7f869ffff700 1 -- 172.21.15.157:0/327698121 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a00c8e50 con 0x7f86a0150410 2023-10-06T03:57:42.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.819+0000 7f869ffff700 1 --2- 172.21.15.157:0/327698121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86a0150410 0x7f86a0155f70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f86980098e0 tx=0x7f86980091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9f81b87f7d1fd62 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:42.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.819+0000 7f869e7fc700 1 -- 172.21.15.157:0/327698121 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8694001470 con 0x7f86a0150410 2023-10-06T03:57:42.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.819+0000 7f869e7fc700 1 -- 172.21.15.157:0/327698121 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8698011420 con 0x7f86a0150410 2023-10-06T03:57:42.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.819+0000 7f869e7fc700 1 -- 172.21.15.157:0/327698121 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86980117c0 con 0x7f86a0150410 2023-10-06T03:57:42.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.819+0000 7f86a6234700 1 -- 172.21.15.157:0/327698121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86a0150410 msgr2=0x7f86a0155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:42.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.819+0000 7f86a6234700 1 --2- 172.21.15.157:0/327698121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86a0150410 0x7f86a0155f70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f86980098e0 tx=0x7f86980091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.819+0000 7f86a6234700 1 -- 172.21.15.157:0/327698121 shutdown_connections 2023-10-06T03:57:42.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.819+0000 7f86a6234700 1 --2- 172.21.15.157:0/327698121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86a0150410 0x7f86a0155f70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.819+0000 7f86a6234700 1 -- 172.21.15.157:0/327698121 >> 172.21.15.157:0/327698121 conn(0x7f86a006f230 msgr2=0x7f86a006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:42.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.819+0000 7f86a6234700 1 -- 172.21.15.157:0/327698121 shutdown_connections 2023-10-06T03:57:42.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.819+0000 7f86a6234700 1 -- 172.21.15.157:0/327698121 wait complete. 2023-10-06T03:57:42.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.821+0000 7f86a6234700 1 Processor -- start 2023-10-06T03:57:42.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.821+0000 7f86a6234700 1 -- start start 2023-10-06T03:57:42.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.822+0000 7f86a6234700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86a006cbb0 0x7f86a0157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.822+0000 7f86a6234700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86a0150410 0x7f86a0159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.822+0000 7f86a6234700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86a00c3220 0x7f86a0159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.822+0000 7f86a6234700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a0158240 con 0x7f86a0150410 2023-10-06T03:57:42.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.822+0000 7f86a6234700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a0158ec0 con 0x7f86a00c3220 2023-10-06T03:57:42.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.822+0000 7f86a6234700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a0166fb0 con 0x7f86a006cbb0 2023-10-06T03:57:42.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.822+0000 7f869effd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86a0150410 0x7f86a0159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.822+0000 7f869effd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86a0150410 0x7f86a0159350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35782/0 (socket says 172.21.15.157:35782) 2023-10-06T03:57:42.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.822+0000 7f869effd700 1 -- 172.21.15.157:0/2076256249 learned_addr learned my addr 172.21.15.157:0/2076256249 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:42.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.822+0000 7f869f7fe700 1 --2- 172.21.15.157:0/2076256249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86a006cbb0 0x7f86a0157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.822+0000 7f869ffff700 1 --2- 172.21.15.157:0/2076256249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86a00c3220 0x7f86a0159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.822+0000 7f869effd700 1 -- 172.21.15.157:0/2076256249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86a006cbb0 msgr2=0x7f86a0157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:42.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.822+0000 7f869effd700 1 --2- 172.21.15.157:0/2076256249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86a006cbb0 0x7f86a0157300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.822+0000 7f869effd700 1 -- 172.21.15.157:0/2076256249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86a00c3220 msgr2=0x7f86a0159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:42.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.822+0000 7f869effd700 1 --2- 172.21.15.157:0/2076256249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86a00c3220 0x7f86a0159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.822+0000 7f869effd700 1 -- 172.21.15.157:0/2076256249 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8698009420 con 0x7f86a0150410 2023-10-06T03:57:42.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.822+0000 7f869effd700 1 --2- 172.21.15.157:0/2076256249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86a0150410 0x7f86a0159350 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f869400cbc0 tx=0x7f8694001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:42.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.822+0000 7f86877fe700 1 -- 172.21.15.157:0/2076256249 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8694007e90 con 0x7f86a0150410 2023-10-06T03:57:42.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.822+0000 7f86877fe700 1 -- 172.21.15.157:0/2076256249 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8694007360 con 0x7f86a0150410 2023-10-06T03:57:42.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.822+0000 7f86a6234700 1 -- 172.21.15.157:0/2076256249 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86a0166cd0 con 0x7f86a0150410 2023-10-06T03:57:42.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.822+0000 7f86877fe700 1 -- 172.21.15.157:0/2076256249 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8694004830 con 0x7f86a0150410 2023-10-06T03:57:42.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.822+0000 7f86a6234700 1 -- 172.21.15.157:0/2076256249 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a0157f60 con 0x7f86a0150410 2023-10-06T03:57:42.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.824+0000 7f86877fe700 1 -- 172.21.15.157:0/2076256249 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8694004be0 con 0x7f86a0150410 2023-10-06T03:57:42.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.824+0000 7f86877fe700 1 --2- 172.21.15.157:0/2076256249 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f868c066f60 0x7f868c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.824+0000 7f86877fe700 1 -- 172.21.15.157:0/2076256249 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8694083a80 con 0x7f86a0150410 2023-10-06T03:57:42.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.824+0000 7f869f7fe700 1 --2- 172.21.15.157:0/2076256249 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f868c066f60 0x7f868c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.825+0000 7f869f7fe700 1 --2- 172.21.15.157:0/2076256249 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f868c066f60 0x7f868c069410 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f869000e1b0 tx=0x7f8690001770 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:42.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.826+0000 7f86af52cb80 1 Processor -- start 2023-10-06T03:57:42.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.826+0000 7f86af52cb80 1 -- start start 2023-10-06T03:57:42.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.827+0000 7f86af52cb80 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3c45a3f50 0x55b3c45a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.827+0000 7f86af52cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3c4560bd0 0x55b3c45a83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.827+0000 7f86af52cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3c45a8920 0x55b3c459e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.827+0000 7f86af52cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c44e2c80 con 0x55b3c45a8920 2023-10-06T03:57:42.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.827+0000 7f86af52cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c44b9a70 con 0x55b3c4560bd0 2023-10-06T03:57:42.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.827+0000 7f86af52cb80 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3c44b7270 con 0x55b3c45a3f50 2023-10-06T03:57:42.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.827+0000 7f869ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3c45a8920 0x55b3c459e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.827+0000 7f869ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3c45a8920 0x55b3c459e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35796/0 (socket says 172.21.15.157:35796) 2023-10-06T03:57:42.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.827+0000 7f869ffff700 1 -- 172.21.15.157:0/1276485096 learned_addr learned my addr 172.21.15.157:0/1276485096 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:42.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.827+0000 7f869f7fe700 1 --2- 172.21.15.157:0/1276485096 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3c45a3f50 0x55b3c45a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.827+0000 7f869effd700 1 --2- 172.21.15.157:0/1276485096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3c4560bd0 0x55b3c45a83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.827+0000 7f869ffff700 1 -- 172.21.15.157:0/1276485096 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3c45a3f50 msgr2=0x55b3c45a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:42.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.827+0000 7f869ffff700 1 --2- 172.21.15.157:0/1276485096 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3c45a3f50 0x55b3c45a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.827+0000 7f869ffff700 1 -- 172.21.15.157:0/1276485096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3c4560bd0 msgr2=0x55b3c45a83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:42.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.827+0000 7f869ffff700 1 --2- 172.21.15.157:0/1276485096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3c4560bd0 0x55b3c45a83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.827+0000 7f869ffff700 1 -- 172.21.15.157:0/1276485096 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3c434d270 con 0x55b3c45a8920 2023-10-06T03:57:42.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.827+0000 7f869effd700 1 --2- 172.21.15.157:0/1276485096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3c4560bd0 0x55b3c45a83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:57:42.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.828+0000 7f869ffff700 1 --2- 172.21.15.157:0/1276485096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3c45a8920 0x55b3c459e040 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f86980030c0 tx=0x7f8698017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6188bba92d3b40 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:42.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.828+0000 7f8684ff9700 1 -- 172.21.15.157:0/1276485096 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8698011b70 con 0x55b3c45a8920 2023-10-06T03:57:42.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.828+0000 7f8684ff9700 1 -- 172.21.15.157:0/1276485096 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8698011cd0 con 0x55b3c45a8920 2023-10-06T03:57:42.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.828+0000 7f8684ff9700 1 -- 172.21.15.157:0/1276485096 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8698027640 con 0x55b3c45a8920 2023-10-06T03:57:42.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.828+0000 7f86af52cb80 1 -- 172.21.15.157:0/1276485096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3c45a8920 msgr2=0x55b3c459e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:42.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.828+0000 7f86af52cb80 1 --2- 172.21.15.157:0/1276485096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3c45a8920 0x55b3c459e040 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f86980030c0 tx=0x7f8698017000 comp rx=0 tx=0).stop 2023-10-06T03:57:42.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.828+0000 7f86af52cb80 1 -- 172.21.15.157:0/1276485096 shutdown_connections 2023-10-06T03:57:42.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.828+0000 7f86af52cb80 1 --2- 172.21.15.157:0/1276485096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3c45a8920 0x55b3c459e040 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.828+0000 7f86af52cb80 1 --2- 172.21.15.157:0/1276485096 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3c45a3f50 0x55b3c45a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.828+0000 7f86af52cb80 1 --2- 172.21.15.157:0/1276485096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3c4560bd0 0x55b3c45a83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.828+0000 7f86af52cb80 1 -- 172.21.15.157:0/1276485096 >> 172.21.15.157:0/1276485096 conn(0x55b3c4561fb0 msgr2=0x55b3c4562380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:42.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.828+0000 7f86af52cb80 1 -- 172.21.15.157:0/1276485096 shutdown_connections 2023-10-06T03:57:42.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.828+0000 7f86af52cb80 1 -- 172.21.15.157:0/1276485096 wait complete. 2023-10-06T03:57:42.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.829+0000 7f86af52cb80 1 -- start start 2023-10-06T03:57:42.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.829+0000 7f86af52cb80 1 Processor -- start 2023-10-06T03:57:42.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.830+0000 7f86af52cb80 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3c4560bd0 0x55b3c459e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.830+0000 7f86af52cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3c45a3f50 0x55b3c45a5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.830+0000 7f86af52cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3c45a8920 0x55b3c45aa850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.830+0000 7f86af52cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c44b3060 con 0x55b3c45a3f50 2023-10-06T03:57:42.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.830+0000 7f86af52cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c445e890 con 0x55b3c45a8920 2023-10-06T03:57:42.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.830+0000 7f86af52cb80 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3c4479ea0 con 0x55b3c4560bd0 2023-10-06T03:57:42.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.830+0000 7f86af52cb80 10 client.0 _ll_register_callbacks cb 0x55b3c434ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:57:42.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.830+0000 7f869effd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3c45a3f50 0x55b3c45a5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.830+0000 7f869effd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3c45a3f50 0x55b3c45a5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35806/0 (socket says 172.21.15.157:35806) 2023-10-06T03:57:42.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.830+0000 7f869effd700 1 -- 172.21.15.157:0/1517424811 learned_addr learned my addr 172.21.15.157:0/1517424811 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:42.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.830+0000 7f869ffff700 1 --2- 172.21.15.157:0/1517424811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3c45a8920 0x55b3c45aa850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.830+0000 7f869f7fe700 1 --2- 172.21.15.157:0/1517424811 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3c4560bd0 0x55b3c459e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.830+0000 7f869effd700 1 -- 172.21.15.157:0/1517424811 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3c4560bd0 msgr2=0x55b3c459e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:42.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.830+0000 7f869effd700 1 --2- 172.21.15.157:0/1517424811 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3c4560bd0 0x55b3c459e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.830+0000 7f869effd700 1 -- 172.21.15.157:0/1517424811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3c45a8920 msgr2=0x55b3c45aa850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:42.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.830+0000 7f869effd700 1 --2- 172.21.15.157:0/1517424811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3c45a8920 0x55b3c45aa850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.830+0000 7f869effd700 1 -- 172.21.15.157:0/1517424811 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a00c8e50 con 0x55b3c45a3f50 2023-10-06T03:57:42.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.830+0000 7f869effd700 1 --2- 172.21.15.157:0/1517424811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3c45a3f50 0x55b3c45a5f90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f869401f030 tx=0x7f8694009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa554ff3849689cd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:42.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.830+0000 7f867dffb700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-10-06T03:57:42.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.830+0000 7f867dffb700 1 -- 172.21.15.157:0/1517424811 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8694054850 con 0x55b3c45a3f50 2023-10-06T03:57:42.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.830+0000 7f867dffb700 1 -- 172.21.15.157:0/1517424811 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8694057aa0 con 0x55b3c45a3f50 2023-10-06T03:57:42.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.830+0000 7f86af52cb80 1 -- 172.21.15.157:0/1517424811 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b3c4557690 con 0x55b3c45a3f50 2023-10-06T03:57:42.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.830+0000 7f867dffb700 1 -- 172.21.15.157:0/1517424811 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f869405b7f0 con 0x55b3c45a3f50 2023-10-06T03:57:42.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.831+0000 7f86af52cb80 10 client.4779 fetch_fsmap learned FSMap version 16 2023-10-06T03:57:42.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.831+0000 7f86af52cb80 1 -- 172.21.15.157:0/1517424811 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55b3c42effe0 con 0x55b3c45a3f50 2023-10-06T03:57:42.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.831+0000 7f867dffb700 1 -- 172.21.15.157:0/1517424811 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b3c4557690 con 0x55b3c45a3f50 2023-10-06T03:57:42.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.831+0000 7f867dffb700 1 -- 172.21.15.157:0/1517424811 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 16) v1 ==== 4238+0+0 (secure 0 0 0) 0x7f86940702a0 con 0x55b3c45a3f50 2023-10-06T03:57:42.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.831+0000 7f86af52cb80 10 client.4779 fetch_fsmap finished waiting for FSMap version 16 2023-10-06T03:57:42.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.831+0000 7f86af52cb80 10 client.4779 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T03:57:42.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.831+0000 7f86af52cb80 20 client.4779 populate_metadata read hostname 'smithi157' 2023-10-06T03:57:42.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.831+0000 7f86af52cb80 1 --2- 172.21.15.157:0/1517424811 >> [v2:172.21.15.175:6834/1596042861,v1:172.21.15.175:6835/1596042861] conn(0x55b3c45a76a0 0x55b3c45a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.831+0000 7f86af52cb80 4 client.4779 mds_command: new command op to 4270 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T03:57:42.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.831+0000 7f86af52cb80 1 -- 172.21.15.157:0/1517424811 --> [v2:172.21.15.175:6834/1596042861,v1:172.21.15.175:6835/1596042861] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b3c45345c0 con 0x55b3c45a76a0 2023-10-06T03:57:42.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.831+0000 7f869f7fe700 1 --2- 172.21.15.157:0/1517424811 >> [v2:172.21.15.175:6834/1596042861,v1:172.21.15.175:6835/1596042861] conn(0x55b3c45a76a0 0x55b3c45a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.831+0000 7f867dffb700 10 client.4779 ms_handle_connect on v2:172.21.15.175:6834/1596042861 2023-10-06T03:57:42.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.831+0000 7f869f7fe700 1 --2- 172.21.15.157:0/1517424811 >> [v2:172.21.15.175:6834/1596042861,v1:172.21.15.175:6835/1596042861] conn(0x55b3c45a76a0 0x55b3c45a7a70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c18496f149944ba7 server_cookie=7500a1bfa4f5801d in_seq=0 out_seq=0 2023-10-06T03:57:42.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.832+0000 7f867dffb700 1 -- 172.21.15.157:0/1517424811 <== mds.3 v2:172.21.15.175:6834/1596042861 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55b3c45345c0 con 0x55b3c45a76a0 2023-10-06T03:57:42.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.832+0000 7f867dffb700 10 client.4779 handle_command_reply: tid=0 2023-10-06T03:57:42.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.832+0000 7f86af52cb80 1 client.4779 shutdown 2023-10-06T03:57:42.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.832+0000 7f86af52cb80 1 -- 172.21.15.157:0/1517424811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3c45a3f50 msgr2=0x55b3c45a5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:42.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.832+0000 7f86af52cb80 1 --2- 172.21.15.157:0/1517424811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3c45a3f50 0x55b3c45a5f90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f869401f030 tx=0x7f8694009040 comp rx=0 tx=0).stop 2023-10-06T03:57:42.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.832+0000 7f86af52cb80 1 -- 172.21.15.157:0/1517424811 shutdown_connections 2023-10-06T03:57:42.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.832+0000 7f86af52cb80 1 --2- 172.21.15.157:0/1517424811 >> [v2:172.21.15.175:6834/1596042861,v1:172.21.15.175:6835/1596042861] conn(0x55b3c45a76a0 0x55b3c45a7a70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.832+0000 7f86af52cb80 1 --2- 172.21.15.157:0/1517424811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3c45a8920 0x55b3c45aa850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.832+0000 7f86af52cb80 1 --2- 172.21.15.157:0/1517424811 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3c4560bd0 0x55b3c459e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.832+0000 7f86af52cb80 1 --2- 172.21.15.157:0/1517424811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3c45a3f50 0x55b3c45a5f90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.832+0000 7f867dffb700 0 client.4779 ms_handle_reset on v2:172.21.15.175:6834/1596042861 2023-10-06T03:57:42.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.832+0000 7f86af52cb80 1 -- 172.21.15.157:0/1517424811 >> 172.21.15.157:0/1517424811 conn(0x55b3c4561fb0 msgr2=0x55b3c45a4320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:42.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.832+0000 7f86af52cb80 1 -- 172.21.15.157:0/1517424811 shutdown_connections 2023-10-06T03:57:42.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.832+0000 7f86af52cb80 1 -- 172.21.15.157:0/1517424811 wait complete. 2023-10-06T03:57:42.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.832+0000 7f86af52cb80 20 client.4779 trim_cache size 0 max 16384 2023-10-06T03:57:42.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.849+0000 7f86af52cb80 1 Processor -- start 2023-10-06T03:57:42.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.849+0000 7f86af52cb80 1 -- start start 2023-10-06T03:57:42.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.850+0000 7f86af52cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3c4560bd0 0x55b3c4596800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.850+0000 7f86af52cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3c45a3f50 0x55b3c45a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.850+0000 7f86af52cb80 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3c45a8920 0x55b3c45a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.850+0000 7f86af52cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c45be8a0 con 0x55b3c45a3f50 2023-10-06T03:57:42.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.850+0000 7f86af52cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c4335e30 con 0x55b3c4560bd0 2023-10-06T03:57:42.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.850+0000 7f86af52cb80 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3c45649b0 con 0x55b3c45a8920 2023-10-06T03:57:42.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.850+0000 7f869effd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3c45a3f50 0x55b3c45a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.850+0000 7f869effd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3c45a3f50 0x55b3c45a7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35812/0 (socket says 172.21.15.157:35812) 2023-10-06T03:57:42.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.850+0000 7f869effd700 1 -- 172.21.15.157:0/1430952495 learned_addr learned my addr 172.21.15.157:0/1430952495 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:42.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.850+0000 7f869ffff700 1 --2- 172.21.15.157:0/1430952495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3c45a8920 0x55b3c45a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.850+0000 7f869f7fe700 1 --2- 172.21.15.157:0/1430952495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3c4560bd0 0x55b3c4596800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.850+0000 7f869effd700 1 -- 172.21.15.157:0/1430952495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3c45a8920 msgr2=0x55b3c45a7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:42.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.850+0000 7f869effd700 1 --2- 172.21.15.157:0/1430952495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3c45a8920 0x55b3c45a7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.850+0000 7f869effd700 1 -- 172.21.15.157:0/1430952495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3c4560bd0 msgr2=0x55b3c4596800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:42.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.850+0000 7f869effd700 1 --2- 172.21.15.157:0/1430952495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3c4560bd0 0x55b3c4596800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.850+0000 7f869effd700 1 -- 172.21.15.157:0/1430952495 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3c4481780 con 0x55b3c45a3f50 2023-10-06T03:57:42.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.850+0000 7f869ffff700 1 --2- 172.21.15.157:0/1430952495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3c45a8920 0x55b3c45a7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:57:42.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.850+0000 7f869f7fe700 1 --2- 172.21.15.157:0/1430952495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3c4560bd0 0x55b3c4596800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:57:42.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.850+0000 7f869effd700 1 --2- 172.21.15.157:0/1430952495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3c45a3f50 0x55b3c45a7a70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f869400a340 tx=0x7f8694069000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6695e4ca5c957d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:42.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.850+0000 7f86857fa700 1 -- 172.21.15.157:0/1430952495 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8694057aa0 con 0x55b3c45a3f50 2023-10-06T03:57:42.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.851+0000 7f86857fa700 1 -- 172.21.15.157:0/1430952495 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8694069070 con 0x55b3c45a3f50 2023-10-06T03:57:42.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.851+0000 7f86857fa700 1 -- 172.21.15.157:0/1430952495 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869405b7f0 con 0x55b3c45a3f50 2023-10-06T03:57:42.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.851+0000 7f86af52cb80 1 -- 172.21.15.157:0/1430952495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3c45a3f50 msgr2=0x55b3c45a7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:42.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.851+0000 7f86af52cb80 1 --2- 172.21.15.157:0/1430952495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3c45a3f50 0x55b3c45a7a70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f869400a340 tx=0x7f8694069000 comp rx=0 tx=0).stop 2023-10-06T03:57:42.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.851+0000 7f86af52cb80 1 -- 172.21.15.157:0/1430952495 shutdown_connections 2023-10-06T03:57:42.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.851+0000 7f86af52cb80 1 --2- 172.21.15.157:0/1430952495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3c4560bd0 0x55b3c4596800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.851+0000 7f86af52cb80 1 --2- 172.21.15.157:0/1430952495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3c45a8920 0x55b3c45a7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.851+0000 7f86af52cb80 1 --2- 172.21.15.157:0/1430952495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3c45a3f50 0x55b3c45a7a70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.851+0000 7f86af52cb80 1 -- 172.21.15.157:0/1430952495 >> 172.21.15.157:0/1430952495 conn(0x55b3c4561fb0 msgr2=0x55b3c4562380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:42.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.851+0000 7f86af52cb80 1 -- 172.21.15.157:0/1430952495 shutdown_connections 2023-10-06T03:57:42.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.851+0000 7f86af52cb80 1 -- 172.21.15.157:0/1430952495 wait complete. 2023-10-06T03:57:42.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.852+0000 7f86af52cb80 1 -- start start 2023-10-06T03:57:42.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.852+0000 7f86af52cb80 1 Processor -- start 2023-10-06T03:57:42.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.852+0000 7f86af52cb80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3c4560bd0 0x55b3c4594f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.852+0000 7f86af52cb80 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3c45a3f50 0x55b3c45a6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.852+0000 7f86af52cb80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3c45a8920 0x55b3c4606a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.852+0000 7f86af52cb80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c45b7a60 con 0x55b3c45a8920 2023-10-06T03:57:42.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.852+0000 7f86af52cb80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c45a7380 con 0x55b3c4560bd0 2023-10-06T03:57:42.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.852+0000 7f86af52cb80 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3c460d5c0 con 0x55b3c45a3f50 2023-10-06T03:57:42.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.852+0000 7f86af52cb80 10 client.0 _ll_register_callbacks cb 0x55b3c434ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:57:42.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.852+0000 7f869ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3c45a8920 0x55b3c4606a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.852+0000 7f869ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3c45a8920 0x55b3c4606a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35814/0 (socket says 172.21.15.157:35814) 2023-10-06T03:57:42.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.852+0000 7f869ffff700 1 -- 172.21.15.157:0/1591337829 learned_addr learned my addr 172.21.15.157:0/1591337829 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:42.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.853+0000 7f869effd700 1 --2- 172.21.15.157:0/1591337829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3c45a3f50 0x55b3c45a6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.853+0000 7f869f7fe700 1 --2- 172.21.15.157:0/1591337829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3c4560bd0 0x55b3c4594f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.853+0000 7f869ffff700 1 -- 172.21.15.157:0/1591337829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3c45a3f50 msgr2=0x55b3c45a6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:42.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.853+0000 7f869ffff700 1 --2- 172.21.15.157:0/1591337829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3c45a3f50 0x55b3c45a6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.853+0000 7f869ffff700 1 -- 172.21.15.157:0/1591337829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3c4560bd0 msgr2=0x55b3c4594f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:42.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.853+0000 7f869ffff700 1 --2- 172.21.15.157:0/1591337829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3c4560bd0 0x55b3c4594f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.853+0000 7f869ffff700 1 -- 172.21.15.157:0/1591337829 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b3c42effe0 con 0x55b3c45a8920 2023-10-06T03:57:42.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.853+0000 7f869f7fe700 1 --2- 172.21.15.157:0/1591337829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3c4560bd0 0x55b3c4594f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:42.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.853+0000 7f869ffff700 1 --2- 172.21.15.157:0/1591337829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3c45a8920 0x55b3c4606a10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f86980030c0 tx=0x7f8698020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9481195500d2ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:42.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.853+0000 7f867dffb700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-10-06T03:57:42.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.853+0000 7f867dffb700 1 -- 172.21.15.157:0/1591337829 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8698011a00 con 0x55b3c45a8920 2023-10-06T03:57:42.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.853+0000 7f867dffb700 1 -- 172.21.15.157:0/1591337829 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f86980279f0 con 0x55b3c45a8920 2023-10-06T03:57:42.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.853+0000 7f86af52cb80 1 -- 172.21.15.157:0/1591337829 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b3c45a68f0 con 0x55b3c45a8920 2023-10-06T03:57:42.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.853+0000 7f867dffb700 1 -- 172.21.15.157:0/1591337829 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8698011b60 con 0x55b3c45a8920 2023-10-06T03:57:42.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.853+0000 7f86af52cb80 10 client.4785 fetch_fsmap learned FSMap version 16 2023-10-06T03:57:42.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.853+0000 7f86af52cb80 1 -- 172.21.15.157:0/1591337829 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55b3c4468470 con 0x55b3c45a8920 2023-10-06T03:57:42.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.853+0000 7f867dffb700 1 -- 172.21.15.157:0/1591337829 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b3c45a68f0 con 0x55b3c45a8920 2023-10-06T03:57:42.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.853+0000 7f867dffb700 1 -- 172.21.15.157:0/1591337829 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 16) v1 ==== 4238+0+0 (secure 0 0 0) 0x7f8698018050 con 0x55b3c45a8920 2023-10-06T03:57:42.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.853+0000 7f86af52cb80 10 client.4785 fetch_fsmap finished waiting for FSMap version 16 2023-10-06T03:57:42.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.853+0000 7f86af52cb80 10 client.4785 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T03:57:42.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.853+0000 7f86af52cb80 20 client.4785 populate_metadata read hostname 'smithi157' 2023-10-06T03:57:42.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.853+0000 7f86af52cb80 1 --2- 172.21.15.157:0/1591337829 >> [v2:172.21.15.175:6834/1596042861,v1:172.21.15.175:6835/1596042861] conn(0x55b3c4562db0 0x55b3c4563180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.853+0000 7f86af52cb80 4 client.4785 mds_command: new command op to 4270 tid=0[{"prefix": "status"}] 2023-10-06T03:57:42.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.853+0000 7f86af52cb80 1 -- 172.21.15.157:0/1591337829 --> [v2:172.21.15.175:6834/1596042861,v1:172.21.15.175:6835/1596042861] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f8698011cd0 con 0x55b3c4562db0 2023-10-06T03:57:42.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.854+0000 7f869f7fe700 1 --2- 172.21.15.157:0/1591337829 >> [v2:172.21.15.175:6834/1596042861,v1:172.21.15.175:6835/1596042861] conn(0x55b3c4562db0 0x55b3c4563180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.854+0000 7f869f7fe700 1 --2- 172.21.15.157:0/1591337829 >> [v2:172.21.15.175:6834/1596042861,v1:172.21.15.175:6835/1596042861] conn(0x55b3c4562db0 0x55b3c4563180 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9ae4609ca376aecf server_cookie=d44dd3a3b1cf83ab in_seq=0 out_seq=0 2023-10-06T03:57:42.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.854+0000 7f867dffb700 10 client.4785 ms_handle_connect on v2:172.21.15.175:6834/1596042861 2023-10-06T03:57:42.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.854+0000 7f867dffb700 1 -- 172.21.15.157:0/1591337829 <== mds.3 v2:172.21.15.175:6834/1596042861 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x7f8698011cd0 con 0x55b3c4562db0 2023-10-06T03:57:42.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.854+0000 7f867dffb700 10 client.4785 handle_command_reply: tid=0 2023-10-06T03:57:42.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.854+0000 7f86af52cb80 1 client.4785 shutdown 2023-10-06T03:57:42.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.854+0000 7f86af52cb80 1 -- 172.21.15.157:0/1591337829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3c45a8920 msgr2=0x55b3c4606a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:42.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.854+0000 7f86af52cb80 1 --2- 172.21.15.157:0/1591337829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3c45a8920 0x55b3c4606a10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f86980030c0 tx=0x7f8698020040 comp rx=0 tx=0).stop 2023-10-06T03:57:42.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.854+0000 7f86af52cb80 1 -- 172.21.15.157:0/1591337829 shutdown_connections 2023-10-06T03:57:42.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.854+0000 7f86af52cb80 1 --2- 172.21.15.157:0/1591337829 >> [v2:172.21.15.175:6834/1596042861,v1:172.21.15.175:6835/1596042861] conn(0x55b3c4562db0 0x55b3c4563180 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.854+0000 7f86af52cb80 1 --2- 172.21.15.157:0/1591337829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b3c4560bd0 0x55b3c4594f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.854+0000 7f867dffb700 0 client.4785 ms_handle_reset on v2:172.21.15.175:6834/1596042861 2023-10-06T03:57:42.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.854+0000 7f86af52cb80 1 --2- 172.21.15.157:0/1591337829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b3c45a8920 0x55b3c4606a10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.854+0000 7f86af52cb80 1 --2- 172.21.15.157:0/1591337829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55b3c45a3f50 0x55b3c45a6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.854+0000 7f86af52cb80 1 -- 172.21.15.157:0/1591337829 >> 172.21.15.157:0/1591337829 conn(0x55b3c4561fb0 msgr2=0x55b3c45a4810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:42.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.854+0000 7f86af52cb80 1 -- 172.21.15.157:0/1591337829 shutdown_connections 2023-10-06T03:57:42.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.854+0000 7f86af52cb80 1 -- 172.21.15.157:0/1591337829 wait complete. 2023-10-06T03:57:42.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.855+0000 7f86af52cb80 20 client.4785 trim_cache size 0 max 16384 2023-10-06T03:57:42.865 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-10-06T03:57:42.865 INFO:teuthology.orchestra.run.smithi157.stdout: "cluster_fsid": "28664a7a-fe30-4efe-be5a-6abf9318b700", 2023-10-06T03:57:42.865 INFO:teuthology.orchestra.run.smithi157.stdout: "whoami": 3, 2023-10-06T03:57:42.865 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 4270, 2023-10-06T03:57:42.865 INFO:teuthology.orchestra.run.smithi157.stdout: "want_state": "up:active", 2023-10-06T03:57:42.865 INFO:teuthology.orchestra.run.smithi157.stdout: "state": "up:active", 2023-10-06T03:57:42.865 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs", 2023-10-06T03:57:42.866 INFO:teuthology.orchestra.run.smithi157.stdout: "rank_uptime": 15.320788771, 2023-10-06T03:57:42.866 INFO:teuthology.orchestra.run.smithi157.stdout: "mdsmap_epoch": 16, 2023-10-06T03:57:42.866 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch": 21, 2023-10-06T03:57:42.866 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch_barrier": 21, 2023-10-06T03:57:42.866 INFO:teuthology.orchestra.run.smithi157.stdout: "uptime": 37.423643169999998 2023-10-06T03:57:42.866 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-10-06T03:57:42.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.855+0000 7f86a6234700 1 -- 172.21.15.157:0/2076256249 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f868c066f60 msgr2=0x7f868c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:42.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.855+0000 7f86a6234700 1 --2- 172.21.15.157:0/2076256249 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f868c066f60 0x7f868c069410 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f869000e1b0 tx=0x7f8690001770 comp rx=0 tx=0).stop 2023-10-06T03:57:42.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.855+0000 7f86a6234700 1 -- 172.21.15.157:0/2076256249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86a0150410 msgr2=0x7f86a0159350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:42.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.855+0000 7f86a6234700 1 --2- 172.21.15.157:0/2076256249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86a0150410 0x7f86a0159350 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f869400cbc0 tx=0x7f8694001b50 comp rx=0 tx=0).stop 2023-10-06T03:57:42.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.855+0000 7f86a6234700 1 -- 172.21.15.157:0/2076256249 shutdown_connections 2023-10-06T03:57:42.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.855+0000 7f86a6234700 1 --2- 172.21.15.157:0/2076256249 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f868c066f60 0x7f868c069410 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.855+0000 7f86a6234700 1 --2- 172.21.15.157:0/2076256249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86a00c3220 0x7f86a0159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.855+0000 7f86a6234700 1 --2- 172.21.15.157:0/2076256249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f86a006cbb0 0x7f86a0157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.855+0000 7f86a6234700 1 --2- 172.21.15.157:0/2076256249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86a0150410 0x7f86a0159350 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.855+0000 7f86a6234700 1 -- 172.21.15.157:0/2076256249 >> 172.21.15.157:0/2076256249 conn(0x7f86a006f230 msgr2=0x7f86a014c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:42.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.855+0000 7f86a6234700 1 -- 172.21.15.157:0/2076256249 shutdown_connections 2023-10-06T03:57:42.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.855+0000 7f86a6234700 1 -- 172.21.15.157:0/2076256249 wait complete. 2023-10-06T03:57:42.869 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T03:57:42.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.983+0000 7fe78d429700 1 Processor -- start 2023-10-06T03:57:42.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.983+0000 7fe78d429700 1 -- start start 2023-10-06T03:57:42.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.984+0000 7fe78d429700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe78806a580 0x7fe78806c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.984+0000 7fe78d429700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe788157f60 0x7fe78814dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.984+0000 7fe78d429700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe78806b700 con 0x7fe78806a950 2023-10-06T03:57:42.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.984+0000 7fe78d429700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe78806ad90 con 0x7fe788157f60 2023-10-06T03:57:42.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.984+0000 7fe78d429700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7881583e0 con 0x7fe78806a580 2023-10-06T03:57:42.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.984+0000 7fe787fff700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe78806a580 0x7fe78806c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.984+0000 7fe78cc28700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe788157f60 0x7fe78814dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.984+0000 7fe787fff700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe78806a580 0x7fe78806c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:47486/0 (socket says 172.21.15.157:47486) 2023-10-06T03:57:42.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.984+0000 7fe787fff700 1 -- 172.21.15.157:0/2060491640 learned_addr learned my addr 172.21.15.157:0/2060491640 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:42.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.984+0000 7fe787fff700 1 -- 172.21.15.157:0/2060491640 >> v2:172.21.15.175:3300/0 conn(0x7fe788157f60 msgr2=0x7fe78814dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:42.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.984+0000 7fe787fff700 1 --2- 172.21.15.157:0/2060491640 >> v2:172.21.15.175:3300/0 conn(0x7fe788157f60 0x7fe78814dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.984+0000 7fe787fff700 1 -- 172.21.15.157:0/2060491640 >> v1:172.21.15.175:6789/0 conn(0x7fe78806a950 legacy=0x7fe7880c3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:42.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.984+0000 7fe787fff700 1 -- 172.21.15.157:0/2060491640 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7880c43d0 con 0x7fe78806a580 2023-10-06T03:57:42.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.985+0000 7fe787fff700 1 --2- 172.21.15.157:0/2060491640 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe78806a580 0x7fe78806c1f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe7780099b0 tx=0x7fe778009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cc87c6f69a37318 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:42.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.985+0000 7fe786ffd700 1 -- 172.21.15.157:0/2060491640 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe778009ea0 con 0x7fe78806a580 2023-10-06T03:57:42.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.985+0000 7fe786ffd700 1 -- 172.21.15.157:0/2060491640 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77800b860 con 0x7fe78806a580 2023-10-06T03:57:42.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.985+0000 7fe786ffd700 1 -- 172.21.15.157:0/2060491640 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77800bc00 con 0x7fe78806a580 2023-10-06T03:57:42.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.985+0000 7fe78d429700 1 -- 172.21.15.157:0/2060491640 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe78806a580 msgr2=0x7fe78806c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:42.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.985+0000 7fe78d429700 1 --2- 172.21.15.157:0/2060491640 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe78806a580 0x7fe78806c1f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe7780099b0 tx=0x7fe778009250 comp rx=0 tx=0).stop 2023-10-06T03:57:42.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.985+0000 7fe78d429700 1 -- 172.21.15.157:0/2060491640 shutdown_connections 2023-10-06T03:57:42.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.985+0000 7fe78d429700 1 --2- 172.21.15.157:0/2060491640 >> v2:172.21.15.175:3300/0 conn(0x7fe788157f60 0x7fe78814dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.985+0000 7fe78d429700 1 --2- 172.21.15.157:0/2060491640 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe78806a580 0x7fe78806c1f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.985+0000 7fe78d429700 1 -- 172.21.15.157:0/2060491640 >> 172.21.15.157:0/2060491640 conn(0x7fe788068020 msgr2=0x7fe788068420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:42.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.985+0000 7fe78d429700 1 -- 172.21.15.157:0/2060491640 shutdown_connections 2023-10-06T03:57:42.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.985+0000 7fe78d429700 1 -- 172.21.15.157:0/2060491640 wait complete. 2023-10-06T03:57:42.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.986+0000 7fe78d429700 1 Processor -- start 2023-10-06T03:57:42.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.987+0000 7fe78d429700 1 -- start start 2023-10-06T03:57:42.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.987+0000 7fe78d429700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe78806a580 0x7fe7880c1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.987+0000 7fe78d429700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe78806a950 0x7fe788145d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.987+0000 7fe78d429700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe788157f60 0x7fe7881462d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.987+0000 7fe78d429700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe788157440 con 0x7fe788157f60 2023-10-06T03:57:42.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.987+0000 7fe78d429700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe788146810 con 0x7fe78806a580 2023-10-06T03:57:42.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.987+0000 7fe78d429700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe788152a10 con 0x7fe78806a950 2023-10-06T03:57:42.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.987+0000 7fe78cc28700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe788157f60 0x7fe7881462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.987+0000 7fe78cc28700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe788157f60 0x7fe7881462d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35824/0 (socket says 172.21.15.157:35824) 2023-10-06T03:57:42.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.987+0000 7fe78cc28700 1 -- 172.21.15.157:0/1918197479 learned_addr learned my addr 172.21.15.157:0/1918197479 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:42.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.987+0000 7fe787fff700 1 --2- 172.21.15.157:0/1918197479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe78806a580 0x7fe7880c1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.987+0000 7fe7877fe700 1 --2- 172.21.15.157:0/1918197479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe78806a950 0x7fe788145d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:42.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.988+0000 7fe78cc28700 1 -- 172.21.15.157:0/1918197479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe78806a950 msgr2=0x7fe788145d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:42.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.988+0000 7fe78cc28700 1 --2- 172.21.15.157:0/1918197479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe78806a950 0x7fe788145d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.988+0000 7fe78cc28700 1 -- 172.21.15.157:0/1918197479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe78806a580 msgr2=0x7fe7880c1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:42.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.988+0000 7fe78cc28700 1 --2- 172.21.15.157:0/1918197479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe78806a580 0x7fe7880c1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:42.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.988+0000 7fe78cc28700 1 -- 172.21.15.157:0/1918197479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7780094f0 con 0x7fe788157f60 2023-10-06T03:57:42.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.988+0000 7fe78cc28700 1 --2- 172.21.15.157:0/1918197479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe788157f60 0x7fe7881462d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe780009e20 tx=0x7fe78000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:42.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.988+0000 7fe76ffff700 1 -- 172.21.15.157:0/1918197479 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78000cb30 con 0x7fe788157f60 2023-10-06T03:57:42.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.988+0000 7fe76ffff700 1 -- 172.21.15.157:0/1918197479 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78000f040 con 0x7fe788157f60 2023-10-06T03:57:42.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.988+0000 7fe78d429700 1 -- 172.21.15.157:0/1918197479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7881565e0 con 0x7fe788157f60 2023-10-06T03:57:42.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.988+0000 7fe76ffff700 1 -- 172.21.15.157:0/1918197479 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe780014480 con 0x7fe788157f60 2023-10-06T03:57:42.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.988+0000 7fe78d429700 1 -- 172.21.15.157:0/1918197479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe788157160 con 0x7fe788157f60 2023-10-06T03:57:42.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.989+0000 7fe76ffff700 1 -- 172.21.15.157:0/1918197479 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe780014830 con 0x7fe788157f60 2023-10-06T03:57:42.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.990+0000 7fe76ffff700 1 --2- 172.21.15.157:0/1918197479 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe774066f30 0x7fe7740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:42.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.990+0000 7fe76ffff700 1 -- 172.21.15.157:0/1918197479 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe780084d30 con 0x7fe788157f60 2023-10-06T03:57:42.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.990+0000 7fe787fff700 1 --2- 172.21.15.157:0/1918197479 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe774066f30 0x7fe7740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.990+0000 7fe787fff700 1 --2- 172.21.15.157:0/1918197479 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe774066f30 0x7fe7740693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe7780099b0 tx=0x7fe778017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:43.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.992+0000 7fe797983b80 1 Processor -- start 2023-10-06T03:57:43.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.992+0000 7fe797983b80 1 -- start start 2023-10-06T03:57:43.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.992+0000 7fe797983b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581b43661e0 0x5581b4321230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.992+0000 7fe797983b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581b4365300 0x5581b4321770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.992+0000 7fe797983b80 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5581b43665b0 0x5581b434f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.992+0000 7fe797983b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581b42a4c80 con 0x5581b43661e0 2023-10-06T03:57:43.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.992+0000 7fe797983b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581b427ba70 con 0x5581b4365300 2023-10-06T03:57:43.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.992+0000 7fe797983b80 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5581b4279270 con 0x5581b43665b0 2023-10-06T03:57:43.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.992+0000 7fe787fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581b43661e0 0x5581b4321230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.993+0000 7fe787fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581b43661e0 0x5581b4321230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35826/0 (socket says 172.21.15.157:35826) 2023-10-06T03:57:43.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.993+0000 7fe787fff700 1 -- 172.21.15.157:0/2405298078 learned_addr learned my addr 172.21.15.157:0/2405298078 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:43.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.993+0000 7fe78cc28700 1 --2- 172.21.15.157:0/2405298078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5581b43665b0 0x5581b434f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.993+0000 7fe7877fe700 1 --2- 172.21.15.157:0/2405298078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581b4365300 0x5581b4321770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.993+0000 7fe787fff700 1 -- 172.21.15.157:0/2405298078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5581b43665b0 msgr2=0x5581b434f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:43.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.993+0000 7fe787fff700 1 --2- 172.21.15.157:0/2405298078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5581b43665b0 0x5581b434f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.993+0000 7fe787fff700 1 -- 172.21.15.157:0/2405298078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581b4365300 msgr2=0x5581b4321770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:43.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.993+0000 7fe787fff700 1 --2- 172.21.15.157:0/2405298078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581b4365300 0x5581b4321770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.993+0000 7fe787fff700 1 -- 172.21.15.157:0/2405298078 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581b410f270 con 0x5581b43661e0 2023-10-06T03:57:43.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.993+0000 7fe787fff700 1 --2- 172.21.15.157:0/2405298078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581b43661e0 0x5581b4321230 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe77800b3c0 tx=0x7fe778021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f099494dfd3fcd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:43.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.993+0000 7fe76d7fa700 1 -- 172.21.15.157:0/2405298078 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe778021070 con 0x5581b43661e0 2023-10-06T03:57:43.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.993+0000 7fe76d7fa700 1 -- 172.21.15.157:0/2405298078 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77800bb90 con 0x5581b43661e0 2023-10-06T03:57:43.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.993+0000 7fe76d7fa700 1 -- 172.21.15.157:0/2405298078 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77802aba0 con 0x5581b43661e0 2023-10-06T03:57:43.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.994+0000 7fe797983b80 1 -- 172.21.15.157:0/2405298078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581b43661e0 msgr2=0x5581b4321230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.994+0000 7fe797983b80 1 --2- 172.21.15.157:0/2405298078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581b43661e0 0x5581b4321230 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe77800b3c0 tx=0x7fe778021000 comp rx=0 tx=0).stop 2023-10-06T03:57:43.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.994+0000 7fe797983b80 1 -- 172.21.15.157:0/2405298078 shutdown_connections 2023-10-06T03:57:43.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.994+0000 7fe797983b80 1 --2- 172.21.15.157:0/2405298078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5581b43665b0 0x5581b434f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.994+0000 7fe797983b80 1 --2- 172.21.15.157:0/2405298078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581b43661e0 0x5581b4321230 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.994+0000 7fe797983b80 1 --2- 172.21.15.157:0/2405298078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581b4365300 0x5581b4321770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.994+0000 7fe797983b80 1 -- 172.21.15.157:0/2405298078 >> 172.21.15.157:0/2405298078 conn(0x5581b43269b0 msgr2=0x5581b4326d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:43.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.994+0000 7fe797983b80 1 -- 172.21.15.157:0/2405298078 shutdown_connections 2023-10-06T03:57:43.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.994+0000 7fe797983b80 1 -- 172.21.15.157:0/2405298078 wait complete. 2023-10-06T03:57:43.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.994+0000 7fe797983b80 1 -- start start 2023-10-06T03:57:43.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.994+0000 7fe797983b80 1 Processor -- start 2023-10-06T03:57:43.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.995+0000 7fe797983b80 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5581b4365300 0x5581b4321580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.995+0000 7fe797983b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581b43661e0 0x5581b437e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.995+0000 7fe797983b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581b43665b0 0x5581b434c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.995+0000 7fe797983b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581b4275060 con 0x5581b43661e0 2023-10-06T03:57:43.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.995+0000 7fe797983b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581b4220890 con 0x5581b43665b0 2023-10-06T03:57:43.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.995+0000 7fe797983b80 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5581b423bea0 con 0x5581b4365300 2023-10-06T03:57:43.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.995+0000 7fe797983b80 10 client.0 _ll_register_callbacks cb 0x5581b4111f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:57:43.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.995+0000 7fe7877fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581b43661e0 0x5581b437e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.995+0000 7fe7877fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581b43661e0 0x5581b437e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35834/0 (socket says 172.21.15.157:35834) 2023-10-06T03:57:43.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.995+0000 7fe7877fe700 1 -- 172.21.15.157:0/3487104680 learned_addr learned my addr 172.21.15.157:0/3487104680 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:43.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.995+0000 7fe787fff700 1 --2- 172.21.15.157:0/3487104680 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5581b4365300 0x5581b4321580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.995+0000 7fe78cc28700 1 --2- 172.21.15.157:0/3487104680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581b43665b0 0x5581b434c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.996+0000 7fe7877fe700 1 -- 172.21.15.157:0/3487104680 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5581b4365300 msgr2=0x5581b4321580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:43.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.996+0000 7fe7877fe700 1 --2- 172.21.15.157:0/3487104680 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5581b4365300 0x5581b4321580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.996+0000 7fe7877fe700 1 -- 172.21.15.157:0/3487104680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581b43665b0 msgr2=0x5581b434c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:43.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.996+0000 7fe7877fe700 1 --2- 172.21.15.157:0/3487104680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581b43665b0 0x5581b434c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.996+0000 7fe7877fe700 1 -- 172.21.15.157:0/3487104680 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7880c43d0 con 0x5581b43661e0 2023-10-06T03:57:43.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.996+0000 7fe7877fe700 1 --2- 172.21.15.157:0/3487104680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581b43661e0 0x5581b437e660 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe77c0125f0 tx=0x7fe77c007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5f806da85582a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:43.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.996+0000 7fe7667fc700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-10-06T03:57:43.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.996+0000 7fe7667fc700 1 -- 172.21.15.157:0/3487104680 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c01a030 con 0x5581b43661e0 2023-10-06T03:57:43.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.996+0000 7fe7667fc700 1 -- 172.21.15.157:0/3487104680 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe77c01b070 con 0x5581b43661e0 2023-10-06T03:57:43.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.996+0000 7fe797983b80 1 -- 172.21.15.157:0/3487104680 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581b4319690 con 0x5581b43661e0 2023-10-06T03:57:43.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.996+0000 7fe7667fc700 1 -- 172.21.15.157:0/3487104680 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe77c006820 con 0x5581b43661e0 2023-10-06T03:57:43.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.996+0000 7fe797983b80 10 client.4794 fetch_fsmap learned FSMap version 16 2023-10-06T03:57:43.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.996+0000 7fe797983b80 1 -- 172.21.15.157:0/3487104680 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x5581b40b1fe0 con 0x5581b43661e0 2023-10-06T03:57:43.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.996+0000 7fe7667fc700 1 -- 172.21.15.157:0/3487104680 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77c025020 con 0x5581b43661e0 2023-10-06T03:57:43.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.996+0000 7fe7667fc700 1 -- 172.21.15.157:0/3487104680 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 16) v1 ==== 4238+0+0 (secure 0 0 0) 0x7fe77c006980 con 0x5581b43661e0 2023-10-06T03:57:43.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.996+0000 7fe797983b80 10 client.4794 fetch_fsmap finished waiting for FSMap version 16 2023-10-06T03:57:43.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.996+0000 7fe797983b80 10 client.4794 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T03:57:43.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.996+0000 7fe797983b80 20 client.4794 populate_metadata read hostname 'smithi157' 2023-10-06T03:57:43.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.996+0000 7fe797983b80 1 --2- 172.21.15.157:0/3487104680 >> [v2:172.21.15.175:6836/1002404870,v1:172.21.15.175:6837/1002404870] conn(0x5581b437abc0 0x5581b437af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.996+0000 7fe797983b80 4 client.4794 mds_command: new command op to 4279 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T03:57:43.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.996+0000 7fe797983b80 1 -- 172.21.15.157:0/3487104680 --> [v2:172.21.15.175:6836/1002404870,v1:172.21.15.175:6837/1002404870] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581b42f65c0 con 0x5581b437abc0 2023-10-06T03:57:43.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.996+0000 7fe787fff700 1 --2- 172.21.15.157:0/3487104680 >> [v2:172.21.15.175:6836/1002404870,v1:172.21.15.175:6837/1002404870] conn(0x5581b437abc0 0x5581b437af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.997+0000 7fe7667fc700 10 client.4794 ms_handle_connect on v2:172.21.15.175:6836/1002404870 2023-10-06T03:57:43.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.997+0000 7fe787fff700 1 --2- 172.21.15.157:0/3487104680 >> [v2:172.21.15.175:6836/1002404870,v1:172.21.15.175:6837/1002404870] conn(0x5581b437abc0 0x5581b437af90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9a2a5d443dfb6cfb server_cookie=133f66c781ec2190 in_seq=0 out_seq=0 2023-10-06T03:57:43.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.997+0000 7fe7667fc700 1 -- 172.21.15.157:0/3487104680 <== mds.2 v2:172.21.15.175:6836/1002404870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5581b42f65c0 con 0x5581b437abc0 2023-10-06T03:57:43.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.997+0000 7fe7667fc700 10 client.4794 handle_command_reply: tid=0 2023-10-06T03:57:43.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.997+0000 7fe797983b80 1 client.4794 shutdown 2023-10-06T03:57:43.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.998+0000 7fe797983b80 1 -- 172.21.15.157:0/3487104680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581b43661e0 msgr2=0x5581b437e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.998+0000 7fe797983b80 1 --2- 172.21.15.157:0/3487104680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581b43661e0 0x5581b437e660 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe77c0125f0 tx=0x7fe77c007ec0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.998+0000 7fe797983b80 1 -- 172.21.15.157:0/3487104680 shutdown_connections 2023-10-06T03:57:43.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.998+0000 7fe797983b80 1 --2- 172.21.15.157:0/3487104680 >> [v2:172.21.15.175:6836/1002404870,v1:172.21.15.175:6837/1002404870] conn(0x5581b437abc0 0x5581b437af90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.998+0000 7fe797983b80 1 --2- 172.21.15.157:0/3487104680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581b43665b0 0x5581b434c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.998+0000 7fe797983b80 1 --2- 172.21.15.157:0/3487104680 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5581b4365300 0x5581b4321580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.998+0000 7fe7667fc700 0 client.4794 ms_handle_reset on v2:172.21.15.175:6836/1002404870 2023-10-06T03:57:43.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.998+0000 7fe797983b80 1 --2- 172.21.15.157:0/3487104680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581b43661e0 0x5581b437e660 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.998+0000 7fe797983b80 1 -- 172.21.15.157:0/3487104680 >> 172.21.15.157:0/3487104680 conn(0x5581b43269b0 msgr2=0x5581b4366cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:43.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.998+0000 7fe797983b80 1 -- 172.21.15.157:0/3487104680 shutdown_connections 2023-10-06T03:57:43.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.998+0000 7fe797983b80 1 -- 172.21.15.157:0/3487104680 wait complete. 2023-10-06T03:57:43.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:42.998+0000 7fe797983b80 20 client.4794 trim_cache size 0 max 16384 2023-10-06T03:57:43.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.014+0000 7fe797983b80 1 Processor -- start 2023-10-06T03:57:43.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.014+0000 7fe797983b80 1 -- start start 2023-10-06T03:57:43.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.015+0000 7fe797983b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581b4365300 0x5581b4321750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.015+0000 7fe797983b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581b43661e0 0x5581b4360fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.015+0000 7fe797983b80 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5581b43665b0 0x5581b4361500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.015+0000 7fe797983b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581b40f7e30 con 0x5581b43661e0 2023-10-06T03:57:43.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.015+0000 7fe797983b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581b4385100 con 0x5581b4365300 2023-10-06T03:57:43.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.015+0000 7fe797983b80 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5581b4322dc0 con 0x5581b43665b0 2023-10-06T03:57:43.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.015+0000 7fe7877fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581b43661e0 0x5581b4360fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.015+0000 7fe7877fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581b43661e0 0x5581b4360fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35848/0 (socket says 172.21.15.157:35848) 2023-10-06T03:57:43.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.015+0000 7fe7877fe700 1 -- 172.21.15.157:0/4230480476 learned_addr learned my addr 172.21.15.157:0/4230480476 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:43.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.015+0000 7fe78cc28700 1 --2- 172.21.15.157:0/4230480476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5581b43665b0 0x5581b4361500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.015+0000 7fe787fff700 1 --2- 172.21.15.157:0/4230480476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581b4365300 0x5581b4321750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.015+0000 7fe7877fe700 1 -- 172.21.15.157:0/4230480476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5581b43665b0 msgr2=0x5581b4361500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:43.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.015+0000 7fe7877fe700 1 --2- 172.21.15.157:0/4230480476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5581b43665b0 0x5581b4361500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.015+0000 7fe7877fe700 1 -- 172.21.15.157:0/4230480476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581b4365300 msgr2=0x5581b4321750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:43.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.015+0000 7fe7877fe700 1 --2- 172.21.15.157:0/4230480476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581b4365300 0x5581b4321750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.015+0000 7fe7877fe700 1 -- 172.21.15.157:0/4230480476 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581b4327a70 con 0x5581b43661e0 2023-10-06T03:57:43.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.015+0000 7fe787fff700 1 --2- 172.21.15.157:0/4230480476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581b4365300 0x5581b4321750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:43.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.016+0000 7fe7877fe700 1 --2- 172.21.15.157:0/4230480476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581b43661e0 0x5581b4360fc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe77c016f20 tx=0x7fe77c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f326ed0bb4917505 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:43.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.016+0000 7fe76dffb700 1 -- 172.21.15.157:0/4230480476 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c01a070 con 0x5581b43661e0 2023-10-06T03:57:43.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.016+0000 7fe76dffb700 1 -- 172.21.15.157:0/4230480476 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77c025030 con 0x5581b43661e0 2023-10-06T03:57:43.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.016+0000 7fe76dffb700 1 -- 172.21.15.157:0/4230480476 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c012330 con 0x5581b43661e0 2023-10-06T03:57:43.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.016+0000 7fe797983b80 1 -- 172.21.15.157:0/4230480476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581b43661e0 msgr2=0x5581b4360fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.016+0000 7fe797983b80 1 --2- 172.21.15.157:0/4230480476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581b43661e0 0x5581b4360fc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe77c016f20 tx=0x7fe77c017000 comp rx=0 tx=0).stop 2023-10-06T03:57:43.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.016+0000 7fe797983b80 1 -- 172.21.15.157:0/4230480476 shutdown_connections 2023-10-06T03:57:43.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.016+0000 7fe797983b80 1 --2- 172.21.15.157:0/4230480476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581b4365300 0x5581b4321750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.016+0000 7fe797983b80 1 --2- 172.21.15.157:0/4230480476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5581b43665b0 0x5581b4361500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.016+0000 7fe797983b80 1 --2- 172.21.15.157:0/4230480476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581b43661e0 0x5581b4360fc0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.016+0000 7fe797983b80 1 -- 172.21.15.157:0/4230480476 >> 172.21.15.157:0/4230480476 conn(0x5581b43269b0 msgr2=0x5581b4362c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:43.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.016+0000 7fe797983b80 1 -- 172.21.15.157:0/4230480476 shutdown_connections 2023-10-06T03:57:43.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.016+0000 7fe797983b80 1 -- 172.21.15.157:0/4230480476 wait complete. 2023-10-06T03:57:43.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.017+0000 7fe797983b80 1 -- start start 2023-10-06T03:57:43.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.017+0000 7fe797983b80 1 Processor -- start 2023-10-06T03:57:43.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.018+0000 7fe797983b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581b4365300 0x5581b4364030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.018+0000 7fe797983b80 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5581b43661e0 0x5581b43d53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.018+0000 7fe797983b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581b43665b0 0x5581b43cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.018+0000 7fe797983b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581b4364570 con 0x5581b43665b0 2023-10-06T03:57:43.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.018+0000 7fe797983b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581b4327d50 con 0x5581b4365300 2023-10-06T03:57:43.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.018+0000 7fe797983b80 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x5581b43d5dd0 con 0x5581b43661e0 2023-10-06T03:57:43.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.018+0000 7fe797983b80 10 client.0 _ll_register_callbacks cb 0x5581b4111f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:57:43.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.018+0000 7fe78cc28700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581b43665b0 0x5581b43cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.018+0000 7fe78cc28700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581b43665b0 0x5581b43cbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35850/0 (socket says 172.21.15.157:35850) 2023-10-06T03:57:43.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.018+0000 7fe78cc28700 1 -- 172.21.15.157:0/1805951216 learned_addr learned my addr 172.21.15.157:0/1805951216 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:43.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.018+0000 7fe787fff700 1 --2- 172.21.15.157:0/1805951216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581b4365300 0x5581b4364030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.018+0000 7fe7877fe700 1 --2- 172.21.15.157:0/1805951216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5581b43661e0 0x5581b43d53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.018+0000 7fe78cc28700 1 -- 172.21.15.157:0/1805951216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5581b43661e0 msgr2=0x5581b43d53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:43.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.018+0000 7fe78cc28700 1 --2- 172.21.15.157:0/1805951216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5581b43661e0 0x5581b43d53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.018+0000 7fe78cc28700 1 -- 172.21.15.157:0/1805951216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581b4365300 msgr2=0x5581b4364030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:43.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.018+0000 7fe78cc28700 1 --2- 172.21.15.157:0/1805951216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581b4365300 0x5581b4364030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.018+0000 7fe78cc28700 1 -- 172.21.15.157:0/1805951216 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581b40b1fe0 con 0x5581b43665b0 2023-10-06T03:57:43.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.018+0000 7fe78cc28700 1 --2- 172.21.15.157:0/1805951216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581b43665b0 0x5581b43cbab0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe780055800 tx=0x7fe780016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a83f0426e163a0b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:43.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.018+0000 7fe7667fc700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-10-06T03:57:43.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.018+0000 7fe7667fc700 1 -- 172.21.15.157:0/1805951216 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe780055d60 con 0x5581b43665b0 2023-10-06T03:57:43.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.018+0000 7fe7667fc700 1 -- 172.21.15.157:0/1805951216 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe78006e070 con 0x5581b43665b0 2023-10-06T03:57:43.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.018+0000 7fe797983b80 1 -- 172.21.15.157:0/1805951216 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe77800bb90 con 0x5581b43665b0 2023-10-06T03:57:43.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.018+0000 7fe7667fc700 1 -- 172.21.15.157:0/1805951216 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe780054ff0 con 0x5581b43665b0 2023-10-06T03:57:43.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.018+0000 7fe797983b80 10 client.4800 fetch_fsmap learned FSMap version 16 2023-10-06T03:57:43.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.018+0000 7fe797983b80 1 -- 172.21.15.157:0/1805951216 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x5581b4243780 con 0x5581b43665b0 2023-10-06T03:57:43.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.018+0000 7fe7667fc700 1 -- 172.21.15.157:0/1805951216 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77800bb90 con 0x5581b43665b0 2023-10-06T03:57:43.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.019+0000 7fe7667fc700 1 -- 172.21.15.157:0/1805951216 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 16) v1 ==== 4238+0+0 (secure 0 0 0) 0x7fe78005c610 con 0x5581b43665b0 2023-10-06T03:57:43.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.019+0000 7fe797983b80 10 client.4800 fetch_fsmap finished waiting for FSMap version 16 2023-10-06T03:57:43.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.019+0000 7fe797983b80 10 client.4800 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T03:57:43.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.019+0000 7fe797983b80 20 client.4800 populate_metadata read hostname 'smithi157' 2023-10-06T03:57:43.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.019+0000 7fe797983b80 1 --2- 172.21.15.157:0/1805951216 >> [v2:172.21.15.175:6836/1002404870,v1:172.21.15.175:6837/1002404870] conn(0x5581b4366ab0 0x5581b4366e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.019+0000 7fe797983b80 4 client.4800 mds_command: new command op to 4279 tid=0[{"prefix": "status"}] 2023-10-06T03:57:43.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.019+0000 7fe797983b80 1 -- 172.21.15.157:0/1805951216 --> [v2:172.21.15.175:6836/1002404870,v1:172.21.15.175:6837/1002404870] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5581b4217e00 con 0x5581b4366ab0 2023-10-06T03:57:43.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.019+0000 7fe787fff700 1 --2- 172.21.15.157:0/1805951216 >> [v2:172.21.15.175:6836/1002404870,v1:172.21.15.175:6837/1002404870] conn(0x5581b4366ab0 0x5581b4366e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.019+0000 7fe7667fc700 10 client.4800 ms_handle_connect on v2:172.21.15.175:6836/1002404870 2023-10-06T03:57:43.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.019+0000 7fe787fff700 1 --2- 172.21.15.157:0/1805951216 >> [v2:172.21.15.175:6836/1002404870,v1:172.21.15.175:6837/1002404870] conn(0x5581b4366ab0 0x5581b4366e80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b5f9b61fe263cd1a server_cookie=f14fab3985a9b4c8 in_seq=0 out_seq=0 2023-10-06T03:57:43.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.019+0000 7fe7667fc700 1 -- 172.21.15.157:0/1805951216 <== mds.2 v2:172.21.15.175:6836/1002404870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5581b4217e00 con 0x5581b4366ab0 2023-10-06T03:57:43.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.019+0000 7fe7667fc700 10 client.4800 handle_command_reply: tid=0 2023-10-06T03:57:43.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.019+0000 7fe797983b80 1 client.4800 shutdown 2023-10-06T03:57:43.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.019+0000 7fe797983b80 1 -- 172.21.15.157:0/1805951216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581b43665b0 msgr2=0x5581b43cbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.019+0000 7fe797983b80 1 --2- 172.21.15.157:0/1805951216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581b43665b0 0x5581b43cbab0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe780055800 tx=0x7fe780016040 comp rx=0 tx=0).stop 2023-10-06T03:57:43.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.019+0000 7fe797983b80 1 -- 172.21.15.157:0/1805951216 shutdown_connections 2023-10-06T03:57:43.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.019+0000 7fe797983b80 1 --2- 172.21.15.157:0/1805951216 >> [v2:172.21.15.175:6836/1002404870,v1:172.21.15.175:6837/1002404870] conn(0x5581b4366ab0 0x5581b4366e80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.019+0000 7fe797983b80 1 --2- 172.21.15.157:0/1805951216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5581b4365300 0x5581b4364030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.019+0000 7fe797983b80 1 --2- 172.21.15.157:0/1805951216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5581b43665b0 0x5581b43cbab0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.019+0000 7fe7667fc700 0 client.4800 ms_handle_reset on v2:172.21.15.175:6836/1002404870 2023-10-06T03:57:43.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.019+0000 7fe797983b80 1 --2- 172.21.15.157:0/1805951216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5581b43661e0 0x5581b43d53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.019+0000 7fe797983b80 1 -- 172.21.15.157:0/1805951216 >> 172.21.15.157:0/1805951216 conn(0x5581b43269b0 msgr2=0x5581b4360fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:43.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.020+0000 7fe797983b80 1 -- 172.21.15.157:0/1805951216 shutdown_connections 2023-10-06T03:57:43.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.020+0000 7fe797983b80 1 -- 172.21.15.157:0/1805951216 wait complete. 2023-10-06T03:57:43.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.020+0000 7fe797983b80 20 client.4800 trim_cache size 0 max 16384 2023-10-06T03:57:43.029 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-10-06T03:57:43.029 INFO:teuthology.orchestra.run.smithi157.stdout: "cluster_fsid": "28664a7a-fe30-4efe-be5a-6abf9318b700", 2023-10-06T03:57:43.029 INFO:teuthology.orchestra.run.smithi157.stdout: "whoami": 2, 2023-10-06T03:57:43.029 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 4279, 2023-10-06T03:57:43.030 INFO:teuthology.orchestra.run.smithi157.stdout: "want_state": "up:active", 2023-10-06T03:57:43.030 INFO:teuthology.orchestra.run.smithi157.stdout: "state": "up:active", 2023-10-06T03:57:43.030 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs", 2023-10-06T03:57:43.030 INFO:teuthology.orchestra.run.smithi157.stdout: "rank_uptime": 21.494938904000001, 2023-10-06T03:57:43.030 INFO:teuthology.orchestra.run.smithi157.stdout: "mdsmap_epoch": 16, 2023-10-06T03:57:43.030 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch": 21, 2023-10-06T03:57:43.030 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch_barrier": 21, 2023-10-06T03:57:43.030 INFO:teuthology.orchestra.run.smithi157.stdout: "uptime": 37.567655731999999 2023-10-06T03:57:43.030 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-10-06T03:57:43.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.020+0000 7fe78d429700 1 -- 172.21.15.157:0/1918197479 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe774066f30 msgr2=0x7fe7740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.020+0000 7fe78d429700 1 --2- 172.21.15.157:0/1918197479 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe774066f30 0x7fe7740693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe7780099b0 tx=0x7fe778017000 comp rx=0 tx=0).stop 2023-10-06T03:57:43.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.020+0000 7fe78d429700 1 -- 172.21.15.157:0/1918197479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe788157f60 msgr2=0x7fe7881462d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.020+0000 7fe78d429700 1 --2- 172.21.15.157:0/1918197479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe788157f60 0x7fe7881462d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe780009e20 tx=0x7fe78000b040 comp rx=0 tx=0).stop 2023-10-06T03:57:43.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.020+0000 7fe78d429700 1 -- 172.21.15.157:0/1918197479 shutdown_connections 2023-10-06T03:57:43.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.020+0000 7fe78d429700 1 --2- 172.21.15.157:0/1918197479 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe774066f30 0x7fe7740693e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.020+0000 7fe78d429700 1 --2- 172.21.15.157:0/1918197479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe78806a580 0x7fe7880c1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.020+0000 7fe78d429700 1 --2- 172.21.15.157:0/1918197479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe788157f60 0x7fe7881462d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.020+0000 7fe78d429700 1 --2- 172.21.15.157:0/1918197479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe78806a950 0x7fe788145d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.020+0000 7fe78d429700 1 -- 172.21.15.157:0/1918197479 >> 172.21.15.157:0/1918197479 conn(0x7fe788068020 msgr2=0x7fe788154000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:43.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.020+0000 7fe78d429700 1 -- 172.21.15.157:0/1918197479 shutdown_connections 2023-10-06T03:57:43.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.020+0000 7fe78d429700 1 -- 172.21.15.157:0/1918197479 wait complete. 2023-10-06T03:57:43.033 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T03:57:43.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.147+0000 7fb9ea8f9700 1 Processor -- start 2023-10-06T03:57:43.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.148+0000 7fb9ea8f9700 1 -- start start 2023-10-06T03:57:43.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.148+0000 7fb9ea8f9700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9e40b1e40 0x7fb9e4157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.148+0000 7fb9ea8f9700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb9e406c2d0 0x7fb9e406dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.148+0000 7fb9ea8f9700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e406d8d0 con 0x7fb9e40b04d0 2023-10-06T03:57:43.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.148+0000 7fb9ea8f9700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e40af8f0 con 0x7fb9e406c2d0 2023-10-06T03:57:43.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.148+0000 7fb9ea8f9700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e415a690 con 0x7fb9e40b1e40 2023-10-06T03:57:43.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.148+0000 7fb9ea0f8700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb9e406c2d0 0x7fb9e406dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.148+0000 7fb9e90f6700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb9e40b04d0 0x7fb9e41563b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39754/0 (socket says 172.21.15.157:39754) 2023-10-06T03:57:43.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.148+0000 7fb9ea0f8700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb9e406c2d0 0x7fb9e406dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35862/0 (socket says 172.21.15.157:35862) 2023-10-06T03:57:43.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.148+0000 7fb9e90f6700 1 -- 172.21.15.157:0/3404480753 learned_addr learned my addr 172.21.15.157:0/3404480753 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:43.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.149+0000 7fb9e98f7700 1 --2- 172.21.15.157:0/3404480753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9e40b1e40 0x7fb9e4157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.149+0000 7fb9ea0f8700 1 -- 172.21.15.157:0/3404480753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9e40b1e40 msgr2=0x7fb9e4157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:43.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.149+0000 7fb9ea0f8700 1 --2- 172.21.15.157:0/3404480753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9e40b1e40 0x7fb9e4157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.149+0000 7fb9ea0f8700 1 -- 172.21.15.157:0/3404480753 >> v1:172.21.15.157:6789/0 conn(0x7fb9e40b04d0 legacy=0x7fb9e41563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.149+0000 7fb9ea0f8700 1 -- 172.21.15.157:0/3404480753 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e406ccc0 con 0x7fb9e406c2d0 2023-10-06T03:57:43.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.149+0000 7fb9e88f5700 1 -- 172.21.15.157:0/3404480753 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1885238358 0 0) 0x7fb9e406d8d0 con 0x7fb9e40b04d0 2023-10-06T03:57:43.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.149+0000 7fb9ea0f8700 1 --2- 172.21.15.157:0/3404480753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9e406c2d0 0x7fb9e406dd60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc0098e0 tx=0x7fb9dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5baa6ce4b670de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:43.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.149+0000 7fb9e88f5700 1 -- 172.21.15.157:0/3404480753 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc00b640 con 0x7fb9e406c2d0 2023-10-06T03:57:43.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.149+0000 7fb9e88f5700 1 -- 172.21.15.157:0/3404480753 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9dc00b7a0 con 0x7fb9e406c2d0 2023-10-06T03:57:43.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.149+0000 7fb9e88f5700 1 -- 172.21.15.157:0/3404480753 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc00bb40 con 0x7fb9e406c2d0 2023-10-06T03:57:43.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.150+0000 7fb9ea8f9700 1 -- 172.21.15.157:0/3404480753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9e406c2d0 msgr2=0x7fb9e406dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.150+0000 7fb9ea8f9700 1 --2- 172.21.15.157:0/3404480753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9e406c2d0 0x7fb9e406dd60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc0098e0 tx=0x7fb9dc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.150+0000 7fb9ea8f9700 1 -- 172.21.15.157:0/3404480753 shutdown_connections 2023-10-06T03:57:43.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.150+0000 7fb9ea8f9700 1 --2- 172.21.15.157:0/3404480753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9e406c2d0 0x7fb9e406dd60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.150+0000 7fb9ea8f9700 1 --2- 172.21.15.157:0/3404480753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9e40b1e40 0x7fb9e4157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.150+0000 7fb9ea8f9700 1 -- 172.21.15.157:0/3404480753 >> 172.21.15.157:0/3404480753 conn(0x7fb9e40cc540 msgr2=0x7fb9e40cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:43.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.150+0000 7fb9ea8f9700 1 -- 172.21.15.157:0/3404480753 shutdown_connections 2023-10-06T03:57:43.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.150+0000 7fb9ea8f9700 1 -- 172.21.15.157:0/3404480753 wait complete. 2023-10-06T03:57:43.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.151+0000 7fb9ea8f9700 1 Processor -- start 2023-10-06T03:57:43.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.151+0000 7fb9ea8f9700 1 -- start start 2023-10-06T03:57:43.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.151+0000 7fb9ea8f9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9e406c2d0 0x7fb9e414fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.151+0000 7fb9ea8f9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9e40b04d0 0x7fb9e4150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.151+0000 7fb9ea8f9700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9e40b1e40 0x7fb9e40c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.151+0000 7fb9ea8f9700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e40c7d40 con 0x7fb9e40b04d0 2023-10-06T03:57:43.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.151+0000 7fb9ea8f9700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e40c7280 con 0x7fb9e406c2d0 2023-10-06T03:57:43.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.151+0000 7fb9ea8f9700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e40c6fa0 con 0x7fb9e40b1e40 2023-10-06T03:57:43.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.151+0000 7fb9e90f6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9e40b04d0 0x7fb9e4150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.151+0000 7fb9e90f6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9e40b04d0 0x7fb9e4150340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35874/0 (socket says 172.21.15.157:35874) 2023-10-06T03:57:43.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.152+0000 7fb9e90f6700 1 -- 172.21.15.157:0/22630982 learned_addr learned my addr 172.21.15.157:0/22630982 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:43.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.152+0000 7fb9ea0f8700 1 --2- 172.21.15.157:0/22630982 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9e40b1e40 0x7fb9e40c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.152+0000 7fb9e98f7700 1 --2- 172.21.15.157:0/22630982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9e406c2d0 0x7fb9e414fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.152+0000 7fb9e90f6700 1 -- 172.21.15.157:0/22630982 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9e40b1e40 msgr2=0x7fb9e40c8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.152+0000 7fb9e90f6700 1 --2- 172.21.15.157:0/22630982 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9e40b1e40 0x7fb9e40c8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.152+0000 7fb9e90f6700 1 -- 172.21.15.157:0/22630982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9e406c2d0 msgr2=0x7fb9e414fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.152+0000 7fb9e90f6700 1 --2- 172.21.15.157:0/22630982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9e406c2d0 0x7fb9e414fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.152+0000 7fb9e90f6700 1 -- 172.21.15.157:0/22630982 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9dc009420 con 0x7fb9e40b04d0 2023-10-06T03:57:43.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.152+0000 7fb9e98f7700 1 --2- 172.21.15.157:0/22630982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9e406c2d0 0x7fb9e414fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:57:43.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.152+0000 7fb9e90f6700 1 --2- 172.21.15.157:0/22630982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9e40b04d0 0x7fb9e4150340 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400c7e0 tx=0x7fb9d4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:43.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.152+0000 7fb9d9ffb700 1 -- 172.21.15.157:0/22630982 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4005b30 con 0x7fb9e40b04d0 2023-10-06T03:57:43.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.152+0000 7fb9d9ffb700 1 -- 172.21.15.157:0/22630982 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d4005c90 con 0x7fb9e40b04d0 2023-10-06T03:57:43.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.152+0000 7fb9ea8f9700 1 -- 172.21.15.157:0/22630982 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e4153550 con 0x7fb9e40b04d0 2023-10-06T03:57:43.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.152+0000 7fb9d9ffb700 1 -- 172.21.15.157:0/22630982 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4007300 con 0x7fb9e40b04d0 2023-10-06T03:57:43.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.152+0000 7fb9ea8f9700 1 -- 172.21.15.157:0/22630982 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e4153b10 con 0x7fb9e40b04d0 2023-10-06T03:57:43.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.154+0000 7fb9d9ffb700 1 -- 172.21.15.157:0/22630982 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb9d401b400 con 0x7fb9e40b04d0 2023-10-06T03:57:43.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.154+0000 7fb9d9ffb700 1 --2- 172.21.15.157:0/22630982 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb9c8066f30 0x7fb9c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.154+0000 7fb9d9ffb700 1 -- 172.21.15.157:0/22630982 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb9d40842a0 con 0x7fb9e40b04d0 2023-10-06T03:57:43.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.154+0000 7fb9e98f7700 1 --2- 172.21.15.157:0/22630982 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb9c8066f30 0x7fb9c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.155+0000 7fb9e98f7700 1 --2- 172.21.15.157:0/22630982 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb9c8066f30 0x7fb9c80693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0007780 tx=0x7fb9e000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:43.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.156+0000 7fb9f4e53b80 1 Processor -- start 2023-10-06T03:57:43.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.156+0000 7fb9f4e53b80 1 -- start start 2023-10-06T03:57:43.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.156+0000 7fb9f4e53b80 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a5602701e0 0x55a56026f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.156+0000 7fb9f4e53b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5602705b0 0x55a56022b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.156+0000 7fb9f4e53b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a56022b6b0 0x55a5602584f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.156+0000 7fb9f4e53b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5601aec80 con 0x55a56022b6b0 2023-10-06T03:57:43.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.156+0000 7fb9f4e53b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a560185a70 con 0x55a5602705b0 2023-10-06T03:57:43.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.156+0000 7fb9f4e53b80 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a560183270 con 0x55a5602701e0 2023-10-06T03:57:43.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.157+0000 7fb9ea0f8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a56022b6b0 0x55a5602584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.157+0000 7fb9ea0f8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a56022b6b0 0x55a5602584f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35890/0 (socket says 172.21.15.157:35890) 2023-10-06T03:57:43.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.157+0000 7fb9ea0f8700 1 -- 172.21.15.157:0/3676776378 learned_addr learned my addr 172.21.15.157:0/3676776378 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:43.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.157+0000 7fb9e98f7700 1 --2- 172.21.15.157:0/3676776378 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a5602701e0 0x55a56026f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.157+0000 7fb9e90f6700 1 --2- 172.21.15.157:0/3676776378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5602705b0 0x55a56022b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.157+0000 7fb9ea0f8700 1 -- 172.21.15.157:0/3676776378 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a5602701e0 msgr2=0x55a56026f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:43.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.157+0000 7fb9ea0f8700 1 --2- 172.21.15.157:0/3676776378 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a5602701e0 0x55a56026f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.157+0000 7fb9ea0f8700 1 -- 172.21.15.157:0/3676776378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5602705b0 msgr2=0x55a56022b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:43.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.157+0000 7fb9ea0f8700 1 --2- 172.21.15.157:0/3676776378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5602705b0 0x55a56022b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.157+0000 7fb9ea0f8700 1 -- 172.21.15.157:0/3676776378 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a560019270 con 0x55a56022b6b0 2023-10-06T03:57:43.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.157+0000 7fb9e90f6700 1 --2- 172.21.15.157:0/3676776378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5602705b0 0x55a56022b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:57:43.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.157+0000 7fb9ea0f8700 1 --2- 172.21.15.157:0/3676776378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a56022b6b0 0x55a5602584f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb9e40899e0 tx=0x7fb9dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=589ca4eac5ad4e6c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:43.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.157+0000 7fb9ceffd700 1 -- 172.21.15.157:0/3676776378 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc020070 con 0x55a56022b6b0 2023-10-06T03:57:43.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.157+0000 7fb9ceffd700 1 -- 172.21.15.157:0/3676776378 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9dc00beb0 con 0x55a56022b6b0 2023-10-06T03:57:43.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.157+0000 7fb9ceffd700 1 -- 172.21.15.157:0/3676776378 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc00bb20 con 0x55a56022b6b0 2023-10-06T03:57:43.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.158+0000 7fb9f4e53b80 1 -- 172.21.15.157:0/3676776378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a56022b6b0 msgr2=0x55a5602584f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.158+0000 7fb9f4e53b80 1 --2- 172.21.15.157:0/3676776378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a56022b6b0 0x55a5602584f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb9e40899e0 tx=0x7fb9dc017000 comp rx=0 tx=0).stop 2023-10-06T03:57:43.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.158+0000 7fb9f4e53b80 1 -- 172.21.15.157:0/3676776378 shutdown_connections 2023-10-06T03:57:43.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.158+0000 7fb9f4e53b80 1 --2- 172.21.15.157:0/3676776378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a56022b6b0 0x55a5602584f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.158+0000 7fb9f4e53b80 1 --2- 172.21.15.157:0/3676776378 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a5602701e0 0x55a56026f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.158+0000 7fb9f4e53b80 1 --2- 172.21.15.157:0/3676776378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5602705b0 0x55a56022b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.158+0000 7fb9f4e53b80 1 -- 172.21.15.157:0/3676776378 >> 172.21.15.157:0/3676776378 conn(0x55a5602309b0 msgr2=0x55a560230d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:43.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.158+0000 7fb9f4e53b80 1 -- 172.21.15.157:0/3676776378 shutdown_connections 2023-10-06T03:57:43.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.158+0000 7fb9f4e53b80 1 -- 172.21.15.157:0/3676776378 wait complete. 2023-10-06T03:57:43.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.158+0000 7fb9f4e53b80 1 -- start start 2023-10-06T03:57:43.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.158+0000 7fb9f4e53b80 1 Processor -- start 2023-10-06T03:57:43.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.159+0000 7fb9f4e53b80 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a56022b6b0 0x55a56026dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.159+0000 7fb9f4e53b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5602701e0 0x55a5602586a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.159+0000 7fb9f4e53b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5602705b0 0x55a560274690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.159+0000 7fb9f4e53b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56017f060 con 0x55a5602705b0 2023-10-06T03:57:43.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.159+0000 7fb9f4e53b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56012a890 con 0x55a5602701e0 2023-10-06T03:57:43.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.159+0000 7fb9f4e53b80 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a560145ea0 con 0x55a56022b6b0 2023-10-06T03:57:43.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.159+0000 7fb9f4e53b80 10 client.0 _ll_register_callbacks cb 0x55a56001bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:57:43.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.159+0000 7fb9ea0f8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5602705b0 0x55a560274690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.159+0000 7fb9ea0f8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5602705b0 0x55a560274690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44918/0 (socket says 172.21.15.157:44918) 2023-10-06T03:57:43.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.159+0000 7fb9ea0f8700 1 -- 172.21.15.157:0/2537733101 learned_addr learned my addr 172.21.15.157:0/2537733101 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:43.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.159+0000 7fb9e98f7700 1 --2- 172.21.15.157:0/2537733101 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a56022b6b0 0x55a56026dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.159+0000 7fb9e90f6700 1 --2- 172.21.15.157:0/2537733101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5602701e0 0x55a5602586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.160+0000 7fb9ea0f8700 1 -- 172.21.15.157:0/2537733101 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a56022b6b0 msgr2=0x55a56026dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:43.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.160+0000 7fb9ea0f8700 1 --2- 172.21.15.157:0/2537733101 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a56022b6b0 0x55a56026dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.160+0000 7fb9ea0f8700 1 -- 172.21.15.157:0/2537733101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5602701e0 msgr2=0x55a5602586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:43.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.160+0000 7fb9ea0f8700 1 --2- 172.21.15.157:0/2537733101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5602701e0 0x55a5602586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.160+0000 7fb9ea0f8700 1 -- 172.21.15.157:0/2537733101 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e406ccc0 con 0x55a5602705b0 2023-10-06T03:57:43.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.160+0000 7fb9ea0f8700 1 --2- 172.21.15.157:0/2537733101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5602705b0 0x55a560274690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc003050 tx=0x7fb9dc020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc7ae0175c6e555 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:43.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.160+0000 7fb9c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-10-06T03:57:43.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.160+0000 7fb9c3fff700 1 -- 172.21.15.157:0/2537733101 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc01a840 con 0x55a5602705b0 2023-10-06T03:57:43.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.160+0000 7fb9c3fff700 1 -- 172.21.15.157:0/2537733101 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb9dc030c60 con 0x55a5602705b0 2023-10-06T03:57:43.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.160+0000 7fb9f4e53b80 1 -- 172.21.15.157:0/2537733101 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a560223690 con 0x55a5602705b0 2023-10-06T03:57:43.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.160+0000 7fb9c3fff700 1 -- 172.21.15.157:0/2537733101 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9dc030e40 con 0x55a5602705b0 2023-10-06T03:57:43.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.160+0000 7fb9f4e53b80 10 client.4815 fetch_fsmap learned FSMap version 16 2023-10-06T03:57:43.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.160+0000 7fb9f4e53b80 1 -- 172.21.15.157:0/2537733101 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55a55ffbbfe0 con 0x55a5602705b0 2023-10-06T03:57:43.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.160+0000 7fb9c3fff700 1 -- 172.21.15.157:0/2537733101 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a560223690 con 0x55a5602705b0 2023-10-06T03:57:43.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.160+0000 7fb9c3fff700 1 -- 172.21.15.157:0/2537733101 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 16) v1 ==== 4238+0+0 (secure 0 0 0) 0x7fb9dc01abb0 con 0x55a5602705b0 2023-10-06T03:57:43.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.160+0000 7fb9f4e53b80 10 client.4815 fetch_fsmap finished waiting for FSMap version 16 2023-10-06T03:57:43.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.160+0000 7fb9f4e53b80 10 client.4815 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T03:57:43.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.160+0000 7fb9f4e53b80 20 client.4815 populate_metadata read hostname 'smithi157' 2023-10-06T03:57:43.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.160+0000 7fb9f4e53b80 1 --2- 172.21.15.157:0/2537733101 >> [v2:172.21.15.157:6832/1179753185,v1:172.21.15.157:6834/1179753185] conn(0x55a560275260 0x55a5602563a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.160+0000 7fb9f4e53b80 4 client.4815 mds_command: new command op to 4287 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T03:57:43.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.160+0000 7fb9f4e53b80 1 -- 172.21.15.157:0/2537733101 --> [v2:172.21.15.157:6832/1179753185,v1:172.21.15.157:6834/1179753185] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a5602005c0 con 0x55a560275260 2023-10-06T03:57:43.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.160+0000 7fb9e98f7700 1 --2- 172.21.15.157:0/2537733101 >> [v2:172.21.15.157:6832/1179753185,v1:172.21.15.157:6834/1179753185] conn(0x55a560275260 0x55a5602563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.161+0000 7fb9e98f7700 1 --2- 172.21.15.157:0/2537733101 >> [v2:172.21.15.157:6832/1179753185,v1:172.21.15.157:6834/1179753185] conn(0x55a560275260 0x55a5602563a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5b24066dc7b589f server_cookie=609cd7bfafbf0733 in_seq=0 out_seq=0 2023-10-06T03:57:43.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.161+0000 7fb9c3fff700 10 client.4815 ms_handle_connect on v2:172.21.15.157:6832/1179753185 2023-10-06T03:57:43.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.161+0000 7fb9c3fff700 1 -- 172.21.15.157:0/2537733101 <== mds.1 v2:172.21.15.157:6832/1179753185 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55a5602005c0 con 0x55a560275260 2023-10-06T03:57:43.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.161+0000 7fb9c3fff700 10 client.4815 handle_command_reply: tid=0 2023-10-06T03:57:43.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.161+0000 7fb9f4e53b80 1 client.4815 shutdown 2023-10-06T03:57:43.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.161+0000 7fb9f4e53b80 1 -- 172.21.15.157:0/2537733101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5602705b0 msgr2=0x55a560274690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.161+0000 7fb9f4e53b80 1 --2- 172.21.15.157:0/2537733101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5602705b0 0x55a560274690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc003050 tx=0x7fb9dc020000 comp rx=0 tx=0).stop 2023-10-06T03:57:43.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.161+0000 7fb9f4e53b80 1 -- 172.21.15.157:0/2537733101 shutdown_connections 2023-10-06T03:57:43.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.161+0000 7fb9f4e53b80 1 --2- 172.21.15.157:0/2537733101 >> [v2:172.21.15.157:6832/1179753185,v1:172.21.15.157:6834/1179753185] conn(0x55a560275260 0x55a5602563a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.161+0000 7fb9f4e53b80 1 --2- 172.21.15.157:0/2537733101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5602705b0 0x55a560274690 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.161+0000 7fb9f4e53b80 1 --2- 172.21.15.157:0/2537733101 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a56022b6b0 0x55a56026dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.161+0000 7fb9f4e53b80 1 --2- 172.21.15.157:0/2537733101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5602701e0 0x55a5602586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.161+0000 7fb9f4e53b80 1 -- 172.21.15.157:0/2537733101 >> 172.21.15.157:0/2537733101 conn(0x55a5602309b0 msgr2=0x55a56022baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:43.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.162+0000 7fb9c3fff700 0 client.4815 ms_handle_reset on v2:172.21.15.157:6832/1179753185 2023-10-06T03:57:43.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.162+0000 7fb9f4e53b80 1 -- 172.21.15.157:0/2537733101 shutdown_connections 2023-10-06T03:57:43.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.162+0000 7fb9f4e53b80 1 -- 172.21.15.157:0/2537733101 wait complete. 2023-10-06T03:57:43.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.162+0000 7fb9f4e53b80 20 client.4815 trim_cache size 0 max 16384 2023-10-06T03:57:43.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.178+0000 7fb9f4e53b80 1 Processor -- start 2023-10-06T03:57:43.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.178+0000 7fb9f4e53b80 1 -- start start 2023-10-06T03:57:43.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.179+0000 7fb9f4e53b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a56022b6b0 0x55a56022a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.179+0000 7fb9f4e53b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5602701e0 0x55a560273fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.179+0000 7fb9f4e53b80 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a5602705b0 0x55a56022f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.179+0000 7fb9f4e53b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a560001e30 con 0x55a5602701e0 2023-10-06T03:57:43.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.179+0000 7fb9f4e53b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a560225d90 con 0x55a56022b6b0 2023-10-06T03:57:43.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.179+0000 7fb9f4e53b80 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5602251a0 con 0x55a5602705b0 2023-10-06T03:57:43.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.179+0000 7fb9e90f6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5602701e0 0x55a560273fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.179+0000 7fb9e90f6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5602701e0 0x55a560273fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44930/0 (socket says 172.21.15.157:44930) 2023-10-06T03:57:43.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.179+0000 7fb9e90f6700 1 -- 172.21.15.157:0/1820274331 learned_addr learned my addr 172.21.15.157:0/1820274331 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:43.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.179+0000 7fb9ea0f8700 1 --2- 172.21.15.157:0/1820274331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a5602705b0 0x55a56022f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.179+0000 7fb9e98f7700 1 --2- 172.21.15.157:0/1820274331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a56022b6b0 0x55a56022a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.179+0000 7fb9e90f6700 1 -- 172.21.15.157:0/1820274331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a5602705b0 msgr2=0x55a56022f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:43.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.179+0000 7fb9e90f6700 1 --2- 172.21.15.157:0/1820274331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a5602705b0 0x55a56022f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.179+0000 7fb9e90f6700 1 -- 172.21.15.157:0/1820274331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a56022b6b0 msgr2=0x55a56022a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:43.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.179+0000 7fb9e90f6700 1 --2- 172.21.15.157:0/1820274331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a56022b6b0 0x55a56022a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.179+0000 7fb9e90f6700 1 -- 172.21.15.157:0/1820274331 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5602579e0 con 0x55a5602701e0 2023-10-06T03:57:43.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.179+0000 7fb9ea0f8700 1 --2- 172.21.15.157:0/1820274331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a5602705b0 0x55a56022f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:43.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.180+0000 7fb9e90f6700 1 --2- 172.21.15.157:0/1820274331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5602701e0 0x55a560273fb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400cf30 tx=0x7fb9d40825c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28103e95e4b985c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:43.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.180+0000 7fb9cf7fe700 1 -- 172.21.15.157:0/1820274331 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d40840d0 con 0x55a5602701e0 2023-10-06T03:57:43.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.180+0000 7fb9cf7fe700 1 -- 172.21.15.157:0/1820274331 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d405c090 con 0x55a5602701e0 2023-10-06T03:57:43.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.180+0000 7fb9cf7fe700 1 -- 172.21.15.157:0/1820274331 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4005e00 con 0x55a5602701e0 2023-10-06T03:57:43.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.180+0000 7fb9f4e53b80 1 -- 172.21.15.157:0/1820274331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5602701e0 msgr2=0x55a560273fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.180+0000 7fb9f4e53b80 1 --2- 172.21.15.157:0/1820274331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5602701e0 0x55a560273fb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400cf30 tx=0x7fb9d40825c0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.180+0000 7fb9f4e53b80 1 -- 172.21.15.157:0/1820274331 shutdown_connections 2023-10-06T03:57:43.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.180+0000 7fb9f4e53b80 1 --2- 172.21.15.157:0/1820274331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a56022b6b0 0x55a56022a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.180+0000 7fb9f4e53b80 1 --2- 172.21.15.157:0/1820274331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a5602705b0 0x55a56022f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.180+0000 7fb9f4e53b80 1 --2- 172.21.15.157:0/1820274331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5602701e0 0x55a560273fb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.180+0000 7fb9f4e53b80 1 -- 172.21.15.157:0/1820274331 >> 172.21.15.157:0/1820274331 conn(0x55a5602309b0 msgr2=0x55a5602311c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:43.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.180+0000 7fb9f4e53b80 1 -- 172.21.15.157:0/1820274331 shutdown_connections 2023-10-06T03:57:43.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.180+0000 7fb9f4e53b80 1 -- 172.21.15.157:0/1820274331 wait complete. 2023-10-06T03:57:43.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.181+0000 7fb9f4e53b80 1 -- start start 2023-10-06T03:57:43.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.181+0000 7fb9f4e53b80 1 Processor -- start 2023-10-06T03:57:43.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.182+0000 7fb9f4e53b80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a56022b6b0 0x55a560271090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.182+0000 7fb9f4e53b80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5602701e0 0x55a5602da410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.182+0000 7fb9f4e53b80 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a5602705b0 0x55a56022e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.182+0000 7fb9f4e53b80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a560283250 con 0x55a56022b6b0 2023-10-06T03:57:43.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.182+0000 7fb9f4e53b80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a560271ad0 con 0x55a5602701e0 2023-10-06T03:57:43.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.182+0000 7fb9f4e53b80 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5602d9ce0 con 0x55a5602705b0 2023-10-06T03:57:43.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.182+0000 7fb9f4e53b80 10 client.0 _ll_register_callbacks cb 0x55a56001bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:57:43.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.182+0000 7fb9e98f7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a56022b6b0 0x55a560271090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.182+0000 7fb9e98f7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a56022b6b0 0x55a560271090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44938/0 (socket says 172.21.15.157:44938) 2023-10-06T03:57:43.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.182+0000 7fb9e98f7700 1 -- 172.21.15.157:0/1274789309 learned_addr learned my addr 172.21.15.157:0/1274789309 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:43.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.182+0000 7fb9e90f6700 1 --2- 172.21.15.157:0/1274789309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5602701e0 0x55a5602da410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.182+0000 7fb9ea0f8700 1 --2- 172.21.15.157:0/1274789309 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a5602705b0 0x55a56022e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.182+0000 7fb9e98f7700 1 -- 172.21.15.157:0/1274789309 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a5602705b0 msgr2=0x55a56022e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:43.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.182+0000 7fb9e98f7700 1 --2- 172.21.15.157:0/1274789309 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a5602705b0 0x55a56022e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.182+0000 7fb9e98f7700 1 -- 172.21.15.157:0/1274789309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5602701e0 msgr2=0x55a5602da410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:43.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.182+0000 7fb9e98f7700 1 --2- 172.21.15.157:0/1274789309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5602701e0 0x55a5602da410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.182+0000 7fb9e98f7700 1 -- 172.21.15.157:0/1274789309 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e4153b10 con 0x55a56022b6b0 2023-10-06T03:57:43.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.182+0000 7fb9e98f7700 1 --2- 172.21.15.157:0/1274789309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a56022b6b0 0x55a560271090 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0007ed0 tx=0x7fb9e0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c352a59dc0dbe7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:43.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.182+0000 7fb9c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-10-06T03:57:43.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.182+0000 7fb9c3fff700 1 -- 172.21.15.157:0/1274789309 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e0028830 con 0x55a56022b6b0 2023-10-06T03:57:43.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.182+0000 7fb9c3fff700 1 -- 172.21.15.157:0/1274789309 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb9e0028990 con 0x55a56022b6b0 2023-10-06T03:57:43.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.182+0000 7fb9f4e53b80 1 -- 172.21.15.157:0/1274789309 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9dc00beb0 con 0x55a56022b6b0 2023-10-06T03:57:43.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.182+0000 7fb9c3fff700 1 -- 172.21.15.157:0/1274789309 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9e0028d60 con 0x55a56022b6b0 2023-10-06T03:57:43.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.182+0000 7fb9f4e53b80 10 client.4821 fetch_fsmap learned FSMap version 16 2023-10-06T03:57:43.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.182+0000 7fb9f4e53b80 1 -- 172.21.15.157:0/1274789309 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55a56014d780 con 0x55a56022b6b0 2023-10-06T03:57:43.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.182+0000 7fb9c3fff700 1 -- 172.21.15.157:0/1274789309 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9dc00beb0 con 0x55a56022b6b0 2023-10-06T03:57:43.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.183+0000 7fb9c3fff700 1 -- 172.21.15.157:0/1274789309 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 16) v1 ==== 4238+0+0 (secure 0 0 0) 0x7fb9e0024410 con 0x55a56022b6b0 2023-10-06T03:57:43.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.183+0000 7fb9f4e53b80 10 client.4821 fetch_fsmap finished waiting for FSMap version 16 2023-10-06T03:57:43.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.183+0000 7fb9f4e53b80 10 client.4821 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T03:57:43.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.183+0000 7fb9f4e53b80 20 client.4821 populate_metadata read hostname 'smithi157' 2023-10-06T03:57:43.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.183+0000 7fb9f4e53b80 1 --2- 172.21.15.157:0/1274789309 >> [v2:172.21.15.157:6832/1179753185,v1:172.21.15.157:6834/1179753185] conn(0x55a56022eaa0 0x55a5602cff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.183+0000 7fb9f4e53b80 4 client.4821 mds_command: new command op to 4287 tid=0[{"prefix": "status"}] 2023-10-06T03:57:43.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.183+0000 7fb9f4e53b80 1 -- 172.21.15.157:0/1274789309 --> [v2:172.21.15.157:6832/1179753185,v1:172.21.15.157:6834/1179753185] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a560121e00 con 0x55a56022eaa0 2023-10-06T03:57:43.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.183+0000 7fb9e90f6700 1 --2- 172.21.15.157:0/1274789309 >> [v2:172.21.15.157:6832/1179753185,v1:172.21.15.157:6834/1179753185] conn(0x55a56022eaa0 0x55a5602cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.183+0000 7fb9c3fff700 10 client.4821 ms_handle_connect on v2:172.21.15.157:6832/1179753185 2023-10-06T03:57:43.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.183+0000 7fb9e90f6700 1 --2- 172.21.15.157:0/1274789309 >> [v2:172.21.15.157:6832/1179753185,v1:172.21.15.157:6834/1179753185] conn(0x55a56022eaa0 0x55a5602cff40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a64d2ee1114f67d0 server_cookie=a977ff696466d45 in_seq=0 out_seq=0 2023-10-06T03:57:43.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.183+0000 7fb9c3fff700 1 -- 172.21.15.157:0/1274789309 <== mds.1 v2:172.21.15.157:6832/1179753185 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55a560121e00 con 0x55a56022eaa0 2023-10-06T03:57:43.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.183+0000 7fb9c3fff700 10 client.4821 handle_command_reply: tid=0 2023-10-06T03:57:43.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.183+0000 7fb9f4e53b80 1 client.4821 shutdown 2023-10-06T03:57:43.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.183+0000 7fb9f4e53b80 1 -- 172.21.15.157:0/1274789309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a56022b6b0 msgr2=0x55a560271090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.183+0000 7fb9f4e53b80 1 --2- 172.21.15.157:0/1274789309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a56022b6b0 0x55a560271090 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0007ed0 tx=0x7fb9e0016000 comp rx=0 tx=0).stop 2023-10-06T03:57:43.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.183+0000 7fb9f4e53b80 1 -- 172.21.15.157:0/1274789309 shutdown_connections 2023-10-06T03:57:43.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.183+0000 7fb9f4e53b80 1 --2- 172.21.15.157:0/1274789309 >> [v2:172.21.15.157:6832/1179753185,v1:172.21.15.157:6834/1179753185] conn(0x55a56022eaa0 0x55a5602cff40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.183+0000 7fb9f4e53b80 1 --2- 172.21.15.157:0/1274789309 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a5602705b0 0x55a56022e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.183+0000 7fb9c3fff700 0 client.4821 ms_handle_reset on v2:172.21.15.157:6832/1179753185 2023-10-06T03:57:43.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.183+0000 7fb9f4e53b80 1 --2- 172.21.15.157:0/1274789309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a56022b6b0 0x55a560271090 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.183+0000 7fb9f4e53b80 1 --2- 172.21.15.157:0/1274789309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a5602701e0 0x55a5602da410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.183+0000 7fb9f4e53b80 1 -- 172.21.15.157:0/1274789309 >> 172.21.15.157:0/1274789309 conn(0x55a5602309b0 msgr2=0x55a5602766d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:43.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.184+0000 7fb9f4e53b80 1 -- 172.21.15.157:0/1274789309 shutdown_connections 2023-10-06T03:57:43.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.184+0000 7fb9f4e53b80 1 -- 172.21.15.157:0/1274789309 wait complete. 2023-10-06T03:57:43.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.184+0000 7fb9f4e53b80 20 client.4821 trim_cache size 0 max 16384 2023-10-06T03:57:43.194 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-10-06T03:57:43.194 INFO:teuthology.orchestra.run.smithi157.stdout: "cluster_fsid": "28664a7a-fe30-4efe-be5a-6abf9318b700", 2023-10-06T03:57:43.194 INFO:teuthology.orchestra.run.smithi157.stdout: "whoami": 1, 2023-10-06T03:57:43.194 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 4287, 2023-10-06T03:57:43.194 INFO:teuthology.orchestra.run.smithi157.stdout: "want_state": "up:active", 2023-10-06T03:57:43.194 INFO:teuthology.orchestra.run.smithi157.stdout: "state": "up:active", 2023-10-06T03:57:43.195 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs", 2023-10-06T03:57:43.195 INFO:teuthology.orchestra.run.smithi157.stdout: "rank_uptime": 26.795275471, 2023-10-06T03:57:43.195 INFO:teuthology.orchestra.run.smithi157.stdout: "mdsmap_epoch": 16, 2023-10-06T03:57:43.195 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch": 21, 2023-10-06T03:57:43.195 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch_barrier": 21, 2023-10-06T03:57:43.195 INFO:teuthology.orchestra.run.smithi157.stdout: "uptime": 37.776530102000002 2023-10-06T03:57:43.195 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-10-06T03:57:43.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.184+0000 7fb9ea8f9700 1 -- 172.21.15.157:0/22630982 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb9c8066f30 msgr2=0x7fb9c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.184+0000 7fb9ea8f9700 1 --2- 172.21.15.157:0/22630982 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb9c8066f30 0x7fb9c80693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0007780 tx=0x7fb9e000b040 comp rx=0 tx=0).stop 2023-10-06T03:57:43.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.184+0000 7fb9ea8f9700 1 -- 172.21.15.157:0/22630982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9e40b04d0 msgr2=0x7fb9e4150340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.184+0000 7fb9ea8f9700 1 --2- 172.21.15.157:0/22630982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9e40b04d0 0x7fb9e4150340 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400c7e0 tx=0x7fb9d4007df0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.184+0000 7fb9ea8f9700 1 -- 172.21.15.157:0/22630982 shutdown_connections 2023-10-06T03:57:43.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.184+0000 7fb9ea8f9700 1 --2- 172.21.15.157:0/22630982 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb9c8066f30 0x7fb9c80693e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.184+0000 7fb9ea8f9700 1 --2- 172.21.15.157:0/22630982 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9e406c2d0 0x7fb9e414fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.184+0000 7fb9ea8f9700 1 --2- 172.21.15.157:0/22630982 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9e40b1e40 0x7fb9e40c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.184+0000 7fb9ea8f9700 1 --2- 172.21.15.157:0/22630982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9e40b04d0 0x7fb9e4150340 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.184+0000 7fb9ea8f9700 1 -- 172.21.15.157:0/22630982 >> 172.21.15.157:0/22630982 conn(0x7fb9e40cc540 msgr2=0x7fb9e4157440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:43.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.184+0000 7fb9ea8f9700 1 -- 172.21.15.157:0/22630982 shutdown_connections 2023-10-06T03:57:43.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.184+0000 7fb9ea8f9700 1 -- 172.21.15.157:0/22630982 wait complete. 2023-10-06T03:57:43.198 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2023-10-06T03:57:43.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.313+0000 7f9bebd45700 1 Processor -- start 2023-10-06T03:57:43.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.313+0000 7f9bebd45700 1 -- start start 2023-10-06T03:57:43.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.314+0000 7f9bebd45700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9be406dc40 0x7f9be414fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.314+0000 7f9bebd45700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9be40c9780 con 0x7f9be406cbb0 2023-10-06T03:57:43.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.314+0000 7f9bebd45700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9be406e0c0 con 0x7f9be406c1f0 2023-10-06T03:57:43.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.314+0000 7f9bebd45700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be40c8c40 con 0x7f9be406dc40 2023-10-06T03:57:43.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.314+0000 7f9be9ae1700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f9be406cbb0 0x7f9be406d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40742/0 (socket says 172.21.15.157:40742) 2023-10-06T03:57:43.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.314+0000 7f9be9ae1700 1 -- 172.21.15.157:0/1432552531 learned_addr learned my addr 172.21.15.157:0/1432552531 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:43.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.315+0000 7f9be92e0700 1 --2- 172.21.15.157:0/1432552531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9be406dc40 0x7f9be414fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.315+0000 7f9be8adf700 1 -- 172.21.15.157:0/1432552531 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3820345436 0 0) 0x7f9be40c9780 con 0x7f9be406cbb0 2023-10-06T03:57:43.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.315+0000 7f9be8adf700 1 -- 172.21.15.157:0/1432552531 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bd0003570 con 0x7f9be406cbb0 2023-10-06T03:57:43.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.315+0000 7f9be8adf700 1 -- 172.21.15.157:0/1432552531 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2574042494 0 0) 0x7f9be406e0c0 con 0x7f9be406c1f0 2023-10-06T03:57:43.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.315+0000 7f9be8adf700 1 -- 172.21.15.157:0/1432552531 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9be40c9780 con 0x7f9be406c1f0 2023-10-06T03:57:43.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.315+0000 7f9be92e0700 1 -- 172.21.15.157:0/1432552531 >> v1:172.21.15.175:6789/0 conn(0x7f9be406c1f0 legacy=0x7f9be4155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.315+0000 7f9be92e0700 1 -- 172.21.15.157:0/1432552531 >> v1:172.21.15.157:6789/0 conn(0x7f9be406cbb0 legacy=0x7f9be406d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.315+0000 7f9be8adf700 1 -- 172.21.15.157:0/1432552531 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 26326332 0 0) 0x7f9bd0003570 con 0x7f9be406cbb0 2023-10-06T03:57:43.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.315+0000 7f9be92e0700 1 -- 172.21.15.157:0/1432552531 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be40c8e50 con 0x7f9be406dc40 2023-10-06T03:57:43.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.315+0000 7f9be92e0700 1 --2- 172.21.15.157:0/1432552531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9be406dc40 0x7f9be414fd00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009910 tx=0x7f9bd40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e181d9c721fd81f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:43.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.315+0000 7f9be8adf700 1 -- 172.21.15.157:0/1432552531 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be00017f0 con 0x7f9be406dc40 2023-10-06T03:57:43.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.315+0000 7f9be8adf700 1 -- 172.21.15.157:0/1432552531 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd4011420 con 0x7f9be406dc40 2023-10-06T03:57:43.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.315+0000 7f9be8adf700 1 -- 172.21.15.157:0/1432552531 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd40117c0 con 0x7f9be406dc40 2023-10-06T03:57:43.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.316+0000 7f9bebd45700 1 -- 172.21.15.157:0/1432552531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9be406dc40 msgr2=0x7f9be414fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.316+0000 7f9bebd45700 1 --2- 172.21.15.157:0/1432552531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9be406dc40 0x7f9be414fd00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009910 tx=0x7f9bd40091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.316+0000 7f9bebd45700 1 -- 172.21.15.157:0/1432552531 shutdown_connections 2023-10-06T03:57:43.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.316+0000 7f9bebd45700 1 --2- 172.21.15.157:0/1432552531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9be406dc40 0x7f9be414fd00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.316+0000 7f9bebd45700 1 -- 172.21.15.157:0/1432552531 >> 172.21.15.157:0/1432552531 conn(0x7f9be406f230 msgr2=0x7f9be406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:43.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.316+0000 7f9bebd45700 1 -- 172.21.15.157:0/1432552531 shutdown_connections 2023-10-06T03:57:43.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.316+0000 7f9bebd45700 1 -- 172.21.15.157:0/1432552531 wait complete. 2023-10-06T03:57:43.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.317+0000 7f9bebd45700 1 Processor -- start 2023-10-06T03:57:43.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.317+0000 7f9bebd45700 1 -- start start 2023-10-06T03:57:43.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.318+0000 7f9bebd45700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9be406c1f0 0x7f9be4152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.318+0000 7f9bebd45700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9be406cbb0 0x7f9be40afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.318+0000 7f9bebd45700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9be416a050 0x7f9be416a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.318+0000 7f9bebd45700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be4166750 con 0x7f9be406c1f0 2023-10-06T03:57:43.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.318+0000 7f9bebd45700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be4148920 con 0x7f9be416a050 2023-10-06T03:57:43.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.318+0000 7f9bebd45700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be4166fb0 con 0x7f9be406cbb0 2023-10-06T03:57:43.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.318+0000 7f9be9ae1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9be406c1f0 0x7f9be4152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.318+0000 7f9be9ae1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9be406c1f0 0x7f9be4152310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44948/0 (socket says 172.21.15.157:44948) 2023-10-06T03:57:43.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.318+0000 7f9be9ae1700 1 -- 172.21.15.157:0/887953343 learned_addr learned my addr 172.21.15.157:0/887953343 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:43.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.318+0000 7f9be92e0700 1 --2- 172.21.15.157:0/887953343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9be406cbb0 0x7f9be40afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.318+0000 7f9bea2e2700 1 --2- 172.21.15.157:0/887953343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9be416a050 0x7f9be416a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.318+0000 7f9be9ae1700 1 -- 172.21.15.157:0/887953343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9be406cbb0 msgr2=0x7f9be40afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.318+0000 7f9be9ae1700 1 --2- 172.21.15.157:0/887953343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9be406cbb0 0x7f9be40afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.318+0000 7f9be9ae1700 1 -- 172.21.15.157:0/887953343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9be416a050 msgr2=0x7f9be416a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.318+0000 7f9be9ae1700 1 --2- 172.21.15.157:0/887953343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9be416a050 0x7f9be416a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.318+0000 7f9be9ae1700 1 -- 172.21.15.157:0/887953343 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd4009420 con 0x7f9be406c1f0 2023-10-06T03:57:43.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.318+0000 7f9bea2e2700 1 --2- 172.21.15.157:0/887953343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9be416a050 0x7f9be416a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:43.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.318+0000 7f9be9ae1700 1 --2- 172.21.15.157:0/887953343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9be406c1f0 0x7f9be4152310 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9be000ccd0 tx=0x7f9be0007da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:43.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.318+0000 7f9bd9ffb700 1 -- 172.21.15.157:0/887953343 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be0004e90 con 0x7f9be406c1f0 2023-10-06T03:57:43.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.318+0000 7f9bd9ffb700 1 -- 172.21.15.157:0/887953343 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be000a6f0 con 0x7f9be406c1f0 2023-10-06T03:57:43.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.318+0000 7f9bebd45700 1 -- 172.21.15.157:0/887953343 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be4148b50 con 0x7f9be406c1f0 2023-10-06T03:57:43.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.318+0000 7f9bd9ffb700 1 -- 172.21.15.157:0/887953343 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be000aa90 con 0x7f9be406c1f0 2023-10-06T03:57:43.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.319+0000 7f9bebd45700 1 -- 172.21.15.157:0/887953343 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be4166960 con 0x7f9be406c1f0 2023-10-06T03:57:43.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.321+0000 7f9bd9ffb700 1 -- 172.21.15.157:0/887953343 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9be0007150 con 0x7f9be406c1f0 2023-10-06T03:57:43.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.322+0000 7f9bd9ffb700 1 --2- 172.21.15.157:0/887953343 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9bc8066f60 0x7f9bc8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.322+0000 7f9bf503db80 1 Processor -- start 2023-10-06T03:57:43.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.322+0000 7f9bd9ffb700 1 -- 172.21.15.157:0/887953343 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9be00841e0 con 0x7f9be406c1f0 2023-10-06T03:57:43.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.322+0000 7f9bf503db80 1 -- start start 2023-10-06T03:57:43.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.322+0000 7f9be92e0700 1 --2- 172.21.15.157:0/887953343 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9bc8066f60 0x7f9bc8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.323+0000 7f9be92e0700 1 --2- 172.21.15.157:0/887953343 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9bc8066f60 0x7f9bc8069410 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40030c0 tx=0x7f9bd4016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:43.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.323+0000 7f9bf503db80 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b9b441f50 0x559b9b443730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.323+0000 7f9bf503db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b9b3febd0 0x559b9b4463e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.323+0000 7f9bf503db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9b446920 0x559b9b43c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.323+0000 7f9bf503db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b9b380c80 con 0x559b9b3febd0 2023-10-06T03:57:43.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.323+0000 7f9bf503db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b9b357a70 con 0x559b9b446920 2023-10-06T03:57:43.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.323+0000 7f9bf503db80 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b9b355270 con 0x559b9b441f50 2023-10-06T03:57:43.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.323+0000 7f9be92e0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b9b3febd0 0x559b9b4463e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.323+0000 7f9be92e0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b9b3febd0 0x559b9b4463e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44958/0 (socket says 172.21.15.157:44958) 2023-10-06T03:57:43.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.323+0000 7f9be92e0700 1 -- 172.21.15.157:0/3583635224 learned_addr learned my addr 172.21.15.157:0/3583635224 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:43.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.323+0000 7f9be9ae1700 1 --2- 172.21.15.157:0/3583635224 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b9b441f50 0x559b9b443730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.323+0000 7f9bea2e2700 1 --2- 172.21.15.157:0/3583635224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9b446920 0x559b9b43c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.323+0000 7f9be92e0700 1 -- 172.21.15.157:0/3583635224 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b9b441f50 msgr2=0x559b9b443730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:43.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.323+0000 7f9be92e0700 1 --2- 172.21.15.157:0/3583635224 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b9b441f50 0x559b9b443730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.323+0000 7f9be92e0700 1 -- 172.21.15.157:0/3583635224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9b446920 msgr2=0x559b9b43c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:43.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.323+0000 7f9be92e0700 1 --2- 172.21.15.157:0/3583635224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9b446920 0x559b9b43c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.323+0000 7f9be92e0700 1 -- 172.21.15.157:0/3583635224 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b9b1eb270 con 0x559b9b3febd0 2023-10-06T03:57:43.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.323+0000 7f9be92e0700 1 --2- 172.21.15.157:0/3583635224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b9b3febd0 0x559b9b4463e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9be40c3960 tx=0x7f9bd4022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e545ba1bf467dd68 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:43.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.323+0000 7f9bceffd700 1 -- 172.21.15.157:0/3583635224 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4022070 con 0x559b9b3febd0 2023-10-06T03:57:43.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.323+0000 7f9bceffd700 1 -- 172.21.15.157:0/3583635224 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd4011d60 con 0x559b9b3febd0 2023-10-06T03:57:43.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.323+0000 7f9bceffd700 1 -- 172.21.15.157:0/3583635224 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd401c8b0 con 0x559b9b3febd0 2023-10-06T03:57:43.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.324+0000 7f9bf503db80 1 -- 172.21.15.157:0/3583635224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b9b3febd0 msgr2=0x559b9b4463e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.324+0000 7f9bf503db80 1 --2- 172.21.15.157:0/3583635224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b9b3febd0 0x559b9b4463e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9be40c3960 tx=0x7f9bd4022000 comp rx=0 tx=0).stop 2023-10-06T03:57:43.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.324+0000 7f9bf503db80 1 -- 172.21.15.157:0/3583635224 shutdown_connections 2023-10-06T03:57:43.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.324+0000 7f9bf503db80 1 --2- 172.21.15.157:0/3583635224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9b446920 0x559b9b43c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.324+0000 7f9bf503db80 1 --2- 172.21.15.157:0/3583635224 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b9b441f50 0x559b9b443730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.324+0000 7f9bf503db80 1 --2- 172.21.15.157:0/3583635224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b9b3febd0 0x559b9b4463e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.324+0000 7f9bf503db80 1 -- 172.21.15.157:0/3583635224 >> 172.21.15.157:0/3583635224 conn(0x559b9b3fffb0 msgr2=0x559b9b400380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:43.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.324+0000 7f9bf503db80 1 -- 172.21.15.157:0/3583635224 shutdown_connections 2023-10-06T03:57:43.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.324+0000 7f9bf503db80 1 -- 172.21.15.157:0/3583635224 wait complete. 2023-10-06T03:57:43.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.325+0000 7f9bf503db80 1 -- start start 2023-10-06T03:57:43.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.325+0000 7f9bf503db80 1 Processor -- start 2023-10-06T03:57:43.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.325+0000 7f9bf503db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9b3febd0 0x559b9b452050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.325+0000 7f9bf503db80 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b9b441f50 0x559b9b440a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.325+0000 7f9bf503db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b9b446920 0x559b9b433020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.325+0000 7f9bf503db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b9b351060 con 0x559b9b446920 2023-10-06T03:57:43.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.325+0000 7f9bf503db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b9b2fc890 con 0x559b9b3febd0 2023-10-06T03:57:43.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.325+0000 7f9bf503db80 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b9b317ea0 con 0x559b9b441f50 2023-10-06T03:57:43.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.326+0000 7f9bf503db80 10 client.0 _ll_register_callbacks cb 0x559b9b1edf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:57:43.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.326+0000 7f9bea2e2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b9b446920 0x559b9b433020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.326+0000 7f9bea2e2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b9b446920 0x559b9b433020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44968/0 (socket says 172.21.15.157:44968) 2023-10-06T03:57:43.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.326+0000 7f9bea2e2700 1 -- 172.21.15.157:0/1655179081 learned_addr learned my addr 172.21.15.157:0/1655179081 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:43.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.326+0000 7f9be9ae1700 1 --2- 172.21.15.157:0/1655179081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9b3febd0 0x559b9b452050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.326+0000 7f9be92e0700 1 --2- 172.21.15.157:0/1655179081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b9b441f50 0x559b9b440a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.326+0000 7f9bea2e2700 1 -- 172.21.15.157:0/1655179081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b9b441f50 msgr2=0x559b9b440a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:43.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.326+0000 7f9bea2e2700 1 --2- 172.21.15.157:0/1655179081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b9b441f50 0x559b9b440a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.326+0000 7f9bea2e2700 1 -- 172.21.15.157:0/1655179081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9b3febd0 msgr2=0x559b9b452050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:43.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.326+0000 7f9bea2e2700 1 --2- 172.21.15.157:0/1655179081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9b3febd0 0x559b9b452050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.326+0000 7f9bea2e2700 1 -- 172.21.15.157:0/1655179081 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be40c8e50 con 0x559b9b446920 2023-10-06T03:57:43.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.326+0000 7f9bea2e2700 1 --2- 172.21.15.157:0/1655179081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b9b446920 0x559b9b433020 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc001430 tx=0x7f9bdc009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a66e9ae7b53ff553 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:43.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.326+0000 7f9bc3fff700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-10-06T03:57:43.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.326+0000 7f9bc3fff700 1 -- 172.21.15.157:0/1655179081 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc014400 con 0x559b9b446920 2023-10-06T03:57:43.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.326+0000 7f9bf503db80 1 -- 172.21.15.157:0/1655179081 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b9b3f5690 con 0x559b9b446920 2023-10-06T03:57:43.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.326+0000 7f9bc3fff700 1 -- 172.21.15.157:0/1655179081 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9bdc014560 con 0x559b9b446920 2023-10-06T03:57:43.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.326+0000 7f9bc3fff700 1 -- 172.21.15.157:0/1655179081 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bdc003560 con 0x559b9b446920 2023-10-06T03:57:43.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.326+0000 7f9bf503db80 10 client.4833 fetch_fsmap learned FSMap version 16 2023-10-06T03:57:43.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.326+0000 7f9bf503db80 1 -- 172.21.15.157:0/1655179081 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x559b9b18dfe0 con 0x559b9b446920 2023-10-06T03:57:43.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.326+0000 7f9bc3fff700 1 -- 172.21.15.157:0/1655179081 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559b9b3f5690 con 0x559b9b446920 2023-10-06T03:57:43.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.327+0000 7f9bc3fff700 1 -- 172.21.15.157:0/1655179081 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 16) v1 ==== 4238+0+0 (secure 0 0 0) 0x7f9bdc014930 con 0x559b9b446920 2023-10-06T03:57:43.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.327+0000 7f9bf503db80 10 client.4833 fetch_fsmap finished waiting for FSMap version 16 2023-10-06T03:57:43.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.327+0000 7f9bf503db80 10 client.4833 resolve_mds: resolved name 'e' to daemon mds.e 2023-10-06T03:57:43.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.327+0000 7f9bf503db80 20 client.4833 populate_metadata read hostname 'smithi157' 2023-10-06T03:57:43.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.327+0000 7f9bf503db80 1 --2- 172.21.15.157:0/1655179081 >> [v2:172.21.15.157:6836/4188652695,v1:172.21.15.157:6837/4188652695] conn(0x559b9b3fc830 0x559b9b3fcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.327+0000 7f9bf503db80 4 client.4833 mds_command: new command op to 4296 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T03:57:43.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.327+0000 7f9bf503db80 1 -- 172.21.15.157:0/1655179081 --> [v2:172.21.15.157:6836/4188652695,v1:172.21.15.157:6837/4188652695] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b9b3d25c0 con 0x559b9b3fc830 2023-10-06T03:57:43.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.327+0000 7f9be9ae1700 1 --2- 172.21.15.157:0/1655179081 >> [v2:172.21.15.157:6836/4188652695,v1:172.21.15.157:6837/4188652695] conn(0x559b9b3fc830 0x559b9b3fcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.327+0000 7f9bc3fff700 10 client.4833 ms_handle_connect on v2:172.21.15.157:6836/4188652695 2023-10-06T03:57:43.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.327+0000 7f9be9ae1700 1 --2- 172.21.15.157:0/1655179081 >> [v2:172.21.15.157:6836/4188652695,v1:172.21.15.157:6837/4188652695] conn(0x559b9b3fc830 0x559b9b3fcc00 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a0fc7d89bd42c1d server_cookie=759681f402032d64 in_seq=0 out_seq=0 2023-10-06T03:57:43.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.328+0000 7f9bc3fff700 1 -- 172.21.15.157:0/1655179081 <== mds.0 v2:172.21.15.157:6836/4188652695 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x559b9b3d25c0 con 0x559b9b3fc830 2023-10-06T03:57:43.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.328+0000 7f9bc3fff700 10 client.4833 handle_command_reply: tid=0 2023-10-06T03:57:43.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.328+0000 7f9bf503db80 1 client.4833 shutdown 2023-10-06T03:57:43.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.328+0000 7f9bf503db80 1 -- 172.21.15.157:0/1655179081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b9b446920 msgr2=0x559b9b433020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.328+0000 7f9bf503db80 1 --2- 172.21.15.157:0/1655179081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b9b446920 0x559b9b433020 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc001430 tx=0x7f9bdc009040 comp rx=0 tx=0).stop 2023-10-06T03:57:43.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.328+0000 7f9bf503db80 1 -- 172.21.15.157:0/1655179081 shutdown_connections 2023-10-06T03:57:43.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.328+0000 7f9bf503db80 1 --2- 172.21.15.157:0/1655179081 >> [v2:172.21.15.157:6836/4188652695,v1:172.21.15.157:6837/4188652695] conn(0x559b9b3fc830 0x559b9b3fcc00 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.328+0000 7f9bf503db80 1 --2- 172.21.15.157:0/1655179081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9b3febd0 0x559b9b452050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.328+0000 7f9bf503db80 1 --2- 172.21.15.157:0/1655179081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b9b446920 0x559b9b433020 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.328+0000 7f9bc3fff700 0 client.4833 ms_handle_reset on v2:172.21.15.157:6836/4188652695 2023-10-06T03:57:43.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.328+0000 7f9bf503db80 1 --2- 172.21.15.157:0/1655179081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b9b441f50 0x559b9b440a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.328+0000 7f9bf503db80 1 -- 172.21.15.157:0/1655179081 >> 172.21.15.157:0/1655179081 conn(0x559b9b3fffb0 msgr2=0x559b9b446d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:43.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.328+0000 7f9bf503db80 1 -- 172.21.15.157:0/1655179081 shutdown_connections 2023-10-06T03:57:43.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.328+0000 7f9bf503db80 1 -- 172.21.15.157:0/1655179081 wait complete. 2023-10-06T03:57:43.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.328+0000 7f9bf503db80 20 client.4833 trim_cache size 0 max 16384 2023-10-06T03:57:43.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.345+0000 7f9bf503db80 1 Processor -- start 2023-10-06T03:57:43.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.345+0000 7f9bf503db80 1 -- start start 2023-10-06T03:57:43.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.346+0000 7f9bf503db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9b3fffb0 0x559b9b3fce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.346+0000 7f9bf503db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b9b3febd0 0x559b9b4021f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.346+0000 7f9bf503db80 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b9b441f50 0x559b9b402730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.346+0000 7f9bf503db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b9b1d3e30 con 0x559b9b3febd0 2023-10-06T03:57:43.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.346+0000 7f9bf503db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b9b447160 con 0x559b9b3fffb0 2023-10-06T03:57:43.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.346+0000 7f9bf503db80 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b9b445c00 con 0x559b9b441f50 2023-10-06T03:57:43.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.346+0000 7f9be92e0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b9b3febd0 0x559b9b4021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.346+0000 7f9be92e0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b9b3febd0 0x559b9b4021f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44978/0 (socket says 172.21.15.157:44978) 2023-10-06T03:57:43.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.346+0000 7f9be92e0700 1 -- 172.21.15.157:0/2600623082 learned_addr learned my addr 172.21.15.157:0/2600623082 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:43.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.346+0000 7f9be9ae1700 1 --2- 172.21.15.157:0/2600623082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9b3fffb0 0x559b9b3fce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.346+0000 7f9bea2e2700 1 --2- 172.21.15.157:0/2600623082 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b9b441f50 0x559b9b402730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.346+0000 7f9be92e0700 1 -- 172.21.15.157:0/2600623082 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b9b441f50 msgr2=0x559b9b402730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:43.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.346+0000 7f9be92e0700 1 --2- 172.21.15.157:0/2600623082 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b9b441f50 0x559b9b402730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.346+0000 7f9be92e0700 1 -- 172.21.15.157:0/2600623082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9b3fffb0 msgr2=0x559b9b3fce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:43.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.346+0000 7f9be92e0700 1 --2- 172.21.15.157:0/2600623082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9b3fffb0 0x559b9b3fce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.346+0000 7f9be92e0700 1 -- 172.21.15.157:0/2600623082 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b9b31f780 con 0x559b9b3febd0 2023-10-06T03:57:43.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.347+0000 7f9be92e0700 1 --2- 172.21.15.157:0/2600623082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b9b3febd0 0x559b9b4021f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4006b90 tx=0x7f9bd402d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f6cd5a826d06eb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:43.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.347+0000 7f9bcf7fe700 1 -- 172.21.15.157:0/2600623082 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4022040 con 0x559b9b3febd0 2023-10-06T03:57:43.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.347+0000 7f9bcf7fe700 1 -- 172.21.15.157:0/2600623082 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd4034470 con 0x559b9b3febd0 2023-10-06T03:57:43.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.347+0000 7f9bcf7fe700 1 -- 172.21.15.157:0/2600623082 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4034810 con 0x559b9b3febd0 2023-10-06T03:57:43.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.347+0000 7f9bf503db80 1 -- 172.21.15.157:0/2600623082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b9b3febd0 msgr2=0x559b9b4021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.347+0000 7f9bf503db80 1 --2- 172.21.15.157:0/2600623082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b9b3febd0 0x559b9b4021f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4006b90 tx=0x7f9bd402d070 comp rx=0 tx=0).stop 2023-10-06T03:57:43.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.347+0000 7f9bf503db80 1 -- 172.21.15.157:0/2600623082 shutdown_connections 2023-10-06T03:57:43.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.347+0000 7f9bf503db80 1 --2- 172.21.15.157:0/2600623082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9b3fffb0 0x559b9b3fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.347+0000 7f9bf503db80 1 --2- 172.21.15.157:0/2600623082 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b9b441f50 0x559b9b402730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.347+0000 7f9bf503db80 1 --2- 172.21.15.157:0/2600623082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b9b3febd0 0x559b9b4021f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.347+0000 7f9bf503db80 1 -- 172.21.15.157:0/2600623082 >> 172.21.15.157:0/2600623082 conn(0x559b9b3fbbe0 msgr2=0x559b9b440a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:43.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.347+0000 7f9bf503db80 1 -- 172.21.15.157:0/2600623082 shutdown_connections 2023-10-06T03:57:43.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.347+0000 7f9bf503db80 1 -- 172.21.15.157:0/2600623082 wait complete. 2023-10-06T03:57:43.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.348+0000 7f9bf503db80 1 -- start start 2023-10-06T03:57:43.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.348+0000 7f9bf503db80 1 Processor -- start 2023-10-06T03:57:43.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.349+0000 7f9bf503db80 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9b3febd0 0x559b9b434250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.349+0000 7f9bf503db80 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b9b3fffb0 0x559b9b44ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.349+0000 7f9bf503db80 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b9b441f50 0x559b9b44ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.349+0000 7f9bf503db80 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b9b434790 con 0x559b9b3fffb0 2023-10-06T03:57:43.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.349+0000 7f9bf503db80 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b9b4abaa0 con 0x559b9b3febd0 2023-10-06T03:57:43.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.349+0000 7f9bf503db80 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b9b402a30 con 0x559b9b441f50 2023-10-06T03:57:43.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.349+0000 7f9bf503db80 10 client.0 _ll_register_callbacks cb 0x559b9b1edf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:57:43.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.349+0000 7f9be92e0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b9b3fffb0 0x559b9b44ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.349+0000 7f9be92e0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b9b3fffb0 0x559b9b44ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44990/0 (socket says 172.21.15.157:44990) 2023-10-06T03:57:43.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.349+0000 7f9be92e0700 1 -- 172.21.15.157:0/933870100 learned_addr learned my addr 172.21.15.157:0/933870100 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:43.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.349+0000 7f9be9ae1700 1 --2- 172.21.15.157:0/933870100 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9b3febd0 0x559b9b434250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.349+0000 7f9bea2e2700 1 --2- 172.21.15.157:0/933870100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b9b441f50 0x559b9b44ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.349+0000 7f9be92e0700 1 -- 172.21.15.157:0/933870100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b9b441f50 msgr2=0x559b9b44ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:43.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.349+0000 7f9be92e0700 1 --2- 172.21.15.157:0/933870100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b9b441f50 0x559b9b44ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.349+0000 7f9be92e0700 1 -- 172.21.15.157:0/933870100 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9b3febd0 msgr2=0x559b9b434250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:43.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.349+0000 7f9be92e0700 1 --2- 172.21.15.157:0/933870100 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9b3febd0 0x559b9b434250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.349+0000 7f9be92e0700 1 -- 172.21.15.157:0/933870100 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b9b1eb270 con 0x559b9b3fffb0 2023-10-06T03:57:43.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.349+0000 7f9bea2e2700 1 --2- 172.21.15.157:0/933870100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b9b441f50 0x559b9b44ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:43.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.349+0000 7f9be92e0700 1 --2- 172.21.15.157:0/933870100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b9b3fffb0 0x559b9b44ea40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4011660 tx=0x7f9bd4037040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b549403b630dcad8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:43.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.349+0000 7f9bc3fff700 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-10-06T03:57:43.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.349+0000 7f9bc3fff700 1 -- 172.21.15.157:0/933870100 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4022040 con 0x559b9b3fffb0 2023-10-06T03:57:43.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.349+0000 7f9bc3fff700 1 -- 172.21.15.157:0/933870100 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9bd4034dd0 con 0x559b9b3fffb0 2023-10-06T03:57:43.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.349+0000 7f9bf503db80 1 -- 172.21.15.157:0/933870100 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9bd4011d60 con 0x559b9b3fffb0 2023-10-06T03:57:43.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.349+0000 7f9bc3fff700 1 -- 172.21.15.157:0/933870100 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bd4034820 con 0x559b9b3fffb0 2023-10-06T03:57:43.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.349+0000 7f9bf503db80 10 client.4839 fetch_fsmap learned FSMap version 16 2023-10-06T03:57:43.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.349+0000 7f9bf503db80 1 -- 172.21.15.157:0/933870100 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x559b9b306470 con 0x559b9b3fffb0 2023-10-06T03:57:43.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.350+0000 7f9bc3fff700 1 -- 172.21.15.157:0/933870100 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd4011d60 con 0x559b9b3fffb0 2023-10-06T03:57:43.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.350+0000 7f9bc3fff700 1 -- 172.21.15.157:0/933870100 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 16) v1 ==== 4238+0+0 (secure 0 0 0) 0x7f9bd402b610 con 0x559b9b3fffb0 2023-10-06T03:57:43.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.350+0000 7f9bf503db80 10 client.4839 fetch_fsmap finished waiting for FSMap version 16 2023-10-06T03:57:43.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.350+0000 7f9bf503db80 10 client.4839 resolve_mds: resolved name 'e' to daemon mds.e 2023-10-06T03:57:43.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.350+0000 7f9bf503db80 20 client.4839 populate_metadata read hostname 'smithi157' 2023-10-06T03:57:43.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.350+0000 7f9bf503db80 1 --2- 172.21.15.157:0/933870100 >> [v2:172.21.15.157:6836/4188652695,v1:172.21.15.157:6837/4188652695] conn(0x559b9b446920 0x559b9b44f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.350+0000 7f9bf503db80 4 client.4839 mds_command: new command op to 4296 tid=0[{"prefix": "status"}] 2023-10-06T03:57:43.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.350+0000 7f9bf503db80 1 -- 172.21.15.157:0/933870100 --> [v2:172.21.15.157:6836/4188652695,v1:172.21.15.157:6837/4188652695] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559b9b2f3e00 con 0x559b9b446920 2023-10-06T03:57:43.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.350+0000 7f9be9ae1700 1 --2- 172.21.15.157:0/933870100 >> [v2:172.21.15.157:6836/4188652695,v1:172.21.15.157:6837/4188652695] conn(0x559b9b446920 0x559b9b44f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.350+0000 7f9bc3fff700 10 client.4839 ms_handle_connect on v2:172.21.15.157:6836/4188652695 2023-10-06T03:57:43.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.350+0000 7f9be9ae1700 1 --2- 172.21.15.157:0/933870100 >> [v2:172.21.15.157:6836/4188652695,v1:172.21.15.157:6837/4188652695] conn(0x559b9b446920 0x559b9b44f4c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b5131ea52047f7e server_cookie=fbf4e55ba62c352d in_seq=0 out_seq=0 2023-10-06T03:57:43.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.350+0000 7f9bc3fff700 1 -- 172.21.15.157:0/933870100 <== mds.0 v2:172.21.15.157:6836/4188652695 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x559b9b2f3e00 con 0x559b9b446920 2023-10-06T03:57:43.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.350+0000 7f9bc3fff700 10 client.4839 handle_command_reply: tid=0 2023-10-06T03:57:43.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.350+0000 7f9bf503db80 1 client.4839 shutdown 2023-10-06T03:57:43.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.350+0000 7f9bf503db80 1 -- 172.21.15.157:0/933870100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b9b3fffb0 msgr2=0x559b9b44ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.350+0000 7f9bf503db80 1 --2- 172.21.15.157:0/933870100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b9b3fffb0 0x559b9b44ea40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4011660 tx=0x7f9bd4037040 comp rx=0 tx=0).stop 2023-10-06T03:57:43.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.350+0000 7f9bf503db80 1 -- 172.21.15.157:0/933870100 shutdown_connections 2023-10-06T03:57:43.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.350+0000 7f9bf503db80 1 --2- 172.21.15.157:0/933870100 >> [v2:172.21.15.157:6836/4188652695,v1:172.21.15.157:6837/4188652695] conn(0x559b9b446920 0x559b9b44f4c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.350+0000 7f9bf503db80 1 --2- 172.21.15.157:0/933870100 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b9b3febd0 0x559b9b434250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.350+0000 7f9bc3fff700 0 client.4839 ms_handle_reset on v2:172.21.15.157:6836/4188652695 2023-10-06T03:57:43.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.350+0000 7f9bf503db80 1 --2- 172.21.15.157:0/933870100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559b9b441f50 0x559b9b44ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.350+0000 7f9bf503db80 1 --2- 172.21.15.157:0/933870100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559b9b3fffb0 0x559b9b44ea40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.350+0000 7f9bf503db80 1 -- 172.21.15.157:0/933870100 >> 172.21.15.157:0/933870100 conn(0x559b9b3fbbe0 msgr2=0x559b9b4021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:43.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.351+0000 7f9bf503db80 1 -- 172.21.15.157:0/933870100 shutdown_connections 2023-10-06T03:57:43.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.351+0000 7f9bf503db80 1 -- 172.21.15.157:0/933870100 wait complete. 2023-10-06T03:57:43.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.351+0000 7f9bf503db80 20 client.4839 trim_cache size 0 max 16384 2023-10-06T03:57:43.360 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-10-06T03:57:43.360 INFO:teuthology.orchestra.run.smithi157.stdout: "cluster_fsid": "28664a7a-fe30-4efe-be5a-6abf9318b700", 2023-10-06T03:57:43.361 INFO:teuthology.orchestra.run.smithi157.stdout: "whoami": 0, 2023-10-06T03:57:43.361 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 4296, 2023-10-06T03:57:43.361 INFO:teuthology.orchestra.run.smithi157.stdout: "want_state": "up:active", 2023-10-06T03:57:43.361 INFO:teuthology.orchestra.run.smithi157.stdout: "state": "up:active", 2023-10-06T03:57:43.361 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs", 2023-10-06T03:57:43.361 INFO:teuthology.orchestra.run.smithi157.stdout: "rank_uptime": 31.988599321999999, 2023-10-06T03:57:43.361 INFO:teuthology.orchestra.run.smithi157.stdout: "mdsmap_epoch": 16, 2023-10-06T03:57:43.361 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch": 21, 2023-10-06T03:57:43.361 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch_barrier": 21, 2023-10-06T03:57:43.361 INFO:teuthology.orchestra.run.smithi157.stdout: "uptime": 37.939016998 2023-10-06T03:57:43.362 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-10-06T03:57:43.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.351+0000 7f9bebd45700 1 -- 172.21.15.157:0/887953343 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9bc8066f60 msgr2=0x7f9bc8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.351+0000 7f9bebd45700 1 --2- 172.21.15.157:0/887953343 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9bc8066f60 0x7f9bc8069410 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40030c0 tx=0x7f9bd4016000 comp rx=0 tx=0).stop 2023-10-06T03:57:43.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.351+0000 7f9bebd45700 1 -- 172.21.15.157:0/887953343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9be406c1f0 msgr2=0x7f9be4152310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.351+0000 7f9bebd45700 1 --2- 172.21.15.157:0/887953343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9be406c1f0 0x7f9be4152310 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9be000ccd0 tx=0x7f9be0007da0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.351+0000 7f9bebd45700 1 -- 172.21.15.157:0/887953343 shutdown_connections 2023-10-06T03:57:43.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.351+0000 7f9bebd45700 1 --2- 172.21.15.157:0/887953343 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9bc8066f60 0x7f9bc8069410 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.351+0000 7f9bebd45700 1 --2- 172.21.15.157:0/887953343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9be416a050 0x7f9be416a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.351+0000 7f9bebd45700 1 --2- 172.21.15.157:0/887953343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9be406c1f0 0x7f9be4152310 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.351+0000 7f9bebd45700 1 --2- 172.21.15.157:0/887953343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9be406cbb0 0x7f9be40afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.351+0000 7f9bebd45700 1 -- 172.21.15.157:0/887953343 >> 172.21.15.157:0/887953343 conn(0x7f9be406f230 msgr2=0x7f9be414fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:43.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.351+0000 7f9bebd45700 1 -- 172.21.15.157:0/887953343 shutdown_connections 2023-10-06T03:57:43.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.351+0000 7f9bebd45700 1 -- 172.21.15.157:0/887953343 wait complete. 2023-10-06T03:57:43.365 INFO:teuthology.run_tasks:Running task kclient... 2023-10-06T03:57:43.400 INFO:tasks.kclient:Mounting kernel clients... 2023-10-06T03:57:43.400 INFO:tasks.kclient:config is {'client.0': {}} 2023-10-06T03:57:43.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:57:43.400 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-10-06T03:57:43.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:57:43.415 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2023-10-06T03:57:43.481 INFO:teuthology.orchestra.run.smithi157.stderr:Cannot find device "ceph-brx" 2023-10-06T03:57:43.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:57:43.483 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-10-06T03:57:43.483 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:57:43.486 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:57:43.526 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:57:43.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.638+0000 7fdb9e8fd700 1 Processor -- start 2023-10-06T03:57:43.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.638+0000 7fdb9e8fd700 1 -- start start 2023-10-06T03:57:43.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.639+0000 7fdb9e8fd700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb980b04d0 0x7fdb98157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.639+0000 7fdb9e8fd700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb9806d8d0 con 0x7fdb980b1e40 2023-10-06T03:57:43.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.639+0000 7fdb9e8fd700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb980af8f0 con 0x7fdb9806c2d0 2023-10-06T03:57:43.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.639+0000 7fdb9e8fd700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9815a690 con 0x7fdb980b04d0 2023-10-06T03:57:43.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.639+0000 7fdb9d8fb700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fdb980b1e40 0x7fdb981563b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40748/0 (socket says 172.21.15.157:40748) 2023-10-06T03:57:43.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.639+0000 7fdb9d8fb700 1 -- 172.21.15.157:0/3300488573 learned_addr learned my addr 172.21.15.157:0/3300488573 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:43.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.639+0000 7fdb9c8f9700 1 -- 172.21.15.157:0/3300488573 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3408725123 0 0) 0x7fdb9806d8d0 con 0x7fdb980b1e40 2023-10-06T03:57:43.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.639+0000 7fdb9d0fa700 1 --2- 172.21.15.157:0/3300488573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb980b04d0 0x7fdb98157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.639+0000 7fdb9c8f9700 1 -- 172.21.15.157:0/3300488573 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb84003570 con 0x7fdb980b1e40 2023-10-06T03:57:43.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.640+0000 7fdb9c8f9700 1 -- 172.21.15.157:0/3300488573 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2504097127 0 0) 0x7fdb84003570 con 0x7fdb980b1e40 2023-10-06T03:57:43.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.640+0000 7fdb9c8f9700 1 -- 172.21.15.157:0/3300488573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb980b04d0 msgr2=0x7fdb98157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:43.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.640+0000 7fdb9c8f9700 1 --2- 172.21.15.157:0/3300488573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb980b04d0 0x7fdb98157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.640+0000 7fdb9c8f9700 1 -- 172.21.15.157:0/3300488573 >> v1:172.21.15.175:6789/0 conn(0x7fdb9806c2d0 legacy=0x7fdb98156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.640+0000 7fdb9c8f9700 1 -- 172.21.15.157:0/3300488573 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb981571d0 con 0x7fdb980b1e40 2023-10-06T03:57:43.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.640+0000 7fdb9c8f9700 1 -- 172.21.15.157:0/3300488573 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7fdb94001470 con 0x7fdb980b1e40 2023-10-06T03:57:43.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.640+0000 7fdb9c8f9700 1 -- 172.21.15.157:0/3300488573 >> v1:172.21.15.157:6789/0 conn(0x7fdb980b1e40 legacy=0x7fdb981563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.640+0000 7fdb9c8f9700 1 --2- 172.21.15.157:0/3300488573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb980b04d0 0x7fdb84003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.640+0000 7fdb9c8f9700 1 --2- 172.21.15.157:0/3300488573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb84004010 0x7fdb84004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.640+0000 7fdb9c8f9700 1 --2- 172.21.15.157:0/3300488573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb84004d50 0x7fdb840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.640+0000 7fdb9c8f9700 1 -- 172.21.15.157:0/3300488573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9815a690 con 0x7fdb84004d50 2023-10-06T03:57:43.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.640+0000 7fdb9c8f9700 1 -- 172.21.15.157:0/3300488573 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8400bbf0 con 0x7fdb84004010 2023-10-06T03:57:43.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.640+0000 7fdb9c8f9700 1 -- 172.21.15.157:0/3300488573 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8400bdb0 con 0x7fdb980b04d0 2023-10-06T03:57:43.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.640+0000 7fdb9e0fc700 1 --2- 172.21.15.157:0/3300488573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb84004d50 0x7fdb840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.640+0000 7fdb9d8fb700 1 --2- 172.21.15.157:0/3300488573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb980b04d0 0x7fdb84003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.640+0000 7fdb9d0fa700 1 --2- 172.21.15.157:0/3300488573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb84004010 0x7fdb84004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.640+0000 7fdb9e0fc700 1 -- 172.21.15.157:0/3300488573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb980b04d0 msgr2=0x7fdb84003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:43.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.640+0000 7fdb9e0fc700 1 --2- 172.21.15.157:0/3300488573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb980b04d0 0x7fdb84003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.640+0000 7fdb9e0fc700 1 -- 172.21.15.157:0/3300488573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb84004010 msgr2=0x7fdb84004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:43.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.640+0000 7fdb9e0fc700 1 --2- 172.21.15.157:0/3300488573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb84004010 0x7fdb84004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.640+0000 7fdb9e0fc700 1 -- 172.21.15.157:0/3300488573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb84003570 con 0x7fdb84004d50 2023-10-06T03:57:43.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.640+0000 7fdb9e0fc700 1 --2- 172.21.15.157:0/3300488573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb84004d50 0x7fdb840051c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdb90001f90 tx=0x7fdb90001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3682e718be53cb5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:43.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.640+0000 7fdb9c8f9700 1 -- 172.21.15.157:0/3300488573 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb90001950 con 0x7fdb84004d50 2023-10-06T03:57:43.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.640+0000 7fdb9c8f9700 1 -- 172.21.15.157:0/3300488573 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9000c8b0 con 0x7fdb84004d50 2023-10-06T03:57:43.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.640+0000 7fdb9c8f9700 1 -- 172.21.15.157:0/3300488573 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9000cc70 con 0x7fdb84004d50 2023-10-06T03:57:43.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.641+0000 7fdb9e8fd700 1 -- 172.21.15.157:0/3300488573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb84004d50 msgr2=0x7fdb840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.641+0000 7fdb9e8fd700 1 --2- 172.21.15.157:0/3300488573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb84004d50 0x7fdb840051c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdb90001f90 tx=0x7fdb90001640 comp rx=0 tx=0).stop 2023-10-06T03:57:43.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.641+0000 7fdb9e0fc700 1 -- 172.21.15.157:0/3300488573 reap_dead start 2023-10-06T03:57:43.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.641+0000 7fdb9e8fd700 1 -- 172.21.15.157:0/3300488573 shutdown_connections 2023-10-06T03:57:43.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.641+0000 7fdb9e8fd700 1 -- 172.21.15.157:0/3300488573 >> 172.21.15.157:0/3300488573 conn(0x7fdb980cc540 msgr2=0x7fdb980cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:43.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.641+0000 7fdb9e8fd700 1 -- 172.21.15.157:0/3300488573 shutdown_connections 2023-10-06T03:57:43.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.641+0000 7fdb9e8fd700 1 -- 172.21.15.157:0/3300488573 wait complete. 2023-10-06T03:57:43.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.642+0000 7fdb9e8fd700 1 Processor -- start 2023-10-06T03:57:43.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.642+0000 7fdb9e8fd700 1 -- start start 2023-10-06T03:57:43.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.643+0000 7fdb9e8fd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb84004d50 0x7fdb980c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.643+0000 7fdb9e8fd700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb98152110 0x7fdb9814fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.643+0000 7fdb9e8fd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb980c9630 0x7fdb98169310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.643+0000 7fdb9e8fd700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb98153410 con 0x7fdb980c9630 2023-10-06T03:57:43.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.643+0000 7fdb9e8fd700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb98152e90 con 0x7fdb84004d50 2023-10-06T03:57:43.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.643+0000 7fdb9e8fd700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb98153130 con 0x7fdb98152110 2023-10-06T03:57:43.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.643+0000 7fdb9e0fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb980c9630 0x7fdb98169310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.643+0000 7fdb9d8fb700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb84004d50 0x7fdb980c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.643+0000 7fdb9d0fa700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb98152110 0x7fdb9814fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.643+0000 7fdb9e0fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb980c9630 0x7fdb98169310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45016/0 (socket says 172.21.15.157:45016) 2023-10-06T03:57:43.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.643+0000 7fdb9e0fc700 1 -- 172.21.15.157:0/2199486973 learned_addr learned my addr 172.21.15.157:0/2199486973 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:43.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.643+0000 7fdb9e0fc700 1 -- 172.21.15.157:0/2199486973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb98152110 msgr2=0x7fdb9814fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.643+0000 7fdb9e0fc700 1 --2- 172.21.15.157:0/2199486973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb98152110 0x7fdb9814fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.643+0000 7fdb9e0fc700 1 -- 172.21.15.157:0/2199486973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb84004d50 msgr2=0x7fdb980c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.643+0000 7fdb9e0fc700 1 --2- 172.21.15.157:0/2199486973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb84004d50 0x7fdb980c80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.643+0000 7fdb9e0fc700 1 -- 172.21.15.157:0/2199486973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb980c6cb0 con 0x7fdb980c9630 2023-10-06T03:57:43.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.643+0000 7fdb9e0fc700 1 --2- 172.21.15.157:0/2199486973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb980c9630 0x7fdb98169310 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdb90001220 tx=0x7fdb90015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:43.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.643+0000 7fdb8dffb700 1 -- 172.21.15.157:0/2199486973 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb90001950 con 0x7fdb980c9630 2023-10-06T03:57:43.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.643+0000 7fdb8dffb700 1 -- 172.21.15.157:0/2199486973 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9000c6d0 con 0x7fdb980c9630 2023-10-06T03:57:43.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.643+0000 7fdb8dffb700 1 -- 172.21.15.157:0/2199486973 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9001f640 con 0x7fdb980c9630 2023-10-06T03:57:43.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.643+0000 7fdb9e8fd700 1 -- 172.21.15.157:0/2199486973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb98153410 con 0x7fdb980c9630 2023-10-06T03:57:43.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.644+0000 7fdb9e8fd700 1 -- 172.21.15.157:0/2199486973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb98150650 con 0x7fdb980c9630 2023-10-06T03:57:43.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.645+0000 7fdb9e8fd700 1 -- 172.21.15.157:0/2199486973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb980c9350 con 0x7fdb980c9630 2023-10-06T03:57:43.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.645+0000 7fdb8dffb700 1 -- 172.21.15.157:0/2199486973 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdb9001fa70 con 0x7fdb980c9630 2023-10-06T03:57:43.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.645+0000 7fdb8dffb700 1 --2- 172.21.15.157:0/2199486973 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdb7c066f30 0x7fdb7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:43.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.645+0000 7fdb8dffb700 1 -- 172.21.15.157:0/2199486973 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdb90086320 con 0x7fdb980c9630 2023-10-06T03:57:43.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.646+0000 7fdb9d8fb700 1 --2- 172.21.15.157:0/2199486973 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdb7c066f30 0x7fdb7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:43.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.646+0000 7fdb9d8fb700 1 --2- 172.21.15.157:0/2199486973 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdb7c066f30 0x7fdb7c0693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdb940010e0 tx=0x7fdb94001e50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:43.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.650+0000 7fdb8dffb700 1 -- 172.21.15.157:0/2199486973 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdb980c9350 con 0x7fdb980c9630 2023-10-06T03:57:43.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.877+0000 7fdb9e8fd700 1 -- 172.21.15.157:0/2199486973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdb980c9350 con 0x7fdb980c9630 2023-10-06T03:57:43.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.877+0000 7fdb8dffb700 1 -- 172.21.15.157:0/2199486973 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdb90053390 con 0x7fdb980c9630 2023-10-06T03:57:43.879 INFO:teuthology.orchestra.run.smithi157.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:57:43.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.879+0000 7fdb9e8fd700 1 -- 172.21.15.157:0/2199486973 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdb7c066f30 msgr2=0x7fdb7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.879+0000 7fdb9e8fd700 1 --2- 172.21.15.157:0/2199486973 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdb7c066f30 0x7fdb7c0693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdb940010e0 tx=0x7fdb94001e50 comp rx=0 tx=0).stop 2023-10-06T03:57:43.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.879+0000 7fdb9e8fd700 1 -- 172.21.15.157:0/2199486973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb980c9630 msgr2=0x7fdb98169310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:43.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.879+0000 7fdb9e8fd700 1 --2- 172.21.15.157:0/2199486973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb980c9630 0x7fdb98169310 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdb90001220 tx=0x7fdb90015000 comp rx=0 tx=0).stop 2023-10-06T03:57:43.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.879+0000 7fdb9e8fd700 1 -- 172.21.15.157:0/2199486973 shutdown_connections 2023-10-06T03:57:43.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.879+0000 7fdb9e8fd700 1 --2- 172.21.15.157:0/2199486973 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdb7c066f30 0x7fdb7c0693e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.879+0000 7fdb9e8fd700 1 --2- 172.21.15.157:0/2199486973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb84004d50 0x7fdb980c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.879+0000 7fdb9e8fd700 1 --2- 172.21.15.157:0/2199486973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb980c9630 0x7fdb98169310 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.879+0000 7fdb9e8fd700 1 --2- 172.21.15.157:0/2199486973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb98152110 0x7fdb9814fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:43.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.879+0000 7fdb9e8fd700 1 -- 172.21.15.157:0/2199486973 >> 172.21.15.157:0/2199486973 conn(0x7fdb980cc540 msgr2=0x7fdb9806dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:43.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.879+0000 7fdb9e8fd700 1 -- 172.21.15.157:0/2199486973 shutdown_connections 2023-10-06T03:57:43.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:43.879+0000 7fdb9e8fd700 1 -- 172.21.15.157:0/2199486973 wait complete. 2023-10-06T03:57:43.893 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T03:57:43.893 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:57:43.893 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2023-10-06T03:57:43.893 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T03:57:43.893 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T03:57:43.893 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T03:57:43.893 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T03:57:43.893 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:57:43.893 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:57:43.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:57:43.894 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2023-10-06T03:57:43.948 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:57:43.948 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:57:43.948 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:57:43.948 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:57:43.948 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2023-10-06T03:57:43.949 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:57:43.949 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:57:43.949 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:57:43.949 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2023-10-06T03:57:43.949 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:57:43.949 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:57:43.949 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2023-10-06T03:57:43.949 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:57:43.949 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:57:43.949 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:57:43.950 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:57:43.950 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:57:43.950 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42481sec preferred_lft 42481sec 2023-10-06T03:57:43.950 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2023-10-06T03:57:43.950 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:57:43.950 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T03:57:43.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:57:43.951 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-10-06T03:57:43.951 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-10-06T03:57:43.951 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add name ceph-brx type bridge 2023-10-06T03:57:43.951 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip addr flush dev ceph-brx 2023-10-06T03:57:43.951 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set ceph-brx up 2023-10-06T03:57:43.951 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T03:57:43.951 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-10-06T03:57:44.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:57:44.098 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:57:44.139 INFO:teuthology.orchestra.run.smithi157.stdout:1 2023-10-06T03:57:44.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:57:44.141 DEBUG:teuthology.orchestra.run.smithi157:> route 2023-10-06T03:57:44.204 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2023-10-06T03:57:44.204 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:57:44.205 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:57:44.205 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:57:44.205 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:57:44.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:57:44.205 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-10-06T03:57:44.205 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-10-06T03:57:44.206 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:57:44.206 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:57:44.206 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:57:44.206 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-10-06T03:57:44.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:57:44.472 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-10-06T03:57:44.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:57:44.487 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list-id 2023-10-06T03:57:44.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:57:44.541 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-10-06T03:57:44.541 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-10-06T03:57:44.541 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T03:57:44.541 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-06T03:57:44.541 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-10-06T03:57:44.631 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-06T03:57:44.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:57:44.632 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-10-06T03:57:44.632 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-10-06T03:57:44.632 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-06T03:57:44.632 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T03:57:44.632 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-06T03:57:44.632 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-06T03:57:44.632 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-06T03:57:44.632 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-10-06T03:57:44.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:57:44.834 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-10-06T03:57:44.834 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-10-06T03:57:44.834 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2023-10-06T03:57:44.834 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T03:57:44.834 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-10-06T03:57:44.927 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:57:44.927 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T03:57:44.941 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:57:44.942 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:57:44.942 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:57:44.996 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T03:57:44.996 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:57:44.996 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:57:45.080 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:57:45.080 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:57:45.165 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:57:45.165 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:57:45.165 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.157:3300,172.21.15.175:3300,172.21.15.175:3301" 2023-10-06T03:57:45.165 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 0@28664a7a-fe30-4efe-be5a-6abf9318b700.cephfs=/ 2023-10-06T03:57:45.165 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.157:3300/172.21.15.175:3300/172.21.15.175:3301" will pass to kernel 2023-10-06T03:57:45.165 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:57:45.165 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:57:45.165 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:57:45.188 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:57:45.188 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:57:45.188 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.157:3300,172.21.15.175:3300,172.21.15.175:3301" 2023-10-06T03:57:45.188 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 0@28664a7a-fe30-4efe-be5a-6abf9318b700.cephfs=/ 2023-10-06T03:57:45.188 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.157:3300/172.21.15.175:3300/172.21.15.175:3301" will pass to kernel 2023-10-06T03:57:45.188 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:57:45.188 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:57:45.189 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:57:45.189 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.157:6789,172.21.15.175:6789,172.21.15.175:6790" 2023-10-06T03:57:45.189 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.157:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2023-10-06T03:57:45.189 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=28664a7a-fe30-4efe-be5a-6abf9318b700" will pass to kernel 2023-10-06T03:57:45.191 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:57:45.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:57:45.191 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:57:45.223 INFO:teuthology.run_tasks:Running task mds_thrash... 2023-10-06T03:57:45.248 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:57:45.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.378+0000 7f083820b700 1 Processor -- start 2023-10-06T03:57:45.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.379+0000 7f083820b700 1 -- start start 2023-10-06T03:57:45.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.379+0000 7f083820b700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f083006f5f0 0x7f0830152950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:45.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.379+0000 7f083820b700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f083006fc70 con 0x7f083006f220 2023-10-06T03:57:45.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.379+0000 7f083820b700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08300bd880 con 0x7f0830068f10 2023-10-06T03:57:45.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.379+0000 7f083820b700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0830152e90 con 0x7f083006f5f0 2023-10-06T03:57:45.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.380+0000 7f0835fa7700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f083006f220 0x7f08300c35f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40750/0 (socket says 172.21.15.157:40750) 2023-10-06T03:57:45.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.380+0000 7f0835fa7700 1 -- 172.21.15.157:0/1650916689 learned_addr learned my addr 172.21.15.157:0/1650916689 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:45.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.380+0000 7f08367a8700 1 --2- 172.21.15.157:0/1650916689 >> v2:172.21.15.175:3300/0 conn(0x7f083006f5f0 0x7f0830152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:45.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.380+0000 7f0834fa5700 1 -- 172.21.15.157:0/1650916689 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2431008231 0 0) 0x7f083006fc70 con 0x7f083006f220 2023-10-06T03:57:45.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.380+0000 7f0834fa5700 1 -- 172.21.15.157:0/1650916689 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f081c003570 con 0x7f083006f220 2023-10-06T03:57:45.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.380+0000 7f0834fa5700 1 -- 172.21.15.157:0/1650916689 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2121691221 0 0) 0x7f08300bd880 con 0x7f0830068f10 2023-10-06T03:57:45.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.380+0000 7f0834fa5700 1 -- 172.21.15.157:0/1650916689 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f083006fc70 con 0x7f0830068f10 2023-10-06T03:57:45.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.380+0000 7f0834fa5700 1 -- 172.21.15.157:0/1650916689 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 973710475 0 0) 0x7f081c003570 con 0x7f083006f220 2023-10-06T03:57:45.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.380+0000 7f0834fa5700 1 -- 172.21.15.157:0/1650916689 >> v2:172.21.15.175:3300/0 conn(0x7f083006f5f0 msgr2=0x7f0830152950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:45.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.380+0000 7f0834fa5700 1 --2- 172.21.15.157:0/1650916689 >> v2:172.21.15.175:3300/0 conn(0x7f083006f5f0 0x7f0830152950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:45.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.380+0000 7f0834fa5700 1 -- 172.21.15.157:0/1650916689 >> v1:172.21.15.175:6789/0 conn(0x7f0830068f10 legacy=0x7f083014f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:45.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.380+0000 7f0834fa5700 1 -- 172.21.15.157:0/1650916689 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08300c3160 con 0x7f083006f220 2023-10-06T03:57:45.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.380+0000 7f0834fa5700 1 -- 172.21.15.157:0/1650916689 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f082c0017f0 con 0x7f083006f220 2023-10-06T03:57:45.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.380+0000 7f0834fa5700 1 -- 172.21.15.157:0/1650916689 >> v1:172.21.15.157:6789/0 conn(0x7f083006f220 legacy=0x7f08300c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:45.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.380+0000 7f0834fa5700 1 --2- 172.21.15.157:0/1650916689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081c003bf0 0x7f081c003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:45.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.380+0000 7f0834fa5700 1 --2- 172.21.15.157:0/1650916689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f081c004810 0x7f081c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:45.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.380+0000 7f0834fa5700 1 --2- 172.21.15.157:0/1650916689 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f081c0051c0 0x7f081c00cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:45.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.380+0000 7f0834fa5700 1 -- 172.21.15.157:0/1650916689 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0830152e90 con 0x7f081c003bf0 2023-10-06T03:57:45.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.380+0000 7f0834fa5700 1 -- 172.21.15.157:0/1650916689 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081c00d1c0 con 0x7f081c004810 2023-10-06T03:57:45.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.380+0000 7f0834fa5700 1 -- 172.21.15.157:0/1650916689 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081c005ab0 con 0x7f081c0051c0 2023-10-06T03:57:45.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.380+0000 7f0835fa7700 1 --2- 172.21.15.157:0/1650916689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081c003bf0 0x7f081c003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:45.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.380+0000 7f08357a6700 1 --2- 172.21.15.157:0/1650916689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f081c004810 0x7f081c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:45.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.380+0000 7f08367a8700 1 --2- 172.21.15.157:0/1650916689 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f081c0051c0 0x7f081c00cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:45.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.380+0000 7f0835fa7700 1 -- 172.21.15.157:0/1650916689 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f081c0051c0 msgr2=0x7f081c00cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:45.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.380+0000 7f0835fa7700 1 --2- 172.21.15.157:0/1650916689 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f081c0051c0 0x7f081c00cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:45.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.380+0000 7f0835fa7700 1 -- 172.21.15.157:0/1650916689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f081c004810 msgr2=0x7f081c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:45.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.380+0000 7f0835fa7700 1 --2- 172.21.15.157:0/1650916689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f081c004810 0x7f081c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:45.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.380+0000 7f0835fa7700 1 -- 172.21.15.157:0/1650916689 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081c003570 con 0x7f081c003bf0 2023-10-06T03:57:45.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.381+0000 7f08367a8700 1 -- 172.21.15.157:0/1650916689 reap_dead start 2023-10-06T03:57:45.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.381+0000 7f0835fa7700 1 --2- 172.21.15.157:0/1650916689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081c003bf0 0x7f081c003fe0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f082c00a8f0 tx=0x7f08300c48b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fad8699f40c25d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:45.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.381+0000 7f0834fa5700 1 -- 172.21.15.157:0/1650916689 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082c0070d0 con 0x7f081c003bf0 2023-10-06T03:57:45.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.381+0000 7f0834fa5700 1 -- 172.21.15.157:0/1650916689 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f082c007250 con 0x7f081c003bf0 2023-10-06T03:57:45.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.381+0000 7f0834fa5700 1 -- 172.21.15.157:0/1650916689 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082c007610 con 0x7f081c003bf0 2023-10-06T03:57:45.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.381+0000 7f083820b700 1 -- 172.21.15.157:0/1650916689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081c003bf0 msgr2=0x7f081c003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:45.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.381+0000 7f083820b700 1 --2- 172.21.15.157:0/1650916689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081c003bf0 0x7f081c003fe0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f082c00a8f0 tx=0x7f08300c48b0 comp rx=0 tx=0).stop 2023-10-06T03:57:45.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.381+0000 7f083820b700 1 -- 172.21.15.157:0/1650916689 shutdown_connections 2023-10-06T03:57:45.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.381+0000 7f083820b700 1 --2- 172.21.15.157:0/1650916689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f081c003bf0 0x7f081c003fe0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:45.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.381+0000 7f083820b700 1 -- 172.21.15.157:0/1650916689 >> 172.21.15.157:0/1650916689 conn(0x7f08300c8d70 msgr2=0x7f08300c4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:45.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.381+0000 7f083820b700 1 -- 172.21.15.157:0/1650916689 shutdown_connections 2023-10-06T03:57:45.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.381+0000 7f083820b700 1 -- 172.21.15.157:0/1650916689 wait complete. 2023-10-06T03:57:45.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.383+0000 7f083820b700 1 Processor -- start 2023-10-06T03:57:45.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.383+0000 7f083820b700 1 -- start start 2023-10-06T03:57:45.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.383+0000 7f083820b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f081c003bf0 0x7f0830156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:45.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.383+0000 7f083820b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08300c7030 0x7f083014d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:45.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.383+0000 7f083820b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08300c2110 0x7f083014d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:45.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.383+0000 7f083820b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08300c12a0 con 0x7f08300c2110 2023-10-06T03:57:45.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.383+0000 7f083820b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083014e9d0 con 0x7f08300c7030 2023-10-06T03:57:45.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.383+0000 7f083820b700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08300c1cf0 con 0x7f081c003bf0 2023-10-06T03:57:45.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.383+0000 7f08367a8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08300c2110 0x7f083014d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:45.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.383+0000 7f08367a8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08300c2110 0x7f083014d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45042/0 (socket says 172.21.15.157:45042) 2023-10-06T03:57:45.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.383+0000 7f08367a8700 1 -- 172.21.15.157:0/4194888432 learned_addr learned my addr 172.21.15.157:0/4194888432 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:45.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.384+0000 7f0835fa7700 1 --2- 172.21.15.157:0/4194888432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f081c003bf0 0x7f0830156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:45.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.384+0000 7f08357a6700 1 --2- 172.21.15.157:0/4194888432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08300c7030 0x7f083014d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:45.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.384+0000 7f08367a8700 1 -- 172.21.15.157:0/4194888432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f081c003bf0 msgr2=0x7f0830156370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:45.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.384+0000 7f08367a8700 1 --2- 172.21.15.157:0/4194888432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f081c003bf0 0x7f0830156370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:45.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.384+0000 7f08367a8700 1 -- 172.21.15.157:0/4194888432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08300c7030 msgr2=0x7f083014d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:45.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.384+0000 7f08367a8700 1 --2- 172.21.15.157:0/4194888432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08300c7030 0x7f083014d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:45.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.384+0000 7f08367a8700 1 -- 172.21.15.157:0/4194888432 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08301528b0 con 0x7f08300c2110 2023-10-06T03:57:45.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.384+0000 7f08357a6700 1 --2- 172.21.15.157:0/4194888432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08300c7030 0x7f083014d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:57:45.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.384+0000 7f08367a8700 1 --2- 172.21.15.157:0/4194888432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08300c2110 0x7f083014d8b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f08300c8270 tx=0x7f08280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:45.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.384+0000 7f0825ffb700 1 -- 172.21.15.157:0/4194888432 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082800f910 con 0x7f08300c2110 2023-10-06T03:57:45.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.384+0000 7f0825ffb700 1 -- 172.21.15.157:0/4194888432 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f082800fa70 con 0x7f08300c2110 2023-10-06T03:57:45.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.384+0000 7f083820b700 1 -- 172.21.15.157:0/4194888432 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08300bff30 con 0x7f08300c2110 2023-10-06T03:57:45.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.384+0000 7f0825ffb700 1 -- 172.21.15.157:0/4194888432 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082800fe10 con 0x7f08300c2110 2023-10-06T03:57:45.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.384+0000 7f083820b700 1 -- 172.21.15.157:0/4194888432 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08300c04f0 con 0x7f08300c2110 2023-10-06T03:57:45.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.385+0000 7f083820b700 1 -- 172.21.15.157:0/4194888432 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08300716b0 con 0x7f08300c2110 2023-10-06T03:57:45.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.386+0000 7f0825ffb700 1 -- 172.21.15.157:0/4194888432 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0828014650 con 0x7f08300c2110 2023-10-06T03:57:45.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.386+0000 7f0825ffb700 1 --2- 172.21.15.157:0/4194888432 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0814066f30 0x7f08140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:45.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.386+0000 7f0825ffb700 1 -- 172.21.15.157:0/4194888432 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0828084cf0 con 0x7f08300c2110 2023-10-06T03:57:45.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.386+0000 7f0835fa7700 1 --2- 172.21.15.157:0/4194888432 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0814066f30 0x7f08140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:45.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.387+0000 7f0835fa7700 1 --2- 172.21.15.157:0/4194888432 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0814066f30 0x7f08140693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f082c015000 tx=0x7f082c013000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:45.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.390+0000 7f0825ffb700 1 -- 172.21.15.157:0/4194888432 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0828089070 con 0x7f08300c2110 2023-10-06T03:57:45.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.578+0000 7f083820b700 1 -- 172.21.15.157:0/4194888432 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08300c9170 con 0x7f08300c2110 2023-10-06T03:57:45.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.579+0000 7f0825ffb700 1 -- 172.21.15.157:0/4194888432 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f0828022090 con 0x7f08300c2110 2023-10-06T03:57:45.581 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:45.581 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":21,"fsid":"28664a7a-fe30-4efe-be5a-6abf9318b700","created":"2023-10-06T03:56:51.887538+0000","modified":"2023-10-06T03:57:12.362537+0000","last_up_change":"2023-10-06T03:57:02.299440+0000","last_in_change":"2023-10-06T03:56:55.141505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:57:03.455121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:57:03.905661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:57:08.377186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:57:09.709582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f677cac-e9f4-4608-acac-718b072ce984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":90640},{"type":"v1","addr":"172.21.15.157:6809","nonce":90640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":90640},{"type":"v1","addr":"172.21.15.157:6811","nonce":90640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":90640},{"type":"v1","addr":"172.21.15.157:6815","nonce":90640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":90640},{"type":"v1","addr":"172.21.15.157:6813","nonce":90640}]},"public_addr":"172.21.15.157:6809/90640","cluster_addr":"172.21.15.157:6811/90640","heartbeat_back_addr":"172.21.15.157:6815/90640","heartbeat_front_addr":"172.21.15.157:6813/90640","state":["exists","up"]},{"osd":1,"uuid":"7d771892-bb91-4e9b-9610-051861c11247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":90654},{"type":"v1","addr":"172.21.15.157:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":90654},{"type":"v1","addr":"172.21.15.157:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":90654},{"type":"v1","addr":"172.21.15.157:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":90654},{"type":"v1","addr":"172.21.15.157:6805","nonce":90654}]},"public_addr":"172.21.15.157:6801/90654","cluster_addr":"172.21.15.157:6803/90654","heartbeat_back_addr":"172.21.15.157:6807/90654","heartbeat_front_addr":"172.21.15.157:6805/90654","state":["exists","up"]},{"osd":2,"uuid":"5c1f7d4e-d185-47cf-ab6c-453b4e3009c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":90657},{"type":"v1","addr":"172.21.15.157:6825","nonce":90657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":90657},{"type":"v1","addr":"172.21.15.157:6827","nonce":90657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":90657},{"type":"v1","addr":"172.21.15.157:6831","nonce":90657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":90657},{"type":"v1","addr":"172.21.15.157:6829","nonce":90657}]},"public_addr":"172.21.15.157:6825/90657","cluster_addr":"172.21.15.157:6827/90657","heartbeat_back_addr":"172.21.15.157:6831/90657","heartbeat_front_addr":"172.21.15.157:6829/90657","state":["exists","up"]},{"osd":3,"uuid":"f6a1e825-43c7-4afd-8daf-e51d45afc75e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":90641},{"type":"v1","addr":"172.21.15.157:6817","nonce":90641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":90641},{"type":"v1","addr":"172.21.15.157:6819","nonce":90641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":90641},{"type":"v1","addr":"172.21.15.157:6823","nonce":90641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":90641},{"type":"v1","addr":"172.21.15.157:6821","nonce":90641}]},"public_addr":"172.21.15.157:6817/90641","cluster_addr":"172.21.15.157:6819/90641","heartbeat_back_addr":"172.21.15.157:6823/90641","heartbeat_front_addr":"172.21.15.157:6821/90641","state":["exists","up"]},{"osd":4,"uuid":"8ad6e3d8-9a31-4f85-9410-417b7ac5da97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":91261},{"type":"v1","addr":"172.21.15.175:6803","nonce":91261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":91261},{"type":"v1","addr":"172.21.15.175:6806","nonce":91261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":91261},{"type":"v1","addr":"172.21.15.175:6814","nonce":91261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":91261},{"type":"v1","addr":"172.21.15.175:6810","nonce":91261}]},"public_addr":"172.21.15.175:6803/91261","cluster_addr":"172.21.15.175:6806/91261","heartbeat_back_addr":"172.21.15.175:6814/91261","heartbeat_front_addr":"172.21.15.175:6810/91261","state":["exists","up"]},{"osd":5,"uuid":"f68aca2a-f182-4c00-8e21-241d09c02ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":91262},{"type":"v1","addr":"172.21.15.175:6817","nonce":91262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":91262},{"type":"v1","addr":"172.21.15.175:6819","nonce":91262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":91262},{"type":"v1","addr":"172.21.15.175:6823","nonce":91262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":91262},{"type":"v1","addr":"172.21.15.175:6821","nonce":91262}]},"public_addr":"172.21.15.175:6817/91262","cluster_addr":"172.21.15.175:6819/91262","heartbeat_back_addr":"172.21.15.175:6823/91262","heartbeat_front_addr":"172.21.15.175:6821/91262","state":["exists","up"]},{"osd":6,"uuid":"d867cd89-3430-40ca-862f-d2bea3de9d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":91263},{"type":"v1","addr":"172.21.15.175:6801","nonce":91263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6805","nonce":91263},{"type":"v1","addr":"172.21.15.175:6807","nonce":91263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6813","nonce":91263},{"type":"v1","addr":"172.21.15.175:6815","nonce":91263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":91263},{"type":"v1","addr":"172.21.15.175:6811","nonce":91263}]},"public_addr":"172.21.15.175:6801/91263","cluster_addr":"172.21.15.175:6807/91263","heartbeat_back_addr":"172.21.15.175:6815/91263","heartbeat_front_addr":"172.21.15.175:6811/91263","state":["exists","up"]},{"osd":7,"uuid":"930cd464-79f1-47cb-b4ff-188ccde74fff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":91306},{"type":"v1","addr":"172.21.15.175:6825","nonce":91306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":91306},{"type":"v1","addr":"172.21.15.175:6827","nonce":91306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":91306},{"type":"v1","addr":"172.21.15.175:6831","nonce":91306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":91306},{"type":"v1","addr":"172.21.15.175:6829","nonce":91306}]},"public_addr":"172.21.15.175:6825/91306","cluster_addr":"172.21.15.175:6827/91306","heartbeat_back_addr":"172.21.15.175:6831/91306","heartbeat_front_addr":"172.21.15.175:6829/91306","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.290673+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.056390+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.292903+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.275861+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.332095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.327453+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.317585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.309330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:57:45.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.581+0000 7f083820b700 1 -- 172.21.15.157:0/4194888432 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0814066f30 msgr2=0x7f08140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:45.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.581+0000 7f083820b700 1 --2- 172.21.15.157:0/4194888432 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0814066f30 0x7f08140693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f082c015000 tx=0x7f082c013000 comp rx=0 tx=0).stop 2023-10-06T03:57:45.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.581+0000 7f083820b700 1 -- 172.21.15.157:0/4194888432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08300c2110 msgr2=0x7f083014d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:45.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.581+0000 7f083820b700 1 --2- 172.21.15.157:0/4194888432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08300c2110 0x7f083014d8b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f08300c8270 tx=0x7f08280091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:45.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.581+0000 7f083820b700 1 -- 172.21.15.157:0/4194888432 shutdown_connections 2023-10-06T03:57:45.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.581+0000 7f083820b700 1 --2- 172.21.15.157:0/4194888432 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0814066f30 0x7f08140693e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:45.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.581+0000 7f083820b700 1 --2- 172.21.15.157:0/4194888432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08300c2110 0x7f083014d8b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:45.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.581+0000 7f083820b700 1 --2- 172.21.15.157:0/4194888432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f081c003bf0 0x7f0830156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:45.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.581+0000 7f083820b700 1 --2- 172.21.15.157:0/4194888432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08300c7030 0x7f083014d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:45.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.581+0000 7f083820b700 1 -- 172.21.15.157:0/4194888432 >> 172.21.15.157:0/4194888432 conn(0x7f08300c8d70 msgr2=0x7f08300c4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:45.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.581+0000 7f083820b700 1 -- 172.21.15.157:0/4194888432 shutdown_connections 2023-10-06T03:57:45.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.581+0000 7f083820b700 1 -- 172.21.15.157:0/4194888432 wait complete. 2023-10-06T03:57:45.594 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:57:03.455121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:57:03.905661+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:57:08.377186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:57:09.709582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:57:45.595 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:57:45.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.714+0000 7f8f1d3f7700 1 Processor -- start 2023-10-06T03:57:45.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.714+0000 7f8f1d3f7700 1 -- start start 2023-10-06T03:57:45.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.715+0000 7f8f1d3f7700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8f1806cbb0 0x7f8f1806d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:45.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.715+0000 7f8f1d3f7700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8f1806c1f0 0x7f8f1806da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:45.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.715+0000 7f8f1d3f7700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f1806cf80 0x7f8f1814fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:45.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.715+0000 7f8f1d3f7700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f180c8c40 con 0x7f8f1806c1f0 2023-10-06T03:57:45.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.715+0000 7f8f1d3f7700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f180c9780 con 0x7f8f1806cbb0 2023-10-06T03:57:45.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.715+0000 7f8f1d3f7700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1806e0c0 con 0x7f8f1806cf80 2023-10-06T03:57:45.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.715+0000 7f8f167fc700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8f1806c1f0 0x7f8f1806da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:45.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.715+0000 7f8f167fc700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8f1806c1f0 0x7f8f1806da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45048/0 (socket says 172.21.15.157:45048) 2023-10-06T03:57:45.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.715+0000 7f8f167fc700 1 -- 172.21.15.157:0/212027233 learned_addr learned my addr 172.21.15.157:0/212027233 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:45.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.715+0000 7f8f177fe700 1 --2- 172.21.15.157:0/212027233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f1806cf80 0x7f8f1814fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:45.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.715+0000 7f8f16ffd700 1 --2- 172.21.15.157:0/212027233 >> v2:172.21.15.175:3300/0 conn(0x7f8f1806cbb0 0x7f8f1806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:45.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.715+0000 7f8f167fc700 1 -- 172.21.15.157:0/212027233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f1806cf80 msgr2=0x7f8f1814fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:45.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.715+0000 7f8f167fc700 1 --2- 172.21.15.157:0/212027233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f1806cf80 0x7f8f1814fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:45.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.715+0000 7f8f167fc700 1 -- 172.21.15.157:0/212027233 >> v2:172.21.15.175:3300/0 conn(0x7f8f1806cbb0 msgr2=0x7f8f1806d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:45.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.715+0000 7f8f167fc700 1 --2- 172.21.15.157:0/212027233 >> v2:172.21.15.175:3300/0 conn(0x7f8f1806cbb0 0x7f8f1806d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:45.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.716+0000 7f8f167fc700 1 -- 172.21.15.157:0/212027233 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f180c8e50 con 0x7f8f1806c1f0 2023-10-06T03:57:45.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.716+0000 7f8f16ffd700 1 --2- 172.21.15.157:0/212027233 >> v2:172.21.15.175:3300/0 conn(0x7f8f1806cbb0 0x7f8f1806d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:57:45.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.716+0000 7f8f167fc700 1 --2- 172.21.15.157:0/212027233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1806c1f0 0x7f8f1806da70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8f1806fd70 tx=0x7f8f0c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f15bf639d5cc0b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:45.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.716+0000 7f8f15ffb700 1 -- 172.21.15.157:0/212027233 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c00b8f0 con 0x7f8f1806c1f0 2023-10-06T03:57:45.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.716+0000 7f8f15ffb700 1 -- 172.21.15.157:0/212027233 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f0c00ba50 con 0x7f8f1806c1f0 2023-10-06T03:57:45.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.716+0000 7f8f15ffb700 1 -- 172.21.15.157:0/212027233 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c00bdf0 con 0x7f8f1806c1f0 2023-10-06T03:57:45.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.716+0000 7f8f1d3f7700 1 -- 172.21.15.157:0/212027233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1806c1f0 msgr2=0x7f8f1806da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:45.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.716+0000 7f8f1d3f7700 1 --2- 172.21.15.157:0/212027233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1806c1f0 0x7f8f1806da70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8f1806fd70 tx=0x7f8f0c009250 comp rx=0 tx=0).stop 2023-10-06T03:57:45.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.716+0000 7f8f1d3f7700 1 -- 172.21.15.157:0/212027233 shutdown_connections 2023-10-06T03:57:45.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.716+0000 7f8f1d3f7700 1 --2- 172.21.15.157:0/212027233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f1806cf80 0x7f8f1814fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:45.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.716+0000 7f8f1d3f7700 1 --2- 172.21.15.157:0/212027233 >> v2:172.21.15.175:3300/0 conn(0x7f8f1806cbb0 0x7f8f1806d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:45.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.716+0000 7f8f1d3f7700 1 --2- 172.21.15.157:0/212027233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1806c1f0 0x7f8f1806da70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:45.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.716+0000 7f8f1d3f7700 1 -- 172.21.15.157:0/212027233 >> 172.21.15.157:0/212027233 conn(0x7f8f1806f230 msgr2=0x7f8f1806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:45.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.716+0000 7f8f1d3f7700 1 -- 172.21.15.157:0/212027233 shutdown_connections 2023-10-06T03:57:45.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.716+0000 7f8f1d3f7700 1 -- 172.21.15.157:0/212027233 wait complete. 2023-10-06T03:57:45.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.718+0000 7f8f1d3f7700 1 Processor -- start 2023-10-06T03:57:45.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.718+0000 7f8f1d3f7700 1 -- start start 2023-10-06T03:57:45.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.718+0000 7f8f1d3f7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1806c1f0 0x7f8f181467b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:45.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.718+0000 7f8f1d3f7700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f1806cbb0 0x7f8f180c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:45.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.718+0000 7f8f1d3f7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f1806cf80 0x7f8f180c62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:45.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.718+0000 7f8f1d3f7700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f180c7530 con 0x7f8f1806c1f0 2023-10-06T03:57:45.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.718+0000 7f8f1d3f7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f181566d0 con 0x7f8f1806cf80 2023-10-06T03:57:45.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.718+0000 7f8f1d3f7700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f18156a70 con 0x7f8f1806cbb0 2023-10-06T03:57:45.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.718+0000 7f8f16ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1806c1f0 0x7f8f181467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:45.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.718+0000 7f8f16ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1806c1f0 0x7f8f181467b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45058/0 (socket says 172.21.15.157:45058) 2023-10-06T03:57:45.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.718+0000 7f8f16ffd700 1 -- 172.21.15.157:0/320046274 learned_addr learned my addr 172.21.15.157:0/320046274 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:45.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.719+0000 7f8f167fc700 1 --2- 172.21.15.157:0/320046274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f1806cbb0 0x7f8f180c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:45.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.719+0000 7f8f177fe700 1 --2- 172.21.15.157:0/320046274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f1806cf80 0x7f8f180c62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:45.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.719+0000 7f8f16ffd700 1 -- 172.21.15.157:0/320046274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f1806cbb0 msgr2=0x7f8f180c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:45.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.719+0000 7f8f16ffd700 1 --2- 172.21.15.157:0/320046274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f1806cbb0 0x7f8f180c5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:45.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.719+0000 7f8f16ffd700 1 -- 172.21.15.157:0/320046274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f1806cf80 msgr2=0x7f8f180c62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:45.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.719+0000 7f8f16ffd700 1 --2- 172.21.15.157:0/320046274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f1806cf80 0x7f8f180c62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:45.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.719+0000 7f8f16ffd700 1 -- 172.21.15.157:0/320046274 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0c0094f0 con 0x7f8f1806c1f0 2023-10-06T03:57:45.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.719+0000 7f8f177fe700 1 --2- 172.21.15.157:0/320046274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f1806cf80 0x7f8f180c62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:45.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.719+0000 7f8f16ffd700 1 --2- 172.21.15.157:0/320046274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1806c1f0 0x7f8f181467b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8f0800a400 tx=0x7f8f08007910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:45.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.719+0000 7f8efeffd700 1 -- 172.21.15.157:0/320046274 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0800ae20 con 0x7f8f1806c1f0 2023-10-06T03:57:45.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.719+0000 7f8efeffd700 1 -- 172.21.15.157:0/320046274 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f0800d650 con 0x7f8f1806c1f0 2023-10-06T03:57:45.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.719+0000 7f8f1d3f7700 1 -- 172.21.15.157:0/320046274 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f18156080 con 0x7f8f1806c1f0 2023-10-06T03:57:45.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.719+0000 7f8efeffd700 1 -- 172.21.15.157:0/320046274 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0800d9f0 con 0x7f8f1806c1f0 2023-10-06T03:57:45.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.719+0000 7f8f1d3f7700 1 -- 172.21.15.157:0/320046274 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f18145d80 con 0x7f8f1806c1f0 2023-10-06T03:57:45.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.720+0000 7f8f1d3f7700 1 -- 172.21.15.157:0/320046274 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f18089ca0 con 0x7f8f1806c1f0 2023-10-06T03:57:45.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.721+0000 7f8efeffd700 1 -- 172.21.15.157:0/320046274 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8f08015440 con 0x7f8f1806c1f0 2023-10-06T03:57:45.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.721+0000 7f8efeffd700 1 --2- 172.21.15.157:0/320046274 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8f04066f30 0x7f8f040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:45.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.721+0000 7f8efeffd700 1 -- 172.21.15.157:0/320046274 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8f08084e70 con 0x7f8f1806c1f0 2023-10-06T03:57:45.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.721+0000 7f8f167fc700 1 --2- 172.21.15.157:0/320046274 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8f04066f30 0x7f8f040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:45.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.722+0000 7f8f167fc700 1 --2- 172.21.15.157:0/320046274 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8f04066f30 0x7f8f040693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8f1806fd70 tx=0x7f8f0c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:45.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.725+0000 7f8efeffd700 1 -- 172.21.15.157:0/320046274 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8f0804e5a0 con 0x7f8f1806c1f0 2023-10-06T03:57:45.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.908+0000 7f8f1d3f7700 1 -- 172.21.15.157:0/320046274 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8f180c9130 con 0x7f8f1806c1f0 2023-10-06T03:57:45.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.909+0000 7f8efeffd700 1 -- 172.21.15.157:0/320046274 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8f0801a0a0 con 0x7f8f1806c1f0 2023-10-06T03:57:45.910 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-10-06T03:57:45.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.910+0000 7f8f1d3f7700 1 -- 172.21.15.157:0/320046274 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8f04066f30 msgr2=0x7f8f040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:45.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.910+0000 7f8f1d3f7700 1 --2- 172.21.15.157:0/320046274 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8f04066f30 0x7f8f040693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8f1806fd70 tx=0x7f8f0c017000 comp rx=0 tx=0).stop 2023-10-06T03:57:45.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.910+0000 7f8f1d3f7700 1 -- 172.21.15.157:0/320046274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1806c1f0 msgr2=0x7f8f181467b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:45.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.910+0000 7f8f1d3f7700 1 --2- 172.21.15.157:0/320046274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1806c1f0 0x7f8f181467b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8f0800a400 tx=0x7f8f08007910 comp rx=0 tx=0).stop 2023-10-06T03:57:45.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.910+0000 7f8f1d3f7700 1 -- 172.21.15.157:0/320046274 shutdown_connections 2023-10-06T03:57:45.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.910+0000 7f8f1d3f7700 1 --2- 172.21.15.157:0/320046274 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8f04066f30 0x7f8f040693e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:45.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.910+0000 7f8f1d3f7700 1 --2- 172.21.15.157:0/320046274 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f1806cf80 0x7f8f180c62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:45.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.910+0000 7f8f1d3f7700 1 --2- 172.21.15.157:0/320046274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f1806c1f0 0x7f8f181467b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:45.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.910+0000 7f8f1d3f7700 1 --2- 172.21.15.157:0/320046274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f1806cbb0 0x7f8f180c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:45.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.910+0000 7f8f1d3f7700 1 -- 172.21.15.157:0/320046274 >> 172.21.15.157:0/320046274 conn(0x7f8f1806f230 msgr2=0x7f8f180c6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:45.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.911+0000 7f8f1d3f7700 1 -- 172.21.15.157:0/320046274 shutdown_connections 2023-10-06T03:57:45.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:45.911+0000 7f8f1d3f7700 1 -- 172.21.15.157:0/320046274 wait complete. 2023-10-06T03:57:45.924 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:57:46.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.039+0000 7f9a2360f700 1 Processor -- start 2023-10-06T03:57:46.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.040+0000 7f9a2360f700 1 -- start start 2023-10-06T03:57:46.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.040+0000 7f9a2360f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9a1c0c00c0 0x7f9a1c0c29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:46.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.040+0000 7f9a2360f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a1c0631f0 0x7f9a1c1540e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:46.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.040+0000 7f9a2360f700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a1c153be0 con 0x7f9a1c0c0490 2023-10-06T03:57:46.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.040+0000 7f9a2360f700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a1c0bf550 con 0x7f9a1c0c00c0 2023-10-06T03:57:46.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.040+0000 7f9a2360f700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c0c3630 con 0x7f9a1c0631f0 2023-10-06T03:57:46.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.041+0000 7f9a20baa700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a1c0631f0 0x7f9a1c1540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:46.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.041+0000 7f9a213ab700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9a1c0c00c0 0x7f9a1c0c29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:46.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.041+0000 7f9a20baa700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a1c0631f0 0x7f9a1c1540e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:53608/0 (socket says 172.21.15.157:53608) 2023-10-06T03:57:46.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.041+0000 7f9a21bac700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f9a1c0c0490 0x7f9a1c14d650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.157:48158/0 (socket says 172.21.15.157:48158) 2023-10-06T03:57:46.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.041+0000 7f9a20baa700 1 -- 172.21.15.157:0/2065785840 learned_addr learned my addr 172.21.15.157:0/2065785840 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:46.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.041+0000 7f9a20baa700 1 -- 172.21.15.157:0/2065785840 >> v2:172.21.15.175:3300/0 conn(0x7f9a1c0c00c0 msgr2=0x7f9a1c0c29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:46.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.041+0000 7f9a20baa700 1 --2- 172.21.15.157:0/2065785840 >> v2:172.21.15.175:3300/0 conn(0x7f9a1c0c00c0 0x7f9a1c0c29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:46.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.041+0000 7f9a20baa700 1 -- 172.21.15.157:0/2065785840 >> v1:172.21.15.175:6789/0 conn(0x7f9a1c0c0490 legacy=0x7f9a1c14d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:46.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.041+0000 7f9a20baa700 1 -- 172.21.15.157:0/2065785840 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1c0c3910 con 0x7f9a1c0631f0 2023-10-06T03:57:46.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.041+0000 7f9a20baa700 1 --2- 172.21.15.157:0/2065785840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a1c0631f0 0x7f9a1c1540e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0099b0 tx=0x7f9a0c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=410ec07ddac5261b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:46.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.041+0000 7f9a13fff700 1 -- 172.21.15.157:0/2065785840 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c009ea0 con 0x7f9a1c0631f0 2023-10-06T03:57:46.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.041+0000 7f9a13fff700 1 -- 172.21.15.157:0/2065785840 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0c00b860 con 0x7f9a1c0631f0 2023-10-06T03:57:46.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.041+0000 7f9a13fff700 1 -- 172.21.15.157:0/2065785840 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c00bc00 con 0x7f9a1c0631f0 2023-10-06T03:57:46.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.042+0000 7f9a2360f700 1 -- 172.21.15.157:0/2065785840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a1c0631f0 msgr2=0x7f9a1c1540e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:46.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.042+0000 7f9a2360f700 1 --2- 172.21.15.157:0/2065785840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a1c0631f0 0x7f9a1c1540e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0099b0 tx=0x7f9a0c009250 comp rx=0 tx=0).stop 2023-10-06T03:57:46.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.042+0000 7f9a2360f700 1 -- 172.21.15.157:0/2065785840 shutdown_connections 2023-10-06T03:57:46.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.042+0000 7f9a2360f700 1 --2- 172.21.15.157:0/2065785840 >> v2:172.21.15.175:3300/0 conn(0x7f9a1c0c00c0 0x7f9a1c0c29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:46.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.042+0000 7f9a2360f700 1 --2- 172.21.15.157:0/2065785840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a1c0631f0 0x7f9a1c1540e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:46.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.042+0000 7f9a2360f700 1 -- 172.21.15.157:0/2065785840 >> 172.21.15.157:0/2065785840 conn(0x7f9a1c0c8b20 msgr2=0x7f9a1c0c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:46.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.042+0000 7f9a2360f700 1 -- 172.21.15.157:0/2065785840 shutdown_connections 2023-10-06T03:57:46.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.042+0000 7f9a2360f700 1 -- 172.21.15.157:0/2065785840 wait complete. 2023-10-06T03:57:46.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.043+0000 7f9a2360f700 1 Processor -- start 2023-10-06T03:57:46.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.043+0000 7f9a2360f700 1 -- start start 2023-10-06T03:57:46.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.044+0000 7f9a2360f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a1c0631f0 0x7f9a1c0c40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:46.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.044+0000 7f9a2360f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a1c0c00c0 0x7f9a1c157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:46.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.044+0000 7f9a2360f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a1c0c0490 0x7f9a1c1425a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:46.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.044+0000 7f9a2360f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c0c7510 con 0x7f9a1c0c00c0 2023-10-06T03:57:46.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.044+0000 7f9a2360f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c1575c0 con 0x7f9a1c0c0490 2023-10-06T03:57:46.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.044+0000 7f9a2360f700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c166f20 con 0x7f9a1c0631f0 2023-10-06T03:57:46.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.044+0000 7f9a20baa700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a1c0c00c0 0x7f9a1c157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:46.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.044+0000 7f9a20baa700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a1c0c00c0 0x7f9a1c157a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45060/0 (socket says 172.21.15.157:45060) 2023-10-06T03:57:46.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.044+0000 7f9a20baa700 1 -- 172.21.15.157:0/2045094631 learned_addr learned my addr 172.21.15.157:0/2045094631 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:46.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.044+0000 7f9a213ab700 1 --2- 172.21.15.157:0/2045094631 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a1c0631f0 0x7f9a1c0c40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:46.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.044+0000 7f9a21bac700 1 --2- 172.21.15.157:0/2045094631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a1c0c0490 0x7f9a1c1425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:46.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.044+0000 7f9a20baa700 1 -- 172.21.15.157:0/2045094631 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a1c0631f0 msgr2=0x7f9a1c0c40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:46.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.044+0000 7f9a20baa700 1 --2- 172.21.15.157:0/2045094631 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a1c0631f0 0x7f9a1c0c40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:46.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.044+0000 7f9a20baa700 1 -- 172.21.15.157:0/2045094631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a1c0c0490 msgr2=0x7f9a1c1425a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:46.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.044+0000 7f9a20baa700 1 --2- 172.21.15.157:0/2045094631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a1c0c0490 0x7f9a1c1425a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:46.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.044+0000 7f9a20baa700 1 -- 172.21.15.157:0/2045094631 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c0094f0 con 0x7f9a1c0c00c0 2023-10-06T03:57:46.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.045+0000 7f9a20baa700 1 --2- 172.21.15.157:0/2045094631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a1c0c00c0 0x7f9a1c157a50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0bf690 tx=0x7f9a0c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:46.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.045+0000 7f9a117fa700 1 -- 172.21.15.157:0/2045094631 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c000d10 con 0x7f9a1c0c00c0 2023-10-06T03:57:46.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.045+0000 7f9a117fa700 1 -- 172.21.15.157:0/2045094631 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0c00bbe0 con 0x7f9a1c0c00c0 2023-10-06T03:57:46.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.045+0000 7f9a2360f700 1 -- 172.21.15.157:0/2045094631 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a1c0c8450 con 0x7f9a1c0c00c0 2023-10-06T03:57:46.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.045+0000 7f9a117fa700 1 -- 172.21.15.157:0/2045094631 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c011670 con 0x7f9a1c0c00c0 2023-10-06T03:57:46.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.045+0000 7f9a2360f700 1 -- 172.21.15.157:0/2045094631 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1c159d40 con 0x7f9a1c0c00c0 2023-10-06T03:57:46.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.046+0000 7f9a2360f700 1 -- 172.21.15.157:0/2045094631 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a1c159ba0 con 0x7f9a1c0c00c0 2023-10-06T03:57:46.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.046+0000 7f9a117fa700 1 -- 172.21.15.157:0/2045094631 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9a0c011aa0 con 0x7f9a1c0c00c0 2023-10-06T03:57:46.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.046+0000 7f9a117fa700 1 --2- 172.21.15.157:0/2045094631 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9a00066f30 0x7f9a000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:46.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.046+0000 7f9a117fa700 1 -- 172.21.15.157:0/2045094631 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9a0c085e30 con 0x7f9a1c0c00c0 2023-10-06T03:57:46.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.047+0000 7f9a213ab700 1 --2- 172.21.15.157:0/2045094631 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9a00066f30 0x7f9a000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:46.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.047+0000 7f9a213ab700 1 --2- 172.21.15.157:0/2045094631 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9a00066f30 0x7f9a000693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800f570 tx=0x7f9a1800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:46.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.051+0000 7f9a117fa700 1 -- 172.21.15.157:0/2045094631 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a1c159ba0 con 0x7f9a1c0c00c0 2023-10-06T03:57:46.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.234+0000 7f9a2360f700 1 -- 172.21.15.157:0/2045094631 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9a1c157770 con 0x7f9a1c0c00c0 2023-10-06T03:57:46.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.234+0000 7f9a117fa700 1 -- 172.21.15.157:0/2045094631 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9a1c157770 con 0x7f9a1c0c00c0 2023-10-06T03:57:46.236 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-10-06T03:57:46.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.236+0000 7f9a2360f700 1 -- 172.21.15.157:0/2045094631 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9a00066f30 msgr2=0x7f9a000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:46.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.236+0000 7f9a2360f700 1 --2- 172.21.15.157:0/2045094631 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9a00066f30 0x7f9a000693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800f570 tx=0x7f9a1800b040 comp rx=0 tx=0).stop 2023-10-06T03:57:46.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.236+0000 7f9a2360f700 1 -- 172.21.15.157:0/2045094631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a1c0c00c0 msgr2=0x7f9a1c157a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:46.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.236+0000 7f9a2360f700 1 --2- 172.21.15.157:0/2045094631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a1c0c00c0 0x7f9a1c157a50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0bf690 tx=0x7f9a0c017000 comp rx=0 tx=0).stop 2023-10-06T03:57:46.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.236+0000 7f9a2360f700 1 -- 172.21.15.157:0/2045094631 shutdown_connections 2023-10-06T03:57:46.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.236+0000 7f9a2360f700 1 --2- 172.21.15.157:0/2045094631 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9a00066f30 0x7f9a000693e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:46.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.236+0000 7f9a2360f700 1 --2- 172.21.15.157:0/2045094631 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a1c0c0490 0x7f9a1c1425a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:46.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.236+0000 7f9a2360f700 1 --2- 172.21.15.157:0/2045094631 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a1c0631f0 0x7f9a1c0c40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:46.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.236+0000 7f9a2360f700 1 --2- 172.21.15.157:0/2045094631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a1c0c00c0 0x7f9a1c157a50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:46.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.236+0000 7f9a2360f700 1 -- 172.21.15.157:0/2045094631 >> 172.21.15.157:0/2045094631 conn(0x7f9a1c0c8b20 msgr2=0x7f9a1c0c8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:46.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.236+0000 7f9a2360f700 1 -- 172.21.15.157:0/2045094631 shutdown_connections 2023-10-06T03:57:46.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.236+0000 7f9a2360f700 1 -- 172.21.15.157:0/2045094631 wait complete. 2023-10-06T03:57:46.250 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:57:46.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.366+0000 7fdd2ec6e700 1 Processor -- start 2023-10-06T03:57:46.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.366+0000 7fdd2ec6e700 1 -- start start 2023-10-06T03:57:46.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.367+0000 7fdd2ec6e700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdd2806cbb0 0x7fdd2806d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:46.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.367+0000 7fdd2ec6e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdd28150410 0x7fdd2806da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:46.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.367+0000 7fdd2ec6e700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd280c9780 con 0x7fdd2806c1f0 2023-10-06T03:57:46.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.367+0000 7fdd2ec6e700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd280c8c40 con 0x7fdd2806cbb0 2023-10-06T03:57:46.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.367+0000 7fdd2ec6e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd2806e0c0 con 0x7fdd28150410 2023-10-06T03:57:46.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.367+0000 7fdd1ffff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fdd2806c1f0 0x7fdd2814fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40760/0 (socket says 172.21.15.157:40760) 2023-10-06T03:57:46.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.367+0000 7fdd2ca0a700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdd2806cbb0 0x7fdd2806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:46.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.367+0000 7fdd1ffff700 1 -- 172.21.15.157:0/2628635276 learned_addr learned my addr 172.21.15.157:0/2628635276 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:46.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.367+0000 7fdd2d20b700 1 --2- 172.21.15.157:0/2628635276 >> v2:172.21.15.175:3300/0 conn(0x7fdd28150410 0x7fdd2806da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:46.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.368+0000 7fdd2ca0a700 1 -- 172.21.15.157:0/2628635276 >> v2:172.21.15.175:3300/0 conn(0x7fdd28150410 msgr2=0x7fdd2806da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:46.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.368+0000 7fdd2ca0a700 1 --2- 172.21.15.157:0/2628635276 >> v2:172.21.15.175:3300/0 conn(0x7fdd28150410 0x7fdd2806da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:46.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.368+0000 7fdd2ca0a700 1 -- 172.21.15.157:0/2628635276 >> v1:172.21.15.157:6789/0 conn(0x7fdd2806c1f0 legacy=0x7fdd2814fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:46.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.368+0000 7fdd2ca0a700 1 -- 172.21.15.157:0/2628635276 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd280c8e50 con 0x7fdd2806cbb0 2023-10-06T03:57:46.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.368+0000 7fdd1f7fe700 1 -- 172.21.15.157:0/2628635276 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 95430543 0 0) 0x7fdd280c9780 con 0x7fdd2806c1f0 2023-10-06T03:57:46.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.368+0000 7fdd2ca0a700 1 --2- 172.21.15.157:0/2628635276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd2806cbb0 0x7fdd2806d530 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdd280b1b50 tx=0x7fdd24009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c61f7291dcbaee45 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:46.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.368+0000 7fdd1f7fe700 1 -- 172.21.15.157:0/2628635276 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2400e070 con 0x7fdd2806cbb0 2023-10-06T03:57:46.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.368+0000 7fdd1f7fe700 1 -- 172.21.15.157:0/2628635276 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd2400bcc0 con 0x7fdd2806cbb0 2023-10-06T03:57:46.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.368+0000 7fdd1f7fe700 1 -- 172.21.15.157:0/2628635276 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd24012680 con 0x7fdd2806cbb0 2023-10-06T03:57:46.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.368+0000 7fdd2ec6e700 1 -- 172.21.15.157:0/2628635276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd2806cbb0 msgr2=0x7fdd2806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:46.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.368+0000 7fdd2ec6e700 1 --2- 172.21.15.157:0/2628635276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd2806cbb0 0x7fdd2806d530 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdd280b1b50 tx=0x7fdd24009780 comp rx=0 tx=0).stop 2023-10-06T03:57:46.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.368+0000 7fdd2ec6e700 1 -- 172.21.15.157:0/2628635276 shutdown_connections 2023-10-06T03:57:46.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.368+0000 7fdd2ec6e700 1 --2- 172.21.15.157:0/2628635276 >> v2:172.21.15.175:3300/0 conn(0x7fdd28150410 0x7fdd2806da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:46.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.368+0000 7fdd2ec6e700 1 --2- 172.21.15.157:0/2628635276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd2806cbb0 0x7fdd2806d530 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:46.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.368+0000 7fdd2ec6e700 1 -- 172.21.15.157:0/2628635276 >> 172.21.15.157:0/2628635276 conn(0x7fdd2806f230 msgr2=0x7fdd2806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:46.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.368+0000 7fdd2ec6e700 1 -- 172.21.15.157:0/2628635276 shutdown_connections 2023-10-06T03:57:46.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.368+0000 7fdd2ec6e700 1 -- 172.21.15.157:0/2628635276 wait complete. 2023-10-06T03:57:46.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.369+0000 7fdd2ec6e700 1 Processor -- start 2023-10-06T03:57:46.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.370+0000 7fdd2ec6e700 1 -- start start 2023-10-06T03:57:46.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.370+0000 7fdd2ec6e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd2806c1f0 0x7fdd28166880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:46.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.370+0000 7fdd2ec6e700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd2806cbb0 0x7fdd28166dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:46.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.370+0000 7fdd2ec6e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd28150410 0x7fdd28144980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:46.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.370+0000 7fdd2ec6e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd28147e60 con 0x7fdd2806c1f0 2023-10-06T03:57:46.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.370+0000 7fdd2ec6e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd28145a30 con 0x7fdd28150410 2023-10-06T03:57:46.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.370+0000 7fdd2ec6e700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd281673b0 con 0x7fdd2806cbb0 2023-10-06T03:57:46.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.370+0000 7fdd2ca0a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd2806c1f0 0x7fdd28166880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:46.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.370+0000 7fdd2ca0a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd2806c1f0 0x7fdd28166880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45078/0 (socket says 172.21.15.157:45078) 2023-10-06T03:57:46.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.370+0000 7fdd2ca0a700 1 -- 172.21.15.157:0/359030082 learned_addr learned my addr 172.21.15.157:0/359030082 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:46.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.371+0000 7fdd1ffff700 1 --2- 172.21.15.157:0/359030082 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd2806cbb0 0x7fdd28166dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:46.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.371+0000 7fdd2d20b700 1 --2- 172.21.15.157:0/359030082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd28150410 0x7fdd28144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:46.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.371+0000 7fdd2ca0a700 1 -- 172.21.15.157:0/359030082 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd2806cbb0 msgr2=0x7fdd28166dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:46.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.371+0000 7fdd2ca0a700 1 --2- 172.21.15.157:0/359030082 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd2806cbb0 0x7fdd28166dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:46.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.371+0000 7fdd2ca0a700 1 -- 172.21.15.157:0/359030082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd28150410 msgr2=0x7fdd28144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:46.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.371+0000 7fdd2ca0a700 1 --2- 172.21.15.157:0/359030082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd28150410 0x7fdd28144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:46.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.371+0000 7fdd2ca0a700 1 -- 172.21.15.157:0/359030082 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd24009a20 con 0x7fdd2806c1f0 2023-10-06T03:57:46.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.371+0000 7fdd1ffff700 1 --2- 172.21.15.157:0/359030082 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd2806cbb0 0x7fdd28166dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:57:46.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.371+0000 7fdd2ca0a700 1 --2- 172.21.15.157:0/359030082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd2806c1f0 0x7fdd28166880 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdd280c3960 tx=0x7fdd24019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:46.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.371+0000 7fdd1cff9700 1 -- 172.21.15.157:0/359030082 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd24012e20 con 0x7fdd2806c1f0 2023-10-06T03:57:46.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.371+0000 7fdd1cff9700 1 -- 172.21.15.157:0/359030082 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd24027470 con 0x7fdd2806c1f0 2023-10-06T03:57:46.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.371+0000 7fdd2ec6e700 1 -- 172.21.15.157:0/359030082 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd28148070 con 0x7fdd2806c1f0 2023-10-06T03:57:46.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.371+0000 7fdd1cff9700 1 -- 172.21.15.157:0/359030082 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd24027830 con 0x7fdd2806c1f0 2023-10-06T03:57:46.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.371+0000 7fdd2ec6e700 1 -- 172.21.15.157:0/359030082 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd2814ea80 con 0x7fdd2806c1f0 2023-10-06T03:57:46.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.372+0000 7fdd2ec6e700 1 -- 172.21.15.157:0/359030082 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd28148510 con 0x7fdd2806c1f0 2023-10-06T03:57:46.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.372+0000 7fdd1cff9700 1 -- 172.21.15.157:0/359030082 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdd24024070 con 0x7fdd2806c1f0 2023-10-06T03:57:46.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.373+0000 7fdd1cff9700 1 --2- 172.21.15.157:0/359030082 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdd14066f60 0x7fdd14069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:46.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.373+0000 7fdd1cff9700 1 -- 172.21.15.157:0/359030082 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdd24087130 con 0x7fdd2806c1f0 2023-10-06T03:57:46.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.373+0000 7fdd1ffff700 1 --2- 172.21.15.157:0/359030082 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdd14066f60 0x7fdd14069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:46.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.373+0000 7fdd1ffff700 1 --2- 172.21.15.157:0/359030082 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdd14066f60 0x7fdd14069410 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdd280c7990 tx=0x7fdd18007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:46.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.377+0000 7fdd1cff9700 1 -- 172.21.15.157:0/359030082 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd28148510 con 0x7fdd2806c1f0 2023-10-06T03:57:46.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.561+0000 7fdd2ec6e700 1 -- 172.21.15.157:0/359030082 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd28148510 con 0x7fdd2806c1f0 2023-10-06T03:57:46.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.561+0000 7fdd1cff9700 1 -- 172.21.15.157:0/359030082 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdd28148510 con 0x7fdd2806c1f0 2023-10-06T03:57:46.563 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-10-06T03:57:46.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.563+0000 7fdd2ec6e700 1 -- 172.21.15.157:0/359030082 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdd14066f60 msgr2=0x7fdd14069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:46.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.563+0000 7fdd2ec6e700 1 --2- 172.21.15.157:0/359030082 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdd14066f60 0x7fdd14069410 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdd280c7990 tx=0x7fdd18007d30 comp rx=0 tx=0).stop 2023-10-06T03:57:46.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.563+0000 7fdd2ec6e700 1 -- 172.21.15.157:0/359030082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd2806c1f0 msgr2=0x7fdd28166880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:46.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.563+0000 7fdd2ec6e700 1 --2- 172.21.15.157:0/359030082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd2806c1f0 0x7fdd28166880 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdd280c3960 tx=0x7fdd24019000 comp rx=0 tx=0).stop 2023-10-06T03:57:46.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.563+0000 7fdd2ec6e700 1 -- 172.21.15.157:0/359030082 shutdown_connections 2023-10-06T03:57:46.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.563+0000 7fdd2ec6e700 1 --2- 172.21.15.157:0/359030082 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdd14066f60 0x7fdd14069410 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:46.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.563+0000 7fdd2ec6e700 1 --2- 172.21.15.157:0/359030082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd28150410 0x7fdd28144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:46.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.563+0000 7fdd2ec6e700 1 --2- 172.21.15.157:0/359030082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd2806c1f0 0x7fdd28166880 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:46.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.563+0000 7fdd2ec6e700 1 --2- 172.21.15.157:0/359030082 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd2806cbb0 0x7fdd28166dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:46.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.563+0000 7fdd2ec6e700 1 -- 172.21.15.157:0/359030082 >> 172.21.15.157:0/359030082 conn(0x7fdd2806f230 msgr2=0x7fdd280cbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:46.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.563+0000 7fdd2ec6e700 1 -- 172.21.15.157:0/359030082 shutdown_connections 2023-10-06T03:57:46.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.563+0000 7fdd2ec6e700 1 -- 172.21.15.157:0/359030082 wait complete. 2023-10-06T03:57:46.577 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:57:46.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.695+0000 7f27ed19f700 1 Processor -- start 2023-10-06T03:57:46.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.696+0000 7f27ed19f700 1 -- start start 2023-10-06T03:57:46.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.696+0000 7f27ed19f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f27e80c6630 0x7f27e8152e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:46.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.696+0000 7f27ed19f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27e806a330 0x7f27e80c9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:46.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.696+0000 7f27ed19f700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27e806b460 con 0x7f27e80bfa80 2023-10-06T03:57:46.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.696+0000 7f27ed19f700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e8153350 con 0x7f27e80c6630 2023-10-06T03:57:46.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.696+0000 7f27ed19f700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e81508a0 con 0x7f27e806a330 2023-10-06T03:57:46.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.696+0000 7f27e77fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f27e80bfa80 0x7f27e806b8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40774/0 (socket says 172.21.15.157:40774) 2023-10-06T03:57:46.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.697+0000 7f27e77fe700 1 -- 172.21.15.157:0/1364791701 learned_addr learned my addr 172.21.15.157:0/1364791701 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:46.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.697+0000 7f27ec99e700 1 --2- 172.21.15.157:0/1364791701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27e806a330 0x7f27e80c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:46.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.697+0000 7f27e7fff700 1 --2- 172.21.15.157:0/1364791701 >> v2:172.21.15.175:3300/0 conn(0x7f27e80c6630 0x7f27e8152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:46.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.697+0000 7f27e6ffd700 1 -- 172.21.15.157:0/1364791701 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1940589817 0 0) 0x7f27e806b460 con 0x7f27e80bfa80 2023-10-06T03:57:46.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.697+0000 7f27e6ffd700 1 -- 172.21.15.157:0/1364791701 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27d0003570 con 0x7f27e80bfa80 2023-10-06T03:57:46.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.697+0000 7f27e6ffd700 1 -- 172.21.15.157:0/1364791701 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 342561778 0 0) 0x7f27d0003570 con 0x7f27e80bfa80 2023-10-06T03:57:46.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.697+0000 7f27e6ffd700 1 -- 172.21.15.157:0/1364791701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27e806a330 msgr2=0x7f27e80c9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:46.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.697+0000 7f27e6ffd700 1 --2- 172.21.15.157:0/1364791701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27e806a330 0x7f27e80c9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:46.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.697+0000 7f27e6ffd700 1 -- 172.21.15.157:0/1364791701 >> v2:172.21.15.175:3300/0 conn(0x7f27e80c6630 msgr2=0x7f27e8152e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:46.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.697+0000 7f27e6ffd700 1 --2- 172.21.15.157:0/1364791701 >> v2:172.21.15.175:3300/0 conn(0x7f27e80c6630 0x7f27e8152e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:46.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.697+0000 7f27e6ffd700 1 -- 172.21.15.157:0/1364791701 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e8153530 con 0x7f27e80bfa80 2023-10-06T03:57:46.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.697+0000 7f27e6ffd700 1 -- 172.21.15.157:0/1364791701 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f27dc001470 con 0x7f27e80bfa80 2023-10-06T03:57:46.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.697+0000 7f27e6ffd700 1 -- 172.21.15.157:0/1364791701 >> v1:172.21.15.157:6789/0 conn(0x7f27e80bfa80 legacy=0x7f27e806b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:46.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.697+0000 7f27e6ffd700 1 --2- 172.21.15.157:0/1364791701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27d0003a60 0x7f27d0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:46.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.697+0000 7f27e6ffd700 1 --2- 172.21.15.157:0/1364791701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d0004810 0x7f27d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:46.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.697+0000 7f27e6ffd700 1 --2- 172.21.15.157:0/1364791701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27e806a330 0x7f27d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:46.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.697+0000 7f27e6ffd700 1 -- 172.21.15.157:0/1364791701 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e8153350 con 0x7f27d0004810 2023-10-06T03:57:46.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.697+0000 7f27e6ffd700 1 -- 172.21.15.157:0/1364791701 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d000ba60 con 0x7f27d0003a60 2023-10-06T03:57:46.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.697+0000 7f27e6ffd700 1 -- 172.21.15.157:0/1364791701 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27d000bba0 con 0x7f27e806a330 2023-10-06T03:57:46.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.697+0000 7f27e77fe700 1 --2- 172.21.15.157:0/1364791701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d0004810 0x7f27d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:46.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.697+0000 7f27ec99e700 1 --2- 172.21.15.157:0/1364791701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27e806a330 0x7f27d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:46.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.697+0000 7f27e7fff700 1 --2- 172.21.15.157:0/1364791701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27d0003a60 0x7f27d0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:46.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.697+0000 7f27e77fe700 1 -- 172.21.15.157:0/1364791701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27e806a330 msgr2=0x7f27d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:46.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.697+0000 7f27e77fe700 1 --2- 172.21.15.157:0/1364791701 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27e806a330 0x7f27d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:46.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.697+0000 7f27e77fe700 1 -- 172.21.15.157:0/1364791701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27d0003a60 msgr2=0x7f27d0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:46.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.698+0000 7f27e77fe700 1 --2- 172.21.15.157:0/1364791701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27d0003a60 0x7f27d0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:46.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.698+0000 7f27e77fe700 1 -- 172.21.15.157:0/1364791701 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d0003570 con 0x7f27d0004810 2023-10-06T03:57:46.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.698+0000 7f27e77fe700 1 --2- 172.21.15.157:0/1364791701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d0004810 0x7f27d0004c80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00c3f0 tx=0x7f27dc001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f062008da54f19 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:46.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.698+0000 7f27e6ffd700 1 -- 172.21.15.157:0/1364791701 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc00cc80 con 0x7f27d0004810 2023-10-06T03:57:46.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.698+0000 7f27e6ffd700 1 -- 172.21.15.157:0/1364791701 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27dc00ce20 con 0x7f27d0004810 2023-10-06T03:57:46.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.698+0000 7f27e6ffd700 1 -- 172.21.15.157:0/1364791701 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc005b70 con 0x7f27d0004810 2023-10-06T03:57:46.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.698+0000 7f27ed19f700 1 -- 172.21.15.157:0/1364791701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d0004810 msgr2=0x7f27d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:46.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.698+0000 7f27ed19f700 1 --2- 172.21.15.157:0/1364791701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d0004810 0x7f27d0004c80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00c3f0 tx=0x7f27dc001e90 comp rx=0 tx=0).stop 2023-10-06T03:57:46.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.698+0000 7f27ec99e700 1 -- 172.21.15.157:0/1364791701 reap_dead start 2023-10-06T03:57:46.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.698+0000 7f27ed19f700 1 -- 172.21.15.157:0/1364791701 shutdown_connections 2023-10-06T03:57:46.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.698+0000 7f27ed19f700 1 -- 172.21.15.157:0/1364791701 >> 172.21.15.157:0/1364791701 conn(0x7f27e8068020 msgr2=0x7f27e8068420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:46.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.698+0000 7f27ed19f700 1 -- 172.21.15.157:0/1364791701 shutdown_connections 2023-10-06T03:57:46.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.698+0000 7f27ed19f700 1 -- 172.21.15.157:0/1364791701 wait complete. 2023-10-06T03:57:46.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.699+0000 7f27ed19f700 1 Processor -- start 2023-10-06T03:57:46.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.700+0000 7f27ed19f700 1 -- start start 2023-10-06T03:57:46.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.700+0000 7f27ed19f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d0004810 0x7f27e80ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:46.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.700+0000 7f27ed19f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27e80c4490 0x7f27e8166390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:46.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.700+0000 7f27ed19f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27e80c48e0 0x7f27e81668d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:46.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.700+0000 7f27ed19f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e80c1a80 con 0x7f27d0004810 2023-10-06T03:57:46.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.700+0000 7f27ed19f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e8169990 con 0x7f27e80c48e0 2023-10-06T03:57:46.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.700+0000 7f27ed19f700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e816a4d0 con 0x7f27e80c4490 2023-10-06T03:57:46.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.700+0000 7f27e7fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d0004810 0x7f27e80ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:46.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.700+0000 7f27e7fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d0004810 0x7f27e80ad130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45106/0 (socket says 172.21.15.157:45106) 2023-10-06T03:57:46.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.700+0000 7f27e7fff700 1 -- 172.21.15.157:0/779234110 learned_addr learned my addr 172.21.15.157:0/779234110 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:46.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.701+0000 7f27e77fe700 1 --2- 172.21.15.157:0/779234110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27e80c4490 0x7f27e8166390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:46.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.701+0000 7f27ec99e700 1 --2- 172.21.15.157:0/779234110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27e80c48e0 0x7f27e81668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:46.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.701+0000 7f27e7fff700 1 -- 172.21.15.157:0/779234110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27e80c4490 msgr2=0x7f27e8166390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:46.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.701+0000 7f27e7fff700 1 --2- 172.21.15.157:0/779234110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27e80c4490 0x7f27e8166390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:46.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.701+0000 7f27e7fff700 1 -- 172.21.15.157:0/779234110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27e80c48e0 msgr2=0x7f27e81668d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:46.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.701+0000 7f27e7fff700 1 --2- 172.21.15.157:0/779234110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27e80c48e0 0x7f27e81668d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:46.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.701+0000 7f27e7fff700 1 -- 172.21.15.157:0/779234110 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e80ad750 con 0x7f27d0004810 2023-10-06T03:57:46.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.701+0000 7f27e77fe700 1 --2- 172.21.15.157:0/779234110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27e80c4490 0x7f27e8166390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:46.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.701+0000 7f27e7fff700 1 --2- 172.21.15.157:0/779234110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d0004810 0x7f27e80ad130 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f27d8011d70 tx=0x7f27d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:46.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.701+0000 7f27cffff700 1 -- 172.21.15.157:0/779234110 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d8009e20 con 0x7f27d0004810 2023-10-06T03:57:46.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.701+0000 7f27cffff700 1 -- 172.21.15.157:0/779234110 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d800fce0 con 0x7f27d0004810 2023-10-06T03:57:46.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.701+0000 7f27ed19f700 1 -- 172.21.15.157:0/779234110 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e8169ba0 con 0x7f27d0004810 2023-10-06T03:57:46.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.701+0000 7f27cffff700 1 -- 172.21.15.157:0/779234110 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d8014650 con 0x7f27d0004810 2023-10-06T03:57:46.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.701+0000 7f27ed19f700 1 -- 172.21.15.157:0/779234110 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e816a160 con 0x7f27d0004810 2023-10-06T03:57:46.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.702+0000 7f27ed19f700 1 -- 172.21.15.157:0/779234110 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e816a7b0 con 0x7f27d0004810 2023-10-06T03:57:46.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.702+0000 7f27cffff700 1 -- 172.21.15.157:0/779234110 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f27d8014a80 con 0x7f27d0004810 2023-10-06T03:57:46.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.703+0000 7f27cffff700 1 --2- 172.21.15.157:0/779234110 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f27d4066f60 0x7f27d4069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:46.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.703+0000 7f27cffff700 1 -- 172.21.15.157:0/779234110 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f27d8018030 con 0x7f27d0004810 2023-10-06T03:57:46.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.703+0000 7f27e77fe700 1 --2- 172.21.15.157:0/779234110 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f27d4066f60 0x7f27d4069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:46.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.703+0000 7f27e77fe700 1 --2- 172.21.15.157:0/779234110 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f27d4066f60 0x7f27d4069410 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00c3f0 tx=0x7f27dc015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:46.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.707+0000 7f27cffff700 1 -- 172.21.15.157:0/779234110 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f27d8052340 con 0x7f27d0004810 2023-10-06T03:57:46.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.890+0000 7f27ed19f700 1 -- 172.21.15.157:0/779234110 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f27e80ab960 con 0x7f27d0004810 2023-10-06T03:57:46.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.891+0000 7f27cffff700 1 -- 172.21.15.157:0/779234110 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f27d8055960 con 0x7f27d0004810 2023-10-06T03:57:46.893 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-10-06T03:57:46.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.893+0000 7f27ed19f700 1 -- 172.21.15.157:0/779234110 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f27d4066f60 msgr2=0x7f27d4069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:46.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.893+0000 7f27ed19f700 1 --2- 172.21.15.157:0/779234110 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f27d4066f60 0x7f27d4069410 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00c3f0 tx=0x7f27dc015000 comp rx=0 tx=0).stop 2023-10-06T03:57:46.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.893+0000 7f27ed19f700 1 -- 172.21.15.157:0/779234110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d0004810 msgr2=0x7f27e80ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:46.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.893+0000 7f27ed19f700 1 --2- 172.21.15.157:0/779234110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d0004810 0x7f27e80ad130 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f27d8011d70 tx=0x7f27d80091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:46.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.893+0000 7f27ed19f700 1 -- 172.21.15.157:0/779234110 shutdown_connections 2023-10-06T03:57:46.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.893+0000 7f27ed19f700 1 --2- 172.21.15.157:0/779234110 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f27d4066f60 0x7f27d4069410 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:46.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.893+0000 7f27ed19f700 1 --2- 172.21.15.157:0/779234110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27e80c48e0 0x7f27e81668d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:46.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.893+0000 7f27ed19f700 1 --2- 172.21.15.157:0/779234110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d0004810 0x7f27e80ad130 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:46.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.893+0000 7f27ed19f700 1 --2- 172.21.15.157:0/779234110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27e80c4490 0x7f27e8166390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:46.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.893+0000 7f27ed19f700 1 -- 172.21.15.157:0/779234110 >> 172.21.15.157:0/779234110 conn(0x7f27e8068020 msgr2=0x7f27e8068420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:46.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.893+0000 7f27ed19f700 1 -- 172.21.15.157:0/779234110 shutdown_connections 2023-10-06T03:57:46.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:46.893+0000 7f27ed19f700 1 -- 172.21.15.157:0/779234110 wait complete. 2023-10-06T03:57:46.907 INFO:tasks.mds_thrash:mds thrasher using random seed: 1696564666 2023-10-06T03:57:46.909 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:57:47.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.022+0000 7f20426d6700 1 Processor -- start 2023-10-06T03:57:47.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.022+0000 7f20426d6700 1 -- start start 2023-10-06T03:57:47.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.023+0000 7f20426d6700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f203c068df0 0x7f203c150d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:47.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.023+0000 7f20426d6700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f203c06e2e0 0x7f203c151270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:47.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.023+0000 7f20426d6700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203c1517b0 con 0x7f203c06a780 2023-10-06T03:57:47.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.023+0000 7f20426d6700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f203c0c36d0 con 0x7f203c068df0 2023-10-06T03:57:47.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.023+0000 7f20426d6700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203c0c3e30 con 0x7f203c06e2e0 2023-10-06T03:57:47.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.023+0000 7f203b7fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f203c068df0 0x7f203c150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:47.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.023+0000 7f203b7fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f203c068df0 0x7f203c150d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45116/0 (socket says 172.21.15.157:45116) 2023-10-06T03:57:47.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.023+0000 7f203b7fe700 1 -- 172.21.15.157:0/1078606102 learned_addr learned my addr 172.21.15.157:0/1078606102 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:47.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.023+0000 7f2040c73700 1 --2- 172.21.15.157:0/1078606102 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f203c06e2e0 0x7f203c151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:47.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.024+0000 7f203b7fe700 1 -- 172.21.15.157:0/1078606102 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f203c06e2e0 msgr2=0x7f203c151270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:47.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.024+0000 7f203b7fe700 1 --2- 172.21.15.157:0/1078606102 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f203c06e2e0 0x7f203c151270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:47.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.024+0000 7f203b7fe700 1 -- 172.21.15.157:0/1078606102 >> v1:172.21.15.175:6789/0 conn(0x7f203c06a780 legacy=0x7f203c154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:47.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.024+0000 7f203b7fe700 1 -- 172.21.15.157:0/1078606102 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c0c4110 con 0x7f203c068df0 2023-10-06T03:57:47.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.024+0000 7f203b7fe700 1 --2- 172.21.15.157:0/1078606102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f203c068df0 0x7f203c150d30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f20300099b0 tx=0x7f2030009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a56a6734439427e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:47.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.024+0000 7f203affd700 1 -- 172.21.15.157:0/1078606102 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2030009ea0 con 0x7f203c068df0 2023-10-06T03:57:47.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.024+0000 7f203affd700 1 -- 172.21.15.157:0/1078606102 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203000b8f0 con 0x7f203c068df0 2023-10-06T03:57:47.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.024+0000 7f203affd700 1 -- 172.21.15.157:0/1078606102 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203000bc90 con 0x7f203c068df0 2023-10-06T03:57:47.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.024+0000 7f20426d6700 1 -- 172.21.15.157:0/1078606102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f203c068df0 msgr2=0x7f203c150d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:47.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.024+0000 7f20426d6700 1 --2- 172.21.15.157:0/1078606102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f203c068df0 0x7f203c150d30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f20300099b0 tx=0x7f2030009250 comp rx=0 tx=0).stop 2023-10-06T03:57:47.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.025+0000 7f20426d6700 1 -- 172.21.15.157:0/1078606102 shutdown_connections 2023-10-06T03:57:47.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.025+0000 7f20426d6700 1 --2- 172.21.15.157:0/1078606102 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f203c06e2e0 0x7f203c151270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:47.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.025+0000 7f20426d6700 1 --2- 172.21.15.157:0/1078606102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f203c068df0 0x7f203c150d30 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:47.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.025+0000 7f20426d6700 1 -- 172.21.15.157:0/1078606102 >> 172.21.15.157:0/1078606102 conn(0x7f203c06b510 msgr2=0x7f203c06b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:47.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.025+0000 7f20426d6700 1 -- 172.21.15.157:0/1078606102 shutdown_connections 2023-10-06T03:57:47.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.025+0000 7f20426d6700 1 -- 172.21.15.157:0/1078606102 wait complete. 2023-10-06T03:57:47.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.026+0000 7f20426d6700 1 Processor -- start 2023-10-06T03:57:47.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.026+0000 7f20426d6700 1 -- start start 2023-10-06T03:57:47.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.026+0000 7f20426d6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f203c068df0 0x7f203c156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:47.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.026+0000 7f20426d6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f203c06a780 0x7f203c147d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:47.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.027+0000 7f20426d6700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f203c06e2e0 0x7f203c148290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:47.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.027+0000 7f20426d6700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c1559a0 con 0x7f203c06a780 2023-10-06T03:57:47.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.027+0000 7f20426d6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c1565b0 con 0x7f203c068df0 2023-10-06T03:57:47.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.027+0000 7f20426d6700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203c156f80 con 0x7f203c06e2e0 2023-10-06T03:57:47.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.027+0000 7f203b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f203c06a780 0x7f203c147d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:47.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.027+0000 7f203b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f203c06a780 0x7f203c147d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45118/0 (socket says 172.21.15.157:45118) 2023-10-06T03:57:47.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.027+0000 7f203b7fe700 1 -- 172.21.15.157:0/3580253938 learned_addr learned my addr 172.21.15.157:0/3580253938 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:47.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.027+0000 7f203bfff700 1 --2- 172.21.15.157:0/3580253938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f203c068df0 0x7f203c156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:47.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.027+0000 7f2040c73700 1 --2- 172.21.15.157:0/3580253938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f203c06e2e0 0x7f203c148290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:47.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.027+0000 7f203b7fe700 1 -- 172.21.15.157:0/3580253938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f203c06e2e0 msgr2=0x7f203c148290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:47.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.027+0000 7f203b7fe700 1 --2- 172.21.15.157:0/3580253938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f203c06e2e0 0x7f203c148290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:47.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.027+0000 7f203b7fe700 1 -- 172.21.15.157:0/3580253938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f203c068df0 msgr2=0x7f203c156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:47.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.027+0000 7f203b7fe700 1 --2- 172.21.15.157:0/3580253938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f203c068df0 0x7f203c156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:47.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.027+0000 7f203b7fe700 1 -- 172.21.15.157:0/3580253938 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20300094f0 con 0x7f203c06a780 2023-10-06T03:57:47.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.027+0000 7f203b7fe700 1 --2- 172.21.15.157:0/3580253938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f203c06a780 0x7f203c147d50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2030009290 tx=0x7f2030017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:47.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.027+0000 7f2023fff700 1 -- 172.21.15.157:0/3580253938 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203000ba60 con 0x7f203c06a780 2023-10-06T03:57:47.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.027+0000 7f2023fff700 1 -- 172.21.15.157:0/3580253938 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203000bbe0 con 0x7f203c06a780 2023-10-06T03:57:47.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.027+0000 7f2023fff700 1 -- 172.21.15.157:0/3580253938 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2030026680 con 0x7f203c06a780 2023-10-06T03:57:47.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.027+0000 7f20426d6700 1 -- 172.21.15.157:0/3580253938 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203c14a700 con 0x7f203c06a780 2023-10-06T03:57:47.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.027+0000 7f20426d6700 1 -- 172.21.15.157:0/3580253938 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f203c155010 con 0x7f203c06a780 2023-10-06T03:57:47.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.028+0000 7f20426d6700 1 -- 172.21.15.157:0/3580253938 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f203c0692b0 con 0x7f203c06a780 2023-10-06T03:57:47.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.029+0000 7f2023fff700 1 -- 172.21.15.157:0/3580253938 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2030023030 con 0x7f203c06a780 2023-10-06T03:57:47.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.029+0000 7f2023fff700 1 --2- 172.21.15.157:0/3580253938 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2028066f60 0x7f2028069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:47.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.029+0000 7f2023fff700 1 -- 172.21.15.157:0/3580253938 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2030085640 con 0x7f203c06a780 2023-10-06T03:57:47.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.029+0000 7f203bfff700 1 --2- 172.21.15.157:0/3580253938 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2028066f60 0x7f2028069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:47.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.030+0000 7f203bfff700 1 --2- 172.21.15.157:0/3580253938 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2028066f60 0x7f2028069410 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f202c003a80 tx=0x7f202c007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:47.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.033+0000 7f2023fff700 1 -- 172.21.15.157:0/3580253938 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f20300527d0 con 0x7f203c06a780 2023-10-06T03:57:47.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.219+0000 7f20426d6700 1 -- 172.21.15.157:0/3580253938 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f203c06c1f0 con 0x7f203c06a780 2023-10-06T03:57:47.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.220+0000 7f2023fff700 1 -- 172.21.15.157:0/3580253938 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f2030055df0 con 0x7f203c06a780 2023-10-06T03:57:47.221 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:47.222 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":21,"fsid":"28664a7a-fe30-4efe-be5a-6abf9318b700","created":"2023-10-06T03:56:51.887538+0000","modified":"2023-10-06T03:57:12.362537+0000","last_up_change":"2023-10-06T03:57:02.299440+0000","last_in_change":"2023-10-06T03:56:55.141505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:57:03.455121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:57:03.905661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:57:08.377186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:57:09.709582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f677cac-e9f4-4608-acac-718b072ce984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":90640},{"type":"v1","addr":"172.21.15.157:6809","nonce":90640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":90640},{"type":"v1","addr":"172.21.15.157:6811","nonce":90640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":90640},{"type":"v1","addr":"172.21.15.157:6815","nonce":90640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":90640},{"type":"v1","addr":"172.21.15.157:6813","nonce":90640}]},"public_addr":"172.21.15.157:6809/90640","cluster_addr":"172.21.15.157:6811/90640","heartbeat_back_addr":"172.21.15.157:6815/90640","heartbeat_front_addr":"172.21.15.157:6813/90640","state":["exists","up"]},{"osd":1,"uuid":"7d771892-bb91-4e9b-9610-051861c11247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":90654},{"type":"v1","addr":"172.21.15.157:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":90654},{"type":"v1","addr":"172.21.15.157:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":90654},{"type":"v1","addr":"172.21.15.157:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":90654},{"type":"v1","addr":"172.21.15.157:6805","nonce":90654}]},"public_addr":"172.21.15.157:6801/90654","cluster_addr":"172.21.15.157:6803/90654","heartbeat_back_addr":"172.21.15.157:6807/90654","heartbeat_front_addr":"172.21.15.157:6805/90654","state":["exists","up"]},{"osd":2,"uuid":"5c1f7d4e-d185-47cf-ab6c-453b4e3009c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":90657},{"type":"v1","addr":"172.21.15.157:6825","nonce":90657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":90657},{"type":"v1","addr":"172.21.15.157:6827","nonce":90657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":90657},{"type":"v1","addr":"172.21.15.157:6831","nonce":90657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":90657},{"type":"v1","addr":"172.21.15.157:6829","nonce":90657}]},"public_addr":"172.21.15.157:6825/90657","cluster_addr":"172.21.15.157:6827/90657","heartbeat_back_addr":"172.21.15.157:6831/90657","heartbeat_front_addr":"172.21.15.157:6829/90657","state":["exists","up"]},{"osd":3,"uuid":"f6a1e825-43c7-4afd-8daf-e51d45afc75e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":90641},{"type":"v1","addr":"172.21.15.157:6817","nonce":90641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":90641},{"type":"v1","addr":"172.21.15.157:6819","nonce":90641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":90641},{"type":"v1","addr":"172.21.15.157:6823","nonce":90641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":90641},{"type":"v1","addr":"172.21.15.157:6821","nonce":90641}]},"public_addr":"172.21.15.157:6817/90641","cluster_addr":"172.21.15.157:6819/90641","heartbeat_back_addr":"172.21.15.157:6823/90641","heartbeat_front_addr":"172.21.15.157:6821/90641","state":["exists","up"]},{"osd":4,"uuid":"8ad6e3d8-9a31-4f85-9410-417b7ac5da97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":91261},{"type":"v1","addr":"172.21.15.175:6803","nonce":91261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":91261},{"type":"v1","addr":"172.21.15.175:6806","nonce":91261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":91261},{"type":"v1","addr":"172.21.15.175:6814","nonce":91261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":91261},{"type":"v1","addr":"172.21.15.175:6810","nonce":91261}]},"public_addr":"172.21.15.175:6803/91261","cluster_addr":"172.21.15.175:6806/91261","heartbeat_back_addr":"172.21.15.175:6814/91261","heartbeat_front_addr":"172.21.15.175:6810/91261","state":["exists","up"]},{"osd":5,"uuid":"f68aca2a-f182-4c00-8e21-241d09c02ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":91262},{"type":"v1","addr":"172.21.15.175:6817","nonce":91262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":91262},{"type":"v1","addr":"172.21.15.175:6819","nonce":91262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":91262},{"type":"v1","addr":"172.21.15.175:6823","nonce":91262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":91262},{"type":"v1","addr":"172.21.15.175:6821","nonce":91262}]},"public_addr":"172.21.15.175:6817/91262","cluster_addr":"172.21.15.175:6819/91262","heartbeat_back_addr":"172.21.15.175:6823/91262","heartbeat_front_addr":"172.21.15.175:6821/91262","state":["exists","up"]},{"osd":6,"uuid":"d867cd89-3430-40ca-862f-d2bea3de9d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":91263},{"type":"v1","addr":"172.21.15.175:6801","nonce":91263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6805","nonce":91263},{"type":"v1","addr":"172.21.15.175:6807","nonce":91263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6813","nonce":91263},{"type":"v1","addr":"172.21.15.175:6815","nonce":91263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":91263},{"type":"v1","addr":"172.21.15.175:6811","nonce":91263}]},"public_addr":"172.21.15.175:6801/91263","cluster_addr":"172.21.15.175:6807/91263","heartbeat_back_addr":"172.21.15.175:6815/91263","heartbeat_front_addr":"172.21.15.175:6811/91263","state":["exists","up"]},{"osd":7,"uuid":"930cd464-79f1-47cb-b4ff-188ccde74fff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":91306},{"type":"v1","addr":"172.21.15.175:6825","nonce":91306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":91306},{"type":"v1","addr":"172.21.15.175:6827","nonce":91306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":91306},{"type":"v1","addr":"172.21.15.175:6831","nonce":91306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":91306},{"type":"v1","addr":"172.21.15.175:6829","nonce":91306}]},"public_addr":"172.21.15.175:6825/91306","cluster_addr":"172.21.15.175:6827/91306","heartbeat_back_addr":"172.21.15.175:6831/91306","heartbeat_front_addr":"172.21.15.175:6829/91306","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.290673+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.056390+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.292903+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.275861+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.332095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.327453+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.317585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.309330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:57:47.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.222+0000 7f20426d6700 1 -- 172.21.15.157:0/3580253938 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2028066f60 msgr2=0x7f2028069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:47.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.222+0000 7f20426d6700 1 --2- 172.21.15.157:0/3580253938 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2028066f60 0x7f2028069410 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f202c003a80 tx=0x7f202c007ec0 comp rx=0 tx=0).stop 2023-10-06T03:57:47.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.222+0000 7f20426d6700 1 -- 172.21.15.157:0/3580253938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f203c06a780 msgr2=0x7f203c147d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:47.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.222+0000 7f20426d6700 1 --2- 172.21.15.157:0/3580253938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f203c06a780 0x7f203c147d50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2030009290 tx=0x7f2030017000 comp rx=0 tx=0).stop 2023-10-06T03:57:47.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.222+0000 7f20426d6700 1 -- 172.21.15.157:0/3580253938 shutdown_connections 2023-10-06T03:57:47.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.222+0000 7f20426d6700 1 --2- 172.21.15.157:0/3580253938 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2028066f60 0x7f2028069410 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:47.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.222+0000 7f20426d6700 1 --2- 172.21.15.157:0/3580253938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f203c068df0 0x7f203c156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:47.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.222+0000 7f20426d6700 1 --2- 172.21.15.157:0/3580253938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f203c06e2e0 0x7f203c148290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:47.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.222+0000 7f20426d6700 1 --2- 172.21.15.157:0/3580253938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f203c06a780 0x7f203c147d50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:47.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.222+0000 7f20426d6700 1 -- 172.21.15.157:0/3580253938 >> 172.21.15.157:0/3580253938 conn(0x7f203c06b510 msgr2=0x7f203c0c9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:47.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.222+0000 7f20426d6700 1 -- 172.21.15.157:0/3580253938 shutdown_connections 2023-10-06T03:57:47.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.222+0000 7f20426d6700 1 -- 172.21.15.157:0/3580253938 wait complete. 2023-10-06T03:57:47.236 INFO:tasks.mds_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:57:03.455121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:57:03.905661+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:57:08.377186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:57:09.709582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:57:47.237 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:57:47.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.352+0000 7fe0d4965700 1 Processor -- start 2023-10-06T03:57:47.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.352+0000 7fe0d4965700 1 -- start start 2023-10-06T03:57:47.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.353+0000 7fe0d4965700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0d00ca450 0x7fe0d00c16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:47.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.353+0000 7fe0d4965700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe0d0157e20 0x7fe0d00c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:47.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.353+0000 7fe0d4965700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0d00c5d10 con 0x7fe0d0068600 2023-10-06T03:57:47.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.353+0000 7fe0d4965700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0d0067630 con 0x7fe0d0157e20 2023-10-06T03:57:47.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.353+0000 7fe0d4965700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d0067c00 con 0x7fe0d00ca450 2023-10-06T03:57:47.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.353+0000 7fe0cffff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe0d0157e20 0x7fe0d00c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:47.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.353+0000 7fe0cffff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe0d0157e20 0x7fe0d00c61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45132/0 (socket says 172.21.15.157:45132) 2023-10-06T03:57:47.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.353+0000 7fe0cffff700 1 -- 172.21.15.157:0/3981231986 learned_addr learned my addr 172.21.15.157:0/3981231986 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:47.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.353+0000 7fe0cf7fe700 1 --2- 172.21.15.157:0/3981231986 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0d00ca450 0x7fe0d00c16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:47.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.353+0000 7fe0cffff700 1 -- 172.21.15.157:0/3981231986 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0d00ca450 msgr2=0x7fe0d00c16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:47.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.353+0000 7fe0cffff700 1 --2- 172.21.15.157:0/3981231986 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0d00ca450 0x7fe0d00c16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:47.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.353+0000 7fe0cffff700 1 -- 172.21.15.157:0/3981231986 >> v1:172.21.15.175:6789/0 conn(0x7fe0d0068600 legacy=0x7fe0d0157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:47.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.353+0000 7fe0cffff700 1 -- 172.21.15.157:0/3981231986 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d00c4290 con 0x7fe0d0157e20 2023-10-06T03:57:47.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.353+0000 7fe0cffff700 1 --2- 172.21.15.157:0/3981231986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0d0157e20 0x7fe0d00c61a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8009f20 tx=0x7fe0c8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604a9ed904dca0be server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:47.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.354+0000 7fe0ce7fc700 1 -- 172.21.15.157:0/3981231986 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c800e070 con 0x7fe0d0157e20 2023-10-06T03:57:47.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.354+0000 7fe0ce7fc700 1 -- 172.21.15.157:0/3981231986 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c800b9d0 con 0x7fe0d0157e20 2023-10-06T03:57:47.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.354+0000 7fe0ce7fc700 1 -- 172.21.15.157:0/3981231986 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c80092e0 con 0x7fe0d0157e20 2023-10-06T03:57:47.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.354+0000 7fe0d4965700 1 -- 172.21.15.157:0/3981231986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0d0157e20 msgr2=0x7fe0d00c61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:47.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.354+0000 7fe0d4965700 1 --2- 172.21.15.157:0/3981231986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0d0157e20 0x7fe0d00c61a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8009f20 tx=0x7fe0c8009840 comp rx=0 tx=0).stop 2023-10-06T03:57:47.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.354+0000 7fe0d4965700 1 -- 172.21.15.157:0/3981231986 shutdown_connections 2023-10-06T03:57:47.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.354+0000 7fe0d4965700 1 --2- 172.21.15.157:0/3981231986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0d0157e20 0x7fe0d00c61a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:47.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.354+0000 7fe0d4965700 1 --2- 172.21.15.157:0/3981231986 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0d00ca450 0x7fe0d00c16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:47.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.354+0000 7fe0d4965700 1 -- 172.21.15.157:0/3981231986 >> 172.21.15.157:0/3981231986 conn(0x7fe0d0069100 msgr2=0x7fe0d0069500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:47.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.354+0000 7fe0d4965700 1 -- 172.21.15.157:0/3981231986 shutdown_connections 2023-10-06T03:57:47.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.354+0000 7fe0d4965700 1 -- 172.21.15.157:0/3981231986 wait complete. 2023-10-06T03:57:47.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.355+0000 7fe0d4965700 1 Processor -- start 2023-10-06T03:57:47.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.355+0000 7fe0d4965700 1 -- start start 2023-10-06T03:57:47.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.356+0000 7fe0d4965700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0d0068600 0x7fe0d0153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:47.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.356+0000 7fe0d4965700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0d00ca450 0x7fe0d01492a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:47.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.356+0000 7fe0d4965700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0d0157e20 0x7fe0d01439b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:47.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.356+0000 7fe0d4965700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d01645e0 con 0x7fe0d00ca450 2023-10-06T03:57:47.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.356+0000 7fe0d4965700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d01498c0 con 0x7fe0d0068600 2023-10-06T03:57:47.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.356+0000 7fe0d4965700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d0163cb0 con 0x7fe0d0157e20 2023-10-06T03:57:47.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.356+0000 7fe0ceffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0d00ca450 0x7fe0d01492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:47.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.356+0000 7fe0ceffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0d00ca450 0x7fe0d01492a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45140/0 (socket says 172.21.15.157:45140) 2023-10-06T03:57:47.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.356+0000 7fe0ceffd700 1 -- 172.21.15.157:0/2037561071 learned_addr learned my addr 172.21.15.157:0/2037561071 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:47.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.356+0000 7fe0cf7fe700 1 --2- 172.21.15.157:0/2037561071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0d0068600 0x7fe0d0153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:47.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.356+0000 7fe0cffff700 1 --2- 172.21.15.157:0/2037561071 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0d0157e20 0x7fe0d01439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:47.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.357+0000 7fe0ceffd700 1 -- 172.21.15.157:0/2037561071 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0d0157e20 msgr2=0x7fe0d01439b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:47.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.357+0000 7fe0ceffd700 1 --2- 172.21.15.157:0/2037561071 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0d0157e20 0x7fe0d01439b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:47.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.357+0000 7fe0ceffd700 1 -- 172.21.15.157:0/2037561071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0d0068600 msgr2=0x7fe0d0153130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:47.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.357+0000 7fe0ceffd700 1 --2- 172.21.15.157:0/2037561071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0d0068600 0x7fe0d0153130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:47.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.357+0000 7fe0ceffd700 1 -- 172.21.15.157:0/2037561071 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c8009ae0 con 0x7fe0d00ca450 2023-10-06T03:57:47.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.357+0000 7fe0ceffd700 1 --2- 172.21.15.157:0/2037561071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0d00ca450 0x7fe0d01492a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe0c400c590 tx=0x7fe0c400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:47.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.357+0000 7fe0b77fe700 1 -- 172.21.15.157:0/2037561071 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c4011590 con 0x7fe0d00ca450 2023-10-06T03:57:47.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.357+0000 7fe0b77fe700 1 -- 172.21.15.157:0/2037561071 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c4011710 con 0x7fe0d00ca450 2023-10-06T03:57:47.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.357+0000 7fe0d4965700 1 -- 172.21.15.157:0/2037561071 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0d0163f90 con 0x7fe0d00ca450 2023-10-06T03:57:47.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.357+0000 7fe0b77fe700 1 -- 172.21.15.157:0/2037561071 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c4011ad0 con 0x7fe0d00ca450 2023-10-06T03:57:47.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.357+0000 7fe0d4965700 1 -- 172.21.15.157:0/2037561071 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0d0143ef0 con 0x7fe0d00ca450 2023-10-06T03:57:47.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.358+0000 7fe0d4965700 1 -- 172.21.15.157:0/2037561071 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0d006a1b0 con 0x7fe0d00ca450 2023-10-06T03:57:47.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.358+0000 7fe0b77fe700 1 -- 172.21.15.157:0/2037561071 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe0c401c440 con 0x7fe0d00ca450 2023-10-06T03:57:47.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.359+0000 7fe0b77fe700 1 --2- 172.21.15.157:0/2037561071 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe0bc066f30 0x7fe0bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:47.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.359+0000 7fe0b77fe700 1 -- 172.21.15.157:0/2037561071 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe0c4083a70 con 0x7fe0d00ca450 2023-10-06T03:57:47.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.359+0000 7fe0cf7fe700 1 --2- 172.21.15.157:0/2037561071 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe0bc066f30 0x7fe0bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:47.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.359+0000 7fe0cf7fe700 1 --2- 172.21.15.157:0/2037561071 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe0bc066f30 0x7fe0bc0693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009db0 tx=0x7fe0c00091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:47.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.363+0000 7fe0b77fe700 1 -- 172.21.15.157:0/2037561071 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe0c4050b80 con 0x7fe0d00ca450 2023-10-06T03:57:47.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.546+0000 7fe0d4965700 1 -- 172.21.15.157:0/2037561071 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe0d00c6f50 con 0x7fe0d00ca450 2023-10-06T03:57:47.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.546+0000 7fe0b77fe700 1 -- 172.21.15.157:0/2037561071 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe0c4017070 con 0x7fe0d00ca450 2023-10-06T03:57:47.548 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-10-06T03:57:47.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.548+0000 7fe0d4965700 1 -- 172.21.15.157:0/2037561071 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe0bc066f30 msgr2=0x7fe0bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:47.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.548+0000 7fe0d4965700 1 --2- 172.21.15.157:0/2037561071 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe0bc066f30 0x7fe0bc0693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009db0 tx=0x7fe0c00091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:47.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.548+0000 7fe0d4965700 1 -- 172.21.15.157:0/2037561071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0d00ca450 msgr2=0x7fe0d01492a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:47.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.548+0000 7fe0d4965700 1 --2- 172.21.15.157:0/2037561071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0d00ca450 0x7fe0d01492a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe0c400c590 tx=0x7fe0c400c8a0 comp rx=0 tx=0).stop 2023-10-06T03:57:47.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.548+0000 7fe0d4965700 1 -- 172.21.15.157:0/2037561071 shutdown_connections 2023-10-06T03:57:47.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.548+0000 7fe0d4965700 1 --2- 172.21.15.157:0/2037561071 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe0bc066f30 0x7fe0bc0693e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:47.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.548+0000 7fe0d4965700 1 --2- 172.21.15.157:0/2037561071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0d0068600 0x7fe0d0153130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:47.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.548+0000 7fe0d4965700 1 --2- 172.21.15.157:0/2037561071 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0d0157e20 0x7fe0d01439b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:47.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.548+0000 7fe0d4965700 1 --2- 172.21.15.157:0/2037561071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0d00ca450 0x7fe0d01492a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:47.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.548+0000 7fe0d4965700 1 -- 172.21.15.157:0/2037561071 >> 172.21.15.157:0/2037561071 conn(0x7fe0d0069100 msgr2=0x7fe0d014c700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:47.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.549+0000 7fe0d4965700 1 -- 172.21.15.157:0/2037561071 shutdown_connections 2023-10-06T03:57:47.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.549+0000 7fe0d4965700 1 -- 172.21.15.157:0/2037561071 wait complete. 2023-10-06T03:57:47.562 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:57:47.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.679+0000 7f1931ace700 1 Processor -- start 2023-10-06T03:57:47.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.679+0000 7f1931ace700 1 -- start start 2023-10-06T03:57:47.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.679+0000 7f1931ace700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f192c06dc40 0x7f192c14fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:47.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.679+0000 7f1931ace700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f192c0c9780 con 0x7f192c06cbb0 2023-10-06T03:57:47.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.679+0000 7f1931ace700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f192c06e0c0 con 0x7f192c06c1f0 2023-10-06T03:57:47.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.679+0000 7f1931ace700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f192c0c8c40 con 0x7f192c06dc40 2023-10-06T03:57:47.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.679+0000 7f192b7fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f192c06cbb0 0x7f192c06d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40786/0 (socket says 172.21.15.157:40786) 2023-10-06T03:57:47.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.679+0000 7f192b7fe700 1 -- 172.21.15.157:0/585802323 learned_addr learned my addr 172.21.15.157:0/585802323 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:47.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.679+0000 7f192affd700 1 --2- 172.21.15.157:0/585802323 >> v2:172.21.15.157:3300/0 conn(0x7f192c06dc40 0x7f192c14fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:47.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.680+0000 7f192affd700 1 -- 172.21.15.157:0/585802323 >> v1:172.21.15.175:6789/0 conn(0x7f192c06c1f0 legacy=0x7f192c155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:47.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.680+0000 7f192affd700 1 -- 172.21.15.157:0/585802323 >> v1:172.21.15.157:6789/0 conn(0x7f192c06cbb0 legacy=0x7f192c06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:47.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.680+0000 7f192affd700 1 -- 172.21.15.157:0/585802323 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192c0c8e50 con 0x7f192c06dc40 2023-10-06T03:57:47.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.680+0000 7f192affd700 1 --2- 172.21.15.157:0/585802323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f192c06dc40 0x7f192c14fd00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f191c0098e0 tx=0x7f191c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d1e6fd7fe33929 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:47.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.680+0000 7f192a7fc700 1 -- 172.21.15.157:0/585802323 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c009e80 con 0x7f192c06dc40 2023-10-06T03:57:47.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.680+0000 7f192a7fc700 1 -- 172.21.15.157:0/585802323 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191c00b8b0 con 0x7f192c06dc40 2023-10-06T03:57:47.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.680+0000 7f192a7fc700 1 -- 172.21.15.157:0/585802323 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c00bc30 con 0x7f192c06dc40 2023-10-06T03:57:47.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.681+0000 7f1931ace700 1 -- 172.21.15.157:0/585802323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f192c06dc40 msgr2=0x7f192c14fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:47.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.681+0000 7f1931ace700 1 --2- 172.21.15.157:0/585802323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f192c06dc40 0x7f192c14fd00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f191c0098e0 tx=0x7f191c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:47.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.681+0000 7f1931ace700 1 -- 172.21.15.157:0/585802323 shutdown_connections 2023-10-06T03:57:47.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.681+0000 7f1931ace700 1 --2- 172.21.15.157:0/585802323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f192c06dc40 0x7f192c14fd00 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:47.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.681+0000 7f1931ace700 1 -- 172.21.15.157:0/585802323 >> 172.21.15.157:0/585802323 conn(0x7f192c06f230 msgr2=0x7f192c06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:47.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.681+0000 7f1931ace700 1 -- 172.21.15.157:0/585802323 shutdown_connections 2023-10-06T03:57:47.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.681+0000 7f1931ace700 1 -- 172.21.15.157:0/585802323 wait complete. 2023-10-06T03:57:47.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.682+0000 7f1931ace700 1 Processor -- start 2023-10-06T03:57:47.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.682+0000 7f1931ace700 1 -- start start 2023-10-06T03:57:47.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.682+0000 7f1931ace700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f192c06c1f0 0x7f192c0af400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:47.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.682+0000 7f1931ace700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f192c06cbb0 0x7f192c15dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:47.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.682+0000 7f1931ace700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f192c0c4350 0x7f192c15e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:47.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.683+0000 7f1931ace700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192c1673d0 con 0x7f192c06c1f0 2023-10-06T03:57:47.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.683+0000 7f1931ace700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192c15d820 con 0x7f192c06cbb0 2023-10-06T03:57:47.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.683+0000 7f1931ace700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f192c0c3480 con 0x7f192c0c4350 2023-10-06T03:57:47.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.683+0000 7f192b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f192c06c1f0 0x7f192c0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:47.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.683+0000 7f192b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f192c06c1f0 0x7f192c0af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45162/0 (socket says 172.21.15.157:45162) 2023-10-06T03:57:47.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.683+0000 7f192b7fe700 1 -- 172.21.15.157:0/3749931560 learned_addr learned my addr 172.21.15.157:0/3749931560 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:47.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.683+0000 7f192affd700 1 --2- 172.21.15.157:0/3749931560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f192c06cbb0 0x7f192c15dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:47.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.683+0000 7f192bfff700 1 --2- 172.21.15.157:0/3749931560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f192c0c4350 0x7f192c15e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:47.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.683+0000 7f192b7fe700 1 -- 172.21.15.157:0/3749931560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f192c0c4350 msgr2=0x7f192c15e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:47.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.683+0000 7f192b7fe700 1 --2- 172.21.15.157:0/3749931560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f192c0c4350 0x7f192c15e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:47.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.683+0000 7f192b7fe700 1 -- 172.21.15.157:0/3749931560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f192c06cbb0 msgr2=0x7f192c15dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:47.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.683+0000 7f192b7fe700 1 --2- 172.21.15.157:0/3749931560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f192c06cbb0 0x7f192c15dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:47.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.683+0000 7f192b7fe700 1 -- 172.21.15.157:0/3749931560 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191c009420 con 0x7f192c06c1f0 2023-10-06T03:57:47.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.683+0000 7f192affd700 1 --2- 172.21.15.157:0/3749931560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f192c06cbb0 0x7f192c15dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:47.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.683+0000 7f192bfff700 1 --2- 172.21.15.157:0/3749931560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f192c0c4350 0x7f192c15e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:47.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.683+0000 7f192b7fe700 1 --2- 172.21.15.157:0/3749931560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f192c06c1f0 0x7f192c0af400 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f191400c7f0 tx=0x7f1914007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:47.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.683+0000 7f19237fe700 1 -- 172.21.15.157:0/3749931560 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1914005b30 con 0x7f192c06c1f0 2023-10-06T03:57:47.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.683+0000 7f19237fe700 1 -- 172.21.15.157:0/3749931560 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1914005cb0 con 0x7f192c06c1f0 2023-10-06T03:57:47.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.683+0000 7f19237fe700 1 -- 172.21.15.157:0/3749931560 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1914004a80 con 0x7f192c06c1f0 2023-10-06T03:57:47.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.683+0000 7f1931ace700 1 -- 172.21.15.157:0/3749931560 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f192c166870 con 0x7f192c06c1f0 2023-10-06T03:57:47.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.683+0000 7f1931ace700 1 -- 172.21.15.157:0/3749931560 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f192c166e10 con 0x7f192c06c1f0 2023-10-06T03:57:47.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.684+0000 7f1931ace700 1 -- 172.21.15.157:0/3749931560 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f192c089ca0 con 0x7f192c06c1f0 2023-10-06T03:57:47.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.684+0000 7f19237fe700 1 -- 172.21.15.157:0/3749931560 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f19140070f0 con 0x7f192c06c1f0 2023-10-06T03:57:47.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.685+0000 7f19237fe700 1 --2- 172.21.15.157:0/3749931560 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f190c066f30 0x7f190c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:47.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.685+0000 7f19237fe700 1 -- 172.21.15.157:0/3749931560 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1914083b00 con 0x7f192c06c1f0 2023-10-06T03:57:47.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.685+0000 7f192affd700 1 --2- 172.21.15.157:0/3749931560 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f190c066f30 0x7f190c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:47.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.685+0000 7f192affd700 1 --2- 172.21.15.157:0/3749931560 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f190c066f30 0x7f190c0693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f191c0030c0 tx=0x7f191c016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:47.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.689+0000 7f19237fe700 1 -- 172.21.15.157:0/3749931560 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1914050c90 con 0x7f192c06c1f0 2023-10-06T03:57:47.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.873+0000 7f1931ace700 1 -- 172.21.15.157:0/3749931560 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f192c0c9130 con 0x7f192c06c1f0 2023-10-06T03:57:47.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.873+0000 7f19237fe700 1 -- 172.21.15.157:0/3749931560 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f19140542b0 con 0x7f192c06c1f0 2023-10-06T03:57:47.875 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-10-06T03:57:47.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.876+0000 7f1931ace700 1 -- 172.21.15.157:0/3749931560 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f190c066f30 msgr2=0x7f190c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:47.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.876+0000 7f1931ace700 1 --2- 172.21.15.157:0/3749931560 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f190c066f30 0x7f190c0693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f191c0030c0 tx=0x7f191c016000 comp rx=0 tx=0).stop 2023-10-06T03:57:47.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.876+0000 7f1931ace700 1 -- 172.21.15.157:0/3749931560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f192c06c1f0 msgr2=0x7f192c0af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:47.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.876+0000 7f1931ace700 1 --2- 172.21.15.157:0/3749931560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f192c06c1f0 0x7f192c0af400 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f191400c7f0 tx=0x7f1914007f70 comp rx=0 tx=0).stop 2023-10-06T03:57:47.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.876+0000 7f1931ace700 1 -- 172.21.15.157:0/3749931560 shutdown_connections 2023-10-06T03:57:47.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.876+0000 7f1931ace700 1 --2- 172.21.15.157:0/3749931560 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f190c066f30 0x7f190c0693e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:47.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.876+0000 7f1931ace700 1 --2- 172.21.15.157:0/3749931560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f192c0c4350 0x7f192c15e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:47.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.876+0000 7f1931ace700 1 --2- 172.21.15.157:0/3749931560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f192c06c1f0 0x7f192c0af400 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:47.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.876+0000 7f1931ace700 1 --2- 172.21.15.157:0/3749931560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f192c06cbb0 0x7f192c15dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:47.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.876+0000 7f1931ace700 1 -- 172.21.15.157:0/3749931560 >> 172.21.15.157:0/3749931560 conn(0x7f192c06f230 msgr2=0x7f192c14e610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:47.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.876+0000 7f1931ace700 1 -- 172.21.15.157:0/3749931560 shutdown_connections 2023-10-06T03:57:47.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:47.876+0000 7f1931ace700 1 -- 172.21.15.157:0/3749931560 wait complete. 2023-10-06T03:57:47.889 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:57:48.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.005+0000 7f94f0d78700 1 Processor -- start 2023-10-06T03:57:48.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.005+0000 7f94f0d78700 1 -- start start 2023-10-06T03:57:48.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.006+0000 7f94f0d78700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f94ec06cbb0 0x7f94ec06d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:48.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.006+0000 7f94f0d78700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f94ec150410 0x7f94ec06da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:48.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.006+0000 7f94f0d78700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94ec0c9780 con 0x7f94ec06c1f0 2023-10-06T03:57:48.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.006+0000 7f94f0d78700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94ec0c8c40 con 0x7f94ec150410 2023-10-06T03:57:48.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.006+0000 7f94f0d78700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94ec06e0c0 con 0x7f94ec06cbb0 2023-10-06T03:57:48.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.006+0000 7f94ead9d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f94ec150410 0x7f94ec06da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:48.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.006+0000 7f94ead9d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f94ec150410 0x7f94ec06da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45164/0 (socket says 172.21.15.157:45164) 2023-10-06T03:57:48.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.007+0000 7f94ead9d700 1 -- 172.21.15.157:0/326249368 learned_addr learned my addr 172.21.15.157:0/326249368 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:48.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.006+0000 7f94e9d9b700 1 --1- 172.21.15.157:0/326249368 >> v1:172.21.15.157:6789/0 conn(0x7f94ec06c1f0 0x7f94ec14fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40802/0 (socket says 172.21.15.157:40802) 2023-10-06T03:57:48.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.007+0000 7f94ea59c700 1 --2- 172.21.15.157:0/326249368 >> v2:172.21.15.175:3300/0 conn(0x7f94ec06cbb0 0x7f94ec06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:48.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.007+0000 7f94ead9d700 1 -- 172.21.15.157:0/326249368 >> v2:172.21.15.175:3300/0 conn(0x7f94ec06cbb0 msgr2=0x7f94ec06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:48.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.007+0000 7f94ead9d700 1 --2- 172.21.15.157:0/326249368 >> v2:172.21.15.175:3300/0 conn(0x7f94ec06cbb0 0x7f94ec06d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:48.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.007+0000 7f94ead9d700 1 -- 172.21.15.157:0/326249368 >> v1:172.21.15.157:6789/0 conn(0x7f94ec06c1f0 legacy=0x7f94ec14fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:48.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.007+0000 7f94ead9d700 1 -- 172.21.15.157:0/326249368 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ec0c8e50 con 0x7f94ec150410 2023-10-06T03:57:48.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.007+0000 7f94e959a700 1 -- 172.21.15.157:0/326249368 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3842618538 0 0) 0x7f94ec0c9780 con 0x7f94ec06c1f0 2023-10-06T03:57:48.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.007+0000 7f94ea59c700 1 --2- 172.21.15.157:0/326249368 >> v2:172.21.15.175:3300/0 conn(0x7f94ec06cbb0 0x7f94ec06d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:57:48.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.007+0000 7f94ead9d700 1 --2- 172.21.15.157:0/326249368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94ec150410 0x7f94ec06da70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f94ec06fd70 tx=0x7f94e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3108cd3daf192 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:48.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.007+0000 7f94e959a700 1 -- 172.21.15.157:0/326249368 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e400b8f0 con 0x7f94ec150410 2023-10-06T03:57:48.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.007+0000 7f94e959a700 1 -- 172.21.15.157:0/326249368 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e400ba50 con 0x7f94ec150410 2023-10-06T03:57:48.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.007+0000 7f94e959a700 1 -- 172.21.15.157:0/326249368 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e400bdf0 con 0x7f94ec150410 2023-10-06T03:57:48.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.008+0000 7f94f0d78700 1 -- 172.21.15.157:0/326249368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94ec150410 msgr2=0x7f94ec06da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:48.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.008+0000 7f94f0d78700 1 --2- 172.21.15.157:0/326249368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94ec150410 0x7f94ec06da70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f94ec06fd70 tx=0x7f94e4009250 comp rx=0 tx=0).stop 2023-10-06T03:57:48.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.008+0000 7f94f0d78700 1 -- 172.21.15.157:0/326249368 shutdown_connections 2023-10-06T03:57:48.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.008+0000 7f94f0d78700 1 --2- 172.21.15.157:0/326249368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94ec150410 0x7f94ec06da70 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:48.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.008+0000 7f94f0d78700 1 --2- 172.21.15.157:0/326249368 >> v2:172.21.15.175:3300/0 conn(0x7f94ec06cbb0 0x7f94ec06d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:48.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.008+0000 7f94f0d78700 1 -- 172.21.15.157:0/326249368 >> 172.21.15.157:0/326249368 conn(0x7f94ec06f230 msgr2=0x7f94ec06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:48.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.008+0000 7f94f0d78700 1 -- 172.21.15.157:0/326249368 shutdown_connections 2023-10-06T03:57:48.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.008+0000 7f94f0d78700 1 -- 172.21.15.157:0/326249368 wait complete. 2023-10-06T03:57:48.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.009+0000 7f94f0d78700 1 Processor -- start 2023-10-06T03:57:48.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.009+0000 7f94f0d78700 1 -- start start 2023-10-06T03:57:48.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.009+0000 7f94f0d78700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94ec06c1f0 0x7f94ec156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:48.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.009+0000 7f94f0d78700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94ec06cbb0 0x7f94ec14d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:48.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.009+0000 7f94f0d78700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94ec150410 0x7f94ec154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:48.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.009+0000 7f94f0d78700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec155b80 con 0x7f94ec150410 2023-10-06T03:57:48.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.009+0000 7f94f0d78700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec16a1d0 con 0x7f94ec06c1f0 2023-10-06T03:57:48.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.009+0000 7f94f0d78700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec16a470 con 0x7f94ec06cbb0 2023-10-06T03:57:48.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.010+0000 7f94ead9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94ec150410 0x7f94ec154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:48.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.010+0000 7f94ead9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94ec150410 0x7f94ec154240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45172/0 (socket says 172.21.15.157:45172) 2023-10-06T03:57:48.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.010+0000 7f94ead9d700 1 -- 172.21.15.157:0/2245278675 learned_addr learned my addr 172.21.15.157:0/2245278675 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:48.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.010+0000 7f94e9d9b700 1 --2- 172.21.15.157:0/2245278675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94ec06cbb0 0x7f94ec14d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:48.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.010+0000 7f94ea59c700 1 --2- 172.21.15.157:0/2245278675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94ec06c1f0 0x7f94ec156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:48.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.010+0000 7f94ead9d700 1 -- 172.21.15.157:0/2245278675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94ec06cbb0 msgr2=0x7f94ec14d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:48.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.010+0000 7f94ead9d700 1 --2- 172.21.15.157:0/2245278675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94ec06cbb0 0x7f94ec14d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:48.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.010+0000 7f94ead9d700 1 -- 172.21.15.157:0/2245278675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94ec06c1f0 msgr2=0x7f94ec156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:48.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.010+0000 7f94ead9d700 1 --2- 172.21.15.157:0/2245278675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94ec06c1f0 0x7f94ec156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:48.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.010+0000 7f94ead9d700 1 -- 172.21.15.157:0/2245278675 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e40094f0 con 0x7f94ec150410 2023-10-06T03:57:48.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.010+0000 7f94ead9d700 1 --2- 172.21.15.157:0/2245278675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94ec150410 0x7f94ec154240 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f94ec0c3960 tx=0x7f94e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:48.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.010+0000 7f94da7fc700 1 -- 172.21.15.157:0/2245278675 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e400b8f0 con 0x7f94ec150410 2023-10-06T03:57:48.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.010+0000 7f94da7fc700 1 -- 172.21.15.157:0/2245278675 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e401a630 con 0x7f94ec150410 2023-10-06T03:57:48.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.010+0000 7f94f0d78700 1 -- 172.21.15.157:0/2245278675 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94ec155530 con 0x7f94ec150410 2023-10-06T03:57:48.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.010+0000 7f94da7fc700 1 -- 172.21.15.157:0/2245278675 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e401a9f0 con 0x7f94ec150410 2023-10-06T03:57:48.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.010+0000 7f94f0d78700 1 -- 172.21.15.157:0/2245278675 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ec0ca770 con 0x7f94ec150410 2023-10-06T03:57:48.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.011+0000 7f94f0d78700 1 -- 172.21.15.157:0/2245278675 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94ec155390 con 0x7f94ec150410 2023-10-06T03:57:48.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.012+0000 7f94da7fc700 1 -- 172.21.15.157:0/2245278675 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f94e40233e0 con 0x7f94ec150410 2023-10-06T03:57:48.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.012+0000 7f94da7fc700 1 --2- 172.21.15.157:0/2245278675 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f94cc066f30 0x7f94cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:48.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.012+0000 7f94da7fc700 1 -- 172.21.15.157:0/2245278675 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f94e40861a0 con 0x7f94ec150410 2023-10-06T03:57:48.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.012+0000 7f94ea59c700 1 --2- 172.21.15.157:0/2245278675 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f94cc066f30 0x7f94cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:48.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.013+0000 7f94ea59c700 1 --2- 172.21.15.157:0/2245278675 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f94cc066f30 0x7f94cc0693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00d6e0 tx=0x7f94dc007850 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:48.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.016+0000 7f94da7fc700 1 -- 172.21.15.157:0/2245278675 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f94ec155390 con 0x7f94ec150410 2023-10-06T03:57:48.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.200+0000 7f94f0d78700 1 -- 172.21.15.157:0/2245278675 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f94ec155390 con 0x7f94ec150410 2023-10-06T03:57:48.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.200+0000 7f94da7fc700 1 -- 172.21.15.157:0/2245278675 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f94ec155390 con 0x7f94ec150410 2023-10-06T03:57:48.202 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-10-06T03:57:48.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.202+0000 7f94f0d78700 1 -- 172.21.15.157:0/2245278675 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f94cc066f30 msgr2=0x7f94cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:48.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.202+0000 7f94f0d78700 1 --2- 172.21.15.157:0/2245278675 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f94cc066f30 0x7f94cc0693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00d6e0 tx=0x7f94dc007850 comp rx=0 tx=0).stop 2023-10-06T03:57:48.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.202+0000 7f94f0d78700 1 -- 172.21.15.157:0/2245278675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94ec150410 msgr2=0x7f94ec154240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:48.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.202+0000 7f94f0d78700 1 --2- 172.21.15.157:0/2245278675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94ec150410 0x7f94ec154240 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f94ec0c3960 tx=0x7f94e4017000 comp rx=0 tx=0).stop 2023-10-06T03:57:48.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.202+0000 7f94f0d78700 1 -- 172.21.15.157:0/2245278675 shutdown_connections 2023-10-06T03:57:48.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.202+0000 7f94f0d78700 1 --2- 172.21.15.157:0/2245278675 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f94cc066f30 0x7f94cc0693e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:48.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.202+0000 7f94f0d78700 1 --2- 172.21.15.157:0/2245278675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94ec06c1f0 0x7f94ec156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:48.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.202+0000 7f94f0d78700 1 --2- 172.21.15.157:0/2245278675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94ec150410 0x7f94ec154240 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:48.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.202+0000 7f94f0d78700 1 --2- 172.21.15.157:0/2245278675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94ec06cbb0 0x7f94ec14d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:48.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.202+0000 7f94f0d78700 1 -- 172.21.15.157:0/2245278675 >> 172.21.15.157:0/2245278675 conn(0x7f94ec06f230 msgr2=0x7f94ec0c9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:48.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.202+0000 7f94f0d78700 1 -- 172.21.15.157:0/2245278675 shutdown_connections 2023-10-06T03:57:48.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.202+0000 7f94f0d78700 1 -- 172.21.15.157:0/2245278675 wait complete. 2023-10-06T03:57:48.216 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:57:48.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.331+0000 7f4c7a7b4700 1 Processor -- start 2023-10-06T03:57:48.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.331+0000 7f4c7a7b4700 1 -- start start 2023-10-06T03:57:48.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.332+0000 7f4c7a7b4700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4c7406dc40 0x7f4c7414fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:48.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.332+0000 7f4c7a7b4700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4c7406c1f0 0x7f4c74150240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:48.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.332+0000 7f4c7a7b4700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c740c9780 con 0x7f4c7406cbb0 2023-10-06T03:57:48.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.332+0000 7f4c7a7b4700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c740c8c40 con 0x7f4c7406dc40 2023-10-06T03:57:48.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.332+0000 7f4c7a7b4700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c7406e0c0 con 0x7f4c7406c1f0 2023-10-06T03:57:48.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.332+0000 7f4c737fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4c7406dc40 0x7f4c7414fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:48.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.332+0000 7f4c73fff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f4c7406cbb0 0x7f4c7406d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40818/0 (socket says 172.21.15.157:40818) 2023-10-06T03:57:48.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.332+0000 7f4c737fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4c7406dc40 0x7f4c7414fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45174/0 (socket says 172.21.15.157:45174) 2023-10-06T03:57:48.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.332+0000 7f4c73fff700 1 -- 172.21.15.157:0/3932363425 learned_addr learned my addr 172.21.15.157:0/3932363425 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:48.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.332+0000 7f4c78d51700 1 --2- 172.21.15.157:0/3932363425 >> v2:172.21.15.175:3300/0 conn(0x7f4c7406c1f0 0x7f4c74150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:48.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.332+0000 7f4c737fe700 1 -- 172.21.15.157:0/3932363425 >> v2:172.21.15.175:3300/0 conn(0x7f4c7406c1f0 msgr2=0x7f4c74150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:48.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.332+0000 7f4c737fe700 1 --2- 172.21.15.157:0/3932363425 >> v2:172.21.15.175:3300/0 conn(0x7f4c7406c1f0 0x7f4c74150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:48.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.332+0000 7f4c737fe700 1 -- 172.21.15.157:0/3932363425 >> v1:172.21.15.157:6789/0 conn(0x7f4c7406cbb0 legacy=0x7f4c7406d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:48.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.332+0000 7f4c737fe700 1 -- 172.21.15.157:0/3932363425 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c740c8e50 con 0x7f4c7406dc40 2023-10-06T03:57:48.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.332+0000 7f4c72ffd700 1 -- 172.21.15.157:0/3932363425 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4007943869 0 0) 0x7f4c740c9780 con 0x7f4c7406cbb0 2023-10-06T03:57:48.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.333+0000 7f4c737fe700 1 --2- 172.21.15.157:0/3932363425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c7406dc40 0x7f4c7414fd00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4c740c96f0 tx=0x7f4c680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28eca631ed744dbb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:48.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.333+0000 7f4c72ffd700 1 -- 172.21.15.157:0/3932363425 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6800b690 con 0x7f4c7406dc40 2023-10-06T03:57:48.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.333+0000 7f4c72ffd700 1 -- 172.21.15.157:0/3932363425 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6800b7f0 con 0x7f4c7406dc40 2023-10-06T03:57:48.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.333+0000 7f4c72ffd700 1 -- 172.21.15.157:0/3932363425 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6800bb90 con 0x7f4c7406dc40 2023-10-06T03:57:48.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.333+0000 7f4c7a7b4700 1 -- 172.21.15.157:0/3932363425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c7406dc40 msgr2=0x7f4c7414fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:48.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.333+0000 7f4c7a7b4700 1 --2- 172.21.15.157:0/3932363425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c7406dc40 0x7f4c7414fd00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4c740c96f0 tx=0x7f4c680091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:48.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.333+0000 7f4c7a7b4700 1 -- 172.21.15.157:0/3932363425 shutdown_connections 2023-10-06T03:57:48.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.333+0000 7f4c7a7b4700 1 --2- 172.21.15.157:0/3932363425 >> v2:172.21.15.175:3300/0 conn(0x7f4c7406c1f0 0x7f4c74150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:48.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.333+0000 7f4c7a7b4700 1 --2- 172.21.15.157:0/3932363425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c7406dc40 0x7f4c7414fd00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:48.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.333+0000 7f4c7a7b4700 1 -- 172.21.15.157:0/3932363425 >> 172.21.15.157:0/3932363425 conn(0x7f4c7406f230 msgr2=0x7f4c7406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:48.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.333+0000 7f4c7a7b4700 1 -- 172.21.15.157:0/3932363425 shutdown_connections 2023-10-06T03:57:48.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.333+0000 7f4c7a7b4700 1 -- 172.21.15.157:0/3932363425 wait complete. 2023-10-06T03:57:48.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.334+0000 7f4c7a7b4700 1 Processor -- start 2023-10-06T03:57:48.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.335+0000 7f4c7a7b4700 1 -- start start 2023-10-06T03:57:48.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.335+0000 7f4c7a7b4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c7406c1f0 0x7f4c7414fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:48.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.335+0000 7f4c7a7b4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c7406cbb0 0x7f4c741520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:48.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.335+0000 7f4c7a7b4700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c74147bf0 0x7f4c74152600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:48.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.335+0000 7f4c7a7b4700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c74146950 con 0x7f4c7406c1f0 2023-10-06T03:57:48.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.335+0000 7f4c7a7b4700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c74146fc0 con 0x7f4c7406cbb0 2023-10-06T03:57:48.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.335+0000 7f4c7a7b4700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c74145cd0 con 0x7f4c74147bf0 2023-10-06T03:57:48.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.335+0000 7f4c73fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c7406c1f0 0x7f4c7414fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:48.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.335+0000 7f4c73fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c7406c1f0 0x7f4c7414fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45188/0 (socket says 172.21.15.157:45188) 2023-10-06T03:57:48.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.335+0000 7f4c73fff700 1 -- 172.21.15.157:0/3581628026 learned_addr learned my addr 172.21.15.157:0/3581628026 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:48.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.336+0000 7f4c737fe700 1 --2- 172.21.15.157:0/3581628026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c7406cbb0 0x7f4c741520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:48.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.336+0000 7f4c78d51700 1 --2- 172.21.15.157:0/3581628026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c74147bf0 0x7f4c74152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:48.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.336+0000 7f4c73fff700 1 -- 172.21.15.157:0/3581628026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c74147bf0 msgr2=0x7f4c74152600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:48.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.336+0000 7f4c73fff700 1 --2- 172.21.15.157:0/3581628026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c74147bf0 0x7f4c74152600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:48.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.336+0000 7f4c73fff700 1 -- 172.21.15.157:0/3581628026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c7406cbb0 msgr2=0x7f4c741520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:48.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.336+0000 7f4c73fff700 1 --2- 172.21.15.157:0/3581628026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c7406cbb0 0x7f4c741520c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:48.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.336+0000 7f4c73fff700 1 -- 172.21.15.157:0/3581628026 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c68009420 con 0x7f4c7406c1f0 2023-10-06T03:57:48.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.336+0000 7f4c73fff700 1 --2- 172.21.15.157:0/3581628026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c7406c1f0 0x7f4c7414fd20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4c6400c7e0 tx=0x7f4c64007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:48.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.336+0000 7f4c5bfff700 1 -- 172.21.15.157:0/3581628026 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64005b30 con 0x7f4c7406c1f0 2023-10-06T03:57:48.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.336+0000 7f4c5bfff700 1 -- 172.21.15.157:0/3581628026 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c64005c90 con 0x7f4c7406c1f0 2023-10-06T03:57:48.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.336+0000 7f4c7a7b4700 1 -- 172.21.15.157:0/3581628026 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c74145fb0 con 0x7f4c7406c1f0 2023-10-06T03:57:48.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.336+0000 7f4c5bfff700 1 -- 172.21.15.157:0/3581628026 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64007300 con 0x7f4c7406c1f0 2023-10-06T03:57:48.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.336+0000 7f4c7a7b4700 1 -- 172.21.15.157:0/3581628026 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c74146570 con 0x7f4c7406c1f0 2023-10-06T03:57:48.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.337+0000 7f4c7a7b4700 1 -- 172.21.15.157:0/3581628026 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c74089ca0 con 0x7f4c7406c1f0 2023-10-06T03:57:48.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.337+0000 7f4c5bfff700 1 -- 172.21.15.157:0/3581628026 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4c6401b400 con 0x7f4c7406c1f0 2023-10-06T03:57:48.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.338+0000 7f4c5bfff700 1 --2- 172.21.15.157:0/3581628026 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4c60066f30 0x7f4c600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:48.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.338+0000 7f4c5bfff700 1 -- 172.21.15.157:0/3581628026 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4c64084210 con 0x7f4c7406c1f0 2023-10-06T03:57:48.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.338+0000 7f4c737fe700 1 --2- 172.21.15.157:0/3581628026 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4c60066f30 0x7f4c600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:48.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.338+0000 7f4c737fe700 1 --2- 172.21.15.157:0/3581628026 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4c60066f30 0x7f4c600693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4c680093f0 tx=0x7f4c68017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:48.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.342+0000 7f4c5bfff700 1 -- 172.21.15.157:0/3581628026 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c640513a0 con 0x7f4c7406c1f0 2023-10-06T03:57:48.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.524+0000 7f4c7a7b4700 1 -- 172.21.15.157:0/3581628026 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4c740c9130 con 0x7f4c7406c1f0 2023-10-06T03:57:48.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.525+0000 7f4c5bfff700 1 -- 172.21.15.157:0/3581628026 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4c640549c0 con 0x7f4c7406c1f0 2023-10-06T03:57:48.526 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-10-06T03:57:48.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.526+0000 7f4c7a7b4700 1 -- 172.21.15.157:0/3581628026 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4c60066f30 msgr2=0x7f4c600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:48.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.526+0000 7f4c7a7b4700 1 --2- 172.21.15.157:0/3581628026 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4c60066f30 0x7f4c600693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4c680093f0 tx=0x7f4c68017000 comp rx=0 tx=0).stop 2023-10-06T03:57:48.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.526+0000 7f4c7a7b4700 1 -- 172.21.15.157:0/3581628026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c7406c1f0 msgr2=0x7f4c7414fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:48.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.526+0000 7f4c7a7b4700 1 --2- 172.21.15.157:0/3581628026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c7406c1f0 0x7f4c7414fd20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4c6400c7e0 tx=0x7f4c64007df0 comp rx=0 tx=0).stop 2023-10-06T03:57:48.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.526+0000 7f4c7a7b4700 1 -- 172.21.15.157:0/3581628026 shutdown_connections 2023-10-06T03:57:48.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.526+0000 7f4c7a7b4700 1 --2- 172.21.15.157:0/3581628026 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4c60066f30 0x7f4c600693e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:48.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.526+0000 7f4c7a7b4700 1 --2- 172.21.15.157:0/3581628026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4c74147bf0 0x7f4c74152600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:48.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.526+0000 7f4c7a7b4700 1 --2- 172.21.15.157:0/3581628026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c7406c1f0 0x7f4c7414fd20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:48.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.526+0000 7f4c7a7b4700 1 --2- 172.21.15.157:0/3581628026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4c7406cbb0 0x7f4c741520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:48.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.527+0000 7f4c7a7b4700 1 -- 172.21.15.157:0/3581628026 >> 172.21.15.157:0/3581628026 conn(0x7f4c7406f230 msgr2=0x7f4c740affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:48.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.527+0000 7f4c7a7b4700 1 -- 172.21.15.157:0/3581628026 shutdown_connections 2023-10-06T03:57:48.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.527+0000 7f4c7a7b4700 1 -- 172.21.15.157:0/3581628026 wait complete. 2023-10-06T03:57:48.540 INFO:tasks.mds_thrash:Wait for all MDSs to reach steady state... 2023-10-06T03:57:48.540 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:57:48.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.653+0000 7f5c50e57700 1 Processor -- start 2023-10-06T03:57:48.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.653+0000 7f5c50e57700 1 -- start start 2023-10-06T03:57:48.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.654+0000 7f5c50e57700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c4c0b1e40 0x7f5c4c157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:48.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.654+0000 7f5c50e57700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5c4c06c2d0 0x7f5c4c06dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:48.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.654+0000 7f5c50e57700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c4c06d8d0 con 0x7f5c4c0b04d0 2023-10-06T03:57:48.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.654+0000 7f5c50e57700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c4c0af8f0 con 0x7f5c4c06c2d0 2023-10-06T03:57:48.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.654+0000 7f5c50e57700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c15a690 con 0x7f5c4c0b1e40 2023-10-06T03:57:48.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.654+0000 7f5c4bfff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5c4c06c2d0 0x7f5c4c06dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:48.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.654+0000 7f5c4bfff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5c4c06c2d0 0x7f5c4c06dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45196/0 (socket says 172.21.15.157:45196) 2023-10-06T03:57:48.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.654+0000 7f5c4bfff700 1 -- 172.21.15.157:0/2594660504 learned_addr learned my addr 172.21.15.157:0/2594660504 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:48.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.654+0000 7f5c4b7fe700 1 --2- 172.21.15.157:0/2594660504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c4c0b1e40 0x7f5c4c157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:48.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.654+0000 7f5c4bfff700 1 -- 172.21.15.157:0/2594660504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c4c0b1e40 msgr2=0x7f5c4c157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:48.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.654+0000 7f5c4bfff700 1 --2- 172.21.15.157:0/2594660504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c4c0b1e40 0x7f5c4c157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:48.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.654+0000 7f5c4bfff700 1 -- 172.21.15.157:0/2594660504 >> v1:172.21.15.175:6789/0 conn(0x7f5c4c0b04d0 legacy=0x7f5c4c1563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:48.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.654+0000 7f5c4a7fc700 1 -- 172.21.15.157:0/2594660504 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 141929248 0 0) 0x7f5c4c06d8d0 con 0x7f5c4c0b04d0 2023-10-06T03:57:48.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.654+0000 7f5c4bfff700 1 -- 172.21.15.157:0/2594660504 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4c06ccc0 con 0x7f5c4c06c2d0 2023-10-06T03:57:48.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.655+0000 7f5c4bfff700 1 --2- 172.21.15.157:0/2594660504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4c06c2d0 0x7f5c4c06dd60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5c440099b0 tx=0x7f5c44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=889430994fb6aedd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:48.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.655+0000 7f5c4a7fc700 1 -- 172.21.15.157:0/2594660504 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c44009ea0 con 0x7f5c4c06c2d0 2023-10-06T03:57:48.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.655+0000 7f5c4a7fc700 1 -- 172.21.15.157:0/2594660504 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c4400b860 con 0x7f5c4c06c2d0 2023-10-06T03:57:48.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.655+0000 7f5c4a7fc700 1 -- 172.21.15.157:0/2594660504 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4400bc20 con 0x7f5c4c06c2d0 2023-10-06T03:57:48.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.655+0000 7f5c50e57700 1 -- 172.21.15.157:0/2594660504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4c06c2d0 msgr2=0x7f5c4c06dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:48.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.655+0000 7f5c50e57700 1 --2- 172.21.15.157:0/2594660504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4c06c2d0 0x7f5c4c06dd60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5c440099b0 tx=0x7f5c44009250 comp rx=0 tx=0).stop 2023-10-06T03:57:48.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.655+0000 7f5c50e57700 1 -- 172.21.15.157:0/2594660504 shutdown_connections 2023-10-06T03:57:48.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.655+0000 7f5c50e57700 1 --2- 172.21.15.157:0/2594660504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4c06c2d0 0x7f5c4c06dd60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:48.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.655+0000 7f5c50e57700 1 --2- 172.21.15.157:0/2594660504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c4c0b1e40 0x7f5c4c157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:48.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.655+0000 7f5c50e57700 1 -- 172.21.15.157:0/2594660504 >> 172.21.15.157:0/2594660504 conn(0x7f5c4c0cc540 msgr2=0x7f5c4c0cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:48.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.655+0000 7f5c50e57700 1 -- 172.21.15.157:0/2594660504 shutdown_connections 2023-10-06T03:57:48.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.655+0000 7f5c50e57700 1 -- 172.21.15.157:0/2594660504 wait complete. 2023-10-06T03:57:48.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.657+0000 7f5c50e57700 1 Processor -- start 2023-10-06T03:57:48.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.657+0000 7f5c50e57700 1 -- start start 2023-10-06T03:57:48.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.657+0000 7f5c50e57700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4c06c2d0 0x7f5c4c166600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:48.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.657+0000 7f5c50e57700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c4c0b04d0 0x7f5c4c166b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:48.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.657+0000 7f5c50e57700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c4c0b1e40 0x7f5c4c152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:48.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.657+0000 7f5c50e57700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c151010 con 0x7f5c4c06c2d0 2023-10-06T03:57:48.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.657+0000 7f5c50e57700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c0c80c0 con 0x7f5c4c0b04d0 2023-10-06T03:57:48.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.657+0000 7f5c50e57700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c166100 con 0x7f5c4c0b1e40 2023-10-06T03:57:48.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.657+0000 7f5c4b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4c06c2d0 0x7f5c4c166600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:48.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.657+0000 7f5c4b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4c06c2d0 0x7f5c4c166600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45200/0 (socket says 172.21.15.157:45200) 2023-10-06T03:57:48.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.657+0000 7f5c4b7fe700 1 -- 172.21.15.157:0/3340612851 learned_addr learned my addr 172.21.15.157:0/3340612851 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:48.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.657+0000 7f5c4bfff700 1 --2- 172.21.15.157:0/3340612851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c4c0b1e40 0x7f5c4c152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:48.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.657+0000 7f5c4affd700 1 --2- 172.21.15.157:0/3340612851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c4c0b04d0 0x7f5c4c166b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:48.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.658+0000 7f5c4b7fe700 1 -- 172.21.15.157:0/3340612851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c4c0b1e40 msgr2=0x7f5c4c152010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:48.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.658+0000 7f5c4b7fe700 1 --2- 172.21.15.157:0/3340612851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c4c0b1e40 0x7f5c4c152010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:48.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.658+0000 7f5c4b7fe700 1 -- 172.21.15.157:0/3340612851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c4c0b04d0 msgr2=0x7f5c4c166b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:48.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.658+0000 7f5c4b7fe700 1 --2- 172.21.15.157:0/3340612851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c4c0b04d0 0x7f5c4c166b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:48.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.658+0000 7f5c4b7fe700 1 -- 172.21.15.157:0/3340612851 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c440094f0 con 0x7f5c4c06c2d0 2023-10-06T03:57:48.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.658+0000 7f5c4b7fe700 1 --2- 172.21.15.157:0/3340612851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4c06c2d0 0x7f5c4c166600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c009ef0 tx=0x7f5c3c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:48.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.658+0000 7f5c337fe700 1 -- 172.21.15.157:0/3340612851 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c00cc00 con 0x7f5c4c06c2d0 2023-10-06T03:57:48.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.658+0000 7f5c337fe700 1 -- 172.21.15.157:0/3340612851 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3c00f040 con 0x7f5c4c06c2d0 2023-10-06T03:57:48.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.658+0000 7f5c50e57700 1 -- 172.21.15.157:0/3340612851 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c4c0cccc0 con 0x7f5c4c06c2d0 2023-10-06T03:57:48.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.658+0000 7f5c337fe700 1 -- 172.21.15.157:0/3340612851 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c014480 con 0x7f5c4c06c2d0 2023-10-06T03:57:48.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.658+0000 7f5c50e57700 1 -- 172.21.15.157:0/3340612851 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c4c150d30 con 0x7f5c4c06c2d0 2023-10-06T03:57:48.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.659+0000 7f5c50e57700 1 -- 172.21.15.157:0/3340612851 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c4c166450 con 0x7f5c4c06c2d0 2023-10-06T03:57:48.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.659+0000 7f5c337fe700 1 -- 172.21.15.157:0/3340612851 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5c3c0147f0 con 0x7f5c4c06c2d0 2023-10-06T03:57:48.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.660+0000 7f5c337fe700 1 --2- 172.21.15.157:0/3340612851 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5c38066f60 0x7f5c38069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:48.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.660+0000 7f5c337fe700 1 -- 172.21.15.157:0/3340612851 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5c3c084fe0 con 0x7f5c4c06c2d0 2023-10-06T03:57:48.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.660+0000 7f5c4affd700 1 --2- 172.21.15.157:0/3340612851 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5c38066f60 0x7f5c38069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:48.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.660+0000 7f5c4affd700 1 --2- 172.21.15.157:0/3340612851 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5c38066f60 0x7f5c38069410 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c0c58c0 tx=0x7f5c40007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:48.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.664+0000 7f5c337fe700 1 -- 172.21.15.157:0/3340612851 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c3c0520f0 con 0x7f5c4c06c2d0 2023-10-06T03:57:48.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.892+0000 7f5c50e57700 1 -- 172.21.15.157:0/3340612851 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c4c0b0340 con 0x7f5c4c06c2d0 2023-10-06T03:57:48.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.892+0000 7f5c337fe700 1 -- 172.21.15.157:0/3340612851 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+6184 (secure 0 0 0) 0x7f5c3c055710 con 0x7f5c4c06c2d0 2023-10-06T03:57:48.894 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:48.894 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T03:57:37.649946+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"f","rank":4,"incarnation":15,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/4177493830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4177493830},{"type":"v1","addr":"172.21.15.175:6839","nonce":4177493830}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:57:48.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.894+0000 7f5c50e57700 1 -- 172.21.15.157:0/3340612851 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5c38066f60 msgr2=0x7f5c38069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:48.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.894+0000 7f5c50e57700 1 --2- 172.21.15.157:0/3340612851 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5c38066f60 0x7f5c38069410 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c0c58c0 tx=0x7f5c40007d30 comp rx=0 tx=0).stop 2023-10-06T03:57:48.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.894+0000 7f5c50e57700 1 -- 172.21.15.157:0/3340612851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4c06c2d0 msgr2=0x7f5c4c166600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:48.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.894+0000 7f5c50e57700 1 --2- 172.21.15.157:0/3340612851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4c06c2d0 0x7f5c4c166600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c009ef0 tx=0x7f5c3c00b040 comp rx=0 tx=0).stop 2023-10-06T03:57:48.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.894+0000 7f5c50e57700 1 -- 172.21.15.157:0/3340612851 shutdown_connections 2023-10-06T03:57:48.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.894+0000 7f5c50e57700 1 --2- 172.21.15.157:0/3340612851 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5c38066f60 0x7f5c38069410 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:48.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.894+0000 7f5c50e57700 1 --2- 172.21.15.157:0/3340612851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c4c0b1e40 0x7f5c4c152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:48.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.894+0000 7f5c50e57700 1 --2- 172.21.15.157:0/3340612851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4c06c2d0 0x7f5c4c166600 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:48.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.894+0000 7f5c50e57700 1 --2- 172.21.15.157:0/3340612851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c4c0b04d0 0x7f5c4c166b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:48.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.894+0000 7f5c50e57700 1 -- 172.21.15.157:0/3340612851 >> 172.21.15.157:0/3340612851 conn(0x7f5c4c0cc540 msgr2=0x7f5c4c157440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:48.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.894+0000 7f5c50e57700 1 -- 172.21.15.157:0/3340612851 shutdown_connections 2023-10-06T03:57:48.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:48.894+0000 7f5c50e57700 1 -- 172.21.15.157:0/3340612851 wait complete. 2023-10-06T03:57:48.897 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 16 2023-10-06T03:57:48.908 INFO:tasks.mds_thrash:Ready to start thrashing 2023-10-06T03:57:48.908 INFO:tasks.mds_thrash.ceph_manager:waiting for clean 2023-10-06T03:57:48.908 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:57:49.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.021+0000 7f85ff4b7700 1 Processor -- start 2023-10-06T03:57:49.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.021+0000 7f85ff4b7700 1 -- start start 2023-10-06T03:57:49.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.022+0000 7f85ff4b7700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f85f806cbb0 0x7f85f806d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:49.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.022+0000 7f85ff4b7700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f806c1f0 0x7f85f806da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:49.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.022+0000 7f85ff4b7700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85f80c9780 con 0x7f85f806cf80 2023-10-06T03:57:49.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.022+0000 7f85ff4b7700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85f80c8c40 con 0x7f85f806cbb0 2023-10-06T03:57:49.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.022+0000 7f85ff4b7700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f806e0c0 con 0x7f85f806c1f0 2023-10-06T03:57:49.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.023+0000 7f85fd253700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f85f806cbb0 0x7f85f806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:49.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.023+0000 7f85fca52700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f806c1f0 0x7f85f806da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:49.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.023+0000 7f85fd253700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f85f806cbb0 0x7f85f806d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:59832/0 (socket says 172.21.15.157:59832) 2023-10-06T03:57:49.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.023+0000 7f85fda54700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f85f806cf80 0x7f85f814fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.157:48196/0 (socket says 172.21.15.157:48196) 2023-10-06T03:57:49.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.023+0000 7f85fd253700 1 -- 172.21.15.157:0/2554496372 learned_addr learned my addr 172.21.15.157:0/2554496372 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:49.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.023+0000 7f85fca52700 1 -- 172.21.15.157:0/2554496372 >> v2:172.21.15.175:3300/0 conn(0x7f85f806cbb0 msgr2=0x7f85f806d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:49.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.023+0000 7f85fca52700 1 --2- 172.21.15.157:0/2554496372 >> v2:172.21.15.175:3300/0 conn(0x7f85f806cbb0 0x7f85f806d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:49.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.023+0000 7f85fca52700 1 -- 172.21.15.157:0/2554496372 >> v1:172.21.15.175:6789/0 conn(0x7f85f806cf80 legacy=0x7f85f814fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:49.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.023+0000 7f85effff700 1 -- 172.21.15.157:0/2554496372 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3712282499 0 0) 0x7f85f80c9780 con 0x7f85f806cf80 2023-10-06T03:57:49.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.023+0000 7f85fca52700 1 -- 172.21.15.157:0/2554496372 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f80c8e50 con 0x7f85f806c1f0 2023-10-06T03:57:49.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.023+0000 7f85fd253700 1 --2- 172.21.15.157:0/2554496372 >> v2:172.21.15.175:3300/0 conn(0x7f85f806cbb0 0x7f85f806d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:57:49.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.023+0000 7f85fca52700 1 --2- 172.21.15.157:0/2554496372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f806c1f0 0x7f85f806da70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009910 tx=0x7f85e80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=495d41be6d9ace6b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:49.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.023+0000 7f85effff700 1 -- 172.21.15.157:0/2554496372 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e800b6d0 con 0x7f85f806c1f0 2023-10-06T03:57:49.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.023+0000 7f85effff700 1 -- 172.21.15.157:0/2554496372 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e800b830 con 0x7f85f806c1f0 2023-10-06T03:57:49.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.023+0000 7f85effff700 1 -- 172.21.15.157:0/2554496372 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e800bbf0 con 0x7f85f806c1f0 2023-10-06T03:57:49.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.024+0000 7f85ff4b7700 1 -- 172.21.15.157:0/2554496372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f806c1f0 msgr2=0x7f85f806da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:49.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.024+0000 7f85ff4b7700 1 --2- 172.21.15.157:0/2554496372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f806c1f0 0x7f85f806da70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009910 tx=0x7f85e80091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:49.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.024+0000 7f85ff4b7700 1 -- 172.21.15.157:0/2554496372 shutdown_connections 2023-10-06T03:57:49.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.024+0000 7f85ff4b7700 1 --2- 172.21.15.157:0/2554496372 >> v2:172.21.15.175:3300/0 conn(0x7f85f806cbb0 0x7f85f806d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:49.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.024+0000 7f85ff4b7700 1 --2- 172.21.15.157:0/2554496372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f806c1f0 0x7f85f806da70 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:49.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.024+0000 7f85ff4b7700 1 -- 172.21.15.157:0/2554496372 >> 172.21.15.157:0/2554496372 conn(0x7f85f806f230 msgr2=0x7f85f806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:49.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.024+0000 7f85ff4b7700 1 -- 172.21.15.157:0/2554496372 shutdown_connections 2023-10-06T03:57:49.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.024+0000 7f85ff4b7700 1 -- 172.21.15.157:0/2554496372 wait complete. 2023-10-06T03:57:49.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.025+0000 7f85ff4b7700 1 Processor -- start 2023-10-06T03:57:49.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.025+0000 7f85ff4b7700 1 -- start start 2023-10-06T03:57:49.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.026+0000 7f85ff4b7700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f806c1f0 0x7f85f80af9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:49.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.026+0000 7f85ff4b7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85f806cbb0 0x7f85f8146cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:49.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.026+0000 7f85ff4b7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f806cf80 0x7f85f8147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:49.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.026+0000 7f85ff4b7700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f81488a0 con 0x7f85f806cbb0 2023-10-06T03:57:49.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.026+0000 7f85ff4b7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f8148550 con 0x7f85f806cf80 2023-10-06T03:57:49.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.026+0000 7f85ff4b7700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f8166e30 con 0x7f85f806c1f0 2023-10-06T03:57:49.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.026+0000 7f85fca52700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85f806cbb0 0x7f85f8146cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:49.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.026+0000 7f85fca52700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85f806cbb0 0x7f85f8146cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45212/0 (socket says 172.21.15.157:45212) 2023-10-06T03:57:49.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.026+0000 7f85fca52700 1 -- 172.21.15.157:0/4127176243 learned_addr learned my addr 172.21.15.157:0/4127176243 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:49.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.026+0000 7f85fd253700 1 --2- 172.21.15.157:0/4127176243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f806c1f0 0x7f85f80af9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:49.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.026+0000 7f85fda54700 1 --2- 172.21.15.157:0/4127176243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f806cf80 0x7f85f8147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:49.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.026+0000 7f85fca52700 1 -- 172.21.15.157:0/4127176243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f806c1f0 msgr2=0x7f85f80af9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:49.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.026+0000 7f85fca52700 1 --2- 172.21.15.157:0/4127176243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f806c1f0 0x7f85f80af9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:49.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.026+0000 7f85fca52700 1 -- 172.21.15.157:0/4127176243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f806cf80 msgr2=0x7f85f8147210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:49.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.026+0000 7f85fca52700 1 --2- 172.21.15.157:0/4127176243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f806cf80 0x7f85f8147210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:49.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.026+0000 7f85fca52700 1 -- 172.21.15.157:0/4127176243 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f40094f0 con 0x7f85f806cbb0 2023-10-06T03:57:49.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.027+0000 7f85fca52700 1 --2- 172.21.15.157:0/4127176243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85f806cbb0 0x7f85f8146cd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009fb0 tx=0x7f85e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:49.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.027+0000 7f85ed7fa700 1 -- 172.21.15.157:0/4127176243 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e801f070 con 0x7f85f806cbb0 2023-10-06T03:57:49.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.027+0000 7f85ed7fa700 1 -- 172.21.15.157:0/4127176243 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e8000bc0 con 0x7f85f806cbb0 2023-10-06T03:57:49.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.027+0000 7f85ff4b7700 1 -- 172.21.15.157:0/4127176243 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e8009420 con 0x7f85f806cbb0 2023-10-06T03:57:49.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.027+0000 7f85ed7fa700 1 -- 172.21.15.157:0/4127176243 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e800bbf0 con 0x7f85f806cbb0 2023-10-06T03:57:49.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.027+0000 7f85ff4b7700 1 -- 172.21.15.157:0/4127176243 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f80c40c0 con 0x7f85f806cbb0 2023-10-06T03:57:49.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.028+0000 7f85ff4b7700 1 -- 172.21.15.157:0/4127176243 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85f81489e0 con 0x7f85f806cbb0 2023-10-06T03:57:49.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.028+0000 7f85ed7fa700 1 -- 172.21.15.157:0/4127176243 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f85e8023070 con 0x7f85f806cbb0 2023-10-06T03:57:49.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.029+0000 7f85ed7fa700 1 --2- 172.21.15.157:0/4127176243 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f85dc066f30 0x7f85dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:49.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.029+0000 7f85ed7fa700 1 -- 172.21.15.157:0/4127176243 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f85e8085aa0 con 0x7f85f806cbb0 2023-10-06T03:57:49.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.029+0000 7f85fd253700 1 --2- 172.21.15.157:0/4127176243 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f85dc066f30 0x7f85dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:49.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.029+0000 7f85fd253700 1 --2- 172.21.15.157:0/4127176243 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f85dc066f30 0x7f85dc0693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f85f4006d20 tx=0x7f85f4009250 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:49.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.033+0000 7f85ed7fa700 1 -- 172.21.15.157:0/4127176243 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f85f81489e0 con 0x7f85f806cbb0 2023-10-06T03:57:49.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.211+0000 7f85ff4b7700 1 -- 172.21.15.157:0/4127176243 --> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f85f81489e0 con 0x7f85dc066f30 2023-10-06T03:57:49.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.219+0000 7f85ed7fa700 1 -- 172.21.15.157:0/4127176243 <== mgr.4107 v2:172.21.15.175:6832/90661 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286868 (secure 0 0 0) 0x7f85f81489e0 con 0x7f85dc066f30 2023-10-06T03:57:49.224 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:49.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.223+0000 7f85ff4b7700 1 -- 172.21.15.157:0/4127176243 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f85dc066f30 msgr2=0x7f85dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:49.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.223+0000 7f85ff4b7700 1 --2- 172.21.15.157:0/4127176243 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f85dc066f30 0x7f85dc0693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f85f4006d20 tx=0x7f85f4009250 comp rx=0 tx=0).stop 2023-10-06T03:57:49.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.223+0000 7f85ff4b7700 1 -- 172.21.15.157:0/4127176243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85f806cbb0 msgr2=0x7f85f8146cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:49.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.223+0000 7f85ff4b7700 1 --2- 172.21.15.157:0/4127176243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85f806cbb0 0x7f85f8146cd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009fb0 tx=0x7f85e8017000 comp rx=0 tx=0).stop 2023-10-06T03:57:49.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.223+0000 7f85ff4b7700 1 -- 172.21.15.157:0/4127176243 shutdown_connections 2023-10-06T03:57:49.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.223+0000 7f85ff4b7700 1 --2- 172.21.15.157:0/4127176243 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f85dc066f30 0x7f85dc0693e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:49.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.223+0000 7f85ff4b7700 1 --2- 172.21.15.157:0/4127176243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85f806cf80 0x7f85f8147210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:49.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.223+0000 7f85ff4b7700 1 --2- 172.21.15.157:0/4127176243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85f806c1f0 0x7f85f80af9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:49.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.223+0000 7f85ff4b7700 1 --2- 172.21.15.157:0/4127176243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85f806cbb0 0x7f85f8146cd0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:49.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.223+0000 7f85ff4b7700 1 -- 172.21.15.157:0/4127176243 >> 172.21.15.157:0/4127176243 conn(0x7f85f806f230 msgr2=0x7f85f806d530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:49.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.223+0000 7f85ff4b7700 1 -- 172.21.15.157:0/4127176243 shutdown_connections 2023-10-06T03:57:49.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.223+0000 7f85ff4b7700 1 -- 172.21.15.157:0/4127176243 wait complete. 2023-10-06T03:57:49.227 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-10-06T03:57:49.237 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2023-10-06T03:57:47.434780+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597994,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":363,"num_write_kb":2473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":295,"ondisk_log_size":295,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2380380,"kb_used_data":2076,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752594340,"statfs":{"total":773094113280,"available":770656604160,"internally_reserved":0,"allocated":2125824,"data_stored":1825715,"data_compressed":64858,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2660,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003151"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378564+0000","last_change":"2023-10-06T03:57:11.372019+0000","last_active":"2023-10-06T03:57:11.378564+0000","last_peered":"2023-10-06T03:57:11.378564+0000","last_clean":"2023-10-06T03:57:11.378564+0000","last_became_active":"2023-10-06T03:57:11.371756+0000","last_became_peered":"2023-10-06T03:57:11.371756+0000","last_unstale":"2023-10-06T03:57:11.378564+0000","last_undegraded":"2023-10-06T03:57:11.378564+0000","last_fullsized":"2023-10-06T03:57:11.378564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:50.086528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372505+0000","last_change":"2023-10-06T03:57:11.370371+0000","last_active":"2023-10-06T03:57:11.372505+0000","last_peered":"2023-10-06T03:57:11.372505+0000","last_clean":"2023-10-06T03:57:11.372505+0000","last_became_active":"2023-10-06T03:57:11.370070+0000","last_became_peered":"2023-10-06T03:57:11.370070+0000","last_unstale":"2023-10-06T03:57:11.372505+0000","last_undegraded":"2023-10-06T03:57:11.372505+0000","last_fullsized":"2023-10-06T03:57:11.372505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:00.917607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371725+0000","last_change":"2023-10-06T03:57:11.366298+0000","last_active":"2023-10-06T03:57:11.371725+0000","last_peered":"2023-10-06T03:57:11.371725+0000","last_clean":"2023-10-06T03:57:11.371725+0000","last_became_active":"2023-10-06T03:57:11.366012+0000","last_became_peered":"2023-10-06T03:57:11.366012+0000","last_unstale":"2023-10-06T03:57:11.371725+0000","last_undegraded":"2023-10-06T03:57:11.371725+0000","last_fullsized":"2023-10-06T03:57:11.371725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:45.802287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.374334+0000","last_change":"2023-10-06T03:57:11.365933+0000","last_active":"2023-10-06T03:57:11.374334+0000","last_peered":"2023-10-06T03:57:11.374334+0000","last_clean":"2023-10-06T03:57:11.374334+0000","last_became_active":"2023-10-06T03:57:11.365673+0000","last_became_peered":"2023-10-06T03:57:11.365673+0000","last_unstale":"2023-10-06T03:57:11.374334+0000","last_undegraded":"2023-10-06T03:57:11.374334+0000","last_fullsized":"2023-10-06T03:57:11.374334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:40.953382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.365753+0000","last_change":"2023-10-06T03:57:11.361569+0000","last_active":"2023-10-06T03:57:11.365753+0000","last_peered":"2023-10-06T03:57:11.365753+0000","last_clean":"2023-10-06T03:57:11.365753+0000","last_became_active":"2023-10-06T03:57:11.361302+0000","last_became_peered":"2023-10-06T03:57:11.361302+0000","last_unstale":"2023-10-06T03:57:11.365753+0000","last_undegraded":"2023-10-06T03:57:11.365753+0000","last_fullsized":"2023-10-06T03:57:11.365753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:57.552493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372859+0000","last_change":"2023-10-06T03:57:11.369953+0000","last_active":"2023-10-06T03:57:11.372859+0000","last_peered":"2023-10-06T03:57:11.372859+0000","last_clean":"2023-10-06T03:57:11.372859+0000","last_became_active":"2023-10-06T03:57:11.368775+0000","last_became_peered":"2023-10-06T03:57:11.368775+0000","last_unstale":"2023-10-06T03:57:11.372859+0000","last_undegraded":"2023-10-06T03:57:11.372859+0000","last_fullsized":"2023-10-06T03:57:11.372859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:22.368117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.380867+0000","last_change":"2023-10-06T03:57:11.378048+0000","last_active":"2023-10-06T03:57:11.380867+0000","last_peered":"2023-10-06T03:57:11.380867+0000","last_clean":"2023-10-06T03:57:11.380867+0000","last_became_active":"2023-10-06T03:57:11.377786+0000","last_became_peered":"2023-10-06T03:57:11.377786+0000","last_unstale":"2023-10-06T03:57:11.380867+0000","last_undegraded":"2023-10-06T03:57:11.380867+0000","last_fullsized":"2023-10-06T03:57:11.380867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:12:54.218593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377264+0000","last_change":"2023-10-06T03:57:11.374003+0000","last_active":"2023-10-06T03:57:11.377264+0000","last_peered":"2023-10-06T03:57:11.377264+0000","last_clean":"2023-10-06T03:57:11.377264+0000","last_became_active":"2023-10-06T03:57:11.373765+0000","last_became_peered":"2023-10-06T03:57:11.373765+0000","last_unstale":"2023-10-06T03:57:11.377264+0000","last_undegraded":"2023-10-06T03:57:11.377264+0000","last_fullsized":"2023-10-06T03:57:11.377264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:05.838007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378921+0000","last_change":"2023-10-06T03:57:11.373794+0000","last_active":"2023-10-06T03:57:11.378921+0000","last_peered":"2023-10-06T03:57:11.378921+0000","last_clean":"2023-10-06T03:57:11.378921+0000","last_became_active":"2023-10-06T03:57:11.368004+0000","last_became_peered":"2023-10-06T03:57:11.368004+0000","last_unstale":"2023-10-06T03:57:11.378921+0000","last_undegraded":"2023-10-06T03:57:11.378921+0000","last_fullsized":"2023-10-06T03:57:11.378921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:16.760049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.382630+0000","last_change":"2023-10-06T03:57:11.379370+0000","last_active":"2023-10-06T03:57:11.382630+0000","last_peered":"2023-10-06T03:57:11.382630+0000","last_clean":"2023-10-06T03:57:11.382630+0000","last_became_active":"2023-10-06T03:57:11.379107+0000","last_became_peered":"2023-10-06T03:57:11.379107+0000","last_unstale":"2023-10-06T03:57:11.382630+0000","last_undegraded":"2023-10-06T03:57:11.382630+0000","last_fullsized":"2023-10-06T03:57:11.382630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:18.373103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371742+0000","last_change":"2023-10-06T03:57:11.369172+0000","last_active":"2023-10-06T03:57:11.371742+0000","last_peered":"2023-10-06T03:57:11.371742+0000","last_clean":"2023-10-06T03:57:11.371742+0000","last_became_active":"2023-10-06T03:57:11.368256+0000","last_became_peered":"2023-10-06T03:57:11.368256+0000","last_unstale":"2023-10-06T03:57:11.371742+0000","last_undegraded":"2023-10-06T03:57:11.371742+0000","last_fullsized":"2023-10-06T03:57:11.371742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:50.736367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373531+0000","last_change":"2023-10-06T03:57:11.368685+0000","last_active":"2023-10-06T03:57:11.373531+0000","last_peered":"2023-10-06T03:57:11.373531+0000","last_clean":"2023-10-06T03:57:11.373531+0000","last_became_active":"2023-10-06T03:57:11.368425+0000","last_became_peered":"2023-10-06T03:57:11.368425+0000","last_unstale":"2023-10-06T03:57:11.373531+0000","last_undegraded":"2023-10-06T03:57:11.373531+0000","last_fullsized":"2023-10-06T03:57:11.373531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:06.004423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372891+0000","last_change":"2023-10-06T03:57:11.367646+0000","last_active":"2023-10-06T03:57:11.372891+0000","last_peered":"2023-10-06T03:57:11.372891+0000","last_clean":"2023-10-06T03:57:11.372891+0000","last_became_active":"2023-10-06T03:57:11.367079+0000","last_became_peered":"2023-10-06T03:57:11.367079+0000","last_unstale":"2023-10-06T03:57:11.372891+0000","last_undegraded":"2023-10-06T03:57:11.372891+0000","last_fullsized":"2023-10-06T03:57:11.372891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:09:54.331206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377012+0000","last_change":"2023-10-06T03:57:11.373481+0000","last_active":"2023-10-06T03:57:11.377012+0000","last_peered":"2023-10-06T03:57:11.377012+0000","last_clean":"2023-10-06T03:57:11.377012+0000","last_became_active":"2023-10-06T03:57:11.373216+0000","last_became_peered":"2023-10-06T03:57:11.373216+0000","last_unstale":"2023-10-06T03:57:11.377012+0000","last_undegraded":"2023-10-06T03:57:11.377012+0000","last_fullsized":"2023-10-06T03:57:11.377012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:56.083203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.379250+0000","last_change":"2023-10-06T03:57:11.376012+0000","last_active":"2023-10-06T03:57:11.379250+0000","last_peered":"2023-10-06T03:57:11.379250+0000","last_clean":"2023-10-06T03:57:11.379250+0000","last_became_active":"2023-10-06T03:57:11.375689+0000","last_became_peered":"2023-10-06T03:57:11.375689+0000","last_unstale":"2023-10-06T03:57:11.379250+0000","last_undegraded":"2023-10-06T03:57:11.379250+0000","last_fullsized":"2023-10-06T03:57:11.379250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:42.953739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372989+0000","last_change":"2023-10-06T03:57:11.366808+0000","last_active":"2023-10-06T03:57:11.372989+0000","last_peered":"2023-10-06T03:57:11.372989+0000","last_clean":"2023-10-06T03:57:11.372989+0000","last_became_active":"2023-10-06T03:57:11.366543+0000","last_became_peered":"2023-10-06T03:57:11.366543+0000","last_unstale":"2023-10-06T03:57:11.372989+0000","last_undegraded":"2023-10-06T03:57:11.372989+0000","last_fullsized":"2023-10-06T03:57:11.372989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:16.212967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378753+0000","last_change":"2023-10-06T03:57:11.367276+0000","last_active":"2023-10-06T03:57:11.378753+0000","last_peered":"2023-10-06T03:57:11.378753+0000","last_clean":"2023-10-06T03:57:11.378753+0000","last_became_active":"2023-10-06T03:57:11.366767+0000","last_became_peered":"2023-10-06T03:57:11.366767+0000","last_unstale":"2023-10-06T03:57:11.378753+0000","last_undegraded":"2023-10-06T03:57:11.378753+0000","last_fullsized":"2023-10-06T03:57:11.378753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:43.113575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367129+0000","last_change":"2023-10-06T03:57:11.362667+0000","last_active":"2023-10-06T03:57:11.367129+0000","last_peered":"2023-10-06T03:57:11.367129+0000","last_clean":"2023-10-06T03:57:11.367129+0000","last_became_active":"2023-10-06T03:57:11.362377+0000","last_became_peered":"2023-10-06T03:57:11.362377+0000","last_unstale":"2023-10-06T03:57:11.367129+0000","last_undegraded":"2023-10-06T03:57:11.367129+0000","last_fullsized":"2023-10-06T03:57:11.367129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:40.600383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.365567+0000","last_change":"2023-10-06T03:57:11.364126+0000","last_active":"2023-10-06T03:57:11.365567+0000","last_peered":"2023-10-06T03:57:11.365567+0000","last_clean":"2023-10-06T03:57:11.365567+0000","last_became_active":"2023-10-06T03:57:11.363792+0000","last_became_peered":"2023-10-06T03:57:11.363792+0000","last_unstale":"2023-10-06T03:57:11.365567+0000","last_undegraded":"2023-10-06T03:57:11.365567+0000","last_fullsized":"2023-10-06T03:57:11.365567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:49.185450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372624+0000","last_change":"2023-10-06T03:57:11.364974+0000","last_active":"2023-10-06T03:57:11.372624+0000","last_peered":"2023-10-06T03:57:11.372624+0000","last_clean":"2023-10-06T03:57:11.372624+0000","last_became_active":"2023-10-06T03:57:11.364726+0000","last_became_peered":"2023-10-06T03:57:11.364726+0000","last_unstale":"2023-10-06T03:57:11.372624+0000","last_undegraded":"2023-10-06T03:57:11.372624+0000","last_fullsized":"2023-10-06T03:57:11.372624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:56.866733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.379227+0000","last_change":"2023-10-06T03:57:11.375987+0000","last_active":"2023-10-06T03:57:11.379227+0000","last_peered":"2023-10-06T03:57:11.379227+0000","last_clean":"2023-10-06T03:57:11.379227+0000","last_became_active":"2023-10-06T03:57:11.375668+0000","last_became_peered":"2023-10-06T03:57:11.375668+0000","last_unstale":"2023-10-06T03:57:11.379227+0000","last_undegraded":"2023-10-06T03:57:11.379227+0000","last_fullsized":"2023-10-06T03:57:11.379227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:53.810875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.366478+0000","last_change":"2023-10-06T03:57:11.365414+0000","last_active":"2023-10-06T03:57:11.366478+0000","last_peered":"2023-10-06T03:57:11.366478+0000","last_clean":"2023-10-06T03:57:11.366478+0000","last_became_active":"2023-10-06T03:57:11.365149+0000","last_became_peered":"2023-10-06T03:57:11.365149+0000","last_unstale":"2023-10-06T03:57:11.366478+0000","last_undegraded":"2023-10-06T03:57:11.366478+0000","last_fullsized":"2023-10-06T03:57:11.366478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:20.911710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.370638+0000","last_change":"2023-10-06T03:57:11.362366+0000","last_active":"2023-10-06T03:57:11.370638+0000","last_peered":"2023-10-06T03:57:11.370638+0000","last_clean":"2023-10-06T03:57:11.370638+0000","last_became_active":"2023-10-06T03:57:11.362128+0000","last_became_peered":"2023-10-06T03:57:11.362128+0000","last_unstale":"2023-10-06T03:57:11.370638+0000","last_undegraded":"2023-10-06T03:57:11.370638+0000","last_fullsized":"2023-10-06T03:57:11.370638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:53.744674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.379282+0000","last_change":"2023-10-06T03:57:11.375969+0000","last_active":"2023-10-06T03:57:11.379282+0000","last_peered":"2023-10-06T03:57:11.379282+0000","last_clean":"2023-10-06T03:57:11.379282+0000","last_became_active":"2023-10-06T03:57:11.375702+0000","last_became_peered":"2023-10-06T03:57:11.375702+0000","last_unstale":"2023-10-06T03:57:11.379282+0000","last_undegraded":"2023-10-06T03:57:11.379282+0000","last_fullsized":"2023-10-06T03:57:11.379282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:31.204231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377763+0000","last_change":"2023-10-06T03:57:11.375099+0000","last_active":"2023-10-06T03:57:11.377763+0000","last_peered":"2023-10-06T03:57:11.377763+0000","last_clean":"2023-10-06T03:57:11.377763+0000","last_became_active":"2023-10-06T03:57:11.374842+0000","last_became_peered":"2023-10-06T03:57:11.374842+0000","last_unstale":"2023-10-06T03:57:11.377763+0000","last_undegraded":"2023-10-06T03:57:11.377763+0000","last_fullsized":"2023-10-06T03:57:11.377763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:54.438974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.369108+0000","last_change":"2023-10-06T03:57:11.365900+0000","last_active":"2023-10-06T03:57:11.369108+0000","last_peered":"2023-10-06T03:57:11.369108+0000","last_clean":"2023-10-06T03:57:11.369108+0000","last_became_active":"2023-10-06T03:57:11.361349+0000","last_became_peered":"2023-10-06T03:57:11.361349+0000","last_unstale":"2023-10-06T03:57:11.369108+0000","last_undegraded":"2023-10-06T03:57:11.369108+0000","last_fullsized":"2023-10-06T03:57:11.369108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:30.646781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378945+0000","last_change":"2023-10-06T03:57:11.376082+0000","last_active":"2023-10-06T03:57:11.378945+0000","last_peered":"2023-10-06T03:57:11.378945+0000","last_clean":"2023-10-06T03:57:11.378945+0000","last_became_active":"2023-10-06T03:57:11.375846+0000","last_became_peered":"2023-10-06T03:57:11.375846+0000","last_unstale":"2023-10-06T03:57:11.378945+0000","last_undegraded":"2023-10-06T03:57:11.378945+0000","last_fullsized":"2023-10-06T03:57:11.378945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:28.080614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"21'7","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.546542+0000","last_change":"2023-10-06T03:57:10.348317+0000","last_active":"2023-10-06T03:57:32.546542+0000","last_peered":"2023-10-06T03:57:32.546542+0000","last_clean":"2023-10-06T03:57:32.546542+0000","last_became_active":"2023-10-06T03:57:10.348078+0000","last_became_peered":"2023-10-06T03:57:10.348078+0000","last_unstale":"2023-10-06T03:57:32.546542+0000","last_undegraded":"2023-10-06T03:57:32.546542+0000","last_fullsized":"2023-10-06T03:57:32.546542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:55.058324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372610+0000","last_change":"2023-10-06T03:57:11.370506+0000","last_active":"2023-10-06T03:57:11.372610+0000","last_peered":"2023-10-06T03:57:11.372610+0000","last_clean":"2023-10-06T03:57:11.372610+0000","last_became_active":"2023-10-06T03:57:11.370245+0000","last_became_peered":"2023-10-06T03:57:11.370245+0000","last_unstale":"2023-10-06T03:57:11.372610+0000","last_undegraded":"2023-10-06T03:57:11.372610+0000","last_fullsized":"2023-10-06T03:57:11.372610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:47:18.189772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:26.532954+0000","last_change":"2023-10-06T03:57:10.345401+0000","last_active":"2023-10-06T03:57:26.532954+0000","last_peered":"2023-10-06T03:57:26.532954+0000","last_clean":"2023-10-06T03:57:26.532954+0000","last_became_active":"2023-10-06T03:57:10.345150+0000","last_became_peered":"2023-10-06T03:57:10.345150+0000","last_unstale":"2023-10-06T03:57:26.532954+0000","last_undegraded":"2023-10-06T03:57:26.532954+0000","last_fullsized":"2023-10-06T03:57:26.532954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:25.758193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.379135+0000","last_change":"2023-10-06T03:57:11.376046+0000","last_active":"2023-10-06T03:57:11.379135+0000","last_peered":"2023-10-06T03:57:11.379135+0000","last_clean":"2023-10-06T03:57:11.379135+0000","last_became_active":"2023-10-06T03:57:11.375790+0000","last_became_peered":"2023-10-06T03:57:11.375790+0000","last_unstale":"2023-10-06T03:57:11.379135+0000","last_undegraded":"2023-10-06T03:57:11.379135+0000","last_fullsized":"2023-10-06T03:57:11.379135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:33.775431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.540480+0000","last_change":"2023-10-06T03:57:10.347562+0000","last_active":"2023-10-06T03:57:32.540480+0000","last_peered":"2023-10-06T03:57:32.540480+0000","last_clean":"2023-10-06T03:57:32.540480+0000","last_became_active":"2023-10-06T03:57:10.347294+0000","last_became_peered":"2023-10-06T03:57:10.347294+0000","last_unstale":"2023-10-06T03:57:32.540480+0000","last_undegraded":"2023-10-06T03:57:32.540480+0000","last_fullsized":"2023-10-06T03:57:32.540480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:25.090992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.375571+0000","last_change":"2023-10-06T03:57:11.372022+0000","last_active":"2023-10-06T03:57:11.375571+0000","last_peered":"2023-10-06T03:57:11.375571+0000","last_clean":"2023-10-06T03:57:11.375571+0000","last_became_active":"2023-10-06T03:57:11.371758+0000","last_became_peered":"2023-10-06T03:57:11.371758+0000","last_unstale":"2023-10-06T03:57:11.375571+0000","last_undegraded":"2023-10-06T03:57:11.375571+0000","last_fullsized":"2023-10-06T03:57:11.375571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:10.922157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:21.441650+0000","last_change":"2023-10-06T03:57:10.347406+0000","last_active":"2023-10-06T03:57:21.441650+0000","last_peered":"2023-10-06T03:57:21.441650+0000","last_clean":"2023-10-06T03:57:21.441650+0000","last_became_active":"2023-10-06T03:57:10.347151+0000","last_became_peered":"2023-10-06T03:57:10.347151+0000","last_unstale":"2023-10-06T03:57:21.441650+0000","last_undegraded":"2023-10-06T03:57:21.441650+0000","last_fullsized":"2023-10-06T03:57:21.441650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:20.427620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372580+0000","last_change":"2023-10-06T03:57:11.370240+0000","last_active":"2023-10-06T03:57:11.372580+0000","last_peered":"2023-10-06T03:57:11.372580+0000","last_clean":"2023-10-06T03:57:11.372580+0000","last_became_active":"2023-10-06T03:57:11.369938+0000","last_became_peered":"2023-10-06T03:57:11.369938+0000","last_unstale":"2023-10-06T03:57:11.372580+0000","last_undegraded":"2023-10-06T03:57:11.372580+0000","last_fullsized":"2023-10-06T03:57:11.372580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:07.105832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.646156+0000","last_change":"2023-10-06T03:57:10.346054+0000","last_active":"2023-10-06T03:57:37.646156+0000","last_peered":"2023-10-06T03:57:37.646156+0000","last_clean":"2023-10-06T03:57:37.646156+0000","last_became_active":"2023-10-06T03:57:10.345721+0000","last_became_peered":"2023-10-06T03:57:10.345721+0000","last_unstale":"2023-10-06T03:57:37.646156+0000","last_undegraded":"2023-10-06T03:57:37.646156+0000","last_fullsized":"2023-10-06T03:57:37.646156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:54.481304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378702+0000","last_change":"2023-10-06T03:57:11.376525+0000","last_active":"2023-10-06T03:57:11.378702+0000","last_peered":"2023-10-06T03:57:11.378702+0000","last_clean":"2023-10-06T03:57:11.378702+0000","last_became_active":"2023-10-06T03:57:11.376291+0000","last_became_peered":"2023-10-06T03:57:11.376291+0000","last_unstale":"2023-10-06T03:57:11.378702+0000","last_undegraded":"2023-10-06T03:57:11.378702+0000","last_fullsized":"2023-10-06T03:57:11.378702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:31.993426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"21'5","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:21.435357+0000","last_change":"2023-10-06T03:57:10.340531+0000","last_active":"2023-10-06T03:57:21.435357+0000","last_peered":"2023-10-06T03:57:21.435357+0000","last_clean":"2023-10-06T03:57:21.435357+0000","last_became_active":"2023-10-06T03:57:10.340232+0000","last_became_peered":"2023-10-06T03:57:10.340232+0000","last_unstale":"2023-10-06T03:57:21.435357+0000","last_undegraded":"2023-10-06T03:57:21.435357+0000","last_fullsized":"2023-10-06T03:57:21.435357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:43:26.332055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377803+0000","last_change":"2023-10-06T03:57:11.375070+0000","last_active":"2023-10-06T03:57:11.377803+0000","last_peered":"2023-10-06T03:57:11.377803+0000","last_clean":"2023-10-06T03:57:11.377803+0000","last_became_active":"2023-10-06T03:57:11.374892+0000","last_became_peered":"2023-10-06T03:57:11.374892+0000","last_unstale":"2023-10-06T03:57:11.377803+0000","last_undegraded":"2023-10-06T03:57:11.377803+0000","last_fullsized":"2023-10-06T03:57:11.377803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:51.198745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"21'10","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.645490+0000","last_change":"2023-10-06T03:57:10.346758+0000","last_active":"2023-10-06T03:57:37.645490+0000","last_peered":"2023-10-06T03:57:37.645490+0000","last_clean":"2023-10-06T03:57:37.645490+0000","last_became_active":"2023-10-06T03:57:10.346430+0000","last_became_peered":"2023-10-06T03:57:10.346430+0000","last_unstale":"2023-10-06T03:57:37.645490+0000","last_undegraded":"2023-10-06T03:57:37.645490+0000","last_fullsized":"2023-10-06T03:57:37.645490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:10.809242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371658+0000","last_change":"2023-10-06T03:57:11.367808+0000","last_active":"2023-10-06T03:57:11.371658+0000","last_peered":"2023-10-06T03:57:11.371658+0000","last_clean":"2023-10-06T03:57:11.371658+0000","last_became_active":"2023-10-06T03:57:11.367565+0000","last_became_peered":"2023-10-06T03:57:11.367565+0000","last_unstale":"2023-10-06T03:57:11.371658+0000","last_undegraded":"2023-10-06T03:57:11.371658+0000","last_fullsized":"2023-10-06T03:57:11.371658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:14.409116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"21'9","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.646018+0000","last_change":"2023-10-06T03:57:10.351062+0000","last_active":"2023-10-06T03:57:37.646018+0000","last_peered":"2023-10-06T03:57:37.646018+0000","last_clean":"2023-10-06T03:57:37.646018+0000","last_became_active":"2023-10-06T03:57:10.350795+0000","last_became_peered":"2023-10-06T03:57:10.350795+0000","last_unstale":"2023-10-06T03:57:37.646018+0000","last_undegraded":"2023-10-06T03:57:37.646018+0000","last_fullsized":"2023-10-06T03:57:37.646018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:54:45.714518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.370549+0000","last_change":"2023-10-06T03:57:11.367489+0000","last_active":"2023-10-06T03:57:11.370549+0000","last_peered":"2023-10-06T03:57:11.370549+0000","last_clean":"2023-10-06T03:57:11.370549+0000","last_became_active":"2023-10-06T03:57:11.367254+0000","last_became_peered":"2023-10-06T03:57:11.367254+0000","last_unstale":"2023-10-06T03:57:11.370549+0000","last_undegraded":"2023-10-06T03:57:11.370549+0000","last_fullsized":"2023-10-06T03:57:11.370549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:04.706485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"21'5","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.544381+0000","last_change":"2023-10-06T03:57:10.338125+0000","last_active":"2023-10-06T03:57:32.544381+0000","last_peered":"2023-10-06T03:57:32.544381+0000","last_clean":"2023-10-06T03:57:32.544381+0000","last_became_active":"2023-10-06T03:57:10.337798+0000","last_became_peered":"2023-10-06T03:57:10.337798+0000","last_unstale":"2023-10-06T03:57:32.544381+0000","last_undegraded":"2023-10-06T03:57:32.544381+0000","last_fullsized":"2023-10-06T03:57:32.544381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:00.482657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378957+0000","last_change":"2023-10-06T03:57:11.376461+0000","last_active":"2023-10-06T03:57:11.378957+0000","last_peered":"2023-10-06T03:57:11.378957+0000","last_clean":"2023-10-06T03:57:11.378957+0000","last_became_active":"2023-10-06T03:57:11.376223+0000","last_became_peered":"2023-10-06T03:57:11.376223+0000","last_unstale":"2023-10-06T03:57:11.378957+0000","last_undegraded":"2023-10-06T03:57:11.378957+0000","last_fullsized":"2023-10-06T03:57:11.378957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:04:37.408871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"21'5","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.547051+0000","last_change":"2023-10-06T03:57:10.350511+0000","last_active":"2023-10-06T03:57:32.547051+0000","last_peered":"2023-10-06T03:57:32.547051+0000","last_clean":"2023-10-06T03:57:32.547051+0000","last_became_active":"2023-10-06T03:57:10.350219+0000","last_became_peered":"2023-10-06T03:57:10.350219+0000","last_unstale":"2023-10-06T03:57:32.547051+0000","last_undegraded":"2023-10-06T03:57:32.547051+0000","last_fullsized":"2023-10-06T03:57:32.547051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:56.043506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.364939+0000","last_change":"2023-10-06T03:57:11.362700+0000","last_active":"2023-10-06T03:57:11.364939+0000","last_peered":"2023-10-06T03:57:11.364939+0000","last_clean":"2023-10-06T03:57:11.364939+0000","last_became_active":"2023-10-06T03:57:11.362443+0000","last_became_peered":"2023-10-06T03:57:11.362443+0000","last_unstale":"2023-10-06T03:57:11.364939+0000","last_undegraded":"2023-10-06T03:57:11.364939+0000","last_fullsized":"2023-10-06T03:57:11.364939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:21:09.143085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"21'17","reported_seq":27,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.541795+0000","last_change":"2023-10-06T03:57:10.346259+0000","last_active":"2023-10-06T03:57:32.541795+0000","last_peered":"2023-10-06T03:57:32.541795+0000","last_clean":"2023-10-06T03:57:32.541795+0000","last_became_active":"2023-10-06T03:57:10.345424+0000","last_became_peered":"2023-10-06T03:57:10.345424+0000","last_unstale":"2023-10-06T03:57:32.541795+0000","last_undegraded":"2023-10-06T03:57:32.541795+0000","last_fullsized":"2023-10-06T03:57:32.541795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:19.278206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378006+0000","last_change":"2023-10-06T03:57:11.376053+0000","last_active":"2023-10-06T03:57:11.378006+0000","last_peered":"2023-10-06T03:57:11.378006+0000","last_clean":"2023-10-06T03:57:11.378006+0000","last_became_active":"2023-10-06T03:57:11.375824+0000","last_became_peered":"2023-10-06T03:57:11.375824+0000","last_unstale":"2023-10-06T03:57:11.378006+0000","last_undegraded":"2023-10-06T03:57:11.378006+0000","last_fullsized":"2023-10-06T03:57:11.378006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:24:26.156866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:16.374858+0000","last_change":"2023-10-06T03:57:10.350408+0000","last_active":"2023-10-06T03:57:16.374858+0000","last_peered":"2023-10-06T03:57:16.374858+0000","last_clean":"2023-10-06T03:57:16.374858+0000","last_became_active":"2023-10-06T03:57:10.350135+0000","last_became_peered":"2023-10-06T03:57:10.350135+0000","last_unstale":"2023-10-06T03:57:16.374858+0000","last_undegraded":"2023-10-06T03:57:16.374858+0000","last_fullsized":"2023-10-06T03:57:16.374858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:43.522111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378954+0000","last_change":"2023-10-06T03:57:11.376524+0000","last_active":"2023-10-06T03:57:11.378954+0000","last_peered":"2023-10-06T03:57:11.378954+0000","last_clean":"2023-10-06T03:57:11.378954+0000","last_became_active":"2023-10-06T03:57:11.376236+0000","last_became_peered":"2023-10-06T03:57:11.376236+0000","last_unstale":"2023-10-06T03:57:11.378954+0000","last_undegraded":"2023-10-06T03:57:11.378954+0000","last_fullsized":"2023-10-06T03:57:11.378954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:33.854818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"21'5","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.644387+0000","last_change":"2023-10-06T03:57:10.345799+0000","last_active":"2023-10-06T03:57:37.644387+0000","last_peered":"2023-10-06T03:57:37.644387+0000","last_clean":"2023-10-06T03:57:37.644387+0000","last_became_active":"2023-10-06T03:57:10.345510+0000","last_became_peered":"2023-10-06T03:57:10.345510+0000","last_unstale":"2023-10-06T03:57:37.644387+0000","last_undegraded":"2023-10-06T03:57:37.644387+0000","last_fullsized":"2023-10-06T03:57:37.644387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:09.567314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372383+0000","last_change":"2023-10-06T03:57:11.368662+0000","last_active":"2023-10-06T03:57:11.372383+0000","last_peered":"2023-10-06T03:57:11.372383+0000","last_clean":"2023-10-06T03:57:11.372383+0000","last_became_active":"2023-10-06T03:57:11.368375+0000","last_became_peered":"2023-10-06T03:57:11.368375+0000","last_unstale":"2023-10-06T03:57:11.372383+0000","last_undegraded":"2023-10-06T03:57:11.372383+0000","last_fullsized":"2023-10-06T03:57:11.372383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:27.125523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.543509+0000","last_change":"2023-10-06T03:57:10.346945+0000","last_active":"2023-10-06T03:57:32.543509+0000","last_peered":"2023-10-06T03:57:32.543509+0000","last_clean":"2023-10-06T03:57:32.543509+0000","last_became_active":"2023-10-06T03:57:10.346689+0000","last_became_peered":"2023-10-06T03:57:10.346689+0000","last_unstale":"2023-10-06T03:57:32.543509+0000","last_undegraded":"2023-10-06T03:57:32.543509+0000","last_fullsized":"2023-10-06T03:57:32.543509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:43.420455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.375632+0000","last_change":"2023-10-06T03:57:11.372308+0000","last_active":"2023-10-06T03:57:11.375632+0000","last_peered":"2023-10-06T03:57:11.375632+0000","last_clean":"2023-10-06T03:57:11.375632+0000","last_became_active":"2023-10-06T03:57:11.371795+0000","last_became_peered":"2023-10-06T03:57:11.371795+0000","last_unstale":"2023-10-06T03:57:11.375632+0000","last_undegraded":"2023-10-06T03:57:11.375632+0000","last_fullsized":"2023-10-06T03:57:11.375632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:00.324478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"21'5","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.542306+0000","last_change":"2023-10-06T03:57:10.348592+0000","last_active":"2023-10-06T03:57:32.542306+0000","last_peered":"2023-10-06T03:57:32.542306+0000","last_clean":"2023-10-06T03:57:32.542306+0000","last_became_active":"2023-10-06T03:57:10.348291+0000","last_became_peered":"2023-10-06T03:57:10.348291+0000","last_unstale":"2023-10-06T03:57:32.542306+0000","last_undegraded":"2023-10-06T03:57:32.542306+0000","last_fullsized":"2023-10-06T03:57:32.542306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:00.660348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1094,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378672+0000","last_change":"2023-10-06T03:57:11.366510+0000","last_active":"2023-10-06T03:57:11.378672+0000","last_peered":"2023-10-06T03:57:11.378672+0000","last_clean":"2023-10-06T03:57:11.378672+0000","last_became_active":"2023-10-06T03:57:11.365077+0000","last_became_peered":"2023-10-06T03:57:11.365077+0000","last_unstale":"2023-10-06T03:57:11.378672+0000","last_undegraded":"2023-10-06T03:57:11.378672+0000","last_fullsized":"2023-10-06T03:57:11.378672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:25.199391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:21.407239+0000","last_change":"2023-10-06T03:57:10.348418+0000","last_active":"2023-10-06T03:57:21.407239+0000","last_peered":"2023-10-06T03:57:21.407239+0000","last_clean":"2023-10-06T03:57:21.407239+0000","last_became_active":"2023-10-06T03:57:10.348055+0000","last_became_peered":"2023-10-06T03:57:10.348055+0000","last_unstale":"2023-10-06T03:57:21.407239+0000","last_undegraded":"2023-10-06T03:57:21.407239+0000","last_fullsized":"2023-10-06T03:57:21.407239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:26.102022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373629+0000","last_change":"2023-10-06T03:57:11.370493+0000","last_active":"2023-10-06T03:57:11.373629+0000","last_peered":"2023-10-06T03:57:11.373629+0000","last_clean":"2023-10-06T03:57:11.373629+0000","last_became_active":"2023-10-06T03:57:11.370073+0000","last_became_peered":"2023-10-06T03:57:11.370073+0000","last_unstale":"2023-10-06T03:57:11.373629+0000","last_undegraded":"2023-10-06T03:57:11.373629+0000","last_fullsized":"2023-10-06T03:57:11.373629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:58.417951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.544169+0000","last_change":"2023-10-06T03:57:10.352966+0000","last_active":"2023-10-06T03:57:32.544169+0000","last_peered":"2023-10-06T03:57:32.544169+0000","last_clean":"2023-10-06T03:57:32.544169+0000","last_became_active":"2023-10-06T03:57:10.352653+0000","last_became_peered":"2023-10-06T03:57:10.352653+0000","last_unstale":"2023-10-06T03:57:32.544169+0000","last_undegraded":"2023-10-06T03:57:32.544169+0000","last_fullsized":"2023-10-06T03:57:32.544169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:32.092557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371704+0000","last_change":"2023-10-06T03:57:11.364052+0000","last_active":"2023-10-06T03:57:11.371704+0000","last_peered":"2023-10-06T03:57:11.371704+0000","last_clean":"2023-10-06T03:57:11.371704+0000","last_became_active":"2023-10-06T03:57:11.363830+0000","last_became_peered":"2023-10-06T03:57:11.363830+0000","last_unstale":"2023-10-06T03:57:11.371704+0000","last_undegraded":"2023-10-06T03:57:11.371704+0000","last_fullsized":"2023-10-06T03:57:11.371704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:04.084019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"21'7","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.645676+0000","last_change":"2023-10-06T03:57:10.349421+0000","last_active":"2023-10-06T03:57:37.645676+0000","last_peered":"2023-10-06T03:57:37.645676+0000","last_clean":"2023-10-06T03:57:37.645676+0000","last_became_active":"2023-10-06T03:57:10.349114+0000","last_became_peered":"2023-10-06T03:57:10.349114+0000","last_unstale":"2023-10-06T03:57:37.645676+0000","last_undegraded":"2023-10-06T03:57:37.645676+0000","last_fullsized":"2023-10-06T03:57:37.645676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:08.209791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372296+0000","last_change":"2023-10-06T03:57:11.367516+0000","last_active":"2023-10-06T03:57:11.372296+0000","last_peered":"2023-10-06T03:57:11.372296+0000","last_clean":"2023-10-06T03:57:11.372296+0000","last_became_active":"2023-10-06T03:57:11.367257+0000","last_became_peered":"2023-10-06T03:57:11.367257+0000","last_unstale":"2023-10-06T03:57:11.372296+0000","last_undegraded":"2023-10-06T03:57:11.372296+0000","last_fullsized":"2023-10-06T03:57:11.372296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:43.628819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:21.429070+0000","last_change":"2023-10-06T03:57:10.347649+0000","last_active":"2023-10-06T03:57:21.429070+0000","last_peered":"2023-10-06T03:57:21.429070+0000","last_clean":"2023-10-06T03:57:21.429070+0000","last_became_active":"2023-10-06T03:57:10.347433+0000","last_became_peered":"2023-10-06T03:57:10.347433+0000","last_unstale":"2023-10-06T03:57:21.429070+0000","last_undegraded":"2023-10-06T03:57:21.429070+0000","last_fullsized":"2023-10-06T03:57:21.429070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:55.951494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372561+0000","last_change":"2023-10-06T03:57:11.370092+0000","last_active":"2023-10-06T03:57:11.372561+0000","last_peered":"2023-10-06T03:57:11.372561+0000","last_clean":"2023-10-06T03:57:11.372561+0000","last_became_active":"2023-10-06T03:57:11.369825+0000","last_became_peered":"2023-10-06T03:57:11.369825+0000","last_unstale":"2023-10-06T03:57:11.372561+0000","last_undegraded":"2023-10-06T03:57:11.372561+0000","last_fullsized":"2023-10-06T03:57:11.372561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:43.415929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367709+0000","last_change":"2023-10-06T03:57:05.663312+0000","last_active":"2023-10-06T03:57:11.367709+0000","last_peered":"2023-10-06T03:57:11.367709+0000","last_clean":"2023-10-06T03:57:11.367709+0000","last_became_active":"2023-10-06T03:57:05.663046+0000","last_became_peered":"2023-10-06T03:57:05.663046+0000","last_unstale":"2023-10-06T03:57:11.367709+0000","last_undegraded":"2023-10-06T03:57:11.367709+0000","last_fullsized":"2023-10-06T03:57:11.367709+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:31.174359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.542752+0000","last_change":"2023-10-06T03:57:10.348930+0000","last_active":"2023-10-06T03:57:32.542752+0000","last_peered":"2023-10-06T03:57:32.542752+0000","last_clean":"2023-10-06T03:57:32.542752+0000","last_became_active":"2023-10-06T03:57:10.348608+0000","last_became_peered":"2023-10-06T03:57:10.348608+0000","last_unstale":"2023-10-06T03:57:32.542752+0000","last_undegraded":"2023-10-06T03:57:32.542752+0000","last_fullsized":"2023-10-06T03:57:32.542752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:00:09.283718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377785+0000","last_change":"2023-10-06T03:57:11.371108+0000","last_active":"2023-10-06T03:57:11.377785+0000","last_peered":"2023-10-06T03:57:11.377785+0000","last_clean":"2023-10-06T03:57:11.377785+0000","last_became_active":"2023-10-06T03:57:11.370847+0000","last_became_peered":"2023-10-06T03:57:11.370847+0000","last_unstale":"2023-10-06T03:57:11.377785+0000","last_undegraded":"2023-10-06T03:57:11.377785+0000","last_fullsized":"2023-10-06T03:57:11.377785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:38.111171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.362274+0000","last_change":"2023-10-06T03:57:05.318834+0000","last_active":"2023-10-06T03:57:11.362274+0000","last_peered":"2023-10-06T03:57:11.362274+0000","last_clean":"2023-10-06T03:57:11.362274+0000","last_became_active":"2023-10-06T03:57:05.318572+0000","last_became_peered":"2023-10-06T03:57:05.318572+0000","last_unstale":"2023-10-06T03:57:11.362274+0000","last_undegraded":"2023-10-06T03:57:11.362274+0000","last_fullsized":"2023-10-06T03:57:11.362274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:21.951095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"21'6","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.546545+0000","last_change":"2023-10-06T03:57:10.348556+0000","last_active":"2023-10-06T03:57:32.546545+0000","last_peered":"2023-10-06T03:57:32.546545+0000","last_clean":"2023-10-06T03:57:32.546545+0000","last_became_active":"2023-10-06T03:57:10.348271+0000","last_became_peered":"2023-10-06T03:57:10.348271+0000","last_unstale":"2023-10-06T03:57:32.546545+0000","last_undegraded":"2023-10-06T03:57:32.546545+0000","last_fullsized":"2023-10-06T03:57:32.546545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:58.470989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.368920+0000","last_change":"2023-10-06T03:57:11.365237+0000","last_active":"2023-10-06T03:57:11.368920+0000","last_peered":"2023-10-06T03:57:11.368920+0000","last_clean":"2023-10-06T03:57:11.368920+0000","last_became_active":"2023-10-06T03:57:11.365006+0000","last_became_peered":"2023-10-06T03:57:11.365006+0000","last_unstale":"2023-10-06T03:57:11.368920+0000","last_undegraded":"2023-10-06T03:57:11.368920+0000","last_fullsized":"2023-10-06T03:57:11.368920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:33.005015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.355768+0000","last_change":"2023-10-06T03:57:05.314695+0000","last_active":"2023-10-06T03:57:11.355768+0000","last_peered":"2023-10-06T03:57:11.355768+0000","last_clean":"2023-10-06T03:57:11.355768+0000","last_became_active":"2023-10-06T03:57:05.314408+0000","last_became_peered":"2023-10-06T03:57:05.314408+0000","last_unstale":"2023-10-06T03:57:11.355768+0000","last_undegraded":"2023-10-06T03:57:11.355768+0000","last_fullsized":"2023-10-06T03:57:11.355768+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:19.042145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:26.532613+0000","last_change":"2023-10-06T03:57:10.346078+0000","last_active":"2023-10-06T03:57:26.532613+0000","last_peered":"2023-10-06T03:57:26.532613+0000","last_clean":"2023-10-06T03:57:26.532613+0000","last_became_active":"2023-10-06T03:57:10.345769+0000","last_became_peered":"2023-10-06T03:57:10.345769+0000","last_unstale":"2023-10-06T03:57:26.532613+0000","last_undegraded":"2023-10-06T03:57:26.532613+0000","last_fullsized":"2023-10-06T03:57:26.532613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:16.630681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.376583+0000","last_change":"2023-10-06T03:57:11.370372+0000","last_active":"2023-10-06T03:57:11.376583+0000","last_peered":"2023-10-06T03:57:11.376583+0000","last_clean":"2023-10-06T03:57:11.376583+0000","last_became_active":"2023-10-06T03:57:11.370076+0000","last_became_peered":"2023-10-06T03:57:11.370076+0000","last_unstale":"2023-10-06T03:57:11.376583+0000","last_undegraded":"2023-10-06T03:57:11.376583+0000","last_fullsized":"2023-10-06T03:57:11.376583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:18:59.517155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.362848+0000","last_change":"2023-10-06T03:57:05.318778+0000","last_active":"2023-10-06T03:57:11.362848+0000","last_peered":"2023-10-06T03:57:11.362848+0000","last_clean":"2023-10-06T03:57:11.362848+0000","last_became_active":"2023-10-06T03:57:05.318456+0000","last_became_peered":"2023-10-06T03:57:05.318456+0000","last_unstale":"2023-10-06T03:57:11.362848+0000","last_undegraded":"2023-10-06T03:57:11.362848+0000","last_fullsized":"2023-10-06T03:57:11.362848+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:39.769241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.362382+0000","last_change":"2023-10-06T03:57:05.664245+0000","last_active":"2023-10-06T03:57:11.362382+0000","last_peered":"2023-10-06T03:57:11.362382+0000","last_clean":"2023-10-06T03:57:11.362382+0000","last_became_active":"2023-10-06T03:57:05.663426+0000","last_became_peered":"2023-10-06T03:57:05.663426+0000","last_unstale":"2023-10-06T03:57:11.362382+0000","last_undegraded":"2023-10-06T03:57:11.362382+0000","last_fullsized":"2023-10-06T03:57:11.362382+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:29.322599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"21'5","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.646148+0000","last_change":"2023-10-06T03:57:10.351059+0000","last_active":"2023-10-06T03:57:37.646148+0000","last_peered":"2023-10-06T03:57:37.646148+0000","last_clean":"2023-10-06T03:57:37.646148+0000","last_became_active":"2023-10-06T03:57:10.348879+0000","last_became_peered":"2023-10-06T03:57:10.348879+0000","last_unstale":"2023-10-06T03:57:37.646148+0000","last_undegraded":"2023-10-06T03:57:37.646148+0000","last_fullsized":"2023-10-06T03:57:37.646148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:35.764148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.375601+0000","last_change":"2023-10-06T03:57:11.373786+0000","last_active":"2023-10-06T03:57:11.375601+0000","last_peered":"2023-10-06T03:57:11.375601+0000","last_clean":"2023-10-06T03:57:11.375601+0000","last_became_active":"2023-10-06T03:57:11.373452+0000","last_became_peered":"2023-10-06T03:57:11.373452+0000","last_unstale":"2023-10-06T03:57:11.375601+0000","last_undegraded":"2023-10-06T03:57:11.375601+0000","last_fullsized":"2023-10-06T03:57:11.375601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:20.524065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.364987+0000","last_change":"2023-10-06T03:57:05.318790+0000","last_active":"2023-10-06T03:57:11.364987+0000","last_peered":"2023-10-06T03:57:11.364987+0000","last_clean":"2023-10-06T03:57:11.364987+0000","last_became_active":"2023-10-06T03:57:05.318531+0000","last_became_peered":"2023-10-06T03:57:05.318531+0000","last_unstale":"2023-10-06T03:57:11.364987+0000","last_undegraded":"2023-10-06T03:57:11.364987+0000","last_fullsized":"2023-10-06T03:57:11.364987+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:12.876998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"21'5","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:26.534499+0000","last_change":"2023-10-06T03:57:10.343328+0000","last_active":"2023-10-06T03:57:26.534499+0000","last_peered":"2023-10-06T03:57:26.534499+0000","last_clean":"2023-10-06T03:57:26.534499+0000","last_became_active":"2023-10-06T03:57:10.343051+0000","last_became_peered":"2023-10-06T03:57:10.343051+0000","last_unstale":"2023-10-06T03:57:26.534499+0000","last_undegraded":"2023-10-06T03:57:26.534499+0000","last_fullsized":"2023-10-06T03:57:26.534499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:27.722475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378907+0000","last_change":"2023-10-06T03:57:11.373581+0000","last_active":"2023-10-06T03:57:11.378907+0000","last_peered":"2023-10-06T03:57:11.378907+0000","last_clean":"2023-10-06T03:57:11.378907+0000","last_became_active":"2023-10-06T03:57:11.373264+0000","last_became_peered":"2023-10-06T03:57:11.373264+0000","last_unstale":"2023-10-06T03:57:11.378907+0000","last_undegraded":"2023-10-06T03:57:11.378907+0000","last_fullsized":"2023-10-06T03:57:11.378907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:15.950245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.366791+0000","last_change":"2023-10-06T03:57:05.663770+0000","last_active":"2023-10-06T03:57:11.366791+0000","last_peered":"2023-10-06T03:57:11.366791+0000","last_clean":"2023-10-06T03:57:11.366791+0000","last_became_active":"2023-10-06T03:57:05.663501+0000","last_became_peered":"2023-10-06T03:57:05.663501+0000","last_unstale":"2023-10-06T03:57:11.366791+0000","last_undegraded":"2023-10-06T03:57:11.366791+0000","last_fullsized":"2023-10-06T03:57:11.366791+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:09.806915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:12.368542+0000","last_change":"2023-10-06T03:57:10.346585+0000","last_active":"2023-10-06T03:57:12.368542+0000","last_peered":"2023-10-06T03:57:12.368542+0000","last_clean":"2023-10-06T03:57:12.368542+0000","last_became_active":"2023-10-06T03:57:10.345743+0000","last_became_peered":"2023-10-06T03:57:10.345743+0000","last_unstale":"2023-10-06T03:57:12.368542+0000","last_undegraded":"2023-10-06T03:57:12.368542+0000","last_fullsized":"2023-10-06T03:57:12.368542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:54.589702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378948+0000","last_change":"2023-10-06T03:57:11.373844+0000","last_active":"2023-10-06T03:57:11.378948+0000","last_peered":"2023-10-06T03:57:11.378948+0000","last_clean":"2023-10-06T03:57:11.378948+0000","last_became_active":"2023-10-06T03:57:11.373543+0000","last_became_peered":"2023-10-06T03:57:11.373543+0000","last_unstale":"2023-10-06T03:57:11.378948+0000","last_undegraded":"2023-10-06T03:57:11.378948+0000","last_fullsized":"2023-10-06T03:57:11.378948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:15.152354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"21'7","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.645077+0000","last_change":"2023-10-06T03:57:10.344618+0000","last_active":"2023-10-06T03:57:37.645077+0000","last_peered":"2023-10-06T03:57:37.645077+0000","last_clean":"2023-10-06T03:57:37.645077+0000","last_became_active":"2023-10-06T03:57:10.344324+0000","last_became_peered":"2023-10-06T03:57:10.344324+0000","last_unstale":"2023-10-06T03:57:37.645077+0000","last_undegraded":"2023-10-06T03:57:37.645077+0000","last_fullsized":"2023-10-06T03:57:37.645077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:06.062331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.368350+0000","last_change":"2023-10-06T03:57:05.317973+0000","last_active":"2023-10-06T03:57:11.368350+0000","last_peered":"2023-10-06T03:57:11.368350+0000","last_clean":"2023-10-06T03:57:11.368350+0000","last_became_active":"2023-10-06T03:57:05.317715+0000","last_became_peered":"2023-10-06T03:57:05.317715+0000","last_unstale":"2023-10-06T03:57:11.368350+0000","last_undegraded":"2023-10-06T03:57:11.368350+0000","last_fullsized":"2023-10-06T03:57:11.368350+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:37:51.858909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372084+0000","last_change":"2023-10-06T03:57:11.369706+0000","last_active":"2023-10-06T03:57:11.372084+0000","last_peered":"2023-10-06T03:57:11.372084+0000","last_clean":"2023-10-06T03:57:11.372084+0000","last_became_active":"2023-10-06T03:57:11.369445+0000","last_became_peered":"2023-10-06T03:57:11.369445+0000","last_unstale":"2023-10-06T03:57:11.372084+0000","last_undegraded":"2023-10-06T03:57:11.372084+0000","last_fullsized":"2023-10-06T03:57:11.372084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:52.999421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.362125+0000","last_change":"2023-10-06T03:57:05.320847+0000","last_active":"2023-10-06T03:57:11.362125+0000","last_peered":"2023-10-06T03:57:11.362125+0000","last_clean":"2023-10-06T03:57:11.362125+0000","last_became_active":"2023-10-06T03:57:05.320583+0000","last_became_peered":"2023-10-06T03:57:05.320583+0000","last_unstale":"2023-10-06T03:57:11.362125+0000","last_undegraded":"2023-10-06T03:57:11.362125+0000","last_fullsized":"2023-10-06T03:57:11.362125+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:12.772449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"21'7","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.542436+0000","last_change":"2023-10-06T03:57:10.341132+0000","last_active":"2023-10-06T03:57:32.542436+0000","last_peered":"2023-10-06T03:57:32.542436+0000","last_clean":"2023-10-06T03:57:32.542436+0000","last_became_active":"2023-10-06T03:57:10.339424+0000","last_became_peered":"2023-10-06T03:57:10.339424+0000","last_unstale":"2023-10-06T03:57:32.542436+0000","last_undegraded":"2023-10-06T03:57:32.542436+0000","last_fullsized":"2023-10-06T03:57:32.542436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:39.675711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.374372+0000","last_change":"2023-10-06T03:57:11.372497+0000","last_active":"2023-10-06T03:57:11.374372+0000","last_peered":"2023-10-06T03:57:11.374372+0000","last_clean":"2023-10-06T03:57:11.374372+0000","last_became_active":"2023-10-06T03:57:11.372215+0000","last_became_peered":"2023-10-06T03:57:11.372215+0000","last_unstale":"2023-10-06T03:57:11.374372+0000","last_undegraded":"2023-10-06T03:57:11.374372+0000","last_fullsized":"2023-10-06T03:57:11.374372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:23.559480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"21'5","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:21.447011+0000","last_change":"2023-10-06T03:57:10.344051+0000","last_active":"2023-10-06T03:57:21.447011+0000","last_peered":"2023-10-06T03:57:21.447011+0000","last_clean":"2023-10-06T03:57:21.447011+0000","last_became_active":"2023-10-06T03:57:10.343865+0000","last_became_peered":"2023-10-06T03:57:10.343865+0000","last_unstale":"2023-10-06T03:57:21.447011+0000","last_undegraded":"2023-10-06T03:57:21.447011+0000","last_fullsized":"2023-10-06T03:57:21.447011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:32.522470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.376665+0000","last_change":"2023-10-06T03:57:11.369927+0000","last_active":"2023-10-06T03:57:11.376665+0000","last_peered":"2023-10-06T03:57:11.376665+0000","last_clean":"2023-10-06T03:57:11.376665+0000","last_became_active":"2023-10-06T03:57:11.369681+0000","last_became_peered":"2023-10-06T03:57:11.369681+0000","last_unstale":"2023-10-06T03:57:11.376665+0000","last_undegraded":"2023-10-06T03:57:11.376665+0000","last_fullsized":"2023-10-06T03:57:11.376665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:53.858179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"21'5","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.646820+0000","last_change":"2023-10-06T03:57:10.349666+0000","last_active":"2023-10-06T03:57:37.646820+0000","last_peered":"2023-10-06T03:57:37.646820+0000","last_clean":"2023-10-06T03:57:37.646820+0000","last_became_active":"2023-10-06T03:57:10.349320+0000","last_became_peered":"2023-10-06T03:57:10.349320+0000","last_unstale":"2023-10-06T03:57:37.646820+0000","last_undegraded":"2023-10-06T03:57:37.646820+0000","last_fullsized":"2023-10-06T03:57:37.646820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:45:49.709519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373619+0000","last_change":"2023-10-06T03:57:11.370442+0000","last_active":"2023-10-06T03:57:11.373619+0000","last_peered":"2023-10-06T03:57:11.373619+0000","last_clean":"2023-10-06T03:57:11.373619+0000","last_became_active":"2023-10-06T03:57:11.370229+0000","last_became_peered":"2023-10-06T03:57:11.370229+0000","last_unstale":"2023-10-06T03:57:11.373619+0000","last_undegraded":"2023-10-06T03:57:11.373619+0000","last_fullsized":"2023-10-06T03:57:11.373619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:26.174331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"21'11","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.646774+0000","last_change":"2023-10-06T03:57:10.350263+0000","last_active":"2023-10-06T03:57:37.646774+0000","last_peered":"2023-10-06T03:57:37.646774+0000","last_clean":"2023-10-06T03:57:37.646774+0000","last_became_active":"2023-10-06T03:57:10.350083+0000","last_became_peered":"2023-10-06T03:57:10.350083+0000","last_unstale":"2023-10-06T03:57:37.646774+0000","last_undegraded":"2023-10-06T03:57:37.646774+0000","last_fullsized":"2023-10-06T03:57:37.646774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:05.433032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372850+0000","last_change":"2023-10-06T03:57:11.369397+0000","last_active":"2023-10-06T03:57:11.372850+0000","last_peered":"2023-10-06T03:57:11.372850+0000","last_clean":"2023-10-06T03:57:11.372850+0000","last_became_active":"2023-10-06T03:57:11.369123+0000","last_became_peered":"2023-10-06T03:57:11.369123+0000","last_unstale":"2023-10-06T03:57:11.372850+0000","last_undegraded":"2023-10-06T03:57:11.372850+0000","last_fullsized":"2023-10-06T03:57:11.372850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:28.139036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:16.370129+0000","last_change":"2023-10-06T03:57:10.347049+0000","last_active":"2023-10-06T03:57:16.370129+0000","last_peered":"2023-10-06T03:57:16.370129+0000","last_clean":"2023-10-06T03:57:16.370129+0000","last_became_active":"2023-10-06T03:57:10.344583+0000","last_became_peered":"2023-10-06T03:57:10.344583+0000","last_unstale":"2023-10-06T03:57:16.370129+0000","last_undegraded":"2023-10-06T03:57:16.370129+0000","last_fullsized":"2023-10-06T03:57:16.370129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:32.280257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.370657+0000","last_change":"2023-10-06T03:57:11.366741+0000","last_active":"2023-10-06T03:57:11.370657+0000","last_peered":"2023-10-06T03:57:11.370657+0000","last_clean":"2023-10-06T03:57:11.370657+0000","last_became_active":"2023-10-06T03:57:11.366500+0000","last_became_peered":"2023-10-06T03:57:11.366500+0000","last_unstale":"2023-10-06T03:57:11.370657+0000","last_undegraded":"2023-10-06T03:57:11.370657+0000","last_fullsized":"2023-10-06T03:57:11.370657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:24.714603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"21'13","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.645297+0000","last_change":"2023-10-06T03:57:10.348170+0000","last_active":"2023-10-06T03:57:37.645297+0000","last_peered":"2023-10-06T03:57:37.645297+0000","last_clean":"2023-10-06T03:57:37.645297+0000","last_became_active":"2023-10-06T03:57:10.347897+0000","last_became_peered":"2023-10-06T03:57:10.347897+0000","last_unstale":"2023-10-06T03:57:37.645297+0000","last_undegraded":"2023-10-06T03:57:37.645297+0000","last_fullsized":"2023-10-06T03:57:37.645297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:40.756246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372969+0000","last_change":"2023-10-06T03:57:11.364368+0000","last_active":"2023-10-06T03:57:11.372969+0000","last_peered":"2023-10-06T03:57:11.372969+0000","last_clean":"2023-10-06T03:57:11.372969+0000","last_became_active":"2023-10-06T03:57:11.364150+0000","last_became_peered":"2023-10-06T03:57:11.364150+0000","last_unstale":"2023-10-06T03:57:11.372969+0000","last_undegraded":"2023-10-06T03:57:11.372969+0000","last_fullsized":"2023-10-06T03:57:11.372969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:11.038080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.374372+0000","last_change":"2023-10-06T03:57:11.366721+0000","last_active":"2023-10-06T03:57:11.374372+0000","last_peered":"2023-10-06T03:57:11.374372+0000","last_clean":"2023-10-06T03:57:11.374372+0000","last_became_active":"2023-10-06T03:57:11.366524+0000","last_became_peered":"2023-10-06T03:57:11.366524+0000","last_unstale":"2023-10-06T03:57:11.374372+0000","last_undegraded":"2023-10-06T03:57:11.374372+0000","last_fullsized":"2023-10-06T03:57:11.374372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:33.044858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373497+0000","last_change":"2023-10-06T03:57:11.371109+0000","last_active":"2023-10-06T03:57:11.373497+0000","last_peered":"2023-10-06T03:57:11.373497+0000","last_clean":"2023-10-06T03:57:11.373497+0000","last_became_active":"2023-10-06T03:57:11.370848+0000","last_became_peered":"2023-10-06T03:57:11.370848+0000","last_unstale":"2023-10-06T03:57:11.373497+0000","last_undegraded":"2023-10-06T03:57:11.373497+0000","last_fullsized":"2023-10-06T03:57:11.373497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:15.919090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372839+0000","last_change":"2023-10-06T03:57:11.364188+0000","last_active":"2023-10-06T03:57:11.372839+0000","last_peered":"2023-10-06T03:57:11.372839+0000","last_clean":"2023-10-06T03:57:11.372839+0000","last_became_active":"2023-10-06T03:57:11.363875+0000","last_became_peered":"2023-10-06T03:57:11.363875+0000","last_unstale":"2023-10-06T03:57:11.372839+0000","last_undegraded":"2023-10-06T03:57:11.372839+0000","last_fullsized":"2023-10-06T03:57:11.372839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:29.236344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378301+0000","last_change":"2023-10-06T03:57:11.377420+0000","last_active":"2023-10-06T03:57:11.378301+0000","last_peered":"2023-10-06T03:57:11.378301+0000","last_clean":"2023-10-06T03:57:11.378301+0000","last_became_active":"2023-10-06T03:57:11.377126+0000","last_became_peered":"2023-10-06T03:57:11.377126+0000","last_unstale":"2023-10-06T03:57:11.378301+0000","last_undegraded":"2023-10-06T03:57:11.378301+0000","last_fullsized":"2023-10-06T03:57:11.378301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:05.041737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377733+0000","last_change":"2023-10-06T03:57:11.375777+0000","last_active":"2023-10-06T03:57:11.377733+0000","last_peered":"2023-10-06T03:57:11.377733+0000","last_clean":"2023-10-06T03:57:11.377733+0000","last_became_active":"2023-10-06T03:57:11.375539+0000","last_became_peered":"2023-10-06T03:57:11.375539+0000","last_unstale":"2023-10-06T03:57:11.377733+0000","last_undegraded":"2023-10-06T03:57:11.377733+0000","last_fullsized":"2023-10-06T03:57:11.377733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:42.707488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":7626,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":204,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":15252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":178,"ondisk_log_size":178,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64858,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297612,"kb_used_data":324,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074228,"statfs":{"total":96636764160,"available":96332009472,"internally_reserved":0,"allocated":331776,"data_stored":672195,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":78947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":78835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297616,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074224,"statfs":{"total":96636764160,"available":96332005376,"internally_reserved":0,"allocated":335872,"data_stored":670584,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297504,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074336,"statfs":{"total":96636764160,"available":96332120064,"internally_reserved":0,"allocated":221184,"data_stored":78925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":253952,"data_stored":81801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297560,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074280,"statfs":{"total":96636764160,"available":96332062720,"internally_reserved":0,"allocated":278528,"data_stored":82399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":82029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":2334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":3564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":2992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:57:49.243 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:57:49.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.391+0000 7ff48f4bd700 1 Processor -- start 2023-10-06T03:57:49.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.391+0000 7ff48f4bd700 1 -- start start 2023-10-06T03:57:49.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.392+0000 7ff48f4bd700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff488150410 0x7ff488155f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:49.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.392+0000 7ff48f4bd700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4880c9780 con 0x7ff48806cbb0 2023-10-06T03:57:49.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.392+0000 7ff48f4bd700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff48806e0c0 con 0x7ff48806dc40 2023-10-06T03:57:49.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.392+0000 7ff48f4bd700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4880c8c40 con 0x7ff488150410 2023-10-06T03:57:49.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.392+0000 7ff48d259700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff48806cbb0 0x7ff48806d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40824/0 (socket says 172.21.15.157:40824) 2023-10-06T03:57:49.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.392+0000 7ff48d259700 1 -- 172.21.15.157:0/3452449518 learned_addr learned my addr 172.21.15.157:0/3452449518 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:49.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.392+0000 7ff48da5a700 1 --2- 172.21.15.157:0/3452449518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff488150410 0x7ff488155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:49.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.392+0000 7ff47ffff700 1 -- 172.21.15.157:0/3452449518 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2498310601 0 0) 0x7ff4880c9780 con 0x7ff48806cbb0 2023-10-06T03:57:49.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.392+0000 7ff47ffff700 1 -- 172.21.15.157:0/3452449518 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff474003570 con 0x7ff48806cbb0 2023-10-06T03:57:49.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.392+0000 7ff47ffff700 1 -- 172.21.15.157:0/3452449518 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4155782675 0 0) 0x7ff48806e0c0 con 0x7ff48806dc40 2023-10-06T03:57:49.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.392+0000 7ff47ffff700 1 -- 172.21.15.157:0/3452449518 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4880c9780 con 0x7ff48806dc40 2023-10-06T03:57:49.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.392+0000 7ff48da5a700 1 -- 172.21.15.157:0/3452449518 >> v1:172.21.15.175:6789/0 conn(0x7ff48806dc40 legacy=0x7ff48814fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:49.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.392+0000 7ff48da5a700 1 -- 172.21.15.157:0/3452449518 >> v1:172.21.15.157:6789/0 conn(0x7ff48806cbb0 legacy=0x7ff48806d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:49.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.392+0000 7ff48da5a700 1 -- 172.21.15.157:0/3452449518 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4880c8e50 con 0x7ff488150410 2023-10-06T03:57:49.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.393+0000 7ff48da5a700 1 --2- 172.21.15.157:0/3452449518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff488150410 0x7ff488155f70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff480009e90 tx=0x7ff480009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5314f077bb4654e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:49.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.393+0000 7ff47ffff700 1 -- 172.21.15.157:0/3452449518 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48000e070 con 0x7ff488150410 2023-10-06T03:57:49.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.393+0000 7ff47ffff700 1 -- 172.21.15.157:0/3452449518 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48000bc10 con 0x7ff488150410 2023-10-06T03:57:49.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.393+0000 7ff47ffff700 1 -- 172.21.15.157:0/3452449518 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff480012430 con 0x7ff488150410 2023-10-06T03:57:49.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.393+0000 7ff48f4bd700 1 -- 172.21.15.157:0/3452449518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff488150410 msgr2=0x7ff488155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:49.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.393+0000 7ff48f4bd700 1 --2- 172.21.15.157:0/3452449518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff488150410 0x7ff488155f70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff480009e90 tx=0x7ff480009780 comp rx=0 tx=0).stop 2023-10-06T03:57:49.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.393+0000 7ff48f4bd700 1 -- 172.21.15.157:0/3452449518 shutdown_connections 2023-10-06T03:57:49.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.393+0000 7ff48f4bd700 1 --2- 172.21.15.157:0/3452449518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff488150410 0x7ff488155f70 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:49.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.393+0000 7ff48f4bd700 1 -- 172.21.15.157:0/3452449518 >> 172.21.15.157:0/3452449518 conn(0x7ff48806f230 msgr2=0x7ff48806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:49.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.393+0000 7ff48f4bd700 1 -- 172.21.15.157:0/3452449518 shutdown_connections 2023-10-06T03:57:49.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.393+0000 7ff48f4bd700 1 -- 172.21.15.157:0/3452449518 wait complete. 2023-10-06T03:57:49.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.394+0000 7ff48f4bd700 1 Processor -- start 2023-10-06T03:57:49.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.394+0000 7ff48f4bd700 1 -- start start 2023-10-06T03:57:49.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.395+0000 7ff48f4bd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff48806cbb0 0x7ff488169450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:49.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.395+0000 7ff48f4bd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff488150410 0x7ff4880c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:49.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.395+0000 7ff48f4bd700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4880c3a00 0x7ff48814a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:49.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.395+0000 7ff48f4bd700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff488155c00 con 0x7ff48806cbb0 2023-10-06T03:57:49.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.395+0000 7ff48f4bd700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff488154440 con 0x7ff488150410 2023-10-06T03:57:49.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.395+0000 7ff48f4bd700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff488156b30 con 0x7ff4880c3a00 2023-10-06T03:57:49.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.395+0000 7ff48d259700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff48806cbb0 0x7ff488169450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:49.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.395+0000 7ff48d259700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff48806cbb0 0x7ff488169450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45218/0 (socket says 172.21.15.157:45218) 2023-10-06T03:57:49.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.395+0000 7ff48d259700 1 -- 172.21.15.157:0/2932677964 learned_addr learned my addr 172.21.15.157:0/2932677964 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:49.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.396+0000 7ff48ca58700 1 --2- 172.21.15.157:0/2932677964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff488150410 0x7ff4880c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:49.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.396+0000 7ff48da5a700 1 --2- 172.21.15.157:0/2932677964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4880c3a00 0x7ff48814a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:49.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.396+0000 7ff48d259700 1 -- 172.21.15.157:0/2932677964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4880c3a00 msgr2=0x7ff48814a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:49.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.396+0000 7ff48d259700 1 --2- 172.21.15.157:0/2932677964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4880c3a00 0x7ff48814a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:49.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.396+0000 7ff48d259700 1 -- 172.21.15.157:0/2932677964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff488150410 msgr2=0x7ff4880c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:49.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.396+0000 7ff48d259700 1 --2- 172.21.15.157:0/2932677964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff488150410 0x7ff4880c6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:49.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.396+0000 7ff48d259700 1 -- 172.21.15.157:0/2932677964 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff480009a20 con 0x7ff48806cbb0 2023-10-06T03:57:49.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.396+0000 7ff48ca58700 1 --2- 172.21.15.157:0/2932677964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff488150410 0x7ff4880c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:49.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.396+0000 7ff48d259700 1 --2- 172.21.15.157:0/2932677964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff48806cbb0 0x7ff488169450 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff47800c790 tx=0x7ff478007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:49.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.396+0000 7ff47d7fa700 1 -- 172.21.15.157:0/2932677964 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff478005b30 con 0x7ff48806cbb0 2023-10-06T03:57:49.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.396+0000 7ff47d7fa700 1 -- 172.21.15.157:0/2932677964 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff478005c90 con 0x7ff48806cbb0 2023-10-06T03:57:49.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.396+0000 7ff47d7fa700 1 -- 172.21.15.157:0/2932677964 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff478007300 con 0x7ff48806cbb0 2023-10-06T03:57:49.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.396+0000 7ff48f4bd700 1 -- 172.21.15.157:0/2932677964 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff488156df0 con 0x7ff48806cbb0 2023-10-06T03:57:49.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.396+0000 7ff48f4bd700 1 -- 172.21.15.157:0/2932677964 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff488154160 con 0x7ff48806cbb0 2023-10-06T03:57:49.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.397+0000 7ff48f4bd700 1 -- 172.21.15.157:0/2932677964 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff488155a20 con 0x7ff48806cbb0 2023-10-06T03:57:49.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.397+0000 7ff47d7fa700 1 -- 172.21.15.157:0/2932677964 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff47801b400 con 0x7ff48806cbb0 2023-10-06T03:57:49.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.398+0000 7ff47d7fa700 1 --2- 172.21.15.157:0/2932677964 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff46c066f60 0x7ff46c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:49.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.398+0000 7ff47d7fa700 1 -- 172.21.15.157:0/2932677964 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff478084330 con 0x7ff48806cbb0 2023-10-06T03:57:49.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.398+0000 7ff48ca58700 1 --2- 172.21.15.157:0/2932677964 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff46c066f60 0x7ff46c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:49.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.398+0000 7ff48ca58700 1 --2- 172.21.15.157:0/2932677964 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff46c066f60 0x7ff46c069410 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff48815c780 tx=0x7ff484009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:49.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.402+0000 7ff47d7fa700 1 -- 172.21.15.157:0/2932677964 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff478054e30 con 0x7ff48806cbb0 2023-10-06T03:57:49.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.583+0000 7ff48f4bd700 1 -- 172.21.15.157:0/2932677964 --> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff4880c9130 con 0x7ff46c066f60 2023-10-06T03:57:49.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.591+0000 7ff47d7fa700 1 -- 172.21.15.157:0/2932677964 <== mgr.4107 v2:172.21.15.175:6832/90661 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286811 (secure 0 0 0) 0x7ff4880c9130 con 0x7ff46c066f60 2023-10-06T03:57:49.596 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:49.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.594+0000 7ff48f4bd700 1 -- 172.21.15.157:0/2932677964 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff46c066f60 msgr2=0x7ff46c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:49.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.594+0000 7ff48f4bd700 1 --2- 172.21.15.157:0/2932677964 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff46c066f60 0x7ff46c069410 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff48815c780 tx=0x7ff484009040 comp rx=0 tx=0).stop 2023-10-06T03:57:49.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.594+0000 7ff48f4bd700 1 -- 172.21.15.157:0/2932677964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff48806cbb0 msgr2=0x7ff488169450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:49.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.594+0000 7ff48f4bd700 1 --2- 172.21.15.157:0/2932677964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff48806cbb0 0x7ff488169450 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff47800c790 tx=0x7ff478007ea0 comp rx=0 tx=0).stop 2023-10-06T03:57:49.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.594+0000 7ff48f4bd700 1 -- 172.21.15.157:0/2932677964 shutdown_connections 2023-10-06T03:57:49.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.594+0000 7ff48f4bd700 1 --2- 172.21.15.157:0/2932677964 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff46c066f60 0x7ff46c069410 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:49.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.594+0000 7ff48f4bd700 1 --2- 172.21.15.157:0/2932677964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4880c3a00 0x7ff48814a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:49.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.594+0000 7ff48f4bd700 1 --2- 172.21.15.157:0/2932677964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff48806cbb0 0x7ff488169450 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:49.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.594+0000 7ff48f4bd700 1 --2- 172.21.15.157:0/2932677964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff488150410 0x7ff4880c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:49.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.594+0000 7ff48f4bd700 1 -- 172.21.15.157:0/2932677964 >> 172.21.15.157:0/2932677964 conn(0x7ff48806f230 msgr2=0x7ff4880b00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:49.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.594+0000 7ff48f4bd700 1 -- 172.21.15.157:0/2932677964 shutdown_connections 2023-10-06T03:57:49.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.594+0000 7ff48f4bd700 1 -- 172.21.15.157:0/2932677964 wait complete. 2023-10-06T03:57:49.600 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-10-06T03:57:49.607 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2023-10-06T03:57:49.435196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":599301,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":369,"num_write_kb":2478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":301,"ondisk_log_size":301,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2380380,"kb_used_data":2076,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752594340,"statfs":{"total":773094113280,"available":770656604160,"internally_reserved":0,"allocated":2125824,"data_stored":1828329,"data_compressed":64858,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3967,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003133"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378564+0000","last_change":"2023-10-06T03:57:11.372019+0000","last_active":"2023-10-06T03:57:11.378564+0000","last_peered":"2023-10-06T03:57:11.378564+0000","last_clean":"2023-10-06T03:57:11.378564+0000","last_became_active":"2023-10-06T03:57:11.371756+0000","last_became_peered":"2023-10-06T03:57:11.371756+0000","last_unstale":"2023-10-06T03:57:11.378564+0000","last_undegraded":"2023-10-06T03:57:11.378564+0000","last_fullsized":"2023-10-06T03:57:11.378564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:50.086528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372505+0000","last_change":"2023-10-06T03:57:11.370371+0000","last_active":"2023-10-06T03:57:11.372505+0000","last_peered":"2023-10-06T03:57:11.372505+0000","last_clean":"2023-10-06T03:57:11.372505+0000","last_became_active":"2023-10-06T03:57:11.370070+0000","last_became_peered":"2023-10-06T03:57:11.370070+0000","last_unstale":"2023-10-06T03:57:11.372505+0000","last_undegraded":"2023-10-06T03:57:11.372505+0000","last_fullsized":"2023-10-06T03:57:11.372505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:00.917607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371725+0000","last_change":"2023-10-06T03:57:11.366298+0000","last_active":"2023-10-06T03:57:11.371725+0000","last_peered":"2023-10-06T03:57:11.371725+0000","last_clean":"2023-10-06T03:57:11.371725+0000","last_became_active":"2023-10-06T03:57:11.366012+0000","last_became_peered":"2023-10-06T03:57:11.366012+0000","last_unstale":"2023-10-06T03:57:11.371725+0000","last_undegraded":"2023-10-06T03:57:11.371725+0000","last_fullsized":"2023-10-06T03:57:11.371725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:45.802287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.374334+0000","last_change":"2023-10-06T03:57:11.365933+0000","last_active":"2023-10-06T03:57:11.374334+0000","last_peered":"2023-10-06T03:57:11.374334+0000","last_clean":"2023-10-06T03:57:11.374334+0000","last_became_active":"2023-10-06T03:57:11.365673+0000","last_became_peered":"2023-10-06T03:57:11.365673+0000","last_unstale":"2023-10-06T03:57:11.374334+0000","last_undegraded":"2023-10-06T03:57:11.374334+0000","last_fullsized":"2023-10-06T03:57:11.374334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:40.953382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.365753+0000","last_change":"2023-10-06T03:57:11.361569+0000","last_active":"2023-10-06T03:57:11.365753+0000","last_peered":"2023-10-06T03:57:11.365753+0000","last_clean":"2023-10-06T03:57:11.365753+0000","last_became_active":"2023-10-06T03:57:11.361302+0000","last_became_peered":"2023-10-06T03:57:11.361302+0000","last_unstale":"2023-10-06T03:57:11.365753+0000","last_undegraded":"2023-10-06T03:57:11.365753+0000","last_fullsized":"2023-10-06T03:57:11.365753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:57.552493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372859+0000","last_change":"2023-10-06T03:57:11.369953+0000","last_active":"2023-10-06T03:57:11.372859+0000","last_peered":"2023-10-06T03:57:11.372859+0000","last_clean":"2023-10-06T03:57:11.372859+0000","last_became_active":"2023-10-06T03:57:11.368775+0000","last_became_peered":"2023-10-06T03:57:11.368775+0000","last_unstale":"2023-10-06T03:57:11.372859+0000","last_undegraded":"2023-10-06T03:57:11.372859+0000","last_fullsized":"2023-10-06T03:57:11.372859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:22.368117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.380867+0000","last_change":"2023-10-06T03:57:11.378048+0000","last_active":"2023-10-06T03:57:11.380867+0000","last_peered":"2023-10-06T03:57:11.380867+0000","last_clean":"2023-10-06T03:57:11.380867+0000","last_became_active":"2023-10-06T03:57:11.377786+0000","last_became_peered":"2023-10-06T03:57:11.377786+0000","last_unstale":"2023-10-06T03:57:11.380867+0000","last_undegraded":"2023-10-06T03:57:11.380867+0000","last_fullsized":"2023-10-06T03:57:11.380867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:12:54.218593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377264+0000","last_change":"2023-10-06T03:57:11.374003+0000","last_active":"2023-10-06T03:57:11.377264+0000","last_peered":"2023-10-06T03:57:11.377264+0000","last_clean":"2023-10-06T03:57:11.377264+0000","last_became_active":"2023-10-06T03:57:11.373765+0000","last_became_peered":"2023-10-06T03:57:11.373765+0000","last_unstale":"2023-10-06T03:57:11.377264+0000","last_undegraded":"2023-10-06T03:57:11.377264+0000","last_fullsized":"2023-10-06T03:57:11.377264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:05.838007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378921+0000","last_change":"2023-10-06T03:57:11.373794+0000","last_active":"2023-10-06T03:57:11.378921+0000","last_peered":"2023-10-06T03:57:11.378921+0000","last_clean":"2023-10-06T03:57:11.378921+0000","last_became_active":"2023-10-06T03:57:11.368004+0000","last_became_peered":"2023-10-06T03:57:11.368004+0000","last_unstale":"2023-10-06T03:57:11.378921+0000","last_undegraded":"2023-10-06T03:57:11.378921+0000","last_fullsized":"2023-10-06T03:57:11.378921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:16.760049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.382630+0000","last_change":"2023-10-06T03:57:11.379370+0000","last_active":"2023-10-06T03:57:11.382630+0000","last_peered":"2023-10-06T03:57:11.382630+0000","last_clean":"2023-10-06T03:57:11.382630+0000","last_became_active":"2023-10-06T03:57:11.379107+0000","last_became_peered":"2023-10-06T03:57:11.379107+0000","last_unstale":"2023-10-06T03:57:11.382630+0000","last_undegraded":"2023-10-06T03:57:11.382630+0000","last_fullsized":"2023-10-06T03:57:11.382630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:18.373103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371742+0000","last_change":"2023-10-06T03:57:11.369172+0000","last_active":"2023-10-06T03:57:11.371742+0000","last_peered":"2023-10-06T03:57:11.371742+0000","last_clean":"2023-10-06T03:57:11.371742+0000","last_became_active":"2023-10-06T03:57:11.368256+0000","last_became_peered":"2023-10-06T03:57:11.368256+0000","last_unstale":"2023-10-06T03:57:11.371742+0000","last_undegraded":"2023-10-06T03:57:11.371742+0000","last_fullsized":"2023-10-06T03:57:11.371742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:50.736367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373531+0000","last_change":"2023-10-06T03:57:11.368685+0000","last_active":"2023-10-06T03:57:11.373531+0000","last_peered":"2023-10-06T03:57:11.373531+0000","last_clean":"2023-10-06T03:57:11.373531+0000","last_became_active":"2023-10-06T03:57:11.368425+0000","last_became_peered":"2023-10-06T03:57:11.368425+0000","last_unstale":"2023-10-06T03:57:11.373531+0000","last_undegraded":"2023-10-06T03:57:11.373531+0000","last_fullsized":"2023-10-06T03:57:11.373531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:06.004423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372891+0000","last_change":"2023-10-06T03:57:11.367646+0000","last_active":"2023-10-06T03:57:11.372891+0000","last_peered":"2023-10-06T03:57:11.372891+0000","last_clean":"2023-10-06T03:57:11.372891+0000","last_became_active":"2023-10-06T03:57:11.367079+0000","last_became_peered":"2023-10-06T03:57:11.367079+0000","last_unstale":"2023-10-06T03:57:11.372891+0000","last_undegraded":"2023-10-06T03:57:11.372891+0000","last_fullsized":"2023-10-06T03:57:11.372891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:09:54.331206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377012+0000","last_change":"2023-10-06T03:57:11.373481+0000","last_active":"2023-10-06T03:57:11.377012+0000","last_peered":"2023-10-06T03:57:11.377012+0000","last_clean":"2023-10-06T03:57:11.377012+0000","last_became_active":"2023-10-06T03:57:11.373216+0000","last_became_peered":"2023-10-06T03:57:11.373216+0000","last_unstale":"2023-10-06T03:57:11.377012+0000","last_undegraded":"2023-10-06T03:57:11.377012+0000","last_fullsized":"2023-10-06T03:57:11.377012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:56.083203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.379250+0000","last_change":"2023-10-06T03:57:11.376012+0000","last_active":"2023-10-06T03:57:11.379250+0000","last_peered":"2023-10-06T03:57:11.379250+0000","last_clean":"2023-10-06T03:57:11.379250+0000","last_became_active":"2023-10-06T03:57:11.375689+0000","last_became_peered":"2023-10-06T03:57:11.375689+0000","last_unstale":"2023-10-06T03:57:11.379250+0000","last_undegraded":"2023-10-06T03:57:11.379250+0000","last_fullsized":"2023-10-06T03:57:11.379250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:42.953739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372989+0000","last_change":"2023-10-06T03:57:11.366808+0000","last_active":"2023-10-06T03:57:11.372989+0000","last_peered":"2023-10-06T03:57:11.372989+0000","last_clean":"2023-10-06T03:57:11.372989+0000","last_became_active":"2023-10-06T03:57:11.366543+0000","last_became_peered":"2023-10-06T03:57:11.366543+0000","last_unstale":"2023-10-06T03:57:11.372989+0000","last_undegraded":"2023-10-06T03:57:11.372989+0000","last_fullsized":"2023-10-06T03:57:11.372989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:16.212967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378753+0000","last_change":"2023-10-06T03:57:11.367276+0000","last_active":"2023-10-06T03:57:11.378753+0000","last_peered":"2023-10-06T03:57:11.378753+0000","last_clean":"2023-10-06T03:57:11.378753+0000","last_became_active":"2023-10-06T03:57:11.366767+0000","last_became_peered":"2023-10-06T03:57:11.366767+0000","last_unstale":"2023-10-06T03:57:11.378753+0000","last_undegraded":"2023-10-06T03:57:11.378753+0000","last_fullsized":"2023-10-06T03:57:11.378753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:43.113575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367129+0000","last_change":"2023-10-06T03:57:11.362667+0000","last_active":"2023-10-06T03:57:11.367129+0000","last_peered":"2023-10-06T03:57:11.367129+0000","last_clean":"2023-10-06T03:57:11.367129+0000","last_became_active":"2023-10-06T03:57:11.362377+0000","last_became_peered":"2023-10-06T03:57:11.362377+0000","last_unstale":"2023-10-06T03:57:11.367129+0000","last_undegraded":"2023-10-06T03:57:11.367129+0000","last_fullsized":"2023-10-06T03:57:11.367129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:40.600383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.365567+0000","last_change":"2023-10-06T03:57:11.364126+0000","last_active":"2023-10-06T03:57:11.365567+0000","last_peered":"2023-10-06T03:57:11.365567+0000","last_clean":"2023-10-06T03:57:11.365567+0000","last_became_active":"2023-10-06T03:57:11.363792+0000","last_became_peered":"2023-10-06T03:57:11.363792+0000","last_unstale":"2023-10-06T03:57:11.365567+0000","last_undegraded":"2023-10-06T03:57:11.365567+0000","last_fullsized":"2023-10-06T03:57:11.365567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:49.185450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372624+0000","last_change":"2023-10-06T03:57:11.364974+0000","last_active":"2023-10-06T03:57:11.372624+0000","last_peered":"2023-10-06T03:57:11.372624+0000","last_clean":"2023-10-06T03:57:11.372624+0000","last_became_active":"2023-10-06T03:57:11.364726+0000","last_became_peered":"2023-10-06T03:57:11.364726+0000","last_unstale":"2023-10-06T03:57:11.372624+0000","last_undegraded":"2023-10-06T03:57:11.372624+0000","last_fullsized":"2023-10-06T03:57:11.372624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:56.866733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.379227+0000","last_change":"2023-10-06T03:57:11.375987+0000","last_active":"2023-10-06T03:57:11.379227+0000","last_peered":"2023-10-06T03:57:11.379227+0000","last_clean":"2023-10-06T03:57:11.379227+0000","last_became_active":"2023-10-06T03:57:11.375668+0000","last_became_peered":"2023-10-06T03:57:11.375668+0000","last_unstale":"2023-10-06T03:57:11.379227+0000","last_undegraded":"2023-10-06T03:57:11.379227+0000","last_fullsized":"2023-10-06T03:57:11.379227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:53.810875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.366478+0000","last_change":"2023-10-06T03:57:11.365414+0000","last_active":"2023-10-06T03:57:11.366478+0000","last_peered":"2023-10-06T03:57:11.366478+0000","last_clean":"2023-10-06T03:57:11.366478+0000","last_became_active":"2023-10-06T03:57:11.365149+0000","last_became_peered":"2023-10-06T03:57:11.365149+0000","last_unstale":"2023-10-06T03:57:11.366478+0000","last_undegraded":"2023-10-06T03:57:11.366478+0000","last_fullsized":"2023-10-06T03:57:11.366478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:20.911710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.370638+0000","last_change":"2023-10-06T03:57:11.362366+0000","last_active":"2023-10-06T03:57:11.370638+0000","last_peered":"2023-10-06T03:57:11.370638+0000","last_clean":"2023-10-06T03:57:11.370638+0000","last_became_active":"2023-10-06T03:57:11.362128+0000","last_became_peered":"2023-10-06T03:57:11.362128+0000","last_unstale":"2023-10-06T03:57:11.370638+0000","last_undegraded":"2023-10-06T03:57:11.370638+0000","last_fullsized":"2023-10-06T03:57:11.370638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:53.744674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.379282+0000","last_change":"2023-10-06T03:57:11.375969+0000","last_active":"2023-10-06T03:57:11.379282+0000","last_peered":"2023-10-06T03:57:11.379282+0000","last_clean":"2023-10-06T03:57:11.379282+0000","last_became_active":"2023-10-06T03:57:11.375702+0000","last_became_peered":"2023-10-06T03:57:11.375702+0000","last_unstale":"2023-10-06T03:57:11.379282+0000","last_undegraded":"2023-10-06T03:57:11.379282+0000","last_fullsized":"2023-10-06T03:57:11.379282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:31.204231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377763+0000","last_change":"2023-10-06T03:57:11.375099+0000","last_active":"2023-10-06T03:57:11.377763+0000","last_peered":"2023-10-06T03:57:11.377763+0000","last_clean":"2023-10-06T03:57:11.377763+0000","last_became_active":"2023-10-06T03:57:11.374842+0000","last_became_peered":"2023-10-06T03:57:11.374842+0000","last_unstale":"2023-10-06T03:57:11.377763+0000","last_undegraded":"2023-10-06T03:57:11.377763+0000","last_fullsized":"2023-10-06T03:57:11.377763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:54.438974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.369108+0000","last_change":"2023-10-06T03:57:11.365900+0000","last_active":"2023-10-06T03:57:11.369108+0000","last_peered":"2023-10-06T03:57:11.369108+0000","last_clean":"2023-10-06T03:57:11.369108+0000","last_became_active":"2023-10-06T03:57:11.361349+0000","last_became_peered":"2023-10-06T03:57:11.361349+0000","last_unstale":"2023-10-06T03:57:11.369108+0000","last_undegraded":"2023-10-06T03:57:11.369108+0000","last_fullsized":"2023-10-06T03:57:11.369108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:30.646781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378945+0000","last_change":"2023-10-06T03:57:11.376082+0000","last_active":"2023-10-06T03:57:11.378945+0000","last_peered":"2023-10-06T03:57:11.378945+0000","last_clean":"2023-10-06T03:57:11.378945+0000","last_became_active":"2023-10-06T03:57:11.375846+0000","last_became_peered":"2023-10-06T03:57:11.375846+0000","last_unstale":"2023-10-06T03:57:11.378945+0000","last_undegraded":"2023-10-06T03:57:11.378945+0000","last_fullsized":"2023-10-06T03:57:11.378945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:28.080614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"21'7","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.546542+0000","last_change":"2023-10-06T03:57:10.348317+0000","last_active":"2023-10-06T03:57:32.546542+0000","last_peered":"2023-10-06T03:57:32.546542+0000","last_clean":"2023-10-06T03:57:32.546542+0000","last_became_active":"2023-10-06T03:57:10.348078+0000","last_became_peered":"2023-10-06T03:57:10.348078+0000","last_unstale":"2023-10-06T03:57:32.546542+0000","last_undegraded":"2023-10-06T03:57:32.546542+0000","last_fullsized":"2023-10-06T03:57:32.546542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:55.058324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372610+0000","last_change":"2023-10-06T03:57:11.370506+0000","last_active":"2023-10-06T03:57:11.372610+0000","last_peered":"2023-10-06T03:57:11.372610+0000","last_clean":"2023-10-06T03:57:11.372610+0000","last_became_active":"2023-10-06T03:57:11.370245+0000","last_became_peered":"2023-10-06T03:57:11.370245+0000","last_unstale":"2023-10-06T03:57:11.372610+0000","last_undegraded":"2023-10-06T03:57:11.372610+0000","last_fullsized":"2023-10-06T03:57:11.372610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:47:18.189772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:26.532954+0000","last_change":"2023-10-06T03:57:10.345401+0000","last_active":"2023-10-06T03:57:26.532954+0000","last_peered":"2023-10-06T03:57:26.532954+0000","last_clean":"2023-10-06T03:57:26.532954+0000","last_became_active":"2023-10-06T03:57:10.345150+0000","last_became_peered":"2023-10-06T03:57:10.345150+0000","last_unstale":"2023-10-06T03:57:26.532954+0000","last_undegraded":"2023-10-06T03:57:26.532954+0000","last_fullsized":"2023-10-06T03:57:26.532954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:25.758193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.379135+0000","last_change":"2023-10-06T03:57:11.376046+0000","last_active":"2023-10-06T03:57:11.379135+0000","last_peered":"2023-10-06T03:57:11.379135+0000","last_clean":"2023-10-06T03:57:11.379135+0000","last_became_active":"2023-10-06T03:57:11.375790+0000","last_became_peered":"2023-10-06T03:57:11.375790+0000","last_unstale":"2023-10-06T03:57:11.379135+0000","last_undegraded":"2023-10-06T03:57:11.379135+0000","last_fullsized":"2023-10-06T03:57:11.379135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:33.775431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.540480+0000","last_change":"2023-10-06T03:57:10.347562+0000","last_active":"2023-10-06T03:57:32.540480+0000","last_peered":"2023-10-06T03:57:32.540480+0000","last_clean":"2023-10-06T03:57:32.540480+0000","last_became_active":"2023-10-06T03:57:10.347294+0000","last_became_peered":"2023-10-06T03:57:10.347294+0000","last_unstale":"2023-10-06T03:57:32.540480+0000","last_undegraded":"2023-10-06T03:57:32.540480+0000","last_fullsized":"2023-10-06T03:57:32.540480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:25.090992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.375571+0000","last_change":"2023-10-06T03:57:11.372022+0000","last_active":"2023-10-06T03:57:11.375571+0000","last_peered":"2023-10-06T03:57:11.375571+0000","last_clean":"2023-10-06T03:57:11.375571+0000","last_became_active":"2023-10-06T03:57:11.371758+0000","last_became_peered":"2023-10-06T03:57:11.371758+0000","last_unstale":"2023-10-06T03:57:11.375571+0000","last_undegraded":"2023-10-06T03:57:11.375571+0000","last_fullsized":"2023-10-06T03:57:11.375571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:10.922157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:21.441650+0000","last_change":"2023-10-06T03:57:10.347406+0000","last_active":"2023-10-06T03:57:21.441650+0000","last_peered":"2023-10-06T03:57:21.441650+0000","last_clean":"2023-10-06T03:57:21.441650+0000","last_became_active":"2023-10-06T03:57:10.347151+0000","last_became_peered":"2023-10-06T03:57:10.347151+0000","last_unstale":"2023-10-06T03:57:21.441650+0000","last_undegraded":"2023-10-06T03:57:21.441650+0000","last_fullsized":"2023-10-06T03:57:21.441650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:20.427620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372580+0000","last_change":"2023-10-06T03:57:11.370240+0000","last_active":"2023-10-06T03:57:11.372580+0000","last_peered":"2023-10-06T03:57:11.372580+0000","last_clean":"2023-10-06T03:57:11.372580+0000","last_became_active":"2023-10-06T03:57:11.369938+0000","last_became_peered":"2023-10-06T03:57:11.369938+0000","last_unstale":"2023-10-06T03:57:11.372580+0000","last_undegraded":"2023-10-06T03:57:11.372580+0000","last_fullsized":"2023-10-06T03:57:11.372580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:07.105832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.646156+0000","last_change":"2023-10-06T03:57:10.346054+0000","last_active":"2023-10-06T03:57:37.646156+0000","last_peered":"2023-10-06T03:57:37.646156+0000","last_clean":"2023-10-06T03:57:37.646156+0000","last_became_active":"2023-10-06T03:57:10.345721+0000","last_became_peered":"2023-10-06T03:57:10.345721+0000","last_unstale":"2023-10-06T03:57:37.646156+0000","last_undegraded":"2023-10-06T03:57:37.646156+0000","last_fullsized":"2023-10-06T03:57:37.646156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:54.481304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378702+0000","last_change":"2023-10-06T03:57:11.376525+0000","last_active":"2023-10-06T03:57:11.378702+0000","last_peered":"2023-10-06T03:57:11.378702+0000","last_clean":"2023-10-06T03:57:11.378702+0000","last_became_active":"2023-10-06T03:57:11.376291+0000","last_became_peered":"2023-10-06T03:57:11.376291+0000","last_unstale":"2023-10-06T03:57:11.378702+0000","last_undegraded":"2023-10-06T03:57:11.378702+0000","last_fullsized":"2023-10-06T03:57:11.378702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:31.993426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"21'5","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:21.435357+0000","last_change":"2023-10-06T03:57:10.340531+0000","last_active":"2023-10-06T03:57:21.435357+0000","last_peered":"2023-10-06T03:57:21.435357+0000","last_clean":"2023-10-06T03:57:21.435357+0000","last_became_active":"2023-10-06T03:57:10.340232+0000","last_became_peered":"2023-10-06T03:57:10.340232+0000","last_unstale":"2023-10-06T03:57:21.435357+0000","last_undegraded":"2023-10-06T03:57:21.435357+0000","last_fullsized":"2023-10-06T03:57:21.435357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:43:26.332055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377803+0000","last_change":"2023-10-06T03:57:11.375070+0000","last_active":"2023-10-06T03:57:11.377803+0000","last_peered":"2023-10-06T03:57:11.377803+0000","last_clean":"2023-10-06T03:57:11.377803+0000","last_became_active":"2023-10-06T03:57:11.374892+0000","last_became_peered":"2023-10-06T03:57:11.374892+0000","last_unstale":"2023-10-06T03:57:11.377803+0000","last_undegraded":"2023-10-06T03:57:11.377803+0000","last_fullsized":"2023-10-06T03:57:11.377803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:51.198745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"21'13","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:45.217086+0000","last_change":"2023-10-06T03:57:10.346758+0000","last_active":"2023-10-06T03:57:45.217086+0000","last_peered":"2023-10-06T03:57:45.217086+0000","last_clean":"2023-10-06T03:57:45.217086+0000","last_became_active":"2023-10-06T03:57:10.346430+0000","last_became_peered":"2023-10-06T03:57:10.346430+0000","last_unstale":"2023-10-06T03:57:45.217086+0000","last_undegraded":"2023-10-06T03:57:45.217086+0000","last_fullsized":"2023-10-06T03:57:45.217086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:10.809242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2583,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371658+0000","last_change":"2023-10-06T03:57:11.367808+0000","last_active":"2023-10-06T03:57:11.371658+0000","last_peered":"2023-10-06T03:57:11.371658+0000","last_clean":"2023-10-06T03:57:11.371658+0000","last_became_active":"2023-10-06T03:57:11.367565+0000","last_became_peered":"2023-10-06T03:57:11.367565+0000","last_unstale":"2023-10-06T03:57:11.371658+0000","last_undegraded":"2023-10-06T03:57:11.371658+0000","last_fullsized":"2023-10-06T03:57:11.371658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:14.409116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"21'9","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.646018+0000","last_change":"2023-10-06T03:57:10.351062+0000","last_active":"2023-10-06T03:57:37.646018+0000","last_peered":"2023-10-06T03:57:37.646018+0000","last_clean":"2023-10-06T03:57:37.646018+0000","last_became_active":"2023-10-06T03:57:10.350795+0000","last_became_peered":"2023-10-06T03:57:10.350795+0000","last_unstale":"2023-10-06T03:57:37.646018+0000","last_undegraded":"2023-10-06T03:57:37.646018+0000","last_fullsized":"2023-10-06T03:57:37.646018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:54:45.714518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.370549+0000","last_change":"2023-10-06T03:57:11.367489+0000","last_active":"2023-10-06T03:57:11.370549+0000","last_peered":"2023-10-06T03:57:11.370549+0000","last_clean":"2023-10-06T03:57:11.370549+0000","last_became_active":"2023-10-06T03:57:11.367254+0000","last_became_peered":"2023-10-06T03:57:11.367254+0000","last_unstale":"2023-10-06T03:57:11.370549+0000","last_undegraded":"2023-10-06T03:57:11.370549+0000","last_fullsized":"2023-10-06T03:57:11.370549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:04.706485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"21'5","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.544381+0000","last_change":"2023-10-06T03:57:10.338125+0000","last_active":"2023-10-06T03:57:32.544381+0000","last_peered":"2023-10-06T03:57:32.544381+0000","last_clean":"2023-10-06T03:57:32.544381+0000","last_became_active":"2023-10-06T03:57:10.337798+0000","last_became_peered":"2023-10-06T03:57:10.337798+0000","last_unstale":"2023-10-06T03:57:32.544381+0000","last_undegraded":"2023-10-06T03:57:32.544381+0000","last_fullsized":"2023-10-06T03:57:32.544381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:00.482657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378957+0000","last_change":"2023-10-06T03:57:11.376461+0000","last_active":"2023-10-06T03:57:11.378957+0000","last_peered":"2023-10-06T03:57:11.378957+0000","last_clean":"2023-10-06T03:57:11.378957+0000","last_became_active":"2023-10-06T03:57:11.376223+0000","last_became_peered":"2023-10-06T03:57:11.376223+0000","last_unstale":"2023-10-06T03:57:11.378957+0000","last_undegraded":"2023-10-06T03:57:11.378957+0000","last_fullsized":"2023-10-06T03:57:11.378957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:04:37.408871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"21'6","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.547205+0000","last_change":"2023-10-06T03:57:10.350511+0000","last_active":"2023-10-06T03:57:32.547205+0000","last_peered":"2023-10-06T03:57:32.547205+0000","last_clean":"2023-10-06T03:57:32.547205+0000","last_became_active":"2023-10-06T03:57:10.350219+0000","last_became_peered":"2023-10-06T03:57:10.350219+0000","last_unstale":"2023-10-06T03:57:32.547205+0000","last_undegraded":"2023-10-06T03:57:32.547205+0000","last_fullsized":"2023-10-06T03:57:32.547205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:56.043506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":800,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.364939+0000","last_change":"2023-10-06T03:57:11.362700+0000","last_active":"2023-10-06T03:57:11.364939+0000","last_peered":"2023-10-06T03:57:11.364939+0000","last_clean":"2023-10-06T03:57:11.364939+0000","last_became_active":"2023-10-06T03:57:11.362443+0000","last_became_peered":"2023-10-06T03:57:11.362443+0000","last_unstale":"2023-10-06T03:57:11.364939+0000","last_undegraded":"2023-10-06T03:57:11.364939+0000","last_fullsized":"2023-10-06T03:57:11.364939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:21:09.143085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"21'17","reported_seq":27,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.541795+0000","last_change":"2023-10-06T03:57:10.346259+0000","last_active":"2023-10-06T03:57:32.541795+0000","last_peered":"2023-10-06T03:57:32.541795+0000","last_clean":"2023-10-06T03:57:32.541795+0000","last_became_active":"2023-10-06T03:57:10.345424+0000","last_became_peered":"2023-10-06T03:57:10.345424+0000","last_unstale":"2023-10-06T03:57:32.541795+0000","last_undegraded":"2023-10-06T03:57:32.541795+0000","last_fullsized":"2023-10-06T03:57:32.541795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:19.278206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378006+0000","last_change":"2023-10-06T03:57:11.376053+0000","last_active":"2023-10-06T03:57:11.378006+0000","last_peered":"2023-10-06T03:57:11.378006+0000","last_clean":"2023-10-06T03:57:11.378006+0000","last_became_active":"2023-10-06T03:57:11.375824+0000","last_became_peered":"2023-10-06T03:57:11.375824+0000","last_unstale":"2023-10-06T03:57:11.378006+0000","last_undegraded":"2023-10-06T03:57:11.378006+0000","last_fullsized":"2023-10-06T03:57:11.378006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:24:26.156866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:16.374858+0000","last_change":"2023-10-06T03:57:10.350408+0000","last_active":"2023-10-06T03:57:16.374858+0000","last_peered":"2023-10-06T03:57:16.374858+0000","last_clean":"2023-10-06T03:57:16.374858+0000","last_became_active":"2023-10-06T03:57:10.350135+0000","last_became_peered":"2023-10-06T03:57:10.350135+0000","last_unstale":"2023-10-06T03:57:16.374858+0000","last_undegraded":"2023-10-06T03:57:16.374858+0000","last_fullsized":"2023-10-06T03:57:16.374858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:43.522111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378954+0000","last_change":"2023-10-06T03:57:11.376524+0000","last_active":"2023-10-06T03:57:11.378954+0000","last_peered":"2023-10-06T03:57:11.378954+0000","last_clean":"2023-10-06T03:57:11.378954+0000","last_became_active":"2023-10-06T03:57:11.376236+0000","last_became_peered":"2023-10-06T03:57:11.376236+0000","last_unstale":"2023-10-06T03:57:11.378954+0000","last_undegraded":"2023-10-06T03:57:11.378954+0000","last_fullsized":"2023-10-06T03:57:11.378954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:33.854818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"21'5","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.644387+0000","last_change":"2023-10-06T03:57:10.345799+0000","last_active":"2023-10-06T03:57:37.644387+0000","last_peered":"2023-10-06T03:57:37.644387+0000","last_clean":"2023-10-06T03:57:37.644387+0000","last_became_active":"2023-10-06T03:57:10.345510+0000","last_became_peered":"2023-10-06T03:57:10.345510+0000","last_unstale":"2023-10-06T03:57:37.644387+0000","last_undegraded":"2023-10-06T03:57:37.644387+0000","last_fullsized":"2023-10-06T03:57:37.644387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:09.567314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372383+0000","last_change":"2023-10-06T03:57:11.368662+0000","last_active":"2023-10-06T03:57:11.372383+0000","last_peered":"2023-10-06T03:57:11.372383+0000","last_clean":"2023-10-06T03:57:11.372383+0000","last_became_active":"2023-10-06T03:57:11.368375+0000","last_became_peered":"2023-10-06T03:57:11.368375+0000","last_unstale":"2023-10-06T03:57:11.372383+0000","last_undegraded":"2023-10-06T03:57:11.372383+0000","last_fullsized":"2023-10-06T03:57:11.372383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:27.125523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.543509+0000","last_change":"2023-10-06T03:57:10.346945+0000","last_active":"2023-10-06T03:57:32.543509+0000","last_peered":"2023-10-06T03:57:32.543509+0000","last_clean":"2023-10-06T03:57:32.543509+0000","last_became_active":"2023-10-06T03:57:10.346689+0000","last_became_peered":"2023-10-06T03:57:10.346689+0000","last_unstale":"2023-10-06T03:57:32.543509+0000","last_undegraded":"2023-10-06T03:57:32.543509+0000","last_fullsized":"2023-10-06T03:57:32.543509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:43.420455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.375632+0000","last_change":"2023-10-06T03:57:11.372308+0000","last_active":"2023-10-06T03:57:11.375632+0000","last_peered":"2023-10-06T03:57:11.375632+0000","last_clean":"2023-10-06T03:57:11.375632+0000","last_became_active":"2023-10-06T03:57:11.371795+0000","last_became_peered":"2023-10-06T03:57:11.371795+0000","last_unstale":"2023-10-06T03:57:11.375632+0000","last_undegraded":"2023-10-06T03:57:11.375632+0000","last_fullsized":"2023-10-06T03:57:11.375632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:00.324478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"21'5","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.542306+0000","last_change":"2023-10-06T03:57:10.348592+0000","last_active":"2023-10-06T03:57:32.542306+0000","last_peered":"2023-10-06T03:57:32.542306+0000","last_clean":"2023-10-06T03:57:32.542306+0000","last_became_active":"2023-10-06T03:57:10.348291+0000","last_became_peered":"2023-10-06T03:57:10.348291+0000","last_unstale":"2023-10-06T03:57:32.542306+0000","last_undegraded":"2023-10-06T03:57:32.542306+0000","last_fullsized":"2023-10-06T03:57:32.542306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:00.660348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1094,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378672+0000","last_change":"2023-10-06T03:57:11.366510+0000","last_active":"2023-10-06T03:57:11.378672+0000","last_peered":"2023-10-06T03:57:11.378672+0000","last_clean":"2023-10-06T03:57:11.378672+0000","last_became_active":"2023-10-06T03:57:11.365077+0000","last_became_peered":"2023-10-06T03:57:11.365077+0000","last_unstale":"2023-10-06T03:57:11.378672+0000","last_undegraded":"2023-10-06T03:57:11.378672+0000","last_fullsized":"2023-10-06T03:57:11.378672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:25.199391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:21.407239+0000","last_change":"2023-10-06T03:57:10.348418+0000","last_active":"2023-10-06T03:57:21.407239+0000","last_peered":"2023-10-06T03:57:21.407239+0000","last_clean":"2023-10-06T03:57:21.407239+0000","last_became_active":"2023-10-06T03:57:10.348055+0000","last_became_peered":"2023-10-06T03:57:10.348055+0000","last_unstale":"2023-10-06T03:57:21.407239+0000","last_undegraded":"2023-10-06T03:57:21.407239+0000","last_fullsized":"2023-10-06T03:57:21.407239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:26.102022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373629+0000","last_change":"2023-10-06T03:57:11.370493+0000","last_active":"2023-10-06T03:57:11.373629+0000","last_peered":"2023-10-06T03:57:11.373629+0000","last_clean":"2023-10-06T03:57:11.373629+0000","last_became_active":"2023-10-06T03:57:11.370073+0000","last_became_peered":"2023-10-06T03:57:11.370073+0000","last_unstale":"2023-10-06T03:57:11.373629+0000","last_undegraded":"2023-10-06T03:57:11.373629+0000","last_fullsized":"2023-10-06T03:57:11.373629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:58.417951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.544169+0000","last_change":"2023-10-06T03:57:10.352966+0000","last_active":"2023-10-06T03:57:32.544169+0000","last_peered":"2023-10-06T03:57:32.544169+0000","last_clean":"2023-10-06T03:57:32.544169+0000","last_became_active":"2023-10-06T03:57:10.352653+0000","last_became_peered":"2023-10-06T03:57:10.352653+0000","last_unstale":"2023-10-06T03:57:32.544169+0000","last_undegraded":"2023-10-06T03:57:32.544169+0000","last_fullsized":"2023-10-06T03:57:32.544169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:32.092557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371704+0000","last_change":"2023-10-06T03:57:11.364052+0000","last_active":"2023-10-06T03:57:11.371704+0000","last_peered":"2023-10-06T03:57:11.371704+0000","last_clean":"2023-10-06T03:57:11.371704+0000","last_became_active":"2023-10-06T03:57:11.363830+0000","last_became_peered":"2023-10-06T03:57:11.363830+0000","last_unstale":"2023-10-06T03:57:11.371704+0000","last_undegraded":"2023-10-06T03:57:11.371704+0000","last_fullsized":"2023-10-06T03:57:11.371704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:04.084019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"21'7","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.645676+0000","last_change":"2023-10-06T03:57:10.349421+0000","last_active":"2023-10-06T03:57:37.645676+0000","last_peered":"2023-10-06T03:57:37.645676+0000","last_clean":"2023-10-06T03:57:37.645676+0000","last_became_active":"2023-10-06T03:57:10.349114+0000","last_became_peered":"2023-10-06T03:57:10.349114+0000","last_unstale":"2023-10-06T03:57:37.645676+0000","last_undegraded":"2023-10-06T03:57:37.645676+0000","last_fullsized":"2023-10-06T03:57:37.645676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:08.209791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372296+0000","last_change":"2023-10-06T03:57:11.367516+0000","last_active":"2023-10-06T03:57:11.372296+0000","last_peered":"2023-10-06T03:57:11.372296+0000","last_clean":"2023-10-06T03:57:11.372296+0000","last_became_active":"2023-10-06T03:57:11.367257+0000","last_became_peered":"2023-10-06T03:57:11.367257+0000","last_unstale":"2023-10-06T03:57:11.372296+0000","last_undegraded":"2023-10-06T03:57:11.372296+0000","last_fullsized":"2023-10-06T03:57:11.372296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:43.628819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:21.429070+0000","last_change":"2023-10-06T03:57:10.347649+0000","last_active":"2023-10-06T03:57:21.429070+0000","last_peered":"2023-10-06T03:57:21.429070+0000","last_clean":"2023-10-06T03:57:21.429070+0000","last_became_active":"2023-10-06T03:57:10.347433+0000","last_became_peered":"2023-10-06T03:57:10.347433+0000","last_unstale":"2023-10-06T03:57:21.429070+0000","last_undegraded":"2023-10-06T03:57:21.429070+0000","last_fullsized":"2023-10-06T03:57:21.429070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:55.951494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372561+0000","last_change":"2023-10-06T03:57:11.370092+0000","last_active":"2023-10-06T03:57:11.372561+0000","last_peered":"2023-10-06T03:57:11.372561+0000","last_clean":"2023-10-06T03:57:11.372561+0000","last_became_active":"2023-10-06T03:57:11.369825+0000","last_became_peered":"2023-10-06T03:57:11.369825+0000","last_unstale":"2023-10-06T03:57:11.372561+0000","last_undegraded":"2023-10-06T03:57:11.372561+0000","last_fullsized":"2023-10-06T03:57:11.372561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:43.415929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367709+0000","last_change":"2023-10-06T03:57:05.663312+0000","last_active":"2023-10-06T03:57:11.367709+0000","last_peered":"2023-10-06T03:57:11.367709+0000","last_clean":"2023-10-06T03:57:11.367709+0000","last_became_active":"2023-10-06T03:57:05.663046+0000","last_became_peered":"2023-10-06T03:57:05.663046+0000","last_unstale":"2023-10-06T03:57:11.367709+0000","last_undegraded":"2023-10-06T03:57:11.367709+0000","last_fullsized":"2023-10-06T03:57:11.367709+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:31.174359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.542752+0000","last_change":"2023-10-06T03:57:10.348930+0000","last_active":"2023-10-06T03:57:32.542752+0000","last_peered":"2023-10-06T03:57:32.542752+0000","last_clean":"2023-10-06T03:57:32.542752+0000","last_became_active":"2023-10-06T03:57:10.348608+0000","last_became_peered":"2023-10-06T03:57:10.348608+0000","last_unstale":"2023-10-06T03:57:32.542752+0000","last_undegraded":"2023-10-06T03:57:32.542752+0000","last_fullsized":"2023-10-06T03:57:32.542752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:00:09.283718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377785+0000","last_change":"2023-10-06T03:57:11.371108+0000","last_active":"2023-10-06T03:57:11.377785+0000","last_peered":"2023-10-06T03:57:11.377785+0000","last_clean":"2023-10-06T03:57:11.377785+0000","last_became_active":"2023-10-06T03:57:11.370847+0000","last_became_peered":"2023-10-06T03:57:11.370847+0000","last_unstale":"2023-10-06T03:57:11.377785+0000","last_undegraded":"2023-10-06T03:57:11.377785+0000","last_fullsized":"2023-10-06T03:57:11.377785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:38.111171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.362274+0000","last_change":"2023-10-06T03:57:05.318834+0000","last_active":"2023-10-06T03:57:11.362274+0000","last_peered":"2023-10-06T03:57:11.362274+0000","last_clean":"2023-10-06T03:57:11.362274+0000","last_became_active":"2023-10-06T03:57:05.318572+0000","last_became_peered":"2023-10-06T03:57:05.318572+0000","last_unstale":"2023-10-06T03:57:11.362274+0000","last_undegraded":"2023-10-06T03:57:11.362274+0000","last_fullsized":"2023-10-06T03:57:11.362274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:21.951095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"21'6","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.546545+0000","last_change":"2023-10-06T03:57:10.348556+0000","last_active":"2023-10-06T03:57:32.546545+0000","last_peered":"2023-10-06T03:57:32.546545+0000","last_clean":"2023-10-06T03:57:32.546545+0000","last_became_active":"2023-10-06T03:57:10.348271+0000","last_became_peered":"2023-10-06T03:57:10.348271+0000","last_unstale":"2023-10-06T03:57:32.546545+0000","last_undegraded":"2023-10-06T03:57:32.546545+0000","last_fullsized":"2023-10-06T03:57:32.546545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:58.470989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.368920+0000","last_change":"2023-10-06T03:57:11.365237+0000","last_active":"2023-10-06T03:57:11.368920+0000","last_peered":"2023-10-06T03:57:11.368920+0000","last_clean":"2023-10-06T03:57:11.368920+0000","last_became_active":"2023-10-06T03:57:11.365006+0000","last_became_peered":"2023-10-06T03:57:11.365006+0000","last_unstale":"2023-10-06T03:57:11.368920+0000","last_undegraded":"2023-10-06T03:57:11.368920+0000","last_fullsized":"2023-10-06T03:57:11.368920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:33.005015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.355768+0000","last_change":"2023-10-06T03:57:05.314695+0000","last_active":"2023-10-06T03:57:11.355768+0000","last_peered":"2023-10-06T03:57:11.355768+0000","last_clean":"2023-10-06T03:57:11.355768+0000","last_became_active":"2023-10-06T03:57:05.314408+0000","last_became_peered":"2023-10-06T03:57:05.314408+0000","last_unstale":"2023-10-06T03:57:11.355768+0000","last_undegraded":"2023-10-06T03:57:11.355768+0000","last_fullsized":"2023-10-06T03:57:11.355768+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:19.042145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:26.532613+0000","last_change":"2023-10-06T03:57:10.346078+0000","last_active":"2023-10-06T03:57:26.532613+0000","last_peered":"2023-10-06T03:57:26.532613+0000","last_clean":"2023-10-06T03:57:26.532613+0000","last_became_active":"2023-10-06T03:57:10.345769+0000","last_became_peered":"2023-10-06T03:57:10.345769+0000","last_unstale":"2023-10-06T03:57:26.532613+0000","last_undegraded":"2023-10-06T03:57:26.532613+0000","last_fullsized":"2023-10-06T03:57:26.532613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:16.630681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.376583+0000","last_change":"2023-10-06T03:57:11.370372+0000","last_active":"2023-10-06T03:57:11.376583+0000","last_peered":"2023-10-06T03:57:11.376583+0000","last_clean":"2023-10-06T03:57:11.376583+0000","last_became_active":"2023-10-06T03:57:11.370076+0000","last_became_peered":"2023-10-06T03:57:11.370076+0000","last_unstale":"2023-10-06T03:57:11.376583+0000","last_undegraded":"2023-10-06T03:57:11.376583+0000","last_fullsized":"2023-10-06T03:57:11.376583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:18:59.517155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.362848+0000","last_change":"2023-10-06T03:57:05.318778+0000","last_active":"2023-10-06T03:57:11.362848+0000","last_peered":"2023-10-06T03:57:11.362848+0000","last_clean":"2023-10-06T03:57:11.362848+0000","last_became_active":"2023-10-06T03:57:05.318456+0000","last_became_peered":"2023-10-06T03:57:05.318456+0000","last_unstale":"2023-10-06T03:57:11.362848+0000","last_undegraded":"2023-10-06T03:57:11.362848+0000","last_fullsized":"2023-10-06T03:57:11.362848+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:39.769241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.362382+0000","last_change":"2023-10-06T03:57:05.664245+0000","last_active":"2023-10-06T03:57:11.362382+0000","last_peered":"2023-10-06T03:57:11.362382+0000","last_clean":"2023-10-06T03:57:11.362382+0000","last_became_active":"2023-10-06T03:57:05.663426+0000","last_became_peered":"2023-10-06T03:57:05.663426+0000","last_unstale":"2023-10-06T03:57:11.362382+0000","last_undegraded":"2023-10-06T03:57:11.362382+0000","last_fullsized":"2023-10-06T03:57:11.362382+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:29.322599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"21'5","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.646148+0000","last_change":"2023-10-06T03:57:10.351059+0000","last_active":"2023-10-06T03:57:37.646148+0000","last_peered":"2023-10-06T03:57:37.646148+0000","last_clean":"2023-10-06T03:57:37.646148+0000","last_became_active":"2023-10-06T03:57:10.348879+0000","last_became_peered":"2023-10-06T03:57:10.348879+0000","last_unstale":"2023-10-06T03:57:37.646148+0000","last_undegraded":"2023-10-06T03:57:37.646148+0000","last_fullsized":"2023-10-06T03:57:37.646148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:35.764148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.375601+0000","last_change":"2023-10-06T03:57:11.373786+0000","last_active":"2023-10-06T03:57:11.375601+0000","last_peered":"2023-10-06T03:57:11.375601+0000","last_clean":"2023-10-06T03:57:11.375601+0000","last_became_active":"2023-10-06T03:57:11.373452+0000","last_became_peered":"2023-10-06T03:57:11.373452+0000","last_unstale":"2023-10-06T03:57:11.375601+0000","last_undegraded":"2023-10-06T03:57:11.375601+0000","last_fullsized":"2023-10-06T03:57:11.375601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:20.524065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.364987+0000","last_change":"2023-10-06T03:57:05.318790+0000","last_active":"2023-10-06T03:57:11.364987+0000","last_peered":"2023-10-06T03:57:11.364987+0000","last_clean":"2023-10-06T03:57:11.364987+0000","last_became_active":"2023-10-06T03:57:05.318531+0000","last_became_peered":"2023-10-06T03:57:05.318531+0000","last_unstale":"2023-10-06T03:57:11.364987+0000","last_undegraded":"2023-10-06T03:57:11.364987+0000","last_fullsized":"2023-10-06T03:57:11.364987+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:12.876998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"21'6","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.646566+0000","last_change":"2023-10-06T03:57:10.343328+0000","last_active":"2023-10-06T03:57:37.646566+0000","last_peered":"2023-10-06T03:57:37.646566+0000","last_clean":"2023-10-06T03:57:37.646566+0000","last_became_active":"2023-10-06T03:57:10.343051+0000","last_became_peered":"2023-10-06T03:57:10.343051+0000","last_unstale":"2023-10-06T03:57:37.646566+0000","last_undegraded":"2023-10-06T03:57:37.646566+0000","last_fullsized":"2023-10-06T03:57:37.646566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:27.722475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378907+0000","last_change":"2023-10-06T03:57:11.373581+0000","last_active":"2023-10-06T03:57:11.378907+0000","last_peered":"2023-10-06T03:57:11.378907+0000","last_clean":"2023-10-06T03:57:11.378907+0000","last_became_active":"2023-10-06T03:57:11.373264+0000","last_became_peered":"2023-10-06T03:57:11.373264+0000","last_unstale":"2023-10-06T03:57:11.378907+0000","last_undegraded":"2023-10-06T03:57:11.378907+0000","last_fullsized":"2023-10-06T03:57:11.378907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:15.950245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.366791+0000","last_change":"2023-10-06T03:57:05.663770+0000","last_active":"2023-10-06T03:57:11.366791+0000","last_peered":"2023-10-06T03:57:11.366791+0000","last_clean":"2023-10-06T03:57:11.366791+0000","last_became_active":"2023-10-06T03:57:05.663501+0000","last_became_peered":"2023-10-06T03:57:05.663501+0000","last_unstale":"2023-10-06T03:57:11.366791+0000","last_undegraded":"2023-10-06T03:57:11.366791+0000","last_fullsized":"2023-10-06T03:57:11.366791+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:09.806915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:12.368542+0000","last_change":"2023-10-06T03:57:10.346585+0000","last_active":"2023-10-06T03:57:12.368542+0000","last_peered":"2023-10-06T03:57:12.368542+0000","last_clean":"2023-10-06T03:57:12.368542+0000","last_became_active":"2023-10-06T03:57:10.345743+0000","last_became_peered":"2023-10-06T03:57:10.345743+0000","last_unstale":"2023-10-06T03:57:12.368542+0000","last_undegraded":"2023-10-06T03:57:12.368542+0000","last_fullsized":"2023-10-06T03:57:12.368542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:54.589702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378948+0000","last_change":"2023-10-06T03:57:11.373844+0000","last_active":"2023-10-06T03:57:11.378948+0000","last_peered":"2023-10-06T03:57:11.378948+0000","last_clean":"2023-10-06T03:57:11.378948+0000","last_became_active":"2023-10-06T03:57:11.373543+0000","last_became_peered":"2023-10-06T03:57:11.373543+0000","last_unstale":"2023-10-06T03:57:11.378948+0000","last_undegraded":"2023-10-06T03:57:11.378948+0000","last_fullsized":"2023-10-06T03:57:11.378948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:15.152354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"21'7","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.645077+0000","last_change":"2023-10-06T03:57:10.344618+0000","last_active":"2023-10-06T03:57:37.645077+0000","last_peered":"2023-10-06T03:57:37.645077+0000","last_clean":"2023-10-06T03:57:37.645077+0000","last_became_active":"2023-10-06T03:57:10.344324+0000","last_became_peered":"2023-10-06T03:57:10.344324+0000","last_unstale":"2023-10-06T03:57:37.645077+0000","last_undegraded":"2023-10-06T03:57:37.645077+0000","last_fullsized":"2023-10-06T03:57:37.645077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:06.062331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.368350+0000","last_change":"2023-10-06T03:57:05.317973+0000","last_active":"2023-10-06T03:57:11.368350+0000","last_peered":"2023-10-06T03:57:11.368350+0000","last_clean":"2023-10-06T03:57:11.368350+0000","last_became_active":"2023-10-06T03:57:05.317715+0000","last_became_peered":"2023-10-06T03:57:05.317715+0000","last_unstale":"2023-10-06T03:57:11.368350+0000","last_undegraded":"2023-10-06T03:57:11.368350+0000","last_fullsized":"2023-10-06T03:57:11.368350+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:37:51.858909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372084+0000","last_change":"2023-10-06T03:57:11.369706+0000","last_active":"2023-10-06T03:57:11.372084+0000","last_peered":"2023-10-06T03:57:11.372084+0000","last_clean":"2023-10-06T03:57:11.372084+0000","last_became_active":"2023-10-06T03:57:11.369445+0000","last_became_peered":"2023-10-06T03:57:11.369445+0000","last_unstale":"2023-10-06T03:57:11.372084+0000","last_undegraded":"2023-10-06T03:57:11.372084+0000","last_fullsized":"2023-10-06T03:57:11.372084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:52.999421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.362125+0000","last_change":"2023-10-06T03:57:05.320847+0000","last_active":"2023-10-06T03:57:11.362125+0000","last_peered":"2023-10-06T03:57:11.362125+0000","last_clean":"2023-10-06T03:57:11.362125+0000","last_became_active":"2023-10-06T03:57:05.320583+0000","last_became_peered":"2023-10-06T03:57:05.320583+0000","last_unstale":"2023-10-06T03:57:11.362125+0000","last_undegraded":"2023-10-06T03:57:11.362125+0000","last_fullsized":"2023-10-06T03:57:11.362125+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:12.772449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"21'7","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.542436+0000","last_change":"2023-10-06T03:57:10.341132+0000","last_active":"2023-10-06T03:57:32.542436+0000","last_peered":"2023-10-06T03:57:32.542436+0000","last_clean":"2023-10-06T03:57:32.542436+0000","last_became_active":"2023-10-06T03:57:10.339424+0000","last_became_peered":"2023-10-06T03:57:10.339424+0000","last_unstale":"2023-10-06T03:57:32.542436+0000","last_undegraded":"2023-10-06T03:57:32.542436+0000","last_fullsized":"2023-10-06T03:57:32.542436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:39.675711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.374372+0000","last_change":"2023-10-06T03:57:11.372497+0000","last_active":"2023-10-06T03:57:11.374372+0000","last_peered":"2023-10-06T03:57:11.374372+0000","last_clean":"2023-10-06T03:57:11.374372+0000","last_became_active":"2023-10-06T03:57:11.372215+0000","last_became_peered":"2023-10-06T03:57:11.372215+0000","last_unstale":"2023-10-06T03:57:11.374372+0000","last_undegraded":"2023-10-06T03:57:11.374372+0000","last_fullsized":"2023-10-06T03:57:11.374372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:23.559480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"21'5","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:21.447011+0000","last_change":"2023-10-06T03:57:10.344051+0000","last_active":"2023-10-06T03:57:21.447011+0000","last_peered":"2023-10-06T03:57:21.447011+0000","last_clean":"2023-10-06T03:57:21.447011+0000","last_became_active":"2023-10-06T03:57:10.343865+0000","last_became_peered":"2023-10-06T03:57:10.343865+0000","last_unstale":"2023-10-06T03:57:21.447011+0000","last_undegraded":"2023-10-06T03:57:21.447011+0000","last_fullsized":"2023-10-06T03:57:21.447011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:32.522470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.376665+0000","last_change":"2023-10-06T03:57:11.369927+0000","last_active":"2023-10-06T03:57:11.376665+0000","last_peered":"2023-10-06T03:57:11.376665+0000","last_clean":"2023-10-06T03:57:11.376665+0000","last_became_active":"2023-10-06T03:57:11.369681+0000","last_became_peered":"2023-10-06T03:57:11.369681+0000","last_unstale":"2023-10-06T03:57:11.376665+0000","last_undegraded":"2023-10-06T03:57:11.376665+0000","last_fullsized":"2023-10-06T03:57:11.376665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:53.858179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"21'5","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.646820+0000","last_change":"2023-10-06T03:57:10.349666+0000","last_active":"2023-10-06T03:57:37.646820+0000","last_peered":"2023-10-06T03:57:37.646820+0000","last_clean":"2023-10-06T03:57:37.646820+0000","last_became_active":"2023-10-06T03:57:10.349320+0000","last_became_peered":"2023-10-06T03:57:10.349320+0000","last_unstale":"2023-10-06T03:57:37.646820+0000","last_undegraded":"2023-10-06T03:57:37.646820+0000","last_fullsized":"2023-10-06T03:57:37.646820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:45:49.709519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373619+0000","last_change":"2023-10-06T03:57:11.370442+0000","last_active":"2023-10-06T03:57:11.373619+0000","last_peered":"2023-10-06T03:57:11.373619+0000","last_clean":"2023-10-06T03:57:11.373619+0000","last_became_active":"2023-10-06T03:57:11.370229+0000","last_became_peered":"2023-10-06T03:57:11.370229+0000","last_unstale":"2023-10-06T03:57:11.373619+0000","last_undegraded":"2023-10-06T03:57:11.373619+0000","last_fullsized":"2023-10-06T03:57:11.373619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:26.174331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"21'12","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.647341+0000","last_change":"2023-10-06T03:57:10.350263+0000","last_active":"2023-10-06T03:57:37.647341+0000","last_peered":"2023-10-06T03:57:37.647341+0000","last_clean":"2023-10-06T03:57:37.647341+0000","last_became_active":"2023-10-06T03:57:10.350083+0000","last_became_peered":"2023-10-06T03:57:10.350083+0000","last_unstale":"2023-10-06T03:57:37.647341+0000","last_undegraded":"2023-10-06T03:57:37.647341+0000","last_fullsized":"2023-10-06T03:57:37.647341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:05.433032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372850+0000","last_change":"2023-10-06T03:57:11.369397+0000","last_active":"2023-10-06T03:57:11.372850+0000","last_peered":"2023-10-06T03:57:11.372850+0000","last_clean":"2023-10-06T03:57:11.372850+0000","last_became_active":"2023-10-06T03:57:11.369123+0000","last_became_peered":"2023-10-06T03:57:11.369123+0000","last_unstale":"2023-10-06T03:57:11.372850+0000","last_undegraded":"2023-10-06T03:57:11.372850+0000","last_fullsized":"2023-10-06T03:57:11.372850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:28.139036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:16.370129+0000","last_change":"2023-10-06T03:57:10.347049+0000","last_active":"2023-10-06T03:57:16.370129+0000","last_peered":"2023-10-06T03:57:16.370129+0000","last_clean":"2023-10-06T03:57:16.370129+0000","last_became_active":"2023-10-06T03:57:10.344583+0000","last_became_peered":"2023-10-06T03:57:10.344583+0000","last_unstale":"2023-10-06T03:57:16.370129+0000","last_undegraded":"2023-10-06T03:57:16.370129+0000","last_fullsized":"2023-10-06T03:57:16.370129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:32.280257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.370657+0000","last_change":"2023-10-06T03:57:11.366741+0000","last_active":"2023-10-06T03:57:11.370657+0000","last_peered":"2023-10-06T03:57:11.370657+0000","last_clean":"2023-10-06T03:57:11.370657+0000","last_became_active":"2023-10-06T03:57:11.366500+0000","last_became_peered":"2023-10-06T03:57:11.366500+0000","last_unstale":"2023-10-06T03:57:11.370657+0000","last_undegraded":"2023-10-06T03:57:11.370657+0000","last_fullsized":"2023-10-06T03:57:11.370657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:24.714603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"21'13","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.645297+0000","last_change":"2023-10-06T03:57:10.348170+0000","last_active":"2023-10-06T03:57:37.645297+0000","last_peered":"2023-10-06T03:57:37.645297+0000","last_clean":"2023-10-06T03:57:37.645297+0000","last_became_active":"2023-10-06T03:57:10.347897+0000","last_became_peered":"2023-10-06T03:57:10.347897+0000","last_unstale":"2023-10-06T03:57:37.645297+0000","last_undegraded":"2023-10-06T03:57:37.645297+0000","last_fullsized":"2023-10-06T03:57:37.645297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:40.756246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372969+0000","last_change":"2023-10-06T03:57:11.364368+0000","last_active":"2023-10-06T03:57:11.372969+0000","last_peered":"2023-10-06T03:57:11.372969+0000","last_clean":"2023-10-06T03:57:11.372969+0000","last_became_active":"2023-10-06T03:57:11.364150+0000","last_became_peered":"2023-10-06T03:57:11.364150+0000","last_unstale":"2023-10-06T03:57:11.372969+0000","last_undegraded":"2023-10-06T03:57:11.372969+0000","last_fullsized":"2023-10-06T03:57:11.372969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:11.038080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.374372+0000","last_change":"2023-10-06T03:57:11.366721+0000","last_active":"2023-10-06T03:57:11.374372+0000","last_peered":"2023-10-06T03:57:11.374372+0000","last_clean":"2023-10-06T03:57:11.374372+0000","last_became_active":"2023-10-06T03:57:11.366524+0000","last_became_peered":"2023-10-06T03:57:11.366524+0000","last_unstale":"2023-10-06T03:57:11.374372+0000","last_undegraded":"2023-10-06T03:57:11.374372+0000","last_fullsized":"2023-10-06T03:57:11.374372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:33.044858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373497+0000","last_change":"2023-10-06T03:57:11.371109+0000","last_active":"2023-10-06T03:57:11.373497+0000","last_peered":"2023-10-06T03:57:11.373497+0000","last_clean":"2023-10-06T03:57:11.373497+0000","last_became_active":"2023-10-06T03:57:11.370848+0000","last_became_peered":"2023-10-06T03:57:11.370848+0000","last_unstale":"2023-10-06T03:57:11.373497+0000","last_undegraded":"2023-10-06T03:57:11.373497+0000","last_fullsized":"2023-10-06T03:57:11.373497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:15.919090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372839+0000","last_change":"2023-10-06T03:57:11.364188+0000","last_active":"2023-10-06T03:57:11.372839+0000","last_peered":"2023-10-06T03:57:11.372839+0000","last_clean":"2023-10-06T03:57:11.372839+0000","last_became_active":"2023-10-06T03:57:11.363875+0000","last_became_peered":"2023-10-06T03:57:11.363875+0000","last_unstale":"2023-10-06T03:57:11.372839+0000","last_undegraded":"2023-10-06T03:57:11.372839+0000","last_fullsized":"2023-10-06T03:57:11.372839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:29.236344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378301+0000","last_change":"2023-10-06T03:57:11.377420+0000","last_active":"2023-10-06T03:57:11.378301+0000","last_peered":"2023-10-06T03:57:11.378301+0000","last_clean":"2023-10-06T03:57:11.378301+0000","last_became_active":"2023-10-06T03:57:11.377126+0000","last_became_peered":"2023-10-06T03:57:11.377126+0000","last_unstale":"2023-10-06T03:57:11.378301+0000","last_undegraded":"2023-10-06T03:57:11.378301+0000","last_fullsized":"2023-10-06T03:57:11.378301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:05.041737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377733+0000","last_change":"2023-10-06T03:57:11.375777+0000","last_active":"2023-10-06T03:57:11.377733+0000","last_peered":"2023-10-06T03:57:11.377733+0000","last_clean":"2023-10-06T03:57:11.377733+0000","last_became_active":"2023-10-06T03:57:11.375539+0000","last_became_peered":"2023-10-06T03:57:11.375539+0000","last_unstale":"2023-10-06T03:57:11.377733+0000","last_undegraded":"2023-10-06T03:57:11.377733+0000","last_fullsized":"2023-10-06T03:57:11.377733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:42.707488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":8933,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":210,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":17866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":184,"ondisk_log_size":184,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64858,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297612,"kb_used_data":324,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074228,"statfs":{"total":96636764160,"available":96332009472,"internally_reserved":0,"allocated":331776,"data_stored":673502,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":78947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574860,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":78835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297616,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074224,"statfs":{"total":96636764160,"available":96332005376,"internally_reserved":0,"allocated":335872,"data_stored":670584,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297504,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074336,"statfs":{"total":96636764160,"available":96332120064,"internally_reserved":0,"allocated":221184,"data_stored":78925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":253952,"data_stored":81801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297560,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074280,"statfs":{"total":96636764160,"available":96332062720,"internally_reserved":0,"allocated":278528,"data_stored":83706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":82029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":2334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:57:49.616 INFO:tasks.mds_thrash.ceph_manager:clean! 2023-10-06T03:57:49.616 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:57:49.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.762+0000 7f80d57b9700 1 Processor -- start 2023-10-06T03:57:49.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.762+0000 7f80d57b9700 1 -- start start 2023-10-06T03:57:49.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.763+0000 7f80d57b9700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80d00bc3c0 0x7f80d006a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:49.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.763+0000 7f80d57b9700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f80d00c5db0 0x7f80d006a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:49.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.763+0000 7f80d57b9700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80d00c3270 con 0x7f80d0156b80 2023-10-06T03:57:49.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.763+0000 7f80d57b9700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80d0157070 con 0x7f80d00c5db0 2023-10-06T03:57:49.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.763+0000 7f80d57b9700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d006dc20 con 0x7f80d00bc3c0 2023-10-06T03:57:49.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.763+0000 7f80c67fc700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f80d00c5db0 0x7f80d006a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:49.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.763+0000 7f80cf7fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f80d0156b80 0x7f80d0152e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40838/0 (socket says 172.21.15.157:40838) 2023-10-06T03:57:49.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.763+0000 7f80cf7fe700 1 -- 172.21.15.157:0/2868350387 learned_addr learned my addr 172.21.15.157:0/2868350387 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:49.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.764+0000 7f80ceffd700 1 --2- 172.21.15.157:0/2868350387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80d00bc3c0 0x7f80d006a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:49.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.764+0000 7f80c67fc700 1 -- 172.21.15.157:0/2868350387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80d00bc3c0 msgr2=0x7f80d006a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:49.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.764+0000 7f80c67fc700 1 --2- 172.21.15.157:0/2868350387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80d00bc3c0 0x7f80d006a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:49.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.764+0000 7f80c67fc700 1 -- 172.21.15.157:0/2868350387 >> v1:172.21.15.157:6789/0 conn(0x7f80d0156b80 legacy=0x7f80d0152e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:49.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.764+0000 7f80ce7fc700 1 -- 172.21.15.157:0/2868350387 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 73997958 0 0) 0x7f80d00c3270 con 0x7f80d0156b80 2023-10-06T03:57:49.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.764+0000 7f80c67fc700 1 -- 172.21.15.157:0/2868350387 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d00c3510 con 0x7f80d00c5db0 2023-10-06T03:57:49.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.764+0000 7f80c67fc700 1 --2- 172.21.15.157:0/2868350387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80d00c5db0 0x7f80d006a9e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f80b80098e0 tx=0x7f80b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559b0082101209fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:49.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.764+0000 7f80ce7fc700 1 -- 172.21.15.157:0/2868350387 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b8009e80 con 0x7f80d00c5db0 2023-10-06T03:57:49.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.764+0000 7f80ce7fc700 1 -- 172.21.15.157:0/2868350387 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b800b8f0 con 0x7f80d00c5db0 2023-10-06T03:57:49.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.764+0000 7f80ce7fc700 1 -- 172.21.15.157:0/2868350387 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b800bc70 con 0x7f80d00c5db0 2023-10-06T03:57:49.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.764+0000 7f80d57b9700 1 -- 172.21.15.157:0/2868350387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80d00c5db0 msgr2=0x7f80d006a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:49.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.764+0000 7f80d57b9700 1 --2- 172.21.15.157:0/2868350387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80d00c5db0 0x7f80d006a9e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f80b80098e0 tx=0x7f80b80091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:49.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.764+0000 7f80d57b9700 1 -- 172.21.15.157:0/2868350387 shutdown_connections 2023-10-06T03:57:49.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.764+0000 7f80d57b9700 1 --2- 172.21.15.157:0/2868350387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80d00bc3c0 0x7f80d006a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:49.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.764+0000 7f80d57b9700 1 --2- 172.21.15.157:0/2868350387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80d00c5db0 0x7f80d006a9e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:49.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.764+0000 7f80d57b9700 1 -- 172.21.15.157:0/2868350387 >> 172.21.15.157:0/2868350387 conn(0x7f80d006b510 msgr2=0x7f80d006b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:49.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.765+0000 7f80d57b9700 1 -- 172.21.15.157:0/2868350387 shutdown_connections 2023-10-06T03:57:49.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.765+0000 7f80d57b9700 1 -- 172.21.15.157:0/2868350387 wait complete. 2023-10-06T03:57:49.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.766+0000 7f80d57b9700 1 Processor -- start 2023-10-06T03:57:49.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.766+0000 7f80d57b9700 1 -- start start 2023-10-06T03:57:49.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.766+0000 7f80d57b9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80d00bc3c0 0x7f80d006c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:49.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.766+0000 7f80d57b9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80d00c5db0 0x7f80d00c7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:49.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.766+0000 7f80d57b9700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80d0156b80 0x7f80d014ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:49.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.766+0000 7f80d57b9700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d0166840 con 0x7f80d00c5db0 2023-10-06T03:57:49.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.766+0000 7f80d57b9700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d00bba70 con 0x7f80d00bc3c0 2023-10-06T03:57:49.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.766+0000 7f80d57b9700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d0165f30 con 0x7f80d0156b80 2023-10-06T03:57:49.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.766+0000 7f80c67fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80d00c5db0 0x7f80d00c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:49.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.767+0000 7f80c67fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80d00c5db0 0x7f80d00c7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45230/0 (socket says 172.21.15.157:45230) 2023-10-06T03:57:49.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.767+0000 7f80c67fc700 1 -- 172.21.15.157:0/151542081 learned_addr learned my addr 172.21.15.157:0/151542081 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:49.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.767+0000 7f80ceffd700 1 --2- 172.21.15.157:0/151542081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80d00bc3c0 0x7f80d006c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:49.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.767+0000 7f80cf7fe700 1 --2- 172.21.15.157:0/151542081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80d0156b80 0x7f80d014ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:49.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.767+0000 7f80c67fc700 1 -- 172.21.15.157:0/151542081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80d0156b80 msgr2=0x7f80d014ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:49.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.767+0000 7f80c67fc700 1 --2- 172.21.15.157:0/151542081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80d0156b80 0x7f80d014ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:49.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.767+0000 7f80c67fc700 1 -- 172.21.15.157:0/151542081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80d00bc3c0 msgr2=0x7f80d006c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:49.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.767+0000 7f80c67fc700 1 --2- 172.21.15.157:0/151542081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80d00bc3c0 0x7f80d006c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:49.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.767+0000 7f80c67fc700 1 -- 172.21.15.157:0/151542081 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b8009420 con 0x7f80d00c5db0 2023-10-06T03:57:49.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.767+0000 7f80ceffd700 1 --2- 172.21.15.157:0/151542081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80d00bc3c0 0x7f80d006c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:57:49.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.767+0000 7f80c67fc700 1 --2- 172.21.15.157:0/151542081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80d00c5db0 0x7f80d00c7230 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f80b800b3c0 tx=0x7f80b8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:49.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.767+0000 7f80c77fe700 1 -- 172.21.15.157:0/151542081 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b8011420 con 0x7f80d00c5db0 2023-10-06T03:57:49.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.767+0000 7f80c77fe700 1 -- 172.21.15.157:0/151542081 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b8011580 con 0x7f80d00c5db0 2023-10-06T03:57:49.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.767+0000 7f80d57b9700 1 -- 172.21.15.157:0/151542081 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80d00bbd10 con 0x7f80d00c5db0 2023-10-06T03:57:49.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.767+0000 7f80c77fe700 1 -- 172.21.15.157:0/151542081 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b8011940 con 0x7f80d00c5db0 2023-10-06T03:57:49.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.767+0000 7f80d57b9700 1 -- 172.21.15.157:0/151542081 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d01664d0 con 0x7f80d00c5db0 2023-10-06T03:57:49.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.768+0000 7f80d57b9700 1 -- 172.21.15.157:0/151542081 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80d01669b0 con 0x7f80d00c5db0 2023-10-06T03:57:49.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.768+0000 7f80c77fe700 1 -- 172.21.15.157:0/151542081 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f80b802c3b0 con 0x7f80d00c5db0 2023-10-06T03:57:49.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.769+0000 7f80c77fe700 1 --2- 172.21.15.157:0/151542081 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f80b0066f30 0x7f80b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:49.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.769+0000 7f80c77fe700 1 -- 172.21.15.157:0/151542081 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f80b8085dd0 con 0x7f80d00c5db0 2023-10-06T03:57:49.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.769+0000 7f80ceffd700 1 --2- 172.21.15.157:0/151542081 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f80b0066f30 0x7f80b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:49.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.770+0000 7f80ceffd700 1 --2- 172.21.15.157:0/151542081 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f80b0066f30 0x7f80b00693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f80c0006fd0 tx=0x7f80c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:49.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.773+0000 7f80c77fe700 1 -- 172.21.15.157:0/151542081 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f80b8052f60 con 0x7f80d00c5db0 2023-10-06T03:57:49.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.952+0000 7f80d57b9700 1 -- 172.21.15.157:0/151542081 --> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f80d004bb40 con 0x7f80b0066f30 2023-10-06T03:57:49.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.960+0000 7f80c77fe700 1 -- 172.21.15.157:0/151542081 <== mgr.4107 v2:172.21.15.175:6832/90661 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286811 (secure 0 0 0) 0x7f80d004bb40 con 0x7f80b0066f30 2023-10-06T03:57:49.965 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:49.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.964+0000 7f80d57b9700 1 -- 172.21.15.157:0/151542081 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f80b0066f30 msgr2=0x7f80b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:49.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.964+0000 7f80d57b9700 1 --2- 172.21.15.157:0/151542081 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f80b0066f30 0x7f80b00693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f80c0006fd0 tx=0x7f80c0008040 comp rx=0 tx=0).stop 2023-10-06T03:57:49.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.964+0000 7f80d57b9700 1 -- 172.21.15.157:0/151542081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80d00c5db0 msgr2=0x7f80d00c7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:49.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.964+0000 7f80d57b9700 1 --2- 172.21.15.157:0/151542081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80d00c5db0 0x7f80d00c7230 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f80b800b3c0 tx=0x7f80b8017000 comp rx=0 tx=0).stop 2023-10-06T03:57:49.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.964+0000 7f80d57b9700 1 -- 172.21.15.157:0/151542081 shutdown_connections 2023-10-06T03:57:49.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.964+0000 7f80d57b9700 1 --2- 172.21.15.157:0/151542081 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f80b0066f30 0x7f80b00693e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:49.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.964+0000 7f80d57b9700 1 --2- 172.21.15.157:0/151542081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80d00bc3c0 0x7f80d006c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:49.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.964+0000 7f80d57b9700 1 --2- 172.21.15.157:0/151542081 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80d0156b80 0x7f80d014ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:49.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.964+0000 7f80d57b9700 1 --2- 172.21.15.157:0/151542081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80d00c5db0 0x7f80d00c7230 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:49.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.964+0000 7f80d57b9700 1 -- 172.21.15.157:0/151542081 >> 172.21.15.157:0/151542081 conn(0x7f80d006b510 msgr2=0x7f80d006a480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:49.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.964+0000 7f80d57b9700 1 -- 172.21.15.157:0/151542081 shutdown_connections 2023-10-06T03:57:49.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:49.964+0000 7f80d57b9700 1 -- 172.21.15.157:0/151542081 wait complete. 2023-10-06T03:57:49.969 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-10-06T03:57:49.977 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2023-10-06T03:57:49.435196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":599301,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":369,"num_write_kb":2478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":301,"ondisk_log_size":301,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2380380,"kb_used_data":2076,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752594340,"statfs":{"total":773094113280,"available":770656604160,"internally_reserved":0,"allocated":2125824,"data_stored":1828329,"data_compressed":64858,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3967,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003133"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378564+0000","last_change":"2023-10-06T03:57:11.372019+0000","last_active":"2023-10-06T03:57:11.378564+0000","last_peered":"2023-10-06T03:57:11.378564+0000","last_clean":"2023-10-06T03:57:11.378564+0000","last_became_active":"2023-10-06T03:57:11.371756+0000","last_became_peered":"2023-10-06T03:57:11.371756+0000","last_unstale":"2023-10-06T03:57:11.378564+0000","last_undegraded":"2023-10-06T03:57:11.378564+0000","last_fullsized":"2023-10-06T03:57:11.378564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:50.086528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372505+0000","last_change":"2023-10-06T03:57:11.370371+0000","last_active":"2023-10-06T03:57:11.372505+0000","last_peered":"2023-10-06T03:57:11.372505+0000","last_clean":"2023-10-06T03:57:11.372505+0000","last_became_active":"2023-10-06T03:57:11.370070+0000","last_became_peered":"2023-10-06T03:57:11.370070+0000","last_unstale":"2023-10-06T03:57:11.372505+0000","last_undegraded":"2023-10-06T03:57:11.372505+0000","last_fullsized":"2023-10-06T03:57:11.372505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:00.917607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371725+0000","last_change":"2023-10-06T03:57:11.366298+0000","last_active":"2023-10-06T03:57:11.371725+0000","last_peered":"2023-10-06T03:57:11.371725+0000","last_clean":"2023-10-06T03:57:11.371725+0000","last_became_active":"2023-10-06T03:57:11.366012+0000","last_became_peered":"2023-10-06T03:57:11.366012+0000","last_unstale":"2023-10-06T03:57:11.371725+0000","last_undegraded":"2023-10-06T03:57:11.371725+0000","last_fullsized":"2023-10-06T03:57:11.371725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:45.802287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.374334+0000","last_change":"2023-10-06T03:57:11.365933+0000","last_active":"2023-10-06T03:57:11.374334+0000","last_peered":"2023-10-06T03:57:11.374334+0000","last_clean":"2023-10-06T03:57:11.374334+0000","last_became_active":"2023-10-06T03:57:11.365673+0000","last_became_peered":"2023-10-06T03:57:11.365673+0000","last_unstale":"2023-10-06T03:57:11.374334+0000","last_undegraded":"2023-10-06T03:57:11.374334+0000","last_fullsized":"2023-10-06T03:57:11.374334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:40.953382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.365753+0000","last_change":"2023-10-06T03:57:11.361569+0000","last_active":"2023-10-06T03:57:11.365753+0000","last_peered":"2023-10-06T03:57:11.365753+0000","last_clean":"2023-10-06T03:57:11.365753+0000","last_became_active":"2023-10-06T03:57:11.361302+0000","last_became_peered":"2023-10-06T03:57:11.361302+0000","last_unstale":"2023-10-06T03:57:11.365753+0000","last_undegraded":"2023-10-06T03:57:11.365753+0000","last_fullsized":"2023-10-06T03:57:11.365753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:57.552493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372859+0000","last_change":"2023-10-06T03:57:11.369953+0000","last_active":"2023-10-06T03:57:11.372859+0000","last_peered":"2023-10-06T03:57:11.372859+0000","last_clean":"2023-10-06T03:57:11.372859+0000","last_became_active":"2023-10-06T03:57:11.368775+0000","last_became_peered":"2023-10-06T03:57:11.368775+0000","last_unstale":"2023-10-06T03:57:11.372859+0000","last_undegraded":"2023-10-06T03:57:11.372859+0000","last_fullsized":"2023-10-06T03:57:11.372859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:22.368117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.380867+0000","last_change":"2023-10-06T03:57:11.378048+0000","last_active":"2023-10-06T03:57:11.380867+0000","last_peered":"2023-10-06T03:57:11.380867+0000","last_clean":"2023-10-06T03:57:11.380867+0000","last_became_active":"2023-10-06T03:57:11.377786+0000","last_became_peered":"2023-10-06T03:57:11.377786+0000","last_unstale":"2023-10-06T03:57:11.380867+0000","last_undegraded":"2023-10-06T03:57:11.380867+0000","last_fullsized":"2023-10-06T03:57:11.380867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:12:54.218593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377264+0000","last_change":"2023-10-06T03:57:11.374003+0000","last_active":"2023-10-06T03:57:11.377264+0000","last_peered":"2023-10-06T03:57:11.377264+0000","last_clean":"2023-10-06T03:57:11.377264+0000","last_became_active":"2023-10-06T03:57:11.373765+0000","last_became_peered":"2023-10-06T03:57:11.373765+0000","last_unstale":"2023-10-06T03:57:11.377264+0000","last_undegraded":"2023-10-06T03:57:11.377264+0000","last_fullsized":"2023-10-06T03:57:11.377264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:05.838007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378921+0000","last_change":"2023-10-06T03:57:11.373794+0000","last_active":"2023-10-06T03:57:11.378921+0000","last_peered":"2023-10-06T03:57:11.378921+0000","last_clean":"2023-10-06T03:57:11.378921+0000","last_became_active":"2023-10-06T03:57:11.368004+0000","last_became_peered":"2023-10-06T03:57:11.368004+0000","last_unstale":"2023-10-06T03:57:11.378921+0000","last_undegraded":"2023-10-06T03:57:11.378921+0000","last_fullsized":"2023-10-06T03:57:11.378921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:16.760049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.382630+0000","last_change":"2023-10-06T03:57:11.379370+0000","last_active":"2023-10-06T03:57:11.382630+0000","last_peered":"2023-10-06T03:57:11.382630+0000","last_clean":"2023-10-06T03:57:11.382630+0000","last_became_active":"2023-10-06T03:57:11.379107+0000","last_became_peered":"2023-10-06T03:57:11.379107+0000","last_unstale":"2023-10-06T03:57:11.382630+0000","last_undegraded":"2023-10-06T03:57:11.382630+0000","last_fullsized":"2023-10-06T03:57:11.382630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:18.373103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371742+0000","last_change":"2023-10-06T03:57:11.369172+0000","last_active":"2023-10-06T03:57:11.371742+0000","last_peered":"2023-10-06T03:57:11.371742+0000","last_clean":"2023-10-06T03:57:11.371742+0000","last_became_active":"2023-10-06T03:57:11.368256+0000","last_became_peered":"2023-10-06T03:57:11.368256+0000","last_unstale":"2023-10-06T03:57:11.371742+0000","last_undegraded":"2023-10-06T03:57:11.371742+0000","last_fullsized":"2023-10-06T03:57:11.371742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:50.736367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373531+0000","last_change":"2023-10-06T03:57:11.368685+0000","last_active":"2023-10-06T03:57:11.373531+0000","last_peered":"2023-10-06T03:57:11.373531+0000","last_clean":"2023-10-06T03:57:11.373531+0000","last_became_active":"2023-10-06T03:57:11.368425+0000","last_became_peered":"2023-10-06T03:57:11.368425+0000","last_unstale":"2023-10-06T03:57:11.373531+0000","last_undegraded":"2023-10-06T03:57:11.373531+0000","last_fullsized":"2023-10-06T03:57:11.373531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:06.004423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372891+0000","last_change":"2023-10-06T03:57:11.367646+0000","last_active":"2023-10-06T03:57:11.372891+0000","last_peered":"2023-10-06T03:57:11.372891+0000","last_clean":"2023-10-06T03:57:11.372891+0000","last_became_active":"2023-10-06T03:57:11.367079+0000","last_became_peered":"2023-10-06T03:57:11.367079+0000","last_unstale":"2023-10-06T03:57:11.372891+0000","last_undegraded":"2023-10-06T03:57:11.372891+0000","last_fullsized":"2023-10-06T03:57:11.372891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:09:54.331206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377012+0000","last_change":"2023-10-06T03:57:11.373481+0000","last_active":"2023-10-06T03:57:11.377012+0000","last_peered":"2023-10-06T03:57:11.377012+0000","last_clean":"2023-10-06T03:57:11.377012+0000","last_became_active":"2023-10-06T03:57:11.373216+0000","last_became_peered":"2023-10-06T03:57:11.373216+0000","last_unstale":"2023-10-06T03:57:11.377012+0000","last_undegraded":"2023-10-06T03:57:11.377012+0000","last_fullsized":"2023-10-06T03:57:11.377012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:56.083203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.379250+0000","last_change":"2023-10-06T03:57:11.376012+0000","last_active":"2023-10-06T03:57:11.379250+0000","last_peered":"2023-10-06T03:57:11.379250+0000","last_clean":"2023-10-06T03:57:11.379250+0000","last_became_active":"2023-10-06T03:57:11.375689+0000","last_became_peered":"2023-10-06T03:57:11.375689+0000","last_unstale":"2023-10-06T03:57:11.379250+0000","last_undegraded":"2023-10-06T03:57:11.379250+0000","last_fullsized":"2023-10-06T03:57:11.379250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:42.953739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372989+0000","last_change":"2023-10-06T03:57:11.366808+0000","last_active":"2023-10-06T03:57:11.372989+0000","last_peered":"2023-10-06T03:57:11.372989+0000","last_clean":"2023-10-06T03:57:11.372989+0000","last_became_active":"2023-10-06T03:57:11.366543+0000","last_became_peered":"2023-10-06T03:57:11.366543+0000","last_unstale":"2023-10-06T03:57:11.372989+0000","last_undegraded":"2023-10-06T03:57:11.372989+0000","last_fullsized":"2023-10-06T03:57:11.372989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:16.212967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378753+0000","last_change":"2023-10-06T03:57:11.367276+0000","last_active":"2023-10-06T03:57:11.378753+0000","last_peered":"2023-10-06T03:57:11.378753+0000","last_clean":"2023-10-06T03:57:11.378753+0000","last_became_active":"2023-10-06T03:57:11.366767+0000","last_became_peered":"2023-10-06T03:57:11.366767+0000","last_unstale":"2023-10-06T03:57:11.378753+0000","last_undegraded":"2023-10-06T03:57:11.378753+0000","last_fullsized":"2023-10-06T03:57:11.378753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:43.113575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367129+0000","last_change":"2023-10-06T03:57:11.362667+0000","last_active":"2023-10-06T03:57:11.367129+0000","last_peered":"2023-10-06T03:57:11.367129+0000","last_clean":"2023-10-06T03:57:11.367129+0000","last_became_active":"2023-10-06T03:57:11.362377+0000","last_became_peered":"2023-10-06T03:57:11.362377+0000","last_unstale":"2023-10-06T03:57:11.367129+0000","last_undegraded":"2023-10-06T03:57:11.367129+0000","last_fullsized":"2023-10-06T03:57:11.367129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:40.600383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.365567+0000","last_change":"2023-10-06T03:57:11.364126+0000","last_active":"2023-10-06T03:57:11.365567+0000","last_peered":"2023-10-06T03:57:11.365567+0000","last_clean":"2023-10-06T03:57:11.365567+0000","last_became_active":"2023-10-06T03:57:11.363792+0000","last_became_peered":"2023-10-06T03:57:11.363792+0000","last_unstale":"2023-10-06T03:57:11.365567+0000","last_undegraded":"2023-10-06T03:57:11.365567+0000","last_fullsized":"2023-10-06T03:57:11.365567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:49.185450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372624+0000","last_change":"2023-10-06T03:57:11.364974+0000","last_active":"2023-10-06T03:57:11.372624+0000","last_peered":"2023-10-06T03:57:11.372624+0000","last_clean":"2023-10-06T03:57:11.372624+0000","last_became_active":"2023-10-06T03:57:11.364726+0000","last_became_peered":"2023-10-06T03:57:11.364726+0000","last_unstale":"2023-10-06T03:57:11.372624+0000","last_undegraded":"2023-10-06T03:57:11.372624+0000","last_fullsized":"2023-10-06T03:57:11.372624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:56.866733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.379227+0000","last_change":"2023-10-06T03:57:11.375987+0000","last_active":"2023-10-06T03:57:11.379227+0000","last_peered":"2023-10-06T03:57:11.379227+0000","last_clean":"2023-10-06T03:57:11.379227+0000","last_became_active":"2023-10-06T03:57:11.375668+0000","last_became_peered":"2023-10-06T03:57:11.375668+0000","last_unstale":"2023-10-06T03:57:11.379227+0000","last_undegraded":"2023-10-06T03:57:11.379227+0000","last_fullsized":"2023-10-06T03:57:11.379227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:53.810875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.366478+0000","last_change":"2023-10-06T03:57:11.365414+0000","last_active":"2023-10-06T03:57:11.366478+0000","last_peered":"2023-10-06T03:57:11.366478+0000","last_clean":"2023-10-06T03:57:11.366478+0000","last_became_active":"2023-10-06T03:57:11.365149+0000","last_became_peered":"2023-10-06T03:57:11.365149+0000","last_unstale":"2023-10-06T03:57:11.366478+0000","last_undegraded":"2023-10-06T03:57:11.366478+0000","last_fullsized":"2023-10-06T03:57:11.366478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:20.911710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.370638+0000","last_change":"2023-10-06T03:57:11.362366+0000","last_active":"2023-10-06T03:57:11.370638+0000","last_peered":"2023-10-06T03:57:11.370638+0000","last_clean":"2023-10-06T03:57:11.370638+0000","last_became_active":"2023-10-06T03:57:11.362128+0000","last_became_peered":"2023-10-06T03:57:11.362128+0000","last_unstale":"2023-10-06T03:57:11.370638+0000","last_undegraded":"2023-10-06T03:57:11.370638+0000","last_fullsized":"2023-10-06T03:57:11.370638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:53.744674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.379282+0000","last_change":"2023-10-06T03:57:11.375969+0000","last_active":"2023-10-06T03:57:11.379282+0000","last_peered":"2023-10-06T03:57:11.379282+0000","last_clean":"2023-10-06T03:57:11.379282+0000","last_became_active":"2023-10-06T03:57:11.375702+0000","last_became_peered":"2023-10-06T03:57:11.375702+0000","last_unstale":"2023-10-06T03:57:11.379282+0000","last_undegraded":"2023-10-06T03:57:11.379282+0000","last_fullsized":"2023-10-06T03:57:11.379282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:31.204231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377763+0000","last_change":"2023-10-06T03:57:11.375099+0000","last_active":"2023-10-06T03:57:11.377763+0000","last_peered":"2023-10-06T03:57:11.377763+0000","last_clean":"2023-10-06T03:57:11.377763+0000","last_became_active":"2023-10-06T03:57:11.374842+0000","last_became_peered":"2023-10-06T03:57:11.374842+0000","last_unstale":"2023-10-06T03:57:11.377763+0000","last_undegraded":"2023-10-06T03:57:11.377763+0000","last_fullsized":"2023-10-06T03:57:11.377763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:54.438974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.369108+0000","last_change":"2023-10-06T03:57:11.365900+0000","last_active":"2023-10-06T03:57:11.369108+0000","last_peered":"2023-10-06T03:57:11.369108+0000","last_clean":"2023-10-06T03:57:11.369108+0000","last_became_active":"2023-10-06T03:57:11.361349+0000","last_became_peered":"2023-10-06T03:57:11.361349+0000","last_unstale":"2023-10-06T03:57:11.369108+0000","last_undegraded":"2023-10-06T03:57:11.369108+0000","last_fullsized":"2023-10-06T03:57:11.369108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:30.646781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378945+0000","last_change":"2023-10-06T03:57:11.376082+0000","last_active":"2023-10-06T03:57:11.378945+0000","last_peered":"2023-10-06T03:57:11.378945+0000","last_clean":"2023-10-06T03:57:11.378945+0000","last_became_active":"2023-10-06T03:57:11.375846+0000","last_became_peered":"2023-10-06T03:57:11.375846+0000","last_unstale":"2023-10-06T03:57:11.378945+0000","last_undegraded":"2023-10-06T03:57:11.378945+0000","last_fullsized":"2023-10-06T03:57:11.378945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:28.080614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"21'7","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.546542+0000","last_change":"2023-10-06T03:57:10.348317+0000","last_active":"2023-10-06T03:57:32.546542+0000","last_peered":"2023-10-06T03:57:32.546542+0000","last_clean":"2023-10-06T03:57:32.546542+0000","last_became_active":"2023-10-06T03:57:10.348078+0000","last_became_peered":"2023-10-06T03:57:10.348078+0000","last_unstale":"2023-10-06T03:57:32.546542+0000","last_undegraded":"2023-10-06T03:57:32.546542+0000","last_fullsized":"2023-10-06T03:57:32.546542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:55.058324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372610+0000","last_change":"2023-10-06T03:57:11.370506+0000","last_active":"2023-10-06T03:57:11.372610+0000","last_peered":"2023-10-06T03:57:11.372610+0000","last_clean":"2023-10-06T03:57:11.372610+0000","last_became_active":"2023-10-06T03:57:11.370245+0000","last_became_peered":"2023-10-06T03:57:11.370245+0000","last_unstale":"2023-10-06T03:57:11.372610+0000","last_undegraded":"2023-10-06T03:57:11.372610+0000","last_fullsized":"2023-10-06T03:57:11.372610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:47:18.189772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:26.532954+0000","last_change":"2023-10-06T03:57:10.345401+0000","last_active":"2023-10-06T03:57:26.532954+0000","last_peered":"2023-10-06T03:57:26.532954+0000","last_clean":"2023-10-06T03:57:26.532954+0000","last_became_active":"2023-10-06T03:57:10.345150+0000","last_became_peered":"2023-10-06T03:57:10.345150+0000","last_unstale":"2023-10-06T03:57:26.532954+0000","last_undegraded":"2023-10-06T03:57:26.532954+0000","last_fullsized":"2023-10-06T03:57:26.532954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:25.758193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.379135+0000","last_change":"2023-10-06T03:57:11.376046+0000","last_active":"2023-10-06T03:57:11.379135+0000","last_peered":"2023-10-06T03:57:11.379135+0000","last_clean":"2023-10-06T03:57:11.379135+0000","last_became_active":"2023-10-06T03:57:11.375790+0000","last_became_peered":"2023-10-06T03:57:11.375790+0000","last_unstale":"2023-10-06T03:57:11.379135+0000","last_undegraded":"2023-10-06T03:57:11.379135+0000","last_fullsized":"2023-10-06T03:57:11.379135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:33.775431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.540480+0000","last_change":"2023-10-06T03:57:10.347562+0000","last_active":"2023-10-06T03:57:32.540480+0000","last_peered":"2023-10-06T03:57:32.540480+0000","last_clean":"2023-10-06T03:57:32.540480+0000","last_became_active":"2023-10-06T03:57:10.347294+0000","last_became_peered":"2023-10-06T03:57:10.347294+0000","last_unstale":"2023-10-06T03:57:32.540480+0000","last_undegraded":"2023-10-06T03:57:32.540480+0000","last_fullsized":"2023-10-06T03:57:32.540480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:25.090992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.375571+0000","last_change":"2023-10-06T03:57:11.372022+0000","last_active":"2023-10-06T03:57:11.375571+0000","last_peered":"2023-10-06T03:57:11.375571+0000","last_clean":"2023-10-06T03:57:11.375571+0000","last_became_active":"2023-10-06T03:57:11.371758+0000","last_became_peered":"2023-10-06T03:57:11.371758+0000","last_unstale":"2023-10-06T03:57:11.375571+0000","last_undegraded":"2023-10-06T03:57:11.375571+0000","last_fullsized":"2023-10-06T03:57:11.375571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:10.922157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:21.441650+0000","last_change":"2023-10-06T03:57:10.347406+0000","last_active":"2023-10-06T03:57:21.441650+0000","last_peered":"2023-10-06T03:57:21.441650+0000","last_clean":"2023-10-06T03:57:21.441650+0000","last_became_active":"2023-10-06T03:57:10.347151+0000","last_became_peered":"2023-10-06T03:57:10.347151+0000","last_unstale":"2023-10-06T03:57:21.441650+0000","last_undegraded":"2023-10-06T03:57:21.441650+0000","last_fullsized":"2023-10-06T03:57:21.441650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:20.427620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372580+0000","last_change":"2023-10-06T03:57:11.370240+0000","last_active":"2023-10-06T03:57:11.372580+0000","last_peered":"2023-10-06T03:57:11.372580+0000","last_clean":"2023-10-06T03:57:11.372580+0000","last_became_active":"2023-10-06T03:57:11.369938+0000","last_became_peered":"2023-10-06T03:57:11.369938+0000","last_unstale":"2023-10-06T03:57:11.372580+0000","last_undegraded":"2023-10-06T03:57:11.372580+0000","last_fullsized":"2023-10-06T03:57:11.372580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:07.105832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.646156+0000","last_change":"2023-10-06T03:57:10.346054+0000","last_active":"2023-10-06T03:57:37.646156+0000","last_peered":"2023-10-06T03:57:37.646156+0000","last_clean":"2023-10-06T03:57:37.646156+0000","last_became_active":"2023-10-06T03:57:10.345721+0000","last_became_peered":"2023-10-06T03:57:10.345721+0000","last_unstale":"2023-10-06T03:57:37.646156+0000","last_undegraded":"2023-10-06T03:57:37.646156+0000","last_fullsized":"2023-10-06T03:57:37.646156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:54.481304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378702+0000","last_change":"2023-10-06T03:57:11.376525+0000","last_active":"2023-10-06T03:57:11.378702+0000","last_peered":"2023-10-06T03:57:11.378702+0000","last_clean":"2023-10-06T03:57:11.378702+0000","last_became_active":"2023-10-06T03:57:11.376291+0000","last_became_peered":"2023-10-06T03:57:11.376291+0000","last_unstale":"2023-10-06T03:57:11.378702+0000","last_undegraded":"2023-10-06T03:57:11.378702+0000","last_fullsized":"2023-10-06T03:57:11.378702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:31.993426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"21'5","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:21.435357+0000","last_change":"2023-10-06T03:57:10.340531+0000","last_active":"2023-10-06T03:57:21.435357+0000","last_peered":"2023-10-06T03:57:21.435357+0000","last_clean":"2023-10-06T03:57:21.435357+0000","last_became_active":"2023-10-06T03:57:10.340232+0000","last_became_peered":"2023-10-06T03:57:10.340232+0000","last_unstale":"2023-10-06T03:57:21.435357+0000","last_undegraded":"2023-10-06T03:57:21.435357+0000","last_fullsized":"2023-10-06T03:57:21.435357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:43:26.332055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377803+0000","last_change":"2023-10-06T03:57:11.375070+0000","last_active":"2023-10-06T03:57:11.377803+0000","last_peered":"2023-10-06T03:57:11.377803+0000","last_clean":"2023-10-06T03:57:11.377803+0000","last_became_active":"2023-10-06T03:57:11.374892+0000","last_became_peered":"2023-10-06T03:57:11.374892+0000","last_unstale":"2023-10-06T03:57:11.377803+0000","last_undegraded":"2023-10-06T03:57:11.377803+0000","last_fullsized":"2023-10-06T03:57:11.377803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:51.198745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"21'13","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:45.217086+0000","last_change":"2023-10-06T03:57:10.346758+0000","last_active":"2023-10-06T03:57:45.217086+0000","last_peered":"2023-10-06T03:57:45.217086+0000","last_clean":"2023-10-06T03:57:45.217086+0000","last_became_active":"2023-10-06T03:57:10.346430+0000","last_became_peered":"2023-10-06T03:57:10.346430+0000","last_unstale":"2023-10-06T03:57:45.217086+0000","last_undegraded":"2023-10-06T03:57:45.217086+0000","last_fullsized":"2023-10-06T03:57:45.217086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:10.809242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2583,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371658+0000","last_change":"2023-10-06T03:57:11.367808+0000","last_active":"2023-10-06T03:57:11.371658+0000","last_peered":"2023-10-06T03:57:11.371658+0000","last_clean":"2023-10-06T03:57:11.371658+0000","last_became_active":"2023-10-06T03:57:11.367565+0000","last_became_peered":"2023-10-06T03:57:11.367565+0000","last_unstale":"2023-10-06T03:57:11.371658+0000","last_undegraded":"2023-10-06T03:57:11.371658+0000","last_fullsized":"2023-10-06T03:57:11.371658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:14.409116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"21'9","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.646018+0000","last_change":"2023-10-06T03:57:10.351062+0000","last_active":"2023-10-06T03:57:37.646018+0000","last_peered":"2023-10-06T03:57:37.646018+0000","last_clean":"2023-10-06T03:57:37.646018+0000","last_became_active":"2023-10-06T03:57:10.350795+0000","last_became_peered":"2023-10-06T03:57:10.350795+0000","last_unstale":"2023-10-06T03:57:37.646018+0000","last_undegraded":"2023-10-06T03:57:37.646018+0000","last_fullsized":"2023-10-06T03:57:37.646018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:54:45.714518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.370549+0000","last_change":"2023-10-06T03:57:11.367489+0000","last_active":"2023-10-06T03:57:11.370549+0000","last_peered":"2023-10-06T03:57:11.370549+0000","last_clean":"2023-10-06T03:57:11.370549+0000","last_became_active":"2023-10-06T03:57:11.367254+0000","last_became_peered":"2023-10-06T03:57:11.367254+0000","last_unstale":"2023-10-06T03:57:11.370549+0000","last_undegraded":"2023-10-06T03:57:11.370549+0000","last_fullsized":"2023-10-06T03:57:11.370549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:04.706485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"21'5","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.544381+0000","last_change":"2023-10-06T03:57:10.338125+0000","last_active":"2023-10-06T03:57:32.544381+0000","last_peered":"2023-10-06T03:57:32.544381+0000","last_clean":"2023-10-06T03:57:32.544381+0000","last_became_active":"2023-10-06T03:57:10.337798+0000","last_became_peered":"2023-10-06T03:57:10.337798+0000","last_unstale":"2023-10-06T03:57:32.544381+0000","last_undegraded":"2023-10-06T03:57:32.544381+0000","last_fullsized":"2023-10-06T03:57:32.544381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:00.482657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378957+0000","last_change":"2023-10-06T03:57:11.376461+0000","last_active":"2023-10-06T03:57:11.378957+0000","last_peered":"2023-10-06T03:57:11.378957+0000","last_clean":"2023-10-06T03:57:11.378957+0000","last_became_active":"2023-10-06T03:57:11.376223+0000","last_became_peered":"2023-10-06T03:57:11.376223+0000","last_unstale":"2023-10-06T03:57:11.378957+0000","last_undegraded":"2023-10-06T03:57:11.378957+0000","last_fullsized":"2023-10-06T03:57:11.378957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:04:37.408871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"21'6","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.547205+0000","last_change":"2023-10-06T03:57:10.350511+0000","last_active":"2023-10-06T03:57:32.547205+0000","last_peered":"2023-10-06T03:57:32.547205+0000","last_clean":"2023-10-06T03:57:32.547205+0000","last_became_active":"2023-10-06T03:57:10.350219+0000","last_became_peered":"2023-10-06T03:57:10.350219+0000","last_unstale":"2023-10-06T03:57:32.547205+0000","last_undegraded":"2023-10-06T03:57:32.547205+0000","last_fullsized":"2023-10-06T03:57:32.547205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:56.043506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":800,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.364939+0000","last_change":"2023-10-06T03:57:11.362700+0000","last_active":"2023-10-06T03:57:11.364939+0000","last_peered":"2023-10-06T03:57:11.364939+0000","last_clean":"2023-10-06T03:57:11.364939+0000","last_became_active":"2023-10-06T03:57:11.362443+0000","last_became_peered":"2023-10-06T03:57:11.362443+0000","last_unstale":"2023-10-06T03:57:11.364939+0000","last_undegraded":"2023-10-06T03:57:11.364939+0000","last_fullsized":"2023-10-06T03:57:11.364939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:21:09.143085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"21'17","reported_seq":27,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.541795+0000","last_change":"2023-10-06T03:57:10.346259+0000","last_active":"2023-10-06T03:57:32.541795+0000","last_peered":"2023-10-06T03:57:32.541795+0000","last_clean":"2023-10-06T03:57:32.541795+0000","last_became_active":"2023-10-06T03:57:10.345424+0000","last_became_peered":"2023-10-06T03:57:10.345424+0000","last_unstale":"2023-10-06T03:57:32.541795+0000","last_undegraded":"2023-10-06T03:57:32.541795+0000","last_fullsized":"2023-10-06T03:57:32.541795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:19.278206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378006+0000","last_change":"2023-10-06T03:57:11.376053+0000","last_active":"2023-10-06T03:57:11.378006+0000","last_peered":"2023-10-06T03:57:11.378006+0000","last_clean":"2023-10-06T03:57:11.378006+0000","last_became_active":"2023-10-06T03:57:11.375824+0000","last_became_peered":"2023-10-06T03:57:11.375824+0000","last_unstale":"2023-10-06T03:57:11.378006+0000","last_undegraded":"2023-10-06T03:57:11.378006+0000","last_fullsized":"2023-10-06T03:57:11.378006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:24:26.156866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:16.374858+0000","last_change":"2023-10-06T03:57:10.350408+0000","last_active":"2023-10-06T03:57:16.374858+0000","last_peered":"2023-10-06T03:57:16.374858+0000","last_clean":"2023-10-06T03:57:16.374858+0000","last_became_active":"2023-10-06T03:57:10.350135+0000","last_became_peered":"2023-10-06T03:57:10.350135+0000","last_unstale":"2023-10-06T03:57:16.374858+0000","last_undegraded":"2023-10-06T03:57:16.374858+0000","last_fullsized":"2023-10-06T03:57:16.374858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:43.522111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378954+0000","last_change":"2023-10-06T03:57:11.376524+0000","last_active":"2023-10-06T03:57:11.378954+0000","last_peered":"2023-10-06T03:57:11.378954+0000","last_clean":"2023-10-06T03:57:11.378954+0000","last_became_active":"2023-10-06T03:57:11.376236+0000","last_became_peered":"2023-10-06T03:57:11.376236+0000","last_unstale":"2023-10-06T03:57:11.378954+0000","last_undegraded":"2023-10-06T03:57:11.378954+0000","last_fullsized":"2023-10-06T03:57:11.378954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:33.854818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"21'5","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.644387+0000","last_change":"2023-10-06T03:57:10.345799+0000","last_active":"2023-10-06T03:57:37.644387+0000","last_peered":"2023-10-06T03:57:37.644387+0000","last_clean":"2023-10-06T03:57:37.644387+0000","last_became_active":"2023-10-06T03:57:10.345510+0000","last_became_peered":"2023-10-06T03:57:10.345510+0000","last_unstale":"2023-10-06T03:57:37.644387+0000","last_undegraded":"2023-10-06T03:57:37.644387+0000","last_fullsized":"2023-10-06T03:57:37.644387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:09.567314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372383+0000","last_change":"2023-10-06T03:57:11.368662+0000","last_active":"2023-10-06T03:57:11.372383+0000","last_peered":"2023-10-06T03:57:11.372383+0000","last_clean":"2023-10-06T03:57:11.372383+0000","last_became_active":"2023-10-06T03:57:11.368375+0000","last_became_peered":"2023-10-06T03:57:11.368375+0000","last_unstale":"2023-10-06T03:57:11.372383+0000","last_undegraded":"2023-10-06T03:57:11.372383+0000","last_fullsized":"2023-10-06T03:57:11.372383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:27.125523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.543509+0000","last_change":"2023-10-06T03:57:10.346945+0000","last_active":"2023-10-06T03:57:32.543509+0000","last_peered":"2023-10-06T03:57:32.543509+0000","last_clean":"2023-10-06T03:57:32.543509+0000","last_became_active":"2023-10-06T03:57:10.346689+0000","last_became_peered":"2023-10-06T03:57:10.346689+0000","last_unstale":"2023-10-06T03:57:32.543509+0000","last_undegraded":"2023-10-06T03:57:32.543509+0000","last_fullsized":"2023-10-06T03:57:32.543509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:43.420455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.375632+0000","last_change":"2023-10-06T03:57:11.372308+0000","last_active":"2023-10-06T03:57:11.375632+0000","last_peered":"2023-10-06T03:57:11.375632+0000","last_clean":"2023-10-06T03:57:11.375632+0000","last_became_active":"2023-10-06T03:57:11.371795+0000","last_became_peered":"2023-10-06T03:57:11.371795+0000","last_unstale":"2023-10-06T03:57:11.375632+0000","last_undegraded":"2023-10-06T03:57:11.375632+0000","last_fullsized":"2023-10-06T03:57:11.375632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:00.324478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"21'5","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.542306+0000","last_change":"2023-10-06T03:57:10.348592+0000","last_active":"2023-10-06T03:57:32.542306+0000","last_peered":"2023-10-06T03:57:32.542306+0000","last_clean":"2023-10-06T03:57:32.542306+0000","last_became_active":"2023-10-06T03:57:10.348291+0000","last_became_peered":"2023-10-06T03:57:10.348291+0000","last_unstale":"2023-10-06T03:57:32.542306+0000","last_undegraded":"2023-10-06T03:57:32.542306+0000","last_fullsized":"2023-10-06T03:57:32.542306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:00.660348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1094,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378672+0000","last_change":"2023-10-06T03:57:11.366510+0000","last_active":"2023-10-06T03:57:11.378672+0000","last_peered":"2023-10-06T03:57:11.378672+0000","last_clean":"2023-10-06T03:57:11.378672+0000","last_became_active":"2023-10-06T03:57:11.365077+0000","last_became_peered":"2023-10-06T03:57:11.365077+0000","last_unstale":"2023-10-06T03:57:11.378672+0000","last_undegraded":"2023-10-06T03:57:11.378672+0000","last_fullsized":"2023-10-06T03:57:11.378672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:25.199391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:21.407239+0000","last_change":"2023-10-06T03:57:10.348418+0000","last_active":"2023-10-06T03:57:21.407239+0000","last_peered":"2023-10-06T03:57:21.407239+0000","last_clean":"2023-10-06T03:57:21.407239+0000","last_became_active":"2023-10-06T03:57:10.348055+0000","last_became_peered":"2023-10-06T03:57:10.348055+0000","last_unstale":"2023-10-06T03:57:21.407239+0000","last_undegraded":"2023-10-06T03:57:21.407239+0000","last_fullsized":"2023-10-06T03:57:21.407239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:26.102022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373629+0000","last_change":"2023-10-06T03:57:11.370493+0000","last_active":"2023-10-06T03:57:11.373629+0000","last_peered":"2023-10-06T03:57:11.373629+0000","last_clean":"2023-10-06T03:57:11.373629+0000","last_became_active":"2023-10-06T03:57:11.370073+0000","last_became_peered":"2023-10-06T03:57:11.370073+0000","last_unstale":"2023-10-06T03:57:11.373629+0000","last_undegraded":"2023-10-06T03:57:11.373629+0000","last_fullsized":"2023-10-06T03:57:11.373629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:58.417951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.544169+0000","last_change":"2023-10-06T03:57:10.352966+0000","last_active":"2023-10-06T03:57:32.544169+0000","last_peered":"2023-10-06T03:57:32.544169+0000","last_clean":"2023-10-06T03:57:32.544169+0000","last_became_active":"2023-10-06T03:57:10.352653+0000","last_became_peered":"2023-10-06T03:57:10.352653+0000","last_unstale":"2023-10-06T03:57:32.544169+0000","last_undegraded":"2023-10-06T03:57:32.544169+0000","last_fullsized":"2023-10-06T03:57:32.544169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:32.092557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.371704+0000","last_change":"2023-10-06T03:57:11.364052+0000","last_active":"2023-10-06T03:57:11.371704+0000","last_peered":"2023-10-06T03:57:11.371704+0000","last_clean":"2023-10-06T03:57:11.371704+0000","last_became_active":"2023-10-06T03:57:11.363830+0000","last_became_peered":"2023-10-06T03:57:11.363830+0000","last_unstale":"2023-10-06T03:57:11.371704+0000","last_undegraded":"2023-10-06T03:57:11.371704+0000","last_fullsized":"2023-10-06T03:57:11.371704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:04.084019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"21'7","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.645676+0000","last_change":"2023-10-06T03:57:10.349421+0000","last_active":"2023-10-06T03:57:37.645676+0000","last_peered":"2023-10-06T03:57:37.645676+0000","last_clean":"2023-10-06T03:57:37.645676+0000","last_became_active":"2023-10-06T03:57:10.349114+0000","last_became_peered":"2023-10-06T03:57:10.349114+0000","last_unstale":"2023-10-06T03:57:37.645676+0000","last_undegraded":"2023-10-06T03:57:37.645676+0000","last_fullsized":"2023-10-06T03:57:37.645676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:08.209791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372296+0000","last_change":"2023-10-06T03:57:11.367516+0000","last_active":"2023-10-06T03:57:11.372296+0000","last_peered":"2023-10-06T03:57:11.372296+0000","last_clean":"2023-10-06T03:57:11.372296+0000","last_became_active":"2023-10-06T03:57:11.367257+0000","last_became_peered":"2023-10-06T03:57:11.367257+0000","last_unstale":"2023-10-06T03:57:11.372296+0000","last_undegraded":"2023-10-06T03:57:11.372296+0000","last_fullsized":"2023-10-06T03:57:11.372296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:43.628819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:21.429070+0000","last_change":"2023-10-06T03:57:10.347649+0000","last_active":"2023-10-06T03:57:21.429070+0000","last_peered":"2023-10-06T03:57:21.429070+0000","last_clean":"2023-10-06T03:57:21.429070+0000","last_became_active":"2023-10-06T03:57:10.347433+0000","last_became_peered":"2023-10-06T03:57:10.347433+0000","last_unstale":"2023-10-06T03:57:21.429070+0000","last_undegraded":"2023-10-06T03:57:21.429070+0000","last_fullsized":"2023-10-06T03:57:21.429070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:55.951494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372561+0000","last_change":"2023-10-06T03:57:11.370092+0000","last_active":"2023-10-06T03:57:11.372561+0000","last_peered":"2023-10-06T03:57:11.372561+0000","last_clean":"2023-10-06T03:57:11.372561+0000","last_became_active":"2023-10-06T03:57:11.369825+0000","last_became_peered":"2023-10-06T03:57:11.369825+0000","last_unstale":"2023-10-06T03:57:11.372561+0000","last_undegraded":"2023-10-06T03:57:11.372561+0000","last_fullsized":"2023-10-06T03:57:11.372561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:43.415929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.367709+0000","last_change":"2023-10-06T03:57:05.663312+0000","last_active":"2023-10-06T03:57:11.367709+0000","last_peered":"2023-10-06T03:57:11.367709+0000","last_clean":"2023-10-06T03:57:11.367709+0000","last_became_active":"2023-10-06T03:57:05.663046+0000","last_became_peered":"2023-10-06T03:57:05.663046+0000","last_unstale":"2023-10-06T03:57:11.367709+0000","last_undegraded":"2023-10-06T03:57:11.367709+0000","last_fullsized":"2023-10-06T03:57:11.367709+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:31.174359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.542752+0000","last_change":"2023-10-06T03:57:10.348930+0000","last_active":"2023-10-06T03:57:32.542752+0000","last_peered":"2023-10-06T03:57:32.542752+0000","last_clean":"2023-10-06T03:57:32.542752+0000","last_became_active":"2023-10-06T03:57:10.348608+0000","last_became_peered":"2023-10-06T03:57:10.348608+0000","last_unstale":"2023-10-06T03:57:32.542752+0000","last_undegraded":"2023-10-06T03:57:32.542752+0000","last_fullsized":"2023-10-06T03:57:32.542752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:00:09.283718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377785+0000","last_change":"2023-10-06T03:57:11.371108+0000","last_active":"2023-10-06T03:57:11.377785+0000","last_peered":"2023-10-06T03:57:11.377785+0000","last_clean":"2023-10-06T03:57:11.377785+0000","last_became_active":"2023-10-06T03:57:11.370847+0000","last_became_peered":"2023-10-06T03:57:11.370847+0000","last_unstale":"2023-10-06T03:57:11.377785+0000","last_undegraded":"2023-10-06T03:57:11.377785+0000","last_fullsized":"2023-10-06T03:57:11.377785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:38.111171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.362274+0000","last_change":"2023-10-06T03:57:05.318834+0000","last_active":"2023-10-06T03:57:11.362274+0000","last_peered":"2023-10-06T03:57:11.362274+0000","last_clean":"2023-10-06T03:57:11.362274+0000","last_became_active":"2023-10-06T03:57:05.318572+0000","last_became_peered":"2023-10-06T03:57:05.318572+0000","last_unstale":"2023-10-06T03:57:11.362274+0000","last_undegraded":"2023-10-06T03:57:11.362274+0000","last_fullsized":"2023-10-06T03:57:11.362274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:21.951095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"21'6","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.546545+0000","last_change":"2023-10-06T03:57:10.348556+0000","last_active":"2023-10-06T03:57:32.546545+0000","last_peered":"2023-10-06T03:57:32.546545+0000","last_clean":"2023-10-06T03:57:32.546545+0000","last_became_active":"2023-10-06T03:57:10.348271+0000","last_became_peered":"2023-10-06T03:57:10.348271+0000","last_unstale":"2023-10-06T03:57:32.546545+0000","last_undegraded":"2023-10-06T03:57:32.546545+0000","last_fullsized":"2023-10-06T03:57:32.546545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:58.470989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.368920+0000","last_change":"2023-10-06T03:57:11.365237+0000","last_active":"2023-10-06T03:57:11.368920+0000","last_peered":"2023-10-06T03:57:11.368920+0000","last_clean":"2023-10-06T03:57:11.368920+0000","last_became_active":"2023-10-06T03:57:11.365006+0000","last_became_peered":"2023-10-06T03:57:11.365006+0000","last_unstale":"2023-10-06T03:57:11.368920+0000","last_undegraded":"2023-10-06T03:57:11.368920+0000","last_fullsized":"2023-10-06T03:57:11.368920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:33.005015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.355768+0000","last_change":"2023-10-06T03:57:05.314695+0000","last_active":"2023-10-06T03:57:11.355768+0000","last_peered":"2023-10-06T03:57:11.355768+0000","last_clean":"2023-10-06T03:57:11.355768+0000","last_became_active":"2023-10-06T03:57:05.314408+0000","last_became_peered":"2023-10-06T03:57:05.314408+0000","last_unstale":"2023-10-06T03:57:11.355768+0000","last_undegraded":"2023-10-06T03:57:11.355768+0000","last_fullsized":"2023-10-06T03:57:11.355768+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:19.042145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:26.532613+0000","last_change":"2023-10-06T03:57:10.346078+0000","last_active":"2023-10-06T03:57:26.532613+0000","last_peered":"2023-10-06T03:57:26.532613+0000","last_clean":"2023-10-06T03:57:26.532613+0000","last_became_active":"2023-10-06T03:57:10.345769+0000","last_became_peered":"2023-10-06T03:57:10.345769+0000","last_unstale":"2023-10-06T03:57:26.532613+0000","last_undegraded":"2023-10-06T03:57:26.532613+0000","last_fullsized":"2023-10-06T03:57:26.532613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:16.630681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.376583+0000","last_change":"2023-10-06T03:57:11.370372+0000","last_active":"2023-10-06T03:57:11.376583+0000","last_peered":"2023-10-06T03:57:11.376583+0000","last_clean":"2023-10-06T03:57:11.376583+0000","last_became_active":"2023-10-06T03:57:11.370076+0000","last_became_peered":"2023-10-06T03:57:11.370076+0000","last_unstale":"2023-10-06T03:57:11.376583+0000","last_undegraded":"2023-10-06T03:57:11.376583+0000","last_fullsized":"2023-10-06T03:57:11.376583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:18:59.517155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.362848+0000","last_change":"2023-10-06T03:57:05.318778+0000","last_active":"2023-10-06T03:57:11.362848+0000","last_peered":"2023-10-06T03:57:11.362848+0000","last_clean":"2023-10-06T03:57:11.362848+0000","last_became_active":"2023-10-06T03:57:05.318456+0000","last_became_peered":"2023-10-06T03:57:05.318456+0000","last_unstale":"2023-10-06T03:57:11.362848+0000","last_undegraded":"2023-10-06T03:57:11.362848+0000","last_fullsized":"2023-10-06T03:57:11.362848+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:39.769241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.362382+0000","last_change":"2023-10-06T03:57:05.664245+0000","last_active":"2023-10-06T03:57:11.362382+0000","last_peered":"2023-10-06T03:57:11.362382+0000","last_clean":"2023-10-06T03:57:11.362382+0000","last_became_active":"2023-10-06T03:57:05.663426+0000","last_became_peered":"2023-10-06T03:57:05.663426+0000","last_unstale":"2023-10-06T03:57:11.362382+0000","last_undegraded":"2023-10-06T03:57:11.362382+0000","last_fullsized":"2023-10-06T03:57:11.362382+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:29.322599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"21'5","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.646148+0000","last_change":"2023-10-06T03:57:10.351059+0000","last_active":"2023-10-06T03:57:37.646148+0000","last_peered":"2023-10-06T03:57:37.646148+0000","last_clean":"2023-10-06T03:57:37.646148+0000","last_became_active":"2023-10-06T03:57:10.348879+0000","last_became_peered":"2023-10-06T03:57:10.348879+0000","last_unstale":"2023-10-06T03:57:37.646148+0000","last_undegraded":"2023-10-06T03:57:37.646148+0000","last_fullsized":"2023-10-06T03:57:37.646148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:35.764148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.375601+0000","last_change":"2023-10-06T03:57:11.373786+0000","last_active":"2023-10-06T03:57:11.375601+0000","last_peered":"2023-10-06T03:57:11.375601+0000","last_clean":"2023-10-06T03:57:11.375601+0000","last_became_active":"2023-10-06T03:57:11.373452+0000","last_became_peered":"2023-10-06T03:57:11.373452+0000","last_unstale":"2023-10-06T03:57:11.375601+0000","last_undegraded":"2023-10-06T03:57:11.375601+0000","last_fullsized":"2023-10-06T03:57:11.375601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:20.524065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.364987+0000","last_change":"2023-10-06T03:57:05.318790+0000","last_active":"2023-10-06T03:57:11.364987+0000","last_peered":"2023-10-06T03:57:11.364987+0000","last_clean":"2023-10-06T03:57:11.364987+0000","last_became_active":"2023-10-06T03:57:05.318531+0000","last_became_peered":"2023-10-06T03:57:05.318531+0000","last_unstale":"2023-10-06T03:57:11.364987+0000","last_undegraded":"2023-10-06T03:57:11.364987+0000","last_fullsized":"2023-10-06T03:57:11.364987+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:12.876998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"21'6","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.646566+0000","last_change":"2023-10-06T03:57:10.343328+0000","last_active":"2023-10-06T03:57:37.646566+0000","last_peered":"2023-10-06T03:57:37.646566+0000","last_clean":"2023-10-06T03:57:37.646566+0000","last_became_active":"2023-10-06T03:57:10.343051+0000","last_became_peered":"2023-10-06T03:57:10.343051+0000","last_unstale":"2023-10-06T03:57:37.646566+0000","last_undegraded":"2023-10-06T03:57:37.646566+0000","last_fullsized":"2023-10-06T03:57:37.646566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:27.722475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378907+0000","last_change":"2023-10-06T03:57:11.373581+0000","last_active":"2023-10-06T03:57:11.378907+0000","last_peered":"2023-10-06T03:57:11.378907+0000","last_clean":"2023-10-06T03:57:11.378907+0000","last_became_active":"2023-10-06T03:57:11.373264+0000","last_became_peered":"2023-10-06T03:57:11.373264+0000","last_unstale":"2023-10-06T03:57:11.378907+0000","last_undegraded":"2023-10-06T03:57:11.378907+0000","last_fullsized":"2023-10-06T03:57:11.378907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:15.950245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.366791+0000","last_change":"2023-10-06T03:57:05.663770+0000","last_active":"2023-10-06T03:57:11.366791+0000","last_peered":"2023-10-06T03:57:11.366791+0000","last_clean":"2023-10-06T03:57:11.366791+0000","last_became_active":"2023-10-06T03:57:05.663501+0000","last_became_peered":"2023-10-06T03:57:05.663501+0000","last_unstale":"2023-10-06T03:57:11.366791+0000","last_undegraded":"2023-10-06T03:57:11.366791+0000","last_fullsized":"2023-10-06T03:57:11.366791+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:09.806915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:12.368542+0000","last_change":"2023-10-06T03:57:10.346585+0000","last_active":"2023-10-06T03:57:12.368542+0000","last_peered":"2023-10-06T03:57:12.368542+0000","last_clean":"2023-10-06T03:57:12.368542+0000","last_became_active":"2023-10-06T03:57:10.345743+0000","last_became_peered":"2023-10-06T03:57:10.345743+0000","last_unstale":"2023-10-06T03:57:12.368542+0000","last_undegraded":"2023-10-06T03:57:12.368542+0000","last_fullsized":"2023-10-06T03:57:12.368542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:54.589702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378948+0000","last_change":"2023-10-06T03:57:11.373844+0000","last_active":"2023-10-06T03:57:11.378948+0000","last_peered":"2023-10-06T03:57:11.378948+0000","last_clean":"2023-10-06T03:57:11.378948+0000","last_became_active":"2023-10-06T03:57:11.373543+0000","last_became_peered":"2023-10-06T03:57:11.373543+0000","last_unstale":"2023-10-06T03:57:11.378948+0000","last_undegraded":"2023-10-06T03:57:11.378948+0000","last_fullsized":"2023-10-06T03:57:11.378948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:15.152354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"21'7","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.645077+0000","last_change":"2023-10-06T03:57:10.344618+0000","last_active":"2023-10-06T03:57:37.645077+0000","last_peered":"2023-10-06T03:57:37.645077+0000","last_clean":"2023-10-06T03:57:37.645077+0000","last_became_active":"2023-10-06T03:57:10.344324+0000","last_became_peered":"2023-10-06T03:57:10.344324+0000","last_unstale":"2023-10-06T03:57:37.645077+0000","last_undegraded":"2023-10-06T03:57:37.645077+0000","last_fullsized":"2023-10-06T03:57:37.645077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:06.062331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.368350+0000","last_change":"2023-10-06T03:57:05.317973+0000","last_active":"2023-10-06T03:57:11.368350+0000","last_peered":"2023-10-06T03:57:11.368350+0000","last_clean":"2023-10-06T03:57:11.368350+0000","last_became_active":"2023-10-06T03:57:05.317715+0000","last_became_peered":"2023-10-06T03:57:05.317715+0000","last_unstale":"2023-10-06T03:57:11.368350+0000","last_undegraded":"2023-10-06T03:57:11.368350+0000","last_fullsized":"2023-10-06T03:57:11.368350+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:37:51.858909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372084+0000","last_change":"2023-10-06T03:57:11.369706+0000","last_active":"2023-10-06T03:57:11.372084+0000","last_peered":"2023-10-06T03:57:11.372084+0000","last_clean":"2023-10-06T03:57:11.372084+0000","last_became_active":"2023-10-06T03:57:11.369445+0000","last_became_peered":"2023-10-06T03:57:11.369445+0000","last_unstale":"2023-10-06T03:57:11.372084+0000","last_undegraded":"2023-10-06T03:57:11.372084+0000","last_fullsized":"2023-10-06T03:57:11.372084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:52.999421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.362125+0000","last_change":"2023-10-06T03:57:05.320847+0000","last_active":"2023-10-06T03:57:11.362125+0000","last_peered":"2023-10-06T03:57:11.362125+0000","last_clean":"2023-10-06T03:57:11.362125+0000","last_became_active":"2023-10-06T03:57:05.320583+0000","last_became_peered":"2023-10-06T03:57:05.320583+0000","last_unstale":"2023-10-06T03:57:11.362125+0000","last_undegraded":"2023-10-06T03:57:11.362125+0000","last_fullsized":"2023-10-06T03:57:11.362125+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:12.772449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"21'7","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:32.542436+0000","last_change":"2023-10-06T03:57:10.341132+0000","last_active":"2023-10-06T03:57:32.542436+0000","last_peered":"2023-10-06T03:57:32.542436+0000","last_clean":"2023-10-06T03:57:32.542436+0000","last_became_active":"2023-10-06T03:57:10.339424+0000","last_became_peered":"2023-10-06T03:57:10.339424+0000","last_unstale":"2023-10-06T03:57:32.542436+0000","last_undegraded":"2023-10-06T03:57:32.542436+0000","last_fullsized":"2023-10-06T03:57:32.542436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:39.675711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.374372+0000","last_change":"2023-10-06T03:57:11.372497+0000","last_active":"2023-10-06T03:57:11.374372+0000","last_peered":"2023-10-06T03:57:11.374372+0000","last_clean":"2023-10-06T03:57:11.374372+0000","last_became_active":"2023-10-06T03:57:11.372215+0000","last_became_peered":"2023-10-06T03:57:11.372215+0000","last_unstale":"2023-10-06T03:57:11.374372+0000","last_undegraded":"2023-10-06T03:57:11.374372+0000","last_fullsized":"2023-10-06T03:57:11.374372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:23.559480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"21'5","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:21.447011+0000","last_change":"2023-10-06T03:57:10.344051+0000","last_active":"2023-10-06T03:57:21.447011+0000","last_peered":"2023-10-06T03:57:21.447011+0000","last_clean":"2023-10-06T03:57:21.447011+0000","last_became_active":"2023-10-06T03:57:10.343865+0000","last_became_peered":"2023-10-06T03:57:10.343865+0000","last_unstale":"2023-10-06T03:57:21.447011+0000","last_undegraded":"2023-10-06T03:57:21.447011+0000","last_fullsized":"2023-10-06T03:57:21.447011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:32.522470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.376665+0000","last_change":"2023-10-06T03:57:11.369927+0000","last_active":"2023-10-06T03:57:11.376665+0000","last_peered":"2023-10-06T03:57:11.376665+0000","last_clean":"2023-10-06T03:57:11.376665+0000","last_became_active":"2023-10-06T03:57:11.369681+0000","last_became_peered":"2023-10-06T03:57:11.369681+0000","last_unstale":"2023-10-06T03:57:11.376665+0000","last_undegraded":"2023-10-06T03:57:11.376665+0000","last_fullsized":"2023-10-06T03:57:11.376665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:53.858179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"21'5","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.646820+0000","last_change":"2023-10-06T03:57:10.349666+0000","last_active":"2023-10-06T03:57:37.646820+0000","last_peered":"2023-10-06T03:57:37.646820+0000","last_clean":"2023-10-06T03:57:37.646820+0000","last_became_active":"2023-10-06T03:57:10.349320+0000","last_became_peered":"2023-10-06T03:57:10.349320+0000","last_unstale":"2023-10-06T03:57:37.646820+0000","last_undegraded":"2023-10-06T03:57:37.646820+0000","last_fullsized":"2023-10-06T03:57:37.646820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:45:49.709519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373619+0000","last_change":"2023-10-06T03:57:11.370442+0000","last_active":"2023-10-06T03:57:11.373619+0000","last_peered":"2023-10-06T03:57:11.373619+0000","last_clean":"2023-10-06T03:57:11.373619+0000","last_became_active":"2023-10-06T03:57:11.370229+0000","last_became_peered":"2023-10-06T03:57:11.370229+0000","last_unstale":"2023-10-06T03:57:11.373619+0000","last_undegraded":"2023-10-06T03:57:11.373619+0000","last_fullsized":"2023-10-06T03:57:11.373619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:26.174331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"21'12","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.647341+0000","last_change":"2023-10-06T03:57:10.350263+0000","last_active":"2023-10-06T03:57:37.647341+0000","last_peered":"2023-10-06T03:57:37.647341+0000","last_clean":"2023-10-06T03:57:37.647341+0000","last_became_active":"2023-10-06T03:57:10.350083+0000","last_became_peered":"2023-10-06T03:57:10.350083+0000","last_unstale":"2023-10-06T03:57:37.647341+0000","last_undegraded":"2023-10-06T03:57:37.647341+0000","last_fullsized":"2023-10-06T03:57:37.647341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:05.433032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372850+0000","last_change":"2023-10-06T03:57:11.369397+0000","last_active":"2023-10-06T03:57:11.372850+0000","last_peered":"2023-10-06T03:57:11.372850+0000","last_clean":"2023-10-06T03:57:11.372850+0000","last_became_active":"2023-10-06T03:57:11.369123+0000","last_became_peered":"2023-10-06T03:57:11.369123+0000","last_unstale":"2023-10-06T03:57:11.372850+0000","last_undegraded":"2023-10-06T03:57:11.372850+0000","last_fullsized":"2023-10-06T03:57:11.372850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:28.139036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:16.370129+0000","last_change":"2023-10-06T03:57:10.347049+0000","last_active":"2023-10-06T03:57:16.370129+0000","last_peered":"2023-10-06T03:57:16.370129+0000","last_clean":"2023-10-06T03:57:16.370129+0000","last_became_active":"2023-10-06T03:57:10.344583+0000","last_became_peered":"2023-10-06T03:57:10.344583+0000","last_unstale":"2023-10-06T03:57:16.370129+0000","last_undegraded":"2023-10-06T03:57:16.370129+0000","last_fullsized":"2023-10-06T03:57:16.370129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:32.280257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.370657+0000","last_change":"2023-10-06T03:57:11.366741+0000","last_active":"2023-10-06T03:57:11.370657+0000","last_peered":"2023-10-06T03:57:11.370657+0000","last_clean":"2023-10-06T03:57:11.370657+0000","last_became_active":"2023-10-06T03:57:11.366500+0000","last_became_peered":"2023-10-06T03:57:11.366500+0000","last_unstale":"2023-10-06T03:57:11.370657+0000","last_undegraded":"2023-10-06T03:57:11.370657+0000","last_fullsized":"2023-10-06T03:57:11.370657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:24.714603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"21'13","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:57:37.645297+0000","last_change":"2023-10-06T03:57:10.348170+0000","last_active":"2023-10-06T03:57:37.645297+0000","last_peered":"2023-10-06T03:57:37.645297+0000","last_clean":"2023-10-06T03:57:37.645297+0000","last_became_active":"2023-10-06T03:57:10.347897+0000","last_became_peered":"2023-10-06T03:57:10.347897+0000","last_unstale":"2023-10-06T03:57:37.645297+0000","last_undegraded":"2023-10-06T03:57:37.645297+0000","last_fullsized":"2023-10-06T03:57:37.645297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:40.756246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372969+0000","last_change":"2023-10-06T03:57:11.364368+0000","last_active":"2023-10-06T03:57:11.372969+0000","last_peered":"2023-10-06T03:57:11.372969+0000","last_clean":"2023-10-06T03:57:11.372969+0000","last_became_active":"2023-10-06T03:57:11.364150+0000","last_became_peered":"2023-10-06T03:57:11.364150+0000","last_unstale":"2023-10-06T03:57:11.372969+0000","last_undegraded":"2023-10-06T03:57:11.372969+0000","last_fullsized":"2023-10-06T03:57:11.372969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:11.038080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.374372+0000","last_change":"2023-10-06T03:57:11.366721+0000","last_active":"2023-10-06T03:57:11.374372+0000","last_peered":"2023-10-06T03:57:11.374372+0000","last_clean":"2023-10-06T03:57:11.374372+0000","last_became_active":"2023-10-06T03:57:11.366524+0000","last_became_peered":"2023-10-06T03:57:11.366524+0000","last_unstale":"2023-10-06T03:57:11.374372+0000","last_undegraded":"2023-10-06T03:57:11.374372+0000","last_fullsized":"2023-10-06T03:57:11.374372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:33.044858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.373497+0000","last_change":"2023-10-06T03:57:11.371109+0000","last_active":"2023-10-06T03:57:11.373497+0000","last_peered":"2023-10-06T03:57:11.373497+0000","last_clean":"2023-10-06T03:57:11.373497+0000","last_became_active":"2023-10-06T03:57:11.370848+0000","last_became_peered":"2023-10-06T03:57:11.370848+0000","last_unstale":"2023-10-06T03:57:11.373497+0000","last_undegraded":"2023-10-06T03:57:11.373497+0000","last_fullsized":"2023-10-06T03:57:11.373497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:15.919090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.372839+0000","last_change":"2023-10-06T03:57:11.364188+0000","last_active":"2023-10-06T03:57:11.372839+0000","last_peered":"2023-10-06T03:57:11.372839+0000","last_clean":"2023-10-06T03:57:11.372839+0000","last_became_active":"2023-10-06T03:57:11.363875+0000","last_became_peered":"2023-10-06T03:57:11.363875+0000","last_unstale":"2023-10-06T03:57:11.372839+0000","last_undegraded":"2023-10-06T03:57:11.372839+0000","last_fullsized":"2023-10-06T03:57:11.372839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:29.236344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.378301+0000","last_change":"2023-10-06T03:57:11.377420+0000","last_active":"2023-10-06T03:57:11.378301+0000","last_peered":"2023-10-06T03:57:11.378301+0000","last_clean":"2023-10-06T03:57:11.378301+0000","last_became_active":"2023-10-06T03:57:11.377126+0000","last_became_peered":"2023-10-06T03:57:11.377126+0000","last_unstale":"2023-10-06T03:57:11.378301+0000","last_undegraded":"2023-10-06T03:57:11.378301+0000","last_fullsized":"2023-10-06T03:57:11.378301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:05.041737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:57:11.377733+0000","last_change":"2023-10-06T03:57:11.375777+0000","last_active":"2023-10-06T03:57:11.377733+0000","last_peered":"2023-10-06T03:57:11.377733+0000","last_clean":"2023-10-06T03:57:11.377733+0000","last_became_active":"2023-10-06T03:57:11.375539+0000","last_became_peered":"2023-10-06T03:57:11.375539+0000","last_unstale":"2023-10-06T03:57:11.377733+0000","last_undegraded":"2023-10-06T03:57:11.377733+0000","last_fullsized":"2023-10-06T03:57:11.377733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:42.707488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":8933,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":210,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":17866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":184,"ondisk_log_size":184,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64858,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297612,"kb_used_data":324,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074228,"statfs":{"total":96636764160,"available":96332009472,"internally_reserved":0,"allocated":331776,"data_stored":673502,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":78947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574860,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":78835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297616,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074224,"statfs":{"total":96636764160,"available":96332005376,"internally_reserved":0,"allocated":335872,"data_stored":670584,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297504,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074336,"statfs":{"total":96636764160,"available":96332120064,"internally_reserved":0,"allocated":221184,"data_stored":78925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":253952,"data_stored":81801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297560,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074280,"statfs":{"total":96636764160,"available":96332062720,"internally_reserved":0,"allocated":278528,"data_stored":83706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":82029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":2334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:57:49.984 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:57:50.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.130+0000 7f32cb1fb700 1 Processor -- start 2023-10-06T03:57:50.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.130+0000 7f32cb1fb700 1 -- start start 2023-10-06T03:57:50.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.131+0000 7f32cb1fb700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32c406ab50 0x7f32c406d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:50.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.131+0000 7f32cb1fb700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f32c406ddf0 0x7f32c406e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:50.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.131+0000 7f32cb1fb700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32c40c6160 con 0x7f32c406a780 2023-10-06T03:57:50.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.131+0000 7f32cb1fb700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32c40c5c00 con 0x7f32c406ddf0 2023-10-06T03:57:50.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.131+0000 7f32cb1fb700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c4069040 con 0x7f32c406ab50 2023-10-06T03:57:50.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.131+0000 7f32c9798700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f32c406ddf0 0x7f32c406e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:50.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.131+0000 7f32c8f97700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f32c406a780 0x7f32c40c65f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40854/0 (socket says 172.21.15.157:40854) 2023-10-06T03:57:50.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.131+0000 7f32c8f97700 1 -- 172.21.15.157:0/927390772 learned_addr learned my addr 172.21.15.157:0/927390772 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:50.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.132+0000 7f32c3fff700 1 --2- 172.21.15.157:0/927390772 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32c406ab50 0x7f32c406d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:50.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.132+0000 7f32c37fe700 1 -- 172.21.15.157:0/927390772 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4273493919 0 0) 0x7f32c40c6160 con 0x7f32c406a780 2023-10-06T03:57:50.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.132+0000 7f32c37fe700 1 -- 172.21.15.157:0/927390772 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32ac003570 con 0x7f32c406a780 2023-10-06T03:57:50.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.132+0000 7f32c9798700 1 -- 172.21.15.157:0/927390772 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32c406ab50 msgr2=0x7f32c406d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:50.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.132+0000 7f32c9798700 1 --2- 172.21.15.157:0/927390772 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32c406ab50 0x7f32c406d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.132+0000 7f32c9798700 1 -- 172.21.15.157:0/927390772 >> v1:172.21.15.157:6789/0 conn(0x7f32c406a780 legacy=0x7f32c40c65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:50.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.132+0000 7f32c9798700 1 -- 172.21.15.157:0/927390772 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c4153930 con 0x7f32c406ddf0 2023-10-06T03:57:50.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.132+0000 7f32c9798700 1 --2- 172.21.15.157:0/927390772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c406ddf0 0x7f32c406e230 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f32bc0098e0 tx=0x7f32bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b433d4489fd083 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:50.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.132+0000 7f32c37fe700 1 -- 172.21.15.157:0/927390772 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc00b640 con 0x7f32c406ddf0 2023-10-06T03:57:50.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.132+0000 7f32c37fe700 1 -- 172.21.15.157:0/927390772 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32bc00b7a0 con 0x7f32c406ddf0 2023-10-06T03:57:50.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.132+0000 7f32c37fe700 1 -- 172.21.15.157:0/927390772 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc00bb40 con 0x7f32c406ddf0 2023-10-06T03:57:50.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.132+0000 7f32cb1fb700 1 -- 172.21.15.157:0/927390772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c406ddf0 msgr2=0x7f32c406e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:50.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.132+0000 7f32cb1fb700 1 --2- 172.21.15.157:0/927390772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c406ddf0 0x7f32c406e230 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f32bc0098e0 tx=0x7f32bc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.132+0000 7f32cb1fb700 1 -- 172.21.15.157:0/927390772 shutdown_connections 2023-10-06T03:57:50.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.132+0000 7f32cb1fb700 1 --2- 172.21.15.157:0/927390772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c406ddf0 0x7f32c406e230 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.132+0000 7f32cb1fb700 1 --2- 172.21.15.157:0/927390772 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32c406ab50 0x7f32c406d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.132+0000 7f32cb1fb700 1 -- 172.21.15.157:0/927390772 >> 172.21.15.157:0/927390772 conn(0x7f32c406b510 msgr2=0x7f32c406b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:50.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.133+0000 7f32cb1fb700 1 -- 172.21.15.157:0/927390772 shutdown_connections 2023-10-06T03:57:50.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.133+0000 7f32cb1fb700 1 -- 172.21.15.157:0/927390772 wait complete. 2023-10-06T03:57:50.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.134+0000 7f32cb1fb700 1 Processor -- start 2023-10-06T03:57:50.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.134+0000 7f32cb1fb700 1 -- start start 2023-10-06T03:57:50.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.134+0000 7f32cb1fb700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32c406a780 0x7f32c414c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:50.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.134+0000 7f32cb1fb700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c406ab50 0x7f32c4165b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:50.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.134+0000 7f32cb1fb700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32c406ddf0 0x7f32c41660c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:50.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.134+0000 7f32cb1fb700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c414d3a0 con 0x7f32c406ab50 2023-10-06T03:57:50.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.134+0000 7f32cb1fb700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c41656c0 con 0x7f32c406ddf0 2023-10-06T03:57:50.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.134+0000 7f32cb1fb700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c4169930 con 0x7f32c406a780 2023-10-06T03:57:50.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.135+0000 7f32c3fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c406ab50 0x7f32c4165b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:50.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.135+0000 7f32c3fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c406ab50 0x7f32c4165b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45258/0 (socket says 172.21.15.157:45258) 2023-10-06T03:57:50.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.135+0000 7f32c3fff700 1 -- 172.21.15.157:0/2791157495 learned_addr learned my addr 172.21.15.157:0/2791157495 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:50.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.135+0000 7f32c8f97700 1 --2- 172.21.15.157:0/2791157495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32c406a780 0x7f32c414c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:50.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.135+0000 7f32c9798700 1 --2- 172.21.15.157:0/2791157495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32c406ddf0 0x7f32c41660c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:50.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.135+0000 7f32c3fff700 1 -- 172.21.15.157:0/2791157495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32c406a780 msgr2=0x7f32c414c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:50.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.135+0000 7f32c3fff700 1 --2- 172.21.15.157:0/2791157495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32c406a780 0x7f32c414c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.135+0000 7f32c3fff700 1 -- 172.21.15.157:0/2791157495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32c406ddf0 msgr2=0x7f32c41660c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:50.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.135+0000 7f32c3fff700 1 --2- 172.21.15.157:0/2791157495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32c406ddf0 0x7f32c41660c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.135+0000 7f32c3fff700 1 -- 172.21.15.157:0/2791157495 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32bc009420 con 0x7f32c406ab50 2023-10-06T03:57:50.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.135+0000 7f32c8f97700 1 --2- 172.21.15.157:0/2791157495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32c406a780 0x7f32c414c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:50.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.135+0000 7f32c9798700 1 --2- 172.21.15.157:0/2791157495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32c406ddf0 0x7f32c41660c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:50.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.135+0000 7f32c3fff700 1 --2- 172.21.15.157:0/2791157495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c406ab50 0x7f32c4165b80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f32b800d890 tx=0x7f32b8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:50.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.135+0000 7f32c0ff9700 1 -- 172.21.15.157:0/2791157495 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b800b600 con 0x7f32c406ab50 2023-10-06T03:57:50.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.135+0000 7f32c0ff9700 1 -- 172.21.15.157:0/2791157495 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b800b760 con 0x7f32c406ab50 2023-10-06T03:57:50.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.135+0000 7f32c0ff9700 1 -- 172.21.15.157:0/2791157495 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b800bae0 con 0x7f32c406ab50 2023-10-06T03:57:50.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.135+0000 7f32cb1fb700 1 -- 172.21.15.157:0/2791157495 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c414d5b0 con 0x7f32c406ab50 2023-10-06T03:57:50.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.135+0000 7f32cb1fb700 1 -- 172.21.15.157:0/2791157495 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c406da10 con 0x7f32c406ab50 2023-10-06T03:57:50.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.136+0000 7f32cb1fb700 1 -- 172.21.15.157:0/2791157495 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c41658a0 con 0x7f32c406ab50 2023-10-06T03:57:50.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.136+0000 7f32c0ff9700 1 -- 172.21.15.157:0/2791157495 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f32b801e460 con 0x7f32c406ab50 2023-10-06T03:57:50.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.137+0000 7f32c0ff9700 1 --2- 172.21.15.157:0/2791157495 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f32b0066f30 0x7f32b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:50.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.137+0000 7f32c0ff9700 1 -- 172.21.15.157:0/2791157495 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f32b8084d70 con 0x7f32c406ab50 2023-10-06T03:57:50.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.137+0000 7f32c8f97700 1 --2- 172.21.15.157:0/2791157495 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f32b0066f30 0x7f32b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:50.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.137+0000 7f32c8f97700 1 --2- 172.21.15.157:0/2791157495 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f32b0066f30 0x7f32b00693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f32b4005c50 tx=0x7f32b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:50.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.141+0000 7f32c0ff9700 1 -- 172.21.15.157:0/2791157495 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32c41658a0 con 0x7f32c406ab50 2023-10-06T03:57:50.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.325+0000 7f32cb1fb700 1 -- 172.21.15.157:0/2791157495 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32c41658a0 con 0x7f32c406ab50 2023-10-06T03:57:50.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.326+0000 7f32c0ff9700 1 -- 172.21.15.157:0/2791157495 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f32c41658a0 con 0x7f32c406ab50 2023-10-06T03:57:50.328 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:50.328 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":21,"fsid":"28664a7a-fe30-4efe-be5a-6abf9318b700","created":"2023-10-06T03:56:51.887538+0000","modified":"2023-10-06T03:57:12.362537+0000","last_up_change":"2023-10-06T03:57:02.299440+0000","last_in_change":"2023-10-06T03:56:55.141505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:57:03.455121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:57:03.905661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:57:08.377186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:57:09.709582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f677cac-e9f4-4608-acac-718b072ce984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":90640},{"type":"v1","addr":"172.21.15.157:6809","nonce":90640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":90640},{"type":"v1","addr":"172.21.15.157:6811","nonce":90640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":90640},{"type":"v1","addr":"172.21.15.157:6815","nonce":90640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":90640},{"type":"v1","addr":"172.21.15.157:6813","nonce":90640}]},"public_addr":"172.21.15.157:6809/90640","cluster_addr":"172.21.15.157:6811/90640","heartbeat_back_addr":"172.21.15.157:6815/90640","heartbeat_front_addr":"172.21.15.157:6813/90640","state":["exists","up"]},{"osd":1,"uuid":"7d771892-bb91-4e9b-9610-051861c11247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":90654},{"type":"v1","addr":"172.21.15.157:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":90654},{"type":"v1","addr":"172.21.15.157:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":90654},{"type":"v1","addr":"172.21.15.157:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":90654},{"type":"v1","addr":"172.21.15.157:6805","nonce":90654}]},"public_addr":"172.21.15.157:6801/90654","cluster_addr":"172.21.15.157:6803/90654","heartbeat_back_addr":"172.21.15.157:6807/90654","heartbeat_front_addr":"172.21.15.157:6805/90654","state":["exists","up"]},{"osd":2,"uuid":"5c1f7d4e-d185-47cf-ab6c-453b4e3009c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":90657},{"type":"v1","addr":"172.21.15.157:6825","nonce":90657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":90657},{"type":"v1","addr":"172.21.15.157:6827","nonce":90657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":90657},{"type":"v1","addr":"172.21.15.157:6831","nonce":90657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":90657},{"type":"v1","addr":"172.21.15.157:6829","nonce":90657}]},"public_addr":"172.21.15.157:6825/90657","cluster_addr":"172.21.15.157:6827/90657","heartbeat_back_addr":"172.21.15.157:6831/90657","heartbeat_front_addr":"172.21.15.157:6829/90657","state":["exists","up"]},{"osd":3,"uuid":"f6a1e825-43c7-4afd-8daf-e51d45afc75e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":90641},{"type":"v1","addr":"172.21.15.157:6817","nonce":90641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":90641},{"type":"v1","addr":"172.21.15.157:6819","nonce":90641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":90641},{"type":"v1","addr":"172.21.15.157:6823","nonce":90641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":90641},{"type":"v1","addr":"172.21.15.157:6821","nonce":90641}]},"public_addr":"172.21.15.157:6817/90641","cluster_addr":"172.21.15.157:6819/90641","heartbeat_back_addr":"172.21.15.157:6823/90641","heartbeat_front_addr":"172.21.15.157:6821/90641","state":["exists","up"]},{"osd":4,"uuid":"8ad6e3d8-9a31-4f85-9410-417b7ac5da97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":91261},{"type":"v1","addr":"172.21.15.175:6803","nonce":91261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":91261},{"type":"v1","addr":"172.21.15.175:6806","nonce":91261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":91261},{"type":"v1","addr":"172.21.15.175:6814","nonce":91261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":91261},{"type":"v1","addr":"172.21.15.175:6810","nonce":91261}]},"public_addr":"172.21.15.175:6803/91261","cluster_addr":"172.21.15.175:6806/91261","heartbeat_back_addr":"172.21.15.175:6814/91261","heartbeat_front_addr":"172.21.15.175:6810/91261","state":["exists","up"]},{"osd":5,"uuid":"f68aca2a-f182-4c00-8e21-241d09c02ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":91262},{"type":"v1","addr":"172.21.15.175:6817","nonce":91262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":91262},{"type":"v1","addr":"172.21.15.175:6819","nonce":91262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":91262},{"type":"v1","addr":"172.21.15.175:6823","nonce":91262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":91262},{"type":"v1","addr":"172.21.15.175:6821","nonce":91262}]},"public_addr":"172.21.15.175:6817/91262","cluster_addr":"172.21.15.175:6819/91262","heartbeat_back_addr":"172.21.15.175:6823/91262","heartbeat_front_addr":"172.21.15.175:6821/91262","state":["exists","up"]},{"osd":6,"uuid":"d867cd89-3430-40ca-862f-d2bea3de9d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":91263},{"type":"v1","addr":"172.21.15.175:6801","nonce":91263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6805","nonce":91263},{"type":"v1","addr":"172.21.15.175:6807","nonce":91263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6813","nonce":91263},{"type":"v1","addr":"172.21.15.175:6815","nonce":91263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":91263},{"type":"v1","addr":"172.21.15.175:6811","nonce":91263}]},"public_addr":"172.21.15.175:6801/91263","cluster_addr":"172.21.15.175:6807/91263","heartbeat_back_addr":"172.21.15.175:6815/91263","heartbeat_front_addr":"172.21.15.175:6811/91263","state":["exists","up"]},{"osd":7,"uuid":"930cd464-79f1-47cb-b4ff-188ccde74fff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":91306},{"type":"v1","addr":"172.21.15.175:6825","nonce":91306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":91306},{"type":"v1","addr":"172.21.15.175:6827","nonce":91306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":91306},{"type":"v1","addr":"172.21.15.175:6831","nonce":91306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":91306},{"type":"v1","addr":"172.21.15.175:6829","nonce":91306}]},"public_addr":"172.21.15.175:6825/91306","cluster_addr":"172.21.15.175:6827/91306","heartbeat_back_addr":"172.21.15.175:6831/91306","heartbeat_front_addr":"172.21.15.175:6829/91306","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.290673+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.056390+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.292903+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.275861+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.332095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.327453+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.317585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.309330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:57:50.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.328+0000 7f32cb1fb700 1 -- 172.21.15.157:0/2791157495 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f32b0066f30 msgr2=0x7f32b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:50.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.328+0000 7f32cb1fb700 1 --2- 172.21.15.157:0/2791157495 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f32b0066f30 0x7f32b00693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f32b4005c50 tx=0x7f32b4009040 comp rx=0 tx=0).stop 2023-10-06T03:57:50.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.328+0000 7f32cb1fb700 1 -- 172.21.15.157:0/2791157495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c406ab50 msgr2=0x7f32c4165b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:50.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.328+0000 7f32cb1fb700 1 --2- 172.21.15.157:0/2791157495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c406ab50 0x7f32c4165b80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f32b800d890 tx=0x7f32b8008040 comp rx=0 tx=0).stop 2023-10-06T03:57:50.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.328+0000 7f32cb1fb700 1 -- 172.21.15.157:0/2791157495 shutdown_connections 2023-10-06T03:57:50.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.328+0000 7f32cb1fb700 1 --2- 172.21.15.157:0/2791157495 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f32b0066f30 0x7f32b00693e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.328+0000 7f32cb1fb700 1 --2- 172.21.15.157:0/2791157495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32c406ddf0 0x7f32c41660c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.328+0000 7f32cb1fb700 1 --2- 172.21.15.157:0/2791157495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32c406a780 0x7f32c414c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.328+0000 7f32cb1fb700 1 --2- 172.21.15.157:0/2791157495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c406ab50 0x7f32c4165b80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.328+0000 7f32cb1fb700 1 -- 172.21.15.157:0/2791157495 >> 172.21.15.157:0/2791157495 conn(0x7f32c406b510 msgr2=0x7f32c406d300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:50.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.328+0000 7f32cb1fb700 1 -- 172.21.15.157:0/2791157495 shutdown_connections 2023-10-06T03:57:50.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.328+0000 7f32cb1fb700 1 -- 172.21.15.157:0/2791157495 wait complete. 2023-10-06T03:57:50.342 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:57:03.455121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:57:03.905661+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:57:08.377186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:57:09.709582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:57:50.343 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:57:50.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.459+0000 7f45c847a700 1 Processor -- start 2023-10-06T03:57:50.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.460+0000 7f45c847a700 1 -- start start 2023-10-06T03:57:50.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.460+0000 7f45c847a700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f45c006cbb0 0x7f45c006d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:50.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.460+0000 7f45c847a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45c006c1f0 0x7f45c006da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:50.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.460+0000 7f45c847a700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45c00c9780 con 0x7f45c006cf80 2023-10-06T03:57:50.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.460+0000 7f45c847a700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45c00c8c40 con 0x7f45c006cbb0 2023-10-06T03:57:50.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.460+0000 7f45c847a700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c006e0c0 con 0x7f45c006c1f0 2023-10-06T03:57:50.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.461+0000 7f45c5a15700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45c006c1f0 0x7f45c006da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:50.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.461+0000 7f45c6216700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f45c006cbb0 0x7f45c006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:50.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.461+0000 7f45c5a15700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45c006c1f0 0x7f45c006da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:53786/0 (socket says 172.21.15.157:53786) 2023-10-06T03:57:50.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.461+0000 7f45c5a15700 1 -- 172.21.15.157:0/1753318926 learned_addr learned my addr 172.21.15.157:0/1753318926 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:50.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.461+0000 7f45c6a17700 1 --1- 172.21.15.157:0/1753318926 >> v1:172.21.15.175:6789/0 conn(0x7f45c006cf80 0x7f45c014fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.157:48216/0 (socket says 172.21.15.157:48216) 2023-10-06T03:57:50.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.461+0000 7f45c5a15700 1 -- 172.21.15.157:0/1753318926 >> v2:172.21.15.175:3300/0 conn(0x7f45c006cbb0 msgr2=0x7f45c006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:50.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.461+0000 7f45c5a15700 1 --2- 172.21.15.157:0/1753318926 >> v2:172.21.15.175:3300/0 conn(0x7f45c006cbb0 0x7f45c006d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.461+0000 7f45c5a15700 1 -- 172.21.15.157:0/1753318926 >> v1:172.21.15.175:6789/0 conn(0x7f45c006cf80 legacy=0x7f45c014fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:50.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.461+0000 7f45c5a15700 1 -- 172.21.15.157:0/1753318926 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c00c8e50 con 0x7f45c006c1f0 2023-10-06T03:57:50.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.461+0000 7f45c5a15700 1 --2- 172.21.15.157:0/1753318926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45c006c1f0 0x7f45c006da70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f45b00099e0 tx=0x7f45b0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9305562e3adb67a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:50.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.461+0000 7f45c5214700 1 -- 172.21.15.157:0/1753318926 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b000b8b0 con 0x7f45c006c1f0 2023-10-06T03:57:50.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.461+0000 7f45c5214700 1 -- 172.21.15.157:0/1753318926 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b000ba10 con 0x7f45c006c1f0 2023-10-06T03:57:50.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.461+0000 7f45c5214700 1 -- 172.21.15.157:0/1753318926 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b000bdb0 con 0x7f45c006c1f0 2023-10-06T03:57:50.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.462+0000 7f45c847a700 1 -- 172.21.15.157:0/1753318926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45c006c1f0 msgr2=0x7f45c006da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:50.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.462+0000 7f45c847a700 1 --2- 172.21.15.157:0/1753318926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45c006c1f0 0x7f45c006da70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f45b00099e0 tx=0x7f45b0009250 comp rx=0 tx=0).stop 2023-10-06T03:57:50.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.462+0000 7f45c847a700 1 -- 172.21.15.157:0/1753318926 shutdown_connections 2023-10-06T03:57:50.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.462+0000 7f45c847a700 1 --2- 172.21.15.157:0/1753318926 >> v2:172.21.15.175:3300/0 conn(0x7f45c006cbb0 0x7f45c006d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.462+0000 7f45c847a700 1 --2- 172.21.15.157:0/1753318926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45c006c1f0 0x7f45c006da70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.462+0000 7f45c847a700 1 -- 172.21.15.157:0/1753318926 >> 172.21.15.157:0/1753318926 conn(0x7f45c006f230 msgr2=0x7f45c006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:50.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.462+0000 7f45c847a700 1 -- 172.21.15.157:0/1753318926 shutdown_connections 2023-10-06T03:57:50.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.462+0000 7f45c847a700 1 -- 172.21.15.157:0/1753318926 wait complete. 2023-10-06T03:57:50.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.463+0000 7f45c847a700 1 Processor -- start 2023-10-06T03:57:50.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.463+0000 7f45c847a700 1 -- start start 2023-10-06T03:57:50.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.464+0000 7f45c847a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45c006c1f0 0x7f45c0155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:50.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.464+0000 7f45c847a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45c006cbb0 0x7f45c01569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:50.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.464+0000 7f45c847a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45c006cf80 0x7f45c00bfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:50.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.464+0000 7f45c847a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c00c16e0 con 0x7f45c006cf80 2023-10-06T03:57:50.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.464+0000 7f45c847a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c00c01b0 con 0x7f45c006cbb0 2023-10-06T03:57:50.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.464+0000 7f45c847a700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c0166b60 con 0x7f45c006c1f0 2023-10-06T03:57:50.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.464+0000 7f45c6a17700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45c006cf80 0x7f45c00bfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:50.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.464+0000 7f45c6a17700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45c006cf80 0x7f45c00bfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45274/0 (socket says 172.21.15.157:45274) 2023-10-06T03:57:50.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.464+0000 7f45c6a17700 1 -- 172.21.15.157:0/386158768 learned_addr learned my addr 172.21.15.157:0/386158768 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:50.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.464+0000 7f45c6216700 1 --2- 172.21.15.157:0/386158768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45c006c1f0 0x7f45c0155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:50.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.464+0000 7f45c5a15700 1 --2- 172.21.15.157:0/386158768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45c006cbb0 0x7f45c01569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:50.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.464+0000 7f45c6a17700 1 -- 172.21.15.157:0/386158768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45c006c1f0 msgr2=0x7f45c0155a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:50.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.464+0000 7f45c6a17700 1 --2- 172.21.15.157:0/386158768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45c006c1f0 0x7f45c0155a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.464+0000 7f45c6a17700 1 -- 172.21.15.157:0/386158768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45c006cbb0 msgr2=0x7f45c01569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:50.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.464+0000 7f45c6a17700 1 --2- 172.21.15.157:0/386158768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45c006cbb0 0x7f45c01569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.464+0000 7f45c6a17700 1 -- 172.21.15.157:0/386158768 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b00094f0 con 0x7f45c006cf80 2023-10-06T03:57:50.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.464+0000 7f45c6a17700 1 --2- 172.21.15.157:0/386158768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45c006cf80 0x7f45c00bfc70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f45b800c510 tx=0x7f45b8007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:50.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.464+0000 7f45b67fc700 1 -- 172.21.15.157:0/386158768 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b800ce80 con 0x7f45c006cf80 2023-10-06T03:57:50.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.465+0000 7f45b67fc700 1 -- 172.21.15.157:0/386158768 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b8005a70 con 0x7f45c006cf80 2023-10-06T03:57:50.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.465+0000 7f45c847a700 1 -- 172.21.15.157:0/386158768 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45c0166e40 con 0x7f45c006cf80 2023-10-06T03:57:50.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.465+0000 7f45b67fc700 1 -- 172.21.15.157:0/386158768 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b8005df0 con 0x7f45c006cf80 2023-10-06T03:57:50.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.465+0000 7f45c847a700 1 -- 172.21.15.157:0/386158768 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c00c1090 con 0x7f45c006cf80 2023-10-06T03:57:50.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.466+0000 7f45c847a700 1 -- 172.21.15.157:0/386158768 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45c00c1890 con 0x7f45c006cf80 2023-10-06T03:57:50.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.466+0000 7f45b67fc700 1 -- 172.21.15.157:0/386158768 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f45b80072e0 con 0x7f45c006cf80 2023-10-06T03:57:50.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.466+0000 7f45b67fc700 1 --2- 172.21.15.157:0/386158768 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f45a4066f30 0x7f45a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:50.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.467+0000 7f45b67fc700 1 -- 172.21.15.157:0/386158768 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f45b8083980 con 0x7f45c006cf80 2023-10-06T03:57:50.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.467+0000 7f45c6216700 1 --2- 172.21.15.157:0/386158768 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f45a4066f30 0x7f45a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:50.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.467+0000 7f45c6216700 1 --2- 172.21.15.157:0/386158768 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f45a4066f30 0x7f45a40693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f45bc00f570 tx=0x7f45bc00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:50.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.471+0000 7f45b67fc700 1 -- 172.21.15.157:0/386158768 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f45c00c1890 con 0x7f45c006cf80 2023-10-06T03:57:50.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.652+0000 7f45c847a700 1 -- 172.21.15.157:0/386158768 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f45c00c1890 con 0x7f45c006cf80 2023-10-06T03:57:50.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.652+0000 7f45b67fc700 1 -- 172.21.15.157:0/386158768 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f45c00c1890 con 0x7f45c006cf80 2023-10-06T03:57:50.654 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-10-06T03:57:50.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.655+0000 7f45c847a700 1 -- 172.21.15.157:0/386158768 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f45a4066f30 msgr2=0x7f45a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:50.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.655+0000 7f45c847a700 1 --2- 172.21.15.157:0/386158768 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f45a4066f30 0x7f45a40693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f45bc00f570 tx=0x7f45bc00b040 comp rx=0 tx=0).stop 2023-10-06T03:57:50.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.655+0000 7f45c847a700 1 -- 172.21.15.157:0/386158768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45c006cf80 msgr2=0x7f45c00bfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:50.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.655+0000 7f45c847a700 1 --2- 172.21.15.157:0/386158768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45c006cf80 0x7f45c00bfc70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f45b800c510 tx=0x7f45b8007f50 comp rx=0 tx=0).stop 2023-10-06T03:57:50.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.655+0000 7f45c847a700 1 -- 172.21.15.157:0/386158768 shutdown_connections 2023-10-06T03:57:50.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.655+0000 7f45c847a700 1 --2- 172.21.15.157:0/386158768 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f45a4066f30 0x7f45a40693e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.655+0000 7f45c847a700 1 --2- 172.21.15.157:0/386158768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45c006cf80 0x7f45c00bfc70 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.655+0000 7f45c847a700 1 --2- 172.21.15.157:0/386158768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45c006c1f0 0x7f45c0155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.655+0000 7f45c847a700 1 --2- 172.21.15.157:0/386158768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45c006cbb0 0x7f45c01569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.655+0000 7f45c847a700 1 -- 172.21.15.157:0/386158768 >> 172.21.15.157:0/386158768 conn(0x7f45c006f230 msgr2=0x7f45c014e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:50.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.655+0000 7f45c847a700 1 -- 172.21.15.157:0/386158768 shutdown_connections 2023-10-06T03:57:50.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.655+0000 7f45c847a700 1 -- 172.21.15.157:0/386158768 wait complete. 2023-10-06T03:57:50.668 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:57:50.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.784+0000 7fb707093700 1 Processor -- start 2023-10-06T03:57:50.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.784+0000 7fb707093700 1 -- start start 2023-10-06T03:57:50.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.785+0000 7fb707093700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb70006cbb0 0x7fb70006d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:50.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.785+0000 7fb707093700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb70006c1f0 0x7fb70006da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:50.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.785+0000 7fb707093700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb70006cf80 0x7fb70014fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:50.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.785+0000 7fb707093700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7000c8c40 con 0x7fb70006c1f0 2023-10-06T03:57:50.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.785+0000 7fb707093700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7000c9780 con 0x7fb70006cbb0 2023-10-06T03:57:50.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.785+0000 7fb707093700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb70006e0c0 con 0x7fb70006cf80 2023-10-06T03:57:50.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.785+0000 7fb6fffff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb70006c1f0 0x7fb70006da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:50.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.785+0000 7fb6fffff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb70006c1f0 0x7fb70006da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45286/0 (socket says 172.21.15.157:45286) 2023-10-06T03:57:50.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.785+0000 7fb6fffff700 1 -- 172.21.15.157:0/2092488232 learned_addr learned my addr 172.21.15.157:0/2092488232 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:50.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.785+0000 7fb704e2f700 1 --2- 172.21.15.157:0/2092488232 >> v2:172.21.15.175:3300/0 conn(0x7fb70006cbb0 0x7fb70006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:50.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.785+0000 7fb705630700 1 --2- 172.21.15.157:0/2092488232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb70006cf80 0x7fb70014fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:50.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.785+0000 7fb6fffff700 1 -- 172.21.15.157:0/2092488232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb70006cf80 msgr2=0x7fb70014fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:50.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.785+0000 7fb6fffff700 1 --2- 172.21.15.157:0/2092488232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb70006cf80 0x7fb70014fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.785+0000 7fb6fffff700 1 -- 172.21.15.157:0/2092488232 >> v2:172.21.15.175:3300/0 conn(0x7fb70006cbb0 msgr2=0x7fb70006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:50.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.785+0000 7fb6fffff700 1 --2- 172.21.15.157:0/2092488232 >> v2:172.21.15.175:3300/0 conn(0x7fb70006cbb0 0x7fb70006d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.785+0000 7fb6fffff700 1 -- 172.21.15.157:0/2092488232 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7000c8e50 con 0x7fb70006c1f0 2023-10-06T03:57:50.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.785+0000 7fb6fffff700 1 --2- 172.21.15.157:0/2092488232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb70006c1f0 0x7fb70006da70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb70006fd70 tx=0x7fb6f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1632b4388494 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:50.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.786+0000 7fb6ff7fe700 1 -- 172.21.15.157:0/2092488232 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f000b6d0 con 0x7fb70006c1f0 2023-10-06T03:57:50.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.786+0000 7fb6ff7fe700 1 -- 172.21.15.157:0/2092488232 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f000b830 con 0x7fb70006c1f0 2023-10-06T03:57:50.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.786+0000 7fb6ff7fe700 1 -- 172.21.15.157:0/2092488232 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f000bbf0 con 0x7fb70006c1f0 2023-10-06T03:57:50.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.786+0000 7fb707093700 1 -- 172.21.15.157:0/2092488232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb70006c1f0 msgr2=0x7fb70006da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:50.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.786+0000 7fb707093700 1 --2- 172.21.15.157:0/2092488232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb70006c1f0 0x7fb70006da70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb70006fd70 tx=0x7fb6f0009250 comp rx=0 tx=0).stop 2023-10-06T03:57:50.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.786+0000 7fb707093700 1 -- 172.21.15.157:0/2092488232 shutdown_connections 2023-10-06T03:57:50.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.786+0000 7fb707093700 1 --2- 172.21.15.157:0/2092488232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb70006cf80 0x7fb70014fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.786+0000 7fb707093700 1 --2- 172.21.15.157:0/2092488232 >> v2:172.21.15.175:3300/0 conn(0x7fb70006cbb0 0x7fb70006d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.786+0000 7fb707093700 1 --2- 172.21.15.157:0/2092488232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb70006c1f0 0x7fb70006da70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.786+0000 7fb707093700 1 -- 172.21.15.157:0/2092488232 >> 172.21.15.157:0/2092488232 conn(0x7fb70006f230 msgr2=0x7fb70006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:50.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.786+0000 7fb707093700 1 -- 172.21.15.157:0/2092488232 shutdown_connections 2023-10-06T03:57:50.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.786+0000 7fb707093700 1 -- 172.21.15.157:0/2092488232 wait complete. 2023-10-06T03:57:50.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.787+0000 7fb707093700 1 Processor -- start 2023-10-06T03:57:50.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.787+0000 7fb707093700 1 -- start start 2023-10-06T03:57:50.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.788+0000 7fb707093700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb70006c1f0 0x7fb7001580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:50.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.788+0000 7fb707093700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb70006cbb0 0x7fb700155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:50.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.788+0000 7fb707093700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb70006cf80 0x7fb700158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:50.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.788+0000 7fb707093700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb700153bb0 con 0x7fb70006cbb0 2023-10-06T03:57:50.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.788+0000 7fb707093700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7001588e0 con 0x7fb70006c1f0 2023-10-06T03:57:50.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.788+0000 7fb707093700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7001585e0 con 0x7fb70006cf80 2023-10-06T03:57:50.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.788+0000 7fb6fffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb70006cbb0 0x7fb700155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:50.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.788+0000 7fb6fffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb70006cbb0 0x7fb700155e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45294/0 (socket says 172.21.15.157:45294) 2023-10-06T03:57:50.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.788+0000 7fb6fffff700 1 -- 172.21.15.157:0/372850876 learned_addr learned my addr 172.21.15.157:0/372850876 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:50.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.788+0000 7fb704e2f700 1 --2- 172.21.15.157:0/372850876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb70006c1f0 0x7fb7001580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:50.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.788+0000 7fb705630700 1 --2- 172.21.15.157:0/372850876 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb70006cf80 0x7fb700158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:50.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.788+0000 7fb6fffff700 1 -- 172.21.15.157:0/372850876 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb70006cf80 msgr2=0x7fb700158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:50.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.788+0000 7fb6fffff700 1 --2- 172.21.15.157:0/372850876 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb70006cf80 0x7fb700158d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.788+0000 7fb6fffff700 1 -- 172.21.15.157:0/372850876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb70006c1f0 msgr2=0x7fb7001580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:50.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.788+0000 7fb6fffff700 1 --2- 172.21.15.157:0/372850876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb70006c1f0 0x7fb7001580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.789+0000 7fb6fffff700 1 -- 172.21.15.157:0/372850876 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f00094f0 con 0x7fb70006cbb0 2023-10-06T03:57:50.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.789+0000 7fb6fffff700 1 --2- 172.21.15.157:0/372850876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb70006cbb0 0x7fb700155e30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb7000c3960 tx=0x7fb6f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:50.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.789+0000 7fb6fcff9700 1 -- 172.21.15.157:0/372850876 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0000d10 con 0x7fb70006cbb0 2023-10-06T03:57:50.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.789+0000 7fb6fcff9700 1 -- 172.21.15.157:0/372850876 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f000bbb0 con 0x7fb70006cbb0 2023-10-06T03:57:50.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.789+0000 7fb707093700 1 -- 172.21.15.157:0/372850876 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb700166660 con 0x7fb70006cbb0 2023-10-06T03:57:50.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.789+0000 7fb6fcff9700 1 -- 172.21.15.157:0/372850876 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0011670 con 0x7fb70006cbb0 2023-10-06T03:57:50.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.789+0000 7fb707093700 1 -- 172.21.15.157:0/372850876 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb700166c20 con 0x7fb70006cbb0 2023-10-06T03:57:50.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.790+0000 7fb707093700 1 -- 172.21.15.157:0/372850876 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb700158a70 con 0x7fb70006cbb0 2023-10-06T03:57:50.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.790+0000 7fb6fcff9700 1 -- 172.21.15.157:0/372850876 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb6f00119e0 con 0x7fb70006cbb0 2023-10-06T03:57:50.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.790+0000 7fb6fcff9700 1 --2- 172.21.15.157:0/372850876 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb6e0066f60 0x7fb6e0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:50.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.791+0000 7fb6fcff9700 1 -- 172.21.15.157:0/372850876 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb6f0085e60 con 0x7fb70006cbb0 2023-10-06T03:57:50.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.791+0000 7fb704e2f700 1 --2- 172.21.15.157:0/372850876 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb6e0066f60 0x7fb6e0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:50.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.791+0000 7fb704e2f700 1 --2- 172.21.15.157:0/372850876 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb6e0066f60 0x7fb6e0069410 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb6f800f570 tx=0x7fb6f800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:50.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.795+0000 7fb6fcff9700 1 -- 172.21.15.157:0/372850876 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6f0052f70 con 0x7fb70006cbb0 2023-10-06T03:57:50.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.977+0000 7fb707093700 1 -- 172.21.15.157:0/372850876 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb7000c9130 con 0x7fb70006cbb0 2023-10-06T03:57:50.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.977+0000 7fb6fcff9700 1 -- 172.21.15.157:0/372850876 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb6f0056590 con 0x7fb70006cbb0 2023-10-06T03:57:50.979 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-10-06T03:57:50.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.979+0000 7fb707093700 1 -- 172.21.15.157:0/372850876 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb6e0066f60 msgr2=0x7fb6e0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:50.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.979+0000 7fb707093700 1 --2- 172.21.15.157:0/372850876 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb6e0066f60 0x7fb6e0069410 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb6f800f570 tx=0x7fb6f800b040 comp rx=0 tx=0).stop 2023-10-06T03:57:50.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.979+0000 7fb707093700 1 -- 172.21.15.157:0/372850876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb70006cbb0 msgr2=0x7fb700155e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:50.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.979+0000 7fb707093700 1 --2- 172.21.15.157:0/372850876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb70006cbb0 0x7fb700155e30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb7000c3960 tx=0x7fb6f0017000 comp rx=0 tx=0).stop 2023-10-06T03:57:50.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.979+0000 7fb707093700 1 -- 172.21.15.157:0/372850876 shutdown_connections 2023-10-06T03:57:50.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.979+0000 7fb707093700 1 --2- 172.21.15.157:0/372850876 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb6e0066f60 0x7fb6e0069410 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.979+0000 7fb707093700 1 --2- 172.21.15.157:0/372850876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb70006c1f0 0x7fb7001580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.979+0000 7fb707093700 1 --2- 172.21.15.157:0/372850876 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb70006cf80 0x7fb700158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.979+0000 7fb707093700 1 --2- 172.21.15.157:0/372850876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb70006cbb0 0x7fb700155e30 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:50.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.979+0000 7fb707093700 1 -- 172.21.15.157:0/372850876 >> 172.21.15.157:0/372850876 conn(0x7fb70006f230 msgr2=0x7fb70006d530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:50.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.980+0000 7fb707093700 1 -- 172.21.15.157:0/372850876 shutdown_connections 2023-10-06T03:57:50.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:50.980+0000 7fb707093700 1 -- 172.21.15.157:0/372850876 wait complete. 2023-10-06T03:57:50.993 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:57:51.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.108+0000 7f2ce78bf700 1 Processor -- start 2023-10-06T03:57:51.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.109+0000 7f2ce78bf700 1 -- start start 2023-10-06T03:57:51.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.109+0000 7f2ce78bf700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2ce006a780 0x7f2ce0154010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:51.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.109+0000 7f2ce78bf700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2ce0068df0 0x7f2ce0154550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:51.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.109+0000 7f2ce78bf700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce00c36d0 con 0x7f2ce006e2e0 2023-10-06T03:57:51.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.109+0000 7f2ce78bf700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ce00c3e30 con 0x7f2ce0068df0 2023-10-06T03:57:51.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.109+0000 7f2ce78bf700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ce00c4110 con 0x7f2ce006a780 2023-10-06T03:57:51.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.110+0000 7f2ce4e5a700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2ce0068df0 0x7f2ce0154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:51.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.110+0000 7f2ce5e5c700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f2ce006e2e0 0x7f2ce0150d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40858/0 (socket says 172.21.15.157:40858) 2023-10-06T03:57:51.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.110+0000 7f2ce5e5c700 1 -- 172.21.15.157:0/797125745 learned_addr learned my addr 172.21.15.157:0/797125745 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:51.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.110+0000 7f2ce565b700 1 --2- 172.21.15.157:0/797125745 >> v2:172.21.15.175:3300/0 conn(0x7f2ce006a780 0x7f2ce0154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:51.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.110+0000 7f2ce4e5a700 1 -- 172.21.15.157:0/797125745 >> v2:172.21.15.175:3300/0 conn(0x7f2ce006a780 msgr2=0x7f2ce0154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:51.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.110+0000 7f2ce4e5a700 1 --2- 172.21.15.157:0/797125745 >> v2:172.21.15.175:3300/0 conn(0x7f2ce006a780 0x7f2ce0154010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:51.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.110+0000 7f2ce4e5a700 1 -- 172.21.15.157:0/797125745 >> v1:172.21.15.157:6789/0 conn(0x7f2ce006e2e0 legacy=0x7f2ce0150d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:51.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.110+0000 7f2ce4e5a700 1 -- 172.21.15.157:0/797125745 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce00c43f0 con 0x7f2ce0068df0 2023-10-06T03:57:51.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.110+0000 7f2cd7fff700 1 -- 172.21.15.157:0/797125745 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3097693362 0 0) 0x7f2ce00c36d0 con 0x7f2ce006e2e0 2023-10-06T03:57:51.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.110+0000 7f2ce565b700 1 --2- 172.21.15.157:0/797125745 >> v2:172.21.15.175:3300/0 conn(0x7f2ce006a780 0x7f2ce0154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:51.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.110+0000 7f2ce4e5a700 1 --2- 172.21.15.157:0/797125745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce0068df0 0x7f2ce0154550 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2ce006a430 tx=0x7f2cd00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83870f682228b17 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:51.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.110+0000 7f2cd7fff700 1 -- 172.21.15.157:0/797125745 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd000b740 con 0x7f2ce0068df0 2023-10-06T03:57:51.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.110+0000 7f2cd7fff700 1 -- 172.21.15.157:0/797125745 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd000b8a0 con 0x7f2ce0068df0 2023-10-06T03:57:51.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.110+0000 7f2cd7fff700 1 -- 172.21.15.157:0/797125745 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd000bc40 con 0x7f2ce0068df0 2023-10-06T03:57:51.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.111+0000 7f2ce78bf700 1 -- 172.21.15.157:0/797125745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce0068df0 msgr2=0x7f2ce0154550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:51.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.111+0000 7f2ce78bf700 1 --2- 172.21.15.157:0/797125745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce0068df0 0x7f2ce0154550 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2ce006a430 tx=0x7f2cd00091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:51.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.111+0000 7f2ce78bf700 1 -- 172.21.15.157:0/797125745 shutdown_connections 2023-10-06T03:57:51.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.111+0000 7f2ce78bf700 1 --2- 172.21.15.157:0/797125745 >> v2:172.21.15.175:3300/0 conn(0x7f2ce006a780 0x7f2ce0154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:51.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.111+0000 7f2ce78bf700 1 --2- 172.21.15.157:0/797125745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce0068df0 0x7f2ce0154550 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:51.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.111+0000 7f2ce78bf700 1 -- 172.21.15.157:0/797125745 >> 172.21.15.157:0/797125745 conn(0x7f2ce006b510 msgr2=0x7f2ce006b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:51.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.111+0000 7f2ce78bf700 1 -- 172.21.15.157:0/797125745 shutdown_connections 2023-10-06T03:57:51.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.111+0000 7f2ce78bf700 1 -- 172.21.15.157:0/797125745 wait complete. 2023-10-06T03:57:51.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.112+0000 7f2ce78bf700 1 Processor -- start 2023-10-06T03:57:51.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.112+0000 7f2ce78bf700 1 -- start start 2023-10-06T03:57:51.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.113+0000 7f2ce78bf700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ce0068df0 0x7f2ce0151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:51.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.113+0000 7f2ce78bf700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ce006a780 0x7f2ce00bb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:51.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.113+0000 7f2ce78bf700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce006e2e0 0x7f2ce00c0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:51.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.113+0000 7f2ce78bf700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce014bba0 con 0x7f2ce006e2e0 2023-10-06T03:57:51.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.113+0000 7f2ce78bf700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce00b9d30 con 0x7f2ce006a780 2023-10-06T03:57:51.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.113+0000 7f2ce78bf700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce014bdf0 con 0x7f2ce0068df0 2023-10-06T03:57:51.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.113+0000 7f2ce5e5c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce006e2e0 0x7f2ce00c0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:51.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.113+0000 7f2ce5e5c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce006e2e0 0x7f2ce00c0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45314/0 (socket says 172.21.15.157:45314) 2023-10-06T03:57:51.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.113+0000 7f2ce5e5c700 1 -- 172.21.15.157:0/1867173163 learned_addr learned my addr 172.21.15.157:0/1867173163 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:51.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.113+0000 7f2ce565b700 1 --2- 172.21.15.157:0/1867173163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ce0068df0 0x7f2ce0151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:51.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.113+0000 7f2ce4e5a700 1 --2- 172.21.15.157:0/1867173163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ce006a780 0x7f2ce00bb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:51.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.113+0000 7f2ce5e5c700 1 -- 172.21.15.157:0/1867173163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ce0068df0 msgr2=0x7f2ce0151960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:51.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.113+0000 7f2ce5e5c700 1 --2- 172.21.15.157:0/1867173163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ce0068df0 0x7f2ce0151960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:51.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.113+0000 7f2ce5e5c700 1 -- 172.21.15.157:0/1867173163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ce006a780 msgr2=0x7f2ce00bb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:51.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.113+0000 7f2ce5e5c700 1 --2- 172.21.15.157:0/1867173163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ce006a780 0x7f2ce00bb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:51.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.113+0000 7f2ce5e5c700 1 -- 172.21.15.157:0/1867173163 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd0009420 con 0x7f2ce006e2e0 2023-10-06T03:57:51.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.113+0000 7f2ce565b700 1 --2- 172.21.15.157:0/1867173163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ce0068df0 0x7f2ce0151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:51.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.113+0000 7f2ce4e5a700 1 --2- 172.21.15.157:0/1867173163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ce006a780 0x7f2ce00bb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:51.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.113+0000 7f2ce5e5c700 1 --2- 172.21.15.157:0/1867173163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce006e2e0 0x7f2ce00c0c90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800c530 tx=0x7f2cd8007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:51.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.114+0000 7f2cd57fa700 1 -- 172.21.15.157:0/1867173163 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8005b30 con 0x7f2ce006e2e0 2023-10-06T03:57:51.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.114+0000 7f2cd57fa700 1 -- 172.21.15.157:0/1867173163 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd8005c90 con 0x7f2ce006e2e0 2023-10-06T03:57:51.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.114+0000 7f2cd57fa700 1 -- 172.21.15.157:0/1867173163 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8007320 con 0x7f2ce006e2e0 2023-10-06T03:57:51.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.114+0000 7f2ce78bf700 1 -- 172.21.15.157:0/1867173163 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce014c0d0 con 0x7f2ce006e2e0 2023-10-06T03:57:51.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.114+0000 7f2ce78bf700 1 -- 172.21.15.157:0/1867173163 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce014c690 con 0x7f2ce006e2e0 2023-10-06T03:57:51.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.115+0000 7f2ce78bf700 1 -- 172.21.15.157:0/1867173163 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce00692b0 con 0x7f2ce006e2e0 2023-10-06T03:57:51.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.115+0000 7f2cd57fa700 1 -- 172.21.15.157:0/1867173163 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2cd801b3b0 con 0x7f2ce006e2e0 2023-10-06T03:57:51.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.115+0000 7f2cd57fa700 1 --2- 172.21.15.157:0/1867173163 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2cc4066f30 0x7f2cc40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:51.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.115+0000 7f2cd57fa700 1 -- 172.21.15.157:0/1867173163 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2cd8083c50 con 0x7f2ce006e2e0 2023-10-06T03:57:51.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.116+0000 7f2ce565b700 1 --2- 172.21.15.157:0/1867173163 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2cc4066f30 0x7f2cc40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:51.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.116+0000 7f2ce565b700 1 --2- 172.21.15.157:0/1867173163 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2cc4066f30 0x7f2cc40693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc0076e0 tx=0x7f2cdc0098f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:51.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.119+0000 7f2cd57fa700 1 -- 172.21.15.157:0/1867173163 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2cd8050cc0 con 0x7f2ce006e2e0 2023-10-06T03:57:51.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.302+0000 7f2ce78bf700 1 -- 172.21.15.157:0/1867173163 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2ce006c1f0 con 0x7f2ce006e2e0 2023-10-06T03:57:51.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.303+0000 7f2cd57fa700 1 -- 172.21.15.157:0/1867173163 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2cd8017020 con 0x7f2ce006e2e0 2023-10-06T03:57:51.304 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-10-06T03:57:51.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.305+0000 7f2ce78bf700 1 -- 172.21.15.157:0/1867173163 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2cc4066f30 msgr2=0x7f2cc40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:51.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.305+0000 7f2ce78bf700 1 --2- 172.21.15.157:0/1867173163 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2cc4066f30 0x7f2cc40693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc0076e0 tx=0x7f2cdc0098f0 comp rx=0 tx=0).stop 2023-10-06T03:57:51.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.305+0000 7f2ce78bf700 1 -- 172.21.15.157:0/1867173163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce006e2e0 msgr2=0x7f2ce00c0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:51.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.305+0000 7f2ce78bf700 1 --2- 172.21.15.157:0/1867173163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce006e2e0 0x7f2ce00c0c90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800c530 tx=0x7f2cd8007f90 comp rx=0 tx=0).stop 2023-10-06T03:57:51.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.305+0000 7f2ce78bf700 1 -- 172.21.15.157:0/1867173163 shutdown_connections 2023-10-06T03:57:51.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.305+0000 7f2ce78bf700 1 --2- 172.21.15.157:0/1867173163 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2cc4066f30 0x7f2cc40693e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:51.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.305+0000 7f2ce78bf700 1 --2- 172.21.15.157:0/1867173163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce006e2e0 0x7f2ce00c0c90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:51.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.305+0000 7f2ce78bf700 1 --2- 172.21.15.157:0/1867173163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ce0068df0 0x7f2ce0151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:51.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.305+0000 7f2ce78bf700 1 --2- 172.21.15.157:0/1867173163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ce006a780 0x7f2ce00bb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:51.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.305+0000 7f2ce78bf700 1 -- 172.21.15.157:0/1867173163 >> 172.21.15.157:0/1867173163 conn(0x7f2ce006b510 msgr2=0x7f2ce014f260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:51.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.305+0000 7f2ce78bf700 1 -- 172.21.15.157:0/1867173163 shutdown_connections 2023-10-06T03:57:51.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.305+0000 7f2ce78bf700 1 -- 172.21.15.157:0/1867173163 wait complete. 2023-10-06T03:57:51.318 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:57:51.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.439+0000 7f2c186f0700 1 Processor -- start 2023-10-06T03:57:51.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.439+0000 7f2c186f0700 1 -- start start 2023-10-06T03:57:51.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.440+0000 7f2c186f0700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2c1006cbb0 0x7f2c1006d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:51.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.440+0000 7f2c186f0700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2c1006c1f0 0x7f2c1006da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:51.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.440+0000 7f2c186f0700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c100c9780 con 0x7f2c1006cf80 2023-10-06T03:57:51.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.440+0000 7f2c186f0700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c100c8c40 con 0x7f2c1006cbb0 2023-10-06T03:57:51.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.440+0000 7f2c186f0700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c1006e0c0 con 0x7f2c1006c1f0 2023-10-06T03:57:51.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.440+0000 7f2c1648c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2c1006cbb0 0x7f2c1006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:51.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.440+0000 7f2c16c8d700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f2c1006cf80 0x7f2c1014fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40864/0 (socket says 172.21.15.157:40864) 2023-10-06T03:57:51.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.440+0000 7f2c16c8d700 1 -- 172.21.15.157:0/28351320 learned_addr learned my addr 172.21.15.157:0/28351320 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:51.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.440+0000 7f2c15c8b700 1 --2- 172.21.15.157:0/28351320 >> v2:172.21.15.175:3300/0 conn(0x7f2c1006c1f0 0x7f2c1006da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:51.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.440+0000 7f2c1648c700 1 -- 172.21.15.157:0/28351320 >> v2:172.21.15.175:3300/0 conn(0x7f2c1006c1f0 msgr2=0x7f2c1006da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:51.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.440+0000 7f2c1648c700 1 --2- 172.21.15.157:0/28351320 >> v2:172.21.15.175:3300/0 conn(0x7f2c1006c1f0 0x7f2c1006da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:51.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.440+0000 7f2c1648c700 1 -- 172.21.15.157:0/28351320 >> v1:172.21.15.157:6789/0 conn(0x7f2c1006cf80 legacy=0x7f2c1014fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:51.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.440+0000 7f2c1648c700 1 -- 172.21.15.157:0/28351320 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c100c8e50 con 0x7f2c1006cbb0 2023-10-06T03:57:51.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.440+0000 7f2c1648c700 1 --2- 172.21.15.157:0/28351320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c1006cbb0 0x7f2c1006d530 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2c10157070 tx=0x7f2c0c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd127505b95fb2b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:51.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.440+0000 7f2c1548a700 1 -- 172.21.15.157:0/28351320 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c00e070 con 0x7f2c1006cbb0 2023-10-06T03:57:51.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.440+0000 7f2c1548a700 1 -- 172.21.15.157:0/28351320 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c0c00bcc0 con 0x7f2c1006cbb0 2023-10-06T03:57:51.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.440+0000 7f2c1548a700 1 -- 172.21.15.157:0/28351320 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c012680 con 0x7f2c1006cbb0 2023-10-06T03:57:51.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.441+0000 7f2c186f0700 1 -- 172.21.15.157:0/28351320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c1006cbb0 msgr2=0x7f2c1006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:51.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.441+0000 7f2c186f0700 1 --2- 172.21.15.157:0/28351320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c1006cbb0 0x7f2c1006d530 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2c10157070 tx=0x7f2c0c009780 comp rx=0 tx=0).stop 2023-10-06T03:57:51.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.441+0000 7f2c186f0700 1 -- 172.21.15.157:0/28351320 shutdown_connections 2023-10-06T03:57:51.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.441+0000 7f2c186f0700 1 --2- 172.21.15.157:0/28351320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c1006cbb0 0x7f2c1006d530 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:51.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.441+0000 7f2c186f0700 1 --2- 172.21.15.157:0/28351320 >> v2:172.21.15.175:3300/0 conn(0x7f2c1006c1f0 0x7f2c1006da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:51.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.441+0000 7f2c186f0700 1 -- 172.21.15.157:0/28351320 >> 172.21.15.157:0/28351320 conn(0x7f2c1006f230 msgr2=0x7f2c1006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:51.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.441+0000 7f2c186f0700 1 -- 172.21.15.157:0/28351320 shutdown_connections 2023-10-06T03:57:51.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.441+0000 7f2c186f0700 1 -- 172.21.15.157:0/28351320 wait complete. 2023-10-06T03:57:51.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.442+0000 7f2c186f0700 1 Processor -- start 2023-10-06T03:57:51.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.442+0000 7f2c186f0700 1 -- start start 2023-10-06T03:57:51.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.443+0000 7f2c186f0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c1006c1f0 0x7f2c10151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:51.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.443+0000 7f2c186f0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c1006cbb0 0x7f2c10169e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:51.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.443+0000 7f2c186f0700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c1006cf80 0x7f2c1016a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:51.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.443+0000 7f2c186f0700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c10157630 con 0x7f2c1006c1f0 2023-10-06T03:57:51.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.443+0000 7f2c186f0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c10157140 con 0x7f2c1006cbb0 2023-10-06T03:57:51.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.443+0000 7f2c186f0700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c10151fd0 con 0x7f2c1006cf80 2023-10-06T03:57:51.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.443+0000 7f2c1648c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c1006c1f0 0x7f2c10151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:51.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.443+0000 7f2c1648c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c1006c1f0 0x7f2c10151220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45334/0 (socket says 172.21.15.157:45334) 2023-10-06T03:57:51.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.443+0000 7f2c1648c700 1 -- 172.21.15.157:0/3280601609 learned_addr learned my addr 172.21.15.157:0/3280601609 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:51.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.443+0000 7f2c15c8b700 1 --2- 172.21.15.157:0/3280601609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c1006cbb0 0x7f2c10169e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:51.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.443+0000 7f2c16c8d700 1 --2- 172.21.15.157:0/3280601609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c1006cf80 0x7f2c1016a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:51.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.443+0000 7f2c1648c700 1 -- 172.21.15.157:0/3280601609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c1006cf80 msgr2=0x7f2c1016a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:51.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.443+0000 7f2c1648c700 1 --2- 172.21.15.157:0/3280601609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c1006cf80 0x7f2c1016a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:51.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.443+0000 7f2c1648c700 1 -- 172.21.15.157:0/3280601609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c1006cbb0 msgr2=0x7f2c10169e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:51.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.443+0000 7f2c1648c700 1 --2- 172.21.15.157:0/3280601609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c1006cbb0 0x7f2c10169e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:51.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.443+0000 7f2c1648c700 1 -- 172.21.15.157:0/3280601609 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c0c009a20 con 0x7f2c1006c1f0 2023-10-06T03:57:51.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.443+0000 7f2c16c8d700 1 --2- 172.21.15.157:0/3280601609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c1006cf80 0x7f2c1016a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:51.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.443+0000 7f2c15c8b700 1 --2- 172.21.15.157:0/3280601609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c1006cbb0 0x7f2c10169e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:57:51.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.443+0000 7f2c1648c700 1 --2- 172.21.15.157:0/3280601609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c1006c1f0 0x7f2c10151220 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2c100c3960 tx=0x7f2c0c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:51.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.443+0000 7f2c067fc700 1 -- 172.21.15.157:0/3280601609 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c027470 con 0x7f2c1006c1f0 2023-10-06T03:57:51.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.443+0000 7f2c067fc700 1 -- 172.21.15.157:0/3280601609 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c0c0275f0 con 0x7f2c1006c1f0 2023-10-06T03:57:51.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.443+0000 7f2c067fc700 1 -- 172.21.15.157:0/3280601609 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c0279b0 con 0x7f2c1006c1f0 2023-10-06T03:57:51.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.443+0000 7f2c186f0700 1 -- 172.21.15.157:0/3280601609 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c1016a9a0 con 0x7f2c1006c1f0 2023-10-06T03:57:51.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.443+0000 7f2c186f0700 1 -- 172.21.15.157:0/3280601609 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c1014dd30 con 0x7f2c1006c1f0 2023-10-06T03:57:51.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.444+0000 7f2c186f0700 1 -- 172.21.15.157:0/3280601609 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c10151d80 con 0x7f2c1006c1f0 2023-10-06T03:57:51.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.445+0000 7f2c067fc700 1 -- 172.21.15.157:0/3280601609 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2c0c024070 con 0x7f2c1006c1f0 2023-10-06T03:57:51.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.445+0000 7f2c067fc700 1 --2- 172.21.15.157:0/3280601609 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2bf4066f30 0x7f2bf40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:51.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.445+0000 7f2c067fc700 1 -- 172.21.15.157:0/3280601609 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2c0c086fe0 con 0x7f2c1006c1f0 2023-10-06T03:57:51.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.446+0000 7f2c15c8b700 1 --2- 172.21.15.157:0/3280601609 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2bf4066f30 0x7f2bf40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:51.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.446+0000 7f2c15c8b700 1 --2- 172.21.15.157:0/3280601609 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2bf4066f30 0x7f2bf40693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2c10157040 tx=0x7f2c00008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:51.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.450+0000 7f2c067fc700 1 -- 172.21.15.157:0/3280601609 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2c0c0540f0 con 0x7f2c1006c1f0 2023-10-06T03:57:51.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.634+0000 7f2c186f0700 1 -- 172.21.15.157:0/3280601609 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2c100c9130 con 0x7f2c1006c1f0 2023-10-06T03:57:51.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.634+0000 7f2c067fc700 1 -- 172.21.15.157:0/3280601609 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2c0c057710 con 0x7f2c1006c1f0 2023-10-06T03:57:51.636 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-10-06T03:57:51.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.637+0000 7f2c186f0700 1 -- 172.21.15.157:0/3280601609 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2bf4066f30 msgr2=0x7f2bf40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:51.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.637+0000 7f2c186f0700 1 --2- 172.21.15.157:0/3280601609 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2bf4066f30 0x7f2bf40693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2c10157040 tx=0x7f2c00008040 comp rx=0 tx=0).stop 2023-10-06T03:57:51.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.637+0000 7f2c186f0700 1 -- 172.21.15.157:0/3280601609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c1006c1f0 msgr2=0x7f2c10151220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:51.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.637+0000 7f2c186f0700 1 --2- 172.21.15.157:0/3280601609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c1006c1f0 0x7f2c10151220 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2c100c3960 tx=0x7f2c0c019000 comp rx=0 tx=0).stop 2023-10-06T03:57:51.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.637+0000 7f2c186f0700 1 -- 172.21.15.157:0/3280601609 shutdown_connections 2023-10-06T03:57:51.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.637+0000 7f2c186f0700 1 --2- 172.21.15.157:0/3280601609 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2bf4066f30 0x7f2bf40693e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:51.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.637+0000 7f2c186f0700 1 --2- 172.21.15.157:0/3280601609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c1006cf80 0x7f2c1016a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:51.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.637+0000 7f2c186f0700 1 --2- 172.21.15.157:0/3280601609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c1006c1f0 0x7f2c10151220 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:51.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.637+0000 7f2c186f0700 1 --2- 172.21.15.157:0/3280601609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c1006cbb0 0x7f2c10169e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:51.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.637+0000 7f2c186f0700 1 -- 172.21.15.157:0/3280601609 >> 172.21.15.157:0/3280601609 conn(0x7f2c1006f230 msgr2=0x7f2c1006d530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:51.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.637+0000 7f2c186f0700 1 -- 172.21.15.157:0/3280601609 shutdown_connections 2023-10-06T03:57:51.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.637+0000 7f2c186f0700 1 -- 172.21.15.157:0/3280601609 wait complete. 2023-10-06T03:57:51.650 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:57:51.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.765+0000 7f2fe4245700 1 Processor -- start 2023-10-06T03:57:51.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.765+0000 7f2fe4245700 1 -- start start 2023-10-06T03:57:51.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.766+0000 7f2fe4245700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fdc069ea0 0x7f2fdc0631f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:51.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.766+0000 7f2fe4245700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2fdc06a270 0x7f2fdc156870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:51.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.766+0000 7f2fe4245700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fdc0c48f0 con 0x7f2fdc06a860 2023-10-06T03:57:51.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.766+0000 7f2fe4245700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fdc156db0 con 0x7f2fdc06a270 2023-10-06T03:57:51.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.766+0000 7f2fe4245700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc0c3d20 con 0x7f2fdc069ea0 2023-10-06T03:57:51.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.766+0000 7f2fe17e0700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f2fdc06a860 0x7f2fdc0c4d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40870/0 (socket says 172.21.15.157:40870) 2023-10-06T03:57:51.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.766+0000 7f2fe17e0700 1 -- 172.21.15.157:0/3588904504 learned_addr learned my addr 172.21.15.157:0/3588904504 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:51.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.766+0000 7f2fe1fe1700 1 --2- 172.21.15.157:0/3588904504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fdc069ea0 0x7f2fdc0631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:51.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.766+0000 7f2fe27e2700 1 --2- 172.21.15.157:0/3588904504 >> v2:172.21.15.175:3300/0 conn(0x7f2fdc06a270 0x7f2fdc156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:51.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.766+0000 7f2fe0fdf700 1 -- 172.21.15.157:0/3588904504 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 830557032 0 0) 0x7f2fdc0c48f0 con 0x7f2fdc06a860 2023-10-06T03:57:51.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.766+0000 7f2fe0fdf700 1 -- 172.21.15.157:0/3588904504 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fc8003570 con 0x7f2fdc06a860 2023-10-06T03:57:51.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.766+0000 7f2fe0fdf700 1 -- 172.21.15.157:0/3588904504 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1326481103 0 0) 0x7f2fc8003570 con 0x7f2fdc06a860 2023-10-06T03:57:51.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.766+0000 7f2fe0fdf700 1 -- 172.21.15.157:0/3588904504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fdc069ea0 msgr2=0x7f2fdc0631f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:51.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.766+0000 7f2fe0fdf700 1 --2- 172.21.15.157:0/3588904504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fdc069ea0 0x7f2fdc0631f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:51.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.766+0000 7f2fe0fdf700 1 -- 172.21.15.157:0/3588904504 >> v2:172.21.15.175:3300/0 conn(0x7f2fdc06a270 msgr2=0x7f2fdc156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:51.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.766+0000 7f2fe0fdf700 1 --2- 172.21.15.157:0/3588904504 >> v2:172.21.15.175:3300/0 conn(0x7f2fdc06a270 0x7f2fdc156870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:51.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.766+0000 7f2fe0fdf700 1 -- 172.21.15.157:0/3588904504 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fdc063840 con 0x7f2fdc06a860 2023-10-06T03:57:51.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.766+0000 7f2fe0fdf700 1 -- 172.21.15.157:0/3588904504 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f2fcc001470 con 0x7f2fdc06a860 2023-10-06T03:57:51.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.766+0000 7f2fe0fdf700 1 -- 172.21.15.157:0/3588904504 >> v1:172.21.15.157:6789/0 conn(0x7f2fdc06a860 legacy=0x7f2fdc0c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:51.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.766+0000 7f2fe0fdf700 1 --2- 172.21.15.157:0/3588904504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fc8003a60 0x7f2fc8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:51.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.766+0000 7f2fe0fdf700 1 --2- 172.21.15.157:0/3588904504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fc8004810 0x7f2fc8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:51.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.766+0000 7f2fe0fdf700 1 --2- 172.21.15.157:0/3588904504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fdc069ea0 0x7f2fc80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:51.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.766+0000 7f2fe0fdf700 1 -- 172.21.15.157:0/3588904504 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc156db0 con 0x7f2fc8004810 2023-10-06T03:57:51.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.766+0000 7f2fe0fdf700 1 -- 172.21.15.157:0/3588904504 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc0c3d20 con 0x7f2fc8003a60 2023-10-06T03:57:51.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.766+0000 7f2fe0fdf700 1 -- 172.21.15.157:0/3588904504 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc800ba60 con 0x7f2fdc069ea0 2023-10-06T03:57:51.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.766+0000 7f2fe17e0700 1 --2- 172.21.15.157:0/3588904504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fc8004810 0x7f2fc8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:51.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.766+0000 7f2fe1fe1700 1 --2- 172.21.15.157:0/3588904504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fc8003a60 0x7f2fc8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:51.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.767+0000 7f2fe27e2700 1 --2- 172.21.15.157:0/3588904504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fdc069ea0 0x7f2fc80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:51.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.767+0000 7f2fe17e0700 1 -- 172.21.15.157:0/3588904504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fdc069ea0 msgr2=0x7f2fc80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:51.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.767+0000 7f2fe17e0700 1 --2- 172.21.15.157:0/3588904504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fdc069ea0 0x7f2fc80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:51.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.767+0000 7f2fe17e0700 1 -- 172.21.15.157:0/3588904504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fc8003a60 msgr2=0x7f2fc8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:51.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.767+0000 7f2fe17e0700 1 --2- 172.21.15.157:0/3588904504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fc8003a60 0x7f2fc8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:51.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.767+0000 7f2fe17e0700 1 -- 172.21.15.157:0/3588904504 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc8003570 con 0x7f2fc8004810 2023-10-06T03:57:51.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.767+0000 7f2fe17e0700 1 --2- 172.21.15.157:0/3588904504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fc8004810 0x7f2fc8004c80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00c3f0 tx=0x7f2fdc069670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770f950f63c9c75a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:51.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.767+0000 7f2fe0fdf700 1 -- 172.21.15.157:0/3588904504 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc00cc80 con 0x7f2fc8004810 2023-10-06T03:57:51.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.767+0000 7f2fe0fdf700 1 -- 172.21.15.157:0/3588904504 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fcc00cde0 con 0x7f2fc8004810 2023-10-06T03:57:51.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.767+0000 7f2fe0fdf700 1 -- 172.21.15.157:0/3588904504 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc005b70 con 0x7f2fc8004810 2023-10-06T03:57:51.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.768+0000 7f2fe4245700 1 -- 172.21.15.157:0/3588904504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fc8004810 msgr2=0x7f2fc8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:51.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.768+0000 7f2fe4245700 1 --2- 172.21.15.157:0/3588904504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fc8004810 0x7f2fc8004c80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00c3f0 tx=0x7f2fdc069670 comp rx=0 tx=0).stop 2023-10-06T03:57:51.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.768+0000 7f2fe27e2700 1 -- 172.21.15.157:0/3588904504 reap_dead start 2023-10-06T03:57:51.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.768+0000 7f2fe4245700 1 -- 172.21.15.157:0/3588904504 shutdown_connections 2023-10-06T03:57:51.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.768+0000 7f2fe4245700 1 -- 172.21.15.157:0/3588904504 >> 172.21.15.157:0/3588904504 conn(0x7f2fdc0c8b20 msgr2=0x7f2fdc0c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:51.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.768+0000 7f2fe4245700 1 -- 172.21.15.157:0/3588904504 shutdown_connections 2023-10-06T03:57:51.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.768+0000 7f2fe4245700 1 -- 172.21.15.157:0/3588904504 wait complete. 2023-10-06T03:57:51.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.769+0000 7f2fe4245700 1 Processor -- start 2023-10-06T03:57:51.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.769+0000 7f2fe4245700 1 -- start start 2023-10-06T03:57:51.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.769+0000 7f2fe4245700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fc8004810 0x7f2fdc1673f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:51.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.769+0000 7f2fe4245700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fdc14b410 0x7f2fdc146ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:51.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.769+0000 7f2fe4245700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fdc149f90 0x7f2fdc1413d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:51.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.769+0000 7f2fe4245700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc154460 con 0x7f2fdc149f90 2023-10-06T03:57:51.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.769+0000 7f2fe4245700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc167930 con 0x7f2fc8004810 2023-10-06T03:57:51.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.769+0000 7f2fe4245700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc158a40 con 0x7f2fdc14b410 2023-10-06T03:57:51.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.770+0000 7f2fe27e2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fdc149f90 0x7f2fdc1413d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:51.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.770+0000 7f2fe27e2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fdc149f90 0x7f2fdc1413d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45350/0 (socket says 172.21.15.157:45350) 2023-10-06T03:57:51.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.770+0000 7f2fe27e2700 1 -- 172.21.15.157:0/1584941903 learned_addr learned my addr 172.21.15.157:0/1584941903 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:51.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.770+0000 7f2fe1fe1700 1 --2- 172.21.15.157:0/1584941903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fc8004810 0x7f2fdc1673f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:51.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.770+0000 7f2fe17e0700 1 --2- 172.21.15.157:0/1584941903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fdc14b410 0x7f2fdc146ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:51.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.770+0000 7f2fe27e2700 1 -- 172.21.15.157:0/1584941903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fdc14b410 msgr2=0x7f2fdc146ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:51.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.770+0000 7f2fe27e2700 1 --2- 172.21.15.157:0/1584941903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fdc14b410 0x7f2fdc146ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:51.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.770+0000 7f2fe27e2700 1 -- 172.21.15.157:0/1584941903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fc8004810 msgr2=0x7f2fdc1673f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:51.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.770+0000 7f2fe27e2700 1 --2- 172.21.15.157:0/1584941903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fc8004810 0x7f2fdc1673f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:51.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.770+0000 7f2fe27e2700 1 -- 172.21.15.157:0/1584941903 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fdc153fd0 con 0x7f2fdc149f90 2023-10-06T03:57:51.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.770+0000 7f2fe27e2700 1 --2- 172.21.15.157:0/1584941903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fdc149f90 0x7f2fdc1413d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2fd40094d0 tx=0x7f2fd400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:51.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.770+0000 7f2fd1ffb700 1 -- 172.21.15.157:0/1584941903 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd400e040 con 0x7f2fdc149f90 2023-10-06T03:57:51.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.770+0000 7f2fd1ffb700 1 -- 172.21.15.157:0/1584941903 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd400f040 con 0x7f2fdc149f90 2023-10-06T03:57:51.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.770+0000 7f2fe4245700 1 -- 172.21.15.157:0/1584941903 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fdc14bad0 con 0x7f2fdc149f90 2023-10-06T03:57:51.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.770+0000 7f2fd1ffb700 1 -- 172.21.15.157:0/1584941903 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd400fda0 con 0x7f2fdc149f90 2023-10-06T03:57:51.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.770+0000 7f2fe4245700 1 -- 172.21.15.157:0/1584941903 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fdc14bd50 con 0x7f2fdc149f90 2023-10-06T03:57:51.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.771+0000 7f2fe4245700 1 -- 172.21.15.157:0/1584941903 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fdc154160 con 0x7f2fdc149f90 2023-10-06T03:57:51.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.772+0000 7f2fd1ffb700 1 -- 172.21.15.157:0/1584941903 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2fd4015640 con 0x7f2fdc149f90 2023-10-06T03:57:51.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.772+0000 7f2fd1ffb700 1 --2- 172.21.15.157:0/1584941903 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2fc0066f60 0x7f2fc0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:51.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.772+0000 7f2fd1ffb700 1 -- 172.21.15.157:0/1584941903 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2fd4084f60 con 0x7f2fdc149f90 2023-10-06T03:57:51.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.772+0000 7f2fe1fe1700 1 --2- 172.21.15.157:0/1584941903 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2fc0066f60 0x7f2fc0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:51.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.773+0000 7f2fe1fe1700 1 --2- 172.21.15.157:0/1584941903 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2fc0066f60 0x7f2fc0069410 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8006f00 tx=0x7f2fd80091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:51.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:51.777+0000 7f2fd1ffb700 1 -- 172.21.15.157:0/1584941903 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2fd4052070 con 0x7f2fdc149f90 2023-10-06T03:57:52.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.002+0000 7f2fe4245700 1 -- 172.21.15.157:0/1584941903 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2fdc0642a0 con 0x7f2fdc149f90 2023-10-06T03:57:52.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.002+0000 7f2fd1ffb700 1 -- 172.21.15.157:0/1584941903 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+6184 (secure 0 0 0) 0x7f2fd4055690 con 0x7f2fdc149f90 2023-10-06T03:57:52.004 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:52.004 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T03:57:37.649946+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"f","rank":4,"incarnation":15,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/4177493830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4177493830},{"type":"v1","addr":"172.21.15.175:6839","nonce":4177493830}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:57:52.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.005+0000 7f2fe4245700 1 -- 172.21.15.157:0/1584941903 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2fc0066f60 msgr2=0x7f2fc0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:52.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.005+0000 7f2fe4245700 1 --2- 172.21.15.157:0/1584941903 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2fc0066f60 0x7f2fc0069410 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8006f00 tx=0x7f2fd80091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:52.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.005+0000 7f2fe4245700 1 -- 172.21.15.157:0/1584941903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fdc149f90 msgr2=0x7f2fdc1413d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:52.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.005+0000 7f2fe4245700 1 --2- 172.21.15.157:0/1584941903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fdc149f90 0x7f2fdc1413d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2fd40094d0 tx=0x7f2fd400b040 comp rx=0 tx=0).stop 2023-10-06T03:57:52.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.005+0000 7f2fe4245700 1 -- 172.21.15.157:0/1584941903 shutdown_connections 2023-10-06T03:57:52.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.005+0000 7f2fe4245700 1 --2- 172.21.15.157:0/1584941903 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2fc0066f60 0x7f2fc0069410 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:52.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.005+0000 7f2fe4245700 1 --2- 172.21.15.157:0/1584941903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fc8004810 0x7f2fdc1673f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:52.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.005+0000 7f2fe4245700 1 --2- 172.21.15.157:0/1584941903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fdc149f90 0x7f2fdc1413d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:52.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.005+0000 7f2fe4245700 1 --2- 172.21.15.157:0/1584941903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fdc14b410 0x7f2fdc146ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:52.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.005+0000 7f2fe4245700 1 -- 172.21.15.157:0/1584941903 >> 172.21.15.157:0/1584941903 conn(0x7f2fdc0c8b20 msgr2=0x7f2fdc14d250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:52.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.005+0000 7f2fe4245700 1 -- 172.21.15.157:0/1584941903 shutdown_connections 2023-10-06T03:57:52.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.005+0000 7f2fe4245700 1 -- 172.21.15.157:0/1584941903 wait complete. 2023-10-06T03:57:52.008 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 16 2023-10-06T03:57:52.018 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:57:52.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.131+0000 7ff6fe8af700 1 Processor -- start 2023-10-06T03:57:52.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.131+0000 7ff6fe8af700 1 -- start start 2023-10-06T03:57:52.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.132+0000 7ff6fe8af700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6f80be970 0x7ff6f806df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:52.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.132+0000 7ff6fe8af700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff6f806e450 0x7ff6f806a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:52.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.132+0000 7ff6fe8af700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6f80bed40 con 0x7ff6f8068d10 2023-10-06T03:57:52.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.132+0000 7ff6fe8af700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6f8157010 con 0x7ff6f806e450 2023-10-06T03:57:52.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.132+0000 7ff6fe8af700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f806f9f0 con 0x7ff6f80be970 2023-10-06T03:57:52.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.132+0000 7ff6fce4c700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff6f8068d10 0x7ff6f81530c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40872/0 (socket says 172.21.15.157:40872) 2023-10-06T03:57:52.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.132+0000 7ff6fce4c700 1 -- 172.21.15.157:0/1929947398 learned_addr learned my addr 172.21.15.157:0/1929947398 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:57:52.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.132+0000 7ff6f7fff700 1 --2- 172.21.15.157:0/1929947398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6f80be970 0x7ff6f806df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:52.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.132+0000 7ff6f77fe700 1 --2- 172.21.15.157:0/1929947398 >> v2:172.21.15.175:3300/0 conn(0x7ff6f806e450 0x7ff6f806a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:52.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.133+0000 7ff6f6ffd700 1 -- 172.21.15.157:0/1929947398 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4089341439 0 0) 0x7ff6f80bed40 con 0x7ff6f8068d10 2023-10-06T03:57:52.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.133+0000 7ff6f6ffd700 1 -- 172.21.15.157:0/1929947398 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6e0003570 con 0x7ff6f8068d10 2023-10-06T03:57:52.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.133+0000 7ff6f7fff700 1 -- 172.21.15.157:0/1929947398 >> v2:172.21.15.175:3300/0 conn(0x7ff6f806e450 msgr2=0x7ff6f806a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:57:52.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.133+0000 7ff6f7fff700 1 --2- 172.21.15.157:0/1929947398 >> v2:172.21.15.175:3300/0 conn(0x7ff6f806e450 0x7ff6f806a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:52.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.133+0000 7ff6f7fff700 1 -- 172.21.15.157:0/1929947398 >> v1:172.21.15.157:6789/0 conn(0x7ff6f8068d10 legacy=0x7ff6f81530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:52.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.133+0000 7ff6f7fff700 1 -- 172.21.15.157:0/1929947398 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f81574a0 con 0x7ff6f80be970 2023-10-06T03:57:52.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.133+0000 7ff6f7fff700 1 --2- 172.21.15.157:0/1929947398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6f80be970 0x7ff6f806df10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80098e0 tx=0x7ff6e80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f34189c2da89afe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:52.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.133+0000 7ff6f6ffd700 1 -- 172.21.15.157:0/1929947398 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8011630 con 0x7ff6f80be970 2023-10-06T03:57:52.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.133+0000 7ff6f6ffd700 1 -- 172.21.15.157:0/1929947398 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e8011790 con 0x7ff6f80be970 2023-10-06T03:57:52.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.133+0000 7ff6f6ffd700 1 -- 172.21.15.157:0/1929947398 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8011b30 con 0x7ff6f80be970 2023-10-06T03:57:52.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.134+0000 7ff6fe8af700 1 -- 172.21.15.157:0/1929947398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6f80be970 msgr2=0x7ff6f806df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:52.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.134+0000 7ff6fe8af700 1 --2- 172.21.15.157:0/1929947398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6f80be970 0x7ff6f806df10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80098e0 tx=0x7ff6e80091a0 comp rx=0 tx=0).stop 2023-10-06T03:57:52.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.134+0000 7ff6fe8af700 1 -- 172.21.15.157:0/1929947398 shutdown_connections 2023-10-06T03:57:52.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.134+0000 7ff6fe8af700 1 --2- 172.21.15.157:0/1929947398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6f80be970 0x7ff6f806df10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:52.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.134+0000 7ff6fe8af700 1 --2- 172.21.15.157:0/1929947398 >> v2:172.21.15.175:3300/0 conn(0x7ff6f806e450 0x7ff6f806a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:52.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.134+0000 7ff6fe8af700 1 -- 172.21.15.157:0/1929947398 >> 172.21.15.157:0/1929947398 conn(0x7ff6f80c8d70 msgr2=0x7ff6f80c4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:52.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.134+0000 7ff6fe8af700 1 -- 172.21.15.157:0/1929947398 shutdown_connections 2023-10-06T03:57:52.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.134+0000 7ff6fe8af700 1 -- 172.21.15.157:0/1929947398 wait complete. 2023-10-06T03:57:52.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.135+0000 7ff6fe8af700 1 Processor -- start 2023-10-06T03:57:52.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.135+0000 7ff6fe8af700 1 -- start start 2023-10-06T03:57:52.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.135+0000 7ff6fe8af700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6f8068d10 0x7ff6f8166e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:52.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.135+0000 7ff6fe8af700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6f806e450 0x7ff6f8156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:52.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.135+0000 7ff6fe8af700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f80be970 0x7ff6f8148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:52.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.135+0000 7ff6fe8af700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f8150900 con 0x7ff6f8068d10 2023-10-06T03:57:52.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.136+0000 7ff6fe8af700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f8163e40 con 0x7ff6f80be970 2023-10-06T03:57:52.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.136+0000 7ff6fe8af700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f8150340 con 0x7ff6f806e450 2023-10-06T03:57:52.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.136+0000 7ff6f7fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6f8068d10 0x7ff6f8166e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:52.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.136+0000 7ff6f7fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6f8068d10 0x7ff6f8166e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45356/0 (socket says 172.21.15.157:45356) 2023-10-06T03:57:52.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.136+0000 7ff6f7fff700 1 -- 172.21.15.157:0/2568430752 learned_addr learned my addr 172.21.15.157:0/2568430752 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:57:52.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.136+0000 7ff6f77fe700 1 --2- 172.21.15.157:0/2568430752 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6f806e450 0x7ff6f8156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:52.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.136+0000 7ff6fce4c700 1 --2- 172.21.15.157:0/2568430752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f80be970 0x7ff6f8148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:52.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.136+0000 7ff6f7fff700 1 -- 172.21.15.157:0/2568430752 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6f806e450 msgr2=0x7ff6f8156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:52.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.136+0000 7ff6f7fff700 1 --2- 172.21.15.157:0/2568430752 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6f806e450 0x7ff6f8156080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:52.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.136+0000 7ff6f7fff700 1 -- 172.21.15.157:0/2568430752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f80be970 msgr2=0x7ff6f8148aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:52.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.136+0000 7ff6f7fff700 1 --2- 172.21.15.157:0/2568430752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f80be970 0x7ff6f8148aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:52.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.136+0000 7ff6f7fff700 1 -- 172.21.15.157:0/2568430752 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e8009420 con 0x7ff6f8068d10 2023-10-06T03:57:52.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.136+0000 7ff6f7fff700 1 --2- 172.21.15.157:0/2568430752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6f8068d10 0x7ff6f8166e70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff6f806fb30 tx=0x7ff6e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:52.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.136+0000 7ff6dffff700 1 -- 172.21.15.157:0/2568430752 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e801f070 con 0x7ff6f8068d10 2023-10-06T03:57:52.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.136+0000 7ff6dffff700 1 -- 172.21.15.157:0/2568430752 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e8000c90 con 0x7ff6f8068d10 2023-10-06T03:57:52.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.136+0000 7ff6fe8af700 1 -- 172.21.15.157:0/2568430752 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f8150620 con 0x7ff6f8068d10 2023-10-06T03:57:52.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.136+0000 7ff6dffff700 1 -- 172.21.15.157:0/2568430752 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8011b30 con 0x7ff6f8068d10 2023-10-06T03:57:52.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.136+0000 7ff6fe8af700 1 -- 172.21.15.157:0/2568430752 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f8166950 con 0x7ff6f8068d10 2023-10-06T03:57:52.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.137+0000 7ff6fe8af700 1 -- 172.21.15.157:0/2568430752 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f80716b0 con 0x7ff6f8068d10 2023-10-06T03:57:52.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.138+0000 7ff6dffff700 1 -- 172.21.15.157:0/2568430752 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff6e8023070 con 0x7ff6f8068d10 2023-10-06T03:57:52.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.138+0000 7ff6dffff700 1 --2- 172.21.15.157:0/2568430752 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff6e4066f60 0x7ff6e4069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:57:52.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.138+0000 7ff6dffff700 1 -- 172.21.15.157:0/2568430752 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff6e8085970 con 0x7ff6f8068d10 2023-10-06T03:57:52.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.138+0000 7ff6f77fe700 1 --2- 172.21.15.157:0/2568430752 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff6e4066f60 0x7ff6e4069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:57:52.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.139+0000 7ff6f77fe700 1 --2- 172.21.15.157:0/2568430752 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff6e4066f60 0x7ff6e4069410 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec0093f0 tx=0x7ff6ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:57:52.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.142+0000 7ff6dffff700 1 -- 172.21.15.157:0/2568430752 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6e8052b30 con 0x7ff6f8068d10 2023-10-06T03:57:52.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.327+0000 7ff6fe8af700 1 -- 172.21.15.157:0/2568430752 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6f80c9170 con 0x7ff6f8068d10 2023-10-06T03:57:52.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.328+0000 7ff6dffff700 1 -- 172.21.15.157:0/2568430752 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16914 (secure 0 0 0) 0x7ff6e802a070 con 0x7ff6f8068d10 2023-10-06T03:57:52.329 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:57:52.330 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":21,"fsid":"28664a7a-fe30-4efe-be5a-6abf9318b700","created":"2023-10-06T03:56:51.887538+0000","modified":"2023-10-06T03:57:12.362537+0000","last_up_change":"2023-10-06T03:57:02.299440+0000","last_in_change":"2023-10-06T03:56:55.141505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:57:03.455121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:57:03.905661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:57:08.377186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:57:09.709582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f677cac-e9f4-4608-acac-718b072ce984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":90640},{"type":"v1","addr":"172.21.15.157:6809","nonce":90640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":90640},{"type":"v1","addr":"172.21.15.157:6811","nonce":90640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":90640},{"type":"v1","addr":"172.21.15.157:6815","nonce":90640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":90640},{"type":"v1","addr":"172.21.15.157:6813","nonce":90640}]},"public_addr":"172.21.15.157:6809/90640","cluster_addr":"172.21.15.157:6811/90640","heartbeat_back_addr":"172.21.15.157:6815/90640","heartbeat_front_addr":"172.21.15.157:6813/90640","state":["exists","up"]},{"osd":1,"uuid":"7d771892-bb91-4e9b-9610-051861c11247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":90654},{"type":"v1","addr":"172.21.15.157:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":90654},{"type":"v1","addr":"172.21.15.157:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":90654},{"type":"v1","addr":"172.21.15.157:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":90654},{"type":"v1","addr":"172.21.15.157:6805","nonce":90654}]},"public_addr":"172.21.15.157:6801/90654","cluster_addr":"172.21.15.157:6803/90654","heartbeat_back_addr":"172.21.15.157:6807/90654","heartbeat_front_addr":"172.21.15.157:6805/90654","state":["exists","up"]},{"osd":2,"uuid":"5c1f7d4e-d185-47cf-ab6c-453b4e3009c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":90657},{"type":"v1","addr":"172.21.15.157:6825","nonce":90657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":90657},{"type":"v1","addr":"172.21.15.157:6827","nonce":90657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":90657},{"type":"v1","addr":"172.21.15.157:6831","nonce":90657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":90657},{"type":"v1","addr":"172.21.15.157:6829","nonce":90657}]},"public_addr":"172.21.15.157:6825/90657","cluster_addr":"172.21.15.157:6827/90657","heartbeat_back_addr":"172.21.15.157:6831/90657","heartbeat_front_addr":"172.21.15.157:6829/90657","state":["exists","up"]},{"osd":3,"uuid":"f6a1e825-43c7-4afd-8daf-e51d45afc75e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":90641},{"type":"v1","addr":"172.21.15.157:6817","nonce":90641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":90641},{"type":"v1","addr":"172.21.15.157:6819","nonce":90641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":90641},{"type":"v1","addr":"172.21.15.157:6823","nonce":90641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":90641},{"type":"v1","addr":"172.21.15.157:6821","nonce":90641}]},"public_addr":"172.21.15.157:6817/90641","cluster_addr":"172.21.15.157:6819/90641","heartbeat_back_addr":"172.21.15.157:6823/90641","heartbeat_front_addr":"172.21.15.157:6821/90641","state":["exists","up"]},{"osd":4,"uuid":"8ad6e3d8-9a31-4f85-9410-417b7ac5da97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":91261},{"type":"v1","addr":"172.21.15.175:6803","nonce":91261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":91261},{"type":"v1","addr":"172.21.15.175:6806","nonce":91261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":91261},{"type":"v1","addr":"172.21.15.175:6814","nonce":91261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":91261},{"type":"v1","addr":"172.21.15.175:6810","nonce":91261}]},"public_addr":"172.21.15.175:6803/91261","cluster_addr":"172.21.15.175:6806/91261","heartbeat_back_addr":"172.21.15.175:6814/91261","heartbeat_front_addr":"172.21.15.175:6810/91261","state":["exists","up"]},{"osd":5,"uuid":"f68aca2a-f182-4c00-8e21-241d09c02ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":91262},{"type":"v1","addr":"172.21.15.175:6817","nonce":91262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":91262},{"type":"v1","addr":"172.21.15.175:6819","nonce":91262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":91262},{"type":"v1","addr":"172.21.15.175:6823","nonce":91262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":91262},{"type":"v1","addr":"172.21.15.175:6821","nonce":91262}]},"public_addr":"172.21.15.175:6817/91262","cluster_addr":"172.21.15.175:6819/91262","heartbeat_back_addr":"172.21.15.175:6823/91262","heartbeat_front_addr":"172.21.15.175:6821/91262","state":["exists","up"]},{"osd":6,"uuid":"d867cd89-3430-40ca-862f-d2bea3de9d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":91263},{"type":"v1","addr":"172.21.15.175:6801","nonce":91263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6805","nonce":91263},{"type":"v1","addr":"172.21.15.175:6807","nonce":91263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6813","nonce":91263},{"type":"v1","addr":"172.21.15.175:6815","nonce":91263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":91263},{"type":"v1","addr":"172.21.15.175:6811","nonce":91263}]},"public_addr":"172.21.15.175:6801/91263","cluster_addr":"172.21.15.175:6807/91263","heartbeat_back_addr":"172.21.15.175:6815/91263","heartbeat_front_addr":"172.21.15.175:6811/91263","state":["exists","up"]},{"osd":7,"uuid":"930cd464-79f1-47cb-b4ff-188ccde74fff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":91306},{"type":"v1","addr":"172.21.15.175:6825","nonce":91306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":91306},{"type":"v1","addr":"172.21.15.175:6827","nonce":91306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":91306},{"type":"v1","addr":"172.21.15.175:6831","nonce":91306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":91306},{"type":"v1","addr":"172.21.15.175:6829","nonce":91306}]},"public_addr":"172.21.15.175:6825/91306","cluster_addr":"172.21.15.175:6827/91306","heartbeat_back_addr":"172.21.15.175:6831/91306","heartbeat_front_addr":"172.21.15.175:6829/91306","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.290673+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.056390+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.292903+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.275861+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.332095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.327453+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.317585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.309330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:57:52.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.330+0000 7ff6fe8af700 1 -- 172.21.15.157:0/2568430752 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff6e4066f60 msgr2=0x7ff6e4069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:52.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.330+0000 7ff6fe8af700 1 --2- 172.21.15.157:0/2568430752 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff6e4066f60 0x7ff6e4069410 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec0093f0 tx=0x7ff6ec008040 comp rx=0 tx=0).stop 2023-10-06T03:57:52.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.330+0000 7ff6fe8af700 1 -- 172.21.15.157:0/2568430752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6f8068d10 msgr2=0x7ff6f8166e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:57:52.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.330+0000 7ff6fe8af700 1 --2- 172.21.15.157:0/2568430752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6f8068d10 0x7ff6f8166e70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff6f806fb30 tx=0x7ff6e8017000 comp rx=0 tx=0).stop 2023-10-06T03:57:52.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.330+0000 7ff6fe8af700 1 -- 172.21.15.157:0/2568430752 shutdown_connections 2023-10-06T03:57:52.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.330+0000 7ff6fe8af700 1 --2- 172.21.15.157:0/2568430752 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff6e4066f60 0x7ff6e4069410 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:52.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.330+0000 7ff6fe8af700 1 --2- 172.21.15.157:0/2568430752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6f80be970 0x7ff6f8148aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:52.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.330+0000 7ff6fe8af700 1 --2- 172.21.15.157:0/2568430752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6f8068d10 0x7ff6f8166e70 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:52.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.330+0000 7ff6fe8af700 1 --2- 172.21.15.157:0/2568430752 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff6f806e450 0x7ff6f8156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:57:52.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.330+0000 7ff6fe8af700 1 -- 172.21.15.157:0/2568430752 >> 172.21.15.157:0/2568430752 conn(0x7ff6f80c8d70 msgr2=0x7ff6f80c4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:57:52.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.330+0000 7ff6fe8af700 1 -- 172.21.15.157:0/2568430752 shutdown_connections 2023-10-06T03:57:52.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:57:52.330+0000 7ff6fe8af700 1 -- 172.21.15.157:0/2568430752 wait complete. 2023-10-06T03:57:52.344 DEBUG:tasks.mds_thrash:Yielding 2023-10-06T03:57:52.344 INFO:teuthology.run_tasks:Running task workunit... 2023-10-06T03:57:52.357 INFO:tasks.workunit:Pulling workunits from ref 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T03:57:52.357 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-10-06T03:57:52.357 INFO:tasks.workunit:timeout=3h 2023-10-06T03:57:52.357 INFO:tasks.workunit:cleanup=True 2023-10-06T03:57:52.358 DEBUG:teuthology.orchestra.run.smithi157:> stat -- /home/ubuntu/cephtest/mnt.0 2023-10-06T03:57:52.358 INFO:tasks.mds_thrash.fs.[cephfs]:starting mds_do_thrash for fs cephfs 2023-10-06T03:57:52.358 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 59 secs before thrashing 2023-10-06T03:57:52.374 INFO:teuthology.orchestra.run.smithi157.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-10-06T03:57:52.374 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-10-06T03:57:52.374 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 2eh/46d Inode: 1 Links: 2 2023-10-06T03:57:52.374 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T03:57:52.374 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:cephfs_t:s0 2023-10-06T03:57:52.374 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2023-10-06T03:57:52.374 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2023-10-06 03:57:16.363784876 +0000 2023-10-06T03:57:52.374 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2023-10-06 03:57:45.212639838 +0000 2023-10-06T03:57:52.374 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2023-10-06T03:57:52.374 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-10-06T03:57:52.375 DEBUG:teuthology.orchestra.run.smithi157:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-10-06T03:57:52.443 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T03:57:52.498 INFO:tasks.workunit.client.0.smithi157.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-10-06T03:58:13.618 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:58:13.621 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:58:36.897 INFO:tasks.workunit.client.0.smithi157.stderr:Note: switching to '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'. 2023-10-06T03:58:36.897 INFO:tasks.workunit.client.0.smithi157.stderr: 2023-10-06T03:58:36.897 INFO:tasks.workunit.client.0.smithi157.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-10-06T03:58:36.897 INFO:tasks.workunit.client.0.smithi157.stderr:changes and commit them, and you can discard any commits you make in this 2023-10-06T03:58:36.897 INFO:tasks.workunit.client.0.smithi157.stderr:state without impacting any branches by switching back to a branch. 2023-10-06T03:58:36.897 INFO:tasks.workunit.client.0.smithi157.stderr: 2023-10-06T03:58:36.897 INFO:tasks.workunit.client.0.smithi157.stderr:If you want to create a new branch to retain commits you create, you may 2023-10-06T03:58:36.897 INFO:tasks.workunit.client.0.smithi157.stderr:do so (now or later) by using -c with the switch command. Example: 2023-10-06T03:58:36.898 INFO:tasks.workunit.client.0.smithi157.stderr: 2023-10-06T03:58:36.898 INFO:tasks.workunit.client.0.smithi157.stderr: git switch -c 2023-10-06T03:58:36.898 INFO:tasks.workunit.client.0.smithi157.stderr: 2023-10-06T03:58:36.898 INFO:tasks.workunit.client.0.smithi157.stderr:Or undo this operation with: 2023-10-06T03:58:36.898 INFO:tasks.workunit.client.0.smithi157.stderr: 2023-10-06T03:58:36.898 INFO:tasks.workunit.client.0.smithi157.stderr: git switch - 2023-10-06T03:58:36.898 INFO:tasks.workunit.client.0.smithi157.stderr: 2023-10-06T03:58:36.898 INFO:tasks.workunit.client.0.smithi157.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-10-06T03:58:36.898 INFO:tasks.workunit.client.0.smithi157.stderr: 2023-10-06T03:58:36.898 INFO:tasks.workunit.client.0.smithi157.stderr:HEAD is now at 68e53a4a7d4 Merge branch 'wip-ksirivad-quincy-backport-51923' of https://github.com/kamoltat/ceph into wip-yuri3-testing-2023-10-05-0943-quincy 2023-10-06T03:58:36.902 DEBUG:teuthology.orchestra.run.smithi157:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-10-06T03:58:36.930 INFO:tasks.workunit.client.0.smithi157.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-10-06T03:58:36.933 INFO:tasks.workunit.client.0.smithi157.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-06T03:58:36.933 INFO:tasks.workunit.client.0.smithi157.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-10-06T03:58:37.473 INFO:tasks.workunit.client.0.smithi157.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-10-06T03:58:37.532 INFO:tasks.workunit.client.0.smithi157.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-10-06T03:58:37.570 INFO:tasks.workunit.client.0.smithi157.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-06T03:58:37.571 INFO:tasks.workunit.client.0.smithi157.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-06T03:58:37.571 INFO:tasks.workunit.client.0.smithi157.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-10-06T03:58:37.608 INFO:tasks.workunit.client.0.smithi157.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-06T03:58:37.611 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-10-06T03:58:37.611 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-10-06T03:58:37.666 INFO:tasks.workunit:Running workunits matching fs/snaps on client.0... 2023-10-06T03:58:37.667 INFO:tasks.workunit:Running workunit fs/snaps/snap-rm-diff.sh... 2023-10-06T03:58:37.667 DEBUG:teuthology.orchestra.run.smithi157:workunit test fs/snaps/snap-rm-diff.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh 2023-10-06T03:58:37.727 INFO:tasks.workunit.client.0.smithi157.stderr:+ wget -q http://download.ceph.com/qa/linux-2.6.33.tar.bz2 2023-10-06T03:58:41.712 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir foo 2023-10-06T03:58:41.717 INFO:tasks.workunit.client.0.smithi157.stderr:+ cp linux-2.6.33.tar.bz2 foo 2023-10-06T03:58:41.756 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir foo/.snap/barsnap 2023-10-06T03:58:41.778 INFO:tasks.workunit.client.0.smithi157.stderr:+ rm foo/linux-2.6.33.tar.bz2 2023-10-06T03:58:42.291 INFO:tasks.workunit.client.0.smithi157.stderr:+ diff -q foo/.snap/barsnap/linux-2.6.33.tar.bz2 linux-2.6.33.tar.bz2 2023-10-06T03:58:42.490 INFO:tasks.workunit.client.0.smithi157.stderr:+ echo 'passed: files are identical' 2023-10-06T03:58:42.490 INFO:tasks.workunit.client.0.smithi157.stdout:passed: files are identical 2023-10-06T03:58:42.490 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir foo/.snap/barsnap 2023-10-06T03:58:42.499 INFO:tasks.workunit.client.0.smithi157.stdout:OK 2023-10-06T03:58:42.500 INFO:tasks.workunit.client.0.smithi157.stderr:+ echo OK 2023-10-06T03:58:42.500 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-06T03:58:42.501 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-06T03:58:43.687 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:58:43.690 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:58:43.790 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-1.sh... 2023-10-06T03:58:43.790 DEBUG:teuthology.orchestra.run.smithi157:workunit test fs/snaps/snaptest-1.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-1.sh 2023-10-06T03:58:45.572 INFO:tasks.workunit.client.0.smithi157.stderr:+ echo 1 2023-10-06T03:58:45.573 INFO:tasks.workunit.client.0.smithi157.stderr:+ echo 2 2023-10-06T03:58:45.573 INFO:tasks.workunit.client.0.smithi157.stderr:+ echo 3 2023-10-06T03:58:45.574 INFO:tasks.workunit.client.0.smithi157.stderr:+ '[' -e file4 ']' 2023-10-06T03:58:45.574 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir .snap/snap1 2023-10-06T03:58:45.599 INFO:tasks.workunit.client.0.smithi157.stderr:+ echo 4 2023-10-06T03:58:45.600 INFO:tasks.workunit.client.0.smithi157.stderr:++ ls 2023-10-06T03:58:45.605 INFO:tasks.workunit.client.0.smithi157.stderr:+ now='file1 2023-10-06T03:58:45.605 INFO:tasks.workunit.client.0.smithi157.stderr:file2 2023-10-06T03:58:45.605 INFO:tasks.workunit.client.0.smithi157.stderr:file3 2023-10-06T03:58:45.605 INFO:tasks.workunit.client.0.smithi157.stderr:file4' 2023-10-06T03:58:45.605 INFO:tasks.workunit.client.0.smithi157.stderr:++ ls .snap/snap1 2023-10-06T03:58:45.612 INFO:tasks.workunit.client.0.smithi157.stderr:+ then='file1 2023-10-06T03:58:45.612 INFO:tasks.workunit.client.0.smithi157.stderr:file2 2023-10-06T03:58:45.612 INFO:tasks.workunit.client.0.smithi157.stderr:file3' 2023-10-06T03:58:45.612 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir .snap/snap1 2023-10-06T03:58:45.618 INFO:tasks.workunit.client.0.smithi157.stderr:+ '[' 'file1 2023-10-06T03:58:45.618 INFO:tasks.workunit.client.0.smithi157.stderr:file2 2023-10-06T03:58:45.618 INFO:tasks.workunit.client.0.smithi157.stderr:file3 2023-10-06T03:58:45.618 INFO:tasks.workunit.client.0.smithi157.stderr:file4' = 'file1 2023-10-06T03:58:45.618 INFO:tasks.workunit.client.0.smithi157.stderr:file2 2023-10-06T03:58:45.618 INFO:tasks.workunit.client.0.smithi157.stderr:file3' ']' 2023-10-06T03:58:45.618 INFO:tasks.workunit.client.0.smithi157.stderr:+ echo 1 2023-10-06T03:58:45.623 INFO:tasks.workunit.client.0.smithi157.stderr:+ echo 2 2023-10-06T03:58:45.629 INFO:tasks.workunit.client.0.smithi157.stderr:+ echo 3 2023-10-06T03:58:45.636 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir .snap/snap1 2023-10-06T03:58:45.657 INFO:tasks.workunit.client.0.smithi157.stderr:+ echo 4 2023-10-06T03:58:45.663 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir .snap/snap1 2023-10-06T03:58:46.168 INFO:tasks.workunit.client.0.smithi157.stderr:+ rm file1 file2 file3 file4 2023-10-06T03:58:46.175 INFO:tasks.workunit.client.0.smithi157.stdout:OK 2023-10-06T03:58:46.176 INFO:tasks.workunit.client.0.smithi157.stderr:+ echo OK 2023-10-06T03:58:46.176 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-06T03:58:46.176 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-06T03:58:46.215 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-2.sh... 2023-10-06T03:58:46.215 DEBUG:teuthology.orchestra.run.smithi157:workunit test fs/snaps/snaptest-2.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-2.sh 2023-10-06T03:58:46.422 INFO:tasks.workunit.client.0.smithi157.stdout:Create dir 100 to 199 ... 2023-10-06T03:58:46.430 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 100 2023-10-06T03:58:46.440 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 101 2023-10-06T03:58:46.450 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 102 2023-10-06T03:58:46.460 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 103 2023-10-06T03:58:46.469 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 104 2023-10-06T03:58:46.478 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 105 2023-10-06T03:58:46.675 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 106 2023-10-06T03:58:46.685 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 107 2023-10-06T03:58:46.694 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 108 2023-10-06T03:58:46.704 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 109 2023-10-06T03:58:46.713 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 110 2023-10-06T03:58:46.722 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 111 2023-10-06T03:58:46.730 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 112 2023-10-06T03:58:46.739 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 113 2023-10-06T03:58:46.747 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 114 2023-10-06T03:58:46.756 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 115 2023-10-06T03:58:46.770 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 116 2023-10-06T03:58:46.793 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 117 2023-10-06T03:58:47.068 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 118 2023-10-06T03:58:47.076 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 119 2023-10-06T03:58:47.085 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 120 2023-10-06T03:58:47.094 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 121 2023-10-06T03:58:47.366 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 122 2023-10-06T03:58:47.376 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 123 2023-10-06T03:58:47.384 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 124 2023-10-06T03:58:47.393 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 125 2023-10-06T03:58:47.402 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 126 2023-10-06T03:58:47.818 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 127 2023-10-06T03:58:47.827 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 128 2023-10-06T03:58:47.836 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 129 2023-10-06T03:58:47.844 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 130 2023-10-06T03:58:47.853 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 131 2023-10-06T03:58:47.862 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 132 2023-10-06T03:58:48.302 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 133 2023-10-06T03:58:48.311 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 134 2023-10-06T03:58:48.332 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 135 2023-10-06T03:58:48.341 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 136 2023-10-06T03:58:48.350 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 137 2023-10-06T03:58:48.358 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 138 2023-10-06T03:58:48.367 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 139 2023-10-06T03:58:48.375 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 140 2023-10-06T03:58:48.384 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 141 2023-10-06T03:58:48.392 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 142 2023-10-06T03:58:48.401 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 143 2023-10-06T03:58:49.225 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 144 2023-10-06T03:58:49.236 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 145 2023-10-06T03:58:49.246 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 146 2023-10-06T03:58:49.256 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 147 2023-10-06T03:58:49.266 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 148 2023-10-06T03:58:49.275 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 149 2023-10-06T03:58:49.285 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 150 2023-10-06T03:58:49.294 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 151 2023-10-06T03:58:49.303 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 152 2023-10-06T03:58:49.312 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 153 2023-10-06T03:58:49.321 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 154 2023-10-06T03:58:49.330 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 155 2023-10-06T03:58:49.339 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 156 2023-10-06T03:58:49.348 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 157 2023-10-06T03:58:49.357 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 158 2023-10-06T03:58:49.705 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 159 2023-10-06T03:58:49.714 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 160 2023-10-06T03:58:49.723 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 161 2023-10-06T03:58:49.732 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 162 2023-10-06T03:58:49.740 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 163 2023-10-06T03:58:50.169 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 164 2023-10-06T03:58:50.539 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 165 2023-10-06T03:58:50.550 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 166 2023-10-06T03:58:50.561 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 167 2023-10-06T03:58:50.571 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 168 2023-10-06T03:58:50.581 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 169 2023-10-06T03:58:50.591 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 170 2023-10-06T03:58:50.600 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 171 2023-10-06T03:58:50.610 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 172 2023-10-06T03:58:50.619 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 173 2023-10-06T03:58:50.628 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 174 2023-10-06T03:58:50.637 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 175 2023-10-06T03:58:50.646 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 176 2023-10-06T03:58:50.655 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 177 2023-10-06T03:58:50.664 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 178 2023-10-06T03:58:50.673 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 179 2023-10-06T03:58:51.198 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 180 2023-10-06T03:58:51.208 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 181 2023-10-06T03:58:51.217 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 182 2023-10-06T03:58:51.359 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:58:51.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.480+0000 7fae3ed1f700 1 Processor -- start 2023-10-06T03:58:51.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.480+0000 7fae3ed1f700 1 -- start start 2023-10-06T03:58:51.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.481+0000 7fae3ed1f700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fae380b04d0 0x7fae38157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:51.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.481+0000 7fae3ed1f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fae3806c2d0 0x7fae3806dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:51.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.481+0000 7fae3ed1f700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae3806d8d0 con 0x7fae380b1e40 2023-10-06T03:58:51.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.481+0000 7fae3ed1f700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae380af8f0 con 0x7fae380b04d0 2023-10-06T03:58:51.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.481+0000 7fae3ed1f700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae3815a690 con 0x7fae3806c2d0 2023-10-06T03:58:51.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.481+0000 7fae3dd1d700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fae380b1e40 0x7fae381563b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:36706/0 (socket says 172.21.15.157:36706) 2023-10-06T03:58:51.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.481+0000 7fae3dd1d700 1 -- 172.21.15.157:0/2051435876 learned_addr learned my addr 172.21.15.157:0/2051435876 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:58:51.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.481+0000 7fae3d51c700 1 --2- 172.21.15.157:0/2051435876 >> v2:172.21.15.157:3300/0 conn(0x7fae380b04d0 0x7fae38157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:51.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.481+0000 7fae3e51e700 1 --2- 172.21.15.157:0/2051435876 >> v2:172.21.15.175:3300/0 conn(0x7fae3806c2d0 0x7fae3806dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:51.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.481+0000 7fae3cd1b700 1 -- 172.21.15.157:0/2051435876 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2913600342 0 0) 0x7fae3806d8d0 con 0x7fae380b1e40 2023-10-06T03:58:51.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.481+0000 7fae3cd1b700 1 -- 172.21.15.157:0/2051435876 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae24003570 con 0x7fae380b1e40 2023-10-06T03:58:51.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.481+0000 7fae3d51c700 1 -- 172.21.15.157:0/2051435876 >> v2:172.21.15.175:3300/0 conn(0x7fae3806c2d0 msgr2=0x7fae3806dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:58:51.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.481+0000 7fae3d51c700 1 --2- 172.21.15.157:0/2051435876 >> v2:172.21.15.175:3300/0 conn(0x7fae3806c2d0 0x7fae3806dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:51.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.481+0000 7fae3d51c700 1 -- 172.21.15.157:0/2051435876 >> v1:172.21.15.157:6789/0 conn(0x7fae380b1e40 legacy=0x7fae381563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:58:51.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.481+0000 7fae3d51c700 1 -- 172.21.15.157:0/2051435876 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3806ccc0 con 0x7fae380b04d0 2023-10-06T03:58:51.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.482+0000 7fae3d51c700 1 --2- 172.21.15.157:0/2051435876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae380b04d0 0x7fae38157750 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fae380c34e0 tx=0x7fae280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f729e8c6e43eee30 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:58:51.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.482+0000 7fae3cd1b700 1 -- 172.21.15.157:0/2051435876 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae28011630 con 0x7fae380b04d0 2023-10-06T03:58:51.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.482+0000 7fae3cd1b700 1 -- 172.21.15.157:0/2051435876 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae28011790 con 0x7fae380b04d0 2023-10-06T03:58:51.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.482+0000 7fae3cd1b700 1 -- 172.21.15.157:0/2051435876 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae28011b30 con 0x7fae380b04d0 2023-10-06T03:58:51.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.482+0000 7fae3ed1f700 1 -- 172.21.15.157:0/2051435876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae380b04d0 msgr2=0x7fae38157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:58:51.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.482+0000 7fae3ed1f700 1 --2- 172.21.15.157:0/2051435876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae380b04d0 0x7fae38157750 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fae380c34e0 tx=0x7fae280091a0 comp rx=0 tx=0).stop 2023-10-06T03:58:51.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.482+0000 7fae3ed1f700 1 -- 172.21.15.157:0/2051435876 shutdown_connections 2023-10-06T03:58:51.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.482+0000 7fae3ed1f700 1 --2- 172.21.15.157:0/2051435876 >> v2:172.21.15.175:3300/0 conn(0x7fae3806c2d0 0x7fae3806dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:51.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.482+0000 7fae3ed1f700 1 --2- 172.21.15.157:0/2051435876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae380b04d0 0x7fae38157750 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:51.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.482+0000 7fae3ed1f700 1 -- 172.21.15.157:0/2051435876 >> 172.21.15.157:0/2051435876 conn(0x7fae380cc540 msgr2=0x7fae380cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:58:51.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.482+0000 7fae3ed1f700 1 -- 172.21.15.157:0/2051435876 shutdown_connections 2023-10-06T03:58:51.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.482+0000 7fae3ed1f700 1 -- 172.21.15.157:0/2051435876 wait complete. 2023-10-06T03:58:51.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.484+0000 7fae3ed1f700 1 Processor -- start 2023-10-06T03:58:51.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.484+0000 7fae3ed1f700 1 -- start start 2023-10-06T03:58:51.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.484+0000 7fae3ed1f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae3806c2d0 0x7fae38166830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:51.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.484+0000 7fae3ed1f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae380b04d0 0x7fae38156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:51.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.484+0000 7fae3ed1f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae380b1e40 0x7fae38169b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:51.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.484+0000 7fae3ed1f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae38150110 con 0x7fae380b1e40 2023-10-06T03:58:51.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.484+0000 7fae3ed1f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae381663a0 con 0x7fae380b04d0 2023-10-06T03:58:51.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.484+0000 7fae3ed1f700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae380cb960 con 0x7fae3806c2d0 2023-10-06T03:58:51.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.484+0000 7fae3e51e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae380b1e40 0x7fae38169b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:51.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.484+0000 7fae3e51e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae380b1e40 0x7fae38169b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52084/0 (socket says 172.21.15.157:52084) 2023-10-06T03:58:51.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.484+0000 7fae3e51e700 1 -- 172.21.15.157:0/1531908329 learned_addr learned my addr 172.21.15.157:0/1531908329 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:58:51.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.485+0000 7fae3d51c700 1 --2- 172.21.15.157:0/1531908329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae380b04d0 0x7fae38156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:51.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.485+0000 7fae3dd1d700 1 --2- 172.21.15.157:0/1531908329 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae3806c2d0 0x7fae38166830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:51.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.485+0000 7fae3e51e700 1 -- 172.21.15.157:0/1531908329 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae3806c2d0 msgr2=0x7fae38166830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:58:51.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.485+0000 7fae3e51e700 1 --2- 172.21.15.157:0/1531908329 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae3806c2d0 0x7fae38166830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:51.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.485+0000 7fae3e51e700 1 -- 172.21.15.157:0/1531908329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae380b04d0 msgr2=0x7fae38156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:58:51.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.485+0000 7fae3e51e700 1 --2- 172.21.15.157:0/1531908329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae380b04d0 0x7fae38156660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:51.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.485+0000 7fae3e51e700 1 -- 172.21.15.157:0/1531908329 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae28009420 con 0x7fae380b1e40 2023-10-06T03:58:51.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.485+0000 7fae3d51c700 1 --2- 172.21.15.157:0/1531908329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae380b04d0 0x7fae38156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:58:51.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.485+0000 7fae3e51e700 1 --2- 172.21.15.157:0/1531908329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae380b1e40 0x7fae38169b50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fae30007f50 tx=0x7fae30009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:58:51.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.485+0000 7fae2dffb700 1 -- 172.21.15.157:0/1531908329 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3000ed60 con 0x7fae380b1e40 2023-10-06T03:58:51.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.485+0000 7fae2dffb700 1 -- 172.21.15.157:0/1531908329 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae30014420 con 0x7fae380b1e40 2023-10-06T03:58:51.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.485+0000 7fae3ed1f700 1 -- 172.21.15.157:0/1531908329 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae380ca320 con 0x7fae380b1e40 2023-10-06T03:58:51.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.485+0000 7fae2dffb700 1 -- 172.21.15.157:0/1531908329 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3000eee0 con 0x7fae380b1e40 2023-10-06T03:58:51.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.485+0000 7fae3ed1f700 1 -- 172.21.15.157:0/1531908329 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae380ca8e0 con 0x7fae380b1e40 2023-10-06T03:58:51.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.486+0000 7fae3ed1f700 1 -- 172.21.15.157:0/1531908329 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae38088f90 con 0x7fae380b1e40 2023-10-06T03:58:51.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.486+0000 7fae2dffb700 1 -- 172.21.15.157:0/1531908329 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fae300149f0 con 0x7fae380b1e40 2023-10-06T03:58:51.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.487+0000 7fae2dffb700 1 --2- 172.21.15.157:0/1531908329 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fae1c066f30 0x7fae1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:51.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.487+0000 7fae2dffb700 1 -- 172.21.15.157:0/1531908329 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6172+0+0 (secure 0 0 0) 0x7fae30084ed0 con 0x7fae380b1e40 2023-10-06T03:58:51.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.487+0000 7fae3dd1d700 1 --2- 172.21.15.157:0/1531908329 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fae1c066f30 0x7fae1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:51.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.487+0000 7fae3dd1d700 1 --2- 172.21.15.157:0/1531908329 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fae1c066f30 0x7fae1c0693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fae340058d0 tx=0x7fae34009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:58:51.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.491+0000 7fae2dffb700 1 -- 172.21.15.157:0/1531908329 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae30051f70 con 0x7fae380b1e40 2023-10-06T03:58:51.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.726+0000 7fae3ed1f700 1 -- 172.21.15.157:0/1531908329 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae3806cfa0 con 0x7fae380b1e40 2023-10-06T03:58:51.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.726+0000 7fae2dffb700 1 -- 172.21.15.157:0/1531908329 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+6184 (secure 0 0 0) 0x7fae30055590 con 0x7fae380b1e40 2023-10-06T03:58:51.728 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:58:51.728 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T03:57:37.649946+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"f","rank":4,"incarnation":15,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/4177493830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4177493830},{"type":"v1","addr":"172.21.15.175:6839","nonce":4177493830}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:58:51.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.728+0000 7fae3ed1f700 1 -- 172.21.15.157:0/1531908329 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fae1c066f30 msgr2=0x7fae1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:58:51.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.728+0000 7fae3ed1f700 1 --2- 172.21.15.157:0/1531908329 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fae1c066f30 0x7fae1c0693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fae340058d0 tx=0x7fae34009040 comp rx=0 tx=0).stop 2023-10-06T03:58:51.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.728+0000 7fae3ed1f700 1 -- 172.21.15.157:0/1531908329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae380b1e40 msgr2=0x7fae38169b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:58:51.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.728+0000 7fae3ed1f700 1 --2- 172.21.15.157:0/1531908329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae380b1e40 0x7fae38169b50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fae30007f50 tx=0x7fae30009860 comp rx=0 tx=0).stop 2023-10-06T03:58:51.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.728+0000 7fae3ed1f700 1 -- 172.21.15.157:0/1531908329 shutdown_connections 2023-10-06T03:58:51.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.729+0000 7fae3ed1f700 1 --2- 172.21.15.157:0/1531908329 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fae1c066f30 0x7fae1c0693e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:51.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.729+0000 7fae3ed1f700 1 --2- 172.21.15.157:0/1531908329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae380b1e40 0x7fae38169b50 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:51.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.729+0000 7fae3ed1f700 1 --2- 172.21.15.157:0/1531908329 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae3806c2d0 0x7fae38166830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:51.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.729+0000 7fae3ed1f700 1 --2- 172.21.15.157:0/1531908329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae380b04d0 0x7fae38156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:51.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.729+0000 7fae3ed1f700 1 -- 172.21.15.157:0/1531908329 >> 172.21.15.157:0/1531908329 conn(0x7fae380cc540 msgr2=0x7fae381572c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:58:51.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.729+0000 7fae3ed1f700 1 -- 172.21.15.157:0/1531908329 shutdown_connections 2023-10-06T03:58:51.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.729+0000 7fae3ed1f700 1 -- 172.21.15.157:0/1531908329 wait complete. 2023-10-06T03:58:51.732 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 16 2023-10-06T03:58:51.743 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.f (rank=4) 2023-10-06T03:58:51.743 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2023-10-06T03:58:51.743 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:58:51.754 INFO:tasks.ceph.mds.f:Stopped 2023-10-06T03:58:51.754 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:58:51.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.867+0000 7fd99c95a700 1 Processor -- start 2023-10-06T03:58:51.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.867+0000 7fd99c95a700 1 -- start start 2023-10-06T03:58:51.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.868+0000 7fd99c95a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9980c94c0 0x7fd99806a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:51.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.868+0000 7fd99c95a700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd9980c9890 0x7fd9980c5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:51.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.868+0000 7fd99c95a700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd99806c1f0 0x7fd99814fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:51.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.868+0000 7fd99c95a700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd998156a80 con 0x7fd9980c9890 2023-10-06T03:58:51.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.868+0000 7fd99c95a700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd998156190 con 0x7fd99806c1f0 2023-10-06T03:58:51.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.868+0000 7fd99c95a700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99806ac50 con 0x7fd9980c94c0 2023-10-06T03:58:51.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.868+0000 7fd99659c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd9980c9890 0x7fd9980c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:51.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.868+0000 7fd99659c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd9980c9890 0x7fd9980c5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52086/0 (socket says 172.21.15.157:52086) 2023-10-06T03:58:51.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.868+0000 7fd99659c700 1 -- 172.21.15.157:0/1831329660 learned_addr learned my addr 172.21.15.157:0/1831329660 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:58:51.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.868+0000 7fd996d9d700 1 --2- 172.21.15.157:0/1831329660 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9980c94c0 0x7fd99806a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:51.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.868+0000 7fd99759e700 1 --2- 172.21.15.157:0/1831329660 >> v2:172.21.15.175:3300/0 conn(0x7fd99806c1f0 0x7fd99814fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:51.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.868+0000 7fd99659c700 1 -- 172.21.15.157:0/1831329660 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9980c94c0 msgr2=0x7fd99806a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:58:51.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.868+0000 7fd99659c700 1 --2- 172.21.15.157:0/1831329660 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9980c94c0 0x7fd99806a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:51.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.868+0000 7fd99659c700 1 -- 172.21.15.157:0/1831329660 >> v2:172.21.15.175:3300/0 conn(0x7fd99806c1f0 msgr2=0x7fd99814fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:58:51.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.868+0000 7fd99659c700 1 --2- 172.21.15.157:0/1831329660 >> v2:172.21.15.175:3300/0 conn(0x7fd99806c1f0 0x7fd99814fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:51.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.868+0000 7fd99659c700 1 -- 172.21.15.157:0/1831329660 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd998156430 con 0x7fd9980c9890 2023-10-06T03:58:51.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.868+0000 7fd99759e700 1 --2- 172.21.15.157:0/1831329660 >> v2:172.21.15.175:3300/0 conn(0x7fd99806c1f0 0x7fd99814fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:58:51.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.868+0000 7fd996d9d700 1 --2- 172.21.15.157:0/1831329660 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9980c94c0 0x7fd99806a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:58:51.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.868+0000 7fd99659c700 1 --2- 172.21.15.157:0/1831329660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9980c9890 0x7fd9980c5180 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd998066c90 tx=0x7fd980009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfab3bb4070f7179 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:58:51.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.868+0000 7fd995d9b700 1 -- 172.21.15.157:0/1831329660 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98000b8f0 con 0x7fd9980c9890 2023-10-06T03:58:51.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.868+0000 7fd995d9b700 1 -- 172.21.15.157:0/1831329660 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98000ba50 con 0x7fd9980c9890 2023-10-06T03:58:51.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.868+0000 7fd995d9b700 1 -- 172.21.15.157:0/1831329660 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98000bdf0 con 0x7fd9980c9890 2023-10-06T03:58:51.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.869+0000 7fd99c95a700 1 -- 172.21.15.157:0/1831329660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9980c9890 msgr2=0x7fd9980c5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:58:51.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.869+0000 7fd99c95a700 1 --2- 172.21.15.157:0/1831329660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9980c9890 0x7fd9980c5180 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd998066c90 tx=0x7fd980009250 comp rx=0 tx=0).stop 2023-10-06T03:58:51.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.869+0000 7fd99c95a700 1 -- 172.21.15.157:0/1831329660 shutdown_connections 2023-10-06T03:58:51.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.869+0000 7fd99c95a700 1 --2- 172.21.15.157:0/1831329660 >> v2:172.21.15.175:3300/0 conn(0x7fd99806c1f0 0x7fd99814fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:51.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.869+0000 7fd99c95a700 1 --2- 172.21.15.157:0/1831329660 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9980c94c0 0x7fd99806a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:51.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.869+0000 7fd99c95a700 1 --2- 172.21.15.157:0/1831329660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9980c9890 0x7fd9980c5180 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:51.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.869+0000 7fd99c95a700 1 -- 172.21.15.157:0/1831329660 >> 172.21.15.157:0/1831329660 conn(0x7fd998068020 msgr2=0x7fd998068420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:58:51.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.869+0000 7fd99c95a700 1 -- 172.21.15.157:0/1831329660 shutdown_connections 2023-10-06T03:58:51.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.869+0000 7fd99c95a700 1 -- 172.21.15.157:0/1831329660 wait complete. 2023-10-06T03:58:51.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.870+0000 7fd99c95a700 1 Processor -- start 2023-10-06T03:58:51.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.870+0000 7fd99c95a700 1 -- start start 2023-10-06T03:58:51.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.871+0000 7fd99c95a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd99806c1f0 0x7fd998150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:51.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.871+0000 7fd99c95a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9980c94c0 0x7fd99816a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:51.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.871+0000 7fd99c95a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9980c9890 0x7fd9980c1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:51.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.871+0000 7fd99c95a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd998159410 con 0x7fd9980c94c0 2023-10-06T03:58:51.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.871+0000 7fd99c95a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9980c2200 con 0x7fd99806c1f0 2023-10-06T03:58:51.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.871+0000 7fd99c95a700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd998158e50 con 0x7fd9980c9890 2023-10-06T03:58:51.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.871+0000 7fd99659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9980c94c0 0x7fd99816a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:51.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.871+0000 7fd99659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9980c94c0 0x7fd99816a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52088/0 (socket says 172.21.15.157:52088) 2023-10-06T03:58:51.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.871+0000 7fd99659c700 1 -- 172.21.15.157:0/2745364603 learned_addr learned my addr 172.21.15.157:0/2745364603 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:58:51.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.871+0000 7fd996d9d700 1 --2- 172.21.15.157:0/2745364603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd99806c1f0 0x7fd998150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:51.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.871+0000 7fd99759e700 1 --2- 172.21.15.157:0/2745364603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9980c9890 0x7fd9980c1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:51.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.871+0000 7fd99659c700 1 -- 172.21.15.157:0/2745364603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9980c9890 msgr2=0x7fd9980c1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:58:51.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.871+0000 7fd99659c700 1 --2- 172.21.15.157:0/2745364603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9980c9890 0x7fd9980c1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:51.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.871+0000 7fd99659c700 1 -- 172.21.15.157:0/2745364603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd99806c1f0 msgr2=0x7fd998150640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:58:51.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.871+0000 7fd99659c700 1 --2- 172.21.15.157:0/2745364603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd99806c1f0 0x7fd998150640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:51.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.871+0000 7fd99659c700 1 -- 172.21.15.157:0/2745364603 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9800094f0 con 0x7fd9980c94c0 2023-10-06T03:58:51.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.871+0000 7fd99759e700 1 --2- 172.21.15.157:0/2745364603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9980c9890 0x7fd9980c1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:58:51.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.871+0000 7fd99659c700 1 --2- 172.21.15.157:0/2745364603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9980c94c0 0x7fd99816a050 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd9800090b0 tx=0x7fd980017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:58:51.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.871+0000 7fd98effd700 1 -- 172.21.15.157:0/2745364603 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd980024440 con 0x7fd9980c94c0 2023-10-06T03:58:51.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.872+0000 7fd98effd700 1 -- 172.21.15.157:0/2745364603 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9800245e0 con 0x7fd9980c94c0 2023-10-06T03:58:51.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.872+0000 7fd99c95a700 1 -- 172.21.15.157:0/2745364603 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd998159130 con 0x7fd9980c94c0 2023-10-06T03:58:51.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.872+0000 7fd98effd700 1 -- 172.21.15.157:0/2745364603 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9800249a0 con 0x7fd9980c94c0 2023-10-06T03:58:51.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.872+0000 7fd99c95a700 1 -- 172.21.15.157:0/2745364603 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9981664c0 con 0x7fd9980c94c0 2023-10-06T03:58:51.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.873+0000 7fd99c95a700 1 -- 172.21.15.157:0/2745364603 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd998166040 con 0x7fd9980c94c0 2023-10-06T03:58:51.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.873+0000 7fd98effd700 1 -- 172.21.15.157:0/2745364603 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd98001a420 con 0x7fd9980c94c0 2023-10-06T03:58:51.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.873+0000 7fd98effd700 1 --2- 172.21.15.157:0/2745364603 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd978066f30 0x7fd9780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:51.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.873+0000 7fd98effd700 1 -- 172.21.15.157:0/2745364603 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fd980086720 con 0x7fd9980c94c0 2023-10-06T03:58:51.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.873+0000 7fd996d9d700 1 --2- 172.21.15.157:0/2745364603 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd978066f30 0x7fd9780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:51.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.874+0000 7fd996d9d700 1 --2- 172.21.15.157:0/2745364603 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd978066f30 0x7fd9780693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd990009fd0 tx=0x7fd990009960 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:58:51.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:51.878+0000 7fd98effd700 1 -- 172.21.15.157:0/2745364603 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd998166040 con 0x7fd9980c94c0 2023-10-06T03:58:52.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:52.101+0000 7fd99c95a700 1 -- 172.21.15.157:0/2745364603 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd998166040 con 0x7fd9980c94c0 2023-10-06T03:58:52.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:52.102+0000 7fd98effd700 1 -- 172.21.15.157:0/2745364603 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+6184 (secure 0 0 0) 0x7fd998166040 con 0x7fd9980c94c0 2023-10-06T03:58:52.105 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:58:52.106 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T03:57:37.649946+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"f","rank":4,"incarnation":15,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/4177493830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4177493830},{"type":"v1","addr":"172.21.15.175:6839","nonce":4177493830}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:58:52.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:52.104+0000 7fd99c95a700 1 -- 172.21.15.157:0/2745364603 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd978066f30 msgr2=0x7fd9780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:58:52.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:52.104+0000 7fd99c95a700 1 --2- 172.21.15.157:0/2745364603 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd978066f30 0x7fd9780693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd990009fd0 tx=0x7fd990009960 comp rx=0 tx=0).stop 2023-10-06T03:58:52.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:52.104+0000 7fd99c95a700 1 -- 172.21.15.157:0/2745364603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9980c94c0 msgr2=0x7fd99816a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:58:52.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:52.104+0000 7fd99c95a700 1 --2- 172.21.15.157:0/2745364603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9980c94c0 0x7fd99816a050 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd9800090b0 tx=0x7fd980017000 comp rx=0 tx=0).stop 2023-10-06T03:58:52.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:52.104+0000 7fd99c95a700 1 -- 172.21.15.157:0/2745364603 shutdown_connections 2023-10-06T03:58:52.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:52.104+0000 7fd99c95a700 1 --2- 172.21.15.157:0/2745364603 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd978066f30 0x7fd9780693e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:52.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:52.104+0000 7fd99c95a700 1 --2- 172.21.15.157:0/2745364603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd99806c1f0 0x7fd998150640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:52.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:52.104+0000 7fd99c95a700 1 --2- 172.21.15.157:0/2745364603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9980c9890 0x7fd9980c1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:52.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:52.104+0000 7fd99c95a700 1 --2- 172.21.15.157:0/2745364603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9980c94c0 0x7fd99816a050 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:52.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:52.104+0000 7fd99c95a700 1 -- 172.21.15.157:0/2745364603 >> 172.21.15.157:0/2745364603 conn(0x7fd998068020 msgr2=0x7fd99814ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:58:52.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:52.104+0000 7fd99c95a700 1 -- 172.21.15.157:0/2745364603 shutdown_connections 2023-10-06T03:58:52.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:52.104+0000 7fd99c95a700 1 -- 172.21.15.157:0/2745364603 wait complete. 2023-10-06T03:58:52.112 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 16 2023-10-06T03:58:52.118 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-10-06T03:58:52.226 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 183 2023-10-06T03:58:53.182 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 184 2023-10-06T03:58:53.192 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 185 2023-10-06T03:58:53.201 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 186 2023-10-06T03:58:53.211 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 187 2023-10-06T03:58:53.220 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 188 2023-10-06T03:58:53.230 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 189 2023-10-06T03:58:53.238 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 190 2023-10-06T03:58:53.247 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 191 2023-10-06T03:58:53.260 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 192 2023-10-06T03:58:53.272 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 193 2023-10-06T03:58:53.285 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 194 2023-10-06T03:58:53.298 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 195 2023-10-06T03:58:53.310 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 196 2023-10-06T03:58:53.322 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 197 2023-10-06T03:58:53.338 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 198 2023-10-06T03:58:53.347 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 199 2023-10-06T03:58:53.360 INFO:tasks.workunit.client.0.smithi157.stdout:Take first snapshot .snap/test1 2023-10-06T03:58:54.119 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:58:54.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.233+0000 7f2f18167700 1 Processor -- start 2023-10-06T03:58:54.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.233+0000 7f2f18167700 1 -- start start 2023-10-06T03:58:54.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.234+0000 7f2f18167700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2f1006cbb0 0x7f2f1006d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:54.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.234+0000 7f2f18167700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2f1006c1f0 0x7f2f1006da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:54.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.234+0000 7f2f18167700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f1006cf80 0x7f2f1014fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:54.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.234+0000 7f2f18167700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f100c8c40 con 0x7f2f1006cbb0 2023-10-06T03:58:54.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.234+0000 7f2f18167700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f100c9780 con 0x7f2f1006c1f0 2023-10-06T03:58:54.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.234+0000 7f2f18167700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1006e0c0 con 0x7f2f1006cf80 2023-10-06T03:58:54.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.234+0000 7f2f15f03700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2f1006cbb0 0x7f2f1006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:54.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.234+0000 7f2f15f03700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2f1006cbb0 0x7f2f1006d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56598/0 (socket says 172.21.15.157:56598) 2023-10-06T03:58:54.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.234+0000 7f2f15f03700 1 -- 172.21.15.157:0/357717323 learned_addr learned my addr 172.21.15.157:0/357717323 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:58:54.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.234+0000 7f2f15702700 1 --2- 172.21.15.157:0/357717323 >> v2:172.21.15.175:3300/0 conn(0x7f2f1006c1f0 0x7f2f1006da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:54.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.234+0000 7f2f16704700 1 --2- 172.21.15.157:0/357717323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f1006cf80 0x7f2f1014fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:54.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.234+0000 7f2f15f03700 1 -- 172.21.15.157:0/357717323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f1006cf80 msgr2=0x7f2f1014fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:58:54.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.234+0000 7f2f15f03700 1 --2- 172.21.15.157:0/357717323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f1006cf80 0x7f2f1014fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:54.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.234+0000 7f2f15f03700 1 -- 172.21.15.157:0/357717323 >> v2:172.21.15.175:3300/0 conn(0x7f2f1006c1f0 msgr2=0x7f2f1006da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:58:54.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.234+0000 7f2f15f03700 1 --2- 172.21.15.157:0/357717323 >> v2:172.21.15.175:3300/0 conn(0x7f2f1006c1f0 0x7f2f1006da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:54.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.234+0000 7f2f15f03700 1 -- 172.21.15.157:0/357717323 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f100c8e50 con 0x7f2f1006cbb0 2023-10-06T03:58:54.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.234+0000 7f2f15f03700 1 --2- 172.21.15.157:0/357717323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f1006cbb0 0x7f2f1006d530 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2f10157070 tx=0x7f2f0c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d89ff07f7650320c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:58:54.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.234+0000 7f2f14f01700 1 -- 172.21.15.157:0/357717323 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c00e070 con 0x7f2f1006cbb0 2023-10-06T03:58:54.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.234+0000 7f2f14f01700 1 -- 172.21.15.157:0/357717323 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0c00bae0 con 0x7f2f1006cbb0 2023-10-06T03:58:54.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.234+0000 7f2f14f01700 1 -- 172.21.15.157:0/357717323 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c0092e0 con 0x7f2f1006cbb0 2023-10-06T03:58:54.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.235+0000 7f2f18167700 1 -- 172.21.15.157:0/357717323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f1006cbb0 msgr2=0x7f2f1006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:58:54.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.235+0000 7f2f18167700 1 --2- 172.21.15.157:0/357717323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f1006cbb0 0x7f2f1006d530 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2f10157070 tx=0x7f2f0c009840 comp rx=0 tx=0).stop 2023-10-06T03:58:54.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.235+0000 7f2f18167700 1 -- 172.21.15.157:0/357717323 shutdown_connections 2023-10-06T03:58:54.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.235+0000 7f2f18167700 1 --2- 172.21.15.157:0/357717323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f1006cf80 0x7f2f1014fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:54.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.235+0000 7f2f18167700 1 --2- 172.21.15.157:0/357717323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f1006cbb0 0x7f2f1006d530 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:54.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.235+0000 7f2f18167700 1 --2- 172.21.15.157:0/357717323 >> v2:172.21.15.175:3300/0 conn(0x7f2f1006c1f0 0x7f2f1006da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:54.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.235+0000 7f2f18167700 1 -- 172.21.15.157:0/357717323 >> 172.21.15.157:0/357717323 conn(0x7f2f1006f230 msgr2=0x7f2f1006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:58:54.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.235+0000 7f2f18167700 1 -- 172.21.15.157:0/357717323 shutdown_connections 2023-10-06T03:58:54.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.235+0000 7f2f18167700 1 -- 172.21.15.157:0/357717323 wait complete. 2023-10-06T03:58:54.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.236+0000 7f2f18167700 1 Processor -- start 2023-10-06T03:58:54.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.236+0000 7f2f18167700 1 -- start start 2023-10-06T03:58:54.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.237+0000 7f2f18167700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f1006c1f0 0x7f2f10166e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:54.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.237+0000 7f2f18167700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f1006cbb0 0x7f2f10150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:54.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.237+0000 7f2f18167700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f1006cf80 0x7f2f101527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:54.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.237+0000 7f2f18167700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f10156b70 con 0x7f2f1006cbb0 2023-10-06T03:58:54.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.237+0000 7f2f18167700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f101514c0 con 0x7f2f1006c1f0 2023-10-06T03:58:54.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.237+0000 7f2f18167700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f10151710 con 0x7f2f1006cf80 2023-10-06T03:58:54.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.237+0000 7f2f15702700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f1006cbb0 0x7f2f10150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:54.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.237+0000 7f2f15702700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f1006cbb0 0x7f2f10150320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56600/0 (socket says 172.21.15.157:56600) 2023-10-06T03:58:54.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.237+0000 7f2f15702700 1 -- 172.21.15.157:0/691775702 learned_addr learned my addr 172.21.15.157:0/691775702 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:58:54.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.237+0000 7f2f15f03700 1 --2- 172.21.15.157:0/691775702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f1006c1f0 0x7f2f10166e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:54.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.237+0000 7f2f16704700 1 --2- 172.21.15.157:0/691775702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f1006cf80 0x7f2f101527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:54.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.237+0000 7f2f15702700 1 -- 172.21.15.157:0/691775702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f1006cf80 msgr2=0x7f2f101527b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:58:54.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.237+0000 7f2f15702700 1 --2- 172.21.15.157:0/691775702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f1006cf80 0x7f2f101527b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:54.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.237+0000 7f2f15702700 1 -- 172.21.15.157:0/691775702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f1006c1f0 msgr2=0x7f2f10166e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:58:54.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.237+0000 7f2f15702700 1 --2- 172.21.15.157:0/691775702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f1006c1f0 0x7f2f10166e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:54.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.237+0000 7f2f15702700 1 -- 172.21.15.157:0/691775702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f0c009ae0 con 0x7f2f1006cbb0 2023-10-06T03:58:54.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.237+0000 7f2f15f03700 1 --2- 172.21.15.157:0/691775702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f1006c1f0 0x7f2f10166e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:58:54.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.237+0000 7f2f15702700 1 --2- 172.21.15.157:0/691775702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f1006cbb0 0x7f2f10150320 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000d890 tx=0x7f2f00008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:58:54.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.237+0000 7f2f05ffb700 1 -- 172.21.15.157:0/691775702 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0000b600 con 0x7f2f1006cbb0 2023-10-06T03:58:54.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.237+0000 7f2f05ffb700 1 -- 172.21.15.157:0/691775702 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0000b760 con 0x7f2f1006cbb0 2023-10-06T03:58:54.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.237+0000 7f2f05ffb700 1 -- 172.21.15.157:0/691775702 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0000bae0 con 0x7f2f1006cbb0 2023-10-06T03:58:54.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.237+0000 7f2f18167700 1 -- 172.21.15.157:0/691775702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f101519f0 con 0x7f2f1006cbb0 2023-10-06T03:58:54.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.238+0000 7f2f18167700 1 -- 172.21.15.157:0/691775702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f10151fb0 con 0x7f2f1006cbb0 2023-10-06T03:58:54.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.239+0000 7f2f18167700 1 -- 172.21.15.157:0/691775702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f10089ca0 con 0x7f2f1006cbb0 2023-10-06T03:58:54.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.239+0000 7f2f05ffb700 1 -- 172.21.15.157:0/691775702 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2f0001e460 con 0x7f2f1006cbb0 2023-10-06T03:58:54.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.239+0000 7f2f05ffb700 1 --2- 172.21.15.157:0/691775702 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2ef4066f30 0x7f2ef40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:54.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.239+0000 7f2f05ffb700 1 -- 172.21.15.157:0/691775702 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f2f00084df0 con 0x7f2f1006cbb0 2023-10-06T03:58:54.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.239+0000 7f2f15f03700 1 --2- 172.21.15.157:0/691775702 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2ef4066f30 0x7f2ef40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:54.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.240+0000 7f2f15f03700 1 --2- 172.21.15.157:0/691775702 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2ef4066f30 0x7f2ef40693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c009880 tx=0x7f2f0c018000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:58:54.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.244+0000 7f2f05ffb700 1 -- 172.21.15.157:0/691775702 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2f00051eb0 con 0x7f2f1006cbb0 2023-10-06T03:58:54.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.473+0000 7f2f18167700 1 -- 172.21.15.157:0/691775702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f100c1450 con 0x7f2f1006cbb0 2023-10-06T03:58:54.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.473+0000 7f2f05ffb700 1 -- 172.21.15.157:0/691775702 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+6184 (secure 0 0 0) 0x7f2f000554d0 con 0x7f2f1006cbb0 2023-10-06T03:58:54.475 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:58:54.475 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T03:57:37.649946+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"f","rank":4,"incarnation":15,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/4177493830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4177493830},{"type":"v1","addr":"172.21.15.175:6839","nonce":4177493830}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:58:54.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.475+0000 7f2f18167700 1 -- 172.21.15.157:0/691775702 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2ef4066f30 msgr2=0x7f2ef40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:58:54.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.475+0000 7f2f18167700 1 --2- 172.21.15.157:0/691775702 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2ef4066f30 0x7f2ef40693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c009880 tx=0x7f2f0c018000 comp rx=0 tx=0).stop 2023-10-06T03:58:54.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.475+0000 7f2f18167700 1 -- 172.21.15.157:0/691775702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f1006cbb0 msgr2=0x7f2f10150320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:58:54.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.475+0000 7f2f18167700 1 --2- 172.21.15.157:0/691775702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f1006cbb0 0x7f2f10150320 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000d890 tx=0x7f2f00008040 comp rx=0 tx=0).stop 2023-10-06T03:58:54.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.476+0000 7f2f18167700 1 -- 172.21.15.157:0/691775702 shutdown_connections 2023-10-06T03:58:54.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.476+0000 7f2f18167700 1 --2- 172.21.15.157:0/691775702 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2ef4066f30 0x7f2ef40693e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:54.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.476+0000 7f2f18167700 1 --2- 172.21.15.157:0/691775702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f1006c1f0 0x7f2f10166e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:54.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.476+0000 7f2f18167700 1 --2- 172.21.15.157:0/691775702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f1006cf80 0x7f2f101527b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:54.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.476+0000 7f2f18167700 1 --2- 172.21.15.157:0/691775702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f1006cbb0 0x7f2f10150320 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:54.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.476+0000 7f2f18167700 1 -- 172.21.15.157:0/691775702 >> 172.21.15.157:0/691775702 conn(0x7f2f1006f230 msgr2=0x7f2f100c6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:58:54.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.476+0000 7f2f18167700 1 -- 172.21.15.157:0/691775702 shutdown_connections 2023-10-06T03:58:54.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:54.476+0000 7f2f18167700 1 -- 172.21.15.157:0/691775702 wait complete. 2023-10-06T03:58:54.479 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 16 2023-10-06T03:58:54.489 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-10-06T03:58:56.490 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:58:56.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.618+0000 7f75e273a700 1 Processor -- start 2023-10-06T03:58:56.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.618+0000 7f75e273a700 1 -- start start 2023-10-06T03:58:56.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.618+0000 7f75e273a700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f75dc14e0a0 0x7f75dc153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:56.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.618+0000 7f75e273a700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75dc0c5770 con 0x7f75dc14dcd0 2023-10-06T03:58:56.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.618+0000 7f75e273a700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75dc1543e0 con 0x7f75dc0c5c00 2023-10-06T03:58:56.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.618+0000 7f75e273a700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75dc154c80 con 0x7f75dc14e0a0 2023-10-06T03:58:56.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.619+0000 7f75dbfff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f75dc14dcd0 0x7f75dc0c18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:45586/0 (socket says 172.21.15.157:45586) 2023-10-06T03:58:56.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.619+0000 7f75dbfff700 1 -- 172.21.15.157:0/619006616 learned_addr learned my addr 172.21.15.157:0/619006616 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:58:56.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.619+0000 7f75db7fe700 1 --2- 172.21.15.157:0/619006616 >> v2:172.21.15.175:3300/0 conn(0x7f75dc14e0a0 0x7f75dc153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:56.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.619+0000 7f75daffd700 1 -- 172.21.15.157:0/619006616 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4102806094 0 0) 0x7f75dc0c5770 con 0x7f75dc14dcd0 2023-10-06T03:58:56.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.619+0000 7f75daffd700 1 -- 172.21.15.157:0/619006616 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75c4003570 con 0x7f75dc14dcd0 2023-10-06T03:58:56.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.619+0000 7f75daffd700 1 -- 172.21.15.157:0/619006616 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 622604446 0 0) 0x7f75c4003570 con 0x7f75dc14dcd0 2023-10-06T03:58:56.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.619+0000 7f75daffd700 1 -- 172.21.15.157:0/619006616 >> v2:172.21.15.175:3300/0 conn(0x7f75dc14e0a0 msgr2=0x7f75dc153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:58:56.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.619+0000 7f75daffd700 1 --2- 172.21.15.157:0/619006616 >> v2:172.21.15.175:3300/0 conn(0x7f75dc14e0a0 0x7f75dc153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:56.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.619+0000 7f75daffd700 1 -- 172.21.15.157:0/619006616 >> v1:172.21.15.175:6789/0 conn(0x7f75dc0c5c00 legacy=0x7f75dc0ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:58:56.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.619+0000 7f75daffd700 1 -- 172.21.15.157:0/619006616 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75dc1546c0 con 0x7f75dc14dcd0 2023-10-06T03:58:56.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.619+0000 7f75daffd700 1 -- 172.21.15.157:0/619006616 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f75cc001470 con 0x7f75dc14dcd0 2023-10-06T03:58:56.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.619+0000 7f75daffd700 1 -- 172.21.15.157:0/619006616 >> v1:172.21.15.157:6789/0 conn(0x7f75dc14dcd0 legacy=0x7f75dc0c18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:58:56.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.619+0000 7f75daffd700 1 --2- 172.21.15.157:0/619006616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75c4003a60 0x7f75c4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:56.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.619+0000 7f75daffd700 1 --2- 172.21.15.157:0/619006616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75c4004810 0x7f75c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:56.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.619+0000 7f75daffd700 1 --2- 172.21.15.157:0/619006616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75c40051c0 0x7f75c400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:56.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.619+0000 7f75daffd700 1 -- 172.21.15.157:0/619006616 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc154c80 con 0x7f75c4004810 2023-10-06T03:58:56.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.619+0000 7f75daffd700 1 -- 172.21.15.157:0/619006616 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c400c0d0 con 0x7f75c40051c0 2023-10-06T03:58:56.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.619+0000 7f75daffd700 1 -- 172.21.15.157:0/619006616 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c400c290 con 0x7f75c4003a60 2023-10-06T03:58:56.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.619+0000 7f75db7fe700 1 --2- 172.21.15.157:0/619006616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75c4004810 0x7f75c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:56.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.619+0000 7f75dbfff700 1 --2- 172.21.15.157:0/619006616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75c4003a60 0x7f75c4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:56.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.619+0000 7f75e0cd7700 1 --2- 172.21.15.157:0/619006616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75c40051c0 0x7f75c400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:56.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.620+0000 7f75db7fe700 1 -- 172.21.15.157:0/619006616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75c4003a60 msgr2=0x7f75c4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:58:56.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.620+0000 7f75db7fe700 1 --2- 172.21.15.157:0/619006616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75c4003a60 0x7f75c4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:56.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.620+0000 7f75db7fe700 1 -- 172.21.15.157:0/619006616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75c40051c0 msgr2=0x7f75c400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:58:56.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.620+0000 7f75db7fe700 1 --2- 172.21.15.157:0/619006616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75c40051c0 0x7f75c400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:56.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.620+0000 7f75db7fe700 1 -- 172.21.15.157:0/619006616 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c4003570 con 0x7f75c4004810 2023-10-06T03:58:56.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.620+0000 7f75e0cd7700 1 -- 172.21.15.157:0/619006616 reap_dead start 2023-10-06T03:58:56.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.620+0000 7f75db7fe700 1 --2- 172.21.15.157:0/619006616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75c4004810 0x7f75c4004c80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f75d0009c00 tx=0x7f75d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba4bf0bab657d33 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:58:56.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.620+0000 7f75daffd700 1 -- 172.21.15.157:0/619006616 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d000e710 con 0x7f75c4004810 2023-10-06T03:58:56.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.620+0000 7f75daffd700 1 -- 172.21.15.157:0/619006616 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d000e870 con 0x7f75c4004810 2023-10-06T03:58:56.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.620+0000 7f75daffd700 1 -- 172.21.15.157:0/619006616 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d000ec30 con 0x7f75c4004810 2023-10-06T03:58:56.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.620+0000 7f75e273a700 1 -- 172.21.15.157:0/619006616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75c4004810 msgr2=0x7f75c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:58:56.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.620+0000 7f75e273a700 1 --2- 172.21.15.157:0/619006616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75c4004810 0x7f75c4004c80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f75d0009c00 tx=0x7f75d00091a0 comp rx=0 tx=0).stop 2023-10-06T03:58:56.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.620+0000 7f75e273a700 1 -- 172.21.15.157:0/619006616 shutdown_connections 2023-10-06T03:58:56.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.620+0000 7f75e273a700 1 --2- 172.21.15.157:0/619006616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75c4004810 0x7f75c4004c80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:56.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.620+0000 7f75e273a700 1 -- 172.21.15.157:0/619006616 >> 172.21.15.157:0/619006616 conn(0x7f75dc0692b0 msgr2=0x7f75dc0696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:58:56.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.621+0000 7f75e273a700 1 -- 172.21.15.157:0/619006616 shutdown_connections 2023-10-06T03:58:56.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.621+0000 7f75e273a700 1 -- 172.21.15.157:0/619006616 wait complete. 2023-10-06T03:58:56.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.622+0000 7f75e273a700 1 Processor -- start 2023-10-06T03:58:56.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.622+0000 7f75e273a700 1 -- start start 2023-10-06T03:58:56.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.622+0000 7f75e273a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75c4004810 0x7f75dc164a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:56.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.622+0000 7f75e273a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75dc149fe0 0x7f75dc149720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:56.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.622+0000 7f75e273a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75dc1561c0 0x7f75dc165440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:56.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.622+0000 7f75e273a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc154200 con 0x7f75dc149fe0 2023-10-06T03:58:56.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.622+0000 7f75e273a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc167de0 con 0x7f75dc1561c0 2023-10-06T03:58:56.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.622+0000 7f75e273a700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc167500 con 0x7f75c4004810 2023-10-06T03:58:56.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.623+0000 7f75db7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75dc149fe0 0x7f75dc149720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:56.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.623+0000 7f75db7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75dc149fe0 0x7f75dc149720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56614/0 (socket says 172.21.15.157:56614) 2023-10-06T03:58:56.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.623+0000 7f75db7fe700 1 -- 172.21.15.157:0/2029506695 learned_addr learned my addr 172.21.15.157:0/2029506695 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:58:56.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.623+0000 7f75dbfff700 1 --2- 172.21.15.157:0/2029506695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75c4004810 0x7f75dc164a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:56.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.623+0000 7f75e0cd7700 1 --2- 172.21.15.157:0/2029506695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75dc1561c0 0x7f75dc165440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:56.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.623+0000 7f75db7fe700 1 -- 172.21.15.157:0/2029506695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75c4004810 msgr2=0x7f75dc164a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:58:56.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.623+0000 7f75db7fe700 1 --2- 172.21.15.157:0/2029506695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75c4004810 0x7f75dc164a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:56.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.623+0000 7f75db7fe700 1 -- 172.21.15.157:0/2029506695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75dc1561c0 msgr2=0x7f75dc165440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:58:56.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.623+0000 7f75db7fe700 1 --2- 172.21.15.157:0/2029506695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75dc1561c0 0x7f75dc165440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:56.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.623+0000 7f75db7fe700 1 -- 172.21.15.157:0/2029506695 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75dc156b40 con 0x7f75dc149fe0 2023-10-06T03:58:56.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.623+0000 7f75dbfff700 1 --2- 172.21.15.157:0/2029506695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75c4004810 0x7f75dc164a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:58:56.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.623+0000 7f75db7fe700 1 --2- 172.21.15.157:0/2029506695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75dc149fe0 0x7f75dc149720 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f75d000e4b0 tx=0x7f75d001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:58:56.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.623+0000 7f75c3fff700 1 -- 172.21.15.157:0/2029506695 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d0022070 con 0x7f75dc149fe0 2023-10-06T03:58:56.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.623+0000 7f75c3fff700 1 -- 172.21.15.157:0/2029506695 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d000e9e0 con 0x7f75dc149fe0 2023-10-06T03:58:56.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.623+0000 7f75e273a700 1 -- 172.21.15.157:0/2029506695 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75dc156de0 con 0x7f75dc149fe0 2023-10-06T03:58:56.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.623+0000 7f75c3fff700 1 -- 172.21.15.157:0/2029506695 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d001c450 con 0x7f75dc149fe0 2023-10-06T03:58:56.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.623+0000 7f75e273a700 1 -- 172.21.15.157:0/2029506695 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75dc167a70 con 0x7f75dc149fe0 2023-10-06T03:58:56.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.624+0000 7f75e273a700 1 -- 172.21.15.157:0/2029506695 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75dc167f90 con 0x7f75dc149fe0 2023-10-06T03:58:56.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.625+0000 7f75c3fff700 1 -- 172.21.15.157:0/2029506695 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f75d001c8a0 con 0x7f75dc149fe0 2023-10-06T03:58:56.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.625+0000 7f75c3fff700 1 --2- 172.21.15.157:0/2029506695 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f75c8066f60 0x7f75c8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:56.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.625+0000 7f75c3fff700 1 -- 172.21.15.157:0/2029506695 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f75d0085f60 con 0x7f75dc149fe0 2023-10-06T03:58:56.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.625+0000 7f75dbfff700 1 --2- 172.21.15.157:0/2029506695 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f75c8066f60 0x7f75c8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:56.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.626+0000 7f75dbfff700 1 --2- 172.21.15.157:0/2029506695 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f75c8066f60 0x7f75c8069410 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f75cc001ce0 tx=0x7f75cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:58:56.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.629+0000 7f75c3fff700 1 -- 172.21.15.157:0/2029506695 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f75dc167f90 con 0x7f75dc149fe0 2023-10-06T03:58:56.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.855+0000 7f75e273a700 1 -- 172.21.15.157:0/2029506695 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75dc167f90 con 0x7f75dc149fe0 2023-10-06T03:58:56.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.856+0000 7f75c3fff700 1 -- 172.21.15.157:0/2029506695 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+6184 (secure 0 0 0) 0x7f75dc167f90 con 0x7f75dc149fe0 2023-10-06T03:58:56.858 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:58:56.858 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T03:57:37.649946+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"f","rank":4,"incarnation":15,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/4177493830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4177493830},{"type":"v1","addr":"172.21.15.175:6839","nonce":4177493830}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:58:56.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.858+0000 7f75e273a700 1 -- 172.21.15.157:0/2029506695 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f75c8066f60 msgr2=0x7f75c8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:58:56.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.858+0000 7f75e273a700 1 --2- 172.21.15.157:0/2029506695 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f75c8066f60 0x7f75c8069410 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f75cc001ce0 tx=0x7f75cc009040 comp rx=0 tx=0).stop 2023-10-06T03:58:56.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.858+0000 7f75e273a700 1 -- 172.21.15.157:0/2029506695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75dc149fe0 msgr2=0x7f75dc149720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:58:56.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.858+0000 7f75e273a700 1 --2- 172.21.15.157:0/2029506695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75dc149fe0 0x7f75dc149720 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f75d000e4b0 tx=0x7f75d001a000 comp rx=0 tx=0).stop 2023-10-06T03:58:56.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.858+0000 7f75e273a700 1 -- 172.21.15.157:0/2029506695 shutdown_connections 2023-10-06T03:58:56.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.858+0000 7f75e273a700 1 --2- 172.21.15.157:0/2029506695 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f75c8066f60 0x7f75c8069410 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:56.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.858+0000 7f75e273a700 1 --2- 172.21.15.157:0/2029506695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75dc1561c0 0x7f75dc165440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:56.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.858+0000 7f75e273a700 1 --2- 172.21.15.157:0/2029506695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75c4004810 0x7f75dc164a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:56.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.858+0000 7f75e273a700 1 --2- 172.21.15.157:0/2029506695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75dc149fe0 0x7f75dc149720 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:56.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.858+0000 7f75e273a700 1 -- 172.21.15.157:0/2029506695 >> 172.21.15.157:0/2029506695 conn(0x7f75dc0692b0 msgr2=0x7f75dc1539e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:58:56.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.858+0000 7f75e273a700 1 -- 172.21.15.157:0/2029506695 shutdown_connections 2023-10-06T03:58:56.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:56.858+0000 7f75e273a700 1 -- 172.21.15.157:0/2029506695 wait complete. 2023-10-06T03:58:56.861 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 16 2023-10-06T03:58:56.872 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-10-06T03:58:58.873 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:58:59.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:58.999+0000 7fb7b755c700 1 Processor -- start 2023-10-06T03:58:59.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:58.999+0000 7fb7b755c700 1 -- start start 2023-10-06T03:58:59.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.000+0000 7fb7b755c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb7b006dc40 0x7fb7b014fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:59.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.000+0000 7fb7b755c700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7b00c9780 con 0x7fb7b006c1f0 2023-10-06T03:58:59.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.000+0000 7fb7b755c700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7b006e0c0 con 0x7fb7b006cbb0 2023-10-06T03:58:59.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.000+0000 7fb7b755c700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7b00c8c40 con 0x7fb7b006dc40 2023-10-06T03:58:59.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.000+0000 7fb7b4af7700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb7b006dc40 0x7fb7b014fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:59.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.000+0000 7fb7b5af9700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb7b006c1f0 0x7fb7b0155f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:45594/0 (socket says 172.21.15.157:45594) 2023-10-06T03:58:59.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.000+0000 7fb7b5af9700 1 -- 172.21.15.157:0/3424541771 learned_addr learned my addr 172.21.15.157:0/3424541771 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:58:59.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.000+0000 7fb7b4af7700 1 -- 172.21.15.157:0/3424541771 >> v1:172.21.15.175:6789/0 conn(0x7fb7b006cbb0 legacy=0x7fb7b006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:58:59.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.000+0000 7fb7b4af7700 1 -- 172.21.15.157:0/3424541771 >> v1:172.21.15.157:6789/0 conn(0x7fb7b006c1f0 legacy=0x7fb7b0155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:58:59.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.000+0000 7fb7b4af7700 1 -- 172.21.15.157:0/3424541771 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b00c8e50 con 0x7fb7b006dc40 2023-10-06T03:58:59.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.000+0000 7fb7a7fff700 1 -- 172.21.15.157:0/3424541771 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1351024916 0 0) 0x7fb7b00c9780 con 0x7fb7b006c1f0 2023-10-06T03:58:59.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.000+0000 7fb7b4af7700 1 --2- 172.21.15.157:0/3424541771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7b006dc40 0x7fb7b014fd00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00098e0 tx=0x7fb7a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3992326a740fd0a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:58:59.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.000+0000 7fb7a7fff700 1 -- 172.21.15.157:0/3424541771 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a0009e80 con 0x7fb7b006dc40 2023-10-06T03:58:59.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.000+0000 7fb7a7fff700 1 -- 172.21.15.157:0/3424541771 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7a000b8f0 con 0x7fb7b006dc40 2023-10-06T03:58:59.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.000+0000 7fb7a7fff700 1 -- 172.21.15.157:0/3424541771 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a000bc70 con 0x7fb7b006dc40 2023-10-06T03:58:59.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.001+0000 7fb7b755c700 1 -- 172.21.15.157:0/3424541771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7b006dc40 msgr2=0x7fb7b014fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:58:59.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.001+0000 7fb7b755c700 1 --2- 172.21.15.157:0/3424541771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7b006dc40 0x7fb7b014fd00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00098e0 tx=0x7fb7a00091a0 comp rx=0 tx=0).stop 2023-10-06T03:58:59.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.001+0000 7fb7b755c700 1 -- 172.21.15.157:0/3424541771 shutdown_connections 2023-10-06T03:58:59.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.001+0000 7fb7b755c700 1 --2- 172.21.15.157:0/3424541771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7b006dc40 0x7fb7b014fd00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:59.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.001+0000 7fb7b755c700 1 -- 172.21.15.157:0/3424541771 >> 172.21.15.157:0/3424541771 conn(0x7fb7b006f230 msgr2=0x7fb7b006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:58:59.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.001+0000 7fb7b755c700 1 -- 172.21.15.157:0/3424541771 shutdown_connections 2023-10-06T03:58:59.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.001+0000 7fb7b755c700 1 -- 172.21.15.157:0/3424541771 wait complete. 2023-10-06T03:58:59.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.002+0000 7fb7b755c700 1 Processor -- start 2023-10-06T03:58:59.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.002+0000 7fb7b755c700 1 -- start start 2023-10-06T03:58:59.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.003+0000 7fb7b755c700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7b006c1f0 0x7fb7b00c7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:59.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.003+0000 7fb7b755c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7b006cbb0 0x7fb7b00c7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:59.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.003+0000 7fb7b755c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7b00c6750 0x7fb7b014af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:59.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.003+0000 7fb7b755c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b0150600 con 0x7fb7b006cbb0 2023-10-06T03:58:59.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.003+0000 7fb7b755c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b01508e0 con 0x7fb7b00c6750 2023-10-06T03:58:59.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.003+0000 7fb7b755c700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b00c0eb0 con 0x7fb7b006c1f0 2023-10-06T03:58:59.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.003+0000 7fb7b4af7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7b006cbb0 0x7fb7b00c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:59.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.003+0000 7fb7b4af7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7b006cbb0 0x7fb7b00c7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56634/0 (socket says 172.21.15.157:56634) 2023-10-06T03:58:59.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.003+0000 7fb7b4af7700 1 -- 172.21.15.157:0/2606984249 learned_addr learned my addr 172.21.15.157:0/2606984249 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:58:59.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.003+0000 7fb7b5af9700 1 --2- 172.21.15.157:0/2606984249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7b00c6750 0x7fb7b014af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:59.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.003+0000 7fb7b52f8700 1 --2- 172.21.15.157:0/2606984249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7b006c1f0 0x7fb7b00c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:59.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.003+0000 7fb7b4af7700 1 -- 172.21.15.157:0/2606984249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7b006c1f0 msgr2=0x7fb7b00c7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:58:59.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.003+0000 7fb7b4af7700 1 --2- 172.21.15.157:0/2606984249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7b006c1f0 0x7fb7b00c7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:59.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.003+0000 7fb7b4af7700 1 -- 172.21.15.157:0/2606984249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7b00c6750 msgr2=0x7fb7b014af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:58:59.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.003+0000 7fb7b4af7700 1 --2- 172.21.15.157:0/2606984249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7b00c6750 0x7fb7b014af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:59.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.003+0000 7fb7b4af7700 1 -- 172.21.15.157:0/2606984249 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a0009420 con 0x7fb7b006cbb0 2023-10-06T03:58:59.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.003+0000 7fb7b5af9700 1 --2- 172.21.15.157:0/2606984249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7b00c6750 0x7fb7b014af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:58:59.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.003+0000 7fb7b4af7700 1 --2- 172.21.15.157:0/2606984249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7b006cbb0 0x7fb7b00c7740 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000b3c0 tx=0x7fb7a0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:58:59.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.004+0000 7fb7a57fa700 1 -- 172.21.15.157:0/2606984249 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a001f070 con 0x7fb7b006cbb0 2023-10-06T03:58:59.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.004+0000 7fb7a57fa700 1 -- 172.21.15.157:0/2606984249 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7a001a420 con 0x7fb7b006cbb0 2023-10-06T03:58:59.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.004+0000 7fb7b755c700 1 -- 172.21.15.157:0/2606984249 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7b014ff50 con 0x7fb7b006cbb0 2023-10-06T03:58:59.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.004+0000 7fb7a57fa700 1 -- 172.21.15.157:0/2606984249 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a000bc70 con 0x7fb7b006cbb0 2023-10-06T03:58:59.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.004+0000 7fb7b755c700 1 -- 172.21.15.157:0/2606984249 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b0152ae0 con 0x7fb7b006cbb0 2023-10-06T03:58:59.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.005+0000 7fb7b755c700 1 -- 172.21.15.157:0/2606984249 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b0089ca0 con 0x7fb7b006cbb0 2023-10-06T03:58:59.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.005+0000 7fb7a57fa700 1 -- 172.21.15.157:0/2606984249 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb7a001aa70 con 0x7fb7b006cbb0 2023-10-06T03:58:59.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.005+0000 7fb7a57fa700 1 --2- 172.21.15.157:0/2606984249 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb794066f30 0x7fb7940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:58:59.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.006+0000 7fb7a57fa700 1 -- 172.21.15.157:0/2606984249 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fb7a0085c60 con 0x7fb7b006cbb0 2023-10-06T03:58:59.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.006+0000 7fb7b52f8700 1 --2- 172.21.15.157:0/2606984249 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb794066f30 0x7fb7940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:58:59.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.006+0000 7fb7b52f8700 1 --2- 172.21.15.157:0/2606984249 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb794066f30 0x7fb7940693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac007fd0 tx=0x7fb7ac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:58:59.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.010+0000 7fb7a57fa700 1 -- 172.21.15.157:0/2606984249 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb7a0052ca0 con 0x7fb7b006cbb0 2023-10-06T03:58:59.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.234+0000 7fb7b755c700 1 -- 172.21.15.157:0/2606984249 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7b0150a20 con 0x7fb7b006cbb0 2023-10-06T03:58:59.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.235+0000 7fb7a57fa700 1 -- 172.21.15.157:0/2606984249 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+6184 (secure 0 0 0) 0x7fb7a0028020 con 0x7fb7b006cbb0 2023-10-06T03:58:59.236 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:58:59.236 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T03:57:37.649946+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"f","rank":4,"incarnation":15,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/4177493830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4177493830},{"type":"v1","addr":"172.21.15.175:6839","nonce":4177493830}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:58:59.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.236+0000 7fb7b755c700 1 -- 172.21.15.157:0/2606984249 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb794066f30 msgr2=0x7fb7940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:58:59.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.236+0000 7fb7b755c700 1 --2- 172.21.15.157:0/2606984249 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb794066f30 0x7fb7940693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac007fd0 tx=0x7fb7ac009040 comp rx=0 tx=0).stop 2023-10-06T03:58:59.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.236+0000 7fb7b755c700 1 -- 172.21.15.157:0/2606984249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7b006cbb0 msgr2=0x7fb7b00c7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:58:59.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.236+0000 7fb7b755c700 1 --2- 172.21.15.157:0/2606984249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7b006cbb0 0x7fb7b00c7740 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000b3c0 tx=0x7fb7a0017000 comp rx=0 tx=0).stop 2023-10-06T03:58:59.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.237+0000 7fb7b755c700 1 -- 172.21.15.157:0/2606984249 shutdown_connections 2023-10-06T03:58:59.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.237+0000 7fb7b755c700 1 --2- 172.21.15.157:0/2606984249 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb794066f30 0x7fb7940693e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:59.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.237+0000 7fb7b755c700 1 --2- 172.21.15.157:0/2606984249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7b00c6750 0x7fb7b014af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:59.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.237+0000 7fb7b755c700 1 --2- 172.21.15.157:0/2606984249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7b006c1f0 0x7fb7b00c7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:59.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.237+0000 7fb7b755c700 1 --2- 172.21.15.157:0/2606984249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7b006cbb0 0x7fb7b00c7740 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:58:59.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.237+0000 7fb7b755c700 1 -- 172.21.15.157:0/2606984249 >> 172.21.15.157:0/2606984249 conn(0x7fb7b006f230 msgr2=0x7fb7b00af240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:58:59.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.237+0000 7fb7b755c700 1 -- 172.21.15.157:0/2606984249 shutdown_connections 2023-10-06T03:58:59.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:58:59.237+0000 7fb7b755c700 1 -- 172.21.15.157:0/2606984249 wait complete. 2023-10-06T03:58:59.240 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 16 2023-10-06T03:58:59.250 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-10-06T03:59:01.250 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:59:01.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.376+0000 7f5f354ad700 1 Processor -- start 2023-10-06T03:59:01.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.376+0000 7f5f354ad700 1 -- start start 2023-10-06T03:59:01.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.376+0000 7f5f354ad700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5f300b2110 0x7f5f30159df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:01.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.376+0000 7f5f354ad700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f3006c1f0 0x7f5f3014fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:01.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.376+0000 7f5f354ad700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f3006ced0 con 0x7f5f300b1d40 2023-10-06T03:59:01.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.376+0000 7f5f354ad700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f300afb10 con 0x7f5f300b2110 2023-10-06T03:59:01.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.376+0000 7f5f354ad700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3006cc10 con 0x7f5f3006c1f0 2023-10-06T03:59:01.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.376+0000 7f5f267fc700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5f300b2110 0x7f5f30159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:01.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.376+0000 7f5f2effd700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f5f300b1d40 0x7f5f300af400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:45606/0 (socket says 172.21.15.157:45606) 2023-10-06T03:59:01.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.377+0000 7f5f2effd700 1 -- 172.21.15.157:0/2194435558 learned_addr learned my addr 172.21.15.157:0/2194435558 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:59:01.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.377+0000 7f5f2f7fe700 1 --2- 172.21.15.157:0/2194435558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f3006c1f0 0x7f5f3014fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:01.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.377+0000 7f5f267fc700 1 -- 172.21.15.157:0/2194435558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f3006c1f0 msgr2=0x7f5f3014fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:59:01.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.377+0000 7f5f267fc700 1 --2- 172.21.15.157:0/2194435558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f3006c1f0 0x7f5f3014fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:01.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.377+0000 7f5f267fc700 1 -- 172.21.15.157:0/2194435558 >> v1:172.21.15.157:6789/0 conn(0x7f5f300b1d40 legacy=0x7f5f300af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:01.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.377+0000 7f5f267fc700 1 -- 172.21.15.157:0/2194435558 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3006e570 con 0x7f5f300b2110 2023-10-06T03:59:01.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.377+0000 7f5f2e7fc700 1 -- 172.21.15.157:0/2194435558 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1480122268 0 0) 0x7f5f3006ced0 con 0x7f5f300b1d40 2023-10-06T03:59:01.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.377+0000 7f5f267fc700 1 --2- 172.21.15.157:0/2194435558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f300b2110 0x7f5f30159df0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5f180098e0 tx=0x7f5f180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ad0bdfd11c7a6b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:01.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.377+0000 7f5f2e7fc700 1 -- 172.21.15.157:0/2194435558 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1800b6d0 con 0x7f5f300b2110 2023-10-06T03:59:01.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.377+0000 7f5f2e7fc700 1 -- 172.21.15.157:0/2194435558 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1800b830 con 0x7f5f300b2110 2023-10-06T03:59:01.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.377+0000 7f5f2e7fc700 1 -- 172.21.15.157:0/2194435558 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1800bbd0 con 0x7f5f300b2110 2023-10-06T03:59:01.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.378+0000 7f5f354ad700 1 -- 172.21.15.157:0/2194435558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f300b2110 msgr2=0x7f5f30159df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:01.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.378+0000 7f5f354ad700 1 --2- 172.21.15.157:0/2194435558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f300b2110 0x7f5f30159df0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5f180098e0 tx=0x7f5f180091a0 comp rx=0 tx=0).stop 2023-10-06T03:59:01.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.378+0000 7f5f354ad700 1 -- 172.21.15.157:0/2194435558 shutdown_connections 2023-10-06T03:59:01.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.378+0000 7f5f354ad700 1 --2- 172.21.15.157:0/2194435558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f3006c1f0 0x7f5f3014fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:01.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.378+0000 7f5f354ad700 1 --2- 172.21.15.157:0/2194435558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f300b2110 0x7f5f30159df0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:01.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.378+0000 7f5f354ad700 1 -- 172.21.15.157:0/2194435558 >> 172.21.15.157:0/2194435558 conn(0x7f5f300cc6e0 msgr2=0x7f5f300ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:59:01.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.378+0000 7f5f354ad700 1 -- 172.21.15.157:0/2194435558 shutdown_connections 2023-10-06T03:59:01.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.378+0000 7f5f354ad700 1 -- 172.21.15.157:0/2194435558 wait complete. 2023-10-06T03:59:01.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.379+0000 7f5f354ad700 1 Processor -- start 2023-10-06T03:59:01.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.379+0000 7f5f354ad700 1 -- start start 2023-10-06T03:59:01.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.380+0000 7f5f354ad700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f3006c1f0 0x7f5f300c2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:01.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.380+0000 7f5f354ad700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f300b1d40 0x7f5f300c1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:01.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.380+0000 7f5f354ad700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f300b2110 0x7f5f3014c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:01.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.380+0000 7f5f354ad700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f301551e0 con 0x7f5f3006c1f0 2023-10-06T03:59:01.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.380+0000 7f5f354ad700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f301532f0 con 0x7f5f300b2110 2023-10-06T03:59:01.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.380+0000 7f5f354ad700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f30169b80 con 0x7f5f300b1d40 2023-10-06T03:59:01.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.380+0000 7f5f2effd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f3006c1f0 0x7f5f300c2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:01.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.380+0000 7f5f2effd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f3006c1f0 0x7f5f300c2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56662/0 (socket says 172.21.15.157:56662) 2023-10-06T03:59:01.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.380+0000 7f5f2effd700 1 -- 172.21.15.157:0/1180236129 learned_addr learned my addr 172.21.15.157:0/1180236129 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:59:01.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.380+0000 7f5f2effd700 1 -- 172.21.15.157:0/1180236129 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f300b1d40 msgr2=0x7f5f300c1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:01.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.380+0000 7f5f267fc700 1 --2- 172.21.15.157:0/1180236129 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f300b1d40 0x7f5f300c1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:01.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.380+0000 7f5f2effd700 1 --2- 172.21.15.157:0/1180236129 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f300b1d40 0x7f5f300c1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:01.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.380+0000 7f5f2effd700 1 -- 172.21.15.157:0/1180236129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f300b2110 msgr2=0x7f5f3014c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:01.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.380+0000 7f5f2f7fe700 1 --2- 172.21.15.157:0/1180236129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f300b2110 0x7f5f3014c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:01.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.380+0000 7f5f2effd700 1 --2- 172.21.15.157:0/1180236129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f300b2110 0x7f5f3014c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:01.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.380+0000 7f5f2effd700 1 -- 172.21.15.157:0/1180236129 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f18009420 con 0x7f5f3006c1f0 2023-10-06T03:59:01.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.380+0000 7f5f2effd700 1 --2- 172.21.15.157:0/1180236129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f3006c1f0 0x7f5f300c2300 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000c810 tx=0x7f5f20007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:01.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.380+0000 7f5f277fe700 1 -- 172.21.15.157:0/1180236129 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20005b30 con 0x7f5f3006c1f0 2023-10-06T03:59:01.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.380+0000 7f5f277fe700 1 -- 172.21.15.157:0/1180236129 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f20005c90 con 0x7f5f3006c1f0 2023-10-06T03:59:01.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.380+0000 7f5f354ad700 1 -- 172.21.15.157:0/1180236129 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f30152d70 con 0x7f5f3006c1f0 2023-10-06T03:59:01.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.380+0000 7f5f277fe700 1 -- 172.21.15.157:0/1180236129 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20007300 con 0x7f5f3006c1f0 2023-10-06T03:59:01.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.380+0000 7f5f354ad700 1 -- 172.21.15.157:0/1180236129 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f301695b0 con 0x7f5f3006c1f0 2023-10-06T03:59:01.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.381+0000 7f5f354ad700 1 -- 172.21.15.157:0/1180236129 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f30169930 con 0x7f5f3006c1f0 2023-10-06T03:59:01.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.382+0000 7f5f277fe700 1 -- 172.21.15.157:0/1180236129 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5f2001b400 con 0x7f5f3006c1f0 2023-10-06T03:59:01.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.382+0000 7f5f277fe700 1 --2- 172.21.15.157:0/1180236129 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5f10066f30 0x7f5f100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:01.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.382+0000 7f5f277fe700 1 -- 172.21.15.157:0/1180236129 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f5f200841b0 con 0x7f5f3006c1f0 2023-10-06T03:59:01.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.382+0000 7f5f267fc700 1 --2- 172.21.15.157:0/1180236129 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5f10066f30 0x7f5f100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:01.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.383+0000 7f5f267fc700 1 --2- 172.21.15.157:0/1180236129 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5f10066f30 0x7f5f100693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5f30064410 tx=0x7f5f18017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:01.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.386+0000 7f5f277fe700 1 -- 172.21.15.157:0/1180236129 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5f200512f0 con 0x7f5f3006c1f0 2023-10-06T03:59:01.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.612+0000 7f5f354ad700 1 -- 172.21.15.157:0/1180236129 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f3006d7b0 con 0x7f5f3006c1f0 2023-10-06T03:59:01.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.612+0000 7f5f277fe700 1 -- 172.21.15.157:0/1180236129 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+6184 (secure 0 0 0) 0x7f5f20054910 con 0x7f5f3006c1f0 2023-10-06T03:59:01.614 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:59:01.614 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T03:57:37.649946+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"f","rank":4,"incarnation":15,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/4177493830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4177493830},{"type":"v1","addr":"172.21.15.175:6839","nonce":4177493830}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:59:01.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.614+0000 7f5f354ad700 1 -- 172.21.15.157:0/1180236129 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5f10066f30 msgr2=0x7f5f100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:01.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.614+0000 7f5f354ad700 1 --2- 172.21.15.157:0/1180236129 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5f10066f30 0x7f5f100693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5f30064410 tx=0x7f5f18017000 comp rx=0 tx=0).stop 2023-10-06T03:59:01.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.614+0000 7f5f354ad700 1 -- 172.21.15.157:0/1180236129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f3006c1f0 msgr2=0x7f5f300c2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:01.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.614+0000 7f5f354ad700 1 --2- 172.21.15.157:0/1180236129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f3006c1f0 0x7f5f300c2300 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000c810 tx=0x7f5f20007df0 comp rx=0 tx=0).stop 2023-10-06T03:59:01.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.614+0000 7f5f354ad700 1 -- 172.21.15.157:0/1180236129 shutdown_connections 2023-10-06T03:59:01.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.614+0000 7f5f354ad700 1 --2- 172.21.15.157:0/1180236129 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5f10066f30 0x7f5f100693e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:01.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.614+0000 7f5f354ad700 1 --2- 172.21.15.157:0/1180236129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f300b2110 0x7f5f3014c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:01.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.614+0000 7f5f354ad700 1 --2- 172.21.15.157:0/1180236129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f3006c1f0 0x7f5f300c2300 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:01.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.614+0000 7f5f354ad700 1 --2- 172.21.15.157:0/1180236129 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f300b1d40 0x7f5f300c1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:01.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.614+0000 7f5f354ad700 1 -- 172.21.15.157:0/1180236129 >> 172.21.15.157:0/1180236129 conn(0x7f5f300cc6e0 msgr2=0x7f5f300cbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:59:01.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.614+0000 7f5f354ad700 1 -- 172.21.15.157:0/1180236129 shutdown_connections 2023-10-06T03:59:01.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:01.614+0000 7f5f354ad700 1 -- 172.21.15.157:0/1180236129 wait complete. 2023-10-06T03:59:01.617 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 16 2023-10-06T03:59:01.628 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-10-06T03:59:03.629 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:59:03.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.758+0000 7fe8e7492700 1 Processor -- start 2023-10-06T03:59:03.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.758+0000 7fe8e7492700 1 -- start start 2023-10-06T03:59:03.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.759+0000 7fe8e7492700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8e006dc40 0x7fe8e014fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:03.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.759+0000 7fe8e7492700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e00c9780 con 0x7fe8e006cbb0 2023-10-06T03:59:03.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.759+0000 7fe8e7492700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e006e0c0 con 0x7fe8e006c1f0 2023-10-06T03:59:03.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.759+0000 7fe8e7492700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e00c8c40 con 0x7fe8e006dc40 2023-10-06T03:59:03.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.759+0000 7fe8e522e700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe8e006cbb0 0x7fe8e006d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:36826/0 (socket says 172.21.15.157:36826) 2023-10-06T03:59:03.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.759+0000 7fe8e522e700 1 -- 172.21.15.157:0/2278452340 learned_addr learned my addr 172.21.15.157:0/2278452340 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:59:03.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.759+0000 7fe8e4a2d700 1 --2- 172.21.15.157:0/2278452340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8e006dc40 0x7fe8e014fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:03.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.759+0000 7fe8d7fff700 1 -- 172.21.15.157:0/2278452340 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 433501422 0 0) 0x7fe8e00c9780 con 0x7fe8e006cbb0 2023-10-06T03:59:03.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.759+0000 7fe8d7fff700 1 -- 172.21.15.157:0/2278452340 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8cc003570 con 0x7fe8e006cbb0 2023-10-06T03:59:03.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.759+0000 7fe8d7fff700 1 -- 172.21.15.157:0/2278452340 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4197801 0 0) 0x7fe8e006e0c0 con 0x7fe8e006c1f0 2023-10-06T03:59:03.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.759+0000 7fe8d7fff700 1 -- 172.21.15.157:0/2278452340 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8e00c9780 con 0x7fe8e006c1f0 2023-10-06T03:59:03.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.759+0000 7fe8d7fff700 1 -- 172.21.15.157:0/2278452340 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1209207853 0 0) 0x7fe8cc003570 con 0x7fe8e006cbb0 2023-10-06T03:59:03.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.759+0000 7fe8d7fff700 1 -- 172.21.15.157:0/2278452340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8e006dc40 msgr2=0x7fe8e014fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:59:03.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.759+0000 7fe8d7fff700 1 --2- 172.21.15.157:0/2278452340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8e006dc40 0x7fe8e014fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:03.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.759+0000 7fe8d7fff700 1 -- 172.21.15.157:0/2278452340 >> v1:172.21.15.175:6789/0 conn(0x7fe8e006c1f0 legacy=0x7fe8e0155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:03.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.759+0000 7fe8d7fff700 1 -- 172.21.15.157:0/2278452340 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e00c8e50 con 0x7fe8e006cbb0 2023-10-06T03:59:03.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.759+0000 7fe8d7fff700 1 -- 172.21.15.157:0/2278452340 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7fe8dc0017f0 con 0x7fe8e006cbb0 2023-10-06T03:59:03.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.760+0000 7fe8d7fff700 1 -- 172.21.15.157:0/2278452340 >> v1:172.21.15.157:6789/0 conn(0x7fe8e006cbb0 legacy=0x7fe8e006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:03.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.760+0000 7fe8d7fff700 1 --2- 172.21.15.157:0/2278452340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8cc003c20 0x7fe8cc004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:03.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.760+0000 7fe8d7fff700 1 --2- 172.21.15.157:0/2278452340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8cc004810 0x7fe8cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:03.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.760+0000 7fe8d7fff700 1 --2- 172.21.15.157:0/2278452340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8cc005180 0x7fe8cc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:03.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.760+0000 7fe8d7fff700 1 -- 172.21.15.157:0/2278452340 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e00c8c40 con 0x7fe8cc005180 2023-10-06T03:59:03.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.760+0000 7fe8d7fff700 1 -- 172.21.15.157:0/2278452340 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc00bfa0 con 0x7fe8cc003c20 2023-10-06T03:59:03.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.760+0000 7fe8d7fff700 1 -- 172.21.15.157:0/2278452340 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc00c0e0 con 0x7fe8cc004810 2023-10-06T03:59:03.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.760+0000 7fe8e522e700 1 --2- 172.21.15.157:0/2278452340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8cc003c20 0x7fe8cc004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:03.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.760+0000 7fe8e4a2d700 1 --2- 172.21.15.157:0/2278452340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8cc004810 0x7fe8cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:03.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.760+0000 7fe8e5a2f700 1 --2- 172.21.15.157:0/2278452340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8cc005180 0x7fe8cc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:03.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.760+0000 7fe8e5a2f700 1 -- 172.21.15.157:0/2278452340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8cc004810 msgr2=0x7fe8cc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:59:03.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.760+0000 7fe8e5a2f700 1 --2- 172.21.15.157:0/2278452340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8cc004810 0x7fe8cc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:03.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.760+0000 7fe8e5a2f700 1 -- 172.21.15.157:0/2278452340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8cc003c20 msgr2=0x7fe8cc004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:59:03.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.760+0000 7fe8e5a2f700 1 --2- 172.21.15.157:0/2278452340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8cc003c20 0x7fe8cc004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:03.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.760+0000 7fe8e5a2f700 1 -- 172.21.15.157:0/2278452340 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8cc003570 con 0x7fe8cc005180 2023-10-06T03:59:03.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.760+0000 7fe8e522e700 1 --2- 172.21.15.157:0/2278452340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8cc003c20 0x7fe8cc004010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:59:03.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.760+0000 7fe8e5a2f700 1 --2- 172.21.15.157:0/2278452340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8cc005180 0x7fe8cc00ba60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8001a40 tx=0x7fe8d8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223cd3a405e2bfd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:03.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.760+0000 7fe8d7fff700 1 -- 172.21.15.157:0/2278452340 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d800c590 con 0x7fe8cc005180 2023-10-06T03:59:03.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.760+0000 7fe8d7fff700 1 -- 172.21.15.157:0/2278452340 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d800c6f0 con 0x7fe8cc005180 2023-10-06T03:59:03.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.760+0000 7fe8d7fff700 1 -- 172.21.15.157:0/2278452340 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d800ca90 con 0x7fe8cc005180 2023-10-06T03:59:03.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.761+0000 7fe8e7492700 1 -- 172.21.15.157:0/2278452340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8cc005180 msgr2=0x7fe8cc00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:03.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.761+0000 7fe8e7492700 1 --2- 172.21.15.157:0/2278452340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8cc005180 0x7fe8cc00ba60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8001a40 tx=0x7fe8d8007d30 comp rx=0 tx=0).stop 2023-10-06T03:59:03.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.761+0000 7fe8e5a2f700 1 -- 172.21.15.157:0/2278452340 reap_dead start 2023-10-06T03:59:03.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.761+0000 7fe8e7492700 1 -- 172.21.15.157:0/2278452340 shutdown_connections 2023-10-06T03:59:03.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.761+0000 7fe8e7492700 1 -- 172.21.15.157:0/2278452340 >> 172.21.15.157:0/2278452340 conn(0x7fe8e006f230 msgr2=0x7fe8e006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:59:03.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.761+0000 7fe8e7492700 1 -- 172.21.15.157:0/2278452340 shutdown_connections 2023-10-06T03:59:03.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.761+0000 7fe8e7492700 1 -- 172.21.15.157:0/2278452340 wait complete. 2023-10-06T03:59:03.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.762+0000 7fe8e7492700 1 Processor -- start 2023-10-06T03:59:03.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.762+0000 7fe8e7492700 1 -- start start 2023-10-06T03:59:03.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.763+0000 7fe8e7492700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8cc005180 0x7fe8e0157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:03.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.763+0000 7fe8e7492700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8e01586e0 0x7fe8e014bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:03.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.763+0000 7fe8e7492700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8e0169a80 0x7fe8e0150930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:03.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.763+0000 7fe8e7492700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e0158530 con 0x7fe8e0169a80 2023-10-06T03:59:03.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.763+0000 7fe8e7492700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e00c2de0 con 0x7fe8e01586e0 2023-10-06T03:59:03.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.763+0000 7fe8e7492700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e0158d40 con 0x7fe8cc005180 2023-10-06T03:59:03.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.763+0000 7fe8e5a2f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8e0169a80 0x7fe8e0150930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:03.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.763+0000 7fe8e5a2f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8e0169a80 0x7fe8e0150930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45562/0 (socket says 172.21.15.157:45562) 2023-10-06T03:59:03.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.763+0000 7fe8e5a2f700 1 -- 172.21.15.157:0/4113878970 learned_addr learned my addr 172.21.15.157:0/4113878970 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:59:03.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.763+0000 7fe8e4a2d700 1 --2- 172.21.15.157:0/4113878970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8e01586e0 0x7fe8e014bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:03.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.763+0000 7fe8e522e700 1 --2- 172.21.15.157:0/4113878970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8cc005180 0x7fe8e0157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:03.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.763+0000 7fe8e5a2f700 1 -- 172.21.15.157:0/4113878970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8cc005180 msgr2=0x7fe8e0157e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:03.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.763+0000 7fe8e5a2f700 1 --2- 172.21.15.157:0/4113878970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8cc005180 0x7fe8e0157e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:03.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.763+0000 7fe8e5a2f700 1 -- 172.21.15.157:0/4113878970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8e01586e0 msgr2=0x7fe8e014bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:03.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.763+0000 7fe8e5a2f700 1 --2- 172.21.15.157:0/4113878970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8e01586e0 0x7fe8e014bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:03.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.763+0000 7fe8e5a2f700 1 -- 172.21.15.157:0/4113878970 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e01695f0 con 0x7fe8e0169a80 2023-10-06T03:59:03.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.764+0000 7fe8e5a2f700 1 --2- 172.21.15.157:0/4113878970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8e0169a80 0x7fe8e0150930 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8002000 tx=0x7fe8d8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:03.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.764+0000 7fe8d57fa700 1 -- 172.21.15.157:0/4113878970 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8000fe0 con 0x7fe8e0169a80 2023-10-06T03:59:03.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.764+0000 7fe8d57fa700 1 -- 172.21.15.157:0/4113878970 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d8001360 con 0x7fe8e0169a80 2023-10-06T03:59:03.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.764+0000 7fe8e7492700 1 -- 172.21.15.157:0/4113878970 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e0153c40 con 0x7fe8e0169a80 2023-10-06T03:59:03.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.764+0000 7fe8d57fa700 1 -- 172.21.15.157:0/4113878970 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d800c860 con 0x7fe8e0169a80 2023-10-06T03:59:03.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.764+0000 7fe8e7492700 1 -- 172.21.15.157:0/4113878970 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e00c2540 con 0x7fe8e0169a80 2023-10-06T03:59:03.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.765+0000 7fe8e7492700 1 -- 172.21.15.157:0/4113878970 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e01693c0 con 0x7fe8e0169a80 2023-10-06T03:59:03.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.765+0000 7fe8d57fa700 1 -- 172.21.15.157:0/4113878970 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe8d8019070 con 0x7fe8e0169a80 2023-10-06T03:59:03.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.765+0000 7fe8d57fa700 1 --2- 172.21.15.157:0/4113878970 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe8c4066f30 0x7fe8c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:03.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.765+0000 7fe8d57fa700 1 -- 172.21.15.157:0/4113878970 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fe8d8085710 con 0x7fe8e0169a80 2023-10-06T03:59:03.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.766+0000 7fe8e522e700 1 --2- 172.21.15.157:0/4113878970 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe8c4066f30 0x7fe8c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:03.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.766+0000 7fe8e522e700 1 --2- 172.21.15.157:0/4113878970 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe8c4066f30 0x7fe8c40693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc001580 tx=0x7fe8dc006040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:03.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.770+0000 7fe8d57fa700 1 -- 172.21.15.157:0/4113878970 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe8e01693c0 con 0x7fe8e0169a80 2023-10-06T03:59:03.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.996+0000 7fe8e7492700 1 -- 172.21.15.157:0/4113878970 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8e01693c0 con 0x7fe8e0169a80 2023-10-06T03:59:03.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.996+0000 7fe8d57fa700 1 -- 172.21.15.157:0/4113878970 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+6184 (secure 0 0 0) 0x7fe8e01693c0 con 0x7fe8e0169a80 2023-10-06T03:59:03.999 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:59:03.999 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T03:57:37.649946+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"f","rank":4,"incarnation":15,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/4177493830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4177493830},{"type":"v1","addr":"172.21.15.175:6839","nonce":4177493830}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:59:04.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.998+0000 7fe8e7492700 1 -- 172.21.15.157:0/4113878970 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe8c4066f30 msgr2=0x7fe8c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:04.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.998+0000 7fe8e7492700 1 --2- 172.21.15.157:0/4113878970 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe8c4066f30 0x7fe8c40693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc001580 tx=0x7fe8dc006040 comp rx=0 tx=0).stop 2023-10-06T03:59:04.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.998+0000 7fe8e7492700 1 -- 172.21.15.157:0/4113878970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8e0169a80 msgr2=0x7fe8e0150930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:04.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.999+0000 7fe8e7492700 1 --2- 172.21.15.157:0/4113878970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8e0169a80 0x7fe8e0150930 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8002000 tx=0x7fe8d8015000 comp rx=0 tx=0).stop 2023-10-06T03:59:04.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.999+0000 7fe8e7492700 1 -- 172.21.15.157:0/4113878970 shutdown_connections 2023-10-06T03:59:04.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.999+0000 7fe8e7492700 1 --2- 172.21.15.157:0/4113878970 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe8c4066f30 0x7fe8c40693e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:04.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.999+0000 7fe8e7492700 1 --2- 172.21.15.157:0/4113878970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8e0169a80 0x7fe8e0150930 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:04.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.999+0000 7fe8e7492700 1 --2- 172.21.15.157:0/4113878970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8cc005180 0x7fe8e0157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:04.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.999+0000 7fe8e7492700 1 --2- 172.21.15.157:0/4113878970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8e01586e0 0x7fe8e014bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:04.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.999+0000 7fe8e7492700 1 -- 172.21.15.157:0/4113878970 >> 172.21.15.157:0/4113878970 conn(0x7fe8e006f230 msgr2=0x7fe8e014c970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:59:04.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.999+0000 7fe8e7492700 1 -- 172.21.15.157:0/4113878970 shutdown_connections 2023-10-06T03:59:04.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:03.999+0000 7fe8e7492700 1 -- 172.21.15.157:0/4113878970 wait complete. 2023-10-06T03:59:04.002 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 16 2023-10-06T03:59:04.012 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-10-06T03:59:06.013 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:59:06.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.140+0000 7f9dc087d700 1 Processor -- start 2023-10-06T03:59:06.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.140+0000 7f9dc087d700 1 -- start start 2023-10-06T03:59:06.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.141+0000 7f9dc087d700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9dbc0b1e40 0x7f9dbc157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:06.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.141+0000 7f9dc087d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9dbc0b04d0 0x7f9dbc06dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:06.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.141+0000 7f9dc087d700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dbc06d8d0 con 0x7f9dbc06c2d0 2023-10-06T03:59:06.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.141+0000 7f9dc087d700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dbc0af8f0 con 0x7f9dbc0b04d0 2023-10-06T03:59:06.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.141+0000 7f9dc087d700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc15a690 con 0x7f9dbc0b1e40 2023-10-06T03:59:06.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.141+0000 7f9db259c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9dbc0b04d0 0x7f9dbc06dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:06.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.141+0000 7f9db259c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9dbc0b04d0 0x7f9dbc06dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45568/0 (socket says 172.21.15.157:45568) 2023-10-06T03:59:06.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.141+0000 7f9db259c700 1 -- 172.21.15.157:0/541727520 learned_addr learned my addr 172.21.15.157:0/541727520 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:59:06.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.141+0000 7f9dbad9d700 1 --2- 172.21.15.157:0/541727520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9dbc0b1e40 0x7f9dbc157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:06.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.141+0000 7f9db259c700 1 -- 172.21.15.157:0/541727520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9dbc0b1e40 msgr2=0x7f9dbc157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:59:06.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.141+0000 7f9db259c700 1 --2- 172.21.15.157:0/541727520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9dbc0b1e40 0x7f9dbc157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:06.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.141+0000 7f9db259c700 1 -- 172.21.15.157:0/541727520 >> v1:172.21.15.175:6789/0 conn(0x7f9dbc06c2d0 legacy=0x7f9dbc1563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:06.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.141+0000 7f9db259c700 1 -- 172.21.15.157:0/541727520 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dbc06ccc0 con 0x7f9dbc0b04d0 2023-10-06T03:59:06.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.142+0000 7f9db259c700 1 --2- 172.21.15.157:0/541727520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dbc0b04d0 0x7f9dbc06dd60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9da40099b0 tx=0x7f9da4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988c69a172a69534 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:06.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.142+0000 7f9dba59c700 1 -- 172.21.15.157:0/541727520 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da4009ea0 con 0x7f9dbc0b04d0 2023-10-06T03:59:06.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.142+0000 7f9dba59c700 1 -- 172.21.15.157:0/541727520 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da400b860 con 0x7f9dbc0b04d0 2023-10-06T03:59:06.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.142+0000 7f9dba59c700 1 -- 172.21.15.157:0/541727520 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da400bc20 con 0x7f9dbc0b04d0 2023-10-06T03:59:06.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.142+0000 7f9dc087d700 1 -- 172.21.15.157:0/541727520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dbc0b04d0 msgr2=0x7f9dbc06dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:06.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.142+0000 7f9dc087d700 1 --2- 172.21.15.157:0/541727520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dbc0b04d0 0x7f9dbc06dd60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9da40099b0 tx=0x7f9da4009250 comp rx=0 tx=0).stop 2023-10-06T03:59:06.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.142+0000 7f9dc087d700 1 -- 172.21.15.157:0/541727520 shutdown_connections 2023-10-06T03:59:06.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.142+0000 7f9dc087d700 1 --2- 172.21.15.157:0/541727520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9dbc0b1e40 0x7f9dbc157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:06.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.142+0000 7f9dc087d700 1 --2- 172.21.15.157:0/541727520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dbc0b04d0 0x7f9dbc06dd60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:06.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.142+0000 7f9dc087d700 1 -- 172.21.15.157:0/541727520 >> 172.21.15.157:0/541727520 conn(0x7f9dbc0cc540 msgr2=0x7f9dbc0cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:59:06.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.142+0000 7f9dc087d700 1 -- 172.21.15.157:0/541727520 shutdown_connections 2023-10-06T03:59:06.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.142+0000 7f9dc087d700 1 -- 172.21.15.157:0/541727520 wait complete. 2023-10-06T03:59:06.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.144+0000 7f9dc087d700 1 Processor -- start 2023-10-06T03:59:06.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.144+0000 7f9dc087d700 1 -- start start 2023-10-06T03:59:06.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.144+0000 7f9dc087d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9dbc06c2d0 0x7f9dbc154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:06.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.144+0000 7f9dc087d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dbc0b04d0 0x7f9dbc166ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:06.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.144+0000 7f9dc087d700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9dbc0b1e40 0x7f9dbc167410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:06.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.144+0000 7f9dc087d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc153b10 con 0x7f9dbc0b04d0 2023-10-06T03:59:06.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.144+0000 7f9dc087d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc152940 con 0x7f9dbc06c2d0 2023-10-06T03:59:06.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.144+0000 7f9dc087d700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc153550 con 0x7f9dbc0b1e40 2023-10-06T03:59:06.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.144+0000 7f9db259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dbc0b04d0 0x7f9dbc166ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:06.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.144+0000 7f9db259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dbc0b04d0 0x7f9dbc166ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45584/0 (socket says 172.21.15.157:45584) 2023-10-06T03:59:06.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.144+0000 7f9db259c700 1 -- 172.21.15.157:0/2152377874 learned_addr learned my addr 172.21.15.157:0/2152377874 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:59:06.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.144+0000 7f9dbad9d700 1 --2- 172.21.15.157:0/2152377874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9dbc06c2d0 0x7f9dbc154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:06.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.144+0000 7f9dbb59e700 1 --2- 172.21.15.157:0/2152377874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9dbc0b1e40 0x7f9dbc167410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:06.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.145+0000 7f9db259c700 1 -- 172.21.15.157:0/2152377874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9dbc0b1e40 msgr2=0x7f9dbc167410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:06.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.145+0000 7f9db259c700 1 --2- 172.21.15.157:0/2152377874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9dbc0b1e40 0x7f9dbc167410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:06.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.145+0000 7f9db259c700 1 -- 172.21.15.157:0/2152377874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9dbc06c2d0 msgr2=0x7f9dbc154580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:06.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.145+0000 7f9db259c700 1 --2- 172.21.15.157:0/2152377874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9dbc06c2d0 0x7f9dbc154580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:06.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.145+0000 7f9db259c700 1 -- 172.21.15.157:0/2152377874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da40094f0 con 0x7f9dbc0b04d0 2023-10-06T03:59:06.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.145+0000 7f9db259c700 1 --2- 172.21.15.157:0/2152377874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dbc0b04d0 0x7f9dbc166ed0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9da4006fd0 tx=0x7f9da4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:06.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.145+0000 7f9db37fe700 1 -- 172.21.15.157:0/2152377874 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da401f070 con 0x7f9dbc0b04d0 2023-10-06T03:59:06.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.145+0000 7f9db37fe700 1 -- 172.21.15.157:0/2152377874 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da4006b10 con 0x7f9dbc0b04d0 2023-10-06T03:59:06.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.145+0000 7f9dc087d700 1 -- 172.21.15.157:0/2152377874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dbc153830 con 0x7f9dbc0b04d0 2023-10-06T03:59:06.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.145+0000 7f9db37fe700 1 -- 172.21.15.157:0/2152377874 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da400bc00 con 0x7f9dbc0b04d0 2023-10-06T03:59:06.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.145+0000 7f9dc087d700 1 -- 172.21.15.157:0/2152377874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dbc152ec0 con 0x7f9dbc0b04d0 2023-10-06T03:59:06.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.146+0000 7f9dc087d700 1 -- 172.21.15.157:0/2152377874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dbc1527a0 con 0x7f9dbc0b04d0 2023-10-06T03:59:06.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.146+0000 7f9db37fe700 1 -- 172.21.15.157:0/2152377874 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9da4011630 con 0x7f9dbc0b04d0 2023-10-06T03:59:06.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.147+0000 7f9db37fe700 1 --2- 172.21.15.157:0/2152377874 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9d9c066f30 0x7f9d9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:06.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.147+0000 7f9db37fe700 1 -- 172.21.15.157:0/2152377874 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f9da40859e0 con 0x7f9dbc0b04d0 2023-10-06T03:59:06.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.147+0000 7f9dbad9d700 1 --2- 172.21.15.157:0/2152377874 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9d9c066f30 0x7f9d9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:06.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.148+0000 7f9dbad9d700 1 --2- 172.21.15.157:0/2152377874 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9d9c066f30 0x7f9d9c0693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9dac009db0 tx=0x7f9dac0091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:06.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.151+0000 7f9db37fe700 1 -- 172.21.15.157:0/2152377874 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9dbc1527a0 con 0x7f9dbc0b04d0 2023-10-06T03:59:06.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.381+0000 7f9dc087d700 1 -- 172.21.15.157:0/2152377874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9dbc1527a0 con 0x7f9dbc0b04d0 2023-10-06T03:59:06.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.381+0000 7f9db37fe700 1 -- 172.21.15.157:0/2152377874 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+6184 (secure 0 0 0) 0x7f9dbc1527a0 con 0x7f9dbc0b04d0 2023-10-06T03:59:06.383 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:59:06.383 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T03:57:37.649946+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4241":{"gid":4241,"name":"f","rank":4,"incarnation":15,"state":"up:active","state_seq":10,"addr":"172.21.15.175:6839/4177493830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":4177493830},{"type":"v1","addr":"172.21.15.175:6839","nonce":4177493830}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:59:06.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.383+0000 7f9dc087d700 1 -- 172.21.15.157:0/2152377874 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9d9c066f30 msgr2=0x7f9d9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:06.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.383+0000 7f9dc087d700 1 --2- 172.21.15.157:0/2152377874 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9d9c066f30 0x7f9d9c0693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9dac009db0 tx=0x7f9dac0091a0 comp rx=0 tx=0).stop 2023-10-06T03:59:06.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.383+0000 7f9dc087d700 1 -- 172.21.15.157:0/2152377874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dbc0b04d0 msgr2=0x7f9dbc166ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:06.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.383+0000 7f9dc087d700 1 --2- 172.21.15.157:0/2152377874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dbc0b04d0 0x7f9dbc166ed0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9da4006fd0 tx=0x7f9da4017000 comp rx=0 tx=0).stop 2023-10-06T03:59:06.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.384+0000 7f9dc087d700 1 -- 172.21.15.157:0/2152377874 shutdown_connections 2023-10-06T03:59:06.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.384+0000 7f9dc087d700 1 --2- 172.21.15.157:0/2152377874 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9d9c066f30 0x7f9d9c0693e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:06.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.384+0000 7f9dc087d700 1 --2- 172.21.15.157:0/2152377874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9dbc06c2d0 0x7f9dbc154580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:06.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.384+0000 7f9dc087d700 1 --2- 172.21.15.157:0/2152377874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9dbc0b1e40 0x7f9dbc167410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:06.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.384+0000 7f9dc087d700 1 --2- 172.21.15.157:0/2152377874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dbc0b04d0 0x7f9dbc166ed0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:06.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.384+0000 7f9dc087d700 1 -- 172.21.15.157:0/2152377874 >> 172.21.15.157:0/2152377874 conn(0x7f9dbc0cc540 msgr2=0x7f9dbc0afac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:59:06.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.384+0000 7f9dc087d700 1 -- 172.21.15.157:0/2152377874 shutdown_connections 2023-10-06T03:59:06.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:06.384+0000 7f9dc087d700 1 -- 172.21.15.157:0/2152377874 wait complete. 2023-10-06T03:59:06.387 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 16 2023-10-06T03:59:06.398 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-10-06T03:59:08.399 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:59:08.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.524+0000 7faec47dd700 1 Processor -- start 2023-10-06T03:59:08.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.525+0000 7faec47dd700 1 -- start start 2023-10-06T03:59:08.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.525+0000 7faec47dd700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7faebc06cbb0 0x7faebc06d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:08.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.525+0000 7faec47dd700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faebc06c1f0 0x7faebc06da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:08.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.525+0000 7faec47dd700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faebc0c9780 con 0x7faebc06cf80 2023-10-06T03:59:08.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.525+0000 7faec47dd700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faebc0c8c40 con 0x7faebc06cbb0 2023-10-06T03:59:08.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.525+0000 7faec47dd700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faebc06e0c0 con 0x7faebc06c1f0 2023-10-06T03:59:08.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.526+0000 7faec1d78700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faebc06c1f0 0x7faebc06da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:08.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.526+0000 7faec2579700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7faebc06cbb0 0x7faebc06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:08.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.526+0000 7faec2d7a700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7faebc06cf80 0x7faebc14fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.157:40880/0 (socket says 172.21.15.157:40880) 2023-10-06T03:59:08.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.526+0000 7faec1d78700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faebc06c1f0 0x7faebc06da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:50744/0 (socket says 172.21.15.157:50744) 2023-10-06T03:59:08.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.526+0000 7faec2579700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7faebc06cbb0 0x7faebc06d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:50900/0 (socket says 172.21.15.157:50900) 2023-10-06T03:59:08.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.526+0000 7faec2d7a700 1 -- 172.21.15.157:0/6835355 learned_addr learned my addr 172.21.15.157:0/6835355 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:59:08.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.526+0000 7faec1d78700 1 -- 172.21.15.157:0/6835355 >> v2:172.21.15.175:3300/0 conn(0x7faebc06cbb0 msgr2=0x7faebc06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:59:08.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.526+0000 7faec1d78700 1 --2- 172.21.15.157:0/6835355 >> v2:172.21.15.175:3300/0 conn(0x7faebc06cbb0 0x7faebc06d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:08.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.526+0000 7faec1d78700 1 -- 172.21.15.157:0/6835355 >> v1:172.21.15.175:6789/0 conn(0x7faebc06cf80 legacy=0x7faebc14fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:08.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.526+0000 7faec1d78700 1 -- 172.21.15.157:0/6835355 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faebc0c8e50 con 0x7faebc06c1f0 2023-10-06T03:59:08.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.526+0000 7faec2579700 1 --2- 172.21.15.157:0/6835355 >> v2:172.21.15.175:3300/0 conn(0x7faebc06cbb0 0x7faebc06d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:59:08.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.526+0000 7faec1d78700 1 --2- 172.21.15.157:0/6835355 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faebc06c1f0 0x7faebc06da70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faeac009910 tx=0x7faeac0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca1034e6b3c49de2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:08.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.526+0000 7faec1577700 1 -- 172.21.15.157:0/6835355 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac00b6d0 con 0x7faebc06c1f0 2023-10-06T03:59:08.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.527+0000 7faec1577700 1 -- 172.21.15.157:0/6835355 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeac00b830 con 0x7faebc06c1f0 2023-10-06T03:59:08.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.527+0000 7faec1577700 1 -- 172.21.15.157:0/6835355 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac00bbf0 con 0x7faebc06c1f0 2023-10-06T03:59:08.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.527+0000 7faec47dd700 1 -- 172.21.15.157:0/6835355 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faebc06c1f0 msgr2=0x7faebc06da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:08.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.527+0000 7faec47dd700 1 --2- 172.21.15.157:0/6835355 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faebc06c1f0 0x7faebc06da70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faeac009910 tx=0x7faeac0091a0 comp rx=0 tx=0).stop 2023-10-06T03:59:08.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.527+0000 7faec47dd700 1 -- 172.21.15.157:0/6835355 shutdown_connections 2023-10-06T03:59:08.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.527+0000 7faec47dd700 1 --2- 172.21.15.157:0/6835355 >> v2:172.21.15.175:3300/0 conn(0x7faebc06cbb0 0x7faebc06d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:08.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.527+0000 7faec47dd700 1 --2- 172.21.15.157:0/6835355 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faebc06c1f0 0x7faebc06da70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:08.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.527+0000 7faec47dd700 1 -- 172.21.15.157:0/6835355 >> 172.21.15.157:0/6835355 conn(0x7faebc06f230 msgr2=0x7faebc06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:59:08.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.527+0000 7faec47dd700 1 -- 172.21.15.157:0/6835355 shutdown_connections 2023-10-06T03:59:08.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.527+0000 7faec47dd700 1 -- 172.21.15.157:0/6835355 wait complete. 2023-10-06T03:59:08.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.528+0000 7faec47dd700 1 Processor -- start 2023-10-06T03:59:08.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.528+0000 7faec47dd700 1 -- start start 2023-10-06T03:59:08.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.529+0000 7faec47dd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faebc06c1f0 0x7faebc155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:08.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.529+0000 7faec47dd700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faebc06cbb0 0x7faebc1569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:08.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.529+0000 7faec47dd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faebc06cf80 0x7faebc0bfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:08.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.529+0000 7faec47dd700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faebc0c16e0 con 0x7faebc06cf80 2023-10-06T03:59:08.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.529+0000 7faec47dd700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faebc0c01b0 con 0x7faebc06c1f0 2023-10-06T03:59:08.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.529+0000 7faec47dd700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faebc166b60 con 0x7faebc06cbb0 2023-10-06T03:59:08.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.529+0000 7faec2d7a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faebc06cf80 0x7faebc0bfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:08.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.529+0000 7faec2d7a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faebc06cf80 0x7faebc0bfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45598/0 (socket says 172.21.15.157:45598) 2023-10-06T03:59:08.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.529+0000 7faec2d7a700 1 -- 172.21.15.157:0/227379412 learned_addr learned my addr 172.21.15.157:0/227379412 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:59:08.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.529+0000 7faec1d78700 1 --2- 172.21.15.157:0/227379412 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faebc06cbb0 0x7faebc1569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:08.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.529+0000 7faec2579700 1 --2- 172.21.15.157:0/227379412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faebc06c1f0 0x7faebc155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:08.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.529+0000 7faec2d7a700 1 -- 172.21.15.157:0/227379412 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faebc06cbb0 msgr2=0x7faebc1569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:08.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.529+0000 7faec2d7a700 1 --2- 172.21.15.157:0/227379412 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faebc06cbb0 0x7faebc1569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:08.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.529+0000 7faec2d7a700 1 -- 172.21.15.157:0/227379412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faebc06c1f0 msgr2=0x7faebc155a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:08.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.529+0000 7faec2d7a700 1 --2- 172.21.15.157:0/227379412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faebc06c1f0 0x7faebc155a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:08.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.529+0000 7faec2d7a700 1 -- 172.21.15.157:0/227379412 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb8009420 con 0x7faebc06cf80 2023-10-06T03:59:08.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.529+0000 7faec2d7a700 1 --2- 172.21.15.157:0/227379412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faebc06cf80 0x7faebc0bfc70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7faeb400cbf0 tx=0x7faeb40019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:08.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.530+0000 7faeb27fc700 1 -- 172.21.15.157:0/227379412 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb400a870 con 0x7faebc06cf80 2023-10-06T03:59:08.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.530+0000 7faeb27fc700 1 -- 172.21.15.157:0/227379412 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb400a9d0 con 0x7faebc06cf80 2023-10-06T03:59:08.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.530+0000 7faec47dd700 1 -- 172.21.15.157:0/227379412 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeac009420 con 0x7faebc06cf80 2023-10-06T03:59:08.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.530+0000 7faeb27fc700 1 -- 172.21.15.157:0/227379412 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb400ad50 con 0x7faebc06cf80 2023-10-06T03:59:08.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.530+0000 7faec47dd700 1 -- 172.21.15.157:0/227379412 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faebc0c0db0 con 0x7faebc06cf80 2023-10-06T03:59:08.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.531+0000 7faec47dd700 1 -- 172.21.15.157:0/227379412 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faebc0c1890 con 0x7faebc06cf80 2023-10-06T03:59:08.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.531+0000 7faeb27fc700 1 -- 172.21.15.157:0/227379412 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faeb4007300 con 0x7faebc06cf80 2023-10-06T03:59:08.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.531+0000 7faeb27fc700 1 --2- 172.21.15.157:0/227379412 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faea0066f30 0x7faea00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:08.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.531+0000 7faeb27fc700 1 -- 172.21.15.157:0/227379412 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6202+0+0 (secure 0 0 0) 0x7faeb4084a60 con 0x7faebc06cf80 2023-10-06T03:59:08.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.532+0000 7faec2579700 1 --2- 172.21.15.157:0/227379412 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faea0066f30 0x7faea00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:08.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.532+0000 7faec2579700 1 --2- 172.21.15.157:0/227379412 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faea0066f30 0x7faea00693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faeb8009f40 tx=0x7faeb80091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:08.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.535+0000 7faeb27fc700 1 -- 172.21.15.157:0/227379412 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faebc0c1890 con 0x7faebc06cf80 2023-10-06T03:59:08.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.904+0000 7faec47dd700 1 -- 172.21.15.157:0/227379412 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faebc0c1890 con 0x7faebc06cf80 2023-10-06T03:59:08.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.904+0000 7faeb27fc700 1 -- 172.21.15.157:0/227379412 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+5426 (secure 0 0 0) 0x7faebc0c1890 con 0x7faebc06cf80 2023-10-06T03:59:08.906 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:59:08.906 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T03:59:06.641696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4223":{"gid":4223,"name":"a","rank":4,"incarnation":17,"state":"up:replay","state_seq":1,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:59:08.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.906+0000 7faec47dd700 1 -- 172.21.15.157:0/227379412 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faea0066f30 msgr2=0x7faea00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:08.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.906+0000 7faec47dd700 1 --2- 172.21.15.157:0/227379412 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faea0066f30 0x7faea00693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faeb8009f40 tx=0x7faeb80091a0 comp rx=0 tx=0).stop 2023-10-06T03:59:08.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.906+0000 7faec47dd700 1 -- 172.21.15.157:0/227379412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faebc06cf80 msgr2=0x7faebc0bfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:08.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.906+0000 7faec47dd700 1 --2- 172.21.15.157:0/227379412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faebc06cf80 0x7faebc0bfc70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7faeb400cbf0 tx=0x7faeb40019c0 comp rx=0 tx=0).stop 2023-10-06T03:59:08.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.906+0000 7faec47dd700 1 -- 172.21.15.157:0/227379412 shutdown_connections 2023-10-06T03:59:08.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.906+0000 7faec47dd700 1 --2- 172.21.15.157:0/227379412 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faea0066f30 0x7faea00693e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:08.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.906+0000 7faec47dd700 1 --2- 172.21.15.157:0/227379412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faebc06c1f0 0x7faebc155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:08.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.906+0000 7faec47dd700 1 --2- 172.21.15.157:0/227379412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faebc06cf80 0x7faebc0bfc70 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:08.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.906+0000 7faec47dd700 1 --2- 172.21.15.157:0/227379412 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faebc06cbb0 0x7faebc1569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:08.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.906+0000 7faec47dd700 1 -- 172.21.15.157:0/227379412 >> 172.21.15.157:0/227379412 conn(0x7faebc06f230 msgr2=0x7faebc14e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:59:08.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.907+0000 7faec47dd700 1 -- 172.21.15.157:0/227379412 shutdown_connections 2023-10-06T03:59:08.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:08.907+0000 7faec47dd700 1 -- 172.21.15.157:0/227379412 wait complete. 2023-10-06T03:59:08.911 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 17 2023-10-06T03:59:08.920 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f down, removed from mdsmap 2023-10-06T03:59:08.920 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-10-06T03:59:08.920 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:59:09.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.034+0000 7fd463ae5700 1 Processor -- start 2023-10-06T03:59:09.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.035+0000 7fd463ae5700 1 -- start start 2023-10-06T03:59:09.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.035+0000 7fd463ae5700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd45c14dcd0 0x7fd45c153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:09.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.035+0000 7fd463ae5700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd45c14e0a0 0x7fd45c0c18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:09.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.035+0000 7fd463ae5700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd45c0c5770 con 0x7fd45c0c5c00 2023-10-06T03:59:09.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.035+0000 7fd463ae5700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd45c154c80 con 0x7fd45c14dcd0 2023-10-06T03:59:09.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.035+0000 7fd463ae5700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd45c154200 con 0x7fd45c14e0a0 2023-10-06T03:59:09.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.035+0000 7fd461881700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd45c14dcd0 0x7fd45c153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:09.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.035+0000 7fd462082700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd45c0c5c00 0x7fd45c0ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:36834/0 (socket says 172.21.15.157:36834) 2023-10-06T03:59:09.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.035+0000 7fd462082700 1 -- 172.21.15.157:0/304001580 learned_addr learned my addr 172.21.15.157:0/304001580 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:59:09.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.035+0000 7fd461080700 1 --2- 172.21.15.157:0/304001580 >> v2:172.21.15.175:3300/0 conn(0x7fd45c14e0a0 0x7fd45c0c18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:09.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.036+0000 7fd461881700 1 -- 172.21.15.157:0/304001580 >> v2:172.21.15.175:3300/0 conn(0x7fd45c14e0a0 msgr2=0x7fd45c0c18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:59:09.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.036+0000 7fd461881700 1 --2- 172.21.15.157:0/304001580 >> v2:172.21.15.175:3300/0 conn(0x7fd45c14e0a0 0x7fd45c0c18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:09.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.036+0000 7fd461881700 1 -- 172.21.15.157:0/304001580 >> v1:172.21.15.157:6789/0 conn(0x7fd45c0c5c00 legacy=0x7fd45c0ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:09.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.036+0000 7fd461881700 1 -- 172.21.15.157:0/304001580 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd45c1543e0 con 0x7fd45c14dcd0 2023-10-06T03:59:09.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.036+0000 7fd46087f700 1 -- 172.21.15.157:0/304001580 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1414423776 0 0) 0x7fd45c0c5770 con 0x7fd45c0c5c00 2023-10-06T03:59:09.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.036+0000 7fd461881700 1 --2- 172.21.15.157:0/304001580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd45c14dcd0 0x7fd45c153cc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0c4920 tx=0x7fd458009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5eaf2cdd0307bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:09.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.036+0000 7fd46087f700 1 -- 172.21.15.157:0/304001580 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45800e070 con 0x7fd45c14dcd0 2023-10-06T03:59:09.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.036+0000 7fd46087f700 1 -- 172.21.15.157:0/304001580 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45800bbf0 con 0x7fd45c14dcd0 2023-10-06T03:59:09.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.036+0000 7fd46087f700 1 -- 172.21.15.157:0/304001580 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd458012430 con 0x7fd45c14dcd0 2023-10-06T03:59:09.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.037+0000 7fd463ae5700 1 -- 172.21.15.157:0/304001580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd45c14dcd0 msgr2=0x7fd45c153cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:09.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.037+0000 7fd463ae5700 1 --2- 172.21.15.157:0/304001580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd45c14dcd0 0x7fd45c153cc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0c4920 tx=0x7fd458009780 comp rx=0 tx=0).stop 2023-10-06T03:59:09.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.037+0000 7fd463ae5700 1 -- 172.21.15.157:0/304001580 shutdown_connections 2023-10-06T03:59:09.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.037+0000 7fd463ae5700 1 --2- 172.21.15.157:0/304001580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd45c14dcd0 0x7fd45c153cc0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:09.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.037+0000 7fd463ae5700 1 --2- 172.21.15.157:0/304001580 >> v2:172.21.15.175:3300/0 conn(0x7fd45c14e0a0 0x7fd45c0c18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:09.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.037+0000 7fd463ae5700 1 -- 172.21.15.157:0/304001580 >> 172.21.15.157:0/304001580 conn(0x7fd45c0692b0 msgr2=0x7fd45c0696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:59:09.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.037+0000 7fd463ae5700 1 -- 172.21.15.157:0/304001580 shutdown_connections 2023-10-06T03:59:09.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.037+0000 7fd463ae5700 1 -- 172.21.15.157:0/304001580 wait complete. 2023-10-06T03:59:09.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.038+0000 7fd463ae5700 1 Processor -- start 2023-10-06T03:59:09.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.038+0000 7fd463ae5700 1 -- start start 2023-10-06T03:59:09.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.038+0000 7fd463ae5700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd45c0c5c00 0x7fd45c14b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:09.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.038+0000 7fd463ae5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd45c14dcd0 0x7fd45c164230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:09.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.038+0000 7fd463ae5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd45c14e0a0 0x7fd45c164770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:09.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.038+0000 7fd463ae5700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c153a90 con 0x7fd45c14dcd0 2023-10-06T03:59:09.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.038+0000 7fd463ae5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c163da0 con 0x7fd45c14e0a0 2023-10-06T03:59:09.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.038+0000 7fd463ae5700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c167e30 con 0x7fd45c0c5c00 2023-10-06T03:59:09.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.039+0000 7fd461080700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd45c14dcd0 0x7fd45c164230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:09.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.039+0000 7fd461080700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd45c14dcd0 0x7fd45c164230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45630/0 (socket says 172.21.15.157:45630) 2023-10-06T03:59:09.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.039+0000 7fd461080700 1 -- 172.21.15.157:0/103280752 learned_addr learned my addr 172.21.15.157:0/103280752 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:59:09.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.039+0000 7fd461881700 1 --2- 172.21.15.157:0/103280752 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd45c0c5c00 0x7fd45c14b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:09.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.039+0000 7fd462082700 1 --2- 172.21.15.157:0/103280752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd45c14e0a0 0x7fd45c164770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:09.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.039+0000 7fd461080700 1 -- 172.21.15.157:0/103280752 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd45c0c5c00 msgr2=0x7fd45c14b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:09.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.039+0000 7fd461080700 1 --2- 172.21.15.157:0/103280752 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd45c0c5c00 0x7fd45c14b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:09.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.039+0000 7fd461080700 1 -- 172.21.15.157:0/103280752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd45c14e0a0 msgr2=0x7fd45c164770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:09.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.039+0000 7fd461080700 1 --2- 172.21.15.157:0/103280752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd45c14e0a0 0x7fd45c164770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:09.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.039+0000 7fd461080700 1 -- 172.21.15.157:0/103280752 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd458009a20 con 0x7fd45c14dcd0 2023-10-06T03:59:09.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.039+0000 7fd461080700 1 --2- 172.21.15.157:0/103280752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd45c14dcd0 0x7fd45c164230 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00e930 tx=0x7fd44c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:09.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.039+0000 7fd451ffb700 1 -- 172.21.15.157:0/103280752 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c00eed0 con 0x7fd45c14dcd0 2023-10-06T03:59:09.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.039+0000 7fd451ffb700 1 -- 172.21.15.157:0/103280752 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44c00c850 con 0x7fd45c14dcd0 2023-10-06T03:59:09.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.039+0000 7fd463ae5700 1 -- 172.21.15.157:0/103280752 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd45c153ce0 con 0x7fd45c14dcd0 2023-10-06T03:59:09.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.039+0000 7fd451ffb700 1 -- 172.21.15.157:0/103280752 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c00cbf0 con 0x7fd45c14dcd0 2023-10-06T03:59:09.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.039+0000 7fd463ae5700 1 -- 172.21.15.157:0/103280752 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd45c167b40 con 0x7fd45c14dcd0 2023-10-06T03:59:09.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.040+0000 7fd463ae5700 1 -- 172.21.15.157:0/103280752 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd45c163f10 con 0x7fd45c14dcd0 2023-10-06T03:59:09.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.041+0000 7fd451ffb700 1 -- 172.21.15.157:0/103280752 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd44c013470 con 0x7fd45c14dcd0 2023-10-06T03:59:09.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.041+0000 7fd451ffb700 1 --2- 172.21.15.157:0/103280752 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd440066f30 0x7fd4400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:09.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.041+0000 7fd451ffb700 1 -- 172.21.15.157:0/103280752 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6202+0+0 (secure 0 0 0) 0x7fd44c084ca0 con 0x7fd45c14dcd0 2023-10-06T03:59:09.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.041+0000 7fd461881700 1 --2- 172.21.15.157:0/103280752 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd440066f30 0x7fd4400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:09.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.041+0000 7fd461881700 1 --2- 172.21.15.157:0/103280752 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd440066f30 0x7fd4400693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd4580097c0 tx=0x7fd458019000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:09.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.045+0000 7fd451ffb700 1 -- 172.21.15.157:0/103280752 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd44c051d20 con 0x7fd45c14dcd0 2023-10-06T03:59:09.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.275+0000 7fd463ae5700 1 -- 172.21.15.157:0/103280752 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd45c04bb40 con 0x7fd45c14dcd0 2023-10-06T03:59:09.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.276+0000 7fd451ffb700 1 -- 172.21.15.157:0/103280752 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+5426 (secure 0 0 0) 0x7fd44c017090 con 0x7fd45c14dcd0 2023-10-06T03:59:09.277 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:59:09.277 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T03:59:06.641696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4223":{"gid":4223,"name":"a","rank":4,"incarnation":17,"state":"up:replay","state_seq":1,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:59:09.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.278+0000 7fd463ae5700 1 -- 172.21.15.157:0/103280752 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd440066f30 msgr2=0x7fd4400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:09.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.278+0000 7fd463ae5700 1 --2- 172.21.15.157:0/103280752 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd440066f30 0x7fd4400693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd4580097c0 tx=0x7fd458019000 comp rx=0 tx=0).stop 2023-10-06T03:59:09.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.278+0000 7fd463ae5700 1 -- 172.21.15.157:0/103280752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd45c14dcd0 msgr2=0x7fd45c164230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:09.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.278+0000 7fd463ae5700 1 --2- 172.21.15.157:0/103280752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd45c14dcd0 0x7fd45c164230 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00e930 tx=0x7fd44c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:59:09.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.278+0000 7fd463ae5700 1 -- 172.21.15.157:0/103280752 shutdown_connections 2023-10-06T03:59:09.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.278+0000 7fd463ae5700 1 --2- 172.21.15.157:0/103280752 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd440066f30 0x7fd4400693e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:09.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.278+0000 7fd463ae5700 1 --2- 172.21.15.157:0/103280752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd45c14e0a0 0x7fd45c164770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:09.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.278+0000 7fd463ae5700 1 --2- 172.21.15.157:0/103280752 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd45c0c5c00 0x7fd45c14b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:09.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.278+0000 7fd463ae5700 1 --2- 172.21.15.157:0/103280752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd45c14dcd0 0x7fd45c164230 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:09.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.278+0000 7fd463ae5700 1 -- 172.21.15.157:0/103280752 >> 172.21.15.157:0/103280752 conn(0x7fd45c0692b0 msgr2=0x7fd45c14c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:59:09.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.278+0000 7fd463ae5700 1 -- 172.21.15.157:0/103280752 shutdown_connections 2023-10-06T03:59:09.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:09.278+0000 7fd463ae5700 1 -- 172.21.15.157:0/103280752 wait complete. 2023-10-06T03:59:09.281 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 17 2023-10-06T03:59:09.292 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 17, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-06T03:57:10.718782+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 17, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0, 1, 2, 3, 4 ], "info": { "gid_4223": { "addr": "172.21.15.157:6835/3932336008", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6833", "nonce": 3932336008, "type": "v2" }, { "addr": "172.21.15.157:6835", "nonce": 3932336008, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4223, "incarnation": 17, "join_fscid": -1, "name": "a", "rank": 4, "state": "up:replay", "state_seq": 1 }, "gid_4270": { "addr": "172.21.15.175:6835/1596042861", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6834", "nonce": 1596042861, "type": "v2" }, { "addr": "172.21.15.175:6835", "nonce": 1596042861, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4270, "incarnation": 13, "join_fscid": -1, "name": "b", "rank": 3, "state": "up:active", "state_seq": 8 }, "gid_4279": { "addr": "172.21.15.175:6837/1002404870", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6836", "nonce": 1002404870, "type": "v2" }, { "addr": "172.21.15.175:6837", "nonce": 1002404870, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4279, "incarnation": 11, "join_fscid": -1, "name": "d", "rank": 2, "state": "up:active", "state_seq": 7 }, "gid_4287": { "addr": "172.21.15.157:6834/1179753185", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6832", "nonce": 1179753185, "type": "v2" }, { "addr": "172.21.15.157:6834", "nonce": 1179753185, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4287, "incarnation": 9, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4296": { "addr": "172.21.15.157:6837/4188652695", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6836", "nonce": 4188652695, "type": "v2" }, { "addr": "172.21.15.157:6837", "nonce": 4188652695, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4296, "incarnation": 4, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 26, "max_file_size": 1099511627776, "max_mds": 5, "metadata_pool": 3, "modified": "2023-10-06T03:59:06.641696+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4296, "mds_1": 4287, "mds_2": 4279, "mds_3": 4270, "mds_4": 4223 } } } ], "standbys": [] } 2023-10-06T03:59:11.294 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:59:11.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.422+0000 7f77f37c5700 1 Processor -- start 2023-10-06T03:59:11.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.422+0000 7f77f37c5700 1 -- start start 2023-10-06T03:59:11.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.423+0000 7f77f37c5700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77ec0ca630 0x7f77ec0c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:11.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.423+0000 7f77f37c5700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f77ec068920 0x7f77ec156840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:11.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.423+0000 7f77f37c5700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f77ec0c52e0 0x7f77ec156d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:11.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.423+0000 7f77f37c5700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77ec06d530 con 0x7f77ec0c52e0 2023-10-06T03:59:11.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.423+0000 7f77f37c5700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77ec06d9c0 con 0x7f77ec068920 2023-10-06T03:59:11.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.423+0000 7f77f37c5700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77ec06f0c0 con 0x7f77ec0ca630 2023-10-06T03:59:11.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.423+0000 7f77f1d62700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f77ec0c52e0 0x7f77ec156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:11.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.423+0000 7f77f1d62700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f77ec0c52e0 0x7f77ec156d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45632/0 (socket says 172.21.15.157:45632) 2023-10-06T03:59:11.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.423+0000 7f77f1d62700 1 -- 172.21.15.157:0/957378879 learned_addr learned my addr 172.21.15.157:0/957378879 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:59:11.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.424+0000 7f77f0d60700 1 --2- 172.21.15.157:0/957378879 >> v2:172.21.15.175:3300/0 conn(0x7f77ec068920 0x7f77ec156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:11.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.424+0000 7f77f1561700 1 --2- 172.21.15.157:0/957378879 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77ec0ca630 0x7f77ec0c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:11.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.424+0000 7f77f1d62700 1 -- 172.21.15.157:0/957378879 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77ec0ca630 msgr2=0x7f77ec0c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:59:11.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.424+0000 7f77f1d62700 1 --2- 172.21.15.157:0/957378879 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77ec0ca630 0x7f77ec0c6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:11.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.424+0000 7f77f1d62700 1 -- 172.21.15.157:0/957378879 >> v2:172.21.15.175:3300/0 conn(0x7f77ec068920 msgr2=0x7f77ec156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:59:11.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.424+0000 7f77f1d62700 1 --2- 172.21.15.157:0/957378879 >> v2:172.21.15.175:3300/0 conn(0x7f77ec068920 0x7f77ec156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:11.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.424+0000 7f77f1d62700 1 -- 172.21.15.157:0/957378879 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ec06f3a0 con 0x7f77ec0c52e0 2023-10-06T03:59:11.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.424+0000 7f77f1d62700 1 --2- 172.21.15.157:0/957378879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77ec0c52e0 0x7f77ec156d80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f77ec06cd50 tx=0x7f77e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f6cd58fc6e47fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:11.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.424+0000 7f77e3fff700 1 -- 172.21.15.157:0/957378879 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e400b860 con 0x7f77ec0c52e0 2023-10-06T03:59:11.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.424+0000 7f77e3fff700 1 -- 172.21.15.157:0/957378879 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e400b9c0 con 0x7f77ec0c52e0 2023-10-06T03:59:11.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.424+0000 7f77e3fff700 1 -- 172.21.15.157:0/957378879 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e400bd60 con 0x7f77ec0c52e0 2023-10-06T03:59:11.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.424+0000 7f77f37c5700 1 -- 172.21.15.157:0/957378879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77ec0c52e0 msgr2=0x7f77ec156d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:11.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.424+0000 7f77f37c5700 1 --2- 172.21.15.157:0/957378879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77ec0c52e0 0x7f77ec156d80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f77ec06cd50 tx=0x7f77e4009250 comp rx=0 tx=0).stop 2023-10-06T03:59:11.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.425+0000 7f77f37c5700 1 -- 172.21.15.157:0/957378879 shutdown_connections 2023-10-06T03:59:11.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.425+0000 7f77f37c5700 1 --2- 172.21.15.157:0/957378879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77ec0c52e0 0x7f77ec156d80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:11.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.425+0000 7f77f37c5700 1 --2- 172.21.15.157:0/957378879 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77ec0ca630 0x7f77ec0c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:11.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.425+0000 7f77f37c5700 1 --2- 172.21.15.157:0/957378879 >> v2:172.21.15.175:3300/0 conn(0x7f77ec068920 0x7f77ec156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:11.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.425+0000 7f77f37c5700 1 -- 172.21.15.157:0/957378879 >> 172.21.15.157:0/957378879 conn(0x7f77ec0692b0 msgr2=0x7f77ec0696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:59:11.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.425+0000 7f77f37c5700 1 -- 172.21.15.157:0/957378879 shutdown_connections 2023-10-06T03:59:11.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.425+0000 7f77f37c5700 1 -- 172.21.15.157:0/957378879 wait complete. 2023-10-06T03:59:11.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.426+0000 7f77f37c5700 1 Processor -- start 2023-10-06T03:59:11.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.426+0000 7f77f37c5700 1 -- start start 2023-10-06T03:59:11.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.427+0000 7f77f37c5700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77ec068920 0x7f77ec151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:11.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.427+0000 7f77f37c5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77ec0c52e0 0x7f77ec167610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:11.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.427+0000 7f77f37c5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77ec0ca630 0x7f77ec167b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:11.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.427+0000 7f77f37c5700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ec15af60 con 0x7f77ec0c52e0 2023-10-06T03:59:11.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.427+0000 7f77f37c5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ec0c7470 con 0x7f77ec0ca630 2023-10-06T03:59:11.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.427+0000 7f77f37c5700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77ec166e30 con 0x7f77ec068920 2023-10-06T03:59:11.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.427+0000 7f77f0d60700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77ec0c52e0 0x7f77ec167610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:11.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.427+0000 7f77f0d60700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77ec0c52e0 0x7f77ec167610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45644/0 (socket says 172.21.15.157:45644) 2023-10-06T03:59:11.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.427+0000 7f77f0d60700 1 -- 172.21.15.157:0/1612846604 learned_addr learned my addr 172.21.15.157:0/1612846604 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:59:11.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.427+0000 7f77f1d62700 1 --2- 172.21.15.157:0/1612846604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77ec0ca630 0x7f77ec167b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:11.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.427+0000 7f77f1561700 1 --2- 172.21.15.157:0/1612846604 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77ec068920 0x7f77ec151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:11.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.427+0000 7f77f0d60700 1 -- 172.21.15.157:0/1612846604 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77ec068920 msgr2=0x7f77ec151320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:11.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.427+0000 7f77f0d60700 1 --2- 172.21.15.157:0/1612846604 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77ec068920 0x7f77ec151320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:11.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.427+0000 7f77f0d60700 1 -- 172.21.15.157:0/1612846604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77ec0ca630 msgr2=0x7f77ec167b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:11.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.427+0000 7f77f0d60700 1 --2- 172.21.15.157:0/1612846604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77ec0ca630 0x7f77ec167b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:11.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.427+0000 7f77f0d60700 1 -- 172.21.15.157:0/1612846604 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e40094f0 con 0x7f77ec0c52e0 2023-10-06T03:59:11.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.427+0000 7f77f1d62700 1 --2- 172.21.15.157:0/1612846604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77ec0ca630 0x7f77ec167b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:59:11.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.427+0000 7f77f0d60700 1 --2- 172.21.15.157:0/1612846604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77ec0c52e0 0x7f77ec167610 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00d890 tx=0x7f77dc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:11.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.427+0000 7f77e17fa700 1 -- 172.21.15.157:0/1612846604 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc00b600 con 0x7f77ec0c52e0 2023-10-06T03:59:11.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.427+0000 7f77e17fa700 1 -- 172.21.15.157:0/1612846604 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77dc00b760 con 0x7f77ec0c52e0 2023-10-06T03:59:11.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.427+0000 7f77f37c5700 1 -- 172.21.15.157:0/1612846604 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77ec0c43e0 con 0x7f77ec0c52e0 2023-10-06T03:59:11.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.427+0000 7f77e17fa700 1 -- 172.21.15.157:0/1612846604 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc00bae0 con 0x7f77ec0c52e0 2023-10-06T03:59:11.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.427+0000 7f77f37c5700 1 -- 172.21.15.157:0/1612846604 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77ec1670f0 con 0x7f77ec0c52e0 2023-10-06T03:59:11.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.428+0000 7f77f37c5700 1 -- 172.21.15.157:0/1612846604 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77ec06a8f0 con 0x7f77ec0c52e0 2023-10-06T03:59:11.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.429+0000 7f77e17fa700 1 -- 172.21.15.157:0/1612846604 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f77dc01e460 con 0x7f77ec0c52e0 2023-10-06T03:59:11.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.429+0000 7f77e17fa700 1 --2- 172.21.15.157:0/1612846604 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f77d0066f30 0x7f77d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:11.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.429+0000 7f77e17fa700 1 -- 172.21.15.157:0/1612846604 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f77dc084de0 con 0x7f77ec0c52e0 2023-10-06T03:59:11.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.429+0000 7f77f1561700 1 --2- 172.21.15.157:0/1612846604 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f77d0066f30 0x7f77d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:11.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.430+0000 7f77f1561700 1 --2- 172.21.15.157:0/1612846604 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f77d0066f30 0x7f77d00693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009fd0 tx=0x7f77e800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:11.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.433+0000 7f77e17fa700 1 -- 172.21.15.157:0/1612846604 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f77dc051e60 con 0x7f77ec0c52e0 2023-10-06T03:59:11.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.664+0000 7f77f37c5700 1 -- 172.21.15.157:0/1612846604 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77ec0c4770 con 0x7f77ec0c52e0 2023-10-06T03:59:11.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.665+0000 7f77e17fa700 1 -- 172.21.15.157:0/1612846604 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+5426 (secure 0 0 0) 0x7f77dc055480 con 0x7f77ec0c52e0 2023-10-06T03:59:11.667 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:59:11.667 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T03:59:06.641696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4223":{"gid":4223,"name":"a","rank":4,"incarnation":17,"state":"up:replay","state_seq":1,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:59:11.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.667+0000 7f77f37c5700 1 -- 172.21.15.157:0/1612846604 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f77d0066f30 msgr2=0x7f77d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:11.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.667+0000 7f77f37c5700 1 --2- 172.21.15.157:0/1612846604 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f77d0066f30 0x7f77d00693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009fd0 tx=0x7f77e800b040 comp rx=0 tx=0).stop 2023-10-06T03:59:11.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.667+0000 7f77f37c5700 1 -- 172.21.15.157:0/1612846604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77ec0c52e0 msgr2=0x7f77ec167610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:11.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.667+0000 7f77f37c5700 1 --2- 172.21.15.157:0/1612846604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77ec0c52e0 0x7f77ec167610 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00d890 tx=0x7f77dc008040 comp rx=0 tx=0).stop 2023-10-06T03:59:11.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.667+0000 7f77f37c5700 1 -- 172.21.15.157:0/1612846604 shutdown_connections 2023-10-06T03:59:11.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.667+0000 7f77f37c5700 1 --2- 172.21.15.157:0/1612846604 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f77d0066f30 0x7f77d00693e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:11.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.667+0000 7f77f37c5700 1 --2- 172.21.15.157:0/1612846604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77ec0ca630 0x7f77ec167b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:11.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.667+0000 7f77f37c5700 1 --2- 172.21.15.157:0/1612846604 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77ec068920 0x7f77ec151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:11.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.667+0000 7f77f37c5700 1 --2- 172.21.15.157:0/1612846604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77ec0c52e0 0x7f77ec167610 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:11.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.667+0000 7f77f37c5700 1 -- 172.21.15.157:0/1612846604 >> 172.21.15.157:0/1612846604 conn(0x7f77ec0692b0 msgr2=0x7f77ec0c94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:59:11.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.667+0000 7f77f37c5700 1 -- 172.21.15.157:0/1612846604 shutdown_connections 2023-10-06T03:59:11.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:11.667+0000 7f77f37c5700 1 -- 172.21.15.157:0/1612846604 wait complete. 2023-10-06T03:59:11.672 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 17 2023-10-06T03:59:11.681 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T03:59:13.681 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:59:13.749 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:59:13.751 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:59:13.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.809+0000 7f9239266700 1 Processor -- start 2023-10-06T03:59:13.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.809+0000 7f9239266700 1 -- start start 2023-10-06T03:59:13.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.810+0000 7f9239266700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f92340ca630 0x7f92340c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:13.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.810+0000 7f9239266700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f923406d530 con 0x7f9234156f50 2023-10-06T03:59:13.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.810+0000 7f9239266700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f923406d9c0 con 0x7f9234068920 2023-10-06T03:59:13.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.810+0000 7f9239266700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f923406f0c0 con 0x7f92340ca630 2023-10-06T03:59:13.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.810+0000 7f92337fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f9234156f50 0x7f92341608c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:46878/0 (socket says 172.21.15.157:46878) 2023-10-06T03:59:13.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.810+0000 7f92337fe700 1 -- 172.21.15.157:0/2313225826 learned_addr learned my addr 172.21.15.157:0/2313225826 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:59:13.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.811+0000 7f9232ffd700 1 --2- 172.21.15.157:0/2313225826 >> v2:172.21.15.175:3300/0 conn(0x7f92340ca630 0x7f92340c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:13.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.811+0000 7f92327fc700 1 -- 172.21.15.157:0/2313225826 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3813201531 0 0) 0x7f923406d530 con 0x7f9234156f50 2023-10-06T03:59:13.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.811+0000 7f92327fc700 1 -- 172.21.15.157:0/2313225826 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9220003570 con 0x7f9234156f50 2023-10-06T03:59:13.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.811+0000 7f92327fc700 1 -- 172.21.15.157:0/2313225826 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 140207360 0 0) 0x7f9220003570 con 0x7f9234156f50 2023-10-06T03:59:13.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.811+0000 7f92327fc700 1 -- 172.21.15.157:0/2313225826 >> v2:172.21.15.175:3300/0 conn(0x7f92340ca630 msgr2=0x7f92340c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:59:13.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.811+0000 7f92327fc700 1 --2- 172.21.15.157:0/2313225826 >> v2:172.21.15.175:3300/0 conn(0x7f92340ca630 0x7f92340c6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:13.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.811+0000 7f92327fc700 1 -- 172.21.15.157:0/2313225826 >> v1:172.21.15.175:6789/0 conn(0x7f9234068920 legacy=0x7f9234156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:13.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.811+0000 7f92327fc700 1 -- 172.21.15.157:0/2313225826 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923406f3a0 con 0x7f9234156f50 2023-10-06T03:59:13.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.811+0000 7f92327fc700 1 -- 172.21.15.157:0/2313225826 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f922c0017f0 con 0x7f9234156f50 2023-10-06T03:59:13.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.811+0000 7f92327fc700 1 -- 172.21.15.157:0/2313225826 >> v1:172.21.15.157:6789/0 conn(0x7f9234156f50 legacy=0x7f92341608c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:13.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.811+0000 7f92327fc700 1 --2- 172.21.15.157:0/2313225826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9220003a60 0x7f9220003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:13.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.811+0000 7f92327fc700 1 --2- 172.21.15.157:0/2313225826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9220004810 0x7f9220004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:13.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.811+0000 7f92327fc700 1 --2- 172.21.15.157:0/2313225826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92200051c0 0x7f922000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:13.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.811+0000 7f92327fc700 1 -- 172.21.15.157:0/2313225826 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923406f0c0 con 0x7f92200051c0 2023-10-06T03:59:13.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.811+0000 7f92327fc700 1 -- 172.21.15.157:0/2313225826 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922000c160 con 0x7f9220003a60 2023-10-06T03:59:13.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.811+0000 7f92327fc700 1 -- 172.21.15.157:0/2313225826 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f922000c320 con 0x7f9220004810 2023-10-06T03:59:13.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.811+0000 7f92337fe700 1 --2- 172.21.15.157:0/2313225826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92200051c0 0x7f922000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:13.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.811+0000 7f9232ffd700 1 --2- 172.21.15.157:0/2313225826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9220003a60 0x7f9220003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:13.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.811+0000 7f922a7fc700 1 --2- 172.21.15.157:0/2313225826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9220004810 0x7f9220004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:13.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.811+0000 7f92337fe700 1 -- 172.21.15.157:0/2313225826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9220004810 msgr2=0x7f9220004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:59:13.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.811+0000 7f92337fe700 1 --2- 172.21.15.157:0/2313225826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9220004810 0x7f9220004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:13.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.811+0000 7f92337fe700 1 -- 172.21.15.157:0/2313225826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9220003a60 msgr2=0x7f9220003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:59:13.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.811+0000 7f92337fe700 1 --2- 172.21.15.157:0/2313225826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9220003a60 0x7f9220003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:13.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.811+0000 7f92337fe700 1 -- 172.21.15.157:0/2313225826 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9220003570 con 0x7f92200051c0 2023-10-06T03:59:13.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.812+0000 7f92337fe700 1 -- 172.21.15.157:0/2313225826 reap_dead start 2023-10-06T03:59:13.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.812+0000 7f92337fe700 1 --2- 172.21.15.157:0/2313225826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92200051c0 0x7f922000ba60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9234157350 tx=0x7f922c0021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23756e82577f1f3f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:13.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.812+0000 7f92327fc700 1 -- 172.21.15.157:0/2313225826 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922c0070d0 con 0x7f92200051c0 2023-10-06T03:59:13.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.812+0000 7f92327fc700 1 -- 172.21.15.157:0/2313225826 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f922c007230 con 0x7f92200051c0 2023-10-06T03:59:13.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.812+0000 7f92327fc700 1 -- 172.21.15.157:0/2313225826 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922c0075d0 con 0x7f92200051c0 2023-10-06T03:59:13.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.812+0000 7f9239266700 1 -- 172.21.15.157:0/2313225826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92200051c0 msgr2=0x7f922000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:13.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.812+0000 7f9239266700 1 --2- 172.21.15.157:0/2313225826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92200051c0 0x7f922000ba60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9234157350 tx=0x7f922c0021d0 comp rx=0 tx=0).stop 2023-10-06T03:59:13.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.812+0000 7f9239266700 1 -- 172.21.15.157:0/2313225826 shutdown_connections 2023-10-06T03:59:13.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.812+0000 7f9239266700 1 --2- 172.21.15.157:0/2313225826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92200051c0 0x7f922000ba60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:13.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.812+0000 7f9239266700 1 -- 172.21.15.157:0/2313225826 >> 172.21.15.157:0/2313225826 conn(0x7f92340692b0 msgr2=0x7f92340696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:59:13.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.812+0000 7f9239266700 1 -- 172.21.15.157:0/2313225826 shutdown_connections 2023-10-06T03:59:13.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.812+0000 7f9239266700 1 -- 172.21.15.157:0/2313225826 wait complete. 2023-10-06T03:59:13.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.814+0000 7f9239266700 1 Processor -- start 2023-10-06T03:59:13.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.814+0000 7f9239266700 1 -- start start 2023-10-06T03:59:13.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.814+0000 7f9239266700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92200051c0 0x7f9234157ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:13.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.814+0000 7f9239266700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9234167cf0 0x7f92340c0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:13.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.814+0000 7f9239266700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92340c0de0 0x7f92340af070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:13.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.814+0000 7f9239266700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92340abcb0 con 0x7f92200051c0 2023-10-06T03:59:13.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.814+0000 7f9239266700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9234158410 con 0x7f92340c0de0 2023-10-06T03:59:13.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.814+0000 7f9239266700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9234158b50 con 0x7f9234167cf0 2023-10-06T03:59:13.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.814+0000 7f9232ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92200051c0 0x7f9234157ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:13.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.814+0000 7f9232ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92200051c0 0x7f9234157ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33666/0 (socket says 172.21.15.157:33666) 2023-10-06T03:59:13.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.814+0000 7f9232ffd700 1 -- 172.21.15.157:0/591341022 learned_addr learned my addr 172.21.15.157:0/591341022 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:59:13.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.815+0000 7f922a7fc700 1 --2- 172.21.15.157:0/591341022 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9234167cf0 0x7f92340c0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:13.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.815+0000 7f92337fe700 1 --2- 172.21.15.157:0/591341022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92340c0de0 0x7f92340af070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:13.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.815+0000 7f9232ffd700 1 -- 172.21.15.157:0/591341022 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9234167cf0 msgr2=0x7f92340c0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:13.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.815+0000 7f9232ffd700 1 --2- 172.21.15.157:0/591341022 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9234167cf0 0x7f92340c0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:13.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.815+0000 7f9232ffd700 1 -- 172.21.15.157:0/591341022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92340c0de0 msgr2=0x7f92340af070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:13.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.815+0000 7f9232ffd700 1 --2- 172.21.15.157:0/591341022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92340c0de0 0x7f92340af070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:13.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.815+0000 7f9232ffd700 1 -- 172.21.15.157:0/591341022 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9234157a40 con 0x7f92200051c0 2023-10-06T03:59:13.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.815+0000 7f9232ffd700 1 --2- 172.21.15.157:0/591341022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92200051c0 0x7f9234157ed0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9224011b10 tx=0x7f92240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:13.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.815+0000 7f922b7fe700 1 -- 172.21.15.157:0/591341022 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922400f9d0 con 0x7f92200051c0 2023-10-06T03:59:13.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.815+0000 7f922b7fe700 1 -- 172.21.15.157:0/591341022 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f922400fb30 con 0x7f92200051c0 2023-10-06T03:59:13.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.815+0000 7f9239266700 1 -- 172.21.15.157:0/591341022 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92341673a0 con 0x7f92200051c0 2023-10-06T03:59:13.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.815+0000 7f922b7fe700 1 -- 172.21.15.157:0/591341022 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922400feb0 con 0x7f92200051c0 2023-10-06T03:59:13.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.815+0000 7f9239266700 1 -- 172.21.15.157:0/591341022 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92341573f0 con 0x7f92200051c0 2023-10-06T03:59:13.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.816+0000 7f9239266700 1 -- 172.21.15.157:0/591341022 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92340ca230 con 0x7f92200051c0 2023-10-06T03:59:13.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.817+0000 7f922b7fe700 1 -- 172.21.15.157:0/591341022 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9224014650 con 0x7f92200051c0 2023-10-06T03:59:13.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.817+0000 7f922b7fe700 1 --2- 172.21.15.157:0/591341022 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9214066f30 0x7f92140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:13.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.817+0000 7f922b7fe700 1 -- 172.21.15.157:0/591341022 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f9224018030 con 0x7f92200051c0 2023-10-06T03:59:13.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.817+0000 7f922a7fc700 1 --2- 172.21.15.157:0/591341022 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9214066f30 0x7f92140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:13.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.818+0000 7f922a7fc700 1 --2- 172.21.15.157:0/591341022 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9214066f30 0x7f92140693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f923415b460 tx=0x7f921c007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:13.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:13.821+0000 7f922b7fe700 1 -- 172.21.15.157:0/591341022 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f92240520d0 con 0x7f92200051c0 2023-10-06T03:59:14.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:14.048+0000 7f9239266700 1 -- 172.21.15.157:0/591341022 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f923404bb40 con 0x7f92200051c0 2023-10-06T03:59:14.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:14.049+0000 7f922b7fe700 1 -- 172.21.15.157:0/591341022 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+5428 (secure 0 0 0) 0x7f9224022090 con 0x7f92200051c0 2023-10-06T03:59:14.051 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:59:14.052 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T03:59:12.644601+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4223":{"gid":4223,"name":"a","rank":4,"incarnation":17,"state":"up:resolve","state_seq":33,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:59:14.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:14.051+0000 7f9239266700 1 -- 172.21.15.157:0/591341022 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9214066f30 msgr2=0x7f92140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:14.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:14.051+0000 7f9239266700 1 --2- 172.21.15.157:0/591341022 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9214066f30 0x7f92140693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f923415b460 tx=0x7f921c007d30 comp rx=0 tx=0).stop 2023-10-06T03:59:14.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:14.051+0000 7f9239266700 1 -- 172.21.15.157:0/591341022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92200051c0 msgr2=0x7f9234157ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:14.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:14.051+0000 7f9239266700 1 --2- 172.21.15.157:0/591341022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92200051c0 0x7f9234157ed0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9224011b10 tx=0x7f92240091a0 comp rx=0 tx=0).stop 2023-10-06T03:59:14.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:14.051+0000 7f9239266700 1 -- 172.21.15.157:0/591341022 shutdown_connections 2023-10-06T03:59:14.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:14.051+0000 7f9239266700 1 --2- 172.21.15.157:0/591341022 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9214066f30 0x7f92140693e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:14.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:14.051+0000 7f9239266700 1 --2- 172.21.15.157:0/591341022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92340c0de0 0x7f92340af070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:14.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:14.051+0000 7f9239266700 1 --2- 172.21.15.157:0/591341022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92200051c0 0x7f9234157ed0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:14.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:14.051+0000 7f9239266700 1 --2- 172.21.15.157:0/591341022 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9234167cf0 0x7f92340c0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:14.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:14.051+0000 7f9239266700 1 -- 172.21.15.157:0/591341022 >> 172.21.15.157:0/591341022 conn(0x7f92340692b0 msgr2=0x7f92340c9500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:59:14.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:14.051+0000 7f9239266700 1 -- 172.21.15.157:0/591341022 shutdown_connections 2023-10-06T03:59:14.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:14.051+0000 7f9239266700 1 -- 172.21.15.157:0/591341022 wait complete. 2023-10-06T03:59:14.055 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 18 2023-10-06T03:59:14.065 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T03:59:16.066 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:59:16.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.192+0000 7fd2b3fff700 1 Processor -- start 2023-10-06T03:59:16.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.192+0000 7fd2b3fff700 1 -- start start 2023-10-06T03:59:16.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.193+0000 7fd2b3fff700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2b40b1e40 0x7fd2b4157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:16.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.193+0000 7fd2b3fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd2b40b04d0 0x7fd2b406dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:16.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.193+0000 7fd2b3fff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd2b406c2d0 0x7fd2b41563b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:16.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.193+0000 7fd2b3fff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b406d8d0 con 0x7fd2b40b04d0 2023-10-06T03:59:16.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.193+0000 7fd2b3fff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b40af8f0 con 0x7fd2b406c2d0 2023-10-06T03:59:16.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.193+0000 7fd2b3fff700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b406ccc0 con 0x7fd2b40b1e40 2023-10-06T03:59:16.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.193+0000 7fd2b27fc700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd2b40b04d0 0x7fd2b406dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:16.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.193+0000 7fd2b27fc700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd2b40b04d0 0x7fd2b406dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33672/0 (socket says 172.21.15.157:33672) 2023-10-06T03:59:16.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.193+0000 7fd2b27fc700 1 -- 172.21.15.157:0/3851148309 learned_addr learned my addr 172.21.15.157:0/3851148309 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:59:16.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.193+0000 7fd2b27fc700 1 -- 172.21.15.157:0/3851148309 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2b40b1e40 msgr2=0x7fd2b4157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:59:16.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.193+0000 7fd2b2ffd700 1 --2- 172.21.15.157:0/3851148309 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2b40b1e40 0x7fd2b4157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:16.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.193+0000 7fd2b37fe700 1 --2- 172.21.15.157:0/3851148309 >> v2:172.21.15.175:3300/0 conn(0x7fd2b406c2d0 0x7fd2b41563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:16.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.193+0000 7fd2b27fc700 1 --2- 172.21.15.157:0/3851148309 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2b40b1e40 0x7fd2b4157750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:16.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.193+0000 7fd2b27fc700 1 -- 172.21.15.157:0/3851148309 >> v2:172.21.15.175:3300/0 conn(0x7fd2b406c2d0 msgr2=0x7fd2b41563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:59:16.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.193+0000 7fd2b27fc700 1 --2- 172.21.15.157:0/3851148309 >> v2:172.21.15.175:3300/0 conn(0x7fd2b406c2d0 0x7fd2b41563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:16.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.193+0000 7fd2b27fc700 1 -- 172.21.15.157:0/3851148309 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b406cfa0 con 0x7fd2b40b04d0 2023-10-06T03:59:16.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.194+0000 7fd2b27fc700 1 --2- 172.21.15.157:0/3851148309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2b40b04d0 0x7fd2b406dd60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd2b408a550 tx=0x7fd2a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a631015a9dde6b52 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:16.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.194+0000 7fd2b1ffb700 1 -- 172.21.15.157:0/3851148309 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a800b860 con 0x7fd2b40b04d0 2023-10-06T03:59:16.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.194+0000 7fd2b1ffb700 1 -- 172.21.15.157:0/3851148309 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a800b9c0 con 0x7fd2b40b04d0 2023-10-06T03:59:16.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.194+0000 7fd2b1ffb700 1 -- 172.21.15.157:0/3851148309 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a800bd80 con 0x7fd2b40b04d0 2023-10-06T03:59:16.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.194+0000 7fd2b3fff700 1 -- 172.21.15.157:0/3851148309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2b40b04d0 msgr2=0x7fd2b406dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:16.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.194+0000 7fd2b3fff700 1 --2- 172.21.15.157:0/3851148309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2b40b04d0 0x7fd2b406dd60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd2b408a550 tx=0x7fd2a8009250 comp rx=0 tx=0).stop 2023-10-06T03:59:16.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.194+0000 7fd2b3fff700 1 -- 172.21.15.157:0/3851148309 shutdown_connections 2023-10-06T03:59:16.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.194+0000 7fd2b3fff700 1 --2- 172.21.15.157:0/3851148309 >> v2:172.21.15.175:3300/0 conn(0x7fd2b406c2d0 0x7fd2b41563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:16.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.194+0000 7fd2b3fff700 1 --2- 172.21.15.157:0/3851148309 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2b40b1e40 0x7fd2b4157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:16.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.194+0000 7fd2b3fff700 1 --2- 172.21.15.157:0/3851148309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2b40b04d0 0x7fd2b406dd60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:16.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.194+0000 7fd2b3fff700 1 -- 172.21.15.157:0/3851148309 >> 172.21.15.157:0/3851148309 conn(0x7fd2b40cc540 msgr2=0x7fd2b40cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:59:16.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.194+0000 7fd2b3fff700 1 -- 172.21.15.157:0/3851148309 shutdown_connections 2023-10-06T03:59:16.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.194+0000 7fd2b3fff700 1 -- 172.21.15.157:0/3851148309 wait complete. 2023-10-06T03:59:16.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.195+0000 7fd2b3fff700 1 Processor -- start 2023-10-06T03:59:16.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.196+0000 7fd2b3fff700 1 -- start start 2023-10-06T03:59:16.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.196+0000 7fd2b3fff700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2b406c2d0 0x7fd2b40c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:16.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.196+0000 7fd2b3fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2b40b04d0 0x7fd2b40c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:16.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.196+0000 7fd2b3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2b40b1e40 0x7fd2b40c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:16.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.196+0000 7fd2b3fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b4151d20 con 0x7fd2b40b04d0 2023-10-06T03:59:16.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.196+0000 7fd2b3fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b4169180 con 0x7fd2b40b1e40 2023-10-06T03:59:16.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.196+0000 7fd2b3fff700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b4169460 con 0x7fd2b406c2d0 2023-10-06T03:59:16.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.196+0000 7fd2b27fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2b40b04d0 0x7fd2b40c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:16.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.196+0000 7fd2b27fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2b40b04d0 0x7fd2b40c97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33680/0 (socket says 172.21.15.157:33680) 2023-10-06T03:59:16.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.196+0000 7fd2b27fc700 1 -- 172.21.15.157:0/3877529328 learned_addr learned my addr 172.21.15.157:0/3877529328 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:59:16.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.196+0000 7fd2b2ffd700 1 --2- 172.21.15.157:0/3877529328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2b406c2d0 0x7fd2b40c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:16.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.197+0000 7fd2b37fe700 1 --2- 172.21.15.157:0/3877529328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2b40b1e40 0x7fd2b40c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:16.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.197+0000 7fd2b27fc700 1 -- 172.21.15.157:0/3877529328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2b406c2d0 msgr2=0x7fd2b40c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:16.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.197+0000 7fd2b27fc700 1 --2- 172.21.15.157:0/3877529328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2b406c2d0 0x7fd2b40c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:16.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.197+0000 7fd2b27fc700 1 -- 172.21.15.157:0/3877529328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2b40b1e40 msgr2=0x7fd2b40c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:16.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.197+0000 7fd2b27fc700 1 --2- 172.21.15.157:0/3877529328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2b40b1e40 0x7fd2b40c6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:16.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.197+0000 7fd2b27fc700 1 -- 172.21.15.157:0/3877529328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a80094f0 con 0x7fd2b40b04d0 2023-10-06T03:59:16.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.197+0000 7fd2b27fc700 1 --2- 172.21.15.157:0/3877529328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2b40b04d0 0x7fd2b40c97b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8006fd0 tx=0x7fd2a8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:16.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.197+0000 7fd29affd700 1 -- 172.21.15.157:0/3877529328 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a801f070 con 0x7fd2b40b04d0 2023-10-06T03:59:16.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.197+0000 7fd29affd700 1 -- 172.21.15.157:0/3877529328 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a801a420 con 0x7fd2b40b04d0 2023-10-06T03:59:16.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.197+0000 7fd2b3fff700 1 -- 172.21.15.157:0/3877529328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b4169740 con 0x7fd2b40b04d0 2023-10-06T03:59:16.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.197+0000 7fd29affd700 1 -- 172.21.15.157:0/3877529328 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a801a7e0 con 0x7fd2b40b04d0 2023-10-06T03:59:16.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.197+0000 7fd2b3fff700 1 -- 172.21.15.157:0/3877529328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b40c9cf0 con 0x7fd2b40b04d0 2023-10-06T03:59:16.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.198+0000 7fd2b3fff700 1 -- 172.21.15.157:0/3877529328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b4088f90 con 0x7fd2b40b04d0 2023-10-06T03:59:16.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.198+0000 7fd29affd700 1 -- 172.21.15.157:0/3877529328 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd2a801ab90 con 0x7fd2b40b04d0 2023-10-06T03:59:16.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.199+0000 7fd29affd700 1 --2- 172.21.15.157:0/3877529328 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd2a0066f30 0x7fd2a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:16.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.199+0000 7fd29affd700 1 -- 172.21.15.157:0/3877529328 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6202+0+0 (secure 0 0 0) 0x7fd2a8085cd0 con 0x7fd2b40b04d0 2023-10-06T03:59:16.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.199+0000 7fd2b2ffd700 1 --2- 172.21.15.157:0/3877529328 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd2a0066f30 0x7fd2a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:16.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.199+0000 7fd2b2ffd700 1 --2- 172.21.15.157:0/3877529328 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd2a0066f30 0x7fd2a00693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd2a400ab30 tx=0x7fd2a4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:16.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.203+0000 7fd29affd700 1 -- 172.21.15.157:0/3877529328 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd2a8052dd0 con 0x7fd2b40b04d0 2023-10-06T03:59:16.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.428+0000 7fd2b3fff700 1 -- 172.21.15.157:0/3877529328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2b406d280 con 0x7fd2b40b04d0 2023-10-06T03:59:16.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.428+0000 7fd29affd700 1 -- 172.21.15.157:0/3877529328 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+5428 (secure 0 0 0) 0x7fd2a8028090 con 0x7fd2b40b04d0 2023-10-06T03:59:16.431 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:59:16.431 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T03:59:12.644601+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4223":{"gid":4223,"name":"a","rank":4,"incarnation":17,"state":"up:resolve","state_seq":33,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:59:16.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.430+0000 7fd2b3fff700 1 -- 172.21.15.157:0/3877529328 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd2a0066f30 msgr2=0x7fd2a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:16.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.430+0000 7fd2b3fff700 1 --2- 172.21.15.157:0/3877529328 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd2a0066f30 0x7fd2a00693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd2a400ab30 tx=0x7fd2a4008040 comp rx=0 tx=0).stop 2023-10-06T03:59:16.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.430+0000 7fd2b3fff700 1 -- 172.21.15.157:0/3877529328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2b40b04d0 msgr2=0x7fd2b40c97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:16.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.430+0000 7fd2b3fff700 1 --2- 172.21.15.157:0/3877529328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2b40b04d0 0x7fd2b40c97b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8006fd0 tx=0x7fd2a8017000 comp rx=0 tx=0).stop 2023-10-06T03:59:16.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.431+0000 7fd2b3fff700 1 -- 172.21.15.157:0/3877529328 shutdown_connections 2023-10-06T03:59:16.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.431+0000 7fd2b3fff700 1 --2- 172.21.15.157:0/3877529328 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd2a0066f30 0x7fd2a00693e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:16.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.431+0000 7fd2b3fff700 1 --2- 172.21.15.157:0/3877529328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2b40b1e40 0x7fd2b40c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:16.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.431+0000 7fd2b3fff700 1 --2- 172.21.15.157:0/3877529328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2b406c2d0 0x7fd2b40c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:16.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.431+0000 7fd2b3fff700 1 --2- 172.21.15.157:0/3877529328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2b40b04d0 0x7fd2b40c97b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:16.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.431+0000 7fd2b3fff700 1 -- 172.21.15.157:0/3877529328 >> 172.21.15.157:0/3877529328 conn(0x7fd2b40cc540 msgr2=0x7fd2b40c41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:59:16.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.431+0000 7fd2b3fff700 1 -- 172.21.15.157:0/3877529328 shutdown_connections 2023-10-06T03:59:16.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:16.431+0000 7fd2b3fff700 1 -- 172.21.15.157:0/3877529328 wait complete. 2023-10-06T03:59:16.435 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 18 2023-10-06T03:59:16.445 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T03:59:18.446 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:59:18.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.571+0000 7fbb5d7ba700 1 Processor -- start 2023-10-06T03:59:18.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.571+0000 7fbb5d7ba700 1 -- start start 2023-10-06T03:59:18.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.572+0000 7fbb5d7ba700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb580b2110 0x7fbb58159df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:18.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.572+0000 7fbb5d7ba700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbb5806c1f0 0x7fbb5814fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:18.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.572+0000 7fbb5d7ba700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb5806ced0 con 0x7fbb580b1d40 2023-10-06T03:59:18.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.572+0000 7fbb5d7ba700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb580afb10 con 0x7fbb5806c1f0 2023-10-06T03:59:18.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.572+0000 7fbb5d7ba700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5806cc10 con 0x7fbb580b2110 2023-10-06T03:59:18.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.573+0000 7fbb4e7fc700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb580b2110 0x7fbb58159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:18.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.573+0000 7fbb577fe700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbb5806c1f0 0x7fbb5814fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:18.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.573+0000 7fbb56ffd700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fbb580b1d40 0x7fbb580af400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.157:43658/0 (socket says 172.21.15.157:43658) 2023-10-06T03:59:18.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.573+0000 7fbb56ffd700 1 -- 172.21.15.157:0/3974475096 learned_addr learned my addr 172.21.15.157:0/3974475096 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:59:18.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.573+0000 7fbb577fe700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbb5806c1f0 0x7fbb5814fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:42252/0 (socket says 172.21.15.157:42252) 2023-10-06T03:59:18.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.573+0000 7fbb4e7fc700 1 -- 172.21.15.157:0/3974475096 >> v2:172.21.15.175:3300/0 conn(0x7fbb5806c1f0 msgr2=0x7fbb5814fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:59:18.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.573+0000 7fbb4e7fc700 1 --2- 172.21.15.157:0/3974475096 >> v2:172.21.15.175:3300/0 conn(0x7fbb5806c1f0 0x7fbb5814fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:18.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.573+0000 7fbb4e7fc700 1 -- 172.21.15.157:0/3974475096 >> v1:172.21.15.175:6789/0 conn(0x7fbb580b1d40 legacy=0x7fbb580af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:18.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.573+0000 7fbb4e7fc700 1 -- 172.21.15.157:0/3974475096 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5806e570 con 0x7fbb580b2110 2023-10-06T03:59:18.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.573+0000 7fbb567fc700 1 -- 172.21.15.157:0/3974475096 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2933809146 0 0) 0x7fbb5806ced0 con 0x7fbb580b1d40 2023-10-06T03:59:18.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.573+0000 7fbb577fe700 1 --2- 172.21.15.157:0/3974475096 >> v2:172.21.15.175:3300/0 conn(0x7fbb5806c1f0 0x7fbb5814fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:59:18.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.573+0000 7fbb4e7fc700 1 --2- 172.21.15.157:0/3974475096 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb580b2110 0x7fbb58159df0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbb400098e0 tx=0x7fbb400091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bc2600bbd57aea9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:18.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.573+0000 7fbb567fc700 1 -- 172.21.15.157:0/3974475096 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4000b6d0 con 0x7fbb580b2110 2023-10-06T03:59:18.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.574+0000 7fbb567fc700 1 -- 172.21.15.157:0/3974475096 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4000b830 con 0x7fbb580b2110 2023-10-06T03:59:18.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.574+0000 7fbb567fc700 1 -- 172.21.15.157:0/3974475096 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4000bbd0 con 0x7fbb580b2110 2023-10-06T03:59:18.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.574+0000 7fbb5d7ba700 1 -- 172.21.15.157:0/3974475096 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb580b2110 msgr2=0x7fbb58159df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:18.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.574+0000 7fbb5d7ba700 1 --2- 172.21.15.157:0/3974475096 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb580b2110 0x7fbb58159df0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbb400098e0 tx=0x7fbb400091a0 comp rx=0 tx=0).stop 2023-10-06T03:59:18.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.574+0000 7fbb5d7ba700 1 -- 172.21.15.157:0/3974475096 shutdown_connections 2023-10-06T03:59:18.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.574+0000 7fbb5d7ba700 1 --2- 172.21.15.157:0/3974475096 >> v2:172.21.15.175:3300/0 conn(0x7fbb5806c1f0 0x7fbb5814fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:18.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.574+0000 7fbb5d7ba700 1 --2- 172.21.15.157:0/3974475096 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb580b2110 0x7fbb58159df0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:18.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.574+0000 7fbb5d7ba700 1 -- 172.21.15.157:0/3974475096 >> 172.21.15.157:0/3974475096 conn(0x7fbb580cc6e0 msgr2=0x7fbb580ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:59:18.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.574+0000 7fbb5d7ba700 1 -- 172.21.15.157:0/3974475096 shutdown_connections 2023-10-06T03:59:18.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.574+0000 7fbb5d7ba700 1 -- 172.21.15.157:0/3974475096 wait complete. 2023-10-06T03:59:18.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.575+0000 7fbb5d7ba700 1 Processor -- start 2023-10-06T03:59:18.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.575+0000 7fbb5d7ba700 1 -- start start 2023-10-06T03:59:18.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.576+0000 7fbb5d7ba700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb5806c1f0 0x7fbb580c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:18.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.576+0000 7fbb5d7ba700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb580b1d40 0x7fbb58169320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:18.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.576+0000 7fbb5d7ba700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb580b2110 0x7fbb580c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:18.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.576+0000 7fbb5d7ba700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb58165da0 con 0x7fbb5806c1f0 2023-10-06T03:59:18.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.576+0000 7fbb5d7ba700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb580c34d0 con 0x7fbb580b1d40 2023-10-06T03:59:18.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.576+0000 7fbb5d7ba700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb581665c0 con 0x7fbb580b2110 2023-10-06T03:59:18.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.576+0000 7fbb56ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb5806c1f0 0x7fbb580c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:18.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.576+0000 7fbb56ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb5806c1f0 0x7fbb580c8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33682/0 (socket says 172.21.15.157:33682) 2023-10-06T03:59:18.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.576+0000 7fbb56ffd700 1 -- 172.21.15.157:0/434649951 learned_addr learned my addr 172.21.15.157:0/434649951 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:59:18.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.576+0000 7fbb4e7fc700 1 --2- 172.21.15.157:0/434649951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb580b1d40 0x7fbb58169320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:18.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.576+0000 7fbb577fe700 1 --2- 172.21.15.157:0/434649951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb580b2110 0x7fbb580c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:18.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.577+0000 7fbb56ffd700 1 -- 172.21.15.157:0/434649951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb580b2110 msgr2=0x7fbb580c61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:18.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.577+0000 7fbb56ffd700 1 --2- 172.21.15.157:0/434649951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb580b2110 0x7fbb580c61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:18.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.577+0000 7fbb56ffd700 1 -- 172.21.15.157:0/434649951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb580b1d40 msgr2=0x7fbb58169320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:18.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.577+0000 7fbb56ffd700 1 --2- 172.21.15.157:0/434649951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb580b1d40 0x7fbb58169320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:18.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.577+0000 7fbb56ffd700 1 -- 172.21.15.157:0/434649951 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb50009a20 con 0x7fbb5806c1f0 2023-10-06T03:59:18.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.577+0000 7fbb56ffd700 1 --2- 172.21.15.157:0/434649951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb5806c1f0 0x7fbb580c8920 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800c810 tx=0x7fbb48007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:18.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.577+0000 7fbb4f7fe700 1 -- 172.21.15.157:0/434649951 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb48005b30 con 0x7fbb5806c1f0 2023-10-06T03:59:18.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.577+0000 7fbb4f7fe700 1 -- 172.21.15.157:0/434649951 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb48005c90 con 0x7fbb5806c1f0 2023-10-06T03:59:18.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.577+0000 7fbb5d7ba700 1 -- 172.21.15.157:0/434649951 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb40009420 con 0x7fbb5806c1f0 2023-10-06T03:59:18.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.577+0000 7fbb4f7fe700 1 -- 172.21.15.157:0/434649951 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb480072e0 con 0x7fbb5806c1f0 2023-10-06T03:59:18.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.577+0000 7fbb5d7ba700 1 -- 172.21.15.157:0/434649951 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb58169860 con 0x7fbb5806c1f0 2023-10-06T03:59:18.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.578+0000 7fbb5d7ba700 1 -- 172.21.15.157:0/434649951 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb580896c0 con 0x7fbb5806c1f0 2023-10-06T03:59:18.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.578+0000 7fbb4f7fe700 1 -- 172.21.15.157:0/434649951 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbb4801b3b0 con 0x7fbb5806c1f0 2023-10-06T03:59:18.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.579+0000 7fbb4f7fe700 1 --2- 172.21.15.157:0/434649951 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbb38066f30 0x7fbb380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:18.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.579+0000 7fbb4f7fe700 1 -- 172.21.15.157:0/434649951 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6202+0+0 (secure 0 0 0) 0x7fbb48083ac0 con 0x7fbb5806c1f0 2023-10-06T03:59:18.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.579+0000 7fbb4e7fc700 1 --2- 172.21.15.157:0/434649951 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbb38066f30 0x7fbb380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:18.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.580+0000 7fbb4e7fc700 1 --2- 172.21.15.157:0/434649951 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbb38066f30 0x7fbb380693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbb400093f0 tx=0x7fbb40017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:18.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.583+0000 7fbb4f7fe700 1 -- 172.21.15.157:0/434649951 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb48050cb0 con 0x7fbb5806c1f0 2023-10-06T03:59:18.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.806+0000 7fbb5d7ba700 1 -- 172.21.15.157:0/434649951 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb5806d7b0 con 0x7fbb5806c1f0 2023-10-06T03:59:18.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.807+0000 7fbb4f7fe700 1 -- 172.21.15.157:0/434649951 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+5430 (secure 0 0 0) 0x7fbb48083e60 con 0x7fbb5806c1f0 2023-10-06T03:59:18.809 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:59:18.810 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T03:59:17.658156+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4223":{"gid":4223,"name":"a","rank":4,"incarnation":17,"state":"up:reconnect","state_seq":35,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:59:18.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.808+0000 7fbb5d7ba700 1 -- 172.21.15.157:0/434649951 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbb38066f30 msgr2=0x7fbb380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:18.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.808+0000 7fbb5d7ba700 1 --2- 172.21.15.157:0/434649951 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbb38066f30 0x7fbb380693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbb400093f0 tx=0x7fbb40017000 comp rx=0 tx=0).stop 2023-10-06T03:59:18.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.808+0000 7fbb5d7ba700 1 -- 172.21.15.157:0/434649951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb5806c1f0 msgr2=0x7fbb580c8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:18.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.808+0000 7fbb5d7ba700 1 --2- 172.21.15.157:0/434649951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb5806c1f0 0x7fbb580c8920 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800c810 tx=0x7fbb48007df0 comp rx=0 tx=0).stop 2023-10-06T03:59:18.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.808+0000 7fbb5d7ba700 1 -- 172.21.15.157:0/434649951 shutdown_connections 2023-10-06T03:59:18.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.809+0000 7fbb5d7ba700 1 --2- 172.21.15.157:0/434649951 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbb38066f30 0x7fbb380693e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:18.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.809+0000 7fbb5d7ba700 1 --2- 172.21.15.157:0/434649951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbb580b2110 0x7fbb580c61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:18.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.809+0000 7fbb5d7ba700 1 --2- 172.21.15.157:0/434649951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb5806c1f0 0x7fbb580c8920 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:18.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.809+0000 7fbb5d7ba700 1 --2- 172.21.15.157:0/434649951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb580b1d40 0x7fbb58169320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:18.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.809+0000 7fbb5d7ba700 1 -- 172.21.15.157:0/434649951 >> 172.21.15.157:0/434649951 conn(0x7fbb580cc6e0 msgr2=0x7fbb580caf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:59:18.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.809+0000 7fbb5d7ba700 1 -- 172.21.15.157:0/434649951 shutdown_connections 2023-10-06T03:59:18.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:18.809+0000 7fbb5d7ba700 1 -- 172.21.15.157:0/434649951 wait complete. 2023-10-06T03:59:18.812 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 19 2023-10-06T03:59:18.823 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T03:59:20.825 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:59:20.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.949+0000 7fa8aa8fd700 1 Processor -- start 2023-10-06T03:59:20.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.949+0000 7fa8aa8fd700 1 -- start start 2023-10-06T03:59:20.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.950+0000 7fa8aa8fd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa8a406cbb0 0x7fa8a406d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:20.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.950+0000 7fa8aa8fd700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a40c9780 con 0x7fa8a406c1f0 2023-10-06T03:59:20.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.950+0000 7fa8aa8fd700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a406e0c0 con 0x7fa8a4150410 2023-10-06T03:59:20.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.950+0000 7fa8aa8fd700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8a40c8c40 con 0x7fa8a406cbb0 2023-10-06T03:59:20.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.950+0000 7fa8a3fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa8a406cbb0 0x7fa8a406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:20.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.950+0000 7fa8a37fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa8a406c1f0 0x7fa8a414fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:46892/0 (socket says 172.21.15.157:46892) 2023-10-06T03:59:20.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.950+0000 7fa8a3fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa8a406cbb0 0x7fa8a406d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33696/0 (socket says 172.21.15.157:33696) 2023-10-06T03:59:20.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.950+0000 7fa8a37fe700 1 -- 172.21.15.157:0/550056769 learned_addr learned my addr 172.21.15.157:0/550056769 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:59:20.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.950+0000 7fa8a3fff700 1 -- 172.21.15.157:0/550056769 >> v1:172.21.15.175:6789/0 conn(0x7fa8a4150410 legacy=0x7fa8a4155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:20.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.950+0000 7fa8a3fff700 1 -- 172.21.15.157:0/550056769 >> v1:172.21.15.157:6789/0 conn(0x7fa8a406c1f0 legacy=0x7fa8a414fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:20.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.950+0000 7fa8a3fff700 1 -- 172.21.15.157:0/550056769 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a40c8e50 con 0x7fa8a406cbb0 2023-10-06T03:59:20.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.950+0000 7fa8a2ffd700 1 -- 172.21.15.157:0/550056769 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3924687456 0 0) 0x7fa8a40c9780 con 0x7fa8a406c1f0 2023-10-06T03:59:20.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.950+0000 7fa8a3fff700 1 --2- 172.21.15.157:0/550056769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8a406cbb0 0x7fa8a406d530 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa8940098e0 tx=0x7fa8940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5310ffbdb600e6d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:20.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.950+0000 7fa8a2ffd700 1 -- 172.21.15.157:0/550056769 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa894009e80 con 0x7fa8a406cbb0 2023-10-06T03:59:20.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.950+0000 7fa8a2ffd700 1 -- 172.21.15.157:0/550056769 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89400b8f0 con 0x7fa8a406cbb0 2023-10-06T03:59:20.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.950+0000 7fa8a2ffd700 1 -- 172.21.15.157:0/550056769 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89400bc70 con 0x7fa8a406cbb0 2023-10-06T03:59:20.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.951+0000 7fa8aa8fd700 1 -- 172.21.15.157:0/550056769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8a406cbb0 msgr2=0x7fa8a406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:20.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.951+0000 7fa8aa8fd700 1 --2- 172.21.15.157:0/550056769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8a406cbb0 0x7fa8a406d530 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa8940098e0 tx=0x7fa8940091a0 comp rx=0 tx=0).stop 2023-10-06T03:59:20.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.951+0000 7fa8aa8fd700 1 -- 172.21.15.157:0/550056769 shutdown_connections 2023-10-06T03:59:20.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.951+0000 7fa8aa8fd700 1 --2- 172.21.15.157:0/550056769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8a406cbb0 0x7fa8a406d530 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:20.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.951+0000 7fa8aa8fd700 1 -- 172.21.15.157:0/550056769 >> 172.21.15.157:0/550056769 conn(0x7fa8a406f230 msgr2=0x7fa8a406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:59:20.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.951+0000 7fa8aa8fd700 1 -- 172.21.15.157:0/550056769 shutdown_connections 2023-10-06T03:59:20.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.951+0000 7fa8aa8fd700 1 -- 172.21.15.157:0/550056769 wait complete. 2023-10-06T03:59:20.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.952+0000 7fa8aa8fd700 1 Processor -- start 2023-10-06T03:59:20.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.952+0000 7fa8aa8fd700 1 -- start start 2023-10-06T03:59:20.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.953+0000 7fa8aa8fd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8a406c1f0 0x7fa8a40c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:20.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.953+0000 7fa8aa8fd700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8a406cbb0 0x7fa8a4153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:20.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.953+0000 7fa8aa8fd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8a4150410 0x7fa8a41541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:20.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.953+0000 7fa8aa8fd700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a40b1ea0 con 0x7fa8a4150410 2023-10-06T03:59:20.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.953+0000 7fa8aa8fd700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a40c6560 con 0x7fa8a406c1f0 2023-10-06T03:59:20.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.953+0000 7fa8aa8fd700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a41533c0 con 0x7fa8a406cbb0 2023-10-06T03:59:20.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.953+0000 7fa8a8e9a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8a4150410 0x7fa8a41541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:20.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.953+0000 7fa8a8e9a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8a4150410 0x7fa8a41541d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33702/0 (socket says 172.21.15.157:33702) 2023-10-06T03:59:20.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.953+0000 7fa8a8e9a700 1 -- 172.21.15.157:0/2008242147 learned_addr learned my addr 172.21.15.157:0/2008242147 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:59:20.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.953+0000 7fa8a3fff700 1 --2- 172.21.15.157:0/2008242147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8a406c1f0 0x7fa8a40c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:20.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.953+0000 7fa8a37fe700 1 --2- 172.21.15.157:0/2008242147 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8a406cbb0 0x7fa8a4153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:20.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.953+0000 7fa8a8e9a700 1 -- 172.21.15.157:0/2008242147 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8a406cbb0 msgr2=0x7fa8a4153c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:20.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.953+0000 7fa8a8e9a700 1 --2- 172.21.15.157:0/2008242147 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8a406cbb0 0x7fa8a4153c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:20.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.953+0000 7fa8a8e9a700 1 -- 172.21.15.157:0/2008242147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8a406c1f0 msgr2=0x7fa8a40c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:20.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.953+0000 7fa8a8e9a700 1 --2- 172.21.15.157:0/2008242147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8a406c1f0 0x7fa8a40c5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:20.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.953+0000 7fa8a8e9a700 1 -- 172.21.15.157:0/2008242147 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa894009420 con 0x7fa8a4150410 2023-10-06T03:59:20.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.954+0000 7fa8a8e9a700 1 --2- 172.21.15.157:0/2008242147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8a4150410 0x7fa8a41541d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa89c00c8e0 tx=0x7fa89c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:20.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.954+0000 7fa88bfff700 1 -- 172.21.15.157:0/2008242147 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c012630 con 0x7fa8a4150410 2023-10-06T03:59:20.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.954+0000 7fa88bfff700 1 -- 172.21.15.157:0/2008242147 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89c012790 con 0x7fa8a4150410 2023-10-06T03:59:20.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.954+0000 7fa88bfff700 1 -- 172.21.15.157:0/2008242147 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c012b50 con 0x7fa8a4150410 2023-10-06T03:59:20.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.954+0000 7fa8aa8fd700 1 -- 172.21.15.157:0/2008242147 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a41536a0 con 0x7fa8a4150410 2023-10-06T03:59:20.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.954+0000 7fa8aa8fd700 1 -- 172.21.15.157:0/2008242147 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a4156950 con 0x7fa8a4150410 2023-10-06T03:59:20.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.955+0000 7fa8aa8fd700 1 -- 172.21.15.157:0/2008242147 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8a4153aa0 con 0x7fa8a4150410 2023-10-06T03:59:20.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.955+0000 7fa88bfff700 1 -- 172.21.15.157:0/2008242147 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa89c01a3a0 con 0x7fa8a4150410 2023-10-06T03:59:20.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.956+0000 7fa88bfff700 1 --2- 172.21.15.157:0/2008242147 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa890066f30 0x7fa8900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:20.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.956+0000 7fa88bfff700 1 -- 172.21.15.157:0/2008242147 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6202+0+0 (secure 0 0 0) 0x7fa89c0849b0 con 0x7fa8a4150410 2023-10-06T03:59:20.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.956+0000 7fa8a3fff700 1 --2- 172.21.15.157:0/2008242147 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa890066f30 0x7fa8900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:20.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.956+0000 7fa8a3fff700 1 --2- 172.21.15.157:0/2008242147 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa890066f30 0x7fa8900693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa8940093f0 tx=0x7fa894017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:20.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:20.960+0000 7fa88bfff700 1 -- 172.21.15.157:0/2008242147 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa89c051a30 con 0x7fa8a4150410 2023-10-06T03:59:21.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:21.182+0000 7fa8aa8fd700 1 -- 172.21.15.157:0/2008242147 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8a40c9130 con 0x7fa8a4150410 2023-10-06T03:59:21.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:21.183+0000 7fa88bfff700 1 -- 172.21.15.157:0/2008242147 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+5430 (secure 0 0 0) 0x7fa89c055050 con 0x7fa8a4150410 2023-10-06T03:59:21.185 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:59:21.185 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T03:59:17.658156+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4223":{"gid":4223,"name":"a","rank":4,"incarnation":17,"state":"up:reconnect","state_seq":35,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:59:21.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:21.184+0000 7fa8aa8fd700 1 -- 172.21.15.157:0/2008242147 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa890066f30 msgr2=0x7fa8900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:21.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:21.184+0000 7fa8aa8fd700 1 --2- 172.21.15.157:0/2008242147 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa890066f30 0x7fa8900693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa8940093f0 tx=0x7fa894017000 comp rx=0 tx=0).stop 2023-10-06T03:59:21.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:21.184+0000 7fa8aa8fd700 1 -- 172.21.15.157:0/2008242147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8a4150410 msgr2=0x7fa8a41541d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:21.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:21.184+0000 7fa8aa8fd700 1 --2- 172.21.15.157:0/2008242147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8a4150410 0x7fa8a41541d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa89c00c8e0 tx=0x7fa89c001710 comp rx=0 tx=0).stop 2023-10-06T03:59:21.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:21.184+0000 7fa8aa8fd700 1 -- 172.21.15.157:0/2008242147 shutdown_connections 2023-10-06T03:59:21.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:21.184+0000 7fa8aa8fd700 1 --2- 172.21.15.157:0/2008242147 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa890066f30 0x7fa8900693e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:21.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:21.184+0000 7fa8aa8fd700 1 --2- 172.21.15.157:0/2008242147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8a406c1f0 0x7fa8a40c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:21.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:21.184+0000 7fa8aa8fd700 1 --2- 172.21.15.157:0/2008242147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8a4150410 0x7fa8a41541d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:21.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:21.185+0000 7fa8aa8fd700 1 --2- 172.21.15.157:0/2008242147 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8a406cbb0 0x7fa8a4153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:21.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:21.185+0000 7fa8aa8fd700 1 -- 172.21.15.157:0/2008242147 >> 172.21.15.157:0/2008242147 conn(0x7fa8a406f230 msgr2=0x7fa8a406d530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:59:21.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:21.185+0000 7fa8aa8fd700 1 -- 172.21.15.157:0/2008242147 shutdown_connections 2023-10-06T03:59:21.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:21.185+0000 7fa8aa8fd700 1 -- 172.21.15.157:0/2008242147 wait complete. 2023-10-06T03:59:21.189 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 19 2023-10-06T03:59:21.200 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 19, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-06T03:57:10.718782+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 19, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0, 1, 2, 3, 4 ], "info": { "gid_4223": { "addr": "172.21.15.157:6835/3932336008", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6833", "nonce": 3932336008, "type": "v2" }, { "addr": "172.21.15.157:6835", "nonce": 3932336008, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4223, "incarnation": 17, "join_fscid": -1, "name": "a", "rank": 4, "state": "up:reconnect", "state_seq": 35 }, "gid_4270": { "addr": "172.21.15.175:6835/1596042861", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6834", "nonce": 1596042861, "type": "v2" }, { "addr": "172.21.15.175:6835", "nonce": 1596042861, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4270, "incarnation": 13, "join_fscid": -1, "name": "b", "rank": 3, "state": "up:active", "state_seq": 8 }, "gid_4279": { "addr": "172.21.15.175:6837/1002404870", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6836", "nonce": 1002404870, "type": "v2" }, { "addr": "172.21.15.175:6837", "nonce": 1002404870, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4279, "incarnation": 11, "join_fscid": -1, "name": "d", "rank": 2, "state": "up:active", "state_seq": 7 }, "gid_4287": { "addr": "172.21.15.157:6834/1179753185", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6832", "nonce": 1179753185, "type": "v2" }, { "addr": "172.21.15.157:6834", "nonce": 1179753185, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4287, "incarnation": 9, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4296": { "addr": "172.21.15.157:6837/4188652695", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6836", "nonce": 4188652695, "type": "v2" }, { "addr": "172.21.15.157:6837", "nonce": 4188652695, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4296, "incarnation": 4, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 26, "max_file_size": 1099511627776, "max_mds": 5, "metadata_pool": 3, "modified": "2023-10-06T03:59:17.658156+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4296, "mds_1": 4287, "mds_2": 4279, "mds_3": 4270, "mds_4": 4223 } } } ], "standbys": [] } 2023-10-06T03:59:23.202 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:59:23.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.327+0000 7f7330686700 1 Processor -- start 2023-10-06T03:59:23.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.327+0000 7f7330686700 1 -- start start 2023-10-06T03:59:23.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.328+0000 7f7330686700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f732806cbb0 0x7f732806d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:23.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.328+0000 7f7330686700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f732806c1f0 0x7f732806da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:23.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.328+0000 7f7330686700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f732806cf80 0x7f732814fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:23.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.328+0000 7f7330686700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73280c8c40 con 0x7f732806cf80 2023-10-06T03:59:23.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.328+0000 7f7330686700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73280c9780 con 0x7f732806cbb0 2023-10-06T03:59:23.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.328+0000 7f7330686700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732806e0c0 con 0x7f732806c1f0 2023-10-06T03:59:23.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.328+0000 7f732ec23700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f732806cf80 0x7f732814fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:23.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.328+0000 7f732ec23700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f732806cf80 0x7f732814fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59470/0 (socket says 172.21.15.157:59470) 2023-10-06T03:59:23.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.328+0000 7f732ec23700 1 -- 172.21.15.157:0/442689671 learned_addr learned my addr 172.21.15.157:0/442689671 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:59:23.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.328+0000 7f732e422700 1 --2- 172.21.15.157:0/442689671 >> v2:172.21.15.175:3300/0 conn(0x7f732806cbb0 0x7f732806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:23.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.328+0000 7f732dc21700 1 --2- 172.21.15.157:0/442689671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f732806c1f0 0x7f732806da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:23.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.328+0000 7f732ec23700 1 -- 172.21.15.157:0/442689671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f732806c1f0 msgr2=0x7f732806da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:59:23.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.328+0000 7f732ec23700 1 --2- 172.21.15.157:0/442689671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f732806c1f0 0x7f732806da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:23.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.328+0000 7f732ec23700 1 -- 172.21.15.157:0/442689671 >> v2:172.21.15.175:3300/0 conn(0x7f732806cbb0 msgr2=0x7f732806d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:59:23.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.328+0000 7f732ec23700 1 --2- 172.21.15.157:0/442689671 >> v2:172.21.15.175:3300/0 conn(0x7f732806cbb0 0x7f732806d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:23.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.328+0000 7f732ec23700 1 -- 172.21.15.157:0/442689671 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73280c8e50 con 0x7f732806cf80 2023-10-06T03:59:23.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.328+0000 7f732ec23700 1 --2- 172.21.15.157:0/442689671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732806cf80 0x7f732814fd00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7320009f20 tx=0x7f7320009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba3ff7169215680 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:23.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.328+0000 7f732d420700 1 -- 172.21.15.157:0/442689671 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732000e070 con 0x7f732806cf80 2023-10-06T03:59:23.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.329+0000 7f732d420700 1 -- 172.21.15.157:0/442689671 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732000b9c0 con 0x7f732806cf80 2023-10-06T03:59:23.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.329+0000 7f732d420700 1 -- 172.21.15.157:0/442689671 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73200092e0 con 0x7f732806cf80 2023-10-06T03:59:23.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.329+0000 7f7330686700 1 -- 172.21.15.157:0/442689671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732806cf80 msgr2=0x7f732814fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:23.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.329+0000 7f7330686700 1 --2- 172.21.15.157:0/442689671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732806cf80 0x7f732814fd00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7320009f20 tx=0x7f7320009840 comp rx=0 tx=0).stop 2023-10-06T03:59:23.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.329+0000 7f7330686700 1 -- 172.21.15.157:0/442689671 shutdown_connections 2023-10-06T03:59:23.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.329+0000 7f7330686700 1 --2- 172.21.15.157:0/442689671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732806cf80 0x7f732814fd00 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:23.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.329+0000 7f7330686700 1 --2- 172.21.15.157:0/442689671 >> v2:172.21.15.175:3300/0 conn(0x7f732806cbb0 0x7f732806d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:23.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.329+0000 7f7330686700 1 --2- 172.21.15.157:0/442689671 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f732806c1f0 0x7f732806da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:23.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.329+0000 7f7330686700 1 -- 172.21.15.157:0/442689671 >> 172.21.15.157:0/442689671 conn(0x7f732806f230 msgr2=0x7f732806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:59:23.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.329+0000 7f7330686700 1 -- 172.21.15.157:0/442689671 shutdown_connections 2023-10-06T03:59:23.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.329+0000 7f7330686700 1 -- 172.21.15.157:0/442689671 wait complete. 2023-10-06T03:59:23.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.330+0000 7f7330686700 1 Processor -- start 2023-10-06T03:59:23.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.330+0000 7f7330686700 1 -- start start 2023-10-06T03:59:23.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.331+0000 7f7330686700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f732806c1f0 0x7f732814d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:23.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.331+0000 7f7330686700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f732806cbb0 0x7f732814d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:23.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.331+0000 7f7330686700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732806cf80 0x7f73281699e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:23.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.331+0000 7f7330686700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73281472a0 con 0x7f732806cf80 2023-10-06T03:59:23.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.331+0000 7f7330686700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73281515e0 con 0x7f732806c1f0 2023-10-06T03:59:23.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.331+0000 7f7330686700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73281518c0 con 0x7f732806cbb0 2023-10-06T03:59:23.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.331+0000 7f732ec23700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732806cf80 0x7f73281699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:23.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.331+0000 7f732ec23700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732806cf80 0x7f73281699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59480/0 (socket says 172.21.15.157:59480) 2023-10-06T03:59:23.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.331+0000 7f732ec23700 1 -- 172.21.15.157:0/2526964880 learned_addr learned my addr 172.21.15.157:0/2526964880 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:59:23.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.331+0000 7f732e422700 1 --2- 172.21.15.157:0/2526964880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f732806c1f0 0x7f732814d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:23.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.331+0000 7f732dc21700 1 --2- 172.21.15.157:0/2526964880 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f732806cbb0 0x7f732814d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:23.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.331+0000 7f732ec23700 1 -- 172.21.15.157:0/2526964880 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f732806cbb0 msgr2=0x7f732814d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:23.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.331+0000 7f732ec23700 1 --2- 172.21.15.157:0/2526964880 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f732806cbb0 0x7f732814d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:23.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.332+0000 7f732ec23700 1 -- 172.21.15.157:0/2526964880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f732806c1f0 msgr2=0x7f732814d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:23.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.332+0000 7f732ec23700 1 --2- 172.21.15.157:0/2526964880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f732806c1f0 0x7f732814d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:23.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.332+0000 7f732ec23700 1 -- 172.21.15.157:0/2526964880 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7320009ae0 con 0x7f732806cf80 2023-10-06T03:59:23.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.332+0000 7f732ec23700 1 --2- 172.21.15.157:0/2526964880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732806cf80 0x7f73281699e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7320009880 tx=0x7f7320018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:23.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.332+0000 7f731e7fc700 1 -- 172.21.15.157:0/2526964880 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7320008ad0 con 0x7f732806cf80 2023-10-06T03:59:23.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.332+0000 7f731e7fc700 1 -- 172.21.15.157:0/2526964880 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7320008c50 con 0x7f732806cf80 2023-10-06T03:59:23.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.332+0000 7f7330686700 1 -- 172.21.15.157:0/2526964880 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7328147520 con 0x7f732806cf80 2023-10-06T03:59:23.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.332+0000 7f731e7fc700 1 -- 172.21.15.157:0/2526964880 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732001c410 con 0x7f732806cf80 2023-10-06T03:59:23.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.332+0000 7f7330686700 1 -- 172.21.15.157:0/2526964880 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7328147ae0 con 0x7f732806cf80 2023-10-06T03:59:23.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.333+0000 7f7330686700 1 -- 172.21.15.157:0/2526964880 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7328089ca0 con 0x7f732806cf80 2023-10-06T03:59:23.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.333+0000 7f731e7fc700 1 -- 172.21.15.157:0/2526964880 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7320003040 con 0x7f732806cf80 2023-10-06T03:59:23.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.334+0000 7f731e7fc700 1 --2- 172.21.15.157:0/2526964880 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f730c066f60 0x7f730c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:23.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.334+0000 7f731e7fc700 1 -- 172.21.15.157:0/2526964880 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f7320085e00 con 0x7f732806cf80 2023-10-06T03:59:23.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.334+0000 7f732e422700 1 --2- 172.21.15.157:0/2526964880 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f730c066f60 0x7f730c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:23.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.334+0000 7f732e422700 1 --2- 172.21.15.157:0/2526964880 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f730c066f60 0x7f730c069410 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7324009db0 tx=0x7f73240091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:23.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.338+0000 7f731e7fc700 1 -- 172.21.15.157:0/2526964880 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7320052e80 con 0x7f732806cf80 2023-10-06T03:59:23.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.562+0000 7f7330686700 1 -- 172.21.15.157:0/2526964880 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7328147cd0 con 0x7f732806cf80 2023-10-06T03:59:23.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.563+0000 7f731e7fc700 1 -- 172.21.15.157:0/2526964880 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+5427 (secure 0 0 0) 0x7f73200564a0 con 0x7f732806cf80 2023-10-06T03:59:23.565 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:59:23.565 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T03:59:22.672561+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4223":{"gid":4223,"name":"a","rank":4,"incarnation":17,"state":"up:rejoin","state_seq":37,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:59:23.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.565+0000 7f7330686700 1 -- 172.21.15.157:0/2526964880 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f730c066f60 msgr2=0x7f730c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:23.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.565+0000 7f7330686700 1 --2- 172.21.15.157:0/2526964880 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f730c066f60 0x7f730c069410 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7324009db0 tx=0x7f73240091a0 comp rx=0 tx=0).stop 2023-10-06T03:59:23.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.565+0000 7f7330686700 1 -- 172.21.15.157:0/2526964880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732806cf80 msgr2=0x7f73281699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:23.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.565+0000 7f7330686700 1 --2- 172.21.15.157:0/2526964880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732806cf80 0x7f73281699e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7320009880 tx=0x7f7320018000 comp rx=0 tx=0).stop 2023-10-06T03:59:23.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.565+0000 7f7330686700 1 -- 172.21.15.157:0/2526964880 shutdown_connections 2023-10-06T03:59:23.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.565+0000 7f7330686700 1 --2- 172.21.15.157:0/2526964880 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f730c066f60 0x7f730c069410 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:23.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.565+0000 7f7330686700 1 --2- 172.21.15.157:0/2526964880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f732806c1f0 0x7f732814d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:23.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.565+0000 7f7330686700 1 --2- 172.21.15.157:0/2526964880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732806cf80 0x7f73281699e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:23.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.565+0000 7f7330686700 1 --2- 172.21.15.157:0/2526964880 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f732806cbb0 0x7f732814d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:23.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.565+0000 7f7330686700 1 -- 172.21.15.157:0/2526964880 >> 172.21.15.157:0/2526964880 conn(0x7f732806f230 msgr2=0x7f732814ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:59:23.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.565+0000 7f7330686700 1 -- 172.21.15.157:0/2526964880 shutdown_connections 2023-10-06T03:59:23.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:23.565+0000 7f7330686700 1 -- 172.21.15.157:0/2526964880 wait complete. 2023-10-06T03:59:23.568 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 20 2023-10-06T03:59:23.579 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T03:59:25.580 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:59:25.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.781+0000 7f5abeef1700 1 Processor -- start 2023-10-06T03:59:25.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.781+0000 7f5abeef1700 1 -- start start 2023-10-06T03:59:25.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.782+0000 7f5abeef1700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5ab80c51c0 0x7f5ab81560b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:25.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.782+0000 7f5abeef1700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ab806d710 con 0x7f5ab814fde0 2023-10-06T03:59:25.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.782+0000 7f5abeef1700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ab80c4fb0 con 0x7f5ab81565f0 2023-10-06T03:59:25.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.782+0000 7f5abeef1700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ab815a210 con 0x7f5ab80c51c0 2023-10-06T03:59:25.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.782+0000 7f5abcc8d700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f5ab814fde0 0x7f5ab806c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:53668/0 (socket says 172.21.15.157:53668) 2023-10-06T03:59:25.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.782+0000 7f5abcc8d700 1 -- 172.21.15.157:0/2597170726 learned_addr learned my addr 172.21.15.157:0/2597170726 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:59:25.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.782+0000 7f5ab7fff700 1 --2- 172.21.15.157:0/2597170726 >> v2:172.21.15.175:3300/0 conn(0x7f5ab80c51c0 0x7f5ab81560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:25.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.782+0000 7f5ab77fe700 1 -- 172.21.15.157:0/2597170726 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 622948541 0 0) 0x7f5ab806d710 con 0x7f5ab814fde0 2023-10-06T03:59:25.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.782+0000 7f5ab77fe700 1 -- 172.21.15.157:0/2597170726 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5aa8003570 con 0x7f5ab814fde0 2023-10-06T03:59:25.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.782+0000 7f5ab7fff700 1 -- 172.21.15.157:0/2597170726 >> v1:172.21.15.175:6789/0 conn(0x7f5ab81565f0 legacy=0x7f5ab81608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:25.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.782+0000 7f5ab77fe700 1 -- 172.21.15.157:0/2597170726 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3542762259 0 0) 0x7f5aa8003570 con 0x7f5ab814fde0 2023-10-06T03:59:25.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.782+0000 7f5ab7fff700 1 -- 172.21.15.157:0/2597170726 >> v1:172.21.15.157:6789/0 conn(0x7f5ab814fde0 legacy=0x7f5ab806c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:25.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.782+0000 7f5ab7fff700 1 -- 172.21.15.157:0/2597170726 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab806f090 con 0x7f5ab80c51c0 2023-10-06T03:59:25.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.783+0000 7f5ab7fff700 1 --2- 172.21.15.157:0/2597170726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ab80c51c0 0x7f5ab81560b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5aa40098e0 tx=0x7f5aa40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=731ea93a777798bd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:25.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.783+0000 7f5ab77fe700 1 -- 172.21.15.157:0/2597170726 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa400b640 con 0x7f5ab80c51c0 2023-10-06T03:59:25.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.783+0000 7f5ab77fe700 1 -- 172.21.15.157:0/2597170726 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa400b7a0 con 0x7f5ab80c51c0 2023-10-06T03:59:25.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.783+0000 7f5ab77fe700 1 -- 172.21.15.157:0/2597170726 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa400bb40 con 0x7f5ab80c51c0 2023-10-06T03:59:25.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.783+0000 7f5abeef1700 1 -- 172.21.15.157:0/2597170726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ab80c51c0 msgr2=0x7f5ab81560b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:25.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.783+0000 7f5abeef1700 1 --2- 172.21.15.157:0/2597170726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ab80c51c0 0x7f5ab81560b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5aa40098e0 tx=0x7f5aa40091a0 comp rx=0 tx=0).stop 2023-10-06T03:59:25.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.783+0000 7f5abeef1700 1 -- 172.21.15.157:0/2597170726 shutdown_connections 2023-10-06T03:59:25.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.783+0000 7f5abeef1700 1 --2- 172.21.15.157:0/2597170726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ab80c51c0 0x7f5ab81560b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:25.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.783+0000 7f5abeef1700 1 -- 172.21.15.157:0/2597170726 >> 172.21.15.157:0/2597170726 conn(0x7f5ab8067850 msgr2=0x7f5ab8067c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:59:25.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.783+0000 7f5abeef1700 1 -- 172.21.15.157:0/2597170726 shutdown_connections 2023-10-06T03:59:25.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.784+0000 7f5abeef1700 1 -- 172.21.15.157:0/2597170726 wait complete. 2023-10-06T03:59:25.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.785+0000 7f5abeef1700 1 Processor -- start 2023-10-06T03:59:25.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.785+0000 7f5abeef1700 1 -- start start 2023-10-06T03:59:25.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.785+0000 7f5abeef1700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ab80c51c0 0x7f5ab80c2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:25.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.785+0000 7f5abeef1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ab814fde0 0x7f5ab8167410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:25.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.786+0000 7f5abeef1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ab81565f0 0x7f5ab80ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:25.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.786+0000 7f5abeef1700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab8159260 con 0x7f5ab81565f0 2023-10-06T03:59:25.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.786+0000 7f5abeef1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab8153070 con 0x7f5ab814fde0 2023-10-06T03:59:25.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.786+0000 7f5abeef1700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab816a200 con 0x7f5ab80c51c0 2023-10-06T03:59:25.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.786+0000 7f5abd48e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ab81565f0 0x7f5ab80ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:25.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.786+0000 7f5abd48e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ab81565f0 0x7f5ab80ca340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59494/0 (socket says 172.21.15.157:59494) 2023-10-06T03:59:25.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.786+0000 7f5abd48e700 1 -- 172.21.15.157:0/4086832792 learned_addr learned my addr 172.21.15.157:0/4086832792 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:59:25.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.786+0000 7f5ab7fff700 1 --2- 172.21.15.157:0/4086832792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ab814fde0 0x7f5ab8167410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:25.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.786+0000 7f5abcc8d700 1 --2- 172.21.15.157:0/4086832792 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ab80c51c0 0x7f5ab80c2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:25.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.786+0000 7f5abd48e700 1 -- 172.21.15.157:0/4086832792 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ab80c51c0 msgr2=0x7f5ab80c2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:25.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.786+0000 7f5abd48e700 1 --2- 172.21.15.157:0/4086832792 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ab80c51c0 0x7f5ab80c2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:25.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.786+0000 7f5abd48e700 1 -- 172.21.15.157:0/4086832792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ab814fde0 msgr2=0x7f5ab8167410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:25.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.786+0000 7f5abd48e700 1 --2- 172.21.15.157:0/4086832792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ab814fde0 0x7f5ab8167410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:25.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.786+0000 7f5abd48e700 1 -- 172.21.15.157:0/4086832792 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa4009420 con 0x7f5ab81565f0 2023-10-06T03:59:25.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.786+0000 7f5ab7fff700 1 --2- 172.21.15.157:0/4086832792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ab814fde0 0x7f5ab8167410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:59:25.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.786+0000 7f5abcc8d700 1 --2- 172.21.15.157:0/4086832792 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ab80c51c0 0x7f5ab80c2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:59:25.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.786+0000 7f5abd48e700 1 --2- 172.21.15.157:0/4086832792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ab81565f0 0x7f5ab80ca340 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000c840 tx=0x7f5ab0001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:25.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.786+0000 7f5ab4ff9700 1 -- 172.21.15.157:0/4086832792 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0012630 con 0x7f5ab81565f0 2023-10-06T03:59:25.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.786+0000 7f5ab4ff9700 1 -- 172.21.15.157:0/4086832792 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab00127b0 con 0x7f5ab81565f0 2023-10-06T03:59:25.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.786+0000 7f5ab4ff9700 1 -- 172.21.15.157:0/4086832792 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0012b70 con 0x7f5ab81565f0 2023-10-06T03:59:25.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.786+0000 7f5abeef1700 1 -- 172.21.15.157:0/4086832792 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab8159470 con 0x7f5ab81565f0 2023-10-06T03:59:25.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.786+0000 7f5abeef1700 1 -- 172.21.15.157:0/4086832792 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab80c4510 con 0x7f5ab81565f0 2023-10-06T03:59:25.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.787+0000 7f5abeef1700 1 -- 172.21.15.157:0/4086832792 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ab816a3c0 con 0x7f5ab81565f0 2023-10-06T03:59:25.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.788+0000 7f5ab4ff9700 1 -- 172.21.15.157:0/4086832792 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5ab0003030 con 0x7f5ab81565f0 2023-10-06T03:59:25.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.788+0000 7f5ab4ff9700 1 --2- 172.21.15.157:0/4086832792 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5a9c066f30 0x7f5a9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:25.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.788+0000 7f5ab4ff9700 1 -- 172.21.15.157:0/4086832792 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f5ab00846e0 con 0x7f5ab81565f0 2023-10-06T03:59:25.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.788+0000 7f5abcc8d700 1 --2- 172.21.15.157:0/4086832792 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5a9c066f30 0x7f5a9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:25.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.789+0000 7f5abcc8d700 1 --2- 172.21.15.157:0/4086832792 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5a9c066f30 0x7f5a9c0693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0011e0 tx=0x7f5aac007a90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:25.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:25.792+0000 7f5ab4ff9700 1 -- 172.21.15.157:0/4086832792 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ab0051760 con 0x7f5ab81565f0 2023-10-06T03:59:26.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:26.017+0000 7f5abeef1700 1 -- 172.21.15.157:0/4086832792 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ab804bb40 con 0x7f5ab81565f0 2023-10-06T03:59:26.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:26.018+0000 7f5ab4ff9700 1 -- 172.21.15.157:0/4086832792 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+5427 (secure 0 0 0) 0x7f5ab0054d80 con 0x7f5ab81565f0 2023-10-06T03:59:26.020 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:59:26.020 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T03:59:22.672561+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4223":{"gid":4223,"name":"a","rank":4,"incarnation":17,"state":"up:rejoin","state_seq":37,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:59:26.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:26.020+0000 7f5abeef1700 1 -- 172.21.15.157:0/4086832792 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5a9c066f30 msgr2=0x7f5a9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:26.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:26.020+0000 7f5abeef1700 1 --2- 172.21.15.157:0/4086832792 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5a9c066f30 0x7f5a9c0693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0011e0 tx=0x7f5aac007a90 comp rx=0 tx=0).stop 2023-10-06T03:59:26.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:26.020+0000 7f5abeef1700 1 -- 172.21.15.157:0/4086832792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ab81565f0 msgr2=0x7f5ab80ca340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:26.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:26.020+0000 7f5abeef1700 1 --2- 172.21.15.157:0/4086832792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ab81565f0 0x7f5ab80ca340 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000c840 tx=0x7f5ab0001670 comp rx=0 tx=0).stop 2023-10-06T03:59:26.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:26.020+0000 7f5abeef1700 1 -- 172.21.15.157:0/4086832792 shutdown_connections 2023-10-06T03:59:26.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:26.020+0000 7f5abeef1700 1 --2- 172.21.15.157:0/4086832792 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5a9c066f30 0x7f5a9c0693e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:26.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:26.020+0000 7f5abeef1700 1 --2- 172.21.15.157:0/4086832792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ab81565f0 0x7f5ab80ca340 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:26.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:26.020+0000 7f5abeef1700 1 --2- 172.21.15.157:0/4086832792 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ab80c51c0 0x7f5ab80c2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:26.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:26.020+0000 7f5abeef1700 1 --2- 172.21.15.157:0/4086832792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ab814fde0 0x7f5ab8167410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:26.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:26.020+0000 7f5abeef1700 1 -- 172.21.15.157:0/4086832792 >> 172.21.15.157:0/4086832792 conn(0x7f5ab8067850 msgr2=0x7f5ab80c6090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:59:26.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:26.020+0000 7f5abeef1700 1 -- 172.21.15.157:0/4086832792 shutdown_connections 2023-10-06T03:59:26.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:26.020+0000 7f5abeef1700 1 -- 172.21.15.157:0/4086832792 wait complete. 2023-10-06T03:59:26.023 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 20 2023-10-06T03:59:26.034 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T03:59:28.034 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:59:28.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.161+0000 7f02f1eb3700 1 Processor -- start 2023-10-06T03:59:28.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.161+0000 7f02f1eb3700 1 -- start start 2023-10-06T03:59:28.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.162+0000 7f02f1eb3700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f02ec150410 0x7f02ec155f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:28.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.162+0000 7f02f1eb3700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02ec0c9780 con 0x7f02ec06dc40 2023-10-06T03:59:28.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.162+0000 7f02f1eb3700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02ec06e0c0 con 0x7f02ec06cbb0 2023-10-06T03:59:28.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.162+0000 7f02f1eb3700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02ec0c8c40 con 0x7f02ec150410 2023-10-06T03:59:28.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.162+0000 7f02ebfff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f02ec150410 0x7f02ec155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:28.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.162+0000 7f02e3fff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f02ec06dc40 0x7f02ec14fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:53676/0 (socket says 172.21.15.157:53676) 2023-10-06T03:59:28.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.162+0000 7f02e3fff700 1 -- 172.21.15.157:0/1576004756 learned_addr learned my addr 172.21.15.157:0/1576004756 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:59:28.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.163+0000 7f02ebfff700 1 -- 172.21.15.157:0/1576004756 >> v1:172.21.15.175:6789/0 conn(0x7f02ec06cbb0 legacy=0x7f02ec06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:28.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.163+0000 7f02ebfff700 1 -- 172.21.15.157:0/1576004756 >> v1:172.21.15.157:6789/0 conn(0x7f02ec06dc40 legacy=0x7f02ec14fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:28.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.163+0000 7f02eaffd700 1 -- 172.21.15.157:0/1576004756 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 169687055 0 0) 0x7f02ec0c9780 con 0x7f02ec06dc40 2023-10-06T03:59:28.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.163+0000 7f02ebfff700 1 -- 172.21.15.157:0/1576004756 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ec0c8e50 con 0x7f02ec150410 2023-10-06T03:59:28.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.163+0000 7f02ebfff700 1 --2- 172.21.15.157:0/1576004756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02ec150410 0x7f02ec155f70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f02e4009e60 tx=0x7f02e4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f7cacd9c6a8eb8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:28.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.163+0000 7f02eaffd700 1 -- 172.21.15.157:0/1576004756 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e400e070 con 0x7f02ec150410 2023-10-06T03:59:28.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.163+0000 7f02eaffd700 1 -- 172.21.15.157:0/1576004756 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e400bcc0 con 0x7f02ec150410 2023-10-06T03:59:28.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.163+0000 7f02eaffd700 1 -- 172.21.15.157:0/1576004756 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e4012680 con 0x7f02ec150410 2023-10-06T03:59:28.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.163+0000 7f02f1eb3700 1 -- 172.21.15.157:0/1576004756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02ec150410 msgr2=0x7f02ec155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:28.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.163+0000 7f02f1eb3700 1 --2- 172.21.15.157:0/1576004756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02ec150410 0x7f02ec155f70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f02e4009e60 tx=0x7f02e4009780 comp rx=0 tx=0).stop 2023-10-06T03:59:28.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.163+0000 7f02f1eb3700 1 -- 172.21.15.157:0/1576004756 shutdown_connections 2023-10-06T03:59:28.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.163+0000 7f02f1eb3700 1 --2- 172.21.15.157:0/1576004756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02ec150410 0x7f02ec155f70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:28.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.163+0000 7f02f1eb3700 1 -- 172.21.15.157:0/1576004756 >> 172.21.15.157:0/1576004756 conn(0x7f02ec06f230 msgr2=0x7f02ec06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:59:28.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.164+0000 7f02f1eb3700 1 -- 172.21.15.157:0/1576004756 shutdown_connections 2023-10-06T03:59:28.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.164+0000 7f02f1eb3700 1 -- 172.21.15.157:0/1576004756 wait complete. 2023-10-06T03:59:28.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.165+0000 7f02f1eb3700 1 Processor -- start 2023-10-06T03:59:28.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.165+0000 7f02f1eb3700 1 -- start start 2023-10-06T03:59:28.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.165+0000 7f02f1eb3700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02ec06cbb0 0x7f02ec15e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:28.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.165+0000 7f02f1eb3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02ec150410 0x7f02ec156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:28.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.165+0000 7f02f1eb3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02ec14bde0 0x7f02ec169330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:28.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.165+0000 7f02f1eb3700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec155860 con 0x7f02ec150410 2023-10-06T03:59:28.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.165+0000 7f02f1eb3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec156290 con 0x7f02ec14bde0 2023-10-06T03:59:28.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.165+0000 7f02f1eb3700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec0b0060 con 0x7f02ec06cbb0 2023-10-06T03:59:28.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.165+0000 7f02e3fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02ec150410 0x7f02ec156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:28.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.165+0000 7f02e3fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02ec150410 0x7f02ec156720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59512/0 (socket says 172.21.15.157:59512) 2023-10-06T03:59:28.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.165+0000 7f02e3fff700 1 -- 172.21.15.157:0/3993912038 learned_addr learned my addr 172.21.15.157:0/3993912038 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:59:28.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.166+0000 7f02ebfff700 1 --2- 172.21.15.157:0/3993912038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02ec14bde0 0x7f02ec169330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:28.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.166+0000 7f02eb7fe700 1 --2- 172.21.15.157:0/3993912038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02ec06cbb0 0x7f02ec15e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:28.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.166+0000 7f02e3fff700 1 -- 172.21.15.157:0/3993912038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02ec06cbb0 msgr2=0x7f02ec15e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:28.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.166+0000 7f02e3fff700 1 --2- 172.21.15.157:0/3993912038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02ec06cbb0 0x7f02ec15e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:28.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.166+0000 7f02e3fff700 1 -- 172.21.15.157:0/3993912038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02ec14bde0 msgr2=0x7f02ec169330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:28.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.166+0000 7f02e3fff700 1 --2- 172.21.15.157:0/3993912038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02ec14bde0 0x7f02ec169330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:28.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.166+0000 7f02e3fff700 1 -- 172.21.15.157:0/3993912038 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e4009a20 con 0x7f02ec150410 2023-10-06T03:59:28.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.166+0000 7f02e3fff700 1 --2- 172.21.15.157:0/3993912038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02ec150410 0x7f02ec156720 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f02d400c810 tx=0x7f02d4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:28.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.166+0000 7f02e37fe700 1 -- 172.21.15.157:0/3993912038 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d4005b30 con 0x7f02ec150410 2023-10-06T03:59:28.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.166+0000 7f02e37fe700 1 -- 172.21.15.157:0/3993912038 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d4005c90 con 0x7f02ec150410 2023-10-06T03:59:28.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.166+0000 7f02f1eb3700 1 -- 172.21.15.157:0/3993912038 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02ec1552c0 con 0x7f02ec150410 2023-10-06T03:59:28.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.166+0000 7f02e37fe700 1 -- 172.21.15.157:0/3993912038 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d4007300 con 0x7f02ec150410 2023-10-06T03:59:28.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.166+0000 7f02f1eb3700 1 -- 172.21.15.157:0/3993912038 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02ec155c40 con 0x7f02ec150410 2023-10-06T03:59:28.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.167+0000 7f02f1eb3700 1 -- 172.21.15.157:0/3993912038 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02ec089ca0 con 0x7f02ec150410 2023-10-06T03:59:28.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.168+0000 7f02e37fe700 1 -- 172.21.15.157:0/3993912038 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f02d401b3b0 con 0x7f02ec150410 2023-10-06T03:59:28.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.168+0000 7f02e37fe700 1 --2- 172.21.15.157:0/3993912038 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f02cc066f60 0x7f02cc069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:28.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.168+0000 7f02e37fe700 1 -- 172.21.15.157:0/3993912038 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f02d4083b30 con 0x7f02ec150410 2023-10-06T03:59:28.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.168+0000 7f02eb7fe700 1 --2- 172.21.15.157:0/3993912038 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f02cc066f60 0x7f02cc069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:28.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.169+0000 7f02eb7fe700 1 --2- 172.21.15.157:0/3993912038 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f02cc066f60 0x7f02cc069410 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f02dc0039e0 tx=0x7f02dc007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:28.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.172+0000 7f02e37fe700 1 -- 172.21.15.157:0/3993912038 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02d4050bb0 con 0x7f02ec150410 2023-10-06T03:59:28.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.399+0000 7f02f1eb3700 1 -- 172.21.15.157:0/3993912038 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02ec1563d0 con 0x7f02ec150410 2023-10-06T03:59:28.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.399+0000 7f02e37fe700 1 -- 172.21.15.157:0/3993912038 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+5427 (secure 0 0 0) 0x7f02d40541d0 con 0x7f02ec150410 2023-10-06T03:59:28.401 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:59:28.401 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T03:59:27.684344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4223":{"gid":4223,"name":"a","rank":4,"incarnation":17,"state":"up:active","state_seq":39,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:59:28.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.401+0000 7f02f1eb3700 1 -- 172.21.15.157:0/3993912038 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f02cc066f60 msgr2=0x7f02cc069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:28.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.401+0000 7f02f1eb3700 1 --2- 172.21.15.157:0/3993912038 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f02cc066f60 0x7f02cc069410 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f02dc0039e0 tx=0x7f02dc007ec0 comp rx=0 tx=0).stop 2023-10-06T03:59:28.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.401+0000 7f02f1eb3700 1 -- 172.21.15.157:0/3993912038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02ec150410 msgr2=0x7f02ec156720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:28.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.401+0000 7f02f1eb3700 1 --2- 172.21.15.157:0/3993912038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02ec150410 0x7f02ec156720 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f02d400c810 tx=0x7f02d4007df0 comp rx=0 tx=0).stop 2023-10-06T03:59:28.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.401+0000 7f02f1eb3700 1 -- 172.21.15.157:0/3993912038 shutdown_connections 2023-10-06T03:59:28.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.401+0000 7f02f1eb3700 1 --2- 172.21.15.157:0/3993912038 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f02cc066f60 0x7f02cc069410 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:28.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.401+0000 7f02f1eb3700 1 --2- 172.21.15.157:0/3993912038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02ec14bde0 0x7f02ec169330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:28.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.401+0000 7f02f1eb3700 1 --2- 172.21.15.157:0/3993912038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02ec06cbb0 0x7f02ec15e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:28.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.401+0000 7f02f1eb3700 1 --2- 172.21.15.157:0/3993912038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02ec150410 0x7f02ec156720 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:28.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.401+0000 7f02f1eb3700 1 -- 172.21.15.157:0/3993912038 >> 172.21.15.157:0/3993912038 conn(0x7f02ec06f230 msgr2=0x7f02ec14fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:59:28.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.401+0000 7f02f1eb3700 1 -- 172.21.15.157:0/3993912038 shutdown_connections 2023-10-06T03:59:28.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:28.401+0000 7f02f1eb3700 1 -- 172.21.15.157:0/3993912038 wait complete. 2023-10-06T03:59:28.404 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 21 2023-10-06T03:59:28.415 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=4, replacing gid=4241 2023-10-06T03:59:28.415 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 1 secs before reviving mds.f 2023-10-06T03:59:29.415 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.f 2023-10-06T03:59:29.416 INFO:tasks.ceph.mds.f:Restarting daemon 2023-10-06T03:59:29.416 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2023-10-06T03:59:29.419 INFO:tasks.ceph.mds.f:Started 2023-10-06T03:59:29.419 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:59:29.499 INFO:tasks.ceph.mds.f.smithi175.stdout:starting mds.f at 2023-10-06T03:59:29.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.538+0000 7feaf4901700 1 Processor -- start 2023-10-06T03:59:29.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.538+0000 7feaf4901700 1 -- start start 2023-10-06T03:59:29.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.539+0000 7feaf4901700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7feaf006dc40 0x7feaf014fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:29.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.539+0000 7feaf4901700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf00c9780 con 0x7feaf006cbb0 2023-10-06T03:59:29.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.539+0000 7feaf4901700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf006e0c0 con 0x7feaf006c1f0 2023-10-06T03:59:29.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.539+0000 7feaf4901700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf00c8c40 con 0x7feaf006dc40 2023-10-06T03:59:29.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.539+0000 7feaee59c700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7feaf006cbb0 0x7feaf006d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:53688/0 (socket says 172.21.15.157:53688) 2023-10-06T03:59:29.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.539+0000 7feaee59c700 1 -- 172.21.15.157:0/2330661234 learned_addr learned my addr 172.21.15.157:0/2330661234 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T03:59:29.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.539+0000 7feaedd9b700 1 --2- 172.21.15.157:0/2330661234 >> v2:172.21.15.175:3300/0 conn(0x7feaf006dc40 0x7feaf014fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:29.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.539+0000 7feaed59a700 1 -- 172.21.15.157:0/2330661234 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2565761391 0 0) 0x7feaf00c9780 con 0x7feaf006cbb0 2023-10-06T03:59:29.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.539+0000 7feaed59a700 1 -- 172.21.15.157:0/2330661234 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fead8003570 con 0x7feaf006cbb0 2023-10-06T03:59:29.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.539+0000 7feaed59a700 1 -- 172.21.15.157:0/2330661234 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1934051505 0 0) 0x7feaf006e0c0 con 0x7feaf006c1f0 2023-10-06T03:59:29.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.539+0000 7feaed59a700 1 -- 172.21.15.157:0/2330661234 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaf00c9780 con 0x7feaf006c1f0 2023-10-06T03:59:29.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.539+0000 7feaedd9b700 1 -- 172.21.15.157:0/2330661234 >> v1:172.21.15.175:6789/0 conn(0x7feaf006c1f0 legacy=0x7feaf0155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:29.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.539+0000 7feaedd9b700 1 -- 172.21.15.157:0/2330661234 >> v1:172.21.15.157:6789/0 conn(0x7feaf006cbb0 legacy=0x7feaf006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:29.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.539+0000 7feaedd9b700 1 -- 172.21.15.157:0/2330661234 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf00c8e50 con 0x7feaf006dc40 2023-10-06T03:59:29.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.540+0000 7feaedd9b700 1 --2- 172.21.15.157:0/2330661234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaf006dc40 0x7feaf014fd00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feae40098e0 tx=0x7feae40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=668812e54502569c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:29.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.540+0000 7feaed59a700 1 -- 172.21.15.157:0/2330661234 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae4011630 con 0x7feaf006dc40 2023-10-06T03:59:29.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.540+0000 7feaed59a700 1 -- 172.21.15.157:0/2330661234 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae4011790 con 0x7feaf006dc40 2023-10-06T03:59:29.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.540+0000 7feaed59a700 1 -- 172.21.15.157:0/2330661234 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae4011b30 con 0x7feaf006dc40 2023-10-06T03:59:29.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.540+0000 7feaf4901700 1 -- 172.21.15.157:0/2330661234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaf006dc40 msgr2=0x7feaf014fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:29.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.540+0000 7feaf4901700 1 --2- 172.21.15.157:0/2330661234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaf006dc40 0x7feaf014fd00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feae40098e0 tx=0x7feae40091a0 comp rx=0 tx=0).stop 2023-10-06T03:59:29.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.540+0000 7feaf4901700 1 -- 172.21.15.157:0/2330661234 shutdown_connections 2023-10-06T03:59:29.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.540+0000 7feaf4901700 1 --2- 172.21.15.157:0/2330661234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaf006dc40 0x7feaf014fd00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:29.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.540+0000 7feaf4901700 1 -- 172.21.15.157:0/2330661234 >> 172.21.15.157:0/2330661234 conn(0x7feaf006f230 msgr2=0x7feaf006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:59:29.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.540+0000 7feaf4901700 1 -- 172.21.15.157:0/2330661234 shutdown_connections 2023-10-06T03:59:29.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.540+0000 7feaf4901700 1 -- 172.21.15.157:0/2330661234 wait complete. 2023-10-06T03:59:29.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.541+0000 7feaf4901700 1 Processor -- start 2023-10-06T03:59:29.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.541+0000 7feaf4901700 1 -- start start 2023-10-06T03:59:29.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.542+0000 7feaf4901700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaf006c1f0 0x7feaf0154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:29.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.542+0000 7feaf4901700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaf006cbb0 0x7feaf0154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:29.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.542+0000 7feaf4901700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feaf01565c0 0x7feaf0145e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:29.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.542+0000 7feaf4901700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf0155d70 con 0x7feaf01565c0 2023-10-06T03:59:29.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.542+0000 7feaf4901700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf01562d0 con 0x7feaf006cbb0 2023-10-06T03:59:29.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.542+0000 7feaf4901700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf0147de0 con 0x7feaf006c1f0 2023-10-06T03:59:29.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.542+0000 7feaeed9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feaf01565c0 0x7feaf0145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:29.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.542+0000 7feaeed9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feaf01565c0 0x7feaf0145e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59526/0 (socket says 172.21.15.157:59526) 2023-10-06T03:59:29.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.542+0000 7feaeed9d700 1 -- 172.21.15.157:0/237107529 learned_addr learned my addr 172.21.15.157:0/237107529 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T03:59:29.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.542+0000 7feaedd9b700 1 --2- 172.21.15.157:0/237107529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaf006cbb0 0x7feaf0154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:29.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.542+0000 7feaee59c700 1 --2- 172.21.15.157:0/237107529 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaf006c1f0 0x7feaf0154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:29.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.543+0000 7feaeed9d700 1 -- 172.21.15.157:0/237107529 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaf006c1f0 msgr2=0x7feaf0154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:29.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.543+0000 7feaeed9d700 1 --2- 172.21.15.157:0/237107529 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaf006c1f0 0x7feaf0154150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:29.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.543+0000 7feaeed9d700 1 -- 172.21.15.157:0/237107529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaf006cbb0 msgr2=0x7feaf0154690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:29.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.543+0000 7feaeed9d700 1 --2- 172.21.15.157:0/237107529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaf006cbb0 0x7feaf0154690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:29.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.543+0000 7feaeed9d700 1 -- 172.21.15.157:0/237107529 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae4009420 con 0x7feaf01565c0 2023-10-06T03:59:29.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.543+0000 7feaedd9b700 1 --2- 172.21.15.157:0/237107529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaf006cbb0 0x7feaf0154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:59:29.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.543+0000 7feaeed9d700 1 --2- 172.21.15.157:0/237107529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feaf01565c0 0x7feaf0145e10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7feae800c910 tx=0x7feae80027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:29.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.543+0000 7feade7fc700 1 -- 172.21.15.157:0/237107529 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae8012630 con 0x7feaf01565c0 2023-10-06T03:59:29.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.543+0000 7feaf4901700 1 -- 172.21.15.157:0/237107529 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf0155ff0 con 0x7feaf01565c0 2023-10-06T03:59:29.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.543+0000 7feade7fc700 1 -- 172.21.15.157:0/237107529 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae80127b0 con 0x7feaf01565c0 2023-10-06T03:59:29.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.543+0000 7feade7fc700 1 -- 172.21.15.157:0/237107529 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae8012b70 con 0x7feaf01565c0 2023-10-06T03:59:29.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.543+0000 7feaf4901700 1 -- 172.21.15.157:0/237107529 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf00c8410 con 0x7feaf01565c0 2023-10-06T03:59:29.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.544+0000 7feaf4901700 1 -- 172.21.15.157:0/237107529 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf0153e70 con 0x7feaf01565c0 2023-10-06T03:59:29.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.544+0000 7feade7fc700 1 -- 172.21.15.157:0/237107529 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feae801b400 con 0x7feaf01565c0 2023-10-06T03:59:29.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.545+0000 7feade7fc700 1 --2- 172.21.15.157:0/237107529 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fead0066f30 0x7fead00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:59:29.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.545+0000 7feade7fc700 1 -- 172.21.15.157:0/237107529 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6202+0+0 (secure 0 0 0) 0x7feae8084b50 con 0x7feaf01565c0 2023-10-06T03:59:29.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.545+0000 7feaee59c700 1 --2- 172.21.15.157:0/237107529 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fead0066f30 0x7fead00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:59:29.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.545+0000 7feaee59c700 1 --2- 172.21.15.157:0/237107529 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fead0066f30 0x7fead00693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7feae0007fd0 tx=0x7feae0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:59:29.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.549+0000 7feade7fc700 1 -- 172.21.15.157:0/237107529 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feaf0153e70 con 0x7feaf01565c0 2023-10-06T03:59:29.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.776+0000 7feaf4901700 1 -- 172.21.15.157:0/237107529 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feaf0153e70 con 0x7feaf01565c0 2023-10-06T03:59:29.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.777+0000 7feade7fc700 1 -- 172.21.15.157:0/237107529 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+6186 (secure 0 0 0) 0x7feaf0153e70 con 0x7feaf01565c0 2023-10-06T03:59:29.778 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T03:59:29.779 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4936,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T03:59:27.684344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4223":{"gid":4223,"name":"a","rank":4,"incarnation":17,"state":"up:active","state_seq":39,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:59:29.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.779+0000 7feaf4901700 1 -- 172.21.15.157:0/237107529 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fead0066f30 msgr2=0x7fead00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:29.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.779+0000 7feaf4901700 1 --2- 172.21.15.157:0/237107529 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fead0066f30 0x7fead00693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7feae0007fd0 tx=0x7feae0009040 comp rx=0 tx=0).stop 2023-10-06T03:59:29.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.779+0000 7feaf4901700 1 -- 172.21.15.157:0/237107529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feaf01565c0 msgr2=0x7feaf0145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:59:29.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.779+0000 7feaf4901700 1 --2- 172.21.15.157:0/237107529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feaf01565c0 0x7feaf0145e10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7feae800c910 tx=0x7feae80027a0 comp rx=0 tx=0).stop 2023-10-06T03:59:29.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.779+0000 7feaf4901700 1 -- 172.21.15.157:0/237107529 shutdown_connections 2023-10-06T03:59:29.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.779+0000 7feaf4901700 1 --2- 172.21.15.157:0/237107529 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fead0066f30 0x7fead00693e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:29.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.779+0000 7feaf4901700 1 --2- 172.21.15.157:0/237107529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feaf01565c0 0x7feaf0145e10 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:29.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.779+0000 7feaf4901700 1 --2- 172.21.15.157:0/237107529 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaf006c1f0 0x7feaf0154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:29.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.779+0000 7feaf4901700 1 --2- 172.21.15.157:0/237107529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaf006cbb0 0x7feaf0154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:59:29.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.779+0000 7feaf4901700 1 -- 172.21.15.157:0/237107529 >> 172.21.15.157:0/237107529 conn(0x7feaf006f230 msgr2=0x7feaf014e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:59:29.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.779+0000 7feaf4901700 1 -- 172.21.15.157:0/237107529 shutdown_connections 2023-10-06T03:59:29.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T03:59:29.779+0000 7feaf4901700 1 -- 172.21.15.157:0/237107529 wait complete. 2023-10-06T03:59:29.782 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 22 2023-10-06T03:59:29.792 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f reported in up:standby state 2023-10-06T03:59:29.793 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 74 secs before thrashing 2023-10-06T03:59:32.709 INFO:tasks.workunit.client.0.smithi157.stdout:Create dir 200 to 299 ... 2023-10-06T03:59:32.710 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 200 2023-10-06T03:59:32.724 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 201 2023-10-06T03:59:32.737 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 202 2023-10-06T03:59:32.750 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 203 2023-10-06T03:59:32.763 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 204 2023-10-06T03:59:32.776 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 205 2023-10-06T03:59:32.789 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 206 2023-10-06T03:59:32.801 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 207 2023-10-06T03:59:32.814 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 208 2023-10-06T03:59:32.826 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 209 2023-10-06T03:59:32.838 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 210 2023-10-06T03:59:32.850 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 211 2023-10-06T03:59:32.860 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 212 2023-10-06T03:59:32.868 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 213 2023-10-06T03:59:32.876 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 214 2023-10-06T03:59:32.885 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 215 2023-10-06T03:59:32.894 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 216 2023-10-06T03:59:32.902 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 217 2023-10-06T03:59:32.911 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 218 2023-10-06T03:59:32.919 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 219 2023-10-06T03:59:32.928 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 220 2023-10-06T03:59:32.937 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 221 2023-10-06T03:59:32.945 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 222 2023-10-06T03:59:32.954 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 223 2023-10-06T03:59:32.962 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 224 2023-10-06T03:59:33.109 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 225 2023-10-06T03:59:33.118 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 226 2023-10-06T03:59:33.127 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 227 2023-10-06T03:59:33.136 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 228 2023-10-06T03:59:33.144 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 229 2023-10-06T03:59:33.153 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 230 2023-10-06T03:59:33.161 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 231 2023-10-06T03:59:33.170 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 232 2023-10-06T03:59:33.179 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 233 2023-10-06T03:59:33.187 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 234 2023-10-06T03:59:33.196 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 235 2023-10-06T03:59:33.205 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 236 2023-10-06T03:59:33.213 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 237 2023-10-06T03:59:33.222 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 238 2023-10-06T03:59:33.231 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 239 2023-10-06T03:59:33.239 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 240 2023-10-06T03:59:33.252 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 241 2023-10-06T03:59:33.263 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 242 2023-10-06T03:59:33.272 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 243 2023-10-06T03:59:33.280 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 244 2023-10-06T03:59:33.289 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 245 2023-10-06T03:59:33.297 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 246 2023-10-06T03:59:33.306 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 247 2023-10-06T03:59:33.315 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 248 2023-10-06T03:59:33.323 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 249 2023-10-06T03:59:33.332 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 250 2023-10-06T03:59:33.347 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 251 2023-10-06T03:59:36.436 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 252 2023-10-06T03:59:36.445 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 253 2023-10-06T03:59:36.454 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 254 2023-10-06T03:59:36.463 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 255 2023-10-06T03:59:36.472 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 256 2023-10-06T03:59:36.481 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 257 2023-10-06T03:59:36.489 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 258 2023-10-06T03:59:36.498 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 259 2023-10-06T03:59:36.507 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 260 2023-10-06T03:59:36.515 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 261 2023-10-06T03:59:36.524 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 262 2023-10-06T03:59:36.793 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 263 2023-10-06T03:59:36.850 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 264 2023-10-06T03:59:36.858 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 265 2023-10-06T03:59:36.867 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 266 2023-10-06T03:59:36.876 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 267 2023-10-06T03:59:36.885 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 268 2023-10-06T03:59:36.893 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 269 2023-10-06T03:59:36.902 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 270 2023-10-06T03:59:36.911 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 271 2023-10-06T03:59:36.919 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 272 2023-10-06T03:59:36.928 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 273 2023-10-06T03:59:36.937 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 274 2023-10-06T03:59:36.946 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 275 2023-10-06T03:59:36.955 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 276 2023-10-06T03:59:36.963 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 277 2023-10-06T03:59:36.972 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 278 2023-10-06T03:59:36.981 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 279 2023-10-06T03:59:37.380 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 280 2023-10-06T03:59:37.388 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 281 2023-10-06T03:59:37.397 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 282 2023-10-06T03:59:37.405 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 283 2023-10-06T03:59:37.413 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 284 2023-10-06T03:59:37.421 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 285 2023-10-06T03:59:37.889 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 286 2023-10-06T03:59:37.897 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 287 2023-10-06T03:59:37.906 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 288 2023-10-06T03:59:37.914 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 289 2023-10-06T03:59:37.923 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 290 2023-10-06T03:59:37.931 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 291 2023-10-06T03:59:37.940 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 292 2023-10-06T03:59:37.949 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 293 2023-10-06T03:59:38.172 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 294 2023-10-06T03:59:38.182 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 295 2023-10-06T03:59:38.190 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 296 2023-10-06T03:59:38.199 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 297 2023-10-06T03:59:38.208 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 298 2023-10-06T03:59:38.216 INFO:tasks.workunit.client.0.smithi157.stdout: create dir 299 2023-10-06T03:59:38.224 INFO:tasks.workunit.client.0.smithi157.stdout:Create a snapshot in every first level dir ... 2023-10-06T03:59:38.364 INFO:tasks.workunit.client.0.smithi157.stdout: create 100/.snap/snap-subdir-test 2023-10-06T03:59:39.395 INFO:tasks.workunit.client.0.smithi157.stdout: create 100/30 file after the snapshot 2023-10-06T03:59:39.396 INFO:tasks.workunit.client.0.smithi157.stdout: create 100/31 file after the snapshot 2023-10-06T03:59:39.397 INFO:tasks.workunit.client.0.smithi157.stdout: create 100/32 file after the snapshot 2023-10-06T03:59:39.397 INFO:tasks.workunit.client.0.smithi157.stdout: create 100/33 file after the snapshot 2023-10-06T03:59:39.398 INFO:tasks.workunit.client.0.smithi157.stdout: create 100/34 file after the snapshot 2023-10-06T03:59:39.399 INFO:tasks.workunit.client.0.smithi157.stdout: create 100/35 file after the snapshot 2023-10-06T03:59:39.399 INFO:tasks.workunit.client.0.smithi157.stdout: create 100/36 file after the snapshot 2023-10-06T03:59:39.400 INFO:tasks.workunit.client.0.smithi157.stdout: create 100/37 file after the snapshot 2023-10-06T03:59:39.401 INFO:tasks.workunit.client.0.smithi157.stdout: create 100/38 file after the snapshot 2023-10-06T03:59:39.401 INFO:tasks.workunit.client.0.smithi157.stdout: create 100/39 file after the snapshot 2023-10-06T03:59:39.402 INFO:tasks.workunit.client.0.smithi157.stdout: create 101/.snap/snap-subdir-test 2023-10-06T03:59:39.440 INFO:tasks.workunit.client.0.smithi157.stdout: create 101/30 file after the snapshot 2023-10-06T03:59:39.440 INFO:tasks.workunit.client.0.smithi157.stdout: create 101/31 file after the snapshot 2023-10-06T03:59:39.441 INFO:tasks.workunit.client.0.smithi157.stdout: create 101/32 file after the snapshot 2023-10-06T03:59:39.442 INFO:tasks.workunit.client.0.smithi157.stdout: create 101/33 file after the snapshot 2023-10-06T03:59:39.442 INFO:tasks.workunit.client.0.smithi157.stdout: create 101/34 file after the snapshot 2023-10-06T03:59:39.443 INFO:tasks.workunit.client.0.smithi157.stdout: create 101/35 file after the snapshot 2023-10-06T03:59:39.444 INFO:tasks.workunit.client.0.smithi157.stdout: create 101/36 file after the snapshot 2023-10-06T03:59:39.446 INFO:tasks.workunit.client.0.smithi157.stdout: create 101/37 file after the snapshot 2023-10-06T03:59:39.446 INFO:tasks.workunit.client.0.smithi157.stdout: create 101/38 file after the snapshot 2023-10-06T03:59:39.446 INFO:tasks.workunit.client.0.smithi157.stdout: create 101/39 file after the snapshot 2023-10-06T03:59:39.447 INFO:tasks.workunit.client.0.smithi157.stdout: create 102/.snap/snap-subdir-test 2023-10-06T03:59:39.521 INFO:tasks.workunit.client.0.smithi157.stdout: create 102/30 file after the snapshot 2023-10-06T03:59:39.521 INFO:tasks.workunit.client.0.smithi157.stdout: create 102/31 file after the snapshot 2023-10-06T03:59:39.522 INFO:tasks.workunit.client.0.smithi157.stdout: create 102/32 file after the snapshot 2023-10-06T03:59:39.523 INFO:tasks.workunit.client.0.smithi157.stdout: create 102/33 file after the snapshot 2023-10-06T03:59:39.523 INFO:tasks.workunit.client.0.smithi157.stdout: create 102/34 file after the snapshot 2023-10-06T03:59:39.524 INFO:tasks.workunit.client.0.smithi157.stdout: create 102/35 file after the snapshot 2023-10-06T03:59:39.525 INFO:tasks.workunit.client.0.smithi157.stdout: create 102/36 file after the snapshot 2023-10-06T03:59:39.525 INFO:tasks.workunit.client.0.smithi157.stdout: create 102/37 file after the snapshot 2023-10-06T03:59:39.526 INFO:tasks.workunit.client.0.smithi157.stdout: create 102/38 file after the snapshot 2023-10-06T03:59:39.526 INFO:tasks.workunit.client.0.smithi157.stdout: create 102/39 file after the snapshot 2023-10-06T03:59:39.527 INFO:tasks.workunit.client.0.smithi157.stdout: create 103/.snap/snap-subdir-test 2023-10-06T03:59:39.558 INFO:tasks.workunit.client.0.smithi157.stdout: create 103/30 file after the snapshot 2023-10-06T03:59:39.559 INFO:tasks.workunit.client.0.smithi157.stdout: create 103/31 file after the snapshot 2023-10-06T03:59:39.559 INFO:tasks.workunit.client.0.smithi157.stdout: create 103/32 file after the snapshot 2023-10-06T03:59:39.560 INFO:tasks.workunit.client.0.smithi157.stdout: create 103/33 file after the snapshot 2023-10-06T03:59:39.561 INFO:tasks.workunit.client.0.smithi157.stdout: create 103/34 file after the snapshot 2023-10-06T03:59:39.561 INFO:tasks.workunit.client.0.smithi157.stdout: create 103/35 file after the snapshot 2023-10-06T03:59:39.562 INFO:tasks.workunit.client.0.smithi157.stdout: create 103/36 file after the snapshot 2023-10-06T03:59:39.563 INFO:tasks.workunit.client.0.smithi157.stdout: create 103/37 file after the snapshot 2023-10-06T03:59:39.563 INFO:tasks.workunit.client.0.smithi157.stdout: create 103/38 file after the snapshot 2023-10-06T03:59:39.564 INFO:tasks.workunit.client.0.smithi157.stdout: create 103/39 file after the snapshot 2023-10-06T03:59:39.565 INFO:tasks.workunit.client.0.smithi157.stdout: create 104/.snap/snap-subdir-test 2023-10-06T03:59:39.595 INFO:tasks.workunit.client.0.smithi157.stdout: create 104/30 file after the snapshot 2023-10-06T03:59:39.596 INFO:tasks.workunit.client.0.smithi157.stdout: create 104/31 file after the snapshot 2023-10-06T03:59:39.596 INFO:tasks.workunit.client.0.smithi157.stdout: create 104/32 file after the snapshot 2023-10-06T03:59:39.597 INFO:tasks.workunit.client.0.smithi157.stdout: create 104/33 file after the snapshot 2023-10-06T03:59:39.598 INFO:tasks.workunit.client.0.smithi157.stdout: create 104/34 file after the snapshot 2023-10-06T03:59:39.598 INFO:tasks.workunit.client.0.smithi157.stdout: create 104/35 file after the snapshot 2023-10-06T03:59:39.599 INFO:tasks.workunit.client.0.smithi157.stdout: create 104/36 file after the snapshot 2023-10-06T03:59:39.600 INFO:tasks.workunit.client.0.smithi157.stdout: create 104/37 file after the snapshot 2023-10-06T03:59:39.600 INFO:tasks.workunit.client.0.smithi157.stdout: create 104/38 file after the snapshot 2023-10-06T03:59:39.601 INFO:tasks.workunit.client.0.smithi157.stdout: create 104/39 file after the snapshot 2023-10-06T03:59:39.602 INFO:tasks.workunit.client.0.smithi157.stdout: create 105/.snap/snap-subdir-test 2023-10-06T03:59:39.632 INFO:tasks.workunit.client.0.smithi157.stdout: create 105/30 file after the snapshot 2023-10-06T03:59:39.633 INFO:tasks.workunit.client.0.smithi157.stdout: create 105/31 file after the snapshot 2023-10-06T03:59:39.634 INFO:tasks.workunit.client.0.smithi157.stdout: create 105/32 file after the snapshot 2023-10-06T03:59:39.634 INFO:tasks.workunit.client.0.smithi157.stdout: create 105/33 file after the snapshot 2023-10-06T03:59:39.635 INFO:tasks.workunit.client.0.smithi157.stdout: create 105/34 file after the snapshot 2023-10-06T03:59:39.636 INFO:tasks.workunit.client.0.smithi157.stdout: create 105/35 file after the snapshot 2023-10-06T03:59:39.636 INFO:tasks.workunit.client.0.smithi157.stdout: create 105/36 file after the snapshot 2023-10-06T03:59:39.637 INFO:tasks.workunit.client.0.smithi157.stdout: create 105/37 file after the snapshot 2023-10-06T03:59:39.638 INFO:tasks.workunit.client.0.smithi157.stdout: create 105/38 file after the snapshot 2023-10-06T03:59:39.638 INFO:tasks.workunit.client.0.smithi157.stdout: create 105/39 file after the snapshot 2023-10-06T03:59:39.639 INFO:tasks.workunit.client.0.smithi157.stdout: create 106/.snap/snap-subdir-test 2023-10-06T03:59:39.669 INFO:tasks.workunit.client.0.smithi157.stdout: create 106/30 file after the snapshot 2023-10-06T03:59:39.670 INFO:tasks.workunit.client.0.smithi157.stdout: create 106/31 file after the snapshot 2023-10-06T03:59:39.671 INFO:tasks.workunit.client.0.smithi157.stdout: create 106/32 file after the snapshot 2023-10-06T03:59:39.671 INFO:tasks.workunit.client.0.smithi157.stdout: create 106/33 file after the snapshot 2023-10-06T03:59:39.672 INFO:tasks.workunit.client.0.smithi157.stdout: create 106/34 file after the snapshot 2023-10-06T03:59:39.673 INFO:tasks.workunit.client.0.smithi157.stdout: create 106/35 file after the snapshot 2023-10-06T03:59:39.673 INFO:tasks.workunit.client.0.smithi157.stdout: create 106/36 file after the snapshot 2023-10-06T03:59:39.674 INFO:tasks.workunit.client.0.smithi157.stdout: create 106/37 file after the snapshot 2023-10-06T03:59:39.675 INFO:tasks.workunit.client.0.smithi157.stdout: create 106/38 file after the snapshot 2023-10-06T03:59:39.675 INFO:tasks.workunit.client.0.smithi157.stdout: create 106/39 file after the snapshot 2023-10-06T03:59:39.676 INFO:tasks.workunit.client.0.smithi157.stdout: create 107/.snap/snap-subdir-test 2023-10-06T03:59:39.707 INFO:tasks.workunit.client.0.smithi157.stdout: create 107/30 file after the snapshot 2023-10-06T03:59:39.708 INFO:tasks.workunit.client.0.smithi157.stdout: create 107/31 file after the snapshot 2023-10-06T03:59:39.708 INFO:tasks.workunit.client.0.smithi157.stdout: create 107/32 file after the snapshot 2023-10-06T03:59:39.709 INFO:tasks.workunit.client.0.smithi157.stdout: create 107/33 file after the snapshot 2023-10-06T03:59:39.709 INFO:tasks.workunit.client.0.smithi157.stdout: create 107/34 file after the snapshot 2023-10-06T03:59:39.710 INFO:tasks.workunit.client.0.smithi157.stdout: create 107/35 file after the snapshot 2023-10-06T03:59:39.711 INFO:tasks.workunit.client.0.smithi157.stdout: create 107/36 file after the snapshot 2023-10-06T03:59:39.711 INFO:tasks.workunit.client.0.smithi157.stdout: create 107/37 file after the snapshot 2023-10-06T03:59:39.712 INFO:tasks.workunit.client.0.smithi157.stdout: create 107/38 file after the snapshot 2023-10-06T03:59:39.713 INFO:tasks.workunit.client.0.smithi157.stdout: create 107/39 file after the snapshot 2023-10-06T03:59:39.713 INFO:tasks.workunit.client.0.smithi157.stdout: create 108/.snap/snap-subdir-test 2023-10-06T03:59:39.745 INFO:tasks.workunit.client.0.smithi157.stdout: create 108/30 file after the snapshot 2023-10-06T03:59:39.746 INFO:tasks.workunit.client.0.smithi157.stdout: create 108/31 file after the snapshot 2023-10-06T03:59:39.746 INFO:tasks.workunit.client.0.smithi157.stdout: create 108/32 file after the snapshot 2023-10-06T03:59:39.747 INFO:tasks.workunit.client.0.smithi157.stdout: create 108/33 file after the snapshot 2023-10-06T03:59:39.747 INFO:tasks.workunit.client.0.smithi157.stdout: create 108/34 file after the snapshot 2023-10-06T03:59:39.748 INFO:tasks.workunit.client.0.smithi157.stdout: create 108/35 file after the snapshot 2023-10-06T03:59:39.749 INFO:tasks.workunit.client.0.smithi157.stdout: create 108/36 file after the snapshot 2023-10-06T03:59:39.749 INFO:tasks.workunit.client.0.smithi157.stdout: create 108/37 file after the snapshot 2023-10-06T03:59:39.750 INFO:tasks.workunit.client.0.smithi157.stdout: create 108/38 file after the snapshot 2023-10-06T03:59:39.751 INFO:tasks.workunit.client.0.smithi157.stdout: create 108/39 file after the snapshot 2023-10-06T03:59:39.751 INFO:tasks.workunit.client.0.smithi157.stdout: create 109/.snap/snap-subdir-test 2023-10-06T03:59:39.782 INFO:tasks.workunit.client.0.smithi157.stdout: create 109/30 file after the snapshot 2023-10-06T03:59:39.783 INFO:tasks.workunit.client.0.smithi157.stdout: create 109/31 file after the snapshot 2023-10-06T03:59:39.784 INFO:tasks.workunit.client.0.smithi157.stdout: create 109/32 file after the snapshot 2023-10-06T03:59:39.785 INFO:tasks.workunit.client.0.smithi157.stdout: create 109/33 file after the snapshot 2023-10-06T03:59:39.785 INFO:tasks.workunit.client.0.smithi157.stdout: create 109/34 file after the snapshot 2023-10-06T03:59:39.786 INFO:tasks.workunit.client.0.smithi157.stdout: create 109/35 file after the snapshot 2023-10-06T03:59:39.787 INFO:tasks.workunit.client.0.smithi157.stdout: create 109/36 file after the snapshot 2023-10-06T03:59:39.787 INFO:tasks.workunit.client.0.smithi157.stdout: create 109/37 file after the snapshot 2023-10-06T03:59:39.788 INFO:tasks.workunit.client.0.smithi157.stdout: create 109/38 file after the snapshot 2023-10-06T03:59:39.789 INFO:tasks.workunit.client.0.smithi157.stdout: create 109/39 file after the snapshot 2023-10-06T03:59:39.789 INFO:tasks.workunit.client.0.smithi157.stdout: create 110/.snap/snap-subdir-test 2023-10-06T03:59:39.814 INFO:tasks.workunit.client.0.smithi157.stdout: create 110/30 file after the snapshot 2023-10-06T03:59:39.815 INFO:tasks.workunit.client.0.smithi157.stdout: create 110/31 file after the snapshot 2023-10-06T03:59:39.816 INFO:tasks.workunit.client.0.smithi157.stdout: create 110/32 file after the snapshot 2023-10-06T03:59:39.816 INFO:tasks.workunit.client.0.smithi157.stdout: create 110/33 file after the snapshot 2023-10-06T03:59:39.817 INFO:tasks.workunit.client.0.smithi157.stdout: create 110/34 file after the snapshot 2023-10-06T03:59:39.818 INFO:tasks.workunit.client.0.smithi157.stdout: create 110/35 file after the snapshot 2023-10-06T03:59:39.937 INFO:tasks.workunit.client.0.smithi157.stdout: create 110/36 file after the snapshot 2023-10-06T03:59:39.938 INFO:tasks.workunit.client.0.smithi157.stdout: create 110/37 file after the snapshot 2023-10-06T03:59:39.939 INFO:tasks.workunit.client.0.smithi157.stdout: create 110/38 file after the snapshot 2023-10-06T03:59:39.939 INFO:tasks.workunit.client.0.smithi157.stdout: create 110/39 file after the snapshot 2023-10-06T03:59:39.940 INFO:tasks.workunit.client.0.smithi157.stdout: create 111/.snap/snap-subdir-test 2023-10-06T03:59:40.699 INFO:tasks.workunit.client.0.smithi157.stdout: create 111/30 file after the snapshot 2023-10-06T03:59:40.700 INFO:tasks.workunit.client.0.smithi157.stdout: create 111/31 file after the snapshot 2023-10-06T03:59:40.701 INFO:tasks.workunit.client.0.smithi157.stdout: create 111/32 file after the snapshot 2023-10-06T03:59:40.702 INFO:tasks.workunit.client.0.smithi157.stdout: create 111/33 file after the snapshot 2023-10-06T03:59:40.702 INFO:tasks.workunit.client.0.smithi157.stdout: create 111/34 file after the snapshot 2023-10-06T03:59:40.703 INFO:tasks.workunit.client.0.smithi157.stdout: create 111/35 file after the snapshot 2023-10-06T03:59:40.704 INFO:tasks.workunit.client.0.smithi157.stdout: create 111/36 file after the snapshot 2023-10-06T03:59:40.705 INFO:tasks.workunit.client.0.smithi157.stdout: create 111/37 file after the snapshot 2023-10-06T03:59:40.705 INFO:tasks.workunit.client.0.smithi157.stdout: create 111/38 file after the snapshot 2023-10-06T03:59:40.706 INFO:tasks.workunit.client.0.smithi157.stdout: create 111/39 file after the snapshot 2023-10-06T03:59:40.707 INFO:tasks.workunit.client.0.smithi157.stdout: create 112/.snap/snap-subdir-test 2023-10-06T03:59:40.733 INFO:tasks.workunit.client.0.smithi157.stdout: create 112/30 file after the snapshot 2023-10-06T03:59:40.734 INFO:tasks.workunit.client.0.smithi157.stdout: create 112/31 file after the snapshot 2023-10-06T03:59:40.735 INFO:tasks.workunit.client.0.smithi157.stdout: create 112/32 file after the snapshot 2023-10-06T03:59:40.736 INFO:tasks.workunit.client.0.smithi157.stdout: create 112/33 file after the snapshot 2023-10-06T03:59:40.736 INFO:tasks.workunit.client.0.smithi157.stdout: create 112/34 file after the snapshot 2023-10-06T03:59:40.737 INFO:tasks.workunit.client.0.smithi157.stdout: create 112/35 file after the snapshot 2023-10-06T03:59:40.738 INFO:tasks.workunit.client.0.smithi157.stdout: create 112/36 file after the snapshot 2023-10-06T03:59:40.738 INFO:tasks.workunit.client.0.smithi157.stdout: create 112/37 file after the snapshot 2023-10-06T03:59:40.739 INFO:tasks.workunit.client.0.smithi157.stdout: create 112/38 file after the snapshot 2023-10-06T03:59:40.740 INFO:tasks.workunit.client.0.smithi157.stdout: create 112/39 file after the snapshot 2023-10-06T03:59:40.740 INFO:tasks.workunit.client.0.smithi157.stdout: create 113/.snap/snap-subdir-test 2023-10-06T03:59:40.765 INFO:tasks.workunit.client.0.smithi157.stdout: create 113/30 file after the snapshot 2023-10-06T03:59:40.766 INFO:tasks.workunit.client.0.smithi157.stdout: create 113/31 file after the snapshot 2023-10-06T03:59:40.767 INFO:tasks.workunit.client.0.smithi157.stdout: create 113/32 file after the snapshot 2023-10-06T03:59:40.767 INFO:tasks.workunit.client.0.smithi157.stdout: create 113/33 file after the snapshot 2023-10-06T03:59:40.768 INFO:tasks.workunit.client.0.smithi157.stdout: create 113/34 file after the snapshot 2023-10-06T03:59:40.769 INFO:tasks.workunit.client.0.smithi157.stdout: create 113/35 file after the snapshot 2023-10-06T03:59:40.769 INFO:tasks.workunit.client.0.smithi157.stdout: create 113/36 file after the snapshot 2023-10-06T03:59:40.770 INFO:tasks.workunit.client.0.smithi157.stdout: create 113/37 file after the snapshot 2023-10-06T03:59:40.771 INFO:tasks.workunit.client.0.smithi157.stdout: create 113/38 file after the snapshot 2023-10-06T03:59:40.771 INFO:tasks.workunit.client.0.smithi157.stdout: create 113/39 file after the snapshot 2023-10-06T03:59:40.772 INFO:tasks.workunit.client.0.smithi157.stdout: create 114/.snap/snap-subdir-test 2023-10-06T03:59:40.797 INFO:tasks.workunit.client.0.smithi157.stdout: create 114/30 file after the snapshot 2023-10-06T03:59:40.798 INFO:tasks.workunit.client.0.smithi157.stdout: create 114/31 file after the snapshot 2023-10-06T03:59:40.799 INFO:tasks.workunit.client.0.smithi157.stdout: create 114/32 file after the snapshot 2023-10-06T03:59:41.064 INFO:tasks.workunit.client.0.smithi157.stdout: create 114/33 file after the snapshot 2023-10-06T03:59:41.065 INFO:tasks.workunit.client.0.smithi157.stdout: create 114/34 file after the snapshot 2023-10-06T03:59:41.066 INFO:tasks.workunit.client.0.smithi157.stdout: create 114/35 file after the snapshot 2023-10-06T03:59:41.067 INFO:tasks.workunit.client.0.smithi157.stdout: create 114/36 file after the snapshot 2023-10-06T03:59:41.068 INFO:tasks.workunit.client.0.smithi157.stdout: create 114/37 file after the snapshot 2023-10-06T03:59:41.069 INFO:tasks.workunit.client.0.smithi157.stdout: create 114/38 file after the snapshot 2023-10-06T03:59:41.070 INFO:tasks.workunit.client.0.smithi157.stdout: create 114/39 file after the snapshot 2023-10-06T03:59:41.071 INFO:tasks.workunit.client.0.smithi157.stdout: create 115/.snap/snap-subdir-test 2023-10-06T03:59:41.097 INFO:tasks.workunit.client.0.smithi157.stdout: create 115/30 file after the snapshot 2023-10-06T03:59:41.098 INFO:tasks.workunit.client.0.smithi157.stdout: create 115/31 file after the snapshot 2023-10-06T03:59:41.099 INFO:tasks.workunit.client.0.smithi157.stdout: create 115/32 file after the snapshot 2023-10-06T03:59:41.100 INFO:tasks.workunit.client.0.smithi157.stdout: create 115/33 file after the snapshot 2023-10-06T03:59:41.101 INFO:tasks.workunit.client.0.smithi157.stdout: create 115/34 file after the snapshot 2023-10-06T03:59:41.102 INFO:tasks.workunit.client.0.smithi157.stdout: create 115/35 file after the snapshot 2023-10-06T03:59:41.103 INFO:tasks.workunit.client.0.smithi157.stdout: create 115/36 file after the snapshot 2023-10-06T03:59:41.104 INFO:tasks.workunit.client.0.smithi157.stdout: create 115/37 file after the snapshot 2023-10-06T03:59:41.105 INFO:tasks.workunit.client.0.smithi157.stdout: create 115/38 file after the snapshot 2023-10-06T03:59:41.106 INFO:tasks.workunit.client.0.smithi157.stdout: create 115/39 file after the snapshot 2023-10-06T03:59:41.107 INFO:tasks.workunit.client.0.smithi157.stdout: create 116/.snap/snap-subdir-test 2023-10-06T03:59:41.132 INFO:tasks.workunit.client.0.smithi157.stdout: create 116/30 file after the snapshot 2023-10-06T03:59:41.133 INFO:tasks.workunit.client.0.smithi157.stdout: create 116/31 file after the snapshot 2023-10-06T03:59:41.134 INFO:tasks.workunit.client.0.smithi157.stdout: create 116/32 file after the snapshot 2023-10-06T03:59:41.135 INFO:tasks.workunit.client.0.smithi157.stdout: create 116/33 file after the snapshot 2023-10-06T03:59:41.136 INFO:tasks.workunit.client.0.smithi157.stdout: create 116/34 file after the snapshot 2023-10-06T03:59:41.137 INFO:tasks.workunit.client.0.smithi157.stdout: create 116/35 file after the snapshot 2023-10-06T03:59:41.138 INFO:tasks.workunit.client.0.smithi157.stdout: create 116/36 file after the snapshot 2023-10-06T03:59:41.139 INFO:tasks.workunit.client.0.smithi157.stdout: create 116/37 file after the snapshot 2023-10-06T03:59:41.140 INFO:tasks.workunit.client.0.smithi157.stdout: create 116/38 file after the snapshot 2023-10-06T03:59:41.141 INFO:tasks.workunit.client.0.smithi157.stdout: create 116/39 file after the snapshot 2023-10-06T03:59:41.142 INFO:tasks.workunit.client.0.smithi157.stdout: create 117/.snap/snap-subdir-test 2023-10-06T03:59:41.167 INFO:tasks.workunit.client.0.smithi157.stdout: create 117/30 file after the snapshot 2023-10-06T03:59:41.168 INFO:tasks.workunit.client.0.smithi157.stdout: create 117/31 file after the snapshot 2023-10-06T03:59:41.168 INFO:tasks.workunit.client.0.smithi157.stdout: create 117/32 file after the snapshot 2023-10-06T03:59:41.169 INFO:tasks.workunit.client.0.smithi157.stdout: create 117/33 file after the snapshot 2023-10-06T03:59:41.170 INFO:tasks.workunit.client.0.smithi157.stdout: create 117/34 file after the snapshot 2023-10-06T03:59:41.170 INFO:tasks.workunit.client.0.smithi157.stdout: create 117/35 file after the snapshot 2023-10-06T03:59:41.171 INFO:tasks.workunit.client.0.smithi157.stdout: create 117/36 file after the snapshot 2023-10-06T03:59:41.172 INFO:tasks.workunit.client.0.smithi157.stdout: create 117/37 file after the snapshot 2023-10-06T03:59:41.172 INFO:tasks.workunit.client.0.smithi157.stdout: create 117/38 file after the snapshot 2023-10-06T03:59:41.173 INFO:tasks.workunit.client.0.smithi157.stdout: create 117/39 file after the snapshot 2023-10-06T03:59:41.174 INFO:tasks.workunit.client.0.smithi157.stdout: create 118/.snap/snap-subdir-test 2023-10-06T03:59:41.198 INFO:tasks.workunit.client.0.smithi157.stdout: create 118/30 file after the snapshot 2023-10-06T03:59:41.199 INFO:tasks.workunit.client.0.smithi157.stdout: create 118/31 file after the snapshot 2023-10-06T03:59:41.199 INFO:tasks.workunit.client.0.smithi157.stdout: create 118/32 file after the snapshot 2023-10-06T03:59:41.200 INFO:tasks.workunit.client.0.smithi157.stdout: create 118/33 file after the snapshot 2023-10-06T03:59:41.201 INFO:tasks.workunit.client.0.smithi157.stdout: create 118/34 file after the snapshot 2023-10-06T03:59:41.201 INFO:tasks.workunit.client.0.smithi157.stdout: create 118/35 file after the snapshot 2023-10-06T03:59:41.202 INFO:tasks.workunit.client.0.smithi157.stdout: create 118/36 file after the snapshot 2023-10-06T03:59:41.203 INFO:tasks.workunit.client.0.smithi157.stdout: create 118/37 file after the snapshot 2023-10-06T03:59:41.203 INFO:tasks.workunit.client.0.smithi157.stdout: create 118/38 file after the snapshot 2023-10-06T03:59:41.204 INFO:tasks.workunit.client.0.smithi157.stdout: create 118/39 file after the snapshot 2023-10-06T03:59:41.205 INFO:tasks.workunit.client.0.smithi157.stdout: create 119/.snap/snap-subdir-test 2023-10-06T03:59:41.229 INFO:tasks.workunit.client.0.smithi157.stdout: create 119/30 file after the snapshot 2023-10-06T03:59:41.230 INFO:tasks.workunit.client.0.smithi157.stdout: create 119/31 file after the snapshot 2023-10-06T03:59:41.230 INFO:tasks.workunit.client.0.smithi157.stdout: create 119/32 file after the snapshot 2023-10-06T03:59:41.231 INFO:tasks.workunit.client.0.smithi157.stdout: create 119/33 file after the snapshot 2023-10-06T03:59:41.232 INFO:tasks.workunit.client.0.smithi157.stdout: create 119/34 file after the snapshot 2023-10-06T03:59:41.232 INFO:tasks.workunit.client.0.smithi157.stdout: create 119/35 file after the snapshot 2023-10-06T03:59:41.233 INFO:tasks.workunit.client.0.smithi157.stdout: create 119/36 file after the snapshot 2023-10-06T03:59:41.234 INFO:tasks.workunit.client.0.smithi157.stdout: create 119/37 file after the snapshot 2023-10-06T03:59:41.234 INFO:tasks.workunit.client.0.smithi157.stdout: create 119/38 file after the snapshot 2023-10-06T03:59:41.235 INFO:tasks.workunit.client.0.smithi157.stdout: create 119/39 file after the snapshot 2023-10-06T03:59:41.236 INFO:tasks.workunit.client.0.smithi157.stdout: create 120/.snap/snap-subdir-test 2023-10-06T03:59:41.260 INFO:tasks.workunit.client.0.smithi157.stdout: create 120/30 file after the snapshot 2023-10-06T03:59:41.262 INFO:tasks.workunit.client.0.smithi157.stdout: create 120/31 file after the snapshot 2023-10-06T03:59:41.263 INFO:tasks.workunit.client.0.smithi157.stdout: create 120/32 file after the snapshot 2023-10-06T03:59:41.264 INFO:tasks.workunit.client.0.smithi157.stdout: create 120/33 file after the snapshot 2023-10-06T03:59:41.265 INFO:tasks.workunit.client.0.smithi157.stdout: create 120/34 file after the snapshot 2023-10-06T03:59:41.266 INFO:tasks.workunit.client.0.smithi157.stdout: create 120/35 file after the snapshot 2023-10-06T03:59:41.267 INFO:tasks.workunit.client.0.smithi157.stdout: create 120/36 file after the snapshot 2023-10-06T03:59:41.268 INFO:tasks.workunit.client.0.smithi157.stdout: create 120/37 file after the snapshot 2023-10-06T03:59:41.269 INFO:tasks.workunit.client.0.smithi157.stdout: create 120/38 file after the snapshot 2023-10-06T03:59:41.270 INFO:tasks.workunit.client.0.smithi157.stdout: create 120/39 file after the snapshot 2023-10-06T03:59:41.271 INFO:tasks.workunit.client.0.smithi157.stdout: create 121/.snap/snap-subdir-test 2023-10-06T03:59:41.297 INFO:tasks.workunit.client.0.smithi157.stdout: create 121/30 file after the snapshot 2023-10-06T03:59:41.298 INFO:tasks.workunit.client.0.smithi157.stdout: create 121/31 file after the snapshot 2023-10-06T03:59:41.298 INFO:tasks.workunit.client.0.smithi157.stdout: create 121/32 file after the snapshot 2023-10-06T03:59:41.299 INFO:tasks.workunit.client.0.smithi157.stdout: create 121/33 file after the snapshot 2023-10-06T03:59:41.300 INFO:tasks.workunit.client.0.smithi157.stdout: create 121/34 file after the snapshot 2023-10-06T03:59:41.300 INFO:tasks.workunit.client.0.smithi157.stdout: create 121/35 file after the snapshot 2023-10-06T03:59:41.301 INFO:tasks.workunit.client.0.smithi157.stdout: create 121/36 file after the snapshot 2023-10-06T03:59:41.302 INFO:tasks.workunit.client.0.smithi157.stdout: create 121/37 file after the snapshot 2023-10-06T03:59:41.302 INFO:tasks.workunit.client.0.smithi157.stdout: create 121/38 file after the snapshot 2023-10-06T03:59:41.303 INFO:tasks.workunit.client.0.smithi157.stdout: create 121/39 file after the snapshot 2023-10-06T03:59:41.304 INFO:tasks.workunit.client.0.smithi157.stdout: create 122/.snap/snap-subdir-test 2023-10-06T03:59:41.328 INFO:tasks.workunit.client.0.smithi157.stdout: create 122/30 file after the snapshot 2023-10-06T03:59:41.329 INFO:tasks.workunit.client.0.smithi157.stdout: create 122/31 file after the snapshot 2023-10-06T03:59:41.330 INFO:tasks.workunit.client.0.smithi157.stdout: create 122/32 file after the snapshot 2023-10-06T03:59:41.331 INFO:tasks.workunit.client.0.smithi157.stdout: create 122/33 file after the snapshot 2023-10-06T03:59:41.331 INFO:tasks.workunit.client.0.smithi157.stdout: create 122/34 file after the snapshot 2023-10-06T03:59:41.332 INFO:tasks.workunit.client.0.smithi157.stdout: create 122/35 file after the snapshot 2023-10-06T03:59:41.333 INFO:tasks.workunit.client.0.smithi157.stdout: create 122/36 file after the snapshot 2023-10-06T03:59:41.333 INFO:tasks.workunit.client.0.smithi157.stdout: create 122/37 file after the snapshot 2023-10-06T03:59:41.334 INFO:tasks.workunit.client.0.smithi157.stdout: create 122/38 file after the snapshot 2023-10-06T03:59:41.335 INFO:tasks.workunit.client.0.smithi157.stdout: create 122/39 file after the snapshot 2023-10-06T03:59:41.336 INFO:tasks.workunit.client.0.smithi157.stdout: create 123/.snap/snap-subdir-test 2023-10-06T03:59:41.358 INFO:tasks.workunit.client.0.smithi157.stdout: create 123/30 file after the snapshot 2023-10-06T03:59:41.359 INFO:tasks.workunit.client.0.smithi157.stdout: create 123/31 file after the snapshot 2023-10-06T03:59:41.360 INFO:tasks.workunit.client.0.smithi157.stdout: create 123/32 file after the snapshot 2023-10-06T03:59:41.361 INFO:tasks.workunit.client.0.smithi157.stdout: create 123/33 file after the snapshot 2023-10-06T03:59:41.361 INFO:tasks.workunit.client.0.smithi157.stdout: create 123/34 file after the snapshot 2023-10-06T03:59:41.362 INFO:tasks.workunit.client.0.smithi157.stdout: create 123/35 file after the snapshot 2023-10-06T03:59:41.363 INFO:tasks.workunit.client.0.smithi157.stdout: create 123/36 file after the snapshot 2023-10-06T03:59:41.363 INFO:tasks.workunit.client.0.smithi157.stdout: create 123/37 file after the snapshot 2023-10-06T03:59:41.364 INFO:tasks.workunit.client.0.smithi157.stdout: create 123/38 file after the snapshot 2023-10-06T03:59:41.365 INFO:tasks.workunit.client.0.smithi157.stdout: create 123/39 file after the snapshot 2023-10-06T03:59:41.365 INFO:tasks.workunit.client.0.smithi157.stdout: create 124/.snap/snap-subdir-test 2023-10-06T03:59:41.965 INFO:tasks.workunit.client.0.smithi157.stdout: create 124/30 file after the snapshot 2023-10-06T03:59:41.966 INFO:tasks.workunit.client.0.smithi157.stdout: create 124/31 file after the snapshot 2023-10-06T03:59:41.967 INFO:tasks.workunit.client.0.smithi157.stdout: create 124/32 file after the snapshot 2023-10-06T03:59:41.968 INFO:tasks.workunit.client.0.smithi157.stdout: create 124/33 file after the snapshot 2023-10-06T03:59:41.968 INFO:tasks.workunit.client.0.smithi157.stdout: create 124/34 file after the snapshot 2023-10-06T03:59:41.969 INFO:tasks.workunit.client.0.smithi157.stdout: create 124/35 file after the snapshot 2023-10-06T03:59:41.970 INFO:tasks.workunit.client.0.smithi157.stdout: create 124/36 file after the snapshot 2023-10-06T03:59:41.970 INFO:tasks.workunit.client.0.smithi157.stdout: create 124/37 file after the snapshot 2023-10-06T03:59:41.971 INFO:tasks.workunit.client.0.smithi157.stdout: create 124/38 file after the snapshot 2023-10-06T03:59:41.972 INFO:tasks.workunit.client.0.smithi157.stdout: create 124/39 file after the snapshot 2023-10-06T03:59:41.972 INFO:tasks.workunit.client.0.smithi157.stdout: create 125/.snap/snap-subdir-test 2023-10-06T03:59:41.997 INFO:tasks.workunit.client.0.smithi157.stdout: create 125/30 file after the snapshot 2023-10-06T03:59:41.998 INFO:tasks.workunit.client.0.smithi157.stdout: create 125/31 file after the snapshot 2023-10-06T03:59:41.998 INFO:tasks.workunit.client.0.smithi157.stdout: create 125/32 file after the snapshot 2023-10-06T03:59:41.999 INFO:tasks.workunit.client.0.smithi157.stdout: create 125/33 file after the snapshot 2023-10-06T03:59:42.000 INFO:tasks.workunit.client.0.smithi157.stdout: create 125/34 file after the snapshot 2023-10-06T03:59:42.000 INFO:tasks.workunit.client.0.smithi157.stdout: create 125/35 file after the snapshot 2023-10-06T03:59:42.001 INFO:tasks.workunit.client.0.smithi157.stdout: create 125/36 file after the snapshot 2023-10-06T03:59:42.002 INFO:tasks.workunit.client.0.smithi157.stdout: create 125/37 file after the snapshot 2023-10-06T03:59:42.002 INFO:tasks.workunit.client.0.smithi157.stdout: create 125/38 file after the snapshot 2023-10-06T03:59:42.003 INFO:tasks.workunit.client.0.smithi157.stdout: create 125/39 file after the snapshot 2023-10-06T03:59:42.004 INFO:tasks.workunit.client.0.smithi157.stdout: create 126/.snap/snap-subdir-test 2023-10-06T03:59:42.027 INFO:tasks.workunit.client.0.smithi157.stdout: create 126/30 file after the snapshot 2023-10-06T03:59:42.029 INFO:tasks.workunit.client.0.smithi157.stdout: create 126/31 file after the snapshot 2023-10-06T03:59:42.029 INFO:tasks.workunit.client.0.smithi157.stdout: create 126/32 file after the snapshot 2023-10-06T03:59:42.030 INFO:tasks.workunit.client.0.smithi157.stdout: create 126/33 file after the snapshot 2023-10-06T03:59:42.030 INFO:tasks.workunit.client.0.smithi157.stdout: create 126/34 file after the snapshot 2023-10-06T03:59:42.031 INFO:tasks.workunit.client.0.smithi157.stdout: create 126/35 file after the snapshot 2023-10-06T03:59:42.032 INFO:tasks.workunit.client.0.smithi157.stdout: create 126/36 file after the snapshot 2023-10-06T03:59:42.032 INFO:tasks.workunit.client.0.smithi157.stdout: create 126/37 file after the snapshot 2023-10-06T03:59:42.033 INFO:tasks.workunit.client.0.smithi157.stdout: create 126/38 file after the snapshot 2023-10-06T03:59:42.034 INFO:tasks.workunit.client.0.smithi157.stdout: create 126/39 file after the snapshot 2023-10-06T03:59:42.034 INFO:tasks.workunit.client.0.smithi157.stdout: create 127/.snap/snap-subdir-test 2023-10-06T03:59:42.058 INFO:tasks.workunit.client.0.smithi157.stdout: create 127/30 file after the snapshot 2023-10-06T03:59:42.059 INFO:tasks.workunit.client.0.smithi157.stdout: create 127/31 file after the snapshot 2023-10-06T03:59:42.060 INFO:tasks.workunit.client.0.smithi157.stdout: create 127/32 file after the snapshot 2023-10-06T03:59:42.060 INFO:tasks.workunit.client.0.smithi157.stdout: create 127/33 file after the snapshot 2023-10-06T03:59:42.061 INFO:tasks.workunit.client.0.smithi157.stdout: create 127/34 file after the snapshot 2023-10-06T03:59:42.062 INFO:tasks.workunit.client.0.smithi157.stdout: create 127/35 file after the snapshot 2023-10-06T03:59:42.062 INFO:tasks.workunit.client.0.smithi157.stdout: create 127/36 file after the snapshot 2023-10-06T03:59:42.063 INFO:tasks.workunit.client.0.smithi157.stdout: create 127/37 file after the snapshot 2023-10-06T03:59:42.064 INFO:tasks.workunit.client.0.smithi157.stdout: create 127/38 file after the snapshot 2023-10-06T03:59:42.064 INFO:tasks.workunit.client.0.smithi157.stdout: create 127/39 file after the snapshot 2023-10-06T03:59:42.065 INFO:tasks.workunit.client.0.smithi157.stdout: create 128/.snap/snap-subdir-test 2023-10-06T03:59:42.089 INFO:tasks.workunit.client.0.smithi157.stdout: create 128/30 file after the snapshot 2023-10-06T03:59:42.090 INFO:tasks.workunit.client.0.smithi157.stdout: create 128/31 file after the snapshot 2023-10-06T03:59:42.091 INFO:tasks.workunit.client.0.smithi157.stdout: create 128/32 file after the snapshot 2023-10-06T03:59:42.091 INFO:tasks.workunit.client.0.smithi157.stdout: create 128/33 file after the snapshot 2023-10-06T03:59:42.092 INFO:tasks.workunit.client.0.smithi157.stdout: create 128/34 file after the snapshot 2023-10-06T03:59:42.093 INFO:tasks.workunit.client.0.smithi157.stdout: create 128/35 file after the snapshot 2023-10-06T03:59:42.093 INFO:tasks.workunit.client.0.smithi157.stdout: create 128/36 file after the snapshot 2023-10-06T03:59:42.094 INFO:tasks.workunit.client.0.smithi157.stdout: create 128/37 file after the snapshot 2023-10-06T03:59:42.095 INFO:tasks.workunit.client.0.smithi157.stdout: create 128/38 file after the snapshot 2023-10-06T03:59:42.095 INFO:tasks.workunit.client.0.smithi157.stdout: create 128/39 file after the snapshot 2023-10-06T03:59:42.096 INFO:tasks.workunit.client.0.smithi157.stdout: create 129/.snap/snap-subdir-test 2023-10-06T03:59:42.120 INFO:tasks.workunit.client.0.smithi157.stdout: create 129/30 file after the snapshot 2023-10-06T03:59:42.121 INFO:tasks.workunit.client.0.smithi157.stdout: create 129/31 file after the snapshot 2023-10-06T03:59:42.121 INFO:tasks.workunit.client.0.smithi157.stdout: create 129/32 file after the snapshot 2023-10-06T03:59:42.122 INFO:tasks.workunit.client.0.smithi157.stdout: create 129/33 file after the snapshot 2023-10-06T03:59:42.123 INFO:tasks.workunit.client.0.smithi157.stdout: create 129/34 file after the snapshot 2023-10-06T03:59:42.123 INFO:tasks.workunit.client.0.smithi157.stdout: create 129/35 file after the snapshot 2023-10-06T03:59:42.124 INFO:tasks.workunit.client.0.smithi157.stdout: create 129/36 file after the snapshot 2023-10-06T03:59:42.125 INFO:tasks.workunit.client.0.smithi157.stdout: create 129/37 file after the snapshot 2023-10-06T03:59:42.125 INFO:tasks.workunit.client.0.smithi157.stdout: create 129/38 file after the snapshot 2023-10-06T03:59:42.126 INFO:tasks.workunit.client.0.smithi157.stdout: create 129/39 file after the snapshot 2023-10-06T03:59:42.126 INFO:tasks.workunit.client.0.smithi157.stdout: create 130/.snap/snap-subdir-test 2023-10-06T03:59:42.151 INFO:tasks.workunit.client.0.smithi157.stdout: create 130/30 file after the snapshot 2023-10-06T03:59:42.151 INFO:tasks.workunit.client.0.smithi157.stdout: create 130/31 file after the snapshot 2023-10-06T03:59:42.152 INFO:tasks.workunit.client.0.smithi157.stdout: create 130/32 file after the snapshot 2023-10-06T03:59:42.153 INFO:tasks.workunit.client.0.smithi157.stdout: create 130/33 file after the snapshot 2023-10-06T03:59:42.153 INFO:tasks.workunit.client.0.smithi157.stdout: create 130/34 file after the snapshot 2023-10-06T03:59:42.154 INFO:tasks.workunit.client.0.smithi157.stdout: create 130/35 file after the snapshot 2023-10-06T03:59:42.155 INFO:tasks.workunit.client.0.smithi157.stdout: create 130/36 file after the snapshot 2023-10-06T03:59:42.155 INFO:tasks.workunit.client.0.smithi157.stdout: create 130/37 file after the snapshot 2023-10-06T03:59:42.156 INFO:tasks.workunit.client.0.smithi157.stdout: create 130/38 file after the snapshot 2023-10-06T03:59:42.156 INFO:tasks.workunit.client.0.smithi157.stdout: create 130/39 file after the snapshot 2023-10-06T03:59:42.201 INFO:tasks.workunit.client.0.smithi157.stdout: create 131/.snap/snap-subdir-test 2023-10-06T03:59:42.566 INFO:tasks.workunit.client.0.smithi157.stdout: create 131/30 file after the snapshot 2023-10-06T03:59:42.568 INFO:tasks.workunit.client.0.smithi157.stdout: create 131/31 file after the snapshot 2023-10-06T03:59:42.569 INFO:tasks.workunit.client.0.smithi157.stdout: create 131/32 file after the snapshot 2023-10-06T03:59:42.570 INFO:tasks.workunit.client.0.smithi157.stdout: create 131/33 file after the snapshot 2023-10-06T03:59:42.571 INFO:tasks.workunit.client.0.smithi157.stdout: create 131/34 file after the snapshot 2023-10-06T03:59:42.572 INFO:tasks.workunit.client.0.smithi157.stdout: create 131/35 file after the snapshot 2023-10-06T03:59:42.572 INFO:tasks.workunit.client.0.smithi157.stdout: create 131/36 file after the snapshot 2023-10-06T03:59:42.573 INFO:tasks.workunit.client.0.smithi157.stdout: create 131/37 file after the snapshot 2023-10-06T03:59:42.574 INFO:tasks.workunit.client.0.smithi157.stdout: create 131/38 file after the snapshot 2023-10-06T03:59:42.575 INFO:tasks.workunit.client.0.smithi157.stdout: create 131/39 file after the snapshot 2023-10-06T03:59:42.576 INFO:tasks.workunit.client.0.smithi157.stdout: create 132/.snap/snap-subdir-test 2023-10-06T03:59:42.601 INFO:tasks.workunit.client.0.smithi157.stdout: create 132/30 file after the snapshot 2023-10-06T03:59:42.602 INFO:tasks.workunit.client.0.smithi157.stdout: create 132/31 file after the snapshot 2023-10-06T03:59:42.603 INFO:tasks.workunit.client.0.smithi157.stdout: create 132/32 file after the snapshot 2023-10-06T03:59:42.603 INFO:tasks.workunit.client.0.smithi157.stdout: create 132/33 file after the snapshot 2023-10-06T03:59:42.604 INFO:tasks.workunit.client.0.smithi157.stdout: create 132/34 file after the snapshot 2023-10-06T03:59:42.604 INFO:tasks.workunit.client.0.smithi157.stdout: create 132/35 file after the snapshot 2023-10-06T03:59:42.605 INFO:tasks.workunit.client.0.smithi157.stdout: create 132/36 file after the snapshot 2023-10-06T03:59:42.605 INFO:tasks.workunit.client.0.smithi157.stdout: create 132/37 file after the snapshot 2023-10-06T03:59:42.606 INFO:tasks.workunit.client.0.smithi157.stdout: create 132/38 file after the snapshot 2023-10-06T03:59:42.607 INFO:tasks.workunit.client.0.smithi157.stdout: create 132/39 file after the snapshot 2023-10-06T03:59:42.607 INFO:tasks.workunit.client.0.smithi157.stdout: create 133/.snap/snap-subdir-test 2023-10-06T03:59:42.631 INFO:tasks.workunit.client.0.smithi157.stdout: create 133/30 file after the snapshot 2023-10-06T03:59:42.633 INFO:tasks.workunit.client.0.smithi157.stdout: create 133/31 file after the snapshot 2023-10-06T03:59:42.634 INFO:tasks.workunit.client.0.smithi157.stdout: create 133/32 file after the snapshot 2023-10-06T03:59:42.635 INFO:tasks.workunit.client.0.smithi157.stdout: create 133/33 file after the snapshot 2023-10-06T03:59:42.636 INFO:tasks.workunit.client.0.smithi157.stdout: create 133/34 file after the snapshot 2023-10-06T03:59:42.637 INFO:tasks.workunit.client.0.smithi157.stdout: create 133/35 file after the snapshot 2023-10-06T03:59:42.638 INFO:tasks.workunit.client.0.smithi157.stdout: create 133/36 file after the snapshot 2023-10-06T03:59:42.638 INFO:tasks.workunit.client.0.smithi157.stdout: create 133/37 file after the snapshot 2023-10-06T03:59:42.639 INFO:tasks.workunit.client.0.smithi157.stdout: create 133/38 file after the snapshot 2023-10-06T03:59:42.640 INFO:tasks.workunit.client.0.smithi157.stdout: create 133/39 file after the snapshot 2023-10-06T03:59:42.641 INFO:tasks.workunit.client.0.smithi157.stdout: create 134/.snap/snap-subdir-test 2023-10-06T03:59:42.665 INFO:tasks.workunit.client.0.smithi157.stdout: create 134/30 file after the snapshot 2023-10-06T03:59:42.667 INFO:tasks.workunit.client.0.smithi157.stdout: create 134/31 file after the snapshot 2023-10-06T03:59:42.667 INFO:tasks.workunit.client.0.smithi157.stdout: create 134/32 file after the snapshot 2023-10-06T03:59:42.667 INFO:tasks.workunit.client.0.smithi157.stdout: create 134/33 file after the snapshot 2023-10-06T03:59:42.668 INFO:tasks.workunit.client.0.smithi157.stdout: create 134/34 file after the snapshot 2023-10-06T03:59:42.669 INFO:tasks.workunit.client.0.smithi157.stdout: create 134/35 file after the snapshot 2023-10-06T03:59:42.669 INFO:tasks.workunit.client.0.smithi157.stdout: create 134/36 file after the snapshot 2023-10-06T03:59:42.670 INFO:tasks.workunit.client.0.smithi157.stdout: create 134/37 file after the snapshot 2023-10-06T03:59:42.671 INFO:tasks.workunit.client.0.smithi157.stdout: create 134/38 file after the snapshot 2023-10-06T03:59:42.671 INFO:tasks.workunit.client.0.smithi157.stdout: create 134/39 file after the snapshot 2023-10-06T03:59:42.672 INFO:tasks.workunit.client.0.smithi157.stdout: create 135/.snap/snap-subdir-test 2023-10-06T03:59:42.696 INFO:tasks.workunit.client.0.smithi157.stdout: create 135/30 file after the snapshot 2023-10-06T03:59:42.697 INFO:tasks.workunit.client.0.smithi157.stdout: create 135/31 file after the snapshot 2023-10-06T03:59:42.697 INFO:tasks.workunit.client.0.smithi157.stdout: create 135/32 file after the snapshot 2023-10-06T03:59:42.698 INFO:tasks.workunit.client.0.smithi157.stdout: create 135/33 file after the snapshot 2023-10-06T03:59:42.699 INFO:tasks.workunit.client.0.smithi157.stdout: create 135/34 file after the snapshot 2023-10-06T03:59:42.699 INFO:tasks.workunit.client.0.smithi157.stdout: create 135/35 file after the snapshot 2023-10-06T03:59:42.700 INFO:tasks.workunit.client.0.smithi157.stdout: create 135/36 file after the snapshot 2023-10-06T03:59:42.701 INFO:tasks.workunit.client.0.smithi157.stdout: create 135/37 file after the snapshot 2023-10-06T03:59:42.701 INFO:tasks.workunit.client.0.smithi157.stdout: create 135/38 file after the snapshot 2023-10-06T03:59:42.702 INFO:tasks.workunit.client.0.smithi157.stdout: create 135/39 file after the snapshot 2023-10-06T03:59:42.703 INFO:tasks.workunit.client.0.smithi157.stdout: create 136/.snap/snap-subdir-test 2023-10-06T03:59:42.727 INFO:tasks.workunit.client.0.smithi157.stdout: create 136/30 file after the snapshot 2023-10-06T03:59:42.728 INFO:tasks.workunit.client.0.smithi157.stdout: create 136/31 file after the snapshot 2023-10-06T03:59:42.729 INFO:tasks.workunit.client.0.smithi157.stdout: create 136/32 file after the snapshot 2023-10-06T03:59:42.730 INFO:tasks.workunit.client.0.smithi157.stdout: create 136/33 file after the snapshot 2023-10-06T03:59:42.730 INFO:tasks.workunit.client.0.smithi157.stdout: create 136/34 file after the snapshot 2023-10-06T03:59:42.731 INFO:tasks.workunit.client.0.smithi157.stdout: create 136/35 file after the snapshot 2023-10-06T03:59:42.732 INFO:tasks.workunit.client.0.smithi157.stdout: create 136/36 file after the snapshot 2023-10-06T03:59:42.732 INFO:tasks.workunit.client.0.smithi157.stdout: create 136/37 file after the snapshot 2023-10-06T03:59:42.733 INFO:tasks.workunit.client.0.smithi157.stdout: create 136/38 file after the snapshot 2023-10-06T03:59:42.734 INFO:tasks.workunit.client.0.smithi157.stdout: create 136/39 file after the snapshot 2023-10-06T03:59:42.735 INFO:tasks.workunit.client.0.smithi157.stdout: create 137/.snap/snap-subdir-test 2023-10-06T03:59:42.758 INFO:tasks.workunit.client.0.smithi157.stdout: create 137/30 file after the snapshot 2023-10-06T03:59:42.760 INFO:tasks.workunit.client.0.smithi157.stdout: create 137/31 file after the snapshot 2023-10-06T03:59:42.760 INFO:tasks.workunit.client.0.smithi157.stdout: create 137/32 file after the snapshot 2023-10-06T03:59:42.761 INFO:tasks.workunit.client.0.smithi157.stdout: create 137/33 file after the snapshot 2023-10-06T03:59:42.761 INFO:tasks.workunit.client.0.smithi157.stdout: create 137/34 file after the snapshot 2023-10-06T03:59:42.762 INFO:tasks.workunit.client.0.smithi157.stdout: create 137/35 file after the snapshot 2023-10-06T03:59:42.763 INFO:tasks.workunit.client.0.smithi157.stdout: create 137/36 file after the snapshot 2023-10-06T03:59:42.763 INFO:tasks.workunit.client.0.smithi157.stdout: create 137/37 file after the snapshot 2023-10-06T03:59:42.764 INFO:tasks.workunit.client.0.smithi157.stdout: create 137/38 file after the snapshot 2023-10-06T03:59:42.765 INFO:tasks.workunit.client.0.smithi157.stdout: create 137/39 file after the snapshot 2023-10-06T03:59:42.765 INFO:tasks.workunit.client.0.smithi157.stdout: create 138/.snap/snap-subdir-test 2023-10-06T03:59:42.789 INFO:tasks.workunit.client.0.smithi157.stdout: create 138/30 file after the snapshot 2023-10-06T03:59:42.791 INFO:tasks.workunit.client.0.smithi157.stdout: create 138/31 file after the snapshot 2023-10-06T03:59:42.791 INFO:tasks.workunit.client.0.smithi157.stdout: create 138/32 file after the snapshot 2023-10-06T03:59:42.792 INFO:tasks.workunit.client.0.smithi157.stdout: create 138/33 file after the snapshot 2023-10-06T03:59:42.793 INFO:tasks.workunit.client.0.smithi157.stdout: create 138/34 file after the snapshot 2023-10-06T03:59:42.794 INFO:tasks.workunit.client.0.smithi157.stdout: create 138/35 file after the snapshot 2023-10-06T03:59:42.795 INFO:tasks.workunit.client.0.smithi157.stdout: create 138/36 file after the snapshot 2023-10-06T03:59:42.796 INFO:tasks.workunit.client.0.smithi157.stdout: create 138/37 file after the snapshot 2023-10-06T03:59:42.797 INFO:tasks.workunit.client.0.smithi157.stdout: create 138/38 file after the snapshot 2023-10-06T03:59:42.798 INFO:tasks.workunit.client.0.smithi157.stdout: create 138/39 file after the snapshot 2023-10-06T03:59:42.799 INFO:tasks.workunit.client.0.smithi157.stdout: create 139/.snap/snap-subdir-test 2023-10-06T03:59:42.829 INFO:tasks.workunit.client.0.smithi157.stdout: create 139/30 file after the snapshot 2023-10-06T03:59:42.830 INFO:tasks.workunit.client.0.smithi157.stdout: create 139/31 file after the snapshot 2023-10-06T03:59:42.831 INFO:tasks.workunit.client.0.smithi157.stdout: create 139/32 file after the snapshot 2023-10-06T03:59:42.831 INFO:tasks.workunit.client.0.smithi157.stdout: create 139/33 file after the snapshot 2023-10-06T03:59:42.832 INFO:tasks.workunit.client.0.smithi157.stdout: create 139/34 file after the snapshot 2023-10-06T03:59:42.833 INFO:tasks.workunit.client.0.smithi157.stdout: create 139/35 file after the snapshot 2023-10-06T03:59:42.833 INFO:tasks.workunit.client.0.smithi157.stdout: create 139/36 file after the snapshot 2023-10-06T03:59:42.834 INFO:tasks.workunit.client.0.smithi157.stdout: create 139/37 file after the snapshot 2023-10-06T03:59:42.835 INFO:tasks.workunit.client.0.smithi157.stdout: create 139/38 file after the snapshot 2023-10-06T03:59:42.835 INFO:tasks.workunit.client.0.smithi157.stdout: create 139/39 file after the snapshot 2023-10-06T03:59:42.836 INFO:tasks.workunit.client.0.smithi157.stdout: create 140/.snap/snap-subdir-test 2023-10-06T03:59:42.860 INFO:tasks.workunit.client.0.smithi157.stdout: create 140/30 file after the snapshot 2023-10-06T03:59:42.861 INFO:tasks.workunit.client.0.smithi157.stdout: create 140/31 file after the snapshot 2023-10-06T03:59:42.861 INFO:tasks.workunit.client.0.smithi157.stdout: create 140/32 file after the snapshot 2023-10-06T03:59:42.862 INFO:tasks.workunit.client.0.smithi157.stdout: create 140/33 file after the snapshot 2023-10-06T03:59:42.863 INFO:tasks.workunit.client.0.smithi157.stdout: create 140/34 file after the snapshot 2023-10-06T03:59:42.863 INFO:tasks.workunit.client.0.smithi157.stdout: create 140/35 file after the snapshot 2023-10-06T03:59:42.864 INFO:tasks.workunit.client.0.smithi157.stdout: create 140/36 file after the snapshot 2023-10-06T03:59:42.865 INFO:tasks.workunit.client.0.smithi157.stdout: create 140/37 file after the snapshot 2023-10-06T03:59:42.865 INFO:tasks.workunit.client.0.smithi157.stdout: create 140/38 file after the snapshot 2023-10-06T03:59:42.866 INFO:tasks.workunit.client.0.smithi157.stdout: create 140/39 file after the snapshot 2023-10-06T03:59:42.867 INFO:tasks.workunit.client.0.smithi157.stdout: create 141/.snap/snap-subdir-test 2023-10-06T03:59:42.890 INFO:tasks.workunit.client.0.smithi157.stdout: create 141/30 file after the snapshot 2023-10-06T03:59:42.891 INFO:tasks.workunit.client.0.smithi157.stdout: create 141/31 file after the snapshot 2023-10-06T03:59:42.892 INFO:tasks.workunit.client.0.smithi157.stdout: create 141/32 file after the snapshot 2023-10-06T03:59:42.893 INFO:tasks.workunit.client.0.smithi157.stdout: create 141/33 file after the snapshot 2023-10-06T03:59:42.893 INFO:tasks.workunit.client.0.smithi157.stdout: create 141/34 file after the snapshot 2023-10-06T03:59:42.894 INFO:tasks.workunit.client.0.smithi157.stdout: create 141/35 file after the snapshot 2023-10-06T03:59:42.895 INFO:tasks.workunit.client.0.smithi157.stdout: create 141/36 file after the snapshot 2023-10-06T03:59:42.895 INFO:tasks.workunit.client.0.smithi157.stdout: create 141/37 file after the snapshot 2023-10-06T03:59:42.896 INFO:tasks.workunit.client.0.smithi157.stdout: create 141/38 file after the snapshot 2023-10-06T03:59:42.897 INFO:tasks.workunit.client.0.smithi157.stdout: create 141/39 file after the snapshot 2023-10-06T03:59:42.897 INFO:tasks.workunit.client.0.smithi157.stdout: create 142/.snap/snap-subdir-test 2023-10-06T03:59:42.942 INFO:tasks.workunit.client.0.smithi157.stdout: create 142/30 file after the snapshot 2023-10-06T03:59:42.949 INFO:tasks.workunit.client.0.smithi157.stdout: create 142/31 file after the snapshot 2023-10-06T03:59:42.961 INFO:tasks.workunit.client.0.smithi157.stdout: create 142/32 file after the snapshot 2023-10-06T03:59:43.498 INFO:tasks.workunit.client.0.smithi157.stdout: create 142/33 file after the snapshot 2023-10-06T03:59:43.811 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:59:43.814 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:59:44.185 INFO:tasks.workunit.client.0.smithi157.stdout: create 142/34 file after the snapshot 2023-10-06T03:59:44.450 INFO:tasks.workunit.client.0.smithi157.stdout: create 142/35 file after the snapshot 2023-10-06T03:59:44.450 INFO:tasks.workunit.client.0.smithi157.stdout: create 142/36 file after the snapshot 2023-10-06T03:59:44.451 INFO:tasks.workunit.client.0.smithi157.stdout: create 142/37 file after the snapshot 2023-10-06T03:59:44.453 INFO:tasks.workunit.client.0.smithi157.stdout: create 142/38 file after the snapshot 2023-10-06T03:59:44.454 INFO:tasks.workunit.client.0.smithi157.stdout: create 142/39 file after the snapshot 2023-10-06T03:59:44.455 INFO:tasks.workunit.client.0.smithi157.stdout: create 143/.snap/snap-subdir-test 2023-10-06T03:59:44.557 INFO:tasks.workunit.client.0.smithi157.stdout: create 143/30 file after the snapshot 2023-10-06T03:59:44.558 INFO:tasks.workunit.client.0.smithi157.stdout: create 143/31 file after the snapshot 2023-10-06T03:59:44.558 INFO:tasks.workunit.client.0.smithi157.stdout: create 143/32 file after the snapshot 2023-10-06T03:59:44.559 INFO:tasks.workunit.client.0.smithi157.stdout: create 143/33 file after the snapshot 2023-10-06T03:59:44.560 INFO:tasks.workunit.client.0.smithi157.stdout: create 143/34 file after the snapshot 2023-10-06T03:59:44.561 INFO:tasks.workunit.client.0.smithi157.stdout: create 143/35 file after the snapshot 2023-10-06T03:59:44.561 INFO:tasks.workunit.client.0.smithi157.stdout: create 143/36 file after the snapshot 2023-10-06T03:59:44.562 INFO:tasks.workunit.client.0.smithi157.stdout: create 143/37 file after the snapshot 2023-10-06T03:59:44.563 INFO:tasks.workunit.client.0.smithi157.stdout: create 143/38 file after the snapshot 2023-10-06T03:59:44.563 INFO:tasks.workunit.client.0.smithi157.stdout: create 143/39 file after the snapshot 2023-10-06T03:59:44.564 INFO:tasks.workunit.client.0.smithi157.stdout: create 144/.snap/snap-subdir-test 2023-10-06T03:59:45.225 INFO:tasks.workunit.client.0.smithi157.stdout: create 144/30 file after the snapshot 2023-10-06T03:59:45.226 INFO:tasks.workunit.client.0.smithi157.stdout: create 144/31 file after the snapshot 2023-10-06T03:59:45.227 INFO:tasks.workunit.client.0.smithi157.stdout: create 144/32 file after the snapshot 2023-10-06T03:59:45.228 INFO:tasks.workunit.client.0.smithi157.stdout: create 144/33 file after the snapshot 2023-10-06T03:59:45.229 INFO:tasks.workunit.client.0.smithi157.stdout: create 144/34 file after the snapshot 2023-10-06T03:59:45.229 INFO:tasks.workunit.client.0.smithi157.stdout: create 144/35 file after the snapshot 2023-10-06T03:59:45.230 INFO:tasks.workunit.client.0.smithi157.stdout: create 144/36 file after the snapshot 2023-10-06T03:59:45.231 INFO:tasks.workunit.client.0.smithi157.stdout: create 144/37 file after the snapshot 2023-10-06T03:59:45.231 INFO:tasks.workunit.client.0.smithi157.stdout: create 144/38 file after the snapshot 2023-10-06T03:59:45.232 INFO:tasks.workunit.client.0.smithi157.stdout: create 144/39 file after the snapshot 2023-10-06T03:59:45.233 INFO:tasks.workunit.client.0.smithi157.stdout: create 145/.snap/snap-subdir-test 2023-10-06T03:59:45.266 INFO:tasks.workunit.client.0.smithi157.stdout: create 145/30 file after the snapshot 2023-10-06T03:59:45.267 INFO:tasks.workunit.client.0.smithi157.stdout: create 145/31 file after the snapshot 2023-10-06T03:59:45.268 INFO:tasks.workunit.client.0.smithi157.stdout: create 145/32 file after the snapshot 2023-10-06T03:59:45.269 INFO:tasks.workunit.client.0.smithi157.stdout: create 145/33 file after the snapshot 2023-10-06T03:59:45.269 INFO:tasks.workunit.client.0.smithi157.stdout: create 145/34 file after the snapshot 2023-10-06T03:59:45.270 INFO:tasks.workunit.client.0.smithi157.stdout: create 145/35 file after the snapshot 2023-10-06T03:59:45.270 INFO:tasks.workunit.client.0.smithi157.stdout: create 145/36 file after the snapshot 2023-10-06T03:59:45.271 INFO:tasks.workunit.client.0.smithi157.stdout: create 145/37 file after the snapshot 2023-10-06T03:59:45.272 INFO:tasks.workunit.client.0.smithi157.stdout: create 145/38 file after the snapshot 2023-10-06T03:59:45.273 INFO:tasks.workunit.client.0.smithi157.stdout: create 145/39 file after the snapshot 2023-10-06T03:59:45.273 INFO:tasks.workunit.client.0.smithi157.stdout: create 146/.snap/snap-subdir-test 2023-10-06T03:59:45.305 INFO:tasks.workunit.client.0.smithi157.stdout: create 146/30 file after the snapshot 2023-10-06T03:59:45.306 INFO:tasks.workunit.client.0.smithi157.stdout: create 146/31 file after the snapshot 2023-10-06T03:59:45.307 INFO:tasks.workunit.client.0.smithi157.stdout: create 146/32 file after the snapshot 2023-10-06T03:59:45.307 INFO:tasks.workunit.client.0.smithi157.stdout: create 146/33 file after the snapshot 2023-10-06T03:59:45.308 INFO:tasks.workunit.client.0.smithi157.stdout: create 146/34 file after the snapshot 2023-10-06T03:59:45.309 INFO:tasks.workunit.client.0.smithi157.stdout: create 146/35 file after the snapshot 2023-10-06T03:59:45.310 INFO:tasks.workunit.client.0.smithi157.stdout: create 146/36 file after the snapshot 2023-10-06T03:59:45.310 INFO:tasks.workunit.client.0.smithi157.stdout: create 146/37 file after the snapshot 2023-10-06T03:59:45.311 INFO:tasks.workunit.client.0.smithi157.stdout: create 146/38 file after the snapshot 2023-10-06T03:59:45.311 INFO:tasks.workunit.client.0.smithi157.stdout: create 146/39 file after the snapshot 2023-10-06T03:59:45.312 INFO:tasks.workunit.client.0.smithi157.stdout: create 147/.snap/snap-subdir-test 2023-10-06T03:59:45.343 INFO:tasks.workunit.client.0.smithi157.stdout: create 147/30 file after the snapshot 2023-10-06T03:59:45.344 INFO:tasks.workunit.client.0.smithi157.stdout: create 147/31 file after the snapshot 2023-10-06T03:59:45.345 INFO:tasks.workunit.client.0.smithi157.stdout: create 147/32 file after the snapshot 2023-10-06T03:59:45.346 INFO:tasks.workunit.client.0.smithi157.stdout: create 147/33 file after the snapshot 2023-10-06T03:59:45.346 INFO:tasks.workunit.client.0.smithi157.stdout: create 147/34 file after the snapshot 2023-10-06T03:59:45.347 INFO:tasks.workunit.client.0.smithi157.stdout: create 147/35 file after the snapshot 2023-10-06T03:59:45.348 INFO:tasks.workunit.client.0.smithi157.stdout: create 147/36 file after the snapshot 2023-10-06T03:59:45.348 INFO:tasks.workunit.client.0.smithi157.stdout: create 147/37 file after the snapshot 2023-10-06T03:59:45.349 INFO:tasks.workunit.client.0.smithi157.stdout: create 147/38 file after the snapshot 2023-10-06T03:59:45.350 INFO:tasks.workunit.client.0.smithi157.stdout: create 147/39 file after the snapshot 2023-10-06T03:59:45.350 INFO:tasks.workunit.client.0.smithi157.stdout: create 148/.snap/snap-subdir-test 2023-10-06T03:59:45.382 INFO:tasks.workunit.client.0.smithi157.stdout: create 148/30 file after the snapshot 2023-10-06T03:59:45.383 INFO:tasks.workunit.client.0.smithi157.stdout: create 148/31 file after the snapshot 2023-10-06T03:59:45.383 INFO:tasks.workunit.client.0.smithi157.stdout: create 148/32 file after the snapshot 2023-10-06T03:59:45.384 INFO:tasks.workunit.client.0.smithi157.stdout: create 148/33 file after the snapshot 2023-10-06T03:59:45.385 INFO:tasks.workunit.client.0.smithi157.stdout: create 148/34 file after the snapshot 2023-10-06T03:59:45.385 INFO:tasks.workunit.client.0.smithi157.stdout: create 148/35 file after the snapshot 2023-10-06T03:59:45.386 INFO:tasks.workunit.client.0.smithi157.stdout: create 148/36 file after the snapshot 2023-10-06T03:59:45.386 INFO:tasks.workunit.client.0.smithi157.stdout: create 148/37 file after the snapshot 2023-10-06T03:59:45.387 INFO:tasks.workunit.client.0.smithi157.stdout: create 148/38 file after the snapshot 2023-10-06T03:59:45.388 INFO:tasks.workunit.client.0.smithi157.stdout: create 148/39 file after the snapshot 2023-10-06T03:59:45.388 INFO:tasks.workunit.client.0.smithi157.stdout: create 149/.snap/snap-subdir-test 2023-10-06T03:59:45.420 INFO:tasks.workunit.client.0.smithi157.stdout: create 149/30 file after the snapshot 2023-10-06T03:59:45.421 INFO:tasks.workunit.client.0.smithi157.stdout: create 149/31 file after the snapshot 2023-10-06T03:59:45.421 INFO:tasks.workunit.client.0.smithi157.stdout: create 149/32 file after the snapshot 2023-10-06T03:59:45.422 INFO:tasks.workunit.client.0.smithi157.stdout: create 149/33 file after the snapshot 2023-10-06T03:59:45.422 INFO:tasks.workunit.client.0.smithi157.stdout: create 149/34 file after the snapshot 2023-10-06T03:59:45.423 INFO:tasks.workunit.client.0.smithi157.stdout: create 149/35 file after the snapshot 2023-10-06T03:59:45.424 INFO:tasks.workunit.client.0.smithi157.stdout: create 149/36 file after the snapshot 2023-10-06T03:59:45.425 INFO:tasks.workunit.client.0.smithi157.stdout: create 149/37 file after the snapshot 2023-10-06T03:59:45.425 INFO:tasks.workunit.client.0.smithi157.stdout: create 149/38 file after the snapshot 2023-10-06T03:59:45.426 INFO:tasks.workunit.client.0.smithi157.stdout: create 149/39 file after the snapshot 2023-10-06T03:59:45.427 INFO:tasks.workunit.client.0.smithi157.stdout: create 150/.snap/snap-subdir-test 2023-10-06T03:59:45.458 INFO:tasks.workunit.client.0.smithi157.stdout: create 150/30 file after the snapshot 2023-10-06T03:59:45.460 INFO:tasks.workunit.client.0.smithi157.stdout: create 150/31 file after the snapshot 2023-10-06T03:59:45.461 INFO:tasks.workunit.client.0.smithi157.stdout: create 150/32 file after the snapshot 2023-10-06T03:59:45.462 INFO:tasks.workunit.client.0.smithi157.stdout: create 150/33 file after the snapshot 2023-10-06T03:59:45.463 INFO:tasks.workunit.client.0.smithi157.stdout: create 150/34 file after the snapshot 2023-10-06T03:59:45.464 INFO:tasks.workunit.client.0.smithi157.stdout: create 150/35 file after the snapshot 2023-10-06T03:59:45.464 INFO:tasks.workunit.client.0.smithi157.stdout: create 150/36 file after the snapshot 2023-10-06T03:59:45.466 INFO:tasks.workunit.client.0.smithi157.stdout: create 150/37 file after the snapshot 2023-10-06T03:59:45.466 INFO:tasks.workunit.client.0.smithi157.stdout: create 150/38 file after the snapshot 2023-10-06T03:59:45.467 INFO:tasks.workunit.client.0.smithi157.stdout: create 150/39 file after the snapshot 2023-10-06T03:59:45.468 INFO:tasks.workunit.client.0.smithi157.stdout: create 151/.snap/snap-subdir-test 2023-10-06T03:59:45.501 INFO:tasks.workunit.client.0.smithi157.stdout: create 151/30 file after the snapshot 2023-10-06T03:59:45.502 INFO:tasks.workunit.client.0.smithi157.stdout: create 151/31 file after the snapshot 2023-10-06T03:59:45.503 INFO:tasks.workunit.client.0.smithi157.stdout: create 151/32 file after the snapshot 2023-10-06T03:59:45.503 INFO:tasks.workunit.client.0.smithi157.stdout: create 151/33 file after the snapshot 2023-10-06T03:59:45.504 INFO:tasks.workunit.client.0.smithi157.stdout: create 151/34 file after the snapshot 2023-10-06T03:59:45.504 INFO:tasks.workunit.client.0.smithi157.stdout: create 151/35 file after the snapshot 2023-10-06T03:59:45.505 INFO:tasks.workunit.client.0.smithi157.stdout: create 151/36 file after the snapshot 2023-10-06T03:59:45.506 INFO:tasks.workunit.client.0.smithi157.stdout: create 151/37 file after the snapshot 2023-10-06T03:59:45.506 INFO:tasks.workunit.client.0.smithi157.stdout: create 151/38 file after the snapshot 2023-10-06T03:59:45.507 INFO:tasks.workunit.client.0.smithi157.stdout: create 151/39 file after the snapshot 2023-10-06T03:59:45.508 INFO:tasks.workunit.client.0.smithi157.stdout: create 152/.snap/snap-subdir-test 2023-10-06T03:59:46.269 INFO:tasks.workunit.client.0.smithi157.stdout: create 152/30 file after the snapshot 2023-10-06T03:59:46.270 INFO:tasks.workunit.client.0.smithi157.stdout: create 152/31 file after the snapshot 2023-10-06T03:59:46.271 INFO:tasks.workunit.client.0.smithi157.stdout: create 152/32 file after the snapshot 2023-10-06T03:59:46.272 INFO:tasks.workunit.client.0.smithi157.stdout: create 152/33 file after the snapshot 2023-10-06T03:59:46.273 INFO:tasks.workunit.client.0.smithi157.stdout: create 152/34 file after the snapshot 2023-10-06T03:59:46.274 INFO:tasks.workunit.client.0.smithi157.stdout: create 152/35 file after the snapshot 2023-10-06T03:59:46.274 INFO:tasks.workunit.client.0.smithi157.stdout: create 152/36 file after the snapshot 2023-10-06T03:59:46.275 INFO:tasks.workunit.client.0.smithi157.stdout: create 152/37 file after the snapshot 2023-10-06T03:59:46.276 INFO:tasks.workunit.client.0.smithi157.stdout: create 152/38 file after the snapshot 2023-10-06T03:59:46.277 INFO:tasks.workunit.client.0.smithi157.stdout: create 152/39 file after the snapshot 2023-10-06T03:59:46.278 INFO:tasks.workunit.client.0.smithi157.stdout: create 153/.snap/snap-subdir-test 2023-10-06T03:59:46.307 INFO:tasks.workunit.client.0.smithi157.stdout: create 153/30 file after the snapshot 2023-10-06T03:59:46.308 INFO:tasks.workunit.client.0.smithi157.stdout: create 153/31 file after the snapshot 2023-10-06T03:59:46.309 INFO:tasks.workunit.client.0.smithi157.stdout: create 153/32 file after the snapshot 2023-10-06T03:59:47.172 INFO:tasks.workunit.client.0.smithi157.stdout: create 153/33 file after the snapshot 2023-10-06T03:59:47.173 INFO:tasks.workunit.client.0.smithi157.stdout: create 153/34 file after the snapshot 2023-10-06T03:59:47.173 INFO:tasks.workunit.client.0.smithi157.stdout: create 153/35 file after the snapshot 2023-10-06T03:59:47.174 INFO:tasks.workunit.client.0.smithi157.stdout: create 153/36 file after the snapshot 2023-10-06T03:59:47.175 INFO:tasks.workunit.client.0.smithi157.stdout: create 153/37 file after the snapshot 2023-10-06T03:59:47.176 INFO:tasks.workunit.client.0.smithi157.stdout: create 153/38 file after the snapshot 2023-10-06T03:59:47.177 INFO:tasks.workunit.client.0.smithi157.stdout: create 153/39 file after the snapshot 2023-10-06T03:59:47.178 INFO:tasks.workunit.client.0.smithi157.stdout: create 154/.snap/snap-subdir-test 2023-10-06T03:59:47.206 INFO:tasks.workunit.client.0.smithi157.stdout: create 154/30 file after the snapshot 2023-10-06T03:59:47.207 INFO:tasks.workunit.client.0.smithi157.stdout: create 154/31 file after the snapshot 2023-10-06T03:59:47.207 INFO:tasks.workunit.client.0.smithi157.stdout: create 154/32 file after the snapshot 2023-10-06T03:59:47.208 INFO:tasks.workunit.client.0.smithi157.stdout: create 154/33 file after the snapshot 2023-10-06T03:59:47.209 INFO:tasks.workunit.client.0.smithi157.stdout: create 154/34 file after the snapshot 2023-10-06T03:59:47.210 INFO:tasks.workunit.client.0.smithi157.stdout: create 154/35 file after the snapshot 2023-10-06T03:59:47.210 INFO:tasks.workunit.client.0.smithi157.stdout: create 154/36 file after the snapshot 2023-10-06T03:59:47.211 INFO:tasks.workunit.client.0.smithi157.stdout: create 154/37 file after the snapshot 2023-10-06T03:59:47.212 INFO:tasks.workunit.client.0.smithi157.stdout: create 154/38 file after the snapshot 2023-10-06T03:59:47.213 INFO:tasks.workunit.client.0.smithi157.stdout: create 154/39 file after the snapshot 2023-10-06T03:59:47.213 INFO:tasks.workunit.client.0.smithi157.stdout: create 155/.snap/snap-subdir-test 2023-10-06T03:59:47.241 INFO:tasks.workunit.client.0.smithi157.stdout: create 155/30 file after the snapshot 2023-10-06T03:59:47.242 INFO:tasks.workunit.client.0.smithi157.stdout: create 155/31 file after the snapshot 2023-10-06T03:59:47.242 INFO:tasks.workunit.client.0.smithi157.stdout: create 155/32 file after the snapshot 2023-10-06T03:59:47.243 INFO:tasks.workunit.client.0.smithi157.stdout: create 155/33 file after the snapshot 2023-10-06T03:59:47.244 INFO:tasks.workunit.client.0.smithi157.stdout: create 155/34 file after the snapshot 2023-10-06T03:59:47.245 INFO:tasks.workunit.client.0.smithi157.stdout: create 155/35 file after the snapshot 2023-10-06T03:59:47.245 INFO:tasks.workunit.client.0.smithi157.stdout: create 155/36 file after the snapshot 2023-10-06T03:59:47.246 INFO:tasks.workunit.client.0.smithi157.stdout: create 155/37 file after the snapshot 2023-10-06T03:59:47.247 INFO:tasks.workunit.client.0.smithi157.stdout: create 155/38 file after the snapshot 2023-10-06T03:59:47.247 INFO:tasks.workunit.client.0.smithi157.stdout: create 155/39 file after the snapshot 2023-10-06T03:59:47.248 INFO:tasks.workunit.client.0.smithi157.stdout: create 156/.snap/snap-subdir-test 2023-10-06T03:59:47.274 INFO:tasks.workunit.client.0.smithi157.stdout: create 156/30 file after the snapshot 2023-10-06T03:59:47.275 INFO:tasks.workunit.client.0.smithi157.stdout: create 156/31 file after the snapshot 2023-10-06T03:59:47.276 INFO:tasks.workunit.client.0.smithi157.stdout: create 156/32 file after the snapshot 2023-10-06T03:59:47.277 INFO:tasks.workunit.client.0.smithi157.stdout: create 156/33 file after the snapshot 2023-10-06T03:59:47.278 INFO:tasks.workunit.client.0.smithi157.stdout: create 156/34 file after the snapshot 2023-10-06T03:59:47.278 INFO:tasks.workunit.client.0.smithi157.stdout: create 156/35 file after the snapshot 2023-10-06T03:59:47.279 INFO:tasks.workunit.client.0.smithi157.stdout: create 156/36 file after the snapshot 2023-10-06T03:59:47.280 INFO:tasks.workunit.client.0.smithi157.stdout: create 156/37 file after the snapshot 2023-10-06T03:59:47.280 INFO:tasks.workunit.client.0.smithi157.stdout: create 156/38 file after the snapshot 2023-10-06T03:59:47.281 INFO:tasks.workunit.client.0.smithi157.stdout: create 156/39 file after the snapshot 2023-10-06T03:59:47.282 INFO:tasks.workunit.client.0.smithi157.stdout: create 157/.snap/snap-subdir-test 2023-10-06T03:59:47.308 INFO:tasks.workunit.client.0.smithi157.stdout: create 157/30 file after the snapshot 2023-10-06T03:59:47.309 INFO:tasks.workunit.client.0.smithi157.stdout: create 157/31 file after the snapshot 2023-10-06T03:59:47.309 INFO:tasks.workunit.client.0.smithi157.stdout: create 157/32 file after the snapshot 2023-10-06T03:59:47.310 INFO:tasks.workunit.client.0.smithi157.stdout: create 157/33 file after the snapshot 2023-10-06T03:59:47.311 INFO:tasks.workunit.client.0.smithi157.stdout: create 157/34 file after the snapshot 2023-10-06T03:59:47.311 INFO:tasks.workunit.client.0.smithi157.stdout: create 157/35 file after the snapshot 2023-10-06T03:59:47.312 INFO:tasks.workunit.client.0.smithi157.stdout: create 157/36 file after the snapshot 2023-10-06T03:59:47.313 INFO:tasks.workunit.client.0.smithi157.stdout: create 157/37 file after the snapshot 2023-10-06T03:59:47.313 INFO:tasks.workunit.client.0.smithi157.stdout: create 157/38 file after the snapshot 2023-10-06T03:59:47.314 INFO:tasks.workunit.client.0.smithi157.stdout: create 157/39 file after the snapshot 2023-10-06T03:59:47.315 INFO:tasks.workunit.client.0.smithi157.stdout: create 158/.snap/snap-subdir-test 2023-10-06T03:59:47.340 INFO:tasks.workunit.client.0.smithi157.stdout: create 158/30 file after the snapshot 2023-10-06T03:59:47.341 INFO:tasks.workunit.client.0.smithi157.stdout: create 158/31 file after the snapshot 2023-10-06T03:59:47.341 INFO:tasks.workunit.client.0.smithi157.stdout: create 158/32 file after the snapshot 2023-10-06T03:59:47.342 INFO:tasks.workunit.client.0.smithi157.stdout: create 158/33 file after the snapshot 2023-10-06T03:59:47.343 INFO:tasks.workunit.client.0.smithi157.stdout: create 158/34 file after the snapshot 2023-10-06T03:59:47.343 INFO:tasks.workunit.client.0.smithi157.stdout: create 158/35 file after the snapshot 2023-10-06T03:59:47.344 INFO:tasks.workunit.client.0.smithi157.stdout: create 158/36 file after the snapshot 2023-10-06T03:59:47.344 INFO:tasks.workunit.client.0.smithi157.stdout: create 158/37 file after the snapshot 2023-10-06T03:59:47.345 INFO:tasks.workunit.client.0.smithi157.stdout: create 158/38 file after the snapshot 2023-10-06T03:59:47.346 INFO:tasks.workunit.client.0.smithi157.stdout: create 158/39 file after the snapshot 2023-10-06T03:59:47.346 INFO:tasks.workunit.client.0.smithi157.stdout: create 159/.snap/snap-subdir-test 2023-10-06T03:59:47.371 INFO:tasks.workunit.client.0.smithi157.stdout: create 159/30 file after the snapshot 2023-10-06T03:59:47.373 INFO:tasks.workunit.client.0.smithi157.stdout: create 159/31 file after the snapshot 2023-10-06T03:59:47.373 INFO:tasks.workunit.client.0.smithi157.stdout: create 159/32 file after the snapshot 2023-10-06T03:59:47.374 INFO:tasks.workunit.client.0.smithi157.stdout: create 159/33 file after the snapshot 2023-10-06T03:59:47.374 INFO:tasks.workunit.client.0.smithi157.stdout: create 159/34 file after the snapshot 2023-10-06T03:59:47.375 INFO:tasks.workunit.client.0.smithi157.stdout: create 159/35 file after the snapshot 2023-10-06T03:59:47.376 INFO:tasks.workunit.client.0.smithi157.stdout: create 159/36 file after the snapshot 2023-10-06T03:59:47.376 INFO:tasks.workunit.client.0.smithi157.stdout: create 159/37 file after the snapshot 2023-10-06T03:59:47.377 INFO:tasks.workunit.client.0.smithi157.stdout: create 159/38 file after the snapshot 2023-10-06T03:59:47.378 INFO:tasks.workunit.client.0.smithi157.stdout: create 159/39 file after the snapshot 2023-10-06T03:59:47.378 INFO:tasks.workunit.client.0.smithi157.stdout: create 160/.snap/snap-subdir-test 2023-10-06T03:59:47.403 INFO:tasks.workunit.client.0.smithi157.stdout: create 160/30 file after the snapshot 2023-10-06T03:59:47.404 INFO:tasks.workunit.client.0.smithi157.stdout: create 160/31 file after the snapshot 2023-10-06T03:59:47.405 INFO:tasks.workunit.client.0.smithi157.stdout: create 160/32 file after the snapshot 2023-10-06T03:59:47.405 INFO:tasks.workunit.client.0.smithi157.stdout: create 160/33 file after the snapshot 2023-10-06T03:59:47.406 INFO:tasks.workunit.client.0.smithi157.stdout: create 160/34 file after the snapshot 2023-10-06T03:59:47.407 INFO:tasks.workunit.client.0.smithi157.stdout: create 160/35 file after the snapshot 2023-10-06T03:59:47.407 INFO:tasks.workunit.client.0.smithi157.stdout: create 160/36 file after the snapshot 2023-10-06T03:59:47.408 INFO:tasks.workunit.client.0.smithi157.stdout: create 160/37 file after the snapshot 2023-10-06T03:59:47.409 INFO:tasks.workunit.client.0.smithi157.stdout: create 160/38 file after the snapshot 2023-10-06T03:59:47.410 INFO:tasks.workunit.client.0.smithi157.stdout: create 160/39 file after the snapshot 2023-10-06T03:59:47.410 INFO:tasks.workunit.client.0.smithi157.stdout: create 161/.snap/snap-subdir-test 2023-10-06T03:59:47.436 INFO:tasks.workunit.client.0.smithi157.stdout: create 161/30 file after the snapshot 2023-10-06T03:59:47.436 INFO:tasks.workunit.client.0.smithi157.stdout: create 161/31 file after the snapshot 2023-10-06T03:59:47.437 INFO:tasks.workunit.client.0.smithi157.stdout: create 161/32 file after the snapshot 2023-10-06T03:59:47.438 INFO:tasks.workunit.client.0.smithi157.stdout: create 161/33 file after the snapshot 2023-10-06T03:59:47.438 INFO:tasks.workunit.client.0.smithi157.stdout: create 161/34 file after the snapshot 2023-10-06T03:59:47.439 INFO:tasks.workunit.client.0.smithi157.stdout: create 161/35 file after the snapshot 2023-10-06T03:59:47.440 INFO:tasks.workunit.client.0.smithi157.stdout: create 161/36 file after the snapshot 2023-10-06T03:59:47.440 INFO:tasks.workunit.client.0.smithi157.stdout: create 161/37 file after the snapshot 2023-10-06T03:59:47.441 INFO:tasks.workunit.client.0.smithi157.stdout: create 161/38 file after the snapshot 2023-10-06T03:59:47.442 INFO:tasks.workunit.client.0.smithi157.stdout: create 161/39 file after the snapshot 2023-10-06T03:59:47.442 INFO:tasks.workunit.client.0.smithi157.stdout: create 162/.snap/snap-subdir-test 2023-10-06T03:59:47.468 INFO:tasks.workunit.client.0.smithi157.stdout: create 162/30 file after the snapshot 2023-10-06T03:59:47.469 INFO:tasks.workunit.client.0.smithi157.stdout: create 162/31 file after the snapshot 2023-10-06T03:59:47.469 INFO:tasks.workunit.client.0.smithi157.stdout: create 162/32 file after the snapshot 2023-10-06T03:59:47.470 INFO:tasks.workunit.client.0.smithi157.stdout: create 162/33 file after the snapshot 2023-10-06T03:59:47.471 INFO:tasks.workunit.client.0.smithi157.stdout: create 162/34 file after the snapshot 2023-10-06T03:59:47.471 INFO:tasks.workunit.client.0.smithi157.stdout: create 162/35 file after the snapshot 2023-10-06T03:59:47.472 INFO:tasks.workunit.client.0.smithi157.stdout: create 162/36 file after the snapshot 2023-10-06T03:59:47.473 INFO:tasks.workunit.client.0.smithi157.stdout: create 162/37 file after the snapshot 2023-10-06T03:59:47.473 INFO:tasks.workunit.client.0.smithi157.stdout: create 162/38 file after the snapshot 2023-10-06T03:59:47.474 INFO:tasks.workunit.client.0.smithi157.stdout: create 162/39 file after the snapshot 2023-10-06T03:59:47.474 INFO:tasks.workunit.client.0.smithi157.stdout: create 163/.snap/snap-subdir-test 2023-10-06T03:59:47.841 INFO:tasks.workunit.client.0.smithi157.stdout: create 163/30 file after the snapshot 2023-10-06T03:59:47.842 INFO:tasks.workunit.client.0.smithi157.stdout: create 163/31 file after the snapshot 2023-10-06T03:59:47.843 INFO:tasks.workunit.client.0.smithi157.stdout: create 163/32 file after the snapshot 2023-10-06T03:59:47.843 INFO:tasks.workunit.client.0.smithi157.stdout: create 163/33 file after the snapshot 2023-10-06T03:59:47.844 INFO:tasks.workunit.client.0.smithi157.stdout: create 163/34 file after the snapshot 2023-10-06T03:59:47.845 INFO:tasks.workunit.client.0.smithi157.stdout: create 163/35 file after the snapshot 2023-10-06T03:59:47.845 INFO:tasks.workunit.client.0.smithi157.stdout: create 163/36 file after the snapshot 2023-10-06T03:59:47.846 INFO:tasks.workunit.client.0.smithi157.stdout: create 163/37 file after the snapshot 2023-10-06T03:59:47.846 INFO:tasks.workunit.client.0.smithi157.stdout: create 163/38 file after the snapshot 2023-10-06T03:59:47.847 INFO:tasks.workunit.client.0.smithi157.stdout: create 163/39 file after the snapshot 2023-10-06T03:59:47.848 INFO:tasks.workunit.client.0.smithi157.stdout: create 164/.snap/snap-subdir-test 2023-10-06T03:59:47.873 INFO:tasks.workunit.client.0.smithi157.stdout: create 164/30 file after the snapshot 2023-10-06T03:59:47.874 INFO:tasks.workunit.client.0.smithi157.stdout: create 164/31 file after the snapshot 2023-10-06T03:59:47.875 INFO:tasks.workunit.client.0.smithi157.stdout: create 164/32 file after the snapshot 2023-10-06T03:59:47.875 INFO:tasks.workunit.client.0.smithi157.stdout: create 164/33 file after the snapshot 2023-10-06T03:59:47.876 INFO:tasks.workunit.client.0.smithi157.stdout: create 164/34 file after the snapshot 2023-10-06T03:59:47.876 INFO:tasks.workunit.client.0.smithi157.stdout: create 164/35 file after the snapshot 2023-10-06T03:59:47.877 INFO:tasks.workunit.client.0.smithi157.stdout: create 164/36 file after the snapshot 2023-10-06T03:59:48.072 INFO:tasks.workunit.client.0.smithi157.stdout: create 164/37 file after the snapshot 2023-10-06T03:59:48.072 INFO:tasks.workunit.client.0.smithi157.stdout: create 164/38 file after the snapshot 2023-10-06T03:59:48.073 INFO:tasks.workunit.client.0.smithi157.stdout: create 164/39 file after the snapshot 2023-10-06T03:59:48.073 INFO:tasks.workunit.client.0.smithi157.stdout: create 165/.snap/snap-subdir-test 2023-10-06T03:59:48.099 INFO:tasks.workunit.client.0.smithi157.stdout: create 165/30 file after the snapshot 2023-10-06T03:59:48.100 INFO:tasks.workunit.client.0.smithi157.stdout: create 165/31 file after the snapshot 2023-10-06T03:59:48.101 INFO:tasks.workunit.client.0.smithi157.stdout: create 165/32 file after the snapshot 2023-10-06T03:59:48.102 INFO:tasks.workunit.client.0.smithi157.stdout: create 165/33 file after the snapshot 2023-10-06T03:59:48.102 INFO:tasks.workunit.client.0.smithi157.stdout: create 165/34 file after the snapshot 2023-10-06T03:59:48.103 INFO:tasks.workunit.client.0.smithi157.stdout: create 165/35 file after the snapshot 2023-10-06T03:59:48.104 INFO:tasks.workunit.client.0.smithi157.stdout: create 165/36 file after the snapshot 2023-10-06T03:59:48.104 INFO:tasks.workunit.client.0.smithi157.stdout: create 165/37 file after the snapshot 2023-10-06T03:59:48.105 INFO:tasks.workunit.client.0.smithi157.stdout: create 165/38 file after the snapshot 2023-10-06T03:59:48.369 INFO:tasks.workunit.client.0.smithi157.stdout: create 165/39 file after the snapshot 2023-10-06T03:59:48.369 INFO:tasks.workunit.client.0.smithi157.stdout: create 166/.snap/snap-subdir-test 2023-10-06T03:59:48.396 INFO:tasks.workunit.client.0.smithi157.stdout: create 166/30 file after the snapshot 2023-10-06T03:59:48.397 INFO:tasks.workunit.client.0.smithi157.stdout: create 166/31 file after the snapshot 2023-10-06T03:59:48.397 INFO:tasks.workunit.client.0.smithi157.stdout: create 166/32 file after the snapshot 2023-10-06T03:59:48.398 INFO:tasks.workunit.client.0.smithi157.stdout: create 166/33 file after the snapshot 2023-10-06T03:59:48.399 INFO:tasks.workunit.client.0.smithi157.stdout: create 166/34 file after the snapshot 2023-10-06T03:59:48.399 INFO:tasks.workunit.client.0.smithi157.stdout: create 166/35 file after the snapshot 2023-10-06T03:59:48.400 INFO:tasks.workunit.client.0.smithi157.stdout: create 166/36 file after the snapshot 2023-10-06T03:59:48.400 INFO:tasks.workunit.client.0.smithi157.stdout: create 166/37 file after the snapshot 2023-10-06T03:59:48.401 INFO:tasks.workunit.client.0.smithi157.stdout: create 166/38 file after the snapshot 2023-10-06T03:59:48.402 INFO:tasks.workunit.client.0.smithi157.stdout: create 166/39 file after the snapshot 2023-10-06T03:59:48.402 INFO:tasks.workunit.client.0.smithi157.stdout: create 167/.snap/snap-subdir-test 2023-10-06T03:59:48.428 INFO:tasks.workunit.client.0.smithi157.stdout: create 167/30 file after the snapshot 2023-10-06T03:59:48.429 INFO:tasks.workunit.client.0.smithi157.stdout: create 167/31 file after the snapshot 2023-10-06T03:59:48.429 INFO:tasks.workunit.client.0.smithi157.stdout: create 167/32 file after the snapshot 2023-10-06T03:59:48.430 INFO:tasks.workunit.client.0.smithi157.stdout: create 167/33 file after the snapshot 2023-10-06T03:59:48.431 INFO:tasks.workunit.client.0.smithi157.stdout: create 167/34 file after the snapshot 2023-10-06T03:59:48.431 INFO:tasks.workunit.client.0.smithi157.stdout: create 167/35 file after the snapshot 2023-10-06T03:59:48.432 INFO:tasks.workunit.client.0.smithi157.stdout: create 167/36 file after the snapshot 2023-10-06T03:59:48.433 INFO:tasks.workunit.client.0.smithi157.stdout: create 167/37 file after the snapshot 2023-10-06T03:59:48.433 INFO:tasks.workunit.client.0.smithi157.stdout: create 167/38 file after the snapshot 2023-10-06T03:59:48.434 INFO:tasks.workunit.client.0.smithi157.stdout: create 167/39 file after the snapshot 2023-10-06T03:59:48.435 INFO:tasks.workunit.client.0.smithi157.stdout: create 168/.snap/snap-subdir-test 2023-10-06T03:59:48.460 INFO:tasks.workunit.client.0.smithi157.stdout: create 168/30 file after the snapshot 2023-10-06T03:59:48.461 INFO:tasks.workunit.client.0.smithi157.stdout: create 168/31 file after the snapshot 2023-10-06T03:59:48.462 INFO:tasks.workunit.client.0.smithi157.stdout: create 168/32 file after the snapshot 2023-10-06T03:59:48.463 INFO:tasks.workunit.client.0.smithi157.stdout: create 168/33 file after the snapshot 2023-10-06T03:59:48.463 INFO:tasks.workunit.client.0.smithi157.stdout: create 168/34 file after the snapshot 2023-10-06T03:59:48.464 INFO:tasks.workunit.client.0.smithi157.stdout: create 168/35 file after the snapshot 2023-10-06T03:59:48.465 INFO:tasks.workunit.client.0.smithi157.stdout: create 168/36 file after the snapshot 2023-10-06T03:59:48.465 INFO:tasks.workunit.client.0.smithi157.stdout: create 168/37 file after the snapshot 2023-10-06T03:59:48.466 INFO:tasks.workunit.client.0.smithi157.stdout: create 168/38 file after the snapshot 2023-10-06T03:59:48.467 INFO:tasks.workunit.client.0.smithi157.stdout: create 168/39 file after the snapshot 2023-10-06T03:59:48.467 INFO:tasks.workunit.client.0.smithi157.stdout: create 169/.snap/snap-subdir-test 2023-10-06T03:59:48.493 INFO:tasks.workunit.client.0.smithi157.stdout: create 169/30 file after the snapshot 2023-10-06T03:59:48.494 INFO:tasks.workunit.client.0.smithi157.stdout: create 169/31 file after the snapshot 2023-10-06T03:59:48.494 INFO:tasks.workunit.client.0.smithi157.stdout: create 169/32 file after the snapshot 2023-10-06T03:59:48.495 INFO:tasks.workunit.client.0.smithi157.stdout: create 169/33 file after the snapshot 2023-10-06T03:59:48.496 INFO:tasks.workunit.client.0.smithi157.stdout: create 169/34 file after the snapshot 2023-10-06T03:59:48.496 INFO:tasks.workunit.client.0.smithi157.stdout: create 169/35 file after the snapshot 2023-10-06T03:59:48.497 INFO:tasks.workunit.client.0.smithi157.stdout: create 169/36 file after the snapshot 2023-10-06T03:59:48.498 INFO:tasks.workunit.client.0.smithi157.stdout: create 169/37 file after the snapshot 2023-10-06T03:59:48.498 INFO:tasks.workunit.client.0.smithi157.stdout: create 169/38 file after the snapshot 2023-10-06T03:59:48.499 INFO:tasks.workunit.client.0.smithi157.stdout: create 169/39 file after the snapshot 2023-10-06T03:59:48.500 INFO:tasks.workunit.client.0.smithi157.stdout: create 170/.snap/snap-subdir-test 2023-10-06T03:59:48.524 INFO:tasks.workunit.client.0.smithi157.stdout: create 170/30 file after the snapshot 2023-10-06T03:59:48.525 INFO:tasks.workunit.client.0.smithi157.stdout: create 170/31 file after the snapshot 2023-10-06T03:59:48.526 INFO:tasks.workunit.client.0.smithi157.stdout: create 170/32 file after the snapshot 2023-10-06T03:59:48.527 INFO:tasks.workunit.client.0.smithi157.stdout: create 170/33 file after the snapshot 2023-10-06T03:59:48.527 INFO:tasks.workunit.client.0.smithi157.stdout: create 170/34 file after the snapshot 2023-10-06T03:59:48.528 INFO:tasks.workunit.client.0.smithi157.stdout: create 170/35 file after the snapshot 2023-10-06T03:59:48.529 INFO:tasks.workunit.client.0.smithi157.stdout: create 170/36 file after the snapshot 2023-10-06T03:59:48.529 INFO:tasks.workunit.client.0.smithi157.stdout: create 170/37 file after the snapshot 2023-10-06T03:59:48.530 INFO:tasks.workunit.client.0.smithi157.stdout: create 170/38 file after the snapshot 2023-10-06T03:59:48.531 INFO:tasks.workunit.client.0.smithi157.stdout: create 170/39 file after the snapshot 2023-10-06T03:59:48.531 INFO:tasks.workunit.client.0.smithi157.stdout: create 171/.snap/snap-subdir-test 2023-10-06T03:59:49.050 INFO:tasks.workunit.client.0.smithi157.stdout: create 171/30 file after the snapshot 2023-10-06T03:59:49.051 INFO:tasks.workunit.client.0.smithi157.stdout: create 171/31 file after the snapshot 2023-10-06T03:59:49.052 INFO:tasks.workunit.client.0.smithi157.stdout: create 171/32 file after the snapshot 2023-10-06T03:59:49.052 INFO:tasks.workunit.client.0.smithi157.stdout: create 171/33 file after the snapshot 2023-10-06T03:59:49.053 INFO:tasks.workunit.client.0.smithi157.stdout: create 171/34 file after the snapshot 2023-10-06T03:59:49.054 INFO:tasks.workunit.client.0.smithi157.stdout: create 171/35 file after the snapshot 2023-10-06T03:59:49.055 INFO:tasks.workunit.client.0.smithi157.stdout: create 171/36 file after the snapshot 2023-10-06T03:59:49.055 INFO:tasks.workunit.client.0.smithi157.stdout: create 171/37 file after the snapshot 2023-10-06T03:59:49.056 INFO:tasks.workunit.client.0.smithi157.stdout: create 171/38 file after the snapshot 2023-10-06T03:59:49.057 INFO:tasks.workunit.client.0.smithi157.stdout: create 171/39 file after the snapshot 2023-10-06T03:59:49.057 INFO:tasks.workunit.client.0.smithi157.stdout: create 172/.snap/snap-subdir-test 2023-10-06T03:59:49.083 INFO:tasks.workunit.client.0.smithi157.stdout: create 172/30 file after the snapshot 2023-10-06T03:59:49.084 INFO:tasks.workunit.client.0.smithi157.stdout: create 172/31 file after the snapshot 2023-10-06T03:59:49.084 INFO:tasks.workunit.client.0.smithi157.stdout: create 172/32 file after the snapshot 2023-10-06T03:59:49.085 INFO:tasks.workunit.client.0.smithi157.stdout: create 172/33 file after the snapshot 2023-10-06T03:59:49.086 INFO:tasks.workunit.client.0.smithi157.stdout: create 172/34 file after the snapshot 2023-10-06T03:59:49.086 INFO:tasks.workunit.client.0.smithi157.stdout: create 172/35 file after the snapshot 2023-10-06T03:59:49.087 INFO:tasks.workunit.client.0.smithi157.stdout: create 172/36 file after the snapshot 2023-10-06T03:59:49.088 INFO:tasks.workunit.client.0.smithi157.stdout: create 172/37 file after the snapshot 2023-10-06T03:59:49.088 INFO:tasks.workunit.client.0.smithi157.stdout: create 172/38 file after the snapshot 2023-10-06T03:59:49.089 INFO:tasks.workunit.client.0.smithi157.stdout: create 172/39 file after the snapshot 2023-10-06T03:59:49.090 INFO:tasks.workunit.client.0.smithi157.stdout: create 173/.snap/snap-subdir-test 2023-10-06T03:59:49.115 INFO:tasks.workunit.client.0.smithi157.stdout: create 173/30 file after the snapshot 2023-10-06T03:59:49.116 INFO:tasks.workunit.client.0.smithi157.stdout: create 173/31 file after the snapshot 2023-10-06T03:59:49.116 INFO:tasks.workunit.client.0.smithi157.stdout: create 173/32 file after the snapshot 2023-10-06T03:59:49.117 INFO:tasks.workunit.client.0.smithi157.stdout: create 173/33 file after the snapshot 2023-10-06T03:59:49.118 INFO:tasks.workunit.client.0.smithi157.stdout: create 173/34 file after the snapshot 2023-10-06T03:59:49.118 INFO:tasks.workunit.client.0.smithi157.stdout: create 173/35 file after the snapshot 2023-10-06T03:59:49.119 INFO:tasks.workunit.client.0.smithi157.stdout: create 173/36 file after the snapshot 2023-10-06T03:59:49.120 INFO:tasks.workunit.client.0.smithi157.stdout: create 173/37 file after the snapshot 2023-10-06T03:59:49.120 INFO:tasks.workunit.client.0.smithi157.stdout: create 173/38 file after the snapshot 2023-10-06T03:59:49.121 INFO:tasks.workunit.client.0.smithi157.stdout: create 173/39 file after the snapshot 2023-10-06T03:59:49.122 INFO:tasks.workunit.client.0.smithi157.stdout: create 174/.snap/snap-subdir-test 2023-10-06T03:59:49.147 INFO:tasks.workunit.client.0.smithi157.stdout: create 174/30 file after the snapshot 2023-10-06T03:59:49.148 INFO:tasks.workunit.client.0.smithi157.stdout: create 174/31 file after the snapshot 2023-10-06T03:59:49.149 INFO:tasks.workunit.client.0.smithi157.stdout: create 174/32 file after the snapshot 2023-10-06T03:59:49.150 INFO:tasks.workunit.client.0.smithi157.stdout: create 174/33 file after the snapshot 2023-10-06T03:59:49.150 INFO:tasks.workunit.client.0.smithi157.stdout: create 174/34 file after the snapshot 2023-10-06T03:59:49.151 INFO:tasks.workunit.client.0.smithi157.stdout: create 174/35 file after the snapshot 2023-10-06T03:59:49.151 INFO:tasks.workunit.client.0.smithi157.stdout: create 174/36 file after the snapshot 2023-10-06T03:59:49.152 INFO:tasks.workunit.client.0.smithi157.stdout: create 174/37 file after the snapshot 2023-10-06T03:59:49.153 INFO:tasks.workunit.client.0.smithi157.stdout: create 174/38 file after the snapshot 2023-10-06T03:59:49.153 INFO:tasks.workunit.client.0.smithi157.stdout: create 174/39 file after the snapshot 2023-10-06T03:59:49.154 INFO:tasks.workunit.client.0.smithi157.stdout: create 175/.snap/snap-subdir-test 2023-10-06T03:59:49.179 INFO:tasks.workunit.client.0.smithi157.stdout: create 175/30 file after the snapshot 2023-10-06T03:59:49.180 INFO:tasks.workunit.client.0.smithi157.stdout: create 175/31 file after the snapshot 2023-10-06T03:59:49.181 INFO:tasks.workunit.client.0.smithi157.stdout: create 175/32 file after the snapshot 2023-10-06T03:59:49.181 INFO:tasks.workunit.client.0.smithi157.stdout: create 175/33 file after the snapshot 2023-10-06T03:59:49.182 INFO:tasks.workunit.client.0.smithi157.stdout: create 175/34 file after the snapshot 2023-10-06T03:59:49.183 INFO:tasks.workunit.client.0.smithi157.stdout: create 175/35 file after the snapshot 2023-10-06T03:59:49.183 INFO:tasks.workunit.client.0.smithi157.stdout: create 175/36 file after the snapshot 2023-10-06T03:59:49.184 INFO:tasks.workunit.client.0.smithi157.stdout: create 175/37 file after the snapshot 2023-10-06T03:59:49.185 INFO:tasks.workunit.client.0.smithi157.stdout: create 175/38 file after the snapshot 2023-10-06T03:59:49.185 INFO:tasks.workunit.client.0.smithi157.stdout: create 175/39 file after the snapshot 2023-10-06T03:59:49.186 INFO:tasks.workunit.client.0.smithi157.stdout: create 176/.snap/snap-subdir-test 2023-10-06T03:59:49.212 INFO:tasks.workunit.client.0.smithi157.stdout: create 176/30 file after the snapshot 2023-10-06T03:59:49.213 INFO:tasks.workunit.client.0.smithi157.stdout: create 176/31 file after the snapshot 2023-10-06T03:59:49.213 INFO:tasks.workunit.client.0.smithi157.stdout: create 176/32 file after the snapshot 2023-10-06T03:59:49.214 INFO:tasks.workunit.client.0.smithi157.stdout: create 176/33 file after the snapshot 2023-10-06T03:59:49.214 INFO:tasks.workunit.client.0.smithi157.stdout: create 176/34 file after the snapshot 2023-10-06T03:59:49.215 INFO:tasks.workunit.client.0.smithi157.stdout: create 176/35 file after the snapshot 2023-10-06T03:59:49.216 INFO:tasks.workunit.client.0.smithi157.stdout: create 176/36 file after the snapshot 2023-10-06T03:59:49.217 INFO:tasks.workunit.client.0.smithi157.stdout: create 176/37 file after the snapshot 2023-10-06T03:59:49.218 INFO:tasks.workunit.client.0.smithi157.stdout: create 176/38 file after the snapshot 2023-10-06T03:59:49.220 INFO:tasks.workunit.client.0.smithi157.stdout: create 176/39 file after the snapshot 2023-10-06T03:59:49.220 INFO:tasks.workunit.client.0.smithi157.stdout: create 177/.snap/snap-subdir-test 2023-10-06T03:59:49.246 INFO:tasks.workunit.client.0.smithi157.stdout: create 177/30 file after the snapshot 2023-10-06T03:59:49.247 INFO:tasks.workunit.client.0.smithi157.stdout: create 177/31 file after the snapshot 2023-10-06T03:59:49.248 INFO:tasks.workunit.client.0.smithi157.stdout: create 177/32 file after the snapshot 2023-10-06T03:59:49.248 INFO:tasks.workunit.client.0.smithi157.stdout: create 177/33 file after the snapshot 2023-10-06T03:59:49.249 INFO:tasks.workunit.client.0.smithi157.stdout: create 177/34 file after the snapshot 2023-10-06T03:59:49.250 INFO:tasks.workunit.client.0.smithi157.stdout: create 177/35 file after the snapshot 2023-10-06T03:59:49.250 INFO:tasks.workunit.client.0.smithi157.stdout: create 177/36 file after the snapshot 2023-10-06T03:59:49.251 INFO:tasks.workunit.client.0.smithi157.stdout: create 177/37 file after the snapshot 2023-10-06T03:59:49.252 INFO:tasks.workunit.client.0.smithi157.stdout: create 177/38 file after the snapshot 2023-10-06T03:59:49.252 INFO:tasks.workunit.client.0.smithi157.stdout: create 177/39 file after the snapshot 2023-10-06T03:59:49.253 INFO:tasks.workunit.client.0.smithi157.stdout: create 178/.snap/snap-subdir-test 2023-10-06T03:59:49.279 INFO:tasks.workunit.client.0.smithi157.stdout: create 178/30 file after the snapshot 2023-10-06T03:59:49.280 INFO:tasks.workunit.client.0.smithi157.stdout: create 178/31 file after the snapshot 2023-10-06T03:59:49.281 INFO:tasks.workunit.client.0.smithi157.stdout: create 178/32 file after the snapshot 2023-10-06T03:59:49.282 INFO:tasks.workunit.client.0.smithi157.stdout: create 178/33 file after the snapshot 2023-10-06T03:59:49.283 INFO:tasks.workunit.client.0.smithi157.stdout: create 178/34 file after the snapshot 2023-10-06T03:59:49.284 INFO:tasks.workunit.client.0.smithi157.stdout: create 178/35 file after the snapshot 2023-10-06T03:59:49.285 INFO:tasks.workunit.client.0.smithi157.stdout: create 178/36 file after the snapshot 2023-10-06T03:59:49.286 INFO:tasks.workunit.client.0.smithi157.stdout: create 178/37 file after the snapshot 2023-10-06T03:59:49.287 INFO:tasks.workunit.client.0.smithi157.stdout: create 178/38 file after the snapshot 2023-10-06T03:59:49.288 INFO:tasks.workunit.client.0.smithi157.stdout: create 178/39 file after the snapshot 2023-10-06T03:59:49.289 INFO:tasks.workunit.client.0.smithi157.stdout: create 179/.snap/snap-subdir-test 2023-10-06T03:59:50.000 INFO:tasks.workunit.client.0.smithi157.stdout: create 179/30 file after the snapshot 2023-10-06T03:59:50.001 INFO:tasks.workunit.client.0.smithi157.stdout: create 179/31 file after the snapshot 2023-10-06T03:59:50.001 INFO:tasks.workunit.client.0.smithi157.stdout: create 179/32 file after the snapshot 2023-10-06T03:59:50.002 INFO:tasks.workunit.client.0.smithi157.stdout: create 179/33 file after the snapshot 2023-10-06T03:59:50.003 INFO:tasks.workunit.client.0.smithi157.stdout: create 179/34 file after the snapshot 2023-10-06T03:59:50.004 INFO:tasks.workunit.client.0.smithi157.stdout: create 179/35 file after the snapshot 2023-10-06T03:59:50.005 INFO:tasks.workunit.client.0.smithi157.stdout: create 179/36 file after the snapshot 2023-10-06T03:59:50.006 INFO:tasks.workunit.client.0.smithi157.stdout: create 179/37 file after the snapshot 2023-10-06T03:59:50.007 INFO:tasks.workunit.client.0.smithi157.stdout: create 179/38 file after the snapshot 2023-10-06T03:59:50.008 INFO:tasks.workunit.client.0.smithi157.stdout: create 179/39 file after the snapshot 2023-10-06T03:59:50.009 INFO:tasks.workunit.client.0.smithi157.stdout: create 180/.snap/snap-subdir-test 2023-10-06T03:59:50.042 INFO:tasks.workunit.client.0.smithi157.stdout: create 180/30 file after the snapshot 2023-10-06T03:59:50.043 INFO:tasks.workunit.client.0.smithi157.stdout: create 180/31 file after the snapshot 2023-10-06T03:59:50.044 INFO:tasks.workunit.client.0.smithi157.stdout: create 180/32 file after the snapshot 2023-10-06T03:59:50.045 INFO:tasks.workunit.client.0.smithi157.stdout: create 180/33 file after the snapshot 2023-10-06T03:59:50.046 INFO:tasks.workunit.client.0.smithi157.stdout: create 180/34 file after the snapshot 2023-10-06T03:59:50.047 INFO:tasks.workunit.client.0.smithi157.stdout: create 180/35 file after the snapshot 2023-10-06T03:59:50.048 INFO:tasks.workunit.client.0.smithi157.stdout: create 180/36 file after the snapshot 2023-10-06T03:59:50.049 INFO:tasks.workunit.client.0.smithi157.stdout: create 180/37 file after the snapshot 2023-10-06T03:59:50.050 INFO:tasks.workunit.client.0.smithi157.stdout: create 180/38 file after the snapshot 2023-10-06T03:59:50.051 INFO:tasks.workunit.client.0.smithi157.stdout: create 180/39 file after the snapshot 2023-10-06T03:59:50.052 INFO:tasks.workunit.client.0.smithi157.stdout: create 181/.snap/snap-subdir-test 2023-10-06T03:59:50.084 INFO:tasks.workunit.client.0.smithi157.stdout: create 181/30 file after the snapshot 2023-10-06T03:59:50.086 INFO:tasks.workunit.client.0.smithi157.stdout: create 181/31 file after the snapshot 2023-10-06T03:59:50.086 INFO:tasks.workunit.client.0.smithi157.stdout: create 181/32 file after the snapshot 2023-10-06T03:59:50.090 INFO:tasks.workunit.client.0.smithi157.stdout: create 181/33 file after the snapshot 2023-10-06T03:59:50.090 INFO:tasks.workunit.client.0.smithi157.stdout: create 181/34 file after the snapshot 2023-10-06T03:59:50.090 INFO:tasks.workunit.client.0.smithi157.stdout: create 181/35 file after the snapshot 2023-10-06T03:59:50.090 INFO:tasks.workunit.client.0.smithi157.stdout: create 181/36 file after the snapshot 2023-10-06T03:59:50.090 INFO:tasks.workunit.client.0.smithi157.stdout: create 181/37 file after the snapshot 2023-10-06T03:59:50.090 INFO:tasks.workunit.client.0.smithi157.stdout: create 181/38 file after the snapshot 2023-10-06T03:59:50.091 INFO:tasks.workunit.client.0.smithi157.stdout: create 181/39 file after the snapshot 2023-10-06T03:59:50.091 INFO:tasks.workunit.client.0.smithi157.stdout: create 182/.snap/snap-subdir-test 2023-10-06T03:59:50.123 INFO:tasks.workunit.client.0.smithi157.stdout: create 182/30 file after the snapshot 2023-10-06T03:59:50.124 INFO:tasks.workunit.client.0.smithi157.stdout: create 182/31 file after the snapshot 2023-10-06T03:59:50.124 INFO:tasks.workunit.client.0.smithi157.stdout: create 182/32 file after the snapshot 2023-10-06T03:59:50.125 INFO:tasks.workunit.client.0.smithi157.stdout: create 182/33 file after the snapshot 2023-10-06T03:59:50.126 INFO:tasks.workunit.client.0.smithi157.stdout: create 182/34 file after the snapshot 2023-10-06T03:59:50.126 INFO:tasks.workunit.client.0.smithi157.stdout: create 182/35 file after the snapshot 2023-10-06T03:59:50.127 INFO:tasks.workunit.client.0.smithi157.stdout: create 182/36 file after the snapshot 2023-10-06T03:59:50.127 INFO:tasks.workunit.client.0.smithi157.stdout: create 182/37 file after the snapshot 2023-10-06T03:59:50.128 INFO:tasks.workunit.client.0.smithi157.stdout: create 182/38 file after the snapshot 2023-10-06T03:59:50.129 INFO:tasks.workunit.client.0.smithi157.stdout: create 182/39 file after the snapshot 2023-10-06T03:59:50.129 INFO:tasks.workunit.client.0.smithi157.stdout: create 183/.snap/snap-subdir-test 2023-10-06T03:59:50.161 INFO:tasks.workunit.client.0.smithi157.stdout: create 183/30 file after the snapshot 2023-10-06T03:59:50.162 INFO:tasks.workunit.client.0.smithi157.stdout: create 183/31 file after the snapshot 2023-10-06T03:59:50.162 INFO:tasks.workunit.client.0.smithi157.stdout: create 183/32 file after the snapshot 2023-10-06T03:59:50.163 INFO:tasks.workunit.client.0.smithi157.stdout: create 183/33 file after the snapshot 2023-10-06T03:59:50.164 INFO:tasks.workunit.client.0.smithi157.stdout: create 183/34 file after the snapshot 2023-10-06T03:59:50.164 INFO:tasks.workunit.client.0.smithi157.stdout: create 183/35 file after the snapshot 2023-10-06T03:59:50.165 INFO:tasks.workunit.client.0.smithi157.stdout: create 183/36 file after the snapshot 2023-10-06T03:59:50.166 INFO:tasks.workunit.client.0.smithi157.stdout: create 183/37 file after the snapshot 2023-10-06T03:59:50.166 INFO:tasks.workunit.client.0.smithi157.stdout: create 183/38 file after the snapshot 2023-10-06T03:59:50.167 INFO:tasks.workunit.client.0.smithi157.stdout: create 183/39 file after the snapshot 2023-10-06T03:59:50.168 INFO:tasks.workunit.client.0.smithi157.stdout: create 184/.snap/snap-subdir-test 2023-10-06T03:59:50.199 INFO:tasks.workunit.client.0.smithi157.stdout: create 184/30 file after the snapshot 2023-10-06T03:59:50.201 INFO:tasks.workunit.client.0.smithi157.stdout: create 184/31 file after the snapshot 2023-10-06T03:59:50.201 INFO:tasks.workunit.client.0.smithi157.stdout: create 184/32 file after the snapshot 2023-10-06T03:59:50.202 INFO:tasks.workunit.client.0.smithi157.stdout: create 184/33 file after the snapshot 2023-10-06T03:59:50.203 INFO:tasks.workunit.client.0.smithi157.stdout: create 184/34 file after the snapshot 2023-10-06T03:59:50.203 INFO:tasks.workunit.client.0.smithi157.stdout: create 184/35 file after the snapshot 2023-10-06T03:59:50.204 INFO:tasks.workunit.client.0.smithi157.stdout: create 184/36 file after the snapshot 2023-10-06T03:59:50.205 INFO:tasks.workunit.client.0.smithi157.stdout: create 184/37 file after the snapshot 2023-10-06T03:59:50.205 INFO:tasks.workunit.client.0.smithi157.stdout: create 184/38 file after the snapshot 2023-10-06T03:59:50.206 INFO:tasks.workunit.client.0.smithi157.stdout: create 184/39 file after the snapshot 2023-10-06T03:59:50.206 INFO:tasks.workunit.client.0.smithi157.stdout: create 185/.snap/snap-subdir-test 2023-10-06T03:59:50.239 INFO:tasks.workunit.client.0.smithi157.stdout: create 185/30 file after the snapshot 2023-10-06T03:59:50.240 INFO:tasks.workunit.client.0.smithi157.stdout: create 185/31 file after the snapshot 2023-10-06T03:59:50.241 INFO:tasks.workunit.client.0.smithi157.stdout: create 185/32 file after the snapshot 2023-10-06T03:59:50.242 INFO:tasks.workunit.client.0.smithi157.stdout: create 185/33 file after the snapshot 2023-10-06T03:59:50.243 INFO:tasks.workunit.client.0.smithi157.stdout: create 185/34 file after the snapshot 2023-10-06T03:59:50.244 INFO:tasks.workunit.client.0.smithi157.stdout: create 185/35 file after the snapshot 2023-10-06T03:59:50.245 INFO:tasks.workunit.client.0.smithi157.stdout: create 185/36 file after the snapshot 2023-10-06T03:59:50.246 INFO:tasks.workunit.client.0.smithi157.stdout: create 185/37 file after the snapshot 2023-10-06T03:59:50.247 INFO:tasks.workunit.client.0.smithi157.stdout: create 185/38 file after the snapshot 2023-10-06T03:59:50.248 INFO:tasks.workunit.client.0.smithi157.stdout: create 185/39 file after the snapshot 2023-10-06T03:59:50.249 INFO:tasks.workunit.client.0.smithi157.stdout: create 186/.snap/snap-subdir-test 2023-10-06T03:59:50.281 INFO:tasks.workunit.client.0.smithi157.stdout: create 186/30 file after the snapshot 2023-10-06T03:59:50.282 INFO:tasks.workunit.client.0.smithi157.stdout: create 186/31 file after the snapshot 2023-10-06T03:59:50.283 INFO:tasks.workunit.client.0.smithi157.stdout: create 186/32 file after the snapshot 2023-10-06T03:59:50.283 INFO:tasks.workunit.client.0.smithi157.stdout: create 186/33 file after the snapshot 2023-10-06T03:59:50.284 INFO:tasks.workunit.client.0.smithi157.stdout: create 186/34 file after the snapshot 2023-10-06T03:59:50.285 INFO:tasks.workunit.client.0.smithi157.stdout: create 186/35 file after the snapshot 2023-10-06T03:59:50.285 INFO:tasks.workunit.client.0.smithi157.stdout: create 186/36 file after the snapshot 2023-10-06T03:59:50.286 INFO:tasks.workunit.client.0.smithi157.stdout: create 186/37 file after the snapshot 2023-10-06T03:59:50.287 INFO:tasks.workunit.client.0.smithi157.stdout: create 186/38 file after the snapshot 2023-10-06T03:59:50.287 INFO:tasks.workunit.client.0.smithi157.stdout: create 186/39 file after the snapshot 2023-10-06T03:59:50.288 INFO:tasks.workunit.client.0.smithi157.stdout: create 187/.snap/snap-subdir-test 2023-10-06T03:59:50.325 INFO:tasks.workunit.client.0.smithi157.stdout: create 187/30 file after the snapshot 2023-10-06T03:59:50.326 INFO:tasks.workunit.client.0.smithi157.stdout: create 187/31 file after the snapshot 2023-10-06T03:59:50.326 INFO:tasks.workunit.client.0.smithi157.stdout: create 187/32 file after the snapshot 2023-10-06T03:59:50.327 INFO:tasks.workunit.client.0.smithi157.stdout: create 187/33 file after the snapshot 2023-10-06T03:59:50.328 INFO:tasks.workunit.client.0.smithi157.stdout: create 187/34 file after the snapshot 2023-10-06T03:59:50.329 INFO:tasks.workunit.client.0.smithi157.stdout: create 187/35 file after the snapshot 2023-10-06T03:59:50.329 INFO:tasks.workunit.client.0.smithi157.stdout: create 187/36 file after the snapshot 2023-10-06T03:59:50.330 INFO:tasks.workunit.client.0.smithi157.stdout: create 187/37 file after the snapshot 2023-10-06T03:59:50.330 INFO:tasks.workunit.client.0.smithi157.stdout: create 187/38 file after the snapshot 2023-10-06T03:59:50.331 INFO:tasks.workunit.client.0.smithi157.stdout: create 187/39 file after the snapshot 2023-10-06T03:59:50.332 INFO:tasks.workunit.client.0.smithi157.stdout: create 188/.snap/snap-subdir-test 2023-10-06T03:59:50.365 INFO:tasks.workunit.client.0.smithi157.stdout: create 188/30 file after the snapshot 2023-10-06T03:59:50.366 INFO:tasks.workunit.client.0.smithi157.stdout: create 188/31 file after the snapshot 2023-10-06T03:59:50.367 INFO:tasks.workunit.client.0.smithi157.stdout: create 188/32 file after the snapshot 2023-10-06T03:59:50.368 INFO:tasks.workunit.client.0.smithi157.stdout: create 188/33 file after the snapshot 2023-10-06T03:59:50.369 INFO:tasks.workunit.client.0.smithi157.stdout: create 188/34 file after the snapshot 2023-10-06T03:59:50.370 INFO:tasks.workunit.client.0.smithi157.stdout: create 188/35 file after the snapshot 2023-10-06T03:59:50.371 INFO:tasks.workunit.client.0.smithi157.stdout: create 188/36 file after the snapshot 2023-10-06T03:59:50.372 INFO:tasks.workunit.client.0.smithi157.stdout: create 188/37 file after the snapshot 2023-10-06T03:59:50.373 INFO:tasks.workunit.client.0.smithi157.stdout: create 188/38 file after the snapshot 2023-10-06T03:59:50.374 INFO:tasks.workunit.client.0.smithi157.stdout: create 188/39 file after the snapshot 2023-10-06T03:59:50.375 INFO:tasks.workunit.client.0.smithi157.stdout: create 189/.snap/snap-subdir-test 2023-10-06T03:59:50.720 INFO:tasks.workunit.client.0.smithi157.stdout: create 189/30 file after the snapshot 2023-10-06T03:59:50.721 INFO:tasks.workunit.client.0.smithi157.stdout: create 189/31 file after the snapshot 2023-10-06T03:59:50.722 INFO:tasks.workunit.client.0.smithi157.stdout: create 189/32 file after the snapshot 2023-10-06T03:59:50.723 INFO:tasks.workunit.client.0.smithi157.stdout: create 189/33 file after the snapshot 2023-10-06T03:59:50.724 INFO:tasks.workunit.client.0.smithi157.stdout: create 189/34 file after the snapshot 2023-10-06T03:59:50.725 INFO:tasks.workunit.client.0.smithi157.stdout: create 189/35 file after the snapshot 2023-10-06T03:59:50.726 INFO:tasks.workunit.client.0.smithi157.stdout: create 189/36 file after the snapshot 2023-10-06T03:59:50.727 INFO:tasks.workunit.client.0.smithi157.stdout: create 189/37 file after the snapshot 2023-10-06T03:59:50.728 INFO:tasks.workunit.client.0.smithi157.stdout: create 189/38 file after the snapshot 2023-10-06T03:59:50.729 INFO:tasks.workunit.client.0.smithi157.stdout: create 189/39 file after the snapshot 2023-10-06T03:59:50.730 INFO:tasks.workunit.client.0.smithi157.stdout: create 190/.snap/snap-subdir-test 2023-10-06T03:59:50.756 INFO:tasks.workunit.client.0.smithi157.stdout: create 190/30 file after the snapshot 2023-10-06T03:59:50.757 INFO:tasks.workunit.client.0.smithi157.stdout: create 190/31 file after the snapshot 2023-10-06T03:59:50.757 INFO:tasks.workunit.client.0.smithi157.stdout: create 190/32 file after the snapshot 2023-10-06T03:59:50.758 INFO:tasks.workunit.client.0.smithi157.stdout: create 190/33 file after the snapshot 2023-10-06T03:59:50.759 INFO:tasks.workunit.client.0.smithi157.stdout: create 190/34 file after the snapshot 2023-10-06T03:59:50.759 INFO:tasks.workunit.client.0.smithi157.stdout: create 190/35 file after the snapshot 2023-10-06T03:59:50.760 INFO:tasks.workunit.client.0.smithi157.stdout: create 190/36 file after the snapshot 2023-10-06T03:59:50.761 INFO:tasks.workunit.client.0.smithi157.stdout: create 190/37 file after the snapshot 2023-10-06T03:59:50.761 INFO:tasks.workunit.client.0.smithi157.stdout: create 190/38 file after the snapshot 2023-10-06T03:59:50.762 INFO:tasks.workunit.client.0.smithi157.stdout: create 190/39 file after the snapshot 2023-10-06T03:59:50.763 INFO:tasks.workunit.client.0.smithi157.stdout: create 191/.snap/snap-subdir-test 2023-10-06T03:59:50.788 INFO:tasks.workunit.client.0.smithi157.stdout: create 191/30 file after the snapshot 2023-10-06T03:59:50.789 INFO:tasks.workunit.client.0.smithi157.stdout: create 191/31 file after the snapshot 2023-10-06T03:59:50.790 INFO:tasks.workunit.client.0.smithi157.stdout: create 191/32 file after the snapshot 2023-10-06T03:59:50.790 INFO:tasks.workunit.client.0.smithi157.stdout: create 191/33 file after the snapshot 2023-10-06T03:59:50.791 INFO:tasks.workunit.client.0.smithi157.stdout: create 191/34 file after the snapshot 2023-10-06T03:59:50.792 INFO:tasks.workunit.client.0.smithi157.stdout: create 191/35 file after the snapshot 2023-10-06T03:59:50.792 INFO:tasks.workunit.client.0.smithi157.stdout: create 191/36 file after the snapshot 2023-10-06T03:59:50.793 INFO:tasks.workunit.client.0.smithi157.stdout: create 191/37 file after the snapshot 2023-10-06T03:59:50.794 INFO:tasks.workunit.client.0.smithi157.stdout: create 191/38 file after the snapshot 2023-10-06T03:59:50.794 INFO:tasks.workunit.client.0.smithi157.stdout: create 191/39 file after the snapshot 2023-10-06T03:59:50.795 INFO:tasks.workunit.client.0.smithi157.stdout: create 192/.snap/snap-subdir-test 2023-10-06T03:59:50.821 INFO:tasks.workunit.client.0.smithi157.stdout: create 192/30 file after the snapshot 2023-10-06T03:59:50.822 INFO:tasks.workunit.client.0.smithi157.stdout: create 192/31 file after the snapshot 2023-10-06T03:59:50.824 INFO:tasks.workunit.client.0.smithi157.stdout: create 192/32 file after the snapshot 2023-10-06T03:59:50.825 INFO:tasks.workunit.client.0.smithi157.stdout: create 192/33 file after the snapshot 2023-10-06T03:59:50.825 INFO:tasks.workunit.client.0.smithi157.stdout: create 192/34 file after the snapshot 2023-10-06T03:59:50.826 INFO:tasks.workunit.client.0.smithi157.stdout: create 192/35 file after the snapshot 2023-10-06T03:59:50.827 INFO:tasks.workunit.client.0.smithi157.stdout: create 192/36 file after the snapshot 2023-10-06T03:59:50.827 INFO:tasks.workunit.client.0.smithi157.stdout: create 192/37 file after the snapshot 2023-10-06T03:59:50.828 INFO:tasks.workunit.client.0.smithi157.stdout: create 192/38 file after the snapshot 2023-10-06T03:59:50.828 INFO:tasks.workunit.client.0.smithi157.stdout: create 192/39 file after the snapshot 2023-10-06T03:59:50.829 INFO:tasks.workunit.client.0.smithi157.stdout: create 193/.snap/snap-subdir-test 2023-10-06T03:59:51.726 INFO:tasks.workunit.client.0.smithi157.stdout: create 193/30 file after the snapshot 2023-10-06T03:59:51.726 INFO:tasks.workunit.client.0.smithi157.stdout: create 193/31 file after the snapshot 2023-10-06T03:59:51.727 INFO:tasks.workunit.client.0.smithi157.stdout: create 193/32 file after the snapshot 2023-10-06T03:59:51.727 INFO:tasks.workunit.client.0.smithi157.stdout: create 193/33 file after the snapshot 2023-10-06T03:59:51.727 INFO:tasks.workunit.client.0.smithi157.stdout: create 193/34 file after the snapshot 2023-10-06T03:59:51.728 INFO:tasks.workunit.client.0.smithi157.stdout: create 193/35 file after the snapshot 2023-10-06T03:59:51.729 INFO:tasks.workunit.client.0.smithi157.stdout: create 193/36 file after the snapshot 2023-10-06T03:59:51.730 INFO:tasks.workunit.client.0.smithi157.stdout: create 193/37 file after the snapshot 2023-10-06T03:59:51.730 INFO:tasks.workunit.client.0.smithi157.stdout: create 193/38 file after the snapshot 2023-10-06T03:59:51.731 INFO:tasks.workunit.client.0.smithi157.stdout: create 193/39 file after the snapshot 2023-10-06T03:59:51.732 INFO:tasks.workunit.client.0.smithi157.stdout: create 194/.snap/snap-subdir-test 2023-10-06T03:59:51.758 INFO:tasks.workunit.client.0.smithi157.stdout: create 194/30 file after the snapshot 2023-10-06T03:59:51.759 INFO:tasks.workunit.client.0.smithi157.stdout: create 194/31 file after the snapshot 2023-10-06T03:59:51.759 INFO:tasks.workunit.client.0.smithi157.stdout: create 194/32 file after the snapshot 2023-10-06T03:59:51.760 INFO:tasks.workunit.client.0.smithi157.stdout: create 194/33 file after the snapshot 2023-10-06T03:59:51.761 INFO:tasks.workunit.client.0.smithi157.stdout: create 194/34 file after the snapshot 2023-10-06T03:59:51.761 INFO:tasks.workunit.client.0.smithi157.stdout: create 194/35 file after the snapshot 2023-10-06T03:59:51.762 INFO:tasks.workunit.client.0.smithi157.stdout: create 194/36 file after the snapshot 2023-10-06T03:59:51.763 INFO:tasks.workunit.client.0.smithi157.stdout: create 194/37 file after the snapshot 2023-10-06T03:59:51.763 INFO:tasks.workunit.client.0.smithi157.stdout: create 194/38 file after the snapshot 2023-10-06T03:59:51.764 INFO:tasks.workunit.client.0.smithi157.stdout: create 194/39 file after the snapshot 2023-10-06T03:59:51.765 INFO:tasks.workunit.client.0.smithi157.stdout: create 195/.snap/snap-subdir-test 2023-10-06T03:59:51.791 INFO:tasks.workunit.client.0.smithi157.stdout: create 195/30 file after the snapshot 2023-10-06T03:59:51.792 INFO:tasks.workunit.client.0.smithi157.stdout: create 195/31 file after the snapshot 2023-10-06T03:59:51.792 INFO:tasks.workunit.client.0.smithi157.stdout: create 195/32 file after the snapshot 2023-10-06T03:59:51.794 INFO:tasks.workunit.client.0.smithi157.stdout: create 195/33 file after the snapshot 2023-10-06T03:59:51.794 INFO:tasks.workunit.client.0.smithi157.stdout: create 195/34 file after the snapshot 2023-10-06T03:59:51.795 INFO:tasks.workunit.client.0.smithi157.stdout: create 195/35 file after the snapshot 2023-10-06T03:59:51.795 INFO:tasks.workunit.client.0.smithi157.stdout: create 195/36 file after the snapshot 2023-10-06T03:59:51.796 INFO:tasks.workunit.client.0.smithi157.stdout: create 195/37 file after the snapshot 2023-10-06T03:59:51.796 INFO:tasks.workunit.client.0.smithi157.stdout: create 195/38 file after the snapshot 2023-10-06T03:59:51.797 INFO:tasks.workunit.client.0.smithi157.stdout: create 195/39 file after the snapshot 2023-10-06T03:59:51.798 INFO:tasks.workunit.client.0.smithi157.stdout: create 196/.snap/snap-subdir-test 2023-10-06T03:59:51.823 INFO:tasks.workunit.client.0.smithi157.stdout: create 196/30 file after the snapshot 2023-10-06T03:59:51.824 INFO:tasks.workunit.client.0.smithi157.stdout: create 196/31 file after the snapshot 2023-10-06T03:59:51.825 INFO:tasks.workunit.client.0.smithi157.stdout: create 196/32 file after the snapshot 2023-10-06T03:59:51.826 INFO:tasks.workunit.client.0.smithi157.stdout: create 196/33 file after the snapshot 2023-10-06T03:59:51.826 INFO:tasks.workunit.client.0.smithi157.stdout: create 196/34 file after the snapshot 2023-10-06T03:59:51.827 INFO:tasks.workunit.client.0.smithi157.stdout: create 196/35 file after the snapshot 2023-10-06T03:59:51.828 INFO:tasks.workunit.client.0.smithi157.stdout: create 196/36 file after the snapshot 2023-10-06T03:59:51.829 INFO:tasks.workunit.client.0.smithi157.stdout: create 196/37 file after the snapshot 2023-10-06T03:59:51.829 INFO:tasks.workunit.client.0.smithi157.stdout: create 196/38 file after the snapshot 2023-10-06T03:59:51.830 INFO:tasks.workunit.client.0.smithi157.stdout: create 196/39 file after the snapshot 2023-10-06T03:59:51.831 INFO:tasks.workunit.client.0.smithi157.stdout: create 197/.snap/snap-subdir-test 2023-10-06T03:59:51.857 INFO:tasks.workunit.client.0.smithi157.stdout: create 197/30 file after the snapshot 2023-10-06T03:59:51.858 INFO:tasks.workunit.client.0.smithi157.stdout: create 197/31 file after the snapshot 2023-10-06T03:59:51.858 INFO:tasks.workunit.client.0.smithi157.stdout: create 197/32 file after the snapshot 2023-10-06T03:59:51.859 INFO:tasks.workunit.client.0.smithi157.stdout: create 197/33 file after the snapshot 2023-10-06T03:59:51.860 INFO:tasks.workunit.client.0.smithi157.stdout: create 197/34 file after the snapshot 2023-10-06T03:59:51.860 INFO:tasks.workunit.client.0.smithi157.stdout: create 197/35 file after the snapshot 2023-10-06T03:59:51.861 INFO:tasks.workunit.client.0.smithi157.stdout: create 197/36 file after the snapshot 2023-10-06T03:59:51.862 INFO:tasks.workunit.client.0.smithi157.stdout: create 197/37 file after the snapshot 2023-10-06T03:59:51.862 INFO:tasks.workunit.client.0.smithi157.stdout: create 197/38 file after the snapshot 2023-10-06T03:59:51.863 INFO:tasks.workunit.client.0.smithi157.stdout: create 197/39 file after the snapshot 2023-10-06T03:59:51.864 INFO:tasks.workunit.client.0.smithi157.stdout: create 198/.snap/snap-subdir-test 2023-10-06T03:59:52.099 INFO:tasks.workunit.client.0.smithi157.stdout: create 198/30 file after the snapshot 2023-10-06T03:59:52.100 INFO:tasks.workunit.client.0.smithi157.stdout: create 198/31 file after the snapshot 2023-10-06T03:59:52.100 INFO:tasks.workunit.client.0.smithi157.stdout: create 198/32 file after the snapshot 2023-10-06T03:59:52.101 INFO:tasks.workunit.client.0.smithi157.stdout: create 198/33 file after the snapshot 2023-10-06T03:59:52.102 INFO:tasks.workunit.client.0.smithi157.stdout: create 198/34 file after the snapshot 2023-10-06T03:59:52.103 INFO:tasks.workunit.client.0.smithi157.stdout: create 198/35 file after the snapshot 2023-10-06T03:59:52.103 INFO:tasks.workunit.client.0.smithi157.stdout: create 198/36 file after the snapshot 2023-10-06T03:59:52.104 INFO:tasks.workunit.client.0.smithi157.stdout: create 198/37 file after the snapshot 2023-10-06T03:59:52.105 INFO:tasks.workunit.client.0.smithi157.stdout: create 198/38 file after the snapshot 2023-10-06T03:59:52.105 INFO:tasks.workunit.client.0.smithi157.stdout: create 198/39 file after the snapshot 2023-10-06T03:59:52.106 INFO:tasks.workunit.client.0.smithi157.stdout: create 199/.snap/snap-subdir-test 2023-10-06T03:59:52.132 INFO:tasks.workunit.client.0.smithi157.stdout: create 199/30 file after the snapshot 2023-10-06T03:59:52.132 INFO:tasks.workunit.client.0.smithi157.stdout: create 199/31 file after the snapshot 2023-10-06T03:59:52.133 INFO:tasks.workunit.client.0.smithi157.stdout: create 199/32 file after the snapshot 2023-10-06T03:59:52.134 INFO:tasks.workunit.client.0.smithi157.stdout: create 199/33 file after the snapshot 2023-10-06T03:59:52.134 INFO:tasks.workunit.client.0.smithi157.stdout: create 199/34 file after the snapshot 2023-10-06T03:59:52.135 INFO:tasks.workunit.client.0.smithi157.stdout: create 199/35 file after the snapshot 2023-10-06T03:59:52.136 INFO:tasks.workunit.client.0.smithi157.stdout: create 199/36 file after the snapshot 2023-10-06T03:59:52.136 INFO:tasks.workunit.client.0.smithi157.stdout: create 199/37 file after the snapshot 2023-10-06T03:59:52.137 INFO:tasks.workunit.client.0.smithi157.stdout: create 199/38 file after the snapshot 2023-10-06T03:59:52.138 INFO:tasks.workunit.client.0.smithi157.stdout: create 199/39 file after the snapshot 2023-10-06T03:59:52.138 INFO:tasks.workunit.client.0.smithi157.stdout: create 200/.snap/snap-subdir-test 2023-10-06T03:59:52.164 INFO:tasks.workunit.client.0.smithi157.stdout: create 200/30 file after the snapshot 2023-10-06T03:59:52.165 INFO:tasks.workunit.client.0.smithi157.stdout: create 200/31 file after the snapshot 2023-10-06T03:59:52.165 INFO:tasks.workunit.client.0.smithi157.stdout: create 200/32 file after the snapshot 2023-10-06T03:59:52.166 INFO:tasks.workunit.client.0.smithi157.stdout: create 200/33 file after the snapshot 2023-10-06T03:59:52.167 INFO:tasks.workunit.client.0.smithi157.stdout: create 200/34 file after the snapshot 2023-10-06T03:59:52.167 INFO:tasks.workunit.client.0.smithi157.stdout: create 200/35 file after the snapshot 2023-10-06T03:59:52.168 INFO:tasks.workunit.client.0.smithi157.stdout: create 200/36 file after the snapshot 2023-10-06T03:59:52.168 INFO:tasks.workunit.client.0.smithi157.stdout: create 200/37 file after the snapshot 2023-10-06T03:59:52.169 INFO:tasks.workunit.client.0.smithi157.stdout: create 200/38 file after the snapshot 2023-10-06T03:59:52.170 INFO:tasks.workunit.client.0.smithi157.stdout: create 200/39 file after the snapshot 2023-10-06T03:59:52.170 INFO:tasks.workunit.client.0.smithi157.stdout: create 201/.snap/snap-subdir-test 2023-10-06T03:59:52.197 INFO:tasks.workunit.client.0.smithi157.stdout: create 201/30 file after the snapshot 2023-10-06T03:59:52.198 INFO:tasks.workunit.client.0.smithi157.stdout: create 201/31 file after the snapshot 2023-10-06T03:59:52.198 INFO:tasks.workunit.client.0.smithi157.stdout: create 201/32 file after the snapshot 2023-10-06T03:59:52.199 INFO:tasks.workunit.client.0.smithi157.stdout: create 201/33 file after the snapshot 2023-10-06T03:59:52.200 INFO:tasks.workunit.client.0.smithi157.stdout: create 201/34 file after the snapshot 2023-10-06T03:59:52.200 INFO:tasks.workunit.client.0.smithi157.stdout: create 201/35 file after the snapshot 2023-10-06T03:59:52.201 INFO:tasks.workunit.client.0.smithi157.stdout: create 201/36 file after the snapshot 2023-10-06T03:59:52.202 INFO:tasks.workunit.client.0.smithi157.stdout: create 201/37 file after the snapshot 2023-10-06T03:59:52.202 INFO:tasks.workunit.client.0.smithi157.stdout: create 201/38 file after the snapshot 2023-10-06T03:59:52.203 INFO:tasks.workunit.client.0.smithi157.stdout: create 201/39 file after the snapshot 2023-10-06T03:59:52.204 INFO:tasks.workunit.client.0.smithi157.stdout: create 202/.snap/snap-subdir-test 2023-10-06T03:59:52.231 INFO:tasks.workunit.client.0.smithi157.stdout: create 202/30 file after the snapshot 2023-10-06T03:59:52.231 INFO:tasks.workunit.client.0.smithi157.stdout: create 202/31 file after the snapshot 2023-10-06T03:59:52.232 INFO:tasks.workunit.client.0.smithi157.stdout: create 202/32 file after the snapshot 2023-10-06T03:59:52.232 INFO:tasks.workunit.client.0.smithi157.stdout: create 202/33 file after the snapshot 2023-10-06T03:59:52.233 INFO:tasks.workunit.client.0.smithi157.stdout: create 202/34 file after the snapshot 2023-10-06T03:59:52.234 INFO:tasks.workunit.client.0.smithi157.stdout: create 202/35 file after the snapshot 2023-10-06T03:59:52.234 INFO:tasks.workunit.client.0.smithi157.stdout: create 202/36 file after the snapshot 2023-10-06T03:59:52.235 INFO:tasks.workunit.client.0.smithi157.stdout: create 202/37 file after the snapshot 2023-10-06T03:59:52.236 INFO:tasks.workunit.client.0.smithi157.stdout: create 202/38 file after the snapshot 2023-10-06T03:59:52.236 INFO:tasks.workunit.client.0.smithi157.stdout: create 202/39 file after the snapshot 2023-10-06T03:59:52.237 INFO:tasks.workunit.client.0.smithi157.stdout: create 203/.snap/snap-subdir-test 2023-10-06T03:59:52.264 INFO:tasks.workunit.client.0.smithi157.stdout: create 203/30 file after the snapshot 2023-10-06T03:59:52.264 INFO:tasks.workunit.client.0.smithi157.stdout: create 203/31 file after the snapshot 2023-10-06T03:59:52.265 INFO:tasks.workunit.client.0.smithi157.stdout: create 203/32 file after the snapshot 2023-10-06T03:59:52.266 INFO:tasks.workunit.client.0.smithi157.stdout: create 203/33 file after the snapshot 2023-10-06T03:59:52.529 INFO:tasks.workunit.client.0.smithi157.stdout: create 203/34 file after the snapshot 2023-10-06T03:59:52.529 INFO:tasks.workunit.client.0.smithi157.stdout: create 203/35 file after the snapshot 2023-10-06T03:59:52.530 INFO:tasks.workunit.client.0.smithi157.stdout: create 203/36 file after the snapshot 2023-10-06T03:59:52.530 INFO:tasks.workunit.client.0.smithi157.stdout: create 203/37 file after the snapshot 2023-10-06T03:59:52.531 INFO:tasks.workunit.client.0.smithi157.stdout: create 203/38 file after the snapshot 2023-10-06T03:59:52.532 INFO:tasks.workunit.client.0.smithi157.stdout: create 203/39 file after the snapshot 2023-10-06T03:59:52.532 INFO:tasks.workunit.client.0.smithi157.stdout: create 204/.snap/snap-subdir-test 2023-10-06T03:59:52.560 INFO:tasks.workunit.client.0.smithi157.stdout: create 204/30 file after the snapshot 2023-10-06T03:59:52.561 INFO:tasks.workunit.client.0.smithi157.stdout: create 204/31 file after the snapshot 2023-10-06T03:59:52.562 INFO:tasks.workunit.client.0.smithi157.stdout: create 204/32 file after the snapshot 2023-10-06T03:59:52.563 INFO:tasks.workunit.client.0.smithi157.stdout: create 204/33 file after the snapshot 2023-10-06T03:59:52.563 INFO:tasks.workunit.client.0.smithi157.stdout: create 204/34 file after the snapshot 2023-10-06T03:59:52.564 INFO:tasks.workunit.client.0.smithi157.stdout: create 204/35 file after the snapshot 2023-10-06T03:59:52.564 INFO:tasks.workunit.client.0.smithi157.stdout: create 204/36 file after the snapshot 2023-10-06T03:59:52.565 INFO:tasks.workunit.client.0.smithi157.stdout: create 204/37 file after the snapshot 2023-10-06T03:59:52.566 INFO:tasks.workunit.client.0.smithi157.stdout: create 204/38 file after the snapshot 2023-10-06T03:59:52.567 INFO:tasks.workunit.client.0.smithi157.stdout: create 204/39 file after the snapshot 2023-10-06T03:59:52.567 INFO:tasks.workunit.client.0.smithi157.stdout: create 205/.snap/snap-subdir-test 2023-10-06T03:59:52.593 INFO:tasks.workunit.client.0.smithi157.stdout: create 205/30 file after the snapshot 2023-10-06T03:59:52.593 INFO:tasks.workunit.client.0.smithi157.stdout: create 205/31 file after the snapshot 2023-10-06T03:59:52.594 INFO:tasks.workunit.client.0.smithi157.stdout: create 205/32 file after the snapshot 2023-10-06T03:59:52.595 INFO:tasks.workunit.client.0.smithi157.stdout: create 205/33 file after the snapshot 2023-10-06T03:59:52.595 INFO:tasks.workunit.client.0.smithi157.stdout: create 205/34 file after the snapshot 2023-10-06T03:59:53.053 INFO:tasks.workunit.client.0.smithi157.stdout: create 205/35 file after the snapshot 2023-10-06T03:59:53.054 INFO:tasks.workunit.client.0.smithi157.stdout: create 205/36 file after the snapshot 2023-10-06T03:59:53.054 INFO:tasks.workunit.client.0.smithi157.stdout: create 205/37 file after the snapshot 2023-10-06T03:59:53.055 INFO:tasks.workunit.client.0.smithi157.stdout: create 205/38 file after the snapshot 2023-10-06T03:59:53.056 INFO:tasks.workunit.client.0.smithi157.stdout: create 205/39 file after the snapshot 2023-10-06T03:59:53.057 INFO:tasks.workunit.client.0.smithi157.stdout: create 206/.snap/snap-subdir-test 2023-10-06T03:59:53.084 INFO:tasks.workunit.client.0.smithi157.stdout: create 206/30 file after the snapshot 2023-10-06T03:59:53.085 INFO:tasks.workunit.client.0.smithi157.stdout: create 206/31 file after the snapshot 2023-10-06T03:59:53.086 INFO:tasks.workunit.client.0.smithi157.stdout: create 206/32 file after the snapshot 2023-10-06T03:59:53.086 INFO:tasks.workunit.client.0.smithi157.stdout: create 206/33 file after the snapshot 2023-10-06T03:59:53.087 INFO:tasks.workunit.client.0.smithi157.stdout: create 206/34 file after the snapshot 2023-10-06T03:59:53.088 INFO:tasks.workunit.client.0.smithi157.stdout: create 206/35 file after the snapshot 2023-10-06T03:59:53.088 INFO:tasks.workunit.client.0.smithi157.stdout: create 206/36 file after the snapshot 2023-10-06T03:59:53.089 INFO:tasks.workunit.client.0.smithi157.stdout: create 206/37 file after the snapshot 2023-10-06T03:59:53.090 INFO:tasks.workunit.client.0.smithi157.stdout: create 206/38 file after the snapshot 2023-10-06T03:59:53.091 INFO:tasks.workunit.client.0.smithi157.stdout: create 206/39 file after the snapshot 2023-10-06T03:59:53.091 INFO:tasks.workunit.client.0.smithi157.stdout: create 207/.snap/snap-subdir-test 2023-10-06T03:59:53.117 INFO:tasks.workunit.client.0.smithi157.stdout: create 207/30 file after the snapshot 2023-10-06T03:59:53.118 INFO:tasks.workunit.client.0.smithi157.stdout: create 207/31 file after the snapshot 2023-10-06T03:59:53.118 INFO:tasks.workunit.client.0.smithi157.stdout: create 207/32 file after the snapshot 2023-10-06T03:59:53.119 INFO:tasks.workunit.client.0.smithi157.stdout: create 207/33 file after the snapshot 2023-10-06T03:59:53.120 INFO:tasks.workunit.client.0.smithi157.stdout: create 207/34 file after the snapshot 2023-10-06T03:59:53.120 INFO:tasks.workunit.client.0.smithi157.stdout: create 207/35 file after the snapshot 2023-10-06T03:59:53.121 INFO:tasks.workunit.client.0.smithi157.stdout: create 207/36 file after the snapshot 2023-10-06T03:59:53.121 INFO:tasks.workunit.client.0.smithi157.stdout: create 207/37 file after the snapshot 2023-10-06T03:59:53.122 INFO:tasks.workunit.client.0.smithi157.stdout: create 207/38 file after the snapshot 2023-10-06T03:59:53.123 INFO:tasks.workunit.client.0.smithi157.stdout: create 207/39 file after the snapshot 2023-10-06T03:59:53.124 INFO:tasks.workunit.client.0.smithi157.stdout: create 208/.snap/snap-subdir-test 2023-10-06T03:59:53.149 INFO:tasks.workunit.client.0.smithi157.stdout: create 208/30 file after the snapshot 2023-10-06T03:59:53.150 INFO:tasks.workunit.client.0.smithi157.stdout: create 208/31 file after the snapshot 2023-10-06T03:59:53.151 INFO:tasks.workunit.client.0.smithi157.stdout: create 208/32 file after the snapshot 2023-10-06T03:59:53.152 INFO:tasks.workunit.client.0.smithi157.stdout: create 208/33 file after the snapshot 2023-10-06T03:59:53.152 INFO:tasks.workunit.client.0.smithi157.stdout: create 208/34 file after the snapshot 2023-10-06T03:59:53.153 INFO:tasks.workunit.client.0.smithi157.stdout: create 208/35 file after the snapshot 2023-10-06T03:59:53.154 INFO:tasks.workunit.client.0.smithi157.stdout: create 208/36 file after the snapshot 2023-10-06T03:59:53.154 INFO:tasks.workunit.client.0.smithi157.stdout: create 208/37 file after the snapshot 2023-10-06T03:59:53.155 INFO:tasks.workunit.client.0.smithi157.stdout: create 208/38 file after the snapshot 2023-10-06T03:59:53.156 INFO:tasks.workunit.client.0.smithi157.stdout: create 208/39 file after the snapshot 2023-10-06T03:59:53.156 INFO:tasks.workunit.client.0.smithi157.stdout: create 209/.snap/snap-subdir-test 2023-10-06T03:59:53.182 INFO:tasks.workunit.client.0.smithi157.stdout: create 209/30 file after the snapshot 2023-10-06T03:59:53.183 INFO:tasks.workunit.client.0.smithi157.stdout: create 209/31 file after the snapshot 2023-10-06T03:59:53.183 INFO:tasks.workunit.client.0.smithi157.stdout: create 209/32 file after the snapshot 2023-10-06T03:59:53.184 INFO:tasks.workunit.client.0.smithi157.stdout: create 209/33 file after the snapshot 2023-10-06T03:59:53.185 INFO:tasks.workunit.client.0.smithi157.stdout: create 209/34 file after the snapshot 2023-10-06T03:59:53.185 INFO:tasks.workunit.client.0.smithi157.stdout: create 209/35 file after the snapshot 2023-10-06T03:59:53.186 INFO:tasks.workunit.client.0.smithi157.stdout: create 209/36 file after the snapshot 2023-10-06T03:59:53.187 INFO:tasks.workunit.client.0.smithi157.stdout: create 209/37 file after the snapshot 2023-10-06T03:59:53.187 INFO:tasks.workunit.client.0.smithi157.stdout: create 209/38 file after the snapshot 2023-10-06T03:59:53.188 INFO:tasks.workunit.client.0.smithi157.stdout: create 209/39 file after the snapshot 2023-10-06T03:59:53.189 INFO:tasks.workunit.client.0.smithi157.stdout: create 210/.snap/snap-subdir-test 2023-10-06T03:59:53.214 INFO:tasks.workunit.client.0.smithi157.stdout: create 210/30 file after the snapshot 2023-10-06T03:59:53.215 INFO:tasks.workunit.client.0.smithi157.stdout: create 210/31 file after the snapshot 2023-10-06T03:59:53.216 INFO:tasks.workunit.client.0.smithi157.stdout: create 210/32 file after the snapshot 2023-10-06T03:59:53.216 INFO:tasks.workunit.client.0.smithi157.stdout: create 210/33 file after the snapshot 2023-10-06T03:59:53.217 INFO:tasks.workunit.client.0.smithi157.stdout: create 210/34 file after the snapshot 2023-10-06T03:59:53.218 INFO:tasks.workunit.client.0.smithi157.stdout: create 210/35 file after the snapshot 2023-10-06T03:59:53.218 INFO:tasks.workunit.client.0.smithi157.stdout: create 210/36 file after the snapshot 2023-10-06T03:59:53.219 INFO:tasks.workunit.client.0.smithi157.stdout: create 210/37 file after the snapshot 2023-10-06T03:59:53.220 INFO:tasks.workunit.client.0.smithi157.stdout: create 210/38 file after the snapshot 2023-10-06T03:59:53.220 INFO:tasks.workunit.client.0.smithi157.stdout: create 210/39 file after the snapshot 2023-10-06T03:59:53.221 INFO:tasks.workunit.client.0.smithi157.stdout: create 211/.snap/snap-subdir-test 2023-10-06T03:59:53.247 INFO:tasks.workunit.client.0.smithi157.stdout: create 211/30 file after the snapshot 2023-10-06T03:59:53.248 INFO:tasks.workunit.client.0.smithi157.stdout: create 211/31 file after the snapshot 2023-10-06T03:59:53.249 INFO:tasks.workunit.client.0.smithi157.stdout: create 211/32 file after the snapshot 2023-10-06T03:59:53.249 INFO:tasks.workunit.client.0.smithi157.stdout: create 211/33 file after the snapshot 2023-10-06T03:59:53.250 INFO:tasks.workunit.client.0.smithi157.stdout: create 211/34 file after the snapshot 2023-10-06T03:59:53.251 INFO:tasks.workunit.client.0.smithi157.stdout: create 211/35 file after the snapshot 2023-10-06T03:59:53.251 INFO:tasks.workunit.client.0.smithi157.stdout: create 211/36 file after the snapshot 2023-10-06T03:59:53.252 INFO:tasks.workunit.client.0.smithi157.stdout: create 211/37 file after the snapshot 2023-10-06T03:59:53.252 INFO:tasks.workunit.client.0.smithi157.stdout: create 211/38 file after the snapshot 2023-10-06T03:59:53.253 INFO:tasks.workunit.client.0.smithi157.stdout: create 211/39 file after the snapshot 2023-10-06T03:59:53.254 INFO:tasks.workunit.client.0.smithi157.stdout: create 212/.snap/snap-subdir-test 2023-10-06T03:59:53.280 INFO:tasks.workunit.client.0.smithi157.stdout: create 212/30 file after the snapshot 2023-10-06T03:59:53.281 INFO:tasks.workunit.client.0.smithi157.stdout: create 212/31 file after the snapshot 2023-10-06T03:59:53.282 INFO:tasks.workunit.client.0.smithi157.stdout: create 212/32 file after the snapshot 2023-10-06T03:59:53.283 INFO:tasks.workunit.client.0.smithi157.stdout: create 212/33 file after the snapshot 2023-10-06T03:59:53.283 INFO:tasks.workunit.client.0.smithi157.stdout: create 212/34 file after the snapshot 2023-10-06T03:59:53.284 INFO:tasks.workunit.client.0.smithi157.stdout: create 212/35 file after the snapshot 2023-10-06T03:59:53.285 INFO:tasks.workunit.client.0.smithi157.stdout: create 212/36 file after the snapshot 2023-10-06T03:59:53.285 INFO:tasks.workunit.client.0.smithi157.stdout: create 212/37 file after the snapshot 2023-10-06T03:59:53.286 INFO:tasks.workunit.client.0.smithi157.stdout: create 212/38 file after the snapshot 2023-10-06T03:59:53.287 INFO:tasks.workunit.client.0.smithi157.stdout: create 212/39 file after the snapshot 2023-10-06T03:59:53.287 INFO:tasks.workunit.client.0.smithi157.stdout: create 213/.snap/snap-subdir-test 2023-10-06T03:59:53.313 INFO:tasks.workunit.client.0.smithi157.stdout: create 213/30 file after the snapshot 2023-10-06T03:59:53.314 INFO:tasks.workunit.client.0.smithi157.stdout: create 213/31 file after the snapshot 2023-10-06T03:59:53.315 INFO:tasks.workunit.client.0.smithi157.stdout: create 213/32 file after the snapshot 2023-10-06T03:59:53.315 INFO:tasks.workunit.client.0.smithi157.stdout: create 213/33 file after the snapshot 2023-10-06T03:59:53.316 INFO:tasks.workunit.client.0.smithi157.stdout: create 213/34 file after the snapshot 2023-10-06T03:59:53.317 INFO:tasks.workunit.client.0.smithi157.stdout: create 213/35 file after the snapshot 2023-10-06T03:59:53.317 INFO:tasks.workunit.client.0.smithi157.stdout: create 213/36 file after the snapshot 2023-10-06T03:59:53.318 INFO:tasks.workunit.client.0.smithi157.stdout: create 213/37 file after the snapshot 2023-10-06T03:59:53.318 INFO:tasks.workunit.client.0.smithi157.stdout: create 213/38 file after the snapshot 2023-10-06T03:59:53.319 INFO:tasks.workunit.client.0.smithi157.stdout: create 213/39 file after the snapshot 2023-10-06T03:59:53.320 INFO:tasks.workunit.client.0.smithi157.stdout: create 214/.snap/snap-subdir-test 2023-10-06T03:59:53.344 INFO:tasks.workunit.client.0.smithi157.stdout: create 214/30 file after the snapshot 2023-10-06T03:59:53.345 INFO:tasks.workunit.client.0.smithi157.stdout: create 214/31 file after the snapshot 2023-10-06T03:59:53.346 INFO:tasks.workunit.client.0.smithi157.stdout: create 214/32 file after the snapshot 2023-10-06T03:59:53.347 INFO:tasks.workunit.client.0.smithi157.stdout: create 214/33 file after the snapshot 2023-10-06T03:59:53.347 INFO:tasks.workunit.client.0.smithi157.stdout: create 214/34 file after the snapshot 2023-10-06T03:59:53.348 INFO:tasks.workunit.client.0.smithi157.stdout: create 214/35 file after the snapshot 2023-10-06T03:59:53.349 INFO:tasks.workunit.client.0.smithi157.stdout: create 214/36 file after the snapshot 2023-10-06T03:59:53.349 INFO:tasks.workunit.client.0.smithi157.stdout: create 214/37 file after the snapshot 2023-10-06T03:59:53.350 INFO:tasks.workunit.client.0.smithi157.stdout: create 214/38 file after the snapshot 2023-10-06T03:59:53.351 INFO:tasks.workunit.client.0.smithi157.stdout: create 214/39 file after the snapshot 2023-10-06T03:59:53.351 INFO:tasks.workunit.client.0.smithi157.stdout: create 215/.snap/snap-subdir-test 2023-10-06T03:59:53.376 INFO:tasks.workunit.client.0.smithi157.stdout: create 215/30 file after the snapshot 2023-10-06T03:59:53.378 INFO:tasks.workunit.client.0.smithi157.stdout: create 215/31 file after the snapshot 2023-10-06T03:59:53.379 INFO:tasks.workunit.client.0.smithi157.stdout: create 215/32 file after the snapshot 2023-10-06T03:59:53.380 INFO:tasks.workunit.client.0.smithi157.stdout: create 215/33 file after the snapshot 2023-10-06T03:59:53.381 INFO:tasks.workunit.client.0.smithi157.stdout: create 215/34 file after the snapshot 2023-10-06T03:59:53.381 INFO:tasks.workunit.client.0.smithi157.stdout: create 215/35 file after the snapshot 2023-10-06T03:59:53.382 INFO:tasks.workunit.client.0.smithi157.stdout: create 215/36 file after the snapshot 2023-10-06T03:59:53.383 INFO:tasks.workunit.client.0.smithi157.stdout: create 215/37 file after the snapshot 2023-10-06T03:59:53.384 INFO:tasks.workunit.client.0.smithi157.stdout: create 215/38 file after the snapshot 2023-10-06T03:59:53.385 INFO:tasks.workunit.client.0.smithi157.stdout: create 215/39 file after the snapshot 2023-10-06T03:59:53.386 INFO:tasks.workunit.client.0.smithi157.stdout: create 216/.snap/snap-subdir-test 2023-10-06T03:59:53.412 INFO:tasks.workunit.client.0.smithi157.stdout: create 216/30 file after the snapshot 2023-10-06T03:59:53.413 INFO:tasks.workunit.client.0.smithi157.stdout: create 216/31 file after the snapshot 2023-10-06T03:59:53.414 INFO:tasks.workunit.client.0.smithi157.stdout: create 216/32 file after the snapshot 2023-10-06T03:59:53.414 INFO:tasks.workunit.client.0.smithi157.stdout: create 216/33 file after the snapshot 2023-10-06T03:59:53.415 INFO:tasks.workunit.client.0.smithi157.stdout: create 216/34 file after the snapshot 2023-10-06T03:59:53.416 INFO:tasks.workunit.client.0.smithi157.stdout: create 216/35 file after the snapshot 2023-10-06T03:59:53.416 INFO:tasks.workunit.client.0.smithi157.stdout: create 216/36 file after the snapshot 2023-10-06T03:59:53.417 INFO:tasks.workunit.client.0.smithi157.stdout: create 216/37 file after the snapshot 2023-10-06T03:59:53.418 INFO:tasks.workunit.client.0.smithi157.stdout: create 216/38 file after the snapshot 2023-10-06T03:59:53.418 INFO:tasks.workunit.client.0.smithi157.stdout: create 216/39 file after the snapshot 2023-10-06T03:59:53.419 INFO:tasks.workunit.client.0.smithi157.stdout: create 217/.snap/snap-subdir-test 2023-10-06T03:59:53.445 INFO:tasks.workunit.client.0.smithi157.stdout: create 217/30 file after the snapshot 2023-10-06T03:59:53.446 INFO:tasks.workunit.client.0.smithi157.stdout: create 217/31 file after the snapshot 2023-10-06T03:59:53.447 INFO:tasks.workunit.client.0.smithi157.stdout: create 217/32 file after the snapshot 2023-10-06T03:59:53.737 INFO:tasks.workunit.client.0.smithi157.stdout: create 217/33 file after the snapshot 2023-10-06T03:59:53.737 INFO:tasks.workunit.client.0.smithi157.stdout: create 217/34 file after the snapshot 2023-10-06T03:59:53.738 INFO:tasks.workunit.client.0.smithi157.stdout: create 217/35 file after the snapshot 2023-10-06T03:59:53.739 INFO:tasks.workunit.client.0.smithi157.stdout: create 217/36 file after the snapshot 2023-10-06T03:59:53.739 INFO:tasks.workunit.client.0.smithi157.stdout: create 217/37 file after the snapshot 2023-10-06T03:59:53.740 INFO:tasks.workunit.client.0.smithi157.stdout: create 217/38 file after the snapshot 2023-10-06T03:59:53.741 INFO:tasks.workunit.client.0.smithi157.stdout: create 217/39 file after the snapshot 2023-10-06T03:59:53.741 INFO:tasks.workunit.client.0.smithi157.stdout: create 218/.snap/snap-subdir-test 2023-10-06T03:59:53.768 INFO:tasks.workunit.client.0.smithi157.stdout: create 218/30 file after the snapshot 2023-10-06T03:59:53.769 INFO:tasks.workunit.client.0.smithi157.stdout: create 218/31 file after the snapshot 2023-10-06T03:59:53.769 INFO:tasks.workunit.client.0.smithi157.stdout: create 218/32 file after the snapshot 2023-10-06T03:59:53.770 INFO:tasks.workunit.client.0.smithi157.stdout: create 218/33 file after the snapshot 2023-10-06T03:59:53.771 INFO:tasks.workunit.client.0.smithi157.stdout: create 218/34 file after the snapshot 2023-10-06T03:59:53.771 INFO:tasks.workunit.client.0.smithi157.stdout: create 218/35 file after the snapshot 2023-10-06T03:59:53.772 INFO:tasks.workunit.client.0.smithi157.stdout: create 218/36 file after the snapshot 2023-10-06T03:59:53.773 INFO:tasks.workunit.client.0.smithi157.stdout: create 218/37 file after the snapshot 2023-10-06T03:59:53.773 INFO:tasks.workunit.client.0.smithi157.stdout: create 218/38 file after the snapshot 2023-10-06T03:59:53.774 INFO:tasks.workunit.client.0.smithi157.stdout: create 218/39 file after the snapshot 2023-10-06T03:59:53.775 INFO:tasks.workunit.client.0.smithi157.stdout: create 219/.snap/snap-subdir-test 2023-10-06T03:59:53.802 INFO:tasks.workunit.client.0.smithi157.stdout: create 219/30 file after the snapshot 2023-10-06T03:59:53.803 INFO:tasks.workunit.client.0.smithi157.stdout: create 219/31 file after the snapshot 2023-10-06T03:59:53.803 INFO:tasks.workunit.client.0.smithi157.stdout: create 219/32 file after the snapshot 2023-10-06T03:59:53.804 INFO:tasks.workunit.client.0.smithi157.stdout: create 219/33 file after the snapshot 2023-10-06T03:59:53.804 INFO:tasks.workunit.client.0.smithi157.stdout: create 219/34 file after the snapshot 2023-10-06T03:59:53.805 INFO:tasks.workunit.client.0.smithi157.stdout: create 219/35 file after the snapshot 2023-10-06T03:59:53.806 INFO:tasks.workunit.client.0.smithi157.stdout: create 219/36 file after the snapshot 2023-10-06T03:59:53.806 INFO:tasks.workunit.client.0.smithi157.stdout: create 219/37 file after the snapshot 2023-10-06T03:59:53.807 INFO:tasks.workunit.client.0.smithi157.stdout: create 219/38 file after the snapshot 2023-10-06T03:59:53.807 INFO:tasks.workunit.client.0.smithi157.stdout: create 219/39 file after the snapshot 2023-10-06T03:59:53.808 INFO:tasks.workunit.client.0.smithi157.stdout: create 220/.snap/snap-subdir-test 2023-10-06T03:59:53.834 INFO:tasks.workunit.client.0.smithi157.stdout: create 220/30 file after the snapshot 2023-10-06T03:59:53.834 INFO:tasks.workunit.client.0.smithi157.stdout: create 220/31 file after the snapshot 2023-10-06T03:59:53.835 INFO:tasks.workunit.client.0.smithi157.stdout: create 220/32 file after the snapshot 2023-10-06T03:59:53.836 INFO:tasks.workunit.client.0.smithi157.stdout: create 220/33 file after the snapshot 2023-10-06T03:59:53.836 INFO:tasks.workunit.client.0.smithi157.stdout: create 220/34 file after the snapshot 2023-10-06T03:59:53.837 INFO:tasks.workunit.client.0.smithi157.stdout: create 220/35 file after the snapshot 2023-10-06T03:59:53.838 INFO:tasks.workunit.client.0.smithi157.stdout: create 220/36 file after the snapshot 2023-10-06T03:59:53.838 INFO:tasks.workunit.client.0.smithi157.stdout: create 220/37 file after the snapshot 2023-10-06T03:59:53.839 INFO:tasks.workunit.client.0.smithi157.stdout: create 220/38 file after the snapshot 2023-10-06T03:59:53.840 INFO:tasks.workunit.client.0.smithi157.stdout: create 220/39 file after the snapshot 2023-10-06T03:59:53.840 INFO:tasks.workunit.client.0.smithi157.stdout: create 221/.snap/snap-subdir-test 2023-10-06T03:59:53.866 INFO:tasks.workunit.client.0.smithi157.stdout: create 221/30 file after the snapshot 2023-10-06T03:59:53.867 INFO:tasks.workunit.client.0.smithi157.stdout: create 221/31 file after the snapshot 2023-10-06T03:59:53.868 INFO:tasks.workunit.client.0.smithi157.stdout: create 221/32 file after the snapshot 2023-10-06T03:59:53.869 INFO:tasks.workunit.client.0.smithi157.stdout: create 221/33 file after the snapshot 2023-10-06T03:59:53.870 INFO:tasks.workunit.client.0.smithi157.stdout: create 221/34 file after the snapshot 2023-10-06T03:59:53.871 INFO:tasks.workunit.client.0.smithi157.stdout: create 221/35 file after the snapshot 2023-10-06T03:59:53.872 INFO:tasks.workunit.client.0.smithi157.stdout: create 221/36 file after the snapshot 2023-10-06T03:59:53.873 INFO:tasks.workunit.client.0.smithi157.stdout: create 221/37 file after the snapshot 2023-10-06T03:59:54.169 INFO:tasks.workunit.client.0.smithi157.stdout: create 221/38 file after the snapshot 2023-10-06T03:59:54.847 INFO:tasks.workunit.client.0.smithi157.stdout: create 221/39 file after the snapshot 2023-10-06T03:59:54.847 INFO:tasks.workunit.client.0.smithi157.stdout: create 222/.snap/snap-subdir-test 2023-10-06T03:59:54.873 INFO:tasks.workunit.client.0.smithi157.stdout: create 222/30 file after the snapshot 2023-10-06T03:59:54.875 INFO:tasks.workunit.client.0.smithi157.stdout: create 222/31 file after the snapshot 2023-10-06T03:59:54.876 INFO:tasks.workunit.client.0.smithi157.stdout: create 222/32 file after the snapshot 2023-10-06T03:59:54.877 INFO:tasks.workunit.client.0.smithi157.stdout: create 222/33 file after the snapshot 2023-10-06T03:59:54.878 INFO:tasks.workunit.client.0.smithi157.stdout: create 222/34 file after the snapshot 2023-10-06T03:59:54.879 INFO:tasks.workunit.client.0.smithi157.stdout: create 222/35 file after the snapshot 2023-10-06T03:59:54.880 INFO:tasks.workunit.client.0.smithi157.stdout: create 222/36 file after the snapshot 2023-10-06T03:59:54.881 INFO:tasks.workunit.client.0.smithi157.stdout: create 222/37 file after the snapshot 2023-10-06T03:59:54.881 INFO:tasks.workunit.client.0.smithi157.stdout: create 222/38 file after the snapshot 2023-10-06T03:59:54.883 INFO:tasks.workunit.client.0.smithi157.stdout: create 222/39 file after the snapshot 2023-10-06T03:59:54.884 INFO:tasks.workunit.client.0.smithi157.stdout: create 223/.snap/snap-subdir-test 2023-10-06T03:59:54.909 INFO:tasks.workunit.client.0.smithi157.stdout: create 223/30 file after the snapshot 2023-10-06T03:59:54.910 INFO:tasks.workunit.client.0.smithi157.stdout: create 223/31 file after the snapshot 2023-10-06T03:59:54.911 INFO:tasks.workunit.client.0.smithi157.stdout: create 223/32 file after the snapshot 2023-10-06T03:59:54.912 INFO:tasks.workunit.client.0.smithi157.stdout: create 223/33 file after the snapshot 2023-10-06T03:59:54.913 INFO:tasks.workunit.client.0.smithi157.stdout: create 223/34 file after the snapshot 2023-10-06T03:59:54.914 INFO:tasks.workunit.client.0.smithi157.stdout: create 223/35 file after the snapshot 2023-10-06T03:59:54.915 INFO:tasks.workunit.client.0.smithi157.stdout: create 223/36 file after the snapshot 2023-10-06T03:59:54.916 INFO:tasks.workunit.client.0.smithi157.stdout: create 223/37 file after the snapshot 2023-10-06T03:59:54.917 INFO:tasks.workunit.client.0.smithi157.stdout: create 223/38 file after the snapshot 2023-10-06T03:59:54.918 INFO:tasks.workunit.client.0.smithi157.stdout: create 223/39 file after the snapshot 2023-10-06T03:59:54.919 INFO:tasks.workunit.client.0.smithi157.stdout: create 224/.snap/snap-subdir-test 2023-10-06T03:59:54.944 INFO:tasks.workunit.client.0.smithi157.stdout: create 224/30 file after the snapshot 2023-10-06T03:59:54.945 INFO:tasks.workunit.client.0.smithi157.stdout: create 224/31 file after the snapshot 2023-10-06T03:59:54.946 INFO:tasks.workunit.client.0.smithi157.stdout: create 224/32 file after the snapshot 2023-10-06T03:59:54.946 INFO:tasks.workunit.client.0.smithi157.stdout: create 224/33 file after the snapshot 2023-10-06T03:59:54.947 INFO:tasks.workunit.client.0.smithi157.stdout: create 224/34 file after the snapshot 2023-10-06T03:59:54.948 INFO:tasks.workunit.client.0.smithi157.stdout: create 224/35 file after the snapshot 2023-10-06T03:59:54.948 INFO:tasks.workunit.client.0.smithi157.stdout: create 224/36 file after the snapshot 2023-10-06T03:59:54.949 INFO:tasks.workunit.client.0.smithi157.stdout: create 224/37 file after the snapshot 2023-10-06T03:59:54.950 INFO:tasks.workunit.client.0.smithi157.stdout: create 224/38 file after the snapshot 2023-10-06T03:59:54.950 INFO:tasks.workunit.client.0.smithi157.stdout: create 224/39 file after the snapshot 2023-10-06T03:59:54.951 INFO:tasks.workunit.client.0.smithi157.stdout: create 225/.snap/snap-subdir-test 2023-10-06T03:59:54.975 INFO:tasks.workunit.client.0.smithi157.stdout: create 225/30 file after the snapshot 2023-10-06T03:59:54.976 INFO:tasks.workunit.client.0.smithi157.stdout: create 225/31 file after the snapshot 2023-10-06T03:59:54.977 INFO:tasks.workunit.client.0.smithi157.stdout: create 225/32 file after the snapshot 2023-10-06T03:59:54.978 INFO:tasks.workunit.client.0.smithi157.stdout: create 225/33 file after the snapshot 2023-10-06T03:59:54.978 INFO:tasks.workunit.client.0.smithi157.stdout: create 225/34 file after the snapshot 2023-10-06T03:59:54.979 INFO:tasks.workunit.client.0.smithi157.stdout: create 225/35 file after the snapshot 2023-10-06T03:59:54.980 INFO:tasks.workunit.client.0.smithi157.stdout: create 225/36 file after the snapshot 2023-10-06T03:59:54.980 INFO:tasks.workunit.client.0.smithi157.stdout: create 225/37 file after the snapshot 2023-10-06T03:59:54.981 INFO:tasks.workunit.client.0.smithi157.stdout: create 225/38 file after the snapshot 2023-10-06T03:59:54.981 INFO:tasks.workunit.client.0.smithi157.stdout: create 225/39 file after the snapshot 2023-10-06T03:59:54.982 INFO:tasks.workunit.client.0.smithi157.stdout: create 226/.snap/snap-subdir-test 2023-10-06T03:59:55.018 INFO:tasks.workunit.client.0.smithi157.stdout: create 226/30 file after the snapshot 2023-10-06T03:59:55.019 INFO:tasks.workunit.client.0.smithi157.stdout: create 226/31 file after the snapshot 2023-10-06T03:59:55.020 INFO:tasks.workunit.client.0.smithi157.stdout: create 226/32 file after the snapshot 2023-10-06T03:59:55.021 INFO:tasks.workunit.client.0.smithi157.stdout: create 226/33 file after the snapshot 2023-10-06T03:59:55.022 INFO:tasks.workunit.client.0.smithi157.stdout: create 226/34 file after the snapshot 2023-10-06T03:59:55.023 INFO:tasks.workunit.client.0.smithi157.stdout: create 226/35 file after the snapshot 2023-10-06T03:59:55.024 INFO:tasks.workunit.client.0.smithi157.stdout: create 226/36 file after the snapshot 2023-10-06T03:59:55.025 INFO:tasks.workunit.client.0.smithi157.stdout: create 226/37 file after the snapshot 2023-10-06T03:59:55.026 INFO:tasks.workunit.client.0.smithi157.stdout: create 226/38 file after the snapshot 2023-10-06T03:59:55.027 INFO:tasks.workunit.client.0.smithi157.stdout: create 226/39 file after the snapshot 2023-10-06T03:59:55.028 INFO:tasks.workunit.client.0.smithi157.stdout: create 227/.snap/snap-subdir-test 2023-10-06T03:59:55.060 INFO:tasks.workunit.client.0.smithi157.stdout: create 227/30 file after the snapshot 2023-10-06T03:59:55.060 INFO:tasks.workunit.client.0.smithi157.stdout: create 227/31 file after the snapshot 2023-10-06T03:59:55.061 INFO:tasks.workunit.client.0.smithi157.stdout: create 227/32 file after the snapshot 2023-10-06T03:59:55.062 INFO:tasks.workunit.client.0.smithi157.stdout: create 227/33 file after the snapshot 2023-10-06T03:59:55.062 INFO:tasks.workunit.client.0.smithi157.stdout: create 227/34 file after the snapshot 2023-10-06T03:59:55.063 INFO:tasks.workunit.client.0.smithi157.stdout: create 227/35 file after the snapshot 2023-10-06T03:59:55.064 INFO:tasks.workunit.client.0.smithi157.stdout: create 227/36 file after the snapshot 2023-10-06T03:59:55.064 INFO:tasks.workunit.client.0.smithi157.stdout: create 227/37 file after the snapshot 2023-10-06T03:59:55.065 INFO:tasks.workunit.client.0.smithi157.stdout: create 227/38 file after the snapshot 2023-10-06T03:59:55.066 INFO:tasks.workunit.client.0.smithi157.stdout: create 227/39 file after the snapshot 2023-10-06T03:59:55.067 INFO:tasks.workunit.client.0.smithi157.stdout: create 228/.snap/snap-subdir-test 2023-10-06T03:59:55.098 INFO:tasks.workunit.client.0.smithi157.stdout: create 228/30 file after the snapshot 2023-10-06T03:59:55.098 INFO:tasks.workunit.client.0.smithi157.stdout: create 228/31 file after the snapshot 2023-10-06T03:59:55.099 INFO:tasks.workunit.client.0.smithi157.stdout: create 228/32 file after the snapshot 2023-10-06T03:59:55.100 INFO:tasks.workunit.client.0.smithi157.stdout: create 228/33 file after the snapshot 2023-10-06T03:59:55.100 INFO:tasks.workunit.client.0.smithi157.stdout: create 228/34 file after the snapshot 2023-10-06T03:59:55.101 INFO:tasks.workunit.client.0.smithi157.stdout: create 228/35 file after the snapshot 2023-10-06T03:59:55.102 INFO:tasks.workunit.client.0.smithi157.stdout: create 228/36 file after the snapshot 2023-10-06T03:59:55.102 INFO:tasks.workunit.client.0.smithi157.stdout: create 228/37 file after the snapshot 2023-10-06T03:59:55.103 INFO:tasks.workunit.client.0.smithi157.stdout: create 228/38 file after the snapshot 2023-10-06T03:59:55.103 INFO:tasks.workunit.client.0.smithi157.stdout: create 228/39 file after the snapshot 2023-10-06T03:59:55.104 INFO:tasks.workunit.client.0.smithi157.stdout: create 229/.snap/snap-subdir-test 2023-10-06T03:59:55.136 INFO:tasks.workunit.client.0.smithi157.stdout: create 229/30 file after the snapshot 2023-10-06T03:59:55.137 INFO:tasks.workunit.client.0.smithi157.stdout: create 229/31 file after the snapshot 2023-10-06T03:59:55.137 INFO:tasks.workunit.client.0.smithi157.stdout: create 229/32 file after the snapshot 2023-10-06T03:59:55.138 INFO:tasks.workunit.client.0.smithi157.stdout: create 229/33 file after the snapshot 2023-10-06T03:59:55.139 INFO:tasks.workunit.client.0.smithi157.stdout: create 229/34 file after the snapshot 2023-10-06T03:59:55.140 INFO:tasks.workunit.client.0.smithi157.stdout: create 229/35 file after the snapshot 2023-10-06T03:59:55.141 INFO:tasks.workunit.client.0.smithi157.stdout: create 229/36 file after the snapshot 2023-10-06T03:59:55.141 INFO:tasks.workunit.client.0.smithi157.stdout: create 229/37 file after the snapshot 2023-10-06T03:59:55.142 INFO:tasks.workunit.client.0.smithi157.stdout: create 229/38 file after the snapshot 2023-10-06T03:59:55.143 INFO:tasks.workunit.client.0.smithi157.stdout: create 229/39 file after the snapshot 2023-10-06T03:59:55.144 INFO:tasks.workunit.client.0.smithi157.stdout: create 230/.snap/snap-subdir-test 2023-10-06T03:59:55.179 INFO:tasks.workunit.client.0.smithi157.stdout: create 230/30 file after the snapshot 2023-10-06T03:59:55.180 INFO:tasks.workunit.client.0.smithi157.stdout: create 230/31 file after the snapshot 2023-10-06T03:59:55.180 INFO:tasks.workunit.client.0.smithi157.stdout: create 230/32 file after the snapshot 2023-10-06T03:59:55.181 INFO:tasks.workunit.client.0.smithi157.stdout: create 230/33 file after the snapshot 2023-10-06T03:59:55.182 INFO:tasks.workunit.client.0.smithi157.stdout: create 230/34 file after the snapshot 2023-10-06T03:59:55.182 INFO:tasks.workunit.client.0.smithi157.stdout: create 230/35 file after the snapshot 2023-10-06T03:59:55.183 INFO:tasks.workunit.client.0.smithi157.stdout: create 230/36 file after the snapshot 2023-10-06T03:59:55.184 INFO:tasks.workunit.client.0.smithi157.stdout: create 230/37 file after the snapshot 2023-10-06T03:59:55.184 INFO:tasks.workunit.client.0.smithi157.stdout: create 230/38 file after the snapshot 2023-10-06T03:59:55.185 INFO:tasks.workunit.client.0.smithi157.stdout: create 230/39 file after the snapshot 2023-10-06T03:59:55.186 INFO:tasks.workunit.client.0.smithi157.stdout: create 231/.snap/snap-subdir-test 2023-10-06T03:59:55.217 INFO:tasks.workunit.client.0.smithi157.stdout: create 231/30 file after the snapshot 2023-10-06T03:59:55.219 INFO:tasks.workunit.client.0.smithi157.stdout: create 231/31 file after the snapshot 2023-10-06T03:59:55.220 INFO:tasks.workunit.client.0.smithi157.stdout: create 231/32 file after the snapshot 2023-10-06T03:59:55.221 INFO:tasks.workunit.client.0.smithi157.stdout: create 231/33 file after the snapshot 2023-10-06T03:59:55.221 INFO:tasks.workunit.client.0.smithi157.stdout: create 231/34 file after the snapshot 2023-10-06T03:59:55.222 INFO:tasks.workunit.client.0.smithi157.stdout: create 231/35 file after the snapshot 2023-10-06T03:59:55.223 INFO:tasks.workunit.client.0.smithi157.stdout: create 231/36 file after the snapshot 2023-10-06T03:59:55.224 INFO:tasks.workunit.client.0.smithi157.stdout: create 231/37 file after the snapshot 2023-10-06T03:59:55.225 INFO:tasks.workunit.client.0.smithi157.stdout: create 231/38 file after the snapshot 2023-10-06T03:59:55.227 INFO:tasks.workunit.client.0.smithi157.stdout: create 231/39 file after the snapshot 2023-10-06T03:59:55.227 INFO:tasks.workunit.client.0.smithi157.stdout: create 232/.snap/snap-subdir-test 2023-10-06T03:59:55.259 INFO:tasks.workunit.client.0.smithi157.stdout: create 232/30 file after the snapshot 2023-10-06T03:59:55.260 INFO:tasks.workunit.client.0.smithi157.stdout: create 232/31 file after the snapshot 2023-10-06T03:59:55.260 INFO:tasks.workunit.client.0.smithi157.stdout: create 232/32 file after the snapshot 2023-10-06T03:59:55.261 INFO:tasks.workunit.client.0.smithi157.stdout: create 232/33 file after the snapshot 2023-10-06T03:59:55.262 INFO:tasks.workunit.client.0.smithi157.stdout: create 232/34 file after the snapshot 2023-10-06T03:59:55.262 INFO:tasks.workunit.client.0.smithi157.stdout: create 232/35 file after the snapshot 2023-10-06T03:59:55.263 INFO:tasks.workunit.client.0.smithi157.stdout: create 232/36 file after the snapshot 2023-10-06T03:59:55.264 INFO:tasks.workunit.client.0.smithi157.stdout: create 232/37 file after the snapshot 2023-10-06T03:59:55.264 INFO:tasks.workunit.client.0.smithi157.stdout: create 232/38 file after the snapshot 2023-10-06T03:59:55.265 INFO:tasks.workunit.client.0.smithi157.stdout: create 232/39 file after the snapshot 2023-10-06T03:59:55.266 INFO:tasks.workunit.client.0.smithi157.stdout: create 233/.snap/snap-subdir-test 2023-10-06T03:59:55.297 INFO:tasks.workunit.client.0.smithi157.stdout: create 233/30 file after the snapshot 2023-10-06T03:59:55.298 INFO:tasks.workunit.client.0.smithi157.stdout: create 233/31 file after the snapshot 2023-10-06T03:59:55.299 INFO:tasks.workunit.client.0.smithi157.stdout: create 233/32 file after the snapshot 2023-10-06T03:59:55.299 INFO:tasks.workunit.client.0.smithi157.stdout: create 233/33 file after the snapshot 2023-10-06T03:59:55.300 INFO:tasks.workunit.client.0.smithi157.stdout: create 233/34 file after the snapshot 2023-10-06T03:59:55.301 INFO:tasks.workunit.client.0.smithi157.stdout: create 233/35 file after the snapshot 2023-10-06T03:59:55.301 INFO:tasks.workunit.client.0.smithi157.stdout: create 233/36 file after the snapshot 2023-10-06T03:59:55.302 INFO:tasks.workunit.client.0.smithi157.stdout: create 233/37 file after the snapshot 2023-10-06T03:59:55.302 INFO:tasks.workunit.client.0.smithi157.stdout: create 233/38 file after the snapshot 2023-10-06T03:59:55.303 INFO:tasks.workunit.client.0.smithi157.stdout: create 233/39 file after the snapshot 2023-10-06T03:59:55.304 INFO:tasks.workunit.client.0.smithi157.stdout: create 234/.snap/snap-subdir-test 2023-10-06T03:59:55.335 INFO:tasks.workunit.client.0.smithi157.stdout: create 234/30 file after the snapshot 2023-10-06T03:59:55.336 INFO:tasks.workunit.client.0.smithi157.stdout: create 234/31 file after the snapshot 2023-10-06T03:59:55.336 INFO:tasks.workunit.client.0.smithi157.stdout: create 234/32 file after the snapshot 2023-10-06T03:59:55.337 INFO:tasks.workunit.client.0.smithi157.stdout: create 234/33 file after the snapshot 2023-10-06T03:59:55.338 INFO:tasks.workunit.client.0.smithi157.stdout: create 234/34 file after the snapshot 2023-10-06T03:59:55.338 INFO:tasks.workunit.client.0.smithi157.stdout: create 234/35 file after the snapshot 2023-10-06T03:59:55.339 INFO:tasks.workunit.client.0.smithi157.stdout: create 234/36 file after the snapshot 2023-10-06T03:59:55.339 INFO:tasks.workunit.client.0.smithi157.stdout: create 234/37 file after the snapshot 2023-10-06T03:59:55.340 INFO:tasks.workunit.client.0.smithi157.stdout: create 234/38 file after the snapshot 2023-10-06T03:59:55.341 INFO:tasks.workunit.client.0.smithi157.stdout: create 234/39 file after the snapshot 2023-10-06T03:59:55.341 INFO:tasks.workunit.client.0.smithi157.stdout: create 235/.snap/snap-subdir-test 2023-10-06T03:59:55.372 INFO:tasks.workunit.client.0.smithi157.stdout: create 235/30 file after the snapshot 2023-10-06T03:59:55.373 INFO:tasks.workunit.client.0.smithi157.stdout: create 235/31 file after the snapshot 2023-10-06T03:59:55.374 INFO:tasks.workunit.client.0.smithi157.stdout: create 235/32 file after the snapshot 2023-10-06T03:59:55.374 INFO:tasks.workunit.client.0.smithi157.stdout: create 235/33 file after the snapshot 2023-10-06T03:59:55.375 INFO:tasks.workunit.client.0.smithi157.stdout: create 235/34 file after the snapshot 2023-10-06T03:59:55.376 INFO:tasks.workunit.client.0.smithi157.stdout: create 235/35 file after the snapshot 2023-10-06T03:59:55.376 INFO:tasks.workunit.client.0.smithi157.stdout: create 235/36 file after the snapshot 2023-10-06T03:59:55.377 INFO:tasks.workunit.client.0.smithi157.stdout: create 235/37 file after the snapshot 2023-10-06T03:59:55.378 INFO:tasks.workunit.client.0.smithi157.stdout: create 235/38 file after the snapshot 2023-10-06T03:59:55.378 INFO:tasks.workunit.client.0.smithi157.stdout: create 235/39 file after the snapshot 2023-10-06T03:59:55.379 INFO:tasks.workunit.client.0.smithi157.stdout: create 236/.snap/snap-subdir-test 2023-10-06T03:59:55.403 INFO:tasks.workunit.client.0.smithi157.stdout: create 236/30 file after the snapshot 2023-10-06T03:59:55.404 INFO:tasks.workunit.client.0.smithi157.stdout: create 236/31 file after the snapshot 2023-10-06T03:59:55.406 INFO:tasks.workunit.client.0.smithi157.stdout: create 236/32 file after the snapshot 2023-10-06T03:59:55.407 INFO:tasks.workunit.client.0.smithi157.stdout: create 236/33 file after the snapshot 2023-10-06T03:59:55.407 INFO:tasks.workunit.client.0.smithi157.stdout: create 236/34 file after the snapshot 2023-10-06T03:59:55.408 INFO:tasks.workunit.client.0.smithi157.stdout: create 236/35 file after the snapshot 2023-10-06T03:59:55.409 INFO:tasks.workunit.client.0.smithi157.stdout: create 236/36 file after the snapshot 2023-10-06T03:59:55.410 INFO:tasks.workunit.client.0.smithi157.stdout: create 236/37 file after the snapshot 2023-10-06T03:59:55.411 INFO:tasks.workunit.client.0.smithi157.stdout: create 236/38 file after the snapshot 2023-10-06T03:59:55.412 INFO:tasks.workunit.client.0.smithi157.stdout: create 236/39 file after the snapshot 2023-10-06T03:59:55.413 INFO:tasks.workunit.client.0.smithi157.stdout: create 237/.snap/snap-subdir-test 2023-10-06T03:59:55.437 INFO:tasks.workunit.client.0.smithi157.stdout: create 237/30 file after the snapshot 2023-10-06T03:59:55.438 INFO:tasks.workunit.client.0.smithi157.stdout: create 237/31 file after the snapshot 2023-10-06T03:59:55.439 INFO:tasks.workunit.client.0.smithi157.stdout: create 237/32 file after the snapshot 2023-10-06T03:59:55.439 INFO:tasks.workunit.client.0.smithi157.stdout: create 237/33 file after the snapshot 2023-10-06T03:59:55.440 INFO:tasks.workunit.client.0.smithi157.stdout: create 237/34 file after the snapshot 2023-10-06T03:59:55.441 INFO:tasks.workunit.client.0.smithi157.stdout: create 237/35 file after the snapshot 2023-10-06T03:59:55.441 INFO:tasks.workunit.client.0.smithi157.stdout: create 237/36 file after the snapshot 2023-10-06T03:59:55.442 INFO:tasks.workunit.client.0.smithi157.stdout: create 237/37 file after the snapshot 2023-10-06T03:59:55.443 INFO:tasks.workunit.client.0.smithi157.stdout: create 237/38 file after the snapshot 2023-10-06T03:59:55.443 INFO:tasks.workunit.client.0.smithi157.stdout: create 237/39 file after the snapshot 2023-10-06T03:59:55.444 INFO:tasks.workunit.client.0.smithi157.stdout: create 238/.snap/snap-subdir-test 2023-10-06T03:59:55.468 INFO:tasks.workunit.client.0.smithi157.stdout: create 238/30 file after the snapshot 2023-10-06T03:59:55.469 INFO:tasks.workunit.client.0.smithi157.stdout: create 238/31 file after the snapshot 2023-10-06T03:59:55.470 INFO:tasks.workunit.client.0.smithi157.stdout: create 238/32 file after the snapshot 2023-10-06T03:59:55.470 INFO:tasks.workunit.client.0.smithi157.stdout: create 238/33 file after the snapshot 2023-10-06T03:59:55.471 INFO:tasks.workunit.client.0.smithi157.stdout: create 238/34 file after the snapshot 2023-10-06T03:59:55.472 INFO:tasks.workunit.client.0.smithi157.stdout: create 238/35 file after the snapshot 2023-10-06T03:59:55.472 INFO:tasks.workunit.client.0.smithi157.stdout: create 238/36 file after the snapshot 2023-10-06T03:59:55.473 INFO:tasks.workunit.client.0.smithi157.stdout: create 238/37 file after the snapshot 2023-10-06T03:59:55.474 INFO:tasks.workunit.client.0.smithi157.stdout: create 238/38 file after the snapshot 2023-10-06T03:59:55.474 INFO:tasks.workunit.client.0.smithi157.stdout: create 238/39 file after the snapshot 2023-10-06T03:59:55.475 INFO:tasks.workunit.client.0.smithi157.stdout: create 239/.snap/snap-subdir-test 2023-10-06T03:59:55.499 INFO:tasks.workunit.client.0.smithi157.stdout: create 239/30 file after the snapshot 2023-10-06T03:59:55.500 INFO:tasks.workunit.client.0.smithi157.stdout: create 239/31 file after the snapshot 2023-10-06T03:59:55.500 INFO:tasks.workunit.client.0.smithi157.stdout: create 239/32 file after the snapshot 2023-10-06T03:59:55.501 INFO:tasks.workunit.client.0.smithi157.stdout: create 239/33 file after the snapshot 2023-10-06T03:59:55.501 INFO:tasks.workunit.client.0.smithi157.stdout: create 239/34 file after the snapshot 2023-10-06T03:59:55.502 INFO:tasks.workunit.client.0.smithi157.stdout: create 239/35 file after the snapshot 2023-10-06T03:59:55.503 INFO:tasks.workunit.client.0.smithi157.stdout: create 239/36 file after the snapshot 2023-10-06T03:59:55.503 INFO:tasks.workunit.client.0.smithi157.stdout: create 239/37 file after the snapshot 2023-10-06T03:59:55.504 INFO:tasks.workunit.client.0.smithi157.stdout: create 239/38 file after the snapshot 2023-10-06T03:59:55.505 INFO:tasks.workunit.client.0.smithi157.stdout: create 239/39 file after the snapshot 2023-10-06T03:59:55.506 INFO:tasks.workunit.client.0.smithi157.stdout: create 240/.snap/snap-subdir-test 2023-10-06T03:59:55.531 INFO:tasks.workunit.client.0.smithi157.stdout: create 240/30 file after the snapshot 2023-10-06T03:59:55.531 INFO:tasks.workunit.client.0.smithi157.stdout: create 240/31 file after the snapshot 2023-10-06T03:59:55.532 INFO:tasks.workunit.client.0.smithi157.stdout: create 240/32 file after the snapshot 2023-10-06T03:59:55.533 INFO:tasks.workunit.client.0.smithi157.stdout: create 240/33 file after the snapshot 2023-10-06T03:59:55.533 INFO:tasks.workunit.client.0.smithi157.stdout: create 240/34 file after the snapshot 2023-10-06T03:59:55.534 INFO:tasks.workunit.client.0.smithi157.stdout: create 240/35 file after the snapshot 2023-10-06T03:59:55.535 INFO:tasks.workunit.client.0.smithi157.stdout: create 240/36 file after the snapshot 2023-10-06T03:59:55.535 INFO:tasks.workunit.client.0.smithi157.stdout: create 240/37 file after the snapshot 2023-10-06T03:59:55.536 INFO:tasks.workunit.client.0.smithi157.stdout: create 240/38 file after the snapshot 2023-10-06T03:59:55.536 INFO:tasks.workunit.client.0.smithi157.stdout: create 240/39 file after the snapshot 2023-10-06T03:59:55.537 INFO:tasks.workunit.client.0.smithi157.stdout: create 241/.snap/snap-subdir-test 2023-10-06T03:59:55.565 INFO:tasks.workunit.client.0.smithi157.stdout: create 241/30 file after the snapshot 2023-10-06T03:59:55.566 INFO:tasks.workunit.client.0.smithi157.stdout: create 241/31 file after the snapshot 2023-10-06T03:59:55.567 INFO:tasks.workunit.client.0.smithi157.stdout: create 241/32 file after the snapshot 2023-10-06T03:59:55.567 INFO:tasks.workunit.client.0.smithi157.stdout: create 241/33 file after the snapshot 2023-10-06T03:59:55.568 INFO:tasks.workunit.client.0.smithi157.stdout: create 241/34 file after the snapshot 2023-10-06T03:59:55.569 INFO:tasks.workunit.client.0.smithi157.stdout: create 241/35 file after the snapshot 2023-10-06T03:59:55.569 INFO:tasks.workunit.client.0.smithi157.stdout: create 241/36 file after the snapshot 2023-10-06T03:59:55.570 INFO:tasks.workunit.client.0.smithi157.stdout: create 241/37 file after the snapshot 2023-10-06T03:59:55.571 INFO:tasks.workunit.client.0.smithi157.stdout: create 241/38 file after the snapshot 2023-10-06T03:59:55.571 INFO:tasks.workunit.client.0.smithi157.stdout: create 241/39 file after the snapshot 2023-10-06T03:59:55.572 INFO:tasks.workunit.client.0.smithi157.stdout: create 242/.snap/snap-subdir-test 2023-10-06T03:59:55.599 INFO:tasks.workunit.client.0.smithi157.stdout: create 242/30 file after the snapshot 2023-10-06T03:59:56.060 INFO:tasks.workunit.client.0.smithi157.stdout: create 242/31 file after the snapshot 2023-10-06T03:59:56.060 INFO:tasks.workunit.client.0.smithi157.stdout: create 242/32 file after the snapshot 2023-10-06T03:59:56.061 INFO:tasks.workunit.client.0.smithi157.stdout: create 242/33 file after the snapshot 2023-10-06T03:59:56.062 INFO:tasks.workunit.client.0.smithi157.stdout: create 242/34 file after the snapshot 2023-10-06T03:59:56.062 INFO:tasks.workunit.client.0.smithi157.stdout: create 242/35 file after the snapshot 2023-10-06T03:59:56.063 INFO:tasks.workunit.client.0.smithi157.stdout: create 242/36 file after the snapshot 2023-10-06T03:59:56.064 INFO:tasks.workunit.client.0.smithi157.stdout: create 242/37 file after the snapshot 2023-10-06T03:59:56.064 INFO:tasks.workunit.client.0.smithi157.stdout: create 242/38 file after the snapshot 2023-10-06T03:59:56.065 INFO:tasks.workunit.client.0.smithi157.stdout: create 242/39 file after the snapshot 2023-10-06T03:59:56.066 INFO:tasks.workunit.client.0.smithi157.stdout: create 243/.snap/snap-subdir-test 2023-10-06T03:59:56.091 INFO:tasks.workunit.client.0.smithi157.stdout: create 243/30 file after the snapshot 2023-10-06T03:59:56.092 INFO:tasks.workunit.client.0.smithi157.stdout: create 243/31 file after the snapshot 2023-10-06T03:59:56.093 INFO:tasks.workunit.client.0.smithi157.stdout: create 243/32 file after the snapshot 2023-10-06T03:59:56.093 INFO:tasks.workunit.client.0.smithi157.stdout: create 243/33 file after the snapshot 2023-10-06T03:59:56.094 INFO:tasks.workunit.client.0.smithi157.stdout: create 243/34 file after the snapshot 2023-10-06T03:59:56.095 INFO:tasks.workunit.client.0.smithi157.stdout: create 243/35 file after the snapshot 2023-10-06T03:59:56.095 INFO:tasks.workunit.client.0.smithi157.stdout: create 243/36 file after the snapshot 2023-10-06T03:59:56.096 INFO:tasks.workunit.client.0.smithi157.stdout: create 243/37 file after the snapshot 2023-10-06T03:59:56.097 INFO:tasks.workunit.client.0.smithi157.stdout: create 243/38 file after the snapshot 2023-10-06T03:59:56.097 INFO:tasks.workunit.client.0.smithi157.stdout: create 243/39 file after the snapshot 2023-10-06T03:59:56.098 INFO:tasks.workunit.client.0.smithi157.stdout: create 244/.snap/snap-subdir-test 2023-10-06T03:59:56.122 INFO:tasks.workunit.client.0.smithi157.stdout: create 244/30 file after the snapshot 2023-10-06T03:59:56.123 INFO:tasks.workunit.client.0.smithi157.stdout: create 244/31 file after the snapshot 2023-10-06T03:59:56.123 INFO:tasks.workunit.client.0.smithi157.stdout: create 244/32 file after the snapshot 2023-10-06T03:59:56.124 INFO:tasks.workunit.client.0.smithi157.stdout: create 244/33 file after the snapshot 2023-10-06T03:59:56.125 INFO:tasks.workunit.client.0.smithi157.stdout: create 244/34 file after the snapshot 2023-10-06T03:59:56.126 INFO:tasks.workunit.client.0.smithi157.stdout: create 244/35 file after the snapshot 2023-10-06T03:59:56.126 INFO:tasks.workunit.client.0.smithi157.stdout: create 244/36 file after the snapshot 2023-10-06T03:59:56.127 INFO:tasks.workunit.client.0.smithi157.stdout: create 244/37 file after the snapshot 2023-10-06T03:59:56.127 INFO:tasks.workunit.client.0.smithi157.stdout: create 244/38 file after the snapshot 2023-10-06T03:59:56.128 INFO:tasks.workunit.client.0.smithi157.stdout: create 244/39 file after the snapshot 2023-10-06T03:59:56.129 INFO:tasks.workunit.client.0.smithi157.stdout: create 245/.snap/snap-subdir-test 2023-10-06T03:59:56.154 INFO:tasks.workunit.client.0.smithi157.stdout: create 245/30 file after the snapshot 2023-10-06T03:59:56.155 INFO:tasks.workunit.client.0.smithi157.stdout: create 245/31 file after the snapshot 2023-10-06T03:59:56.156 INFO:tasks.workunit.client.0.smithi157.stdout: create 245/32 file after the snapshot 2023-10-06T03:59:56.157 INFO:tasks.workunit.client.0.smithi157.stdout: create 245/33 file after the snapshot 2023-10-06T03:59:56.158 INFO:tasks.workunit.client.0.smithi157.stdout: create 245/34 file after the snapshot 2023-10-06T03:59:56.159 INFO:tasks.workunit.client.0.smithi157.stdout: create 245/35 file after the snapshot 2023-10-06T03:59:56.160 INFO:tasks.workunit.client.0.smithi157.stdout: create 245/36 file after the snapshot 2023-10-06T03:59:56.161 INFO:tasks.workunit.client.0.smithi157.stdout: create 245/37 file after the snapshot 2023-10-06T03:59:56.162 INFO:tasks.workunit.client.0.smithi157.stdout: create 245/38 file after the snapshot 2023-10-06T03:59:56.162 INFO:tasks.workunit.client.0.smithi157.stdout: create 245/39 file after the snapshot 2023-10-06T03:59:56.163 INFO:tasks.workunit.client.0.smithi157.stdout: create 246/.snap/snap-subdir-test 2023-10-06T03:59:56.187 INFO:tasks.workunit.client.0.smithi157.stdout: create 246/30 file after the snapshot 2023-10-06T03:59:56.188 INFO:tasks.workunit.client.0.smithi157.stdout: create 246/31 file after the snapshot 2023-10-06T03:59:56.189 INFO:tasks.workunit.client.0.smithi157.stdout: create 246/32 file after the snapshot 2023-10-06T03:59:56.190 INFO:tasks.workunit.client.0.smithi157.stdout: create 246/33 file after the snapshot 2023-10-06T03:59:56.190 INFO:tasks.workunit.client.0.smithi157.stdout: create 246/34 file after the snapshot 2023-10-06T03:59:56.191 INFO:tasks.workunit.client.0.smithi157.stdout: create 246/35 file after the snapshot 2023-10-06T03:59:56.192 INFO:tasks.workunit.client.0.smithi157.stdout: create 246/36 file after the snapshot 2023-10-06T03:59:56.192 INFO:tasks.workunit.client.0.smithi157.stdout: create 246/37 file after the snapshot 2023-10-06T03:59:56.193 INFO:tasks.workunit.client.0.smithi157.stdout: create 246/38 file after the snapshot 2023-10-06T03:59:56.193 INFO:tasks.workunit.client.0.smithi157.stdout: create 246/39 file after the snapshot 2023-10-06T03:59:56.194 INFO:tasks.workunit.client.0.smithi157.stdout: create 247/.snap/snap-subdir-test 2023-10-06T03:59:56.218 INFO:tasks.workunit.client.0.smithi157.stdout: create 247/30 file after the snapshot 2023-10-06T03:59:56.219 INFO:tasks.workunit.client.0.smithi157.stdout: create 247/31 file after the snapshot 2023-10-06T03:59:56.220 INFO:tasks.workunit.client.0.smithi157.stdout: create 247/32 file after the snapshot 2023-10-06T03:59:56.220 INFO:tasks.workunit.client.0.smithi157.stdout: create 247/33 file after the snapshot 2023-10-06T03:59:56.221 INFO:tasks.workunit.client.0.smithi157.stdout: create 247/34 file after the snapshot 2023-10-06T03:59:56.222 INFO:tasks.workunit.client.0.smithi157.stdout: create 247/35 file after the snapshot 2023-10-06T03:59:56.223 INFO:tasks.workunit.client.0.smithi157.stdout: create 247/36 file after the snapshot 2023-10-06T03:59:56.223 INFO:tasks.workunit.client.0.smithi157.stdout: create 247/37 file after the snapshot 2023-10-06T03:59:56.224 INFO:tasks.workunit.client.0.smithi157.stdout: create 247/38 file after the snapshot 2023-10-06T03:59:56.225 INFO:tasks.workunit.client.0.smithi157.stdout: create 247/39 file after the snapshot 2023-10-06T03:59:56.225 INFO:tasks.workunit.client.0.smithi157.stdout: create 248/.snap/snap-subdir-test 2023-10-06T03:59:56.260 INFO:tasks.workunit.client.0.smithi157.stdout: create 248/30 file after the snapshot 2023-10-06T03:59:56.261 INFO:tasks.workunit.client.0.smithi157.stdout: create 248/31 file after the snapshot 2023-10-06T03:59:56.262 INFO:tasks.workunit.client.0.smithi157.stdout: create 248/32 file after the snapshot 2023-10-06T03:59:56.262 INFO:tasks.workunit.client.0.smithi157.stdout: create 248/33 file after the snapshot 2023-10-06T03:59:56.263 INFO:tasks.workunit.client.0.smithi157.stdout: create 248/34 file after the snapshot 2023-10-06T03:59:56.263 INFO:tasks.workunit.client.0.smithi157.stdout: create 248/35 file after the snapshot 2023-10-06T03:59:56.264 INFO:tasks.workunit.client.0.smithi157.stdout: create 248/36 file after the snapshot 2023-10-06T03:59:56.265 INFO:tasks.workunit.client.0.smithi157.stdout: create 248/37 file after the snapshot 2023-10-06T03:59:56.265 INFO:tasks.workunit.client.0.smithi157.stdout: create 248/38 file after the snapshot 2023-10-06T03:59:56.266 INFO:tasks.workunit.client.0.smithi157.stdout: create 248/39 file after the snapshot 2023-10-06T03:59:56.267 INFO:tasks.workunit.client.0.smithi157.stdout: create 249/.snap/snap-subdir-test 2023-10-06T03:59:56.292 INFO:tasks.workunit.client.0.smithi157.stdout: create 249/30 file after the snapshot 2023-10-06T03:59:56.374 INFO:tasks.workunit.client.0.smithi157.stdout: create 249/31 file after the snapshot 2023-10-06T03:59:56.375 INFO:tasks.workunit.client.0.smithi157.stdout: create 249/32 file after the snapshot 2023-10-06T03:59:56.375 INFO:tasks.workunit.client.0.smithi157.stdout: create 249/33 file after the snapshot 2023-10-06T03:59:56.375 INFO:tasks.workunit.client.0.smithi157.stdout: create 249/34 file after the snapshot 2023-10-06T03:59:56.375 INFO:tasks.workunit.client.0.smithi157.stdout: create 249/35 file after the snapshot 2023-10-06T03:59:56.375 INFO:tasks.workunit.client.0.smithi157.stdout: create 249/36 file after the snapshot 2023-10-06T03:59:56.375 INFO:tasks.workunit.client.0.smithi157.stdout: create 249/37 file after the snapshot 2023-10-06T03:59:56.375 INFO:tasks.workunit.client.0.smithi157.stdout: create 249/38 file after the snapshot 2023-10-06T03:59:56.375 INFO:tasks.workunit.client.0.smithi157.stdout: create 249/39 file after the snapshot 2023-10-06T03:59:56.375 INFO:tasks.workunit.client.0.smithi157.stdout: create 250/.snap/snap-subdir-test 2023-10-06T03:59:56.375 INFO:tasks.workunit.client.0.smithi157.stdout: create 250/30 file after the snapshot 2023-10-06T03:59:56.375 INFO:tasks.workunit.client.0.smithi157.stdout: create 250/31 file after the snapshot 2023-10-06T03:59:56.376 INFO:tasks.workunit.client.0.smithi157.stdout: create 250/32 file after the snapshot 2023-10-06T03:59:56.376 INFO:tasks.workunit.client.0.smithi157.stdout: create 250/33 file after the snapshot 2023-10-06T03:59:56.376 INFO:tasks.workunit.client.0.smithi157.stdout: create 250/34 file after the snapshot 2023-10-06T03:59:56.376 INFO:tasks.workunit.client.0.smithi157.stdout: create 250/35 file after the snapshot 2023-10-06T03:59:56.376 INFO:tasks.workunit.client.0.smithi157.stdout: create 250/36 file after the snapshot 2023-10-06T03:59:56.376 INFO:tasks.workunit.client.0.smithi157.stdout: create 250/37 file after the snapshot 2023-10-06T03:59:56.376 INFO:tasks.workunit.client.0.smithi157.stdout: create 250/38 file after the snapshot 2023-10-06T03:59:56.376 INFO:tasks.workunit.client.0.smithi157.stdout: create 250/39 file after the snapshot 2023-10-06T03:59:56.376 INFO:tasks.workunit.client.0.smithi157.stdout: create 251/.snap/snap-subdir-test 2023-10-06T03:59:56.376 INFO:tasks.workunit.client.0.smithi157.stdout: create 251/30 file after the snapshot 2023-10-06T03:59:56.376 INFO:tasks.workunit.client.0.smithi157.stdout: create 251/31 file after the snapshot 2023-10-06T03:59:56.376 INFO:tasks.workunit.client.0.smithi157.stdout: create 251/32 file after the snapshot 2023-10-06T03:59:56.376 INFO:tasks.workunit.client.0.smithi157.stdout: create 251/33 file after the snapshot 2023-10-06T03:59:56.377 INFO:tasks.workunit.client.0.smithi157.stdout: create 251/34 file after the snapshot 2023-10-06T03:59:56.377 INFO:tasks.workunit.client.0.smithi157.stdout: create 251/35 file after the snapshot 2023-10-06T03:59:56.377 INFO:tasks.workunit.client.0.smithi157.stdout: create 251/36 file after the snapshot 2023-10-06T03:59:56.377 INFO:tasks.workunit.client.0.smithi157.stdout: create 251/37 file after the snapshot 2023-10-06T03:59:56.377 INFO:tasks.workunit.client.0.smithi157.stdout: create 251/38 file after the snapshot 2023-10-06T03:59:56.377 INFO:tasks.workunit.client.0.smithi157.stdout: create 251/39 file after the snapshot 2023-10-06T03:59:56.377 INFO:tasks.workunit.client.0.smithi157.stdout: create 252/.snap/snap-subdir-test 2023-10-06T03:59:56.383 INFO:tasks.workunit.client.0.smithi157.stdout: create 252/30 file after the snapshot 2023-10-06T03:59:56.384 INFO:tasks.workunit.client.0.smithi157.stdout: create 252/31 file after the snapshot 2023-10-06T03:59:56.384 INFO:tasks.workunit.client.0.smithi157.stdout: create 252/32 file after the snapshot 2023-10-06T03:59:56.385 INFO:tasks.workunit.client.0.smithi157.stdout: create 252/33 file after the snapshot 2023-10-06T03:59:56.386 INFO:tasks.workunit.client.0.smithi157.stdout: create 252/34 file after the snapshot 2023-10-06T03:59:56.386 INFO:tasks.workunit.client.0.smithi157.stdout: create 252/35 file after the snapshot 2023-10-06T03:59:56.387 INFO:tasks.workunit.client.0.smithi157.stdout: create 252/36 file after the snapshot 2023-10-06T03:59:56.388 INFO:tasks.workunit.client.0.smithi157.stdout: create 252/37 file after the snapshot 2023-10-06T03:59:56.388 INFO:tasks.workunit.client.0.smithi157.stdout: create 252/38 file after the snapshot 2023-10-06T03:59:56.389 INFO:tasks.workunit.client.0.smithi157.stdout: create 252/39 file after the snapshot 2023-10-06T03:59:56.390 INFO:tasks.workunit.client.0.smithi157.stdout: create 253/.snap/snap-subdir-test 2023-10-06T03:59:56.413 INFO:tasks.workunit.client.0.smithi157.stdout: create 253/30 file after the snapshot 2023-10-06T03:59:56.673 INFO:tasks.workunit.client.0.smithi157.stdout: create 253/31 file after the snapshot 2023-10-06T03:59:56.673 INFO:tasks.workunit.client.0.smithi157.stdout: create 253/32 file after the snapshot 2023-10-06T03:59:56.674 INFO:tasks.workunit.client.0.smithi157.stdout: create 253/33 file after the snapshot 2023-10-06T03:59:56.675 INFO:tasks.workunit.client.0.smithi157.stdout: create 253/34 file after the snapshot 2023-10-06T03:59:56.675 INFO:tasks.workunit.client.0.smithi157.stdout: create 253/35 file after the snapshot 2023-10-06T03:59:56.676 INFO:tasks.workunit.client.0.smithi157.stdout: create 253/36 file after the snapshot 2023-10-06T03:59:56.676 INFO:tasks.workunit.client.0.smithi157.stdout: create 253/37 file after the snapshot 2023-10-06T03:59:56.677 INFO:tasks.workunit.client.0.smithi157.stdout: create 253/38 file after the snapshot 2023-10-06T03:59:56.678 INFO:tasks.workunit.client.0.smithi157.stdout: create 253/39 file after the snapshot 2023-10-06T03:59:56.678 INFO:tasks.workunit.client.0.smithi157.stdout: create 254/.snap/snap-subdir-test 2023-10-06T03:59:56.703 INFO:tasks.workunit.client.0.smithi157.stdout: create 254/30 file after the snapshot 2023-10-06T03:59:56.703 INFO:tasks.workunit.client.0.smithi157.stdout: create 254/31 file after the snapshot 2023-10-06T03:59:56.704 INFO:tasks.workunit.client.0.smithi157.stdout: create 254/32 file after the snapshot 2023-10-06T03:59:56.705 INFO:tasks.workunit.client.0.smithi157.stdout: create 254/33 file after the snapshot 2023-10-06T03:59:56.705 INFO:tasks.workunit.client.0.smithi157.stdout: create 254/34 file after the snapshot 2023-10-06T03:59:56.706 INFO:tasks.workunit.client.0.smithi157.stdout: create 254/35 file after the snapshot 2023-10-06T03:59:56.707 INFO:tasks.workunit.client.0.smithi157.stdout: create 254/36 file after the snapshot 2023-10-06T03:59:56.707 INFO:tasks.workunit.client.0.smithi157.stdout: create 254/37 file after the snapshot 2023-10-06T03:59:56.708 INFO:tasks.workunit.client.0.smithi157.stdout: create 254/38 file after the snapshot 2023-10-06T03:59:56.969 INFO:tasks.workunit.client.0.smithi157.stdout: create 254/39 file after the snapshot 2023-10-06T03:59:56.978 INFO:tasks.workunit.client.0.smithi157.stdout: create 255/.snap/snap-subdir-test 2023-10-06T03:59:57.845 INFO:tasks.workunit.client.0.smithi157.stdout: create 255/30 file after the snapshot 2023-10-06T03:59:57.848 INFO:tasks.workunit.client.0.smithi157.stdout: create 255/31 file after the snapshot 2023-10-06T03:59:57.849 INFO:tasks.workunit.client.0.smithi157.stdout: create 255/32 file after the snapshot 2023-10-06T03:59:57.849 INFO:tasks.workunit.client.0.smithi157.stdout: create 255/33 file after the snapshot 2023-10-06T03:59:57.850 INFO:tasks.workunit.client.0.smithi157.stdout: create 255/34 file after the snapshot 2023-10-06T03:59:57.851 INFO:tasks.workunit.client.0.smithi157.stdout: create 255/35 file after the snapshot 2023-10-06T03:59:57.852 INFO:tasks.workunit.client.0.smithi157.stdout: create 255/36 file after the snapshot 2023-10-06T03:59:57.852 INFO:tasks.workunit.client.0.smithi157.stdout: create 255/37 file after the snapshot 2023-10-06T03:59:57.853 INFO:tasks.workunit.client.0.smithi157.stdout: create 255/38 file after the snapshot 2023-10-06T03:59:57.853 INFO:tasks.workunit.client.0.smithi157.stdout: create 255/39 file after the snapshot 2023-10-06T03:59:57.854 INFO:tasks.workunit.client.0.smithi157.stdout: create 256/.snap/snap-subdir-test 2023-10-06T03:59:57.881 INFO:tasks.workunit.client.0.smithi157.stdout: create 256/30 file after the snapshot 2023-10-06T03:59:57.882 INFO:tasks.workunit.client.0.smithi157.stdout: create 256/31 file after the snapshot 2023-10-06T03:59:57.882 INFO:tasks.workunit.client.0.smithi157.stdout: create 256/32 file after the snapshot 2023-10-06T03:59:57.882 INFO:tasks.workunit.client.0.smithi157.stdout: create 256/33 file after the snapshot 2023-10-06T03:59:57.882 INFO:tasks.workunit.client.0.smithi157.stdout: create 256/34 file after the snapshot 2023-10-06T03:59:57.883 INFO:tasks.workunit.client.0.smithi157.stdout: create 256/35 file after the snapshot 2023-10-06T03:59:58.201 INFO:tasks.workunit.client.0.smithi157.stdout: create 256/36 file after the snapshot 2023-10-06T03:59:58.291 INFO:tasks.workunit.client.0.smithi157.stdout: create 256/37 file after the snapshot 2023-10-06T03:59:58.293 INFO:tasks.workunit.client.0.smithi157.stdout: create 256/38 file after the snapshot 2023-10-06T03:59:58.303 INFO:tasks.workunit.client.0.smithi157.stdout: create 256/39 file after the snapshot 2023-10-06T03:59:58.319 INFO:tasks.workunit.client.0.smithi157.stdout: create 257/.snap/snap-subdir-test 2023-10-06T04:00:00.472 INFO:tasks.workunit.client.0.smithi157.stdout: create 257/30 file after the snapshot 2023-10-06T04:00:00.473 INFO:tasks.workunit.client.0.smithi157.stdout: create 257/31 file after the snapshot 2023-10-06T04:00:00.473 INFO:tasks.workunit.client.0.smithi157.stdout: create 257/32 file after the snapshot 2023-10-06T04:00:00.474 INFO:tasks.workunit.client.0.smithi157.stdout: create 257/33 file after the snapshot 2023-10-06T04:00:00.475 INFO:tasks.workunit.client.0.smithi157.stdout: create 257/34 file after the snapshot 2023-10-06T04:00:00.476 INFO:tasks.workunit.client.0.smithi157.stdout: create 257/35 file after the snapshot 2023-10-06T04:00:00.476 INFO:tasks.workunit.client.0.smithi157.stdout: create 257/36 file after the snapshot 2023-10-06T04:00:00.477 INFO:tasks.workunit.client.0.smithi157.stdout: create 257/37 file after the snapshot 2023-10-06T04:00:00.478 INFO:tasks.workunit.client.0.smithi157.stdout: create 257/38 file after the snapshot 2023-10-06T04:00:00.478 INFO:tasks.workunit.client.0.smithi157.stdout: create 257/39 file after the snapshot 2023-10-06T04:00:00.479 INFO:tasks.workunit.client.0.smithi157.stdout: create 258/.snap/snap-subdir-test 2023-10-06T04:00:00.761 INFO:tasks.workunit.client.0.smithi157.stdout: create 258/30 file after the snapshot 2023-10-06T04:00:00.762 INFO:tasks.workunit.client.0.smithi157.stdout: create 258/31 file after the snapshot 2023-10-06T04:00:00.763 INFO:tasks.workunit.client.0.smithi157.stdout: create 258/32 file after the snapshot 2023-10-06T04:00:00.763 INFO:tasks.workunit.client.0.smithi157.stdout: create 258/33 file after the snapshot 2023-10-06T04:00:00.764 INFO:tasks.workunit.client.0.smithi157.stdout: create 258/34 file after the snapshot 2023-10-06T04:00:00.765 INFO:tasks.workunit.client.0.smithi157.stdout: create 258/35 file after the snapshot 2023-10-06T04:00:00.766 INFO:tasks.workunit.client.0.smithi157.stdout: create 258/36 file after the snapshot 2023-10-06T04:00:00.766 INFO:tasks.workunit.client.0.smithi157.stdout: create 258/37 file after the snapshot 2023-10-06T04:00:00.767 INFO:tasks.workunit.client.0.smithi157.stdout: create 258/38 file after the snapshot 2023-10-06T04:00:01.012 INFO:tasks.workunit.client.0.smithi157.stdout: create 258/39 file after the snapshot 2023-10-06T04:00:01.013 INFO:tasks.workunit.client.0.smithi157.stdout: create 259/.snap/snap-subdir-test 2023-10-06T04:00:01.045 INFO:tasks.workunit.client.0.smithi157.stdout: create 259/30 file after the snapshot 2023-10-06T04:00:01.045 INFO:tasks.workunit.client.0.smithi157.stdout: create 259/31 file after the snapshot 2023-10-06T04:00:01.046 INFO:tasks.workunit.client.0.smithi157.stdout: create 259/32 file after the snapshot 2023-10-06T04:00:01.047 INFO:tasks.workunit.client.0.smithi157.stdout: create 259/33 file after the snapshot 2023-10-06T04:00:01.047 INFO:tasks.workunit.client.0.smithi157.stdout: create 259/34 file after the snapshot 2023-10-06T04:00:01.048 INFO:tasks.workunit.client.0.smithi157.stdout: create 259/35 file after the snapshot 2023-10-06T04:00:01.049 INFO:tasks.workunit.client.0.smithi157.stdout: create 259/36 file after the snapshot 2023-10-06T04:00:01.049 INFO:tasks.workunit.client.0.smithi157.stdout: create 259/37 file after the snapshot 2023-10-06T04:00:01.050 INFO:tasks.workunit.client.0.smithi157.stdout: create 259/38 file after the snapshot 2023-10-06T04:00:01.051 INFO:tasks.workunit.client.0.smithi157.stdout: create 259/39 file after the snapshot 2023-10-06T04:00:01.052 INFO:tasks.workunit.client.0.smithi157.stdout: create 260/.snap/snap-subdir-test 2023-10-06T04:00:01.084 INFO:tasks.workunit.client.0.smithi157.stdout: create 260/30 file after the snapshot 2023-10-06T04:00:01.084 INFO:tasks.workunit.client.0.smithi157.stdout: create 260/31 file after the snapshot 2023-10-06T04:00:01.085 INFO:tasks.workunit.client.0.smithi157.stdout: create 260/32 file after the snapshot 2023-10-06T04:00:01.086 INFO:tasks.workunit.client.0.smithi157.stdout: create 260/33 file after the snapshot 2023-10-06T04:00:01.086 INFO:tasks.workunit.client.0.smithi157.stdout: create 260/34 file after the snapshot 2023-10-06T04:00:01.087 INFO:tasks.workunit.client.0.smithi157.stdout: create 260/35 file after the snapshot 2023-10-06T04:00:01.088 INFO:tasks.workunit.client.0.smithi157.stdout: create 260/36 file after the snapshot 2023-10-06T04:00:01.088 INFO:tasks.workunit.client.0.smithi157.stdout: create 260/37 file after the snapshot 2023-10-06T04:00:01.089 INFO:tasks.workunit.client.0.smithi157.stdout: create 260/38 file after the snapshot 2023-10-06T04:00:01.090 INFO:tasks.workunit.client.0.smithi157.stdout: create 260/39 file after the snapshot 2023-10-06T04:00:01.090 INFO:tasks.workunit.client.0.smithi157.stdout: create 261/.snap/snap-subdir-test 2023-10-06T04:00:01.120 INFO:tasks.workunit.client.0.smithi157.stdout: create 261/30 file after the snapshot 2023-10-06T04:00:01.121 INFO:tasks.workunit.client.0.smithi157.stdout: create 261/31 file after the snapshot 2023-10-06T04:00:01.122 INFO:tasks.workunit.client.0.smithi157.stdout: create 261/32 file after the snapshot 2023-10-06T04:00:01.123 INFO:tasks.workunit.client.0.smithi157.stdout: create 261/33 file after the snapshot 2023-10-06T04:00:01.123 INFO:tasks.workunit.client.0.smithi157.stdout: create 261/34 file after the snapshot 2023-10-06T04:00:01.124 INFO:tasks.workunit.client.0.smithi157.stdout: create 261/35 file after the snapshot 2023-10-06T04:00:01.124 INFO:tasks.workunit.client.0.smithi157.stdout: create 261/36 file after the snapshot 2023-10-06T04:00:01.125 INFO:tasks.workunit.client.0.smithi157.stdout: create 261/37 file after the snapshot 2023-10-06T04:00:01.126 INFO:tasks.workunit.client.0.smithi157.stdout: create 261/38 file after the snapshot 2023-10-06T04:00:01.126 INFO:tasks.workunit.client.0.smithi157.stdout: create 261/39 file after the snapshot 2023-10-06T04:00:01.127 INFO:tasks.workunit.client.0.smithi157.stdout: create 262/.snap/snap-subdir-test 2023-10-06T04:00:01.157 INFO:tasks.workunit.client.0.smithi157.stdout: create 262/30 file after the snapshot 2023-10-06T04:00:01.158 INFO:tasks.workunit.client.0.smithi157.stdout: create 262/31 file after the snapshot 2023-10-06T04:00:01.159 INFO:tasks.workunit.client.0.smithi157.stdout: create 262/32 file after the snapshot 2023-10-06T04:00:01.159 INFO:tasks.workunit.client.0.smithi157.stdout: create 262/33 file after the snapshot 2023-10-06T04:00:01.160 INFO:tasks.workunit.client.0.smithi157.stdout: create 262/34 file after the snapshot 2023-10-06T04:00:01.161 INFO:tasks.workunit.client.0.smithi157.stdout: create 262/35 file after the snapshot 2023-10-06T04:00:01.161 INFO:tasks.workunit.client.0.smithi157.stdout: create 262/36 file after the snapshot 2023-10-06T04:00:01.162 INFO:tasks.workunit.client.0.smithi157.stdout: create 262/37 file after the snapshot 2023-10-06T04:00:01.163 INFO:tasks.workunit.client.0.smithi157.stdout: create 262/38 file after the snapshot 2023-10-06T04:00:01.163 INFO:tasks.workunit.client.0.smithi157.stdout: create 262/39 file after the snapshot 2023-10-06T04:00:01.164 INFO:tasks.workunit.client.0.smithi157.stdout: create 263/.snap/snap-subdir-test 2023-10-06T04:00:01.190 INFO:tasks.workunit.client.0.smithi157.stdout: create 263/30 file after the snapshot 2023-10-06T04:00:01.191 INFO:tasks.workunit.client.0.smithi157.stdout: create 263/31 file after the snapshot 2023-10-06T04:00:01.192 INFO:tasks.workunit.client.0.smithi157.stdout: create 263/32 file after the snapshot 2023-10-06T04:00:01.192 INFO:tasks.workunit.client.0.smithi157.stdout: create 263/33 file after the snapshot 2023-10-06T04:00:01.193 INFO:tasks.workunit.client.0.smithi157.stdout: create 263/34 file after the snapshot 2023-10-06T04:00:01.194 INFO:tasks.workunit.client.0.smithi157.stdout: create 263/35 file after the snapshot 2023-10-06T04:00:01.194 INFO:tasks.workunit.client.0.smithi157.stdout: create 263/36 file after the snapshot 2023-10-06T04:00:01.195 INFO:tasks.workunit.client.0.smithi157.stdout: create 263/37 file after the snapshot 2023-10-06T04:00:01.195 INFO:tasks.workunit.client.0.smithi157.stdout: create 263/38 file after the snapshot 2023-10-06T04:00:01.196 INFO:tasks.workunit.client.0.smithi157.stdout: create 263/39 file after the snapshot 2023-10-06T04:00:01.197 INFO:tasks.workunit.client.0.smithi157.stdout: create 264/.snap/snap-subdir-test 2023-10-06T04:00:03.195 INFO:tasks.workunit.client.0.smithi157.stdout: create 264/30 file after the snapshot 2023-10-06T04:00:03.197 INFO:tasks.workunit.client.0.smithi157.stdout: create 264/31 file after the snapshot 2023-10-06T04:00:03.197 INFO:tasks.workunit.client.0.smithi157.stdout: create 264/32 file after the snapshot 2023-10-06T04:00:03.198 INFO:tasks.workunit.client.0.smithi157.stdout: create 264/33 file after the snapshot 2023-10-06T04:00:03.198 INFO:tasks.workunit.client.0.smithi157.stdout: create 264/34 file after the snapshot 2023-10-06T04:00:03.199 INFO:tasks.workunit.client.0.smithi157.stdout: create 264/35 file after the snapshot 2023-10-06T04:00:03.200 INFO:tasks.workunit.client.0.smithi157.stdout: create 264/36 file after the snapshot 2023-10-06T04:00:03.201 INFO:tasks.workunit.client.0.smithi157.stdout: create 264/37 file after the snapshot 2023-10-06T04:00:03.201 INFO:tasks.workunit.client.0.smithi157.stdout: create 264/38 file after the snapshot 2023-10-06T04:00:03.202 INFO:tasks.workunit.client.0.smithi157.stdout: create 264/39 file after the snapshot 2023-10-06T04:00:03.203 INFO:tasks.workunit.client.0.smithi157.stdout: create 265/.snap/snap-subdir-test 2023-10-06T04:00:03.511 INFO:tasks.workunit.client.0.smithi157.stdout: create 265/30 file after the snapshot 2023-10-06T04:00:03.514 INFO:tasks.workunit.client.0.smithi157.stdout: create 265/31 file after the snapshot 2023-10-06T04:00:03.515 INFO:tasks.workunit.client.0.smithi157.stdout: create 265/32 file after the snapshot 2023-10-06T04:00:03.516 INFO:tasks.workunit.client.0.smithi157.stdout: create 265/33 file after the snapshot 2023-10-06T04:00:03.517 INFO:tasks.workunit.client.0.smithi157.stdout: create 265/34 file after the snapshot 2023-10-06T04:00:03.518 INFO:tasks.workunit.client.0.smithi157.stdout: create 265/35 file after the snapshot 2023-10-06T04:00:03.519 INFO:tasks.workunit.client.0.smithi157.stdout: create 265/36 file after the snapshot 2023-10-06T04:00:03.520 INFO:tasks.workunit.client.0.smithi157.stdout: create 265/37 file after the snapshot 2023-10-06T04:00:03.521 INFO:tasks.workunit.client.0.smithi157.stdout: create 265/38 file after the snapshot 2023-10-06T04:00:03.522 INFO:tasks.workunit.client.0.smithi157.stdout: create 265/39 file after the snapshot 2023-10-06T04:00:03.523 INFO:tasks.workunit.client.0.smithi157.stdout: create 266/.snap/snap-subdir-test 2023-10-06T04:00:03.552 INFO:tasks.workunit.client.0.smithi157.stdout: create 266/30 file after the snapshot 2023-10-06T04:00:03.553 INFO:tasks.workunit.client.0.smithi157.stdout: create 266/31 file after the snapshot 2023-10-06T04:00:03.554 INFO:tasks.workunit.client.0.smithi157.stdout: create 266/32 file after the snapshot 2023-10-06T04:00:03.555 INFO:tasks.workunit.client.0.smithi157.stdout: create 266/33 file after the snapshot 2023-10-06T04:00:03.556 INFO:tasks.workunit.client.0.smithi157.stdout: create 266/34 file after the snapshot 2023-10-06T04:00:03.557 INFO:tasks.workunit.client.0.smithi157.stdout: create 266/35 file after the snapshot 2023-10-06T04:00:03.558 INFO:tasks.workunit.client.0.smithi157.stdout: create 266/36 file after the snapshot 2023-10-06T04:00:03.559 INFO:tasks.workunit.client.0.smithi157.stdout: create 266/37 file after the snapshot 2023-10-06T04:00:03.560 INFO:tasks.workunit.client.0.smithi157.stdout: create 266/38 file after the snapshot 2023-10-06T04:00:03.561 INFO:tasks.workunit.client.0.smithi157.stdout: create 266/39 file after the snapshot 2023-10-06T04:00:03.562 INFO:tasks.workunit.client.0.smithi157.stdout: create 267/.snap/snap-subdir-test 2023-10-06T04:00:03.584 INFO:tasks.workunit.client.0.smithi157.stdout: create 267/30 file after the snapshot 2023-10-06T04:00:03.585 INFO:tasks.workunit.client.0.smithi157.stdout: create 267/31 file after the snapshot 2023-10-06T04:00:03.586 INFO:tasks.workunit.client.0.smithi157.stdout: create 267/32 file after the snapshot 2023-10-06T04:00:03.587 INFO:tasks.workunit.client.0.smithi157.stdout: create 267/33 file after the snapshot 2023-10-06T04:00:03.588 INFO:tasks.workunit.client.0.smithi157.stdout: create 267/34 file after the snapshot 2023-10-06T04:00:03.589 INFO:tasks.workunit.client.0.smithi157.stdout: create 267/35 file after the snapshot 2023-10-06T04:00:03.590 INFO:tasks.workunit.client.0.smithi157.stdout: create 267/36 file after the snapshot 2023-10-06T04:00:03.592 INFO:tasks.workunit.client.0.smithi157.stdout: create 267/37 file after the snapshot 2023-10-06T04:00:03.592 INFO:tasks.workunit.client.0.smithi157.stdout: create 267/38 file after the snapshot 2023-10-06T04:00:03.593 INFO:tasks.workunit.client.0.smithi157.stdout: create 267/39 file after the snapshot 2023-10-06T04:00:03.594 INFO:tasks.workunit.client.0.smithi157.stdout: create 268/.snap/snap-subdir-test 2023-10-06T04:00:03.618 INFO:tasks.workunit.client.0.smithi157.stdout: create 268/30 file after the snapshot 2023-10-06T04:00:03.619 INFO:tasks.workunit.client.0.smithi157.stdout: create 268/31 file after the snapshot 2023-10-06T04:00:03.619 INFO:tasks.workunit.client.0.smithi157.stdout: create 268/32 file after the snapshot 2023-10-06T04:00:03.619 INFO:tasks.workunit.client.0.smithi157.stdout: create 268/33 file after the snapshot 2023-10-06T04:00:03.620 INFO:tasks.workunit.client.0.smithi157.stdout: create 268/34 file after the snapshot 2023-10-06T04:00:03.620 INFO:tasks.workunit.client.0.smithi157.stdout: create 268/35 file after the snapshot 2023-10-06T04:00:03.621 INFO:tasks.workunit.client.0.smithi157.stdout: create 268/36 file after the snapshot 2023-10-06T04:00:03.622 INFO:tasks.workunit.client.0.smithi157.stdout: create 268/37 file after the snapshot 2023-10-06T04:00:03.623 INFO:tasks.workunit.client.0.smithi157.stdout: create 268/38 file after the snapshot 2023-10-06T04:00:03.624 INFO:tasks.workunit.client.0.smithi157.stdout: create 268/39 file after the snapshot 2023-10-06T04:00:03.625 INFO:tasks.workunit.client.0.smithi157.stdout: create 269/.snap/snap-subdir-test 2023-10-06T04:00:03.647 INFO:tasks.workunit.client.0.smithi157.stdout: create 269/30 file after the snapshot 2023-10-06T04:00:03.648 INFO:tasks.workunit.client.0.smithi157.stdout: create 269/31 file after the snapshot 2023-10-06T04:00:03.649 INFO:tasks.workunit.client.0.smithi157.stdout: create 269/32 file after the snapshot 2023-10-06T04:00:03.650 INFO:tasks.workunit.client.0.smithi157.stdout: create 269/33 file after the snapshot 2023-10-06T04:00:03.651 INFO:tasks.workunit.client.0.smithi157.stdout: create 269/34 file after the snapshot 2023-10-06T04:00:03.652 INFO:tasks.workunit.client.0.smithi157.stdout: create 269/35 file after the snapshot 2023-10-06T04:00:03.653 INFO:tasks.workunit.client.0.smithi157.stdout: create 269/36 file after the snapshot 2023-10-06T04:00:03.655 INFO:tasks.workunit.client.0.smithi157.stdout: create 269/37 file after the snapshot 2023-10-06T04:00:03.656 INFO:tasks.workunit.client.0.smithi157.stdout: create 269/38 file after the snapshot 2023-10-06T04:00:03.657 INFO:tasks.workunit.client.0.smithi157.stdout: create 269/39 file after the snapshot 2023-10-06T04:00:03.657 INFO:tasks.workunit.client.0.smithi157.stdout: create 270/.snap/snap-subdir-test 2023-10-06T04:00:03.679 INFO:tasks.workunit.client.0.smithi157.stdout: create 270/30 file after the snapshot 2023-10-06T04:00:03.679 INFO:tasks.workunit.client.0.smithi157.stdout: create 270/31 file after the snapshot 2023-10-06T04:00:03.680 INFO:tasks.workunit.client.0.smithi157.stdout: create 270/32 file after the snapshot 2023-10-06T04:00:03.682 INFO:tasks.workunit.client.0.smithi157.stdout: create 270/33 file after the snapshot 2023-10-06T04:00:03.683 INFO:tasks.workunit.client.0.smithi157.stdout: create 270/34 file after the snapshot 2023-10-06T04:00:03.684 INFO:tasks.workunit.client.0.smithi157.stdout: create 270/35 file after the snapshot 2023-10-06T04:00:03.685 INFO:tasks.workunit.client.0.smithi157.stdout: create 270/36 file after the snapshot 2023-10-06T04:00:03.685 INFO:tasks.workunit.client.0.smithi157.stdout: create 270/37 file after the snapshot 2023-10-06T04:00:03.686 INFO:tasks.workunit.client.0.smithi157.stdout: create 270/38 file after the snapshot 2023-10-06T04:00:03.688 INFO:tasks.workunit.client.0.smithi157.stdout: create 270/39 file after the snapshot 2023-10-06T04:00:03.689 INFO:tasks.workunit.client.0.smithi157.stdout: create 271/.snap/snap-subdir-test 2023-10-06T04:00:03.706 INFO:tasks.workunit.client.0.smithi157.stdout: create 271/30 file after the snapshot 2023-10-06T04:00:03.707 INFO:tasks.workunit.client.0.smithi157.stdout: create 271/31 file after the snapshot 2023-10-06T04:00:03.708 INFO:tasks.workunit.client.0.smithi157.stdout: create 271/32 file after the snapshot 2023-10-06T04:00:03.708 INFO:tasks.workunit.client.0.smithi157.stdout: create 271/33 file after the snapshot 2023-10-06T04:00:03.709 INFO:tasks.workunit.client.0.smithi157.stdout: create 271/34 file after the snapshot 2023-10-06T04:00:03.710 INFO:tasks.workunit.client.0.smithi157.stdout: create 271/35 file after the snapshot 2023-10-06T04:00:03.711 INFO:tasks.workunit.client.0.smithi157.stdout: create 271/36 file after the snapshot 2023-10-06T04:00:03.711 INFO:tasks.workunit.client.0.smithi157.stdout: create 271/37 file after the snapshot 2023-10-06T04:00:03.712 INFO:tasks.workunit.client.0.smithi157.stdout: create 271/38 file after the snapshot 2023-10-06T04:00:03.712 INFO:tasks.workunit.client.0.smithi157.stdout: create 271/39 file after the snapshot 2023-10-06T04:00:03.713 INFO:tasks.workunit.client.0.smithi157.stdout: create 272/.snap/snap-subdir-test 2023-10-06T04:00:03.737 INFO:tasks.workunit.client.0.smithi157.stdout: create 272/30 file after the snapshot 2023-10-06T04:00:03.737 INFO:tasks.workunit.client.0.smithi157.stdout: create 272/31 file after the snapshot 2023-10-06T04:00:03.738 INFO:tasks.workunit.client.0.smithi157.stdout: create 272/32 file after the snapshot 2023-10-06T04:00:03.739 INFO:tasks.workunit.client.0.smithi157.stdout: create 272/33 file after the snapshot 2023-10-06T04:00:03.739 INFO:tasks.workunit.client.0.smithi157.stdout: create 272/34 file after the snapshot 2023-10-06T04:00:03.740 INFO:tasks.workunit.client.0.smithi157.stdout: create 272/35 file after the snapshot 2023-10-06T04:00:03.741 INFO:tasks.workunit.client.0.smithi157.stdout: create 272/36 file after the snapshot 2023-10-06T04:00:03.741 INFO:tasks.workunit.client.0.smithi157.stdout: create 272/37 file after the snapshot 2023-10-06T04:00:03.742 INFO:tasks.workunit.client.0.smithi157.stdout: create 272/38 file after the snapshot 2023-10-06T04:00:03.743 INFO:tasks.workunit.client.0.smithi157.stdout: create 272/39 file after the snapshot 2023-10-06T04:00:03.743 INFO:tasks.workunit.client.0.smithi157.stdout: create 273/.snap/snap-subdir-test 2023-10-06T04:00:03.768 INFO:tasks.workunit.client.0.smithi157.stdout: create 273/30 file after the snapshot 2023-10-06T04:00:03.769 INFO:tasks.workunit.client.0.smithi157.stdout: create 273/31 file after the snapshot 2023-10-06T04:00:03.770 INFO:tasks.workunit.client.0.smithi157.stdout: create 273/32 file after the snapshot 2023-10-06T04:00:03.770 INFO:tasks.workunit.client.0.smithi157.stdout: create 273/33 file after the snapshot 2023-10-06T04:00:03.771 INFO:tasks.workunit.client.0.smithi157.stdout: create 273/34 file after the snapshot 2023-10-06T04:00:03.773 INFO:tasks.workunit.client.0.smithi157.stdout: create 273/35 file after the snapshot 2023-10-06T04:00:03.774 INFO:tasks.workunit.client.0.smithi157.stdout: create 273/36 file after the snapshot 2023-10-06T04:00:03.775 INFO:tasks.workunit.client.0.smithi157.stdout: create 273/37 file after the snapshot 2023-10-06T04:00:03.776 INFO:tasks.workunit.client.0.smithi157.stdout: create 273/38 file after the snapshot 2023-10-06T04:00:03.777 INFO:tasks.workunit.client.0.smithi157.stdout: create 273/39 file after the snapshot 2023-10-06T04:00:03.778 INFO:tasks.workunit.client.0.smithi157.stdout: create 274/.snap/snap-subdir-test 2023-10-06T04:00:03.798 INFO:tasks.workunit.client.0.smithi157.stdout: create 274/30 file after the snapshot 2023-10-06T04:00:03.799 INFO:tasks.workunit.client.0.smithi157.stdout: create 274/31 file after the snapshot 2023-10-06T04:00:03.799 INFO:tasks.workunit.client.0.smithi157.stdout: create 274/32 file after the snapshot 2023-10-06T04:00:03.800 INFO:tasks.workunit.client.0.smithi157.stdout: create 274/33 file after the snapshot 2023-10-06T04:00:03.801 INFO:tasks.workunit.client.0.smithi157.stdout: create 274/34 file after the snapshot 2023-10-06T04:00:03.802 INFO:tasks.workunit.client.0.smithi157.stdout: create 274/35 file after the snapshot 2023-10-06T04:00:03.802 INFO:tasks.workunit.client.0.smithi157.stdout: create 274/36 file after the snapshot 2023-10-06T04:00:03.803 INFO:tasks.workunit.client.0.smithi157.stdout: create 274/37 file after the snapshot 2023-10-06T04:00:03.804 INFO:tasks.workunit.client.0.smithi157.stdout: create 274/38 file after the snapshot 2023-10-06T04:00:03.805 INFO:tasks.workunit.client.0.smithi157.stdout: create 274/39 file after the snapshot 2023-10-06T04:00:03.806 INFO:tasks.workunit.client.0.smithi157.stdout: create 275/.snap/snap-subdir-test 2023-10-06T04:00:03.827 INFO:tasks.workunit.client.0.smithi157.stdout: create 275/30 file after the snapshot 2023-10-06T04:00:03.828 INFO:tasks.workunit.client.0.smithi157.stdout: create 275/31 file after the snapshot 2023-10-06T04:00:03.829 INFO:tasks.workunit.client.0.smithi157.stdout: create 275/32 file after the snapshot 2023-10-06T04:00:03.830 INFO:tasks.workunit.client.0.smithi157.stdout: create 275/33 file after the snapshot 2023-10-06T04:00:03.830 INFO:tasks.workunit.client.0.smithi157.stdout: create 275/34 file after the snapshot 2023-10-06T04:00:03.830 INFO:tasks.workunit.client.0.smithi157.stdout: create 275/35 file after the snapshot 2023-10-06T04:00:03.831 INFO:tasks.workunit.client.0.smithi157.stdout: create 275/36 file after the snapshot 2023-10-06T04:00:03.832 INFO:tasks.workunit.client.0.smithi157.stdout: create 275/37 file after the snapshot 2023-10-06T04:00:03.832 INFO:tasks.workunit.client.0.smithi157.stdout: create 275/38 file after the snapshot 2023-10-06T04:00:03.833 INFO:tasks.workunit.client.0.smithi157.stdout: create 275/39 file after the snapshot 2023-10-06T04:00:03.834 INFO:tasks.workunit.client.0.smithi157.stdout: create 276/.snap/snap-subdir-test 2023-10-06T04:00:03.856 INFO:tasks.workunit.client.0.smithi157.stdout: create 276/30 file after the snapshot 2023-10-06T04:00:03.856 INFO:tasks.workunit.client.0.smithi157.stdout: create 276/31 file after the snapshot 2023-10-06T04:00:03.857 INFO:tasks.workunit.client.0.smithi157.stdout: create 276/32 file after the snapshot 2023-10-06T04:00:03.858 INFO:tasks.workunit.client.0.smithi157.stdout: create 276/33 file after the snapshot 2023-10-06T04:00:03.859 INFO:tasks.workunit.client.0.smithi157.stdout: create 276/34 file after the snapshot 2023-10-06T04:00:03.860 INFO:tasks.workunit.client.0.smithi157.stdout: create 276/35 file after the snapshot 2023-10-06T04:00:03.861 INFO:tasks.workunit.client.0.smithi157.stdout: create 276/36 file after the snapshot 2023-10-06T04:00:03.862 INFO:tasks.workunit.client.0.smithi157.stdout: create 276/37 file after the snapshot 2023-10-06T04:00:03.863 INFO:tasks.workunit.client.0.smithi157.stdout: create 276/38 file after the snapshot 2023-10-06T04:00:03.864 INFO:tasks.workunit.client.0.smithi157.stdout: create 276/39 file after the snapshot 2023-10-06T04:00:03.865 INFO:tasks.workunit.client.0.smithi157.stdout: create 277/.snap/snap-subdir-test 2023-10-06T04:00:03.884 INFO:tasks.workunit.client.0.smithi157.stdout: create 277/30 file after the snapshot 2023-10-06T04:00:03.885 INFO:tasks.workunit.client.0.smithi157.stdout: create 277/31 file after the snapshot 2023-10-06T04:00:03.885 INFO:tasks.workunit.client.0.smithi157.stdout: create 277/32 file after the snapshot 2023-10-06T04:00:03.886 INFO:tasks.workunit.client.0.smithi157.stdout: create 277/33 file after the snapshot 2023-10-06T04:00:03.887 INFO:tasks.workunit.client.0.smithi157.stdout: create 277/34 file after the snapshot 2023-10-06T04:00:03.887 INFO:tasks.workunit.client.0.smithi157.stdout: create 277/35 file after the snapshot 2023-10-06T04:00:03.888 INFO:tasks.workunit.client.0.smithi157.stdout: create 277/36 file after the snapshot 2023-10-06T04:00:03.888 INFO:tasks.workunit.client.0.smithi157.stdout: create 277/37 file after the snapshot 2023-10-06T04:00:03.889 INFO:tasks.workunit.client.0.smithi157.stdout: create 277/38 file after the snapshot 2023-10-06T04:00:03.890 INFO:tasks.workunit.client.0.smithi157.stdout: create 277/39 file after the snapshot 2023-10-06T04:00:03.890 INFO:tasks.workunit.client.0.smithi157.stdout: create 278/.snap/snap-subdir-test 2023-10-06T04:00:03.912 INFO:tasks.workunit.client.0.smithi157.stdout: create 278/30 file after the snapshot 2023-10-06T04:00:03.912 INFO:tasks.workunit.client.0.smithi157.stdout: create 278/31 file after the snapshot 2023-10-06T04:00:03.913 INFO:tasks.workunit.client.0.smithi157.stdout: create 278/32 file after the snapshot 2023-10-06T04:00:03.914 INFO:tasks.workunit.client.0.smithi157.stdout: create 278/33 file after the snapshot 2023-10-06T04:00:03.914 INFO:tasks.workunit.client.0.smithi157.stdout: create 278/34 file after the snapshot 2023-10-06T04:00:03.915 INFO:tasks.workunit.client.0.smithi157.stdout: create 278/35 file after the snapshot 2023-10-06T04:00:03.916 INFO:tasks.workunit.client.0.smithi157.stdout: create 278/36 file after the snapshot 2023-10-06T04:00:03.916 INFO:tasks.workunit.client.0.smithi157.stdout: create 278/37 file after the snapshot 2023-10-06T04:00:03.917 INFO:tasks.workunit.client.0.smithi157.stdout: create 278/38 file after the snapshot 2023-10-06T04:00:03.918 INFO:tasks.workunit.client.0.smithi157.stdout: create 278/39 file after the snapshot 2023-10-06T04:00:03.918 INFO:tasks.workunit.client.0.smithi157.stdout: create 279/.snap/snap-subdir-test 2023-10-06T04:00:03.936 INFO:tasks.workunit.client.0.smithi157.stdout: create 279/30 file after the snapshot 2023-10-06T04:00:03.937 INFO:tasks.workunit.client.0.smithi157.stdout: create 279/31 file after the snapshot 2023-10-06T04:00:03.938 INFO:tasks.workunit.client.0.smithi157.stdout: create 279/32 file after the snapshot 2023-10-06T04:00:03.938 INFO:tasks.workunit.client.0.smithi157.stdout: create 279/33 file after the snapshot 2023-10-06T04:00:03.939 INFO:tasks.workunit.client.0.smithi157.stdout: create 279/34 file after the snapshot 2023-10-06T04:00:03.940 INFO:tasks.workunit.client.0.smithi157.stdout: create 279/35 file after the snapshot 2023-10-06T04:00:03.941 INFO:tasks.workunit.client.0.smithi157.stdout: create 279/36 file after the snapshot 2023-10-06T04:00:03.941 INFO:tasks.workunit.client.0.smithi157.stdout: create 279/37 file after the snapshot 2023-10-06T04:00:03.942 INFO:tasks.workunit.client.0.smithi157.stdout: create 279/38 file after the snapshot 2023-10-06T04:00:03.942 INFO:tasks.workunit.client.0.smithi157.stdout: create 279/39 file after the snapshot 2023-10-06T04:00:03.943 INFO:tasks.workunit.client.0.smithi157.stdout: create 280/.snap/snap-subdir-test 2023-10-06T04:00:03.965 INFO:tasks.workunit.client.0.smithi157.stdout: create 280/30 file after the snapshot 2023-10-06T04:00:03.966 INFO:tasks.workunit.client.0.smithi157.stdout: create 280/31 file after the snapshot 2023-10-06T04:00:03.966 INFO:tasks.workunit.client.0.smithi157.stdout: create 280/32 file after the snapshot 2023-10-06T04:00:03.967 INFO:tasks.workunit.client.0.smithi157.stdout: create 280/33 file after the snapshot 2023-10-06T04:00:03.968 INFO:tasks.workunit.client.0.smithi157.stdout: create 280/34 file after the snapshot 2023-10-06T04:00:03.968 INFO:tasks.workunit.client.0.smithi157.stdout: create 280/35 file after the snapshot 2023-10-06T04:00:03.969 INFO:tasks.workunit.client.0.smithi157.stdout: create 280/36 file after the snapshot 2023-10-06T04:00:03.969 INFO:tasks.workunit.client.0.smithi157.stdout: create 280/37 file after the snapshot 2023-10-06T04:00:03.970 INFO:tasks.workunit.client.0.smithi157.stdout: create 280/38 file after the snapshot 2023-10-06T04:00:03.971 INFO:tasks.workunit.client.0.smithi157.stdout: create 280/39 file after the snapshot 2023-10-06T04:00:03.971 INFO:tasks.workunit.client.0.smithi157.stdout: create 281/.snap/snap-subdir-test 2023-10-06T04:00:03.996 INFO:tasks.workunit.client.0.smithi157.stdout: create 281/30 file after the snapshot 2023-10-06T04:00:03.997 INFO:tasks.workunit.client.0.smithi157.stdout: create 281/31 file after the snapshot 2023-10-06T04:00:03.998 INFO:tasks.workunit.client.0.smithi157.stdout: create 281/32 file after the snapshot 2023-10-06T04:00:03.999 INFO:tasks.workunit.client.0.smithi157.stdout: create 281/33 file after the snapshot 2023-10-06T04:00:03.999 INFO:tasks.workunit.client.0.smithi157.stdout: create 281/34 file after the snapshot 2023-10-06T04:00:04.000 INFO:tasks.workunit.client.0.smithi157.stdout: create 281/35 file after the snapshot 2023-10-06T04:00:04.001 INFO:tasks.workunit.client.0.smithi157.stdout: create 281/36 file after the snapshot 2023-10-06T04:00:04.002 INFO:tasks.workunit.client.0.smithi157.stdout: create 281/37 file after the snapshot 2023-10-06T04:00:04.003 INFO:tasks.workunit.client.0.smithi157.stdout: create 281/38 file after the snapshot 2023-10-06T04:00:04.004 INFO:tasks.workunit.client.0.smithi157.stdout: create 281/39 file after the snapshot 2023-10-06T04:00:04.005 INFO:tasks.workunit.client.0.smithi157.stdout: create 282/.snap/snap-subdir-test 2023-10-06T04:00:04.026 INFO:tasks.workunit.client.0.smithi157.stdout: create 282/30 file after the snapshot 2023-10-06T04:00:04.028 INFO:tasks.workunit.client.0.smithi157.stdout: create 282/31 file after the snapshot 2023-10-06T04:00:04.028 INFO:tasks.workunit.client.0.smithi157.stdout: create 282/32 file after the snapshot 2023-10-06T04:00:04.028 INFO:tasks.workunit.client.0.smithi157.stdout: create 282/33 file after the snapshot 2023-10-06T04:00:04.029 INFO:tasks.workunit.client.0.smithi157.stdout: create 282/34 file after the snapshot 2023-10-06T04:00:04.030 INFO:tasks.workunit.client.0.smithi157.stdout: create 282/35 file after the snapshot 2023-10-06T04:00:04.032 INFO:tasks.workunit.client.0.smithi157.stdout: create 282/36 file after the snapshot 2023-10-06T04:00:04.032 INFO:tasks.workunit.client.0.smithi157.stdout: create 282/37 file after the snapshot 2023-10-06T04:00:04.033 INFO:tasks.workunit.client.0.smithi157.stdout: create 282/38 file after the snapshot 2023-10-06T04:00:04.034 INFO:tasks.workunit.client.0.smithi157.stdout: create 282/39 file after the snapshot 2023-10-06T04:00:04.035 INFO:tasks.workunit.client.0.smithi157.stdout: create 283/.snap/snap-subdir-test 2023-10-06T04:00:04.054 INFO:tasks.workunit.client.0.smithi157.stdout: create 283/30 file after the snapshot 2023-10-06T04:00:04.055 INFO:tasks.workunit.client.0.smithi157.stdout: create 283/31 file after the snapshot 2023-10-06T04:00:04.056 INFO:tasks.workunit.client.0.smithi157.stdout: create 283/32 file after the snapshot 2023-10-06T04:00:04.057 INFO:tasks.workunit.client.0.smithi157.stdout: create 283/33 file after the snapshot 2023-10-06T04:00:04.058 INFO:tasks.workunit.client.0.smithi157.stdout: create 283/34 file after the snapshot 2023-10-06T04:00:04.058 INFO:tasks.workunit.client.0.smithi157.stdout: create 283/35 file after the snapshot 2023-10-06T04:00:04.059 INFO:tasks.workunit.client.0.smithi157.stdout: create 283/36 file after the snapshot 2023-10-06T04:00:04.852 INFO:tasks.workunit.client.0.smithi157.stdout: create 283/37 file after the snapshot 2023-10-06T04:00:04.853 INFO:tasks.workunit.client.0.smithi157.stdout: create 283/38 file after the snapshot 2023-10-06T04:00:04.854 INFO:tasks.workunit.client.0.smithi157.stdout: create 283/39 file after the snapshot 2023-10-06T04:00:04.855 INFO:tasks.workunit.client.0.smithi157.stdout: create 284/.snap/snap-subdir-test 2023-10-06T04:00:04.877 INFO:tasks.workunit.client.0.smithi157.stdout: create 284/30 file after the snapshot 2023-10-06T04:00:04.878 INFO:tasks.workunit.client.0.smithi157.stdout: create 284/31 file after the snapshot 2023-10-06T04:00:04.879 INFO:tasks.workunit.client.0.smithi157.stdout: create 284/32 file after the snapshot 2023-10-06T04:00:04.879 INFO:tasks.workunit.client.0.smithi157.stdout: create 284/33 file after the snapshot 2023-10-06T04:00:04.880 INFO:tasks.workunit.client.0.smithi157.stdout: create 284/34 file after the snapshot 2023-10-06T04:00:04.881 INFO:tasks.workunit.client.0.smithi157.stdout: create 284/35 file after the snapshot 2023-10-06T04:00:04.881 INFO:tasks.workunit.client.0.smithi157.stdout: create 284/36 file after the snapshot 2023-10-06T04:00:04.882 INFO:tasks.workunit.client.0.smithi157.stdout: create 284/37 file after the snapshot 2023-10-06T04:00:04.883 INFO:tasks.workunit.client.0.smithi157.stdout: create 284/38 file after the snapshot 2023-10-06T04:00:04.884 INFO:tasks.workunit.client.0.smithi157.stdout: create 284/39 file after the snapshot 2023-10-06T04:00:04.885 INFO:tasks.workunit.client.0.smithi157.stdout: create 285/.snap/snap-subdir-test 2023-10-06T04:00:04.909 INFO:tasks.workunit.client.0.smithi157.stdout: create 285/30 file after the snapshot 2023-10-06T04:00:04.910 INFO:tasks.workunit.client.0.smithi157.stdout: create 285/31 file after the snapshot 2023-10-06T04:00:04.911 INFO:tasks.workunit.client.0.smithi157.stdout: create 285/32 file after the snapshot 2023-10-06T04:00:04.912 INFO:tasks.workunit.client.0.smithi157.stdout: create 285/33 file after the snapshot 2023-10-06T04:00:04.914 INFO:tasks.workunit.client.0.smithi157.stdout: create 285/34 file after the snapshot 2023-10-06T04:00:04.915 INFO:tasks.workunit.client.0.smithi157.stdout: create 285/35 file after the snapshot 2023-10-06T04:00:04.916 INFO:tasks.workunit.client.0.smithi157.stdout: create 285/36 file after the snapshot 2023-10-06T04:00:04.917 INFO:tasks.workunit.client.0.smithi157.stdout: create 285/37 file after the snapshot 2023-10-06T04:00:04.919 INFO:tasks.workunit.client.0.smithi157.stdout: create 285/38 file after the snapshot 2023-10-06T04:00:04.920 INFO:tasks.workunit.client.0.smithi157.stdout: create 285/39 file after the snapshot 2023-10-06T04:00:04.921 INFO:tasks.workunit.client.0.smithi157.stdout: create 286/.snap/snap-subdir-test 2023-10-06T04:00:04.941 INFO:tasks.workunit.client.0.smithi157.stdout: create 286/30 file after the snapshot 2023-10-06T04:00:04.942 INFO:tasks.workunit.client.0.smithi157.stdout: create 286/31 file after the snapshot 2023-10-06T04:00:04.942 INFO:tasks.workunit.client.0.smithi157.stdout: create 286/32 file after the snapshot 2023-10-06T04:00:04.943 INFO:tasks.workunit.client.0.smithi157.stdout: create 286/33 file after the snapshot 2023-10-06T04:00:04.944 INFO:tasks.workunit.client.0.smithi157.stdout: create 286/34 file after the snapshot 2023-10-06T04:00:04.944 INFO:tasks.workunit.client.0.smithi157.stdout: create 286/35 file after the snapshot 2023-10-06T04:00:04.945 INFO:tasks.workunit.client.0.smithi157.stdout: create 286/36 file after the snapshot 2023-10-06T04:00:04.946 INFO:tasks.workunit.client.0.smithi157.stdout: create 286/37 file after the snapshot 2023-10-06T04:00:04.946 INFO:tasks.workunit.client.0.smithi157.stdout: create 286/38 file after the snapshot 2023-10-06T04:00:04.947 INFO:tasks.workunit.client.0.smithi157.stdout: create 286/39 file after the snapshot 2023-10-06T04:00:04.948 INFO:tasks.workunit.client.0.smithi157.stdout: create 287/.snap/snap-subdir-test 2023-10-06T04:00:04.966 INFO:tasks.workunit.client.0.smithi157.stdout: create 287/30 file after the snapshot 2023-10-06T04:00:04.967 INFO:tasks.workunit.client.0.smithi157.stdout: create 287/31 file after the snapshot 2023-10-06T04:00:04.968 INFO:tasks.workunit.client.0.smithi157.stdout: create 287/32 file after the snapshot 2023-10-06T04:00:04.968 INFO:tasks.workunit.client.0.smithi157.stdout: create 287/33 file after the snapshot 2023-10-06T04:00:04.969 INFO:tasks.workunit.client.0.smithi157.stdout: create 287/34 file after the snapshot 2023-10-06T04:00:04.970 INFO:tasks.workunit.client.0.smithi157.stdout: create 287/35 file after the snapshot 2023-10-06T04:00:04.971 INFO:tasks.workunit.client.0.smithi157.stdout: create 287/36 file after the snapshot 2023-10-06T04:00:04.971 INFO:tasks.workunit.client.0.smithi157.stdout: create 287/37 file after the snapshot 2023-10-06T04:00:04.972 INFO:tasks.workunit.client.0.smithi157.stdout: create 287/38 file after the snapshot 2023-10-06T04:00:04.972 INFO:tasks.workunit.client.0.smithi157.stdout: create 287/39 file after the snapshot 2023-10-06T04:00:04.973 INFO:tasks.workunit.client.0.smithi157.stdout: create 288/.snap/snap-subdir-test 2023-10-06T04:00:05.000 INFO:tasks.workunit.client.0.smithi157.stdout: create 288/30 file after the snapshot 2023-10-06T04:00:05.001 INFO:tasks.workunit.client.0.smithi157.stdout: create 288/31 file after the snapshot 2023-10-06T04:00:05.002 INFO:tasks.workunit.client.0.smithi157.stdout: create 288/32 file after the snapshot 2023-10-06T04:00:05.003 INFO:tasks.workunit.client.0.smithi157.stdout: create 288/33 file after the snapshot 2023-10-06T04:00:05.004 INFO:tasks.workunit.client.0.smithi157.stdout: create 288/34 file after the snapshot 2023-10-06T04:00:05.006 INFO:tasks.workunit.client.0.smithi157.stdout: create 288/35 file after the snapshot 2023-10-06T04:00:05.007 INFO:tasks.workunit.client.0.smithi157.stdout: create 288/36 file after the snapshot 2023-10-06T04:00:05.008 INFO:tasks.workunit.client.0.smithi157.stdout: create 288/37 file after the snapshot 2023-10-06T04:00:05.009 INFO:tasks.workunit.client.0.smithi157.stdout: create 288/38 file after the snapshot 2023-10-06T04:00:05.010 INFO:tasks.workunit.client.0.smithi157.stdout: create 288/39 file after the snapshot 2023-10-06T04:00:05.011 INFO:tasks.workunit.client.0.smithi157.stdout: create 289/.snap/snap-subdir-test 2023-10-06T04:00:05.030 INFO:tasks.workunit.client.0.smithi157.stdout: create 289/30 file after the snapshot 2023-10-06T04:00:05.031 INFO:tasks.workunit.client.0.smithi157.stdout: create 289/31 file after the snapshot 2023-10-06T04:00:05.032 INFO:tasks.workunit.client.0.smithi157.stdout: create 289/32 file after the snapshot 2023-10-06T04:00:05.033 INFO:tasks.workunit.client.0.smithi157.stdout: create 289/33 file after the snapshot 2023-10-06T04:00:05.034 INFO:tasks.workunit.client.0.smithi157.stdout: create 289/34 file after the snapshot 2023-10-06T04:00:05.034 INFO:tasks.workunit.client.0.smithi157.stdout: create 289/35 file after the snapshot 2023-10-06T04:00:05.035 INFO:tasks.workunit.client.0.smithi157.stdout: create 289/36 file after the snapshot 2023-10-06T04:00:05.036 INFO:tasks.workunit.client.0.smithi157.stdout: create 289/37 file after the snapshot 2023-10-06T04:00:05.037 INFO:tasks.workunit.client.0.smithi157.stdout: create 289/38 file after the snapshot 2023-10-06T04:00:05.038 INFO:tasks.workunit.client.0.smithi157.stdout: create 289/39 file after the snapshot 2023-10-06T04:00:05.039 INFO:tasks.workunit.client.0.smithi157.stdout: create 290/.snap/snap-subdir-test 2023-10-06T04:00:05.059 INFO:tasks.workunit.client.0.smithi157.stdout: create 290/30 file after the snapshot 2023-10-06T04:00:05.060 INFO:tasks.workunit.client.0.smithi157.stdout: create 290/31 file after the snapshot 2023-10-06T04:00:05.061 INFO:tasks.workunit.client.0.smithi157.stdout: create 290/32 file after the snapshot 2023-10-06T04:00:05.062 INFO:tasks.workunit.client.0.smithi157.stdout: create 290/33 file after the snapshot 2023-10-06T04:00:05.063 INFO:tasks.workunit.client.0.smithi157.stdout: create 290/34 file after the snapshot 2023-10-06T04:00:05.064 INFO:tasks.workunit.client.0.smithi157.stdout: create 290/35 file after the snapshot 2023-10-06T04:00:05.065 INFO:tasks.workunit.client.0.smithi157.stdout: create 290/36 file after the snapshot 2023-10-06T04:00:05.066 INFO:tasks.workunit.client.0.smithi157.stdout: create 290/37 file after the snapshot 2023-10-06T04:00:05.067 INFO:tasks.workunit.client.0.smithi157.stdout: create 290/38 file after the snapshot 2023-10-06T04:00:05.068 INFO:tasks.workunit.client.0.smithi157.stdout: create 290/39 file after the snapshot 2023-10-06T04:00:05.069 INFO:tasks.workunit.client.0.smithi157.stdout: create 291/.snap/snap-subdir-test 2023-10-06T04:00:05.310 INFO:tasks.workunit.client.0.smithi157.stdout: create 291/30 file after the snapshot 2023-10-06T04:00:05.311 INFO:tasks.workunit.client.0.smithi157.stdout: create 291/31 file after the snapshot 2023-10-06T04:00:05.312 INFO:tasks.workunit.client.0.smithi157.stdout: create 291/32 file after the snapshot 2023-10-06T04:00:05.314 INFO:tasks.workunit.client.0.smithi157.stdout: create 291/33 file after the snapshot 2023-10-06T04:00:05.315 INFO:tasks.workunit.client.0.smithi157.stdout: create 291/34 file after the snapshot 2023-10-06T04:00:06.093 INFO:tasks.workunit.client.0.smithi157.stdout: create 291/35 file after the snapshot 2023-10-06T04:00:06.094 INFO:tasks.workunit.client.0.smithi157.stdout: create 291/36 file after the snapshot 2023-10-06T04:00:06.095 INFO:tasks.workunit.client.0.smithi157.stdout: create 291/37 file after the snapshot 2023-10-06T04:00:06.096 INFO:tasks.workunit.client.0.smithi157.stdout: create 291/38 file after the snapshot 2023-10-06T04:00:06.098 INFO:tasks.workunit.client.0.smithi157.stdout: create 291/39 file after the snapshot 2023-10-06T04:00:06.099 INFO:tasks.workunit.client.0.smithi157.stdout: create 292/.snap/snap-subdir-test 2023-10-06T04:00:06.498 INFO:tasks.workunit.client.0.smithi157.stdout: create 292/30 file after the snapshot 2023-10-06T04:00:06.499 INFO:tasks.workunit.client.0.smithi157.stdout: create 292/31 file after the snapshot 2023-10-06T04:00:06.500 INFO:tasks.workunit.client.0.smithi157.stdout: create 292/32 file after the snapshot 2023-10-06T04:00:06.502 INFO:tasks.workunit.client.0.smithi157.stdout: create 292/33 file after the snapshot 2023-10-06T04:00:06.503 INFO:tasks.workunit.client.0.smithi157.stdout: create 292/34 file after the snapshot 2023-10-06T04:00:06.504 INFO:tasks.workunit.client.0.smithi157.stdout: create 292/35 file after the snapshot 2023-10-06T04:00:06.505 INFO:tasks.workunit.client.0.smithi157.stdout: create 292/36 file after the snapshot 2023-10-06T04:00:06.505 INFO:tasks.workunit.client.0.smithi157.stdout: create 292/37 file after the snapshot 2023-10-06T04:00:06.507 INFO:tasks.workunit.client.0.smithi157.stdout: create 292/38 file after the snapshot 2023-10-06T04:00:06.508 INFO:tasks.workunit.client.0.smithi157.stdout: create 292/39 file after the snapshot 2023-10-06T04:00:06.509 INFO:tasks.workunit.client.0.smithi157.stdout: create 293/.snap/snap-subdir-test 2023-10-06T04:00:06.536 INFO:tasks.workunit.client.0.smithi157.stdout: create 293/30 file after the snapshot 2023-10-06T04:00:06.537 INFO:tasks.workunit.client.0.smithi157.stdout: create 293/31 file after the snapshot 2023-10-06T04:00:06.538 INFO:tasks.workunit.client.0.smithi157.stdout: create 293/32 file after the snapshot 2023-10-06T04:00:06.539 INFO:tasks.workunit.client.0.smithi157.stdout: create 293/33 file after the snapshot 2023-10-06T04:00:06.540 INFO:tasks.workunit.client.0.smithi157.stdout: create 293/34 file after the snapshot 2023-10-06T04:00:06.541 INFO:tasks.workunit.client.0.smithi157.stdout: create 293/35 file after the snapshot 2023-10-06T04:00:06.542 INFO:tasks.workunit.client.0.smithi157.stdout: create 293/36 file after the snapshot 2023-10-06T04:00:06.543 INFO:tasks.workunit.client.0.smithi157.stdout: create 293/37 file after the snapshot 2023-10-06T04:00:06.544 INFO:tasks.workunit.client.0.smithi157.stdout: create 293/38 file after the snapshot 2023-10-06T04:00:06.544 INFO:tasks.workunit.client.0.smithi157.stdout: create 293/39 file after the snapshot 2023-10-06T04:00:06.545 INFO:tasks.workunit.client.0.smithi157.stdout: create 294/.snap/snap-subdir-test 2023-10-06T04:00:06.569 INFO:tasks.workunit.client.0.smithi157.stdout: create 294/30 file after the snapshot 2023-10-06T04:00:06.570 INFO:tasks.workunit.client.0.smithi157.stdout: create 294/31 file after the snapshot 2023-10-06T04:00:06.571 INFO:tasks.workunit.client.0.smithi157.stdout: create 294/32 file after the snapshot 2023-10-06T04:00:06.572 INFO:tasks.workunit.client.0.smithi157.stdout: create 294/33 file after the snapshot 2023-10-06T04:00:06.573 INFO:tasks.workunit.client.0.smithi157.stdout: create 294/34 file after the snapshot 2023-10-06T04:00:06.573 INFO:tasks.workunit.client.0.smithi157.stdout: create 294/35 file after the snapshot 2023-10-06T04:00:06.574 INFO:tasks.workunit.client.0.smithi157.stdout: create 294/36 file after the snapshot 2023-10-06T04:00:06.575 INFO:tasks.workunit.client.0.smithi157.stdout: create 294/37 file after the snapshot 2023-10-06T04:00:06.575 INFO:tasks.workunit.client.0.smithi157.stdout: create 294/38 file after the snapshot 2023-10-06T04:00:06.576 INFO:tasks.workunit.client.0.smithi157.stdout: create 294/39 file after the snapshot 2023-10-06T04:00:06.577 INFO:tasks.workunit.client.0.smithi157.stdout: create 295/.snap/snap-subdir-test 2023-10-06T04:00:07.536 INFO:tasks.workunit.client.0.smithi157.stdout: create 295/30 file after the snapshot 2023-10-06T04:00:07.803 INFO:tasks.workunit.client.0.smithi157.stdout: create 295/31 file after the snapshot 2023-10-06T04:00:07.804 INFO:tasks.workunit.client.0.smithi157.stdout: create 295/32 file after the snapshot 2023-10-06T04:00:07.804 INFO:tasks.workunit.client.0.smithi157.stdout: create 295/33 file after the snapshot 2023-10-06T04:00:07.805 INFO:tasks.workunit.client.0.smithi157.stdout: create 295/34 file after the snapshot 2023-10-06T04:00:07.806 INFO:tasks.workunit.client.0.smithi157.stdout: create 295/35 file after the snapshot 2023-10-06T04:00:07.807 INFO:tasks.workunit.client.0.smithi157.stdout: create 295/36 file after the snapshot 2023-10-06T04:00:07.807 INFO:tasks.workunit.client.0.smithi157.stdout: create 295/37 file after the snapshot 2023-10-06T04:00:07.808 INFO:tasks.workunit.client.0.smithi157.stdout: create 295/38 file after the snapshot 2023-10-06T04:00:07.809 INFO:tasks.workunit.client.0.smithi157.stdout: create 295/39 file after the snapshot 2023-10-06T04:00:07.809 INFO:tasks.workunit.client.0.smithi157.stdout: create 296/.snap/snap-subdir-test 2023-10-06T04:00:07.839 INFO:tasks.workunit.client.0.smithi157.stdout: create 296/30 file after the snapshot 2023-10-06T04:00:07.841 INFO:tasks.workunit.client.0.smithi157.stdout: create 296/31 file after the snapshot 2023-10-06T04:00:07.842 INFO:tasks.workunit.client.0.smithi157.stdout: create 296/32 file after the snapshot 2023-10-06T04:00:07.843 INFO:tasks.workunit.client.0.smithi157.stdout: create 296/33 file after the snapshot 2023-10-06T04:00:07.843 INFO:tasks.workunit.client.0.smithi157.stdout: create 296/34 file after the snapshot 2023-10-06T04:00:07.844 INFO:tasks.workunit.client.0.smithi157.stdout: create 296/35 file after the snapshot 2023-10-06T04:00:07.845 INFO:tasks.workunit.client.0.smithi157.stdout: create 296/36 file after the snapshot 2023-10-06T04:00:07.846 INFO:tasks.workunit.client.0.smithi157.stdout: create 296/37 file after the snapshot 2023-10-06T04:00:07.846 INFO:tasks.workunit.client.0.smithi157.stdout: create 296/38 file after the snapshot 2023-10-06T04:00:07.847 INFO:tasks.workunit.client.0.smithi157.stdout: create 296/39 file after the snapshot 2023-10-06T04:00:07.848 INFO:tasks.workunit.client.0.smithi157.stdout: create 297/.snap/snap-subdir-test 2023-10-06T04:00:07.873 INFO:tasks.workunit.client.0.smithi157.stdout: create 297/30 file after the snapshot 2023-10-06T04:00:07.875 INFO:tasks.workunit.client.0.smithi157.stdout: create 297/31 file after the snapshot 2023-10-06T04:00:07.876 INFO:tasks.workunit.client.0.smithi157.stdout: create 297/32 file after the snapshot 2023-10-06T04:00:07.877 INFO:tasks.workunit.client.0.smithi157.stdout: create 297/33 file after the snapshot 2023-10-06T04:00:07.878 INFO:tasks.workunit.client.0.smithi157.stdout: create 297/34 file after the snapshot 2023-10-06T04:00:07.879 INFO:tasks.workunit.client.0.smithi157.stdout: create 297/35 file after the snapshot 2023-10-06T04:00:07.880 INFO:tasks.workunit.client.0.smithi157.stdout: create 297/36 file after the snapshot 2023-10-06T04:00:07.881 INFO:tasks.workunit.client.0.smithi157.stdout: create 297/37 file after the snapshot 2023-10-06T04:00:07.882 INFO:tasks.workunit.client.0.smithi157.stdout: create 297/38 file after the snapshot 2023-10-06T04:00:07.883 INFO:tasks.workunit.client.0.smithi157.stdout: create 297/39 file after the snapshot 2023-10-06T04:00:07.884 INFO:tasks.workunit.client.0.smithi157.stdout: create 298/.snap/snap-subdir-test 2023-10-06T04:00:07.914 INFO:tasks.workunit.client.0.smithi157.stdout: create 298/30 file after the snapshot 2023-10-06T04:00:07.914 INFO:tasks.workunit.client.0.smithi157.stdout: create 298/31 file after the snapshot 2023-10-06T04:00:07.914 INFO:tasks.workunit.client.0.smithi157.stdout: create 298/32 file after the snapshot 2023-10-06T04:00:07.914 INFO:tasks.workunit.client.0.smithi157.stdout: create 298/33 file after the snapshot 2023-10-06T04:00:07.915 INFO:tasks.workunit.client.0.smithi157.stdout: create 298/34 file after the snapshot 2023-10-06T04:00:07.916 INFO:tasks.workunit.client.0.smithi157.stdout: create 298/35 file after the snapshot 2023-10-06T04:00:07.916 INFO:tasks.workunit.client.0.smithi157.stdout: create 298/36 file after the snapshot 2023-10-06T04:00:07.918 INFO:tasks.workunit.client.0.smithi157.stdout: create 298/37 file after the snapshot 2023-10-06T04:00:07.919 INFO:tasks.workunit.client.0.smithi157.stdout: create 298/38 file after the snapshot 2023-10-06T04:00:07.919 INFO:tasks.workunit.client.0.smithi157.stdout: create 298/39 file after the snapshot 2023-10-06T04:00:07.920 INFO:tasks.workunit.client.0.smithi157.stdout: create 299/.snap/snap-subdir-test 2023-10-06T04:00:07.947 INFO:tasks.workunit.client.0.smithi157.stdout: create 299/30 file after the snapshot 2023-10-06T04:00:07.948 INFO:tasks.workunit.client.0.smithi157.stdout: create 299/31 file after the snapshot 2023-10-06T04:00:07.949 INFO:tasks.workunit.client.0.smithi157.stdout: create 299/32 file after the snapshot 2023-10-06T04:00:07.950 INFO:tasks.workunit.client.0.smithi157.stdout: create 299/33 file after the snapshot 2023-10-06T04:00:07.951 INFO:tasks.workunit.client.0.smithi157.stdout: create 299/34 file after the snapshot 2023-10-06T04:00:07.952 INFO:tasks.workunit.client.0.smithi157.stdout: create 299/35 file after the snapshot 2023-10-06T04:00:07.952 INFO:tasks.workunit.client.0.smithi157.stdout: create 299/36 file after the snapshot 2023-10-06T04:00:07.953 INFO:tasks.workunit.client.0.smithi157.stdout: create 299/37 file after the snapshot 2023-10-06T04:00:07.954 INFO:tasks.workunit.client.0.smithi157.stdout: create 299/38 file after the snapshot 2023-10-06T04:00:07.955 INFO:tasks.workunit.client.0.smithi157.stdout: create 299/39 file after the snapshot 2023-10-06T04:00:07.956 INFO:tasks.workunit.client.0.smithi157.stdout:Take second snapshot .snap/test2 2023-10-06T04:00:07.987 INFO:tasks.workunit.client.0.smithi157.stdout:Copy content of .snap/test1 to copyofsnap1 ... 2023-10-06T04:00:09.341 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1' -> 'copyofsnap1/test1' 2023-10-06T04:00:09.342 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/100' -> 'copyofsnap1/test1/100' 2023-10-06T04:00:09.342 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/100/10' -> 'copyofsnap1/test1/100/10' 2023-10-06T04:00:09.342 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/100/11' -> 'copyofsnap1/test1/100/11' 2023-10-06T04:00:09.342 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/100/12' -> 'copyofsnap1/test1/100/12' 2023-10-06T04:00:09.342 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/100/13' -> 'copyofsnap1/test1/100/13' 2023-10-06T04:00:09.342 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/100/14' -> 'copyofsnap1/test1/100/14' 2023-10-06T04:00:09.342 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/100/15' -> 'copyofsnap1/test1/100/15' 2023-10-06T04:00:09.343 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/100/16' -> 'copyofsnap1/test1/100/16' 2023-10-06T04:00:09.343 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/100/17' -> 'copyofsnap1/test1/100/17' 2023-10-06T04:00:09.343 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/100/18' -> 'copyofsnap1/test1/100/18' 2023-10-06T04:00:09.343 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/100/19' -> 'copyofsnap1/test1/100/19' 2023-10-06T04:00:09.343 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/100/20' -> 'copyofsnap1/test1/100/20' 2023-10-06T04:00:09.343 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/101' -> 'copyofsnap1/test1/101' 2023-10-06T04:00:09.343 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/101/10' -> 'copyofsnap1/test1/101/10' 2023-10-06T04:00:09.343 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/101/11' -> 'copyofsnap1/test1/101/11' 2023-10-06T04:00:09.343 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/101/12' -> 'copyofsnap1/test1/101/12' 2023-10-06T04:00:09.343 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/101/13' -> 'copyofsnap1/test1/101/13' 2023-10-06T04:00:09.343 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/101/14' -> 'copyofsnap1/test1/101/14' 2023-10-06T04:00:09.344 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/101/15' -> 'copyofsnap1/test1/101/15' 2023-10-06T04:00:09.344 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/101/16' -> 'copyofsnap1/test1/101/16' 2023-10-06T04:00:09.344 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/101/17' -> 'copyofsnap1/test1/101/17' 2023-10-06T04:00:09.344 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/101/18' -> 'copyofsnap1/test1/101/18' 2023-10-06T04:00:09.344 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/101/19' -> 'copyofsnap1/test1/101/19' 2023-10-06T04:00:09.344 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/101/20' -> 'copyofsnap1/test1/101/20' 2023-10-06T04:00:09.344 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/102' -> 'copyofsnap1/test1/102' 2023-10-06T04:00:09.344 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/102/10' -> 'copyofsnap1/test1/102/10' 2023-10-06T04:00:09.344 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/102/11' -> 'copyofsnap1/test1/102/11' 2023-10-06T04:00:09.344 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/102/12' -> 'copyofsnap1/test1/102/12' 2023-10-06T04:00:09.344 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/102/13' -> 'copyofsnap1/test1/102/13' 2023-10-06T04:00:09.344 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/102/14' -> 'copyofsnap1/test1/102/14' 2023-10-06T04:00:09.345 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/102/15' -> 'copyofsnap1/test1/102/15' 2023-10-06T04:00:09.345 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/102/16' -> 'copyofsnap1/test1/102/16' 2023-10-06T04:00:09.345 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/102/17' -> 'copyofsnap1/test1/102/17' 2023-10-06T04:00:09.345 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/102/18' -> 'copyofsnap1/test1/102/18' 2023-10-06T04:00:09.345 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/102/19' -> 'copyofsnap1/test1/102/19' 2023-10-06T04:00:09.345 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/102/20' -> 'copyofsnap1/test1/102/20' 2023-10-06T04:00:09.345 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/103' -> 'copyofsnap1/test1/103' 2023-10-06T04:00:09.345 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/103/10' -> 'copyofsnap1/test1/103/10' 2023-10-06T04:00:09.345 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/103/11' -> 'copyofsnap1/test1/103/11' 2023-10-06T04:00:09.345 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/103/12' -> 'copyofsnap1/test1/103/12' 2023-10-06T04:00:09.345 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/103/13' -> 'copyofsnap1/test1/103/13' 2023-10-06T04:00:09.345 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/103/14' -> 'copyofsnap1/test1/103/14' 2023-10-06T04:00:09.346 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/103/15' -> 'copyofsnap1/test1/103/15' 2023-10-06T04:00:09.346 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/103/16' -> 'copyofsnap1/test1/103/16' 2023-10-06T04:00:09.346 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/103/17' -> 'copyofsnap1/test1/103/17' 2023-10-06T04:00:09.346 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/103/18' -> 'copyofsnap1/test1/103/18' 2023-10-06T04:00:09.346 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/103/19' -> 'copyofsnap1/test1/103/19' 2023-10-06T04:00:09.346 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/103/20' -> 'copyofsnap1/test1/103/20' 2023-10-06T04:00:09.346 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/104' -> 'copyofsnap1/test1/104' 2023-10-06T04:00:09.346 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/104/10' -> 'copyofsnap1/test1/104/10' 2023-10-06T04:00:09.346 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/104/11' -> 'copyofsnap1/test1/104/11' 2023-10-06T04:00:09.346 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/104/12' -> 'copyofsnap1/test1/104/12' 2023-10-06T04:00:09.346 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/104/13' -> 'copyofsnap1/test1/104/13' 2023-10-06T04:00:09.346 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/104/14' -> 'copyofsnap1/test1/104/14' 2023-10-06T04:00:09.347 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/104/15' -> 'copyofsnap1/test1/104/15' 2023-10-06T04:00:09.347 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/104/16' -> 'copyofsnap1/test1/104/16' 2023-10-06T04:00:09.347 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/104/17' -> 'copyofsnap1/test1/104/17' 2023-10-06T04:00:09.347 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/104/18' -> 'copyofsnap1/test1/104/18' 2023-10-06T04:00:09.347 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/104/19' -> 'copyofsnap1/test1/104/19' 2023-10-06T04:00:09.347 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/104/20' -> 'copyofsnap1/test1/104/20' 2023-10-06T04:00:09.347 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/105' -> 'copyofsnap1/test1/105' 2023-10-06T04:00:09.347 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/105/10' -> 'copyofsnap1/test1/105/10' 2023-10-06T04:00:09.347 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/105/11' -> 'copyofsnap1/test1/105/11' 2023-10-06T04:00:09.347 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/105/12' -> 'copyofsnap1/test1/105/12' 2023-10-06T04:00:09.347 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/105/13' -> 'copyofsnap1/test1/105/13' 2023-10-06T04:00:09.347 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/105/14' -> 'copyofsnap1/test1/105/14' 2023-10-06T04:00:09.348 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/105/15' -> 'copyofsnap1/test1/105/15' 2023-10-06T04:00:09.348 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/105/16' -> 'copyofsnap1/test1/105/16' 2023-10-06T04:00:09.348 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/105/17' -> 'copyofsnap1/test1/105/17' 2023-10-06T04:00:09.348 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/105/18' -> 'copyofsnap1/test1/105/18' 2023-10-06T04:00:09.348 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/105/19' -> 'copyofsnap1/test1/105/19' 2023-10-06T04:00:09.348 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/105/20' -> 'copyofsnap1/test1/105/20' 2023-10-06T04:00:09.348 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/106' -> 'copyofsnap1/test1/106' 2023-10-06T04:00:09.348 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/106/10' -> 'copyofsnap1/test1/106/10' 2023-10-06T04:00:09.348 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/106/11' -> 'copyofsnap1/test1/106/11' 2023-10-06T04:00:09.348 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/106/12' -> 'copyofsnap1/test1/106/12' 2023-10-06T04:00:09.348 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/106/13' -> 'copyofsnap1/test1/106/13' 2023-10-06T04:00:09.348 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/106/14' -> 'copyofsnap1/test1/106/14' 2023-10-06T04:00:09.349 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/106/15' -> 'copyofsnap1/test1/106/15' 2023-10-06T04:00:09.349 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/106/16' -> 'copyofsnap1/test1/106/16' 2023-10-06T04:00:11.589 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/106/17' -> 'copyofsnap1/test1/106/17' 2023-10-06T04:00:11.589 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/106/18' -> 'copyofsnap1/test1/106/18' 2023-10-06T04:00:11.589 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/106/19' -> 'copyofsnap1/test1/106/19' 2023-10-06T04:00:11.589 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/106/20' -> 'copyofsnap1/test1/106/20' 2023-10-06T04:00:11.589 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/107' -> 'copyofsnap1/test1/107' 2023-10-06T04:00:11.589 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/107/10' -> 'copyofsnap1/test1/107/10' 2023-10-06T04:00:11.589 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/107/11' -> 'copyofsnap1/test1/107/11' 2023-10-06T04:00:11.589 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/107/12' -> 'copyofsnap1/test1/107/12' 2023-10-06T04:00:11.589 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/107/13' -> 'copyofsnap1/test1/107/13' 2023-10-06T04:00:11.589 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/107/14' -> 'copyofsnap1/test1/107/14' 2023-10-06T04:00:11.589 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/107/15' -> 'copyofsnap1/test1/107/15' 2023-10-06T04:00:11.590 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/107/16' -> 'copyofsnap1/test1/107/16' 2023-10-06T04:00:11.590 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/107/17' -> 'copyofsnap1/test1/107/17' 2023-10-06T04:00:11.590 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/107/18' -> 'copyofsnap1/test1/107/18' 2023-10-06T04:00:11.590 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/107/19' -> 'copyofsnap1/test1/107/19' 2023-10-06T04:00:11.590 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/107/20' -> 'copyofsnap1/test1/107/20' 2023-10-06T04:00:11.590 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/108' -> 'copyofsnap1/test1/108' 2023-10-06T04:00:11.590 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/108/10' -> 'copyofsnap1/test1/108/10' 2023-10-06T04:00:11.590 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/108/11' -> 'copyofsnap1/test1/108/11' 2023-10-06T04:00:11.590 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/108/12' -> 'copyofsnap1/test1/108/12' 2023-10-06T04:00:11.590 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/108/13' -> 'copyofsnap1/test1/108/13' 2023-10-06T04:00:11.590 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/108/14' -> 'copyofsnap1/test1/108/14' 2023-10-06T04:00:11.591 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/108/15' -> 'copyofsnap1/test1/108/15' 2023-10-06T04:00:11.591 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/108/16' -> 'copyofsnap1/test1/108/16' 2023-10-06T04:00:11.591 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/108/17' -> 'copyofsnap1/test1/108/17' 2023-10-06T04:00:11.591 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/108/18' -> 'copyofsnap1/test1/108/18' 2023-10-06T04:00:11.591 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/108/19' -> 'copyofsnap1/test1/108/19' 2023-10-06T04:00:11.591 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/108/20' -> 'copyofsnap1/test1/108/20' 2023-10-06T04:00:11.591 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/109' -> 'copyofsnap1/test1/109' 2023-10-06T04:00:11.591 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/109/10' -> 'copyofsnap1/test1/109/10' 2023-10-06T04:00:11.591 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/109/11' -> 'copyofsnap1/test1/109/11' 2023-10-06T04:00:11.591 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/109/12' -> 'copyofsnap1/test1/109/12' 2023-10-06T04:00:11.593 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/109/13' -> 'copyofsnap1/test1/109/13' 2023-10-06T04:00:11.594 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/109/14' -> 'copyofsnap1/test1/109/14' 2023-10-06T04:00:11.594 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/109/15' -> 'copyofsnap1/test1/109/15' 2023-10-06T04:00:11.594 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/109/16' -> 'copyofsnap1/test1/109/16' 2023-10-06T04:00:11.594 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/109/17' -> 'copyofsnap1/test1/109/17' 2023-10-06T04:00:11.594 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/109/18' -> 'copyofsnap1/test1/109/18' 2023-10-06T04:00:11.594 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/109/19' -> 'copyofsnap1/test1/109/19' 2023-10-06T04:00:11.594 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/109/20' -> 'copyofsnap1/test1/109/20' 2023-10-06T04:00:11.594 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/110' -> 'copyofsnap1/test1/110' 2023-10-06T04:00:11.594 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/110/10' -> 'copyofsnap1/test1/110/10' 2023-10-06T04:00:11.594 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/110/11' -> 'copyofsnap1/test1/110/11' 2023-10-06T04:00:11.594 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/110/12' -> 'copyofsnap1/test1/110/12' 2023-10-06T04:00:11.594 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/110/13' -> 'copyofsnap1/test1/110/13' 2023-10-06T04:00:11.595 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/110/14' -> 'copyofsnap1/test1/110/14' 2023-10-06T04:00:11.595 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/110/15' -> 'copyofsnap1/test1/110/15' 2023-10-06T04:00:11.595 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/110/16' -> 'copyofsnap1/test1/110/16' 2023-10-06T04:00:11.595 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/110/17' -> 'copyofsnap1/test1/110/17' 2023-10-06T04:00:11.595 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/110/18' -> 'copyofsnap1/test1/110/18' 2023-10-06T04:00:11.595 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/110/19' -> 'copyofsnap1/test1/110/19' 2023-10-06T04:00:11.595 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/110/20' -> 'copyofsnap1/test1/110/20' 2023-10-06T04:00:11.595 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/111' -> 'copyofsnap1/test1/111' 2023-10-06T04:00:11.595 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/111/10' -> 'copyofsnap1/test1/111/10' 2023-10-06T04:00:11.595 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/111/11' -> 'copyofsnap1/test1/111/11' 2023-10-06T04:00:11.595 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/111/12' -> 'copyofsnap1/test1/111/12' 2023-10-06T04:00:11.595 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/111/13' -> 'copyofsnap1/test1/111/13' 2023-10-06T04:00:11.596 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/111/14' -> 'copyofsnap1/test1/111/14' 2023-10-06T04:00:11.596 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/111/15' -> 'copyofsnap1/test1/111/15' 2023-10-06T04:00:11.596 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/111/16' -> 'copyofsnap1/test1/111/16' 2023-10-06T04:00:11.596 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/111/17' -> 'copyofsnap1/test1/111/17' 2023-10-06T04:00:11.596 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/111/18' -> 'copyofsnap1/test1/111/18' 2023-10-06T04:00:11.596 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/111/19' -> 'copyofsnap1/test1/111/19' 2023-10-06T04:00:11.596 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/111/20' -> 'copyofsnap1/test1/111/20' 2023-10-06T04:00:11.596 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/112' -> 'copyofsnap1/test1/112' 2023-10-06T04:00:11.596 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/112/10' -> 'copyofsnap1/test1/112/10' 2023-10-06T04:00:11.596 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/112/11' -> 'copyofsnap1/test1/112/11' 2023-10-06T04:00:11.596 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/112/12' -> 'copyofsnap1/test1/112/12' 2023-10-06T04:00:11.596 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/112/13' -> 'copyofsnap1/test1/112/13' 2023-10-06T04:00:11.596 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/112/14' -> 'copyofsnap1/test1/112/14' 2023-10-06T04:00:11.597 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/112/15' -> 'copyofsnap1/test1/112/15' 2023-10-06T04:00:11.597 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/112/16' -> 'copyofsnap1/test1/112/16' 2023-10-06T04:00:11.597 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/112/17' -> 'copyofsnap1/test1/112/17' 2023-10-06T04:00:11.597 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/112/18' -> 'copyofsnap1/test1/112/18' 2023-10-06T04:00:11.597 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/112/19' -> 'copyofsnap1/test1/112/19' 2023-10-06T04:00:11.597 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/112/20' -> 'copyofsnap1/test1/112/20' 2023-10-06T04:00:11.597 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/113' -> 'copyofsnap1/test1/113' 2023-10-06T04:00:11.597 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/113/10' -> 'copyofsnap1/test1/113/10' 2023-10-06T04:00:11.597 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/113/11' -> 'copyofsnap1/test1/113/11' 2023-10-06T04:00:11.597 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/113/12' -> 'copyofsnap1/test1/113/12' 2023-10-06T04:00:11.597 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/113/13' -> 'copyofsnap1/test1/113/13' 2023-10-06T04:00:12.190 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/113/14' -> 'copyofsnap1/test1/113/14' 2023-10-06T04:00:12.191 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/113/15' -> 'copyofsnap1/test1/113/15' 2023-10-06T04:00:12.191 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/113/16' -> 'copyofsnap1/test1/113/16' 2023-10-06T04:00:12.191 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/113/17' -> 'copyofsnap1/test1/113/17' 2023-10-06T04:00:12.191 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/113/18' -> 'copyofsnap1/test1/113/18' 2023-10-06T04:00:12.191 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/113/19' -> 'copyofsnap1/test1/113/19' 2023-10-06T04:00:12.191 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/113/20' -> 'copyofsnap1/test1/113/20' 2023-10-06T04:00:12.191 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/114' -> 'copyofsnap1/test1/114' 2023-10-06T04:00:12.191 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/114/10' -> 'copyofsnap1/test1/114/10' 2023-10-06T04:00:12.191 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/114/11' -> 'copyofsnap1/test1/114/11' 2023-10-06T04:00:12.192 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/114/12' -> 'copyofsnap1/test1/114/12' 2023-10-06T04:00:12.192 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/114/13' -> 'copyofsnap1/test1/114/13' 2023-10-06T04:00:12.192 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/114/14' -> 'copyofsnap1/test1/114/14' 2023-10-06T04:00:12.192 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/114/15' -> 'copyofsnap1/test1/114/15' 2023-10-06T04:00:12.192 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/114/16' -> 'copyofsnap1/test1/114/16' 2023-10-06T04:00:12.192 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/114/17' -> 'copyofsnap1/test1/114/17' 2023-10-06T04:00:12.192 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/114/18' -> 'copyofsnap1/test1/114/18' 2023-10-06T04:00:12.192 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/114/19' -> 'copyofsnap1/test1/114/19' 2023-10-06T04:00:12.192 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/114/20' -> 'copyofsnap1/test1/114/20' 2023-10-06T04:00:12.192 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/115' -> 'copyofsnap1/test1/115' 2023-10-06T04:00:12.192 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/115/10' -> 'copyofsnap1/test1/115/10' 2023-10-06T04:00:12.193 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/115/11' -> 'copyofsnap1/test1/115/11' 2023-10-06T04:00:12.193 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/115/12' -> 'copyofsnap1/test1/115/12' 2023-10-06T04:00:12.193 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/115/13' -> 'copyofsnap1/test1/115/13' 2023-10-06T04:00:12.193 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/115/14' -> 'copyofsnap1/test1/115/14' 2023-10-06T04:00:12.193 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/115/15' -> 'copyofsnap1/test1/115/15' 2023-10-06T04:00:12.193 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/115/16' -> 'copyofsnap1/test1/115/16' 2023-10-06T04:00:12.193 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/115/17' -> 'copyofsnap1/test1/115/17' 2023-10-06T04:00:12.193 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/115/18' -> 'copyofsnap1/test1/115/18' 2023-10-06T04:00:12.193 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/115/19' -> 'copyofsnap1/test1/115/19' 2023-10-06T04:00:12.193 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/115/20' -> 'copyofsnap1/test1/115/20' 2023-10-06T04:00:12.193 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/116' -> 'copyofsnap1/test1/116' 2023-10-06T04:00:12.194 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/116/10' -> 'copyofsnap1/test1/116/10' 2023-10-06T04:00:12.194 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/116/11' -> 'copyofsnap1/test1/116/11' 2023-10-06T04:00:12.194 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/116/12' -> 'copyofsnap1/test1/116/12' 2023-10-06T04:00:12.194 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/116/13' -> 'copyofsnap1/test1/116/13' 2023-10-06T04:00:12.194 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/116/14' -> 'copyofsnap1/test1/116/14' 2023-10-06T04:00:12.194 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/116/15' -> 'copyofsnap1/test1/116/15' 2023-10-06T04:00:12.194 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/116/16' -> 'copyofsnap1/test1/116/16' 2023-10-06T04:00:12.194 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/116/17' -> 'copyofsnap1/test1/116/17' 2023-10-06T04:00:12.194 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/116/18' -> 'copyofsnap1/test1/116/18' 2023-10-06T04:00:12.194 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/116/19' -> 'copyofsnap1/test1/116/19' 2023-10-06T04:00:12.194 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/116/20' -> 'copyofsnap1/test1/116/20' 2023-10-06T04:00:12.195 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/117' -> 'copyofsnap1/test1/117' 2023-10-06T04:00:12.195 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/117/10' -> 'copyofsnap1/test1/117/10' 2023-10-06T04:00:12.195 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/117/11' -> 'copyofsnap1/test1/117/11' 2023-10-06T04:00:12.195 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/117/12' -> 'copyofsnap1/test1/117/12' 2023-10-06T04:00:12.195 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/117/13' -> 'copyofsnap1/test1/117/13' 2023-10-06T04:00:12.195 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/117/14' -> 'copyofsnap1/test1/117/14' 2023-10-06T04:00:12.195 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/117/15' -> 'copyofsnap1/test1/117/15' 2023-10-06T04:00:12.195 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/117/16' -> 'copyofsnap1/test1/117/16' 2023-10-06T04:00:12.196 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/117/17' -> 'copyofsnap1/test1/117/17' 2023-10-06T04:00:12.196 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/117/18' -> 'copyofsnap1/test1/117/18' 2023-10-06T04:00:12.196 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/117/19' -> 'copyofsnap1/test1/117/19' 2023-10-06T04:00:12.196 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/117/20' -> 'copyofsnap1/test1/117/20' 2023-10-06T04:00:12.196 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/118' -> 'copyofsnap1/test1/118' 2023-10-06T04:00:12.196 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/118/10' -> 'copyofsnap1/test1/118/10' 2023-10-06T04:00:12.196 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/118/11' -> 'copyofsnap1/test1/118/11' 2023-10-06T04:00:12.196 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/118/12' -> 'copyofsnap1/test1/118/12' 2023-10-06T04:00:12.196 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/118/13' -> 'copyofsnap1/test1/118/13' 2023-10-06T04:00:12.197 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/118/14' -> 'copyofsnap1/test1/118/14' 2023-10-06T04:00:12.197 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/118/15' -> 'copyofsnap1/test1/118/15' 2023-10-06T04:00:12.197 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/118/16' -> 'copyofsnap1/test1/118/16' 2023-10-06T04:00:12.197 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/118/17' -> 'copyofsnap1/test1/118/17' 2023-10-06T04:00:12.197 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/118/18' -> 'copyofsnap1/test1/118/18' 2023-10-06T04:00:12.197 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/118/19' -> 'copyofsnap1/test1/118/19' 2023-10-06T04:00:12.197 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/118/20' -> 'copyofsnap1/test1/118/20' 2023-10-06T04:00:12.197 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/119' -> 'copyofsnap1/test1/119' 2023-10-06T04:00:12.197 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/119/10' -> 'copyofsnap1/test1/119/10' 2023-10-06T04:00:12.197 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/119/11' -> 'copyofsnap1/test1/119/11' 2023-10-06T04:00:12.197 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/119/12' -> 'copyofsnap1/test1/119/12' 2023-10-06T04:00:12.198 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/119/13' -> 'copyofsnap1/test1/119/13' 2023-10-06T04:00:12.198 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/119/14' -> 'copyofsnap1/test1/119/14' 2023-10-06T04:00:12.198 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/119/15' -> 'copyofsnap1/test1/119/15' 2023-10-06T04:00:12.198 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/119/16' -> 'copyofsnap1/test1/119/16' 2023-10-06T04:00:12.198 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/119/17' -> 'copyofsnap1/test1/119/17' 2023-10-06T04:00:12.198 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/119/18' -> 'copyofsnap1/test1/119/18' 2023-10-06T04:00:12.198 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/119/19' -> 'copyofsnap1/test1/119/19' 2023-10-06T04:00:12.198 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/119/20' -> 'copyofsnap1/test1/119/20' 2023-10-06T04:00:12.198 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/120' -> 'copyofsnap1/test1/120' 2023-10-06T04:00:12.198 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/120/10' -> 'copyofsnap1/test1/120/10' 2023-10-06T04:00:13.599 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/120/11' -> 'copyofsnap1/test1/120/11' 2023-10-06T04:00:13.599 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/120/12' -> 'copyofsnap1/test1/120/12' 2023-10-06T04:00:13.599 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/120/13' -> 'copyofsnap1/test1/120/13' 2023-10-06T04:00:13.599 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/120/14' -> 'copyofsnap1/test1/120/14' 2023-10-06T04:00:13.600 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/120/15' -> 'copyofsnap1/test1/120/15' 2023-10-06T04:00:13.600 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/120/16' -> 'copyofsnap1/test1/120/16' 2023-10-06T04:00:13.600 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/120/17' -> 'copyofsnap1/test1/120/17' 2023-10-06T04:00:13.600 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/120/18' -> 'copyofsnap1/test1/120/18' 2023-10-06T04:00:13.600 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/120/19' -> 'copyofsnap1/test1/120/19' 2023-10-06T04:00:13.600 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/120/20' -> 'copyofsnap1/test1/120/20' 2023-10-06T04:00:13.600 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/121' -> 'copyofsnap1/test1/121' 2023-10-06T04:00:13.600 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/121/10' -> 'copyofsnap1/test1/121/10' 2023-10-06T04:00:13.600 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/121/11' -> 'copyofsnap1/test1/121/11' 2023-10-06T04:00:13.600 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/121/12' -> 'copyofsnap1/test1/121/12' 2023-10-06T04:00:13.600 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/121/13' -> 'copyofsnap1/test1/121/13' 2023-10-06T04:00:13.601 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/121/14' -> 'copyofsnap1/test1/121/14' 2023-10-06T04:00:13.601 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/121/15' -> 'copyofsnap1/test1/121/15' 2023-10-06T04:00:13.601 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/121/16' -> 'copyofsnap1/test1/121/16' 2023-10-06T04:00:13.601 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/121/17' -> 'copyofsnap1/test1/121/17' 2023-10-06T04:00:13.601 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/121/18' -> 'copyofsnap1/test1/121/18' 2023-10-06T04:00:13.601 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/121/19' -> 'copyofsnap1/test1/121/19' 2023-10-06T04:00:13.601 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/121/20' -> 'copyofsnap1/test1/121/20' 2023-10-06T04:00:13.601 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/122' -> 'copyofsnap1/test1/122' 2023-10-06T04:00:13.601 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/122/10' -> 'copyofsnap1/test1/122/10' 2023-10-06T04:00:13.601 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/122/11' -> 'copyofsnap1/test1/122/11' 2023-10-06T04:00:13.601 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/122/12' -> 'copyofsnap1/test1/122/12' 2023-10-06T04:00:13.601 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/122/13' -> 'copyofsnap1/test1/122/13' 2023-10-06T04:00:13.602 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/122/14' -> 'copyofsnap1/test1/122/14' 2023-10-06T04:00:13.602 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/122/15' -> 'copyofsnap1/test1/122/15' 2023-10-06T04:00:13.602 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/122/16' -> 'copyofsnap1/test1/122/16' 2023-10-06T04:00:13.602 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/122/17' -> 'copyofsnap1/test1/122/17' 2023-10-06T04:00:13.602 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/122/18' -> 'copyofsnap1/test1/122/18' 2023-10-06T04:00:13.602 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/122/19' -> 'copyofsnap1/test1/122/19' 2023-10-06T04:00:13.602 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/122/20' -> 'copyofsnap1/test1/122/20' 2023-10-06T04:00:13.602 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/123' -> 'copyofsnap1/test1/123' 2023-10-06T04:00:13.602 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/123/10' -> 'copyofsnap1/test1/123/10' 2023-10-06T04:00:13.602 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/123/11' -> 'copyofsnap1/test1/123/11' 2023-10-06T04:00:13.602 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/123/12' -> 'copyofsnap1/test1/123/12' 2023-10-06T04:00:13.602 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/123/13' -> 'copyofsnap1/test1/123/13' 2023-10-06T04:00:13.603 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/123/14' -> 'copyofsnap1/test1/123/14' 2023-10-06T04:00:13.603 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/123/15' -> 'copyofsnap1/test1/123/15' 2023-10-06T04:00:13.603 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/123/16' -> 'copyofsnap1/test1/123/16' 2023-10-06T04:00:13.603 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/123/17' -> 'copyofsnap1/test1/123/17' 2023-10-06T04:00:13.603 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/123/18' -> 'copyofsnap1/test1/123/18' 2023-10-06T04:00:13.603 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/123/19' -> 'copyofsnap1/test1/123/19' 2023-10-06T04:00:13.603 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/123/20' -> 'copyofsnap1/test1/123/20' 2023-10-06T04:00:13.603 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/124' -> 'copyofsnap1/test1/124' 2023-10-06T04:00:13.603 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/124/10' -> 'copyofsnap1/test1/124/10' 2023-10-06T04:00:13.603 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/124/11' -> 'copyofsnap1/test1/124/11' 2023-10-06T04:00:13.603 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/124/12' -> 'copyofsnap1/test1/124/12' 2023-10-06T04:00:13.603 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/124/13' -> 'copyofsnap1/test1/124/13' 2023-10-06T04:00:13.604 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/124/14' -> 'copyofsnap1/test1/124/14' 2023-10-06T04:00:13.604 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/124/15' -> 'copyofsnap1/test1/124/15' 2023-10-06T04:00:13.604 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/124/16' -> 'copyofsnap1/test1/124/16' 2023-10-06T04:00:13.604 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/124/17' -> 'copyofsnap1/test1/124/17' 2023-10-06T04:00:13.604 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/124/18' -> 'copyofsnap1/test1/124/18' 2023-10-06T04:00:13.604 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/124/19' -> 'copyofsnap1/test1/124/19' 2023-10-06T04:00:13.604 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/124/20' -> 'copyofsnap1/test1/124/20' 2023-10-06T04:00:13.604 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/125' -> 'copyofsnap1/test1/125' 2023-10-06T04:00:13.604 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/125/10' -> 'copyofsnap1/test1/125/10' 2023-10-06T04:00:13.604 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/125/11' -> 'copyofsnap1/test1/125/11' 2023-10-06T04:00:13.604 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/125/12' -> 'copyofsnap1/test1/125/12' 2023-10-06T04:00:13.604 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/125/13' -> 'copyofsnap1/test1/125/13' 2023-10-06T04:00:13.604 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/125/14' -> 'copyofsnap1/test1/125/14' 2023-10-06T04:00:13.605 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/125/15' -> 'copyofsnap1/test1/125/15' 2023-10-06T04:00:13.605 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/125/16' -> 'copyofsnap1/test1/125/16' 2023-10-06T04:00:13.605 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/125/17' -> 'copyofsnap1/test1/125/17' 2023-10-06T04:00:13.605 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/125/18' -> 'copyofsnap1/test1/125/18' 2023-10-06T04:00:13.605 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/125/19' -> 'copyofsnap1/test1/125/19' 2023-10-06T04:00:13.605 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/125/20' -> 'copyofsnap1/test1/125/20' 2023-10-06T04:00:13.605 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/126' -> 'copyofsnap1/test1/126' 2023-10-06T04:00:13.605 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/126/10' -> 'copyofsnap1/test1/126/10' 2023-10-06T04:00:13.605 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/126/11' -> 'copyofsnap1/test1/126/11' 2023-10-06T04:00:13.605 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/126/12' -> 'copyofsnap1/test1/126/12' 2023-10-06T04:00:13.605 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/126/13' -> 'copyofsnap1/test1/126/13' 2023-10-06T04:00:13.606 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/126/14' -> 'copyofsnap1/test1/126/14' 2023-10-06T04:00:13.606 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/126/15' -> 'copyofsnap1/test1/126/15' 2023-10-06T04:00:13.606 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/126/16' -> 'copyofsnap1/test1/126/16' 2023-10-06T04:00:13.606 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/126/17' -> 'copyofsnap1/test1/126/17' 2023-10-06T04:00:13.606 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/126/18' -> 'copyofsnap1/test1/126/18' 2023-10-06T04:00:13.606 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/126/19' -> 'copyofsnap1/test1/126/19' 2023-10-06T04:00:13.910 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:00:13.912 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:00:20.305 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/126/20' -> 'copyofsnap1/test1/126/20' 2023-10-06T04:00:20.305 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/127' -> 'copyofsnap1/test1/127' 2023-10-06T04:00:20.305 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/127/10' -> 'copyofsnap1/test1/127/10' 2023-10-06T04:00:20.305 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/127/11' -> 'copyofsnap1/test1/127/11' 2023-10-06T04:00:20.305 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/127/12' -> 'copyofsnap1/test1/127/12' 2023-10-06T04:00:20.305 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/127/13' -> 'copyofsnap1/test1/127/13' 2023-10-06T04:00:20.305 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/127/14' -> 'copyofsnap1/test1/127/14' 2023-10-06T04:00:20.305 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/127/15' -> 'copyofsnap1/test1/127/15' 2023-10-06T04:00:20.305 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/127/16' -> 'copyofsnap1/test1/127/16' 2023-10-06T04:00:20.306 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/127/17' -> 'copyofsnap1/test1/127/17' 2023-10-06T04:00:20.306 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/127/18' -> 'copyofsnap1/test1/127/18' 2023-10-06T04:00:20.306 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/127/19' -> 'copyofsnap1/test1/127/19' 2023-10-06T04:00:20.306 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/127/20' -> 'copyofsnap1/test1/127/20' 2023-10-06T04:00:20.306 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/128' -> 'copyofsnap1/test1/128' 2023-10-06T04:00:20.306 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/128/10' -> 'copyofsnap1/test1/128/10' 2023-10-06T04:00:20.306 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/128/11' -> 'copyofsnap1/test1/128/11' 2023-10-06T04:00:20.306 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/128/12' -> 'copyofsnap1/test1/128/12' 2023-10-06T04:00:20.306 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/128/13' -> 'copyofsnap1/test1/128/13' 2023-10-06T04:00:20.306 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/128/14' -> 'copyofsnap1/test1/128/14' 2023-10-06T04:00:20.306 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/128/15' -> 'copyofsnap1/test1/128/15' 2023-10-06T04:00:20.307 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/128/16' -> 'copyofsnap1/test1/128/16' 2023-10-06T04:00:20.307 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/128/17' -> 'copyofsnap1/test1/128/17' 2023-10-06T04:00:20.307 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/128/18' -> 'copyofsnap1/test1/128/18' 2023-10-06T04:00:20.307 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/128/19' -> 'copyofsnap1/test1/128/19' 2023-10-06T04:00:20.307 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/128/20' -> 'copyofsnap1/test1/128/20' 2023-10-06T04:00:20.307 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/129' -> 'copyofsnap1/test1/129' 2023-10-06T04:00:20.307 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/129/10' -> 'copyofsnap1/test1/129/10' 2023-10-06T04:00:20.307 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/129/11' -> 'copyofsnap1/test1/129/11' 2023-10-06T04:00:20.307 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/129/12' -> 'copyofsnap1/test1/129/12' 2023-10-06T04:00:20.307 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/129/13' -> 'copyofsnap1/test1/129/13' 2023-10-06T04:00:20.307 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/129/14' -> 'copyofsnap1/test1/129/14' 2023-10-06T04:00:20.308 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/129/15' -> 'copyofsnap1/test1/129/15' 2023-10-06T04:00:20.308 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/129/16' -> 'copyofsnap1/test1/129/16' 2023-10-06T04:00:20.308 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/129/17' -> 'copyofsnap1/test1/129/17' 2023-10-06T04:00:20.308 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/129/18' -> 'copyofsnap1/test1/129/18' 2023-10-06T04:00:20.308 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/129/19' -> 'copyofsnap1/test1/129/19' 2023-10-06T04:00:20.308 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/129/20' -> 'copyofsnap1/test1/129/20' 2023-10-06T04:00:20.308 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/130' -> 'copyofsnap1/test1/130' 2023-10-06T04:00:20.308 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/130/10' -> 'copyofsnap1/test1/130/10' 2023-10-06T04:00:20.308 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/130/11' -> 'copyofsnap1/test1/130/11' 2023-10-06T04:00:20.308 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/130/12' -> 'copyofsnap1/test1/130/12' 2023-10-06T04:00:20.308 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/130/13' -> 'copyofsnap1/test1/130/13' 2023-10-06T04:00:20.308 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/130/14' -> 'copyofsnap1/test1/130/14' 2023-10-06T04:00:20.308 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/130/15' -> 'copyofsnap1/test1/130/15' 2023-10-06T04:00:20.309 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/130/16' -> 'copyofsnap1/test1/130/16' 2023-10-06T04:00:20.309 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/130/17' -> 'copyofsnap1/test1/130/17' 2023-10-06T04:00:20.309 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/130/18' -> 'copyofsnap1/test1/130/18' 2023-10-06T04:00:20.309 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/130/19' -> 'copyofsnap1/test1/130/19' 2023-10-06T04:00:20.309 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/130/20' -> 'copyofsnap1/test1/130/20' 2023-10-06T04:00:20.309 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/131' -> 'copyofsnap1/test1/131' 2023-10-06T04:00:20.309 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/131/10' -> 'copyofsnap1/test1/131/10' 2023-10-06T04:00:20.309 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/131/11' -> 'copyofsnap1/test1/131/11' 2023-10-06T04:00:20.309 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/131/12' -> 'copyofsnap1/test1/131/12' 2023-10-06T04:00:20.309 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/131/13' -> 'copyofsnap1/test1/131/13' 2023-10-06T04:00:20.309 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/131/14' -> 'copyofsnap1/test1/131/14' 2023-10-06T04:00:20.309 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/131/15' -> 'copyofsnap1/test1/131/15' 2023-10-06T04:00:20.310 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/131/16' -> 'copyofsnap1/test1/131/16' 2023-10-06T04:00:20.310 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/131/17' -> 'copyofsnap1/test1/131/17' 2023-10-06T04:00:20.310 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/131/18' -> 'copyofsnap1/test1/131/18' 2023-10-06T04:00:20.310 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/131/19' -> 'copyofsnap1/test1/131/19' 2023-10-06T04:00:20.310 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/131/20' -> 'copyofsnap1/test1/131/20' 2023-10-06T04:00:20.310 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/132' -> 'copyofsnap1/test1/132' 2023-10-06T04:00:20.310 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/132/10' -> 'copyofsnap1/test1/132/10' 2023-10-06T04:00:20.310 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/132/11' -> 'copyofsnap1/test1/132/11' 2023-10-06T04:00:20.310 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/132/12' -> 'copyofsnap1/test1/132/12' 2023-10-06T04:00:20.310 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/132/13' -> 'copyofsnap1/test1/132/13' 2023-10-06T04:00:20.310 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/132/14' -> 'copyofsnap1/test1/132/14' 2023-10-06T04:00:20.310 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/132/15' -> 'copyofsnap1/test1/132/15' 2023-10-06T04:00:20.311 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/132/16' -> 'copyofsnap1/test1/132/16' 2023-10-06T04:00:20.311 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/132/17' -> 'copyofsnap1/test1/132/17' 2023-10-06T04:00:20.311 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/132/18' -> 'copyofsnap1/test1/132/18' 2023-10-06T04:00:20.311 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/132/19' -> 'copyofsnap1/test1/132/19' 2023-10-06T04:00:20.311 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/132/20' -> 'copyofsnap1/test1/132/20' 2023-10-06T04:00:20.311 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/133' -> 'copyofsnap1/test1/133' 2023-10-06T04:00:20.311 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/133/10' -> 'copyofsnap1/test1/133/10' 2023-10-06T04:00:20.311 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/133/11' -> 'copyofsnap1/test1/133/11' 2023-10-06T04:00:20.311 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/133/12' -> 'copyofsnap1/test1/133/12' 2023-10-06T04:00:20.311 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/133/13' -> 'copyofsnap1/test1/133/13' 2023-10-06T04:00:20.311 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/133/14' -> 'copyofsnap1/test1/133/14' 2023-10-06T04:00:20.311 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/133/15' -> 'copyofsnap1/test1/133/15' 2023-10-06T04:00:20.312 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/133/16' -> 'copyofsnap1/test1/133/16' 2023-10-06T04:00:22.318 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/133/17' -> 'copyofsnap1/test1/133/17' 2023-10-06T04:00:22.318 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/133/18' -> 'copyofsnap1/test1/133/18' 2023-10-06T04:00:22.318 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/133/19' -> 'copyofsnap1/test1/133/19' 2023-10-06T04:00:22.318 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/133/20' -> 'copyofsnap1/test1/133/20' 2023-10-06T04:00:22.318 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/134' -> 'copyofsnap1/test1/134' 2023-10-06T04:00:22.319 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/134/10' -> 'copyofsnap1/test1/134/10' 2023-10-06T04:00:22.319 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/134/11' -> 'copyofsnap1/test1/134/11' 2023-10-06T04:00:22.319 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/134/12' -> 'copyofsnap1/test1/134/12' 2023-10-06T04:00:22.319 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/134/13' -> 'copyofsnap1/test1/134/13' 2023-10-06T04:00:22.319 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/134/14' -> 'copyofsnap1/test1/134/14' 2023-10-06T04:00:22.319 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/134/15' -> 'copyofsnap1/test1/134/15' 2023-10-06T04:00:22.319 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/134/16' -> 'copyofsnap1/test1/134/16' 2023-10-06T04:00:22.319 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/134/17' -> 'copyofsnap1/test1/134/17' 2023-10-06T04:00:22.319 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/134/18' -> 'copyofsnap1/test1/134/18' 2023-10-06T04:00:22.319 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/134/19' -> 'copyofsnap1/test1/134/19' 2023-10-06T04:00:22.319 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/134/20' -> 'copyofsnap1/test1/134/20' 2023-10-06T04:00:22.319 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/135' -> 'copyofsnap1/test1/135' 2023-10-06T04:00:22.320 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/135/10' -> 'copyofsnap1/test1/135/10' 2023-10-06T04:00:22.320 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/135/11' -> 'copyofsnap1/test1/135/11' 2023-10-06T04:00:22.320 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/135/12' -> 'copyofsnap1/test1/135/12' 2023-10-06T04:00:22.320 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/135/13' -> 'copyofsnap1/test1/135/13' 2023-10-06T04:00:22.320 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/135/14' -> 'copyofsnap1/test1/135/14' 2023-10-06T04:00:22.320 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/135/15' -> 'copyofsnap1/test1/135/15' 2023-10-06T04:00:22.320 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/135/16' -> 'copyofsnap1/test1/135/16' 2023-10-06T04:00:22.320 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/135/17' -> 'copyofsnap1/test1/135/17' 2023-10-06T04:00:22.320 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/135/18' -> 'copyofsnap1/test1/135/18' 2023-10-06T04:00:22.320 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/135/19' -> 'copyofsnap1/test1/135/19' 2023-10-06T04:00:22.320 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/135/20' -> 'copyofsnap1/test1/135/20' 2023-10-06T04:00:22.320 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/136' -> 'copyofsnap1/test1/136' 2023-10-06T04:00:22.320 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/136/10' -> 'copyofsnap1/test1/136/10' 2023-10-06T04:00:22.321 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/136/11' -> 'copyofsnap1/test1/136/11' 2023-10-06T04:00:22.321 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/136/12' -> 'copyofsnap1/test1/136/12' 2023-10-06T04:00:22.321 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/136/13' -> 'copyofsnap1/test1/136/13' 2023-10-06T04:00:22.321 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/136/14' -> 'copyofsnap1/test1/136/14' 2023-10-06T04:00:22.321 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/136/15' -> 'copyofsnap1/test1/136/15' 2023-10-06T04:00:22.321 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/136/16' -> 'copyofsnap1/test1/136/16' 2023-10-06T04:00:22.321 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/136/17' -> 'copyofsnap1/test1/136/17' 2023-10-06T04:00:22.321 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/136/18' -> 'copyofsnap1/test1/136/18' 2023-10-06T04:00:22.321 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/136/19' -> 'copyofsnap1/test1/136/19' 2023-10-06T04:00:22.321 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/136/20' -> 'copyofsnap1/test1/136/20' 2023-10-06T04:00:22.321 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/137' -> 'copyofsnap1/test1/137' 2023-10-06T04:00:22.321 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/137/10' -> 'copyofsnap1/test1/137/10' 2023-10-06T04:00:22.322 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/137/11' -> 'copyofsnap1/test1/137/11' 2023-10-06T04:00:22.322 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/137/12' -> 'copyofsnap1/test1/137/12' 2023-10-06T04:00:22.322 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/137/13' -> 'copyofsnap1/test1/137/13' 2023-10-06T04:00:22.322 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/137/14' -> 'copyofsnap1/test1/137/14' 2023-10-06T04:00:22.322 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/137/15' -> 'copyofsnap1/test1/137/15' 2023-10-06T04:00:22.322 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/137/16' -> 'copyofsnap1/test1/137/16' 2023-10-06T04:00:22.322 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/137/17' -> 'copyofsnap1/test1/137/17' 2023-10-06T04:00:22.322 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/137/18' -> 'copyofsnap1/test1/137/18' 2023-10-06T04:00:22.322 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/137/19' -> 'copyofsnap1/test1/137/19' 2023-10-06T04:00:22.322 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/137/20' -> 'copyofsnap1/test1/137/20' 2023-10-06T04:00:22.322 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/138' -> 'copyofsnap1/test1/138' 2023-10-06T04:00:22.322 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/138/10' -> 'copyofsnap1/test1/138/10' 2023-10-06T04:00:22.323 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/138/11' -> 'copyofsnap1/test1/138/11' 2023-10-06T04:00:22.323 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/138/12' -> 'copyofsnap1/test1/138/12' 2023-10-06T04:00:22.323 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/138/13' -> 'copyofsnap1/test1/138/13' 2023-10-06T04:00:22.323 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/138/14' -> 'copyofsnap1/test1/138/14' 2023-10-06T04:00:22.323 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/138/15' -> 'copyofsnap1/test1/138/15' 2023-10-06T04:00:22.323 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/138/16' -> 'copyofsnap1/test1/138/16' 2023-10-06T04:00:22.323 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/138/17' -> 'copyofsnap1/test1/138/17' 2023-10-06T04:00:22.323 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/138/18' -> 'copyofsnap1/test1/138/18' 2023-10-06T04:00:22.323 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/138/19' -> 'copyofsnap1/test1/138/19' 2023-10-06T04:00:22.323 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/138/20' -> 'copyofsnap1/test1/138/20' 2023-10-06T04:00:22.323 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/139' -> 'copyofsnap1/test1/139' 2023-10-06T04:00:22.323 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/139/10' -> 'copyofsnap1/test1/139/10' 2023-10-06T04:00:22.324 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/139/11' -> 'copyofsnap1/test1/139/11' 2023-10-06T04:00:22.324 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/139/12' -> 'copyofsnap1/test1/139/12' 2023-10-06T04:00:22.324 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/139/13' -> 'copyofsnap1/test1/139/13' 2023-10-06T04:00:22.324 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/139/14' -> 'copyofsnap1/test1/139/14' 2023-10-06T04:00:22.324 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/139/15' -> 'copyofsnap1/test1/139/15' 2023-10-06T04:00:22.324 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/139/16' -> 'copyofsnap1/test1/139/16' 2023-10-06T04:00:22.324 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/139/17' -> 'copyofsnap1/test1/139/17' 2023-10-06T04:00:22.324 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/139/18' -> 'copyofsnap1/test1/139/18' 2023-10-06T04:00:22.324 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/139/19' -> 'copyofsnap1/test1/139/19' 2023-10-06T04:00:22.324 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/139/20' -> 'copyofsnap1/test1/139/20' 2023-10-06T04:00:22.324 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/140' -> 'copyofsnap1/test1/140' 2023-10-06T04:00:22.325 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/140/10' -> 'copyofsnap1/test1/140/10' 2023-10-06T04:00:22.325 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/140/11' -> 'copyofsnap1/test1/140/11' 2023-10-06T04:00:22.325 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/140/12' -> 'copyofsnap1/test1/140/12' 2023-10-06T04:00:22.325 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/140/13' -> 'copyofsnap1/test1/140/13' 2023-10-06T04:00:22.496 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/140/14' -> 'copyofsnap1/test1/140/14' 2023-10-06T04:00:22.496 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/140/15' -> 'copyofsnap1/test1/140/15' 2023-10-06T04:00:22.496 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/140/16' -> 'copyofsnap1/test1/140/16' 2023-10-06T04:00:22.496 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/140/17' -> 'copyofsnap1/test1/140/17' 2023-10-06T04:00:22.496 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/140/18' -> 'copyofsnap1/test1/140/18' 2023-10-06T04:00:22.496 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/140/19' -> 'copyofsnap1/test1/140/19' 2023-10-06T04:00:22.496 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/140/20' -> 'copyofsnap1/test1/140/20' 2023-10-06T04:00:22.496 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/141' -> 'copyofsnap1/test1/141' 2023-10-06T04:00:22.496 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/141/10' -> 'copyofsnap1/test1/141/10' 2023-10-06T04:00:22.496 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/141/11' -> 'copyofsnap1/test1/141/11' 2023-10-06T04:00:22.497 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/141/12' -> 'copyofsnap1/test1/141/12' 2023-10-06T04:00:22.497 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/141/13' -> 'copyofsnap1/test1/141/13' 2023-10-06T04:00:22.497 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/141/14' -> 'copyofsnap1/test1/141/14' 2023-10-06T04:00:22.497 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/141/15' -> 'copyofsnap1/test1/141/15' 2023-10-06T04:00:22.497 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/141/16' -> 'copyofsnap1/test1/141/16' 2023-10-06T04:00:22.497 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/141/17' -> 'copyofsnap1/test1/141/17' 2023-10-06T04:00:22.497 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/141/18' -> 'copyofsnap1/test1/141/18' 2023-10-06T04:00:22.497 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/141/19' -> 'copyofsnap1/test1/141/19' 2023-10-06T04:00:22.497 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/141/20' -> 'copyofsnap1/test1/141/20' 2023-10-06T04:00:22.497 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/142' -> 'copyofsnap1/test1/142' 2023-10-06T04:00:22.497 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/142/10' -> 'copyofsnap1/test1/142/10' 2023-10-06T04:00:22.498 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/142/11' -> 'copyofsnap1/test1/142/11' 2023-10-06T04:00:22.498 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/142/12' -> 'copyofsnap1/test1/142/12' 2023-10-06T04:00:22.498 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/142/13' -> 'copyofsnap1/test1/142/13' 2023-10-06T04:00:22.498 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/142/14' -> 'copyofsnap1/test1/142/14' 2023-10-06T04:00:22.498 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/142/15' -> 'copyofsnap1/test1/142/15' 2023-10-06T04:00:22.498 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/142/16' -> 'copyofsnap1/test1/142/16' 2023-10-06T04:00:22.498 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/142/17' -> 'copyofsnap1/test1/142/17' 2023-10-06T04:00:22.498 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/142/18' -> 'copyofsnap1/test1/142/18' 2023-10-06T04:00:22.498 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/142/19' -> 'copyofsnap1/test1/142/19' 2023-10-06T04:00:22.498 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/142/20' -> 'copyofsnap1/test1/142/20' 2023-10-06T04:00:22.498 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/143' -> 'copyofsnap1/test1/143' 2023-10-06T04:00:22.499 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/143/10' -> 'copyofsnap1/test1/143/10' 2023-10-06T04:00:22.499 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/143/11' -> 'copyofsnap1/test1/143/11' 2023-10-06T04:00:22.499 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/143/12' -> 'copyofsnap1/test1/143/12' 2023-10-06T04:00:22.499 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/143/13' -> 'copyofsnap1/test1/143/13' 2023-10-06T04:00:22.499 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/143/14' -> 'copyofsnap1/test1/143/14' 2023-10-06T04:00:22.499 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/143/15' -> 'copyofsnap1/test1/143/15' 2023-10-06T04:00:22.499 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/143/16' -> 'copyofsnap1/test1/143/16' 2023-10-06T04:00:22.499 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/143/17' -> 'copyofsnap1/test1/143/17' 2023-10-06T04:00:22.499 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/143/18' -> 'copyofsnap1/test1/143/18' 2023-10-06T04:00:22.499 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/143/19' -> 'copyofsnap1/test1/143/19' 2023-10-06T04:00:22.499 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/143/20' -> 'copyofsnap1/test1/143/20' 2023-10-06T04:00:22.499 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/144' -> 'copyofsnap1/test1/144' 2023-10-06T04:00:22.500 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/144/10' -> 'copyofsnap1/test1/144/10' 2023-10-06T04:00:22.500 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/144/11' -> 'copyofsnap1/test1/144/11' 2023-10-06T04:00:22.500 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/144/12' -> 'copyofsnap1/test1/144/12' 2023-10-06T04:00:22.500 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/144/13' -> 'copyofsnap1/test1/144/13' 2023-10-06T04:00:22.500 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/144/14' -> 'copyofsnap1/test1/144/14' 2023-10-06T04:00:22.500 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/144/15' -> 'copyofsnap1/test1/144/15' 2023-10-06T04:00:22.500 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/144/16' -> 'copyofsnap1/test1/144/16' 2023-10-06T04:00:22.500 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/144/17' -> 'copyofsnap1/test1/144/17' 2023-10-06T04:00:22.500 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/144/18' -> 'copyofsnap1/test1/144/18' 2023-10-06T04:00:22.500 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/144/19' -> 'copyofsnap1/test1/144/19' 2023-10-06T04:00:22.500 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/144/20' -> 'copyofsnap1/test1/144/20' 2023-10-06T04:00:22.500 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/145' -> 'copyofsnap1/test1/145' 2023-10-06T04:00:22.501 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/145/10' -> 'copyofsnap1/test1/145/10' 2023-10-06T04:00:22.501 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/145/11' -> 'copyofsnap1/test1/145/11' 2023-10-06T04:00:22.501 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/145/12' -> 'copyofsnap1/test1/145/12' 2023-10-06T04:00:22.501 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/145/13' -> 'copyofsnap1/test1/145/13' 2023-10-06T04:00:22.501 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/145/14' -> 'copyofsnap1/test1/145/14' 2023-10-06T04:00:22.501 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/145/15' -> 'copyofsnap1/test1/145/15' 2023-10-06T04:00:22.501 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/145/16' -> 'copyofsnap1/test1/145/16' 2023-10-06T04:00:22.501 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/145/17' -> 'copyofsnap1/test1/145/17' 2023-10-06T04:00:22.501 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/145/18' -> 'copyofsnap1/test1/145/18' 2023-10-06T04:00:22.501 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/145/19' -> 'copyofsnap1/test1/145/19' 2023-10-06T04:00:22.501 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/145/20' -> 'copyofsnap1/test1/145/20' 2023-10-06T04:00:22.502 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/146' -> 'copyofsnap1/test1/146' 2023-10-06T04:00:22.502 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/146/10' -> 'copyofsnap1/test1/146/10' 2023-10-06T04:00:22.502 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/146/11' -> 'copyofsnap1/test1/146/11' 2023-10-06T04:00:22.502 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/146/12' -> 'copyofsnap1/test1/146/12' 2023-10-06T04:00:22.502 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/146/13' -> 'copyofsnap1/test1/146/13' 2023-10-06T04:00:22.502 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/146/14' -> 'copyofsnap1/test1/146/14' 2023-10-06T04:00:22.502 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/146/15' -> 'copyofsnap1/test1/146/15' 2023-10-06T04:00:22.502 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/146/16' -> 'copyofsnap1/test1/146/16' 2023-10-06T04:00:22.502 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/146/17' -> 'copyofsnap1/test1/146/17' 2023-10-06T04:00:22.502 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/146/18' -> 'copyofsnap1/test1/146/18' 2023-10-06T04:00:22.502 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/146/19' -> 'copyofsnap1/test1/146/19' 2023-10-06T04:00:22.502 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/146/20' -> 'copyofsnap1/test1/146/20' 2023-10-06T04:00:22.503 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/147' -> 'copyofsnap1/test1/147' 2023-10-06T04:00:22.503 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/147/10' -> 'copyofsnap1/test1/147/10' 2023-10-06T04:00:22.503 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/147/11' -> 'copyofsnap1/test1/147/11' 2023-10-06T04:00:23.265 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/147/12' -> 'copyofsnap1/test1/147/12' 2023-10-06T04:00:23.265 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/147/13' -> 'copyofsnap1/test1/147/13' 2023-10-06T04:00:23.265 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/147/14' -> 'copyofsnap1/test1/147/14' 2023-10-06T04:00:23.265 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/147/15' -> 'copyofsnap1/test1/147/15' 2023-10-06T04:00:23.265 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/147/16' -> 'copyofsnap1/test1/147/16' 2023-10-06T04:00:23.265 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/147/17' -> 'copyofsnap1/test1/147/17' 2023-10-06T04:00:23.265 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/147/18' -> 'copyofsnap1/test1/147/18' 2023-10-06T04:00:23.265 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/147/19' -> 'copyofsnap1/test1/147/19' 2023-10-06T04:00:23.266 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/147/20' -> 'copyofsnap1/test1/147/20' 2023-10-06T04:00:23.266 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/148' -> 'copyofsnap1/test1/148' 2023-10-06T04:00:23.266 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/148/10' -> 'copyofsnap1/test1/148/10' 2023-10-06T04:00:23.266 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/148/11' -> 'copyofsnap1/test1/148/11' 2023-10-06T04:00:23.266 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/148/12' -> 'copyofsnap1/test1/148/12' 2023-10-06T04:00:23.266 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/148/13' -> 'copyofsnap1/test1/148/13' 2023-10-06T04:00:23.266 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/148/14' -> 'copyofsnap1/test1/148/14' 2023-10-06T04:00:23.266 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/148/15' -> 'copyofsnap1/test1/148/15' 2023-10-06T04:00:23.266 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/148/16' -> 'copyofsnap1/test1/148/16' 2023-10-06T04:00:23.266 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/148/17' -> 'copyofsnap1/test1/148/17' 2023-10-06T04:00:23.266 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/148/18' -> 'copyofsnap1/test1/148/18' 2023-10-06T04:00:23.267 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/148/19' -> 'copyofsnap1/test1/148/19' 2023-10-06T04:00:23.267 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/148/20' -> 'copyofsnap1/test1/148/20' 2023-10-06T04:00:23.267 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/149' -> 'copyofsnap1/test1/149' 2023-10-06T04:00:23.267 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/149/10' -> 'copyofsnap1/test1/149/10' 2023-10-06T04:00:23.267 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/149/11' -> 'copyofsnap1/test1/149/11' 2023-10-06T04:00:23.267 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/149/12' -> 'copyofsnap1/test1/149/12' 2023-10-06T04:00:23.267 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/149/13' -> 'copyofsnap1/test1/149/13' 2023-10-06T04:00:23.267 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/149/14' -> 'copyofsnap1/test1/149/14' 2023-10-06T04:00:23.267 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/149/15' -> 'copyofsnap1/test1/149/15' 2023-10-06T04:00:23.267 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/149/16' -> 'copyofsnap1/test1/149/16' 2023-10-06T04:00:23.267 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/149/17' -> 'copyofsnap1/test1/149/17' 2023-10-06T04:00:23.268 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/149/18' -> 'copyofsnap1/test1/149/18' 2023-10-06T04:00:23.268 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/149/19' -> 'copyofsnap1/test1/149/19' 2023-10-06T04:00:23.268 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/149/20' -> 'copyofsnap1/test1/149/20' 2023-10-06T04:00:23.268 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/150' -> 'copyofsnap1/test1/150' 2023-10-06T04:00:23.268 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/150/10' -> 'copyofsnap1/test1/150/10' 2023-10-06T04:00:23.268 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/150/11' -> 'copyofsnap1/test1/150/11' 2023-10-06T04:00:23.268 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/150/12' -> 'copyofsnap1/test1/150/12' 2023-10-06T04:00:23.268 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/150/13' -> 'copyofsnap1/test1/150/13' 2023-10-06T04:00:23.268 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/150/14' -> 'copyofsnap1/test1/150/14' 2023-10-06T04:00:23.268 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/150/15' -> 'copyofsnap1/test1/150/15' 2023-10-06T04:00:23.268 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/150/16' -> 'copyofsnap1/test1/150/16' 2023-10-06T04:00:23.269 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/150/17' -> 'copyofsnap1/test1/150/17' 2023-10-06T04:00:23.269 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/150/18' -> 'copyofsnap1/test1/150/18' 2023-10-06T04:00:23.269 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/150/19' -> 'copyofsnap1/test1/150/19' 2023-10-06T04:00:23.269 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/150/20' -> 'copyofsnap1/test1/150/20' 2023-10-06T04:00:23.269 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/151' -> 'copyofsnap1/test1/151' 2023-10-06T04:00:23.269 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/151/10' -> 'copyofsnap1/test1/151/10' 2023-10-06T04:00:23.269 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/151/11' -> 'copyofsnap1/test1/151/11' 2023-10-06T04:00:23.269 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/151/12' -> 'copyofsnap1/test1/151/12' 2023-10-06T04:00:23.269 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/151/13' -> 'copyofsnap1/test1/151/13' 2023-10-06T04:00:23.269 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/151/14' -> 'copyofsnap1/test1/151/14' 2023-10-06T04:00:23.269 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/151/15' -> 'copyofsnap1/test1/151/15' 2023-10-06T04:00:23.269 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/151/16' -> 'copyofsnap1/test1/151/16' 2023-10-06T04:00:23.270 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/151/17' -> 'copyofsnap1/test1/151/17' 2023-10-06T04:00:23.270 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/151/18' -> 'copyofsnap1/test1/151/18' 2023-10-06T04:00:23.270 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/151/19' -> 'copyofsnap1/test1/151/19' 2023-10-06T04:00:23.270 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/151/20' -> 'copyofsnap1/test1/151/20' 2023-10-06T04:00:23.270 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/152' -> 'copyofsnap1/test1/152' 2023-10-06T04:00:23.270 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/152/10' -> 'copyofsnap1/test1/152/10' 2023-10-06T04:00:23.270 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/152/11' -> 'copyofsnap1/test1/152/11' 2023-10-06T04:00:23.270 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/152/12' -> 'copyofsnap1/test1/152/12' 2023-10-06T04:00:23.270 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/152/13' -> 'copyofsnap1/test1/152/13' 2023-10-06T04:00:23.270 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/152/14' -> 'copyofsnap1/test1/152/14' 2023-10-06T04:00:23.270 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/152/15' -> 'copyofsnap1/test1/152/15' 2023-10-06T04:00:23.271 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/152/16' -> 'copyofsnap1/test1/152/16' 2023-10-06T04:00:23.271 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/152/17' -> 'copyofsnap1/test1/152/17' 2023-10-06T04:00:23.271 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/152/18' -> 'copyofsnap1/test1/152/18' 2023-10-06T04:00:23.271 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/152/19' -> 'copyofsnap1/test1/152/19' 2023-10-06T04:00:23.271 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/152/20' -> 'copyofsnap1/test1/152/20' 2023-10-06T04:00:23.271 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/153' -> 'copyofsnap1/test1/153' 2023-10-06T04:00:23.271 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/153/10' -> 'copyofsnap1/test1/153/10' 2023-10-06T04:00:23.271 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/153/11' -> 'copyofsnap1/test1/153/11' 2023-10-06T04:00:23.271 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/153/12' -> 'copyofsnap1/test1/153/12' 2023-10-06T04:00:23.271 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/153/13' -> 'copyofsnap1/test1/153/13' 2023-10-06T04:00:23.271 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/153/14' -> 'copyofsnap1/test1/153/14' 2023-10-06T04:00:23.271 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/153/15' -> 'copyofsnap1/test1/153/15' 2023-10-06T04:00:23.272 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/153/16' -> 'copyofsnap1/test1/153/16' 2023-10-06T04:00:23.272 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/153/17' -> 'copyofsnap1/test1/153/17' 2023-10-06T04:00:23.272 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/153/18' -> 'copyofsnap1/test1/153/18' 2023-10-06T04:00:23.272 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/153/19' -> 'copyofsnap1/test1/153/19' 2023-10-06T04:00:23.272 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/153/20' -> 'copyofsnap1/test1/153/20' 2023-10-06T04:00:29.601 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/154' -> 'copyofsnap1/test1/154' 2023-10-06T04:00:29.601 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/154/10' -> 'copyofsnap1/test1/154/10' 2023-10-06T04:00:29.601 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/154/11' -> 'copyofsnap1/test1/154/11' 2023-10-06T04:00:29.601 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/154/12' -> 'copyofsnap1/test1/154/12' 2023-10-06T04:00:29.601 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/154/13' -> 'copyofsnap1/test1/154/13' 2023-10-06T04:00:29.601 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/154/14' -> 'copyofsnap1/test1/154/14' 2023-10-06T04:00:29.602 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/154/15' -> 'copyofsnap1/test1/154/15' 2023-10-06T04:00:29.602 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/154/16' -> 'copyofsnap1/test1/154/16' 2023-10-06T04:00:29.602 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/154/17' -> 'copyofsnap1/test1/154/17' 2023-10-06T04:00:29.602 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/154/18' -> 'copyofsnap1/test1/154/18' 2023-10-06T04:00:29.602 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/154/19' -> 'copyofsnap1/test1/154/19' 2023-10-06T04:00:29.602 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/154/20' -> 'copyofsnap1/test1/154/20' 2023-10-06T04:00:29.602 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/155' -> 'copyofsnap1/test1/155' 2023-10-06T04:00:29.602 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/155/10' -> 'copyofsnap1/test1/155/10' 2023-10-06T04:00:29.602 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/155/11' -> 'copyofsnap1/test1/155/11' 2023-10-06T04:00:29.602 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/155/12' -> 'copyofsnap1/test1/155/12' 2023-10-06T04:00:29.602 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/155/13' -> 'copyofsnap1/test1/155/13' 2023-10-06T04:00:29.602 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/155/14' -> 'copyofsnap1/test1/155/14' 2023-10-06T04:00:29.603 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/155/15' -> 'copyofsnap1/test1/155/15' 2023-10-06T04:00:29.603 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/155/16' -> 'copyofsnap1/test1/155/16' 2023-10-06T04:00:29.603 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/155/17' -> 'copyofsnap1/test1/155/17' 2023-10-06T04:00:29.603 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/155/18' -> 'copyofsnap1/test1/155/18' 2023-10-06T04:00:29.603 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/155/19' -> 'copyofsnap1/test1/155/19' 2023-10-06T04:00:29.603 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/155/20' -> 'copyofsnap1/test1/155/20' 2023-10-06T04:00:29.603 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/156' -> 'copyofsnap1/test1/156' 2023-10-06T04:00:29.603 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/156/10' -> 'copyofsnap1/test1/156/10' 2023-10-06T04:00:29.603 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/156/11' -> 'copyofsnap1/test1/156/11' 2023-10-06T04:00:29.603 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/156/12' -> 'copyofsnap1/test1/156/12' 2023-10-06T04:00:29.603 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/156/13' -> 'copyofsnap1/test1/156/13' 2023-10-06T04:00:29.603 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/156/14' -> 'copyofsnap1/test1/156/14' 2023-10-06T04:00:29.604 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/156/15' -> 'copyofsnap1/test1/156/15' 2023-10-06T04:00:29.604 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/156/16' -> 'copyofsnap1/test1/156/16' 2023-10-06T04:00:29.604 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/156/17' -> 'copyofsnap1/test1/156/17' 2023-10-06T04:00:29.604 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/156/18' -> 'copyofsnap1/test1/156/18' 2023-10-06T04:00:29.604 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/156/19' -> 'copyofsnap1/test1/156/19' 2023-10-06T04:00:29.604 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/156/20' -> 'copyofsnap1/test1/156/20' 2023-10-06T04:00:29.604 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/157' -> 'copyofsnap1/test1/157' 2023-10-06T04:00:29.604 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/157/10' -> 'copyofsnap1/test1/157/10' 2023-10-06T04:00:29.604 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/157/11' -> 'copyofsnap1/test1/157/11' 2023-10-06T04:00:29.604 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/157/12' -> 'copyofsnap1/test1/157/12' 2023-10-06T04:00:29.604 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/157/13' -> 'copyofsnap1/test1/157/13' 2023-10-06T04:00:29.604 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/157/14' -> 'copyofsnap1/test1/157/14' 2023-10-06T04:00:29.604 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/157/15' -> 'copyofsnap1/test1/157/15' 2023-10-06T04:00:29.605 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/157/16' -> 'copyofsnap1/test1/157/16' 2023-10-06T04:00:29.605 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/157/17' -> 'copyofsnap1/test1/157/17' 2023-10-06T04:00:29.605 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/157/18' -> 'copyofsnap1/test1/157/18' 2023-10-06T04:00:29.605 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/157/19' -> 'copyofsnap1/test1/157/19' 2023-10-06T04:00:29.605 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/157/20' -> 'copyofsnap1/test1/157/20' 2023-10-06T04:00:29.605 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/158' -> 'copyofsnap1/test1/158' 2023-10-06T04:00:29.605 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/158/10' -> 'copyofsnap1/test1/158/10' 2023-10-06T04:00:29.605 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/158/11' -> 'copyofsnap1/test1/158/11' 2023-10-06T04:00:29.605 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/158/12' -> 'copyofsnap1/test1/158/12' 2023-10-06T04:00:29.605 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/158/13' -> 'copyofsnap1/test1/158/13' 2023-10-06T04:00:29.605 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/158/14' -> 'copyofsnap1/test1/158/14' 2023-10-06T04:00:29.605 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/158/15' -> 'copyofsnap1/test1/158/15' 2023-10-06T04:00:29.606 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/158/16' -> 'copyofsnap1/test1/158/16' 2023-10-06T04:00:29.606 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/158/17' -> 'copyofsnap1/test1/158/17' 2023-10-06T04:00:29.606 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/158/18' -> 'copyofsnap1/test1/158/18' 2023-10-06T04:00:29.606 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/158/19' -> 'copyofsnap1/test1/158/19' 2023-10-06T04:00:29.606 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/158/20' -> 'copyofsnap1/test1/158/20' 2023-10-06T04:00:29.606 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/159' -> 'copyofsnap1/test1/159' 2023-10-06T04:00:29.606 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/159/10' -> 'copyofsnap1/test1/159/10' 2023-10-06T04:00:29.606 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/159/11' -> 'copyofsnap1/test1/159/11' 2023-10-06T04:00:29.606 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/159/12' -> 'copyofsnap1/test1/159/12' 2023-10-06T04:00:29.606 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/159/13' -> 'copyofsnap1/test1/159/13' 2023-10-06T04:00:29.606 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/159/14' -> 'copyofsnap1/test1/159/14' 2023-10-06T04:00:29.606 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/159/15' -> 'copyofsnap1/test1/159/15' 2023-10-06T04:00:29.607 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/159/16' -> 'copyofsnap1/test1/159/16' 2023-10-06T04:00:29.607 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/159/17' -> 'copyofsnap1/test1/159/17' 2023-10-06T04:00:29.607 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/159/18' -> 'copyofsnap1/test1/159/18' 2023-10-06T04:00:29.607 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/159/19' -> 'copyofsnap1/test1/159/19' 2023-10-06T04:00:29.607 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/159/20' -> 'copyofsnap1/test1/159/20' 2023-10-06T04:00:29.607 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/160' -> 'copyofsnap1/test1/160' 2023-10-06T04:00:29.607 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/160/10' -> 'copyofsnap1/test1/160/10' 2023-10-06T04:00:29.607 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/160/11' -> 'copyofsnap1/test1/160/11' 2023-10-06T04:00:29.607 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/160/12' -> 'copyofsnap1/test1/160/12' 2023-10-06T04:00:29.607 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/160/13' -> 'copyofsnap1/test1/160/13' 2023-10-06T04:00:29.607 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/160/14' -> 'copyofsnap1/test1/160/14' 2023-10-06T04:00:29.607 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/160/15' -> 'copyofsnap1/test1/160/15' 2023-10-06T04:00:29.607 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/160/16' -> 'copyofsnap1/test1/160/16' 2023-10-06T04:00:29.608 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/160/17' -> 'copyofsnap1/test1/160/17' 2023-10-06T04:00:35.739 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/160/18' -> 'copyofsnap1/test1/160/18' 2023-10-06T04:00:35.739 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/160/19' -> 'copyofsnap1/test1/160/19' 2023-10-06T04:00:35.740 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/160/20' -> 'copyofsnap1/test1/160/20' 2023-10-06T04:00:35.740 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/161' -> 'copyofsnap1/test1/161' 2023-10-06T04:00:35.740 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/161/10' -> 'copyofsnap1/test1/161/10' 2023-10-06T04:00:35.740 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/161/11' -> 'copyofsnap1/test1/161/11' 2023-10-06T04:00:35.740 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/161/12' -> 'copyofsnap1/test1/161/12' 2023-10-06T04:00:35.740 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/161/13' -> 'copyofsnap1/test1/161/13' 2023-10-06T04:00:35.740 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/161/14' -> 'copyofsnap1/test1/161/14' 2023-10-06T04:00:35.740 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/161/15' -> 'copyofsnap1/test1/161/15' 2023-10-06T04:00:35.740 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/161/16' -> 'copyofsnap1/test1/161/16' 2023-10-06T04:00:35.740 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/161/17' -> 'copyofsnap1/test1/161/17' 2023-10-06T04:00:35.740 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/161/18' -> 'copyofsnap1/test1/161/18' 2023-10-06T04:00:35.741 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/161/19' -> 'copyofsnap1/test1/161/19' 2023-10-06T04:00:35.741 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/161/20' -> 'copyofsnap1/test1/161/20' 2023-10-06T04:00:35.741 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/162' -> 'copyofsnap1/test1/162' 2023-10-06T04:00:35.741 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/162/10' -> 'copyofsnap1/test1/162/10' 2023-10-06T04:00:35.741 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/162/11' -> 'copyofsnap1/test1/162/11' 2023-10-06T04:00:35.741 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/162/12' -> 'copyofsnap1/test1/162/12' 2023-10-06T04:00:35.741 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/162/13' -> 'copyofsnap1/test1/162/13' 2023-10-06T04:00:35.741 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/162/14' -> 'copyofsnap1/test1/162/14' 2023-10-06T04:00:35.741 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/162/15' -> 'copyofsnap1/test1/162/15' 2023-10-06T04:00:35.741 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/162/16' -> 'copyofsnap1/test1/162/16' 2023-10-06T04:00:35.742 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/162/17' -> 'copyofsnap1/test1/162/17' 2023-10-06T04:00:35.742 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/162/18' -> 'copyofsnap1/test1/162/18' 2023-10-06T04:00:35.742 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/162/19' -> 'copyofsnap1/test1/162/19' 2023-10-06T04:00:35.742 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/162/20' -> 'copyofsnap1/test1/162/20' 2023-10-06T04:00:35.742 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/163' -> 'copyofsnap1/test1/163' 2023-10-06T04:00:35.742 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/163/10' -> 'copyofsnap1/test1/163/10' 2023-10-06T04:00:35.742 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/163/11' -> 'copyofsnap1/test1/163/11' 2023-10-06T04:00:35.742 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/163/12' -> 'copyofsnap1/test1/163/12' 2023-10-06T04:00:35.742 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/163/13' -> 'copyofsnap1/test1/163/13' 2023-10-06T04:00:35.742 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/163/14' -> 'copyofsnap1/test1/163/14' 2023-10-06T04:00:35.742 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/163/15' -> 'copyofsnap1/test1/163/15' 2023-10-06T04:00:35.743 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/163/16' -> 'copyofsnap1/test1/163/16' 2023-10-06T04:00:35.743 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/163/17' -> 'copyofsnap1/test1/163/17' 2023-10-06T04:00:35.743 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/163/18' -> 'copyofsnap1/test1/163/18' 2023-10-06T04:00:35.743 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/163/19' -> 'copyofsnap1/test1/163/19' 2023-10-06T04:00:35.743 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/163/20' -> 'copyofsnap1/test1/163/20' 2023-10-06T04:00:35.743 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/164' -> 'copyofsnap1/test1/164' 2023-10-06T04:00:35.743 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/164/10' -> 'copyofsnap1/test1/164/10' 2023-10-06T04:00:35.743 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/164/11' -> 'copyofsnap1/test1/164/11' 2023-10-06T04:00:35.743 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/164/12' -> 'copyofsnap1/test1/164/12' 2023-10-06T04:00:35.743 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/164/13' -> 'copyofsnap1/test1/164/13' 2023-10-06T04:00:35.743 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/164/14' -> 'copyofsnap1/test1/164/14' 2023-10-06T04:00:35.743 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/164/15' -> 'copyofsnap1/test1/164/15' 2023-10-06T04:00:35.744 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/164/16' -> 'copyofsnap1/test1/164/16' 2023-10-06T04:00:35.744 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/164/17' -> 'copyofsnap1/test1/164/17' 2023-10-06T04:00:35.744 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/164/18' -> 'copyofsnap1/test1/164/18' 2023-10-06T04:00:35.744 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/164/19' -> 'copyofsnap1/test1/164/19' 2023-10-06T04:00:35.744 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/164/20' -> 'copyofsnap1/test1/164/20' 2023-10-06T04:00:35.744 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/165' -> 'copyofsnap1/test1/165' 2023-10-06T04:00:35.744 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/165/10' -> 'copyofsnap1/test1/165/10' 2023-10-06T04:00:35.744 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/165/11' -> 'copyofsnap1/test1/165/11' 2023-10-06T04:00:35.744 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/165/12' -> 'copyofsnap1/test1/165/12' 2023-10-06T04:00:35.744 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/165/13' -> 'copyofsnap1/test1/165/13' 2023-10-06T04:00:35.744 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/165/14' -> 'copyofsnap1/test1/165/14' 2023-10-06T04:00:35.744 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/165/15' -> 'copyofsnap1/test1/165/15' 2023-10-06T04:00:35.745 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/165/16' -> 'copyofsnap1/test1/165/16' 2023-10-06T04:00:35.745 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/165/17' -> 'copyofsnap1/test1/165/17' 2023-10-06T04:00:35.745 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/165/18' -> 'copyofsnap1/test1/165/18' 2023-10-06T04:00:35.745 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/165/19' -> 'copyofsnap1/test1/165/19' 2023-10-06T04:00:35.745 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/165/20' -> 'copyofsnap1/test1/165/20' 2023-10-06T04:00:35.745 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/166' -> 'copyofsnap1/test1/166' 2023-10-06T04:00:35.745 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/166/10' -> 'copyofsnap1/test1/166/10' 2023-10-06T04:00:35.745 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/166/11' -> 'copyofsnap1/test1/166/11' 2023-10-06T04:00:35.746 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/166/12' -> 'copyofsnap1/test1/166/12' 2023-10-06T04:00:35.746 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/166/13' -> 'copyofsnap1/test1/166/13' 2023-10-06T04:00:35.746 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/166/14' -> 'copyofsnap1/test1/166/14' 2023-10-06T04:00:35.746 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/166/15' -> 'copyofsnap1/test1/166/15' 2023-10-06T04:00:35.746 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/166/16' -> 'copyofsnap1/test1/166/16' 2023-10-06T04:00:35.746 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/166/17' -> 'copyofsnap1/test1/166/17' 2023-10-06T04:00:35.746 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/166/18' -> 'copyofsnap1/test1/166/18' 2023-10-06T04:00:35.747 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/166/19' -> 'copyofsnap1/test1/166/19' 2023-10-06T04:00:35.747 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/166/20' -> 'copyofsnap1/test1/166/20' 2023-10-06T04:00:35.747 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/167' -> 'copyofsnap1/test1/167' 2023-10-06T04:00:35.747 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/167/10' -> 'copyofsnap1/test1/167/10' 2023-10-06T04:00:35.747 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/167/11' -> 'copyofsnap1/test1/167/11' 2023-10-06T04:00:35.747 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/167/12' -> 'copyofsnap1/test1/167/12' 2023-10-06T04:00:35.747 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/167/13' -> 'copyofsnap1/test1/167/13' 2023-10-06T04:00:35.748 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/167/14' -> 'copyofsnap1/test1/167/14' 2023-10-06T04:00:37.201 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/167/15' -> 'copyofsnap1/test1/167/15' 2023-10-06T04:00:37.201 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/167/16' -> 'copyofsnap1/test1/167/16' 2023-10-06T04:00:37.201 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/167/17' -> 'copyofsnap1/test1/167/17' 2023-10-06T04:00:37.201 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/167/18' -> 'copyofsnap1/test1/167/18' 2023-10-06T04:00:37.202 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/167/19' -> 'copyofsnap1/test1/167/19' 2023-10-06T04:00:37.202 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/167/20' -> 'copyofsnap1/test1/167/20' 2023-10-06T04:00:37.202 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/168' -> 'copyofsnap1/test1/168' 2023-10-06T04:00:37.202 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/168/10' -> 'copyofsnap1/test1/168/10' 2023-10-06T04:00:37.202 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/168/11' -> 'copyofsnap1/test1/168/11' 2023-10-06T04:00:37.202 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/168/12' -> 'copyofsnap1/test1/168/12' 2023-10-06T04:00:37.202 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/168/13' -> 'copyofsnap1/test1/168/13' 2023-10-06T04:00:37.202 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/168/14' -> 'copyofsnap1/test1/168/14' 2023-10-06T04:00:37.202 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/168/15' -> 'copyofsnap1/test1/168/15' 2023-10-06T04:00:37.202 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/168/16' -> 'copyofsnap1/test1/168/16' 2023-10-06T04:00:37.203 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/168/17' -> 'copyofsnap1/test1/168/17' 2023-10-06T04:00:37.203 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/168/18' -> 'copyofsnap1/test1/168/18' 2023-10-06T04:00:37.203 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/168/19' -> 'copyofsnap1/test1/168/19' 2023-10-06T04:00:37.203 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/168/20' -> 'copyofsnap1/test1/168/20' 2023-10-06T04:00:37.203 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/169' -> 'copyofsnap1/test1/169' 2023-10-06T04:00:37.203 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/169/10' -> 'copyofsnap1/test1/169/10' 2023-10-06T04:00:37.203 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/169/11' -> 'copyofsnap1/test1/169/11' 2023-10-06T04:00:37.203 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/169/12' -> 'copyofsnap1/test1/169/12' 2023-10-06T04:00:37.203 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/169/13' -> 'copyofsnap1/test1/169/13' 2023-10-06T04:00:37.203 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/169/14' -> 'copyofsnap1/test1/169/14' 2023-10-06T04:00:37.203 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/169/15' -> 'copyofsnap1/test1/169/15' 2023-10-06T04:00:37.203 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/169/16' -> 'copyofsnap1/test1/169/16' 2023-10-06T04:00:37.203 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/169/17' -> 'copyofsnap1/test1/169/17' 2023-10-06T04:00:37.204 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/169/18' -> 'copyofsnap1/test1/169/18' 2023-10-06T04:00:37.205 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/169/19' -> 'copyofsnap1/test1/169/19' 2023-10-06T04:00:37.205 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/169/20' -> 'copyofsnap1/test1/169/20' 2023-10-06T04:00:37.205 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/170' -> 'copyofsnap1/test1/170' 2023-10-06T04:00:37.205 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/170/10' -> 'copyofsnap1/test1/170/10' 2023-10-06T04:00:37.205 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/170/11' -> 'copyofsnap1/test1/170/11' 2023-10-06T04:00:37.205 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/170/12' -> 'copyofsnap1/test1/170/12' 2023-10-06T04:00:37.205 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/170/13' -> 'copyofsnap1/test1/170/13' 2023-10-06T04:00:37.205 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/170/14' -> 'copyofsnap1/test1/170/14' 2023-10-06T04:00:37.205 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/170/15' -> 'copyofsnap1/test1/170/15' 2023-10-06T04:00:37.205 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/170/16' -> 'copyofsnap1/test1/170/16' 2023-10-06T04:00:37.205 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/170/17' -> 'copyofsnap1/test1/170/17' 2023-10-06T04:00:37.205 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/170/18' -> 'copyofsnap1/test1/170/18' 2023-10-06T04:00:37.206 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/170/19' -> 'copyofsnap1/test1/170/19' 2023-10-06T04:00:37.206 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/170/20' -> 'copyofsnap1/test1/170/20' 2023-10-06T04:00:37.206 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/171' -> 'copyofsnap1/test1/171' 2023-10-06T04:00:37.206 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/171/10' -> 'copyofsnap1/test1/171/10' 2023-10-06T04:00:37.206 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/171/11' -> 'copyofsnap1/test1/171/11' 2023-10-06T04:00:37.206 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/171/12' -> 'copyofsnap1/test1/171/12' 2023-10-06T04:00:37.206 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/171/13' -> 'copyofsnap1/test1/171/13' 2023-10-06T04:00:37.206 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/171/14' -> 'copyofsnap1/test1/171/14' 2023-10-06T04:00:37.206 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/171/15' -> 'copyofsnap1/test1/171/15' 2023-10-06T04:00:37.206 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/171/16' -> 'copyofsnap1/test1/171/16' 2023-10-06T04:00:37.206 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/171/17' -> 'copyofsnap1/test1/171/17' 2023-10-06T04:00:37.207 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/171/18' -> 'copyofsnap1/test1/171/18' 2023-10-06T04:00:37.207 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/171/19' -> 'copyofsnap1/test1/171/19' 2023-10-06T04:00:37.207 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/171/20' -> 'copyofsnap1/test1/171/20' 2023-10-06T04:00:37.207 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/172' -> 'copyofsnap1/test1/172' 2023-10-06T04:00:37.207 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/172/10' -> 'copyofsnap1/test1/172/10' 2023-10-06T04:00:37.207 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/172/11' -> 'copyofsnap1/test1/172/11' 2023-10-06T04:00:37.207 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/172/12' -> 'copyofsnap1/test1/172/12' 2023-10-06T04:00:37.207 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/172/13' -> 'copyofsnap1/test1/172/13' 2023-10-06T04:00:37.207 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/172/14' -> 'copyofsnap1/test1/172/14' 2023-10-06T04:00:37.207 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/172/15' -> 'copyofsnap1/test1/172/15' 2023-10-06T04:00:37.207 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/172/16' -> 'copyofsnap1/test1/172/16' 2023-10-06T04:00:37.207 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/172/17' -> 'copyofsnap1/test1/172/17' 2023-10-06T04:00:37.208 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/172/18' -> 'copyofsnap1/test1/172/18' 2023-10-06T04:00:37.208 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/172/19' -> 'copyofsnap1/test1/172/19' 2023-10-06T04:00:37.208 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/172/20' -> 'copyofsnap1/test1/172/20' 2023-10-06T04:00:37.208 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/173' -> 'copyofsnap1/test1/173' 2023-10-06T04:00:37.208 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/173/10' -> 'copyofsnap1/test1/173/10' 2023-10-06T04:00:37.208 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/173/11' -> 'copyofsnap1/test1/173/11' 2023-10-06T04:00:37.208 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/173/12' -> 'copyofsnap1/test1/173/12' 2023-10-06T04:00:37.208 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/173/13' -> 'copyofsnap1/test1/173/13' 2023-10-06T04:00:37.208 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/173/14' -> 'copyofsnap1/test1/173/14' 2023-10-06T04:00:37.208 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/173/15' -> 'copyofsnap1/test1/173/15' 2023-10-06T04:00:37.208 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/173/16' -> 'copyofsnap1/test1/173/16' 2023-10-06T04:00:37.208 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/173/17' -> 'copyofsnap1/test1/173/17' 2023-10-06T04:00:37.208 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/173/18' -> 'copyofsnap1/test1/173/18' 2023-10-06T04:00:37.209 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/173/19' -> 'copyofsnap1/test1/173/19' 2023-10-06T04:00:37.209 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/173/20' -> 'copyofsnap1/test1/173/20' 2023-10-06T04:00:37.209 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/174' -> 'copyofsnap1/test1/174' 2023-10-06T04:00:37.209 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/174/10' -> 'copyofsnap1/test1/174/10' 2023-10-06T04:00:37.209 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/174/11' -> 'copyofsnap1/test1/174/11' 2023-10-06T04:00:38.568 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/174/12' -> 'copyofsnap1/test1/174/12' 2023-10-06T04:00:38.568 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/174/13' -> 'copyofsnap1/test1/174/13' 2023-10-06T04:00:38.569 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/174/14' -> 'copyofsnap1/test1/174/14' 2023-10-06T04:00:38.569 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/174/15' -> 'copyofsnap1/test1/174/15' 2023-10-06T04:00:38.569 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/174/16' -> 'copyofsnap1/test1/174/16' 2023-10-06T04:00:38.569 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/174/17' -> 'copyofsnap1/test1/174/17' 2023-10-06T04:00:38.569 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/174/18' -> 'copyofsnap1/test1/174/18' 2023-10-06T04:00:38.569 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/174/19' -> 'copyofsnap1/test1/174/19' 2023-10-06T04:00:38.569 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/174/20' -> 'copyofsnap1/test1/174/20' 2023-10-06T04:00:38.569 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/175' -> 'copyofsnap1/test1/175' 2023-10-06T04:00:38.569 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/175/10' -> 'copyofsnap1/test1/175/10' 2023-10-06T04:00:38.569 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/175/11' -> 'copyofsnap1/test1/175/11' 2023-10-06T04:00:38.570 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/175/12' -> 'copyofsnap1/test1/175/12' 2023-10-06T04:00:38.570 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/175/13' -> 'copyofsnap1/test1/175/13' 2023-10-06T04:00:38.570 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/175/14' -> 'copyofsnap1/test1/175/14' 2023-10-06T04:00:38.570 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/175/15' -> 'copyofsnap1/test1/175/15' 2023-10-06T04:00:38.570 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/175/16' -> 'copyofsnap1/test1/175/16' 2023-10-06T04:00:38.570 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/175/17' -> 'copyofsnap1/test1/175/17' 2023-10-06T04:00:38.570 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/175/18' -> 'copyofsnap1/test1/175/18' 2023-10-06T04:00:38.570 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/175/19' -> 'copyofsnap1/test1/175/19' 2023-10-06T04:00:38.570 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/175/20' -> 'copyofsnap1/test1/175/20' 2023-10-06T04:00:38.570 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/176' -> 'copyofsnap1/test1/176' 2023-10-06T04:00:38.570 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/176/10' -> 'copyofsnap1/test1/176/10' 2023-10-06T04:00:38.571 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/176/11' -> 'copyofsnap1/test1/176/11' 2023-10-06T04:00:38.571 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/176/12' -> 'copyofsnap1/test1/176/12' 2023-10-06T04:00:38.571 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/176/13' -> 'copyofsnap1/test1/176/13' 2023-10-06T04:00:38.571 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/176/14' -> 'copyofsnap1/test1/176/14' 2023-10-06T04:00:38.571 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/176/15' -> 'copyofsnap1/test1/176/15' 2023-10-06T04:00:38.571 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/176/16' -> 'copyofsnap1/test1/176/16' 2023-10-06T04:00:38.571 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/176/17' -> 'copyofsnap1/test1/176/17' 2023-10-06T04:00:38.571 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/176/18' -> 'copyofsnap1/test1/176/18' 2023-10-06T04:00:38.571 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/176/19' -> 'copyofsnap1/test1/176/19' 2023-10-06T04:00:38.571 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/176/20' -> 'copyofsnap1/test1/176/20' 2023-10-06T04:00:38.571 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/177' -> 'copyofsnap1/test1/177' 2023-10-06T04:00:38.572 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/177/10' -> 'copyofsnap1/test1/177/10' 2023-10-06T04:00:38.572 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/177/11' -> 'copyofsnap1/test1/177/11' 2023-10-06T04:00:38.572 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/177/12' -> 'copyofsnap1/test1/177/12' 2023-10-06T04:00:38.572 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/177/13' -> 'copyofsnap1/test1/177/13' 2023-10-06T04:00:38.572 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/177/14' -> 'copyofsnap1/test1/177/14' 2023-10-06T04:00:38.572 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/177/15' -> 'copyofsnap1/test1/177/15' 2023-10-06T04:00:38.572 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/177/16' -> 'copyofsnap1/test1/177/16' 2023-10-06T04:00:38.572 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/177/17' -> 'copyofsnap1/test1/177/17' 2023-10-06T04:00:38.572 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/177/18' -> 'copyofsnap1/test1/177/18' 2023-10-06T04:00:38.572 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/177/19' -> 'copyofsnap1/test1/177/19' 2023-10-06T04:00:38.572 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/177/20' -> 'copyofsnap1/test1/177/20' 2023-10-06T04:00:38.573 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/178' -> 'copyofsnap1/test1/178' 2023-10-06T04:00:38.573 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/178/10' -> 'copyofsnap1/test1/178/10' 2023-10-06T04:00:38.573 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/178/11' -> 'copyofsnap1/test1/178/11' 2023-10-06T04:00:38.573 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/178/12' -> 'copyofsnap1/test1/178/12' 2023-10-06T04:00:38.573 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/178/13' -> 'copyofsnap1/test1/178/13' 2023-10-06T04:00:38.573 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/178/14' -> 'copyofsnap1/test1/178/14' 2023-10-06T04:00:38.573 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/178/15' -> 'copyofsnap1/test1/178/15' 2023-10-06T04:00:38.573 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/178/16' -> 'copyofsnap1/test1/178/16' 2023-10-06T04:00:38.573 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/178/17' -> 'copyofsnap1/test1/178/17' 2023-10-06T04:00:38.573 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/178/18' -> 'copyofsnap1/test1/178/18' 2023-10-06T04:00:38.573 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/178/19' -> 'copyofsnap1/test1/178/19' 2023-10-06T04:00:38.573 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/178/20' -> 'copyofsnap1/test1/178/20' 2023-10-06T04:00:38.574 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/179' -> 'copyofsnap1/test1/179' 2023-10-06T04:00:38.574 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/179/10' -> 'copyofsnap1/test1/179/10' 2023-10-06T04:00:38.574 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/179/11' -> 'copyofsnap1/test1/179/11' 2023-10-06T04:00:38.574 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/179/12' -> 'copyofsnap1/test1/179/12' 2023-10-06T04:00:38.574 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/179/13' -> 'copyofsnap1/test1/179/13' 2023-10-06T04:00:38.574 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/179/14' -> 'copyofsnap1/test1/179/14' 2023-10-06T04:00:38.574 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/179/15' -> 'copyofsnap1/test1/179/15' 2023-10-06T04:00:38.574 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/179/16' -> 'copyofsnap1/test1/179/16' 2023-10-06T04:00:38.574 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/179/17' -> 'copyofsnap1/test1/179/17' 2023-10-06T04:00:38.574 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/179/18' -> 'copyofsnap1/test1/179/18' 2023-10-06T04:00:38.575 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/179/19' -> 'copyofsnap1/test1/179/19' 2023-10-06T04:00:38.575 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/179/20' -> 'copyofsnap1/test1/179/20' 2023-10-06T04:00:38.575 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/180' -> 'copyofsnap1/test1/180' 2023-10-06T04:00:38.575 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/180/10' -> 'copyofsnap1/test1/180/10' 2023-10-06T04:00:38.575 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/180/11' -> 'copyofsnap1/test1/180/11' 2023-10-06T04:00:38.575 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/180/12' -> 'copyofsnap1/test1/180/12' 2023-10-06T04:00:38.575 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/180/13' -> 'copyofsnap1/test1/180/13' 2023-10-06T04:00:38.575 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/180/14' -> 'copyofsnap1/test1/180/14' 2023-10-06T04:00:38.575 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/180/15' -> 'copyofsnap1/test1/180/15' 2023-10-06T04:00:38.575 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/180/16' -> 'copyofsnap1/test1/180/16' 2023-10-06T04:00:38.575 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/180/17' -> 'copyofsnap1/test1/180/17' 2023-10-06T04:00:38.576 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/180/18' -> 'copyofsnap1/test1/180/18' 2023-10-06T04:00:38.576 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/180/19' -> 'copyofsnap1/test1/180/19' 2023-10-06T04:00:38.576 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/180/20' -> 'copyofsnap1/test1/180/20' 2023-10-06T04:00:38.709 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/181' -> 'copyofsnap1/test1/181' 2023-10-06T04:00:38.709 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/181/10' -> 'copyofsnap1/test1/181/10' 2023-10-06T04:00:38.709 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/181/11' -> 'copyofsnap1/test1/181/11' 2023-10-06T04:00:38.709 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/181/12' -> 'copyofsnap1/test1/181/12' 2023-10-06T04:00:38.709 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/181/13' -> 'copyofsnap1/test1/181/13' 2023-10-06T04:00:38.709 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/181/14' -> 'copyofsnap1/test1/181/14' 2023-10-06T04:00:38.709 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/181/15' -> 'copyofsnap1/test1/181/15' 2023-10-06T04:00:38.709 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/181/16' -> 'copyofsnap1/test1/181/16' 2023-10-06T04:00:38.709 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/181/17' -> 'copyofsnap1/test1/181/17' 2023-10-06T04:00:38.709 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/181/18' -> 'copyofsnap1/test1/181/18' 2023-10-06T04:00:38.710 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/181/19' -> 'copyofsnap1/test1/181/19' 2023-10-06T04:00:38.710 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/181/20' -> 'copyofsnap1/test1/181/20' 2023-10-06T04:00:38.710 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/182' -> 'copyofsnap1/test1/182' 2023-10-06T04:00:38.710 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/182/10' -> 'copyofsnap1/test1/182/10' 2023-10-06T04:00:38.710 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/182/11' -> 'copyofsnap1/test1/182/11' 2023-10-06T04:00:38.710 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/182/12' -> 'copyofsnap1/test1/182/12' 2023-10-06T04:00:38.710 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/182/13' -> 'copyofsnap1/test1/182/13' 2023-10-06T04:00:38.710 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/182/14' -> 'copyofsnap1/test1/182/14' 2023-10-06T04:00:38.710 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/182/15' -> 'copyofsnap1/test1/182/15' 2023-10-06T04:00:38.710 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/182/16' -> 'copyofsnap1/test1/182/16' 2023-10-06T04:00:38.710 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/182/17' -> 'copyofsnap1/test1/182/17' 2023-10-06T04:00:38.711 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/182/19' -> 'copyofsnap1/test1/182/19' 2023-10-06T04:00:38.711 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/182/20' -> 'copyofsnap1/test1/182/20' 2023-10-06T04:00:38.711 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/182/18' -> 'copyofsnap1/test1/182/18' 2023-10-06T04:00:38.711 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/183' -> 'copyofsnap1/test1/183' 2023-10-06T04:00:38.711 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/183/10' -> 'copyofsnap1/test1/183/10' 2023-10-06T04:00:38.711 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/183/11' -> 'copyofsnap1/test1/183/11' 2023-10-06T04:00:38.711 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/183/12' -> 'copyofsnap1/test1/183/12' 2023-10-06T04:00:38.711 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/183/13' -> 'copyofsnap1/test1/183/13' 2023-10-06T04:00:38.711 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/183/14' -> 'copyofsnap1/test1/183/14' 2023-10-06T04:00:38.711 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/183/15' -> 'copyofsnap1/test1/183/15' 2023-10-06T04:00:38.711 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/183/16' -> 'copyofsnap1/test1/183/16' 2023-10-06T04:00:38.711 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/183/17' -> 'copyofsnap1/test1/183/17' 2023-10-06T04:00:38.712 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/183/18' -> 'copyofsnap1/test1/183/18' 2023-10-06T04:00:38.712 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/183/19' -> 'copyofsnap1/test1/183/19' 2023-10-06T04:00:38.712 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/183/20' -> 'copyofsnap1/test1/183/20' 2023-10-06T04:00:38.712 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/184' -> 'copyofsnap1/test1/184' 2023-10-06T04:00:38.712 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/184/10' -> 'copyofsnap1/test1/184/10' 2023-10-06T04:00:38.712 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/184/11' -> 'copyofsnap1/test1/184/11' 2023-10-06T04:00:38.712 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/184/12' -> 'copyofsnap1/test1/184/12' 2023-10-06T04:00:38.712 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/184/13' -> 'copyofsnap1/test1/184/13' 2023-10-06T04:00:38.712 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/184/14' -> 'copyofsnap1/test1/184/14' 2023-10-06T04:00:38.712 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/184/15' -> 'copyofsnap1/test1/184/15' 2023-10-06T04:00:38.712 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/184/16' -> 'copyofsnap1/test1/184/16' 2023-10-06T04:00:38.712 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/184/17' -> 'copyofsnap1/test1/184/17' 2023-10-06T04:00:38.712 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/184/18' -> 'copyofsnap1/test1/184/18' 2023-10-06T04:00:38.713 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/184/19' -> 'copyofsnap1/test1/184/19' 2023-10-06T04:00:38.713 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/184/20' -> 'copyofsnap1/test1/184/20' 2023-10-06T04:00:38.713 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/185' -> 'copyofsnap1/test1/185' 2023-10-06T04:00:38.713 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/185/10' -> 'copyofsnap1/test1/185/10' 2023-10-06T04:00:38.713 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/185/11' -> 'copyofsnap1/test1/185/11' 2023-10-06T04:00:38.713 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/185/12' -> 'copyofsnap1/test1/185/12' 2023-10-06T04:00:38.713 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/185/13' -> 'copyofsnap1/test1/185/13' 2023-10-06T04:00:38.713 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/185/14' -> 'copyofsnap1/test1/185/14' 2023-10-06T04:00:38.713 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/185/15' -> 'copyofsnap1/test1/185/15' 2023-10-06T04:00:38.713 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/185/16' -> 'copyofsnap1/test1/185/16' 2023-10-06T04:00:38.713 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/185/17' -> 'copyofsnap1/test1/185/17' 2023-10-06T04:00:38.713 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/185/18' -> 'copyofsnap1/test1/185/18' 2023-10-06T04:00:38.714 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/185/19' -> 'copyofsnap1/test1/185/19' 2023-10-06T04:00:38.714 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/185/20' -> 'copyofsnap1/test1/185/20' 2023-10-06T04:00:38.714 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/186' -> 'copyofsnap1/test1/186' 2023-10-06T04:00:38.714 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/186/10' -> 'copyofsnap1/test1/186/10' 2023-10-06T04:00:38.714 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/186/11' -> 'copyofsnap1/test1/186/11' 2023-10-06T04:00:38.714 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/186/12' -> 'copyofsnap1/test1/186/12' 2023-10-06T04:00:38.714 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/186/13' -> 'copyofsnap1/test1/186/13' 2023-10-06T04:00:38.714 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/186/14' -> 'copyofsnap1/test1/186/14' 2023-10-06T04:00:38.714 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/186/15' -> 'copyofsnap1/test1/186/15' 2023-10-06T04:00:38.714 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/186/16' -> 'copyofsnap1/test1/186/16' 2023-10-06T04:00:38.714 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/186/17' -> 'copyofsnap1/test1/186/17' 2023-10-06T04:00:38.714 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/186/18' -> 'copyofsnap1/test1/186/18' 2023-10-06T04:00:38.715 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/186/19' -> 'copyofsnap1/test1/186/19' 2023-10-06T04:00:38.715 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/186/20' -> 'copyofsnap1/test1/186/20' 2023-10-06T04:00:38.715 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/187' -> 'copyofsnap1/test1/187' 2023-10-06T04:00:38.715 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/187/10' -> 'copyofsnap1/test1/187/10' 2023-10-06T04:00:38.715 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/187/11' -> 'copyofsnap1/test1/187/11' 2023-10-06T04:00:38.715 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/187/12' -> 'copyofsnap1/test1/187/12' 2023-10-06T04:00:38.715 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/187/13' -> 'copyofsnap1/test1/187/13' 2023-10-06T04:00:38.715 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/187/14' -> 'copyofsnap1/test1/187/14' 2023-10-06T04:00:38.715 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/187/15' -> 'copyofsnap1/test1/187/15' 2023-10-06T04:00:38.715 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/187/16' -> 'copyofsnap1/test1/187/16' 2023-10-06T04:00:38.715 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/187/17' -> 'copyofsnap1/test1/187/17' 2023-10-06T04:00:40.376 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/187/18' -> 'copyofsnap1/test1/187/18' 2023-10-06T04:00:40.376 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/187/19' -> 'copyofsnap1/test1/187/19' 2023-10-06T04:00:40.376 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/187/20' -> 'copyofsnap1/test1/187/20' 2023-10-06T04:00:40.376 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/188' -> 'copyofsnap1/test1/188' 2023-10-06T04:00:40.376 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/188/10' -> 'copyofsnap1/test1/188/10' 2023-10-06T04:00:40.376 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/188/11' -> 'copyofsnap1/test1/188/11' 2023-10-06T04:00:40.377 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/188/12' -> 'copyofsnap1/test1/188/12' 2023-10-06T04:00:40.377 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/188/13' -> 'copyofsnap1/test1/188/13' 2023-10-06T04:00:40.377 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/188/14' -> 'copyofsnap1/test1/188/14' 2023-10-06T04:00:40.377 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/188/15' -> 'copyofsnap1/test1/188/15' 2023-10-06T04:00:40.377 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/188/16' -> 'copyofsnap1/test1/188/16' 2023-10-06T04:00:40.377 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/188/17' -> 'copyofsnap1/test1/188/17' 2023-10-06T04:00:40.377 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/188/18' -> 'copyofsnap1/test1/188/18' 2023-10-06T04:00:40.377 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/188/19' -> 'copyofsnap1/test1/188/19' 2023-10-06T04:00:40.377 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/188/20' -> 'copyofsnap1/test1/188/20' 2023-10-06T04:00:40.377 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/189' -> 'copyofsnap1/test1/189' 2023-10-06T04:00:40.377 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/189/10' -> 'copyofsnap1/test1/189/10' 2023-10-06T04:00:40.377 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/189/11' -> 'copyofsnap1/test1/189/11' 2023-10-06T04:00:40.378 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/189/12' -> 'copyofsnap1/test1/189/12' 2023-10-06T04:00:40.378 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/189/13' -> 'copyofsnap1/test1/189/13' 2023-10-06T04:00:40.378 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/189/14' -> 'copyofsnap1/test1/189/14' 2023-10-06T04:00:40.378 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/189/15' -> 'copyofsnap1/test1/189/15' 2023-10-06T04:00:40.378 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/189/16' -> 'copyofsnap1/test1/189/16' 2023-10-06T04:00:40.378 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/189/17' -> 'copyofsnap1/test1/189/17' 2023-10-06T04:00:40.378 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/189/18' -> 'copyofsnap1/test1/189/18' 2023-10-06T04:00:40.378 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/189/19' -> 'copyofsnap1/test1/189/19' 2023-10-06T04:00:40.378 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/189/20' -> 'copyofsnap1/test1/189/20' 2023-10-06T04:00:40.378 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/190' -> 'copyofsnap1/test1/190' 2023-10-06T04:00:40.378 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/190/10' -> 'copyofsnap1/test1/190/10' 2023-10-06T04:00:40.379 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/190/11' -> 'copyofsnap1/test1/190/11' 2023-10-06T04:00:40.379 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/190/12' -> 'copyofsnap1/test1/190/12' 2023-10-06T04:00:40.379 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/190/13' -> 'copyofsnap1/test1/190/13' 2023-10-06T04:00:40.379 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/190/14' -> 'copyofsnap1/test1/190/14' 2023-10-06T04:00:40.379 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/190/15' -> 'copyofsnap1/test1/190/15' 2023-10-06T04:00:40.379 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/190/16' -> 'copyofsnap1/test1/190/16' 2023-10-06T04:00:40.379 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/190/17' -> 'copyofsnap1/test1/190/17' 2023-10-06T04:00:40.379 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/190/18' -> 'copyofsnap1/test1/190/18' 2023-10-06T04:00:40.379 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/190/19' -> 'copyofsnap1/test1/190/19' 2023-10-06T04:00:40.379 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/190/20' -> 'copyofsnap1/test1/190/20' 2023-10-06T04:00:40.379 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/191' -> 'copyofsnap1/test1/191' 2023-10-06T04:00:40.380 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/191/10' -> 'copyofsnap1/test1/191/10' 2023-10-06T04:00:40.380 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/191/11' -> 'copyofsnap1/test1/191/11' 2023-10-06T04:00:40.380 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/191/12' -> 'copyofsnap1/test1/191/12' 2023-10-06T04:00:40.380 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/191/13' -> 'copyofsnap1/test1/191/13' 2023-10-06T04:00:40.380 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/191/14' -> 'copyofsnap1/test1/191/14' 2023-10-06T04:00:40.380 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/191/15' -> 'copyofsnap1/test1/191/15' 2023-10-06T04:00:40.380 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/191/16' -> 'copyofsnap1/test1/191/16' 2023-10-06T04:00:40.380 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/191/17' -> 'copyofsnap1/test1/191/17' 2023-10-06T04:00:40.380 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/191/18' -> 'copyofsnap1/test1/191/18' 2023-10-06T04:00:40.380 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/191/19' -> 'copyofsnap1/test1/191/19' 2023-10-06T04:00:40.380 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/191/20' -> 'copyofsnap1/test1/191/20' 2023-10-06T04:00:40.380 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/192' -> 'copyofsnap1/test1/192' 2023-10-06T04:00:40.381 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/192/10' -> 'copyofsnap1/test1/192/10' 2023-10-06T04:00:40.381 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/192/11' -> 'copyofsnap1/test1/192/11' 2023-10-06T04:00:40.381 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/192/12' -> 'copyofsnap1/test1/192/12' 2023-10-06T04:00:40.381 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/192/13' -> 'copyofsnap1/test1/192/13' 2023-10-06T04:00:40.381 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/192/14' -> 'copyofsnap1/test1/192/14' 2023-10-06T04:00:40.381 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/192/15' -> 'copyofsnap1/test1/192/15' 2023-10-06T04:00:40.381 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/192/16' -> 'copyofsnap1/test1/192/16' 2023-10-06T04:00:40.381 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/192/17' -> 'copyofsnap1/test1/192/17' 2023-10-06T04:00:40.381 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/192/18' -> 'copyofsnap1/test1/192/18' 2023-10-06T04:00:40.381 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/192/19' -> 'copyofsnap1/test1/192/19' 2023-10-06T04:00:40.381 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/192/20' -> 'copyofsnap1/test1/192/20' 2023-10-06T04:00:40.381 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/193' -> 'copyofsnap1/test1/193' 2023-10-06T04:00:40.382 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/193/10' -> 'copyofsnap1/test1/193/10' 2023-10-06T04:00:40.382 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/193/11' -> 'copyofsnap1/test1/193/11' 2023-10-06T04:00:40.382 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/193/12' -> 'copyofsnap1/test1/193/12' 2023-10-06T04:00:40.382 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/193/13' -> 'copyofsnap1/test1/193/13' 2023-10-06T04:00:40.382 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/193/14' -> 'copyofsnap1/test1/193/14' 2023-10-06T04:00:40.382 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/193/15' -> 'copyofsnap1/test1/193/15' 2023-10-06T04:00:40.382 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/193/16' -> 'copyofsnap1/test1/193/16' 2023-10-06T04:00:40.382 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/193/17' -> 'copyofsnap1/test1/193/17' 2023-10-06T04:00:40.382 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/193/18' -> 'copyofsnap1/test1/193/18' 2023-10-06T04:00:40.382 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/193/19' -> 'copyofsnap1/test1/193/19' 2023-10-06T04:00:40.382 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/193/20' -> 'copyofsnap1/test1/193/20' 2023-10-06T04:00:40.383 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/194' -> 'copyofsnap1/test1/194' 2023-10-06T04:00:40.383 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/194/10' -> 'copyofsnap1/test1/194/10' 2023-10-06T04:00:40.383 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/194/11' -> 'copyofsnap1/test1/194/11' 2023-10-06T04:00:40.383 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/194/12' -> 'copyofsnap1/test1/194/12' 2023-10-06T04:00:40.383 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/194/13' -> 'copyofsnap1/test1/194/13' 2023-10-06T04:00:40.383 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/194/14' -> 'copyofsnap1/test1/194/14' 2023-10-06T04:00:41.465 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/194/15' -> 'copyofsnap1/test1/194/15' 2023-10-06T04:00:41.465 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/194/16' -> 'copyofsnap1/test1/194/16' 2023-10-06T04:00:41.465 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/194/17' -> 'copyofsnap1/test1/194/17' 2023-10-06T04:00:41.465 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/194/18' -> 'copyofsnap1/test1/194/18' 2023-10-06T04:00:41.466 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/194/19' -> 'copyofsnap1/test1/194/19' 2023-10-06T04:00:41.466 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/194/20' -> 'copyofsnap1/test1/194/20' 2023-10-06T04:00:41.466 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/195' -> 'copyofsnap1/test1/195' 2023-10-06T04:00:41.466 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/195/10' -> 'copyofsnap1/test1/195/10' 2023-10-06T04:00:41.466 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/195/11' -> 'copyofsnap1/test1/195/11' 2023-10-06T04:00:41.466 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/195/12' -> 'copyofsnap1/test1/195/12' 2023-10-06T04:00:41.466 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/195/13' -> 'copyofsnap1/test1/195/13' 2023-10-06T04:00:41.466 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/195/14' -> 'copyofsnap1/test1/195/14' 2023-10-06T04:00:41.466 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/195/15' -> 'copyofsnap1/test1/195/15' 2023-10-06T04:00:41.466 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/195/16' -> 'copyofsnap1/test1/195/16' 2023-10-06T04:00:41.467 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/195/17' -> 'copyofsnap1/test1/195/17' 2023-10-06T04:00:41.467 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/195/18' -> 'copyofsnap1/test1/195/18' 2023-10-06T04:00:41.467 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/195/19' -> 'copyofsnap1/test1/195/19' 2023-10-06T04:00:41.467 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/195/20' -> 'copyofsnap1/test1/195/20' 2023-10-06T04:00:41.467 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/196' -> 'copyofsnap1/test1/196' 2023-10-06T04:00:41.467 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/196/10' -> 'copyofsnap1/test1/196/10' 2023-10-06T04:00:41.467 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/196/11' -> 'copyofsnap1/test1/196/11' 2023-10-06T04:00:41.467 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/196/12' -> 'copyofsnap1/test1/196/12' 2023-10-06T04:00:41.467 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/196/13' -> 'copyofsnap1/test1/196/13' 2023-10-06T04:00:41.468 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/196/14' -> 'copyofsnap1/test1/196/14' 2023-10-06T04:00:41.468 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/196/15' -> 'copyofsnap1/test1/196/15' 2023-10-06T04:00:41.468 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/196/16' -> 'copyofsnap1/test1/196/16' 2023-10-06T04:00:41.468 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/196/17' -> 'copyofsnap1/test1/196/17' 2023-10-06T04:00:41.468 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/196/18' -> 'copyofsnap1/test1/196/18' 2023-10-06T04:00:41.468 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/196/19' -> 'copyofsnap1/test1/196/19' 2023-10-06T04:00:41.468 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/196/20' -> 'copyofsnap1/test1/196/20' 2023-10-06T04:00:41.468 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/197' -> 'copyofsnap1/test1/197' 2023-10-06T04:00:41.468 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/197/10' -> 'copyofsnap1/test1/197/10' 2023-10-06T04:00:41.468 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/197/11' -> 'copyofsnap1/test1/197/11' 2023-10-06T04:00:41.469 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/197/12' -> 'copyofsnap1/test1/197/12' 2023-10-06T04:00:41.469 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/197/13' -> 'copyofsnap1/test1/197/13' 2023-10-06T04:00:41.469 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/197/14' -> 'copyofsnap1/test1/197/14' 2023-10-06T04:00:41.469 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/197/15' -> 'copyofsnap1/test1/197/15' 2023-10-06T04:00:41.469 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/197/16' -> 'copyofsnap1/test1/197/16' 2023-10-06T04:00:41.469 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/197/17' -> 'copyofsnap1/test1/197/17' 2023-10-06T04:00:41.469 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/197/18' -> 'copyofsnap1/test1/197/18' 2023-10-06T04:00:41.469 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/197/19' -> 'copyofsnap1/test1/197/19' 2023-10-06T04:00:41.469 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/197/20' -> 'copyofsnap1/test1/197/20' 2023-10-06T04:00:41.469 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/198' -> 'copyofsnap1/test1/198' 2023-10-06T04:00:41.470 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/198/10' -> 'copyofsnap1/test1/198/10' 2023-10-06T04:00:41.470 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/198/11' -> 'copyofsnap1/test1/198/11' 2023-10-06T04:00:41.470 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/198/12' -> 'copyofsnap1/test1/198/12' 2023-10-06T04:00:41.470 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/198/13' -> 'copyofsnap1/test1/198/13' 2023-10-06T04:00:41.470 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/198/14' -> 'copyofsnap1/test1/198/14' 2023-10-06T04:00:41.470 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/198/15' -> 'copyofsnap1/test1/198/15' 2023-10-06T04:00:41.470 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/198/16' -> 'copyofsnap1/test1/198/16' 2023-10-06T04:00:41.470 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/198/17' -> 'copyofsnap1/test1/198/17' 2023-10-06T04:00:41.470 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/198/18' -> 'copyofsnap1/test1/198/18' 2023-10-06T04:00:41.470 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/198/19' -> 'copyofsnap1/test1/198/19' 2023-10-06T04:00:41.471 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/198/20' -> 'copyofsnap1/test1/198/20' 2023-10-06T04:00:41.471 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/199' -> 'copyofsnap1/test1/199' 2023-10-06T04:00:41.471 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/199/10' -> 'copyofsnap1/test1/199/10' 2023-10-06T04:00:41.471 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/199/11' -> 'copyofsnap1/test1/199/11' 2023-10-06T04:00:41.471 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/199/12' -> 'copyofsnap1/test1/199/12' 2023-10-06T04:00:41.471 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/199/13' -> 'copyofsnap1/test1/199/13' 2023-10-06T04:00:41.471 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/199/14' -> 'copyofsnap1/test1/199/14' 2023-10-06T04:00:41.471 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/199/15' -> 'copyofsnap1/test1/199/15' 2023-10-06T04:00:41.472 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/199/16' -> 'copyofsnap1/test1/199/16' 2023-10-06T04:00:41.472 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/199/17' -> 'copyofsnap1/test1/199/17' 2023-10-06T04:00:41.472 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/199/18' -> 'copyofsnap1/test1/199/18' 2023-10-06T04:00:41.472 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/199/19' -> 'copyofsnap1/test1/199/19' 2023-10-06T04:00:41.472 INFO:tasks.workunit.client.0.smithi157.stdout:'.snap/test1/199/20' -> 'copyofsnap1/test1/199/20' 2023-10-06T04:00:41.472 INFO:tasks.workunit.client.0.smithi157.stdout:Take third snapshot .snap/test3 2023-10-06T04:00:41.493 INFO:tasks.workunit.client.0.smithi157.stdout:Delete the snapshots... 2023-10-06T04:00:43.793 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:00:43.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.938+0000 7fcef60c3700 1 Processor -- start 2023-10-06T04:00:43.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.938+0000 7fcef60c3700 1 -- start start 2023-10-06T04:00:43.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.939+0000 7fcef60c3700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcef00ad780 0x7fcef0152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:43.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.939+0000 7fcef60c3700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef0152a20 con 0x7fcef006f8c0 2023-10-06T04:00:43.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.939+0000 7fcef60c3700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef00c4dc0 con 0x7fcef006bb70 2023-10-06T04:00:43.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.939+0000 7fcef60c3700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef006fcc0 con 0x7fcef00ad780 2023-10-06T04:00:43.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.939+0000 7fcef50c1700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcef00ad780 0x7fcef0152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:43.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.939+0000 7fcef48c0700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fcef006f8c0 0x7fcef006b460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43486/0 (socket says 172.21.15.157:43486) 2023-10-06T04:00:43.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.939+0000 7fcef48c0700 1 -- 172.21.15.157:0/3139558413 learned_addr learned my addr 172.21.15.157:0/3139558413 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:00:43.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.939+0000 7fcef50c1700 1 -- 172.21.15.157:0/3139558413 >> v1:172.21.15.175:6789/0 conn(0x7fcef006bb70 legacy=0x7fcef015cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:00:43.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.939+0000 7fcef50c1700 1 -- 172.21.15.157:0/3139558413 >> v1:172.21.15.157:6789/0 conn(0x7fcef006f8c0 legacy=0x7fcef006b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:43.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.939+0000 7fcef50c1700 1 -- 172.21.15.157:0/3139558413 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef00c5060 con 0x7fcef00ad780 2023-10-06T04:00:43.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.939+0000 7fcee7fff700 1 -- 172.21.15.157:0/3139558413 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 604766815 0 0) 0x7fcef0152a20 con 0x7fcef006f8c0 2023-10-06T04:00:43.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.940+0000 7fcef50c1700 1 --2- 172.21.15.157:0/3139558413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcef00ad780 0x7fcef0152eb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fceec0098e0 tx=0x7fceec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb6bb00ee87563c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:00:43.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.940+0000 7fcee7fff700 1 -- 172.21.15.157:0/3139558413 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec009e80 con 0x7fcef00ad780 2023-10-06T04:00:43.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.940+0000 7fcee7fff700 1 -- 172.21.15.157:0/3139558413 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceec00b8b0 con 0x7fcef00ad780 2023-10-06T04:00:43.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.940+0000 7fcee7fff700 1 -- 172.21.15.157:0/3139558413 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec00bc30 con 0x7fcef00ad780 2023-10-06T04:00:43.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.940+0000 7fcef60c3700 1 -- 172.21.15.157:0/3139558413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcef00ad780 msgr2=0x7fcef0152eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:43.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.940+0000 7fcef60c3700 1 --2- 172.21.15.157:0/3139558413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcef00ad780 0x7fcef0152eb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fceec0098e0 tx=0x7fceec0091a0 comp rx=0 tx=0).stop 2023-10-06T04:00:43.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.940+0000 7fcef60c3700 1 -- 172.21.15.157:0/3139558413 shutdown_connections 2023-10-06T04:00:43.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.940+0000 7fcef60c3700 1 --2- 172.21.15.157:0/3139558413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcef00ad780 0x7fcef0152eb0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:43.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.940+0000 7fcef60c3700 1 -- 172.21.15.157:0/3139558413 >> 172.21.15.157:0/3139558413 conn(0x7fcef006eb20 msgr2=0x7fcef006c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:00:43.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.940+0000 7fcef60c3700 1 -- 172.21.15.157:0/3139558413 shutdown_connections 2023-10-06T04:00:43.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.940+0000 7fcef60c3700 1 -- 172.21.15.157:0/3139558413 wait complete. 2023-10-06T04:00:43.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.941+0000 7fcef60c3700 1 Processor -- start 2023-10-06T04:00:43.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.941+0000 7fcef60c3700 1 -- start start 2023-10-06T04:00:43.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.942+0000 7fcef60c3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcef006bb70 0x7fcef0162b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:43.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.942+0000 7fcef60c3700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcef006f8c0 0x7fcef0150380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:43.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.942+0000 7fcef60c3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcef00ad780 0x7fcef00c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:43.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.942+0000 7fcef60c3700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef0163070 con 0x7fcef00ad780 2023-10-06T04:00:43.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.942+0000 7fcef60c3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef0151590 con 0x7fcef006bb70 2023-10-06T04:00:43.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.942+0000 7fcef60c3700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef00c6930 con 0x7fcef006f8c0 2023-10-06T04:00:43.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.942+0000 7fcef58c2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcef00ad780 0x7fcef00c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:43.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.942+0000 7fcef58c2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcef00ad780 0x7fcef00c6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59354/0 (socket says 172.21.15.157:59354) 2023-10-06T04:00:43.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.942+0000 7fcef58c2700 1 -- 172.21.15.157:0/2395103645 learned_addr learned my addr 172.21.15.157:0/2395103645 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:00:43.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.942+0000 7fcef50c1700 1 --2- 172.21.15.157:0/2395103645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcef006bb70 0x7fcef0162b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:43.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.942+0000 7fcef48c0700 1 --2- 172.21.15.157:0/2395103645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcef006f8c0 0x7fcef0150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:43.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.943+0000 7fcef58c2700 1 -- 172.21.15.157:0/2395103645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcef006f8c0 msgr2=0x7fcef0150380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:43.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.943+0000 7fcef58c2700 1 --2- 172.21.15.157:0/2395103645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcef006f8c0 0x7fcef0150380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:43.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.943+0000 7fcef58c2700 1 -- 172.21.15.157:0/2395103645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcef006bb70 msgr2=0x7fcef0162b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:43.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.943+0000 7fcef58c2700 1 --2- 172.21.15.157:0/2395103645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcef006bb70 0x7fcef0162b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:43.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.943+0000 7fcef58c2700 1 -- 172.21.15.157:0/2395103645 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceec009420 con 0x7fcef00ad780 2023-10-06T04:00:43.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.943+0000 7fcef58c2700 1 --2- 172.21.15.157:0/2395103645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcef00ad780 0x7fcef00c6dc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fcee800bc60 tx=0x7fcee8009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:00:43.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.943+0000 7fcee57fa700 1 -- 172.21.15.157:0/2395103645 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee800c840 con 0x7fcef00ad780 2023-10-06T04:00:43.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.943+0000 7fcee57fa700 1 -- 172.21.15.157:0/2395103645 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee800c9a0 con 0x7fcef00ad780 2023-10-06T04:00:43.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.943+0000 7fcef60c3700 1 -- 172.21.15.157:0/2395103645 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef01512b0 con 0x7fcef00ad780 2023-10-06T04:00:43.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.943+0000 7fcee57fa700 1 -- 172.21.15.157:0/2395103645 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee800cd60 con 0x7fcef00ad780 2023-10-06T04:00:43.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.943+0000 7fcef60c3700 1 -- 172.21.15.157:0/2395103645 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef00c62e0 con 0x7fcef00ad780 2023-10-06T04:00:43.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.944+0000 7fcef60c3700 1 -- 172.21.15.157:0/2395103645 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef00c3760 con 0x7fcef00ad780 2023-10-06T04:00:43.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.944+0000 7fcee57fa700 1 -- 172.21.15.157:0/2395103645 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcee8014620 con 0x7fcef00ad780 2023-10-06T04:00:43.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.945+0000 7fcee57fa700 1 --2- 172.21.15.157:0/2395103645 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fced4066f30 0x7fced40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:43.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.945+0000 7fcee57fa700 1 -- 172.21.15.157:0/2395103645 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6202+0+0 (secure 0 0 0) 0x7fcee80845f0 con 0x7fcef00ad780 2023-10-06T04:00:43.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.945+0000 7fcef50c1700 1 --2- 172.21.15.157:0/2395103645 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fced4066f30 0x7fced40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:43.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.945+0000 7fcef50c1700 1 --2- 172.21.15.157:0/2395103645 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fced4066f30 0x7fced40693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fceec0093f0 tx=0x7fceec016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:00:43.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:43.949+0000 7fcee57fa700 1 -- 172.21.15.157:0/2395103645 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcee804c5a0 con 0x7fcef00ad780 2023-10-06T04:00:43.965 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:00:43.967 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:00:44.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.180+0000 7fcef60c3700 1 -- 172.21.15.157:0/2395103645 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcef014fbd0 con 0x7fcef00ad780 2023-10-06T04:00:44.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.180+0000 7fcee57fa700 1 -- 172.21.15.157:0/2395103645 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+6203 (secure 0 0 0) 0x7fcee8051670 con 0x7fcef00ad780 2023-10-06T04:00:44.182 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:00:44.182 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4936,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:00:42.818220+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4223":{"gid":4223,"name":"a","rank":4,"incarnation":17,"state":"up:active","state_seq":39,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:00:44.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.182+0000 7fcef60c3700 1 -- 172.21.15.157:0/2395103645 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fced4066f30 msgr2=0x7fced40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:44.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.182+0000 7fcef60c3700 1 --2- 172.21.15.157:0/2395103645 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fced4066f30 0x7fced40693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fceec0093f0 tx=0x7fceec016000 comp rx=0 tx=0).stop 2023-10-06T04:00:44.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.182+0000 7fcef60c3700 1 -- 172.21.15.157:0/2395103645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcef00ad780 msgr2=0x7fcef00c6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:44.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.182+0000 7fcef60c3700 1 --2- 172.21.15.157:0/2395103645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcef00ad780 0x7fcef00c6dc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fcee800bc60 tx=0x7fcee8009820 comp rx=0 tx=0).stop 2023-10-06T04:00:44.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.182+0000 7fcef60c3700 1 -- 172.21.15.157:0/2395103645 shutdown_connections 2023-10-06T04:00:44.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.182+0000 7fcef60c3700 1 --2- 172.21.15.157:0/2395103645 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fced4066f30 0x7fced40693e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:44.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.182+0000 7fcef60c3700 1 --2- 172.21.15.157:0/2395103645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcef006bb70 0x7fcef0162b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:44.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.182+0000 7fcef60c3700 1 --2- 172.21.15.157:0/2395103645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcef00ad780 0x7fcef00c6dc0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:44.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.182+0000 7fcef60c3700 1 --2- 172.21.15.157:0/2395103645 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcef006f8c0 0x7fcef0150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:44.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.182+0000 7fcef60c3700 1 -- 172.21.15.157:0/2395103645 >> 172.21.15.157:0/2395103645 conn(0x7fcef006eb20 msgr2=0x7fcef00c2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:00:44.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.182+0000 7fcef60c3700 1 -- 172.21.15.157:0/2395103645 shutdown_connections 2023-10-06T04:00:44.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.182+0000 7fcef60c3700 1 -- 172.21.15.157:0/2395103645 wait complete. 2023-10-06T04:00:44.185 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 28 2023-10-06T04:00:44.196 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=4) 2023-10-06T04:00:44.196 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T04:00:44.196 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:00:44.206 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T04:00:44.206 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:00:44.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.359+0000 7f93ed50f700 1 Processor -- start 2023-10-06T04:00:44.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.359+0000 7f93ed50f700 1 -- start start 2023-10-06T04:00:44.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.360+0000 7f93ed50f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93e8067d40 0x7f93e8152e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:44.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.360+0000 7f93ed50f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f93e80c5ca0 0x7f93e8153390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:44.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.360+0000 7f93ed50f700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93e806d3a0 con 0x7f93e80b0030 2023-10-06T04:00:44.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.360+0000 7f93ed50f700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93e806ed70 con 0x7f93e80c5ca0 2023-10-06T04:00:44.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.360+0000 7f93ed50f700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e80c7930 con 0x7f93e8067d40 2023-10-06T04:00:44.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.360+0000 7f93ecd0e700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f93e80b0030 0x7f93e8159600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43488/0 (socket says 172.21.15.157:43488) 2023-10-06T04:00:44.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.360+0000 7f93ecd0e700 1 -- 172.21.15.157:0/2704185353 learned_addr learned my addr 172.21.15.157:0/2704185353 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:00:44.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.360+0000 7f93e7fff700 1 --2- 172.21.15.157:0/2704185353 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93e8067d40 0x7f93e8152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:44.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.360+0000 7f93e77fe700 1 --2- 172.21.15.157:0/2704185353 >> v2:172.21.15.175:3300/0 conn(0x7f93e80c5ca0 0x7f93e8153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:44.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.360+0000 7f93e6ffd700 1 -- 172.21.15.157:0/2704185353 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 355395257 0 0) 0x7f93e806d3a0 con 0x7f93e80b0030 2023-10-06T04:00:44.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.360+0000 7f93e6ffd700 1 -- 172.21.15.157:0/2704185353 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93d0003570 con 0x7f93e80b0030 2023-10-06T04:00:44.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.360+0000 7f93e6ffd700 1 -- 172.21.15.157:0/2704185353 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3578937567 0 0) 0x7f93d0003570 con 0x7f93e80b0030 2023-10-06T04:00:44.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.360+0000 7f93e6ffd700 1 -- 172.21.15.157:0/2704185353 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93e8067d40 msgr2=0x7f93e8152e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:00:44.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.360+0000 7f93e6ffd700 1 --2- 172.21.15.157:0/2704185353 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93e8067d40 0x7f93e8152e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:44.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.360+0000 7f93e6ffd700 1 -- 172.21.15.157:0/2704185353 >> v2:172.21.15.175:3300/0 conn(0x7f93e80c5ca0 msgr2=0x7f93e8153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:00:44.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.360+0000 7f93e6ffd700 1 --2- 172.21.15.157:0/2704185353 >> v2:172.21.15.175:3300/0 conn(0x7f93e80c5ca0 0x7f93e8153390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:44.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.360+0000 7f93e6ffd700 1 -- 172.21.15.157:0/2704185353 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e806d610 con 0x7f93e80b0030 2023-10-06T04:00:44.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.360+0000 7f93e6ffd700 1 -- 172.21.15.157:0/2704185353 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f93e00017f0 con 0x7f93e80b0030 2023-10-06T04:00:44.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.360+0000 7f93e77fe700 1 --2- 172.21.15.157:0/2704185353 >> v2:172.21.15.175:3300/0 conn(0x7f93e80c5ca0 0x7f93e8153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:00:44.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.361+0000 7f93e6ffd700 1 -- 172.21.15.157:0/2704185353 >> v1:172.21.15.157:6789/0 conn(0x7f93e80b0030 legacy=0x7f93e8159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:44.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.361+0000 7f93e6ffd700 1 --2- 172.21.15.157:0/2704185353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93d0003a60 0x7f93d0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:44.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.361+0000 7f93e6ffd700 1 --2- 172.21.15.157:0/2704185353 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93e8067d40 0x7f93d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:44.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.361+0000 7f93e6ffd700 1 --2- 172.21.15.157:0/2704185353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93d0004d50 0x7f93d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:44.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.361+0000 7f93e6ffd700 1 -- 172.21.15.157:0/2704185353 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e806ed70 con 0x7f93d0004d50 2023-10-06T04:00:44.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.361+0000 7f93e6ffd700 1 -- 172.21.15.157:0/2704185353 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e80c7930 con 0x7f93d0003a60 2023-10-06T04:00:44.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.361+0000 7f93e6ffd700 1 -- 172.21.15.157:0/2704185353 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93d000ba60 con 0x7f93e8067d40 2023-10-06T04:00:44.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.361+0000 7f93ecd0e700 1 --2- 172.21.15.157:0/2704185353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93d0004d50 0x7f93d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:44.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.361+0000 7f93e7fff700 1 --2- 172.21.15.157:0/2704185353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93d0003a60 0x7f93d0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:44.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.361+0000 7f93e77fe700 1 --2- 172.21.15.157:0/2704185353 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93e8067d40 0x7f93d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:44.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.361+0000 7f93ecd0e700 1 -- 172.21.15.157:0/2704185353 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93e8067d40 msgr2=0x7f93d0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:00:44.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.361+0000 7f93ecd0e700 1 --2- 172.21.15.157:0/2704185353 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93e8067d40 0x7f93d0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:44.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.361+0000 7f93ecd0e700 1 -- 172.21.15.157:0/2704185353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93d0003a60 msgr2=0x7f93d0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:00:44.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.361+0000 7f93ecd0e700 1 --2- 172.21.15.157:0/2704185353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93d0003a60 0x7f93d0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:44.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.361+0000 7f93ecd0e700 1 -- 172.21.15.157:0/2704185353 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d0003570 con 0x7f93d0004d50 2023-10-06T04:00:44.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.361+0000 7f93ecd0e700 1 --2- 172.21.15.157:0/2704185353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93d0004d50 0x7f93d00051c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f93e8068c00 tx=0x7f93e806e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54dbe9a190208ad3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:00:44.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.361+0000 7f93e6ffd700 1 -- 172.21.15.157:0/2704185353 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e00070d0 con 0x7f93d0004d50 2023-10-06T04:00:44.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.361+0000 7f93e6ffd700 1 -- 172.21.15.157:0/2704185353 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e0007230 con 0x7f93d0004d50 2023-10-06T04:00:44.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.361+0000 7f93e6ffd700 1 -- 172.21.15.157:0/2704185353 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e00075f0 con 0x7f93d0004d50 2023-10-06T04:00:44.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.362+0000 7f93ed50f700 1 -- 172.21.15.157:0/2704185353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93d0004d50 msgr2=0x7f93d00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:44.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.362+0000 7f93ed50f700 1 --2- 172.21.15.157:0/2704185353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93d0004d50 0x7f93d00051c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f93e8068c00 tx=0x7f93e806e0f0 comp rx=0 tx=0).stop 2023-10-06T04:00:44.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.362+0000 7f93ecd0e700 1 -- 172.21.15.157:0/2704185353 reap_dead start 2023-10-06T04:00:44.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.362+0000 7f93ed50f700 1 -- 172.21.15.157:0/2704185353 shutdown_connections 2023-10-06T04:00:44.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.362+0000 7f93ed50f700 1 -- 172.21.15.157:0/2704185353 >> 172.21.15.157:0/2704185353 conn(0x7f93e806c1f0 msgr2=0x7f93e806c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:00:44.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.362+0000 7f93ed50f700 1 -- 172.21.15.157:0/2704185353 shutdown_connections 2023-10-06T04:00:44.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.362+0000 7f93ed50f700 1 -- 172.21.15.157:0/2704185353 wait complete. 2023-10-06T04:00:44.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.363+0000 7f93ed50f700 1 Processor -- start 2023-10-06T04:00:44.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.363+0000 7f93ed50f700 1 -- start start 2023-10-06T04:00:44.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.364+0000 7f93ed50f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93e80c82e0 0x7f93e8166270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:44.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.364+0000 7f93ed50f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93e80c86b0 0x7f93e8158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:44.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.364+0000 7f93ed50f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93e8158cc0 0x7f93e8168da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:44.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.364+0000 7f93ed50f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e80c9f90 con 0x7f93e80c86b0 2023-10-06T04:00:44.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.364+0000 7f93ed50f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e81582f0 con 0x7f93e8158cc0 2023-10-06T04:00:44.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.364+0000 7f93ed50f700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e8159160 con 0x7f93e80c82e0 2023-10-06T04:00:44.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.364+0000 7f93e77fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93e80c86b0 0x7f93e8158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:44.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.364+0000 7f93e77fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93e80c86b0 0x7f93e8158780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59370/0 (socket says 172.21.15.157:59370) 2023-10-06T04:00:44.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.364+0000 7f93e77fe700 1 -- 172.21.15.157:0/1421287782 learned_addr learned my addr 172.21.15.157:0/1421287782 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:00:44.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.364+0000 7f93ecd0e700 1 --2- 172.21.15.157:0/1421287782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93e8158cc0 0x7f93e8168da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:44.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.364+0000 7f93e7fff700 1 --2- 172.21.15.157:0/1421287782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93e80c82e0 0x7f93e8166270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:44.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.364+0000 7f93e77fe700 1 -- 172.21.15.157:0/1421287782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93e80c82e0 msgr2=0x7f93e8166270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:44.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.364+0000 7f93e77fe700 1 --2- 172.21.15.157:0/1421287782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93e80c82e0 0x7f93e8166270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:44.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.364+0000 7f93e77fe700 1 -- 172.21.15.157:0/1421287782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93e8158cc0 msgr2=0x7f93e8168da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:44.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.364+0000 7f93e77fe700 1 --2- 172.21.15.157:0/1421287782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93e8158cc0 0x7f93e8168da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:44.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.364+0000 7f93e77fe700 1 -- 172.21.15.157:0/1421287782 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e80ca1a0 con 0x7f93e80c86b0 2023-10-06T04:00:44.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.364+0000 7f93e77fe700 1 --2- 172.21.15.157:0/1421287782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93e80c86b0 0x7f93e8158780 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f93dc011da0 tx=0x7f93dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:00:44.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.364+0000 7f93cffff700 1 -- 172.21.15.157:0/1421287782 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc011ea0 con 0x7f93e80c86b0 2023-10-06T04:00:44.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.364+0000 7f93cffff700 1 -- 172.21.15.157:0/1421287782 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93dc00fc20 con 0x7f93e80c86b0 2023-10-06T04:00:44.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.364+0000 7f93ed50f700 1 -- 172.21.15.157:0/1421287782 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e80ca460 con 0x7f93e80c86b0 2023-10-06T04:00:44.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.364+0000 7f93cffff700 1 -- 172.21.15.157:0/1421287782 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc01d480 con 0x7f93e80c86b0 2023-10-06T04:00:44.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.364+0000 7f93ed50f700 1 -- 172.21.15.157:0/1421287782 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e8157ca0 con 0x7f93e80c86b0 2023-10-06T04:00:44.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.365+0000 7f93ed50f700 1 -- 172.21.15.157:0/1421287782 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e806cbf0 con 0x7f93e80c86b0 2023-10-06T04:00:44.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.366+0000 7f93cffff700 1 -- 172.21.15.157:0/1421287782 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f93dc01d830 con 0x7f93e80c86b0 2023-10-06T04:00:44.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.366+0000 7f93cffff700 1 --2- 172.21.15.157:0/1421287782 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f93d4066f30 0x7f93d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:44.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.366+0000 7f93cffff700 1 -- 172.21.15.157:0/1421287782 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f93dc018070 con 0x7f93e80c86b0 2023-10-06T04:00:44.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.366+0000 7f93e7fff700 1 --2- 172.21.15.157:0/1421287782 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f93d4066f30 0x7f93d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:44.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.367+0000 7f93e7fff700 1 --2- 172.21.15.157:0/1421287782 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f93d4066f30 0x7f93d40693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f93d8006f00 tx=0x7f93d80091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:00:44.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.370+0000 7f93cffff700 1 -- 172.21.15.157:0/1421287782 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f93dc052590 con 0x7f93e80c86b0 2023-10-06T04:00:44.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.595+0000 7f93ed50f700 1 -- 172.21.15.157:0/1421287782 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93e806efe0 con 0x7f93e80c86b0 2023-10-06T04:00:44.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.596+0000 7f93cffff700 1 -- 172.21.15.157:0/1421287782 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+6203 (secure 0 0 0) 0x7f93dc055bb0 con 0x7f93e80c86b0 2023-10-06T04:00:44.598 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:00:44.598 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4936,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:00:42.818220+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4223":{"gid":4223,"name":"a","rank":4,"incarnation":17,"state":"up:active","state_seq":39,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:00:44.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.598+0000 7f93ed50f700 1 -- 172.21.15.157:0/1421287782 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f93d4066f30 msgr2=0x7f93d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:44.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.598+0000 7f93ed50f700 1 --2- 172.21.15.157:0/1421287782 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f93d4066f30 0x7f93d40693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f93d8006f00 tx=0x7f93d80091a0 comp rx=0 tx=0).stop 2023-10-06T04:00:44.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.598+0000 7f93ed50f700 1 -- 172.21.15.157:0/1421287782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93e80c86b0 msgr2=0x7f93e8158780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:44.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.598+0000 7f93ed50f700 1 --2- 172.21.15.157:0/1421287782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93e80c86b0 0x7f93e8158780 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f93dc011da0 tx=0x7f93dc0091a0 comp rx=0 tx=0).stop 2023-10-06T04:00:44.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.598+0000 7f93ed50f700 1 -- 172.21.15.157:0/1421287782 shutdown_connections 2023-10-06T04:00:44.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.598+0000 7f93ed50f700 1 --2- 172.21.15.157:0/1421287782 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f93d4066f30 0x7f93d40693e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:44.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.598+0000 7f93ed50f700 1 --2- 172.21.15.157:0/1421287782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93e8158cc0 0x7f93e8168da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:44.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.598+0000 7f93ed50f700 1 --2- 172.21.15.157:0/1421287782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93e80c82e0 0x7f93e8166270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:44.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.598+0000 7f93ed50f700 1 --2- 172.21.15.157:0/1421287782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93e80c86b0 0x7f93e8158780 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:44.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.598+0000 7f93ed50f700 1 -- 172.21.15.157:0/1421287782 >> 172.21.15.157:0/1421287782 conn(0x7f93e806c1f0 msgr2=0x7f93e80c54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:00:44.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.598+0000 7f93ed50f700 1 -- 172.21.15.157:0/1421287782 shutdown_connections 2023-10-06T04:00:44.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:44.598+0000 7f93ed50f700 1 -- 172.21.15.157:0/1421287782 wait complete. 2023-10-06T04:00:44.602 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 28 2023-10-06T04:00:44.612 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-10-06T04:00:46.613 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:00:46.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.740+0000 7ff0a2712700 1 Processor -- start 2023-10-06T04:00:46.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.740+0000 7ff0a2712700 1 -- start start 2023-10-06T04:00:46.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.741+0000 7ff0a2712700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff09c06dc40 0x7ff09c14fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:46.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.741+0000 7ff0a2712700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff09c0c9780 con 0x7ff09c06cbb0 2023-10-06T04:00:46.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.741+0000 7ff0a2712700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff09c06e0c0 con 0x7ff09c06c1f0 2023-10-06T04:00:46.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.741+0000 7ff0a2712700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c0c8c40 con 0x7ff09c06dc40 2023-10-06T04:00:46.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.741+0000 7ff09bfff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff09c06cbb0 0x7ff09c06d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43500/0 (socket says 172.21.15.157:43500) 2023-10-06T04:00:46.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.741+0000 7ff09bfff700 1 -- 172.21.15.157:0/3802143206 learned_addr learned my addr 172.21.15.157:0/3802143206 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:00:46.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.741+0000 7ff09b7fe700 1 --2- 172.21.15.157:0/3802143206 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff09c06dc40 0x7ff09c14fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:46.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.741+0000 7ff09affd700 1 -- 172.21.15.157:0/3802143206 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1383858724 0 0) 0x7ff09c0c9780 con 0x7ff09c06cbb0 2023-10-06T04:00:46.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.741+0000 7ff09affd700 1 -- 172.21.15.157:0/3802143206 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff084003570 con 0x7ff09c06cbb0 2023-10-06T04:00:46.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.742+0000 7ff09affd700 1 -- 172.21.15.157:0/3802143206 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2554285075 0 0) 0x7ff09c06e0c0 con 0x7ff09c06c1f0 2023-10-06T04:00:46.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.742+0000 7ff09affd700 1 -- 172.21.15.157:0/3802143206 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff09c0c9780 con 0x7ff09c06c1f0 2023-10-06T04:00:46.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.742+0000 7ff09affd700 1 -- 172.21.15.157:0/3802143206 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2200903047 0 0) 0x7ff084003570 con 0x7ff09c06cbb0 2023-10-06T04:00:46.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.742+0000 7ff09affd700 1 -- 172.21.15.157:0/3802143206 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff09c06dc40 msgr2=0x7ff09c14fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:00:46.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.742+0000 7ff09affd700 1 --2- 172.21.15.157:0/3802143206 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff09c06dc40 0x7ff09c14fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:46.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.742+0000 7ff09affd700 1 -- 172.21.15.157:0/3802143206 >> v1:172.21.15.175:6789/0 conn(0x7ff09c06c1f0 legacy=0x7ff09c155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:46.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.742+0000 7ff09affd700 1 -- 172.21.15.157:0/3802143206 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c0c8e50 con 0x7ff09c06cbb0 2023-10-06T04:00:46.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.742+0000 7ff09affd700 1 -- 172.21.15.157:0/3802143206 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7ff08c001470 con 0x7ff09c06cbb0 2023-10-06T04:00:46.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.742+0000 7ff09affd700 1 -- 172.21.15.157:0/3802143206 >> v1:172.21.15.157:6789/0 conn(0x7ff09c06cbb0 legacy=0x7ff09c06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:46.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.742+0000 7ff09affd700 1 --2- 172.21.15.157:0/3802143206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff084003c20 0x7ff084004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:46.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.742+0000 7ff09affd700 1 --2- 172.21.15.157:0/3802143206 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff084004810 0x7ff084004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:46.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.742+0000 7ff09affd700 1 --2- 172.21.15.157:0/3802143206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff084005180 0x7ff08400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:46.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.742+0000 7ff09affd700 1 -- 172.21.15.157:0/3802143206 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c0c8c40 con 0x7ff084005180 2023-10-06T04:00:46.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.742+0000 7ff09affd700 1 -- 172.21.15.157:0/3802143206 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08400bfa0 con 0x7ff084003c20 2023-10-06T04:00:46.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.742+0000 7ff09affd700 1 -- 172.21.15.157:0/3802143206 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff08400c0e0 con 0x7ff084004810 2023-10-06T04:00:46.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.742+0000 7ff0a0caf700 1 --2- 172.21.15.157:0/3802143206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff084005180 0x7ff08400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:46.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.742+0000 7ff09bfff700 1 --2- 172.21.15.157:0/3802143206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff084003c20 0x7ff084004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:46.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.742+0000 7ff09b7fe700 1 --2- 172.21.15.157:0/3802143206 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff084004810 0x7ff084004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:46.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.742+0000 7ff0a0caf700 1 -- 172.21.15.157:0/3802143206 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff084004810 msgr2=0x7ff084004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:00:46.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.742+0000 7ff0a0caf700 1 --2- 172.21.15.157:0/3802143206 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff084004810 0x7ff084004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:46.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.742+0000 7ff0a0caf700 1 -- 172.21.15.157:0/3802143206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff084003c20 msgr2=0x7ff084004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:00:46.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.742+0000 7ff0a0caf700 1 --2- 172.21.15.157:0/3802143206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff084003c20 0x7ff084004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:46.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.742+0000 7ff0a0caf700 1 -- 172.21.15.157:0/3802143206 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff084003570 con 0x7ff084005180 2023-10-06T04:00:46.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.742+0000 7ff0a0caf700 1 --2- 172.21.15.157:0/3802143206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff084005180 0x7ff08400ba60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff094002150 tx=0x7ff0940017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4944ea9b722d3a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:00:46.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.742+0000 7ff09affd700 1 -- 172.21.15.157:0/3802143206 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff094001ab0 con 0x7ff084005180 2023-10-06T04:00:46.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.742+0000 7ff09affd700 1 -- 172.21.15.157:0/3802143206 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09400c940 con 0x7ff084005180 2023-10-06T04:00:46.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.743+0000 7ff09affd700 1 -- 172.21.15.157:0/3802143206 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09400cce0 con 0x7ff084005180 2023-10-06T04:00:46.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.743+0000 7ff0a2712700 1 -- 172.21.15.157:0/3802143206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff084005180 msgr2=0x7ff08400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:46.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.743+0000 7ff0a2712700 1 --2- 172.21.15.157:0/3802143206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff084005180 0x7ff08400ba60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff094002150 tx=0x7ff0940017a0 comp rx=0 tx=0).stop 2023-10-06T04:00:46.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.743+0000 7ff0a0caf700 1 -- 172.21.15.157:0/3802143206 reap_dead start 2023-10-06T04:00:46.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.743+0000 7ff0a2712700 1 -- 172.21.15.157:0/3802143206 shutdown_connections 2023-10-06T04:00:46.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.743+0000 7ff0a2712700 1 -- 172.21.15.157:0/3802143206 >> 172.21.15.157:0/3802143206 conn(0x7ff09c06f230 msgr2=0x7ff09c06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:00:46.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.743+0000 7ff0a2712700 1 -- 172.21.15.157:0/3802143206 shutdown_connections 2023-10-06T04:00:46.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.743+0000 7ff0a2712700 1 -- 172.21.15.157:0/3802143206 wait complete. 2023-10-06T04:00:46.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.744+0000 7ff0a2712700 1 Processor -- start 2023-10-06T04:00:46.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.744+0000 7ff0a2712700 1 -- start start 2023-10-06T04:00:46.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.745+0000 7ff0a2712700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff09c14b490 0x7ff09c1678b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:46.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.745+0000 7ff0a2712700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff09c14b860 0x7ff09c167df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:46.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.745+0000 7ff0a2712700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff09c0cc7e0 0x7ff09c1692f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:46.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.745+0000 7ff0a2712700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c1690d0 con 0x7ff09c14b490 2023-10-06T04:00:46.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.745+0000 7ff0a2712700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c167420 con 0x7ff09c0cc7e0 2023-10-06T04:00:46.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.745+0000 7ff0a2712700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c0cbf70 con 0x7ff09c14b860 2023-10-06T04:00:46.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.745+0000 7ff09bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff09c14b490 0x7ff09c1678b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:46.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.745+0000 7ff09bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff09c14b490 0x7ff09c1678b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59386/0 (socket says 172.21.15.157:59386) 2023-10-06T04:00:46.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.745+0000 7ff09bfff700 1 -- 172.21.15.157:0/4154244041 learned_addr learned my addr 172.21.15.157:0/4154244041 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:00:46.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.745+0000 7ff0a0caf700 1 --2- 172.21.15.157:0/4154244041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff09c0cc7e0 0x7ff09c1692f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:46.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.745+0000 7ff09b7fe700 1 --2- 172.21.15.157:0/4154244041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff09c14b860 0x7ff09c167df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:46.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.745+0000 7ff09bfff700 1 -- 172.21.15.157:0/4154244041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff09c14b860 msgr2=0x7ff09c167df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:46.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.745+0000 7ff09bfff700 1 --2- 172.21.15.157:0/4154244041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff09c14b860 0x7ff09c167df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:46.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.745+0000 7ff09bfff700 1 -- 172.21.15.157:0/4154244041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff09c0cc7e0 msgr2=0x7ff09c1692f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:46.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.745+0000 7ff09bfff700 1 --2- 172.21.15.157:0/4154244041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff09c0cc7e0 0x7ff09c1692f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:46.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.745+0000 7ff09bfff700 1 -- 172.21.15.157:0/4154244041 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c0cc1e0 con 0x7ff09c14b490 2023-10-06T04:00:46.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.746+0000 7ff09bfff700 1 --2- 172.21.15.157:0/4154244041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff09c14b490 0x7ff09c1678b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff08c007fd0 tx=0x7ff09c0b1020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:00:46.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.746+0000 7ff083fff700 1 -- 172.21.15.157:0/4154244041 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c011590 con 0x7ff09c14b490 2023-10-06T04:00:46.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.746+0000 7ff083fff700 1 -- 172.21.15.157:0/4154244041 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08c0116f0 con 0x7ff09c14b490 2023-10-06T04:00:46.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.746+0000 7ff083fff700 1 -- 172.21.15.157:0/4154244041 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c011ab0 con 0x7ff09c14b490 2023-10-06T04:00:46.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.746+0000 7ff0a2712700 1 -- 172.21.15.157:0/4154244041 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff09c0ccfe0 con 0x7ff09c14b490 2023-10-06T04:00:46.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.746+0000 7ff0a2712700 1 -- 172.21.15.157:0/4154244041 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff09c14d720 con 0x7ff09c14b490 2023-10-06T04:00:46.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.747+0000 7ff0a2712700 1 -- 172.21.15.157:0/4154244041 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff09c167700 con 0x7ff09c14b490 2023-10-06T04:00:46.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.747+0000 7ff083fff700 1 -- 172.21.15.157:0/4154244041 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff08c01d460 con 0x7ff09c14b490 2023-10-06T04:00:46.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.748+0000 7ff083fff700 1 --2- 172.21.15.157:0/4154244041 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff088066f30 0x7ff0880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:46.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.748+0000 7ff083fff700 1 -- 172.21.15.157:0/4154244041 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6202+0+0 (secure 0 0 0) 0x7ff08c0847e0 con 0x7ff09c14b490 2023-10-06T04:00:46.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.748+0000 7ff09b7fe700 1 --2- 172.21.15.157:0/4154244041 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff088066f30 0x7ff0880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:46.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.748+0000 7ff09b7fe700 1 --2- 172.21.15.157:0/4154244041 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff088066f30 0x7ff0880693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff09c0afbf0 tx=0x7ff0900091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:00:46.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.752+0000 7ff083fff700 1 -- 172.21.15.157:0/4154244041 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff08c051860 con 0x7ff09c14b490 2023-10-06T04:00:46.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.980+0000 7ff0a2712700 1 -- 172.21.15.157:0/4154244041 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff09c1671f0 con 0x7ff09c14b490 2023-10-06T04:00:46.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.980+0000 7ff083fff700 1 -- 172.21.15.157:0/4154244041 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+6203 (secure 0 0 0) 0x7ff08c054e80 con 0x7ff09c14b490 2023-10-06T04:00:46.982 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:00:46.982 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4936,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:00:42.818220+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4223":{"gid":4223,"name":"a","rank":4,"incarnation":17,"state":"up:active","state_seq":39,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:00:46.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.982+0000 7ff0a2712700 1 -- 172.21.15.157:0/4154244041 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff088066f30 msgr2=0x7ff0880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:46.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.982+0000 7ff0a2712700 1 --2- 172.21.15.157:0/4154244041 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff088066f30 0x7ff0880693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff09c0afbf0 tx=0x7ff0900091a0 comp rx=0 tx=0).stop 2023-10-06T04:00:46.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.982+0000 7ff0a2712700 1 -- 172.21.15.157:0/4154244041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff09c14b490 msgr2=0x7ff09c1678b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:46.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.982+0000 7ff0a2712700 1 --2- 172.21.15.157:0/4154244041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff09c14b490 0x7ff09c1678b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff08c007fd0 tx=0x7ff09c0b1020 comp rx=0 tx=0).stop 2023-10-06T04:00:46.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.982+0000 7ff0a2712700 1 -- 172.21.15.157:0/4154244041 shutdown_connections 2023-10-06T04:00:46.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.982+0000 7ff0a2712700 1 --2- 172.21.15.157:0/4154244041 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff088066f30 0x7ff0880693e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:46.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.982+0000 7ff0a2712700 1 --2- 172.21.15.157:0/4154244041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff09c0cc7e0 0x7ff09c1692f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:46.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.982+0000 7ff0a2712700 1 --2- 172.21.15.157:0/4154244041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff09c14b490 0x7ff09c1678b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:46.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.982+0000 7ff0a2712700 1 --2- 172.21.15.157:0/4154244041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff09c14b860 0x7ff09c167df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:46.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.982+0000 7ff0a2712700 1 -- 172.21.15.157:0/4154244041 >> 172.21.15.157:0/4154244041 conn(0x7ff09c06f230 msgr2=0x7ff09c14fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:00:46.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.983+0000 7ff0a2712700 1 -- 172.21.15.157:0/4154244041 shutdown_connections 2023-10-06T04:00:46.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:46.983+0000 7ff0a2712700 1 -- 172.21.15.157:0/4154244041 wait complete. 2023-10-06T04:00:46.986 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 28 2023-10-06T04:00:46.996 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-10-06T04:00:48.997 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:00:49.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.121+0000 7f26c359e700 1 Processor -- start 2023-10-06T04:00:49.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.121+0000 7f26c359e700 1 -- start start 2023-10-06T04:00:49.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.122+0000 7f26c359e700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f26c40b04d0 0x7f26c4157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:49.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.122+0000 7f26c359e700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26c406c2d0 0x7f26c406dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:49.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.122+0000 7f26c359e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26c406d8d0 con 0x7f26c40b1e40 2023-10-06T04:00:49.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.122+0000 7f26c359e700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26c40af8f0 con 0x7f26c40b04d0 2023-10-06T04:00:49.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.122+0000 7f26c359e700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26c415a690 con 0x7f26c406c2d0 2023-10-06T04:00:49.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.122+0000 7f26b9d9b700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f26c40b04d0 0x7f26c4157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:49.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.122+0000 7f26b9d9b700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f26c40b04d0 0x7f26c4157750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59402/0 (socket says 172.21.15.157:59402) 2023-10-06T04:00:49.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.122+0000 7f26b9d9b700 1 -- 172.21.15.157:0/194302580 learned_addr learned my addr 172.21.15.157:0/194302580 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:00:49.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.122+0000 7f26c2d9d700 1 --2- 172.21.15.157:0/194302580 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26c406c2d0 0x7f26c406dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:49.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.122+0000 7f26b9d9b700 1 -- 172.21.15.157:0/194302580 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26c406c2d0 msgr2=0x7f26c406dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:00:49.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.122+0000 7f26b9d9b700 1 --2- 172.21.15.157:0/194302580 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26c406c2d0 0x7f26c406dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:49.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.122+0000 7f26b9d9b700 1 -- 172.21.15.157:0/194302580 >> v1:172.21.15.175:6789/0 conn(0x7f26c40b1e40 legacy=0x7f26c41563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:00:49.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.122+0000 7f26b9d9b700 1 -- 172.21.15.157:0/194302580 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c406ccc0 con 0x7f26c40b04d0 2023-10-06T04:00:49.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.122+0000 7f26b9d9b700 1 --2- 172.21.15.157:0/194302580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26c40b04d0 0x7f26c4157750 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0099b0 tx=0x7f26ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d8551dae1908e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:00:49.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.122+0000 7f26c1d9b700 1 -- 172.21.15.157:0/194302580 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac00b820 con 0x7f26c40b04d0 2023-10-06T04:00:49.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.123+0000 7f26c1d9b700 1 -- 172.21.15.157:0/194302580 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ac00b980 con 0x7f26c40b04d0 2023-10-06T04:00:49.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.123+0000 7f26c1d9b700 1 -- 172.21.15.157:0/194302580 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac00bd40 con 0x7f26c40b04d0 2023-10-06T04:00:49.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.123+0000 7f26c359e700 1 -- 172.21.15.157:0/194302580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26c40b04d0 msgr2=0x7f26c4157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:49.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.123+0000 7f26c359e700 1 --2- 172.21.15.157:0/194302580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26c40b04d0 0x7f26c4157750 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0099b0 tx=0x7f26ac009250 comp rx=0 tx=0).stop 2023-10-06T04:00:49.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.123+0000 7f26c359e700 1 -- 172.21.15.157:0/194302580 shutdown_connections 2023-10-06T04:00:49.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.123+0000 7f26c359e700 1 --2- 172.21.15.157:0/194302580 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26c406c2d0 0x7f26c406dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:49.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.123+0000 7f26c359e700 1 --2- 172.21.15.157:0/194302580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26c40b04d0 0x7f26c4157750 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:49.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.123+0000 7f26c359e700 1 -- 172.21.15.157:0/194302580 >> 172.21.15.157:0/194302580 conn(0x7f26c40cc540 msgr2=0x7f26c40cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:00:49.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.123+0000 7f26c359e700 1 -- 172.21.15.157:0/194302580 shutdown_connections 2023-10-06T04:00:49.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.123+0000 7f26c359e700 1 -- 172.21.15.157:0/194302580 wait complete. 2023-10-06T04:00:49.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.124+0000 7f26c359e700 1 Processor -- start 2023-10-06T04:00:49.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.124+0000 7f26c359e700 1 -- start start 2023-10-06T04:00:49.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.125+0000 7f26c359e700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26c406c2d0 0x7f26c4165bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:49.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.125+0000 7f26c359e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26c40b04d0 0x7f26c415b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:49.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.125+0000 7f26c359e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26c40b1e40 0x7f26c415b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:49.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.125+0000 7f26c359e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c40aad30 con 0x7f26c40b04d0 2023-10-06T04:00:49.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.125+0000 7f26c359e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c40cb3e0 con 0x7f26c40b1e40 2023-10-06T04:00:49.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.125+0000 7f26c359e700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26c40aeed0 con 0x7f26c406c2d0 2023-10-06T04:00:49.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.125+0000 7f26b9d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26c40b04d0 0x7f26c415b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:49.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.125+0000 7f26b9d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26c40b04d0 0x7f26c415b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59412/0 (socket says 172.21.15.157:59412) 2023-10-06T04:00:49.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.125+0000 7f26b9d9b700 1 -- 172.21.15.157:0/2213184387 learned_addr learned my addr 172.21.15.157:0/2213184387 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:00:49.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.125+0000 7f26c259c700 1 --2- 172.21.15.157:0/2213184387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26c406c2d0 0x7f26c4165bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:49.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.125+0000 7f26c2d9d700 1 --2- 172.21.15.157:0/2213184387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26c40b1e40 0x7f26c415b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:49.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.126+0000 7f26b9d9b700 1 -- 172.21.15.157:0/2213184387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26c406c2d0 msgr2=0x7f26c4165bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:49.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.126+0000 7f26b9d9b700 1 --2- 172.21.15.157:0/2213184387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26c406c2d0 0x7f26c4165bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:49.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.126+0000 7f26b9d9b700 1 -- 172.21.15.157:0/2213184387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26c40b1e40 msgr2=0x7f26c415b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:49.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.126+0000 7f26b9d9b700 1 --2- 172.21.15.157:0/2213184387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26c40b1e40 0x7f26c415b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:49.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.126+0000 7f26b9d9b700 1 -- 172.21.15.157:0/2213184387 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ac0094f0 con 0x7f26c40b04d0 2023-10-06T04:00:49.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.126+0000 7f26b9d9b700 1 --2- 172.21.15.157:0/2213184387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26c40b04d0 0x7f26c415b300 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f26c40899e0 tx=0x7f26ac016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:00:49.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.126+0000 7f26baffd700 1 -- 172.21.15.157:0/2213184387 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac011b20 con 0x7f26c40b04d0 2023-10-06T04:00:49.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.126+0000 7f26baffd700 1 -- 172.21.15.157:0/2213184387 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ac020070 con 0x7f26c40b04d0 2023-10-06T04:00:49.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.126+0000 7f26c359e700 1 -- 172.21.15.157:0/2213184387 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26c40cb8d0 con 0x7f26c40b04d0 2023-10-06T04:00:49.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.126+0000 7f26baffd700 1 -- 172.21.15.157:0/2213184387 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac01b450 con 0x7f26c40b04d0 2023-10-06T04:00:49.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.126+0000 7f26c359e700 1 -- 172.21.15.157:0/2213184387 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26c40cb5f0 con 0x7f26c40b04d0 2023-10-06T04:00:49.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.127+0000 7f26c359e700 1 -- 172.21.15.157:0/2213184387 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26c40b0e00 con 0x7f26c40b04d0 2023-10-06T04:00:49.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.127+0000 7f26baffd700 1 -- 172.21.15.157:0/2213184387 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f26ac022030 con 0x7f26c40b04d0 2023-10-06T04:00:49.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.128+0000 7f26baffd700 1 --2- 172.21.15.157:0/2213184387 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f26a4066f30 0x7f26a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:49.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.128+0000 7f26baffd700 1 -- 172.21.15.157:0/2213184387 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f26ac086580 con 0x7f26c40b04d0 2023-10-06T04:00:49.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.128+0000 7f26c259c700 1 --2- 172.21.15.157:0/2213184387 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f26a4066f30 0x7f26a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:49.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.128+0000 7f26c259c700 1 --2- 172.21.15.157:0/2213184387 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f26a4066f30 0x7f26a40693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f26b4003a30 tx=0x7f26b4007ea0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:00:49.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.132+0000 7f26baffd700 1 -- 172.21.15.157:0/2213184387 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f26ac053560 con 0x7f26c40b04d0 2023-10-06T04:00:49.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.360+0000 7f26c359e700 1 -- 172.21.15.157:0/2213184387 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26c40b0e00 con 0x7f26c40b04d0 2023-10-06T04:00:49.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.361+0000 7f26baffd700 1 -- 172.21.15.157:0/2213184387 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+6203 (secure 0 0 0) 0x7f26ac056b80 con 0x7f26c40b04d0 2023-10-06T04:00:49.362 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:00:49.363 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4936,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:00:42.818220+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4223":{"gid":4223,"name":"a","rank":4,"incarnation":17,"state":"up:active","state_seq":39,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:00:49.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.363+0000 7f26c359e700 1 -- 172.21.15.157:0/2213184387 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f26a4066f30 msgr2=0x7f26a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:49.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.363+0000 7f26c359e700 1 --2- 172.21.15.157:0/2213184387 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f26a4066f30 0x7f26a40693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f26b4003a30 tx=0x7f26b4007ea0 comp rx=0 tx=0).stop 2023-10-06T04:00:49.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.363+0000 7f26c359e700 1 -- 172.21.15.157:0/2213184387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26c40b04d0 msgr2=0x7f26c415b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:49.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.363+0000 7f26c359e700 1 --2- 172.21.15.157:0/2213184387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26c40b04d0 0x7f26c415b300 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f26c40899e0 tx=0x7f26ac016000 comp rx=0 tx=0).stop 2023-10-06T04:00:49.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.363+0000 7f26c359e700 1 -- 172.21.15.157:0/2213184387 shutdown_connections 2023-10-06T04:00:49.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.363+0000 7f26c359e700 1 --2- 172.21.15.157:0/2213184387 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f26a4066f30 0x7f26a40693e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:49.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.363+0000 7f26c359e700 1 --2- 172.21.15.157:0/2213184387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26c40b1e40 0x7f26c415b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:49.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.363+0000 7f26c359e700 1 --2- 172.21.15.157:0/2213184387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26c406c2d0 0x7f26c4165bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:49.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.363+0000 7f26c359e700 1 --2- 172.21.15.157:0/2213184387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26c40b04d0 0x7f26c415b300 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:49.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.363+0000 7f26c359e700 1 -- 172.21.15.157:0/2213184387 >> 172.21.15.157:0/2213184387 conn(0x7f26c40cc540 msgr2=0x7f26c4156a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:00:49.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.363+0000 7f26c359e700 1 -- 172.21.15.157:0/2213184387 shutdown_connections 2023-10-06T04:00:49.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:49.363+0000 7f26c359e700 1 -- 172.21.15.157:0/2213184387 wait complete. 2023-10-06T04:00:49.366 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 28 2023-10-06T04:00:49.376 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-10-06T04:00:51.377 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:00:51.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.506+0000 7f88fa4d8700 1 Processor -- start 2023-10-06T04:00:51.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.506+0000 7f88fa4d8700 1 -- start start 2023-10-06T04:00:51.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.507+0000 7f88fa4d8700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f88f406cbb0 0x7f88f406d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:51.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.507+0000 7f88fa4d8700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f88f406c1f0 0x7f88f406da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:51.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.507+0000 7f88fa4d8700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f40c9780 con 0x7f88f406cf80 2023-10-06T04:00:51.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.507+0000 7f88fa4d8700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88f40c8c40 con 0x7f88f406c1f0 2023-10-06T04:00:51.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.507+0000 7f88fa4d8700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88f406e0c0 con 0x7f88f406cbb0 2023-10-06T04:00:51.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.507+0000 7f88f3fff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f88f406cbb0 0x7f88f406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:51.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.507+0000 7f88f37fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f88f406c1f0 0x7f88f406da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:51.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.507+0000 7f88f8a75700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f88f406cf80 0x7f88f414fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.157:37116/0 (socket says 172.21.15.157:37116) 2023-10-06T04:00:51.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.507+0000 7f88f3fff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f88f406cbb0 0x7f88f406d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:38428/0 (socket says 172.21.15.157:38428) 2023-10-06T04:00:51.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.507+0000 7f88f8a75700 1 -- 172.21.15.157:0/1544776014 learned_addr learned my addr 172.21.15.157:0/1544776014 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:00:51.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.507+0000 7f88f37fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f88f406c1f0 0x7f88f406da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59418/0 (socket says 172.21.15.157:59418) 2023-10-06T04:00:51.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.507+0000 7f88f37fe700 1 -- 172.21.15.157:0/1544776014 >> v2:172.21.15.175:3300/0 conn(0x7f88f406cbb0 msgr2=0x7f88f406d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:00:51.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.507+0000 7f88f37fe700 1 --2- 172.21.15.157:0/1544776014 >> v2:172.21.15.175:3300/0 conn(0x7f88f406cbb0 0x7f88f406d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:51.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.507+0000 7f88f37fe700 1 -- 172.21.15.157:0/1544776014 >> v1:172.21.15.175:6789/0 conn(0x7f88f406cf80 legacy=0x7f88f414fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:51.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.507+0000 7f88f2ffd700 1 -- 172.21.15.157:0/1544776014 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1986888886 0 0) 0x7f88f40c9780 con 0x7f88f406cf80 2023-10-06T04:00:51.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.507+0000 7f88f37fe700 1 -- 172.21.15.157:0/1544776014 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f40c8e50 con 0x7f88f406c1f0 2023-10-06T04:00:51.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.508+0000 7f88f37fe700 1 --2- 172.21.15.157:0/1544776014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88f406c1f0 0x7f88f406da70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f88f406fd70 tx=0x7f88e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131739c5e7475a78 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:00:51.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.508+0000 7f88f2ffd700 1 -- 172.21.15.157:0/1544776014 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e800b690 con 0x7f88f406c1f0 2023-10-06T04:00:51.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.508+0000 7f88f2ffd700 1 -- 172.21.15.157:0/1544776014 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e800b7f0 con 0x7f88f406c1f0 2023-10-06T04:00:51.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.508+0000 7f88f2ffd700 1 -- 172.21.15.157:0/1544776014 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e800bb90 con 0x7f88f406c1f0 2023-10-06T04:00:51.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.508+0000 7f88fa4d8700 1 -- 172.21.15.157:0/1544776014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88f406c1f0 msgr2=0x7f88f406da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:51.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.508+0000 7f88fa4d8700 1 --2- 172.21.15.157:0/1544776014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88f406c1f0 0x7f88f406da70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f88f406fd70 tx=0x7f88e80091a0 comp rx=0 tx=0).stop 2023-10-06T04:00:51.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.508+0000 7f88fa4d8700 1 -- 172.21.15.157:0/1544776014 shutdown_connections 2023-10-06T04:00:51.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.508+0000 7f88fa4d8700 1 --2- 172.21.15.157:0/1544776014 >> v2:172.21.15.175:3300/0 conn(0x7f88f406cbb0 0x7f88f406d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:51.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.508+0000 7f88fa4d8700 1 --2- 172.21.15.157:0/1544776014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88f406c1f0 0x7f88f406da70 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:51.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.508+0000 7f88fa4d8700 1 -- 172.21.15.157:0/1544776014 >> 172.21.15.157:0/1544776014 conn(0x7f88f406f230 msgr2=0x7f88f406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:00:51.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.508+0000 7f88fa4d8700 1 -- 172.21.15.157:0/1544776014 shutdown_connections 2023-10-06T04:00:51.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.508+0000 7f88fa4d8700 1 -- 172.21.15.157:0/1544776014 wait complete. 2023-10-06T04:00:51.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.509+0000 7f88fa4d8700 1 Processor -- start 2023-10-06T04:00:51.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.510+0000 7f88fa4d8700 1 -- start start 2023-10-06T04:00:51.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.510+0000 7f88fa4d8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88f406c1f0 0x7f88f40c4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:51.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.510+0000 7f88fa4d8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88f406cbb0 0x7f88f4155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:51.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.510+0000 7f88fa4d8700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88f406cf80 0x7f88f4156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:51.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.510+0000 7f88fa4d8700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f4148be0 con 0x7f88f406cbb0 2023-10-06T04:00:51.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.510+0000 7f88fa4d8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f416a4e0 con 0x7f88f406c1f0 2023-10-06T04:00:51.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.510+0000 7f88fa4d8700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f40c67f0 con 0x7f88f406cf80 2023-10-06T04:00:51.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.510+0000 7f88f37fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88f406cbb0 0x7f88f4155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:51.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.510+0000 7f88f37fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88f406cbb0 0x7f88f4155d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59428/0 (socket says 172.21.15.157:59428) 2023-10-06T04:00:51.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.510+0000 7f88f37fe700 1 -- 172.21.15.157:0/427681357 learned_addr learned my addr 172.21.15.157:0/427681357 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:00:51.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.511+0000 7f88f3fff700 1 --2- 172.21.15.157:0/427681357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88f406c1f0 0x7f88f40c4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:51.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.511+0000 7f88f8a75700 1 --2- 172.21.15.157:0/427681357 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88f406cf80 0x7f88f4156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:51.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.511+0000 7f88f37fe700 1 -- 172.21.15.157:0/427681357 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88f406cf80 msgr2=0x7f88f4156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:51.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.511+0000 7f88f37fe700 1 --2- 172.21.15.157:0/427681357 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88f406cf80 0x7f88f4156280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:51.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.511+0000 7f88f37fe700 1 -- 172.21.15.157:0/427681357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88f406c1f0 msgr2=0x7f88f40c4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:51.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.511+0000 7f88f37fe700 1 --2- 172.21.15.157:0/427681357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88f406c1f0 0x7f88f40c4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:51.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.511+0000 7f88f37fe700 1 -- 172.21.15.157:0/427681357 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e8009420 con 0x7f88f406cbb0 2023-10-06T04:00:51.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.511+0000 7f88f37fe700 1 --2- 172.21.15.157:0/427681357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88f406cbb0 0x7f88f4155d40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f88f40c3960 tx=0x7f88e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:00:51.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.511+0000 7f88dbfff700 1 -- 172.21.15.157:0/427681357 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e800b960 con 0x7f88f406cbb0 2023-10-06T04:00:51.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.511+0000 7f88dbfff700 1 -- 172.21.15.157:0/427681357 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e800bac0 con 0x7f88f406cbb0 2023-10-06T04:00:51.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.511+0000 7f88fa4d8700 1 -- 172.21.15.157:0/427681357 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f40c6a40 con 0x7f88f406cbb0 2023-10-06T04:00:51.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.511+0000 7f88dbfff700 1 -- 172.21.15.157:0/427681357 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e8025670 con 0x7f88f406cbb0 2023-10-06T04:00:51.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.511+0000 7f88fa4d8700 1 -- 172.21.15.157:0/427681357 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f416a6e0 con 0x7f88f406cbb0 2023-10-06T04:00:51.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.512+0000 7f88fa4d8700 1 -- 172.21.15.157:0/427681357 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88f4153de0 con 0x7f88f406cbb0 2023-10-06T04:00:51.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.512+0000 7f88dbfff700 1 -- 172.21.15.157:0/427681357 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f88e8023070 con 0x7f88f406cbb0 2023-10-06T04:00:51.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.513+0000 7f88dbfff700 1 --2- 172.21.15.157:0/427681357 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f88e0066f30 0x7f88e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:51.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.513+0000 7f88dbfff700 1 -- 172.21.15.157:0/427681357 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f88e8085dd0 con 0x7f88f406cbb0 2023-10-06T04:00:51.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.513+0000 7f88f3fff700 1 --2- 172.21.15.157:0/427681357 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f88e0066f30 0x7f88e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:51.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.513+0000 7f88f3fff700 1 --2- 172.21.15.157:0/427681357 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f88e0066f30 0x7f88e00693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f88e4009d40 tx=0x7f88e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:00:51.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.517+0000 7f88dbfff700 1 -- 172.21.15.157:0/427681357 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f88f4153de0 con 0x7f88f406cbb0 2023-10-06T04:00:51.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.743+0000 7f88fa4d8700 1 -- 172.21.15.157:0/427681357 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88f416aa50 con 0x7f88f406cbb0 2023-10-06T04:00:51.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.743+0000 7f88dbfff700 1 -- 172.21.15.157:0/427681357 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+6203 (secure 0 0 0) 0x7f88f416aa50 con 0x7f88f406cbb0 2023-10-06T04:00:51.745 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:00:51.745 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4936,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:00:42.818220+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4223":{"gid":4223,"name":"a","rank":4,"incarnation":17,"state":"up:active","state_seq":39,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:00:51.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.745+0000 7f88fa4d8700 1 -- 172.21.15.157:0/427681357 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f88e0066f30 msgr2=0x7f88e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:51.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.745+0000 7f88fa4d8700 1 --2- 172.21.15.157:0/427681357 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f88e0066f30 0x7f88e00693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f88e4009d40 tx=0x7f88e4008040 comp rx=0 tx=0).stop 2023-10-06T04:00:51.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.746+0000 7f88fa4d8700 1 -- 172.21.15.157:0/427681357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88f406cbb0 msgr2=0x7f88f4155d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:51.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.746+0000 7f88fa4d8700 1 --2- 172.21.15.157:0/427681357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88f406cbb0 0x7f88f4155d40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f88f40c3960 tx=0x7f88e8017000 comp rx=0 tx=0).stop 2023-10-06T04:00:51.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.746+0000 7f88fa4d8700 1 -- 172.21.15.157:0/427681357 shutdown_connections 2023-10-06T04:00:51.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.746+0000 7f88fa4d8700 1 --2- 172.21.15.157:0/427681357 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f88e0066f30 0x7f88e00693e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:51.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.746+0000 7f88fa4d8700 1 --2- 172.21.15.157:0/427681357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88f406c1f0 0x7f88f40c4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:51.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.746+0000 7f88fa4d8700 1 --2- 172.21.15.157:0/427681357 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88f406cf80 0x7f88f4156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:51.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.746+0000 7f88fa4d8700 1 --2- 172.21.15.157:0/427681357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88f406cbb0 0x7f88f4155d40 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:51.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.746+0000 7f88fa4d8700 1 -- 172.21.15.157:0/427681357 >> 172.21.15.157:0/427681357 conn(0x7f88f406f230 msgr2=0x7f88f40c77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:00:51.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.746+0000 7f88fa4d8700 1 -- 172.21.15.157:0/427681357 shutdown_connections 2023-10-06T04:00:51.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:51.746+0000 7f88fa4d8700 1 -- 172.21.15.157:0/427681357 wait complete. 2023-10-06T04:00:51.749 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 28 2023-10-06T04:00:51.759 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-10-06T04:00:53.760 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:00:53.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.884+0000 7fd3b31f0700 1 Processor -- start 2023-10-06T04:00:53.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.884+0000 7fd3b31f0700 1 -- start start 2023-10-06T04:00:53.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.885+0000 7fd3b31f0700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3ac0ca630 0x7fd3ac153e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:53.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.885+0000 7fd3b31f0700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd3ac0caa00 0x7fd3ac0c18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:53.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.885+0000 7fd3b31f0700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3ac0c39c0 con 0x7fd3ac0c3e50 2023-10-06T04:00:53.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.885+0000 7fd3b31f0700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3ac154340 con 0x7fd3ac0caa00 2023-10-06T04:00:53.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.885+0000 7fd3b31f0700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac068b80 con 0x7fd3ac0ca630 2023-10-06T04:00:53.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.885+0000 7fd3a3fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd3ac0caa00 0x7fd3ac0c18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:53.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.885+0000 7fd3a3fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd3ac0caa00 0x7fd3ac0c18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56836/0 (socket says 172.21.15.157:56836) 2023-10-06T04:00:53.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.885+0000 7fd3a3fff700 1 -- 172.21.15.157:0/618807054 learned_addr learned my addr 172.21.15.157:0/618807054 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:00:53.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.885+0000 7fd3b0f8c700 1 --2- 172.21.15.157:0/618807054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3ac0ca630 0x7fd3ac153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:53.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.885+0000 7fd3a3fff700 1 -- 172.21.15.157:0/618807054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3ac0ca630 msgr2=0x7fd3ac153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:00:53.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.885+0000 7fd3a3fff700 1 --2- 172.21.15.157:0/618807054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3ac0ca630 0x7fd3ac153e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:53.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.885+0000 7fd3a3fff700 1 -- 172.21.15.157:0/618807054 >> v1:172.21.15.175:6789/0 conn(0x7fd3ac0c3e50 legacy=0x7fd3ac0c4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:53.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.885+0000 7fd3a3fff700 1 -- 172.21.15.157:0/618807054 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ac0c1e00 con 0x7fd3ac0caa00 2023-10-06T04:00:53.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.886+0000 7fd3a3fff700 1 --2- 172.21.15.157:0/618807054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3ac0caa00 0x7fd3ac0c18c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0099b0 tx=0x7fd39c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e09ba10a907e2f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:00:53.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.886+0000 7fd3a37fe700 1 -- 172.21.15.157:0/618807054 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39c00b8f0 con 0x7fd3ac0caa00 2023-10-06T04:00:53.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.886+0000 7fd3a37fe700 1 -- 172.21.15.157:0/618807054 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39c00ba50 con 0x7fd3ac0caa00 2023-10-06T04:00:53.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.886+0000 7fd3a37fe700 1 -- 172.21.15.157:0/618807054 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39c00bdf0 con 0x7fd3ac0caa00 2023-10-06T04:00:53.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.886+0000 7fd3b31f0700 1 -- 172.21.15.157:0/618807054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3ac0caa00 msgr2=0x7fd3ac0c18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:53.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.886+0000 7fd3b31f0700 1 --2- 172.21.15.157:0/618807054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3ac0caa00 0x7fd3ac0c18c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0099b0 tx=0x7fd39c009250 comp rx=0 tx=0).stop 2023-10-06T04:00:53.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.886+0000 7fd3b31f0700 1 -- 172.21.15.157:0/618807054 shutdown_connections 2023-10-06T04:00:53.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.886+0000 7fd3b31f0700 1 --2- 172.21.15.157:0/618807054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3ac0ca630 0x7fd3ac153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:53.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.886+0000 7fd3b31f0700 1 --2- 172.21.15.157:0/618807054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3ac0caa00 0x7fd3ac0c18c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:53.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.886+0000 7fd3b31f0700 1 -- 172.21.15.157:0/618807054 >> 172.21.15.157:0/618807054 conn(0x7fd3ac0692b0 msgr2=0x7fd3ac0696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:00:53.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.886+0000 7fd3b31f0700 1 -- 172.21.15.157:0/618807054 shutdown_connections 2023-10-06T04:00:53.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.886+0000 7fd3b31f0700 1 -- 172.21.15.157:0/618807054 wait complete. 2023-10-06T04:00:53.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.887+0000 7fd3b31f0700 1 Processor -- start 2023-10-06T04:00:53.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.888+0000 7fd3b31f0700 1 -- start start 2023-10-06T04:00:53.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.888+0000 7fd3b31f0700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3ac0c3e50 0x7fd3ac168090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:53.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.888+0000 7fd3b31f0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3ac0ca630 0x7fd3ac158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:53.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.888+0000 7fd3b31f0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3ac0caa00 0x7fd3ac153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:53.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.888+0000 7fd3b31f0700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac1685d0 con 0x7fd3ac0caa00 2023-10-06T04:00:53.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.888+0000 7fd3b31f0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac152bc0 con 0x7fd3ac0ca630 2023-10-06T04:00:53.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.888+0000 7fd3b31f0700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac0c4d30 con 0x7fd3ac0c3e50 2023-10-06T04:00:53.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.888+0000 7fd3b178d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3ac0caa00 0x7fd3ac153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:53.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.888+0000 7fd3b178d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3ac0caa00 0x7fd3ac153800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56842/0 (socket says 172.21.15.157:56842) 2023-10-06T04:00:53.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.888+0000 7fd3b178d700 1 -- 172.21.15.157:0/4086701499 learned_addr learned my addr 172.21.15.157:0/4086701499 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:00:53.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.889+0000 7fd3b0f8c700 1 --2- 172.21.15.157:0/4086701499 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3ac0c3e50 0x7fd3ac168090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:53.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.889+0000 7fd3a3fff700 1 --2- 172.21.15.157:0/4086701499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3ac0ca630 0x7fd3ac158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:53.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.889+0000 7fd3b178d700 1 -- 172.21.15.157:0/4086701499 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3ac0c3e50 msgr2=0x7fd3ac168090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:53.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.889+0000 7fd3b178d700 1 --2- 172.21.15.157:0/4086701499 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3ac0c3e50 0x7fd3ac168090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:53.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.889+0000 7fd3b178d700 1 -- 172.21.15.157:0/4086701499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3ac0ca630 msgr2=0x7fd3ac158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:53.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.889+0000 7fd3b178d700 1 --2- 172.21.15.157:0/4086701499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3ac0ca630 0x7fd3ac158650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:53.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.889+0000 7fd3b178d700 1 -- 172.21.15.157:0/4086701499 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd39c0094f0 con 0x7fd3ac0caa00 2023-10-06T04:00:53.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.889+0000 7fd3b178d700 1 --2- 172.21.15.157:0/4086701499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3ac0caa00 0x7fd3ac153800 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400c4c0 tx=0x7fd3a4007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:00:53.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.889+0000 7fd3a0ff9700 1 -- 172.21.15.157:0/4086701499 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a400ce70 con 0x7fd3ac0caa00 2023-10-06T04:00:53.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.889+0000 7fd3a0ff9700 1 -- 172.21.15.157:0/4086701499 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a4011590 con 0x7fd3ac0caa00 2023-10-06T04:00:53.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.889+0000 7fd3b31f0700 1 -- 172.21.15.157:0/4086701499 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3ac152340 con 0x7fd3ac0caa00 2023-10-06T04:00:53.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.889+0000 7fd3a0ff9700 1 -- 172.21.15.157:0/4086701499 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4011970 con 0x7fd3ac0caa00 2023-10-06T04:00:53.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.889+0000 7fd3b31f0700 1 -- 172.21.15.157:0/4086701499 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3ac1528e0 con 0x7fd3ac0caa00 2023-10-06T04:00:53.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.890+0000 7fd3b31f0700 1 -- 172.21.15.157:0/4086701499 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3ac06a8f0 con 0x7fd3ac0caa00 2023-10-06T04:00:53.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.891+0000 7fd3a0ff9700 1 -- 172.21.15.157:0/4086701499 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd3a4011ce0 con 0x7fd3ac0caa00 2023-10-06T04:00:53.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.891+0000 7fd3a0ff9700 1 --2- 172.21.15.157:0/4086701499 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd398066f30 0x7fd3980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:53.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.891+0000 7fd3a0ff9700 1 -- 172.21.15.157:0/4086701499 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6202+0+0 (secure 0 0 0) 0x7fd3a4083a00 con 0x7fd3ac0caa00 2023-10-06T04:00:53.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.891+0000 7fd3b0f8c700 1 --2- 172.21.15.157:0/4086701499 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd398066f30 0x7fd3980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:53.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.892+0000 7fd3b0f8c700 1 --2- 172.21.15.157:0/4086701499 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd398066f30 0x7fd3980693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd3a80075a0 tx=0x7fd3a80097e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:00:53.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:53.895+0000 7fd3a0ff9700 1 -- 172.21.15.157:0/4086701499 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd3a4050a80 con 0x7fd3ac0caa00 2023-10-06T04:00:54.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:54.119+0000 7fd3b31f0700 1 -- 172.21.15.157:0/4086701499 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3ac146150 con 0x7fd3ac0caa00 2023-10-06T04:00:54.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:54.120+0000 7fd3a0ff9700 1 -- 172.21.15.157:0/4086701499 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+6203 (secure 0 0 0) 0x7fd3a4017070 con 0x7fd3ac0caa00 2023-10-06T04:00:54.121 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:00:54.122 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4936,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:00:42.818220+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4223":{"gid":4223,"name":"a","rank":4,"incarnation":17,"state":"up:active","state_seq":39,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:00:54.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:54.122+0000 7fd3b31f0700 1 -- 172.21.15.157:0/4086701499 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd398066f30 msgr2=0x7fd3980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:54.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:54.122+0000 7fd3b31f0700 1 --2- 172.21.15.157:0/4086701499 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd398066f30 0x7fd3980693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd3a80075a0 tx=0x7fd3a80097e0 comp rx=0 tx=0).stop 2023-10-06T04:00:54.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:54.122+0000 7fd3b31f0700 1 -- 172.21.15.157:0/4086701499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3ac0caa00 msgr2=0x7fd3ac153800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:54.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:54.122+0000 7fd3b31f0700 1 --2- 172.21.15.157:0/4086701499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3ac0caa00 0x7fd3ac153800 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400c4c0 tx=0x7fd3a4007f50 comp rx=0 tx=0).stop 2023-10-06T04:00:54.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:54.122+0000 7fd3b31f0700 1 -- 172.21.15.157:0/4086701499 shutdown_connections 2023-10-06T04:00:54.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:54.122+0000 7fd3b31f0700 1 --2- 172.21.15.157:0/4086701499 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd398066f30 0x7fd3980693e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:54.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:54.122+0000 7fd3b31f0700 1 --2- 172.21.15.157:0/4086701499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3ac0caa00 0x7fd3ac153800 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:54.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:54.122+0000 7fd3b31f0700 1 --2- 172.21.15.157:0/4086701499 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3ac0c3e50 0x7fd3ac168090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:54.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:54.122+0000 7fd3b31f0700 1 --2- 172.21.15.157:0/4086701499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3ac0ca630 0x7fd3ac158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:54.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:54.122+0000 7fd3b31f0700 1 -- 172.21.15.157:0/4086701499 >> 172.21.15.157:0/4086701499 conn(0x7fd3ac0692b0 msgr2=0x7fd3ac14a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:00:54.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:54.122+0000 7fd3b31f0700 1 -- 172.21.15.157:0/4086701499 shutdown_connections 2023-10-06T04:00:54.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:54.122+0000 7fd3b31f0700 1 -- 172.21.15.157:0/4086701499 wait complete. 2023-10-06T04:00:54.125 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 28 2023-10-06T04:00:54.135 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-10-06T04:00:56.136 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:00:56.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.263+0000 7f797fb9e700 1 Processor -- start 2023-10-06T04:00:56.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.263+0000 7f797fb9e700 1 -- start start 2023-10-06T04:00:56.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.264+0000 7f797fb9e700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f797806cbb0 0x7f797806d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:56.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.264+0000 7f797fb9e700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f797806c1f0 0x7f797806da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:56.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.264+0000 7f797fb9e700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79780c9780 con 0x7f797806cf80 2023-10-06T04:00:56.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.264+0000 7f797fb9e700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79780c8c40 con 0x7f797806c1f0 2023-10-06T04:00:56.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.264+0000 7f797fb9e700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f797806e0c0 con 0x7f797806cbb0 2023-10-06T04:00:56.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.264+0000 7f797d139700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f797806c1f0 0x7f797806da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:56.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.264+0000 7f797e13b700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f797806cf80 0x7f797814fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59156/0 (socket says 172.21.15.157:59156) 2023-10-06T04:00:56.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.264+0000 7f797e13b700 1 -- 172.21.15.157:0/175663066 learned_addr learned my addr 172.21.15.157:0/175663066 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:00:56.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.264+0000 7f797d93a700 1 --2- 172.21.15.157:0/175663066 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f797806cbb0 0x7f797806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:56.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.264+0000 7f797d139700 1 -- 172.21.15.157:0/175663066 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f797806cbb0 msgr2=0x7f797806d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:00:56.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.264+0000 7f797d139700 1 --2- 172.21.15.157:0/175663066 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f797806cbb0 0x7f797806d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:56.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.264+0000 7f797d139700 1 -- 172.21.15.157:0/175663066 >> v1:172.21.15.157:6789/0 conn(0x7f797806cf80 legacy=0x7f797814fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:56.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.264+0000 7f797d139700 1 -- 172.21.15.157:0/175663066 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79780c8e50 con 0x7f797806c1f0 2023-10-06T04:00:56.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.264+0000 7f797c938700 1 -- 172.21.15.157:0/175663066 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2946021535 0 0) 0x7f79780c9780 con 0x7f797806cf80 2023-10-06T04:00:56.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.264+0000 7f797d139700 1 --2- 172.21.15.157:0/175663066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f797806c1f0 0x7f797806da70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7968009910 tx=0x7f79680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca8ad53d97661c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:00:56.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.265+0000 7f797c938700 1 -- 172.21.15.157:0/175663066 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796800b8b0 con 0x7f797806c1f0 2023-10-06T04:00:56.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.265+0000 7f797c938700 1 -- 172.21.15.157:0/175663066 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796800ba10 con 0x7f797806c1f0 2023-10-06T04:00:56.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.265+0000 7f797c938700 1 -- 172.21.15.157:0/175663066 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796800bdb0 con 0x7f797806c1f0 2023-10-06T04:00:56.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.265+0000 7f797fb9e700 1 -- 172.21.15.157:0/175663066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f797806c1f0 msgr2=0x7f797806da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:56.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.265+0000 7f797fb9e700 1 --2- 172.21.15.157:0/175663066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f797806c1f0 0x7f797806da70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7968009910 tx=0x7f79680091a0 comp rx=0 tx=0).stop 2023-10-06T04:00:56.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.265+0000 7f797fb9e700 1 -- 172.21.15.157:0/175663066 shutdown_connections 2023-10-06T04:00:56.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.265+0000 7f797fb9e700 1 --2- 172.21.15.157:0/175663066 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f797806cbb0 0x7f797806d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:56.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.265+0000 7f797fb9e700 1 --2- 172.21.15.157:0/175663066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f797806c1f0 0x7f797806da70 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:56.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.265+0000 7f797fb9e700 1 -- 172.21.15.157:0/175663066 >> 172.21.15.157:0/175663066 conn(0x7f797806f230 msgr2=0x7f797806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:00:56.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.265+0000 7f797fb9e700 1 -- 172.21.15.157:0/175663066 shutdown_connections 2023-10-06T04:00:56.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.265+0000 7f797fb9e700 1 -- 172.21.15.157:0/175663066 wait complete. 2023-10-06T04:00:56.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.266+0000 7f797fb9e700 1 Processor -- start 2023-10-06T04:00:56.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.266+0000 7f797fb9e700 1 -- start start 2023-10-06T04:00:56.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.267+0000 7f797fb9e700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f797806c1f0 0x7f7978156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:56.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.267+0000 7f797fb9e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f797806cbb0 0x7f7978158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:56.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.267+0000 7f797fb9e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f797806cf80 0x7f79781591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:56.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.267+0000 7f797fb9e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797816aac0 con 0x7f797806cf80 2023-10-06T04:00:56.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.267+0000 7f797fb9e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79781540c0 con 0x7f797806cbb0 2023-10-06T04:00:56.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.267+0000 7f797fb9e700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f797816a560 con 0x7f797806c1f0 2023-10-06T04:00:56.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.267+0000 7f797e13b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f797806cf80 0x7f79781591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:56.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.267+0000 7f797d93a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f797806c1f0 0x7f7978156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:56.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.267+0000 7f797d139700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f797806cbb0 0x7f7978158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:56.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.267+0000 7f797e13b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f797806cf80 0x7f79781591e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56850/0 (socket says 172.21.15.157:56850) 2023-10-06T04:00:56.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.267+0000 7f797d93a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f797806c1f0 0x7f7978156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:51090/0 (socket says 172.21.15.157:51090) 2023-10-06T04:00:56.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.267+0000 7f797e13b700 1 -- 172.21.15.157:0/105367265 learned_addr learned my addr 172.21.15.157:0/105367265 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:00:56.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.267+0000 7f797e13b700 1 -- 172.21.15.157:0/105367265 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f797806c1f0 msgr2=0x7f7978156630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:56.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.267+0000 7f797e13b700 1 --2- 172.21.15.157:0/105367265 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f797806c1f0 0x7f7978156630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:56.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.267+0000 7f797e13b700 1 -- 172.21.15.157:0/105367265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f797806cbb0 msgr2=0x7f7978158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:56.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.267+0000 7f797e13b700 1 --2- 172.21.15.157:0/105367265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f797806cbb0 0x7f7978158ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:56.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.267+0000 7f797e13b700 1 -- 172.21.15.157:0/105367265 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7968009420 con 0x7f797806cf80 2023-10-06T04:00:56.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.268+0000 7f797e13b700 1 --2- 172.21.15.157:0/105367265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f797806cf80 0x7f79781591e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f797000cc20 tx=0x7f79700019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:00:56.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.268+0000 7f796dffb700 1 -- 172.21.15.157:0/105367265 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797000a870 con 0x7f797806cf80 2023-10-06T04:00:56.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.268+0000 7f796dffb700 1 -- 172.21.15.157:0/105367265 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797000a9d0 con 0x7f797806cf80 2023-10-06T04:00:56.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.268+0000 7f797fb9e700 1 -- 172.21.15.157:0/105367265 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79780aeee0 con 0x7f797806cf80 2023-10-06T04:00:56.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.268+0000 7f796dffb700 1 -- 172.21.15.157:0/105367265 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797000ad90 con 0x7f797806cf80 2023-10-06T04:00:56.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.268+0000 7f797fb9e700 1 -- 172.21.15.157:0/105367265 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7978157400 con 0x7f797806cf80 2023-10-06T04:00:56.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.269+0000 7f797fb9e700 1 -- 172.21.15.157:0/105367265 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7978154250 con 0x7f797806cf80 2023-10-06T04:00:56.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.269+0000 7f796dffb700 1 -- 172.21.15.157:0/105367265 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7970007300 con 0x7f797806cf80 2023-10-06T04:00:56.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.270+0000 7f796dffb700 1 --2- 172.21.15.157:0/105367265 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f795c066f60 0x7f795c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:56.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.270+0000 7f796dffb700 1 -- 172.21.15.157:0/105367265 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f79700849e0 con 0x7f797806cf80 2023-10-06T04:00:56.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.270+0000 7f797d93a700 1 --2- 172.21.15.157:0/105367265 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f795c066f60 0x7f795c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:56.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.270+0000 7f797d93a700 1 --2- 172.21.15.157:0/105367265 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f795c066f60 0x7f795c069410 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7974006fd0 tx=0x7f7974008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:00:56.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.274+0000 7f796dffb700 1 -- 172.21.15.157:0/105367265 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7970051ae0 con 0x7f797806cf80 2023-10-06T04:00:56.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.501+0000 7f797fb9e700 1 -- 172.21.15.157:0/105367265 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79780c9130 con 0x7f797806cf80 2023-10-06T04:00:56.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.501+0000 7f796dffb700 1 -- 172.21.15.157:0/105367265 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+6203 (secure 0 0 0) 0x7f7970055100 con 0x7f797806cf80 2023-10-06T04:00:56.503 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:00:56.503 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4936,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:00:42.818220+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4223":{"gid":4223,"name":"a","rank":4,"incarnation":17,"state":"up:active","state_seq":39,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:00:56.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.503+0000 7f797fb9e700 1 -- 172.21.15.157:0/105367265 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f795c066f60 msgr2=0x7f795c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:56.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.503+0000 7f797fb9e700 1 --2- 172.21.15.157:0/105367265 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f795c066f60 0x7f795c069410 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7974006fd0 tx=0x7f7974008040 comp rx=0 tx=0).stop 2023-10-06T04:00:56.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.503+0000 7f797fb9e700 1 -- 172.21.15.157:0/105367265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f797806cf80 msgr2=0x7f79781591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:56.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.503+0000 7f797fb9e700 1 --2- 172.21.15.157:0/105367265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f797806cf80 0x7f79781591e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f797000cc20 tx=0x7f79700019b0 comp rx=0 tx=0).stop 2023-10-06T04:00:56.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.503+0000 7f797fb9e700 1 -- 172.21.15.157:0/105367265 shutdown_connections 2023-10-06T04:00:56.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.503+0000 7f797fb9e700 1 --2- 172.21.15.157:0/105367265 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f795c066f60 0x7f795c069410 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:56.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.503+0000 7f797fb9e700 1 --2- 172.21.15.157:0/105367265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f797806cf80 0x7f79781591e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:56.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.503+0000 7f797fb9e700 1 --2- 172.21.15.157:0/105367265 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f797806c1f0 0x7f7978156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:56.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.503+0000 7f797fb9e700 1 --2- 172.21.15.157:0/105367265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f797806cbb0 0x7f7978158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:56.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.503+0000 7f797fb9e700 1 -- 172.21.15.157:0/105367265 >> 172.21.15.157:0/105367265 conn(0x7f797806f230 msgr2=0x7f7978150e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:00:56.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.503+0000 7f797fb9e700 1 -- 172.21.15.157:0/105367265 shutdown_connections 2023-10-06T04:00:56.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:56.503+0000 7f797fb9e700 1 -- 172.21.15.157:0/105367265 wait complete. 2023-10-06T04:00:56.507 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 28 2023-10-06T04:00:56.517 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-10-06T04:00:58.518 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:00:58.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.644+0000 7fb5b7bb4700 1 Processor -- start 2023-10-06T04:00:58.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.645+0000 7fb5b7bb4700 1 -- start start 2023-10-06T04:00:58.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.645+0000 7fb5b7bb4700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5b0150410 0x7fb5b0155f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:58.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.645+0000 7fb5b7bb4700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5b00c9780 con 0x7fb5b006cbb0 2023-10-06T04:00:58.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.645+0000 7fb5b7bb4700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5b006e0c0 con 0x7fb5b006dc40 2023-10-06T04:00:58.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.645+0000 7fb5b7bb4700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b00c8c40 con 0x7fb5b0150410 2023-10-06T04:00:58.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.646+0000 7fb5b5950700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb5b006cbb0 0x7fb5b006d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59164/0 (socket says 172.21.15.157:59164) 2023-10-06T04:00:58.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.646+0000 7fb5b5950700 1 -- 172.21.15.157:0/2748084457 learned_addr learned my addr 172.21.15.157:0/2748084457 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:00:58.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.646+0000 7fb5b6151700 1 --2- 172.21.15.157:0/2748084457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5b0150410 0x7fb5b0155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:58.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.646+0000 7fb5b494e700 1 -- 172.21.15.157:0/2748084457 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3396481514 0 0) 0x7fb5b00c9780 con 0x7fb5b006cbb0 2023-10-06T04:00:58.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.646+0000 7fb5b494e700 1 -- 172.21.15.157:0/2748084457 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb59c003570 con 0x7fb5b006cbb0 2023-10-06T04:00:58.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.646+0000 7fb5b494e700 1 -- 172.21.15.157:0/2748084457 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4262193666 0 0) 0x7fb5b006e0c0 con 0x7fb5b006dc40 2023-10-06T04:00:58.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.646+0000 7fb5b494e700 1 -- 172.21.15.157:0/2748084457 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5b00c9780 con 0x7fb5b006dc40 2023-10-06T04:00:58.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.646+0000 7fb5b494e700 1 -- 172.21.15.157:0/2748084457 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3625672911 0 0) 0x7fb59c003570 con 0x7fb5b006cbb0 2023-10-06T04:00:58.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.646+0000 7fb5b494e700 1 -- 172.21.15.157:0/2748084457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5b0150410 msgr2=0x7fb5b0155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:00:58.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.646+0000 7fb5b494e700 1 --2- 172.21.15.157:0/2748084457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5b0150410 0x7fb5b0155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:58.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.646+0000 7fb5b494e700 1 -- 172.21.15.157:0/2748084457 >> v1:172.21.15.175:6789/0 conn(0x7fb5b006dc40 legacy=0x7fb5b014fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:58.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.646+0000 7fb5b494e700 1 -- 172.21.15.157:0/2748084457 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b00c8e50 con 0x7fb5b006cbb0 2023-10-06T04:00:58.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.646+0000 7fb5b494e700 1 -- 172.21.15.157:0/2748084457 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7fb5ac0017f0 con 0x7fb5b006cbb0 2023-10-06T04:00:58.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.646+0000 7fb5b6151700 1 --2- 172.21.15.157:0/2748084457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5b0150410 0x7fb5b0155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:00:58.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.646+0000 7fb5b494e700 1 -- 172.21.15.157:0/2748084457 >> v1:172.21.15.157:6789/0 conn(0x7fb5b006cbb0 legacy=0x7fb5b006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:58.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.646+0000 7fb5b494e700 1 --2- 172.21.15.157:0/2748084457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5b0150410 0x7fb59c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:58.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.646+0000 7fb5b494e700 1 --2- 172.21.15.157:0/2748084457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb59c004160 0x7fb59c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:58.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.646+0000 7fb5b494e700 1 --2- 172.21.15.157:0/2748084457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb59c004d50 0x7fb59c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:58.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.646+0000 7fb5b494e700 1 -- 172.21.15.157:0/2748084457 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b00c8c40 con 0x7fb59c004160 2023-10-06T04:00:58.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.646+0000 7fb5b494e700 1 -- 172.21.15.157:0/2748084457 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c00bba0 con 0x7fb59c004d50 2023-10-06T04:00:58.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.646+0000 7fb5b494e700 1 -- 172.21.15.157:0/2748084457 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c00bd60 con 0x7fb5b0150410 2023-10-06T04:00:58.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.646+0000 7fb5b514f700 1 --2- 172.21.15.157:0/2748084457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb59c004160 0x7fb59c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:58.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.646+0000 7fb5b5950700 1 --2- 172.21.15.157:0/2748084457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5b0150410 0x7fb59c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:58.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.646+0000 7fb5b6151700 1 --2- 172.21.15.157:0/2748084457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb59c004d50 0x7fb59c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:58.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.647+0000 7fb5b514f700 1 -- 172.21.15.157:0/2748084457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5b0150410 msgr2=0x7fb59c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:00:58.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.647+0000 7fb5b514f700 1 --2- 172.21.15.157:0/2748084457 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5b0150410 0x7fb59c003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:58.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.647+0000 7fb5b514f700 1 -- 172.21.15.157:0/2748084457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb59c004d50 msgr2=0x7fb59c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:00:58.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.647+0000 7fb5b514f700 1 --2- 172.21.15.157:0/2748084457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb59c004d50 0x7fb59c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:58.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.647+0000 7fb5b514f700 1 -- 172.21.15.157:0/2748084457 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a8009420 con 0x7fb59c004160 2023-10-06T04:00:58.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.647+0000 7fb5b514f700 1 --2- 172.21.15.157:0/2748084457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb59c004160 0x7fb59c004810 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0001a10 tx=0x7fb5a0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8f7d6102a5b1b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:00:58.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.647+0000 7fb5b494e700 1 -- 172.21.15.157:0/2748084457 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a000c5d0 con 0x7fb59c004160 2023-10-06T04:00:58.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.647+0000 7fb5b494e700 1 -- 172.21.15.157:0/2748084457 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5a000c730 con 0x7fb59c004160 2023-10-06T04:00:58.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.647+0000 7fb5b494e700 1 -- 172.21.15.157:0/2748084457 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a000cad0 con 0x7fb59c004160 2023-10-06T04:00:58.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.647+0000 7fb5b7bb4700 1 -- 172.21.15.157:0/2748084457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb59c004160 msgr2=0x7fb59c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:58.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.647+0000 7fb5b7bb4700 1 --2- 172.21.15.157:0/2748084457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb59c004160 0x7fb59c004810 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0001a10 tx=0x7fb5a0007d30 comp rx=0 tx=0).stop 2023-10-06T04:00:58.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.647+0000 7fb5b6151700 1 -- 172.21.15.157:0/2748084457 reap_dead start 2023-10-06T04:00:58.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.647+0000 7fb5b7bb4700 1 -- 172.21.15.157:0/2748084457 shutdown_connections 2023-10-06T04:00:58.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.647+0000 7fb5b7bb4700 1 -- 172.21.15.157:0/2748084457 >> 172.21.15.157:0/2748084457 conn(0x7fb5b006f230 msgr2=0x7fb5b006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:00:58.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.648+0000 7fb5b7bb4700 1 -- 172.21.15.157:0/2748084457 shutdown_connections 2023-10-06T04:00:58.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.648+0000 7fb5b7bb4700 1 -- 172.21.15.157:0/2748084457 wait complete. 2023-10-06T04:00:58.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.649+0000 7fb5b7bb4700 1 Processor -- start 2023-10-06T04:00:58.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.649+0000 7fb5b7bb4700 1 -- start start 2023-10-06T04:00:58.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.649+0000 7fb5b7bb4700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb59c004160 0x7fb5b0169430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:58.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.649+0000 7fb5b7bb4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5b00c4ab0 0x7fb5b01668c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:58.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.649+0000 7fb5b7bb4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5b00c20f0 0x7fb5b00cb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:58.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.649+0000 7fb5b7bb4700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b016a310 con 0x7fb5b00c4ab0 2023-10-06T04:00:58.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.649+0000 7fb5b7bb4700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b0165eb0 con 0x7fb5b00c20f0 2023-10-06T04:00:58.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.649+0000 7fb5b7bb4700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b0166e00 con 0x7fb59c004160 2023-10-06T04:00:58.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.649+0000 7fb5b514f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5b00c4ab0 0x7fb5b01668c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:58.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.649+0000 7fb5b514f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5b00c4ab0 0x7fb5b01668c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56866/0 (socket says 172.21.15.157:56866) 2023-10-06T04:00:58.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.649+0000 7fb5b514f700 1 -- 172.21.15.157:0/4021098313 learned_addr learned my addr 172.21.15.157:0/4021098313 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:00:58.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.650+0000 7fb5b5950700 1 --2- 172.21.15.157:0/4021098313 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb59c004160 0x7fb5b0169430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:58.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.650+0000 7fb5b6151700 1 --2- 172.21.15.157:0/4021098313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5b00c20f0 0x7fb5b00cb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:58.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.650+0000 7fb5b514f700 1 -- 172.21.15.157:0/4021098313 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb59c004160 msgr2=0x7fb5b0169430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:58.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.650+0000 7fb5b514f700 1 --2- 172.21.15.157:0/4021098313 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb59c004160 0x7fb5b0169430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:58.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.650+0000 7fb5b514f700 1 -- 172.21.15.157:0/4021098313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5b00c20f0 msgr2=0x7fb5b00cb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:58.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.650+0000 7fb5b514f700 1 --2- 172.21.15.157:0/4021098313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5b00c20f0 0x7fb5b00cb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:58.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.650+0000 7fb5b514f700 1 -- 172.21.15.157:0/4021098313 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b0169a70 con 0x7fb5b00c4ab0 2023-10-06T04:00:58.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.650+0000 7fb5b5950700 1 --2- 172.21.15.157:0/4021098313 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb59c004160 0x7fb5b0169430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:00:58.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.650+0000 7fb5b514f700 1 --2- 172.21.15.157:0/4021098313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5b00c4ab0 0x7fb5b01668c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00bb70 tx=0x7fb5a0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:00:58.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.650+0000 7fb5a5ffb700 1 -- 172.21.15.157:0/4021098313 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a000c5d0 con 0x7fb5b00c4ab0 2023-10-06T04:00:58.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.650+0000 7fb5a5ffb700 1 -- 172.21.15.157:0/4021098313 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5a0001360 con 0x7fb5b00c4ab0 2023-10-06T04:00:58.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.650+0000 7fb5b7bb4700 1 -- 172.21.15.157:0/4021098313 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5b0169d50 con 0x7fb5b00c4ab0 2023-10-06T04:00:58.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.650+0000 7fb5a5ffb700 1 -- 172.21.15.157:0/4021098313 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0006a70 con 0x7fb5b00c4ab0 2023-10-06T04:00:58.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.650+0000 7fb5b7bb4700 1 -- 172.21.15.157:0/4021098313 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5b01660c0 con 0x7fb5b00c4ab0 2023-10-06T04:00:58.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.651+0000 7fb5b7bb4700 1 -- 172.21.15.157:0/4021098313 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5b0089ca0 con 0x7fb5b00c4ab0 2023-10-06T04:00:58.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.652+0000 7fb5a5ffb700 1 -- 172.21.15.157:0/4021098313 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb5a0019070 con 0x7fb5b00c4ab0 2023-10-06T04:00:58.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.652+0000 7fb5a5ffb700 1 --2- 172.21.15.157:0/4021098313 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb594066f30 0x7fb5940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:00:58.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.652+0000 7fb5a5ffb700 1 -- 172.21.15.157:0/4021098313 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6202+0+0 (secure 0 0 0) 0x7fb5a0085fd0 con 0x7fb5b00c4ab0 2023-10-06T04:00:58.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.652+0000 7fb5b5950700 1 --2- 172.21.15.157:0/4021098313 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb594066f30 0x7fb5940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:00:58.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.653+0000 7fb5b5950700 1 --2- 172.21.15.157:0/4021098313 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb594066f30 0x7fb5940693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac001920 tx=0x7fb5ac006040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:00:58.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.656+0000 7fb5a5ffb700 1 -- 172.21.15.157:0/4021098313 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb5a0053050 con 0x7fb5b00c4ab0 2023-10-06T04:00:58.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.881+0000 7fb5b7bb4700 1 -- 172.21.15.157:0/4021098313 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5b00c4790 con 0x7fb5b00c4ab0 2023-10-06T04:00:58.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.882+0000 7fb5a5ffb700 1 -- 172.21.15.157:0/4021098313 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+6203 (secure 0 0 0) 0x7fb5a0022090 con 0x7fb5b00c4ab0 2023-10-06T04:00:58.884 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:00:58.884 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4936,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:00:42.818220+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4223":{"gid":4223,"name":"a","rank":4,"incarnation":17,"state":"up:active","state_seq":39,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:00:58.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.884+0000 7fb5b7bb4700 1 -- 172.21.15.157:0/4021098313 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb594066f30 msgr2=0x7fb5940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:58.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.884+0000 7fb5b7bb4700 1 --2- 172.21.15.157:0/4021098313 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb594066f30 0x7fb5940693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac001920 tx=0x7fb5ac006040 comp rx=0 tx=0).stop 2023-10-06T04:00:58.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.884+0000 7fb5b7bb4700 1 -- 172.21.15.157:0/4021098313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5b00c4ab0 msgr2=0x7fb5b01668c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:00:58.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.884+0000 7fb5b7bb4700 1 --2- 172.21.15.157:0/4021098313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5b00c4ab0 0x7fb5b01668c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00bb70 tx=0x7fb5a0015000 comp rx=0 tx=0).stop 2023-10-06T04:00:58.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.884+0000 7fb5b7bb4700 1 -- 172.21.15.157:0/4021098313 shutdown_connections 2023-10-06T04:00:58.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.884+0000 7fb5b7bb4700 1 --2- 172.21.15.157:0/4021098313 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb594066f30 0x7fb5940693e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:58.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.884+0000 7fb5b7bb4700 1 --2- 172.21.15.157:0/4021098313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5b00c20f0 0x7fb5b00cb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:58.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.884+0000 7fb5b7bb4700 1 --2- 172.21.15.157:0/4021098313 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb59c004160 0x7fb5b0169430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:58.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.884+0000 7fb5b7bb4700 1 --2- 172.21.15.157:0/4021098313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5b00c4ab0 0x7fb5b01668c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:00:58.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.884+0000 7fb5b7bb4700 1 -- 172.21.15.157:0/4021098313 >> 172.21.15.157:0/4021098313 conn(0x7fb5b006f230 msgr2=0x7fb5b014eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:00:58.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.884+0000 7fb5b7bb4700 1 -- 172.21.15.157:0/4021098313 shutdown_connections 2023-10-06T04:00:58.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:00:58.884+0000 7fb5b7bb4700 1 -- 172.21.15.157:0/4021098313 wait complete. 2023-10-06T04:00:58.887 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 28 2023-10-06T04:00:58.898 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-10-06T04:01:00.899 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:01:01.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.023+0000 7f8a7c567700 1 Processor -- start 2023-10-06T04:01:01.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.023+0000 7f8a7c567700 1 -- start start 2023-10-06T04:01:01.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.024+0000 7f8a7c567700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8a740c23c0 0x7f8a74155f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:01.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.024+0000 7f8a7c567700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a7406c1f0 0x7f8a74157470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:01.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.024+0000 7f8a7c567700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8a740c2790 0x7f8a74069c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:01.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.024+0000 7f8a7c567700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a7406d5d0 con 0x7f8a740c2790 2023-10-06T04:01:01.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.024+0000 7f8a7c567700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a7406c6e0 con 0x7f8a740c23c0 2023-10-06T04:01:01.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.024+0000 7f8a7c567700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7406c9c0 con 0x7f8a7406c1f0 2023-10-06T04:01:01.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.024+0000 7f8a7ab04700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8a740c2790 0x7f8a74069c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:01.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.025+0000 7f8a7ab04700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8a740c2790 0x7f8a74069c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56872/0 (socket says 172.21.15.157:56872) 2023-10-06T04:01:01.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.025+0000 7f8a7ab04700 1 -- 172.21.15.157:0/2138884909 learned_addr learned my addr 172.21.15.157:0/2138884909 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:01:01.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.025+0000 7f8a7a303700 1 --2- 172.21.15.157:0/2138884909 >> v2:172.21.15.175:3300/0 conn(0x7f8a740c23c0 0x7f8a74155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:01.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.025+0000 7f8a79b02700 1 --2- 172.21.15.157:0/2138884909 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a7406c1f0 0x7f8a74157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:01.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.025+0000 7f8a7ab04700 1 -- 172.21.15.157:0/2138884909 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a7406c1f0 msgr2=0x7f8a74157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:01:01.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.025+0000 7f8a7ab04700 1 --2- 172.21.15.157:0/2138884909 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a7406c1f0 0x7f8a74157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:01.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.025+0000 7f8a7ab04700 1 -- 172.21.15.157:0/2138884909 >> v2:172.21.15.175:3300/0 conn(0x7f8a740c23c0 msgr2=0x7f8a74155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:01:01.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.025+0000 7f8a7ab04700 1 --2- 172.21.15.157:0/2138884909 >> v2:172.21.15.175:3300/0 conn(0x7f8a740c23c0 0x7f8a74155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:01.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.025+0000 7f8a7ab04700 1 -- 172.21.15.157:0/2138884909 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7406cca0 con 0x7f8a740c2790 2023-10-06T04:01:01.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.025+0000 7f8a7ab04700 1 --2- 172.21.15.157:0/2138884909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a740c2790 0x7f8a74069c20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8a74069540 tx=0x7f8a6c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18709cfc30e2f136 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:01.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.025+0000 7f8a79301700 1 -- 172.21.15.157:0/2138884909 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c00e070 con 0x7f8a740c2790 2023-10-06T04:01:01.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.025+0000 7f8a79301700 1 -- 172.21.15.157:0/2138884909 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6c00ba00 con 0x7f8a740c2790 2023-10-06T04:01:01.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.025+0000 7f8a79301700 1 -- 172.21.15.157:0/2138884909 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c0092e0 con 0x7f8a740c2790 2023-10-06T04:01:01.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.026+0000 7f8a7c567700 1 -- 172.21.15.157:0/2138884909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a740c2790 msgr2=0x7f8a74069c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:01.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.026+0000 7f8a7c567700 1 --2- 172.21.15.157:0/2138884909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a740c2790 0x7f8a74069c20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8a74069540 tx=0x7f8a6c009840 comp rx=0 tx=0).stop 2023-10-06T04:01:01.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.026+0000 7f8a7c567700 1 -- 172.21.15.157:0/2138884909 shutdown_connections 2023-10-06T04:01:01.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.026+0000 7f8a7c567700 1 --2- 172.21.15.157:0/2138884909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a740c2790 0x7f8a74069c20 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:01.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.026+0000 7f8a7c567700 1 --2- 172.21.15.157:0/2138884909 >> v2:172.21.15.175:3300/0 conn(0x7f8a740c23c0 0x7f8a74155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:01.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.026+0000 7f8a7c567700 1 --2- 172.21.15.157:0/2138884909 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a7406c1f0 0x7f8a74157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:01.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.026+0000 7f8a7c567700 1 -- 172.21.15.157:0/2138884909 >> 172.21.15.157:0/2138884909 conn(0x7f8a7406f290 msgr2=0x7f8a7406f690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:01.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.026+0000 7f8a7c567700 1 -- 172.21.15.157:0/2138884909 shutdown_connections 2023-10-06T04:01:01.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.026+0000 7f8a7c567700 1 -- 172.21.15.157:0/2138884909 wait complete. 2023-10-06T04:01:01.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.027+0000 7f8a7c567700 1 Processor -- start 2023-10-06T04:01:01.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.027+0000 7f8a7c567700 1 -- start start 2023-10-06T04:01:01.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.028+0000 7f8a7c567700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7406c1f0 0x7f8a740bad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:01.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.028+0000 7f8a7c567700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a740c23c0 0x7f8a74155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:01.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.028+0000 7f8a7c567700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a740c2790 0x7f8a74152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:01.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.028+0000 7f8a7c567700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a740c1380 con 0x7f8a7406c1f0 2023-10-06T04:01:01.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.028+0000 7f8a7c567700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a740ba2b0 con 0x7f8a740c2790 2023-10-06T04:01:01.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.028+0000 7f8a7c567700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a740ba870 con 0x7f8a740c23c0 2023-10-06T04:01:01.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.028+0000 7f8a7a303700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7406c1f0 0x7f8a740bad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:01.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.028+0000 7f8a7a303700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7406c1f0 0x7f8a740bad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56882/0 (socket says 172.21.15.157:56882) 2023-10-06T04:01:01.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.028+0000 7f8a7a303700 1 -- 172.21.15.157:0/2774180972 learned_addr learned my addr 172.21.15.157:0/2774180972 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:01:01.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.028+0000 7f8a7ab04700 1 --2- 172.21.15.157:0/2774180972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a740c2790 0x7f8a74152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:01.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.028+0000 7f8a79b02700 1 --2- 172.21.15.157:0/2774180972 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a740c23c0 0x7f8a74155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:01.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.028+0000 7f8a7a303700 1 -- 172.21.15.157:0/2774180972 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a740c23c0 msgr2=0x7f8a74155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:01.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.028+0000 7f8a7a303700 1 --2- 172.21.15.157:0/2774180972 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a740c23c0 0x7f8a74155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:01.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.028+0000 7f8a7a303700 1 -- 172.21.15.157:0/2774180972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a740c2790 msgr2=0x7f8a74152290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:01.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.028+0000 7f8a7a303700 1 --2- 172.21.15.157:0/2774180972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a740c2790 0x7f8a74152290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:01.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.028+0000 7f8a7a303700 1 -- 172.21.15.157:0/2774180972 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6c009ae0 con 0x7f8a7406c1f0 2023-10-06T04:01:01.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.028+0000 7f8a7ab04700 1 --2- 172.21.15.157:0/2774180972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a740c2790 0x7f8a74152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:01:01.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.028+0000 7f8a79b02700 1 --2- 172.21.15.157:0/2774180972 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a740c23c0 0x7f8a74155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:01:01.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.029+0000 7f8a7a303700 1 --2- 172.21.15.157:0/2774180972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7406c1f0 0x7f8a740bad00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8a7000ecb0 tx=0x7f8a700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:01.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.029+0000 7f8a6a7fc700 1 -- 172.21.15.157:0/2774180972 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7000c9d0 con 0x7f8a7406c1f0 2023-10-06T04:01:01.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.029+0000 7f8a6a7fc700 1 -- 172.21.15.157:0/2774180972 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7000cb30 con 0x7f8a7406c1f0 2023-10-06T04:01:01.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.029+0000 7f8a7c567700 1 -- 172.21.15.157:0/2774180972 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a740c1940 con 0x7f8a7406c1f0 2023-10-06T04:01:01.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.029+0000 7f8a6a7fc700 1 -- 172.21.15.157:0/2774180972 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7000ced0 con 0x7f8a7406c1f0 2023-10-06T04:01:01.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.029+0000 7f8a7c567700 1 -- 172.21.15.157:0/2774180972 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a741667d0 con 0x7f8a7406c1f0 2023-10-06T04:01:01.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.030+0000 7f8a7c567700 1 -- 172.21.15.157:0/2774180972 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a74068c80 con 0x7f8a7406c1f0 2023-10-06T04:01:01.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.030+0000 7f8a6a7fc700 1 -- 172.21.15.157:0/2774180972 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8a70013680 con 0x7f8a7406c1f0 2023-10-06T04:01:01.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.030+0000 7f8a6a7fc700 1 --2- 172.21.15.157:0/2774180972 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8a58066f30 0x7f8a580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:01.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.030+0000 7f8a6a7fc700 1 -- 172.21.15.157:0/2774180972 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f8a70084ae0 con 0x7f8a7406c1f0 2023-10-06T04:01:01.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.031+0000 7f8a79b02700 1 --2- 172.21.15.157:0/2774180972 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8a58066f30 0x7f8a580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:01.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.031+0000 7f8a79b02700 1 --2- 172.21.15.157:0/2774180972 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8a58066f30 0x7f8a580693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8a64006f00 tx=0x7f8a64008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:01.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.035+0000 7f8a6a7fc700 1 -- 172.21.15.157:0/2774180972 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8a70051b60 con 0x7f8a7406c1f0 2023-10-06T04:01:01.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.259+0000 7f8a7c567700 1 -- 172.21.15.157:0/2774180972 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a7406cf80 con 0x7f8a7406c1f0 2023-10-06T04:01:01.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.260+0000 7f8a6a7fc700 1 -- 172.21.15.157:0/2774180972 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+6203 (secure 0 0 0) 0x7f8a70055180 con 0x7f8a7406c1f0 2023-10-06T04:01:01.263 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:01:01.263 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4936,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:00:42.818220+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4223},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4223":{"gid":4223,"name":"a","rank":4,"incarnation":17,"state":"up:active","state_seq":39,"addr":"172.21.15.157:6835/3932336008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":3932336008},{"type":"v1","addr":"172.21.15.157:6835","nonce":3932336008}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:01:01.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.262+0000 7f8a7c567700 1 -- 172.21.15.157:0/2774180972 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8a58066f30 msgr2=0x7f8a580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:01.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.262+0000 7f8a7c567700 1 --2- 172.21.15.157:0/2774180972 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8a58066f30 0x7f8a580693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8a64006f00 tx=0x7f8a64008040 comp rx=0 tx=0).stop 2023-10-06T04:01:01.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.262+0000 7f8a7c567700 1 -- 172.21.15.157:0/2774180972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7406c1f0 msgr2=0x7f8a740bad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:01.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.262+0000 7f8a7c567700 1 --2- 172.21.15.157:0/2774180972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7406c1f0 0x7f8a740bad00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8a7000ecb0 tx=0x7f8a700091a0 comp rx=0 tx=0).stop 2023-10-06T04:01:01.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.262+0000 7f8a7c567700 1 -- 172.21.15.157:0/2774180972 shutdown_connections 2023-10-06T04:01:01.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.262+0000 7f8a7c567700 1 --2- 172.21.15.157:0/2774180972 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8a58066f30 0x7f8a580693e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:01.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.262+0000 7f8a7c567700 1 --2- 172.21.15.157:0/2774180972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a740c2790 0x7f8a74152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:01.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.262+0000 7f8a7c567700 1 --2- 172.21.15.157:0/2774180972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a7406c1f0 0x7f8a740bad00 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:01.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.262+0000 7f8a7c567700 1 --2- 172.21.15.157:0/2774180972 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a740c23c0 0x7f8a74155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:01.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.262+0000 7f8a7c567700 1 -- 172.21.15.157:0/2774180972 >> 172.21.15.157:0/2774180972 conn(0x7f8a7406f290 msgr2=0x7f8a740c6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:01.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.262+0000 7f8a7c567700 1 -- 172.21.15.157:0/2774180972 shutdown_connections 2023-10-06T04:01:01.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:01.262+0000 7f8a7c567700 1 -- 172.21.15.157:0/2774180972 wait complete. 2023-10-06T04:01:01.266 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 28 2023-10-06T04:01:01.276 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-10-06T04:01:03.277 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:01:03.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.407+0000 7f079af47700 1 Processor -- start 2023-10-06T04:01:03.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.407+0000 7f079af47700 1 -- start start 2023-10-06T04:01:03.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.408+0000 7f079af47700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f079406f220 0x7f0794152950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:03.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.408+0000 7f079af47700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f079406f5f0 0x7f079414f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:03.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.408+0000 7f079af47700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079406fc70 con 0x7f0794068f10 2023-10-06T04:01:03.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.408+0000 7f079af47700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0794152e90 con 0x7f079406f220 2023-10-06T04:01:03.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.408+0000 7f079af47700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07940bd880 con 0x7f079406f5f0 2023-10-06T04:01:03.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.408+0000 7f07994e4700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f079406f5f0 0x7f079414f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:03.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.408+0000 7f0798ce3700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f079406f220 0x7f0794152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:03.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.408+0000 7f0798ce3700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f079406f220 0x7f0794152950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:44666/0 (socket says 172.21.15.157:44666) 2023-10-06T04:01:03.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.408+0000 7f07994e4700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f079406f5f0 0x7f079414f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:35260/0 (socket says 172.21.15.157:35260) 2023-10-06T04:01:03.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.408+0000 7f0798ce3700 1 -- 172.21.15.157:0/3556106950 learned_addr learned my addr 172.21.15.157:0/3556106950 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:01:03.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.408+0000 7f0793fff700 1 --1- 172.21.15.157:0/3556106950 >> v1:172.21.15.175:6789/0 conn(0x7f0794068f10 0x7f07940c35f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.157:37402/0 (socket says 172.21.15.157:37402) 2023-10-06T04:01:03.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.409+0000 7f0798ce3700 1 -- 172.21.15.157:0/3556106950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f079406f5f0 msgr2=0x7f079414f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:01:03.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.409+0000 7f0798ce3700 1 --2- 172.21.15.157:0/3556106950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f079406f5f0 0x7f079414f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:03.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.409+0000 7f0798ce3700 1 -- 172.21.15.157:0/3556106950 >> v1:172.21.15.175:6789/0 conn(0x7f0794068f10 legacy=0x7f07940c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:03.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.409+0000 7f0798ce3700 1 -- 172.21.15.157:0/3556106950 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07940c3160 con 0x7f079406f220 2023-10-06T04:01:03.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.409+0000 7f07937fe700 1 -- 172.21.15.157:0/3556106950 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 816163853 0 0) 0x7f079406fc70 con 0x7f0794068f10 2023-10-06T04:01:03.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.409+0000 7f07994e4700 1 --2- 172.21.15.157:0/3556106950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f079406f5f0 0x7f079414f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:01:03.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.409+0000 7f0798ce3700 1 --2- 172.21.15.157:0/3556106950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f079406f220 0x7f0794152950 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f078c00d1a0 tx=0x7f078c00ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8c08c026f508aec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:03.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.409+0000 7f07937fe700 1 -- 172.21.15.157:0/3556106950 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c010070 con 0x7f079406f220 2023-10-06T04:01:03.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.409+0000 7f07937fe700 1 -- 172.21.15.157:0/3556106950 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078c00b0d0 con 0x7f079406f220 2023-10-06T04:01:03.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.409+0000 7f07937fe700 1 -- 172.21.15.157:0/3556106950 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c00dea0 con 0x7f079406f220 2023-10-06T04:01:03.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.409+0000 7f079af47700 1 -- 172.21.15.157:0/3556106950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f079406f220 msgr2=0x7f0794152950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:03.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.409+0000 7f079af47700 1 --2- 172.21.15.157:0/3556106950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f079406f220 0x7f0794152950 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f078c00d1a0 tx=0x7f078c00ba50 comp rx=0 tx=0).stop 2023-10-06T04:01:03.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.409+0000 7f079af47700 1 -- 172.21.15.157:0/3556106950 shutdown_connections 2023-10-06T04:01:03.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.409+0000 7f079af47700 1 --2- 172.21.15.157:0/3556106950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f079406f5f0 0x7f079414f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:03.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.409+0000 7f079af47700 1 --2- 172.21.15.157:0/3556106950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f079406f220 0x7f0794152950 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:03.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.409+0000 7f079af47700 1 -- 172.21.15.157:0/3556106950 >> 172.21.15.157:0/3556106950 conn(0x7f07940c8d70 msgr2=0x7f07940c4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:03.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.409+0000 7f079af47700 1 -- 172.21.15.157:0/3556106950 shutdown_connections 2023-10-06T04:01:03.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.410+0000 7f079af47700 1 -- 172.21.15.157:0/3556106950 wait complete. 2023-10-06T04:01:03.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.411+0000 7f079af47700 1 Processor -- start 2023-10-06T04:01:03.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.411+0000 7f079af47700 1 -- start start 2023-10-06T04:01:03.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.411+0000 7f079af47700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0794068f10 0x7f07941575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:03.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.411+0000 7f079af47700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f079406f220 0x7f0794150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:03.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.411+0000 7f079af47700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f079406f5f0 0x7f0794151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:03.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.411+0000 7f079af47700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07940acbd0 con 0x7f079406f5f0 2023-10-06T04:01:03.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.411+0000 7f079af47700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07940ad710 con 0x7f0794068f10 2023-10-06T04:01:03.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.411+0000 7f079af47700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0794157ae0 con 0x7f079406f220 2023-10-06T04:01:03.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.412+0000 7f07994e4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f079406f5f0 0x7f0794151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:03.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.412+0000 7f07994e4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f079406f5f0 0x7f0794151040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56378/0 (socket says 172.21.15.157:56378) 2023-10-06T04:01:03.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.412+0000 7f07994e4700 1 -- 172.21.15.157:0/3109408043 learned_addr learned my addr 172.21.15.157:0/3109408043 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:01:03.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.412+0000 7f0793fff700 1 --2- 172.21.15.157:0/3109408043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f079406f220 0x7f0794150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:03.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.412+0000 7f0798ce3700 1 --2- 172.21.15.157:0/3109408043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0794068f10 0x7f07941575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:03.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.412+0000 7f07994e4700 1 -- 172.21.15.157:0/3109408043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f079406f220 msgr2=0x7f0794150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:03.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.412+0000 7f07994e4700 1 --2- 172.21.15.157:0/3109408043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f079406f220 0x7f0794150b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:03.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.412+0000 7f07994e4700 1 -- 172.21.15.157:0/3109408043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0794068f10 msgr2=0x7f07941575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:03.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.412+0000 7f07994e4700 1 --2- 172.21.15.157:0/3109408043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0794068f10 0x7f07941575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:03.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.412+0000 7f07994e4700 1 -- 172.21.15.157:0/3109408043 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c009420 con 0x7f079406f5f0 2023-10-06T04:01:03.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.412+0000 7f07994e4700 1 --2- 172.21.15.157:0/3109408043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f079406f5f0 0x7f0794151040 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f077c00ecb0 tx=0x7f077c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:03.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.412+0000 7f0790ff9700 1 -- 172.21.15.157:0/3109408043 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c00c850 con 0x7f079406f5f0 2023-10-06T04:01:03.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.412+0000 7f0790ff9700 1 -- 172.21.15.157:0/3109408043 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077c00c9b0 con 0x7f079406f5f0 2023-10-06T04:01:03.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.412+0000 7f0790ff9700 1 -- 172.21.15.157:0/3109408043 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c00cd70 con 0x7f079406f5f0 2023-10-06T04:01:03.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.412+0000 7f079af47700 1 -- 172.21.15.157:0/3109408043 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078c00bcf0 con 0x7f079406f5f0 2023-10-06T04:01:03.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.412+0000 7f079af47700 1 -- 172.21.15.157:0/3109408043 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07940acde0 con 0x7f079406f5f0 2023-10-06T04:01:03.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.413+0000 7f079af47700 1 -- 172.21.15.157:0/3109408043 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07940716b0 con 0x7f079406f5f0 2023-10-06T04:01:03.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.414+0000 7f0790ff9700 1 -- 172.21.15.157:0/3109408043 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f077c013680 con 0x7f079406f5f0 2023-10-06T04:01:03.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.414+0000 7f0790ff9700 1 --2- 172.21.15.157:0/3109408043 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0774066f30 0x7f07740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:03.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.414+0000 7f0790ff9700 1 -- 172.21.15.157:0/3109408043 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f077c084750 con 0x7f079406f5f0 2023-10-06T04:01:03.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.414+0000 7f0798ce3700 1 --2- 172.21.15.157:0/3109408043 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0774066f30 0x7f07740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:03.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.415+0000 7f0798ce3700 1 --2- 172.21.15.157:0/3109408043 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0774066f30 0x7f07740693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f078c00ba90 tx=0x7f078c01a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:03.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.418+0000 7f0790ff9700 1 -- 172.21.15.157:0/3109408043 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f077c051780 con 0x7f079406f5f0 2023-10-06T04:01:03.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.647+0000 7f079af47700 1 -- 172.21.15.157:0/3109408043 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07940c9170 con 0x7f079406f5f0 2023-10-06T04:01:03.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.647+0000 7f0790ff9700 1 -- 172.21.15.157:0/3109408043 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+5438 (secure 0 0 0) 0x7f077c054da0 con 0x7f079406f5f0 2023-10-06T04:01:03.650 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:01:03.650 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:01:01.662635+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"f","rank":4,"incarnation":29,"state":"up:replay","state_seq":1,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:01:03.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.649+0000 7f079af47700 1 -- 172.21.15.157:0/3109408043 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0774066f30 msgr2=0x7f07740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:03.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.649+0000 7f079af47700 1 --2- 172.21.15.157:0/3109408043 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0774066f30 0x7f07740693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f078c00ba90 tx=0x7f078c01a000 comp rx=0 tx=0).stop 2023-10-06T04:01:03.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.649+0000 7f079af47700 1 -- 172.21.15.157:0/3109408043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f079406f5f0 msgr2=0x7f0794151040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:03.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.649+0000 7f079af47700 1 --2- 172.21.15.157:0/3109408043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f079406f5f0 0x7f0794151040 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f077c00ecb0 tx=0x7f077c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:01:03.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.649+0000 7f079af47700 1 -- 172.21.15.157:0/3109408043 shutdown_connections 2023-10-06T04:01:03.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.649+0000 7f079af47700 1 --2- 172.21.15.157:0/3109408043 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0774066f30 0x7f07740693e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:03.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.649+0000 7f079af47700 1 --2- 172.21.15.157:0/3109408043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0794068f10 0x7f07941575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:03.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.649+0000 7f079af47700 1 --2- 172.21.15.157:0/3109408043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f079406f5f0 0x7f0794151040 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:03.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.649+0000 7f079af47700 1 --2- 172.21.15.157:0/3109408043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f079406f220 0x7f0794150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:03.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.649+0000 7f079af47700 1 -- 172.21.15.157:0/3109408043 >> 172.21.15.157:0/3109408043 conn(0x7f07940c8d70 msgr2=0x7f079406a290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:03.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.649+0000 7f079af47700 1 -- 172.21.15.157:0/3109408043 shutdown_connections 2023-10-06T04:01:03.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.649+0000 7f079af47700 1 -- 172.21.15.157:0/3109408043 wait complete. 2023-10-06T04:01:03.654 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 29 2023-10-06T04:01:03.664 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2023-10-06T04:01:03.665 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-10-06T04:01:03.665 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:01:03.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.777+0000 7fe48260c700 1 Processor -- start 2023-10-06T04:01:03.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.777+0000 7fe48260c700 1 -- start start 2023-10-06T04:01:03.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.778+0000 7fe48260c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe47c06dc40 0x7fe47c14fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:03.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.778+0000 7fe48260c700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe47c0c9780 con 0x7fe47c06c1f0 2023-10-06T04:01:03.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.778+0000 7fe48260c700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe47c06e0c0 con 0x7fe47c06cbb0 2023-10-06T04:01:03.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.778+0000 7fe48260c700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe47c0c8c40 con 0x7fe47c06dc40 2023-10-06T04:01:03.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.778+0000 7fe47b7fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe47c06dc40 0x7fe47c14fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:03.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.778+0000 7fe480ba9700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe47c06c1f0 0x7fe47c155f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51194/0 (socket says 172.21.15.157:51194) 2023-10-06T04:01:03.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.778+0000 7fe480ba9700 1 -- 172.21.15.157:0/655331734 learned_addr learned my addr 172.21.15.157:0/655331734 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:01:03.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.779+0000 7fe47b7fe700 1 -- 172.21.15.157:0/655331734 >> v1:172.21.15.175:6789/0 conn(0x7fe47c06cbb0 legacy=0x7fe47c06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:03.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.779+0000 7fe47b7fe700 1 -- 172.21.15.157:0/655331734 >> v1:172.21.15.157:6789/0 conn(0x7fe47c06c1f0 legacy=0x7fe47c155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:03.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.779+0000 7fe47affd700 1 -- 172.21.15.157:0/655331734 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3788213734 0 0) 0x7fe47c0c9780 con 0x7fe47c06c1f0 2023-10-06T04:01:03.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.779+0000 7fe47b7fe700 1 -- 172.21.15.157:0/655331734 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe47c0c8e50 con 0x7fe47c06dc40 2023-10-06T04:01:03.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.779+0000 7fe47b7fe700 1 --2- 172.21.15.157:0/655331734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe47c06dc40 0x7fe47c14fd00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe4700098e0 tx=0x7fe4700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b589b0a2ef333f85 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:03.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.779+0000 7fe47affd700 1 -- 172.21.15.157:0/655331734 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe470009e80 con 0x7fe47c06dc40 2023-10-06T04:01:03.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.779+0000 7fe47affd700 1 -- 172.21.15.157:0/655331734 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47000b8f0 con 0x7fe47c06dc40 2023-10-06T04:01:03.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.779+0000 7fe47affd700 1 -- 172.21.15.157:0/655331734 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47000bc70 con 0x7fe47c06dc40 2023-10-06T04:01:03.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.779+0000 7fe48260c700 1 -- 172.21.15.157:0/655331734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe47c06dc40 msgr2=0x7fe47c14fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:03.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.779+0000 7fe48260c700 1 --2- 172.21.15.157:0/655331734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe47c06dc40 0x7fe47c14fd00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe4700098e0 tx=0x7fe4700091a0 comp rx=0 tx=0).stop 2023-10-06T04:01:03.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.779+0000 7fe48260c700 1 -- 172.21.15.157:0/655331734 shutdown_connections 2023-10-06T04:01:03.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.779+0000 7fe48260c700 1 --2- 172.21.15.157:0/655331734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe47c06dc40 0x7fe47c14fd00 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:03.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.779+0000 7fe48260c700 1 -- 172.21.15.157:0/655331734 >> 172.21.15.157:0/655331734 conn(0x7fe47c06f230 msgr2=0x7fe47c06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:03.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.779+0000 7fe48260c700 1 -- 172.21.15.157:0/655331734 shutdown_connections 2023-10-06T04:01:03.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.779+0000 7fe48260c700 1 -- 172.21.15.157:0/655331734 wait complete. 2023-10-06T04:01:03.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.780+0000 7fe48260c700 1 Processor -- start 2023-10-06T04:01:03.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.781+0000 7fe48260c700 1 -- start start 2023-10-06T04:01:03.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.781+0000 7fe48260c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe47c06c1f0 0x7fe47c0c7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:03.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.781+0000 7fe48260c700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe47c06cbb0 0x7fe47c0c7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:03.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.781+0000 7fe48260c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe47c0c6750 0x7fe47c14af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:03.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.781+0000 7fe48260c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47c150600 con 0x7fe47c0c6750 2023-10-06T04:01:03.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.781+0000 7fe48260c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47c1508e0 con 0x7fe47c06c1f0 2023-10-06T04:01:03.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.781+0000 7fe48260c700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe47c0c0eb0 con 0x7fe47c06cbb0 2023-10-06T04:01:03.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.781+0000 7fe480ba9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe47c0c6750 0x7fe47c14af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:03.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.781+0000 7fe480ba9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe47c0c6750 0x7fe47c14af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56406/0 (socket says 172.21.15.157:56406) 2023-10-06T04:01:03.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.781+0000 7fe480ba9700 1 -- 172.21.15.157:0/1507622142 learned_addr learned my addr 172.21.15.157:0/1507622142 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:01:03.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.782+0000 7fe47b7fe700 1 --2- 172.21.15.157:0/1507622142 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe47c06cbb0 0x7fe47c0c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:03.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.782+0000 7fe47bfff700 1 --2- 172.21.15.157:0/1507622142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe47c06c1f0 0x7fe47c0c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:03.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.782+0000 7fe480ba9700 1 -- 172.21.15.157:0/1507622142 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe47c06cbb0 msgr2=0x7fe47c0c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:03.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.782+0000 7fe480ba9700 1 --2- 172.21.15.157:0/1507622142 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe47c06cbb0 0x7fe47c0c7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:03.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.782+0000 7fe480ba9700 1 -- 172.21.15.157:0/1507622142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe47c06c1f0 msgr2=0x7fe47c0c7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:03.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.782+0000 7fe480ba9700 1 --2- 172.21.15.157:0/1507622142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe47c06c1f0 0x7fe47c0c7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:03.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.782+0000 7fe480ba9700 1 -- 172.21.15.157:0/1507622142 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe470009420 con 0x7fe47c0c6750 2023-10-06T04:01:03.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.782+0000 7fe480ba9700 1 --2- 172.21.15.157:0/1507622142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe47c0c6750 0x7fe47c14af70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe47400ccc0 tx=0x7fe474001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:03.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.782+0000 7fe463fff700 1 -- 172.21.15.157:0/1507622142 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47400ab30 con 0x7fe47c0c6750 2023-10-06T04:01:03.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.782+0000 7fe463fff700 1 -- 172.21.15.157:0/1507622142 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47400ac90 con 0x7fe47c0c6750 2023-10-06T04:01:03.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.782+0000 7fe463fff700 1 -- 172.21.15.157:0/1507622142 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe474007340 con 0x7fe47c0c6750 2023-10-06T04:01:03.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.782+0000 7fe48260c700 1 -- 172.21.15.157:0/1507622142 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe47c14ff50 con 0x7fe47c0c6750 2023-10-06T04:01:03.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.782+0000 7fe48260c700 1 -- 172.21.15.157:0/1507622142 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe47c152ae0 con 0x7fe47c0c6750 2023-10-06T04:01:03.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.783+0000 7fe48260c700 1 -- 172.21.15.157:0/1507622142 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe47c089ca0 con 0x7fe47c0c6750 2023-10-06T04:01:03.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.783+0000 7fe463fff700 1 -- 172.21.15.157:0/1507622142 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe474007730 con 0x7fe47c0c6750 2023-10-06T04:01:03.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.784+0000 7fe463fff700 1 --2- 172.21.15.157:0/1507622142 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe468066f30 0x7fe4680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:03.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.784+0000 7fe463fff700 1 -- 172.21.15.157:0/1507622142 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7fe474084bd0 con 0x7fe47c0c6750 2023-10-06T04:01:03.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.784+0000 7fe47bfff700 1 --2- 172.21.15.157:0/1507622142 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe468066f30 0x7fe4680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:03.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.784+0000 7fe47bfff700 1 --2- 172.21.15.157:0/1507622142 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe468066f30 0x7fe4680693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe46c003860 tx=0x7fe46c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:03.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:03.788+0000 7fe463fff700 1 -- 172.21.15.157:0/1507622142 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe474051c80 con 0x7fe47c0c6750 2023-10-06T04:01:04.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:04.012+0000 7fe48260c700 1 -- 172.21.15.157:0/1507622142 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe47c0c9130 con 0x7fe47c0c6750 2023-10-06T04:01:04.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:04.013+0000 7fe463fff700 1 -- 172.21.15.157:0/1507622142 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+5438 (secure 0 0 0) 0x7fe4740552a0 con 0x7fe47c0c6750 2023-10-06T04:01:04.016 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:01:04.016 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:01:01.662635+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"f","rank":4,"incarnation":29,"state":"up:replay","state_seq":1,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:01:04.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:04.015+0000 7fe48260c700 1 -- 172.21.15.157:0/1507622142 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe468066f30 msgr2=0x7fe4680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:04.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:04.015+0000 7fe48260c700 1 --2- 172.21.15.157:0/1507622142 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe468066f30 0x7fe4680693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe46c003860 tx=0x7fe46c009040 comp rx=0 tx=0).stop 2023-10-06T04:01:04.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:04.015+0000 7fe48260c700 1 -- 172.21.15.157:0/1507622142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe47c0c6750 msgr2=0x7fe47c14af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:04.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:04.015+0000 7fe48260c700 1 --2- 172.21.15.157:0/1507622142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe47c0c6750 0x7fe47c14af70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe47400ccc0 tx=0x7fe474001a80 comp rx=0 tx=0).stop 2023-10-06T04:01:04.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:04.015+0000 7fe48260c700 1 -- 172.21.15.157:0/1507622142 shutdown_connections 2023-10-06T04:01:04.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:04.015+0000 7fe48260c700 1 --2- 172.21.15.157:0/1507622142 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe468066f30 0x7fe4680693e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:04.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:04.015+0000 7fe48260c700 1 --2- 172.21.15.157:0/1507622142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe47c06c1f0 0x7fe47c0c7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:04.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:04.015+0000 7fe48260c700 1 --2- 172.21.15.157:0/1507622142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe47c0c6750 0x7fe47c14af70 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:04.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:04.015+0000 7fe48260c700 1 --2- 172.21.15.157:0/1507622142 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe47c06cbb0 0x7fe47c0c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:04.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:04.015+0000 7fe48260c700 1 -- 172.21.15.157:0/1507622142 >> 172.21.15.157:0/1507622142 conn(0x7fe47c06f230 msgr2=0x7fe47c0af240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:04.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:04.015+0000 7fe48260c700 1 -- 172.21.15.157:0/1507622142 shutdown_connections 2023-10-06T04:01:04.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:04.015+0000 7fe48260c700 1 -- 172.21.15.157:0/1507622142 wait complete. 2023-10-06T04:01:04.019 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 29 2023-10-06T04:01:04.031 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 29, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-06T03:57:10.718782+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 29, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0, 1, 2, 3, 4 ], "info": { "gid_4270": { "addr": "172.21.15.175:6835/1596042861", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6834", "nonce": 1596042861, "type": "v2" }, { "addr": "172.21.15.175:6835", "nonce": 1596042861, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4270, "incarnation": 13, "join_fscid": -1, "name": "b", "rank": 3, "state": "up:active", "state_seq": 8 }, "gid_4279": { "addr": "172.21.15.175:6837/1002404870", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6836", "nonce": 1002404870, "type": "v2" }, { "addr": "172.21.15.175:6837", "nonce": 1002404870, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1, 3, 4 ], "features": 4540138320759226367, "flags": 0, "gid": 4279, "incarnation": 11, "join_fscid": -1, "name": "d", "rank": 2, "state": "up:active", "state_seq": 7 }, "gid_4287": { "addr": "172.21.15.157:6834/1179753185", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6832", "nonce": 1179753185, "type": "v2" }, { "addr": "172.21.15.157:6834", "nonce": 1179753185, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4287, "incarnation": 9, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4296": { "addr": "172.21.15.157:6837/4188652695", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6836", "nonce": 4188652695, "type": "v2" }, { "addr": "172.21.15.157:6837", "nonce": 4188652695, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2, 4 ], "features": 4540138320759226367, "flags": 0, "gid": 4296, "incarnation": 4, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 4 }, "gid_4936": { "addr": "172.21.15.175:6839/3353503675", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6838", "nonce": 3353503675, "type": "v2" }, { "addr": "172.21.15.175:6839", "nonce": 3353503675, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4936, "incarnation": 29, "join_fscid": -1, "name": "f", "rank": 4, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 27, "max_file_size": 1099511627776, "max_mds": 5, "metadata_pool": 3, "modified": "2023-10-06T04:01:01.662635+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4296, "mds_1": 4287, "mds_2": 4279, "mds_3": 4270, "mds_4": 4936 } } } ], "standbys": [] } 2023-10-06T04:01:06.032 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:01:06.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.158+0000 7fb974dcb700 1 Processor -- start 2023-10-06T04:01:06.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.158+0000 7fb974dcb700 1 -- start start 2023-10-06T04:01:06.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.159+0000 7fb974dcb700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb97006ab50 0x7fb97006d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:06.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.159+0000 7fb974dcb700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9700c6160 con 0x7fb97006a780 2023-10-06T04:01:06.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.159+0000 7fb974dcb700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb970069040 con 0x7fb97006ddf0 2023-10-06T04:01:06.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.159+0000 7fb974dcb700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9700c5c00 con 0x7fb97006ab50 2023-10-06T04:01:06.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.159+0000 7fb96dd9b700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb97006ab50 0x7fb97006d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:06.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.159+0000 7fb96dd9b700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb97006ab50 0x7fb97006d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56412/0 (socket says 172.21.15.157:56412) 2023-10-06T04:01:06.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.159+0000 7fb96e59c700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb97006a780 0x7fb9700c65f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51202/0 (socket says 172.21.15.157:51202) 2023-10-06T04:01:06.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.159+0000 7fb96dd9b700 1 -- 172.21.15.157:0/1901675873 learned_addr learned my addr 172.21.15.157:0/1901675873 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:01:06.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.159+0000 7fb96dd9b700 1 -- 172.21.15.157:0/1901675873 >> v1:172.21.15.175:6789/0 conn(0x7fb97006ddf0 legacy=0x7fb97006e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:01:06.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.159+0000 7fb96dd9b700 1 -- 172.21.15.157:0/1901675873 >> v1:172.21.15.157:6789/0 conn(0x7fb97006a780 legacy=0x7fb9700c65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:06.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.159+0000 7fb96dd9b700 1 -- 172.21.15.157:0/1901675873 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb970153930 con 0x7fb97006ab50 2023-10-06T04:01:06.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.159+0000 7fb96d59a700 1 -- 172.21.15.157:0/1901675873 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3297768662 0 0) 0x7fb9700c6160 con 0x7fb97006a780 2023-10-06T04:01:06.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.159+0000 7fb96dd9b700 1 --2- 172.21.15.157:0/1901675873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb97006ab50 0x7fb97006d8b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb96400bc50 tx=0x7fb96400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7cb5f7872a7b02c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:06.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.160+0000 7fb96d59a700 1 -- 172.21.15.157:0/1901675873 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96400be60 con 0x7fb97006ab50 2023-10-06T04:01:06.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.160+0000 7fb96d59a700 1 -- 172.21.15.157:0/1901675873 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb96400de50 con 0x7fb97006ab50 2023-10-06T04:01:06.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.160+0000 7fb96d59a700 1 -- 172.21.15.157:0/1901675873 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9640106a0 con 0x7fb97006ab50 2023-10-06T04:01:06.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.160+0000 7fb974dcb700 1 -- 172.21.15.157:0/1901675873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb97006ab50 msgr2=0x7fb97006d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:06.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.160+0000 7fb974dcb700 1 --2- 172.21.15.157:0/1901675873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb97006ab50 0x7fb97006d8b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb96400bc50 tx=0x7fb96400b4f0 comp rx=0 tx=0).stop 2023-10-06T04:01:06.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.160+0000 7fb974dcb700 1 -- 172.21.15.157:0/1901675873 shutdown_connections 2023-10-06T04:01:06.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.160+0000 7fb974dcb700 1 --2- 172.21.15.157:0/1901675873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb97006ab50 0x7fb97006d8b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:06.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.160+0000 7fb974dcb700 1 -- 172.21.15.157:0/1901675873 >> 172.21.15.157:0/1901675873 conn(0x7fb97006b510 msgr2=0x7fb97006b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:06.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.160+0000 7fb974dcb700 1 -- 172.21.15.157:0/1901675873 shutdown_connections 2023-10-06T04:01:06.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.160+0000 7fb974dcb700 1 -- 172.21.15.157:0/1901675873 wait complete. 2023-10-06T04:01:06.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.161+0000 7fb974dcb700 1 Processor -- start 2023-10-06T04:01:06.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.161+0000 7fb974dcb700 1 -- start start 2023-10-06T04:01:06.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.162+0000 7fb974dcb700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb97006a780 0x7fb970153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:06.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.162+0000 7fb974dcb700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb97006ab50 0x7fb970161d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:06.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.162+0000 7fb974dcb700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb97006ddf0 0x7fb970151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:06.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.162+0000 7fb974dcb700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb970154420 con 0x7fb97006ab50 2023-10-06T04:01:06.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.162+0000 7fb974dcb700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9700bf7c0 con 0x7fb97006a780 2023-10-06T04:01:06.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.162+0000 7fb974dcb700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9700bfaa0 con 0x7fb97006ddf0 2023-10-06T04:01:06.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.162+0000 7fb96dd9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb97006ab50 0x7fb970161d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:06.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.162+0000 7fb96dd9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb97006ab50 0x7fb970161d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56416/0 (socket says 172.21.15.157:56416) 2023-10-06T04:01:06.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.162+0000 7fb96dd9b700 1 -- 172.21.15.157:0/2470060561 learned_addr learned my addr 172.21.15.157:0/2470060561 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:01:06.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.162+0000 7fb96e59c700 1 --2- 172.21.15.157:0/2470060561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb97006a780 0x7fb970153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:06.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.162+0000 7fb96dd9b700 1 -- 172.21.15.157:0/2470060561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb97006ddf0 msgr2=0x7fb970151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:06.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.162+0000 7fb96ed9d700 1 --2- 172.21.15.157:0/2470060561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb97006ddf0 0x7fb970151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:06.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.162+0000 7fb96dd9b700 1 --2- 172.21.15.157:0/2470060561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb97006ddf0 0x7fb970151b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:06.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.162+0000 7fb96dd9b700 1 -- 172.21.15.157:0/2470060561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb97006a780 msgr2=0x7fb970153ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:06.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.162+0000 7fb96dd9b700 1 --2- 172.21.15.157:0/2470060561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb97006a780 0x7fb970153ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:06.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.162+0000 7fb96dd9b700 1 -- 172.21.15.157:0/2470060561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96400b790 con 0x7fb97006ab50 2023-10-06T04:01:06.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.162+0000 7fb96e59c700 1 --2- 172.21.15.157:0/2470060561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb97006a780 0x7fb970153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:01:06.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.163+0000 7fb96dd9b700 1 --2- 172.21.15.157:0/2470060561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb97006ab50 0x7fb970161d30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb964006fd0 tx=0x7fb96401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:06.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.163+0000 7fb95e7fc700 1 -- 172.21.15.157:0/2470060561 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb964018930 con 0x7fb97006ab50 2023-10-06T04:01:06.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.163+0000 7fb95e7fc700 1 -- 172.21.15.157:0/2470060561 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb964018ab0 con 0x7fb97006ab50 2023-10-06T04:01:06.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.163+0000 7fb974dcb700 1 -- 172.21.15.157:0/2470060561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9700bd330 con 0x7fb97006ab50 2023-10-06T04:01:06.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.163+0000 7fb95e7fc700 1 -- 172.21.15.157:0/2470060561 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb964018e70 con 0x7fb97006ab50 2023-10-06T04:01:06.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.163+0000 7fb974dcb700 1 -- 172.21.15.157:0/2470060561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb970154a50 con 0x7fb97006ab50 2023-10-06T04:01:06.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.164+0000 7fb974dcb700 1 -- 172.21.15.157:0/2470060561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9700bfc50 con 0x7fb97006ab50 2023-10-06T04:01:06.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.164+0000 7fb95e7fc700 1 -- 172.21.15.157:0/2470060561 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb9640233d0 con 0x7fb97006ab50 2023-10-06T04:01:06.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.164+0000 7fb95e7fc700 1 --2- 172.21.15.157:0/2470060561 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb950066f60 0x7fb950069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:06.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.164+0000 7fb95e7fc700 1 -- 172.21.15.157:0/2470060561 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7fb964014070 con 0x7fb97006ab50 2023-10-06T04:01:06.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.165+0000 7fb96e59c700 1 --2- 172.21.15.157:0/2470060561 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb950066f60 0x7fb950069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:06.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.165+0000 7fb96e59c700 1 --2- 172.21.15.157:0/2470060561 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb950066f60 0x7fb950069410 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb9600018d0 tx=0x7fb960009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:06.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.169+0000 7fb95e7fc700 1 -- 172.21.15.157:0/2470060561 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb9700bfc50 con 0x7fb97006ab50 2023-10-06T04:01:06.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.394+0000 7fb974dcb700 1 -- 172.21.15.157:0/2470060561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9700bfc10 con 0x7fb97006ab50 2023-10-06T04:01:06.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.394+0000 7fb95e7fc700 1 -- 172.21.15.157:0/2470060561 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+5438 (secure 0 0 0) 0x7fb9640558a0 con 0x7fb97006ab50 2023-10-06T04:01:06.398 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:01:06.398 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:01:01.662635+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"f","rank":4,"incarnation":29,"state":"up:replay","state_seq":1,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:01:06.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.396+0000 7fb974dcb700 1 -- 172.21.15.157:0/2470060561 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb950066f60 msgr2=0x7fb950069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:06.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.396+0000 7fb974dcb700 1 --2- 172.21.15.157:0/2470060561 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb950066f60 0x7fb950069410 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb9600018d0 tx=0x7fb960009040 comp rx=0 tx=0).stop 2023-10-06T04:01:06.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.396+0000 7fb974dcb700 1 -- 172.21.15.157:0/2470060561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb97006ab50 msgr2=0x7fb970161d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:06.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.396+0000 7fb974dcb700 1 --2- 172.21.15.157:0/2470060561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb97006ab50 0x7fb970161d30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb964006fd0 tx=0x7fb96401a000 comp rx=0 tx=0).stop 2023-10-06T04:01:06.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.396+0000 7fb974dcb700 1 -- 172.21.15.157:0/2470060561 shutdown_connections 2023-10-06T04:01:06.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.396+0000 7fb974dcb700 1 --2- 172.21.15.157:0/2470060561 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb950066f60 0x7fb950069410 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:06.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.396+0000 7fb974dcb700 1 --2- 172.21.15.157:0/2470060561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb97006a780 0x7fb970153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:06.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.396+0000 7fb974dcb700 1 --2- 172.21.15.157:0/2470060561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb97006ddf0 0x7fb970151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:06.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.396+0000 7fb974dcb700 1 --2- 172.21.15.157:0/2470060561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb97006ab50 0x7fb970161d30 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:06.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.396+0000 7fb974dcb700 1 -- 172.21.15.157:0/2470060561 >> 172.21.15.157:0/2470060561 conn(0x7fb97006b510 msgr2=0x7fb9700c39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:06.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.396+0000 7fb974dcb700 1 -- 172.21.15.157:0/2470060561 shutdown_connections 2023-10-06T04:01:06.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:06.397+0000 7fb974dcb700 1 -- 172.21.15.157:0/2470060561 wait complete. 2023-10-06T04:01:06.401 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 29 2023-10-06T04:01:06.411 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:01:08.412 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:01:08.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.537+0000 7f94401f3700 1 Processor -- start 2023-10-06T04:01:08.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.537+0000 7f94401f3700 1 -- start start 2023-10-06T04:01:08.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.538+0000 7f94401f3700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f943806c2d0 0x7f94380c9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:08.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.538+0000 7f94401f3700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94380b2270 0x7f943815da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:08.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.538+0000 7f94401f3700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94380c7550 con 0x7f94380b1ea0 2023-10-06T04:01:08.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.538+0000 7f94401f3700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f943815e9c0 con 0x7f943806c2d0 2023-10-06T04:01:08.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.538+0000 7f94401f3700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f943815e120 con 0x7f94380b2270 2023-10-06T04:01:08.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.538+0000 7f943d78e700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f943806c2d0 0x7f94380c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:08.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.538+0000 7f943d78e700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f943806c2d0 0x7f94380c9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56426/0 (socket says 172.21.15.157:56426) 2023-10-06T04:01:08.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.538+0000 7f943d78e700 1 -- 172.21.15.157:0/557692684 learned_addr learned my addr 172.21.15.157:0/557692684 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:01:08.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.538+0000 7f943e790700 1 --2- 172.21.15.157:0/557692684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94380b2270 0x7f943815da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:08.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.538+0000 7f943d78e700 1 -- 172.21.15.157:0/557692684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94380b2270 msgr2=0x7f943815da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:01:08.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.538+0000 7f943d78e700 1 --2- 172.21.15.157:0/557692684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94380b2270 0x7f943815da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:08.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.538+0000 7f943d78e700 1 -- 172.21.15.157:0/557692684 >> v1:172.21.15.175:6789/0 conn(0x7f94380b1ea0 legacy=0x7f94380c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:08.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.538+0000 7f943d78e700 1 -- 172.21.15.157:0/557692684 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943815e400 con 0x7f943806c2d0 2023-10-06T04:01:08.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.538+0000 7f943d78e700 1 --2- 172.21.15.157:0/557692684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f943806c2d0 0x7f94380c9620 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f942c0099b0 tx=0x7f942c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c167c71502e2a1f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:08.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.539+0000 7f943cf8d700 1 -- 172.21.15.157:0/557692684 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c009ea0 con 0x7f943806c2d0 2023-10-06T04:01:08.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.539+0000 7f943cf8d700 1 -- 172.21.15.157:0/557692684 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942c00b8f0 con 0x7f943806c2d0 2023-10-06T04:01:08.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.539+0000 7f943cf8d700 1 -- 172.21.15.157:0/557692684 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c00bc90 con 0x7f943806c2d0 2023-10-06T04:01:08.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.539+0000 7f94401f3700 1 -- 172.21.15.157:0/557692684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f943806c2d0 msgr2=0x7f94380c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:08.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.539+0000 7f94401f3700 1 --2- 172.21.15.157:0/557692684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f943806c2d0 0x7f94380c9620 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f942c0099b0 tx=0x7f942c009250 comp rx=0 tx=0).stop 2023-10-06T04:01:08.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.539+0000 7f94401f3700 1 -- 172.21.15.157:0/557692684 shutdown_connections 2023-10-06T04:01:08.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.539+0000 7f94401f3700 1 --2- 172.21.15.157:0/557692684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94380b2270 0x7f943815da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:08.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.539+0000 7f94401f3700 1 --2- 172.21.15.157:0/557692684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f943806c2d0 0x7f94380c9620 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:08.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.539+0000 7f94401f3700 1 -- 172.21.15.157:0/557692684 >> 172.21.15.157:0/557692684 conn(0x7f943806f230 msgr2=0x7f943806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:08.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.539+0000 7f94401f3700 1 -- 172.21.15.157:0/557692684 shutdown_connections 2023-10-06T04:01:08.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.539+0000 7f94401f3700 1 -- 172.21.15.157:0/557692684 wait complete. 2023-10-06T04:01:08.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.540+0000 7f94401f3700 1 Processor -- start 2023-10-06T04:01:08.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.540+0000 7f94401f3700 1 -- start start 2023-10-06T04:01:08.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.541+0000 7f94401f3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f943806c2d0 0x7f943816e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:08.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.541+0000 7f94401f3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94380b1ea0 0x7f94380c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:08.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.541+0000 7f94401f3700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94380b2270 0x7f94380c5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:08.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.541+0000 7f94401f3700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94380c07f0 con 0x7f943806c2d0 2023-10-06T04:01:08.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.541+0000 7f94401f3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94380c4260 con 0x7f94380b1ea0 2023-10-06T04:01:08.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.541+0000 7f94401f3700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f943816f250 con 0x7f94380b2270 2023-10-06T04:01:08.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.541+0000 7f943df8f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f943806c2d0 0x7f943816e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:08.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.541+0000 7f943df8f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f943806c2d0 0x7f943816e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56430/0 (socket says 172.21.15.157:56430) 2023-10-06T04:01:08.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.541+0000 7f943df8f700 1 -- 172.21.15.157:0/2961777143 learned_addr learned my addr 172.21.15.157:0/2961777143 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:01:08.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.541+0000 7f943d78e700 1 --2- 172.21.15.157:0/2961777143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94380b1ea0 0x7f94380c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:08.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.541+0000 7f943e790700 1 --2- 172.21.15.157:0/2961777143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94380b2270 0x7f94380c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:08.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.541+0000 7f943df8f700 1 -- 172.21.15.157:0/2961777143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94380b2270 msgr2=0x7f94380c5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:08.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.541+0000 7f943df8f700 1 --2- 172.21.15.157:0/2961777143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94380b2270 0x7f94380c5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:08.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.541+0000 7f943df8f700 1 -- 172.21.15.157:0/2961777143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94380b1ea0 msgr2=0x7f94380c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:08.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.541+0000 7f943df8f700 1 --2- 172.21.15.157:0/2961777143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94380b1ea0 0x7f94380c50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:08.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.541+0000 7f943df8f700 1 -- 172.21.15.157:0/2961777143 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c0094f0 con 0x7f943806c2d0 2023-10-06T04:01:08.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.542+0000 7f943df8f700 1 --2- 172.21.15.157:0/2961777143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f943806c2d0 0x7f943816e560 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f942800c590 tx=0x7f942800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:08.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.542+0000 7f9435ffb700 1 -- 172.21.15.157:0/2961777143 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9428011590 con 0x7f943806c2d0 2023-10-06T04:01:08.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.542+0000 7f9435ffb700 1 -- 172.21.15.157:0/2961777143 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9428011710 con 0x7f943806c2d0 2023-10-06T04:01:08.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.542+0000 7f9435ffb700 1 -- 172.21.15.157:0/2961777143 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9428011ad0 con 0x7f943806c2d0 2023-10-06T04:01:08.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.542+0000 7f94401f3700 1 -- 172.21.15.157:0/2961777143 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94380c4c30 con 0x7f943806c2d0 2023-10-06T04:01:08.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.542+0000 7f94401f3700 1 -- 172.21.15.157:0/2961777143 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f943816ef70 con 0x7f943806c2d0 2023-10-06T04:01:08.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.543+0000 7f94401f3700 1 -- 172.21.15.157:0/2961777143 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94380c4f10 con 0x7f943806c2d0 2023-10-06T04:01:08.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.543+0000 7f9435ffb700 1 -- 172.21.15.157:0/2961777143 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f942801b3b0 con 0x7f943806c2d0 2023-10-06T04:01:08.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.544+0000 7f9435ffb700 1 --2- 172.21.15.157:0/2961777143 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9424066f30 0x7f94240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:08.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.544+0000 7f9435ffb700 1 -- 172.21.15.157:0/2961777143 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f9428083be0 con 0x7f943806c2d0 2023-10-06T04:01:08.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.544+0000 7f943d78e700 1 --2- 172.21.15.157:0/2961777143 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9424066f30 0x7f94240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:08.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.544+0000 7f943d78e700 1 --2- 172.21.15.157:0/2961777143 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9424066f30 0x7f94240693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f942c0099b0 tx=0x7f942c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:08.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.548+0000 7f9435ffb700 1 -- 172.21.15.157:0/2961777143 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9428050c10 con 0x7f943806c2d0 2023-10-06T04:01:08.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.774+0000 7f94401f3700 1 -- 172.21.15.157:0/2961777143 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94380c9ff0 con 0x7f943806c2d0 2023-10-06T04:01:08.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.774+0000 7f9435ffb700 1 -- 172.21.15.157:0/2961777143 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+5440 (secure 0 0 0) 0x7f9428017070 con 0x7f943806c2d0 2023-10-06T04:01:08.777 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:01:08.777 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:01:07.690676+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"f","rank":4,"incarnation":29,"state":"up:resolve","state_seq":26,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:01:08.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.776+0000 7f94401f3700 1 -- 172.21.15.157:0/2961777143 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9424066f30 msgr2=0x7f94240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:08.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.776+0000 7f94401f3700 1 --2- 172.21.15.157:0/2961777143 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9424066f30 0x7f94240693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f942c0099b0 tx=0x7f942c017000 comp rx=0 tx=0).stop 2023-10-06T04:01:08.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.776+0000 7f94401f3700 1 -- 172.21.15.157:0/2961777143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f943806c2d0 msgr2=0x7f943816e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:08.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.776+0000 7f94401f3700 1 --2- 172.21.15.157:0/2961777143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f943806c2d0 0x7f943816e560 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f942800c590 tx=0x7f942800c8a0 comp rx=0 tx=0).stop 2023-10-06T04:01:08.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.776+0000 7f94401f3700 1 -- 172.21.15.157:0/2961777143 shutdown_connections 2023-10-06T04:01:08.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.776+0000 7f94401f3700 1 --2- 172.21.15.157:0/2961777143 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9424066f30 0x7f94240693e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:08.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.776+0000 7f94401f3700 1 --2- 172.21.15.157:0/2961777143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94380b2270 0x7f94380c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:08.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.776+0000 7f94401f3700 1 --2- 172.21.15.157:0/2961777143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f943806c2d0 0x7f943816e560 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:08.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.776+0000 7f94401f3700 1 --2- 172.21.15.157:0/2961777143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94380b1ea0 0x7f94380c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:08.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.776+0000 7f94401f3700 1 -- 172.21.15.157:0/2961777143 >> 172.21.15.157:0/2961777143 conn(0x7f943806f230 msgr2=0x7f94380c8180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:08.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.776+0000 7f94401f3700 1 -- 172.21.15.157:0/2961777143 shutdown_connections 2023-10-06T04:01:08.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:08.776+0000 7f94401f3700 1 -- 172.21.15.157:0/2961777143 wait complete. 2023-10-06T04:01:08.780 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 30 2023-10-06T04:01:08.791 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:01:10.791 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:01:10.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.916+0000 7f179e079700 1 Processor -- start 2023-10-06T04:01:10.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.916+0000 7f179e079700 1 -- start start 2023-10-06T04:01:10.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.917+0000 7f179e079700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f179806dc40 0x7f179814fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:10.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.917+0000 7f179e079700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f179806c1f0 0x7f1798150240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:10.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.917+0000 7f179e079700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17980c9780 con 0x7f179806cbb0 2023-10-06T04:01:10.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.917+0000 7f179e079700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17980c8c40 con 0x7f179806dc40 2023-10-06T04:01:10.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.917+0000 7f179e079700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f179806e0c0 con 0x7f179806c1f0 2023-10-06T04:01:10.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.917+0000 7f1796ffd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f179806dc40 0x7f179814fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:10.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.917+0000 7f17977fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f179806cbb0 0x7f179806d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51214/0 (socket says 172.21.15.157:51214) 2023-10-06T04:01:10.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.917+0000 7f17977fe700 1 -- 172.21.15.157:0/3411315307 learned_addr learned my addr 172.21.15.157:0/3411315307 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:01:10.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.917+0000 7f1797fff700 1 --2- 172.21.15.157:0/3411315307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f179806c1f0 0x7f1798150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:10.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.917+0000 7f1796ffd700 1 -- 172.21.15.157:0/3411315307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f179806c1f0 msgr2=0x7f1798150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:01:10.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.917+0000 7f1796ffd700 1 --2- 172.21.15.157:0/3411315307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f179806c1f0 0x7f1798150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:10.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.917+0000 7f1796ffd700 1 -- 172.21.15.157:0/3411315307 >> v1:172.21.15.157:6789/0 conn(0x7f179806cbb0 legacy=0x7f179806d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:10.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.917+0000 7f17967fc700 1 -- 172.21.15.157:0/3411315307 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3118976578 0 0) 0x7f17980c9780 con 0x7f179806cbb0 2023-10-06T04:01:10.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.917+0000 7f1796ffd700 1 -- 172.21.15.157:0/3411315307 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17980c8e50 con 0x7f179806dc40 2023-10-06T04:01:10.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.918+0000 7f1796ffd700 1 --2- 172.21.15.157:0/3411315307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f179806dc40 0x7f179814fd00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f178c0098e0 tx=0x7f178c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e36b6797b32b554 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:10.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.918+0000 7f17967fc700 1 -- 172.21.15.157:0/3411315307 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c00b6d0 con 0x7f179806dc40 2023-10-06T04:01:10.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.918+0000 7f17967fc700 1 -- 172.21.15.157:0/3411315307 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178c00b830 con 0x7f179806dc40 2023-10-06T04:01:10.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.918+0000 7f17967fc700 1 -- 172.21.15.157:0/3411315307 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c00bbd0 con 0x7f179806dc40 2023-10-06T04:01:10.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.918+0000 7f179e079700 1 -- 172.21.15.157:0/3411315307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f179806dc40 msgr2=0x7f179814fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:10.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.918+0000 7f179e079700 1 --2- 172.21.15.157:0/3411315307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f179806dc40 0x7f179814fd00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f178c0098e0 tx=0x7f178c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:01:10.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.918+0000 7f179e079700 1 -- 172.21.15.157:0/3411315307 shutdown_connections 2023-10-06T04:01:10.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.918+0000 7f179e079700 1 --2- 172.21.15.157:0/3411315307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f179806c1f0 0x7f1798150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:10.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.918+0000 7f179e079700 1 --2- 172.21.15.157:0/3411315307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f179806dc40 0x7f179814fd00 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:10.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.918+0000 7f179e079700 1 -- 172.21.15.157:0/3411315307 >> 172.21.15.157:0/3411315307 conn(0x7f179806f230 msgr2=0x7f179806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:10.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.919+0000 7f179e079700 1 -- 172.21.15.157:0/3411315307 shutdown_connections 2023-10-06T04:01:10.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.919+0000 7f179e079700 1 -- 172.21.15.157:0/3411315307 wait complete. 2023-10-06T04:01:10.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.920+0000 7f179e079700 1 Processor -- start 2023-10-06T04:01:10.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.920+0000 7f179e079700 1 -- start start 2023-10-06T04:01:10.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.921+0000 7f179e079700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f179806c1f0 0x7f179816a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:10.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.921+0000 7f179e079700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f179806cbb0 0x7f17980c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:10.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.921+0000 7f179e079700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1798166f70 0x7f17980afa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:10.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.921+0000 7f179e079700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179816a030 con 0x7f1798166f70 2023-10-06T04:01:10.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.921+0000 7f179e079700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179816aa00 con 0x7f179806cbb0 2023-10-06T04:01:10.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.921+0000 7f179e079700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1798167400 con 0x7f179806c1f0 2023-10-06T04:01:10.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.921+0000 7f1797fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1798166f70 0x7f17980afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:10.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.921+0000 7f1797fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1798166f70 0x7f17980afa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56458/0 (socket says 172.21.15.157:56458) 2023-10-06T04:01:10.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.921+0000 7f1797fff700 1 -- 172.21.15.157:0/1213110374 learned_addr learned my addr 172.21.15.157:0/1213110374 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:01:10.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.921+0000 7f17977fe700 1 --2- 172.21.15.157:0/1213110374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f179806c1f0 0x7f179816a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:10.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.921+0000 7f1796ffd700 1 --2- 172.21.15.157:0/1213110374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f179806cbb0 0x7f17980c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:10.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.921+0000 7f1797fff700 1 -- 172.21.15.157:0/1213110374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f179806c1f0 msgr2=0x7f179816a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:10.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.921+0000 7f1797fff700 1 --2- 172.21.15.157:0/1213110374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f179806c1f0 0x7f179816a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:10.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.921+0000 7f1797fff700 1 -- 172.21.15.157:0/1213110374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f179806cbb0 msgr2=0x7f17980c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:10.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.921+0000 7f1797fff700 1 --2- 172.21.15.157:0/1213110374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f179806cbb0 0x7f17980c5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:10.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.921+0000 7f1797fff700 1 -- 172.21.15.157:0/1213110374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178c009420 con 0x7f1798166f70 2023-10-06T04:01:10.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.921+0000 7f1797fff700 1 --2- 172.21.15.157:0/1213110374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1798166f70 0x7f17980afa10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f179000ea80 tx=0x7f17900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:10.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.921+0000 7f177f7fe700 1 -- 172.21.15.157:0/1213110374 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179000eec0 con 0x7f1798166f70 2023-10-06T04:01:10.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.921+0000 7f177f7fe700 1 -- 172.21.15.157:0/1213110374 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f179000c8e0 con 0x7f1798166f70 2023-10-06T04:01:10.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.921+0000 7f177f7fe700 1 -- 172.21.15.157:0/1213110374 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179000cca0 con 0x7f1798166f70 2023-10-06T04:01:10.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.921+0000 7f179e079700 1 -- 172.21.15.157:0/1213110374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17980c2f50 con 0x7f1798166f70 2023-10-06T04:01:10.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.921+0000 7f179e079700 1 -- 172.21.15.157:0/1213110374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1798169cc0 con 0x7f1798166f70 2023-10-06T04:01:10.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.922+0000 7f179e079700 1 -- 172.21.15.157:0/1213110374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1798169b20 con 0x7f1798166f70 2023-10-06T04:01:10.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.923+0000 7f177f7fe700 1 -- 172.21.15.157:0/1213110374 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1790013680 con 0x7f1798166f70 2023-10-06T04:01:10.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.923+0000 7f177f7fe700 1 --2- 172.21.15.157:0/1213110374 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1784066f30 0x7f17840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:10.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.923+0000 7f177f7fe700 1 -- 172.21.15.157:0/1213110374 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f17900849f0 con 0x7f1798166f70 2023-10-06T04:01:10.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.923+0000 7f17977fe700 1 --2- 172.21.15.157:0/1213110374 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1784066f30 0x7f17840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:10.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.924+0000 7f17977fe700 1 --2- 172.21.15.157:0/1213110374 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1784066f30 0x7f17840693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1788002250 tx=0x7f1788001a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:10.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:10.927+0000 7f177f7fe700 1 -- 172.21.15.157:0/1213110374 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1798169b20 con 0x7f1798166f70 2023-10-06T04:01:11.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:11.151+0000 7f179e079700 1 -- 172.21.15.157:0/1213110374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1798169b20 con 0x7f1798166f70 2023-10-06T04:01:11.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:11.151+0000 7f177f7fe700 1 -- 172.21.15.157:0/1213110374 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+5440 (secure 0 0 0) 0x7f1798169b20 con 0x7f1798166f70 2023-10-06T04:01:11.153 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:01:11.153 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:01:07.690676+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"f","rank":4,"incarnation":29,"state":"up:resolve","state_seq":26,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:01:11.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:11.153+0000 7f179e079700 1 -- 172.21.15.157:0/1213110374 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1784066f30 msgr2=0x7f17840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:11.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:11.153+0000 7f179e079700 1 --2- 172.21.15.157:0/1213110374 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1784066f30 0x7f17840693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1788002250 tx=0x7f1788001a20 comp rx=0 tx=0).stop 2023-10-06T04:01:11.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:11.153+0000 7f179e079700 1 -- 172.21.15.157:0/1213110374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1798166f70 msgr2=0x7f17980afa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:11.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:11.153+0000 7f179e079700 1 --2- 172.21.15.157:0/1213110374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1798166f70 0x7f17980afa10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f179000ea80 tx=0x7f17900091a0 comp rx=0 tx=0).stop 2023-10-06T04:01:11.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:11.153+0000 7f179e079700 1 -- 172.21.15.157:0/1213110374 shutdown_connections 2023-10-06T04:01:11.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:11.153+0000 7f179e079700 1 --2- 172.21.15.157:0/1213110374 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1784066f30 0x7f17840693e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:11.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:11.153+0000 7f179e079700 1 --2- 172.21.15.157:0/1213110374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1798166f70 0x7f17980afa10 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:11.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:11.153+0000 7f179e079700 1 --2- 172.21.15.157:0/1213110374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f179806c1f0 0x7f179816a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:11.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:11.153+0000 7f179e079700 1 --2- 172.21.15.157:0/1213110374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f179806cbb0 0x7f17980c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:11.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:11.153+0000 7f179e079700 1 -- 172.21.15.157:0/1213110374 >> 172.21.15.157:0/1213110374 conn(0x7f179806f230 msgr2=0x7f179814fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:11.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:11.153+0000 7f179e079700 1 -- 172.21.15.157:0/1213110374 shutdown_connections 2023-10-06T04:01:11.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:11.153+0000 7f179e079700 1 -- 172.21.15.157:0/1213110374 wait complete. 2023-10-06T04:01:11.156 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 30 2023-10-06T04:01:11.167 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:01:13.168 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:01:13.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.301+0000 7fe71af22700 1 Processor -- start 2023-10-06T04:01:13.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.301+0000 7fe71af22700 1 -- start start 2023-10-06T04:01:13.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.302+0000 7fe71af22700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe71406cbb0 0x7fe71406d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:13.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.302+0000 7fe71af22700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe71406c1f0 0x7fe71406da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:13.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.302+0000 7fe71af22700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7140c9780 con 0x7fe71406cf80 2023-10-06T04:01:13.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.302+0000 7fe71af22700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7140c8c40 con 0x7fe71406c1f0 2023-10-06T04:01:13.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.302+0000 7fe71af22700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe71406e0c0 con 0x7fe71406cbb0 2023-10-06T04:01:13.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.302+0000 7fe70bfff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe71406c1f0 0x7fe71406da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:13.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.302+0000 7fe7194bf700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe71406cf80 0x7fe71414fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38318/0 (socket says 172.21.15.157:38318) 2023-10-06T04:01:13.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.302+0000 7fe7194bf700 1 -- 172.21.15.157:0/4018401791 learned_addr learned my addr 172.21.15.157:0/4018401791 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:01:13.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.302+0000 7fe718cbe700 1 --2- 172.21.15.157:0/4018401791 >> v2:172.21.15.175:3300/0 conn(0x7fe71406cbb0 0x7fe71406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:13.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.303+0000 7fe70bfff700 1 -- 172.21.15.157:0/4018401791 >> v2:172.21.15.175:3300/0 conn(0x7fe71406cbb0 msgr2=0x7fe71406d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:01:13.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.303+0000 7fe70bfff700 1 --2- 172.21.15.157:0/4018401791 >> v2:172.21.15.175:3300/0 conn(0x7fe71406cbb0 0x7fe71406d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:13.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.303+0000 7fe70bfff700 1 -- 172.21.15.157:0/4018401791 >> v1:172.21.15.157:6789/0 conn(0x7fe71406cf80 legacy=0x7fe71414fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:13.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.303+0000 7fe70bfff700 1 -- 172.21.15.157:0/4018401791 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7140c8e50 con 0x7fe71406c1f0 2023-10-06T04:01:13.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.303+0000 7fe70b7fe700 1 -- 172.21.15.157:0/4018401791 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2781752511 0 0) 0x7fe7140c9780 con 0x7fe71406cf80 2023-10-06T04:01:13.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.303+0000 7fe718cbe700 1 --2- 172.21.15.157:0/4018401791 >> v2:172.21.15.175:3300/0 conn(0x7fe71406cbb0 0x7fe71406d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:01:13.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.303+0000 7fe70bfff700 1 --2- 172.21.15.157:0/4018401791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71406c1f0 0x7fe71406da70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe71406fd70 tx=0x7fe7040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc29b58bbbf8f343 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:13.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.303+0000 7fe70b7fe700 1 -- 172.21.15.157:0/4018401791 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70400b740 con 0x7fe71406c1f0 2023-10-06T04:01:13.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.303+0000 7fe70b7fe700 1 -- 172.21.15.157:0/4018401791 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70400b8a0 con 0x7fe71406c1f0 2023-10-06T04:01:13.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.303+0000 7fe70b7fe700 1 -- 172.21.15.157:0/4018401791 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70400bc40 con 0x7fe71406c1f0 2023-10-06T04:01:13.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.303+0000 7fe71af22700 1 -- 172.21.15.157:0/4018401791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71406c1f0 msgr2=0x7fe71406da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:13.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.303+0000 7fe71af22700 1 --2- 172.21.15.157:0/4018401791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71406c1f0 0x7fe71406da70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe71406fd70 tx=0x7fe7040091a0 comp rx=0 tx=0).stop 2023-10-06T04:01:13.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.303+0000 7fe71af22700 1 -- 172.21.15.157:0/4018401791 shutdown_connections 2023-10-06T04:01:13.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.303+0000 7fe71af22700 1 --2- 172.21.15.157:0/4018401791 >> v2:172.21.15.175:3300/0 conn(0x7fe71406cbb0 0x7fe71406d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:13.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.303+0000 7fe71af22700 1 --2- 172.21.15.157:0/4018401791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71406c1f0 0x7fe71406da70 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:13.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.303+0000 7fe71af22700 1 -- 172.21.15.157:0/4018401791 >> 172.21.15.157:0/4018401791 conn(0x7fe71406f230 msgr2=0x7fe71406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:13.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.304+0000 7fe71af22700 1 -- 172.21.15.157:0/4018401791 shutdown_connections 2023-10-06T04:01:13.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.304+0000 7fe71af22700 1 -- 172.21.15.157:0/4018401791 wait complete. 2023-10-06T04:01:13.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.305+0000 7fe71af22700 1 Processor -- start 2023-10-06T04:01:13.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.305+0000 7fe71af22700 1 -- start start 2023-10-06T04:01:13.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.305+0000 7fe71af22700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71406c1f0 0x7fe714158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:13.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.305+0000 7fe71af22700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe71406cbb0 0x7fe714157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:13.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.306+0000 7fe71af22700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe71406cf80 0x7fe714146dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:13.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.306+0000 7fe71af22700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe714159480 con 0x7fe71406c1f0 2023-10-06T04:01:13.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.306+0000 7fe71af22700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71414f300 con 0x7fe71406cf80 2023-10-06T04:01:13.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.306+0000 7fe71af22700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe714159180 con 0x7fe71406cbb0 2023-10-06T04:01:13.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.306+0000 7fe718cbe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71406c1f0 0x7fe714158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:13.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.306+0000 7fe718cbe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71406c1f0 0x7fe714158560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37686/0 (socket says 172.21.15.157:37686) 2023-10-06T04:01:13.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.306+0000 7fe718cbe700 1 -- 172.21.15.157:0/3572497411 learned_addr learned my addr 172.21.15.157:0/3572497411 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:01:13.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.306+0000 7fe7194bf700 1 --2- 172.21.15.157:0/3572497411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe71406cf80 0x7fe714146dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:13.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.306+0000 7fe70bfff700 1 --2- 172.21.15.157:0/3572497411 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe71406cbb0 0x7fe714157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:13.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.306+0000 7fe718cbe700 1 -- 172.21.15.157:0/3572497411 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe71406cbb0 msgr2=0x7fe714157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:13.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.306+0000 7fe718cbe700 1 --2- 172.21.15.157:0/3572497411 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe71406cbb0 0x7fe714157390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:13.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.306+0000 7fe718cbe700 1 -- 172.21.15.157:0/3572497411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe71406cf80 msgr2=0x7fe714146dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:13.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.306+0000 7fe718cbe700 1 --2- 172.21.15.157:0/3572497411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe71406cf80 0x7fe714146dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:13.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.306+0000 7fe718cbe700 1 -- 172.21.15.157:0/3572497411 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe704009420 con 0x7fe71406c1f0 2023-10-06T04:01:13.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.306+0000 7fe70bfff700 1 --2- 172.21.15.157:0/3572497411 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe71406cbb0 0x7fe714157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:01:13.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.306+0000 7fe718cbe700 1 --2- 172.21.15.157:0/3572497411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71406c1f0 0x7fe714158560 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe71000e900 tx=0x7fe710008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:13.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.306+0000 7fe708ff9700 1 -- 172.21.15.157:0/3572497411 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71000eea0 con 0x7fe71406c1f0 2023-10-06T04:01:13.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.306+0000 7fe708ff9700 1 -- 172.21.15.157:0/3572497411 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71000c5e0 con 0x7fe71406c1f0 2023-10-06T04:01:13.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.306+0000 7fe71af22700 1 -- 172.21.15.157:0/3572497411 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7141541d0 con 0x7fe71406c1f0 2023-10-06T04:01:13.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.306+0000 7fe708ff9700 1 -- 172.21.15.157:0/3572497411 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71000c980 con 0x7fe71406c1f0 2023-10-06T04:01:13.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.306+0000 7fe71af22700 1 -- 172.21.15.157:0/3572497411 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe71415a150 con 0x7fe71406c1f0 2023-10-06T04:01:13.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.307+0000 7fe71af22700 1 -- 172.21.15.157:0/3572497411 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7141592c0 con 0x7fe71406c1f0 2023-10-06T04:01:13.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.308+0000 7fe708ff9700 1 -- 172.21.15.157:0/3572497411 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe71000cd30 con 0x7fe71406c1f0 2023-10-06T04:01:13.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.308+0000 7fe708ff9700 1 --2- 172.21.15.157:0/3572497411 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe700066f30 0x7fe7000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:13.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.308+0000 7fe708ff9700 1 -- 172.21.15.157:0/3572497411 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7fe710084400 con 0x7fe71406c1f0 2023-10-06T04:01:13.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.308+0000 7fe70bfff700 1 --2- 172.21.15.157:0/3572497411 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe700066f30 0x7fe7000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:13.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.309+0000 7fe70bfff700 1 --2- 172.21.15.157:0/3572497411 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe700066f30 0x7fe7000693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe7140af550 tx=0x7fe704017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:13.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.312+0000 7fe708ff9700 1 -- 172.21.15.157:0/3572497411 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe7141592c0 con 0x7fe71406c1f0 2023-10-06T04:01:13.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.539+0000 7fe71af22700 1 -- 172.21.15.157:0/3572497411 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7141592c0 con 0x7fe71406c1f0 2023-10-06T04:01:13.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.539+0000 7fe708ff9700 1 -- 172.21.15.157:0/3572497411 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+5440 (secure 0 0 0) 0x7fe7141592c0 con 0x7fe71406c1f0 2023-10-06T04:01:13.541 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:01:13.541 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:01:07.690676+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"f","rank":4,"incarnation":29,"state":"up:resolve","state_seq":26,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:01:13.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.541+0000 7fe71af22700 1 -- 172.21.15.157:0/3572497411 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe700066f30 msgr2=0x7fe7000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:13.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.541+0000 7fe71af22700 1 --2- 172.21.15.157:0/3572497411 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe700066f30 0x7fe7000693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe7140af550 tx=0x7fe704017000 comp rx=0 tx=0).stop 2023-10-06T04:01:13.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.541+0000 7fe71af22700 1 -- 172.21.15.157:0/3572497411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71406c1f0 msgr2=0x7fe714158560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:13.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.541+0000 7fe71af22700 1 --2- 172.21.15.157:0/3572497411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71406c1f0 0x7fe714158560 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe71000e900 tx=0x7fe710008040 comp rx=0 tx=0).stop 2023-10-06T04:01:13.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.541+0000 7fe71af22700 1 -- 172.21.15.157:0/3572497411 shutdown_connections 2023-10-06T04:01:13.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.541+0000 7fe71af22700 1 --2- 172.21.15.157:0/3572497411 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe700066f30 0x7fe7000693e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:13.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.541+0000 7fe71af22700 1 --2- 172.21.15.157:0/3572497411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe71406cf80 0x7fe714146dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:13.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.541+0000 7fe71af22700 1 --2- 172.21.15.157:0/3572497411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe71406c1f0 0x7fe714158560 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:13.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.541+0000 7fe71af22700 1 --2- 172.21.15.157:0/3572497411 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe71406cbb0 0x7fe714157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:13.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.541+0000 7fe71af22700 1 -- 172.21.15.157:0/3572497411 >> 172.21.15.157:0/3572497411 conn(0x7fe71406f230 msgr2=0x7fe71406d530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:13.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.541+0000 7fe71af22700 1 -- 172.21.15.157:0/3572497411 shutdown_connections 2023-10-06T04:01:13.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:13.541+0000 7fe71af22700 1 -- 172.21.15.157:0/3572497411 wait complete. 2023-10-06T04:01:13.544 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 30 2023-10-06T04:01:13.555 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:01:14.021 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:01:14.024 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:01:15.558 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:01:15.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.678+0000 7fd584f38700 1 Processor -- start 2023-10-06T04:01:15.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.678+0000 7fd584f38700 1 -- start start 2023-10-06T04:01:15.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.678+0000 7fd584f38700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd58006dc40 0x7fd58014fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:15.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.678+0000 7fd584f38700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd58006c1f0 0x7fd580150240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:15.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.678+0000 7fd584f38700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5800c9780 con 0x7fd58006cbb0 2023-10-06T04:01:15.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.678+0000 7fd584f38700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5800c8c40 con 0x7fd58006dc40 2023-10-06T04:01:15.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.678+0000 7fd584f38700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd58006e0c0 con 0x7fd58006c1f0 2023-10-06T04:01:15.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.678+0000 7fd575d9b700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd58006dc40 0x7fd58014fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:15.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.679+0000 7fd575d9b700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd58006dc40 0x7fd58014fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37700/0 (socket says 172.21.15.157:37700) 2023-10-06T04:01:15.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.679+0000 7fd575d9b700 1 -- 172.21.15.157:0/19127706 learned_addr learned my addr 172.21.15.157:0/19127706 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:01:15.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.679+0000 7fd57ed9d700 1 --2- 172.21.15.157:0/19127706 >> v2:172.21.15.175:3300/0 conn(0x7fd58006c1f0 0x7fd580150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:15.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.679+0000 7fd575d9b700 1 -- 172.21.15.157:0/19127706 >> v2:172.21.15.175:3300/0 conn(0x7fd58006c1f0 msgr2=0x7fd580150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:01:15.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.679+0000 7fd575d9b700 1 --2- 172.21.15.157:0/19127706 >> v2:172.21.15.175:3300/0 conn(0x7fd58006c1f0 0x7fd580150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:15.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.679+0000 7fd575d9b700 1 -- 172.21.15.157:0/19127706 >> v1:172.21.15.175:6789/0 conn(0x7fd58006cbb0 legacy=0x7fd58006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:15.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.679+0000 7fd575d9b700 1 -- 172.21.15.157:0/19127706 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5800c8e50 con 0x7fd58006dc40 2023-10-06T04:01:15.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.679+0000 7fd575d9b700 1 --2- 172.21.15.157:0/19127706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd58006dc40 0x7fd58014fd00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd5800c96f0 tx=0x7fd568009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa8b8029cf13998 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:15.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.679+0000 7fd57dd9b700 1 -- 172.21.15.157:0/19127706 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd568009e70 con 0x7fd58006dc40 2023-10-06T04:01:15.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.679+0000 7fd57dd9b700 1 -- 172.21.15.157:0/19127706 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56800b8f0 con 0x7fd58006dc40 2023-10-06T04:01:15.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.679+0000 7fd57dd9b700 1 -- 172.21.15.157:0/19127706 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56800bc90 con 0x7fd58006dc40 2023-10-06T04:01:15.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.680+0000 7fd584f38700 1 -- 172.21.15.157:0/19127706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd58006dc40 msgr2=0x7fd58014fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:15.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.680+0000 7fd584f38700 1 --2- 172.21.15.157:0/19127706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd58006dc40 0x7fd58014fd00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd5800c96f0 tx=0x7fd568009250 comp rx=0 tx=0).stop 2023-10-06T04:01:15.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.680+0000 7fd584f38700 1 -- 172.21.15.157:0/19127706 shutdown_connections 2023-10-06T04:01:15.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.680+0000 7fd584f38700 1 --2- 172.21.15.157:0/19127706 >> v2:172.21.15.175:3300/0 conn(0x7fd58006c1f0 0x7fd580150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:15.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.680+0000 7fd584f38700 1 --2- 172.21.15.157:0/19127706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd58006dc40 0x7fd58014fd00 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:15.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.680+0000 7fd584f38700 1 -- 172.21.15.157:0/19127706 >> 172.21.15.157:0/19127706 conn(0x7fd58006f230 msgr2=0x7fd58006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:15.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.680+0000 7fd584f38700 1 -- 172.21.15.157:0/19127706 shutdown_connections 2023-10-06T04:01:15.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.680+0000 7fd584f38700 1 -- 172.21.15.157:0/19127706 wait complete. 2023-10-06T04:01:15.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.681+0000 7fd584f38700 1 Processor -- start 2023-10-06T04:01:15.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.681+0000 7fd584f38700 1 -- start start 2023-10-06T04:01:15.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.682+0000 7fd584f38700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd58006c1f0 0x7fd5800c47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:15.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.682+0000 7fd584f38700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd58006cbb0 0x7fd58014af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:15.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.682+0000 7fd584f38700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd580153560 0x7fd5800cad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:15.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.682+0000 7fd584f38700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5800c5740 con 0x7fd58006c1f0 2023-10-06T04:01:15.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.682+0000 7fd584f38700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd580152350 con 0x7fd58006cbb0 2023-10-06T04:01:15.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.682+0000 7fd584f38700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5801525f0 con 0x7fd580153560 2023-10-06T04:01:15.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.682+0000 7fd57e59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd58006c1f0 0x7fd5800c47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:15.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.682+0000 7fd57e59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd58006c1f0 0x7fd5800c47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37712/0 (socket says 172.21.15.157:37712) 2023-10-06T04:01:15.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.682+0000 7fd57e59c700 1 -- 172.21.15.157:0/3504935300 learned_addr learned my addr 172.21.15.157:0/3504935300 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:01:15.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.682+0000 7fd575d9b700 1 --2- 172.21.15.157:0/3504935300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd58006cbb0 0x7fd58014af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:15.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.682+0000 7fd57ed9d700 1 --2- 172.21.15.157:0/3504935300 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd580153560 0x7fd5800cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:15.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.682+0000 7fd57e59c700 1 -- 172.21.15.157:0/3504935300 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd580153560 msgr2=0x7fd5800cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:15.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.682+0000 7fd57e59c700 1 --2- 172.21.15.157:0/3504935300 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd580153560 0x7fd5800cad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:15.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.682+0000 7fd57e59c700 1 -- 172.21.15.157:0/3504935300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd58006cbb0 msgr2=0x7fd58014af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:15.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.682+0000 7fd57e59c700 1 --2- 172.21.15.157:0/3504935300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd58006cbb0 0x7fd58014af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:15.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.682+0000 7fd57e59c700 1 -- 172.21.15.157:0/3504935300 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5680094f0 con 0x7fd58006c1f0 2023-10-06T04:01:15.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.682+0000 7fd575d9b700 1 --2- 172.21.15.157:0/3504935300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd58006cbb0 0x7fd58014af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:01:15.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.682+0000 7fd57e59c700 1 --2- 172.21.15.157:0/3504935300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd58006c1f0 0x7fd5800c47b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd57000c590 tx=0x7fd57000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:15.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.682+0000 7fd576ffd700 1 -- 172.21.15.157:0/3504935300 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570011590 con 0x7fd58006c1f0 2023-10-06T04:01:15.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.682+0000 7fd576ffd700 1 -- 172.21.15.157:0/3504935300 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5700116f0 con 0x7fd58006c1f0 2023-10-06T04:01:15.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.682+0000 7fd584f38700 1 -- 172.21.15.157:0/3504935300 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5801528d0 con 0x7fd58006c1f0 2023-10-06T04:01:15.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.682+0000 7fd576ffd700 1 -- 172.21.15.157:0/3504935300 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570011ab0 con 0x7fd58006c1f0 2023-10-06T04:01:15.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.682+0000 7fd584f38700 1 -- 172.21.15.157:0/3504935300 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5800c53d0 con 0x7fd58006c1f0 2023-10-06T04:01:15.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.683+0000 7fd584f38700 1 -- 172.21.15.157:0/3504935300 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd580089ca0 con 0x7fd58006c1f0 2023-10-06T04:01:15.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.684+0000 7fd576ffd700 1 -- 172.21.15.157:0/3504935300 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd57001b3b0 con 0x7fd58006c1f0 2023-10-06T04:01:15.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.684+0000 7fd576ffd700 1 --2- 172.21.15.157:0/3504935300 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd560066f30 0x7fd5600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:15.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.684+0000 7fd576ffd700 1 -- 172.21.15.157:0/3504935300 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7fd570083d20 con 0x7fd58006c1f0 2023-10-06T04:01:15.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.684+0000 7fd575d9b700 1 --2- 172.21.15.157:0/3504935300 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd560066f30 0x7fd5600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:15.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.685+0000 7fd575d9b700 1 --2- 172.21.15.157:0/3504935300 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd560066f30 0x7fd5600693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd5800c96f0 tx=0x7fd568017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:15.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.688+0000 7fd576ffd700 1 -- 172.21.15.157:0/3504935300 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd570050d50 con 0x7fd58006c1f0 2023-10-06T04:01:15.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.913+0000 7fd584f38700 1 -- 172.21.15.157:0/3504935300 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5800c9130 con 0x7fd58006c1f0 2023-10-06T04:01:15.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.913+0000 7fd576ffd700 1 -- 172.21.15.157:0/3504935300 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+5442 (secure 0 0 0) 0x7fd570017070 con 0x7fd58006c1f0 2023-10-06T04:01:15.915 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:01:15.915 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:01:13.706044+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"f","rank":4,"incarnation":29,"state":"up:reconnect","state_seq":28,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:01:15.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.915+0000 7fd584f38700 1 -- 172.21.15.157:0/3504935300 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd560066f30 msgr2=0x7fd5600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:15.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.915+0000 7fd584f38700 1 --2- 172.21.15.157:0/3504935300 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd560066f30 0x7fd5600693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd5800c96f0 tx=0x7fd568017000 comp rx=0 tx=0).stop 2023-10-06T04:01:15.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.915+0000 7fd584f38700 1 -- 172.21.15.157:0/3504935300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd58006c1f0 msgr2=0x7fd5800c47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:15.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.915+0000 7fd584f38700 1 --2- 172.21.15.157:0/3504935300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd58006c1f0 0x7fd5800c47b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd57000c590 tx=0x7fd57000c950 comp rx=0 tx=0).stop 2023-10-06T04:01:15.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.915+0000 7fd584f38700 1 -- 172.21.15.157:0/3504935300 shutdown_connections 2023-10-06T04:01:15.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.915+0000 7fd584f38700 1 --2- 172.21.15.157:0/3504935300 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd560066f30 0x7fd5600693e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:15.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.915+0000 7fd584f38700 1 --2- 172.21.15.157:0/3504935300 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd580153560 0x7fd5800cad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:15.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.915+0000 7fd584f38700 1 --2- 172.21.15.157:0/3504935300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd58006c1f0 0x7fd5800c47b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:15.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.915+0000 7fd584f38700 1 --2- 172.21.15.157:0/3504935300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd58006cbb0 0x7fd58014af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:15.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.915+0000 7fd584f38700 1 -- 172.21.15.157:0/3504935300 >> 172.21.15.157:0/3504935300 conn(0x7fd58006f230 msgr2=0x7fd5800c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:15.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.915+0000 7fd584f38700 1 -- 172.21.15.157:0/3504935300 shutdown_connections 2023-10-06T04:01:15.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:15.915+0000 7fd584f38700 1 -- 172.21.15.157:0/3504935300 wait complete. 2023-10-06T04:01:15.919 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 31 2023-10-06T04:01:15.929 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 31, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-06T03:57:10.718782+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 31, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0, 1, 2, 3, 4 ], "info": { "gid_4270": { "addr": "172.21.15.175:6835/1596042861", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6834", "nonce": 1596042861, "type": "v2" }, { "addr": "172.21.15.175:6835", "nonce": 1596042861, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4270, "incarnation": 13, "join_fscid": -1, "name": "b", "rank": 3, "state": "up:active", "state_seq": 8 }, "gid_4279": { "addr": "172.21.15.175:6837/1002404870", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6836", "nonce": 1002404870, "type": "v2" }, { "addr": "172.21.15.175:6837", "nonce": 1002404870, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1, 3, 4 ], "features": 4540138320759226367, "flags": 0, "gid": 4279, "incarnation": 11, "join_fscid": -1, "name": "d", "rank": 2, "state": "up:active", "state_seq": 7 }, "gid_4287": { "addr": "172.21.15.157:6834/1179753185", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6832", "nonce": 1179753185, "type": "v2" }, { "addr": "172.21.15.157:6834", "nonce": 1179753185, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4287, "incarnation": 9, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4296": { "addr": "172.21.15.157:6837/4188652695", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6836", "nonce": 4188652695, "type": "v2" }, { "addr": "172.21.15.157:6837", "nonce": 4188652695, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2, 4 ], "features": 4540138320759226367, "flags": 0, "gid": 4296, "incarnation": 4, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 4 }, "gid_4936": { "addr": "172.21.15.175:6839/3353503675", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6838", "nonce": 3353503675, "type": "v2" }, { "addr": "172.21.15.175:6839", "nonce": 3353503675, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4936, "incarnation": 29, "join_fscid": -1, "name": "f", "rank": 4, "state": "up:reconnect", "state_seq": 28 } }, "last_failure": 0, "last_failure_osd_epoch": 27, "max_file_size": 1099511627776, "max_mds": 5, "metadata_pool": 3, "modified": "2023-10-06T04:01:13.706044+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4296, "mds_1": 4287, "mds_2": 4279, "mds_3": 4270, "mds_4": 4936 } } } ], "standbys": [] } 2023-10-06T04:01:17.931 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:01:18.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.054+0000 7fe41d57c700 1 Processor -- start 2023-10-06T04:01:18.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.054+0000 7fe41d57c700 1 -- start start 2023-10-06T04:01:18.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.054+0000 7fe41d57c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe41806cbb0 0x7fe41806d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:18.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.054+0000 7fe41d57c700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe41806c1f0 0x7fe41806da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:18.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.054+0000 7fe41d57c700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4180c9780 con 0x7fe41806cf80 2023-10-06T04:01:18.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.054+0000 7fe41d57c700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4180c8c40 con 0x7fe41806cbb0 2023-10-06T04:01:18.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.054+0000 7fe41d57c700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe41806e0c0 con 0x7fe41806c1f0 2023-10-06T04:01:18.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.055+0000 7fe416ffd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe41806cbb0 0x7fe41806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:18.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.055+0000 7fe416ffd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe41806cbb0 0x7fe41806d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37724/0 (socket says 172.21.15.157:37724) 2023-10-06T04:01:18.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.055+0000 7fe416ffd700 1 -- 172.21.15.157:0/175262460 learned_addr learned my addr 172.21.15.157:0/175262460 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:01:18.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.055+0000 7fe4167fc700 1 --2- 172.21.15.157:0/175262460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe41806c1f0 0x7fe41806da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:18.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.055+0000 7fe416ffd700 1 -- 172.21.15.157:0/175262460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe41806c1f0 msgr2=0x7fe41806da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:01:18.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.055+0000 7fe416ffd700 1 --2- 172.21.15.157:0/175262460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe41806c1f0 0x7fe41806da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:18.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.055+0000 7fe416ffd700 1 -- 172.21.15.157:0/175262460 >> v1:172.21.15.175:6789/0 conn(0x7fe41806cf80 legacy=0x7fe41814fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:18.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.055+0000 7fe416ffd700 1 -- 172.21.15.157:0/175262460 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4180c8e50 con 0x7fe41806cbb0 2023-10-06T04:01:18.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.055+0000 7fe416ffd700 1 --2- 172.21.15.157:0/175262460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe41806cbb0 0x7fe41806d530 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe408009f50 tx=0x7fe408009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98168cca009277f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:18.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.055+0000 7fe415ffb700 1 -- 172.21.15.157:0/175262460 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40800e070 con 0x7fe41806cbb0 2023-10-06T04:01:18.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.055+0000 7fe415ffb700 1 -- 172.21.15.157:0/175262460 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe40800ba50 con 0x7fe41806cbb0 2023-10-06T04:01:18.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.055+0000 7fe415ffb700 1 -- 172.21.15.157:0/175262460 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4080092e0 con 0x7fe41806cbb0 2023-10-06T04:01:18.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.056+0000 7fe41d57c700 1 -- 172.21.15.157:0/175262460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe41806cbb0 msgr2=0x7fe41806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:18.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.056+0000 7fe41d57c700 1 --2- 172.21.15.157:0/175262460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe41806cbb0 0x7fe41806d530 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe408009f50 tx=0x7fe408009840 comp rx=0 tx=0).stop 2023-10-06T04:01:18.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.056+0000 7fe41d57c700 1 -- 172.21.15.157:0/175262460 shutdown_connections 2023-10-06T04:01:18.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.056+0000 7fe41d57c700 1 --2- 172.21.15.157:0/175262460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe41806cbb0 0x7fe41806d530 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:18.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.056+0000 7fe41d57c700 1 --2- 172.21.15.157:0/175262460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe41806c1f0 0x7fe41806da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:18.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.056+0000 7fe41d57c700 1 -- 172.21.15.157:0/175262460 >> 172.21.15.157:0/175262460 conn(0x7fe41806f230 msgr2=0x7fe41806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:18.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.056+0000 7fe41d57c700 1 -- 172.21.15.157:0/175262460 shutdown_connections 2023-10-06T04:01:18.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.056+0000 7fe41d57c700 1 -- 172.21.15.157:0/175262460 wait complete. 2023-10-06T04:01:18.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.057+0000 7fe41d57c700 1 Processor -- start 2023-10-06T04:01:18.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.057+0000 7fe41d57c700 1 -- start start 2023-10-06T04:01:18.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.058+0000 7fe41d57c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe41806c1f0 0x7fe418154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:18.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.058+0000 7fe41d57c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe41806cbb0 0x7fe418154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:18.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.058+0000 7fe41d57c700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe41806cf80 0x7fe418159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:18.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.058+0000 7fe41d57c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe418153bb0 con 0x7fe41806cbb0 2023-10-06T04:01:18.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.058+0000 7fe41d57c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4181588d0 con 0x7fe41806c1f0 2023-10-06T04:01:18.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.058+0000 7fe41d57c700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4181585f0 con 0x7fe41806cf80 2023-10-06T04:01:18.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.058+0000 7fe4167fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe41806cbb0 0x7fe418154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:18.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.058+0000 7fe4167fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe41806cbb0 0x7fe418154390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37736/0 (socket says 172.21.15.157:37736) 2023-10-06T04:01:18.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.058+0000 7fe4167fc700 1 -- 172.21.15.157:0/3772154298 learned_addr learned my addr 172.21.15.157:0/3772154298 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:01:18.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.058+0000 7fe416ffd700 1 --2- 172.21.15.157:0/3772154298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe41806c1f0 0x7fe418154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:18.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.058+0000 7fe4177fe700 1 --2- 172.21.15.157:0/3772154298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe41806cf80 0x7fe418159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:18.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.058+0000 7fe4167fc700 1 -- 172.21.15.157:0/3772154298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe41806cf80 msgr2=0x7fe418159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:18.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.058+0000 7fe4167fc700 1 --2- 172.21.15.157:0/3772154298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe41806cf80 0x7fe418159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:18.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.058+0000 7fe4167fc700 1 -- 172.21.15.157:0/3772154298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe41806c1f0 msgr2=0x7fe418154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:18.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.058+0000 7fe4167fc700 1 --2- 172.21.15.157:0/3772154298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe41806c1f0 0x7fe418154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:18.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.058+0000 7fe4167fc700 1 -- 172.21.15.157:0/3772154298 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe408009ae0 con 0x7fe41806cbb0 2023-10-06T04:01:18.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.058+0000 7fe4167fc700 1 --2- 172.21.15.157:0/3772154298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe41806cbb0 0x7fe418154390 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00ecb0 tx=0x7fe40c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:18.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.058+0000 7fe3feffd700 1 -- 172.21.15.157:0/3772154298 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40c00c920 con 0x7fe41806cbb0 2023-10-06T04:01:18.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.058+0000 7fe3feffd700 1 -- 172.21.15.157:0/3772154298 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe40c00ca80 con 0x7fe41806cbb0 2023-10-06T04:01:18.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.058+0000 7fe41d57c700 1 -- 172.21.15.157:0/3772154298 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4181572c0 con 0x7fe41806cbb0 2023-10-06T04:01:18.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.058+0000 7fe3feffd700 1 -- 172.21.15.157:0/3772154298 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40c00ce40 con 0x7fe41806cbb0 2023-10-06T04:01:18.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.059+0000 7fe41d57c700 1 -- 172.21.15.157:0/3772154298 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4181540a0 con 0x7fe41806cbb0 2023-10-06T04:01:18.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.059+0000 7fe41d57c700 1 -- 172.21.15.157:0/3772154298 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe418089ca0 con 0x7fe41806cbb0 2023-10-06T04:01:18.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.060+0000 7fe3feffd700 1 -- 172.21.15.157:0/3772154298 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe40c013680 con 0x7fe41806cbb0 2023-10-06T04:01:18.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.060+0000 7fe3feffd700 1 --2- 172.21.15.157:0/3772154298 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe404066f30 0x7fe4040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:18.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.060+0000 7fe3feffd700 1 -- 172.21.15.157:0/3772154298 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7fe40c084a50 con 0x7fe41806cbb0 2023-10-06T04:01:18.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.060+0000 7fe416ffd700 1 --2- 172.21.15.157:0/3772154298 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe404066f30 0x7fe4040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:18.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.061+0000 7fe416ffd700 1 --2- 172.21.15.157:0/3772154298 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe404066f30 0x7fe4040693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe408009880 tx=0x7fe408018000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:18.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.064+0000 7fe3feffd700 1 -- 172.21.15.157:0/3772154298 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe40c051a00 con 0x7fe41806cbb0 2023-10-06T04:01:18.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.288+0000 7fe41d57c700 1 -- 172.21.15.157:0/3772154298 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4180c9130 con 0x7fe41806cbb0 2023-10-06T04:01:18.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.288+0000 7fe3feffd700 1 -- 172.21.15.157:0/3772154298 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+5442 (secure 0 0 0) 0x7fe40c055020 con 0x7fe41806cbb0 2023-10-06T04:01:18.290 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:01:18.290 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:01:13.706044+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"f","rank":4,"incarnation":29,"state":"up:reconnect","state_seq":28,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:01:18.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.291+0000 7fe41d57c700 1 -- 172.21.15.157:0/3772154298 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe404066f30 msgr2=0x7fe4040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:18.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.291+0000 7fe41d57c700 1 --2- 172.21.15.157:0/3772154298 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe404066f30 0x7fe4040693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe408009880 tx=0x7fe408018000 comp rx=0 tx=0).stop 2023-10-06T04:01:18.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.291+0000 7fe41d57c700 1 -- 172.21.15.157:0/3772154298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe41806cbb0 msgr2=0x7fe418154390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:18.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.291+0000 7fe41d57c700 1 --2- 172.21.15.157:0/3772154298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe41806cbb0 0x7fe418154390 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00ecb0 tx=0x7fe40c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:01:18.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.291+0000 7fe41d57c700 1 -- 172.21.15.157:0/3772154298 shutdown_connections 2023-10-06T04:01:18.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.291+0000 7fe41d57c700 1 --2- 172.21.15.157:0/3772154298 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe404066f30 0x7fe4040693e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:18.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.291+0000 7fe41d57c700 1 --2- 172.21.15.157:0/3772154298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe41806c1f0 0x7fe418154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:18.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.291+0000 7fe41d57c700 1 --2- 172.21.15.157:0/3772154298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe41806cf80 0x7fe418159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:18.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.291+0000 7fe41d57c700 1 --2- 172.21.15.157:0/3772154298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe41806cbb0 0x7fe418154390 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:18.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.291+0000 7fe41d57c700 1 -- 172.21.15.157:0/3772154298 >> 172.21.15.157:0/3772154298 conn(0x7fe41806f230 msgr2=0x7fe4180af1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:18.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.291+0000 7fe41d57c700 1 -- 172.21.15.157:0/3772154298 shutdown_connections 2023-10-06T04:01:18.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:18.291+0000 7fe41d57c700 1 -- 172.21.15.157:0/3772154298 wait complete. 2023-10-06T04:01:18.294 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 31 2023-10-06T04:01:18.304 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:01:20.305 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:01:20.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.432+0000 7fdefbca5700 1 Processor -- start 2023-10-06T04:01:20.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.432+0000 7fdefbca5700 1 -- start start 2023-10-06T04:01:20.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.433+0000 7fdefbca5700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdef40c97d0 0x7fdef4156a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:20.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.433+0000 7fdefbca5700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdef4157490 0x7fdef4159c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:20.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.433+0000 7fdefbca5700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef41507c0 con 0x7fdef40c81b0 2023-10-06T04:01:20.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.433+0000 7fdefbca5700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef40c60e0 con 0x7fdef40c97d0 2023-10-06T04:01:20.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.433+0000 7fdefbca5700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef40c40b0 con 0x7fdef4157490 2023-10-06T04:01:20.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.433+0000 7fdef9240700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdef40c97d0 0x7fdef4156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:20.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.433+0000 7fdefa242700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdef4157490 0x7fdef4159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:20.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.433+0000 7fdef9a41700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fdef40c81b0 0x7fdef40c4540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.157:49676/0 (socket says 172.21.15.157:49676) 2023-10-06T04:01:20.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.433+0000 7fdefa242700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdef4157490 0x7fdef4159c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:57314/0 (socket says 172.21.15.157:57314) 2023-10-06T04:01:20.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.433+0000 7fdef9a41700 1 -- 172.21.15.157:0/3574625348 learned_addr learned my addr 172.21.15.157:0/3574625348 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:01:20.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.433+0000 7fdef9240700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdef40c97d0 0x7fdef4156a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:45774/0 (socket says 172.21.15.157:45774) 2023-10-06T04:01:20.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.433+0000 7fdefa242700 1 -- 172.21.15.157:0/3574625348 >> v2:172.21.15.175:3300/0 conn(0x7fdef40c97d0 msgr2=0x7fdef4156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:01:20.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.433+0000 7fdefa242700 1 --2- 172.21.15.157:0/3574625348 >> v2:172.21.15.175:3300/0 conn(0x7fdef40c97d0 0x7fdef4156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:20.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.433+0000 7fdefa242700 1 -- 172.21.15.157:0/3574625348 >> v1:172.21.15.175:6789/0 conn(0x7fdef40c81b0 legacy=0x7fdef40c4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:20.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.433+0000 7fdefa242700 1 -- 172.21.15.157:0/3574625348 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef40c6330 con 0x7fdef4157490 2023-10-06T04:01:20.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.433+0000 7fdef9240700 1 --2- 172.21.15.157:0/3574625348 >> v2:172.21.15.175:3300/0 conn(0x7fdef40c97d0 0x7fdef4156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:01:20.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.433+0000 7fdefa242700 1 --2- 172.21.15.157:0/3574625348 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdef4157490 0x7fdef4159c30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0098e0 tx=0x7fdeec0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6800fced11b8166f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:20.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.434+0000 7fdef8a3f700 1 -- 172.21.15.157:0/3574625348 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec00b6d0 con 0x7fdef4157490 2023-10-06T04:01:20.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.434+0000 7fdef8a3f700 1 -- 172.21.15.157:0/3574625348 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeec00b830 con 0x7fdef4157490 2023-10-06T04:01:20.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.434+0000 7fdef8a3f700 1 -- 172.21.15.157:0/3574625348 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec00bbd0 con 0x7fdef4157490 2023-10-06T04:01:20.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.434+0000 7fdefbca5700 1 -- 172.21.15.157:0/3574625348 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdef4157490 msgr2=0x7fdef4159c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:20.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.434+0000 7fdefbca5700 1 --2- 172.21.15.157:0/3574625348 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdef4157490 0x7fdef4159c30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0098e0 tx=0x7fdeec0091a0 comp rx=0 tx=0).stop 2023-10-06T04:01:20.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.434+0000 7fdefbca5700 1 -- 172.21.15.157:0/3574625348 shutdown_connections 2023-10-06T04:01:20.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.434+0000 7fdefbca5700 1 --2- 172.21.15.157:0/3574625348 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdef4157490 0x7fdef4159c30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:20.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.434+0000 7fdefbca5700 1 --2- 172.21.15.157:0/3574625348 >> v2:172.21.15.175:3300/0 conn(0x7fdef40c97d0 0x7fdef4156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:20.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.434+0000 7fdefbca5700 1 -- 172.21.15.157:0/3574625348 >> 172.21.15.157:0/3574625348 conn(0x7fdef40cd0b0 msgr2=0x7fdef40c9050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:20.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.434+0000 7fdefbca5700 1 -- 172.21.15.157:0/3574625348 shutdown_connections 2023-10-06T04:01:20.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.434+0000 7fdefbca5700 1 -- 172.21.15.157:0/3574625348 wait complete. 2023-10-06T04:01:20.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.435+0000 7fdefbca5700 1 Processor -- start 2023-10-06T04:01:20.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.435+0000 7fdefbca5700 1 -- start start 2023-10-06T04:01:20.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.436+0000 7fdefbca5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdef40c81b0 0x7fdef4154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:20.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.436+0000 7fdefbca5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdef40c97d0 0x7fdef4167a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:20.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.436+0000 7fdefbca5700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdef4157490 0x7fdef4167f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:20.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.436+0000 7fdefbca5700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef41675a0 con 0x7fdef40c81b0 2023-10-06T04:01:20.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.436+0000 7fdefbca5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef41593a0 con 0x7fdef40c97d0 2023-10-06T04:01:20.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.436+0000 7fdefbca5700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef4159600 con 0x7fdef4157490 2023-10-06T04:01:20.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.436+0000 7fdef9a41700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdef40c81b0 0x7fdef4154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:20.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.436+0000 7fdef9a41700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdef40c81b0 0x7fdef4154e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37740/0 (socket says 172.21.15.157:37740) 2023-10-06T04:01:20.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.436+0000 7fdef9a41700 1 -- 172.21.15.157:0/576472316 learned_addr learned my addr 172.21.15.157:0/576472316 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:01:20.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.436+0000 7fdef9240700 1 --2- 172.21.15.157:0/576472316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdef40c97d0 0x7fdef4167a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:20.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.436+0000 7fdefa242700 1 --2- 172.21.15.157:0/576472316 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdef4157490 0x7fdef4167f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:20.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.436+0000 7fdef9a41700 1 -- 172.21.15.157:0/576472316 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdef4157490 msgr2=0x7fdef4167f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:20.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.436+0000 7fdef9a41700 1 --2- 172.21.15.157:0/576472316 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdef4157490 0x7fdef4167f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:20.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.436+0000 7fdef9a41700 1 -- 172.21.15.157:0/576472316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdef40c97d0 msgr2=0x7fdef4167a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:20.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.436+0000 7fdef9a41700 1 --2- 172.21.15.157:0/576472316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdef40c97d0 0x7fdef4167a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:20.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.436+0000 7fdef9a41700 1 -- 172.21.15.157:0/576472316 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee4009420 con 0x7fdef40c81b0 2023-10-06T04:01:20.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.436+0000 7fdef9240700 1 --2- 172.21.15.157:0/576472316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdef40c97d0 0x7fdef4167a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:01:20.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.437+0000 7fdef9a41700 1 --2- 172.21.15.157:0/576472316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdef40c81b0 0x7fdef4154e00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fdef000cd30 tx=0x7fdef0001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:20.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.437+0000 7fdee9ffb700 1 -- 172.21.15.157:0/576472316 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef000ce30 con 0x7fdef40c81b0 2023-10-06T04:01:20.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.437+0000 7fdee9ffb700 1 -- 172.21.15.157:0/576472316 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef0004bb0 con 0x7fdef40c81b0 2023-10-06T04:01:20.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.437+0000 7fdefbca5700 1 -- 172.21.15.157:0/576472316 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeec009420 con 0x7fdef40c81b0 2023-10-06T04:01:20.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.437+0000 7fdee9ffb700 1 -- 172.21.15.157:0/576472316 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef000ab50 con 0x7fdef40c81b0 2023-10-06T04:01:20.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.437+0000 7fdefbca5700 1 -- 172.21.15.157:0/576472316 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef4159bc0 con 0x7fdef40c81b0 2023-10-06T04:01:20.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.438+0000 7fdefbca5700 1 -- 172.21.15.157:0/576472316 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef4167880 con 0x7fdef40c81b0 2023-10-06T04:01:20.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.438+0000 7fdee9ffb700 1 -- 172.21.15.157:0/576472316 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdef00070f0 con 0x7fdef40c81b0 2023-10-06T04:01:20.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.439+0000 7fdee9ffb700 1 --2- 172.21.15.157:0/576472316 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fded8066f30 0x7fded80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:20.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.439+0000 7fdee9ffb700 1 -- 172.21.15.157:0/576472316 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7fdef00844d0 con 0x7fdef40c81b0 2023-10-06T04:01:20.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.439+0000 7fdef9240700 1 --2- 172.21.15.157:0/576472316 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fded8066f30 0x7fded80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:20.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.439+0000 7fdef9240700 1 --2- 172.21.15.157:0/576472316 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fded8066f30 0x7fded80693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdef40c93e0 tx=0x7fdee40091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:20.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.443+0000 7fdee9ffb700 1 -- 172.21.15.157:0/576472316 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdef0051580 con 0x7fdef40c81b0 2023-10-06T04:01:20.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.671+0000 7fdefbca5700 1 -- 172.21.15.157:0/576472316 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdef406fc70 con 0x7fdef40c81b0 2023-10-06T04:01:20.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.671+0000 7fdee9ffb700 1 -- 172.21.15.157:0/576472316 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+5439 (secure 0 0 0) 0x7fdef0054ba0 con 0x7fdef40c81b0 2023-10-06T04:01:20.673 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:01:20.673 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:01:19.720894+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"f","rank":4,"incarnation":29,"state":"up:rejoin","state_seq":30,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:01:20.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.674+0000 7fdefbca5700 1 -- 172.21.15.157:0/576472316 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fded8066f30 msgr2=0x7fded80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:20.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.674+0000 7fdefbca5700 1 --2- 172.21.15.157:0/576472316 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fded8066f30 0x7fded80693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdef40c93e0 tx=0x7fdee40091a0 comp rx=0 tx=0).stop 2023-10-06T04:01:20.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.674+0000 7fdefbca5700 1 -- 172.21.15.157:0/576472316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdef40c81b0 msgr2=0x7fdef4154e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:20.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.674+0000 7fdefbca5700 1 --2- 172.21.15.157:0/576472316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdef40c81b0 0x7fdef4154e00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fdef000cd30 tx=0x7fdef0001ad0 comp rx=0 tx=0).stop 2023-10-06T04:01:20.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.674+0000 7fdefbca5700 1 -- 172.21.15.157:0/576472316 shutdown_connections 2023-10-06T04:01:20.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.674+0000 7fdefbca5700 1 --2- 172.21.15.157:0/576472316 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fded8066f30 0x7fded80693e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:20.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.674+0000 7fdefbca5700 1 --2- 172.21.15.157:0/576472316 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdef4157490 0x7fdef4167f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:20.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.674+0000 7fdefbca5700 1 --2- 172.21.15.157:0/576472316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdef40c81b0 0x7fdef4154e00 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:20.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.674+0000 7fdefbca5700 1 --2- 172.21.15.157:0/576472316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdef40c97d0 0x7fdef4167a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:20.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.674+0000 7fdefbca5700 1 -- 172.21.15.157:0/576472316 >> 172.21.15.157:0/576472316 conn(0x7fdef40cd0b0 msgr2=0x7fdef40cb660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:20.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.674+0000 7fdefbca5700 1 -- 172.21.15.157:0/576472316 shutdown_connections 2023-10-06T04:01:20.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:20.674+0000 7fdefbca5700 1 -- 172.21.15.157:0/576472316 wait complete. 2023-10-06T04:01:20.677 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 32 2023-10-06T04:01:20.687 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:01:22.687 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:01:22.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.816+0000 7f90df59e700 1 Processor -- start 2023-10-06T04:01:22.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.816+0000 7f90df59e700 1 -- start start 2023-10-06T04:01:22.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.817+0000 7f90df59e700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90e00b1e40 0x7f90e0157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:22.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.817+0000 7f90df59e700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f90e00b04d0 0x7f90e006dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:22.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.817+0000 7f90df59e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e006d8d0 con 0x7f90e006c2d0 2023-10-06T04:01:22.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.817+0000 7f90df59e700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e00af8f0 con 0x7f90e00b04d0 2023-10-06T04:01:22.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.817+0000 7f90df59e700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e015a690 con 0x7f90e00b1e40 2023-10-06T04:01:22.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.817+0000 7f90d5d9b700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f90e00b04d0 0x7f90e006dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:22.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.817+0000 7f90d5d9b700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f90e00b04d0 0x7f90e006dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37754/0 (socket says 172.21.15.157:37754) 2023-10-06T04:01:22.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.817+0000 7f90d5d9b700 1 -- 172.21.15.157:0/3358031609 learned_addr learned my addr 172.21.15.157:0/3358031609 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:01:22.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.817+0000 7f90de59c700 1 --2- 172.21.15.157:0/3358031609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90e00b1e40 0x7f90e0157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:22.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.817+0000 7f90d5d9b700 1 -- 172.21.15.157:0/3358031609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90e00b1e40 msgr2=0x7f90e0157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:01:22.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.817+0000 7f90d5d9b700 1 --2- 172.21.15.157:0/3358031609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90e00b1e40 0x7f90e0157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:22.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.817+0000 7f90d5d9b700 1 -- 172.21.15.157:0/3358031609 >> v1:172.21.15.175:6789/0 conn(0x7f90e006c2d0 legacy=0x7f90e01563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:22.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.817+0000 7f90d5d9b700 1 -- 172.21.15.157:0/3358031609 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e006ccc0 con 0x7f90e00b04d0 2023-10-06T04:01:22.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.817+0000 7f90de59c700 1 --2- 172.21.15.157:0/3358031609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90e00b1e40 0x7f90e0157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:01:22.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.817+0000 7f90d5d9b700 1 --2- 172.21.15.157:0/3358031609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90e00b04d0 0x7f90e006dd60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f90c80099b0 tx=0x7f90c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4ddeaf857b3d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:22.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.818+0000 7f90ddd9b700 1 -- 172.21.15.157:0/3358031609 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c800b860 con 0x7f90e00b04d0 2023-10-06T04:01:22.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.818+0000 7f90ddd9b700 1 -- 172.21.15.157:0/3358031609 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c800b9c0 con 0x7f90e00b04d0 2023-10-06T04:01:22.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.818+0000 7f90ddd9b700 1 -- 172.21.15.157:0/3358031609 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c800bd80 con 0x7f90e00b04d0 2023-10-06T04:01:22.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.818+0000 7f90df59e700 1 -- 172.21.15.157:0/3358031609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90e00b04d0 msgr2=0x7f90e006dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:22.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.818+0000 7f90df59e700 1 --2- 172.21.15.157:0/3358031609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90e00b04d0 0x7f90e006dd60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f90c80099b0 tx=0x7f90c8009250 comp rx=0 tx=0).stop 2023-10-06T04:01:22.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.818+0000 7f90df59e700 1 -- 172.21.15.157:0/3358031609 shutdown_connections 2023-10-06T04:01:22.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.818+0000 7f90df59e700 1 --2- 172.21.15.157:0/3358031609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90e00b1e40 0x7f90e0157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:22.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.818+0000 7f90df59e700 1 --2- 172.21.15.157:0/3358031609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90e00b04d0 0x7f90e006dd60 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:22.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.818+0000 7f90df59e700 1 -- 172.21.15.157:0/3358031609 >> 172.21.15.157:0/3358031609 conn(0x7f90e00cc540 msgr2=0x7f90e00cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:22.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.818+0000 7f90df59e700 1 -- 172.21.15.157:0/3358031609 shutdown_connections 2023-10-06T04:01:22.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.818+0000 7f90df59e700 1 -- 172.21.15.157:0/3358031609 wait complete. 2023-10-06T04:01:22.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.819+0000 7f90df59e700 1 Processor -- start 2023-10-06T04:01:22.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.820+0000 7f90df59e700 1 -- start start 2023-10-06T04:01:22.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.820+0000 7f90df59e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90e006c2d0 0x7f90e014cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:22.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.820+0000 7f90df59e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90e00b04d0 0x7f90e0169210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:22.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.820+0000 7f90df59e700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90e00b1e40 0x7f90e00c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:22.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.820+0000 7f90df59e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e0152760 con 0x7f90e006c2d0 2023-10-06T04:01:22.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.820+0000 7f90df59e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e014d6f0 con 0x7f90e00b04d0 2023-10-06T04:01:22.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.820+0000 7f90df59e700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e014dc70 con 0x7f90e00b1e40 2023-10-06T04:01:22.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.820+0000 7f90de59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90e006c2d0 0x7f90e014cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:22.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.820+0000 7f90de59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90e006c2d0 0x7f90e014cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37760/0 (socket says 172.21.15.157:37760) 2023-10-06T04:01:22.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.820+0000 7f90de59c700 1 -- 172.21.15.157:0/1176589759 learned_addr learned my addr 172.21.15.157:0/1176589759 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:01:22.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.820+0000 7f90d5d9b700 1 --2- 172.21.15.157:0/1176589759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90e00b04d0 0x7f90e0169210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:22.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.820+0000 7f90ded9d700 1 --2- 172.21.15.157:0/1176589759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90e00b1e40 0x7f90e00c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:22.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.821+0000 7f90de59c700 1 -- 172.21.15.157:0/1176589759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90e00b1e40 msgr2=0x7f90e00c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:22.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.821+0000 7f90de59c700 1 --2- 172.21.15.157:0/1176589759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90e00b1e40 0x7f90e00c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:22.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.821+0000 7f90de59c700 1 -- 172.21.15.157:0/1176589759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90e00b04d0 msgr2=0x7f90e0169210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:22.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.821+0000 7f90de59c700 1 --2- 172.21.15.157:0/1176589759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90e00b04d0 0x7f90e0169210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:22.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.821+0000 7f90de59c700 1 -- 172.21.15.157:0/1176589759 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c80094f0 con 0x7f90e006c2d0 2023-10-06T04:01:22.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.821+0000 7f90d5d9b700 1 --2- 172.21.15.157:0/1176589759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90e00b04d0 0x7f90e0169210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:01:22.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.821+0000 7f90ded9d700 1 --2- 172.21.15.157:0/1176589759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90e00b1e40 0x7f90e00c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:01:22.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.821+0000 7f90de59c700 1 --2- 172.21.15.157:0/1176589759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90e006c2d0 0x7f90e014cf20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f90d000e8d0 tx=0x7f90d0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:22.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.821+0000 7f90d6ffd700 1 -- 172.21.15.157:0/1176589759 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d000ee70 con 0x7f90e006c2d0 2023-10-06T04:01:22.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.821+0000 7f90d6ffd700 1 -- 172.21.15.157:0/1176589759 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d000c670 con 0x7f90e006c2d0 2023-10-06T04:01:22.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.821+0000 7f90df59e700 1 -- 172.21.15.157:0/1176589759 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e00cbeb0 con 0x7f90e006c2d0 2023-10-06T04:01:22.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.821+0000 7f90d6ffd700 1 -- 172.21.15.157:0/1176589759 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d000ca10 con 0x7f90e006c2d0 2023-10-06T04:01:22.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.821+0000 7f90df59e700 1 -- 172.21.15.157:0/1176589759 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e014d460 con 0x7f90e006c2d0 2023-10-06T04:01:22.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.822+0000 7f90df59e700 1 -- 172.21.15.157:0/1176589759 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e0088f90 con 0x7f90e006c2d0 2023-10-06T04:01:22.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.822+0000 7f90d6ffd700 1 -- 172.21.15.157:0/1176589759 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f90d0013410 con 0x7f90e006c2d0 2023-10-06T04:01:22.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.823+0000 7f90d6ffd700 1 --2- 172.21.15.157:0/1176589759 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f90c0066f30 0x7f90c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:22.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.823+0000 7f90d6ffd700 1 -- 172.21.15.157:0/1176589759 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f90d0084640 con 0x7f90e006c2d0 2023-10-06T04:01:22.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.823+0000 7f90d5d9b700 1 --2- 172.21.15.157:0/1176589759 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f90c0066f30 0x7f90c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:22.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.823+0000 7f90d5d9b700 1 --2- 172.21.15.157:0/1176589759 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f90c0066f30 0x7f90c00693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f90c80099b0 tx=0x7f90c8017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:22.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:22.827+0000 7f90d6ffd700 1 -- 172.21.15.157:0/1176589759 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90d0051670 con 0x7f90e006c2d0 2023-10-06T04:01:23.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:23.051+0000 7f90df59e700 1 -- 172.21.15.157:0/1176589759 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90e006cfa0 con 0x7f90e006c2d0 2023-10-06T04:01:23.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:23.052+0000 7f90d6ffd700 1 -- 172.21.15.157:0/1176589759 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+5439 (secure 0 0 0) 0x7f90d0054c90 con 0x7f90e006c2d0 2023-10-06T04:01:23.053 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:01:23.053 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:01:19.720894+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"f","rank":4,"incarnation":29,"state":"up:rejoin","state_seq":30,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:01:23.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:23.054+0000 7f90df59e700 1 -- 172.21.15.157:0/1176589759 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f90c0066f30 msgr2=0x7f90c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:23.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:23.054+0000 7f90df59e700 1 --2- 172.21.15.157:0/1176589759 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f90c0066f30 0x7f90c00693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f90c80099b0 tx=0x7f90c8017000 comp rx=0 tx=0).stop 2023-10-06T04:01:23.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:23.054+0000 7f90df59e700 1 -- 172.21.15.157:0/1176589759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90e006c2d0 msgr2=0x7f90e014cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:23.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:23.054+0000 7f90df59e700 1 --2- 172.21.15.157:0/1176589759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90e006c2d0 0x7f90e014cf20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f90d000e8d0 tx=0x7f90d0008040 comp rx=0 tx=0).stop 2023-10-06T04:01:23.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:23.054+0000 7f90df59e700 1 -- 172.21.15.157:0/1176589759 shutdown_connections 2023-10-06T04:01:23.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:23.054+0000 7f90df59e700 1 --2- 172.21.15.157:0/1176589759 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f90c0066f30 0x7f90c00693e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:23.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:23.054+0000 7f90df59e700 1 --2- 172.21.15.157:0/1176589759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90e00b1e40 0x7f90e00c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:23.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:23.054+0000 7f90df59e700 1 --2- 172.21.15.157:0/1176589759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90e006c2d0 0x7f90e014cf20 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:23.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:23.054+0000 7f90df59e700 1 --2- 172.21.15.157:0/1176589759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90e00b04d0 0x7f90e0169210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:23.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:23.054+0000 7f90df59e700 1 -- 172.21.15.157:0/1176589759 >> 172.21.15.157:0/1176589759 conn(0x7f90e00cc540 msgr2=0x7f90e00c54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:23.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:23.054+0000 7f90df59e700 1 -- 172.21.15.157:0/1176589759 shutdown_connections 2023-10-06T04:01:23.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:23.054+0000 7f90df59e700 1 -- 172.21.15.157:0/1176589759 wait complete. 2023-10-06T04:01:23.057 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 32 2023-10-06T04:01:23.067 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:01:25.068 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:01:25.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.192+0000 7fd98c58b700 1 Processor -- start 2023-10-06T04:01:25.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.192+0000 7fd98c58b700 1 -- start start 2023-10-06T04:01:25.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.193+0000 7fd98c58b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9840ca630 0x7fd9840c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:25.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.193+0000 7fd98c58b700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd984156f50 0x7fd9841608c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:25.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.193+0000 7fd98c58b700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd98406d530 con 0x7fd984068920 2023-10-06T04:01:25.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.193+0000 7fd98c58b700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd98406d9c0 con 0x7fd984156f50 2023-10-06T04:01:25.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.193+0000 7fd98c58b700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd98406f0c0 con 0x7fd9840ca630 2023-10-06T04:01:25.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.193+0000 7fd989b26700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd984068920 0x7fd984156840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:52926/0 (socket says 172.21.15.157:52926) 2023-10-06T04:01:25.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.193+0000 7fd989b26700 1 -- 172.21.15.157:0/668439491 learned_addr learned my addr 172.21.15.157:0/668439491 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:01:25.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.194+0000 7fd98ab28700 1 --2- 172.21.15.157:0/668439491 >> v2:172.21.15.175:3300/0 conn(0x7fd984156f50 0x7fd9841608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:25.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.194+0000 7fd98a327700 1 --2- 172.21.15.157:0/668439491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9840ca630 0x7fd9840c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:25.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.194+0000 7fd989325700 1 -- 172.21.15.157:0/668439491 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1678900233 0 0) 0x7fd98406d530 con 0x7fd984068920 2023-10-06T04:01:25.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.194+0000 7fd989325700 1 -- 172.21.15.157:0/668439491 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd970003570 con 0x7fd984068920 2023-10-06T04:01:25.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.194+0000 7fd989325700 1 -- 172.21.15.157:0/668439491 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3114785135 0 0) 0x7fd970003570 con 0x7fd984068920 2023-10-06T04:01:25.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.194+0000 7fd989325700 1 -- 172.21.15.157:0/668439491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9840ca630 msgr2=0x7fd9840c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:01:25.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.194+0000 7fd989325700 1 --2- 172.21.15.157:0/668439491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9840ca630 0x7fd9840c6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:25.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.194+0000 7fd989325700 1 -- 172.21.15.157:0/668439491 >> v2:172.21.15.175:3300/0 conn(0x7fd984156f50 msgr2=0x7fd9841608c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:01:25.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.194+0000 7fd989325700 1 --2- 172.21.15.157:0/668439491 >> v2:172.21.15.175:3300/0 conn(0x7fd984156f50 0x7fd9841608c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:25.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.194+0000 7fd989325700 1 -- 172.21.15.157:0/668439491 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98406f3a0 con 0x7fd984068920 2023-10-06T04:01:25.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.194+0000 7fd989325700 1 -- 172.21.15.157:0/668439491 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7fd974001470 con 0x7fd984068920 2023-10-06T04:01:25.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.194+0000 7fd989325700 1 -- 172.21.15.157:0/668439491 >> v1:172.21.15.157:6789/0 conn(0x7fd984068920 legacy=0x7fd984156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:25.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.194+0000 7fd989325700 1 --2- 172.21.15.157:0/668439491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9840ca630 0x7fd970003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:25.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.194+0000 7fd989325700 1 --2- 172.21.15.157:0/668439491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd970003fe0 0x7fd970004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:25.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.194+0000 7fd989325700 1 --2- 172.21.15.157:0/668439491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd970004d50 0x7fd9700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:25.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.194+0000 7fd989325700 1 -- 172.21.15.157:0/668439491 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98406d9c0 con 0x7fd970004d50 2023-10-06T04:01:25.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.194+0000 7fd989325700 1 -- 172.21.15.157:0/668439491 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98406f0c0 con 0x7fd970003fe0 2023-10-06T04:01:25.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.194+0000 7fd989325700 1 -- 172.21.15.157:0/668439491 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97000ba90 con 0x7fd9840ca630 2023-10-06T04:01:25.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.194+0000 7fd98ab28700 1 --2- 172.21.15.157:0/668439491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd970004d50 0x7fd9700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:25.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.194+0000 7fd98a327700 1 --2- 172.21.15.157:0/668439491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9840ca630 0x7fd970003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:25.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.194+0000 7fd989b26700 1 --2- 172.21.15.157:0/668439491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd970003fe0 0x7fd970004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:25.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.195+0000 7fd98ab28700 1 -- 172.21.15.157:0/668439491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9840ca630 msgr2=0x7fd970003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:01:25.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.195+0000 7fd98ab28700 1 --2- 172.21.15.157:0/668439491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9840ca630 0x7fd970003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:25.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.195+0000 7fd98ab28700 1 -- 172.21.15.157:0/668439491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd970003fe0 msgr2=0x7fd970004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:01:25.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.195+0000 7fd98ab28700 1 --2- 172.21.15.157:0/668439491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd970003fe0 0x7fd970004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:25.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.195+0000 7fd98ab28700 1 -- 172.21.15.157:0/668439491 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd970003570 con 0x7fd970004d50 2023-10-06T04:01:25.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.195+0000 7fd989b26700 1 --2- 172.21.15.157:0/668439491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd970003fe0 0x7fd970004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:01:25.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.195+0000 7fd98ab28700 1 --2- 172.21.15.157:0/668439491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd970004d50 0x7fd9700051c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd984157350 tx=0x7fd97c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0445ea9f05f1e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:25.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.195+0000 7fd989325700 1 -- 172.21.15.157:0/668439491 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c00ea30 con 0x7fd970004d50 2023-10-06T04:01:25.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.195+0000 7fd989325700 1 -- 172.21.15.157:0/668439491 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97c00f040 con 0x7fd970004d50 2023-10-06T04:01:25.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.195+0000 7fd989325700 1 -- 172.21.15.157:0/668439491 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c00ede0 con 0x7fd970004d50 2023-10-06T04:01:25.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.195+0000 7fd98c58b700 1 -- 172.21.15.157:0/668439491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd970004d50 msgr2=0x7fd9700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:25.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.195+0000 7fd98c58b700 1 --2- 172.21.15.157:0/668439491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd970004d50 0x7fd9700051c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd984157350 tx=0x7fd97c00b040 comp rx=0 tx=0).stop 2023-10-06T04:01:25.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.195+0000 7fd98ab28700 1 -- 172.21.15.157:0/668439491 reap_dead start 2023-10-06T04:01:25.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.195+0000 7fd98c58b700 1 -- 172.21.15.157:0/668439491 shutdown_connections 2023-10-06T04:01:25.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.196+0000 7fd98c58b700 1 -- 172.21.15.157:0/668439491 >> 172.21.15.157:0/668439491 conn(0x7fd9840692b0 msgr2=0x7fd9840696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:25.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.196+0000 7fd98c58b700 1 -- 172.21.15.157:0/668439491 shutdown_connections 2023-10-06T04:01:25.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.196+0000 7fd98c58b700 1 -- 172.21.15.157:0/668439491 wait complete. 2023-10-06T04:01:25.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.197+0000 7fd98c58b700 1 Processor -- start 2023-10-06T04:01:25.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.197+0000 7fd98c58b700 1 -- start start 2023-10-06T04:01:25.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.197+0000 7fd98c58b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd970004d50 0x7fd98415b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:25.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.197+0000 7fd98c58b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9840c3970 0x7fd984157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:25.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.197+0000 7fd98c58b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9840c3dc0 0x7fd984158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:25.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.197+0000 7fd98c58b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd984169ee0 con 0x7fd970004d50 2023-10-06T04:01:25.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.197+0000 7fd98c58b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9841699b0 con 0x7fd9840c3dc0 2023-10-06T04:01:25.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.197+0000 7fd98c58b700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd984157850 con 0x7fd9840c3970 2023-10-06T04:01:25.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.197+0000 7fd98a327700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd970004d50 0x7fd98415b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:25.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.197+0000 7fd98a327700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd970004d50 0x7fd98415b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46232/0 (socket says 172.21.15.157:46232) 2023-10-06T04:01:25.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.197+0000 7fd98a327700 1 -- 172.21.15.157:0/478056108 learned_addr learned my addr 172.21.15.157:0/478056108 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:01:25.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.197+0000 7fd989b26700 1 --2- 172.21.15.157:0/478056108 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9840c3970 0x7fd984157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:25.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.198+0000 7fd98ab28700 1 --2- 172.21.15.157:0/478056108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9840c3dc0 0x7fd984158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:25.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.198+0000 7fd98a327700 1 -- 172.21.15.157:0/478056108 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9840c3970 msgr2=0x7fd984157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:25.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.198+0000 7fd98a327700 1 --2- 172.21.15.157:0/478056108 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9840c3970 0x7fd984157ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:25.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.198+0000 7fd98a327700 1 -- 172.21.15.157:0/478056108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9840c3dc0 msgr2=0x7fd984158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:25.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.198+0000 7fd98a327700 1 --2- 172.21.15.157:0/478056108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9840c3dc0 0x7fd984158220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:25.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.198+0000 7fd98a327700 1 -- 172.21.15.157:0/478056108 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd984157520 con 0x7fd970004d50 2023-10-06T04:01:25.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.198+0000 7fd98a327700 1 --2- 172.21.15.157:0/478056108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd970004d50 0x7fd98415b9e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd980011b10 tx=0x7fd9800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:25.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.198+0000 7fd97a7fc700 1 -- 172.21.15.157:0/478056108 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98000f940 con 0x7fd970004d50 2023-10-06T04:01:25.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.198+0000 7fd97a7fc700 1 -- 172.21.15.157:0/478056108 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98000faa0 con 0x7fd970004d50 2023-10-06T04:01:25.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.198+0000 7fd98c58b700 1 -- 172.21.15.157:0/478056108 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd984158a70 con 0x7fd970004d50 2023-10-06T04:01:25.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.198+0000 7fd97a7fc700 1 -- 172.21.15.157:0/478056108 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98000fe20 con 0x7fd970004d50 2023-10-06T04:01:25.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.198+0000 7fd98c58b700 1 -- 172.21.15.157:0/478056108 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd984169c00 con 0x7fd970004d50 2023-10-06T04:01:25.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.199+0000 7fd98c58b700 1 -- 172.21.15.157:0/478056108 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd984157380 con 0x7fd970004d50 2023-10-06T04:01:25.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.199+0000 7fd97a7fc700 1 -- 172.21.15.157:0/478056108 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd980014650 con 0x7fd970004d50 2023-10-06T04:01:25.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.200+0000 7fd97a7fc700 1 --2- 172.21.15.157:0/478056108 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd968066f60 0x7fd968069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:25.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.200+0000 7fd97a7fc700 1 -- 172.21.15.157:0/478056108 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7fd980018030 con 0x7fd970004d50 2023-10-06T04:01:25.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.200+0000 7fd989b26700 1 --2- 172.21.15.157:0/478056108 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd968066f60 0x7fd968069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:25.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.200+0000 7fd989b26700 1 --2- 172.21.15.157:0/478056108 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd968066f60 0x7fd968069410 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd974001d20 tx=0x7fd974001e90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:25.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.204+0000 7fd97a7fc700 1 -- 172.21.15.157:0/478056108 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd984157380 con 0x7fd970004d50 2023-10-06T04:01:25.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.428+0000 7fd98c58b700 1 -- 172.21.15.157:0/478056108 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd984157380 con 0x7fd970004d50 2023-10-06T04:01:25.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.429+0000 7fd97a7fc700 1 -- 172.21.15.157:0/478056108 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+5439 (secure 0 0 0) 0x7fd984157380 con 0x7fd970004d50 2023-10-06T04:01:25.431 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:01:25.431 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:01:19.720894+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"f","rank":4,"incarnation":29,"state":"up:rejoin","state_seq":30,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:01:25.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.430+0000 7fd98c58b700 1 -- 172.21.15.157:0/478056108 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd968066f60 msgr2=0x7fd968069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:25.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.430+0000 7fd98c58b700 1 --2- 172.21.15.157:0/478056108 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd968066f60 0x7fd968069410 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd974001d20 tx=0x7fd974001e90 comp rx=0 tx=0).stop 2023-10-06T04:01:25.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.430+0000 7fd98c58b700 1 -- 172.21.15.157:0/478056108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd970004d50 msgr2=0x7fd98415b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:25.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.430+0000 7fd98c58b700 1 --2- 172.21.15.157:0/478056108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd970004d50 0x7fd98415b9e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd980011b10 tx=0x7fd9800091a0 comp rx=0 tx=0).stop 2023-10-06T04:01:25.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.430+0000 7fd98c58b700 1 -- 172.21.15.157:0/478056108 shutdown_connections 2023-10-06T04:01:25.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.430+0000 7fd98c58b700 1 --2- 172.21.15.157:0/478056108 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd968066f60 0x7fd968069410 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:25.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.430+0000 7fd98c58b700 1 --2- 172.21.15.157:0/478056108 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9840c3dc0 0x7fd984158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:25.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.430+0000 7fd98c58b700 1 --2- 172.21.15.157:0/478056108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd970004d50 0x7fd98415b9e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:25.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.431+0000 7fd98c58b700 1 --2- 172.21.15.157:0/478056108 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9840c3970 0x7fd984157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:25.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.431+0000 7fd98c58b700 1 -- 172.21.15.157:0/478056108 >> 172.21.15.157:0/478056108 conn(0x7fd9840692b0 msgr2=0x7fd984153ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:25.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.431+0000 7fd98c58b700 1 -- 172.21.15.157:0/478056108 shutdown_connections 2023-10-06T04:01:25.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:25.431+0000 7fd98c58b700 1 -- 172.21.15.157:0/478056108 wait complete. 2023-10-06T04:01:25.434 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 32 2023-10-06T04:01:25.444 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:01:27.445 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:01:27.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.572+0000 7f35490b5700 1 Processor -- start 2023-10-06T04:01:27.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.572+0000 7f35490b5700 1 -- start start 2023-10-06T04:01:27.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.573+0000 7f35490b5700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f354406cbb0 0x7f354406d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:27.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.573+0000 7f35490b5700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35440c9780 con 0x7f3544150410 2023-10-06T04:01:27.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.573+0000 7f35490b5700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f354406e0c0 con 0x7f354406c1f0 2023-10-06T04:01:27.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.573+0000 7f35490b5700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35440c8c40 con 0x7f354406cbb0 2023-10-06T04:01:27.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.573+0000 7f3542d9d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f354406cbb0 0x7f354406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:27.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.573+0000 7f354359e700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f3544150410 0x7f3544155f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:52936/0 (socket says 172.21.15.157:52936) 2023-10-06T04:01:27.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.573+0000 7f354359e700 1 -- 172.21.15.157:0/2393210753 learned_addr learned my addr 172.21.15.157:0/2393210753 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:01:27.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.573+0000 7f3542d9d700 1 -- 172.21.15.157:0/2393210753 >> v1:172.21.15.175:6789/0 conn(0x7f354406c1f0 legacy=0x7f354414fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:01:27.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.573+0000 7f3542d9d700 1 -- 172.21.15.157:0/2393210753 >> v1:172.21.15.157:6789/0 conn(0x7f3544150410 legacy=0x7f3544155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:27.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.573+0000 7f3542d9d700 1 -- 172.21.15.157:0/2393210753 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35440c8e50 con 0x7f354406cbb0 2023-10-06T04:01:27.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.573+0000 7f3542d9d700 1 --2- 172.21.15.157:0/2393210753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f354406cbb0 0x7f354406d530 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f35340098e0 tx=0x7f35340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b0487d7810ed98 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:27.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.573+0000 7f354259c700 1 -- 172.21.15.157:0/2393210753 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3534009e80 con 0x7f354406cbb0 2023-10-06T04:01:27.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.573+0000 7f354259c700 1 -- 172.21.15.157:0/2393210753 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f353400b8b0 con 0x7f354406cbb0 2023-10-06T04:01:27.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.573+0000 7f354259c700 1 -- 172.21.15.157:0/2393210753 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353400bc30 con 0x7f354406cbb0 2023-10-06T04:01:27.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.574+0000 7f35490b5700 1 -- 172.21.15.157:0/2393210753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f354406cbb0 msgr2=0x7f354406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:27.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.574+0000 7f35490b5700 1 --2- 172.21.15.157:0/2393210753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f354406cbb0 0x7f354406d530 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f35340098e0 tx=0x7f35340091a0 comp rx=0 tx=0).stop 2023-10-06T04:01:27.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.574+0000 7f35490b5700 1 -- 172.21.15.157:0/2393210753 shutdown_connections 2023-10-06T04:01:27.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.574+0000 7f35490b5700 1 --2- 172.21.15.157:0/2393210753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f354406cbb0 0x7f354406d530 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:27.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.574+0000 7f35490b5700 1 -- 172.21.15.157:0/2393210753 >> 172.21.15.157:0/2393210753 conn(0x7f354406f230 msgr2=0x7f354406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:27.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.574+0000 7f35490b5700 1 -- 172.21.15.157:0/2393210753 shutdown_connections 2023-10-06T04:01:27.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.574+0000 7f35490b5700 1 -- 172.21.15.157:0/2393210753 wait complete. 2023-10-06T04:01:27.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.575+0000 7f35490b5700 1 Processor -- start 2023-10-06T04:01:27.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.575+0000 7f35490b5700 1 -- start start 2023-10-06T04:01:27.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.576+0000 7f35490b5700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f354406c1f0 0x7f3544167390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:27.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.576+0000 7f35490b5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f354406cbb0 0x7f35440c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:27.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.576+0000 7f35490b5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3544150410 0x7f35440c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:27.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.576+0000 7f35490b5700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35441678d0 con 0x7f3544150410 2023-10-06T04:01:27.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.576+0000 7f35490b5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35440c4b90 con 0x7f354406cbb0 2023-10-06T04:01:27.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.576+0000 7f35490b5700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35440c60b0 con 0x7f354406c1f0 2023-10-06T04:01:27.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.576+0000 7f354359e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3544150410 0x7f35440c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:27.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.576+0000 7f354359e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3544150410 0x7f35440c5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46262/0 (socket says 172.21.15.157:46262) 2023-10-06T04:01:27.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.576+0000 7f354359e700 1 -- 172.21.15.157:0/1647341698 learned_addr learned my addr 172.21.15.157:0/1647341698 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:01:27.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.576+0000 7f353bfff700 1 --2- 172.21.15.157:0/1647341698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f354406cbb0 0x7f35440c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:27.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.576+0000 7f3542d9d700 1 --2- 172.21.15.157:0/1647341698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f354406c1f0 0x7f3544167390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:27.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.576+0000 7f354359e700 1 -- 172.21.15.157:0/1647341698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f354406c1f0 msgr2=0x7f3544167390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:27.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.576+0000 7f354359e700 1 --2- 172.21.15.157:0/1647341698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f354406c1f0 0x7f3544167390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:27.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.576+0000 7f354359e700 1 -- 172.21.15.157:0/1647341698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f354406cbb0 msgr2=0x7f35440c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:27.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.576+0000 7f354359e700 1 --2- 172.21.15.157:0/1647341698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f354406cbb0 0x7f35440c3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:27.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.576+0000 7f354359e700 1 -- 172.21.15.157:0/1647341698 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3534009420 con 0x7f3544150410 2023-10-06T04:01:27.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.576+0000 7f3542d9d700 1 --2- 172.21.15.157:0/1647341698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f354406c1f0 0x7f3544167390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:01:27.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.576+0000 7f353bfff700 1 --2- 172.21.15.157:0/1647341698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f354406cbb0 0x7f35440c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:01:27.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.576+0000 7f354359e700 1 --2- 172.21.15.157:0/1647341698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3544150410 0x7f35440c5020 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f353c00cbc0 tx=0x7f353c0019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:27.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.576+0000 7f353affd700 1 -- 172.21.15.157:0/1647341698 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c00a9d0 con 0x7f3544150410 2023-10-06T04:01:27.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.577+0000 7f353affd700 1 -- 172.21.15.157:0/1647341698 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f353c00ab30 con 0x7f3544150410 2023-10-06T04:01:27.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.577+0000 7f35490b5700 1 -- 172.21.15.157:0/1647341698 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f354414e670 con 0x7f3544150410 2023-10-06T04:01:27.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.577+0000 7f353affd700 1 -- 172.21.15.157:0/1647341698 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c0070f0 con 0x7f3544150410 2023-10-06T04:01:27.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.577+0000 7f35490b5700 1 -- 172.21.15.157:0/1647341698 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35440c6630 con 0x7f3544150410 2023-10-06T04:01:27.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.577+0000 7f35490b5700 1 -- 172.21.15.157:0/1647341698 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35440c9c60 con 0x7f3544150410 2023-10-06T04:01:27.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.578+0000 7f353affd700 1 -- 172.21.15.157:0/1647341698 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f353c0074a0 con 0x7f3544150410 2023-10-06T04:01:27.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.578+0000 7f353affd700 1 --2- 172.21.15.157:0/1647341698 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3524066f60 0x7f3524069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:27.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.578+0000 7f353affd700 1 -- 172.21.15.157:0/1647341698 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f353c084c20 con 0x7f3544150410 2023-10-06T04:01:27.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.579+0000 7f3542d9d700 1 --2- 172.21.15.157:0/1647341698 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3524066f60 0x7f3524069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:27.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.579+0000 7f3542d9d700 1 --2- 172.21.15.157:0/1647341698 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3524066f60 0x7f3524069410 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f35340030c0 tx=0x7f3534016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:27.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.583+0000 7f353affd700 1 -- 172.21.15.157:0/1647341698 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f353c051c40 con 0x7f3544150410 2023-10-06T04:01:27.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.811+0000 7f35490b5700 1 -- 172.21.15.157:0/1647341698 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35440c4cd0 con 0x7f3544150410 2023-10-06T04:01:27.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.811+0000 7f353affd700 1 -- 172.21.15.157:0/1647341698 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+5439 (secure 0 0 0) 0x7f353c055260 con 0x7f3544150410 2023-10-06T04:01:27.813 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:01:27.813 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:01:25.737271+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"f","rank":4,"incarnation":29,"state":"up:active","state_seq":32,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:01:27.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.813+0000 7f35490b5700 1 -- 172.21.15.157:0/1647341698 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3524066f60 msgr2=0x7f3524069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:27.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.814+0000 7f35490b5700 1 --2- 172.21.15.157:0/1647341698 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3524066f60 0x7f3524069410 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f35340030c0 tx=0x7f3534016000 comp rx=0 tx=0).stop 2023-10-06T04:01:27.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.814+0000 7f35490b5700 1 -- 172.21.15.157:0/1647341698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3544150410 msgr2=0x7f35440c5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:27.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.814+0000 7f35490b5700 1 --2- 172.21.15.157:0/1647341698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3544150410 0x7f35440c5020 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f353c00cbc0 tx=0x7f353c0019c0 comp rx=0 tx=0).stop 2023-10-06T04:01:27.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.814+0000 7f35490b5700 1 -- 172.21.15.157:0/1647341698 shutdown_connections 2023-10-06T04:01:27.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.814+0000 7f35490b5700 1 --2- 172.21.15.157:0/1647341698 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3524066f60 0x7f3524069410 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:27.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.814+0000 7f35490b5700 1 --2- 172.21.15.157:0/1647341698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3544150410 0x7f35440c5020 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:27.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.814+0000 7f35490b5700 1 --2- 172.21.15.157:0/1647341698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f354406c1f0 0x7f3544167390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:27.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.814+0000 7f35490b5700 1 --2- 172.21.15.157:0/1647341698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f354406cbb0 0x7f35440c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:27.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.814+0000 7f35490b5700 1 -- 172.21.15.157:0/1647341698 >> 172.21.15.157:0/1647341698 conn(0x7f354406f230 msgr2=0x7f354414b600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:27.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.814+0000 7f35490b5700 1 -- 172.21.15.157:0/1647341698 shutdown_connections 2023-10-06T04:01:27.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:27.814+0000 7f35490b5700 1 -- 172.21.15.157:0/1647341698 wait complete. 2023-10-06T04:01:27.817 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 33 2023-10-06T04:01:27.827 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f has gained rank=4, replacing gid=4223 2023-10-06T04:01:27.827 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 5 secs before reviving mds.a 2023-10-06T04:01:32.828 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2023-10-06T04:01:32.828 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T04:01:32.829 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T04:01:32.832 INFO:tasks.ceph.mds.a:Started 2023-10-06T04:01:32.832 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:01:32.914 INFO:tasks.ceph.mds.a.smithi157.stdout:starting mds.a at 2023-10-06T04:01:32.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.959+0000 7f141e069700 1 Processor -- start 2023-10-06T04:01:32.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.960+0000 7f141e069700 1 -- start start 2023-10-06T04:01:32.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.960+0000 7f141e069700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1418068bc0 0x7f141806a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:32.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.960+0000 7f141e069700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1418156760 con 0x7f14180c2cb0 2023-10-06T04:01:32.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.960+0000 7f141e069700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1418159e40 con 0x7f14180bc850 2023-10-06T04:01:32.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.960+0000 7f141e069700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f141815a3c0 con 0x7f1418068bc0 2023-10-06T04:01:32.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.960+0000 7f14177fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f14180c2cb0 0x7f141806c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:52948/0 (socket says 172.21.15.157:52948) 2023-10-06T04:01:32.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.960+0000 7f14177fe700 1 -- 172.21.15.157:0/1405698023 learned_addr learned my addr 172.21.15.157:0/1405698023 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:01:32.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.961+0000 7f1416ffd700 1 --2- 172.21.15.157:0/1405698023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1418068bc0 0x7f141806a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:32.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.961+0000 7f14167fc700 1 -- 172.21.15.157:0/1405698023 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 722584768 0 0) 0x7f1418156760 con 0x7f14180c2cb0 2023-10-06T04:01:32.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.961+0000 7f14167fc700 1 -- 172.21.15.157:0/1405698023 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1400003570 con 0x7f14180c2cb0 2023-10-06T04:01:32.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.961+0000 7f14167fc700 1 -- 172.21.15.157:0/1405698023 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2416356821 0 0) 0x7f1418159e40 con 0x7f14180bc850 2023-10-06T04:01:32.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.961+0000 7f14167fc700 1 -- 172.21.15.157:0/1405698023 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1418156760 con 0x7f14180bc850 2023-10-06T04:01:32.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.961+0000 7f14167fc700 1 -- 172.21.15.157:0/1405698023 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3304714995 0 0) 0x7f1400003570 con 0x7f14180c2cb0 2023-10-06T04:01:32.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.961+0000 7f14167fc700 1 -- 172.21.15.157:0/1405698023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1418068bc0 msgr2=0x7f141806a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:01:32.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.961+0000 7f14167fc700 1 --2- 172.21.15.157:0/1405698023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1418068bc0 0x7f141806a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:32.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.961+0000 7f14167fc700 1 -- 172.21.15.157:0/1405698023 >> v1:172.21.15.175:6789/0 conn(0x7f14180bc850 legacy=0x7f14181607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:32.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.961+0000 7f14167fc700 1 -- 172.21.15.157:0/1405698023 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141815a0e0 con 0x7f14180c2cb0 2023-10-06T04:01:32.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.961+0000 7f14167fc700 1 -- 172.21.15.157:0/1405698023 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f1408001470 con 0x7f14180c2cb0 2023-10-06T04:01:32.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.961+0000 7f14167fc700 1 -- 172.21.15.157:0/1405698023 >> v1:172.21.15.157:6789/0 conn(0x7f14180c2cb0 legacy=0x7f141806c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:32.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.961+0000 7f14167fc700 1 --2- 172.21.15.157:0/1405698023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1400003c20 0x7f1400004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:32.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.961+0000 7f14167fc700 1 --2- 172.21.15.157:0/1405698023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1418068bc0 0x7f1400004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:32.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.961+0000 7f14167fc700 1 --2- 172.21.15.157:0/1405698023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1400004d50 0x7f14000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:32.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.961+0000 7f14167fc700 1 -- 172.21.15.157:0/1405698023 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141815a3c0 con 0x7f1400003c20 2023-10-06T04:01:32.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.961+0000 7f14167fc700 1 -- 172.21.15.157:0/1405698023 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140000bbf0 con 0x7f1400004d50 2023-10-06T04:01:32.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.961+0000 7f14167fc700 1 -- 172.21.15.157:0/1405698023 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f140000bd60 con 0x7f1418068bc0 2023-10-06T04:01:32.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.961+0000 7f14177fe700 1 --2- 172.21.15.157:0/1405698023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1400003c20 0x7f1400004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:32.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.961+0000 7f1417fff700 1 --2- 172.21.15.157:0/1405698023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1400004d50 0x7f14000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:32.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.961+0000 7f1416ffd700 1 --2- 172.21.15.157:0/1405698023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1418068bc0 0x7f1400004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:32.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.962+0000 7f14177fe700 1 -- 172.21.15.157:0/1405698023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1418068bc0 msgr2=0x7f1400004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:01:32.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.962+0000 7f14177fe700 1 --2- 172.21.15.157:0/1405698023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1418068bc0 0x7f1400004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:32.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.962+0000 7f14177fe700 1 -- 172.21.15.157:0/1405698023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1400004d50 msgr2=0x7f14000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:01:32.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.962+0000 7f14177fe700 1 --2- 172.21.15.157:0/1405698023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1400004d50 0x7f14000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:32.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.962+0000 7f14177fe700 1 -- 172.21.15.157:0/1405698023 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1400003570 con 0x7f1400003c20 2023-10-06T04:01:32.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.962+0000 7f14177fe700 1 --2- 172.21.15.157:0/1405698023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1400003c20 0x7f1400004010 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1408007fd0 tx=0x7f141806e790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5cdba3633c53377 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:32.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.962+0000 7f14167fc700 1 -- 172.21.15.157:0/1405698023 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140800cae0 con 0x7f1400003c20 2023-10-06T04:01:32.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.962+0000 7f14167fc700 1 -- 172.21.15.157:0/1405698023 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140800cc60 con 0x7f1400003c20 2023-10-06T04:01:32.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.962+0000 7f14167fc700 1 -- 172.21.15.157:0/1405698023 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1408005920 con 0x7f1400003c20 2023-10-06T04:01:32.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.962+0000 7f141e069700 1 -- 172.21.15.157:0/1405698023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1400003c20 msgr2=0x7f1400004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:32.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.962+0000 7f141e069700 1 --2- 172.21.15.157:0/1405698023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1400003c20 0x7f1400004010 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1408007fd0 tx=0x7f141806e790 comp rx=0 tx=0).stop 2023-10-06T04:01:32.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.962+0000 7f1417fff700 1 -- 172.21.15.157:0/1405698023 reap_dead start 2023-10-06T04:01:32.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.962+0000 7f141e069700 1 -- 172.21.15.157:0/1405698023 shutdown_connections 2023-10-06T04:01:32.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.962+0000 7f141e069700 1 -- 172.21.15.157:0/1405698023 >> 172.21.15.157:0/1405698023 conn(0x7f141806b510 msgr2=0x7f141806b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:32.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.962+0000 7f141e069700 1 -- 172.21.15.157:0/1405698023 shutdown_connections 2023-10-06T04:01:32.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.962+0000 7f141e069700 1 -- 172.21.15.157:0/1405698023 wait complete. 2023-10-06T04:01:32.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.964+0000 7f141e069700 1 Processor -- start 2023-10-06T04:01:32.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.964+0000 7f141e069700 1 -- start start 2023-10-06T04:01:32.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.964+0000 7f141e069700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1400003c20 0x7f1418165e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:32.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.964+0000 7f141e069700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14180bd710 0x7f1418158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:32.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.964+0000 7f141e069700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1418159500 0x7f141814ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:32.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.964+0000 7f141e069700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14180c5470 con 0x7f1418159500 2023-10-06T04:01:32.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.964+0000 7f141e069700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14180bccc0 con 0x7f14180bd710 2023-10-06T04:01:32.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.964+0000 7f141e069700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14180bd280 con 0x7f1400003c20 2023-10-06T04:01:32.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.964+0000 7f1417fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1418159500 0x7f141814ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:32.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.964+0000 7f1417fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1418159500 0x7f141814ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46298/0 (socket says 172.21.15.157:46298) 2023-10-06T04:01:32.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.964+0000 7f1417fff700 1 -- 172.21.15.157:0/1669428725 learned_addr learned my addr 172.21.15.157:0/1669428725 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:01:32.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.965+0000 7f1416ffd700 1 --2- 172.21.15.157:0/1669428725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14180bd710 0x7f1418158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:32.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.965+0000 7f14177fe700 1 --2- 172.21.15.157:0/1669428725 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1400003c20 0x7f1418165e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:32.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.965+0000 7f1417fff700 1 -- 172.21.15.157:0/1669428725 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1400003c20 msgr2=0x7f1418165e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:32.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.965+0000 7f1417fff700 1 --2- 172.21.15.157:0/1669428725 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1400003c20 0x7f1418165e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:32.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.965+0000 7f1417fff700 1 -- 172.21.15.157:0/1669428725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14180bd710 msgr2=0x7f1418158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:32.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.965+0000 7f1417fff700 1 --2- 172.21.15.157:0/1669428725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14180bd710 0x7f1418158710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:32.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.965+0000 7f1417fff700 1 -- 172.21.15.157:0/1669428725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14180ba800 con 0x7f1418159500 2023-10-06T04:01:32.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.965+0000 7f14177fe700 1 --2- 172.21.15.157:0/1669428725 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1400003c20 0x7f1418165e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:01:32.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.965+0000 7f1417fff700 1 --2- 172.21.15.157:0/1669428725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1418159500 0x7f141814ed10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1418069d60 tx=0x7f14100017e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:32.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.965+0000 7f13ff7fe700 1 -- 172.21.15.157:0/1669428725 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1410006e80 con 0x7f1418159500 2023-10-06T04:01:32.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.965+0000 7f13ff7fe700 1 -- 172.21.15.157:0/1669428725 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f141000c540 con 0x7f1418159500 2023-10-06T04:01:32.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.965+0000 7f141e069700 1 -- 172.21.15.157:0/1669428725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14180baac0 con 0x7f1418159500 2023-10-06T04:01:32.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.965+0000 7f13ff7fe700 1 -- 172.21.15.157:0/1669428725 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141000c920 con 0x7f1418159500 2023-10-06T04:01:32.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.965+0000 7f141e069700 1 -- 172.21.15.157:0/1669428725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14180bb080 con 0x7f1418159500 2023-10-06T04:01:32.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.966+0000 7f141e069700 1 -- 172.21.15.157:0/1669428725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14180bd430 con 0x7f1418159500 2023-10-06T04:01:32.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.966+0000 7f13ff7fe700 1 -- 172.21.15.157:0/1669428725 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f141000cc90 con 0x7f1418159500 2023-10-06T04:01:32.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.967+0000 7f13ff7fe700 1 --2- 172.21.15.157:0/1669428725 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1404066f60 0x7f1404069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:32.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.967+0000 7f13ff7fe700 1 -- 172.21.15.157:0/1669428725 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f1410084070 con 0x7f1418159500 2023-10-06T04:01:32.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.967+0000 7f14177fe700 1 --2- 172.21.15.157:0/1669428725 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1404066f60 0x7f1404069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:32.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.968+0000 7f14177fe700 1 --2- 172.21.15.157:0/1669428725 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1404066f60 0x7f1404069410 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1408007fd0 tx=0x7f1408015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:32.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:32.971+0000 7f13ff7fe700 1 -- 172.21.15.157:0/1669428725 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f141004d5a0 con 0x7f1418159500 2023-10-06T04:01:33.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:33.199+0000 7f141e069700 1 -- 172.21.15.157:0/1669428725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f141804bb40 con 0x7f1418159500 2023-10-06T04:01:33.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:33.200+0000 7f13ff7fe700 1 -- 172.21.15.157:0/1669428725 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+5439 (secure 0 0 0) 0x7f14180bd430 con 0x7f1418159500 2023-10-06T04:01:33.202 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:01:33.202 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:01:25.737271+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"f","rank":4,"incarnation":29,"state":"up:active","state_seq":32,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:01:33.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:33.202+0000 7f141e069700 1 -- 172.21.15.157:0/1669428725 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1404066f60 msgr2=0x7f1404069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:33.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:33.202+0000 7f141e069700 1 --2- 172.21.15.157:0/1669428725 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1404066f60 0x7f1404069410 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1408007fd0 tx=0x7f1408015000 comp rx=0 tx=0).stop 2023-10-06T04:01:33.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:33.202+0000 7f141e069700 1 -- 172.21.15.157:0/1669428725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1418159500 msgr2=0x7f141814ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:33.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:33.202+0000 7f141e069700 1 --2- 172.21.15.157:0/1669428725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1418159500 0x7f141814ed10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1418069d60 tx=0x7f14100017e0 comp rx=0 tx=0).stop 2023-10-06T04:01:33.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:33.202+0000 7f141e069700 1 -- 172.21.15.157:0/1669428725 shutdown_connections 2023-10-06T04:01:33.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:33.202+0000 7f141e069700 1 --2- 172.21.15.157:0/1669428725 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1404066f60 0x7f1404069410 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:33.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:33.202+0000 7f141e069700 1 --2- 172.21.15.157:0/1669428725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1418159500 0x7f141814ed10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:33.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:33.202+0000 7f141e069700 1 --2- 172.21.15.157:0/1669428725 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1400003c20 0x7f1418165e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:33.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:33.202+0000 7f141e069700 1 --2- 172.21.15.157:0/1669428725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14180bd710 0x7f1418158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:33.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:33.202+0000 7f141e069700 1 -- 172.21.15.157:0/1669428725 >> 172.21.15.157:0/1669428725 conn(0x7f141806b510 msgr2=0x7f14180c5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:33.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:33.202+0000 7f141e069700 1 -- 172.21.15.157:0/1669428725 shutdown_connections 2023-10-06T04:01:33.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:33.202+0000 7f141e069700 1 -- 172.21.15.157:0/1669428725 wait complete. 2023-10-06T04:01:33.205 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 33 2023-10-06T04:01:33.216 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2023-10-06T04:01:35.217 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:01:35.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.343+0000 7f21a1c6b700 1 Processor -- start 2023-10-06T04:01:35.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.343+0000 7f21a1c6b700 1 -- start start 2023-10-06T04:01:35.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.344+0000 7f21a1c6b700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f219c06dc40 0x7f219c14fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:35.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.344+0000 7f21a1c6b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f219c06c1f0 0x7f219c150240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:35.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.344+0000 7f21a1c6b700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f219c0c9780 con 0x7f219c06cbb0 2023-10-06T04:01:35.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.344+0000 7f21a1c6b700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f219c0c8c40 con 0x7f219c06dc40 2023-10-06T04:01:35.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.344+0000 7f21a1c6b700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f219c06e0c0 con 0x7f219c06c1f0 2023-10-06T04:01:35.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.344+0000 7f219b7fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f219c06cbb0 0x7f219c06d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:56734/0 (socket says 172.21.15.157:56734) 2023-10-06T04:01:35.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.344+0000 7f219b7fe700 1 -- 172.21.15.157:0/3115579829 learned_addr learned my addr 172.21.15.157:0/3115579829 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:01:35.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.344+0000 7f219affd700 1 --2- 172.21.15.157:0/3115579829 >> v2:172.21.15.175:3300/0 conn(0x7f219c06dc40 0x7f219c14fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:35.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.344+0000 7f219bfff700 1 --2- 172.21.15.157:0/3115579829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f219c06c1f0 0x7f219c150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:35.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.344+0000 7f219a7fc700 1 -- 172.21.15.157:0/3115579829 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2556241882 0 0) 0x7f219c0c9780 con 0x7f219c06cbb0 2023-10-06T04:01:35.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.344+0000 7f219a7fc700 1 -- 172.21.15.157:0/3115579829 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2184003570 con 0x7f219c06cbb0 2023-10-06T04:01:35.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.344+0000 7f219a7fc700 1 -- 172.21.15.157:0/3115579829 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3254214058 0 0) 0x7f2184003570 con 0x7f219c06cbb0 2023-10-06T04:01:35.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.344+0000 7f219a7fc700 1 -- 172.21.15.157:0/3115579829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f219c06c1f0 msgr2=0x7f219c150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:01:35.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.344+0000 7f219a7fc700 1 --2- 172.21.15.157:0/3115579829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f219c06c1f0 0x7f219c150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:35.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.345+0000 7f219a7fc700 1 -- 172.21.15.157:0/3115579829 >> v2:172.21.15.175:3300/0 conn(0x7f219c06dc40 msgr2=0x7f219c14fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:01:35.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.345+0000 7f219a7fc700 1 --2- 172.21.15.157:0/3115579829 >> v2:172.21.15.175:3300/0 conn(0x7f219c06dc40 0x7f219c14fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:35.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.345+0000 7f219a7fc700 1 -- 172.21.15.157:0/3115579829 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219c0c8e50 con 0x7f219c06cbb0 2023-10-06T04:01:35.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.345+0000 7f219a7fc700 1 -- 172.21.15.157:0/3115579829 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f218c0017f0 con 0x7f219c06cbb0 2023-10-06T04:01:35.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.345+0000 7f219a7fc700 1 -- 172.21.15.157:0/3115579829 >> v1:172.21.15.157:6789/0 conn(0x7f219c06cbb0 legacy=0x7f219c06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:35.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.345+0000 7f219a7fc700 1 --2- 172.21.15.157:0/3115579829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f219c06c1f0 0x7f2184003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:35.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.345+0000 7f219a7fc700 1 --2- 172.21.15.157:0/3115579829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2184003fe0 0x7f2184004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:35.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.345+0000 7f219a7fc700 1 --2- 172.21.15.157:0/3115579829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2184004d50 0x7f21840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:35.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.345+0000 7f219a7fc700 1 -- 172.21.15.157:0/3115579829 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219c0c8c40 con 0x7f2184003fe0 2023-10-06T04:01:35.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.345+0000 7f219a7fc700 1 -- 172.21.15.157:0/3115579829 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219c06e0c0 con 0x7f2184004d50 2023-10-06T04:01:35.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.345+0000 7f219a7fc700 1 -- 172.21.15.157:0/3115579829 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f218400ba90 con 0x7f219c06c1f0 2023-10-06T04:01:35.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.345+0000 7f219affd700 1 --2- 172.21.15.157:0/3115579829 >> v2:172.21.15.175:3300/0 conn(0x7f219c06dc40 0x7f219c14fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:01:35.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.345+0000 7f219b7fe700 1 --2- 172.21.15.157:0/3115579829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f219c06c1f0 0x7f2184003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:35.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.345+0000 7f219affd700 1 --2- 172.21.15.157:0/3115579829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2184003fe0 0x7f2184004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:35.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.345+0000 7f219bfff700 1 --2- 172.21.15.157:0/3115579829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2184004d50 0x7f21840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:35.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.345+0000 7f219affd700 1 -- 172.21.15.157:0/3115579829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f219c06c1f0 msgr2=0x7f2184003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:01:35.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.345+0000 7f219affd700 1 --2- 172.21.15.157:0/3115579829 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f219c06c1f0 0x7f2184003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:35.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.345+0000 7f219affd700 1 -- 172.21.15.157:0/3115579829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2184004d50 msgr2=0x7f21840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:01:35.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.345+0000 7f219affd700 1 --2- 172.21.15.157:0/3115579829 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2184004d50 0x7f21840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:35.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.345+0000 7f219affd700 1 -- 172.21.15.157:0/3115579829 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2184003570 con 0x7f2184003fe0 2023-10-06T04:01:35.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.345+0000 7f219affd700 1 --2- 172.21.15.157:0/3115579829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2184003fe0 0x7f2184004810 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2190009db0 tx=0x7f21900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e679b02ce45b8b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:35.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.345+0000 7f219a7fc700 1 -- 172.21.15.157:0/3115579829 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2190009740 con 0x7f2184003fe0 2023-10-06T04:01:35.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.345+0000 7f219a7fc700 1 -- 172.21.15.157:0/3115579829 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219000e8c0 con 0x7f2184003fe0 2023-10-06T04:01:35.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.345+0000 7f219a7fc700 1 -- 172.21.15.157:0/3115579829 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219000ec40 con 0x7f2184003fe0 2023-10-06T04:01:35.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.346+0000 7f21a1c6b700 1 -- 172.21.15.157:0/3115579829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2184003fe0 msgr2=0x7f2184004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:35.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.346+0000 7f21a1c6b700 1 --2- 172.21.15.157:0/3115579829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2184003fe0 0x7f2184004810 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2190009db0 tx=0x7f21900091a0 comp rx=0 tx=0).stop 2023-10-06T04:01:35.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.346+0000 7f219bfff700 1 -- 172.21.15.157:0/3115579829 reap_dead start 2023-10-06T04:01:35.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.346+0000 7f21a1c6b700 1 -- 172.21.15.157:0/3115579829 shutdown_connections 2023-10-06T04:01:35.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.346+0000 7f21a1c6b700 1 -- 172.21.15.157:0/3115579829 >> 172.21.15.157:0/3115579829 conn(0x7f219c06f230 msgr2=0x7f219c06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:35.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.346+0000 7f21a1c6b700 1 -- 172.21.15.157:0/3115579829 shutdown_connections 2023-10-06T04:01:35.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.346+0000 7f21a1c6b700 1 -- 172.21.15.157:0/3115579829 wait complete. 2023-10-06T04:01:35.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.347+0000 7f21a1c6b700 1 Processor -- start 2023-10-06T04:01:35.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.347+0000 7f21a1c6b700 1 -- start start 2023-10-06T04:01:35.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.348+0000 7f21a1c6b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f219c0cb9b0 0x7f219c151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:35.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.348+0000 7f21a1c6b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f219c1459b0 0x7f219c152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:35.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.348+0000 7f21a1c6b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f219c0cbd80 0x7f219c169bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:35.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.348+0000 7f21a1c6b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219c166d90 con 0x7f219c0cbd80 2023-10-06T04:01:35.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.348+0000 7f21a1c6b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219c145e80 con 0x7f219c1459b0 2023-10-06T04:01:35.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.348+0000 7f21a1c6b700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f219c146160 con 0x7f219c0cb9b0 2023-10-06T04:01:35.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.348+0000 7f219bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f219c0cbd80 0x7f219c169bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:35.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.348+0000 7f219bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f219c0cbd80 0x7f219c169bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52120/0 (socket says 172.21.15.157:52120) 2023-10-06T04:01:35.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.348+0000 7f219bfff700 1 -- 172.21.15.157:0/1011569925 learned_addr learned my addr 172.21.15.157:0/1011569925 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:01:35.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.348+0000 7f219affd700 1 --2- 172.21.15.157:0/1011569925 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f219c1459b0 0x7f219c152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:35.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.348+0000 7f219b7fe700 1 --2- 172.21.15.157:0/1011569925 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f219c0cb9b0 0x7f219c151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:35.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.348+0000 7f219bfff700 1 -- 172.21.15.157:0/1011569925 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f219c0cb9b0 msgr2=0x7f219c151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:35.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.348+0000 7f219bfff700 1 --2- 172.21.15.157:0/1011569925 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f219c0cb9b0 0x7f219c151820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:35.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.348+0000 7f219bfff700 1 -- 172.21.15.157:0/1011569925 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f219c1459b0 msgr2=0x7f219c152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:35.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.348+0000 7f219bfff700 1 --2- 172.21.15.157:0/1011569925 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f219c1459b0 0x7f219c152c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:35.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.348+0000 7f219bfff700 1 -- 172.21.15.157:0/1011569925 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219c166740 con 0x7f219c0cbd80 2023-10-06T04:01:35.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.348+0000 7f219affd700 1 --2- 172.21.15.157:0/1011569925 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f219c1459b0 0x7f219c152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:01:35.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.349+0000 7f219bfff700 1 --2- 172.21.15.157:0/1011569925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f219c0cbd80 0x7f219c169bb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f219c0cc630 tx=0x7f21940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:35.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.349+0000 7f21837fe700 1 -- 172.21.15.157:0/1011569925 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2194014720 con 0x7f219c0cbd80 2023-10-06T04:01:35.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.349+0000 7f21837fe700 1 -- 172.21.15.157:0/1011569925 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2194014880 con 0x7f219c0cbd80 2023-10-06T04:01:35.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.349+0000 7f21a1c6b700 1 -- 172.21.15.157:0/1011569925 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f219c166a20 con 0x7f219c0cbd80 2023-10-06T04:01:35.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.349+0000 7f21837fe700 1 -- 172.21.15.157:0/1011569925 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2194014c40 con 0x7f219c0cbd80 2023-10-06T04:01:35.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.349+0000 7f21a1c6b700 1 -- 172.21.15.157:0/1011569925 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f219c1534d0 con 0x7f219c0cbd80 2023-10-06T04:01:35.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.350+0000 7f21a1c6b700 1 -- 172.21.15.157:0/1011569925 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f219c089ca0 con 0x7f219c0cbd80 2023-10-06T04:01:35.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.350+0000 7f21837fe700 1 -- 172.21.15.157:0/1011569925 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f219400f480 con 0x7f219c0cbd80 2023-10-06T04:01:35.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.350+0000 7f21837fe700 1 --2- 172.21.15.157:0/1011569925 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2188066f30 0x7f21880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:35.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.351+0000 7f21837fe700 1 -- 172.21.15.157:0/1011569925 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f2194084c00 con 0x7f219c0cbd80 2023-10-06T04:01:35.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.351+0000 7f219b7fe700 1 --2- 172.21.15.157:0/1011569925 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2188066f30 0x7f21880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:35.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.351+0000 7f219b7fe700 1 --2- 172.21.15.157:0/1011569925 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2188066f30 0x7f21880693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f218c004c10 tx=0x7f218c006040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:35.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.355+0000 7f21837fe700 1 -- 172.21.15.157:0/1011569925 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2194055740 con 0x7f219c0cbd80 2023-10-06T04:01:35.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.585+0000 7f21a1c6b700 1 -- 172.21.15.157:0/1011569925 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f219c166ed0 con 0x7f219c0cbd80 2023-10-06T04:01:35.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.585+0000 7f21837fe700 1 -- 172.21.15.157:0/1011569925 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+6198 (secure 0 0 0) 0x7f21940552d0 con 0x7f219c0cbd80 2023-10-06T04:01:35.588 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:01:35.588 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:01:25.737271+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"f","rank":4,"incarnation":29,"state":"up:active","state_seq":32,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:01:35.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.588+0000 7f21a1c6b700 1 -- 172.21.15.157:0/1011569925 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2188066f30 msgr2=0x7f21880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:35.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.588+0000 7f21a1c6b700 1 --2- 172.21.15.157:0/1011569925 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2188066f30 0x7f21880693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f218c004c10 tx=0x7f218c006040 comp rx=0 tx=0).stop 2023-10-06T04:01:35.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.588+0000 7f21a1c6b700 1 -- 172.21.15.157:0/1011569925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f219c0cbd80 msgr2=0x7f219c169bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:35.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.588+0000 7f21a1c6b700 1 --2- 172.21.15.157:0/1011569925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f219c0cbd80 0x7f219c169bb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f219c0cc630 tx=0x7f21940091a0 comp rx=0 tx=0).stop 2023-10-06T04:01:35.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.588+0000 7f21a1c6b700 1 -- 172.21.15.157:0/1011569925 shutdown_connections 2023-10-06T04:01:35.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.588+0000 7f21a1c6b700 1 --2- 172.21.15.157:0/1011569925 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2188066f30 0x7f21880693e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:35.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.588+0000 7f21a1c6b700 1 --2- 172.21.15.157:0/1011569925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f219c0cbd80 0x7f219c169bb0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:35.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.588+0000 7f21a1c6b700 1 --2- 172.21.15.157:0/1011569925 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f219c0cb9b0 0x7f219c151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:35.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.588+0000 7f21a1c6b700 1 --2- 172.21.15.157:0/1011569925 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f219c1459b0 0x7f219c152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:35.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.588+0000 7f21a1c6b700 1 -- 172.21.15.157:0/1011569925 >> 172.21.15.157:0/1011569925 conn(0x7f219c06f230 msgr2=0x7f219c06d530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:35.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.588+0000 7f21a1c6b700 1 -- 172.21.15.157:0/1011569925 shutdown_connections 2023-10-06T04:01:35.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:35.588+0000 7f21a1c6b700 1 -- 172.21.15.157:0/1011569925 wait complete. 2023-10-06T04:01:35.591 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 34 2023-10-06T04:01:35.601 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2023-10-06T04:01:35.601 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 16 secs before thrashing 2023-10-06T04:01:36.974 INFO:tasks.workunit.client.0.smithi157.stdout:.//.snap/test1 2023-10-06T04:01:36.974 INFO:tasks.workunit.client.0.smithi157.stdout:.//.snap/test2 2023-10-06T04:01:36.974 INFO:tasks.workunit.client.0.smithi157.stdout:.//.snap/test3 2023-10-06T04:01:36.978 INFO:tasks.workunit.client.0.smithi157.stdout:./274/.snap/snap-subdir-test 2023-10-06T04:01:36.983 INFO:tasks.workunit.client.0.smithi157.stdout:./166/.snap/snap-subdir-test 2023-10-06T04:01:37.386 INFO:tasks.workunit.client.0.smithi157.stdout:./256/.snap/snap-subdir-test 2023-10-06T04:01:37.391 INFO:tasks.workunit.client.0.smithi157.stdout:./247/.snap/snap-subdir-test 2023-10-06T04:01:37.395 INFO:tasks.workunit.client.0.smithi157.stdout:./248/.snap/snap-subdir-test 2023-10-06T04:01:37.399 INFO:tasks.workunit.client.0.smithi157.stdout:./147/.snap/snap-subdir-test 2023-10-06T04:01:37.404 INFO:tasks.workunit.client.0.smithi157.stdout:./151/.snap/snap-subdir-test 2023-10-06T04:01:37.409 INFO:tasks.workunit.client.0.smithi157.stdout:./170/.snap/snap-subdir-test 2023-10-06T04:01:37.413 INFO:tasks.workunit.client.0.smithi157.stdout:./167/.snap/snap-subdir-test 2023-10-06T04:01:37.417 INFO:tasks.workunit.client.0.smithi157.stdout:./140/.snap/snap-subdir-test 2023-10-06T04:01:37.421 INFO:tasks.workunit.client.0.smithi157.stdout:./142/.snap/snap-subdir-test 2023-10-06T04:01:37.425 INFO:tasks.workunit.client.0.smithi157.stdout:./150/.snap/snap-subdir-test 2023-10-06T04:01:37.430 INFO:tasks.workunit.client.0.smithi157.stdout:./273/.snap/snap-subdir-test 2023-10-06T04:01:37.434 INFO:tasks.workunit.client.0.smithi157.stdout:./269/.snap/snap-subdir-test 2023-10-06T04:01:37.439 INFO:tasks.workunit.client.0.smithi157.stdout:./105/.snap/snap-subdir-test 2023-10-06T04:01:37.443 INFO:tasks.workunit.client.0.smithi157.stdout:./210/.snap/snap-subdir-test 2023-10-06T04:01:37.446 INFO:tasks.workunit.client.0.smithi157.stdout:./265/.snap/snap-subdir-test 2023-10-06T04:01:37.450 INFO:tasks.workunit.client.0.smithi157.stdout:./218/.snap/snap-subdir-test 2023-10-06T04:01:37.454 INFO:tasks.workunit.client.0.smithi157.stdout:./290/.snap/snap-subdir-test 2023-10-06T04:01:37.458 INFO:tasks.workunit.client.0.smithi157.stdout:./149/.snap/snap-subdir-test 2023-10-06T04:01:37.463 INFO:tasks.workunit.client.0.smithi157.stdout:./109/.snap/snap-subdir-test 2023-10-06T04:01:37.467 INFO:tasks.workunit.client.0.smithi157.stdout:./292/.snap/snap-subdir-test 2023-10-06T04:01:37.470 INFO:tasks.workunit.client.0.smithi157.stdout:./242/.snap/snap-subdir-test 2023-10-06T04:01:37.475 INFO:tasks.workunit.client.0.smithi157.stdout:./158/.snap/snap-subdir-test 2023-10-06T04:01:37.480 INFO:tasks.workunit.client.0.smithi157.stdout:./243/.snap/snap-subdir-test 2023-10-06T04:01:37.486 INFO:tasks.workunit.client.0.smithi157.stdout:./106/.snap/snap-subdir-test 2023-10-06T04:01:37.490 INFO:tasks.workunit.client.0.smithi157.stdout:./250/.snap/snap-subdir-test 2023-10-06T04:01:38.288 INFO:tasks.workunit.client.0.smithi157.stdout:./221/.snap/snap-subdir-test 2023-10-06T04:01:38.293 INFO:tasks.workunit.client.0.smithi157.stdout:./244/.snap/snap-subdir-test 2023-10-06T04:01:38.298 INFO:tasks.workunit.client.0.smithi157.stdout:./258/.snap/snap-subdir-test 2023-10-06T04:01:38.304 INFO:tasks.workunit.client.0.smithi157.stdout:./268/.snap/snap-subdir-test 2023-10-06T04:01:38.308 INFO:tasks.workunit.client.0.smithi157.stdout:./189/.snap/snap-subdir-test 2023-10-06T04:01:38.313 INFO:tasks.workunit.client.0.smithi157.stdout:./285/.snap/snap-subdir-test 2023-10-06T04:01:38.317 INFO:tasks.workunit.client.0.smithi157.stdout:./208/.snap/snap-subdir-test 2023-10-06T04:01:38.322 INFO:tasks.workunit.client.0.smithi157.stdout:./288/.snap/snap-subdir-test 2023-10-06T04:01:38.326 INFO:tasks.workunit.client.0.smithi157.stdout:./209/.snap/snap-subdir-test 2023-10-06T04:01:38.332 INFO:tasks.workunit.client.0.smithi157.stdout:./253/.snap/snap-subdir-test 2023-10-06T04:01:38.336 INFO:tasks.workunit.client.0.smithi157.stdout:./249/.snap/snap-subdir-test 2023-10-06T04:01:38.341 INFO:tasks.workunit.client.0.smithi157.stdout:./133/.snap/snap-subdir-test 2023-10-06T04:01:38.346 INFO:tasks.workunit.client.0.smithi157.stdout:./291/.snap/snap-subdir-test 2023-10-06T04:01:38.350 INFO:tasks.workunit.client.0.smithi157.stdout:./226/.snap/snap-subdir-test 2023-10-06T04:01:38.355 INFO:tasks.workunit.client.0.smithi157.stdout:./173/.snap/snap-subdir-test 2023-10-06T04:01:38.360 INFO:tasks.workunit.client.0.smithi157.stdout:./174/.snap/snap-subdir-test 2023-10-06T04:01:38.364 INFO:tasks.workunit.client.0.smithi157.stdout:./299/.snap/snap-subdir-test 2023-10-06T04:01:38.368 INFO:tasks.workunit.client.0.smithi157.stdout:./262/.snap/snap-subdir-test 2023-10-06T04:01:38.372 INFO:tasks.workunit.client.0.smithi157.stdout:./231/.snap/snap-subdir-test 2023-10-06T04:01:38.377 INFO:tasks.workunit.client.0.smithi157.stdout:./214/.snap/snap-subdir-test 2023-10-06T04:01:38.381 INFO:tasks.workunit.client.0.smithi157.stdout:./281/.snap/snap-subdir-test 2023-10-06T04:01:38.386 INFO:tasks.workunit.client.0.smithi157.stdout:./289/.snap/snap-subdir-test 2023-10-06T04:01:38.390 INFO:tasks.workunit.client.0.smithi157.stdout:./283/.snap/snap-subdir-test 2023-10-06T04:01:38.395 INFO:tasks.workunit.client.0.smithi157.stdout:./232/.snap/snap-subdir-test 2023-10-06T04:01:39.186 INFO:tasks.workunit.client.0.smithi157.stdout:./298/.snap/snap-subdir-test 2023-10-06T04:01:39.191 INFO:tasks.workunit.client.0.smithi157.stdout:./233/.snap/snap-subdir-test 2023-10-06T04:01:39.197 INFO:tasks.workunit.client.0.smithi157.stdout:./155/.snap/snap-subdir-test 2023-10-06T04:01:39.203 INFO:tasks.workunit.client.0.smithi157.stdout:./114/.snap/snap-subdir-test 2023-10-06T04:01:39.208 INFO:tasks.workunit.client.0.smithi157.stdout:./159/.snap/snap-subdir-test 2023-10-06T04:01:39.212 INFO:tasks.workunit.client.0.smithi157.stdout:./187/.snap/snap-subdir-test 2023-10-06T04:01:39.217 INFO:tasks.workunit.client.0.smithi157.stdout:./185/.snap/snap-subdir-test 2023-10-06T04:01:39.222 INFO:tasks.workunit.client.0.smithi157.stdout:./293/.snap/snap-subdir-test 2023-10-06T04:01:39.226 INFO:tasks.workunit.client.0.smithi157.stdout:./228/.snap/snap-subdir-test 2023-10-06T04:01:39.231 INFO:tasks.workunit.client.0.smithi157.stdout:./120/.snap/snap-subdir-test 2023-10-06T04:01:39.236 INFO:tasks.workunit.client.0.smithi157.stdout:./110/.snap/snap-subdir-test 2023-10-06T04:01:39.240 INFO:tasks.workunit.client.0.smithi157.stdout:./267/.snap/snap-subdir-test 2023-10-06T04:01:39.245 INFO:tasks.workunit.client.0.smithi157.stdout:./202/.snap/snap-subdir-test 2023-10-06T04:01:39.250 INFO:tasks.workunit.client.0.smithi157.stdout:./137/.snap/snap-subdir-test 2023-10-06T04:01:39.254 INFO:tasks.workunit.client.0.smithi157.stdout:./246/.snap/snap-subdir-test 2023-10-06T04:01:39.258 INFO:tasks.workunit.client.0.smithi157.stdout:./219/.snap/snap-subdir-test 2023-10-06T04:01:39.263 INFO:tasks.workunit.client.0.smithi157.stdout:./198/.snap/snap-subdir-test 2023-10-06T04:01:39.268 INFO:tasks.workunit.client.0.smithi157.stdout:./136/.snap/snap-subdir-test 2023-10-06T04:01:39.272 INFO:tasks.workunit.client.0.smithi157.stdout:./270/.snap/snap-subdir-test 2023-10-06T04:01:39.276 INFO:tasks.workunit.client.0.smithi157.stdout:./125/.snap/snap-subdir-test 2023-10-06T04:01:39.281 INFO:tasks.workunit.client.0.smithi157.stdout:./194/.snap/snap-subdir-test 2023-10-06T04:01:39.286 INFO:tasks.workunit.client.0.smithi157.stdout:./119/.snap/snap-subdir-test 2023-10-06T04:01:39.290 INFO:tasks.workunit.client.0.smithi157.stdout:./297/.snap/snap-subdir-test 2023-10-06T04:01:39.294 INFO:tasks.workunit.client.0.smithi157.stdout:./230/.snap/snap-subdir-test 2023-10-06T04:01:39.298 INFO:tasks.workunit.client.0.smithi157.stdout:./260/.snap/snap-subdir-test 2023-10-06T04:01:39.302 INFO:tasks.workunit.client.0.smithi157.stdout:./157/.snap/snap-subdir-test 2023-10-06T04:01:39.305 INFO:tasks.workunit.client.0.smithi157.stdout:./172/.snap/snap-subdir-test 2023-10-06T04:01:39.309 INFO:tasks.workunit.client.0.smithi157.stdout:./277/.snap/snap-subdir-test 2023-10-06T04:01:39.313 INFO:tasks.workunit.client.0.smithi157.stdout:./186/.snap/snap-subdir-test 2023-10-06T04:01:39.316 INFO:tasks.workunit.client.0.smithi157.stdout:./237/.snap/snap-subdir-test 2023-10-06T04:01:39.321 INFO:tasks.workunit.client.0.smithi157.stdout:./190/.snap/snap-subdir-test 2023-10-06T04:01:39.324 INFO:tasks.workunit.client.0.smithi157.stdout:./213/.snap/snap-subdir-test 2023-10-06T04:01:39.327 INFO:tasks.workunit.client.0.smithi157.stdout:./104/.snap/snap-subdir-test 2023-10-06T04:01:39.331 INFO:tasks.workunit.client.0.smithi157.stdout:./164/.snap/snap-subdir-test 2023-10-06T04:01:39.335 INFO:tasks.workunit.client.0.smithi157.stdout:./112/.snap/snap-subdir-test 2023-10-06T04:01:39.338 INFO:tasks.workunit.client.0.smithi157.stdout:./295/.snap/snap-subdir-test 2023-10-06T04:01:39.341 INFO:tasks.workunit.client.0.smithi157.stdout:./272/.snap/snap-subdir-test 2023-10-06T04:01:39.345 INFO:tasks.workunit.client.0.smithi157.stdout:./100/.snap/snap-subdir-test 2023-10-06T04:01:39.349 INFO:tasks.workunit.client.0.smithi157.stdout:./134/.snap/snap-subdir-test 2023-10-06T04:01:39.352 INFO:tasks.workunit.client.0.smithi157.stdout:./200/.snap/snap-subdir-test 2023-10-06T04:01:39.356 INFO:tasks.workunit.client.0.smithi157.stdout:./115/.snap/snap-subdir-test 2023-10-06T04:01:39.360 INFO:tasks.workunit.client.0.smithi157.stdout:./284/.snap/snap-subdir-test 2023-10-06T04:01:39.363 INFO:tasks.workunit.client.0.smithi157.stdout:./280/.snap/snap-subdir-test 2023-10-06T04:01:39.367 INFO:tasks.workunit.client.0.smithi157.stdout:./116/.snap/snap-subdir-test 2023-10-06T04:01:39.370 INFO:tasks.workunit.client.0.smithi157.stdout:./245/.snap/snap-subdir-test 2023-10-06T04:01:39.374 INFO:tasks.workunit.client.0.smithi157.stdout:./195/.snap/snap-subdir-test 2023-10-06T04:01:39.377 INFO:tasks.workunit.client.0.smithi157.stdout:./138/.snap/snap-subdir-test 2023-10-06T04:01:39.642 INFO:tasks.workunit.client.0.smithi157.stdout:./135/.snap/snap-subdir-test 2023-10-06T04:01:39.646 INFO:tasks.workunit.client.0.smithi157.stdout:./117/.snap/snap-subdir-test 2023-10-06T04:01:39.650 INFO:tasks.workunit.client.0.smithi157.stdout:./241/.snap/snap-subdir-test 2023-10-06T04:01:39.653 INFO:tasks.workunit.client.0.smithi157.stdout:./143/.snap/snap-subdir-test 2023-10-06T04:01:39.657 INFO:tasks.workunit.client.0.smithi157.stdout:./180/.snap/snap-subdir-test 2023-10-06T04:01:40.302 INFO:tasks.workunit.client.0.smithi157.stdout:./126/.snap/snap-subdir-test 2023-10-06T04:01:40.307 INFO:tasks.workunit.client.0.smithi157.stdout:./161/.snap/snap-subdir-test 2023-10-06T04:01:40.310 INFO:tasks.workunit.client.0.smithi157.stdout:./204/.snap/snap-subdir-test 2023-10-06T04:01:40.314 INFO:tasks.workunit.client.0.smithi157.stdout:./176/.snap/snap-subdir-test 2023-10-06T04:01:40.318 INFO:tasks.workunit.client.0.smithi157.stdout:./255/.snap/snap-subdir-test 2023-10-06T04:01:40.590 INFO:tasks.workunit.client.0.smithi157.stdout:./215/.snap/snap-subdir-test 2023-10-06T04:01:40.596 INFO:tasks.workunit.client.0.smithi157.stdout:./111/.snap/snap-subdir-test 2023-10-06T04:01:40.600 INFO:tasks.workunit.client.0.smithi157.stdout:./238/.snap/snap-subdir-test 2023-10-06T04:01:40.604 INFO:tasks.workunit.client.0.smithi157.stdout:./191/.snap/snap-subdir-test 2023-10-06T04:01:40.609 INFO:tasks.workunit.client.0.smithi157.stdout:./184/.snap/snap-subdir-test 2023-10-06T04:01:40.613 INFO:tasks.workunit.client.0.smithi157.stdout:./192/.snap/snap-subdir-test 2023-10-06T04:01:40.616 INFO:tasks.workunit.client.0.smithi157.stdout:./103/.snap/snap-subdir-test 2023-10-06T04:01:40.622 INFO:tasks.workunit.client.0.smithi157.stdout:./128/.snap/snap-subdir-test 2023-10-06T04:01:40.625 INFO:tasks.workunit.client.0.smithi157.stdout:./229/.snap/snap-subdir-test 2023-10-06T04:01:40.629 INFO:tasks.workunit.client.0.smithi157.stdout:./144/.snap/snap-subdir-test 2023-10-06T04:01:40.633 INFO:tasks.workunit.client.0.smithi157.stdout:./146/.snap/snap-subdir-test 2023-10-06T04:01:40.637 INFO:tasks.workunit.client.0.smithi157.stdout:./227/.snap/snap-subdir-test 2023-10-06T04:01:40.640 INFO:tasks.workunit.client.0.smithi157.stdout:./278/.snap/snap-subdir-test 2023-10-06T04:01:40.644 INFO:tasks.workunit.client.0.smithi157.stdout:./141/.snap/snap-subdir-test 2023-10-06T04:01:40.649 INFO:tasks.workunit.client.0.smithi157.stdout:./234/.snap/snap-subdir-test 2023-10-06T04:01:40.653 INFO:tasks.workunit.client.0.smithi157.stdout:./225/.snap/snap-subdir-test 2023-10-06T04:01:40.657 INFO:tasks.workunit.client.0.smithi157.stdout:./182/.snap/snap-subdir-test 2023-10-06T04:01:40.661 INFO:tasks.workunit.client.0.smithi157.stdout:./107/.snap/snap-subdir-test 2023-10-06T04:01:40.667 INFO:tasks.workunit.client.0.smithi157.stdout:./178/.snap/snap-subdir-test 2023-10-06T04:01:40.671 INFO:tasks.workunit.client.0.smithi157.stdout:./266/.snap/snap-subdir-test 2023-10-06T04:01:40.677 INFO:tasks.workunit.client.0.smithi157.stdout:./240/.snap/snap-subdir-test 2023-10-06T04:01:40.682 INFO:tasks.workunit.client.0.smithi157.stdout:./156/.snap/snap-subdir-test 2023-10-06T04:01:40.687 INFO:tasks.workunit.client.0.smithi157.stdout:./131/.snap/snap-subdir-test 2023-10-06T04:01:40.692 INFO:tasks.workunit.client.0.smithi157.stdout:./123/.snap/snap-subdir-test 2023-10-06T04:01:40.696 INFO:tasks.workunit.client.0.smithi157.stdout:./160/.snap/snap-subdir-test 2023-10-06T04:01:40.701 INFO:tasks.workunit.client.0.smithi157.stdout:./165/.snap/snap-subdir-test 2023-10-06T04:01:40.705 INFO:tasks.workunit.client.0.smithi157.stdout:./101/.snap/snap-subdir-test 2023-10-06T04:01:40.710 INFO:tasks.workunit.client.0.smithi157.stdout:./148/.snap/snap-subdir-test 2023-10-06T04:01:40.715 INFO:tasks.workunit.client.0.smithi157.stdout:./168/.snap/snap-subdir-test 2023-10-06T04:01:40.719 INFO:tasks.workunit.client.0.smithi157.stdout:./179/.snap/snap-subdir-test 2023-10-06T04:01:40.723 INFO:tasks.workunit.client.0.smithi157.stdout:./257/.snap/snap-subdir-test 2023-10-06T04:01:40.727 INFO:tasks.workunit.client.0.smithi157.stdout:./252/.snap/snap-subdir-test 2023-10-06T04:01:40.733 INFO:tasks.workunit.client.0.smithi157.stdout:./132/.snap/snap-subdir-test 2023-10-06T04:01:40.738 INFO:tasks.workunit.client.0.smithi157.stdout:./236/.snap/snap-subdir-test 2023-10-06T04:01:40.743 INFO:tasks.workunit.client.0.smithi157.stdout:./211/.snap/snap-subdir-test 2023-10-06T04:01:40.747 INFO:tasks.workunit.client.0.smithi157.stdout:./199/.snap/snap-subdir-test 2023-10-06T04:01:40.752 INFO:tasks.workunit.client.0.smithi157.stdout:./205/.snap/snap-subdir-test 2023-10-06T04:01:42.202 INFO:tasks.workunit.client.0.smithi157.stdout:./163/.snap/snap-subdir-test 2023-10-06T04:01:42.207 INFO:tasks.workunit.client.0.smithi157.stdout:./261/.snap/snap-subdir-test 2023-10-06T04:01:42.211 INFO:tasks.workunit.client.0.smithi157.stdout:./201/.snap/snap-subdir-test 2023-10-06T04:01:43.060 INFO:tasks.workunit.client.0.smithi157.stdout:./282/.snap/snap-subdir-test 2023-10-06T04:01:43.065 INFO:tasks.workunit.client.0.smithi157.stdout:./296/.snap/snap-subdir-test 2023-10-06T04:01:43.071 INFO:tasks.workunit.client.0.smithi157.stdout:./217/.snap/snap-subdir-test 2023-10-06T04:01:43.076 INFO:tasks.workunit.client.0.smithi157.stdout:./153/.snap/snap-subdir-test 2023-10-06T04:01:43.858 INFO:tasks.workunit.client.0.smithi157.stdout:./206/.snap/snap-subdir-test 2023-10-06T04:01:44.161 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:01:44.165 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:01:44.228 INFO:tasks.workunit.client.0.smithi157.stdout:./122/.snap/snap-subdir-test 2023-10-06T04:01:44.233 INFO:tasks.workunit.client.0.smithi157.stdout:./162/.snap/snap-subdir-test 2023-10-06T04:01:44.238 INFO:tasks.workunit.client.0.smithi157.stdout:./197/.snap/snap-subdir-test 2023-10-06T04:01:44.244 INFO:tasks.workunit.client.0.smithi157.stdout:./216/.snap/snap-subdir-test 2023-10-06T04:01:48.376 INFO:tasks.workunit.client.0.smithi157.stdout:./254/.snap/snap-subdir-test 2023-10-06T04:01:48.379 INFO:tasks.workunit.client.0.smithi157.stdout:./113/.snap/snap-subdir-test 2023-10-06T04:01:48.387 INFO:tasks.workunit.client.0.smithi157.stdout:./152/.snap/snap-subdir-test 2023-10-06T04:01:48.391 INFO:tasks.workunit.client.0.smithi157.stdout:./121/.snap/snap-subdir-test 2023-10-06T04:01:48.394 INFO:tasks.workunit.client.0.smithi157.stdout:./118/.snap/snap-subdir-test 2023-10-06T04:01:48.397 INFO:tasks.workunit.client.0.smithi157.stdout:./196/.snap/snap-subdir-test 2023-10-06T04:01:48.400 INFO:tasks.workunit.client.0.smithi157.stdout:./124/.snap/snap-subdir-test 2023-10-06T04:01:48.403 INFO:tasks.workunit.client.0.smithi157.stdout:./239/.snap/snap-subdir-test 2023-10-06T04:01:48.410 INFO:tasks.workunit.client.0.smithi157.stdout:./188/.snap/snap-subdir-test 2023-10-06T04:01:48.413 INFO:tasks.workunit.client.0.smithi157.stdout:./287/.snap/snap-subdir-test 2023-10-06T04:01:48.416 INFO:tasks.workunit.client.0.smithi157.stdout:./275/.snap/snap-subdir-test 2023-10-06T04:01:48.419 INFO:tasks.workunit.client.0.smithi157.stdout:./259/.snap/snap-subdir-test 2023-10-06T04:01:48.424 INFO:tasks.workunit.client.0.smithi157.stdout:./224/.snap/snap-subdir-test 2023-10-06T04:01:48.427 INFO:tasks.workunit.client.0.smithi157.stdout:./264/.snap/snap-subdir-test 2023-10-06T04:01:48.867 INFO:tasks.workunit.client.0.smithi157.stdout:./145/.snap/snap-subdir-test 2023-10-06T04:01:48.871 INFO:tasks.workunit.client.0.smithi157.stdout:./279/.snap/snap-subdir-test 2023-10-06T04:01:48.879 INFO:tasks.workunit.client.0.smithi157.stdout:./263/.snap/snap-subdir-test 2023-10-06T04:01:48.885 INFO:tasks.workunit.client.0.smithi157.stdout:./130/.snap/snap-subdir-test 2023-10-06T04:01:49.858 INFO:tasks.workunit.client.0.smithi157.stdout:./193/.snap/snap-subdir-test 2023-10-06T04:01:49.865 INFO:tasks.workunit.client.0.smithi157.stdout:./223/.snap/snap-subdir-test 2023-10-06T04:01:49.874 INFO:tasks.workunit.client.0.smithi157.stdout:./207/.snap/snap-subdir-test 2023-10-06T04:01:49.881 INFO:tasks.workunit.client.0.smithi157.stdout:./276/.snap/snap-subdir-test 2023-10-06T04:01:50.150 INFO:tasks.workunit.client.0.smithi157.stdout:./271/.snap/snap-subdir-test 2023-10-06T04:01:50.157 INFO:tasks.workunit.client.0.smithi157.stdout:./251/.snap/snap-subdir-test 2023-10-06T04:01:50.165 INFO:tasks.workunit.client.0.smithi157.stdout:./175/.snap/snap-subdir-test 2023-10-06T04:01:50.173 INFO:tasks.workunit.client.0.smithi157.stdout:./222/.snap/snap-subdir-test 2023-10-06T04:01:50.179 INFO:tasks.workunit.client.0.smithi157.stdout:./220/.snap/snap-subdir-test 2023-10-06T04:01:50.187 INFO:tasks.workunit.client.0.smithi157.stdout:./203/.snap/snap-subdir-test 2023-10-06T04:01:50.458 INFO:tasks.workunit.client.0.smithi157.stdout:./181/.snap/snap-subdir-test 2023-10-06T04:01:50.465 INFO:tasks.workunit.client.0.smithi157.stdout:./286/.snap/snap-subdir-test 2023-10-06T04:01:50.472 INFO:tasks.workunit.client.0.smithi157.stdout:./127/.snap/snap-subdir-test 2023-10-06T04:01:50.479 INFO:tasks.workunit.client.0.smithi157.stdout:./108/.snap/snap-subdir-test 2023-10-06T04:01:50.486 INFO:tasks.workunit.client.0.smithi157.stdout:./183/.snap/snap-subdir-test 2023-10-06T04:01:50.493 INFO:tasks.workunit.client.0.smithi157.stdout:./294/.snap/snap-subdir-test 2023-10-06T04:01:50.499 INFO:tasks.workunit.client.0.smithi157.stdout:./235/.snap/snap-subdir-test 2023-10-06T04:01:50.507 INFO:tasks.workunit.client.0.smithi157.stdout:./177/.snap/snap-subdir-test 2023-10-06T04:01:50.988 INFO:tasks.workunit.client.0.smithi157.stdout:./169/.snap/snap-subdir-test 2023-10-06T04:01:50.998 INFO:tasks.workunit.client.0.smithi157.stdout:./212/.snap/snap-subdir-test 2023-10-06T04:01:51.006 INFO:tasks.workunit.client.0.smithi157.stdout:./154/.snap/snap-subdir-test 2023-10-06T04:01:51.014 INFO:tasks.workunit.client.0.smithi157.stdout:./171/.snap/snap-subdir-test 2023-10-06T04:01:51.023 INFO:tasks.workunit.client.0.smithi157.stdout:./139/.snap/snap-subdir-test 2023-10-06T04:01:51.029 INFO:tasks.workunit.client.0.smithi157.stdout:./102/.snap/snap-subdir-test 2023-10-06T04:01:51.036 INFO:tasks.workunit.client.0.smithi157.stdout:./129/.snap/snap-subdir-test 2023-10-06T04:01:51.602 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:01:51.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.719+0000 7f2830a68700 1 Processor -- start 2023-10-06T04:01:51.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.719+0000 7f2830a68700 1 -- start start 2023-10-06T04:01:51.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.720+0000 7f2830a68700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f282c06cbb0 0x7f282c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:51.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.720+0000 7f2830a68700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f282c06c1f0 0x7f282c06da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:51.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.720+0000 7f2830a68700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f282c0c9780 con 0x7f282c06cf80 2023-10-06T04:01:51.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.720+0000 7f2830a68700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f282c0c8c40 con 0x7f282c06c1f0 2023-10-06T04:01:51.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.720+0000 7f2830a68700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f282c06e0c0 con 0x7f282c06cbb0 2023-10-06T04:01:51.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.720+0000 7f2829d9b700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f282c06c1f0 0x7f282c06da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:51.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.720+0000 7f2829d9b700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f282c06c1f0 0x7f282c06da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45190/0 (socket says 172.21.15.157:45190) 2023-10-06T04:01:51.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.720+0000 7f2829d9b700 1 -- 172.21.15.157:0/3018652651 learned_addr learned my addr 172.21.15.157:0/3018652651 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:01:51.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.720+0000 7f282a59c700 1 --2- 172.21.15.157:0/3018652651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f282c06cbb0 0x7f282c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:51.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.720+0000 7f2829d9b700 1 -- 172.21.15.157:0/3018652651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f282c06cbb0 msgr2=0x7f282c06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:01:51.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.720+0000 7f2829d9b700 1 --2- 172.21.15.157:0/3018652651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f282c06cbb0 0x7f282c06d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:51.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.720+0000 7f2829d9b700 1 -- 172.21.15.157:0/3018652651 >> v1:172.21.15.175:6789/0 conn(0x7f282c06cf80 legacy=0x7f282c14fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:51.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.720+0000 7f2829d9b700 1 -- 172.21.15.157:0/3018652651 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282c0c8e50 con 0x7f282c06c1f0 2023-10-06T04:01:51.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.720+0000 7f2829d9b700 1 --2- 172.21.15.157:0/3018652651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f282c06c1f0 0x7f282c06da70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f28140099e0 tx=0x7f2814009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d7fac25dc36341 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:51.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.720+0000 7f282959a700 1 -- 172.21.15.157:0/3018652651 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281400b8b0 con 0x7f282c06c1f0 2023-10-06T04:01:51.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.720+0000 7f282959a700 1 -- 172.21.15.157:0/3018652651 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f281400ba10 con 0x7f282c06c1f0 2023-10-06T04:01:51.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.720+0000 7f282959a700 1 -- 172.21.15.157:0/3018652651 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281400bdb0 con 0x7f282c06c1f0 2023-10-06T04:01:51.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.721+0000 7f2830a68700 1 -- 172.21.15.157:0/3018652651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f282c06c1f0 msgr2=0x7f282c06da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:51.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.721+0000 7f2830a68700 1 --2- 172.21.15.157:0/3018652651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f282c06c1f0 0x7f282c06da70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f28140099e0 tx=0x7f2814009250 comp rx=0 tx=0).stop 2023-10-06T04:01:51.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.721+0000 7f2830a68700 1 -- 172.21.15.157:0/3018652651 shutdown_connections 2023-10-06T04:01:51.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.721+0000 7f2830a68700 1 --2- 172.21.15.157:0/3018652651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f282c06cbb0 0x7f282c06d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:51.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.721+0000 7f2830a68700 1 --2- 172.21.15.157:0/3018652651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f282c06c1f0 0x7f282c06da70 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:51.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.721+0000 7f2830a68700 1 -- 172.21.15.157:0/3018652651 >> 172.21.15.157:0/3018652651 conn(0x7f282c06f230 msgr2=0x7f282c06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:51.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.721+0000 7f2830a68700 1 -- 172.21.15.157:0/3018652651 shutdown_connections 2023-10-06T04:01:51.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.721+0000 7f2830a68700 1 -- 172.21.15.157:0/3018652651 wait complete. 2023-10-06T04:01:51.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.722+0000 7f2830a68700 1 Processor -- start 2023-10-06T04:01:51.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.722+0000 7f2830a68700 1 -- start start 2023-10-06T04:01:51.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.723+0000 7f2830a68700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f282c06c1f0 0x7f282c14bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:51.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.723+0000 7f2830a68700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f282c06cbb0 0x7f282c0c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:51.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.723+0000 7f2830a68700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f282c06cf80 0x7f282c0c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:51.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.723+0000 7f2830a68700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282c1663f0 con 0x7f282c06cbb0 2023-10-06T04:01:51.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.723+0000 7f2830a68700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282c1666d0 con 0x7f282c06cf80 2023-10-06T04:01:51.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.723+0000 7f2830a68700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f282c1669b0 con 0x7f282c06c1f0 2023-10-06T04:01:51.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.723+0000 7f2829d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f282c06cbb0 0x7f282c0c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:51.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.723+0000 7f2829d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f282c06cbb0 0x7f282c0c60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45192/0 (socket says 172.21.15.157:45192) 2023-10-06T04:01:51.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.723+0000 7f2829d9b700 1 -- 172.21.15.157:0/1330723383 learned_addr learned my addr 172.21.15.157:0/1330723383 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:01:51.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.723+0000 7f282a59c700 1 --2- 172.21.15.157:0/1330723383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f282c06c1f0 0x7f282c14bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:51.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.723+0000 7f282ad9d700 1 --2- 172.21.15.157:0/1330723383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f282c06cf80 0x7f282c0c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:51.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.723+0000 7f2829d9b700 1 -- 172.21.15.157:0/1330723383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f282c06c1f0 msgr2=0x7f282c14bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:51.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.723+0000 7f2829d9b700 1 --2- 172.21.15.157:0/1330723383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f282c06c1f0 0x7f282c14bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:51.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.723+0000 7f2829d9b700 1 -- 172.21.15.157:0/1330723383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f282c06cf80 msgr2=0x7f282c0c3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:51.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.723+0000 7f2829d9b700 1 --2- 172.21.15.157:0/1330723383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f282c06cf80 0x7f282c0c3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:51.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.723+0000 7f2829d9b700 1 -- 172.21.15.157:0/1330723383 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28140094f0 con 0x7f282c06cbb0 2023-10-06T04:01:51.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.723+0000 7f2829d9b700 1 --2- 172.21.15.157:0/1330723383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f282c06cbb0 0x7f282c0c60f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f282c0c3960 tx=0x7f2814017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:51.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.723+0000 7f28227fc700 1 -- 172.21.15.157:0/1330723383 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2814026450 con 0x7f282c06cbb0 2023-10-06T04:01:51.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.723+0000 7f28227fc700 1 -- 172.21.15.157:0/1330723383 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28140265d0 con 0x7f282c06cbb0 2023-10-06T04:01:51.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.723+0000 7f2830a68700 1 -- 172.21.15.157:0/1330723383 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f282c166c90 con 0x7f282c06cbb0 2023-10-06T04:01:51.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.723+0000 7f28227fc700 1 -- 172.21.15.157:0/1330723383 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2814026990 con 0x7f282c06cbb0 2023-10-06T04:01:51.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.723+0000 7f2830a68700 1 -- 172.21.15.157:0/1330723383 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f282c14c1a0 con 0x7f282c06cbb0 2023-10-06T04:01:51.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.724+0000 7f2830a68700 1 -- 172.21.15.157:0/1330723383 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f282c0c3a20 con 0x7f282c06cbb0 2023-10-06T04:01:51.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.725+0000 7f28227fc700 1 -- 172.21.15.157:0/1330723383 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f281401a420 con 0x7f282c06cbb0 2023-10-06T04:01:51.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.725+0000 7f28227fc700 1 --2- 172.21.15.157:0/1330723383 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f280c066f30 0x7f280c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:51.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.725+0000 7f28227fc700 1 -- 172.21.15.157:0/1330723383 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f28140866f0 con 0x7f282c06cbb0 2023-10-06T04:01:51.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.725+0000 7f282a59c700 1 --2- 172.21.15.157:0/1330723383 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f280c066f30 0x7f280c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:51.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.726+0000 7f282a59c700 1 --2- 172.21.15.157:0/1330723383 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f280c066f30 0x7f280c0693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2824009fd0 tx=0x7f2824009880 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:51.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.729+0000 7f28227fc700 1 -- 172.21.15.157:0/1330723383 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f282c0c3a20 con 0x7f282c06cbb0 2023-10-06T04:01:51.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.962+0000 7f2830a68700 1 -- 172.21.15.157:0/1330723383 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f282c167280 con 0x7f282c06cbb0 2023-10-06T04:01:51.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.963+0000 7f28227fc700 1 -- 172.21.15.157:0/1330723383 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+6201 (secure 0 0 0) 0x7f28140537a0 con 0x7f282c06cbb0 2023-10-06T04:01:51.965 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:01:51.965 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:01:46.779174+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"f","rank":4,"incarnation":29,"state":"up:active","state_seq":32,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[0,3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:01:51.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.965+0000 7f2830a68700 1 -- 172.21.15.157:0/1330723383 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f280c066f30 msgr2=0x7f280c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:51.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.965+0000 7f2830a68700 1 --2- 172.21.15.157:0/1330723383 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f280c066f30 0x7f280c0693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2824009fd0 tx=0x7f2824009880 comp rx=0 tx=0).stop 2023-10-06T04:01:51.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.965+0000 7f2830a68700 1 -- 172.21.15.157:0/1330723383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f282c06cbb0 msgr2=0x7f282c0c60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:51.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.965+0000 7f2830a68700 1 --2- 172.21.15.157:0/1330723383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f282c06cbb0 0x7f282c0c60f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f282c0c3960 tx=0x7f2814017000 comp rx=0 tx=0).stop 2023-10-06T04:01:51.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.965+0000 7f2830a68700 1 -- 172.21.15.157:0/1330723383 shutdown_connections 2023-10-06T04:01:51.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.965+0000 7f2830a68700 1 --2- 172.21.15.157:0/1330723383 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f280c066f30 0x7f280c0693e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:51.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.965+0000 7f2830a68700 1 --2- 172.21.15.157:0/1330723383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f282c06cf80 0x7f282c0c3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:51.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.965+0000 7f2830a68700 1 --2- 172.21.15.157:0/1330723383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f282c06c1f0 0x7f282c14bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:51.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.965+0000 7f2830a68700 1 --2- 172.21.15.157:0/1330723383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f282c06cbb0 0x7f282c0c60f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:51.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.965+0000 7f2830a68700 1 -- 172.21.15.157:0/1330723383 >> 172.21.15.157:0/1330723383 conn(0x7f282c06f230 msgr2=0x7f282c14e670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:51.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.965+0000 7f2830a68700 1 -- 172.21.15.157:0/1330723383 shutdown_connections 2023-10-06T04:01:51.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:51.965+0000 7f2830a68700 1 -- 172.21.15.157:0/1330723383 wait complete. 2023-10-06T04:01:51.968 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 36 2023-10-06T04:01:51.979 INFO:tasks.mds_thrash.fs.[cephfs]:thrashing max_mds: 5 -> 2 2023-10-06T04:01:51.979 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2023-10-06T04:01:52.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.101+0000 7f5ffc40c700 1 Processor -- start 2023-10-06T04:01:52.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.101+0000 7f5ffc40c700 1 -- start start 2023-10-06T04:01:52.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.102+0000 7f5ffc40c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5ff406cbb0 0x7f5ff406d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:52.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.102+0000 7f5ffc40c700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5ff406c1f0 0x7f5ff406da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:52.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.102+0000 7f5ffc40c700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff40c9780 con 0x7f5ff406cf80 2023-10-06T04:01:52.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.102+0000 7f5ffc40c700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff40c8c40 con 0x7f5ff406cbb0 2023-10-06T04:01:52.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.102+0000 7f5ffc40c700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff406e0c0 con 0x7f5ff406c1f0 2023-10-06T04:01:52.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.102+0000 7f5ffa1a8700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5ff406cbb0 0x7f5ff406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:52.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.102+0000 7f5ffa9a9700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f5ff406cf80 0x7f5ff414fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49876/0 (socket says 172.21.15.157:49876) 2023-10-06T04:01:52.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.102+0000 7f5ffa9a9700 1 -- 172.21.15.157:0/3429419876 learned_addr learned my addr 172.21.15.157:0/3429419876 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:01:52.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.102+0000 7f5ff99a7700 1 --2- 172.21.15.157:0/3429419876 >> v2:172.21.15.175:3300/0 conn(0x7f5ff406c1f0 0x7f5ff406da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:52.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.102+0000 7f5ffa1a8700 1 -- 172.21.15.157:0/3429419876 >> v2:172.21.15.175:3300/0 conn(0x7f5ff406c1f0 msgr2=0x7f5ff406da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:01:52.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.102+0000 7f5ffa1a8700 1 --2- 172.21.15.157:0/3429419876 >> v2:172.21.15.175:3300/0 conn(0x7f5ff406c1f0 0x7f5ff406da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:52.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.102+0000 7f5ffa1a8700 1 -- 172.21.15.157:0/3429419876 >> v1:172.21.15.157:6789/0 conn(0x7f5ff406cf80 legacy=0x7f5ff414fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:52.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.102+0000 7f5ff91a6700 1 -- 172.21.15.157:0/3429419876 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2041034079 0 0) 0x7f5ff40c9780 con 0x7f5ff406cf80 2023-10-06T04:01:52.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.102+0000 7f5ffa1a8700 1 -- 172.21.15.157:0/3429419876 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff40c8e50 con 0x7f5ff406cbb0 2023-10-06T04:01:52.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.103+0000 7f5ffa1a8700 1 --2- 172.21.15.157:0/3429419876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff406cbb0 0x7f5ff406d530 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4157070 tx=0x7f5ff00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742881cece24323b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:52.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.103+0000 7f5ff91a6700 1 -- 172.21.15.157:0/3429419876 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff000b740 con 0x7f5ff406cbb0 2023-10-06T04:01:52.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.103+0000 7f5ff91a6700 1 -- 172.21.15.157:0/3429419876 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff000b8a0 con 0x7f5ff406cbb0 2023-10-06T04:01:52.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.103+0000 7f5ff91a6700 1 -- 172.21.15.157:0/3429419876 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff000bc40 con 0x7f5ff406cbb0 2023-10-06T04:01:52.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.103+0000 7f5ffc40c700 1 -- 172.21.15.157:0/3429419876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff406cbb0 msgr2=0x7f5ff406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:52.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.103+0000 7f5ffc40c700 1 --2- 172.21.15.157:0/3429419876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff406cbb0 0x7f5ff406d530 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4157070 tx=0x7f5ff00091a0 comp rx=0 tx=0).stop 2023-10-06T04:01:52.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.103+0000 7f5ffc40c700 1 -- 172.21.15.157:0/3429419876 shutdown_connections 2023-10-06T04:01:52.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.103+0000 7f5ffc40c700 1 --2- 172.21.15.157:0/3429419876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff406cbb0 0x7f5ff406d530 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:52.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.103+0000 7f5ffc40c700 1 --2- 172.21.15.157:0/3429419876 >> v2:172.21.15.175:3300/0 conn(0x7f5ff406c1f0 0x7f5ff406da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:52.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.103+0000 7f5ffc40c700 1 -- 172.21.15.157:0/3429419876 >> 172.21.15.157:0/3429419876 conn(0x7f5ff406f230 msgr2=0x7f5ff406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:52.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.103+0000 7f5ffc40c700 1 -- 172.21.15.157:0/3429419876 shutdown_connections 2023-10-06T04:01:52.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.103+0000 7f5ffc40c700 1 -- 172.21.15.157:0/3429419876 wait complete. 2023-10-06T04:01:52.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.104+0000 7f5ffc40c700 1 Processor -- start 2023-10-06T04:01:52.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.105+0000 7f5ffc40c700 1 -- start start 2023-10-06T04:01:52.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.105+0000 7f5ffc40c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ff406c1f0 0x7f5ff40bfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:52.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.105+0000 7f5ffc40c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff406cbb0 0x7f5ff40c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:52.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.105+0000 7f5ffc40c700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ff406cf80 0x7f5ff4147ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:52.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.105+0000 7f5ffc40c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff40c01b0 con 0x7f5ff406cbb0 2023-10-06T04:01:52.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.105+0000 7f5ffc40c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff41667d0 con 0x7f5ff406c1f0 2023-10-06T04:01:52.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.105+0000 7f5ffc40c700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff40c03b0 con 0x7f5ff406cf80 2023-10-06T04:01:52.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.106+0000 7f5ff99a7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff406cbb0 0x7f5ff40c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:52.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.106+0000 7f5ff99a7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff406cbb0 0x7f5ff40c2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45204/0 (socket says 172.21.15.157:45204) 2023-10-06T04:01:52.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.106+0000 7f5ffa1a8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ff406c1f0 0x7f5ff40bfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:52.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.106+0000 7f5ff99a7700 1 -- 172.21.15.157:0/1794910337 learned_addr learned my addr 172.21.15.157:0/1794910337 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:01:52.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.106+0000 7f5ffa9a9700 1 --2- 172.21.15.157:0/1794910337 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ff406cf80 0x7f5ff4147ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:52.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.106+0000 7f5ff99a7700 1 -- 172.21.15.157:0/1794910337 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ff406cf80 msgr2=0x7f5ff4147ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:52.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.106+0000 7f5ff99a7700 1 --2- 172.21.15.157:0/1794910337 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ff406cf80 0x7f5ff4147ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:52.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.106+0000 7f5ff99a7700 1 -- 172.21.15.157:0/1794910337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ff406c1f0 msgr2=0x7f5ff40bfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:52.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.106+0000 7f5ff99a7700 1 --2- 172.21.15.157:0/1794910337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ff406c1f0 0x7f5ff40bfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:52.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.106+0000 7f5ff99a7700 1 -- 172.21.15.157:0/1794910337 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff0009420 con 0x7f5ff406cbb0 2023-10-06T04:01:52.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.106+0000 7f5ff99a7700 1 --2- 172.21.15.157:0/1794910337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff406cbb0 0x7f5ff40c2da0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400d890 tx=0x7f5fe4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:52.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.106+0000 7f5fea7fc700 1 -- 172.21.15.157:0/1794910337 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe40125f0 con 0x7f5ff406cbb0 2023-10-06T04:01:52.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.106+0000 7f5fea7fc700 1 -- 172.21.15.157:0/1794910337 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe4012750 con 0x7f5ff406cbb0 2023-10-06T04:01:52.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.106+0000 7f5ffc40c700 1 -- 172.21.15.157:0/1794910337 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff4155420 con 0x7f5ff406cbb0 2023-10-06T04:01:52.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.106+0000 7f5fea7fc700 1 -- 172.21.15.157:0/1794910337 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4012af0 con 0x7f5ff406cbb0 2023-10-06T04:01:52.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.106+0000 7f5ffc40c700 1 -- 172.21.15.157:0/1794910337 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff40c32e0 con 0x7f5ff406cbb0 2023-10-06T04:01:52.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.107+0000 7f5ffc40c700 1 -- 172.21.15.157:0/1794910337 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff4166940 con 0x7f5ff406cbb0 2023-10-06T04:01:52.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.108+0000 7f5fea7fc700 1 -- 172.21.15.157:0/1794910337 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5fe401e460 con 0x7f5ff406cbb0 2023-10-06T04:01:52.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.108+0000 7f5fea7fc700 1 --2- 172.21.15.157:0/1794910337 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5fd8066f30 0x7f5fd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:52.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.108+0000 7f5fea7fc700 1 -- 172.21.15.157:0/1794910337 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f5fe4084cc0 con 0x7f5ff406cbb0 2023-10-06T04:01:52.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.108+0000 7f5ffa1a8700 1 --2- 172.21.15.157:0/1794910337 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5fd8066f30 0x7f5fd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:52.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.109+0000 7f5ffa1a8700 1 --2- 172.21.15.157:0/1794910337 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5fd8066f30 0x7f5fd80693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00093f0 tx=0x7f5ff0017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:52.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.112+0000 7f5fea7fc700 1 -- 172.21.15.157:0/1794910337 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5fe4055780 con 0x7f5ff406cbb0 2023-10-06T04:01:52.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.341+0000 7f5ffc40c700 1 -- 172.21.15.157:0/1794910337 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f5ff40c9130 con 0x7f5ff406cbb0 2023-10-06T04:01:52.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.800+0000 7f5fea7fc700 1 -- 172.21.15.157:0/1794910337 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v37) v1 ==== 105+0+0 (secure 0 0 0) 0x7f5fe4055310 con 0x7f5ff406cbb0 2023-10-06T04:01:52.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.802+0000 7f5ffc40c700 1 -- 172.21.15.157:0/1794910337 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5fd8066f30 msgr2=0x7f5fd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:52.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.802+0000 7f5ffc40c700 1 --2- 172.21.15.157:0/1794910337 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5fd8066f30 0x7f5fd80693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00093f0 tx=0x7f5ff0017000 comp rx=0 tx=0).stop 2023-10-06T04:01:52.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.802+0000 7f5ffc40c700 1 -- 172.21.15.157:0/1794910337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff406cbb0 msgr2=0x7f5ff40c2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:52.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.802+0000 7f5ffc40c700 1 --2- 172.21.15.157:0/1794910337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff406cbb0 0x7f5ff40c2da0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400d890 tx=0x7f5fe4008040 comp rx=0 tx=0).stop 2023-10-06T04:01:52.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.802+0000 7f5ffc40c700 1 -- 172.21.15.157:0/1794910337 shutdown_connections 2023-10-06T04:01:52.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.802+0000 7f5ffc40c700 1 --2- 172.21.15.157:0/1794910337 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5fd8066f30 0x7f5fd80693e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:52.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.802+0000 7f5ffc40c700 1 --2- 172.21.15.157:0/1794910337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ff406c1f0 0x7f5ff40bfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:52.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.802+0000 7f5ffc40c700 1 --2- 172.21.15.157:0/1794910337 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ff406cf80 0x7f5ff4147ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:52.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.802+0000 7f5ffc40c700 1 --2- 172.21.15.157:0/1794910337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff406cbb0 0x7f5ff40c2da0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:52.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.802+0000 7f5ffc40c700 1 -- 172.21.15.157:0/1794910337 >> 172.21.15.157:0/1794910337 conn(0x7f5ff406f230 msgr2=0x7f5ff406d530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:52.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.802+0000 7f5ffc40c700 1 -- 172.21.15.157:0/1794910337 shutdown_connections 2023-10-06T04:01:52.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.802+0000 7f5ffc40c700 1 -- 172.21.15.157:0/1794910337 wait complete. 2023-10-06T04:01:52.816 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-10-06T04:01:52.816 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:01:52.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.930+0000 7f5edabb8700 1 Processor -- start 2023-10-06T04:01:52.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.930+0000 7f5edabb8700 1 -- start start 2023-10-06T04:01:52.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.931+0000 7f5edabb8700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5ed40b04d0 0x7f5ed4157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:52.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.931+0000 7f5edabb8700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ed406c2d0 0x7f5ed406dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:52.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.931+0000 7f5edabb8700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed406d8d0 con 0x7f5ed40b1e40 2023-10-06T04:01:52.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.931+0000 7f5edabb8700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ed40af8f0 con 0x7f5ed40b04d0 2023-10-06T04:01:52.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.931+0000 7f5edabb8700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed415a690 con 0x7f5ed406c2d0 2023-10-06T04:01:52.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.931+0000 7f5ed93b5700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5ed40b04d0 0x7f5ed4157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:52.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.931+0000 7f5ed9bb6700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f5ed40b1e40 0x7f5ed41563b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49890/0 (socket says 172.21.15.157:49890) 2023-10-06T04:01:52.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.931+0000 7f5ed9bb6700 1 -- 172.21.15.157:0/1559072600 learned_addr learned my addr 172.21.15.157:0/1559072600 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:01:52.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.931+0000 7f5ed93b5700 1 -- 172.21.15.157:0/1559072600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ed406c2d0 msgr2=0x7f5ed406dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:01:52.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.931+0000 7f5eda3b7700 1 --2- 172.21.15.157:0/1559072600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ed406c2d0 0x7f5ed406dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:52.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.931+0000 7f5ed8bb4700 1 -- 172.21.15.157:0/1559072600 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2643558470 0 0) 0x7f5ed406d8d0 con 0x7f5ed40b1e40 2023-10-06T04:01:52.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.931+0000 7f5ed93b5700 1 --2- 172.21.15.157:0/1559072600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ed406c2d0 0x7f5ed406dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:52.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.931+0000 7f5ed93b5700 1 -- 172.21.15.157:0/1559072600 >> v1:172.21.15.157:6789/0 conn(0x7f5ed40b1e40 legacy=0x7f5ed41563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:52.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.931+0000 7f5ed93b5700 1 -- 172.21.15.157:0/1559072600 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed406ccc0 con 0x7f5ed40b04d0 2023-10-06T04:01:52.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.932+0000 7f5ed93b5700 1 --2- 172.21.15.157:0/1559072600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ed40b04d0 0x7f5ed4157750 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5ec40098e0 tx=0x7f5ec40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682cfadda1618fa1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:52.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.932+0000 7f5ed8bb4700 1 -- 172.21.15.157:0/1559072600 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4009e80 con 0x7f5ed40b04d0 2023-10-06T04:01:52.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.932+0000 7f5ed8bb4700 1 -- 172.21.15.157:0/1559072600 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec400b860 con 0x7f5ed40b04d0 2023-10-06T04:01:52.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.932+0000 7f5ed8bb4700 1 -- 172.21.15.157:0/1559072600 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec400bbe0 con 0x7f5ed40b04d0 2023-10-06T04:01:52.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.932+0000 7f5edabb8700 1 -- 172.21.15.157:0/1559072600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ed40b04d0 msgr2=0x7f5ed4157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:52.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.932+0000 7f5edabb8700 1 --2- 172.21.15.157:0/1559072600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ed40b04d0 0x7f5ed4157750 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5ec40098e0 tx=0x7f5ec40091a0 comp rx=0 tx=0).stop 2023-10-06T04:01:52.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.932+0000 7f5edabb8700 1 -- 172.21.15.157:0/1559072600 shutdown_connections 2023-10-06T04:01:52.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.932+0000 7f5edabb8700 1 --2- 172.21.15.157:0/1559072600 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ed406c2d0 0x7f5ed406dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:52.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.932+0000 7f5edabb8700 1 --2- 172.21.15.157:0/1559072600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ed40b04d0 0x7f5ed4157750 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:52.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.932+0000 7f5edabb8700 1 -- 172.21.15.157:0/1559072600 >> 172.21.15.157:0/1559072600 conn(0x7f5ed40cc540 msgr2=0x7f5ed40cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:52.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.932+0000 7f5edabb8700 1 -- 172.21.15.157:0/1559072600 shutdown_connections 2023-10-06T04:01:52.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.932+0000 7f5edabb8700 1 -- 172.21.15.157:0/1559072600 wait complete. 2023-10-06T04:01:52.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.933+0000 7f5edabb8700 1 Processor -- start 2023-10-06T04:01:52.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.933+0000 7f5edabb8700 1 -- start start 2023-10-06T04:01:52.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.934+0000 7f5edabb8700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ed406c2d0 0x7f5ed40c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:52.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.934+0000 7f5edabb8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ed40b04d0 0x7f5ed40c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:52.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.934+0000 7f5edabb8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ed40b1e40 0x7f5ed40c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:52.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.934+0000 7f5edabb8700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed40c9c20 con 0x7f5ed40b04d0 2023-10-06T04:01:52.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.934+0000 7f5edabb8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed4155b10 con 0x7f5ed40b1e40 2023-10-06T04:01:52.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.934+0000 7f5edabb8700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed4155df0 con 0x7f5ed406c2d0 2023-10-06T04:01:52.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.934+0000 7f5ed93b5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ed40b04d0 0x7f5ed40c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:52.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.934+0000 7f5ed93b5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ed40b04d0 0x7f5ed40c6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45214/0 (socket says 172.21.15.157:45214) 2023-10-06T04:01:52.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.934+0000 7f5ed93b5700 1 -- 172.21.15.157:0/1438757467 learned_addr learned my addr 172.21.15.157:0/1438757467 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:01:52.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.934+0000 7f5ed9bb6700 1 --2- 172.21.15.157:0/1438757467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ed406c2d0 0x7f5ed40c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:52.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.934+0000 7f5eda3b7700 1 --2- 172.21.15.157:0/1438757467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ed40b1e40 0x7f5ed40c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:52.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.934+0000 7f5ed93b5700 1 -- 172.21.15.157:0/1438757467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ed406c2d0 msgr2=0x7f5ed40c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:52.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.934+0000 7f5ed93b5700 1 --2- 172.21.15.157:0/1438757467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ed406c2d0 0x7f5ed40c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:52.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.934+0000 7f5ed93b5700 1 -- 172.21.15.157:0/1438757467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ed40b1e40 msgr2=0x7f5ed40c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:52.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.934+0000 7f5ed93b5700 1 --2- 172.21.15.157:0/1438757467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ed40b1e40 0x7f5ed40c7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:52.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.934+0000 7f5ed93b5700 1 -- 172.21.15.157:0/1438757467 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec4009420 con 0x7f5ed40b04d0 2023-10-06T04:01:52.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.934+0000 7f5ed9bb6700 1 --2- 172.21.15.157:0/1438757467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ed406c2d0 0x7f5ed40c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:01:52.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.935+0000 7f5ed93b5700 1 --2- 172.21.15.157:0/1438757467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ed40b04d0 0x7f5ed40c6c10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4013000 tx=0x7f5ec4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:52.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.935+0000 7f5ec9ffb700 1 -- 172.21.15.157:0/1438757467 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec400b9d0 con 0x7f5ed40b04d0 2023-10-06T04:01:52.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.935+0000 7f5ec9ffb700 1 -- 172.21.15.157:0/1438757467 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec400bb50 con 0x7f5ed40b04d0 2023-10-06T04:01:52.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.935+0000 7f5edabb8700 1 -- 172.21.15.157:0/1438757467 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed40c92f0 con 0x7f5ed40b04d0 2023-10-06T04:01:52.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.935+0000 7f5ec9ffb700 1 -- 172.21.15.157:0/1438757467 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4026680 con 0x7f5ed40b04d0 2023-10-06T04:01:52.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.935+0000 7f5edabb8700 1 -- 172.21.15.157:0/1438757467 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed40c98b0 con 0x7f5ed40b04d0 2023-10-06T04:01:52.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.936+0000 7f5edabb8700 1 -- 172.21.15.157:0/1438757467 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed4088f90 con 0x7f5ed40b04d0 2023-10-06T04:01:52.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.936+0000 7f5ec9ffb700 1 -- 172.21.15.157:0/1438757467 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5ec4023030 con 0x7f5ed40b04d0 2023-10-06T04:01:52.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.937+0000 7f5ec9ffb700 1 --2- 172.21.15.157:0/1438757467 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5eb8066f60 0x7f5eb8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:52.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.937+0000 7f5ec9ffb700 1 -- 172.21.15.157:0/1438757467 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f5ec40861a0 con 0x7f5ed40b04d0 2023-10-06T04:01:52.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.937+0000 7f5ed9bb6700 1 --2- 172.21.15.157:0/1438757467 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5eb8066f60 0x7f5eb8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:52.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.937+0000 7f5ed9bb6700 1 --2- 172.21.15.157:0/1438757467 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5eb8066f60 0x7f5eb8069410 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5ed00011e0 tx=0x7f5ed0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:52.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:52.941+0000 7f5ec9ffb700 1 -- 172.21.15.157:0/1438757467 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ec4053250 con 0x7f5ed40b04d0 2023-10-06T04:01:53.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:53.165+0000 7f5edabb8700 1 -- 172.21.15.157:0/1438757467 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ed406cfa0 con 0x7f5ed40b04d0 2023-10-06T04:01:53.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:53.165+0000 7f5ec9ffb700 1 -- 172.21.15.157:0/1438757467 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+6203 (secure 0 0 0) 0x7f5ec4056870 con 0x7f5ed40b04d0 2023-10-06T04:01:53.167 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:01:53.167 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:01:52.801202+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"f","rank":4,"incarnation":29,"state":"up:stopping","state_seq":32,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[0,3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:01:53.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:53.167+0000 7f5edabb8700 1 -- 172.21.15.157:0/1438757467 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5eb8066f60 msgr2=0x7f5eb8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:53.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:53.167+0000 7f5edabb8700 1 --2- 172.21.15.157:0/1438757467 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5eb8066f60 0x7f5eb8069410 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5ed00011e0 tx=0x7f5ed0009040 comp rx=0 tx=0).stop 2023-10-06T04:01:53.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:53.167+0000 7f5edabb8700 1 -- 172.21.15.157:0/1438757467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ed40b04d0 msgr2=0x7f5ed40c6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:53.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:53.167+0000 7f5edabb8700 1 --2- 172.21.15.157:0/1438757467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ed40b04d0 0x7f5ed40c6c10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4013000 tx=0x7f5ec4017000 comp rx=0 tx=0).stop 2023-10-06T04:01:53.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:53.167+0000 7f5edabb8700 1 -- 172.21.15.157:0/1438757467 shutdown_connections 2023-10-06T04:01:53.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:53.167+0000 7f5edabb8700 1 --2- 172.21.15.157:0/1438757467 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5eb8066f60 0x7f5eb8069410 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:53.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:53.167+0000 7f5edabb8700 1 --2- 172.21.15.157:0/1438757467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ed40b1e40 0x7f5ed40c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:53.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:53.167+0000 7f5edabb8700 1 --2- 172.21.15.157:0/1438757467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ed406c2d0 0x7f5ed40c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:53.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:53.167+0000 7f5edabb8700 1 --2- 172.21.15.157:0/1438757467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ed40b04d0 0x7f5ed40c6c10 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:53.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:53.167+0000 7f5edabb8700 1 -- 172.21.15.157:0/1438757467 >> 172.21.15.157:0/1438757467 conn(0x7f5ed40cc540 msgr2=0x7f5ed40c4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:53.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:53.167+0000 7f5edabb8700 1 -- 172.21.15.157:0/1438757467 shutdown_connections 2023-10-06T04:01:53.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:53.167+0000 7f5edabb8700 1 -- 172.21.15.157:0/1438757467 wait complete. 2023-10-06T04:01:53.170 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 38 2023-10-06T04:01:53.183 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2023-10-06T04:01:53.183 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 38, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-06T03:57:10.718782+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 38, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0, 1, 2, 3, 4 ], "info": { "gid_4270": { "addr": "172.21.15.175:6835/1596042861", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6834", "nonce": 1596042861, "type": "v2" }, { "addr": "172.21.15.175:6835", "nonce": 1596042861, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4270, "incarnation": 13, "join_fscid": -1, "name": "b", "rank": 3, "state": "up:active", "state_seq": 8 }, "gid_4279": { "addr": "172.21.15.175:6837/1002404870", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6836", "nonce": 1002404870, "type": "v2" }, { "addr": "172.21.15.175:6837", "nonce": 1002404870, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1, 3, 4 ], "features": 4540138320759226367, "flags": 0, "gid": 4279, "incarnation": 11, "join_fscid": -1, "name": "d", "rank": 2, "state": "up:active", "state_seq": 7 }, "gid_4287": { "addr": "172.21.15.157:6834/1179753185", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6832", "nonce": 1179753185, "type": "v2" }, { "addr": "172.21.15.157:6834", "nonce": 1179753185, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4287, "incarnation": 9, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4296": { "addr": "172.21.15.157:6837/4188652695", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6836", "nonce": 4188652695, "type": "v2" }, { "addr": "172.21.15.157:6837", "nonce": 4188652695, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2, 4 ], "features": 4540138320759226367, "flags": 0, "gid": 4296, "incarnation": 4, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 4 }, "gid_4936": { "addr": "172.21.15.175:6839/3353503675", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6838", "nonce": 3353503675, "type": "v2" }, { "addr": "172.21.15.175:6839", "nonce": 3353503675, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 3 ], "features": 4540138320759226367, "flags": 0, "gid": 4936, "incarnation": 29, "join_fscid": -1, "name": "f", "rank": 4, "state": "up:stopping", "state_seq": 32 } }, "last_failure": 0, "last_failure_osd_epoch": 27, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2023-10-06T04:01:52.801202+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4296, "mds_1": 4287, "mds_2": 4279, "mds_3": 4270, "mds_4": 4936 } } } ], "standbys": [ { "addr": "172.21.15.157:6835/2461751318", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6833", "nonce": 2461751318, "type": "v2" }, { "addr": "172.21.15.157:6835", "nonce": 2461751318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 34, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5268, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-06T04:01:55.185 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:01:55.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.308+0000 7fef24a24700 1 Processor -- start 2023-10-06T04:01:55.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.308+0000 7fef24a24700 1 -- start start 2023-10-06T04:01:55.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.309+0000 7fef24a24700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fef2006cbb0 0x7fef2006d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:55.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.309+0000 7fef24a24700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef200c9780 con 0x7fef2006c1f0 2023-10-06T04:01:55.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.309+0000 7fef24a24700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef2006e0c0 con 0x7fef20150410 2023-10-06T04:01:55.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.309+0000 7fef24a24700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef200c8c40 con 0x7fef2006cbb0 2023-10-06T04:01:55.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.309+0000 7fef15d9b700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fef2006c1f0 0x7fef2014fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:35634/0 (socket says 172.21.15.157:35634) 2023-10-06T04:01:55.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.309+0000 7fef15d9b700 1 -- 172.21.15.157:0/3035714451 learned_addr learned my addr 172.21.15.157:0/3035714451 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:01:55.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.310+0000 7fef1e59c700 1 --2- 172.21.15.157:0/3035714451 >> v2:172.21.15.175:3300/0 conn(0x7fef2006cbb0 0x7fef2006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:55.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.310+0000 7fef1dd9b700 1 -- 172.21.15.157:0/3035714451 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4034405326 0 0) 0x7fef200c9780 con 0x7fef2006c1f0 2023-10-06T04:01:55.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.310+0000 7fef1dd9b700 1 -- 172.21.15.157:0/3035714451 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef0c003570 con 0x7fef2006c1f0 2023-10-06T04:01:55.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.310+0000 7fef1e59c700 1 -- 172.21.15.157:0/3035714451 >> v1:172.21.15.175:6789/0 conn(0x7fef20150410 legacy=0x7fef20155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:55.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.310+0000 7fef1e59c700 1 -- 172.21.15.157:0/3035714451 >> v1:172.21.15.157:6789/0 conn(0x7fef2006c1f0 legacy=0x7fef2014fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:55.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.310+0000 7fef1e59c700 1 -- 172.21.15.157:0/3035714451 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef200c8e50 con 0x7fef2006cbb0 2023-10-06T04:01:55.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.310+0000 7fef1e59c700 1 --2- 172.21.15.157:0/3035714451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef2006cbb0 0x7fef2006d530 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fef100098e0 tx=0x7fef100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f975555d269d578c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:55.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.310+0000 7fef1dd9b700 1 -- 172.21.15.157:0/3035714451 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1000b6d0 con 0x7fef2006cbb0 2023-10-06T04:01:55.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.310+0000 7fef1dd9b700 1 -- 172.21.15.157:0/3035714451 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1000b830 con 0x7fef2006cbb0 2023-10-06T04:01:55.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.310+0000 7fef1dd9b700 1 -- 172.21.15.157:0/3035714451 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1000bbd0 con 0x7fef2006cbb0 2023-10-06T04:01:55.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.311+0000 7fef24a24700 1 -- 172.21.15.157:0/3035714451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef2006cbb0 msgr2=0x7fef2006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:55.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.311+0000 7fef24a24700 1 --2- 172.21.15.157:0/3035714451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef2006cbb0 0x7fef2006d530 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fef100098e0 tx=0x7fef100091a0 comp rx=0 tx=0).stop 2023-10-06T04:01:55.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.311+0000 7fef24a24700 1 -- 172.21.15.157:0/3035714451 shutdown_connections 2023-10-06T04:01:55.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.311+0000 7fef24a24700 1 --2- 172.21.15.157:0/3035714451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef2006cbb0 0x7fef2006d530 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:55.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.311+0000 7fef24a24700 1 -- 172.21.15.157:0/3035714451 >> 172.21.15.157:0/3035714451 conn(0x7fef2006f230 msgr2=0x7fef2006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:55.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.311+0000 7fef24a24700 1 -- 172.21.15.157:0/3035714451 shutdown_connections 2023-10-06T04:01:55.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.311+0000 7fef24a24700 1 -- 172.21.15.157:0/3035714451 wait complete. 2023-10-06T04:01:55.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.312+0000 7fef24a24700 1 Processor -- start 2023-10-06T04:01:55.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.312+0000 7fef24a24700 1 -- start start 2023-10-06T04:01:55.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.313+0000 7fef24a24700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef2006c1f0 0x7fef200c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:55.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.313+0000 7fef24a24700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef2006cbb0 0x7fef200c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:55.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.313+0000 7fef24a24700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef20150410 0x7fef20155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:55.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.313+0000 7fef24a24700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef201543a0 con 0x7fef2006cbb0 2023-10-06T04:01:55.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.313+0000 7fef24a24700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef200c3fe0 con 0x7fef20150410 2023-10-06T04:01:55.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.313+0000 7fef24a24700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef20153bb0 con 0x7fef2006c1f0 2023-10-06T04:01:55.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.313+0000 7fef15d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef2006cbb0 0x7fef200c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:55.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.313+0000 7fef15d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef2006cbb0 0x7fef200c4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47232/0 (socket says 172.21.15.157:47232) 2023-10-06T04:01:55.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.313+0000 7fef15d9b700 1 -- 172.21.15.157:0/2757197619 learned_addr learned my addr 172.21.15.157:0/2757197619 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:01:55.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.313+0000 7fef1ed9d700 1 --2- 172.21.15.157:0/2757197619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef20150410 0x7fef20155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:55.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.313+0000 7fef1e59c700 1 --2- 172.21.15.157:0/2757197619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef2006c1f0 0x7fef200c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:55.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.313+0000 7fef15d9b700 1 -- 172.21.15.157:0/2757197619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef2006c1f0 msgr2=0x7fef200c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:55.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.313+0000 7fef15d9b700 1 --2- 172.21.15.157:0/2757197619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef2006c1f0 0x7fef200c46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:55.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.313+0000 7fef15d9b700 1 -- 172.21.15.157:0/2757197619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef20150410 msgr2=0x7fef20155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:55.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.313+0000 7fef15d9b700 1 --2- 172.21.15.157:0/2757197619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef20150410 0x7fef20155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:55.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.313+0000 7fef15d9b700 1 -- 172.21.15.157:0/2757197619 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef10009420 con 0x7fef2006cbb0 2023-10-06T04:01:55.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.313+0000 7fef1e59c700 1 --2- 172.21.15.157:0/2757197619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef2006c1f0 0x7fef200c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:01:55.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.313+0000 7fef15d9b700 1 --2- 172.21.15.157:0/2757197619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef2006cbb0 0x7fef200c4c10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fef0800c590 tx=0x7fef08007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:55.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.313+0000 7fef16ffd700 1 -- 172.21.15.157:0/2757197619 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef08005b30 con 0x7fef2006cbb0 2023-10-06T04:01:55.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.313+0000 7fef16ffd700 1 -- 172.21.15.157:0/2757197619 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef08005cb0 con 0x7fef2006cbb0 2023-10-06T04:01:55.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.314+0000 7fef16ffd700 1 -- 172.21.15.157:0/2757197619 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef08007320 con 0x7fef2006cbb0 2023-10-06T04:01:55.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.314+0000 7fef24a24700 1 -- 172.21.15.157:0/2757197619 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef20153e50 con 0x7fef2006cbb0 2023-10-06T04:01:55.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.314+0000 7fef24a24700 1 -- 172.21.15.157:0/2757197619 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef2016a780 con 0x7fef2006cbb0 2023-10-06T04:01:55.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.314+0000 7fef24a24700 1 -- 172.21.15.157:0/2757197619 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef200c4120 con 0x7fef2006cbb0 2023-10-06T04:01:55.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.315+0000 7fef16ffd700 1 -- 172.21.15.157:0/2757197619 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fef0801b3b0 con 0x7fef2006cbb0 2023-10-06T04:01:55.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.315+0000 7fef16ffd700 1 --2- 172.21.15.157:0/2757197619 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fef00066f30 0x7fef000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:55.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.315+0000 7fef16ffd700 1 -- 172.21.15.157:0/2757197619 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7fef08083d60 con 0x7fef2006cbb0 2023-10-06T04:01:55.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.315+0000 7fef1e59c700 1 --2- 172.21.15.157:0/2757197619 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fef00066f30 0x7fef000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:55.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.316+0000 7fef1e59c700 1 --2- 172.21.15.157:0/2757197619 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fef00066f30 0x7fef000693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fef100091e0 tx=0x7fef10017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:55.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.319+0000 7fef16ffd700 1 -- 172.21.15.157:0/2757197619 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fef200c4120 con 0x7fef2006cbb0 2023-10-06T04:01:55.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.545+0000 7fef24a24700 1 -- 172.21.15.157:0/2757197619 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef200c4120 con 0x7fef2006cbb0 2023-10-06T04:01:55.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.546+0000 7fef16ffd700 1 -- 172.21.15.157:0/2757197619 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+6203 (secure 0 0 0) 0x7fef200c4120 con 0x7fef2006cbb0 2023-10-06T04:01:55.549 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:01:55.549 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:01:52.801202+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"f","rank":4,"incarnation":29,"state":"up:stopping","state_seq":32,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[0,3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:01:55.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.548+0000 7fef24a24700 1 -- 172.21.15.157:0/2757197619 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fef00066f30 msgr2=0x7fef000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:55.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.548+0000 7fef24a24700 1 --2- 172.21.15.157:0/2757197619 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fef00066f30 0x7fef000693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fef100091e0 tx=0x7fef10017000 comp rx=0 tx=0).stop 2023-10-06T04:01:55.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.548+0000 7fef24a24700 1 -- 172.21.15.157:0/2757197619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef2006cbb0 msgr2=0x7fef200c4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:55.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.548+0000 7fef24a24700 1 --2- 172.21.15.157:0/2757197619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef2006cbb0 0x7fef200c4c10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fef0800c590 tx=0x7fef08007e00 comp rx=0 tx=0).stop 2023-10-06T04:01:55.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.548+0000 7fef24a24700 1 -- 172.21.15.157:0/2757197619 shutdown_connections 2023-10-06T04:01:55.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.548+0000 7fef24a24700 1 --2- 172.21.15.157:0/2757197619 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fef00066f30 0x7fef000693e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:55.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.548+0000 7fef24a24700 1 --2- 172.21.15.157:0/2757197619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef20150410 0x7fef20155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:55.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.548+0000 7fef24a24700 1 --2- 172.21.15.157:0/2757197619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef2006c1f0 0x7fef200c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:55.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.548+0000 7fef24a24700 1 --2- 172.21.15.157:0/2757197619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef2006cbb0 0x7fef200c4c10 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:55.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.548+0000 7fef24a24700 1 -- 172.21.15.157:0/2757197619 >> 172.21.15.157:0/2757197619 conn(0x7fef2006f230 msgr2=0x7fef2014e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:55.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.548+0000 7fef24a24700 1 -- 172.21.15.157:0/2757197619 shutdown_connections 2023-10-06T04:01:55.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:55.548+0000 7fef24a24700 1 -- 172.21.15.157:0/2757197619 wait complete. 2023-10-06T04:01:55.552 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 38 2023-10-06T04:01:55.562 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:01:57.563 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:01:57.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.689+0000 7f96114e3700 1 Processor -- start 2023-10-06T04:01:57.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.689+0000 7f96114e3700 1 -- start start 2023-10-06T04:01:57.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.690+0000 7f96114e3700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f960c06cbb0 0x7f960c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:57.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.690+0000 7f96114e3700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f960c06c1f0 0x7f960c06da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:57.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.690+0000 7f96114e3700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f960c06cf80 0x7f960c14fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:57.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.690+0000 7f96114e3700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f960c0c8c40 con 0x7f960c06cf80 2023-10-06T04:01:57.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.690+0000 7f96114e3700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f960c0c9780 con 0x7f960c06cbb0 2023-10-06T04:01:57.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.690+0000 7f96114e3700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960c06e0c0 con 0x7f960c06c1f0 2023-10-06T04:01:57.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.690+0000 7f960b7fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f960c06cf80 0x7f960c14fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:57.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.690+0000 7f960b7fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f960c06cf80 0x7f960c14fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47246/0 (socket says 172.21.15.157:47246) 2023-10-06T04:01:57.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.690+0000 7f960b7fe700 1 -- 172.21.15.157:0/4279832685 learned_addr learned my addr 172.21.15.157:0/4279832685 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:01:57.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.690+0000 7f960affd700 1 --2- 172.21.15.157:0/4279832685 >> v2:172.21.15.175:3300/0 conn(0x7f960c06cbb0 0x7f960c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:57.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.690+0000 7f960a7fc700 1 --2- 172.21.15.157:0/4279832685 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f960c06c1f0 0x7f960c06da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:57.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.690+0000 7f960b7fe700 1 -- 172.21.15.157:0/4279832685 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f960c06c1f0 msgr2=0x7f960c06da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:01:57.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.690+0000 7f960b7fe700 1 --2- 172.21.15.157:0/4279832685 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f960c06c1f0 0x7f960c06da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:57.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.690+0000 7f960b7fe700 1 -- 172.21.15.157:0/4279832685 >> v2:172.21.15.175:3300/0 conn(0x7f960c06cbb0 msgr2=0x7f960c06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:01:57.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.690+0000 7f960b7fe700 1 --2- 172.21.15.157:0/4279832685 >> v2:172.21.15.175:3300/0 conn(0x7f960c06cbb0 0x7f960c06d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:57.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.690+0000 7f960b7fe700 1 -- 172.21.15.157:0/4279832685 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f960c0c8e50 con 0x7f960c06cf80 2023-10-06T04:01:57.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.690+0000 7f960b7fe700 1 --2- 172.21.15.157:0/4279832685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f960c06cf80 0x7f960c14fd00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9604009f20 tx=0x7f9604009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5338900c003e459 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:57.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.690+0000 7f9609ffb700 1 -- 172.21.15.157:0/4279832685 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960400e070 con 0x7f960c06cf80 2023-10-06T04:01:57.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.690+0000 7f9609ffb700 1 -- 172.21.15.157:0/4279832685 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960400bad0 con 0x7f960c06cf80 2023-10-06T04:01:57.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.690+0000 7f9609ffb700 1 -- 172.21.15.157:0/4279832685 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96040092e0 con 0x7f960c06cf80 2023-10-06T04:01:57.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.691+0000 7f96114e3700 1 -- 172.21.15.157:0/4279832685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f960c06cf80 msgr2=0x7f960c14fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:57.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.691+0000 7f96114e3700 1 --2- 172.21.15.157:0/4279832685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f960c06cf80 0x7f960c14fd00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9604009f20 tx=0x7f9604009840 comp rx=0 tx=0).stop 2023-10-06T04:01:57.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.691+0000 7f96114e3700 1 -- 172.21.15.157:0/4279832685 shutdown_connections 2023-10-06T04:01:57.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.691+0000 7f96114e3700 1 --2- 172.21.15.157:0/4279832685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f960c06cf80 0x7f960c14fd00 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:57.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.691+0000 7f96114e3700 1 --2- 172.21.15.157:0/4279832685 >> v2:172.21.15.175:3300/0 conn(0x7f960c06cbb0 0x7f960c06d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:57.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.691+0000 7f96114e3700 1 --2- 172.21.15.157:0/4279832685 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f960c06c1f0 0x7f960c06da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:57.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.691+0000 7f96114e3700 1 -- 172.21.15.157:0/4279832685 >> 172.21.15.157:0/4279832685 conn(0x7f960c06f230 msgr2=0x7f960c06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:57.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.691+0000 7f96114e3700 1 -- 172.21.15.157:0/4279832685 shutdown_connections 2023-10-06T04:01:57.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.691+0000 7f96114e3700 1 -- 172.21.15.157:0/4279832685 wait complete. 2023-10-06T04:01:57.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.692+0000 7f96114e3700 1 Processor -- start 2023-10-06T04:01:57.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.692+0000 7f96114e3700 1 -- start start 2023-10-06T04:01:57.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.693+0000 7f96114e3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f960c06c1f0 0x7f960c14d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:57.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.693+0000 7f96114e3700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f960c06cbb0 0x7f960c14d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:57.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.693+0000 7f96114e3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f960c06cf80 0x7f960c1699e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:57.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.693+0000 7f96114e3700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960c1472a0 con 0x7f960c06c1f0 2023-10-06T04:01:57.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.693+0000 7f96114e3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960c1515e0 con 0x7f960c06cf80 2023-10-06T04:01:57.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.693+0000 7f96114e3700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960c1518c0 con 0x7f960c06cbb0 2023-10-06T04:01:57.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.693+0000 7f960affd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f960c06c1f0 0x7f960c14d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:57.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.693+0000 7f960affd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f960c06c1f0 0x7f960c14d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47252/0 (socket says 172.21.15.157:47252) 2023-10-06T04:01:57.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.693+0000 7f960affd700 1 -- 172.21.15.157:0/3438168353 learned_addr learned my addr 172.21.15.157:0/3438168353 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:01:57.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.693+0000 7f960a7fc700 1 --2- 172.21.15.157:0/3438168353 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f960c06cbb0 0x7f960c14d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:57.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.693+0000 7f960b7fe700 1 --2- 172.21.15.157:0/3438168353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f960c06cf80 0x7f960c1699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:57.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.693+0000 7f960affd700 1 -- 172.21.15.157:0/3438168353 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f960c06cbb0 msgr2=0x7f960c14d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:57.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.693+0000 7f960affd700 1 --2- 172.21.15.157:0/3438168353 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f960c06cbb0 0x7f960c14d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:57.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.693+0000 7f960affd700 1 -- 172.21.15.157:0/3438168353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f960c06cf80 msgr2=0x7f960c1699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:57.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.693+0000 7f960affd700 1 --2- 172.21.15.157:0/3438168353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f960c06cf80 0x7f960c1699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:57.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.693+0000 7f960affd700 1 -- 172.21.15.157:0/3438168353 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9604009ae0 con 0x7f960c06c1f0 2023-10-06T04:01:57.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.694+0000 7f960affd700 1 --2- 172.21.15.157:0/3438168353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f960c06c1f0 0x7f960c14d4b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00d890 tx=0x7f95fc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:57.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.694+0000 7f95f2ffd700 1 -- 172.21.15.157:0/3438168353 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc0125f0 con 0x7f960c06c1f0 2023-10-06T04:01:57.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.694+0000 7f95f2ffd700 1 -- 172.21.15.157:0/3438168353 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95fc012750 con 0x7f960c06c1f0 2023-10-06T04:01:57.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.694+0000 7f96114e3700 1 -- 172.21.15.157:0/3438168353 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f960c147520 con 0x7f960c06c1f0 2023-10-06T04:01:57.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.694+0000 7f95f2ffd700 1 -- 172.21.15.157:0/3438168353 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc012af0 con 0x7f960c06c1f0 2023-10-06T04:01:57.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.694+0000 7f96114e3700 1 -- 172.21.15.157:0/3438168353 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f960c147ae0 con 0x7f960c06c1f0 2023-10-06T04:01:57.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.695+0000 7f96114e3700 1 -- 172.21.15.157:0/3438168353 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f960c147d00 con 0x7f960c06c1f0 2023-10-06T04:01:57.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.695+0000 7f95f2ffd700 1 -- 172.21.15.157:0/3438168353 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f95fc01e460 con 0x7f960c06c1f0 2023-10-06T04:01:57.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.695+0000 7f95f2ffd700 1 --2- 172.21.15.157:0/3438168353 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f95f8066f60 0x7f95f8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:01:57.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.696+0000 7f95f2ffd700 1 -- 172.21.15.157:0/3438168353 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f95fc084ca0 con 0x7f960c06c1f0 2023-10-06T04:01:57.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.696+0000 7f960a7fc700 1 --2- 172.21.15.157:0/3438168353 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f95f8066f60 0x7f95f8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:01:57.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.696+0000 7f960a7fc700 1 --2- 172.21.15.157:0/3438168353 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f95f8066f60 0x7f95f8069410 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f960c0cb650 tx=0x7f9600008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:01:57.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.700+0000 7f95f2ffd700 1 -- 172.21.15.157:0/3438168353 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f95fc051cd0 con 0x7f960c06c1f0 2023-10-06T04:01:57.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.929+0000 7f96114e3700 1 -- 172.21.15.157:0/3438168353 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f960c0c9130 con 0x7f960c06c1f0 2023-10-06T04:01:57.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.930+0000 7f95f2ffd700 1 -- 172.21.15.157:0/3438168353 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+6203 (secure 0 0 0) 0x7f95fc0552f0 con 0x7f960c06c1f0 2023-10-06T04:01:57.933 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:01:57.933 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:01:52.801202+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"f","rank":4,"incarnation":29,"state":"up:stopping","state_seq":32,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[0,3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:01:57.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.932+0000 7f96114e3700 1 -- 172.21.15.157:0/3438168353 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f95f8066f60 msgr2=0x7f95f8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:57.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.932+0000 7f96114e3700 1 --2- 172.21.15.157:0/3438168353 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f95f8066f60 0x7f95f8069410 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f960c0cb650 tx=0x7f9600008040 comp rx=0 tx=0).stop 2023-10-06T04:01:57.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.932+0000 7f96114e3700 1 -- 172.21.15.157:0/3438168353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f960c06c1f0 msgr2=0x7f960c14d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:01:57.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.932+0000 7f96114e3700 1 --2- 172.21.15.157:0/3438168353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f960c06c1f0 0x7f960c14d4b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00d890 tx=0x7f95fc008040 comp rx=0 tx=0).stop 2023-10-06T04:01:57.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.932+0000 7f96114e3700 1 -- 172.21.15.157:0/3438168353 shutdown_connections 2023-10-06T04:01:57.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.932+0000 7f96114e3700 1 --2- 172.21.15.157:0/3438168353 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f95f8066f60 0x7f95f8069410 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:57.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.932+0000 7f96114e3700 1 --2- 172.21.15.157:0/3438168353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f960c06cf80 0x7f960c1699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:57.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.932+0000 7f96114e3700 1 --2- 172.21.15.157:0/3438168353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f960c06c1f0 0x7f960c14d4b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:57.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.932+0000 7f96114e3700 1 --2- 172.21.15.157:0/3438168353 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f960c06cbb0 0x7f960c14d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:01:57.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.932+0000 7f96114e3700 1 -- 172.21.15.157:0/3438168353 >> 172.21.15.157:0/3438168353 conn(0x7f960c06f230 msgr2=0x7f960c14ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:01:57.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.932+0000 7f96114e3700 1 -- 172.21.15.157:0/3438168353 shutdown_connections 2023-10-06T04:01:57.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:01:57.932+0000 7f96114e3700 1 -- 172.21.15.157:0/3438168353 wait complete. 2023-10-06T04:01:57.936 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 38 2023-10-06T04:01:57.947 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:01:59.948 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:02:00.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.073+0000 7f9fee578700 1 Processor -- start 2023-10-06T04:02:00.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.073+0000 7f9fee578700 1 -- start start 2023-10-06T04:02:00.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.073+0000 7f9fee578700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9fe8068d10 0x7f9fe806a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:00.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.073+0000 7f9fee578700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9fe806d460 0x7f9fe806ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:00.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.073+0000 7f9fee578700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fe80bed40 con 0x7f9fe80be970 2023-10-06T04:02:00.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.073+0000 7f9fee578700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fe8157010 con 0x7f9fe8068d10 2023-10-06T04:02:00.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.073+0000 7f9fee578700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fe806f9f0 con 0x7f9fe806d460 2023-10-06T04:02:00.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.074+0000 7f9fe77fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9fe8068d10 0x7f9fe806a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:00.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.074+0000 7f9fe77fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9fe8068d10 0x7f9fe806a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47258/0 (socket says 172.21.15.157:47258) 2023-10-06T04:02:00.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.074+0000 7f9fe77fe700 1 -- 172.21.15.157:0/1739793363 learned_addr learned my addr 172.21.15.157:0/1739793363 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:00.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.074+0000 7f9fecb15700 1 --2- 172.21.15.157:0/1739793363 >> v2:172.21.15.175:3300/0 conn(0x7f9fe806d460 0x7f9fe806ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:00.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.074+0000 7f9fe77fe700 1 -- 172.21.15.157:0/1739793363 >> v2:172.21.15.175:3300/0 conn(0x7f9fe806d460 msgr2=0x7f9fe806ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:00.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.074+0000 7f9fe77fe700 1 --2- 172.21.15.157:0/1739793363 >> v2:172.21.15.175:3300/0 conn(0x7f9fe806d460 0x7f9fe806ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:00.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.074+0000 7f9fe77fe700 1 -- 172.21.15.157:0/1739793363 >> v1:172.21.15.175:6789/0 conn(0x7f9fe80be970 legacy=0x7f9fe806df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:00.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.074+0000 7f9fe77fe700 1 -- 172.21.15.157:0/1739793363 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe81574a0 con 0x7f9fe8068d10 2023-10-06T04:02:00.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.074+0000 7f9fecb15700 1 --2- 172.21.15.157:0/1739793363 >> v2:172.21.15.175:3300/0 conn(0x7f9fe806d460 0x7f9fe806ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:02:00.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.074+0000 7f9fe77fe700 1 --2- 172.21.15.157:0/1739793363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fe8068d10 0x7f9fe806a600 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9fe80692f0 tx=0x7f9fdc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9267643bba300f10 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:00.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.074+0000 7f9fe6ffd700 1 -- 172.21.15.157:0/1739793363 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc00b820 con 0x7f9fe8068d10 2023-10-06T04:02:00.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.074+0000 7f9fe6ffd700 1 -- 172.21.15.157:0/1739793363 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fdc00b980 con 0x7f9fe8068d10 2023-10-06T04:02:00.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.074+0000 7f9fe6ffd700 1 -- 172.21.15.157:0/1739793363 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc00bd20 con 0x7f9fe8068d10 2023-10-06T04:02:00.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.075+0000 7f9fee578700 1 -- 172.21.15.157:0/1739793363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fe8068d10 msgr2=0x7f9fe806a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:00.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.075+0000 7f9fee578700 1 --2- 172.21.15.157:0/1739793363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fe8068d10 0x7f9fe806a600 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9fe80692f0 tx=0x7f9fdc009250 comp rx=0 tx=0).stop 2023-10-06T04:02:00.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.075+0000 7f9fee578700 1 -- 172.21.15.157:0/1739793363 shutdown_connections 2023-10-06T04:02:00.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.075+0000 7f9fee578700 1 --2- 172.21.15.157:0/1739793363 >> v2:172.21.15.175:3300/0 conn(0x7f9fe806d460 0x7f9fe806ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:00.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.075+0000 7f9fee578700 1 --2- 172.21.15.157:0/1739793363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fe8068d10 0x7f9fe806a600 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:00.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.075+0000 7f9fee578700 1 -- 172.21.15.157:0/1739793363 >> 172.21.15.157:0/1739793363 conn(0x7f9fe80c8d70 msgr2=0x7f9fe80c4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:00.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.075+0000 7f9fee578700 1 -- 172.21.15.157:0/1739793363 shutdown_connections 2023-10-06T04:02:00.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.075+0000 7f9fee578700 1 -- 172.21.15.157:0/1739793363 wait complete. 2023-10-06T04:02:00.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.076+0000 7f9fee578700 1 Processor -- start 2023-10-06T04:02:00.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.076+0000 7f9fee578700 1 -- start start 2023-10-06T04:02:00.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.077+0000 7f9fee578700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fe8068d10 0x7f9fe8166470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:00.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.077+0000 7f9fee578700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fe806d460 0x7f9fe8144fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:00.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.077+0000 7f9fee578700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe80be970 0x7f9fe8153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:00.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.077+0000 7f9fee578700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe8167430 con 0x7f9fe806d460 2023-10-06T04:02:00.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.077+0000 7f9fee578700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe81669b0 con 0x7f9fe80be970 2023-10-06T04:02:00.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.077+0000 7f9fee578700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe806a3e0 con 0x7f9fe8068d10 2023-10-06T04:02:00.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.077+0000 7f9fe77fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fe806d460 0x7f9fe8144fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:00.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.077+0000 7f9fe77fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fe806d460 0x7f9fe8144fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47272/0 (socket says 172.21.15.157:47272) 2023-10-06T04:02:00.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.077+0000 7f9fe77fe700 1 -- 172.21.15.157:0/51369729 learned_addr learned my addr 172.21.15.157:0/51369729 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:00.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.077+0000 7f9fecb15700 1 --2- 172.21.15.157:0/51369729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe80be970 0x7f9fe8153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:00.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.077+0000 7f9fe7fff700 1 --2- 172.21.15.157:0/51369729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fe8068d10 0x7f9fe8166470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:00.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.077+0000 7f9fe77fe700 1 -- 172.21.15.157:0/51369729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fe8068d10 msgr2=0x7f9fe8166470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:00.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.077+0000 7f9fe77fe700 1 --2- 172.21.15.157:0/51369729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fe8068d10 0x7f9fe8166470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:00.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.077+0000 7f9fe77fe700 1 -- 172.21.15.157:0/51369729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe80be970 msgr2=0x7f9fe8153750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:00.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.077+0000 7f9fe77fe700 1 --2- 172.21.15.157:0/51369729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe80be970 0x7f9fe8153750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:00.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.077+0000 7f9fe77fe700 1 -- 172.21.15.157:0/51369729 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fdc0094f0 con 0x7f9fe806d460 2023-10-06T04:02:00.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.077+0000 7f9fe7fff700 1 --2- 172.21.15.157:0/51369729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fe8068d10 0x7f9fe8166470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:02:00.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.077+0000 7f9fe77fe700 1 --2- 172.21.15.157:0/51369729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fe806d460 0x7f9fe8144fe0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8068050 tx=0x7f9fdc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:00.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.077+0000 7f9fcffff700 1 -- 172.21.15.157:0/51369729 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc011e20 con 0x7f9fe806d460 2023-10-06T04:02:00.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.078+0000 7f9fcffff700 1 -- 172.21.15.157:0/51369729 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fdc020070 con 0x7f9fe806d460 2023-10-06T04:02:00.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.078+0000 7f9fee578700 1 -- 172.21.15.157:0/51369729 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe8152700 con 0x7f9fe806d460 2023-10-06T04:02:00.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.078+0000 7f9fcffff700 1 -- 172.21.15.157:0/51369729 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc01b6a0 con 0x7f9fe806d460 2023-10-06T04:02:00.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.078+0000 7f9fee578700 1 -- 172.21.15.157:0/51369729 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe8166e70 con 0x7f9fe806d460 2023-10-06T04:02:00.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.079+0000 7f9fee578700 1 -- 172.21.15.157:0/51369729 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe80716b0 con 0x7f9fe806d460 2023-10-06T04:02:00.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.079+0000 7f9fcffff700 1 -- 172.21.15.157:0/51369729 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9fdc022030 con 0x7f9fe806d460 2023-10-06T04:02:00.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.079+0000 7f9fcffff700 1 --2- 172.21.15.157:0/51369729 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9fd4066f30 0x7f9fd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:00.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.079+0000 7f9fcffff700 1 -- 172.21.15.157:0/51369729 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f9fdc0866f0 con 0x7f9fe806d460 2023-10-06T04:02:00.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.080+0000 7f9fe7fff700 1 --2- 172.21.15.157:0/51369729 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9fd4066f30 0x7f9fd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:00.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.080+0000 7f9fe7fff700 1 --2- 172.21.15.157:0/51369729 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9fd4066f30 0x7f9fd40693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8002880 tx=0x7f9fd8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:00.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.084+0000 7f9fcffff700 1 -- 172.21.15.157:0/51369729 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9fdc053720 con 0x7f9fe806d460 2023-10-06T04:02:00.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.310+0000 7f9fee578700 1 -- 172.21.15.157:0/51369729 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fe80716b0 con 0x7f9fe806d460 2023-10-06T04:02:00.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.311+0000 7f9fcffff700 1 -- 172.21.15.157:0/51369729 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+6203 (secure 0 0 0) 0x7f9fdc056d40 con 0x7f9fe806d460 2023-10-06T04:02:00.313 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:02:00.314 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:01:52.801202+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"f","rank":4,"incarnation":29,"state":"up:stopping","state_seq":32,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[0,3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:02:00.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.313+0000 7f9fee578700 1 -- 172.21.15.157:0/51369729 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9fd4066f30 msgr2=0x7f9fd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:00.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.313+0000 7f9fee578700 1 --2- 172.21.15.157:0/51369729 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9fd4066f30 0x7f9fd40693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8002880 tx=0x7f9fd8009040 comp rx=0 tx=0).stop 2023-10-06T04:02:00.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.313+0000 7f9fee578700 1 -- 172.21.15.157:0/51369729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fe806d460 msgr2=0x7f9fe8144fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:00.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.313+0000 7f9fee578700 1 --2- 172.21.15.157:0/51369729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fe806d460 0x7f9fe8144fe0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8068050 tx=0x7f9fdc016000 comp rx=0 tx=0).stop 2023-10-06T04:02:00.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.313+0000 7f9fee578700 1 -- 172.21.15.157:0/51369729 shutdown_connections 2023-10-06T04:02:00.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.313+0000 7f9fee578700 1 --2- 172.21.15.157:0/51369729 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9fd4066f30 0x7f9fd40693e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:00.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.313+0000 7f9fee578700 1 --2- 172.21.15.157:0/51369729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fe80be970 0x7f9fe8153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:00.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.313+0000 7f9fee578700 1 --2- 172.21.15.157:0/51369729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fe8068d10 0x7f9fe8166470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:00.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.313+0000 7f9fee578700 1 --2- 172.21.15.157:0/51369729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fe806d460 0x7f9fe8144fe0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:00.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.313+0000 7f9fee578700 1 -- 172.21.15.157:0/51369729 >> 172.21.15.157:0/51369729 conn(0x7f9fe80c8d70 msgr2=0x7f9fe814cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:00.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.313+0000 7f9fee578700 1 -- 172.21.15.157:0/51369729 shutdown_connections 2023-10-06T04:02:00.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:00.313+0000 7f9fee578700 1 -- 172.21.15.157:0/51369729 wait complete. 2023-10-06T04:02:00.317 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 38 2023-10-06T04:02:00.327 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:02:02.328 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:02:02.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.442+0000 7f79650f0700 1 Processor -- start 2023-10-06T04:02:02.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.442+0000 7f79650f0700 1 -- start start 2023-10-06T04:02:02.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.442+0000 7f79650f0700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79600b1d40 0x7f7960159df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:02.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.442+0000 7f79650f0700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f796006c1f0 0x7f796014fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:02.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.442+0000 7f79650f0700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f796006ced0 con 0x7f79600b2110 2023-10-06T04:02:02.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.442+0000 7f79650f0700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79600afb10 con 0x7f796006c1f0 2023-10-06T04:02:02.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.442+0000 7f79650f0700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796006cc10 con 0x7f79600b1d40 2023-10-06T04:02:02.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.443+0000 7f795f59e700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f796006c1f0 0x7f796014fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:02.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.443+0000 7f795f59e700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f796006c1f0 0x7f796014fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47284/0 (socket says 172.21.15.157:47284) 2023-10-06T04:02:02.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.443+0000 7f795f59e700 1 -- 172.21.15.157:0/2305452607 learned_addr learned my addr 172.21.15.157:0/2305452607 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:02.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.443+0000 7f795ed9d700 1 --2- 172.21.15.157:0/2305452607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79600b1d40 0x7f7960159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:02.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.443+0000 7f795f59e700 1 -- 172.21.15.157:0/2305452607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79600b1d40 msgr2=0x7f7960159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:02.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.443+0000 7f795f59e700 1 --2- 172.21.15.157:0/2305452607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79600b1d40 0x7f7960159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:02.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.443+0000 7f795f59e700 1 -- 172.21.15.157:0/2305452607 >> v1:172.21.15.175:6789/0 conn(0x7f79600b2110 legacy=0x7f79600af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:02.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.443+0000 7f795f59e700 1 -- 172.21.15.157:0/2305452607 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796006e570 con 0x7f796006c1f0 2023-10-06T04:02:02.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.443+0000 7f795f59e700 1 --2- 172.21.15.157:0/2305452607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f796006c1f0 0x7f796014fbe0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7958009f20 tx=0x7f7958009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd2d5d4865dbdfb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:02.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.443+0000 7f795dd9b700 1 -- 172.21.15.157:0/2305452607 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795800e070 con 0x7f796006c1f0 2023-10-06T04:02:02.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.443+0000 7f795dd9b700 1 -- 172.21.15.157:0/2305452607 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795800ba60 con 0x7f796006c1f0 2023-10-06T04:02:02.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.443+0000 7f795dd9b700 1 -- 172.21.15.157:0/2305452607 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79580092e0 con 0x7f796006c1f0 2023-10-06T04:02:02.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.444+0000 7f79650f0700 1 -- 172.21.15.157:0/2305452607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f796006c1f0 msgr2=0x7f796014fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:02.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.444+0000 7f79650f0700 1 --2- 172.21.15.157:0/2305452607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f796006c1f0 0x7f796014fbe0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7958009f20 tx=0x7f7958009840 comp rx=0 tx=0).stop 2023-10-06T04:02:02.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.444+0000 7f79650f0700 1 -- 172.21.15.157:0/2305452607 shutdown_connections 2023-10-06T04:02:02.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.444+0000 7f79650f0700 1 --2- 172.21.15.157:0/2305452607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f796006c1f0 0x7f796014fbe0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:02.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.444+0000 7f79650f0700 1 --2- 172.21.15.157:0/2305452607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79600b1d40 0x7f7960159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:02.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.444+0000 7f79650f0700 1 -- 172.21.15.157:0/2305452607 >> 172.21.15.157:0/2305452607 conn(0x7f79600cc6e0 msgr2=0x7f79600ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:02.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.444+0000 7f79650f0700 1 -- 172.21.15.157:0/2305452607 shutdown_connections 2023-10-06T04:02:02.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.444+0000 7f79650f0700 1 -- 172.21.15.157:0/2305452607 wait complete. 2023-10-06T04:02:02.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.445+0000 7f79650f0700 1 Processor -- start 2023-10-06T04:02:02.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.445+0000 7f79650f0700 1 -- start start 2023-10-06T04:02:02.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.446+0000 7f79650f0700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f796006c1f0 0x7f79600c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:02.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.446+0000 7f79650f0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79600b1d40 0x7f79600c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:02.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.446+0000 7f79650f0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79600b2110 0x7f7960150720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:02.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.446+0000 7f79650f0700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7960165e10 con 0x7f79600b2110 2023-10-06T04:02:02.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.446+0000 7f79650f0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79600c9ff0 con 0x7f79600b1d40 2023-10-06T04:02:02.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.446+0000 7f79650f0700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79600c93f0 con 0x7f796006c1f0 2023-10-06T04:02:02.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.446+0000 7f795f59e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79600b2110 0x7f7960150720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:02.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.446+0000 7f795f59e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79600b2110 0x7f7960150720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47292/0 (socket says 172.21.15.157:47292) 2023-10-06T04:02:02.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.446+0000 7f795f59e700 1 -- 172.21.15.157:0/2153737727 learned_addr learned my addr 172.21.15.157:0/2153737727 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:02.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.446+0000 7f795ed9d700 1 --2- 172.21.15.157:0/2153737727 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f796006c1f0 0x7f79600c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:02.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.446+0000 7f795e59c700 1 --2- 172.21.15.157:0/2153737727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79600b1d40 0x7f79600c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:02.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.446+0000 7f795f59e700 1 -- 172.21.15.157:0/2153737727 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f796006c1f0 msgr2=0x7f79600c9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:02.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.446+0000 7f795f59e700 1 --2- 172.21.15.157:0/2153737727 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f796006c1f0 0x7f79600c9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:02.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.446+0000 7f795f59e700 1 -- 172.21.15.157:0/2153737727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79600b1d40 msgr2=0x7f79600c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:02.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.446+0000 7f795f59e700 1 --2- 172.21.15.157:0/2153737727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79600b1d40 0x7f79600c5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:02.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.446+0000 7f795f59e700 1 -- 172.21.15.157:0/2153737727 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7958009ae0 con 0x7f79600b2110 2023-10-06T04:02:02.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.446+0000 7f795f59e700 1 --2- 172.21.15.157:0/2153737727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79600b2110 0x7f7960150720 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f795800b610 tx=0x7f7958018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:02.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.447+0000 7f7946ffd700 1 -- 172.21.15.157:0/2153737727 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79580032f0 con 0x7f79600b2110 2023-10-06T04:02:02.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.447+0000 7f7946ffd700 1 -- 172.21.15.157:0/2153737727 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7958008d20 con 0x7f79600b2110 2023-10-06T04:02:02.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.447+0000 7f7946ffd700 1 -- 172.21.15.157:0/2153737727 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7958026670 con 0x7f79600b2110 2023-10-06T04:02:02.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.447+0000 7f79650f0700 1 -- 172.21.15.157:0/2153737727 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7960166020 con 0x7f79600b2110 2023-10-06T04:02:02.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.447+0000 7f79650f0700 1 -- 172.21.15.157:0/2153737727 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79601665e0 con 0x7f79600b2110 2023-10-06T04:02:02.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.447+0000 7f79650f0700 1 -- 172.21.15.157:0/2153737727 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7960166c10 con 0x7f79600b2110 2023-10-06T04:02:02.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.448+0000 7f7946ffd700 1 -- 172.21.15.157:0/2153737727 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7958003040 con 0x7f79600b2110 2023-10-06T04:02:02.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.448+0000 7f7946ffd700 1 --2- 172.21.15.157:0/2153737727 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f794c066f60 0x7f794c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:02.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.448+0000 7f7946ffd700 1 -- 172.21.15.157:0/2153737727 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f7958086020 con 0x7f79600b2110 2023-10-06T04:02:02.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.448+0000 7f795ed9d700 1 --2- 172.21.15.157:0/2153737727 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f794c066f60 0x7f794c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:02.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.449+0000 7f795ed9d700 1 --2- 172.21.15.157:0/2153737727 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f794c066f60 0x7f794c069410 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7950009db0 tx=0x7f79500091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:02.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.453+0000 7f7946ffd700 1 -- 172.21.15.157:0/2153737727 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7958053100 con 0x7f79600b2110 2023-10-06T04:02:02.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.681+0000 7f79650f0700 1 -- 172.21.15.157:0/2153737727 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f796006d7b0 con 0x7f79600b2110 2023-10-06T04:02:02.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.682+0000 7f7946ffd700 1 -- 172.21.15.157:0/2153737727 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+6203 (secure 0 0 0) 0x7f7958056720 con 0x7f79600b2110 2023-10-06T04:02:02.684 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:02:02.685 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:01:52.801202+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"f","rank":4,"incarnation":29,"state":"up:stopping","state_seq":32,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[0,3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:02:02.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.684+0000 7f79650f0700 1 -- 172.21.15.157:0/2153737727 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f794c066f60 msgr2=0x7f794c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:02.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.684+0000 7f79650f0700 1 --2- 172.21.15.157:0/2153737727 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f794c066f60 0x7f794c069410 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7950009db0 tx=0x7f79500091a0 comp rx=0 tx=0).stop 2023-10-06T04:02:02.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.684+0000 7f79650f0700 1 -- 172.21.15.157:0/2153737727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79600b2110 msgr2=0x7f7960150720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:02.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.684+0000 7f79650f0700 1 --2- 172.21.15.157:0/2153737727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79600b2110 0x7f7960150720 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f795800b610 tx=0x7f7958018000 comp rx=0 tx=0).stop 2023-10-06T04:02:02.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.684+0000 7f79650f0700 1 -- 172.21.15.157:0/2153737727 shutdown_connections 2023-10-06T04:02:02.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.684+0000 7f79650f0700 1 --2- 172.21.15.157:0/2153737727 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f794c066f60 0x7f794c069410 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:02.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.684+0000 7f79650f0700 1 --2- 172.21.15.157:0/2153737727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79600b2110 0x7f7960150720 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:02.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.684+0000 7f79650f0700 1 --2- 172.21.15.157:0/2153737727 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f796006c1f0 0x7f79600c9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:02.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.684+0000 7f79650f0700 1 --2- 172.21.15.157:0/2153737727 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79600b1d40 0x7f79600c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:02.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.684+0000 7f79650f0700 1 -- 172.21.15.157:0/2153737727 >> 172.21.15.157:0/2153737727 conn(0x7f79600cc6e0 msgr2=0x7f796014e680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:02.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.684+0000 7f79650f0700 1 -- 172.21.15.157:0/2153737727 shutdown_connections 2023-10-06T04:02:02.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:02.684+0000 7f79650f0700 1 -- 172.21.15.157:0/2153737727 wait complete. 2023-10-06T04:02:02.688 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 38 2023-10-06T04:02:02.701 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:02:04.702 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:02:04.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.829+0000 7f18d9dc5700 1 Processor -- start 2023-10-06T04:02:04.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.829+0000 7f18d9dc5700 1 -- start start 2023-10-06T04:02:04.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.830+0000 7f18d9dc5700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f18d406a580 0x7f18d406c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:04.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.830+0000 7f18d9dc5700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f18d4157f60 0x7f18d414dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:04.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.830+0000 7f18d9dc5700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18d406b700 con 0x7f18d406a950 2023-10-06T04:02:04.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.830+0000 7f18d9dc5700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d406ad90 con 0x7f18d406a580 2023-10-06T04:02:04.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.830+0000 7f18d9dc5700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d41583e0 con 0x7f18d4157f60 2023-10-06T04:02:04.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.830+0000 7f18d3fff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f18d406a950 0x7f18d40c3cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40030/0 (socket says 172.21.15.157:40030) 2023-10-06T04:02:04.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.830+0000 7f18d8dc3700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f18d406a580 0x7f18d406c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:04.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.830+0000 7f18d3fff700 1 -- 172.21.15.157:0/448696392 learned_addr learned my addr 172.21.15.157:0/448696392 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:02:04.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.830+0000 7f18d95c4700 1 --2- 172.21.15.157:0/448696392 >> v2:172.21.15.175:3300/0 conn(0x7f18d4157f60 0x7f18d414dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:04.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.830+0000 7f18d8dc3700 1 -- 172.21.15.157:0/448696392 >> v2:172.21.15.175:3300/0 conn(0x7f18d4157f60 msgr2=0x7f18d414dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:04.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.830+0000 7f18d8dc3700 1 --2- 172.21.15.157:0/448696392 >> v2:172.21.15.175:3300/0 conn(0x7f18d4157f60 0x7f18d414dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:04.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.830+0000 7f18d8dc3700 1 -- 172.21.15.157:0/448696392 >> v1:172.21.15.157:6789/0 conn(0x7f18d406a950 legacy=0x7f18d40c3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:04.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.830+0000 7f18d8dc3700 1 -- 172.21.15.157:0/448696392 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d40c43d0 con 0x7f18d406a580 2023-10-06T04:02:04.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.830+0000 7f18d37fe700 1 -- 172.21.15.157:0/448696392 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4239383805 0 0) 0x7f18d406b700 con 0x7f18d406a950 2023-10-06T04:02:04.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.831+0000 7f18d8dc3700 1 --2- 172.21.15.157:0/448696392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d406a580 0x7f18d406c1f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f18d406c880 tx=0x7f18c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec3ab1dc5219af2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:04.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.831+0000 7f18d37fe700 1 -- 172.21.15.157:0/448696392 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c8011630 con 0x7f18d406a580 2023-10-06T04:02:04.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.831+0000 7f18d37fe700 1 -- 172.21.15.157:0/448696392 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c8011790 con 0x7f18d406a580 2023-10-06T04:02:04.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.831+0000 7f18d37fe700 1 -- 172.21.15.157:0/448696392 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c8011b30 con 0x7f18d406a580 2023-10-06T04:02:04.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.831+0000 7f18d9dc5700 1 -- 172.21.15.157:0/448696392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d406a580 msgr2=0x7f18d406c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:04.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.831+0000 7f18d9dc5700 1 --2- 172.21.15.157:0/448696392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d406a580 0x7f18d406c1f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f18d406c880 tx=0x7f18c80091a0 comp rx=0 tx=0).stop 2023-10-06T04:02:04.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.831+0000 7f18d9dc5700 1 -- 172.21.15.157:0/448696392 shutdown_connections 2023-10-06T04:02:04.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.831+0000 7f18d9dc5700 1 --2- 172.21.15.157:0/448696392 >> v2:172.21.15.175:3300/0 conn(0x7f18d4157f60 0x7f18d414dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:04.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.831+0000 7f18d9dc5700 1 --2- 172.21.15.157:0/448696392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d406a580 0x7f18d406c1f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:04.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.831+0000 7f18d9dc5700 1 -- 172.21.15.157:0/448696392 >> 172.21.15.157:0/448696392 conn(0x7f18d4068020 msgr2=0x7f18d4068420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:04.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.831+0000 7f18d9dc5700 1 -- 172.21.15.157:0/448696392 shutdown_connections 2023-10-06T04:02:04.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.831+0000 7f18d9dc5700 1 -- 172.21.15.157:0/448696392 wait complete. 2023-10-06T04:02:04.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.833+0000 7f18d9dc5700 1 Processor -- start 2023-10-06T04:02:04.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.833+0000 7f18d9dc5700 1 -- start start 2023-10-06T04:02:04.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.833+0000 7f18d9dc5700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18d406a580 0x7f18d414fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:04.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.834+0000 7f18d9dc5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d406a950 0x7f18d41500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:04.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.834+0000 7f18d9dc5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18d4157f60 0x7f18d4149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:04.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.834+0000 7f18d9dc5700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d4143a40 con 0x7f18d406a950 2023-10-06T04:02:04.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.834+0000 7f18d9dc5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d4148cd0 con 0x7f18d4157f60 2023-10-06T04:02:04.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.834+0000 7f18d9dc5700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d41486f0 con 0x7f18d406a580 2023-10-06T04:02:04.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.834+0000 7f18d3fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d406a950 0x7f18d41500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:04.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.834+0000 7f18d3fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d406a950 0x7f18d41500b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49356/0 (socket says 172.21.15.157:49356) 2023-10-06T04:02:04.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.834+0000 7f18d3fff700 1 -- 172.21.15.157:0/95479335 learned_addr learned my addr 172.21.15.157:0/95479335 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:04.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.834+0000 7f18d8dc3700 1 --2- 172.21.15.157:0/95479335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18d406a580 0x7f18d414fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:04.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.834+0000 7f18d95c4700 1 --2- 172.21.15.157:0/95479335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18d4157f60 0x7f18d4149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:04.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.834+0000 7f18d3fff700 1 -- 172.21.15.157:0/95479335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18d406a580 msgr2=0x7f18d414fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:04.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.834+0000 7f18d3fff700 1 --2- 172.21.15.157:0/95479335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18d406a580 0x7f18d414fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:04.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.834+0000 7f18d3fff700 1 -- 172.21.15.157:0/95479335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18d4157f60 msgr2=0x7f18d4149160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:04.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.834+0000 7f18d3fff700 1 --2- 172.21.15.157:0/95479335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18d4157f60 0x7f18d4149160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:04.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.834+0000 7f18d3fff700 1 -- 172.21.15.157:0/95479335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c8009420 con 0x7f18d406a950 2023-10-06T04:02:04.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.834+0000 7f18d3fff700 1 --2- 172.21.15.157:0/95479335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d406a950 0x7f18d41500b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f18c000c810 tx=0x7f18c0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:04.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.834+0000 7f18d0ff9700 1 -- 172.21.15.157:0/95479335 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c0005b30 con 0x7f18d406a950 2023-10-06T04:02:04.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.834+0000 7f18d0ff9700 1 -- 172.21.15.157:0/95479335 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c0005c90 con 0x7f18d406a950 2023-10-06T04:02:04.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.834+0000 7f18d0ff9700 1 -- 172.21.15.157:0/95479335 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c0007300 con 0x7f18d406a950 2023-10-06T04:02:04.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.834+0000 7f18d9dc5700 1 -- 172.21.15.157:0/95479335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d4151310 con 0x7f18d406a950 2023-10-06T04:02:04.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.834+0000 7f18d9dc5700 1 -- 172.21.15.157:0/95479335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d4148960 con 0x7f18d406a950 2023-10-06T04:02:04.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.835+0000 7f18d9dc5700 1 -- 172.21.15.157:0/95479335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d406b930 con 0x7f18d406a950 2023-10-06T04:02:04.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.836+0000 7f18d0ff9700 1 -- 172.21.15.157:0/95479335 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f18c001b3b0 con 0x7f18d406a950 2023-10-06T04:02:04.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.836+0000 7f18d0ff9700 1 --2- 172.21.15.157:0/95479335 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f18b8066f30 0x7f18b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:04.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.836+0000 7f18d0ff9700 1 -- 172.21.15.157:0/95479335 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f18c0083bd0 con 0x7f18d406a950 2023-10-06T04:02:04.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.836+0000 7f18d8dc3700 1 --2- 172.21.15.157:0/95479335 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f18b8066f30 0x7f18b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:04.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.837+0000 7f18d8dc3700 1 --2- 172.21.15.157:0/95479335 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f18b8066f30 0x7f18b80693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f18c80093f0 tx=0x7f18c8017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:04.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:04.841+0000 7f18d0ff9700 1 -- 172.21.15.157:0/95479335 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18c0050c00 con 0x7f18d406a950 2023-10-06T04:02:05.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:05.071+0000 7f18d9dc5700 1 -- 172.21.15.157:0/95479335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18d404bb40 con 0x7f18d406a950 2023-10-06T04:02:05.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:05.071+0000 7f18d0ff9700 1 -- 172.21.15.157:0/95479335 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+6207 (secure 0 0 0) 0x7f18c0017090 con 0x7f18d406a950 2023-10-06T04:02:05.073 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:02:05.073 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:02:02.817947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"f","rank":4,"incarnation":29,"state":"up:stopping","state_seq":32,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:02:05.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:05.073+0000 7f18d9dc5700 1 -- 172.21.15.157:0/95479335 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f18b8066f30 msgr2=0x7f18b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:05.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:05.073+0000 7f18d9dc5700 1 --2- 172.21.15.157:0/95479335 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f18b8066f30 0x7f18b80693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f18c80093f0 tx=0x7f18c8017000 comp rx=0 tx=0).stop 2023-10-06T04:02:05.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:05.074+0000 7f18d9dc5700 1 -- 172.21.15.157:0/95479335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d406a950 msgr2=0x7f18d41500b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:05.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:05.074+0000 7f18d9dc5700 1 --2- 172.21.15.157:0/95479335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d406a950 0x7f18d41500b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f18c000c810 tx=0x7f18c0007df0 comp rx=0 tx=0).stop 2023-10-06T04:02:05.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:05.074+0000 7f18d9dc5700 1 -- 172.21.15.157:0/95479335 shutdown_connections 2023-10-06T04:02:05.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:05.074+0000 7f18d9dc5700 1 --2- 172.21.15.157:0/95479335 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f18b8066f30 0x7f18b80693e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:05.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:05.074+0000 7f18d9dc5700 1 --2- 172.21.15.157:0/95479335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18d4157f60 0x7f18d4149160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:05.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:05.074+0000 7f18d9dc5700 1 --2- 172.21.15.157:0/95479335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18d406a580 0x7f18d414fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:05.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:05.074+0000 7f18d9dc5700 1 --2- 172.21.15.157:0/95479335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18d406a950 0x7f18d41500b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:05.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:05.074+0000 7f18d9dc5700 1 -- 172.21.15.157:0/95479335 >> 172.21.15.157:0/95479335 conn(0x7f18d4068020 msgr2=0x7f18d41528f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:05.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:05.074+0000 7f18d9dc5700 1 -- 172.21.15.157:0/95479335 shutdown_connections 2023-10-06T04:02:05.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:05.074+0000 7f18d9dc5700 1 -- 172.21.15.157:0/95479335 wait complete. 2023-10-06T04:02:05.077 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 39 2023-10-06T04:02:05.087 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2023-10-06T04:02:05.088 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 39, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-06T03:57:10.718782+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 39, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0, 1, 2, 3, 4 ], "info": { "gid_4270": { "addr": "172.21.15.175:6835/1596042861", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6834", "nonce": 1596042861, "type": "v2" }, { "addr": "172.21.15.175:6835", "nonce": 1596042861, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4270, "incarnation": 13, "join_fscid": -1, "name": "b", "rank": 3, "state": "up:active", "state_seq": 8 }, "gid_4279": { "addr": "172.21.15.175:6837/1002404870", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6836", "nonce": 1002404870, "type": "v2" }, { "addr": "172.21.15.175:6837", "nonce": 1002404870, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1, 3, 4 ], "features": 4540138320759226367, "flags": 0, "gid": 4279, "incarnation": 11, "join_fscid": -1, "name": "d", "rank": 2, "state": "up:active", "state_seq": 7 }, "gid_4287": { "addr": "172.21.15.157:6834/1179753185", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6832", "nonce": 1179753185, "type": "v2" }, { "addr": "172.21.15.157:6834", "nonce": 1179753185, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4287, "incarnation": 9, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4296": { "addr": "172.21.15.157:6837/4188652695", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6836", "nonce": 4188652695, "type": "v2" }, { "addr": "172.21.15.157:6837", "nonce": 4188652695, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2, 4 ], "features": 4540138320759226367, "flags": 0, "gid": 4296, "incarnation": 4, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 4 }, "gid_4936": { "addr": "172.21.15.175:6839/3353503675", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6838", "nonce": 3353503675, "type": "v2" }, { "addr": "172.21.15.175:6839", "nonce": 3353503675, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1, 2, 3 ], "features": 4540138320759226367, "flags": 0, "gid": 4936, "incarnation": 29, "join_fscid": -1, "name": "f", "rank": 4, "state": "up:stopping", "state_seq": 32 } }, "last_failure": 0, "last_failure_osd_epoch": 27, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2023-10-06T04:02:02.817947+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4296, "mds_1": 4287, "mds_2": 4279, "mds_3": 4270, "mds_4": 4936 } } } ], "standbys": [ { "addr": "172.21.15.157:6835/2461751318", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6833", "nonce": 2461751318, "type": "v2" }, { "addr": "172.21.15.157:6835", "nonce": 2461751318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 34, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5268, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-06T04:02:07.089 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:02:07.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.214+0000 7f8879728700 1 Processor -- start 2023-10-06T04:02:07.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.214+0000 7f8879728700 1 -- start start 2023-10-06T04:02:07.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.215+0000 7f8879728700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f88740b1e40 0x7f8874157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:07.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.215+0000 7f8879728700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88740b04d0 0x7f887406dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:07.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.215+0000 7f8879728700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f887406d8d0 con 0x7f887406c2d0 2023-10-06T04:02:07.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.215+0000 7f8879728700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88740af8f0 con 0x7f88740b1e40 2023-10-06T04:02:07.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.215+0000 7f8879728700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887415a690 con 0x7f88740b04d0 2023-10-06T04:02:07.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.215+0000 7f8873fff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f88740b1e40 0x7f8874157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:07.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.215+0000 7f88737fe700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88740b04d0 0x7f887406dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:07.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.215+0000 7f8878f27700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f887406c2d0 0x7f88741563b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.157:54616/0 (socket says 172.21.15.157:54616) 2023-10-06T04:02:07.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.215+0000 7f8878f27700 1 -- 172.21.15.157:0/1204845141 learned_addr learned my addr 172.21.15.157:0/1204845141 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:02:07.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.216+0000 7f8872ffd700 1 -- 172.21.15.157:0/1204845141 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1415690598 0 0) 0x7f887406d8d0 con 0x7f887406c2d0 2023-10-06T04:02:07.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.216+0000 7f8872ffd700 1 -- 172.21.15.157:0/1204845141 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f885c003570 con 0x7f887406c2d0 2023-10-06T04:02:07.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.216+0000 7f88737fe700 1 -- 172.21.15.157:0/1204845141 >> v2:172.21.15.175:3300/0 conn(0x7f88740b1e40 msgr2=0x7f8874157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:07.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.216+0000 7f88737fe700 1 --2- 172.21.15.157:0/1204845141 >> v2:172.21.15.175:3300/0 conn(0x7f88740b1e40 0x7f8874157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:07.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.216+0000 7f88737fe700 1 -- 172.21.15.157:0/1204845141 >> v1:172.21.15.175:6789/0 conn(0x7f887406c2d0 legacy=0x7f88741563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:07.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.216+0000 7f88737fe700 1 -- 172.21.15.157:0/1204845141 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887406ccc0 con 0x7f88740b04d0 2023-10-06T04:02:07.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.216+0000 7f8873fff700 1 --2- 172.21.15.157:0/1204845141 >> v2:172.21.15.175:3300/0 conn(0x7f88740b1e40 0x7f8874157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:02:07.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.216+0000 7f88737fe700 1 --2- 172.21.15.157:0/1204845141 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88740b04d0 0x7f887406dd60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f88680098e0 tx=0x7f88680091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4af2f413fd221d42 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:07.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.216+0000 7f8872ffd700 1 -- 172.21.15.157:0/1204845141 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8868011630 con 0x7f88740b04d0 2023-10-06T04:02:07.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.216+0000 7f8872ffd700 1 -- 172.21.15.157:0/1204845141 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8868011790 con 0x7f88740b04d0 2023-10-06T04:02:07.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.216+0000 7f8872ffd700 1 -- 172.21.15.157:0/1204845141 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8868011b10 con 0x7f88740b04d0 2023-10-06T04:02:07.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.217+0000 7f8879728700 1 -- 172.21.15.157:0/1204845141 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88740b04d0 msgr2=0x7f887406dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:07.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.217+0000 7f8879728700 1 --2- 172.21.15.157:0/1204845141 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88740b04d0 0x7f887406dd60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f88680098e0 tx=0x7f88680091a0 comp rx=0 tx=0).stop 2023-10-06T04:02:07.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.217+0000 7f8879728700 1 -- 172.21.15.157:0/1204845141 shutdown_connections 2023-10-06T04:02:07.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.217+0000 7f8879728700 1 --2- 172.21.15.157:0/1204845141 >> v2:172.21.15.175:3300/0 conn(0x7f88740b1e40 0x7f8874157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:07.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.217+0000 7f8879728700 1 --2- 172.21.15.157:0/1204845141 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88740b04d0 0x7f887406dd60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:07.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.217+0000 7f8879728700 1 -- 172.21.15.157:0/1204845141 >> 172.21.15.157:0/1204845141 conn(0x7f88740cc540 msgr2=0x7f88740cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:07.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.217+0000 7f8879728700 1 -- 172.21.15.157:0/1204845141 shutdown_connections 2023-10-06T04:02:07.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.217+0000 7f8879728700 1 -- 172.21.15.157:0/1204845141 wait complete. 2023-10-06T04:02:07.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.218+0000 7f8879728700 1 Processor -- start 2023-10-06T04:02:07.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.218+0000 7f8879728700 1 -- start start 2023-10-06T04:02:07.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.219+0000 7f8879728700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f887406c2d0 0x7f8874168640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:07.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.219+0000 7f8879728700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88740b04d0 0x7f88740c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:07.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.219+0000 7f8879728700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88740b1e40 0x7f88740c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:07.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.219+0000 7f8879728700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887415a370 con 0x7f887406c2d0 2023-10-06T04:02:07.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.219+0000 7f8879728700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8874168b80 con 0x7f88740b1e40 2023-10-06T04:02:07.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.219+0000 7f8879728700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8874159e40 con 0x7f88740b04d0 2023-10-06T04:02:07.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.219+0000 7f8873fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f887406c2d0 0x7f8874168640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:07.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.219+0000 7f8873fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f887406c2d0 0x7f8874168640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49372/0 (socket says 172.21.15.157:49372) 2023-10-06T04:02:07.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.219+0000 7f8873fff700 1 -- 172.21.15.157:0/820232947 learned_addr learned my addr 172.21.15.157:0/820232947 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:07.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.219+0000 7f88737fe700 1 --2- 172.21.15.157:0/820232947 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88740b04d0 0x7f88740c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:07.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.219+0000 7f8878f27700 1 --2- 172.21.15.157:0/820232947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88740b1e40 0x7f88740c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:07.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.219+0000 7f8873fff700 1 -- 172.21.15.157:0/820232947 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88740b04d0 msgr2=0x7f88740c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:07.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.219+0000 7f8873fff700 1 --2- 172.21.15.157:0/820232947 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88740b04d0 0x7f88740c91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:07.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.219+0000 7f8873fff700 1 -- 172.21.15.157:0/820232947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88740b1e40 msgr2=0x7f88740c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:07.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.219+0000 7f8873fff700 1 --2- 172.21.15.157:0/820232947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88740b1e40 0x7f88740c96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:07.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.219+0000 7f8873fff700 1 -- 172.21.15.157:0/820232947 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8864009420 con 0x7f887406c2d0 2023-10-06T04:02:07.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.219+0000 7f8878f27700 1 --2- 172.21.15.157:0/820232947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88740b1e40 0x7f88740c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:02:07.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.219+0000 7f8873fff700 1 --2- 172.21.15.157:0/820232947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f887406c2d0 0x7f8874168640 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f886400ecb0 tx=0x7f88640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:07.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.219+0000 7f885bfff700 1 -- 172.21.15.157:0/820232947 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886400ee60 con 0x7f887406c2d0 2023-10-06T04:02:07.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.220+0000 7f885bfff700 1 -- 172.21.15.157:0/820232947 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886400cc10 con 0x7f887406c2d0 2023-10-06T04:02:07.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.220+0000 7f8879728700 1 -- 172.21.15.157:0/820232947 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8868009420 con 0x7f887406c2d0 2023-10-06T04:02:07.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.220+0000 7f885bfff700 1 -- 172.21.15.157:0/820232947 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8864013470 con 0x7f887406c2d0 2023-10-06T04:02:07.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.220+0000 7f8879728700 1 -- 172.21.15.157:0/820232947 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88741681b0 con 0x7f887406c2d0 2023-10-06T04:02:07.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.221+0000 7f8879728700 1 -- 172.21.15.157:0/820232947 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8874088f90 con 0x7f887406c2d0 2023-10-06T04:02:07.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.221+0000 7f885bfff700 1 -- 172.21.15.157:0/820232947 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8864013820 con 0x7f887406c2d0 2023-10-06T04:02:07.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.221+0000 7f885bfff700 1 --2- 172.21.15.157:0/820232947 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8860066f30 0x7f88600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:07.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.221+0000 7f885bfff700 1 -- 172.21.15.157:0/820232947 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f8864084b50 con 0x7f887406c2d0 2023-10-06T04:02:07.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.222+0000 7f88737fe700 1 --2- 172.21.15.157:0/820232947 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8860066f30 0x7f88600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:07.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.222+0000 7f88737fe700 1 --2- 172.21.15.157:0/820232947 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8860066f30 0x7f88600693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f887406fb70 tx=0x7f8868017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:07.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.226+0000 7f885bfff700 1 -- 172.21.15.157:0/820232947 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8864051b80 con 0x7f887406c2d0 2023-10-06T04:02:07.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.453+0000 7f8879728700 1 -- 172.21.15.157:0/820232947 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f887406cfa0 con 0x7f887406c2d0 2023-10-06T04:02:07.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.454+0000 7f885bfff700 1 -- 172.21.15.157:0/820232947 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+6207 (secure 0 0 0) 0x7f88640551a0 con 0x7f887406c2d0 2023-10-06T04:02:07.456 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:02:07.456 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:02:02.817947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"f","rank":4,"incarnation":29,"state":"up:stopping","state_seq":32,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:02:07.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.456+0000 7f8879728700 1 -- 172.21.15.157:0/820232947 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8860066f30 msgr2=0x7f88600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:07.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.456+0000 7f8879728700 1 --2- 172.21.15.157:0/820232947 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8860066f30 0x7f88600693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f887406fb70 tx=0x7f8868017000 comp rx=0 tx=0).stop 2023-10-06T04:02:07.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.456+0000 7f8879728700 1 -- 172.21.15.157:0/820232947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f887406c2d0 msgr2=0x7f8874168640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:07.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.456+0000 7f8879728700 1 --2- 172.21.15.157:0/820232947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f887406c2d0 0x7f8874168640 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f886400ecb0 tx=0x7f88640091a0 comp rx=0 tx=0).stop 2023-10-06T04:02:07.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.456+0000 7f8879728700 1 -- 172.21.15.157:0/820232947 shutdown_connections 2023-10-06T04:02:07.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.456+0000 7f8879728700 1 --2- 172.21.15.157:0/820232947 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8860066f30 0x7f88600693e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:07.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.456+0000 7f8879728700 1 --2- 172.21.15.157:0/820232947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88740b1e40 0x7f88740c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:07.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.456+0000 7f8879728700 1 --2- 172.21.15.157:0/820232947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f887406c2d0 0x7f8874168640 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:07.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.456+0000 7f8879728700 1 --2- 172.21.15.157:0/820232947 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88740b04d0 0x7f88740c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:07.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.456+0000 7f8879728700 1 -- 172.21.15.157:0/820232947 >> 172.21.15.157:0/820232947 conn(0x7f88740cc540 msgr2=0x7f8874157440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:07.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.457+0000 7f8879728700 1 -- 172.21.15.157:0/820232947 shutdown_connections 2023-10-06T04:02:07.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:07.457+0000 7f8879728700 1 -- 172.21.15.157:0/820232947 wait complete. 2023-10-06T04:02:07.460 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 39 2023-10-06T04:02:07.470 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:02:09.471 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:02:09.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.613+0000 7fd8b5bbc700 1 Processor -- start 2023-10-06T04:02:09.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.614+0000 7fd8b5bbc700 1 -- start start 2023-10-06T04:02:09.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.614+0000 7fd8b5bbc700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8b00ca630 0x7fd8b00c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:09.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.614+0000 7fd8b5bbc700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd8b0068920 0x7fd8b0156840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:09.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.614+0000 7fd8b5bbc700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b006d530 con 0x7fd8b00c52e0 2023-10-06T04:02:09.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.614+0000 7fd8b5bbc700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b006d9c0 con 0x7fd8b0068920 2023-10-06T04:02:09.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.614+0000 7fd8b5bbc700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b006f0c0 con 0x7fd8b00ca630 2023-10-06T04:02:09.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.614+0000 7fd8affff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd8b00c52e0 0x7fd8b0159580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40042/0 (socket says 172.21.15.157:40042) 2023-10-06T04:02:09.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.614+0000 7fd8affff700 1 -- 172.21.15.157:0/2970255352 learned_addr learned my addr 172.21.15.157:0/2970255352 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:02:09.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.615+0000 7fd8aeffd700 1 --2- 172.21.15.157:0/2970255352 >> v2:172.21.15.175:3300/0 conn(0x7fd8b0068920 0x7fd8b0156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:09.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.615+0000 7fd8af7fe700 1 --2- 172.21.15.157:0/2970255352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8b00ca630 0x7fd8b00c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:09.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.615+0000 7fd8ae7fc700 1 -- 172.21.15.157:0/2970255352 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1118000009 0 0) 0x7fd8b006d530 con 0x7fd8b00c52e0 2023-10-06T04:02:09.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.615+0000 7fd8ae7fc700 1 -- 172.21.15.157:0/2970255352 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd898003570 con 0x7fd8b00c52e0 2023-10-06T04:02:09.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.615+0000 7fd8ae7fc700 1 -- 172.21.15.157:0/2970255352 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4031923825 0 0) 0x7fd898003570 con 0x7fd8b00c52e0 2023-10-06T04:02:09.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.615+0000 7fd8ae7fc700 1 -- 172.21.15.157:0/2970255352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8b00ca630 msgr2=0x7fd8b00c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:09.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.615+0000 7fd8ae7fc700 1 --2- 172.21.15.157:0/2970255352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8b00ca630 0x7fd8b00c6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:09.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.615+0000 7fd8ae7fc700 1 -- 172.21.15.157:0/2970255352 >> v2:172.21.15.175:3300/0 conn(0x7fd8b0068920 msgr2=0x7fd8b0156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:09.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.615+0000 7fd8ae7fc700 1 --2- 172.21.15.157:0/2970255352 >> v2:172.21.15.175:3300/0 conn(0x7fd8b0068920 0x7fd8b0156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:09.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.615+0000 7fd8ae7fc700 1 -- 172.21.15.157:0/2970255352 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b006f3a0 con 0x7fd8b00c52e0 2023-10-06T04:02:09.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.615+0000 7fd8ae7fc700 1 -- 172.21.15.157:0/2970255352 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7fd8a8001470 con 0x7fd8b00c52e0 2023-10-06T04:02:09.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.615+0000 7fd8ae7fc700 1 -- 172.21.15.157:0/2970255352 >> v1:172.21.15.157:6789/0 conn(0x7fd8b00c52e0 legacy=0x7fd8b0159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:09.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.615+0000 7fd8ae7fc700 1 --2- 172.21.15.157:0/2970255352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd898003a60 0x7fd898003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:09.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.615+0000 7fd8ae7fc700 1 --2- 172.21.15.157:0/2970255352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd898004810 0x7fd898004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:09.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.615+0000 7fd8ae7fc700 1 --2- 172.21.15.157:0/2970255352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8980051c0 0x7fd89800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:09.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.615+0000 7fd8ae7fc700 1 -- 172.21.15.157:0/2970255352 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b006d9c0 con 0x7fd8980051c0 2023-10-06T04:02:09.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.615+0000 7fd8ae7fc700 1 -- 172.21.15.157:0/2970255352 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b006f0c0 con 0x7fd898004810 2023-10-06T04:02:09.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.615+0000 7fd8ae7fc700 1 -- 172.21.15.157:0/2970255352 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd89800bfa0 con 0x7fd898003a60 2023-10-06T04:02:09.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.615+0000 7fd8af7fe700 1 --2- 172.21.15.157:0/2970255352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8b00ca630 0x7fd8b00c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:02:09.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.615+0000 7fd8aeffd700 1 --2- 172.21.15.157:0/2970255352 >> v2:172.21.15.175:3300/0 conn(0x7fd8b0068920 0x7fd8b0156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:02:09.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.615+0000 7fd8affff700 1 --2- 172.21.15.157:0/2970255352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8980051c0 0x7fd89800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:09.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.615+0000 7fd8af7fe700 1 --2- 172.21.15.157:0/2970255352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd898003a60 0x7fd898003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:09.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.615+0000 7fd8aeffd700 1 --2- 172.21.15.157:0/2970255352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd898004810 0x7fd898004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:09.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.615+0000 7fd8affff700 1 -- 172.21.15.157:0/2970255352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd898003a60 msgr2=0x7fd898003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:09.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.615+0000 7fd8affff700 1 --2- 172.21.15.157:0/2970255352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd898003a60 0x7fd898003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:09.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.615+0000 7fd8affff700 1 -- 172.21.15.157:0/2970255352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd898004810 msgr2=0x7fd898004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:09.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.615+0000 7fd8affff700 1 --2- 172.21.15.157:0/2970255352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd898004810 0x7fd898004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:09.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.615+0000 7fd8affff700 1 -- 172.21.15.157:0/2970255352 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd898003570 con 0x7fd8980051c0 2023-10-06T04:02:09.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.616+0000 7fd8affff700 1 --2- 172.21.15.157:0/2970255352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8980051c0 0x7fd89800ba60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800c3f0 tx=0x7fd8a8001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26681a775fa33617 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:09.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.616+0000 7fd8ae7fc700 1 -- 172.21.15.157:0/2970255352 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a800cc20 con 0x7fd8980051c0 2023-10-06T04:02:09.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.616+0000 7fd8ae7fc700 1 -- 172.21.15.157:0/2970255352 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a800cdc0 con 0x7fd8980051c0 2023-10-06T04:02:09.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.616+0000 7fd8ae7fc700 1 -- 172.21.15.157:0/2970255352 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8005b70 con 0x7fd8980051c0 2023-10-06T04:02:09.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.616+0000 7fd8b5bbc700 1 -- 172.21.15.157:0/2970255352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8980051c0 msgr2=0x7fd89800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:09.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.616+0000 7fd8b5bbc700 1 --2- 172.21.15.157:0/2970255352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8980051c0 0x7fd89800ba60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800c3f0 tx=0x7fd8a8001e90 comp rx=0 tx=0).stop 2023-10-06T04:02:09.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.616+0000 7fd8affff700 1 -- 172.21.15.157:0/2970255352 reap_dead start 2023-10-06T04:02:09.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.616+0000 7fd8b5bbc700 1 -- 172.21.15.157:0/2970255352 shutdown_connections 2023-10-06T04:02:09.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.616+0000 7fd8b5bbc700 1 -- 172.21.15.157:0/2970255352 >> 172.21.15.157:0/2970255352 conn(0x7fd8b00692b0 msgr2=0x7fd8b00696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:09.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.616+0000 7fd8b5bbc700 1 -- 172.21.15.157:0/2970255352 shutdown_connections 2023-10-06T04:02:09.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.616+0000 7fd8b5bbc700 1 -- 172.21.15.157:0/2970255352 wait complete. 2023-10-06T04:02:09.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.618+0000 7fd8b5bbc700 1 Processor -- start 2023-10-06T04:02:09.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.618+0000 7fd8b5bbc700 1 -- start start 2023-10-06T04:02:09.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.618+0000 7fd8b5bbc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8b00c3990 0x7fd8b0157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:09.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.618+0000 7fd8b5bbc700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8b00c3d60 0x7fd8b0166580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:09.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.618+0000 7fd8b5bbc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8b0152660 0x7fd8b015bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:09.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.618+0000 7fd8b5bbc700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b0158710 con 0x7fd8b0152660 2023-10-06T04:02:09.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.618+0000 7fd8b5bbc700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b01524d0 con 0x7fd8b00c3990 2023-10-06T04:02:09.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.618+0000 7fd8b5bbc700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b0165b70 con 0x7fd8b00c3d60 2023-10-06T04:02:09.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.618+0000 7fd8affff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8b0152660 0x7fd8b015bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:09.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.619+0000 7fd8affff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8b0152660 0x7fd8b015bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49394/0 (socket says 172.21.15.157:49394) 2023-10-06T04:02:09.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.619+0000 7fd8affff700 1 -- 172.21.15.157:0/1489126611 learned_addr learned my addr 172.21.15.157:0/1489126611 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:09.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.619+0000 7fd8aeffd700 1 --2- 172.21.15.157:0/1489126611 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8b00c3d60 0x7fd8b0166580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:09.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.619+0000 7fd8af7fe700 1 --2- 172.21.15.157:0/1489126611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8b00c3990 0x7fd8b0157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:09.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.619+0000 7fd8affff700 1 -- 172.21.15.157:0/1489126611 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8b00c3d60 msgr2=0x7fd8b0166580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:09.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.619+0000 7fd8affff700 1 --2- 172.21.15.157:0/1489126611 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8b00c3d60 0x7fd8b0166580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:09.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.619+0000 7fd8affff700 1 -- 172.21.15.157:0/1489126611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8b00c3990 msgr2=0x7fd8b0157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:09.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.619+0000 7fd8affff700 1 --2- 172.21.15.157:0/1489126611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8b00c3990 0x7fd8b0157100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:09.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.619+0000 7fd8affff700 1 -- 172.21.15.157:0/1489126611 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b0166ac0 con 0x7fd8b0152660 2023-10-06T04:02:09.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.619+0000 7fd8affff700 1 --2- 172.21.15.157:0/1489126611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8b0152660 0x7fd8b015bbd0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8011000 tx=0x7fd8a8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:09.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.619+0000 7fd8977fe700 1 -- 172.21.15.157:0/1489126611 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a800cc20 con 0x7fd8b0152660 2023-10-06T04:02:09.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.619+0000 7fd8977fe700 1 -- 172.21.15.157:0/1489126611 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a8001430 con 0x7fd8b0152660 2023-10-06T04:02:09.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.619+0000 7fd8b5bbc700 1 -- 172.21.15.157:0/1489126611 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b0156610 con 0x7fd8b0152660 2023-10-06T04:02:09.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.619+0000 7fd8977fe700 1 -- 172.21.15.157:0/1489126611 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8007550 con 0x7fd8b0152660 2023-10-06T04:02:09.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.619+0000 7fd8b5bbc700 1 -- 172.21.15.157:0/1489126611 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b01660f0 con 0x7fd8b0152660 2023-10-06T04:02:09.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.620+0000 7fd8b5bbc700 1 -- 172.21.15.157:0/1489126611 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b0166280 con 0x7fd8b0152660 2023-10-06T04:02:09.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.620+0000 7fd8977fe700 1 -- 172.21.15.157:0/1489126611 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd8a8027460 con 0x7fd8b0152660 2023-10-06T04:02:09.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.621+0000 7fd8977fe700 1 --2- 172.21.15.157:0/1489126611 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd89c066f30 0x7fd89c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:09.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.621+0000 7fd8977fe700 1 -- 172.21.15.157:0/1489126611 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7fd8a80857d0 con 0x7fd8b0152660 2023-10-06T04:02:09.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.621+0000 7fd8af7fe700 1 --2- 172.21.15.157:0/1489126611 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd89c066f30 0x7fd89c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:09.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.622+0000 7fd8af7fe700 1 --2- 172.21.15.157:0/1489126611 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd89c066f30 0x7fd89c0693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0068360 tx=0x7fd8a000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:09.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.625+0000 7fd8977fe700 1 -- 172.21.15.157:0/1489126611 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd8a8052800 con 0x7fd8b0152660 2023-10-06T04:02:09.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.852+0000 7fd8b5bbc700 1 -- 172.21.15.157:0/1489126611 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8b0166ac0 con 0x7fd8b0152660 2023-10-06T04:02:09.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.852+0000 7fd8977fe700 1 -- 172.21.15.157:0/1489126611 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+6207 (secure 0 0 0) 0x7fd8b0166ac0 con 0x7fd8b0152660 2023-10-06T04:02:09.854 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:02:09.854 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:02:02.817947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"f","rank":4,"incarnation":29,"state":"up:stopping","state_seq":32,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:02:09.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.854+0000 7fd8b5bbc700 1 -- 172.21.15.157:0/1489126611 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd89c066f30 msgr2=0x7fd89c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:09.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.854+0000 7fd8b5bbc700 1 --2- 172.21.15.157:0/1489126611 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd89c066f30 0x7fd89c0693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0068360 tx=0x7fd8a000b040 comp rx=0 tx=0).stop 2023-10-06T04:02:09.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.854+0000 7fd8b5bbc700 1 -- 172.21.15.157:0/1489126611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8b0152660 msgr2=0x7fd8b015bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:09.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.854+0000 7fd8b5bbc700 1 --2- 172.21.15.157:0/1489126611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8b0152660 0x7fd8b015bbd0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8011000 tx=0x7fd8a8015000 comp rx=0 tx=0).stop 2023-10-06T04:02:09.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.854+0000 7fd8b5bbc700 1 -- 172.21.15.157:0/1489126611 shutdown_connections 2023-10-06T04:02:09.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.854+0000 7fd8b5bbc700 1 --2- 172.21.15.157:0/1489126611 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd89c066f30 0x7fd89c0693e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:09.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.854+0000 7fd8b5bbc700 1 --2- 172.21.15.157:0/1489126611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8b00c3990 0x7fd8b0157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:09.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.854+0000 7fd8b5bbc700 1 --2- 172.21.15.157:0/1489126611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8b0152660 0x7fd8b015bbd0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:09.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.854+0000 7fd8b5bbc700 1 --2- 172.21.15.157:0/1489126611 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd8b00c3d60 0x7fd8b0166580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:09.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.854+0000 7fd8b5bbc700 1 -- 172.21.15.157:0/1489126611 >> 172.21.15.157:0/1489126611 conn(0x7fd8b00692b0 msgr2=0x7fd8b00c9460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:09.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.854+0000 7fd8b5bbc700 1 -- 172.21.15.157:0/1489126611 shutdown_connections 2023-10-06T04:02:09.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:09.854+0000 7fd8b5bbc700 1 -- 172.21.15.157:0/1489126611 wait complete. 2023-10-06T04:02:09.857 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 39 2023-10-06T04:02:09.868 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:02:11.868 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:02:11.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.994+0000 7f486549d700 1 Processor -- start 2023-10-06T04:02:11.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.994+0000 7f486549d700 1 -- start start 2023-10-06T04:02:12.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.995+0000 7f486549d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f48600b04d0 0x7f4860157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:12.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.995+0000 7f486549d700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f486006c2d0 0x7f486006dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:12.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.995+0000 7f486549d700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f486006d8d0 con 0x7f48600b1e40 2023-10-06T04:02:12.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.995+0000 7f486549d700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48600af8f0 con 0x7f48600b04d0 2023-10-06T04:02:12.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.995+0000 7f486549d700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f486015a690 con 0x7f486006c2d0 2023-10-06T04:02:12.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.995+0000 7f485ffff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f48600b1e40 0x7f48601563b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40058/0 (socket says 172.21.15.157:40058) 2023-10-06T04:02:12.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.995+0000 7f485ffff700 1 -- 172.21.15.157:0/1584821864 learned_addr learned my addr 172.21.15.157:0/1584821864 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:02:12.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.995+0000 7f485f7fe700 1 --2- 172.21.15.157:0/1584821864 >> v2:172.21.15.157:3300/0 conn(0x7f48600b04d0 0x7f4860157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:12.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.995+0000 7f4864c9c700 1 --2- 172.21.15.157:0/1584821864 >> v2:172.21.15.175:3300/0 conn(0x7f486006c2d0 0x7f486006dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:12.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.995+0000 7f485effd700 1 -- 172.21.15.157:0/1584821864 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3322571464 0 0) 0x7f486006d8d0 con 0x7f48600b1e40 2023-10-06T04:02:12.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.995+0000 7f485effd700 1 -- 172.21.15.157:0/1584821864 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4848003570 con 0x7f48600b1e40 2023-10-06T04:02:12.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.996+0000 7f485f7fe700 1 -- 172.21.15.157:0/1584821864 >> v2:172.21.15.175:3300/0 conn(0x7f486006c2d0 msgr2=0x7f486006dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:12.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.996+0000 7f485f7fe700 1 --2- 172.21.15.157:0/1584821864 >> v2:172.21.15.175:3300/0 conn(0x7f486006c2d0 0x7f486006dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:12.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.996+0000 7f485f7fe700 1 -- 172.21.15.157:0/1584821864 >> v1:172.21.15.157:6789/0 conn(0x7f48600b1e40 legacy=0x7f48601563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:12.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.996+0000 7f485f7fe700 1 -- 172.21.15.157:0/1584821864 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486006ccc0 con 0x7f48600b04d0 2023-10-06T04:02:12.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.996+0000 7f485f7fe700 1 --2- 172.21.15.157:0/1584821864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48600b04d0 0x7f4860157750 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f48600c34e0 tx=0x7f48540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8379a0753e4651a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:12.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.996+0000 7f485effd700 1 -- 172.21.15.157:0/1584821864 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4854011630 con 0x7f48600b04d0 2023-10-06T04:02:12.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.996+0000 7f485effd700 1 -- 172.21.15.157:0/1584821864 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4854011790 con 0x7f48600b04d0 2023-10-06T04:02:12.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.996+0000 7f485effd700 1 -- 172.21.15.157:0/1584821864 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4854011b30 con 0x7f48600b04d0 2023-10-06T04:02:12.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.996+0000 7f486549d700 1 -- 172.21.15.157:0/1584821864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48600b04d0 msgr2=0x7f4860157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:12.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.996+0000 7f486549d700 1 --2- 172.21.15.157:0/1584821864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48600b04d0 0x7f4860157750 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f48600c34e0 tx=0x7f48540091a0 comp rx=0 tx=0).stop 2023-10-06T04:02:12.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.996+0000 7f486549d700 1 -- 172.21.15.157:0/1584821864 shutdown_connections 2023-10-06T04:02:12.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.996+0000 7f486549d700 1 --2- 172.21.15.157:0/1584821864 >> v2:172.21.15.175:3300/0 conn(0x7f486006c2d0 0x7f486006dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:12.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.996+0000 7f486549d700 1 --2- 172.21.15.157:0/1584821864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48600b04d0 0x7f4860157750 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:12.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.996+0000 7f486549d700 1 -- 172.21.15.157:0/1584821864 >> 172.21.15.157:0/1584821864 conn(0x7f48600cc540 msgr2=0x7f48600cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:12.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.996+0000 7f486549d700 1 -- 172.21.15.157:0/1584821864 shutdown_connections 2023-10-06T04:02:12.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.996+0000 7f486549d700 1 -- 172.21.15.157:0/1584821864 wait complete. 2023-10-06T04:02:12.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.997+0000 7f486549d700 1 Processor -- start 2023-10-06T04:02:12.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.998+0000 7f486549d700 1 -- start start 2023-10-06T04:02:12.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.998+0000 7f486549d700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f486006c2d0 0x7f4860166830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:12.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.998+0000 7f486549d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48600b04d0 0x7f4860156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:12.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.998+0000 7f486549d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48600b1e40 0x7f4860169b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:12.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.998+0000 7f486549d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4860150110 con 0x7f48600b1e40 2023-10-06T04:02:12.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.998+0000 7f486549d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48601663a0 con 0x7f48600b04d0 2023-10-06T04:02:12.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.998+0000 7f486549d700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48600cb960 con 0x7f486006c2d0 2023-10-06T04:02:12.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.999+0000 7f4864c9c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48600b1e40 0x7f4860169b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:12.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.999+0000 7f4864c9c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48600b1e40 0x7f4860169b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49400/0 (socket says 172.21.15.157:49400) 2023-10-06T04:02:12.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.999+0000 7f4864c9c700 1 -- 172.21.15.157:0/1317266280 learned_addr learned my addr 172.21.15.157:0/1317266280 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:12.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.999+0000 7f485f7fe700 1 --2- 172.21.15.157:0/1317266280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48600b04d0 0x7f4860156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:12.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.999+0000 7f485ffff700 1 --2- 172.21.15.157:0/1317266280 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f486006c2d0 0x7f4860166830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:12.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.999+0000 7f4864c9c700 1 -- 172.21.15.157:0/1317266280 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f486006c2d0 msgr2=0x7f4860166830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:12.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.999+0000 7f4864c9c700 1 --2- 172.21.15.157:0/1317266280 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f486006c2d0 0x7f4860166830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:12.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.999+0000 7f4864c9c700 1 -- 172.21.15.157:0/1317266280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48600b04d0 msgr2=0x7f4860156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:12.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.999+0000 7f4864c9c700 1 --2- 172.21.15.157:0/1317266280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48600b04d0 0x7f4860156660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:12.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.999+0000 7f4864c9c700 1 -- 172.21.15.157:0/1317266280 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4854009420 con 0x7f48600b1e40 2023-10-06T04:02:12.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.999+0000 7f4864c9c700 1 --2- 172.21.15.157:0/1317266280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48600b1e40 0x7f4860169b50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4858007f50 tx=0x7f4858009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:12.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.999+0000 7f4847fff700 1 -- 172.21.15.157:0/1317266280 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485800ec00 con 0x7f48600b1e40 2023-10-06T04:02:12.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.999+0000 7f4847fff700 1 -- 172.21.15.157:0/1317266280 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485800ed80 con 0x7f48600b1e40 2023-10-06T04:02:12.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.999+0000 7f486549d700 1 -- 172.21.15.157:0/1317266280 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48600ca320 con 0x7f48600b1e40 2023-10-06T04:02:12.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.999+0000 7f4847fff700 1 -- 172.21.15.157:0/1317266280 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4858014670 con 0x7f48600b1e40 2023-10-06T04:02:12.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:11.999+0000 7f486549d700 1 -- 172.21.15.157:0/1317266280 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48600ca8e0 con 0x7f48600b1e40 2023-10-06T04:02:12.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:12.000+0000 7f486549d700 1 -- 172.21.15.157:0/1317266280 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4860088f90 con 0x7f48600b1e40 2023-10-06T04:02:12.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:12.001+0000 7f4847fff700 1 -- 172.21.15.157:0/1317266280 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f48580149e0 con 0x7f48600b1e40 2023-10-06T04:02:12.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:12.001+0000 7f4847fff700 1 --2- 172.21.15.157:0/1317266280 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f484c066f30 0x7f484c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:12.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:12.001+0000 7f4847fff700 1 -- 172.21.15.157:0/1317266280 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f4858084de0 con 0x7f48600b1e40 2023-10-06T04:02:12.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:12.001+0000 7f485ffff700 1 --2- 172.21.15.157:0/1317266280 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f484c066f30 0x7f484c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:12.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:12.002+0000 7f485ffff700 1 --2- 172.21.15.157:0/1317266280 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f484c066f30 0x7f484c0693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4850007fd0 tx=0x7f4850009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:12.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:12.005+0000 7f4847fff700 1 -- 172.21.15.157:0/1317266280 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4858051e10 con 0x7f48600b1e40 2023-10-06T04:02:12.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:12.233+0000 7f486549d700 1 -- 172.21.15.157:0/1317266280 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f486006cfa0 con 0x7f48600b1e40 2023-10-06T04:02:12.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:12.234+0000 7f4847fff700 1 -- 172.21.15.157:0/1317266280 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+6207 (secure 0 0 0) 0x7f4858055430 con 0x7f48600b1e40 2023-10-06T04:02:12.235 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:02:12.235 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:02:02.817947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"f","rank":4,"incarnation":29,"state":"up:stopping","state_seq":32,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:02:12.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:12.236+0000 7f486549d700 1 -- 172.21.15.157:0/1317266280 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f484c066f30 msgr2=0x7f484c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:12.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:12.236+0000 7f486549d700 1 --2- 172.21.15.157:0/1317266280 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f484c066f30 0x7f484c0693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4850007fd0 tx=0x7f4850009040 comp rx=0 tx=0).stop 2023-10-06T04:02:12.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:12.236+0000 7f486549d700 1 -- 172.21.15.157:0/1317266280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48600b1e40 msgr2=0x7f4860169b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:12.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:12.236+0000 7f486549d700 1 --2- 172.21.15.157:0/1317266280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48600b1e40 0x7f4860169b50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4858007f50 tx=0x7f4858009860 comp rx=0 tx=0).stop 2023-10-06T04:02:12.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:12.236+0000 7f486549d700 1 -- 172.21.15.157:0/1317266280 shutdown_connections 2023-10-06T04:02:12.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:12.236+0000 7f486549d700 1 --2- 172.21.15.157:0/1317266280 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f484c066f30 0x7f484c0693e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:12.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:12.236+0000 7f486549d700 1 --2- 172.21.15.157:0/1317266280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48600b1e40 0x7f4860169b50 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:12.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:12.236+0000 7f486549d700 1 --2- 172.21.15.157:0/1317266280 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f486006c2d0 0x7f4860166830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:12.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:12.236+0000 7f486549d700 1 --2- 172.21.15.157:0/1317266280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48600b04d0 0x7f4860156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:12.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:12.236+0000 7f486549d700 1 -- 172.21.15.157:0/1317266280 >> 172.21.15.157:0/1317266280 conn(0x7f48600cc540 msgr2=0x7f48601572c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:12.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:12.236+0000 7f486549d700 1 -- 172.21.15.157:0/1317266280 shutdown_connections 2023-10-06T04:02:12.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:12.236+0000 7f486549d700 1 -- 172.21.15.157:0/1317266280 wait complete. 2023-10-06T04:02:12.239 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 39 2023-10-06T04:02:12.250 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:02:14.242 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:02:14.245 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:02:14.251 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:02:14.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.411+0000 7f6cb4ef8700 1 Processor -- start 2023-10-06T04:02:14.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.411+0000 7f6cb4ef8700 1 -- start start 2023-10-06T04:02:14.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.412+0000 7f6cb4ef8700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6cb00c5fc0 0x7f6cb0069130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:14.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.412+0000 7f6cb4ef8700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6cb006c810 0x7f6cb01569e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:14.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.412+0000 7f6cb4ef8700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cb0155f20 con 0x7f6cb006c440 2023-10-06T04:02:14.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.412+0000 7f6cb4ef8700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cb006f310 con 0x7f6cb006c810 2023-10-06T04:02:14.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.412+0000 7f6cb4ef8700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cb006d4a0 con 0x7f6cb00c5fc0 2023-10-06T04:02:14.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.412+0000 7f6caed9d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6cb006c810 0x7f6cb01569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:14.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.412+0000 7f6caed9d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6cb006c810 0x7f6cb01569e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39486/0 (socket says 172.21.15.157:39486) 2023-10-06T04:02:14.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.412+0000 7f6caed9d700 1 -- 172.21.15.157:0/2959811705 learned_addr learned my addr 172.21.15.157:0/2959811705 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:14.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.413+0000 7f6ca5d9b700 1 --2- 172.21.15.157:0/2959811705 >> v2:172.21.15.175:3300/0 conn(0x7f6cb00c5fc0 0x7f6cb0069130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:14.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.413+0000 7f6caed9d700 1 -- 172.21.15.157:0/2959811705 >> v2:172.21.15.175:3300/0 conn(0x7f6cb00c5fc0 msgr2=0x7f6cb0069130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:14.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.413+0000 7f6caed9d700 1 --2- 172.21.15.157:0/2959811705 >> v2:172.21.15.175:3300/0 conn(0x7f6cb00c5fc0 0x7f6cb0069130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:14.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.413+0000 7f6caed9d700 1 -- 172.21.15.157:0/2959811705 >> v1:172.21.15.175:6789/0 conn(0x7f6cb006c440 legacy=0x7f6cb0152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:14.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.413+0000 7f6caed9d700 1 -- 172.21.15.157:0/2959811705 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb006d780 con 0x7f6cb006c810 2023-10-06T04:02:14.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.413+0000 7f6cadd9b700 1 -- 172.21.15.157:0/2959811705 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2733293489 0 0) 0x7f6cb0155f20 con 0x7f6cb006c440 2023-10-06T04:02:14.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.413+0000 7f6caed9d700 1 --2- 172.21.15.157:0/2959811705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cb006c810 0x7f6cb01569e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6cb00cac60 tx=0x7f6ca8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1156ef707fe316a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:14.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.413+0000 7f6cadd9b700 1 -- 172.21.15.157:0/2959811705 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca800e070 con 0x7f6cb006c810 2023-10-06T04:02:14.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.413+0000 7f6cadd9b700 1 -- 172.21.15.157:0/2959811705 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca800ba20 con 0x7f6cb006c810 2023-10-06T04:02:14.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.413+0000 7f6cadd9b700 1 -- 172.21.15.157:0/2959811705 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca80092e0 con 0x7f6cb006c810 2023-10-06T04:02:14.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.414+0000 7f6cb4ef8700 1 -- 172.21.15.157:0/2959811705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cb006c810 msgr2=0x7f6cb01569e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:14.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.414+0000 7f6cb4ef8700 1 --2- 172.21.15.157:0/2959811705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cb006c810 0x7f6cb01569e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6cb00cac60 tx=0x7f6ca8009840 comp rx=0 tx=0).stop 2023-10-06T04:02:14.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.414+0000 7f6cb4ef8700 1 -- 172.21.15.157:0/2959811705 shutdown_connections 2023-10-06T04:02:14.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.414+0000 7f6cb4ef8700 1 --2- 172.21.15.157:0/2959811705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cb006c810 0x7f6cb01569e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:14.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.414+0000 7f6cb4ef8700 1 --2- 172.21.15.157:0/2959811705 >> v2:172.21.15.175:3300/0 conn(0x7f6cb00c5fc0 0x7f6cb0069130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:14.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.414+0000 7f6cb4ef8700 1 -- 172.21.15.157:0/2959811705 >> 172.21.15.157:0/2959811705 conn(0x7f6cb0068370 msgr2=0x7f6cb0068770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:14.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.414+0000 7f6cb4ef8700 1 -- 172.21.15.157:0/2959811705 shutdown_connections 2023-10-06T04:02:14.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.414+0000 7f6cb4ef8700 1 -- 172.21.15.157:0/2959811705 wait complete. 2023-10-06T04:02:14.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.415+0000 7f6cb4ef8700 1 Processor -- start 2023-10-06T04:02:14.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.415+0000 7f6cb4ef8700 1 -- start start 2023-10-06T04:02:14.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.416+0000 7f6cb4ef8700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6cb006c440 0x7f6cb00ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:14.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.416+0000 7f6cb4ef8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cb006c810 0x7f6cb015be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:14.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.416+0000 7f6cb4ef8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cb00c5fc0 0x7f6cb01574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:14.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.416+0000 7f6cb4ef8700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb0157090 con 0x7f6cb006c810 2023-10-06T04:02:14.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.416+0000 7f6cb4ef8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb01668f0 con 0x7f6cb00c5fc0 2023-10-06T04:02:14.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.416+0000 7f6cb4ef8700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb014fba0 con 0x7f6cb006c440 2023-10-06T04:02:14.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.416+0000 7f6ca5d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cb006c810 0x7f6cb015be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:14.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.416+0000 7f6ca5d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cb006c810 0x7f6cb015be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39494/0 (socket says 172.21.15.157:39494) 2023-10-06T04:02:14.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.416+0000 7f6ca5d9b700 1 -- 172.21.15.157:0/1415354529 learned_addr learned my addr 172.21.15.157:0/1415354529 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:14.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.416+0000 7f6caed9d700 1 --2- 172.21.15.157:0/1415354529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cb00c5fc0 0x7f6cb01574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:14.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.416+0000 7f6cae59c700 1 --2- 172.21.15.157:0/1415354529 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6cb006c440 0x7f6cb00ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:14.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.416+0000 7f6ca5d9b700 1 -- 172.21.15.157:0/1415354529 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6cb006c440 msgr2=0x7f6cb00ae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:14.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.416+0000 7f6ca5d9b700 1 --2- 172.21.15.157:0/1415354529 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6cb006c440 0x7f6cb00ae2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:14.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.416+0000 7f6ca5d9b700 1 -- 172.21.15.157:0/1415354529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cb00c5fc0 msgr2=0x7f6cb01574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:14.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.416+0000 7f6ca5d9b700 1 --2- 172.21.15.157:0/1415354529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cb00c5fc0 0x7f6cb01574e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:14.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.416+0000 7f6ca5d9b700 1 -- 172.21.15.157:0/1415354529 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca8009ae0 con 0x7f6cb006c810 2023-10-06T04:02:14.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.416+0000 7f6cae59c700 1 --2- 172.21.15.157:0/1415354529 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6cb006c440 0x7f6cb00ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:02:14.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.416+0000 7f6ca5d9b700 1 --2- 172.21.15.157:0/1415354529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cb006c810 0x7f6cb015be50 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6c9800ec80 tx=0x7f6c980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:14.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.416+0000 7f6ca6ffd700 1 -- 172.21.15.157:0/1415354529 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9800c850 con 0x7f6cb006c810 2023-10-06T04:02:14.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.416+0000 7f6ca6ffd700 1 -- 172.21.15.157:0/1415354529 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9800c9d0 con 0x7f6cb006c810 2023-10-06T04:02:14.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.416+0000 7f6cb4ef8700 1 -- 172.21.15.157:0/1415354529 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb014fe40 con 0x7f6cb006c810 2023-10-06T04:02:14.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.416+0000 7f6ca6ffd700 1 -- 172.21.15.157:0/1415354529 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9800cd90 con 0x7f6cb006c810 2023-10-06T04:02:14.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.416+0000 7f6cb4ef8700 1 -- 172.21.15.157:0/1415354529 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb00c58f0 con 0x7f6cb006c810 2023-10-06T04:02:14.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.417+0000 7f6cb4ef8700 1 -- 172.21.15.157:0/1415354529 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cb01571e0 con 0x7f6cb006c810 2023-10-06T04:02:14.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.418+0000 7f6ca6ffd700 1 -- 172.21.15.157:0/1415354529 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6c98013680 con 0x7f6cb006c810 2023-10-06T04:02:14.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.418+0000 7f6ca6ffd700 1 --2- 172.21.15.157:0/1415354529 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6c90066f60 0x7f6c90069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:14.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.418+0000 7f6ca6ffd700 1 -- 172.21.15.157:0/1415354529 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f6c98084a30 con 0x7f6cb006c810 2023-10-06T04:02:14.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.418+0000 7f6cae59c700 1 --2- 172.21.15.157:0/1415354529 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6c90066f60 0x7f6c90069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:14.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.419+0000 7f6cae59c700 1 --2- 172.21.15.157:0/1415354529 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6c90066f60 0x7f6c90069410 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0007fd0 tx=0x7f6ca0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:14.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.422+0000 7f6ca6ffd700 1 -- 172.21.15.157:0/1415354529 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c98051a60 con 0x7f6cb006c810 2023-10-06T04:02:14.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.651+0000 7f6cb4ef8700 1 -- 172.21.15.157:0/1415354529 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6cb01561e0 con 0x7f6cb006c810 2023-10-06T04:02:14.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.651+0000 7f6ca6ffd700 1 -- 172.21.15.157:0/1415354529 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+6207 (secure 0 0 0) 0x7f6c98055080 con 0x7f6cb006c810 2023-10-06T04:02:14.653 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:02:14.653 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:02:02.817947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"f","rank":4,"incarnation":29,"state":"up:stopping","state_seq":32,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:02:14.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.653+0000 7f6cb4ef8700 1 -- 172.21.15.157:0/1415354529 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6c90066f60 msgr2=0x7f6c90069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:14.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.653+0000 7f6cb4ef8700 1 --2- 172.21.15.157:0/1415354529 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6c90066f60 0x7f6c90069410 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0007fd0 tx=0x7f6ca0009040 comp rx=0 tx=0).stop 2023-10-06T04:02:14.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.653+0000 7f6cb4ef8700 1 -- 172.21.15.157:0/1415354529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cb006c810 msgr2=0x7f6cb015be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:14.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.653+0000 7f6cb4ef8700 1 --2- 172.21.15.157:0/1415354529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cb006c810 0x7f6cb015be50 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6c9800ec80 tx=0x7f6c980091a0 comp rx=0 tx=0).stop 2023-10-06T04:02:14.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.653+0000 7f6cb4ef8700 1 -- 172.21.15.157:0/1415354529 shutdown_connections 2023-10-06T04:02:14.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.653+0000 7f6cb4ef8700 1 --2- 172.21.15.157:0/1415354529 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6c90066f60 0x7f6c90069410 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:14.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.653+0000 7f6cb4ef8700 1 --2- 172.21.15.157:0/1415354529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cb00c5fc0 0x7f6cb01574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:14.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.653+0000 7f6cb4ef8700 1 --2- 172.21.15.157:0/1415354529 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6cb006c440 0x7f6cb00ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:14.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.653+0000 7f6cb4ef8700 1 --2- 172.21.15.157:0/1415354529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6cb006c810 0x7f6cb015be50 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:14.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.653+0000 7f6cb4ef8700 1 -- 172.21.15.157:0/1415354529 >> 172.21.15.157:0/1415354529 conn(0x7f6cb0068370 msgr2=0x7f6cb0152090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:14.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.653+0000 7f6cb4ef8700 1 -- 172.21.15.157:0/1415354529 shutdown_connections 2023-10-06T04:02:14.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:14.653+0000 7f6cb4ef8700 1 -- 172.21.15.157:0/1415354529 wait complete. 2023-10-06T04:02:14.656 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 39 2023-10-06T04:02:14.667 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:02:16.668 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:02:16.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.793+0000 7faa8b8d5700 1 Processor -- start 2023-10-06T04:02:16.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.793+0000 7faa8b8d5700 1 -- start start 2023-10-06T04:02:16.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.794+0000 7faa8b8d5700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa8406cbb0 0x7faa8406d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:16.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.794+0000 7faa8b8d5700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa840c9780 con 0x7faa84150410 2023-10-06T04:02:16.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.794+0000 7faa8b8d5700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa8406e0c0 con 0x7faa8406c1f0 2023-10-06T04:02:16.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.794+0000 7faa8b8d5700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa840c8c40 con 0x7faa8406cbb0 2023-10-06T04:02:16.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.794+0000 7faa89e72700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7faa84150410 0x7faa84155f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41662/0 (socket says 172.21.15.157:41662) 2023-10-06T04:02:16.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.794+0000 7faa89e72700 1 -- 172.21.15.157:0/1237602686 learned_addr learned my addr 172.21.15.157:0/1237602686 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:02:16.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.794+0000 7faa89671700 1 --2- 172.21.15.157:0/1237602686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa8406cbb0 0x7faa8406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:16.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.795+0000 7faa7bfff700 1 -- 172.21.15.157:0/1237602686 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 389209646 0 0) 0x7faa840c9780 con 0x7faa84150410 2023-10-06T04:02:16.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.795+0000 7faa7bfff700 1 -- 172.21.15.157:0/1237602686 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa70003570 con 0x7faa84150410 2023-10-06T04:02:16.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.795+0000 7faa7bfff700 1 -- 172.21.15.157:0/1237602686 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 335511742 0 0) 0x7faa8406e0c0 con 0x7faa8406c1f0 2023-10-06T04:02:16.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.795+0000 7faa7bfff700 1 -- 172.21.15.157:0/1237602686 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa840c9780 con 0x7faa8406c1f0 2023-10-06T04:02:16.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.795+0000 7faa7bfff700 1 -- 172.21.15.157:0/1237602686 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1467873613 0 0) 0x7faa70003570 con 0x7faa84150410 2023-10-06T04:02:16.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.795+0000 7faa89671700 1 -- 172.21.15.157:0/1237602686 >> v1:172.21.15.175:6789/0 conn(0x7faa8406c1f0 legacy=0x7faa8414fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:16.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.795+0000 7faa89671700 1 -- 172.21.15.157:0/1237602686 >> v1:172.21.15.157:6789/0 conn(0x7faa84150410 legacy=0x7faa84155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:16.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.795+0000 7faa89671700 1 -- 172.21.15.157:0/1237602686 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa840c8e50 con 0x7faa8406cbb0 2023-10-06T04:02:16.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.795+0000 7faa89671700 1 --2- 172.21.15.157:0/1237602686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa8406cbb0 0x7faa8406d530 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faa80009910 tx=0x7faa800091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4edbc6544205a7c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:16.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.795+0000 7faa7bfff700 1 -- 172.21.15.157:0/1237602686 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c0017f0 con 0x7faa8406cbb0 2023-10-06T04:02:16.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.795+0000 7faa7bfff700 1 -- 172.21.15.157:0/1237602686 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa80011420 con 0x7faa8406cbb0 2023-10-06T04:02:16.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.795+0000 7faa7bfff700 1 -- 172.21.15.157:0/1237602686 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa800117c0 con 0x7faa8406cbb0 2023-10-06T04:02:16.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.796+0000 7faa8b8d5700 1 -- 172.21.15.157:0/1237602686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa8406cbb0 msgr2=0x7faa8406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:16.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.796+0000 7faa8b8d5700 1 --2- 172.21.15.157:0/1237602686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa8406cbb0 0x7faa8406d530 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faa80009910 tx=0x7faa800091a0 comp rx=0 tx=0).stop 2023-10-06T04:02:16.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.796+0000 7faa8b8d5700 1 -- 172.21.15.157:0/1237602686 shutdown_connections 2023-10-06T04:02:16.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.796+0000 7faa8b8d5700 1 --2- 172.21.15.157:0/1237602686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa8406cbb0 0x7faa8406d530 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:16.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.796+0000 7faa8b8d5700 1 -- 172.21.15.157:0/1237602686 >> 172.21.15.157:0/1237602686 conn(0x7faa8406f230 msgr2=0x7faa8406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:16.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.796+0000 7faa8b8d5700 1 -- 172.21.15.157:0/1237602686 shutdown_connections 2023-10-06T04:02:16.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.796+0000 7faa8b8d5700 1 -- 172.21.15.157:0/1237602686 wait complete. 2023-10-06T04:02:16.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.797+0000 7faa8b8d5700 1 Processor -- start 2023-10-06T04:02:16.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.797+0000 7faa8b8d5700 1 -- start start 2023-10-06T04:02:16.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.798+0000 7faa8b8d5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa8406c1f0 0x7faa8414d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:16.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.798+0000 7faa8b8d5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa8406cbb0 0x7faa84144b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:16.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.798+0000 7faa8b8d5700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa84150410 0x7faa84145060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:16.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.798+0000 7faa8b8d5700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa841509d0 con 0x7faa8406c1f0 2023-10-06T04:02:16.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.798+0000 7faa8b8d5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa84150c90 con 0x7faa8406cbb0 2023-10-06T04:02:16.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.798+0000 7faa8b8d5700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa84150f70 con 0x7faa84150410 2023-10-06T04:02:16.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.798+0000 7faa89671700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa8406c1f0 0x7faa8414d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:16.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.798+0000 7faa89671700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa8406c1f0 0x7faa8414d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39500/0 (socket says 172.21.15.157:39500) 2023-10-06T04:02:16.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.798+0000 7faa89671700 1 -- 172.21.15.157:0/1655359010 learned_addr learned my addr 172.21.15.157:0/1655359010 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:16.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.798+0000 7faa88e70700 1 --2- 172.21.15.157:0/1655359010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa8406cbb0 0x7faa84144b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:16.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.798+0000 7faa89e72700 1 --2- 172.21.15.157:0/1655359010 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa84150410 0x7faa84145060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:16.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.798+0000 7faa89671700 1 -- 172.21.15.157:0/1655359010 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa84150410 msgr2=0x7faa84145060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:16.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.798+0000 7faa89671700 1 --2- 172.21.15.157:0/1655359010 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa84150410 0x7faa84145060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:16.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.798+0000 7faa89671700 1 -- 172.21.15.157:0/1655359010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa8406cbb0 msgr2=0x7faa84144b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:16.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.798+0000 7faa89671700 1 --2- 172.21.15.157:0/1655359010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa8406cbb0 0x7faa84144b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:16.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.798+0000 7faa89671700 1 -- 172.21.15.157:0/1655359010 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa80009420 con 0x7faa8406c1f0 2023-10-06T04:02:16.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.798+0000 7faa89671700 1 --2- 172.21.15.157:0/1655359010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa8406c1f0 0x7faa8414d9d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7faa800091e0 tx=0x7faa80016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:16.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.798+0000 7faa797fa700 1 -- 172.21.15.157:0/1655359010 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8001cc90 con 0x7faa8406c1f0 2023-10-06T04:02:16.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.798+0000 7faa797fa700 1 -- 172.21.15.157:0/1655359010 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa8001cdf0 con 0x7faa8406c1f0 2023-10-06T04:02:16.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.798+0000 7faa8b8d5700 1 -- 172.21.15.157:0/1655359010 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa84151250 con 0x7faa8406c1f0 2023-10-06T04:02:16.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.798+0000 7faa797fa700 1 -- 172.21.15.157:0/1655359010 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8001a6a0 con 0x7faa8406c1f0 2023-10-06T04:02:16.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.799+0000 7faa8b8d5700 1 -- 172.21.15.157:0/1655359010 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa840c3b60 con 0x7faa8406c1f0 2023-10-06T04:02:16.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.799+0000 7faa8b8d5700 1 -- 172.21.15.157:0/1655359010 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa841516f0 con 0x7faa8406c1f0 2023-10-06T04:02:16.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.800+0000 7faa797fa700 1 -- 172.21.15.157:0/1655359010 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faa8001aa90 con 0x7faa8406c1f0 2023-10-06T04:02:16.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.800+0000 7faa797fa700 1 --2- 172.21.15.157:0/1655359010 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faa68066f30 0x7faa680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:16.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.800+0000 7faa797fa700 1 -- 172.21.15.157:0/1655359010 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7faa80085950 con 0x7faa8406c1f0 2023-10-06T04:02:16.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.801+0000 7faa88e70700 1 --2- 172.21.15.157:0/1655359010 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faa68066f30 0x7faa680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:16.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.801+0000 7faa88e70700 1 --2- 172.21.15.157:0/1655359010 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faa68066f30 0x7faa680693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faa840c6300 tx=0x7faa74009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:16.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:16.805+0000 7faa797fa700 1 -- 172.21.15.157:0/1655359010 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa80052980 con 0x7faa8406c1f0 2023-10-06T04:02:17.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:17.031+0000 7faa8b8d5700 1 -- 172.21.15.157:0/1655359010 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa840c9130 con 0x7faa8406c1f0 2023-10-06T04:02:17.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:17.031+0000 7faa797fa700 1 -- 172.21.15.157:0/1655359010 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+6207 (secure 0 0 0) 0x7faa80055fa0 con 0x7faa8406c1f0 2023-10-06T04:02:17.034 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:02:17.034 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:02:02.817947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"f","rank":4,"incarnation":29,"state":"up:stopping","state_seq":32,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:02:17.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:17.033+0000 7faa8b8d5700 1 -- 172.21.15.157:0/1655359010 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faa68066f30 msgr2=0x7faa680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:17.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:17.033+0000 7faa8b8d5700 1 --2- 172.21.15.157:0/1655359010 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faa68066f30 0x7faa680693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faa840c6300 tx=0x7faa74009040 comp rx=0 tx=0).stop 2023-10-06T04:02:17.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:17.033+0000 7faa8b8d5700 1 -- 172.21.15.157:0/1655359010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa8406c1f0 msgr2=0x7faa8414d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:17.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:17.033+0000 7faa8b8d5700 1 --2- 172.21.15.157:0/1655359010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa8406c1f0 0x7faa8414d9d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7faa800091e0 tx=0x7faa80016000 comp rx=0 tx=0).stop 2023-10-06T04:02:17.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:17.033+0000 7faa8b8d5700 1 -- 172.21.15.157:0/1655359010 shutdown_connections 2023-10-06T04:02:17.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:17.033+0000 7faa8b8d5700 1 --2- 172.21.15.157:0/1655359010 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faa68066f30 0x7faa680693e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:17.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:17.033+0000 7faa8b8d5700 1 --2- 172.21.15.157:0/1655359010 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa84150410 0x7faa84145060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:17.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:17.033+0000 7faa8b8d5700 1 --2- 172.21.15.157:0/1655359010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa8406c1f0 0x7faa8414d9d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:17.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:17.033+0000 7faa8b8d5700 1 --2- 172.21.15.157:0/1655359010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa8406cbb0 0x7faa84144b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:17.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:17.033+0000 7faa8b8d5700 1 -- 172.21.15.157:0/1655359010 >> 172.21.15.157:0/1655359010 conn(0x7faa8406f230 msgr2=0x7faa840b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:17.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:17.033+0000 7faa8b8d5700 1 -- 172.21.15.157:0/1655359010 shutdown_connections 2023-10-06T04:02:17.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:17.033+0000 7faa8b8d5700 1 -- 172.21.15.157:0/1655359010 wait complete. 2023-10-06T04:02:17.038 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 39 2023-10-06T04:02:17.047 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2023-10-06T04:02:17.048 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 39, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-06T03:57:10.718782+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 39, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0, 1, 2, 3, 4 ], "info": { "gid_4270": { "addr": "172.21.15.175:6835/1596042861", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6834", "nonce": 1596042861, "type": "v2" }, { "addr": "172.21.15.175:6835", "nonce": 1596042861, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4270, "incarnation": 13, "join_fscid": -1, "name": "b", "rank": 3, "state": "up:active", "state_seq": 8 }, "gid_4279": { "addr": "172.21.15.175:6837/1002404870", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6836", "nonce": 1002404870, "type": "v2" }, { "addr": "172.21.15.175:6837", "nonce": 1002404870, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1, 3, 4 ], "features": 4540138320759226367, "flags": 0, "gid": 4279, "incarnation": 11, "join_fscid": -1, "name": "d", "rank": 2, "state": "up:active", "state_seq": 7 }, "gid_4287": { "addr": "172.21.15.157:6834/1179753185", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6832", "nonce": 1179753185, "type": "v2" }, { "addr": "172.21.15.157:6834", "nonce": 1179753185, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4287, "incarnation": 9, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4296": { "addr": "172.21.15.157:6837/4188652695", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6836", "nonce": 4188652695, "type": "v2" }, { "addr": "172.21.15.157:6837", "nonce": 4188652695, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2, 4 ], "features": 4540138320759226367, "flags": 0, "gid": 4296, "incarnation": 4, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 4 }, "gid_4936": { "addr": "172.21.15.175:6839/3353503675", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6838", "nonce": 3353503675, "type": "v2" }, { "addr": "172.21.15.175:6839", "nonce": 3353503675, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1, 2, 3 ], "features": 4540138320759226367, "flags": 0, "gid": 4936, "incarnation": 29, "join_fscid": -1, "name": "f", "rank": 4, "state": "up:stopping", "state_seq": 32 } }, "last_failure": 0, "last_failure_osd_epoch": 27, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2023-10-06T04:02:02.817947+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4296, "mds_1": 4287, "mds_2": 4279, "mds_3": 4270, "mds_4": 4936 } } } ], "standbys": [ { "addr": "172.21.15.157:6835/2461751318", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6833", "nonce": 2461751318, "type": "v2" }, { "addr": "172.21.15.157:6835", "nonce": 2461751318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 34, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5268, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-06T04:02:19.049 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:02:19.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.176+0000 7fb142213700 1 Processor -- start 2023-10-06T04:02:19.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.176+0000 7fb142213700 1 -- start start 2023-10-06T04:02:19.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.177+0000 7fb142213700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb13c06dc40 0x7fb13c14fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:19.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.177+0000 7fb142213700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb13c06c1f0 0x7fb13c150240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:19.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.177+0000 7fb142213700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb13c0c9780 con 0x7fb13c06cbb0 2023-10-06T04:02:19.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.177+0000 7fb142213700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb13c0c8c40 con 0x7fb13c06dc40 2023-10-06T04:02:19.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.177+0000 7fb142213700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c06e0c0 con 0x7fb13c06c1f0 2023-10-06T04:02:19.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.177+0000 7fb13b7fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb13c06cbb0 0x7fb13c06d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41668/0 (socket says 172.21.15.157:41668) 2023-10-06T04:02:19.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.177+0000 7fb13b7fe700 1 -- 172.21.15.157:0/4147019990 learned_addr learned my addr 172.21.15.157:0/4147019990 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:02:19.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.177+0000 7fb13affd700 1 --2- 172.21.15.157:0/4147019990 >> v2:172.21.15.157:3300/0 conn(0x7fb13c06dc40 0x7fb13c14fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:19.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.177+0000 7fb13bfff700 1 --2- 172.21.15.157:0/4147019990 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb13c06c1f0 0x7fb13c150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:19.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.177+0000 7fb13affd700 1 -- 172.21.15.157:0/4147019990 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb13c06c1f0 msgr2=0x7fb13c150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:19.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.177+0000 7fb13affd700 1 --2- 172.21.15.157:0/4147019990 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb13c06c1f0 0x7fb13c150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:19.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.177+0000 7fb13affd700 1 -- 172.21.15.157:0/4147019990 >> v1:172.21.15.157:6789/0 conn(0x7fb13c06cbb0 legacy=0x7fb13c06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:19.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.177+0000 7fb13affd700 1 -- 172.21.15.157:0/4147019990 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13c0c8e50 con 0x7fb13c06dc40 2023-10-06T04:02:19.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.177+0000 7fb13affd700 1 --2- 172.21.15.157:0/4147019990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb13c06dc40 0x7fb13c14fd00 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb1240098e0 tx=0x7fb1240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5835fdc620794ff6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:19.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.177+0000 7fb13a7fc700 1 -- 172.21.15.157:0/4147019990 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb124009e80 con 0x7fb13c06dc40 2023-10-06T04:02:19.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.177+0000 7fb13a7fc700 1 -- 172.21.15.157:0/4147019990 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb12400b760 con 0x7fb13c06dc40 2023-10-06T04:02:19.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.177+0000 7fb13a7fc700 1 -- 172.21.15.157:0/4147019990 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12400bae0 con 0x7fb13c06dc40 2023-10-06T04:02:19.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.178+0000 7fb142213700 1 -- 172.21.15.157:0/4147019990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb13c06dc40 msgr2=0x7fb13c14fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:19.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.178+0000 7fb142213700 1 --2- 172.21.15.157:0/4147019990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb13c06dc40 0x7fb13c14fd00 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb1240098e0 tx=0x7fb1240091a0 comp rx=0 tx=0).stop 2023-10-06T04:02:19.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.178+0000 7fb142213700 1 -- 172.21.15.157:0/4147019990 shutdown_connections 2023-10-06T04:02:19.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.178+0000 7fb142213700 1 --2- 172.21.15.157:0/4147019990 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb13c06c1f0 0x7fb13c150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:19.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.178+0000 7fb142213700 1 --2- 172.21.15.157:0/4147019990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb13c06dc40 0x7fb13c14fd00 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:19.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.178+0000 7fb142213700 1 -- 172.21.15.157:0/4147019990 >> 172.21.15.157:0/4147019990 conn(0x7fb13c06f230 msgr2=0x7fb13c06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:19.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.178+0000 7fb142213700 1 -- 172.21.15.157:0/4147019990 shutdown_connections 2023-10-06T04:02:19.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.178+0000 7fb142213700 1 -- 172.21.15.157:0/4147019990 wait complete. 2023-10-06T04:02:19.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.179+0000 7fb142213700 1 Processor -- start 2023-10-06T04:02:19.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.179+0000 7fb142213700 1 -- start start 2023-10-06T04:02:19.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.180+0000 7fb142213700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb13c06c1f0 0x7fb13c169c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:19.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.180+0000 7fb142213700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb13c06cbb0 0x7fb13c1478b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:19.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.180+0000 7fb142213700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb13c147df0 0x7fb13c0c5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:19.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.180+0000 7fb142213700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c153360 con 0x7fb13c06cbb0 2023-10-06T04:02:19.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.180+0000 7fb142213700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c147030 con 0x7fb13c06c1f0 2023-10-06T04:02:19.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.180+0000 7fb142213700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c0c2f40 con 0x7fb13c147df0 2023-10-06T04:02:19.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.180+0000 7fb13affd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb13c06cbb0 0x7fb13c1478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:19.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.180+0000 7fb13affd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb13c06cbb0 0x7fb13c1478b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39520/0 (socket says 172.21.15.157:39520) 2023-10-06T04:02:19.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.180+0000 7fb13affd700 1 -- 172.21.15.157:0/915638723 learned_addr learned my addr 172.21.15.157:0/915638723 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:19.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.180+0000 7fb13bfff700 1 --2- 172.21.15.157:0/915638723 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb13c147df0 0x7fb13c0c5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:19.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.180+0000 7fb13b7fe700 1 --2- 172.21.15.157:0/915638723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb13c06c1f0 0x7fb13c169c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:19.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.180+0000 7fb13affd700 1 -- 172.21.15.157:0/915638723 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb13c147df0 msgr2=0x7fb13c0c5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:19.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.180+0000 7fb13affd700 1 --2- 172.21.15.157:0/915638723 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb13c147df0 0x7fb13c0c5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:19.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.180+0000 7fb13affd700 1 -- 172.21.15.157:0/915638723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb13c06c1f0 msgr2=0x7fb13c169c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:19.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.180+0000 7fb13affd700 1 --2- 172.21.15.157:0/915638723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb13c06c1f0 0x7fb13c169c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:19.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.180+0000 7fb13affd700 1 -- 172.21.15.157:0/915638723 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb124009420 con 0x7fb13c06cbb0 2023-10-06T04:02:19.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.180+0000 7fb13b7fe700 1 --2- 172.21.15.157:0/915638723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb13c06c1f0 0x7fb13c169c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:02:19.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.181+0000 7fb13affd700 1 --2- 172.21.15.157:0/915638723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb13c06cbb0 0x7fb13c1478b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb13c0c3960 tx=0x7fb124016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:19.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.181+0000 7fb1337fe700 1 -- 172.21.15.157:0/915638723 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12400bae0 con 0x7fb13c06cbb0 2023-10-06T04:02:19.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.181+0000 7fb1337fe700 1 -- 172.21.15.157:0/915638723 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb12401c460 con 0x7fb13c06cbb0 2023-10-06T04:02:19.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.181+0000 7fb1337fe700 1 -- 172.21.15.157:0/915638723 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12401c800 con 0x7fb13c06cbb0 2023-10-06T04:02:19.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.181+0000 7fb142213700 1 -- 172.21.15.157:0/915638723 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb13c16a8e0 con 0x7fb13c06cbb0 2023-10-06T04:02:19.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.181+0000 7fb142213700 1 -- 172.21.15.157:0/915638723 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb13c0c33d0 con 0x7fb13c06cbb0 2023-10-06T04:02:19.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.182+0000 7fb142213700 1 -- 172.21.15.157:0/915638723 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb13c0c30d0 con 0x7fb13c06cbb0 2023-10-06T04:02:19.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.182+0000 7fb1337fe700 1 -- 172.21.15.157:0/915638723 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb12401cb70 con 0x7fb13c06cbb0 2023-10-06T04:02:19.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.183+0000 7fb1337fe700 1 --2- 172.21.15.157:0/915638723 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb11c066f30 0x7fb11c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:19.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.183+0000 7fb1337fe700 1 -- 172.21.15.157:0/915638723 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7fb1240862a0 con 0x7fb13c06cbb0 2023-10-06T04:02:19.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.183+0000 7fb13b7fe700 1 --2- 172.21.15.157:0/915638723 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb11c066f30 0x7fb11c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:19.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.183+0000 7fb13b7fe700 1 --2- 172.21.15.157:0/915638723 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb11c066f30 0x7fb11c0693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb12c005c50 tx=0x7fb12c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:19.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.187+0000 7fb1337fe700 1 -- 172.21.15.157:0/915638723 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb124053250 con 0x7fb13c06cbb0 2023-10-06T04:02:19.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.413+0000 7fb142213700 1 -- 172.21.15.157:0/915638723 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb13c0c9130 con 0x7fb13c06cbb0 2023-10-06T04:02:19.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.414+0000 7fb1337fe700 1 -- 172.21.15.157:0/915638723 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+6207 (secure 0 0 0) 0x7fb124056870 con 0x7fb13c06cbb0 2023-10-06T04:02:19.415 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:02:19.416 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:02:02.817947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"f","rank":4,"incarnation":29,"state":"up:stopping","state_seq":32,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:02:19.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.416+0000 7fb142213700 1 -- 172.21.15.157:0/915638723 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb11c066f30 msgr2=0x7fb11c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:19.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.416+0000 7fb142213700 1 --2- 172.21.15.157:0/915638723 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb11c066f30 0x7fb11c0693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb12c005c50 tx=0x7fb12c009040 comp rx=0 tx=0).stop 2023-10-06T04:02:19.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.416+0000 7fb142213700 1 -- 172.21.15.157:0/915638723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb13c06cbb0 msgr2=0x7fb13c1478b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:19.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.416+0000 7fb142213700 1 --2- 172.21.15.157:0/915638723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb13c06cbb0 0x7fb13c1478b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb13c0c3960 tx=0x7fb124016000 comp rx=0 tx=0).stop 2023-10-06T04:02:19.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.416+0000 7fb142213700 1 -- 172.21.15.157:0/915638723 shutdown_connections 2023-10-06T04:02:19.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.416+0000 7fb142213700 1 --2- 172.21.15.157:0/915638723 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb11c066f30 0x7fb11c0693e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:19.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.416+0000 7fb142213700 1 --2- 172.21.15.157:0/915638723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb13c06c1f0 0x7fb13c169c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:19.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.416+0000 7fb142213700 1 --2- 172.21.15.157:0/915638723 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb13c147df0 0x7fb13c0c5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:19.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.416+0000 7fb142213700 1 --2- 172.21.15.157:0/915638723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb13c06cbb0 0x7fb13c1478b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:19.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.416+0000 7fb142213700 1 -- 172.21.15.157:0/915638723 >> 172.21.15.157:0/915638723 conn(0x7fb13c06f230 msgr2=0x7fb13c14e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:19.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.416+0000 7fb142213700 1 -- 172.21.15.157:0/915638723 shutdown_connections 2023-10-06T04:02:19.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:19.416+0000 7fb142213700 1 -- 172.21.15.157:0/915638723 wait complete. 2023-10-06T04:02:19.419 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 39 2023-10-06T04:02:19.429 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:02:21.430 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:02:21.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.557+0000 7fb724279700 1 Processor -- start 2023-10-06T04:02:21.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.557+0000 7fb724279700 1 -- start start 2023-10-06T04:02:21.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.558+0000 7fb724279700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb71c06dc40 0x7fb71c14fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:21.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.558+0000 7fb724279700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb71c06c1f0 0x7fb71c150240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:21.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.558+0000 7fb724279700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb71c0c9780 con 0x7fb71c06cbb0 2023-10-06T04:02:21.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.558+0000 7fb724279700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb71c0c8c40 con 0x7fb71c06dc40 2023-10-06T04:02:21.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.558+0000 7fb724279700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb71c06e0c0 con 0x7fb71c06c1f0 2023-10-06T04:02:21.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.558+0000 7fb721814700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb71c06dc40 0x7fb71c14fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:21.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.558+0000 7fb722015700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb71c06cbb0 0x7fb71c06d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41674/0 (socket says 172.21.15.157:41674) 2023-10-06T04:02:21.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.558+0000 7fb722015700 1 -- 172.21.15.157:0/1099900439 learned_addr learned my addr 172.21.15.157:0/1099900439 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:02:21.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.558+0000 7fb722816700 1 --2- 172.21.15.157:0/1099900439 >> v2:172.21.15.175:3300/0 conn(0x7fb71c06c1f0 0x7fb71c150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:21.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.559+0000 7fb721814700 1 -- 172.21.15.157:0/1099900439 >> v2:172.21.15.175:3300/0 conn(0x7fb71c06c1f0 msgr2=0x7fb71c150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:21.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.559+0000 7fb721814700 1 --2- 172.21.15.157:0/1099900439 >> v2:172.21.15.175:3300/0 conn(0x7fb71c06c1f0 0x7fb71c150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:21.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.559+0000 7fb721814700 1 -- 172.21.15.157:0/1099900439 >> v1:172.21.15.157:6789/0 conn(0x7fb71c06cbb0 legacy=0x7fb71c06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:21.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.559+0000 7fb721814700 1 -- 172.21.15.157:0/1099900439 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb71c0c8e50 con 0x7fb71c06dc40 2023-10-06T04:02:21.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.559+0000 7fb721013700 1 -- 172.21.15.157:0/1099900439 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1340212629 0 0) 0x7fb71c0c9780 con 0x7fb71c06cbb0 2023-10-06T04:02:21.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.559+0000 7fb721814700 1 --2- 172.21.15.157:0/1099900439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb71c06dc40 0x7fb71c14fd00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0c96f0 tx=0x7fb70c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c459b82fab32e620 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:21.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.559+0000 7fb721013700 1 -- 172.21.15.157:0/1099900439 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70c00b740 con 0x7fb71c06dc40 2023-10-06T04:02:21.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.559+0000 7fb721013700 1 -- 172.21.15.157:0/1099900439 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70c00b8a0 con 0x7fb71c06dc40 2023-10-06T04:02:21.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.559+0000 7fb721013700 1 -- 172.21.15.157:0/1099900439 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70c00bc40 con 0x7fb71c06dc40 2023-10-06T04:02:21.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.559+0000 7fb724279700 1 -- 172.21.15.157:0/1099900439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb71c06dc40 msgr2=0x7fb71c14fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:21.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.559+0000 7fb724279700 1 --2- 172.21.15.157:0/1099900439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb71c06dc40 0x7fb71c14fd00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0c96f0 tx=0x7fb70c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:02:21.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.560+0000 7fb724279700 1 -- 172.21.15.157:0/1099900439 shutdown_connections 2023-10-06T04:02:21.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.560+0000 7fb724279700 1 --2- 172.21.15.157:0/1099900439 >> v2:172.21.15.175:3300/0 conn(0x7fb71c06c1f0 0x7fb71c150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:21.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.560+0000 7fb724279700 1 --2- 172.21.15.157:0/1099900439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb71c06dc40 0x7fb71c14fd00 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:21.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.560+0000 7fb724279700 1 -- 172.21.15.157:0/1099900439 >> 172.21.15.157:0/1099900439 conn(0x7fb71c06f230 msgr2=0x7fb71c06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:21.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.560+0000 7fb724279700 1 -- 172.21.15.157:0/1099900439 shutdown_connections 2023-10-06T04:02:21.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.560+0000 7fb724279700 1 -- 172.21.15.157:0/1099900439 wait complete. 2023-10-06T04:02:21.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.561+0000 7fb724279700 1 Processor -- start 2023-10-06T04:02:21.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.561+0000 7fb724279700 1 -- start start 2023-10-06T04:02:21.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.561+0000 7fb724279700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb71c06c1f0 0x7fb71c1533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:21.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.561+0000 7fb724279700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb71c06cbb0 0x7fb71c166b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:21.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.561+0000 7fb724279700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb71c0c3170 0x7fb71c0c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:21.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.561+0000 7fb724279700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c0c4b20 con 0x7fb71c0c3170 2023-10-06T04:02:21.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.561+0000 7fb724279700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c152110 con 0x7fb71c06c1f0 2023-10-06T04:02:21.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.561+0000 7fb724279700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c167760 con 0x7fb71c06cbb0 2023-10-06T04:02:21.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.562+0000 7fb722816700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb71c0c3170 0x7fb71c0c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:21.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.562+0000 7fb722816700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb71c0c3170 0x7fb71c0c4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39532/0 (socket says 172.21.15.157:39532) 2023-10-06T04:02:21.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.562+0000 7fb722816700 1 -- 172.21.15.157:0/2178791769 learned_addr learned my addr 172.21.15.157:0/2178791769 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:21.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.562+0000 7fb722015700 1 --2- 172.21.15.157:0/2178791769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb71c06c1f0 0x7fb71c1533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:21.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.562+0000 7fb721814700 1 --2- 172.21.15.157:0/2178791769 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb71c06cbb0 0x7fb71c166b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:21.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.562+0000 7fb722816700 1 -- 172.21.15.157:0/2178791769 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb71c06cbb0 msgr2=0x7fb71c166b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:21.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.562+0000 7fb722816700 1 --2- 172.21.15.157:0/2178791769 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb71c06cbb0 0x7fb71c166b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:21.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.562+0000 7fb722816700 1 -- 172.21.15.157:0/2178791769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb71c06c1f0 msgr2=0x7fb71c1533c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:21.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.562+0000 7fb722816700 1 --2- 172.21.15.157:0/2178791769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb71c06c1f0 0x7fb71c1533c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:21.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.562+0000 7fb722816700 1 -- 172.21.15.157:0/2178791769 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70c009420 con 0x7fb71c0c3170 2023-10-06T04:02:21.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.562+0000 7fb722816700 1 --2- 172.21.15.157:0/2178791769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb71c0c3170 0x7fb71c0c4e90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb714009fd0 tx=0x7fb714009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:21.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.562+0000 7fb7127fc700 1 -- 172.21.15.157:0/2178791769 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71400ec80 con 0x7fb71c0c3170 2023-10-06T04:02:21.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.562+0000 7fb7127fc700 1 -- 172.21.15.157:0/2178791769 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71400ede0 con 0x7fb71c0c3170 2023-10-06T04:02:21.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.562+0000 7fb724279700 1 -- 172.21.15.157:0/2178791769 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb71c151c80 con 0x7fb71c0c3170 2023-10-06T04:02:21.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.562+0000 7fb7127fc700 1 -- 172.21.15.157:0/2178791769 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb714014670 con 0x7fb71c0c3170 2023-10-06T04:02:21.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.562+0000 7fb724279700 1 -- 172.21.15.157:0/2178791769 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb71c1674a0 con 0x7fb71c0c3170 2023-10-06T04:02:21.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.563+0000 7fb724279700 1 -- 172.21.15.157:0/2178791769 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb71c151e10 con 0x7fb71c0c3170 2023-10-06T04:02:21.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.564+0000 7fb7127fc700 1 -- 172.21.15.157:0/2178791769 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb7140149e0 con 0x7fb71c0c3170 2023-10-06T04:02:21.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.564+0000 7fb7127fc700 1 --2- 172.21.15.157:0/2178791769 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb700066f60 0x7fb700069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:21.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.564+0000 7fb7127fc700 1 -- 172.21.15.157:0/2178791769 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7fb714084e30 con 0x7fb71c0c3170 2023-10-06T04:02:21.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.564+0000 7fb722015700 1 --2- 172.21.15.157:0/2178791769 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb700066f60 0x7fb700069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:21.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.565+0000 7fb722015700 1 --2- 172.21.15.157:0/2178791769 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb700066f60 0x7fb700069410 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb718007fd0 tx=0x7fb718007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:21.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.568+0000 7fb7127fc700 1 -- 172.21.15.157:0/2178791769 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb714051e60 con 0x7fb71c0c3170 2023-10-06T04:02:21.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.795+0000 7fb724279700 1 -- 172.21.15.157:0/2178791769 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb71c0c9130 con 0x7fb71c0c3170 2023-10-06T04:02:21.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.795+0000 7fb7127fc700 1 -- 172.21.15.157:0/2178791769 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+6207 (secure 0 0 0) 0x7fb714055480 con 0x7fb71c0c3170 2023-10-06T04:02:21.797 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:02:21.798 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:02:02.817947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2,3,4],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270,"mds_4":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:active","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"f","rank":4,"incarnation":29,"state":"up:stopping","state_seq":32,"addr":"172.21.15.175:6839/3353503675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":3353503675},{"type":"v1","addr":"172.21.15.175:6839","nonce":3353503675}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:02:21.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.797+0000 7fb724279700 1 -- 172.21.15.157:0/2178791769 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb700066f60 msgr2=0x7fb700069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:21.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.797+0000 7fb724279700 1 --2- 172.21.15.157:0/2178791769 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb700066f60 0x7fb700069410 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb718007fd0 tx=0x7fb718007d30 comp rx=0 tx=0).stop 2023-10-06T04:02:21.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.797+0000 7fb724279700 1 -- 172.21.15.157:0/2178791769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb71c0c3170 msgr2=0x7fb71c0c4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:21.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.797+0000 7fb724279700 1 --2- 172.21.15.157:0/2178791769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb71c0c3170 0x7fb71c0c4e90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb714009fd0 tx=0x7fb714009900 comp rx=0 tx=0).stop 2023-10-06T04:02:21.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.797+0000 7fb724279700 1 -- 172.21.15.157:0/2178791769 shutdown_connections 2023-10-06T04:02:21.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.797+0000 7fb724279700 1 --2- 172.21.15.157:0/2178791769 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb700066f60 0x7fb700069410 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:21.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.797+0000 7fb724279700 1 --2- 172.21.15.157:0/2178791769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb71c06c1f0 0x7fb71c1533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:21.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.797+0000 7fb724279700 1 --2- 172.21.15.157:0/2178791769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb71c0c3170 0x7fb71c0c4e90 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:21.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.797+0000 7fb724279700 1 --2- 172.21.15.157:0/2178791769 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb71c06cbb0 0x7fb71c166b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:21.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.798+0000 7fb724279700 1 -- 172.21.15.157:0/2178791769 >> 172.21.15.157:0/2178791769 conn(0x7fb71c06f230 msgr2=0x7fb71c06d530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:21.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.798+0000 7fb724279700 1 -- 172.21.15.157:0/2178791769 shutdown_connections 2023-10-06T04:02:21.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:21.798+0000 7fb724279700 1 -- 172.21.15.157:0/2178791769 wait complete. 2023-10-06T04:02:21.801 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 39 2023-10-06T04:02:21.811 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:02:22.906 INFO:tasks.ceph.mds.f.smithi175.stdout:starting mds.f at 2023-10-06T04:02:23.813 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:02:23.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.938+0000 7fa925205700 1 Processor -- start 2023-10-06T04:02:23.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.938+0000 7fa925205700 1 -- start start 2023-10-06T04:02:23.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.939+0000 7fa925205700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa920159df0 0x7fa92014fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:23.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.939+0000 7fa925205700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa92006ced0 con 0x7fa9200b2110 2023-10-06T04:02:23.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.939+0000 7fa925205700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9200afb10 con 0x7fa9200b1d40 2023-10-06T04:02:23.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.939+0000 7fa925205700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa92015a270 con 0x7fa920159df0 2023-10-06T04:02:23.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.939+0000 7fa917fff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa9200b2110 0x7fa92006c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:36952/0 (socket says 172.21.15.157:36952) 2023-10-06T04:02:23.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.939+0000 7fa917fff700 1 -- 172.21.15.157:0/3139735275 learned_addr learned my addr 172.21.15.157:0/3139735275 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:02:23.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.940+0000 7fa91f59e700 1 --2- 172.21.15.157:0/3139735275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa920159df0 0x7fa92014fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:23.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.940+0000 7fa91e59c700 1 -- 172.21.15.157:0/3139735275 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 927102916 0 0) 0x7fa92006ced0 con 0x7fa9200b2110 2023-10-06T04:02:23.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.940+0000 7fa91e59c700 1 -- 172.21.15.157:0/3139735275 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa90c003570 con 0x7fa9200b2110 2023-10-06T04:02:23.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.940+0000 7fa91e59c700 1 -- 172.21.15.157:0/3139735275 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4093802538 0 0) 0x7fa90c003570 con 0x7fa9200b2110 2023-10-06T04:02:23.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.940+0000 7fa91e59c700 1 -- 172.21.15.157:0/3139735275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa920159df0 msgr2=0x7fa92014fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:23.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.940+0000 7fa91e59c700 1 --2- 172.21.15.157:0/3139735275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa920159df0 0x7fa92014fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:23.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.940+0000 7fa91e59c700 1 -- 172.21.15.157:0/3139735275 >> v1:172.21.15.175:6789/0 conn(0x7fa9200b1d40 legacy=0x7fa9200af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:23.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.940+0000 7fa91e59c700 1 -- 172.21.15.157:0/3139735275 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa92006cc10 con 0x7fa9200b2110 2023-10-06T04:02:23.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.940+0000 7fa91e59c700 1 -- 172.21.15.157:0/3139735275 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7fa908001470 con 0x7fa9200b2110 2023-10-06T04:02:23.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.940+0000 7fa91e59c700 1 -- 172.21.15.157:0/3139735275 >> v1:172.21.15.157:6789/0 conn(0x7fa9200b2110 legacy=0x7fa92006c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:23.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.940+0000 7fa91e59c700 1 --2- 172.21.15.157:0/3139735275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa90c003a90 0x7fa90c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:23.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.940+0000 7fa91e59c700 1 --2- 172.21.15.157:0/3139735275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa90c004810 0x7fa90c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:23.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.940+0000 7fa91e59c700 1 --2- 172.21.15.157:0/3139735275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa920159df0 0x7fa90c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:23.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.940+0000 7fa91e59c700 1 -- 172.21.15.157:0/3139735275 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92015a270 con 0x7fa90c003a90 2023-10-06T04:02:23.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.940+0000 7fa91e59c700 1 -- 172.21.15.157:0/3139735275 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90c00bb10 con 0x7fa90c004810 2023-10-06T04:02:23.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.940+0000 7fa91e59c700 1 -- 172.21.15.157:0/3139735275 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa90c00bcd0 con 0x7fa920159df0 2023-10-06T04:02:23.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.940+0000 7fa91ed9d700 1 --2- 172.21.15.157:0/3139735275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa90c003a90 0x7fa90c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:23.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.940+0000 7fa917fff700 1 --2- 172.21.15.157:0/3139735275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa90c004810 0x7fa90c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:23.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.940+0000 7fa91f59e700 1 --2- 172.21.15.157:0/3139735275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa920159df0 0x7fa90c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:23.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.940+0000 7fa91ed9d700 1 -- 172.21.15.157:0/3139735275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa920159df0 msgr2=0x7fa90c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:23.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.940+0000 7fa91ed9d700 1 --2- 172.21.15.157:0/3139735275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa920159df0 0x7fa90c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:23.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.940+0000 7fa91ed9d700 1 -- 172.21.15.157:0/3139735275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa90c004810 msgr2=0x7fa90c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:23.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.940+0000 7fa91ed9d700 1 --2- 172.21.15.157:0/3139735275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa90c004810 0x7fa90c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:23.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.940+0000 7fa91ed9d700 1 -- 172.21.15.157:0/3139735275 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90c003570 con 0x7fa90c003a90 2023-10-06T04:02:23.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.941+0000 7fa91ed9d700 1 --2- 172.21.15.157:0/3139735275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa90c003a90 0x7fa90c003e60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa9100019a0 tx=0x7fa910007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb98078b05d11ccb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:23.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.941+0000 7fa91e59c700 1 -- 172.21.15.157:0/3139735275 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91000c5d0 con 0x7fa90c003a90 2023-10-06T04:02:23.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.941+0000 7fa91e59c700 1 -- 172.21.15.157:0/3139735275 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91000c730 con 0x7fa90c003a90 2023-10-06T04:02:23.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.941+0000 7fa91e59c700 1 -- 172.21.15.157:0/3139735275 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91000cad0 con 0x7fa90c003a90 2023-10-06T04:02:23.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.941+0000 7fa925205700 1 -- 172.21.15.157:0/3139735275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa90c003a90 msgr2=0x7fa90c003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:23.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.941+0000 7fa925205700 1 --2- 172.21.15.157:0/3139735275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa90c003a90 0x7fa90c003e60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa9100019a0 tx=0x7fa910007d10 comp rx=0 tx=0).stop 2023-10-06T04:02:23.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.941+0000 7fa91f59e700 1 -- 172.21.15.157:0/3139735275 reap_dead start 2023-10-06T04:02:23.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.941+0000 7fa925205700 1 -- 172.21.15.157:0/3139735275 shutdown_connections 2023-10-06T04:02:23.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.941+0000 7fa925205700 1 -- 172.21.15.157:0/3139735275 >> 172.21.15.157:0/3139735275 conn(0x7fa9200cc6e0 msgr2=0x7fa9200ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:23.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.941+0000 7fa925205700 1 -- 172.21.15.157:0/3139735275 shutdown_connections 2023-10-06T04:02:23.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.941+0000 7fa925205700 1 -- 172.21.15.157:0/3139735275 wait complete. 2023-10-06T04:02:23.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.942+0000 7fa925205700 1 Processor -- start 2023-10-06T04:02:23.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.942+0000 7fa925205700 1 -- start start 2023-10-06T04:02:23.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.943+0000 7fa925205700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa90c003a90 0x7fa920166590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:23.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.943+0000 7fa925205700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa92015b8c0 0x7fa920166fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:23.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.943+0000 7fa925205700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa920169c90 0x7fa92014c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:23.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.943+0000 7fa925205700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa920167500 con 0x7fa90c003a90 2023-10-06T04:02:23.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.943+0000 7fa925205700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa920169800 con 0x7fa920169c90 2023-10-06T04:02:23.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.943+0000 7fa925205700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa92015c330 con 0x7fa92015b8c0 2023-10-06T04:02:23.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.943+0000 7fa91ed9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa90c003a90 0x7fa920166590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:23.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.943+0000 7fa91ed9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa90c003a90 0x7fa920166590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51956/0 (socket says 172.21.15.157:51956) 2023-10-06T04:02:23.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.943+0000 7fa91ed9d700 1 -- 172.21.15.157:0/3299004050 learned_addr learned my addr 172.21.15.157:0/3299004050 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:23.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.943+0000 7fa91f59e700 1 --2- 172.21.15.157:0/3299004050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa920169c90 0x7fa92014c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:23.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.943+0000 7fa917fff700 1 --2- 172.21.15.157:0/3299004050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa92015b8c0 0x7fa920166fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:23.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.944+0000 7fa91ed9d700 1 -- 172.21.15.157:0/3299004050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa92015b8c0 msgr2=0x7fa920166fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:23.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.944+0000 7fa91ed9d700 1 --2- 172.21.15.157:0/3299004050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa92015b8c0 0x7fa920166fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:23.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.944+0000 7fa91ed9d700 1 -- 172.21.15.157:0/3299004050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa920169c90 msgr2=0x7fa92014c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:23.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.944+0000 7fa91ed9d700 1 --2- 172.21.15.157:0/3299004050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa920169c90 0x7fa92014c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:23.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.944+0000 7fa91ed9d700 1 -- 172.21.15.157:0/3299004050 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa920157120 con 0x7fa90c003a90 2023-10-06T04:02:23.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.944+0000 7fa91ed9d700 1 --2- 172.21.15.157:0/3299004050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa90c003a90 0x7fa920166590 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa90c004770 tx=0x7fa910015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:23.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.944+0000 7fa916ffd700 1 -- 172.21.15.157:0/3299004050 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91000c5d0 con 0x7fa90c003a90 2023-10-06T04:02:23.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.944+0000 7fa916ffd700 1 -- 172.21.15.157:0/3299004050 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91000ce40 con 0x7fa90c003a90 2023-10-06T04:02:23.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.944+0000 7fa925205700 1 -- 172.21.15.157:0/3299004050 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9201691f0 con 0x7fa90c003a90 2023-10-06T04:02:23.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.944+0000 7fa916ffd700 1 -- 172.21.15.157:0/3299004050 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa910007af0 con 0x7fa90c003a90 2023-10-06T04:02:23.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.944+0000 7fa925205700 1 -- 172.21.15.157:0/3299004050 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92015c050 con 0x7fa90c003a90 2023-10-06T04:02:23.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.945+0000 7fa925205700 1 -- 172.21.15.157:0/3299004050 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9200896c0 con 0x7fa90c003a90 2023-10-06T04:02:23.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.945+0000 7fa916ffd700 1 -- 172.21.15.157:0/3299004050 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa910019030 con 0x7fa90c003a90 2023-10-06T04:02:23.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.946+0000 7fa916ffd700 1 --2- 172.21.15.157:0/3299004050 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa900066f60 0x7fa900069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:23.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.946+0000 7fa916ffd700 1 -- 172.21.15.157:0/3299004050 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7fa910085d50 con 0x7fa90c003a90 2023-10-06T04:02:23.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.946+0000 7fa917fff700 1 --2- 172.21.15.157:0/3299004050 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa900066f60 0x7fa900069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:23.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.946+0000 7fa917fff700 1 --2- 172.21.15.157:0/3299004050 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa900066f60 0x7fa900069410 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa92006c740 tx=0x7fa908001e70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:23.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:23.950+0000 7fa916ffd700 1 -- 172.21.15.157:0/3299004050 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa910089030 con 0x7fa90c003a90 2023-10-06T04:02:24.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:24.175+0000 7fa925205700 1 -- 172.21.15.157:0/3299004050 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa92004bb40 con 0x7fa90c003a90 2023-10-06T04:02:24.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:24.176+0000 7fa916ffd700 1 -- 172.21.15.157:0/3299004050 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+6177 (secure 0 0 0) 0x7fa910017090 con 0x7fa90c003a90 2023-10-06T04:02:24.178 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:02:24.178 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:02:22.870092+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2,3],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270},"failed":[],"damaged":[],"stopped":[4],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:stopping","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:02:24.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:24.178+0000 7fa925205700 1 -- 172.21.15.157:0/3299004050 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa900066f60 msgr2=0x7fa900069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:24.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:24.178+0000 7fa925205700 1 --2- 172.21.15.157:0/3299004050 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa900066f60 0x7fa900069410 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa92006c740 tx=0x7fa908001e70 comp rx=0 tx=0).stop 2023-10-06T04:02:24.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:24.178+0000 7fa925205700 1 -- 172.21.15.157:0/3299004050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa90c003a90 msgr2=0x7fa920166590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:24.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:24.178+0000 7fa925205700 1 --2- 172.21.15.157:0/3299004050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa90c003a90 0x7fa920166590 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa90c004770 tx=0x7fa910015000 comp rx=0 tx=0).stop 2023-10-06T04:02:24.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:24.178+0000 7fa925205700 1 -- 172.21.15.157:0/3299004050 shutdown_connections 2023-10-06T04:02:24.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:24.178+0000 7fa925205700 1 --2- 172.21.15.157:0/3299004050 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa900066f60 0x7fa900069410 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:24.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:24.178+0000 7fa925205700 1 --2- 172.21.15.157:0/3299004050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa920169c90 0x7fa92014c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:24.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:24.178+0000 7fa925205700 1 --2- 172.21.15.157:0/3299004050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa90c003a90 0x7fa920166590 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:24.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:24.178+0000 7fa925205700 1 --2- 172.21.15.157:0/3299004050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa92015b8c0 0x7fa920166fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:24.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:24.178+0000 7fa925205700 1 -- 172.21.15.157:0/3299004050 >> 172.21.15.157:0/3299004050 conn(0x7fa9200cc6e0 msgr2=0x7fa92014cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:24.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:24.178+0000 7fa925205700 1 -- 172.21.15.157:0/3299004050 shutdown_connections 2023-10-06T04:02:24.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:24.178+0000 7fa925205700 1 -- 172.21.15.157:0/3299004050 wait complete. 2023-10-06T04:02:24.181 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 42 2023-10-06T04:02:24.192 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:02:26.193 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:02:26.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.317+0000 7f087a0e3700 1 Processor -- start 2023-10-06T04:02:26.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.318+0000 7f087a0e3700 1 -- start start 2023-10-06T04:02:26.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.318+0000 7f087a0e3700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f087406dc40 0x7f087414fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:26.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.318+0000 7f087a0e3700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08740c9780 con 0x7f087406cbb0 2023-10-06T04:02:26.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.318+0000 7f087a0e3700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f087406e0c0 con 0x7f087406c1f0 2023-10-06T04:02:26.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.318+0000 7f087a0e3700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08740c8c40 con 0x7f087406dc40 2023-10-06T04:02:26.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.318+0000 7f08737fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f087406cbb0 0x7f087406d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:36964/0 (socket says 172.21.15.157:36964) 2023-10-06T04:02:26.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.318+0000 7f08737fe700 1 -- 172.21.15.157:0/1925387643 learned_addr learned my addr 172.21.15.157:0/1925387643 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:02:26.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.319+0000 7f0872ffd700 1 --2- 172.21.15.157:0/1925387643 >> v2:172.21.15.175:3300/0 conn(0x7f087406dc40 0x7f087414fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:26.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.319+0000 7f08727fc700 1 -- 172.21.15.157:0/1925387643 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4250826426 0 0) 0x7f08740c9780 con 0x7f087406cbb0 2023-10-06T04:02:26.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.319+0000 7f08727fc700 1 -- 172.21.15.157:0/1925387643 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f085c003570 con 0x7f087406cbb0 2023-10-06T04:02:26.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.319+0000 7f08727fc700 1 -- 172.21.15.157:0/1925387643 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1618615012 0 0) 0x7f085c003570 con 0x7f087406cbb0 2023-10-06T04:02:26.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.319+0000 7f08727fc700 1 -- 172.21.15.157:0/1925387643 >> v2:172.21.15.175:3300/0 conn(0x7f087406dc40 msgr2=0x7f087414fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:26.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.319+0000 7f08727fc700 1 --2- 172.21.15.157:0/1925387643 >> v2:172.21.15.175:3300/0 conn(0x7f087406dc40 0x7f087414fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:26.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.319+0000 7f08727fc700 1 -- 172.21.15.157:0/1925387643 >> v1:172.21.15.175:6789/0 conn(0x7f087406c1f0 legacy=0x7f0874155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:26.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.319+0000 7f08727fc700 1 -- 172.21.15.157:0/1925387643 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08740c8e50 con 0x7f087406cbb0 2023-10-06T04:02:26.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.319+0000 7f08727fc700 1 -- 172.21.15.157:0/1925387643 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f0864001470 con 0x7f087406cbb0 2023-10-06T04:02:26.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.319+0000 7f08727fc700 1 -- 172.21.15.157:0/1925387643 >> v1:172.21.15.157:6789/0 conn(0x7f087406cbb0 legacy=0x7f087406d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:26.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.319+0000 7f08727fc700 1 --2- 172.21.15.157:0/1925387643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f085c003a60 0x7f085c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:26.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.319+0000 7f08727fc700 1 --2- 172.21.15.157:0/1925387643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f085c004810 0x7f085c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:26.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.319+0000 7f08727fc700 1 --2- 172.21.15.157:0/1925387643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f085c0051c0 0x7f085c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:26.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.319+0000 7f08727fc700 1 -- 172.21.15.157:0/1925387643 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08740c8c40 con 0x7f085c0051c0 2023-10-06T04:02:26.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.319+0000 7f08727fc700 1 -- 172.21.15.157:0/1925387643 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085c00c0d0 con 0x7f085c004810 2023-10-06T04:02:26.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.319+0000 7f08727fc700 1 -- 172.21.15.157:0/1925387643 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f085c00c290 con 0x7f085c003a60 2023-10-06T04:02:26.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.319+0000 7f0873fff700 1 --2- 172.21.15.157:0/1925387643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f085c0051c0 0x7f085c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:26.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.319+0000 7f0872ffd700 1 --2- 172.21.15.157:0/1925387643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f085c004810 0x7f085c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:26.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.319+0000 7f08737fe700 1 --2- 172.21.15.157:0/1925387643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f085c003a60 0x7f085c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:26.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.319+0000 7f0873fff700 1 -- 172.21.15.157:0/1925387643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f085c003a60 msgr2=0x7f085c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:26.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.320+0000 7f0873fff700 1 --2- 172.21.15.157:0/1925387643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f085c003a60 0x7f085c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:26.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.320+0000 7f0873fff700 1 -- 172.21.15.157:0/1925387643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f085c004810 msgr2=0x7f085c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:26.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.320+0000 7f0873fff700 1 --2- 172.21.15.157:0/1925387643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f085c004810 0x7f085c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:26.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.320+0000 7f0873fff700 1 -- 172.21.15.157:0/1925387643 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f085c003570 con 0x7f085c0051c0 2023-10-06T04:02:26.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.320+0000 7f0873fff700 1 -- 172.21.15.157:0/1925387643 reap_dead start 2023-10-06T04:02:26.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.320+0000 7f0873fff700 1 --2- 172.21.15.157:0/1925387643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f085c0051c0 0x7f085c00ba60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f086c001fc0 tx=0x7f086c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872103d57483b25e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:26.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.320+0000 7f08727fc700 1 -- 172.21.15.157:0/1925387643 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c001950 con 0x7f085c0051c0 2023-10-06T04:02:26.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.320+0000 7f08727fc700 1 -- 172.21.15.157:0/1925387643 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086c00c8b0 con 0x7f085c0051c0 2023-10-06T04:02:26.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.320+0000 7f08727fc700 1 -- 172.21.15.157:0/1925387643 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c00cc90 con 0x7f085c0051c0 2023-10-06T04:02:26.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.320+0000 7f087a0e3700 1 -- 172.21.15.157:0/1925387643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f085c0051c0 msgr2=0x7f085c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:26.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.320+0000 7f087a0e3700 1 --2- 172.21.15.157:0/1925387643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f085c0051c0 0x7f085c00ba60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f086c001fc0 tx=0x7f086c001640 comp rx=0 tx=0).stop 2023-10-06T04:02:26.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.320+0000 7f087a0e3700 1 -- 172.21.15.157:0/1925387643 shutdown_connections 2023-10-06T04:02:26.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.320+0000 7f087a0e3700 1 --2- 172.21.15.157:0/1925387643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f085c0051c0 0x7f085c00ba60 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:26.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.320+0000 7f087a0e3700 1 -- 172.21.15.157:0/1925387643 >> 172.21.15.157:0/1925387643 conn(0x7f087406f230 msgr2=0x7f087406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:26.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.320+0000 7f087a0e3700 1 -- 172.21.15.157:0/1925387643 shutdown_connections 2023-10-06T04:02:26.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.321+0000 7f087a0e3700 1 -- 172.21.15.157:0/1925387643 wait complete. 2023-10-06T04:02:26.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.322+0000 7f087a0e3700 1 Processor -- start 2023-10-06T04:02:26.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.322+0000 7f087a0e3700 1 -- start start 2023-10-06T04:02:26.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.322+0000 7f087a0e3700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f085c0051c0 0x7f08740cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:26.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.322+0000 7f087a0e3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f087416a8c0 0x7f0874152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:26.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.322+0000 7f087a0e3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08740c2610 0x7f0874152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:26.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.322+0000 7f087a0e3700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087416adf0 con 0x7f087416a8c0 2023-10-06T04:02:26.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.322+0000 7f087a0e3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08740c1980 con 0x7f08740c2610 2023-10-06T04:02:26.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.322+0000 7f087a0e3700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0874145c80 con 0x7f085c0051c0 2023-10-06T04:02:26.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.323+0000 7f0872ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f087416a8c0 0x7f0874152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:26.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.323+0000 7f0872ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f087416a8c0 0x7f0874152410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51972/0 (socket says 172.21.15.157:51972) 2023-10-06T04:02:26.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.323+0000 7f0872ffd700 1 -- 172.21.15.157:0/3925900688 learned_addr learned my addr 172.21.15.157:0/3925900688 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:26.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.323+0000 7f0873fff700 1 --2- 172.21.15.157:0/3925900688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08740c2610 0x7f0874152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:26.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.323+0000 7f0872ffd700 1 -- 172.21.15.157:0/3925900688 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f085c0051c0 msgr2=0x7f08740cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:26.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.323+0000 7f0872ffd700 1 --2- 172.21.15.157:0/3925900688 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f085c0051c0 0x7f08740cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:26.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.323+0000 7f0872ffd700 1 -- 172.21.15.157:0/3925900688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08740c2610 msgr2=0x7f0874152950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:26.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.323+0000 7f0872ffd700 1 --2- 172.21.15.157:0/3925900688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08740c2610 0x7f0874152950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:26.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.323+0000 7f0872ffd700 1 -- 172.21.15.157:0/3925900688 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08740c0d70 con 0x7f087416a8c0 2023-10-06T04:02:26.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.323+0000 7f0873fff700 1 --2- 172.21.15.157:0/3925900688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08740c2610 0x7f0874152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:02:26.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.323+0000 7f0872ffd700 1 --2- 172.21.15.157:0/3925900688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f087416a8c0 0x7f0874152410 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0868011b10 tx=0x7f08680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:26.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.323+0000 7f085b7fe700 1 -- 172.21.15.157:0/3925900688 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086800f990 con 0x7f087416a8c0 2023-10-06T04:02:26.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.323+0000 7f085b7fe700 1 -- 172.21.15.157:0/3925900688 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086800faf0 con 0x7f087416a8c0 2023-10-06T04:02:26.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.323+0000 7f087a0e3700 1 -- 172.21.15.157:0/3925900688 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08740c1050 con 0x7f087416a8c0 2023-10-06T04:02:26.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.323+0000 7f085b7fe700 1 -- 172.21.15.157:0/3925900688 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086800fe70 con 0x7f087416a8c0 2023-10-06T04:02:26.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.323+0000 7f087a0e3700 1 -- 172.21.15.157:0/3925900688 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08740c1610 con 0x7f087416a8c0 2023-10-06T04:02:26.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.324+0000 7f087a0e3700 1 -- 172.21.15.157:0/3925900688 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08740c17d0 con 0x7f087416a8c0 2023-10-06T04:02:26.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.324+0000 7f085b7fe700 1 -- 172.21.15.157:0/3925900688 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0868014650 con 0x7f087416a8c0 2023-10-06T04:02:26.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.325+0000 7f085b7fe700 1 --2- 172.21.15.157:0/3925900688 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0860066f30 0x7f08600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:26.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.325+0000 7f085b7fe700 1 -- 172.21.15.157:0/3925900688 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f0868018030 con 0x7f087416a8c0 2023-10-06T04:02:26.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.325+0000 7f08737fe700 1 --2- 172.21.15.157:0/3925900688 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0860066f30 0x7f08600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:26.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.326+0000 7f08737fe700 1 --2- 172.21.15.157:0/3925900688 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0860066f30 0x7f08600693e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0864000e10 tx=0x7f0864001e50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:26.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.329+0000 7f085b7fe700 1 -- 172.21.15.157:0/3925900688 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0868055c30 con 0x7f087416a8c0 2023-10-06T04:02:26.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.553+0000 7f087a0e3700 1 -- 172.21.15.157:0/3925900688 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08740c1ac0 con 0x7f087416a8c0 2023-10-06T04:02:26.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.553+0000 7f085b7fe700 1 -- 172.21.15.157:0/3925900688 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+6177 (secure 0 0 0) 0x7f08680557c0 con 0x7f087416a8c0 2023-10-06T04:02:26.555 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:02:26.555 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:02:22.870092+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2,3],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270},"failed":[],"damaged":[],"stopped":[4],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:stopping","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:02:26.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.556+0000 7f087a0e3700 1 -- 172.21.15.157:0/3925900688 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0860066f30 msgr2=0x7f08600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:26.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.556+0000 7f087a0e3700 1 --2- 172.21.15.157:0/3925900688 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0860066f30 0x7f08600693e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0864000e10 tx=0x7f0864001e50 comp rx=0 tx=0).stop 2023-10-06T04:02:26.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.556+0000 7f087a0e3700 1 -- 172.21.15.157:0/3925900688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f087416a8c0 msgr2=0x7f0874152410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:26.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.556+0000 7f087a0e3700 1 --2- 172.21.15.157:0/3925900688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f087416a8c0 0x7f0874152410 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0868011b10 tx=0x7f08680091a0 comp rx=0 tx=0).stop 2023-10-06T04:02:26.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.556+0000 7f087a0e3700 1 -- 172.21.15.157:0/3925900688 shutdown_connections 2023-10-06T04:02:26.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.556+0000 7f087a0e3700 1 --2- 172.21.15.157:0/3925900688 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0860066f30 0x7f08600693e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:26.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.556+0000 7f087a0e3700 1 --2- 172.21.15.157:0/3925900688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08740c2610 0x7f0874152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:26.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.556+0000 7f087a0e3700 1 --2- 172.21.15.157:0/3925900688 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f085c0051c0 0x7f08740cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:26.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.556+0000 7f087a0e3700 1 --2- 172.21.15.157:0/3925900688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f087416a8c0 0x7f0874152410 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:26.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.556+0000 7f087a0e3700 1 -- 172.21.15.157:0/3925900688 >> 172.21.15.157:0/3925900688 conn(0x7f087406f230 msgr2=0x7f087414fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:26.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.556+0000 7f087a0e3700 1 -- 172.21.15.157:0/3925900688 shutdown_connections 2023-10-06T04:02:26.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:26.556+0000 7f087a0e3700 1 -- 172.21.15.157:0/3925900688 wait complete. 2023-10-06T04:02:26.559 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 42 2023-10-06T04:02:26.569 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:02:28.570 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:02:28.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.683+0000 7f65b4b61700 1 Processor -- start 2023-10-06T04:02:28.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.683+0000 7f65b4b61700 1 -- start start 2023-10-06T04:02:28.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.684+0000 7f65b4b61700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f65b0068920 0x7f65b0156840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:28.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.684+0000 7f65b4b61700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f65b00c52e0 0x7f65b0156d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:28.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.684+0000 7f65b4b61700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65b006d530 con 0x7f65b00ca630 2023-10-06T04:02:28.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.684+0000 7f65b4b61700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65b006d9c0 con 0x7f65b00c52e0 2023-10-06T04:02:28.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.684+0000 7f65b4b61700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65b006f0c0 con 0x7f65b0068920 2023-10-06T04:02:28.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.684+0000 7f65aed9d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f65b00c52e0 0x7f65b0156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:28.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.684+0000 7f65ae59c700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f65b00ca630 0x7f65b00c6ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:36972/0 (socket says 172.21.15.157:36972) 2023-10-06T04:02:28.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.684+0000 7f65ae59c700 1 -- 172.21.15.157:0/2665168266 learned_addr learned my addr 172.21.15.157:0/2665168266 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:02:28.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.685+0000 7f65a7fff700 1 --2- 172.21.15.157:0/2665168266 >> v2:172.21.15.175:3300/0 conn(0x7f65b0068920 0x7f65b0156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:28.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.685+0000 7f65aed9d700 1 -- 172.21.15.157:0/2665168266 >> v2:172.21.15.175:3300/0 conn(0x7f65b0068920 msgr2=0x7f65b0156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:28.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.685+0000 7f65aed9d700 1 --2- 172.21.15.157:0/2665168266 >> v2:172.21.15.175:3300/0 conn(0x7f65b0068920 0x7f65b0156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:28.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.685+0000 7f65aed9d700 1 -- 172.21.15.157:0/2665168266 >> v1:172.21.15.157:6789/0 conn(0x7f65b00ca630 legacy=0x7f65b00c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:28.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.685+0000 7f65aed9d700 1 -- 172.21.15.157:0/2665168266 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b006f3a0 con 0x7f65b00c52e0 2023-10-06T04:02:28.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.685+0000 7f65add9b700 1 -- 172.21.15.157:0/2665168266 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 824278026 0 0) 0x7f65b006d530 con 0x7f65b00ca630 2023-10-06T04:02:28.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.685+0000 7f65aed9d700 1 --2- 172.21.15.157:0/2665168266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65b00c52e0 0x7f65b0156d80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f65b00ca440 tx=0x7f65a8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fedc8ff90972222f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:28.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.685+0000 7f65add9b700 1 -- 172.21.15.157:0/2665168266 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a800e070 con 0x7f65b00c52e0 2023-10-06T04:02:28.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.685+0000 7f65add9b700 1 -- 172.21.15.157:0/2665168266 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a800bc30 con 0x7f65b00c52e0 2023-10-06T04:02:28.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.685+0000 7f65add9b700 1 -- 172.21.15.157:0/2665168266 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a8012430 con 0x7f65b00c52e0 2023-10-06T04:02:28.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.685+0000 7f65b4b61700 1 -- 172.21.15.157:0/2665168266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65b00c52e0 msgr2=0x7f65b0156d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:28.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.685+0000 7f65b4b61700 1 --2- 172.21.15.157:0/2665168266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65b00c52e0 0x7f65b0156d80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f65b00ca440 tx=0x7f65a8009780 comp rx=0 tx=0).stop 2023-10-06T04:02:28.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.686+0000 7f65b4b61700 1 -- 172.21.15.157:0/2665168266 shutdown_connections 2023-10-06T04:02:28.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.686+0000 7f65b4b61700 1 --2- 172.21.15.157:0/2665168266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65b00c52e0 0x7f65b0156d80 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:28.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.686+0000 7f65b4b61700 1 --2- 172.21.15.157:0/2665168266 >> v2:172.21.15.175:3300/0 conn(0x7f65b0068920 0x7f65b0156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:28.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.686+0000 7f65b4b61700 1 -- 172.21.15.157:0/2665168266 >> 172.21.15.157:0/2665168266 conn(0x7f65b00692b0 msgr2=0x7f65b00696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:28.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.686+0000 7f65b4b61700 1 -- 172.21.15.157:0/2665168266 shutdown_connections 2023-10-06T04:02:28.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.686+0000 7f65b4b61700 1 -- 172.21.15.157:0/2665168266 wait complete. 2023-10-06T04:02:28.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.687+0000 7f65b4b61700 1 Processor -- start 2023-10-06T04:02:28.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.687+0000 7f65b4b61700 1 -- start start 2023-10-06T04:02:28.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.687+0000 7f65b4b61700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65b0068920 0x7f65b0150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:28.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.687+0000 7f65b4b61700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65b00c52e0 0x7f65b00c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:28.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.687+0000 7f65b4b61700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65b00ca630 0x7f65b0151100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:28.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.687+0000 7f65b4b61700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b0168cb0 con 0x7f65b0068920 2023-10-06T04:02:28.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.687+0000 7f65b4b61700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b0150ca0 con 0x7f65b00ca630 2023-10-06T04:02:28.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.687+0000 7f65b4b61700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65b01502d0 con 0x7f65b00c52e0 2023-10-06T04:02:28.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.687+0000 7f65ae59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65b0068920 0x7f65b0150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:28.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.687+0000 7f65ae59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65b0068920 0x7f65b0150760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51980/0 (socket says 172.21.15.157:51980) 2023-10-06T04:02:28.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.687+0000 7f65ae59c700 1 -- 172.21.15.157:0/49927874 learned_addr learned my addr 172.21.15.157:0/49927874 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:28.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.688+0000 7f65aed9d700 1 --2- 172.21.15.157:0/49927874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65b00ca630 0x7f65b0151100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:28.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.688+0000 7f65a7fff700 1 --2- 172.21.15.157:0/49927874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65b00c52e0 0x7f65b00c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:28.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.688+0000 7f65ae59c700 1 -- 172.21.15.157:0/49927874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65b00c52e0 msgr2=0x7f65b00c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:28.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.688+0000 7f65ae59c700 1 --2- 172.21.15.157:0/49927874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65b00c52e0 0x7f65b00c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:28.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.688+0000 7f65ae59c700 1 -- 172.21.15.157:0/49927874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65b00ca630 msgr2=0x7f65b0151100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:28.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.688+0000 7f65ae59c700 1 --2- 172.21.15.157:0/49927874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65b00ca630 0x7f65b0151100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:28.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.688+0000 7f65ae59c700 1 -- 172.21.15.157:0/49927874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a8009a20 con 0x7f65b0068920 2023-10-06T04:02:28.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.688+0000 7f65aed9d700 1 --2- 172.21.15.157:0/49927874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65b00ca630 0x7f65b0151100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:02:28.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.688+0000 7f65a7fff700 1 --2- 172.21.15.157:0/49927874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65b00c52e0 0x7f65b00c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:02:28.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.688+0000 7f65ae59c700 1 --2- 172.21.15.157:0/49927874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65b0068920 0x7f65b0150760 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f65a000c810 tx=0x7f65a0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:28.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.688+0000 7f65a67fc700 1 -- 172.21.15.157:0/49927874 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a0005b30 con 0x7f65b0068920 2023-10-06T04:02:28.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.688+0000 7f65a67fc700 1 -- 172.21.15.157:0/49927874 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a0005c90 con 0x7f65b0068920 2023-10-06T04:02:28.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.688+0000 7f65a67fc700 1 -- 172.21.15.157:0/49927874 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a0007300 con 0x7f65b0068920 2023-10-06T04:02:28.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.688+0000 7f65b4b61700 1 -- 172.21.15.157:0/49927874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65b014fca0 con 0x7f65b0068920 2023-10-06T04:02:28.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.688+0000 7f65b4b61700 1 -- 172.21.15.157:0/49927874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65b0168ec0 con 0x7f65b0068920 2023-10-06T04:02:28.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.689+0000 7f65b4b61700 1 -- 172.21.15.157:0/49927874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65b006a8f0 con 0x7f65b0068920 2023-10-06T04:02:28.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.689+0000 7f65a67fc700 1 -- 172.21.15.157:0/49927874 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f65a001b400 con 0x7f65b0068920 2023-10-06T04:02:28.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.690+0000 7f65a67fc700 1 --2- 172.21.15.157:0/49927874 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6590066f60 0x7f6590069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:28.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.690+0000 7f65a67fc700 1 -- 172.21.15.157:0/49927874 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f65a0084290 con 0x7f65b0068920 2023-10-06T04:02:28.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.690+0000 7f65a7fff700 1 --2- 172.21.15.157:0/49927874 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6590066f60 0x7f6590069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:28.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.691+0000 7f65a7fff700 1 --2- 172.21.15.157:0/49927874 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6590066f60 0x7f6590069410 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f65b014c340 tx=0x7f6598008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:28.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.694+0000 7f65a67fc700 1 -- 172.21.15.157:0/49927874 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f65a0051340 con 0x7f65b0068920 2023-10-06T04:02:28.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.934+0000 7f65b4b61700 1 -- 172.21.15.157:0/49927874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65b004bb40 con 0x7f65b0068920 2023-10-06T04:02:28.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.934+0000 7f65a67fc700 1 -- 172.21.15.157:0/49927874 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+6177 (secure 0 0 0) 0x7f65a0054960 con 0x7f65b0068920 2023-10-06T04:02:28.936 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:02:28.936 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:02:22.870092+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2,3],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270},"failed":[],"damaged":[],"stopped":[4],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:stopping","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:02:28.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.937+0000 7f65b4b61700 1 -- 172.21.15.157:0/49927874 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6590066f60 msgr2=0x7f6590069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:28.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.937+0000 7f65b4b61700 1 --2- 172.21.15.157:0/49927874 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6590066f60 0x7f6590069410 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f65b014c340 tx=0x7f6598008040 comp rx=0 tx=0).stop 2023-10-06T04:02:28.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.937+0000 7f65b4b61700 1 -- 172.21.15.157:0/49927874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65b0068920 msgr2=0x7f65b0150760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:28.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.937+0000 7f65b4b61700 1 --2- 172.21.15.157:0/49927874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65b0068920 0x7f65b0150760 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f65a000c810 tx=0x7f65a0007df0 comp rx=0 tx=0).stop 2023-10-06T04:02:28.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.937+0000 7f65b4b61700 1 -- 172.21.15.157:0/49927874 shutdown_connections 2023-10-06T04:02:28.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.937+0000 7f65b4b61700 1 --2- 172.21.15.157:0/49927874 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6590066f60 0x7f6590069410 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:28.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.937+0000 7f65b4b61700 1 --2- 172.21.15.157:0/49927874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65b00ca630 0x7f65b0151100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:28.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.937+0000 7f65b4b61700 1 --2- 172.21.15.157:0/49927874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65b0068920 0x7f65b0150760 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:28.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.937+0000 7f65b4b61700 1 --2- 172.21.15.157:0/49927874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65b00c52e0 0x7f65b00c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:28.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.937+0000 7f65b4b61700 1 -- 172.21.15.157:0/49927874 >> 172.21.15.157:0/49927874 conn(0x7f65b00692b0 msgr2=0x7f65b006e760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:28.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.937+0000 7f65b4b61700 1 -- 172.21.15.157:0/49927874 shutdown_connections 2023-10-06T04:02:28.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:28.937+0000 7f65b4b61700 1 -- 172.21.15.157:0/49927874 wait complete. 2023-10-06T04:02:28.940 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 42 2023-10-06T04:02:28.951 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2023-10-06T04:02:28.951 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 42, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-06T03:57:10.718782+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 41, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0, 1, 2, 3 ], "info": { "gid_4270": { "addr": "172.21.15.175:6835/1596042861", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6834", "nonce": 1596042861, "type": "v2" }, { "addr": "172.21.15.175:6835", "nonce": 1596042861, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4270, "incarnation": 13, "join_fscid": -1, "name": "b", "rank": 3, "state": "up:stopping", "state_seq": 8 }, "gid_4279": { "addr": "172.21.15.175:6837/1002404870", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6836", "nonce": 1002404870, "type": "v2" }, { "addr": "172.21.15.175:6837", "nonce": 1002404870, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 3 ], "features": 4540138320759226367, "flags": 0, "gid": 4279, "incarnation": 11, "join_fscid": -1, "name": "d", "rank": 2, "state": "up:active", "state_seq": 7 }, "gid_4287": { "addr": "172.21.15.157:6834/1179753185", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6832", "nonce": 1179753185, "type": "v2" }, { "addr": "172.21.15.157:6834", "nonce": 1179753185, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4287, "incarnation": 9, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4296": { "addr": "172.21.15.157:6837/4188652695", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6836", "nonce": 4188652695, "type": "v2" }, { "addr": "172.21.15.157:6837", "nonce": 4188652695, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2, 4 ], "features": 4540138320759226367, "flags": 0, "gid": 4296, "incarnation": 4, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 27, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2023-10-06T04:02:22.870092+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 4 ], "tableserver": 0, "up": { "mds_0": 4296, "mds_1": 4287, "mds_2": 4279, "mds_3": 4270 } } } ], "standbys": [ { "addr": "172.21.15.175:6839/745804271", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6838", "nonce": 745804271, "type": "v2" }, { "addr": "172.21.15.175:6839", "nonce": 745804271, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 42, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5164, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.157:6835/2461751318", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6833", "nonce": 2461751318, "type": "v2" }, { "addr": "172.21.15.157:6835", "nonce": 2461751318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 34, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5268, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-06T04:02:30.953 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:02:31.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.071+0000 7ff824012700 1 Processor -- start 2023-10-06T04:02:31.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.072+0000 7ff824012700 1 -- start start 2023-10-06T04:02:31.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.072+0000 7ff824012700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff81c150410 0x7ff81c155f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:31.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.072+0000 7ff824012700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff81c0c9780 con 0x7ff81c06dc40 2023-10-06T04:02:31.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.072+0000 7ff824012700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff81c06e0c0 con 0x7ff81c06cbb0 2023-10-06T04:02:31.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.072+0000 7ff824012700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff81c0c8c40 con 0x7ff81c150410 2023-10-06T04:02:31.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.073+0000 7ff8215ad700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff81c06dc40 0x7ff81c14fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:36988/0 (socket says 172.21.15.157:36988) 2023-10-06T04:02:31.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.073+0000 7ff8215ad700 1 -- 172.21.15.157:0/2801833881 learned_addr learned my addr 172.21.15.157:0/2801833881 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:02:31.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.073+0000 7ff8225af700 1 --2- 172.21.15.157:0/2801833881 >> v2:172.21.15.175:3300/0 conn(0x7ff81c150410 0x7ff81c155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:31.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.073+0000 7ff820dac700 1 -- 172.21.15.157:0/2801833881 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 110980074 0 0) 0x7ff81c0c9780 con 0x7ff81c06dc40 2023-10-06T04:02:31.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.073+0000 7ff820dac700 1 -- 172.21.15.157:0/2801833881 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff808003570 con 0x7ff81c06dc40 2023-10-06T04:02:31.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.073+0000 7ff820dac700 1 -- 172.21.15.157:0/2801833881 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3510966530 0 0) 0x7ff808003570 con 0x7ff81c06dc40 2023-10-06T04:02:31.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.073+0000 7ff820dac700 1 -- 172.21.15.157:0/2801833881 >> v2:172.21.15.175:3300/0 conn(0x7ff81c150410 msgr2=0x7ff81c155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:31.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.073+0000 7ff820dac700 1 --2- 172.21.15.157:0/2801833881 >> v2:172.21.15.175:3300/0 conn(0x7ff81c150410 0x7ff81c155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:31.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.073+0000 7ff820dac700 1 -- 172.21.15.157:0/2801833881 >> v1:172.21.15.175:6789/0 conn(0x7ff81c06cbb0 legacy=0x7ff81c06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:31.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.073+0000 7ff820dac700 1 -- 172.21.15.157:0/2801833881 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c0c8e50 con 0x7ff81c06dc40 2023-10-06T04:02:31.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.073+0000 7ff820dac700 1 -- 172.21.15.157:0/2801833881 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7ff80c001470 con 0x7ff81c06dc40 2023-10-06T04:02:31.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.073+0000 7ff820dac700 1 -- 172.21.15.157:0/2801833881 >> v1:172.21.15.157:6789/0 conn(0x7ff81c06dc40 legacy=0x7ff81c14fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:31.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.073+0000 7ff820dac700 1 --2- 172.21.15.157:0/2801833881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff808003a60 0x7ff808003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:31.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.073+0000 7ff820dac700 1 --2- 172.21.15.157:0/2801833881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff808004810 0x7ff808004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:31.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.073+0000 7ff820dac700 1 --2- 172.21.15.157:0/2801833881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8080051c0 0x7ff80800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:31.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.073+0000 7ff820dac700 1 -- 172.21.15.157:0/2801833881 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c0c8c40 con 0x7ff8080051c0 2023-10-06T04:02:31.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.073+0000 7ff820dac700 1 -- 172.21.15.157:0/2801833881 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80800c0d0 con 0x7ff808004810 2023-10-06T04:02:31.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.073+0000 7ff820dac700 1 -- 172.21.15.157:0/2801833881 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff80800c290 con 0x7ff808003a60 2023-10-06T04:02:31.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.073+0000 7ff8225af700 1 --2- 172.21.15.157:0/2801833881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8080051c0 0x7ff80800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:31.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.073+0000 7ff8215ad700 1 --2- 172.21.15.157:0/2801833881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff808004810 0x7ff808004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:31.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.073+0000 7ff821dae700 1 --2- 172.21.15.157:0/2801833881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff808003a60 0x7ff808003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:31.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.074+0000 7ff8225af700 1 -- 172.21.15.157:0/2801833881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff808003a60 msgr2=0x7ff808003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:31.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.074+0000 7ff8225af700 1 --2- 172.21.15.157:0/2801833881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff808003a60 0x7ff808003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:31.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.074+0000 7ff8225af700 1 -- 172.21.15.157:0/2801833881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff808004810 msgr2=0x7ff808004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:31.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.074+0000 7ff8225af700 1 --2- 172.21.15.157:0/2801833881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff808004810 0x7ff808004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:31.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.074+0000 7ff8225af700 1 -- 172.21.15.157:0/2801833881 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff808003570 con 0x7ff8080051c0 2023-10-06T04:02:31.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.074+0000 7ff8225af700 1 -- 172.21.15.157:0/2801833881 reap_dead start 2023-10-06T04:02:31.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.074+0000 7ff8225af700 1 --2- 172.21.15.157:0/2801833881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8080051c0 0x7ff80800ba60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff814009c00 tx=0x7ff8140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4b082d46e05870 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:31.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.074+0000 7ff820dac700 1 -- 172.21.15.157:0/2801833881 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81400e710 con 0x7ff8080051c0 2023-10-06T04:02:31.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.074+0000 7ff820dac700 1 -- 172.21.15.157:0/2801833881 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81400e870 con 0x7ff8080051c0 2023-10-06T04:02:31.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.074+0000 7ff820dac700 1 -- 172.21.15.157:0/2801833881 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81400ec30 con 0x7ff8080051c0 2023-10-06T04:02:31.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.074+0000 7ff824012700 1 -- 172.21.15.157:0/2801833881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8080051c0 msgr2=0x7ff80800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:31.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.074+0000 7ff824012700 1 --2- 172.21.15.157:0/2801833881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8080051c0 0x7ff80800ba60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff814009c00 tx=0x7ff8140091a0 comp rx=0 tx=0).stop 2023-10-06T04:02:31.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.074+0000 7ff824012700 1 -- 172.21.15.157:0/2801833881 shutdown_connections 2023-10-06T04:02:31.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.074+0000 7ff824012700 1 --2- 172.21.15.157:0/2801833881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8080051c0 0x7ff80800ba60 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:31.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.074+0000 7ff824012700 1 -- 172.21.15.157:0/2801833881 >> 172.21.15.157:0/2801833881 conn(0x7ff81c06f230 msgr2=0x7ff81c06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:31.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.075+0000 7ff824012700 1 -- 172.21.15.157:0/2801833881 shutdown_connections 2023-10-06T04:02:31.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.075+0000 7ff824012700 1 -- 172.21.15.157:0/2801833881 wait complete. 2023-10-06T04:02:31.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.076+0000 7ff824012700 1 Processor -- start 2023-10-06T04:02:31.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.076+0000 7ff824012700 1 -- start start 2023-10-06T04:02:31.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.076+0000 7ff824012700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8080051c0 0x7ff81c147af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:31.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.076+0000 7ff824012700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff81c148030 0x7ff81c166170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:31.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.076+0000 7ff824012700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff81c14ed40 0x7ff81c145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:31.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.076+0000 7ff824012700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c166eb0 con 0x7ff81c14ed40 2023-10-06T04:02:31.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.076+0000 7ff824012700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c0c3530 con 0x7ff8080051c0 2023-10-06T04:02:31.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.076+0000 7ff824012700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c152cc0 con 0x7ff81c148030 2023-10-06T04:02:31.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.076+0000 7ff8225af700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff81c14ed40 0x7ff81c145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:31.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.076+0000 7ff8225af700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff81c14ed40 0x7ff81c145800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52000/0 (socket says 172.21.15.157:52000) 2023-10-06T04:02:31.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.076+0000 7ff8225af700 1 -- 172.21.15.157:0/644846894 learned_addr learned my addr 172.21.15.157:0/644846894 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:31.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.077+0000 7ff821dae700 1 --2- 172.21.15.157:0/644846894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8080051c0 0x7ff81c147af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:31.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.077+0000 7ff8215ad700 1 --2- 172.21.15.157:0/644846894 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff81c148030 0x7ff81c166170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:31.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.077+0000 7ff8225af700 1 -- 172.21.15.157:0/644846894 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff81c148030 msgr2=0x7ff81c166170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:31.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.077+0000 7ff8225af700 1 --2- 172.21.15.157:0/644846894 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff81c148030 0x7ff81c166170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:31.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.077+0000 7ff8225af700 1 -- 172.21.15.157:0/644846894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8080051c0 msgr2=0x7ff81c147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:31.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.077+0000 7ff8225af700 1 --2- 172.21.15.157:0/644846894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8080051c0 0x7ff81c147af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:31.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.077+0000 7ff8225af700 1 -- 172.21.15.157:0/644846894 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c1523d0 con 0x7ff81c14ed40 2023-10-06T04:02:31.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.077+0000 7ff8225af700 1 --2- 172.21.15.157:0/644846894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff81c14ed40 0x7ff81c145800 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff81400e4b0 tx=0x7ff814019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:31.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.077+0000 7ff811ffb700 1 -- 172.21.15.157:0/644846894 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81400e710 con 0x7ff81c14ed40 2023-10-06T04:02:31.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.077+0000 7ff811ffb700 1 -- 172.21.15.157:0/644846894 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81401c450 con 0x7ff81c14ed40 2023-10-06T04:02:31.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.077+0000 7ff824012700 1 -- 172.21.15.157:0/644846894 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81c166bd0 con 0x7ff81c14ed40 2023-10-06T04:02:31.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.077+0000 7ff811ffb700 1 -- 172.21.15.157:0/644846894 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff814006e50 con 0x7ff81c14ed40 2023-10-06T04:02:31.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.077+0000 7ff824012700 1 -- 172.21.15.157:0/644846894 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81c152950 con 0x7ff81c14ed40 2023-10-06T04:02:31.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.078+0000 7ff824012700 1 -- 172.21.15.157:0/644846894 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff81c089ca0 con 0x7ff81c14ed40 2023-10-06T04:02:31.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.079+0000 7ff811ffb700 1 -- 172.21.15.157:0/644846894 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff81401ca80 con 0x7ff81c14ed40 2023-10-06T04:02:31.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.079+0000 7ff811ffb700 1 --2- 172.21.15.157:0/644846894 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff800066f30 0x7ff8000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:31.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.079+0000 7ff811ffb700 1 -- 172.21.15.157:0/644846894 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6288+0+0 (secure 0 0 0) 0x7ff814086530 con 0x7ff81c14ed40 2023-10-06T04:02:31.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.079+0000 7ff821dae700 1 --2- 172.21.15.157:0/644846894 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff800066f30 0x7ff8000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:31.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.080+0000 7ff821dae700 1 --2- 172.21.15.157:0/644846894 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff800066f30 0x7ff8000693e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff818001750 tx=0x7ff818001640 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:31.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.083+0000 7ff811ffb700 1 -- 172.21.15.157:0/644846894 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff814053620 con 0x7ff81c14ed40 2023-10-06T04:02:31.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.313+0000 7ff824012700 1 -- 172.21.15.157:0/644846894 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff81c0c9130 con 0x7ff81c14ed40 2023-10-06T04:02:31.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.314+0000 7ff811ffb700 1 -- 172.21.15.157:0/644846894 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+6177 (secure 0 0 0) 0x7ff8140095b0 con 0x7ff81c14ed40 2023-10-06T04:02:31.316 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:02:31.316 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:02:22.870092+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2,3],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270},"failed":[],"damaged":[],"stopped":[4],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:stopping","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:02:31.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.316+0000 7ff824012700 1 -- 172.21.15.157:0/644846894 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff800066f30 msgr2=0x7ff8000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:31.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.316+0000 7ff824012700 1 --2- 172.21.15.157:0/644846894 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff800066f30 0x7ff8000693e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff818001750 tx=0x7ff818001640 comp rx=0 tx=0).stop 2023-10-06T04:02:31.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.316+0000 7ff824012700 1 -- 172.21.15.157:0/644846894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff81c14ed40 msgr2=0x7ff81c145800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:31.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.316+0000 7ff824012700 1 --2- 172.21.15.157:0/644846894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff81c14ed40 0x7ff81c145800 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff81400e4b0 tx=0x7ff814019000 comp rx=0 tx=0).stop 2023-10-06T04:02:31.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.316+0000 7ff824012700 1 -- 172.21.15.157:0/644846894 shutdown_connections 2023-10-06T04:02:31.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.316+0000 7ff824012700 1 --2- 172.21.15.157:0/644846894 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff800066f30 0x7ff8000693e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:31.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.316+0000 7ff824012700 1 --2- 172.21.15.157:0/644846894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8080051c0 0x7ff81c147af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:31.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.316+0000 7ff824012700 1 --2- 172.21.15.157:0/644846894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff81c14ed40 0x7ff81c145800 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:31.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.316+0000 7ff824012700 1 --2- 172.21.15.157:0/644846894 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff81c148030 0x7ff81c166170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:31.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.316+0000 7ff824012700 1 -- 172.21.15.157:0/644846894 >> 172.21.15.157:0/644846894 conn(0x7ff81c06f230 msgr2=0x7ff81c0c8280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:31.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.316+0000 7ff824012700 1 -- 172.21.15.157:0/644846894 shutdown_connections 2023-10-06T04:02:31.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:31.316+0000 7ff824012700 1 -- 172.21.15.157:0/644846894 wait complete. 2023-10-06T04:02:31.319 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 42 2023-10-06T04:02:31.330 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:02:33.331 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:02:33.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.444+0000 7f12ef92f700 1 Processor -- start 2023-10-06T04:02:33.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.444+0000 7f12ef92f700 1 -- start start 2023-10-06T04:02:33.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.445+0000 7f12ef92f700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f12e80c7bb0 0x7f12e80c7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:33.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.445+0000 7f12ef92f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12e8156040 0x7f12e814fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:33.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.445+0000 7f12ef92f700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12e8159640 con 0x7f12e80c7f80 2023-10-06T04:02:33.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.445+0000 7f12ef92f700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12e80c75c0 con 0x7f12e80c7bb0 2023-10-06T04:02:33.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.445+0000 7f12ef92f700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e8159920 con 0x7f12e8156040 2023-10-06T04:02:33.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.445+0000 7f12ed6cb700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f12e80c7bb0 0x7f12e80c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:33.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.445+0000 7f12ececa700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f12e80c7f80 0x7f12e8159db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37222/0 (socket says 172.21.15.157:37222) 2023-10-06T04:02:33.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.445+0000 7f12ed6cb700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f12e80c7bb0 0x7f12e80c7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53966/0 (socket says 172.21.15.157:53966) 2023-10-06T04:02:33.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.445+0000 7f12ececa700 1 -- 172.21.15.157:0/3461386684 learned_addr learned my addr 172.21.15.157:0/3461386684 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:02:33.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.445+0000 7f12edecc700 1 --2- 172.21.15.157:0/3461386684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12e8156040 0x7f12e814fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:33.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.446+0000 7f12ed6cb700 1 -- 172.21.15.157:0/3461386684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12e8156040 msgr2=0x7f12e814fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:33.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.446+0000 7f12ed6cb700 1 --2- 172.21.15.157:0/3461386684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12e8156040 0x7f12e814fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:33.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.446+0000 7f12ed6cb700 1 -- 172.21.15.157:0/3461386684 >> v1:172.21.15.157:6789/0 conn(0x7f12e80c7f80 legacy=0x7f12e8159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:33.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.446+0000 7f12ed6cb700 1 -- 172.21.15.157:0/3461386684 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e80c2990 con 0x7f12e80c7bb0 2023-10-06T04:02:33.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.446+0000 7f12dffff700 1 -- 172.21.15.157:0/3461386684 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4104628550 0 0) 0x7f12e8159640 con 0x7f12e80c7f80 2023-10-06T04:02:33.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.446+0000 7f12ed6cb700 1 --2- 172.21.15.157:0/3461386684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e80c7bb0 0x7f12e80c7080 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f12e4009910 tx=0x7f12e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b2d151f4cddae5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:33.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.446+0000 7f12dffff700 1 -- 172.21.15.157:0/3461386684 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e400b6d0 con 0x7f12e80c7bb0 2023-10-06T04:02:33.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.446+0000 7f12dffff700 1 -- 172.21.15.157:0/3461386684 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e400b830 con 0x7f12e80c7bb0 2023-10-06T04:02:33.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.446+0000 7f12dffff700 1 -- 172.21.15.157:0/3461386684 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e400bbf0 con 0x7f12e80c7bb0 2023-10-06T04:02:33.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.446+0000 7f12ef92f700 1 -- 172.21.15.157:0/3461386684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e80c7bb0 msgr2=0x7f12e80c7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:33.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.446+0000 7f12ef92f700 1 --2- 172.21.15.157:0/3461386684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e80c7bb0 0x7f12e80c7080 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f12e4009910 tx=0x7f12e40091a0 comp rx=0 tx=0).stop 2023-10-06T04:02:33.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.446+0000 7f12ef92f700 1 -- 172.21.15.157:0/3461386684 shutdown_connections 2023-10-06T04:02:33.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.446+0000 7f12ef92f700 1 --2- 172.21.15.157:0/3461386684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12e8156040 0x7f12e814fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:33.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.446+0000 7f12ef92f700 1 --2- 172.21.15.157:0/3461386684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e80c7bb0 0x7f12e80c7080 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:33.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.446+0000 7f12ef92f700 1 -- 172.21.15.157:0/3461386684 >> 172.21.15.157:0/3461386684 conn(0x7f12e80631f0 msgr2=0x7f12e806f200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:33.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.446+0000 7f12ef92f700 1 -- 172.21.15.157:0/3461386684 shutdown_connections 2023-10-06T04:02:33.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.446+0000 7f12ef92f700 1 -- 172.21.15.157:0/3461386684 wait complete. 2023-10-06T04:02:33.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.448+0000 7f12ef92f700 1 Processor -- start 2023-10-06T04:02:33.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.448+0000 7f12ef92f700 1 -- start start 2023-10-06T04:02:33.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.449+0000 7f12ef92f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12e80c7bb0 0x7f12e80cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:33.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.449+0000 7f12ef92f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e80c7f80 0x7f12e8169b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:33.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.449+0000 7f12ef92f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12e8156040 0x7f12e816a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:33.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.449+0000 7f12ef92f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e80c5740 con 0x7f12e80c7f80 2023-10-06T04:02:33.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.449+0000 7f12ef92f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e80c83e0 con 0x7f12e80c7bb0 2023-10-06T04:02:33.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.449+0000 7f12ef92f700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e80c8c80 con 0x7f12e8156040 2023-10-06T04:02:33.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.449+0000 7f12ececa700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e80c7f80 0x7f12e8169b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:33.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.449+0000 7f12ececa700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e80c7f80 0x7f12e8169b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53982/0 (socket says 172.21.15.157:53982) 2023-10-06T04:02:33.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.449+0000 7f12ececa700 1 -- 172.21.15.157:0/3303160130 learned_addr learned my addr 172.21.15.157:0/3303160130 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:33.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.449+0000 7f12ed6cb700 1 --2- 172.21.15.157:0/3303160130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12e80c7bb0 0x7f12e80cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:33.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.449+0000 7f12edecc700 1 --2- 172.21.15.157:0/3303160130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12e8156040 0x7f12e816a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:33.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.449+0000 7f12ececa700 1 -- 172.21.15.157:0/3303160130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12e8156040 msgr2=0x7f12e816a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:33.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.449+0000 7f12ececa700 1 --2- 172.21.15.157:0/3303160130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12e8156040 0x7f12e816a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:33.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.449+0000 7f12ececa700 1 -- 172.21.15.157:0/3303160130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12e80c7bb0 msgr2=0x7f12e80cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:33.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.449+0000 7f12ececa700 1 --2- 172.21.15.157:0/3303160130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12e80c7bb0 0x7f12e80cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:33.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.449+0000 7f12ececa700 1 -- 172.21.15.157:0/3303160130 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e4009420 con 0x7f12e80c7f80 2023-10-06T04:02:33.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.449+0000 7f12ececa700 1 --2- 172.21.15.157:0/3303160130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e80c7f80 0x7f12e8169b00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f12d800c7e0 tx=0x7f12d8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:33.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.449+0000 7f12dd7fa700 1 -- 172.21.15.157:0/3303160130 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d8005b30 con 0x7f12e80c7f80 2023-10-06T04:02:33.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.449+0000 7f12dd7fa700 1 -- 172.21.15.157:0/3303160130 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d8005c90 con 0x7f12e80c7f80 2023-10-06T04:02:33.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.449+0000 7f12ef92f700 1 -- 172.21.15.157:0/3303160130 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e815cdb0 con 0x7f12e80c7f80 2023-10-06T04:02:33.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.449+0000 7f12dd7fa700 1 -- 172.21.15.157:0/3303160130 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d8007300 con 0x7f12e80c7f80 2023-10-06T04:02:33.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.449+0000 7f12ef92f700 1 -- 172.21.15.157:0/3303160130 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e80cd3d0 con 0x7f12e80c7f80 2023-10-06T04:02:33.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.450+0000 7f12ef92f700 1 -- 172.21.15.157:0/3303160130 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12e80c8e30 con 0x7f12e80c7f80 2023-10-06T04:02:33.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.451+0000 7f12dd7fa700 1 -- 172.21.15.157:0/3303160130 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f12d80076b0 con 0x7f12e80c7f80 2023-10-06T04:02:33.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.451+0000 7f12dd7fa700 1 --2- 172.21.15.157:0/3303160130 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f12cc066f30 0x7f12cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:33.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.451+0000 7f12dd7fa700 1 -- 172.21.15.157:0/3303160130 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 7128+0+0 (secure 0 0 0) 0x7f12d8083f80 con 0x7f12e80c7f80 2023-10-06T04:02:33.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.451+0000 7f12ed6cb700 1 --2- 172.21.15.157:0/3303160130 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f12cc066f30 0x7f12cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:33.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.452+0000 7f12ed6cb700 1 --2- 172.21.15.157:0/3303160130 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f12cc066f30 0x7f12cc0693e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f12e40091e0 tx=0x7f12e4017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:33.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.455+0000 7f12dd7fa700 1 -- 172.21.15.157:0/3303160130 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f12e80c8e30 con 0x7f12e80c7f80 2023-10-06T04:02:33.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.697+0000 7f12ef92f700 1 -- 172.21.15.157:0/3303160130 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12e806d550 con 0x7f12e80c7f80 2023-10-06T04:02:33.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.697+0000 7f12dd7fa700 1 -- 172.21.15.157:0/3303160130 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+6178 (secure 0 0 0) 0x7f12d80546f0 con 0x7f12e80c7f80 2023-10-06T04:02:33.700 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:02:33.700 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:02:32.894779+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2,3],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270},"failed":[],"damaged":[],"stopped":[4],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:stopping","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:02:33.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.699+0000 7f12ef92f700 1 -- 172.21.15.157:0/3303160130 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f12cc066f30 msgr2=0x7f12cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:33.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.699+0000 7f12ef92f700 1 --2- 172.21.15.157:0/3303160130 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f12cc066f30 0x7f12cc0693e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f12e40091e0 tx=0x7f12e4017000 comp rx=0 tx=0).stop 2023-10-06T04:02:33.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.699+0000 7f12ef92f700 1 -- 172.21.15.157:0/3303160130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e80c7f80 msgr2=0x7f12e8169b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:33.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.699+0000 7f12ef92f700 1 --2- 172.21.15.157:0/3303160130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e80c7f80 0x7f12e8169b00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f12d800c7e0 tx=0x7f12d8007df0 comp rx=0 tx=0).stop 2023-10-06T04:02:33.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.700+0000 7f12ef92f700 1 -- 172.21.15.157:0/3303160130 shutdown_connections 2023-10-06T04:02:33.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.700+0000 7f12ef92f700 1 --2- 172.21.15.157:0/3303160130 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f12cc066f30 0x7f12cc0693e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:33.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.700+0000 7f12ef92f700 1 --2- 172.21.15.157:0/3303160130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12e80c7bb0 0x7f12e80cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:33.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.700+0000 7f12ef92f700 1 --2- 172.21.15.157:0/3303160130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12e8156040 0x7f12e816a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:33.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.700+0000 7f12ef92f700 1 --2- 172.21.15.157:0/3303160130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e80c7f80 0x7f12e8169b00 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:33.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.700+0000 7f12ef92f700 1 -- 172.21.15.157:0/3303160130 >> 172.21.15.157:0/3303160130 conn(0x7f12e80631f0 msgr2=0x7f12e8154f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:33.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.700+0000 7f12ef92f700 1 -- 172.21.15.157:0/3303160130 shutdown_connections 2023-10-06T04:02:33.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:33.700+0000 7f12ef92f700 1 -- 172.21.15.157:0/3303160130 wait complete. 2023-10-06T04:02:33.703 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 43 2023-10-06T04:02:33.713 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:02:35.714 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:02:35.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.831+0000 7f8e6d129700 1 Processor -- start 2023-10-06T04:02:35.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.831+0000 7f8e6d129700 1 -- start start 2023-10-06T04:02:35.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.831+0000 7f8e6d129700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8e6806a780 0x7f8e680c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:35.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.831+0000 7f8e6d129700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e68156790 0x7f8e6815d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:35.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.831+0000 7f8e6d129700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e680c6160 con 0x7f8e6806ab50 2023-10-06T04:02:35.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.832+0000 7f8e6d129700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e680c5c00 con 0x7f8e6806a780 2023-10-06T04:02:35.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.832+0000 7f8e6d129700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e68069040 con 0x7f8e68156790 2023-10-06T04:02:35.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.832+0000 7f8e6659c700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f8e6806ab50 0x7f8e68156080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37228/0 (socket says 172.21.15.157:37228) 2023-10-06T04:02:35.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.832+0000 7f8e6659c700 1 -- 172.21.15.157:0/883973854 learned_addr learned my addr 172.21.15.157:0/883973854 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:02:35.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.832+0000 7f8e66d9d700 1 --2- 172.21.15.157:0/883973854 >> v2:172.21.15.175:3300/0 conn(0x7f8e6806a780 0x7f8e680c65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:35.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.832+0000 7f8e6759e700 1 --2- 172.21.15.157:0/883973854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e68156790 0x7f8e6815d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:35.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.832+0000 7f8e65d9b700 1 -- 172.21.15.157:0/883973854 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4058417382 0 0) 0x7f8e680c6160 con 0x7f8e6806ab50 2023-10-06T04:02:35.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.832+0000 7f8e65d9b700 1 -- 172.21.15.157:0/883973854 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e50003570 con 0x7f8e6806ab50 2023-10-06T04:02:35.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.832+0000 7f8e66d9d700 1 -- 172.21.15.157:0/883973854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e68156790 msgr2=0x7f8e6815d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:35.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.832+0000 7f8e66d9d700 1 --2- 172.21.15.157:0/883973854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e68156790 0x7f8e6815d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:35.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.832+0000 7f8e66d9d700 1 -- 172.21.15.157:0/883973854 >> v1:172.21.15.157:6789/0 conn(0x7f8e6806ab50 legacy=0x7f8e68156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:35.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.832+0000 7f8e66d9d700 1 -- 172.21.15.157:0/883973854 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e68153930 con 0x7f8e6806a780 2023-10-06T04:02:35.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.832+0000 7f8e6759e700 1 --2- 172.21.15.157:0/883973854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e68156790 0x7f8e6815d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:02:35.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.832+0000 7f8e66d9d700 1 --2- 172.21.15.157:0/883973854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e6806a780 0x7f8e680c65f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800b9a0 tx=0x7f8e580090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c543ddc3401c3e33 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:35.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.832+0000 7f8e65d9b700 1 -- 172.21.15.157:0/883973854 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5800c8d0 con 0x7f8e6806a780 2023-10-06T04:02:35.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.832+0000 7f8e65d9b700 1 -- 172.21.15.157:0/883973854 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e5800ca30 con 0x7f8e6806a780 2023-10-06T04:02:35.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.832+0000 7f8e65d9b700 1 -- 172.21.15.157:0/883973854 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5800cdf0 con 0x7f8e6806a780 2023-10-06T04:02:35.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.833+0000 7f8e6d129700 1 -- 172.21.15.157:0/883973854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e6806a780 msgr2=0x7f8e680c65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:35.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.833+0000 7f8e6d129700 1 --2- 172.21.15.157:0/883973854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e6806a780 0x7f8e680c65f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800b9a0 tx=0x7f8e580090f0 comp rx=0 tx=0).stop 2023-10-06T04:02:35.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.833+0000 7f8e6d129700 1 -- 172.21.15.157:0/883973854 shutdown_connections 2023-10-06T04:02:35.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.833+0000 7f8e6d129700 1 --2- 172.21.15.157:0/883973854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e68156790 0x7f8e6815d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:35.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.833+0000 7f8e6d129700 1 --2- 172.21.15.157:0/883973854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e6806a780 0x7f8e680c65f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:35.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.833+0000 7f8e6d129700 1 -- 172.21.15.157:0/883973854 >> 172.21.15.157:0/883973854 conn(0x7f8e6806b510 msgr2=0x7f8e6806b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:35.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.833+0000 7f8e6d129700 1 -- 172.21.15.157:0/883973854 shutdown_connections 2023-10-06T04:02:35.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.833+0000 7f8e6d129700 1 -- 172.21.15.157:0/883973854 wait complete. 2023-10-06T04:02:35.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.834+0000 7f8e6d129700 1 Processor -- start 2023-10-06T04:02:35.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.834+0000 7f8e6d129700 1 -- start start 2023-10-06T04:02:35.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.835+0000 7f8e6d129700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e6806a780 0x7f8e680bf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:35.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.835+0000 7f8e6d129700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e6806ab50 0x7f8e68164b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:35.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.835+0000 7f8e6d129700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e68156790 0x7f8e68165090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:35.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.835+0000 7f8e6d129700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e68150880 con 0x7f8e6806a780 2023-10-06T04:02:35.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.835+0000 7f8e6d129700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6806d2f0 con 0x7f8e6806ab50 2023-10-06T04:02:35.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.835+0000 7f8e6d129700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e681505c0 con 0x7f8e68156790 2023-10-06T04:02:35.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.835+0000 7f8e66d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e6806a780 0x7f8e680bf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:35.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.835+0000 7f8e66d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e6806a780 0x7f8e680bf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53990/0 (socket says 172.21.15.157:53990) 2023-10-06T04:02:35.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.835+0000 7f8e66d9d700 1 -- 172.21.15.157:0/493971347 learned_addr learned my addr 172.21.15.157:0/493971347 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:35.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.835+0000 7f8e6659c700 1 --2- 172.21.15.157:0/493971347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e6806ab50 0x7f8e68164b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:35.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.835+0000 7f8e6759e700 1 --2- 172.21.15.157:0/493971347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e68156790 0x7f8e68165090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:35.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.836+0000 7f8e66d9d700 1 -- 172.21.15.157:0/493971347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e68156790 msgr2=0x7f8e68165090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:35.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.836+0000 7f8e66d9d700 1 --2- 172.21.15.157:0/493971347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e68156790 0x7f8e68165090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:35.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.836+0000 7f8e66d9d700 1 -- 172.21.15.157:0/493971347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e6806ab50 msgr2=0x7f8e68164b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:35.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.836+0000 7f8e66d9d700 1 --2- 172.21.15.157:0/493971347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e6806ab50 0x7f8e68164b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:35.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.836+0000 7f8e66d9d700 1 -- 172.21.15.157:0/493971347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e6000de30 con 0x7f8e6806a780 2023-10-06T04:02:35.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.836+0000 7f8e6659c700 1 --2- 172.21.15.157:0/493971347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e6806ab50 0x7f8e68164b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:02:35.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.836+0000 7f8e66d9d700 1 --2- 172.21.15.157:0/493971347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e6806a780 0x7f8e680bf210 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8e6806b1f0 tx=0x7f8e58017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:35.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.836+0000 7f8e4effd700 1 -- 172.21.15.157:0/493971347 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5800fe00 con 0x7f8e6806a780 2023-10-06T04:02:35.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.836+0000 7f8e4effd700 1 -- 172.21.15.157:0/493971347 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e5801d040 con 0x7f8e6806a780 2023-10-06T04:02:35.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.836+0000 7f8e6d129700 1 -- 172.21.15.157:0/493971347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e5800b4e0 con 0x7f8e6806a780 2023-10-06T04:02:35.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.836+0000 7f8e4effd700 1 -- 172.21.15.157:0/493971347 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e58017650 con 0x7f8e6806a780 2023-10-06T04:02:35.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.836+0000 7f8e6d129700 1 -- 172.21.15.157:0/493971347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e6806d870 con 0x7f8e6806a780 2023-10-06T04:02:35.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.837+0000 7f8e6d129700 1 -- 172.21.15.157:0/493971347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e680692b0 con 0x7f8e6806a780 2023-10-06T04:02:35.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.837+0000 7f8e4effd700 1 -- 172.21.15.157:0/493971347 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8e58017a00 con 0x7f8e6806a780 2023-10-06T04:02:35.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.838+0000 7f8e4effd700 1 --2- 172.21.15.157:0/493971347 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8e54066f30 0x7f8e540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:35.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.838+0000 7f8e4effd700 1 -- 172.21.15.157:0/493971347 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7480+0+0 (secure 0 0 0) 0x7f8e58086990 con 0x7f8e6806a780 2023-10-06T04:02:35.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.838+0000 7f8e6659c700 1 --2- 172.21.15.157:0/493971347 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8e54066f30 0x7f8e540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:35.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.839+0000 7f8e6659c700 1 --2- 172.21.15.157:0/493971347 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8e54066f30 0x7f8e540693e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8e680c9d50 tx=0x7f8e5c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:35.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:35.842+0000 7f8e4effd700 1 -- 172.21.15.157:0/493971347 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8e58053510 con 0x7f8e6806a780 2023-10-06T04:02:36.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:36.273+0000 7f8e6d129700 1 -- 172.21.15.157:0/493971347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e680c6b90 con 0x7f8e6806a780 2023-10-06T04:02:36.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:36.273+0000 7f8e4effd700 1 -- 172.21.15.157:0/493971347 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+6178 (secure 0 0 0) 0x7f8e58017cb0 con 0x7f8e6806a780 2023-10-06T04:02:36.275 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:02:36.275 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:02:32.894779+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2,3],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270},"failed":[],"damaged":[],"stopped":[4],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:stopping","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:02:36.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:36.276+0000 7f8e6d129700 1 -- 172.21.15.157:0/493971347 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8e54066f30 msgr2=0x7f8e540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:36.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:36.276+0000 7f8e6d129700 1 --2- 172.21.15.157:0/493971347 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8e54066f30 0x7f8e540693e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8e680c9d50 tx=0x7f8e5c009040 comp rx=0 tx=0).stop 2023-10-06T04:02:36.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:36.276+0000 7f8e6d129700 1 -- 172.21.15.157:0/493971347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e6806a780 msgr2=0x7f8e680bf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:36.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:36.276+0000 7f8e6d129700 1 --2- 172.21.15.157:0/493971347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e6806a780 0x7f8e680bf210 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8e6806b1f0 tx=0x7f8e58017000 comp rx=0 tx=0).stop 2023-10-06T04:02:36.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:36.276+0000 7f8e6d129700 1 -- 172.21.15.157:0/493971347 shutdown_connections 2023-10-06T04:02:36.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:36.276+0000 7f8e6d129700 1 --2- 172.21.15.157:0/493971347 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8e54066f30 0x7f8e540693e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:36.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:36.276+0000 7f8e6d129700 1 --2- 172.21.15.157:0/493971347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e68156790 0x7f8e68165090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:36.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:36.276+0000 7f8e6d129700 1 --2- 172.21.15.157:0/493971347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e6806a780 0x7f8e680bf210 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:36.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:36.276+0000 7f8e6d129700 1 --2- 172.21.15.157:0/493971347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e6806ab50 0x7f8e68164b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:36.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:36.276+0000 7f8e6d129700 1 -- 172.21.15.157:0/493971347 >> 172.21.15.157:0/493971347 conn(0x7f8e6806b510 msgr2=0x7f8e68070940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:36.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:36.276+0000 7f8e6d129700 1 -- 172.21.15.157:0/493971347 shutdown_connections 2023-10-06T04:02:36.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:36.276+0000 7f8e6d129700 1 -- 172.21.15.157:0/493971347 wait complete. 2023-10-06T04:02:36.279 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 43 2023-10-06T04:02:36.290 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:02:38.065 INFO:tasks.workunit.client.0.smithi157.stdout:Delete all the files and directories ... 2023-10-06T04:02:38.291 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:02:38.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.426+0000 7f3698834700 1 Processor -- start 2023-10-06T04:02:38.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.426+0000 7f3698834700 1 -- start start 2023-10-06T04:02:38.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.427+0000 7f3698834700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f369414d530 0x7f369406a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:38.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.427+0000 7f3698834700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f36940bfad0 0x7f369406b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:38.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.427+0000 7f3698834700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f369406af50 con 0x7f36940c9450 2023-10-06T04:02:38.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.427+0000 7f3698834700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3694152f50 con 0x7f36940bfad0 2023-10-06T04:02:38.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.427+0000 7f3698834700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3694153160 con 0x7f369414d530 2023-10-06T04:02:38.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.427+0000 7f3692d9d700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f36940c9450 0x7f369414ce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37242/0 (socket says 172.21.15.157:37242) 2023-10-06T04:02:38.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.427+0000 7f3692d9d700 1 -- 172.21.15.157:0/2049896664 learned_addr learned my addr 172.21.15.157:0/2049896664 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:02:38.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.427+0000 7f369259c700 1 --2- 172.21.15.157:0/2049896664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f369414d530 0x7f369406a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:38.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.427+0000 7f369359e700 1 --2- 172.21.15.157:0/2049896664 >> v2:172.21.15.175:3300/0 conn(0x7f36940bfad0 0x7f369406b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:38.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.428+0000 7f3691d9b700 1 -- 172.21.15.157:0/2049896664 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 215651786 0 0) 0x7f369406af50 con 0x7f36940c9450 2023-10-06T04:02:38.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.428+0000 7f3691d9b700 1 -- 172.21.15.157:0/2049896664 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f367c003570 con 0x7f36940c9450 2023-10-06T04:02:38.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.428+0000 7f369259c700 1 -- 172.21.15.157:0/2049896664 >> v2:172.21.15.175:3300/0 conn(0x7f36940bfad0 msgr2=0x7f369406b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:38.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.428+0000 7f369259c700 1 --2- 172.21.15.157:0/2049896664 >> v2:172.21.15.175:3300/0 conn(0x7f36940bfad0 0x7f369406b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:38.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.428+0000 7f369259c700 1 -- 172.21.15.157:0/2049896664 >> v1:172.21.15.157:6789/0 conn(0x7f36940c9450 legacy=0x7f369414ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:38.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.428+0000 7f369259c700 1 -- 172.21.15.157:0/2049896664 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3694153440 con 0x7f369414d530 2023-10-06T04:02:38.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.428+0000 7f369259c700 1 --2- 172.21.15.157:0/2049896664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f369414d530 0x7f369406a330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f36880098e0 tx=0x7f36880091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3342897edd987397 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:38.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.428+0000 7f3691d9b700 1 -- 172.21.15.157:0/2049896664 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3688011630 con 0x7f369414d530 2023-10-06T04:02:38.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.428+0000 7f3691d9b700 1 -- 172.21.15.157:0/2049896664 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3688011790 con 0x7f369414d530 2023-10-06T04:02:38.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.428+0000 7f3691d9b700 1 -- 172.21.15.157:0/2049896664 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3688011b10 con 0x7f369414d530 2023-10-06T04:02:38.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.429+0000 7f3698834700 1 -- 172.21.15.157:0/2049896664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f369414d530 msgr2=0x7f369406a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:38.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.429+0000 7f3698834700 1 --2- 172.21.15.157:0/2049896664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f369414d530 0x7f369406a330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f36880098e0 tx=0x7f36880091a0 comp rx=0 tx=0).stop 2023-10-06T04:02:38.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.429+0000 7f3698834700 1 -- 172.21.15.157:0/2049896664 shutdown_connections 2023-10-06T04:02:38.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.429+0000 7f3698834700 1 --2- 172.21.15.157:0/2049896664 >> v2:172.21.15.175:3300/0 conn(0x7f36940bfad0 0x7f369406b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:38.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.429+0000 7f3698834700 1 --2- 172.21.15.157:0/2049896664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f369414d530 0x7f369406a330 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:38.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.429+0000 7f3698834700 1 -- 172.21.15.157:0/2049896664 >> 172.21.15.157:0/2049896664 conn(0x7f3694068020 msgr2=0x7f3694068420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:38.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.429+0000 7f3698834700 1 -- 172.21.15.157:0/2049896664 shutdown_connections 2023-10-06T04:02:38.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.429+0000 7f3698834700 1 -- 172.21.15.157:0/2049896664 wait complete. 2023-10-06T04:02:38.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.430+0000 7f3698834700 1 Processor -- start 2023-10-06T04:02:38.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.431+0000 7f3698834700 1 -- start start 2023-10-06T04:02:38.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.432+0000 7f3698834700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36940bfad0 0x7f3694159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:38.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.432+0000 7f3698834700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36940c9450 0x7f36941646e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:38.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.432+0000 7f3698834700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f369414d530 0x7f3694164c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:38.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.432+0000 7f3698834700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3694167980 con 0x7f36940c9450 2023-10-06T04:02:38.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.432+0000 7f3698834700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36941687e0 con 0x7f36940bfad0 2023-10-06T04:02:38.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.432+0000 7f3698834700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3694167bd0 con 0x7f369414d530 2023-10-06T04:02:38.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.432+0000 7f369259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36940c9450 0x7f36941646e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:38.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.432+0000 7f369259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36940c9450 0x7f36941646e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53994/0 (socket says 172.21.15.157:53994) 2023-10-06T04:02:38.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.432+0000 7f369259c700 1 -- 172.21.15.157:0/2860511162 learned_addr learned my addr 172.21.15.157:0/2860511162 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:38.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.432+0000 7f3692d9d700 1 --2- 172.21.15.157:0/2860511162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36940bfad0 0x7f3694159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:38.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.432+0000 7f369359e700 1 --2- 172.21.15.157:0/2860511162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f369414d530 0x7f3694164c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:38.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.432+0000 7f369259c700 1 -- 172.21.15.157:0/2860511162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f369414d530 msgr2=0x7f3694164c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:38.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.432+0000 7f369259c700 1 --2- 172.21.15.157:0/2860511162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f369414d530 0x7f3694164c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:38.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.432+0000 7f369259c700 1 -- 172.21.15.157:0/2860511162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36940bfad0 msgr2=0x7f3694159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:38.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.432+0000 7f369259c700 1 --2- 172.21.15.157:0/2860511162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36940bfad0 0x7f3694159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:38.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.432+0000 7f369259c700 1 -- 172.21.15.157:0/2860511162 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3688009420 con 0x7f36940c9450 2023-10-06T04:02:38.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.432+0000 7f369259c700 1 --2- 172.21.15.157:0/2860511162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36940c9450 0x7f36941646e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3688009ed0 tx=0x7f3688017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:38.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.432+0000 7f367affd700 1 -- 172.21.15.157:0/2860511162 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3688011900 con 0x7f36940c9450 2023-10-06T04:02:38.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.432+0000 7f367affd700 1 -- 172.21.15.157:0/2860511162 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3688011a80 con 0x7f36940c9450 2023-10-06T04:02:38.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.433+0000 7f367affd700 1 -- 172.21.15.157:0/2860511162 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368801a680 con 0x7f36940c9450 2023-10-06T04:02:38.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.433+0000 7f3698834700 1 -- 172.21.15.157:0/2860511162 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3694167eb0 con 0x7f36940c9450 2023-10-06T04:02:38.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.433+0000 7f3698834700 1 -- 172.21.15.157:0/2860511162 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3694168470 con 0x7f36940c9450 2023-10-06T04:02:38.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.433+0000 7f3698834700 1 -- 172.21.15.157:0/2860511162 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3694153720 con 0x7f36940c9450 2023-10-06T04:02:38.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.434+0000 7f367affd700 1 -- 172.21.15.157:0/2860511162 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f368801aab0 con 0x7f36940c9450 2023-10-06T04:02:38.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.434+0000 7f367affd700 1 --2- 172.21.15.157:0/2860511162 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3680066f30 0x7f36800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:38.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.435+0000 7f367affd700 1 -- 172.21.15.157:0/2860511162 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7844+0+0 (secure 0 0 0) 0x7f3688086820 con 0x7f36940c9450 2023-10-06T04:02:38.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.435+0000 7f3692d9d700 1 --2- 172.21.15.157:0/2860511162 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3680066f30 0x7f36800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:38.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.435+0000 7f3692d9d700 1 --2- 172.21.15.157:0/2860511162 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3680066f30 0x7f36800693e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3684007fd0 tx=0x7f3684009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:38.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.439+0000 7f367affd700 1 -- 172.21.15.157:0/2860511162 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3688053230 con 0x7f36940c9450 2023-10-06T04:02:38.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.669+0000 7f3698834700 1 -- 172.21.15.157:0/2860511162 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f369404bb40 con 0x7f36940c9450 2023-10-06T04:02:38.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.670+0000 7f367affd700 1 -- 172.21.15.157:0/2860511162 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+6182 (secure 0 0 0) 0x7f3688015090 con 0x7f36940c9450 2023-10-06T04:02:38.671 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:02:38.672 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:02:37.923949+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2,3],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270},"failed":[],"damaged":[],"stopped":[4],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:stopping","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:02:38.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.672+0000 7f3698834700 1 -- 172.21.15.157:0/2860511162 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3680066f30 msgr2=0x7f36800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:38.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.672+0000 7f3698834700 1 --2- 172.21.15.157:0/2860511162 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3680066f30 0x7f36800693e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3684007fd0 tx=0x7f3684009040 comp rx=0 tx=0).stop 2023-10-06T04:02:38.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.672+0000 7f3698834700 1 -- 172.21.15.157:0/2860511162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36940c9450 msgr2=0x7f36941646e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:38.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.672+0000 7f3698834700 1 --2- 172.21.15.157:0/2860511162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36940c9450 0x7f36941646e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3688009ed0 tx=0x7f3688017000 comp rx=0 tx=0).stop 2023-10-06T04:02:38.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.672+0000 7f3698834700 1 -- 172.21.15.157:0/2860511162 shutdown_connections 2023-10-06T04:02:38.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.672+0000 7f3698834700 1 --2- 172.21.15.157:0/2860511162 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3680066f30 0x7f36800693e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:38.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.672+0000 7f3698834700 1 --2- 172.21.15.157:0/2860511162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36940bfad0 0x7f3694159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:38.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.672+0000 7f3698834700 1 --2- 172.21.15.157:0/2860511162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f369414d530 0x7f3694164c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:38.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.672+0000 7f3698834700 1 --2- 172.21.15.157:0/2860511162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36940c9450 0x7f36941646e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:38.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.672+0000 7f3698834700 1 -- 172.21.15.157:0/2860511162 >> 172.21.15.157:0/2860511162 conn(0x7f3694068020 msgr2=0x7f369414a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:38.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.672+0000 7f3698834700 1 -- 172.21.15.157:0/2860511162 shutdown_connections 2023-10-06T04:02:38.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:38.672+0000 7f3698834700 1 -- 172.21.15.157:0/2860511162 wait complete. 2023-10-06T04:02:38.675 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 44 2023-10-06T04:02:38.685 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:02:40.502 INFO:tasks.workunit.client.0.smithi157.stdout:removed './100/19' 2023-10-06T04:02:40.502 INFO:tasks.workunit.client.0.smithi157.stdout:removed './100/15' 2023-10-06T04:02:40.503 INFO:tasks.workunit.client.0.smithi157.stdout:removed './100/31' 2023-10-06T04:02:40.503 INFO:tasks.workunit.client.0.smithi157.stdout:removed './100/36' 2023-10-06T04:02:40.503 INFO:tasks.workunit.client.0.smithi157.stdout:removed './100/20' 2023-10-06T04:02:40.503 INFO:tasks.workunit.client.0.smithi157.stdout:removed './100/12' 2023-10-06T04:02:40.503 INFO:tasks.workunit.client.0.smithi157.stdout:removed './100/39' 2023-10-06T04:02:40.503 INFO:tasks.workunit.client.0.smithi157.stdout:removed './100/10' 2023-10-06T04:02:40.503 INFO:tasks.workunit.client.0.smithi157.stdout:removed './100/34' 2023-10-06T04:02:40.503 INFO:tasks.workunit.client.0.smithi157.stdout:removed './100/38' 2023-10-06T04:02:40.503 INFO:tasks.workunit.client.0.smithi157.stdout:removed './100/11' 2023-10-06T04:02:40.503 INFO:tasks.workunit.client.0.smithi157.stdout:removed './100/16' 2023-10-06T04:02:40.503 INFO:tasks.workunit.client.0.smithi157.stdout:removed './100/32' 2023-10-06T04:02:40.504 INFO:tasks.workunit.client.0.smithi157.stdout:removed './100/30' 2023-10-06T04:02:40.504 INFO:tasks.workunit.client.0.smithi157.stdout:removed './100/14' 2023-10-06T04:02:40.504 INFO:tasks.workunit.client.0.smithi157.stdout:removed './100/18' 2023-10-06T04:02:40.504 INFO:tasks.workunit.client.0.smithi157.stdout:removed './100/13' 2023-10-06T04:02:40.504 INFO:tasks.workunit.client.0.smithi157.stdout:removed './100/33' 2023-10-06T04:02:40.504 INFO:tasks.workunit.client.0.smithi157.stdout:removed './100/17' 2023-10-06T04:02:40.504 INFO:tasks.workunit.client.0.smithi157.stdout:removed './100/35' 2023-10-06T04:02:40.504 INFO:tasks.workunit.client.0.smithi157.stdout:removed './100/37' 2023-10-06T04:02:40.504 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './100' 2023-10-06T04:02:40.504 INFO:tasks.workunit.client.0.smithi157.stdout:removed './101/19' 2023-10-06T04:02:40.504 INFO:tasks.workunit.client.0.smithi157.stdout:removed './101/15' 2023-10-06T04:02:40.504 INFO:tasks.workunit.client.0.smithi157.stdout:removed './101/31' 2023-10-06T04:02:40.505 INFO:tasks.workunit.client.0.smithi157.stdout:removed './101/36' 2023-10-06T04:02:40.505 INFO:tasks.workunit.client.0.smithi157.stdout:removed './101/20' 2023-10-06T04:02:40.505 INFO:tasks.workunit.client.0.smithi157.stdout:removed './101/12' 2023-10-06T04:02:40.505 INFO:tasks.workunit.client.0.smithi157.stdout:removed './101/39' 2023-10-06T04:02:40.505 INFO:tasks.workunit.client.0.smithi157.stdout:removed './101/10' 2023-10-06T04:02:40.505 INFO:tasks.workunit.client.0.smithi157.stdout:removed './101/34' 2023-10-06T04:02:40.505 INFO:tasks.workunit.client.0.smithi157.stdout:removed './101/38' 2023-10-06T04:02:40.505 INFO:tasks.workunit.client.0.smithi157.stdout:removed './101/11' 2023-10-06T04:02:40.505 INFO:tasks.workunit.client.0.smithi157.stdout:removed './101/16' 2023-10-06T04:02:40.505 INFO:tasks.workunit.client.0.smithi157.stdout:removed './101/32' 2023-10-06T04:02:40.505 INFO:tasks.workunit.client.0.smithi157.stdout:removed './101/30' 2023-10-06T04:02:40.506 INFO:tasks.workunit.client.0.smithi157.stdout:removed './101/14' 2023-10-06T04:02:40.506 INFO:tasks.workunit.client.0.smithi157.stdout:removed './101/18' 2023-10-06T04:02:40.506 INFO:tasks.workunit.client.0.smithi157.stdout:removed './101/13' 2023-10-06T04:02:40.506 INFO:tasks.workunit.client.0.smithi157.stdout:removed './101/33' 2023-10-06T04:02:40.506 INFO:tasks.workunit.client.0.smithi157.stdout:removed './101/17' 2023-10-06T04:02:40.506 INFO:tasks.workunit.client.0.smithi157.stdout:removed './101/35' 2023-10-06T04:02:40.506 INFO:tasks.workunit.client.0.smithi157.stdout:removed './101/37' 2023-10-06T04:02:40.506 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './101' 2023-10-06T04:02:40.506 INFO:tasks.workunit.client.0.smithi157.stdout:removed './102/19' 2023-10-06T04:02:40.506 INFO:tasks.workunit.client.0.smithi157.stdout:removed './102/15' 2023-10-06T04:02:40.506 INFO:tasks.workunit.client.0.smithi157.stdout:removed './102/31' 2023-10-06T04:02:40.506 INFO:tasks.workunit.client.0.smithi157.stdout:removed './102/36' 2023-10-06T04:02:40.507 INFO:tasks.workunit.client.0.smithi157.stdout:removed './102/20' 2023-10-06T04:02:40.507 INFO:tasks.workunit.client.0.smithi157.stdout:removed './102/12' 2023-10-06T04:02:40.507 INFO:tasks.workunit.client.0.smithi157.stdout:removed './102/39' 2023-10-06T04:02:40.507 INFO:tasks.workunit.client.0.smithi157.stdout:removed './102/10' 2023-10-06T04:02:40.507 INFO:tasks.workunit.client.0.smithi157.stdout:removed './102/34' 2023-10-06T04:02:40.507 INFO:tasks.workunit.client.0.smithi157.stdout:removed './102/38' 2023-10-06T04:02:40.507 INFO:tasks.workunit.client.0.smithi157.stdout:removed './102/11' 2023-10-06T04:02:40.507 INFO:tasks.workunit.client.0.smithi157.stdout:removed './102/16' 2023-10-06T04:02:40.507 INFO:tasks.workunit.client.0.smithi157.stdout:removed './102/32' 2023-10-06T04:02:40.507 INFO:tasks.workunit.client.0.smithi157.stdout:removed './102/30' 2023-10-06T04:02:40.507 INFO:tasks.workunit.client.0.smithi157.stdout:removed './102/14' 2023-10-06T04:02:40.507 INFO:tasks.workunit.client.0.smithi157.stdout:removed './102/18' 2023-10-06T04:02:40.508 INFO:tasks.workunit.client.0.smithi157.stdout:removed './102/13' 2023-10-06T04:02:40.508 INFO:tasks.workunit.client.0.smithi157.stdout:removed './102/33' 2023-10-06T04:02:40.508 INFO:tasks.workunit.client.0.smithi157.stdout:removed './102/17' 2023-10-06T04:02:40.508 INFO:tasks.workunit.client.0.smithi157.stdout:removed './102/35' 2023-10-06T04:02:40.508 INFO:tasks.workunit.client.0.smithi157.stdout:removed './102/37' 2023-10-06T04:02:40.508 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './102' 2023-10-06T04:02:40.508 INFO:tasks.workunit.client.0.smithi157.stdout:removed './103/19' 2023-10-06T04:02:40.508 INFO:tasks.workunit.client.0.smithi157.stdout:removed './103/15' 2023-10-06T04:02:40.508 INFO:tasks.workunit.client.0.smithi157.stdout:removed './103/31' 2023-10-06T04:02:40.508 INFO:tasks.workunit.client.0.smithi157.stdout:removed './103/36' 2023-10-06T04:02:40.508 INFO:tasks.workunit.client.0.smithi157.stdout:removed './103/20' 2023-10-06T04:02:40.508 INFO:tasks.workunit.client.0.smithi157.stdout:removed './103/12' 2023-10-06T04:02:40.509 INFO:tasks.workunit.client.0.smithi157.stdout:removed './103/39' 2023-10-06T04:02:40.509 INFO:tasks.workunit.client.0.smithi157.stdout:removed './103/10' 2023-10-06T04:02:40.509 INFO:tasks.workunit.client.0.smithi157.stdout:removed './103/34' 2023-10-06T04:02:40.509 INFO:tasks.workunit.client.0.smithi157.stdout:removed './103/38' 2023-10-06T04:02:40.509 INFO:tasks.workunit.client.0.smithi157.stdout:removed './103/11' 2023-10-06T04:02:40.509 INFO:tasks.workunit.client.0.smithi157.stdout:removed './103/16' 2023-10-06T04:02:40.509 INFO:tasks.workunit.client.0.smithi157.stdout:removed './103/32' 2023-10-06T04:02:40.509 INFO:tasks.workunit.client.0.smithi157.stdout:removed './103/30' 2023-10-06T04:02:40.509 INFO:tasks.workunit.client.0.smithi157.stdout:removed './103/14' 2023-10-06T04:02:40.509 INFO:tasks.workunit.client.0.smithi157.stdout:removed './103/18' 2023-10-06T04:02:40.509 INFO:tasks.workunit.client.0.smithi157.stdout:removed './103/13' 2023-10-06T04:02:40.509 INFO:tasks.workunit.client.0.smithi157.stdout:removed './103/33' 2023-10-06T04:02:40.510 INFO:tasks.workunit.client.0.smithi157.stdout:removed './103/17' 2023-10-06T04:02:40.510 INFO:tasks.workunit.client.0.smithi157.stdout:removed './103/35' 2023-10-06T04:02:40.510 INFO:tasks.workunit.client.0.smithi157.stdout:removed './103/37' 2023-10-06T04:02:40.510 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './103' 2023-10-06T04:02:40.510 INFO:tasks.workunit.client.0.smithi157.stdout:removed './104/19' 2023-10-06T04:02:40.510 INFO:tasks.workunit.client.0.smithi157.stdout:removed './104/15' 2023-10-06T04:02:40.510 INFO:tasks.workunit.client.0.smithi157.stdout:removed './104/31' 2023-10-06T04:02:40.510 INFO:tasks.workunit.client.0.smithi157.stdout:removed './104/36' 2023-10-06T04:02:40.510 INFO:tasks.workunit.client.0.smithi157.stdout:removed './104/20' 2023-10-06T04:02:40.510 INFO:tasks.workunit.client.0.smithi157.stdout:removed './104/12' 2023-10-06T04:02:40.510 INFO:tasks.workunit.client.0.smithi157.stdout:removed './104/39' 2023-10-06T04:02:40.510 INFO:tasks.workunit.client.0.smithi157.stdout:removed './104/10' 2023-10-06T04:02:40.511 INFO:tasks.workunit.client.0.smithi157.stdout:removed './104/34' 2023-10-06T04:02:40.511 INFO:tasks.workunit.client.0.smithi157.stdout:removed './104/38' 2023-10-06T04:02:40.511 INFO:tasks.workunit.client.0.smithi157.stdout:removed './104/11' 2023-10-06T04:02:40.511 INFO:tasks.workunit.client.0.smithi157.stdout:removed './104/16' 2023-10-06T04:02:40.511 INFO:tasks.workunit.client.0.smithi157.stdout:removed './104/32' 2023-10-06T04:02:40.511 INFO:tasks.workunit.client.0.smithi157.stdout:removed './104/30' 2023-10-06T04:02:40.511 INFO:tasks.workunit.client.0.smithi157.stdout:removed './104/14' 2023-10-06T04:02:40.511 INFO:tasks.workunit.client.0.smithi157.stdout:removed './104/18' 2023-10-06T04:02:40.511 INFO:tasks.workunit.client.0.smithi157.stdout:removed './104/13' 2023-10-06T04:02:40.511 INFO:tasks.workunit.client.0.smithi157.stdout:removed './104/33' 2023-10-06T04:02:40.511 INFO:tasks.workunit.client.0.smithi157.stdout:removed './104/17' 2023-10-06T04:02:40.511 INFO:tasks.workunit.client.0.smithi157.stdout:removed './104/35' 2023-10-06T04:02:40.512 INFO:tasks.workunit.client.0.smithi157.stdout:removed './104/37' 2023-10-06T04:02:40.512 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './104' 2023-10-06T04:02:40.512 INFO:tasks.workunit.client.0.smithi157.stdout:removed './105/19' 2023-10-06T04:02:40.512 INFO:tasks.workunit.client.0.smithi157.stdout:removed './105/15' 2023-10-06T04:02:40.512 INFO:tasks.workunit.client.0.smithi157.stdout:removed './105/31' 2023-10-06T04:02:40.512 INFO:tasks.workunit.client.0.smithi157.stdout:removed './105/36' 2023-10-06T04:02:40.512 INFO:tasks.workunit.client.0.smithi157.stdout:removed './105/20' 2023-10-06T04:02:40.512 INFO:tasks.workunit.client.0.smithi157.stdout:removed './105/12' 2023-10-06T04:02:40.512 INFO:tasks.workunit.client.0.smithi157.stdout:removed './105/39' 2023-10-06T04:02:40.512 INFO:tasks.workunit.client.0.smithi157.stdout:removed './105/10' 2023-10-06T04:02:40.512 INFO:tasks.workunit.client.0.smithi157.stdout:removed './105/34' 2023-10-06T04:02:40.512 INFO:tasks.workunit.client.0.smithi157.stdout:removed './105/38' 2023-10-06T04:02:40.513 INFO:tasks.workunit.client.0.smithi157.stdout:removed './105/11' 2023-10-06T04:02:40.513 INFO:tasks.workunit.client.0.smithi157.stdout:removed './105/16' 2023-10-06T04:02:40.513 INFO:tasks.workunit.client.0.smithi157.stdout:removed './105/32' 2023-10-06T04:02:40.513 INFO:tasks.workunit.client.0.smithi157.stdout:removed './105/30' 2023-10-06T04:02:40.513 INFO:tasks.workunit.client.0.smithi157.stdout:removed './105/14' 2023-10-06T04:02:40.513 INFO:tasks.workunit.client.0.smithi157.stdout:removed './105/18' 2023-10-06T04:02:40.513 INFO:tasks.workunit.client.0.smithi157.stdout:removed './105/13' 2023-10-06T04:02:40.513 INFO:tasks.workunit.client.0.smithi157.stdout:removed './105/33' 2023-10-06T04:02:40.513 INFO:tasks.workunit.client.0.smithi157.stdout:removed './105/17' 2023-10-06T04:02:40.513 INFO:tasks.workunit.client.0.smithi157.stdout:removed './105/35' 2023-10-06T04:02:40.513 INFO:tasks.workunit.client.0.smithi157.stdout:removed './105/37' 2023-10-06T04:02:40.513 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './105' 2023-10-06T04:02:40.514 INFO:tasks.workunit.client.0.smithi157.stdout:removed './106/19' 2023-10-06T04:02:40.514 INFO:tasks.workunit.client.0.smithi157.stdout:removed './106/15' 2023-10-06T04:02:40.514 INFO:tasks.workunit.client.0.smithi157.stdout:removed './106/31' 2023-10-06T04:02:40.514 INFO:tasks.workunit.client.0.smithi157.stdout:removed './106/36' 2023-10-06T04:02:40.514 INFO:tasks.workunit.client.0.smithi157.stdout:removed './106/20' 2023-10-06T04:02:40.514 INFO:tasks.workunit.client.0.smithi157.stdout:removed './106/12' 2023-10-06T04:02:40.514 INFO:tasks.workunit.client.0.smithi157.stdout:removed './106/39' 2023-10-06T04:02:40.514 INFO:tasks.workunit.client.0.smithi157.stdout:removed './106/10' 2023-10-06T04:02:40.514 INFO:tasks.workunit.client.0.smithi157.stdout:removed './106/34' 2023-10-06T04:02:40.514 INFO:tasks.workunit.client.0.smithi157.stdout:removed './106/38' 2023-10-06T04:02:40.514 INFO:tasks.workunit.client.0.smithi157.stdout:removed './106/11' 2023-10-06T04:02:40.514 INFO:tasks.workunit.client.0.smithi157.stdout:removed './106/16' 2023-10-06T04:02:40.515 INFO:tasks.workunit.client.0.smithi157.stdout:removed './106/32' 2023-10-06T04:02:40.515 INFO:tasks.workunit.client.0.smithi157.stdout:removed './106/30' 2023-10-06T04:02:40.515 INFO:tasks.workunit.client.0.smithi157.stdout:removed './106/14' 2023-10-06T04:02:40.515 INFO:tasks.workunit.client.0.smithi157.stdout:removed './106/18' 2023-10-06T04:02:40.515 INFO:tasks.workunit.client.0.smithi157.stdout:removed './106/13' 2023-10-06T04:02:40.515 INFO:tasks.workunit.client.0.smithi157.stdout:removed './106/33' 2023-10-06T04:02:40.515 INFO:tasks.workunit.client.0.smithi157.stdout:removed './106/17' 2023-10-06T04:02:40.515 INFO:tasks.workunit.client.0.smithi157.stdout:removed './106/35' 2023-10-06T04:02:40.515 INFO:tasks.workunit.client.0.smithi157.stdout:removed './106/37' 2023-10-06T04:02:40.515 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './106' 2023-10-06T04:02:40.515 INFO:tasks.workunit.client.0.smithi157.stdout:removed './107/19' 2023-10-06T04:02:40.515 INFO:tasks.workunit.client.0.smithi157.stdout:removed './107/15' 2023-10-06T04:02:40.516 INFO:tasks.workunit.client.0.smithi157.stdout:removed './107/31' 2023-10-06T04:02:40.516 INFO:tasks.workunit.client.0.smithi157.stdout:removed './107/36' 2023-10-06T04:02:40.516 INFO:tasks.workunit.client.0.smithi157.stdout:removed './107/20' 2023-10-06T04:02:40.516 INFO:tasks.workunit.client.0.smithi157.stdout:removed './107/12' 2023-10-06T04:02:40.516 INFO:tasks.workunit.client.0.smithi157.stdout:removed './107/39' 2023-10-06T04:02:40.516 INFO:tasks.workunit.client.0.smithi157.stdout:removed './107/10' 2023-10-06T04:02:40.516 INFO:tasks.workunit.client.0.smithi157.stdout:removed './107/34' 2023-10-06T04:02:40.516 INFO:tasks.workunit.client.0.smithi157.stdout:removed './107/38' 2023-10-06T04:02:40.516 INFO:tasks.workunit.client.0.smithi157.stdout:removed './107/11' 2023-10-06T04:02:40.516 INFO:tasks.workunit.client.0.smithi157.stdout:removed './107/16' 2023-10-06T04:02:40.516 INFO:tasks.workunit.client.0.smithi157.stdout:removed './107/32' 2023-10-06T04:02:40.516 INFO:tasks.workunit.client.0.smithi157.stdout:removed './107/30' 2023-10-06T04:02:40.517 INFO:tasks.workunit.client.0.smithi157.stdout:removed './107/14' 2023-10-06T04:02:40.517 INFO:tasks.workunit.client.0.smithi157.stdout:removed './107/18' 2023-10-06T04:02:40.517 INFO:tasks.workunit.client.0.smithi157.stdout:removed './107/13' 2023-10-06T04:02:40.517 INFO:tasks.workunit.client.0.smithi157.stdout:removed './107/33' 2023-10-06T04:02:40.517 INFO:tasks.workunit.client.0.smithi157.stdout:removed './107/17' 2023-10-06T04:02:40.517 INFO:tasks.workunit.client.0.smithi157.stdout:removed './107/35' 2023-10-06T04:02:40.517 INFO:tasks.workunit.client.0.smithi157.stdout:removed './107/37' 2023-10-06T04:02:40.517 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './107' 2023-10-06T04:02:40.517 INFO:tasks.workunit.client.0.smithi157.stdout:removed './108/19' 2023-10-06T04:02:40.517 INFO:tasks.workunit.client.0.smithi157.stdout:removed './108/15' 2023-10-06T04:02:40.517 INFO:tasks.workunit.client.0.smithi157.stdout:removed './108/31' 2023-10-06T04:02:40.517 INFO:tasks.workunit.client.0.smithi157.stdout:removed './108/36' 2023-10-06T04:02:40.518 INFO:tasks.workunit.client.0.smithi157.stdout:removed './108/20' 2023-10-06T04:02:40.518 INFO:tasks.workunit.client.0.smithi157.stdout:removed './108/12' 2023-10-06T04:02:40.518 INFO:tasks.workunit.client.0.smithi157.stdout:removed './108/39' 2023-10-06T04:02:40.518 INFO:tasks.workunit.client.0.smithi157.stdout:removed './108/10' 2023-10-06T04:02:40.518 INFO:tasks.workunit.client.0.smithi157.stdout:removed './108/34' 2023-10-06T04:02:40.518 INFO:tasks.workunit.client.0.smithi157.stdout:removed './108/38' 2023-10-06T04:02:40.518 INFO:tasks.workunit.client.0.smithi157.stdout:removed './108/11' 2023-10-06T04:02:40.518 INFO:tasks.workunit.client.0.smithi157.stdout:removed './108/16' 2023-10-06T04:02:40.518 INFO:tasks.workunit.client.0.smithi157.stdout:removed './108/32' 2023-10-06T04:02:40.518 INFO:tasks.workunit.client.0.smithi157.stdout:removed './108/30' 2023-10-06T04:02:40.518 INFO:tasks.workunit.client.0.smithi157.stdout:removed './108/14' 2023-10-06T04:02:40.518 INFO:tasks.workunit.client.0.smithi157.stdout:removed './108/18' 2023-10-06T04:02:40.519 INFO:tasks.workunit.client.0.smithi157.stdout:removed './108/13' 2023-10-06T04:02:40.519 INFO:tasks.workunit.client.0.smithi157.stdout:removed './108/33' 2023-10-06T04:02:40.519 INFO:tasks.workunit.client.0.smithi157.stdout:removed './108/17' 2023-10-06T04:02:40.519 INFO:tasks.workunit.client.0.smithi157.stdout:removed './108/35' 2023-10-06T04:02:40.519 INFO:tasks.workunit.client.0.smithi157.stdout:removed './108/37' 2023-10-06T04:02:40.519 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './108' 2023-10-06T04:02:40.519 INFO:tasks.workunit.client.0.smithi157.stdout:removed './109/19' 2023-10-06T04:02:40.519 INFO:tasks.workunit.client.0.smithi157.stdout:removed './109/15' 2023-10-06T04:02:40.519 INFO:tasks.workunit.client.0.smithi157.stdout:removed './109/31' 2023-10-06T04:02:40.519 INFO:tasks.workunit.client.0.smithi157.stdout:removed './109/36' 2023-10-06T04:02:40.519 INFO:tasks.workunit.client.0.smithi157.stdout:removed './109/20' 2023-10-06T04:02:40.519 INFO:tasks.workunit.client.0.smithi157.stdout:removed './109/12' 2023-10-06T04:02:40.520 INFO:tasks.workunit.client.0.smithi157.stdout:removed './109/39' 2023-10-06T04:02:40.520 INFO:tasks.workunit.client.0.smithi157.stdout:removed './109/10' 2023-10-06T04:02:40.520 INFO:tasks.workunit.client.0.smithi157.stdout:removed './109/34' 2023-10-06T04:02:40.520 INFO:tasks.workunit.client.0.smithi157.stdout:removed './109/38' 2023-10-06T04:02:40.520 INFO:tasks.workunit.client.0.smithi157.stdout:removed './109/11' 2023-10-06T04:02:40.520 INFO:tasks.workunit.client.0.smithi157.stdout:removed './109/16' 2023-10-06T04:02:40.520 INFO:tasks.workunit.client.0.smithi157.stdout:removed './109/32' 2023-10-06T04:02:40.520 INFO:tasks.workunit.client.0.smithi157.stdout:removed './109/30' 2023-10-06T04:02:40.687 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:02:40.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.806+0000 7f7bbdfde700 1 Processor -- start 2023-10-06T04:02:40.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.806+0000 7f7bbdfde700 1 -- start start 2023-10-06T04:02:40.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.807+0000 7f7bbdfde700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7bb806a780 0x7f7bb80c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:40.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.807+0000 7f7bbdfde700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bb8156790 0x7f7bb815d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:40.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.807+0000 7f7bbdfde700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb80c6160 con 0x7f7bb806ab50 2023-10-06T04:02:40.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.807+0000 7f7bbdfde700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb80c5c00 con 0x7f7bb806a780 2023-10-06T04:02:40.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.807+0000 7f7bbdfde700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb8069040 con 0x7f7bb8156790 2023-10-06T04:02:40.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.807+0000 7f7baffff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7bb806ab50 0x7f7bb8156080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37254/0 (socket says 172.21.15.157:37254) 2023-10-06T04:02:40.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.807+0000 7f7baffff700 1 -- 172.21.15.157:0/3546421378 learned_addr learned my addr 172.21.15.157:0/3546421378 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:02:40.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.807+0000 7f7bb77fe700 1 --2- 172.21.15.157:0/3546421378 >> v2:172.21.15.175:3300/0 conn(0x7f7bb806a780 0x7f7bb80c65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:40.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.807+0000 7f7bb7fff700 1 --2- 172.21.15.157:0/3546421378 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bb8156790 0x7f7bb815d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:40.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.807+0000 7f7bb6ffd700 1 -- 172.21.15.157:0/3546421378 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 683854707 0 0) 0x7f7bb80c6160 con 0x7f7bb806ab50 2023-10-06T04:02:40.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.807+0000 7f7bb6ffd700 1 -- 172.21.15.157:0/3546421378 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ba4003570 con 0x7f7bb806ab50 2023-10-06T04:02:40.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.808+0000 7f7bb77fe700 1 -- 172.21.15.157:0/3546421378 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bb8156790 msgr2=0x7f7bb815d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:40.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.808+0000 7f7bb77fe700 1 --2- 172.21.15.157:0/3546421378 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bb8156790 0x7f7bb815d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:40.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.808+0000 7f7bb77fe700 1 -- 172.21.15.157:0/3546421378 >> v1:172.21.15.157:6789/0 conn(0x7f7bb806ab50 legacy=0x7f7bb8156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:40.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.808+0000 7f7bb77fe700 1 -- 172.21.15.157:0/3546421378 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb8153930 con 0x7f7bb806a780 2023-10-06T04:02:40.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.808+0000 7f7bb7fff700 1 --2- 172.21.15.157:0/3546421378 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bb8156790 0x7f7bb815d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:02:40.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.808+0000 7f7bb77fe700 1 --2- 172.21.15.157:0/3546421378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bb806a780 0x7f7bb80c65f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80098e0 tx=0x7f7ba80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eacded563ffd5fd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:40.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.808+0000 7f7bb6ffd700 1 -- 172.21.15.157:0/3546421378 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8011630 con 0x7f7bb806a780 2023-10-06T04:02:40.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.808+0000 7f7bb6ffd700 1 -- 172.21.15.157:0/3546421378 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba8011790 con 0x7f7bb806a780 2023-10-06T04:02:40.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.808+0000 7f7bb6ffd700 1 -- 172.21.15.157:0/3546421378 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8011b30 con 0x7f7bb806a780 2023-10-06T04:02:40.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.809+0000 7f7bbdfde700 1 -- 172.21.15.157:0/3546421378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bb806a780 msgr2=0x7f7bb80c65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:40.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.809+0000 7f7bbdfde700 1 --2- 172.21.15.157:0/3546421378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bb806a780 0x7f7bb80c65f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80098e0 tx=0x7f7ba80091a0 comp rx=0 tx=0).stop 2023-10-06T04:02:40.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.809+0000 7f7bbdfde700 1 -- 172.21.15.157:0/3546421378 shutdown_connections 2023-10-06T04:02:40.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.809+0000 7f7bbdfde700 1 --2- 172.21.15.157:0/3546421378 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bb8156790 0x7f7bb815d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:40.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.809+0000 7f7bbdfde700 1 --2- 172.21.15.157:0/3546421378 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bb806a780 0x7f7bb80c65f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:40.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.809+0000 7f7bbdfde700 1 -- 172.21.15.157:0/3546421378 >> 172.21.15.157:0/3546421378 conn(0x7f7bb806b510 msgr2=0x7f7bb806b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:40.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.809+0000 7f7bbdfde700 1 -- 172.21.15.157:0/3546421378 shutdown_connections 2023-10-06T04:02:40.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.809+0000 7f7bbdfde700 1 -- 172.21.15.157:0/3546421378 wait complete. 2023-10-06T04:02:40.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.811+0000 7f7bbdfde700 1 Processor -- start 2023-10-06T04:02:40.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.811+0000 7f7bbdfde700 1 -- start start 2023-10-06T04:02:40.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.812+0000 7f7bbdfde700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bb806a780 0x7f7bb80bcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:40.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.812+0000 7f7bbdfde700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bb806ab50 0x7f7bb814f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:40.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.812+0000 7f7bbdfde700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bb8156790 0x7f7bb814fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:40.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.812+0000 7f7bbdfde700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb8156010 con 0x7f7bb806a780 2023-10-06T04:02:40.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.812+0000 7f7bbdfde700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb81556a0 con 0x7f7bb806ab50 2023-10-06T04:02:40.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.812+0000 7f7bbdfde700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb8155a90 con 0x7f7bb8156790 2023-10-06T04:02:40.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.812+0000 7f7bb77fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bb806a780 0x7f7bb80bcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:40.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.812+0000 7f7bb77fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bb806a780 0x7f7bb80bcfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54004/0 (socket says 172.21.15.157:54004) 2023-10-06T04:02:40.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.812+0000 7f7bb77fe700 1 -- 172.21.15.157:0/2998056201 learned_addr learned my addr 172.21.15.157:0/2998056201 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:40.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.812+0000 7f7bb7fff700 1 --2- 172.21.15.157:0/2998056201 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bb8156790 0x7f7bb814fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:40.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.812+0000 7f7baffff700 1 --2- 172.21.15.157:0/2998056201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bb806ab50 0x7f7bb814f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:40.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.812+0000 7f7bb77fe700 1 -- 172.21.15.157:0/2998056201 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bb8156790 msgr2=0x7f7bb814fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:40.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.812+0000 7f7bb77fe700 1 --2- 172.21.15.157:0/2998056201 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bb8156790 0x7f7bb814fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:40.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.812+0000 7f7bb77fe700 1 -- 172.21.15.157:0/2998056201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bb806ab50 msgr2=0x7f7bb814f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:40.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.812+0000 7f7bb77fe700 1 --2- 172.21.15.157:0/2998056201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bb806ab50 0x7f7bb814f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:40.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.812+0000 7f7bb77fe700 1 -- 172.21.15.157:0/2998056201 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb0009a20 con 0x7f7bb806a780 2023-10-06T04:02:40.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.812+0000 7f7bb7fff700 1 --2- 172.21.15.157:0/2998056201 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bb8156790 0x7f7bb814fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:02:40.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.812+0000 7f7baffff700 1 --2- 172.21.15.157:0/2998056201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bb806ab50 0x7f7bb814f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:02:40.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.813+0000 7f7bb77fe700 1 --2- 172.21.15.157:0/2998056201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bb806a780 0x7f7bb80bcfb0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7bb806b1f0 tx=0x7f7ba8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:40.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.813+0000 7f7baf7fe700 1 -- 172.21.15.157:0/2998056201 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8011900 con 0x7f7bb806a780 2023-10-06T04:02:40.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.813+0000 7f7baf7fe700 1 -- 172.21.15.157:0/2998056201 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba8011a60 con 0x7f7bb806a780 2023-10-06T04:02:40.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.813+0000 7f7bbdfde700 1 -- 172.21.15.157:0/2998056201 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba8009420 con 0x7f7bb806a780 2023-10-06T04:02:40.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.813+0000 7f7baf7fe700 1 -- 172.21.15.157:0/2998056201 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba801a680 con 0x7f7bb806a780 2023-10-06T04:02:40.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.813+0000 7f7bbdfde700 1 -- 172.21.15.157:0/2998056201 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb8154d70 con 0x7f7bb806a780 2023-10-06T04:02:40.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.814+0000 7f7bbdfde700 1 -- 172.21.15.157:0/2998056201 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb80692b0 con 0x7f7bb806a780 2023-10-06T04:02:40.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.815+0000 7f7baf7fe700 1 -- 172.21.15.157:0/2998056201 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7ba801a9f0 con 0x7f7bb806a780 2023-10-06T04:02:40.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.815+0000 7f7baf7fe700 1 --2- 172.21.15.157:0/2998056201 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7b98066f30 0x7f7b980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:40.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.815+0000 7f7baf7fe700 1 -- 172.21.15.157:0/2998056201 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7844+0+0 (secure 0 0 0) 0x7f7ba8086910 con 0x7f7bb806a780 2023-10-06T04:02:40.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.815+0000 7f7baffff700 1 --2- 172.21.15.157:0/2998056201 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7b98066f30 0x7f7b980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:40.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.816+0000 7f7baffff700 1 --2- 172.21.15.157:0/2998056201 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7b98066f30 0x7f7b980693e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8156670 tx=0x7f7ba0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:40.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:40.820+0000 7f7baf7fe700 1 -- 172.21.15.157:0/2998056201 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ba80533e0 con 0x7f7bb806a780 2023-10-06T04:02:41.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:41.047+0000 7f7bbdfde700 1 -- 172.21.15.157:0/2998056201 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7bb80c49d0 con 0x7f7bb806a780 2023-10-06T04:02:41.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:41.047+0000 7f7baf7fe700 1 -- 172.21.15.157:0/2998056201 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+6182 (secure 0 0 0) 0x7f7ba8015020 con 0x7f7bb806a780 2023-10-06T04:02:41.049 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:02:41.049 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:02:37.923949+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2,3],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270},"failed":[],"damaged":[],"stopped":[4],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:stopping","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:02:41.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:41.049+0000 7f7bbdfde700 1 -- 172.21.15.157:0/2998056201 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7b98066f30 msgr2=0x7f7b980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:41.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:41.049+0000 7f7bbdfde700 1 --2- 172.21.15.157:0/2998056201 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7b98066f30 0x7f7b980693e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8156670 tx=0x7f7ba0009040 comp rx=0 tx=0).stop 2023-10-06T04:02:41.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:41.049+0000 7f7bbdfde700 1 -- 172.21.15.157:0/2998056201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bb806a780 msgr2=0x7f7bb80bcfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:41.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:41.049+0000 7f7bbdfde700 1 --2- 172.21.15.157:0/2998056201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bb806a780 0x7f7bb80bcfb0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7bb806b1f0 tx=0x7f7ba8017000 comp rx=0 tx=0).stop 2023-10-06T04:02:41.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:41.049+0000 7f7bbdfde700 1 -- 172.21.15.157:0/2998056201 shutdown_connections 2023-10-06T04:02:41.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:41.049+0000 7f7bbdfde700 1 --2- 172.21.15.157:0/2998056201 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7b98066f30 0x7f7b980693e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:41.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:41.049+0000 7f7bbdfde700 1 --2- 172.21.15.157:0/2998056201 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bb8156790 0x7f7bb814fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:41.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:41.049+0000 7f7bbdfde700 1 --2- 172.21.15.157:0/2998056201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bb806a780 0x7f7bb80bcfb0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:41.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:41.049+0000 7f7bbdfde700 1 --2- 172.21.15.157:0/2998056201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bb806ab50 0x7f7bb814f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:41.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:41.049+0000 7f7bbdfde700 1 -- 172.21.15.157:0/2998056201 >> 172.21.15.157:0/2998056201 conn(0x7f7bb806b510 msgr2=0x7f7bb80c39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:41.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:41.050+0000 7f7bbdfde700 1 -- 172.21.15.157:0/2998056201 shutdown_connections 2023-10-06T04:02:41.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:41.050+0000 7f7bbdfde700 1 -- 172.21.15.157:0/2998056201 wait complete. 2023-10-06T04:02:41.054 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 44 2023-10-06T04:02:41.071 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2023-10-06T04:02:41.072 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 44, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-06T03:57:10.718782+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 44, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0, 1, 2, 3 ], "info": { "gid_4270": { "addr": "172.21.15.175:6835/1596042861", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6834", "nonce": 1596042861, "type": "v2" }, { "addr": "172.21.15.175:6835", "nonce": 1596042861, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1, 2 ], "features": 4540138320759226367, "flags": 0, "gid": 4270, "incarnation": 13, "join_fscid": -1, "name": "b", "rank": 3, "state": "up:stopping", "state_seq": 8 }, "gid_4279": { "addr": "172.21.15.175:6837/1002404870", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6836", "nonce": 1002404870, "type": "v2" }, { "addr": "172.21.15.175:6837", "nonce": 1002404870, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 3 ], "features": 4540138320759226367, "flags": 0, "gid": 4279, "incarnation": 11, "join_fscid": -1, "name": "d", "rank": 2, "state": "up:active", "state_seq": 7 }, "gid_4287": { "addr": "172.21.15.157:6834/1179753185", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6832", "nonce": 1179753185, "type": "v2" }, { "addr": "172.21.15.157:6834", "nonce": 1179753185, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4287, "incarnation": 9, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4296": { "addr": "172.21.15.157:6837/4188652695", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6836", "nonce": 4188652695, "type": "v2" }, { "addr": "172.21.15.157:6837", "nonce": 4188652695, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2, 4 ], "features": 4540138320759226367, "flags": 0, "gid": 4296, "incarnation": 4, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 27, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2023-10-06T04:02:37.923949+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 4 ], "tableserver": 0, "up": { "mds_0": 4296, "mds_1": 4287, "mds_2": 4279, "mds_3": 4270 } } } ], "standbys": [ { "addr": "172.21.15.175:6839/745804271", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6838", "nonce": 745804271, "type": "v2" }, { "addr": "172.21.15.175:6839", "nonce": 745804271, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 42, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5164, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.157:6835/2461751318", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6833", "nonce": 2461751318, "type": "v2" }, { "addr": "172.21.15.157:6835", "nonce": 2461751318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 34, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5268, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-06T04:02:41.238 INFO:tasks.workunit.client.0.smithi157.stdout:removed './109/14' 2023-10-06T04:02:41.238 INFO:tasks.workunit.client.0.smithi157.stdout:removed './109/18' 2023-10-06T04:02:41.238 INFO:tasks.workunit.client.0.smithi157.stdout:removed './109/13' 2023-10-06T04:02:41.238 INFO:tasks.workunit.client.0.smithi157.stdout:removed './109/33' 2023-10-06T04:02:41.238 INFO:tasks.workunit.client.0.smithi157.stdout:removed './109/17' 2023-10-06T04:02:41.239 INFO:tasks.workunit.client.0.smithi157.stdout:removed './109/35' 2023-10-06T04:02:41.239 INFO:tasks.workunit.client.0.smithi157.stdout:removed './109/37' 2023-10-06T04:02:41.239 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './109' 2023-10-06T04:02:41.239 INFO:tasks.workunit.client.0.smithi157.stdout:removed './110/19' 2023-10-06T04:02:41.239 INFO:tasks.workunit.client.0.smithi157.stdout:removed './110/15' 2023-10-06T04:02:41.239 INFO:tasks.workunit.client.0.smithi157.stdout:removed './110/31' 2023-10-06T04:02:41.239 INFO:tasks.workunit.client.0.smithi157.stdout:removed './110/36' 2023-10-06T04:02:41.239 INFO:tasks.workunit.client.0.smithi157.stdout:removed './110/20' 2023-10-06T04:02:41.239 INFO:tasks.workunit.client.0.smithi157.stdout:removed './110/12' 2023-10-06T04:02:41.239 INFO:tasks.workunit.client.0.smithi157.stdout:removed './110/39' 2023-10-06T04:02:41.240 INFO:tasks.workunit.client.0.smithi157.stdout:removed './110/10' 2023-10-06T04:02:41.240 INFO:tasks.workunit.client.0.smithi157.stdout:removed './110/34' 2023-10-06T04:02:41.240 INFO:tasks.workunit.client.0.smithi157.stdout:removed './110/38' 2023-10-06T04:02:41.240 INFO:tasks.workunit.client.0.smithi157.stdout:removed './110/11' 2023-10-06T04:02:41.240 INFO:tasks.workunit.client.0.smithi157.stdout:removed './110/16' 2023-10-06T04:02:41.240 INFO:tasks.workunit.client.0.smithi157.stdout:removed './110/32' 2023-10-06T04:02:41.240 INFO:tasks.workunit.client.0.smithi157.stdout:removed './110/30' 2023-10-06T04:02:41.240 INFO:tasks.workunit.client.0.smithi157.stdout:removed './110/14' 2023-10-06T04:02:41.240 INFO:tasks.workunit.client.0.smithi157.stdout:removed './110/18' 2023-10-06T04:02:41.240 INFO:tasks.workunit.client.0.smithi157.stdout:removed './110/13' 2023-10-06T04:02:41.240 INFO:tasks.workunit.client.0.smithi157.stdout:removed './110/33' 2023-10-06T04:02:41.241 INFO:tasks.workunit.client.0.smithi157.stdout:removed './110/17' 2023-10-06T04:02:41.241 INFO:tasks.workunit.client.0.smithi157.stdout:removed './110/35' 2023-10-06T04:02:41.241 INFO:tasks.workunit.client.0.smithi157.stdout:removed './110/37' 2023-10-06T04:02:41.241 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './110' 2023-10-06T04:02:41.241 INFO:tasks.workunit.client.0.smithi157.stdout:removed './111/19' 2023-10-06T04:02:41.241 INFO:tasks.workunit.client.0.smithi157.stdout:removed './111/15' 2023-10-06T04:02:41.241 INFO:tasks.workunit.client.0.smithi157.stdout:removed './111/31' 2023-10-06T04:02:41.241 INFO:tasks.workunit.client.0.smithi157.stdout:removed './111/36' 2023-10-06T04:02:41.241 INFO:tasks.workunit.client.0.smithi157.stdout:removed './111/20' 2023-10-06T04:02:41.241 INFO:tasks.workunit.client.0.smithi157.stdout:removed './111/12' 2023-10-06T04:02:41.242 INFO:tasks.workunit.client.0.smithi157.stdout:removed './111/39' 2023-10-06T04:02:41.242 INFO:tasks.workunit.client.0.smithi157.stdout:removed './111/10' 2023-10-06T04:02:41.242 INFO:tasks.workunit.client.0.smithi157.stdout:removed './111/34' 2023-10-06T04:02:41.242 INFO:tasks.workunit.client.0.smithi157.stdout:removed './111/38' 2023-10-06T04:02:41.242 INFO:tasks.workunit.client.0.smithi157.stdout:removed './111/11' 2023-10-06T04:02:41.242 INFO:tasks.workunit.client.0.smithi157.stdout:removed './111/16' 2023-10-06T04:02:41.242 INFO:tasks.workunit.client.0.smithi157.stdout:removed './111/32' 2023-10-06T04:02:41.242 INFO:tasks.workunit.client.0.smithi157.stdout:removed './111/30' 2023-10-06T04:02:41.242 INFO:tasks.workunit.client.0.smithi157.stdout:removed './111/14' 2023-10-06T04:02:41.242 INFO:tasks.workunit.client.0.smithi157.stdout:removed './111/18' 2023-10-06T04:02:41.243 INFO:tasks.workunit.client.0.smithi157.stdout:removed './111/13' 2023-10-06T04:02:41.243 INFO:tasks.workunit.client.0.smithi157.stdout:removed './111/33' 2023-10-06T04:02:41.243 INFO:tasks.workunit.client.0.smithi157.stdout:removed './111/17' 2023-10-06T04:02:41.243 INFO:tasks.workunit.client.0.smithi157.stdout:removed './111/35' 2023-10-06T04:02:41.243 INFO:tasks.workunit.client.0.smithi157.stdout:removed './111/37' 2023-10-06T04:02:41.243 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './111' 2023-10-06T04:02:41.243 INFO:tasks.workunit.client.0.smithi157.stdout:removed './112/19' 2023-10-06T04:02:41.243 INFO:tasks.workunit.client.0.smithi157.stdout:removed './112/15' 2023-10-06T04:02:41.243 INFO:tasks.workunit.client.0.smithi157.stdout:removed './112/31' 2023-10-06T04:02:41.244 INFO:tasks.workunit.client.0.smithi157.stdout:removed './112/36' 2023-10-06T04:02:41.244 INFO:tasks.workunit.client.0.smithi157.stdout:removed './112/20' 2023-10-06T04:02:41.244 INFO:tasks.workunit.client.0.smithi157.stdout:removed './112/12' 2023-10-06T04:02:41.244 INFO:tasks.workunit.client.0.smithi157.stdout:removed './112/39' 2023-10-06T04:02:41.244 INFO:tasks.workunit.client.0.smithi157.stdout:removed './112/10' 2023-10-06T04:02:41.244 INFO:tasks.workunit.client.0.smithi157.stdout:removed './112/34' 2023-10-06T04:02:41.244 INFO:tasks.workunit.client.0.smithi157.stdout:removed './112/38' 2023-10-06T04:02:41.244 INFO:tasks.workunit.client.0.smithi157.stdout:removed './112/11' 2023-10-06T04:02:41.244 INFO:tasks.workunit.client.0.smithi157.stdout:removed './112/16' 2023-10-06T04:02:41.244 INFO:tasks.workunit.client.0.smithi157.stdout:removed './112/32' 2023-10-06T04:02:41.245 INFO:tasks.workunit.client.0.smithi157.stdout:removed './112/30' 2023-10-06T04:02:41.245 INFO:tasks.workunit.client.0.smithi157.stdout:removed './112/14' 2023-10-06T04:02:41.245 INFO:tasks.workunit.client.0.smithi157.stdout:removed './112/18' 2023-10-06T04:02:41.245 INFO:tasks.workunit.client.0.smithi157.stdout:removed './112/13' 2023-10-06T04:02:41.245 INFO:tasks.workunit.client.0.smithi157.stdout:removed './112/33' 2023-10-06T04:02:41.245 INFO:tasks.workunit.client.0.smithi157.stdout:removed './112/17' 2023-10-06T04:02:41.245 INFO:tasks.workunit.client.0.smithi157.stdout:removed './112/35' 2023-10-06T04:02:41.245 INFO:tasks.workunit.client.0.smithi157.stdout:removed './112/37' 2023-10-06T04:02:41.245 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './112' 2023-10-06T04:02:41.245 INFO:tasks.workunit.client.0.smithi157.stdout:removed './113/19' 2023-10-06T04:02:41.246 INFO:tasks.workunit.client.0.smithi157.stdout:removed './113/15' 2023-10-06T04:02:41.246 INFO:tasks.workunit.client.0.smithi157.stdout:removed './113/31' 2023-10-06T04:02:41.246 INFO:tasks.workunit.client.0.smithi157.stdout:removed './113/36' 2023-10-06T04:02:41.246 INFO:tasks.workunit.client.0.smithi157.stdout:removed './113/20' 2023-10-06T04:02:41.246 INFO:tasks.workunit.client.0.smithi157.stdout:removed './113/12' 2023-10-06T04:02:41.246 INFO:tasks.workunit.client.0.smithi157.stdout:removed './113/39' 2023-10-06T04:02:41.246 INFO:tasks.workunit.client.0.smithi157.stdout:removed './113/10' 2023-10-06T04:02:41.246 INFO:tasks.workunit.client.0.smithi157.stdout:removed './113/34' 2023-10-06T04:02:41.246 INFO:tasks.workunit.client.0.smithi157.stdout:removed './113/38' 2023-10-06T04:02:41.247 INFO:tasks.workunit.client.0.smithi157.stdout:removed './113/11' 2023-10-06T04:02:41.247 INFO:tasks.workunit.client.0.smithi157.stdout:removed './113/16' 2023-10-06T04:02:41.247 INFO:tasks.workunit.client.0.smithi157.stdout:removed './113/32' 2023-10-06T04:02:41.247 INFO:tasks.workunit.client.0.smithi157.stdout:removed './113/30' 2023-10-06T04:02:41.247 INFO:tasks.workunit.client.0.smithi157.stdout:removed './113/14' 2023-10-06T04:02:41.247 INFO:tasks.workunit.client.0.smithi157.stdout:removed './113/18' 2023-10-06T04:02:41.247 INFO:tasks.workunit.client.0.smithi157.stdout:removed './113/13' 2023-10-06T04:02:41.247 INFO:tasks.workunit.client.0.smithi157.stdout:removed './113/33' 2023-10-06T04:02:41.247 INFO:tasks.workunit.client.0.smithi157.stdout:removed './113/17' 2023-10-06T04:02:41.247 INFO:tasks.workunit.client.0.smithi157.stdout:removed './113/35' 2023-10-06T04:02:41.248 INFO:tasks.workunit.client.0.smithi157.stdout:removed './113/37' 2023-10-06T04:02:41.248 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './113' 2023-10-06T04:02:41.248 INFO:tasks.workunit.client.0.smithi157.stdout:removed './114/19' 2023-10-06T04:02:41.248 INFO:tasks.workunit.client.0.smithi157.stdout:removed './114/15' 2023-10-06T04:02:41.248 INFO:tasks.workunit.client.0.smithi157.stdout:removed './114/31' 2023-10-06T04:02:41.248 INFO:tasks.workunit.client.0.smithi157.stdout:removed './114/36' 2023-10-06T04:02:41.248 INFO:tasks.workunit.client.0.smithi157.stdout:removed './114/20' 2023-10-06T04:02:41.248 INFO:tasks.workunit.client.0.smithi157.stdout:removed './114/12' 2023-10-06T04:02:41.249 INFO:tasks.workunit.client.0.smithi157.stdout:removed './114/39' 2023-10-06T04:02:41.249 INFO:tasks.workunit.client.0.smithi157.stdout:removed './114/10' 2023-10-06T04:02:41.249 INFO:tasks.workunit.client.0.smithi157.stdout:removed './114/34' 2023-10-06T04:02:41.249 INFO:tasks.workunit.client.0.smithi157.stdout:removed './114/38' 2023-10-06T04:02:41.249 INFO:tasks.workunit.client.0.smithi157.stdout:removed './114/11' 2023-10-06T04:02:41.249 INFO:tasks.workunit.client.0.smithi157.stdout:removed './114/16' 2023-10-06T04:02:41.249 INFO:tasks.workunit.client.0.smithi157.stdout:removed './114/32' 2023-10-06T04:02:41.250 INFO:tasks.workunit.client.0.smithi157.stdout:removed './114/30' 2023-10-06T04:02:41.250 INFO:tasks.workunit.client.0.smithi157.stdout:removed './114/14' 2023-10-06T04:02:41.250 INFO:tasks.workunit.client.0.smithi157.stdout:removed './114/18' 2023-10-06T04:02:41.250 INFO:tasks.workunit.client.0.smithi157.stdout:removed './114/13' 2023-10-06T04:02:41.250 INFO:tasks.workunit.client.0.smithi157.stdout:removed './114/33' 2023-10-06T04:02:41.251 INFO:tasks.workunit.client.0.smithi157.stdout:removed './114/17' 2023-10-06T04:02:41.251 INFO:tasks.workunit.client.0.smithi157.stdout:removed './114/35' 2023-10-06T04:02:41.251 INFO:tasks.workunit.client.0.smithi157.stdout:removed './114/37' 2023-10-06T04:02:41.251 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './114' 2023-10-06T04:02:41.251 INFO:tasks.workunit.client.0.smithi157.stdout:removed './115/19' 2023-10-06T04:02:41.251 INFO:tasks.workunit.client.0.smithi157.stdout:removed './115/15' 2023-10-06T04:02:41.252 INFO:tasks.workunit.client.0.smithi157.stdout:removed './115/31' 2023-10-06T04:02:41.252 INFO:tasks.workunit.client.0.smithi157.stdout:removed './115/36' 2023-10-06T04:02:41.252 INFO:tasks.workunit.client.0.smithi157.stdout:removed './115/20' 2023-10-06T04:02:41.252 INFO:tasks.workunit.client.0.smithi157.stdout:removed './115/12' 2023-10-06T04:02:41.252 INFO:tasks.workunit.client.0.smithi157.stdout:removed './115/39' 2023-10-06T04:02:41.252 INFO:tasks.workunit.client.0.smithi157.stdout:removed './115/10' 2023-10-06T04:02:41.252 INFO:tasks.workunit.client.0.smithi157.stdout:removed './115/34' 2023-10-06T04:02:41.252 INFO:tasks.workunit.client.0.smithi157.stdout:removed './115/38' 2023-10-06T04:02:41.252 INFO:tasks.workunit.client.0.smithi157.stdout:removed './115/11' 2023-10-06T04:02:41.253 INFO:tasks.workunit.client.0.smithi157.stdout:removed './115/16' 2023-10-06T04:02:41.253 INFO:tasks.workunit.client.0.smithi157.stdout:removed './115/32' 2023-10-06T04:02:41.253 INFO:tasks.workunit.client.0.smithi157.stdout:removed './115/30' 2023-10-06T04:02:41.253 INFO:tasks.workunit.client.0.smithi157.stdout:removed './115/14' 2023-10-06T04:02:41.253 INFO:tasks.workunit.client.0.smithi157.stdout:removed './115/18' 2023-10-06T04:02:41.253 INFO:tasks.workunit.client.0.smithi157.stdout:removed './115/13' 2023-10-06T04:02:41.253 INFO:tasks.workunit.client.0.smithi157.stdout:removed './115/33' 2023-10-06T04:02:41.253 INFO:tasks.workunit.client.0.smithi157.stdout:removed './115/17' 2023-10-06T04:02:41.253 INFO:tasks.workunit.client.0.smithi157.stdout:removed './115/35' 2023-10-06T04:02:41.253 INFO:tasks.workunit.client.0.smithi157.stdout:removed './115/37' 2023-10-06T04:02:41.254 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './115' 2023-10-06T04:02:41.254 INFO:tasks.workunit.client.0.smithi157.stdout:removed './116/19' 2023-10-06T04:02:41.254 INFO:tasks.workunit.client.0.smithi157.stdout:removed './116/15' 2023-10-06T04:02:41.254 INFO:tasks.workunit.client.0.smithi157.stdout:removed './116/31' 2023-10-06T04:02:41.254 INFO:tasks.workunit.client.0.smithi157.stdout:removed './116/36' 2023-10-06T04:02:41.254 INFO:tasks.workunit.client.0.smithi157.stdout:removed './116/20' 2023-10-06T04:02:41.254 INFO:tasks.workunit.client.0.smithi157.stdout:removed './116/12' 2023-10-06T04:02:41.254 INFO:tasks.workunit.client.0.smithi157.stdout:removed './116/39' 2023-10-06T04:02:41.254 INFO:tasks.workunit.client.0.smithi157.stdout:removed './116/10' 2023-10-06T04:02:41.254 INFO:tasks.workunit.client.0.smithi157.stdout:removed './116/34' 2023-10-06T04:02:41.255 INFO:tasks.workunit.client.0.smithi157.stdout:removed './116/38' 2023-10-06T04:02:41.255 INFO:tasks.workunit.client.0.smithi157.stdout:removed './116/11' 2023-10-06T04:02:41.255 INFO:tasks.workunit.client.0.smithi157.stdout:removed './116/16' 2023-10-06T04:02:41.255 INFO:tasks.workunit.client.0.smithi157.stdout:removed './116/32' 2023-10-06T04:02:41.255 INFO:tasks.workunit.client.0.smithi157.stdout:removed './116/30' 2023-10-06T04:02:41.255 INFO:tasks.workunit.client.0.smithi157.stdout:removed './116/14' 2023-10-06T04:02:41.255 INFO:tasks.workunit.client.0.smithi157.stdout:removed './116/18' 2023-10-06T04:02:41.255 INFO:tasks.workunit.client.0.smithi157.stdout:removed './116/13' 2023-10-06T04:02:41.255 INFO:tasks.workunit.client.0.smithi157.stdout:removed './116/33' 2023-10-06T04:02:41.255 INFO:tasks.workunit.client.0.smithi157.stdout:removed './116/17' 2023-10-06T04:02:41.256 INFO:tasks.workunit.client.0.smithi157.stdout:removed './116/35' 2023-10-06T04:02:41.256 INFO:tasks.workunit.client.0.smithi157.stdout:removed './116/37' 2023-10-06T04:02:41.256 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './116' 2023-10-06T04:02:41.256 INFO:tasks.workunit.client.0.smithi157.stdout:removed './117/19' 2023-10-06T04:02:41.256 INFO:tasks.workunit.client.0.smithi157.stdout:removed './117/15' 2023-10-06T04:02:41.256 INFO:tasks.workunit.client.0.smithi157.stdout:removed './117/31' 2023-10-06T04:02:41.256 INFO:tasks.workunit.client.0.smithi157.stdout:removed './117/36' 2023-10-06T04:02:41.256 INFO:tasks.workunit.client.0.smithi157.stdout:removed './117/20' 2023-10-06T04:02:41.256 INFO:tasks.workunit.client.0.smithi157.stdout:removed './117/12' 2023-10-06T04:02:41.256 INFO:tasks.workunit.client.0.smithi157.stdout:removed './117/39' 2023-10-06T04:02:41.257 INFO:tasks.workunit.client.0.smithi157.stdout:removed './117/10' 2023-10-06T04:02:41.257 INFO:tasks.workunit.client.0.smithi157.stdout:removed './117/34' 2023-10-06T04:02:41.257 INFO:tasks.workunit.client.0.smithi157.stdout:removed './117/38' 2023-10-06T04:02:41.257 INFO:tasks.workunit.client.0.smithi157.stdout:removed './117/11' 2023-10-06T04:02:41.257 INFO:tasks.workunit.client.0.smithi157.stdout:removed './117/16' 2023-10-06T04:02:41.257 INFO:tasks.workunit.client.0.smithi157.stdout:removed './117/32' 2023-10-06T04:02:41.257 INFO:tasks.workunit.client.0.smithi157.stdout:removed './117/30' 2023-10-06T04:02:41.257 INFO:tasks.workunit.client.0.smithi157.stdout:removed './117/14' 2023-10-06T04:02:41.257 INFO:tasks.workunit.client.0.smithi157.stdout:removed './117/18' 2023-10-06T04:02:41.258 INFO:tasks.workunit.client.0.smithi157.stdout:removed './117/13' 2023-10-06T04:02:41.258 INFO:tasks.workunit.client.0.smithi157.stdout:removed './117/33' 2023-10-06T04:02:41.258 INFO:tasks.workunit.client.0.smithi157.stdout:removed './117/17' 2023-10-06T04:02:41.258 INFO:tasks.workunit.client.0.smithi157.stdout:removed './117/35' 2023-10-06T04:02:41.258 INFO:tasks.workunit.client.0.smithi157.stdout:removed './117/37' 2023-10-06T04:02:41.258 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './117' 2023-10-06T04:02:41.258 INFO:tasks.workunit.client.0.smithi157.stdout:removed './118/19' 2023-10-06T04:02:41.258 INFO:tasks.workunit.client.0.smithi157.stdout:removed './118/15' 2023-10-06T04:02:41.258 INFO:tasks.workunit.client.0.smithi157.stdout:removed './118/31' 2023-10-06T04:02:41.259 INFO:tasks.workunit.client.0.smithi157.stdout:removed './118/36' 2023-10-06T04:02:41.259 INFO:tasks.workunit.client.0.smithi157.stdout:removed './118/20' 2023-10-06T04:02:41.259 INFO:tasks.workunit.client.0.smithi157.stdout:removed './118/12' 2023-10-06T04:02:41.259 INFO:tasks.workunit.client.0.smithi157.stdout:removed './118/39' 2023-10-06T04:02:41.259 INFO:tasks.workunit.client.0.smithi157.stdout:removed './118/10' 2023-10-06T04:02:41.259 INFO:tasks.workunit.client.0.smithi157.stdout:removed './118/34' 2023-10-06T04:02:41.259 INFO:tasks.workunit.client.0.smithi157.stdout:removed './118/38' 2023-10-06T04:02:41.259 INFO:tasks.workunit.client.0.smithi157.stdout:removed './118/11' 2023-10-06T04:02:41.259 INFO:tasks.workunit.client.0.smithi157.stdout:removed './118/16' 2023-10-06T04:02:41.259 INFO:tasks.workunit.client.0.smithi157.stdout:removed './118/32' 2023-10-06T04:02:41.260 INFO:tasks.workunit.client.0.smithi157.stdout:removed './118/30' 2023-10-06T04:02:41.260 INFO:tasks.workunit.client.0.smithi157.stdout:removed './118/14' 2023-10-06T04:02:41.260 INFO:tasks.workunit.client.0.smithi157.stdout:removed './118/18' 2023-10-06T04:02:41.260 INFO:tasks.workunit.client.0.smithi157.stdout:removed './118/13' 2023-10-06T04:02:41.260 INFO:tasks.workunit.client.0.smithi157.stdout:removed './118/33' 2023-10-06T04:02:41.260 INFO:tasks.workunit.client.0.smithi157.stdout:removed './118/17' 2023-10-06T04:02:41.260 INFO:tasks.workunit.client.0.smithi157.stdout:removed './118/35' 2023-10-06T04:02:41.260 INFO:tasks.workunit.client.0.smithi157.stdout:removed './118/37' 2023-10-06T04:02:41.260 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './118' 2023-10-06T04:02:41.261 INFO:tasks.workunit.client.0.smithi157.stdout:removed './119/19' 2023-10-06T04:02:41.261 INFO:tasks.workunit.client.0.smithi157.stdout:removed './119/15' 2023-10-06T04:02:41.261 INFO:tasks.workunit.client.0.smithi157.stdout:removed './119/31' 2023-10-06T04:02:41.261 INFO:tasks.workunit.client.0.smithi157.stdout:removed './119/36' 2023-10-06T04:02:41.261 INFO:tasks.workunit.client.0.smithi157.stdout:removed './119/20' 2023-10-06T04:02:41.261 INFO:tasks.workunit.client.0.smithi157.stdout:removed './119/12' 2023-10-06T04:02:42.309 INFO:tasks.workunit.client.0.smithi157.stdout:removed './119/39' 2023-10-06T04:02:42.309 INFO:tasks.workunit.client.0.smithi157.stdout:removed './119/10' 2023-10-06T04:02:42.309 INFO:tasks.workunit.client.0.smithi157.stdout:removed './119/34' 2023-10-06T04:02:42.309 INFO:tasks.workunit.client.0.smithi157.stdout:removed './119/38' 2023-10-06T04:02:42.310 INFO:tasks.workunit.client.0.smithi157.stdout:removed './119/11' 2023-10-06T04:02:42.310 INFO:tasks.workunit.client.0.smithi157.stdout:removed './119/16' 2023-10-06T04:02:42.310 INFO:tasks.workunit.client.0.smithi157.stdout:removed './119/32' 2023-10-06T04:02:42.310 INFO:tasks.workunit.client.0.smithi157.stdout:removed './119/30' 2023-10-06T04:02:42.310 INFO:tasks.workunit.client.0.smithi157.stdout:removed './119/14' 2023-10-06T04:02:42.310 INFO:tasks.workunit.client.0.smithi157.stdout:removed './119/18' 2023-10-06T04:02:42.310 INFO:tasks.workunit.client.0.smithi157.stdout:removed './119/13' 2023-10-06T04:02:42.310 INFO:tasks.workunit.client.0.smithi157.stdout:removed './119/33' 2023-10-06T04:02:42.310 INFO:tasks.workunit.client.0.smithi157.stdout:removed './119/17' 2023-10-06T04:02:42.310 INFO:tasks.workunit.client.0.smithi157.stdout:removed './119/35' 2023-10-06T04:02:42.311 INFO:tasks.workunit.client.0.smithi157.stdout:removed './119/37' 2023-10-06T04:02:42.311 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './119' 2023-10-06T04:02:42.311 INFO:tasks.workunit.client.0.smithi157.stdout:removed './120/19' 2023-10-06T04:02:42.311 INFO:tasks.workunit.client.0.smithi157.stdout:removed './120/15' 2023-10-06T04:02:42.311 INFO:tasks.workunit.client.0.smithi157.stdout:removed './120/31' 2023-10-06T04:02:42.311 INFO:tasks.workunit.client.0.smithi157.stdout:removed './120/36' 2023-10-06T04:02:42.311 INFO:tasks.workunit.client.0.smithi157.stdout:removed './120/20' 2023-10-06T04:02:42.311 INFO:tasks.workunit.client.0.smithi157.stdout:removed './120/12' 2023-10-06T04:02:42.311 INFO:tasks.workunit.client.0.smithi157.stdout:removed './120/39' 2023-10-06T04:02:42.311 INFO:tasks.workunit.client.0.smithi157.stdout:removed './120/10' 2023-10-06T04:02:42.312 INFO:tasks.workunit.client.0.smithi157.stdout:removed './120/34' 2023-10-06T04:02:42.312 INFO:tasks.workunit.client.0.smithi157.stdout:removed './120/38' 2023-10-06T04:02:42.312 INFO:tasks.workunit.client.0.smithi157.stdout:removed './120/11' 2023-10-06T04:02:42.312 INFO:tasks.workunit.client.0.smithi157.stdout:removed './120/16' 2023-10-06T04:02:42.312 INFO:tasks.workunit.client.0.smithi157.stdout:removed './120/32' 2023-10-06T04:02:42.312 INFO:tasks.workunit.client.0.smithi157.stdout:removed './120/30' 2023-10-06T04:02:42.312 INFO:tasks.workunit.client.0.smithi157.stdout:removed './120/14' 2023-10-06T04:02:42.312 INFO:tasks.workunit.client.0.smithi157.stdout:removed './120/18' 2023-10-06T04:02:42.312 INFO:tasks.workunit.client.0.smithi157.stdout:removed './120/13' 2023-10-06T04:02:42.312 INFO:tasks.workunit.client.0.smithi157.stdout:removed './120/33' 2023-10-06T04:02:42.313 INFO:tasks.workunit.client.0.smithi157.stdout:removed './120/17' 2023-10-06T04:02:42.313 INFO:tasks.workunit.client.0.smithi157.stdout:removed './120/35' 2023-10-06T04:02:42.313 INFO:tasks.workunit.client.0.smithi157.stdout:removed './120/37' 2023-10-06T04:02:42.313 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './120' 2023-10-06T04:02:42.313 INFO:tasks.workunit.client.0.smithi157.stdout:removed './121/19' 2023-10-06T04:02:42.313 INFO:tasks.workunit.client.0.smithi157.stdout:removed './121/15' 2023-10-06T04:02:42.313 INFO:tasks.workunit.client.0.smithi157.stdout:removed './121/31' 2023-10-06T04:02:42.313 INFO:tasks.workunit.client.0.smithi157.stdout:removed './121/36' 2023-10-06T04:02:42.313 INFO:tasks.workunit.client.0.smithi157.stdout:removed './121/20' 2023-10-06T04:02:42.313 INFO:tasks.workunit.client.0.smithi157.stdout:removed './121/12' 2023-10-06T04:02:42.313 INFO:tasks.workunit.client.0.smithi157.stdout:removed './121/39' 2023-10-06T04:02:42.314 INFO:tasks.workunit.client.0.smithi157.stdout:removed './121/10' 2023-10-06T04:02:42.314 INFO:tasks.workunit.client.0.smithi157.stdout:removed './121/34' 2023-10-06T04:02:42.314 INFO:tasks.workunit.client.0.smithi157.stdout:removed './121/38' 2023-10-06T04:02:42.314 INFO:tasks.workunit.client.0.smithi157.stdout:removed './121/11' 2023-10-06T04:02:42.314 INFO:tasks.workunit.client.0.smithi157.stdout:removed './121/16' 2023-10-06T04:02:42.314 INFO:tasks.workunit.client.0.smithi157.stdout:removed './121/32' 2023-10-06T04:02:42.314 INFO:tasks.workunit.client.0.smithi157.stdout:removed './121/30' 2023-10-06T04:02:42.314 INFO:tasks.workunit.client.0.smithi157.stdout:removed './121/14' 2023-10-06T04:02:42.314 INFO:tasks.workunit.client.0.smithi157.stdout:removed './121/18' 2023-10-06T04:02:42.314 INFO:tasks.workunit.client.0.smithi157.stdout:removed './121/13' 2023-10-06T04:02:42.315 INFO:tasks.workunit.client.0.smithi157.stdout:removed './121/33' 2023-10-06T04:02:42.315 INFO:tasks.workunit.client.0.smithi157.stdout:removed './121/17' 2023-10-06T04:02:42.315 INFO:tasks.workunit.client.0.smithi157.stdout:removed './121/35' 2023-10-06T04:02:42.315 INFO:tasks.workunit.client.0.smithi157.stdout:removed './121/37' 2023-10-06T04:02:42.315 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './121' 2023-10-06T04:02:42.315 INFO:tasks.workunit.client.0.smithi157.stdout:removed './122/19' 2023-10-06T04:02:42.315 INFO:tasks.workunit.client.0.smithi157.stdout:removed './122/15' 2023-10-06T04:02:42.315 INFO:tasks.workunit.client.0.smithi157.stdout:removed './122/31' 2023-10-06T04:02:42.315 INFO:tasks.workunit.client.0.smithi157.stdout:removed './122/36' 2023-10-06T04:02:42.315 INFO:tasks.workunit.client.0.smithi157.stdout:removed './122/20' 2023-10-06T04:02:42.316 INFO:tasks.workunit.client.0.smithi157.stdout:removed './122/12' 2023-10-06T04:02:42.316 INFO:tasks.workunit.client.0.smithi157.stdout:removed './122/39' 2023-10-06T04:02:42.316 INFO:tasks.workunit.client.0.smithi157.stdout:removed './122/10' 2023-10-06T04:02:42.316 INFO:tasks.workunit.client.0.smithi157.stdout:removed './122/34' 2023-10-06T04:02:42.316 INFO:tasks.workunit.client.0.smithi157.stdout:removed './122/38' 2023-10-06T04:02:42.316 INFO:tasks.workunit.client.0.smithi157.stdout:removed './122/11' 2023-10-06T04:02:42.316 INFO:tasks.workunit.client.0.smithi157.stdout:removed './122/16' 2023-10-06T04:02:42.316 INFO:tasks.workunit.client.0.smithi157.stdout:removed './122/32' 2023-10-06T04:02:42.316 INFO:tasks.workunit.client.0.smithi157.stdout:removed './122/30' 2023-10-06T04:02:42.316 INFO:tasks.workunit.client.0.smithi157.stdout:removed './122/14' 2023-10-06T04:02:42.317 INFO:tasks.workunit.client.0.smithi157.stdout:removed './122/18' 2023-10-06T04:02:42.317 INFO:tasks.workunit.client.0.smithi157.stdout:removed './122/13' 2023-10-06T04:02:42.317 INFO:tasks.workunit.client.0.smithi157.stdout:removed './122/33' 2023-10-06T04:02:42.317 INFO:tasks.workunit.client.0.smithi157.stdout:removed './122/17' 2023-10-06T04:02:42.317 INFO:tasks.workunit.client.0.smithi157.stdout:removed './122/35' 2023-10-06T04:02:42.317 INFO:tasks.workunit.client.0.smithi157.stdout:removed './122/37' 2023-10-06T04:02:42.317 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './122' 2023-10-06T04:02:42.317 INFO:tasks.workunit.client.0.smithi157.stdout:removed './123/19' 2023-10-06T04:02:42.317 INFO:tasks.workunit.client.0.smithi157.stdout:removed './123/15' 2023-10-06T04:02:42.317 INFO:tasks.workunit.client.0.smithi157.stdout:removed './123/31' 2023-10-06T04:02:42.317 INFO:tasks.workunit.client.0.smithi157.stdout:removed './123/36' 2023-10-06T04:02:42.318 INFO:tasks.workunit.client.0.smithi157.stdout:removed './123/20' 2023-10-06T04:02:42.318 INFO:tasks.workunit.client.0.smithi157.stdout:removed './123/12' 2023-10-06T04:02:42.318 INFO:tasks.workunit.client.0.smithi157.stdout:removed './123/39' 2023-10-06T04:02:42.318 INFO:tasks.workunit.client.0.smithi157.stdout:removed './123/10' 2023-10-06T04:02:42.318 INFO:tasks.workunit.client.0.smithi157.stdout:removed './123/34' 2023-10-06T04:02:42.318 INFO:tasks.workunit.client.0.smithi157.stdout:removed './123/38' 2023-10-06T04:02:42.318 INFO:tasks.workunit.client.0.smithi157.stdout:removed './123/11' 2023-10-06T04:02:42.318 INFO:tasks.workunit.client.0.smithi157.stdout:removed './123/16' 2023-10-06T04:02:42.318 INFO:tasks.workunit.client.0.smithi157.stdout:removed './123/32' 2023-10-06T04:02:42.318 INFO:tasks.workunit.client.0.smithi157.stdout:removed './123/30' 2023-10-06T04:02:42.319 INFO:tasks.workunit.client.0.smithi157.stdout:removed './123/14' 2023-10-06T04:02:42.319 INFO:tasks.workunit.client.0.smithi157.stdout:removed './123/18' 2023-10-06T04:02:42.319 INFO:tasks.workunit.client.0.smithi157.stdout:removed './123/13' 2023-10-06T04:02:42.319 INFO:tasks.workunit.client.0.smithi157.stdout:removed './123/33' 2023-10-06T04:02:42.319 INFO:tasks.workunit.client.0.smithi157.stdout:removed './123/17' 2023-10-06T04:02:42.319 INFO:tasks.workunit.client.0.smithi157.stdout:removed './123/35' 2023-10-06T04:02:42.319 INFO:tasks.workunit.client.0.smithi157.stdout:removed './123/37' 2023-10-06T04:02:42.319 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './123' 2023-10-06T04:02:42.319 INFO:tasks.workunit.client.0.smithi157.stdout:removed './124/19' 2023-10-06T04:02:42.319 INFO:tasks.workunit.client.0.smithi157.stdout:removed './124/15' 2023-10-06T04:02:42.320 INFO:tasks.workunit.client.0.smithi157.stdout:removed './124/31' 2023-10-06T04:02:42.320 INFO:tasks.workunit.client.0.smithi157.stdout:removed './124/36' 2023-10-06T04:02:42.320 INFO:tasks.workunit.client.0.smithi157.stdout:removed './124/20' 2023-10-06T04:02:42.320 INFO:tasks.workunit.client.0.smithi157.stdout:removed './124/12' 2023-10-06T04:02:42.320 INFO:tasks.workunit.client.0.smithi157.stdout:removed './124/39' 2023-10-06T04:02:42.320 INFO:tasks.workunit.client.0.smithi157.stdout:removed './124/10' 2023-10-06T04:02:42.320 INFO:tasks.workunit.client.0.smithi157.stdout:removed './124/34' 2023-10-06T04:02:42.320 INFO:tasks.workunit.client.0.smithi157.stdout:removed './124/38' 2023-10-06T04:02:42.320 INFO:tasks.workunit.client.0.smithi157.stdout:removed './124/11' 2023-10-06T04:02:42.320 INFO:tasks.workunit.client.0.smithi157.stdout:removed './124/16' 2023-10-06T04:02:42.321 INFO:tasks.workunit.client.0.smithi157.stdout:removed './124/32' 2023-10-06T04:02:42.321 INFO:tasks.workunit.client.0.smithi157.stdout:removed './124/30' 2023-10-06T04:02:42.321 INFO:tasks.workunit.client.0.smithi157.stdout:removed './124/14' 2023-10-06T04:02:42.321 INFO:tasks.workunit.client.0.smithi157.stdout:removed './124/18' 2023-10-06T04:02:42.321 INFO:tasks.workunit.client.0.smithi157.stdout:removed './124/13' 2023-10-06T04:02:42.321 INFO:tasks.workunit.client.0.smithi157.stdout:removed './124/33' 2023-10-06T04:02:42.321 INFO:tasks.workunit.client.0.smithi157.stdout:removed './124/17' 2023-10-06T04:02:42.321 INFO:tasks.workunit.client.0.smithi157.stdout:removed './124/35' 2023-10-06T04:02:42.321 INFO:tasks.workunit.client.0.smithi157.stdout:removed './124/37' 2023-10-06T04:02:42.321 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './124' 2023-10-06T04:02:42.322 INFO:tasks.workunit.client.0.smithi157.stdout:removed './125/19' 2023-10-06T04:02:42.322 INFO:tasks.workunit.client.0.smithi157.stdout:removed './125/15' 2023-10-06T04:02:42.322 INFO:tasks.workunit.client.0.smithi157.stdout:removed './125/31' 2023-10-06T04:02:42.322 INFO:tasks.workunit.client.0.smithi157.stdout:removed './125/36' 2023-10-06T04:02:42.322 INFO:tasks.workunit.client.0.smithi157.stdout:removed './125/20' 2023-10-06T04:02:42.322 INFO:tasks.workunit.client.0.smithi157.stdout:removed './125/12' 2023-10-06T04:02:42.322 INFO:tasks.workunit.client.0.smithi157.stdout:removed './125/39' 2023-10-06T04:02:42.322 INFO:tasks.workunit.client.0.smithi157.stdout:removed './125/10' 2023-10-06T04:02:42.322 INFO:tasks.workunit.client.0.smithi157.stdout:removed './125/34' 2023-10-06T04:02:42.323 INFO:tasks.workunit.client.0.smithi157.stdout:removed './125/38' 2023-10-06T04:02:42.323 INFO:tasks.workunit.client.0.smithi157.stdout:removed './125/11' 2023-10-06T04:02:42.323 INFO:tasks.workunit.client.0.smithi157.stdout:removed './125/16' 2023-10-06T04:02:42.323 INFO:tasks.workunit.client.0.smithi157.stdout:removed './125/32' 2023-10-06T04:02:42.323 INFO:tasks.workunit.client.0.smithi157.stdout:removed './125/30' 2023-10-06T04:02:42.323 INFO:tasks.workunit.client.0.smithi157.stdout:removed './125/14' 2023-10-06T04:02:42.323 INFO:tasks.workunit.client.0.smithi157.stdout:removed './125/18' 2023-10-06T04:02:42.323 INFO:tasks.workunit.client.0.smithi157.stdout:removed './125/13' 2023-10-06T04:02:42.323 INFO:tasks.workunit.client.0.smithi157.stdout:removed './125/33' 2023-10-06T04:02:42.323 INFO:tasks.workunit.client.0.smithi157.stdout:removed './125/17' 2023-10-06T04:02:42.323 INFO:tasks.workunit.client.0.smithi157.stdout:removed './125/35' 2023-10-06T04:02:42.324 INFO:tasks.workunit.client.0.smithi157.stdout:removed './125/37' 2023-10-06T04:02:42.324 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './125' 2023-10-06T04:02:42.324 INFO:tasks.workunit.client.0.smithi157.stdout:removed './126/19' 2023-10-06T04:02:42.324 INFO:tasks.workunit.client.0.smithi157.stdout:removed './126/15' 2023-10-06T04:02:42.324 INFO:tasks.workunit.client.0.smithi157.stdout:removed './126/31' 2023-10-06T04:02:42.324 INFO:tasks.workunit.client.0.smithi157.stdout:removed './126/36' 2023-10-06T04:02:42.324 INFO:tasks.workunit.client.0.smithi157.stdout:removed './126/20' 2023-10-06T04:02:42.324 INFO:tasks.workunit.client.0.smithi157.stdout:removed './126/12' 2023-10-06T04:02:42.324 INFO:tasks.workunit.client.0.smithi157.stdout:removed './126/39' 2023-10-06T04:02:42.325 INFO:tasks.workunit.client.0.smithi157.stdout:removed './126/10' 2023-10-06T04:02:42.325 INFO:tasks.workunit.client.0.smithi157.stdout:removed './126/34' 2023-10-06T04:02:42.325 INFO:tasks.workunit.client.0.smithi157.stdout:removed './126/38' 2023-10-06T04:02:42.325 INFO:tasks.workunit.client.0.smithi157.stdout:removed './126/11' 2023-10-06T04:02:42.325 INFO:tasks.workunit.client.0.smithi157.stdout:removed './126/16' 2023-10-06T04:02:42.325 INFO:tasks.workunit.client.0.smithi157.stdout:removed './126/32' 2023-10-06T04:02:42.325 INFO:tasks.workunit.client.0.smithi157.stdout:removed './126/30' 2023-10-06T04:02:42.325 INFO:tasks.workunit.client.0.smithi157.stdout:removed './126/14' 2023-10-06T04:02:42.325 INFO:tasks.workunit.client.0.smithi157.stdout:removed './126/18' 2023-10-06T04:02:42.325 INFO:tasks.workunit.client.0.smithi157.stdout:removed './126/13' 2023-10-06T04:02:42.326 INFO:tasks.workunit.client.0.smithi157.stdout:removed './126/33' 2023-10-06T04:02:42.326 INFO:tasks.workunit.client.0.smithi157.stdout:removed './126/17' 2023-10-06T04:02:42.326 INFO:tasks.workunit.client.0.smithi157.stdout:removed './126/35' 2023-10-06T04:02:42.326 INFO:tasks.workunit.client.0.smithi157.stdout:removed './126/37' 2023-10-06T04:02:42.326 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './126' 2023-10-06T04:02:42.326 INFO:tasks.workunit.client.0.smithi157.stdout:removed './127/19' 2023-10-06T04:02:42.326 INFO:tasks.workunit.client.0.smithi157.stdout:removed './127/15' 2023-10-06T04:02:42.326 INFO:tasks.workunit.client.0.smithi157.stdout:removed './127/31' 2023-10-06T04:02:42.326 INFO:tasks.workunit.client.0.smithi157.stdout:removed './127/36' 2023-10-06T04:02:42.326 INFO:tasks.workunit.client.0.smithi157.stdout:removed './127/20' 2023-10-06T04:02:42.327 INFO:tasks.workunit.client.0.smithi157.stdout:removed './127/12' 2023-10-06T04:02:42.327 INFO:tasks.workunit.client.0.smithi157.stdout:removed './127/39' 2023-10-06T04:02:42.327 INFO:tasks.workunit.client.0.smithi157.stdout:removed './127/10' 2023-10-06T04:02:42.327 INFO:tasks.workunit.client.0.smithi157.stdout:removed './127/34' 2023-10-06T04:02:42.327 INFO:tasks.workunit.client.0.smithi157.stdout:removed './127/38' 2023-10-06T04:02:42.327 INFO:tasks.workunit.client.0.smithi157.stdout:removed './127/11' 2023-10-06T04:02:42.327 INFO:tasks.workunit.client.0.smithi157.stdout:removed './127/16' 2023-10-06T04:02:42.327 INFO:tasks.workunit.client.0.smithi157.stdout:removed './127/32' 2023-10-06T04:02:42.327 INFO:tasks.workunit.client.0.smithi157.stdout:removed './127/30' 2023-10-06T04:02:42.327 INFO:tasks.workunit.client.0.smithi157.stdout:removed './127/14' 2023-10-06T04:02:42.328 INFO:tasks.workunit.client.0.smithi157.stdout:removed './127/18' 2023-10-06T04:02:42.328 INFO:tasks.workunit.client.0.smithi157.stdout:removed './127/13' 2023-10-06T04:02:42.328 INFO:tasks.workunit.client.0.smithi157.stdout:removed './127/33' 2023-10-06T04:02:42.328 INFO:tasks.workunit.client.0.smithi157.stdout:removed './127/17' 2023-10-06T04:02:42.328 INFO:tasks.workunit.client.0.smithi157.stdout:removed './127/35' 2023-10-06T04:02:42.328 INFO:tasks.workunit.client.0.smithi157.stdout:removed './127/37' 2023-10-06T04:02:42.328 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './127' 2023-10-06T04:02:42.328 INFO:tasks.workunit.client.0.smithi157.stdout:removed './128/19' 2023-10-06T04:02:42.328 INFO:tasks.workunit.client.0.smithi157.stdout:removed './128/15' 2023-10-06T04:02:42.328 INFO:tasks.workunit.client.0.smithi157.stdout:removed './128/31' 2023-10-06T04:02:42.329 INFO:tasks.workunit.client.0.smithi157.stdout:removed './128/36' 2023-10-06T04:02:42.329 INFO:tasks.workunit.client.0.smithi157.stdout:removed './128/20' 2023-10-06T04:02:42.329 INFO:tasks.workunit.client.0.smithi157.stdout:removed './128/12' 2023-10-06T04:02:42.329 INFO:tasks.workunit.client.0.smithi157.stdout:removed './128/39' 2023-10-06T04:02:42.329 INFO:tasks.workunit.client.0.smithi157.stdout:removed './128/10' 2023-10-06T04:02:42.329 INFO:tasks.workunit.client.0.smithi157.stdout:removed './128/34' 2023-10-06T04:02:42.329 INFO:tasks.workunit.client.0.smithi157.stdout:removed './128/38' 2023-10-06T04:02:42.329 INFO:tasks.workunit.client.0.smithi157.stdout:removed './128/11' 2023-10-06T04:02:42.329 INFO:tasks.workunit.client.0.smithi157.stdout:removed './128/16' 2023-10-06T04:02:42.329 INFO:tasks.workunit.client.0.smithi157.stdout:removed './128/32' 2023-10-06T04:02:42.330 INFO:tasks.workunit.client.0.smithi157.stdout:removed './128/30' 2023-10-06T04:02:42.330 INFO:tasks.workunit.client.0.smithi157.stdout:removed './128/14' 2023-10-06T04:02:42.330 INFO:tasks.workunit.client.0.smithi157.stdout:removed './128/18' 2023-10-06T04:02:42.330 INFO:tasks.workunit.client.0.smithi157.stdout:removed './128/13' 2023-10-06T04:02:42.330 INFO:tasks.workunit.client.0.smithi157.stdout:removed './128/33' 2023-10-06T04:02:42.330 INFO:tasks.workunit.client.0.smithi157.stdout:removed './128/17' 2023-10-06T04:02:42.330 INFO:tasks.workunit.client.0.smithi157.stdout:removed './128/35' 2023-10-06T04:02:43.074 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:02:43.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.187+0000 7f53e753b700 1 Processor -- start 2023-10-06T04:02:43.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.188+0000 7f53e753b700 1 -- start start 2023-10-06T04:02:43.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.188+0000 7f53e753b700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f53e006cbb0 0x7f53e006d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:43.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.188+0000 7f53e753b700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f53e0150410 0x7f53e006da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:43.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.188+0000 7f53e753b700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e00c9780 con 0x7f53e006c1f0 2023-10-06T04:02:43.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.188+0000 7f53e753b700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e00c8c40 con 0x7f53e0150410 2023-10-06T04:02:43.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.188+0000 7f53e753b700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e006e0c0 con 0x7f53e006cbb0 2023-10-06T04:02:43.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.188+0000 7f53e5ad8700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f53e0150410 0x7f53e006da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:43.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.188+0000 7f53e4ad6700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f53e006c1f0 0x7f53e014fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:54940/0 (socket says 172.21.15.157:54940) 2023-10-06T04:02:43.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.188+0000 7f53e4ad6700 1 -- 172.21.15.157:0/2655345306 learned_addr learned my addr 172.21.15.157:0/2655345306 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:02:43.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.189+0000 7f53e52d7700 1 --2- 172.21.15.157:0/2655345306 >> v2:172.21.15.175:3300/0 conn(0x7f53e006cbb0 0x7f53e006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:43.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.189+0000 7f53e5ad8700 1 -- 172.21.15.157:0/2655345306 >> v2:172.21.15.175:3300/0 conn(0x7f53e006cbb0 msgr2=0x7f53e006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:43.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.189+0000 7f53e5ad8700 1 --2- 172.21.15.157:0/2655345306 >> v2:172.21.15.175:3300/0 conn(0x7f53e006cbb0 0x7f53e006d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:43.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.189+0000 7f53e5ad8700 1 -- 172.21.15.157:0/2655345306 >> v1:172.21.15.157:6789/0 conn(0x7f53e006c1f0 legacy=0x7f53e014fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:43.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.189+0000 7f53e5ad8700 1 -- 172.21.15.157:0/2655345306 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e00c8e50 con 0x7f53e0150410 2023-10-06T04:02:43.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.189+0000 7f53d7fff700 1 -- 172.21.15.157:0/2655345306 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1889508980 0 0) 0x7f53e00c9780 con 0x7f53e006c1f0 2023-10-06T04:02:43.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.189+0000 7f53e5ad8700 1 --2- 172.21.15.157:0/2655345306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53e0150410 0x7f53e006da70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f53e006fd70 tx=0x7f53d8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6489ba038a29401 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:43.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.189+0000 7f53d7fff700 1 -- 172.21.15.157:0/2655345306 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d800e070 con 0x7f53e0150410 2023-10-06T04:02:43.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.189+0000 7f53d7fff700 1 -- 172.21.15.157:0/2655345306 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d800bbf0 con 0x7f53e0150410 2023-10-06T04:02:43.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.189+0000 7f53d7fff700 1 -- 172.21.15.157:0/2655345306 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d8012430 con 0x7f53e0150410 2023-10-06T04:02:43.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.190+0000 7f53e753b700 1 -- 172.21.15.157:0/2655345306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53e0150410 msgr2=0x7f53e006da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:43.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.190+0000 7f53e753b700 1 --2- 172.21.15.157:0/2655345306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53e0150410 0x7f53e006da70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f53e006fd70 tx=0x7f53d8009780 comp rx=0 tx=0).stop 2023-10-06T04:02:43.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.190+0000 7f53e753b700 1 -- 172.21.15.157:0/2655345306 shutdown_connections 2023-10-06T04:02:43.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.190+0000 7f53e753b700 1 --2- 172.21.15.157:0/2655345306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53e0150410 0x7f53e006da70 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:43.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.190+0000 7f53e753b700 1 --2- 172.21.15.157:0/2655345306 >> v2:172.21.15.175:3300/0 conn(0x7f53e006cbb0 0x7f53e006d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:43.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.190+0000 7f53e753b700 1 -- 172.21.15.157:0/2655345306 >> 172.21.15.157:0/2655345306 conn(0x7f53e006f230 msgr2=0x7f53e006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:43.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.190+0000 7f53e753b700 1 -- 172.21.15.157:0/2655345306 shutdown_connections 2023-10-06T04:02:43.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.190+0000 7f53e753b700 1 -- 172.21.15.157:0/2655345306 wait complete. 2023-10-06T04:02:43.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.191+0000 7f53e753b700 1 Processor -- start 2023-10-06T04:02:43.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.191+0000 7f53e753b700 1 -- start start 2023-10-06T04:02:43.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.192+0000 7f53e753b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53e006c1f0 0x7f53e01589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:43.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.192+0000 7f53e753b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53e006cbb0 0x7f53e00cb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:43.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.192+0000 7f53e753b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53e0150410 0x7f53e00cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:43.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.192+0000 7f53e753b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e0156aa0 con 0x7f53e0150410 2023-10-06T04:02:43.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.192+0000 7f53e753b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e0168e30 con 0x7f53e006c1f0 2023-10-06T04:02:43.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.192+0000 7f53e753b700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e01690b0 con 0x7f53e006cbb0 2023-10-06T04:02:43.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.192+0000 7f53e5ad8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53e0150410 0x7f53e00cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:43.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.192+0000 7f53e5ad8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53e0150410 0x7f53e00cbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44586/0 (socket says 172.21.15.157:44586) 2023-10-06T04:02:43.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.192+0000 7f53e5ad8700 1 -- 172.21.15.157:0/456089312 learned_addr learned my addr 172.21.15.157:0/456089312 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:43.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.192+0000 7f53e52d7700 1 --2- 172.21.15.157:0/456089312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53e006c1f0 0x7f53e01589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:43.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.192+0000 7f53e4ad6700 1 --2- 172.21.15.157:0/456089312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53e006cbb0 0x7f53e00cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:43.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.192+0000 7f53e5ad8700 1 -- 172.21.15.157:0/456089312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53e006cbb0 msgr2=0x7f53e00cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:43.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.192+0000 7f53e5ad8700 1 --2- 172.21.15.157:0/456089312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53e006cbb0 0x7f53e00cb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:43.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.192+0000 7f53e5ad8700 1 -- 172.21.15.157:0/456089312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53e006c1f0 msgr2=0x7f53e01589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:43.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.192+0000 7f53e5ad8700 1 --2- 172.21.15.157:0/456089312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53e006c1f0 0x7f53e01589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:43.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.192+0000 7f53e5ad8700 1 -- 172.21.15.157:0/456089312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d8009a20 con 0x7f53e0150410 2023-10-06T04:02:43.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.192+0000 7f53e5ad8700 1 --2- 172.21.15.157:0/456089312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53e0150410 0x7f53e00cbc50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f53e00c3960 tx=0x7f53d8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:43.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.192+0000 7f53d57fa700 1 -- 172.21.15.157:0/456089312 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d801d630 con 0x7f53e0150410 2023-10-06T04:02:43.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.192+0000 7f53d57fa700 1 -- 172.21.15.157:0/456089312 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d801d7b0 con 0x7f53e0150410 2023-10-06T04:02:43.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.192+0000 7f53e753b700 1 -- 172.21.15.157:0/456089312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e0169390 con 0x7f53e0150410 2023-10-06T04:02:43.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.192+0000 7f53d57fa700 1 -- 172.21.15.157:0/456089312 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d801db70 con 0x7f53e0150410 2023-10-06T04:02:43.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.193+0000 7f53e753b700 1 -- 172.21.15.157:0/456089312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e0169950 con 0x7f53e0150410 2023-10-06T04:02:43.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.193+0000 7f53e753b700 1 -- 172.21.15.157:0/456089312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e00cb430 con 0x7f53e0150410 2023-10-06T04:02:43.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.194+0000 7f53d57fa700 1 -- 172.21.15.157:0/456089312 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f53d8024070 con 0x7f53e0150410 2023-10-06T04:02:43.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.194+0000 7f53d57fa700 1 --2- 172.21.15.157:0/456089312 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f53c4066f30 0x7f53c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:43.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.194+0000 7f53d57fa700 1 -- 172.21.15.157:0/456089312 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7252+0+0 (secure 0 0 0) 0x7f53d80874b0 con 0x7f53e0150410 2023-10-06T04:02:43.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.194+0000 7f53e52d7700 1 --2- 172.21.15.157:0/456089312 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f53c4066f30 0x7f53c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:43.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.195+0000 7f53e52d7700 1 --2- 172.21.15.157:0/456089312 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f53c4066f30 0x7f53c40693e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009d40 tx=0x7f53dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:43.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.198+0000 7f53d57fa700 1 -- 172.21.15.157:0/456089312 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f53d8054110 con 0x7f53e0150410 2023-10-06T04:02:43.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.432+0000 7f53e753b700 1 -- 172.21.15.157:0/456089312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53e00c9130 con 0x7f53e0150410 2023-10-06T04:02:43.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.432+0000 7f53d57fa700 1 -- 172.21.15.157:0/456089312 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+6182 (secure 0 0 0) 0x7f53d8057730 con 0x7f53e0150410 2023-10-06T04:02:43.434 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:02:43.434 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:02:37.923949+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2,3],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270},"failed":[],"damaged":[],"stopped":[4],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:stopping","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:02:43.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.434+0000 7f53e753b700 1 -- 172.21.15.157:0/456089312 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f53c4066f30 msgr2=0x7f53c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:43.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.434+0000 7f53e753b700 1 --2- 172.21.15.157:0/456089312 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f53c4066f30 0x7f53c40693e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009d40 tx=0x7f53dc008040 comp rx=0 tx=0).stop 2023-10-06T04:02:43.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.434+0000 7f53e753b700 1 -- 172.21.15.157:0/456089312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53e0150410 msgr2=0x7f53e00cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:43.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.434+0000 7f53e753b700 1 --2- 172.21.15.157:0/456089312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53e0150410 0x7f53e00cbc50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f53e00c3960 tx=0x7f53d8019000 comp rx=0 tx=0).stop 2023-10-06T04:02:43.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.434+0000 7f53e753b700 1 -- 172.21.15.157:0/456089312 shutdown_connections 2023-10-06T04:02:43.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.434+0000 7f53e753b700 1 --2- 172.21.15.157:0/456089312 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f53c4066f30 0x7f53c40693e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:43.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.434+0000 7f53e753b700 1 --2- 172.21.15.157:0/456089312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53e006c1f0 0x7f53e01589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:43.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.434+0000 7f53e753b700 1 --2- 172.21.15.157:0/456089312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53e0150410 0x7f53e00cbc50 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:43.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.434+0000 7f53e753b700 1 --2- 172.21.15.157:0/456089312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53e006cbb0 0x7f53e00cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:43.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.434+0000 7f53e753b700 1 -- 172.21.15.157:0/456089312 >> 172.21.15.157:0/456089312 conn(0x7f53e006f230 msgr2=0x7f53e014c950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:43.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.434+0000 7f53e753b700 1 -- 172.21.15.157:0/456089312 shutdown_connections 2023-10-06T04:02:43.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:43.434+0000 7f53e753b700 1 -- 172.21.15.157:0/456089312 wait complete. 2023-10-06T04:02:43.437 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 44 2023-10-06T04:02:43.448 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:02:44.336 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:02:44.338 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:02:45.449 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:02:45.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.575+0000 7f7a7f1b0700 1 Processor -- start 2023-10-06T04:02:45.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.575+0000 7f7a7f1b0700 1 -- start start 2023-10-06T04:02:45.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.576+0000 7f7a7f1b0700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a78150410 0x7f7a78155f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:45.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.576+0000 7f7a7f1b0700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a780c9780 con 0x7f7a7806cbb0 2023-10-06T04:02:45.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.576+0000 7f7a7f1b0700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a7806e0c0 con 0x7f7a7806dc40 2023-10-06T04:02:45.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.576+0000 7f7a7f1b0700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a780c8c40 con 0x7f7a78150410 2023-10-06T04:02:45.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.576+0000 7f7a7cf4c700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7a7806cbb0 0x7f7a7806d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:54946/0 (socket says 172.21.15.157:54946) 2023-10-06T04:02:45.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.576+0000 7f7a7cf4c700 1 -- 172.21.15.157:0/3043412161 learned_addr learned my addr 172.21.15.157:0/3043412161 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:02:45.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.576+0000 7f7a7d74d700 1 --2- 172.21.15.157:0/3043412161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a78150410 0x7f7a78155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:45.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.577+0000 7f7a6f7fe700 1 -- 172.21.15.157:0/3043412161 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 951938824 0 0) 0x7f7a780c9780 con 0x7f7a7806cbb0 2023-10-06T04:02:45.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.577+0000 7f7a6f7fe700 1 -- 172.21.15.157:0/3043412161 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a60003570 con 0x7f7a7806cbb0 2023-10-06T04:02:45.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.577+0000 7f7a6f7fe700 1 -- 172.21.15.157:0/3043412161 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2900968274 0 0) 0x7f7a7806e0c0 con 0x7f7a7806dc40 2023-10-06T04:02:45.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.577+0000 7f7a6f7fe700 1 -- 172.21.15.157:0/3043412161 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a780c9780 con 0x7f7a7806dc40 2023-10-06T04:02:45.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.577+0000 7f7a6f7fe700 1 -- 172.21.15.157:0/3043412161 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1817689433 0 0) 0x7f7a60003570 con 0x7f7a7806cbb0 2023-10-06T04:02:45.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.577+0000 7f7a7d74d700 1 -- 172.21.15.157:0/3043412161 >> v1:172.21.15.175:6789/0 conn(0x7f7a7806dc40 legacy=0x7f7a7814fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:45.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.577+0000 7f7a7d74d700 1 -- 172.21.15.157:0/3043412161 >> v1:172.21.15.157:6789/0 conn(0x7f7a7806cbb0 legacy=0x7f7a7806d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:45.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.577+0000 7f7a7d74d700 1 -- 172.21.15.157:0/3043412161 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a780c8e50 con 0x7f7a78150410 2023-10-06T04:02:45.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.577+0000 7f7a7d74d700 1 --2- 172.21.15.157:0/3043412161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a78150410 0x7f7a78155f70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7a70009e90 tx=0x7f7a70009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5353d62dfcd6cf5e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:45.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.577+0000 7f7a6f7fe700 1 -- 172.21.15.157:0/3043412161 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7000e070 con 0x7f7a78150410 2023-10-06T04:02:45.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.577+0000 7f7a6f7fe700 1 -- 172.21.15.157:0/3043412161 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7000b900 con 0x7f7a78150410 2023-10-06T04:02:45.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.577+0000 7f7a6f7fe700 1 -- 172.21.15.157:0/3043412161 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a74001470 con 0x7f7a78150410 2023-10-06T04:02:45.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.578+0000 7f7a7f1b0700 1 -- 172.21.15.157:0/3043412161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a78150410 msgr2=0x7f7a78155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:45.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.578+0000 7f7a7f1b0700 1 --2- 172.21.15.157:0/3043412161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a78150410 0x7f7a78155f70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7a70009e90 tx=0x7f7a70009780 comp rx=0 tx=0).stop 2023-10-06T04:02:45.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.578+0000 7f7a7f1b0700 1 -- 172.21.15.157:0/3043412161 shutdown_connections 2023-10-06T04:02:45.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.578+0000 7f7a7f1b0700 1 --2- 172.21.15.157:0/3043412161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a78150410 0x7f7a78155f70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:45.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.578+0000 7f7a7f1b0700 1 -- 172.21.15.157:0/3043412161 >> 172.21.15.157:0/3043412161 conn(0x7f7a7806f230 msgr2=0x7f7a7806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:45.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.578+0000 7f7a7f1b0700 1 -- 172.21.15.157:0/3043412161 shutdown_connections 2023-10-06T04:02:45.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.578+0000 7f7a7f1b0700 1 -- 172.21.15.157:0/3043412161 wait complete. 2023-10-06T04:02:45.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.579+0000 7f7a7f1b0700 1 Processor -- start 2023-10-06T04:02:45.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.579+0000 7f7a7f1b0700 1 -- start start 2023-10-06T04:02:45.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.580+0000 7f7a7f1b0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a7806cbb0 0x7f7a7815a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:45.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.580+0000 7f7a7f1b0700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a78150410 0x7f7a781660d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:45.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.580+0000 7f7a7f1b0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a780c61f0 0x7f7a7814ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:45.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.580+0000 7f7a7f1b0700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a78155bb0 con 0x7f7a7806cbb0 2023-10-06T04:02:45.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.580+0000 7f7a7f1b0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a780c51e0 con 0x7f7a780c61f0 2023-10-06T04:02:45.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.580+0000 7f7a7f1b0700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a78169e40 con 0x7f7a78150410 2023-10-06T04:02:45.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.580+0000 7f7a7cf4c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a7806cbb0 0x7f7a7815a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:45.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.580+0000 7f7a7cf4c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a7806cbb0 0x7f7a7815a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44598/0 (socket says 172.21.15.157:44598) 2023-10-06T04:02:45.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.580+0000 7f7a7cf4c700 1 -- 172.21.15.157:0/323427162 learned_addr learned my addr 172.21.15.157:0/323427162 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:45.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.580+0000 7f7a7d74d700 1 --2- 172.21.15.157:0/323427162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a780c61f0 0x7f7a7814ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:45.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.580+0000 7f7a6ffff700 1 --2- 172.21.15.157:0/323427162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a78150410 0x7f7a781660d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:45.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.580+0000 7f7a7cf4c700 1 -- 172.21.15.157:0/323427162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a78150410 msgr2=0x7f7a781660d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:45.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.580+0000 7f7a7cf4c700 1 --2- 172.21.15.157:0/323427162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a78150410 0x7f7a781660d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:45.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.580+0000 7f7a7cf4c700 1 -- 172.21.15.157:0/323427162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a780c61f0 msgr2=0x7f7a7814ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:45.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.580+0000 7f7a7cf4c700 1 --2- 172.21.15.157:0/323427162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a780c61f0 0x7f7a7814ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:45.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.580+0000 7f7a7cf4c700 1 -- 172.21.15.157:0/323427162 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a70009a20 con 0x7f7a7806cbb0 2023-10-06T04:02:45.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.580+0000 7f7a6ffff700 1 --2- 172.21.15.157:0/323427162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a78150410 0x7f7a781660d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:02:45.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.580+0000 7f7a7d74d700 1 --2- 172.21.15.157:0/323427162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a780c61f0 0x7f7a7814ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:02:45.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.580+0000 7f7a7cf4c700 1 --2- 172.21.15.157:0/323427162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a7806cbb0 0x7f7a7815a7f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400cba0 tx=0x7f7a74001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:45.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.581+0000 7f7a6cff9700 1 -- 172.21.15.157:0/323427162 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a74007360 con 0x7f7a7806cbb0 2023-10-06T04:02:45.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.581+0000 7f7a6cff9700 1 -- 172.21.15.157:0/323427162 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a740074c0 con 0x7f7a7806cbb0 2023-10-06T04:02:45.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.581+0000 7f7a7f1b0700 1 -- 172.21.15.157:0/323427162 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a780c4c20 con 0x7f7a7806cbb0 2023-10-06T04:02:45.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.581+0000 7f7a6cff9700 1 -- 172.21.15.157:0/323427162 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a74004830 con 0x7f7a7806cbb0 2023-10-06T04:02:45.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.581+0000 7f7a7f1b0700 1 -- 172.21.15.157:0/323427162 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a781697f0 con 0x7f7a7806cbb0 2023-10-06T04:02:45.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.582+0000 7f7a7f1b0700 1 -- 172.21.15.157:0/323427162 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a78089ca0 con 0x7f7a7806cbb0 2023-10-06T04:02:45.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.582+0000 7f7a6cff9700 1 -- 172.21.15.157:0/323427162 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7a74004ba0 con 0x7f7a7806cbb0 2023-10-06T04:02:45.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.582+0000 7f7a6cff9700 1 --2- 172.21.15.157:0/323427162 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7a64066f60 0x7f7a64069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:45.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.583+0000 7f7a6cff9700 1 -- 172.21.15.157:0/323427162 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7252+0+0 (secure 0 0 0) 0x7f7a74084310 con 0x7f7a7806cbb0 2023-10-06T04:02:45.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.583+0000 7f7a6ffff700 1 --2- 172.21.15.157:0/323427162 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7a64066f60 0x7f7a64069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:45.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.583+0000 7f7a6ffff700 1 --2- 172.21.15.157:0/323427162 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7a64066f60 0x7f7a64069410 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7a780c3040 tx=0x7f7a68009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:45.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.587+0000 7f7a6cff9700 1 -- 172.21.15.157:0/323427162 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a74054a80 con 0x7f7a7806cbb0 2023-10-06T04:02:45.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.814+0000 7f7a7f1b0700 1 -- 172.21.15.157:0/323427162 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a781695e0 con 0x7f7a7806cbb0 2023-10-06T04:02:45.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.814+0000 7f7a6cff9700 1 -- 172.21.15.157:0/323427162 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+6182 (secure 0 0 0) 0x7f7a74054610 con 0x7f7a7806cbb0 2023-10-06T04:02:45.816 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:02:45.816 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:02:37.923949+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2,3],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270},"failed":[],"damaged":[],"stopped":[4],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:stopping","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:02:45.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.817+0000 7f7a7f1b0700 1 -- 172.21.15.157:0/323427162 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7a64066f60 msgr2=0x7f7a64069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:45.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.817+0000 7f7a7f1b0700 1 --2- 172.21.15.157:0/323427162 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7a64066f60 0x7f7a64069410 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7a780c3040 tx=0x7f7a68009040 comp rx=0 tx=0).stop 2023-10-06T04:02:45.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.817+0000 7f7a7f1b0700 1 -- 172.21.15.157:0/323427162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a7806cbb0 msgr2=0x7f7a7815a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:45.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.817+0000 7f7a7f1b0700 1 --2- 172.21.15.157:0/323427162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a7806cbb0 0x7f7a7815a7f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400cba0 tx=0x7f7a74001b50 comp rx=0 tx=0).stop 2023-10-06T04:02:45.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.817+0000 7f7a7f1b0700 1 -- 172.21.15.157:0/323427162 shutdown_connections 2023-10-06T04:02:45.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.817+0000 7f7a7f1b0700 1 --2- 172.21.15.157:0/323427162 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7a64066f60 0x7f7a64069410 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:45.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.817+0000 7f7a7f1b0700 1 --2- 172.21.15.157:0/323427162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a780c61f0 0x7f7a7814ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:45.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.817+0000 7f7a7f1b0700 1 --2- 172.21.15.157:0/323427162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a7806cbb0 0x7f7a7815a7f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:45.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.817+0000 7f7a7f1b0700 1 --2- 172.21.15.157:0/323427162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a78150410 0x7f7a781660d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:45.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.817+0000 7f7a7f1b0700 1 -- 172.21.15.157:0/323427162 >> 172.21.15.157:0/323427162 conn(0x7f7a7806f230 msgr2=0x7f7a780cbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:45.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.817+0000 7f7a7f1b0700 1 -- 172.21.15.157:0/323427162 shutdown_connections 2023-10-06T04:02:45.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:45.817+0000 7f7a7f1b0700 1 -- 172.21.15.157:0/323427162 wait complete. 2023-10-06T04:02:45.820 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 44 2023-10-06T04:02:45.830 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:02:47.831 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:02:47.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.958+0000 7f3ab80f2700 1 Processor -- start 2023-10-06T04:02:47.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.958+0000 7f3ab80f2700 1 -- start start 2023-10-06T04:02:47.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.959+0000 7f3ab80f2700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3ab006cbb0 0x7f3ab006d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:47.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.959+0000 7f3ab80f2700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3ab006c1f0 0x7f3ab006da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:47.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.959+0000 7f3ab80f2700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ab00c9780 con 0x7f3ab006cf80 2023-10-06T04:02:47.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.959+0000 7f3ab80f2700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ab00c8c40 con 0x7f3ab006cbb0 2023-10-06T04:02:47.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.959+0000 7f3ab80f2700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ab006e0c0 con 0x7f3ab006c1f0 2023-10-06T04:02:47.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.959+0000 7f3ab5e8e700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3ab006cbb0 0x7f3ab006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:47.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.959+0000 7f3ab5e8e700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3ab006cbb0 0x7f3ab006d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44612/0 (socket says 172.21.15.157:44612) 2023-10-06T04:02:47.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.959+0000 7f3ab5e8e700 1 -- 172.21.15.157:0/3650565116 learned_addr learned my addr 172.21.15.157:0/3650565116 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:47.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.960+0000 7f3ab568d700 1 --2- 172.21.15.157:0/3650565116 >> v2:172.21.15.175:3300/0 conn(0x7f3ab006c1f0 0x7f3ab006da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:47.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.960+0000 7f3ab5e8e700 1 -- 172.21.15.157:0/3650565116 >> v2:172.21.15.175:3300/0 conn(0x7f3ab006c1f0 msgr2=0x7f3ab006da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:47.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.960+0000 7f3ab5e8e700 1 --2- 172.21.15.157:0/3650565116 >> v2:172.21.15.175:3300/0 conn(0x7f3ab006c1f0 0x7f3ab006da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:47.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.960+0000 7f3ab5e8e700 1 -- 172.21.15.157:0/3650565116 >> v1:172.21.15.175:6789/0 conn(0x7f3ab006cf80 legacy=0x7f3ab014fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:47.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.960+0000 7f3ab5e8e700 1 -- 172.21.15.157:0/3650565116 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab00c8e50 con 0x7f3ab006cbb0 2023-10-06T04:02:47.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.960+0000 7f3ab5e8e700 1 --2- 172.21.15.157:0/3650565116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ab006cbb0 0x7f3ab006d530 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0157070 tx=0x7f3aa0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2230c402566654f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:47.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.961+0000 7f3ab4e8c700 1 -- 172.21.15.157:0/3650565116 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa000e070 con 0x7f3ab006cbb0 2023-10-06T04:02:47.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.961+0000 7f3ab4e8c700 1 -- 172.21.15.157:0/3650565116 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa000bab0 con 0x7f3ab006cbb0 2023-10-06T04:02:47.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.962+0000 7f3ab4e8c700 1 -- 172.21.15.157:0/3650565116 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa00092e0 con 0x7f3ab006cbb0 2023-10-06T04:02:47.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.962+0000 7f3ab80f2700 1 -- 172.21.15.157:0/3650565116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ab006cbb0 msgr2=0x7f3ab006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:47.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.962+0000 7f3ab80f2700 1 --2- 172.21.15.157:0/3650565116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ab006cbb0 0x7f3ab006d530 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0157070 tx=0x7f3aa0009840 comp rx=0 tx=0).stop 2023-10-06T04:02:47.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.962+0000 7f3ab80f2700 1 -- 172.21.15.157:0/3650565116 shutdown_connections 2023-10-06T04:02:47.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.962+0000 7f3ab80f2700 1 --2- 172.21.15.157:0/3650565116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ab006cbb0 0x7f3ab006d530 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:47.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.962+0000 7f3ab80f2700 1 --2- 172.21.15.157:0/3650565116 >> v2:172.21.15.175:3300/0 conn(0x7f3ab006c1f0 0x7f3ab006da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:47.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.962+0000 7f3ab80f2700 1 -- 172.21.15.157:0/3650565116 >> 172.21.15.157:0/3650565116 conn(0x7f3ab006f230 msgr2=0x7f3ab006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:47.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.962+0000 7f3ab80f2700 1 -- 172.21.15.157:0/3650565116 shutdown_connections 2023-10-06T04:02:47.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.962+0000 7f3ab80f2700 1 -- 172.21.15.157:0/3650565116 wait complete. 2023-10-06T04:02:47.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.963+0000 7f3ab80f2700 1 Processor -- start 2023-10-06T04:02:47.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.963+0000 7f3ab80f2700 1 -- start start 2023-10-06T04:02:47.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.964+0000 7f3ab80f2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ab006c1f0 0x7f3ab0151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:47.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.964+0000 7f3ab80f2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ab006cbb0 0x7f3ab01670d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:47.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.964+0000 7f3ab80f2700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ab006cf80 0x7f3ab00c3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:47.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.964+0000 7f3ab80f2700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab014ff60 con 0x7f3ab006cbb0 2023-10-06T04:02:47.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.964+0000 7f3ab80f2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab014ba20 con 0x7f3ab006c1f0 2023-10-06T04:02:47.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.964+0000 7f3ab80f2700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab0150190 con 0x7f3ab006cf80 2023-10-06T04:02:47.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.964+0000 7f3ab568d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ab006cbb0 0x7f3ab01670d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:47.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.964+0000 7f3ab568d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ab006cbb0 0x7f3ab01670d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44628/0 (socket says 172.21.15.157:44628) 2023-10-06T04:02:47.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.964+0000 7f3ab568d700 1 -- 172.21.15.157:0/1114326500 learned_addr learned my addr 172.21.15.157:0/1114326500 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:47.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.964+0000 7f3ab5e8e700 1 --2- 172.21.15.157:0/1114326500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ab006c1f0 0x7f3ab0151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:47.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.964+0000 7f3ab668f700 1 --2- 172.21.15.157:0/1114326500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ab006cf80 0x7f3ab00c3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:47.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.965+0000 7f3ab568d700 1 -- 172.21.15.157:0/1114326500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ab006cf80 msgr2=0x7f3ab00c3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:47.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.965+0000 7f3ab568d700 1 --2- 172.21.15.157:0/1114326500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ab006cf80 0x7f3ab00c3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:47.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.965+0000 7f3ab568d700 1 -- 172.21.15.157:0/1114326500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ab006c1f0 msgr2=0x7f3ab0151320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:47.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.965+0000 7f3ab568d700 1 --2- 172.21.15.157:0/1114326500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ab006c1f0 0x7f3ab0151320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:47.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.965+0000 7f3ab568d700 1 -- 172.21.15.157:0/1114326500 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa0009ae0 con 0x7f3ab006cbb0 2023-10-06T04:02:47.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.965+0000 7f3ab668f700 1 --2- 172.21.15.157:0/1114326500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ab006cf80 0x7f3ab00c3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:02:47.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.965+0000 7f3ab568d700 1 --2- 172.21.15.157:0/1114326500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ab006cbb0 0x7f3ab01670d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3aa400ec80 tx=0x7f3aa40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:47.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.965+0000 7f3aadffb700 1 -- 172.21.15.157:0/1114326500 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa400c850 con 0x7f3ab006cbb0 2023-10-06T04:02:47.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.965+0000 7f3aadffb700 1 -- 172.21.15.157:0/1114326500 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa400c9b0 con 0x7f3ab006cbb0 2023-10-06T04:02:47.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.965+0000 7f3ab80f2700 1 -- 172.21.15.157:0/1114326500 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab0150470 con 0x7f3ab006cbb0 2023-10-06T04:02:47.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.965+0000 7f3aadffb700 1 -- 172.21.15.157:0/1114326500 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa400cd50 con 0x7f3ab006cbb0 2023-10-06T04:02:47.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.965+0000 7f3ab80f2700 1 -- 172.21.15.157:0/1114326500 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab00c0f20 con 0x7f3ab006cbb0 2023-10-06T04:02:47.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.966+0000 7f3ab80f2700 1 -- 172.21.15.157:0/1114326500 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab006ddc0 con 0x7f3ab006cbb0 2023-10-06T04:02:47.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.967+0000 7f3aadffb700 1 -- 172.21.15.157:0/1114326500 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3aa4013680 con 0x7f3ab006cbb0 2023-10-06T04:02:47.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.967+0000 7f3aadffb700 1 --2- 172.21.15.157:0/1114326500 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3a9c066f30 0x7f3a9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:47.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.967+0000 7f3aadffb700 1 -- 172.21.15.157:0/1114326500 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7f3aa4084b80 con 0x7f3ab006cbb0 2023-10-06T04:02:47.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.967+0000 7f3ab5e8e700 1 --2- 172.21.15.157:0/1114326500 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3a9c066f30 0x7f3a9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:47.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.968+0000 7f3ab5e8e700 1 --2- 172.21.15.157:0/1114326500 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3a9c066f30 0x7f3a9c0693e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0009880 tx=0x7f3aa0018000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:47.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:47.971+0000 7f3aadffb700 1 -- 172.21.15.157:0/1114326500 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3aa4051a40 con 0x7f3ab006cbb0 2023-10-06T04:02:48.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:48.209+0000 7f3ab80f2700 1 -- 172.21.15.157:0/1114326500 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ab00c9130 con 0x7f3ab006cbb0 2023-10-06T04:02:48.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:48.209+0000 7f3aadffb700 1 -- 172.21.15.157:0/1114326500 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+6178 (secure 0 0 0) 0x7f3aa4055060 con 0x7f3ab006cbb0 2023-10-06T04:02:48.212 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:02:48.212 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:02:46.949420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2,3],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270},"failed":[],"damaged":[],"stopped":[4],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:stopping","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:02:48.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:48.212+0000 7f3ab80f2700 1 -- 172.21.15.157:0/1114326500 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3a9c066f30 msgr2=0x7f3a9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:48.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:48.212+0000 7f3ab80f2700 1 --2- 172.21.15.157:0/1114326500 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3a9c066f30 0x7f3a9c0693e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0009880 tx=0x7f3aa0018000 comp rx=0 tx=0).stop 2023-10-06T04:02:48.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:48.212+0000 7f3ab80f2700 1 -- 172.21.15.157:0/1114326500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ab006cbb0 msgr2=0x7f3ab01670d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:48.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:48.212+0000 7f3ab80f2700 1 --2- 172.21.15.157:0/1114326500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ab006cbb0 0x7f3ab01670d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3aa400ec80 tx=0x7f3aa40091a0 comp rx=0 tx=0).stop 2023-10-06T04:02:48.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:48.212+0000 7f3ab80f2700 1 -- 172.21.15.157:0/1114326500 shutdown_connections 2023-10-06T04:02:48.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:48.212+0000 7f3ab80f2700 1 --2- 172.21.15.157:0/1114326500 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3a9c066f30 0x7f3a9c0693e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:48.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:48.212+0000 7f3ab80f2700 1 --2- 172.21.15.157:0/1114326500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ab006c1f0 0x7f3ab0151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:48.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:48.212+0000 7f3ab80f2700 1 --2- 172.21.15.157:0/1114326500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ab006cf80 0x7f3ab00c3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:48.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:48.212+0000 7f3ab80f2700 1 --2- 172.21.15.157:0/1114326500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ab006cbb0 0x7f3ab01670d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:48.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:48.212+0000 7f3ab80f2700 1 -- 172.21.15.157:0/1114326500 >> 172.21.15.157:0/1114326500 conn(0x7f3ab006f230 msgr2=0x7f3ab014e500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:48.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:48.212+0000 7f3ab80f2700 1 -- 172.21.15.157:0/1114326500 shutdown_connections 2023-10-06T04:02:48.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:48.212+0000 7f3ab80f2700 1 -- 172.21.15.157:0/1114326500 wait complete. 2023-10-06T04:02:48.216 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 45 2023-10-06T04:02:48.227 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:02:48.268 INFO:tasks.workunit.client.0.smithi157.stdout:removed './128/37' 2023-10-06T04:02:48.268 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './128' 2023-10-06T04:02:48.268 INFO:tasks.workunit.client.0.smithi157.stdout:removed './129/19' 2023-10-06T04:02:48.268 INFO:tasks.workunit.client.0.smithi157.stdout:removed './129/15' 2023-10-06T04:02:48.268 INFO:tasks.workunit.client.0.smithi157.stdout:removed './129/31' 2023-10-06T04:02:48.269 INFO:tasks.workunit.client.0.smithi157.stdout:removed './129/36' 2023-10-06T04:02:48.269 INFO:tasks.workunit.client.0.smithi157.stdout:removed './129/20' 2023-10-06T04:02:48.269 INFO:tasks.workunit.client.0.smithi157.stdout:removed './129/12' 2023-10-06T04:02:48.269 INFO:tasks.workunit.client.0.smithi157.stdout:removed './129/39' 2023-10-06T04:02:48.269 INFO:tasks.workunit.client.0.smithi157.stdout:removed './129/10' 2023-10-06T04:02:48.269 INFO:tasks.workunit.client.0.smithi157.stdout:removed './129/34' 2023-10-06T04:02:48.269 INFO:tasks.workunit.client.0.smithi157.stdout:removed './129/38' 2023-10-06T04:02:48.269 INFO:tasks.workunit.client.0.smithi157.stdout:removed './129/11' 2023-10-06T04:02:48.269 INFO:tasks.workunit.client.0.smithi157.stdout:removed './129/16' 2023-10-06T04:02:48.269 INFO:tasks.workunit.client.0.smithi157.stdout:removed './129/32' 2023-10-06T04:02:48.270 INFO:tasks.workunit.client.0.smithi157.stdout:removed './129/30' 2023-10-06T04:02:48.270 INFO:tasks.workunit.client.0.smithi157.stdout:removed './129/14' 2023-10-06T04:02:48.270 INFO:tasks.workunit.client.0.smithi157.stdout:removed './129/18' 2023-10-06T04:02:48.270 INFO:tasks.workunit.client.0.smithi157.stdout:removed './129/13' 2023-10-06T04:02:48.270 INFO:tasks.workunit.client.0.smithi157.stdout:removed './129/33' 2023-10-06T04:02:48.270 INFO:tasks.workunit.client.0.smithi157.stdout:removed './129/17' 2023-10-06T04:02:48.271 INFO:tasks.workunit.client.0.smithi157.stdout:removed './129/35' 2023-10-06T04:02:48.271 INFO:tasks.workunit.client.0.smithi157.stdout:removed './129/37' 2023-10-06T04:02:48.271 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './129' 2023-10-06T04:02:48.271 INFO:tasks.workunit.client.0.smithi157.stdout:removed './130/19' 2023-10-06T04:02:48.271 INFO:tasks.workunit.client.0.smithi157.stdout:removed './130/15' 2023-10-06T04:02:48.271 INFO:tasks.workunit.client.0.smithi157.stdout:removed './130/31' 2023-10-06T04:02:48.271 INFO:tasks.workunit.client.0.smithi157.stdout:removed './130/36' 2023-10-06T04:02:48.271 INFO:tasks.workunit.client.0.smithi157.stdout:removed './130/20' 2023-10-06T04:02:48.271 INFO:tasks.workunit.client.0.smithi157.stdout:removed './130/12' 2023-10-06T04:02:48.271 INFO:tasks.workunit.client.0.smithi157.stdout:removed './130/39' 2023-10-06T04:02:48.272 INFO:tasks.workunit.client.0.smithi157.stdout:removed './130/10' 2023-10-06T04:02:48.272 INFO:tasks.workunit.client.0.smithi157.stdout:removed './130/34' 2023-10-06T04:02:48.272 INFO:tasks.workunit.client.0.smithi157.stdout:removed './130/38' 2023-10-06T04:02:48.272 INFO:tasks.workunit.client.0.smithi157.stdout:removed './130/11' 2023-10-06T04:02:48.272 INFO:tasks.workunit.client.0.smithi157.stdout:removed './130/16' 2023-10-06T04:02:48.273 INFO:tasks.workunit.client.0.smithi157.stdout:removed './130/32' 2023-10-06T04:02:48.273 INFO:tasks.workunit.client.0.smithi157.stdout:removed './130/30' 2023-10-06T04:02:48.273 INFO:tasks.workunit.client.0.smithi157.stdout:removed './130/14' 2023-10-06T04:02:48.273 INFO:tasks.workunit.client.0.smithi157.stdout:removed './130/18' 2023-10-06T04:02:48.273 INFO:tasks.workunit.client.0.smithi157.stdout:removed './130/13' 2023-10-06T04:02:48.273 INFO:tasks.workunit.client.0.smithi157.stdout:removed './130/33' 2023-10-06T04:02:48.273 INFO:tasks.workunit.client.0.smithi157.stdout:removed './130/17' 2023-10-06T04:02:48.273 INFO:tasks.workunit.client.0.smithi157.stdout:removed './130/35' 2023-10-06T04:02:48.273 INFO:tasks.workunit.client.0.smithi157.stdout:removed './130/37' 2023-10-06T04:02:48.273 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './130' 2023-10-06T04:02:48.273 INFO:tasks.workunit.client.0.smithi157.stdout:removed './131/19' 2023-10-06T04:02:48.273 INFO:tasks.workunit.client.0.smithi157.stdout:removed './131/15' 2023-10-06T04:02:48.274 INFO:tasks.workunit.client.0.smithi157.stdout:removed './131/31' 2023-10-06T04:02:48.274 INFO:tasks.workunit.client.0.smithi157.stdout:removed './131/36' 2023-10-06T04:02:48.274 INFO:tasks.workunit.client.0.smithi157.stdout:removed './131/20' 2023-10-06T04:02:48.274 INFO:tasks.workunit.client.0.smithi157.stdout:removed './131/12' 2023-10-06T04:02:48.274 INFO:tasks.workunit.client.0.smithi157.stdout:removed './131/39' 2023-10-06T04:02:48.274 INFO:tasks.workunit.client.0.smithi157.stdout:removed './131/10' 2023-10-06T04:02:48.275 INFO:tasks.workunit.client.0.smithi157.stdout:removed './131/34' 2023-10-06T04:02:48.275 INFO:tasks.workunit.client.0.smithi157.stdout:removed './131/38' 2023-10-06T04:02:48.275 INFO:tasks.workunit.client.0.smithi157.stdout:removed './131/11' 2023-10-06T04:02:48.275 INFO:tasks.workunit.client.0.smithi157.stdout:removed './131/16' 2023-10-06T04:02:48.275 INFO:tasks.workunit.client.0.smithi157.stdout:removed './131/32' 2023-10-06T04:02:48.275 INFO:tasks.workunit.client.0.smithi157.stdout:removed './131/30' 2023-10-06T04:02:48.275 INFO:tasks.workunit.client.0.smithi157.stdout:removed './131/14' 2023-10-06T04:02:48.275 INFO:tasks.workunit.client.0.smithi157.stdout:removed './131/18' 2023-10-06T04:02:48.275 INFO:tasks.workunit.client.0.smithi157.stdout:removed './131/13' 2023-10-06T04:02:48.275 INFO:tasks.workunit.client.0.smithi157.stdout:removed './131/33' 2023-10-06T04:02:48.276 INFO:tasks.workunit.client.0.smithi157.stdout:removed './131/17' 2023-10-06T04:02:48.276 INFO:tasks.workunit.client.0.smithi157.stdout:removed './131/35' 2023-10-06T04:02:48.276 INFO:tasks.workunit.client.0.smithi157.stdout:removed './131/37' 2023-10-06T04:02:48.276 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './131' 2023-10-06T04:02:48.276 INFO:tasks.workunit.client.0.smithi157.stdout:removed './132/19' 2023-10-06T04:02:48.276 INFO:tasks.workunit.client.0.smithi157.stdout:removed './132/15' 2023-10-06T04:02:48.276 INFO:tasks.workunit.client.0.smithi157.stdout:removed './132/31' 2023-10-06T04:02:48.276 INFO:tasks.workunit.client.0.smithi157.stdout:removed './132/36' 2023-10-06T04:02:48.276 INFO:tasks.workunit.client.0.smithi157.stdout:removed './132/20' 2023-10-06T04:02:48.276 INFO:tasks.workunit.client.0.smithi157.stdout:removed './132/12' 2023-10-06T04:02:48.276 INFO:tasks.workunit.client.0.smithi157.stdout:removed './132/39' 2023-10-06T04:02:48.276 INFO:tasks.workunit.client.0.smithi157.stdout:removed './132/10' 2023-10-06T04:02:48.277 INFO:tasks.workunit.client.0.smithi157.stdout:removed './132/34' 2023-10-06T04:02:48.277 INFO:tasks.workunit.client.0.smithi157.stdout:removed './132/38' 2023-10-06T04:02:48.277 INFO:tasks.workunit.client.0.smithi157.stdout:removed './132/11' 2023-10-06T04:02:48.277 INFO:tasks.workunit.client.0.smithi157.stdout:removed './132/16' 2023-10-06T04:02:48.277 INFO:tasks.workunit.client.0.smithi157.stdout:removed './132/32' 2023-10-06T04:02:48.277 INFO:tasks.workunit.client.0.smithi157.stdout:removed './132/30' 2023-10-06T04:02:48.277 INFO:tasks.workunit.client.0.smithi157.stdout:removed './132/14' 2023-10-06T04:02:48.277 INFO:tasks.workunit.client.0.smithi157.stdout:removed './132/18' 2023-10-06T04:02:48.277 INFO:tasks.workunit.client.0.smithi157.stdout:removed './132/13' 2023-10-06T04:02:48.277 INFO:tasks.workunit.client.0.smithi157.stdout:removed './132/33' 2023-10-06T04:02:48.277 INFO:tasks.workunit.client.0.smithi157.stdout:removed './132/17' 2023-10-06T04:02:48.278 INFO:tasks.workunit.client.0.smithi157.stdout:removed './132/35' 2023-10-06T04:02:48.278 INFO:tasks.workunit.client.0.smithi157.stdout:removed './132/37' 2023-10-06T04:02:48.278 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './132' 2023-10-06T04:02:48.278 INFO:tasks.workunit.client.0.smithi157.stdout:removed './133/19' 2023-10-06T04:02:48.278 INFO:tasks.workunit.client.0.smithi157.stdout:removed './133/15' 2023-10-06T04:02:48.278 INFO:tasks.workunit.client.0.smithi157.stdout:removed './133/31' 2023-10-06T04:02:48.278 INFO:tasks.workunit.client.0.smithi157.stdout:removed './133/36' 2023-10-06T04:02:48.278 INFO:tasks.workunit.client.0.smithi157.stdout:removed './133/20' 2023-10-06T04:02:48.278 INFO:tasks.workunit.client.0.smithi157.stdout:removed './133/12' 2023-10-06T04:02:48.278 INFO:tasks.workunit.client.0.smithi157.stdout:removed './133/39' 2023-10-06T04:02:48.278 INFO:tasks.workunit.client.0.smithi157.stdout:removed './133/10' 2023-10-06T04:02:48.279 INFO:tasks.workunit.client.0.smithi157.stdout:removed './133/34' 2023-10-06T04:02:48.279 INFO:tasks.workunit.client.0.smithi157.stdout:removed './133/38' 2023-10-06T04:02:48.279 INFO:tasks.workunit.client.0.smithi157.stdout:removed './133/11' 2023-10-06T04:02:48.279 INFO:tasks.workunit.client.0.smithi157.stdout:removed './133/16' 2023-10-06T04:02:48.279 INFO:tasks.workunit.client.0.smithi157.stdout:removed './133/32' 2023-10-06T04:02:48.279 INFO:tasks.workunit.client.0.smithi157.stdout:removed './133/30' 2023-10-06T04:02:48.279 INFO:tasks.workunit.client.0.smithi157.stdout:removed './133/14' 2023-10-06T04:02:48.279 INFO:tasks.workunit.client.0.smithi157.stdout:removed './133/18' 2023-10-06T04:02:48.279 INFO:tasks.workunit.client.0.smithi157.stdout:removed './133/13' 2023-10-06T04:02:48.280 INFO:tasks.workunit.client.0.smithi157.stdout:removed './133/33' 2023-10-06T04:02:48.280 INFO:tasks.workunit.client.0.smithi157.stdout:removed './133/17' 2023-10-06T04:02:48.280 INFO:tasks.workunit.client.0.smithi157.stdout:removed './133/35' 2023-10-06T04:02:48.280 INFO:tasks.workunit.client.0.smithi157.stdout:removed './133/37' 2023-10-06T04:02:48.280 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './133' 2023-10-06T04:02:48.280 INFO:tasks.workunit.client.0.smithi157.stdout:removed './134/19' 2023-10-06T04:02:48.280 INFO:tasks.workunit.client.0.smithi157.stdout:removed './134/15' 2023-10-06T04:02:48.281 INFO:tasks.workunit.client.0.smithi157.stdout:removed './134/31' 2023-10-06T04:02:48.281 INFO:tasks.workunit.client.0.smithi157.stdout:removed './134/36' 2023-10-06T04:02:48.281 INFO:tasks.workunit.client.0.smithi157.stdout:removed './134/20' 2023-10-06T04:02:48.281 INFO:tasks.workunit.client.0.smithi157.stdout:removed './134/12' 2023-10-06T04:02:48.281 INFO:tasks.workunit.client.0.smithi157.stdout:removed './134/39' 2023-10-06T04:02:48.281 INFO:tasks.workunit.client.0.smithi157.stdout:removed './134/10' 2023-10-06T04:02:48.282 INFO:tasks.workunit.client.0.smithi157.stdout:removed './134/34' 2023-10-06T04:02:48.282 INFO:tasks.workunit.client.0.smithi157.stdout:removed './134/38' 2023-10-06T04:02:48.282 INFO:tasks.workunit.client.0.smithi157.stdout:removed './134/11' 2023-10-06T04:02:48.282 INFO:tasks.workunit.client.0.smithi157.stdout:removed './134/16' 2023-10-06T04:02:48.282 INFO:tasks.workunit.client.0.smithi157.stdout:removed './134/32' 2023-10-06T04:02:48.282 INFO:tasks.workunit.client.0.smithi157.stdout:removed './134/30' 2023-10-06T04:02:48.283 INFO:tasks.workunit.client.0.smithi157.stdout:removed './134/14' 2023-10-06T04:02:48.283 INFO:tasks.workunit.client.0.smithi157.stdout:removed './134/18' 2023-10-06T04:02:48.283 INFO:tasks.workunit.client.0.smithi157.stdout:removed './134/13' 2023-10-06T04:02:48.283 INFO:tasks.workunit.client.0.smithi157.stdout:removed './134/33' 2023-10-06T04:02:48.283 INFO:tasks.workunit.client.0.smithi157.stdout:removed './134/17' 2023-10-06T04:02:48.283 INFO:tasks.workunit.client.0.smithi157.stdout:removed './134/35' 2023-10-06T04:02:48.284 INFO:tasks.workunit.client.0.smithi157.stdout:removed './134/37' 2023-10-06T04:02:48.284 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './134' 2023-10-06T04:02:48.284 INFO:tasks.workunit.client.0.smithi157.stdout:removed './135/19' 2023-10-06T04:02:48.284 INFO:tasks.workunit.client.0.smithi157.stdout:removed './135/15' 2023-10-06T04:02:48.284 INFO:tasks.workunit.client.0.smithi157.stdout:removed './135/31' 2023-10-06T04:02:48.284 INFO:tasks.workunit.client.0.smithi157.stdout:removed './135/36' 2023-10-06T04:02:48.285 INFO:tasks.workunit.client.0.smithi157.stdout:removed './135/20' 2023-10-06T04:02:48.285 INFO:tasks.workunit.client.0.smithi157.stdout:removed './135/12' 2023-10-06T04:02:48.285 INFO:tasks.workunit.client.0.smithi157.stdout:removed './135/39' 2023-10-06T04:02:48.285 INFO:tasks.workunit.client.0.smithi157.stdout:removed './135/10' 2023-10-06T04:02:48.285 INFO:tasks.workunit.client.0.smithi157.stdout:removed './135/34' 2023-10-06T04:02:48.286 INFO:tasks.workunit.client.0.smithi157.stdout:removed './135/38' 2023-10-06T04:02:48.286 INFO:tasks.workunit.client.0.smithi157.stdout:removed './135/11' 2023-10-06T04:02:48.286 INFO:tasks.workunit.client.0.smithi157.stdout:removed './135/16' 2023-10-06T04:02:48.286 INFO:tasks.workunit.client.0.smithi157.stdout:removed './135/32' 2023-10-06T04:02:48.286 INFO:tasks.workunit.client.0.smithi157.stdout:removed './135/30' 2023-10-06T04:02:48.286 INFO:tasks.workunit.client.0.smithi157.stdout:removed './135/14' 2023-10-06T04:02:48.286 INFO:tasks.workunit.client.0.smithi157.stdout:removed './135/18' 2023-10-06T04:02:48.287 INFO:tasks.workunit.client.0.smithi157.stdout:removed './135/13' 2023-10-06T04:02:48.287 INFO:tasks.workunit.client.0.smithi157.stdout:removed './135/33' 2023-10-06T04:02:48.287 INFO:tasks.workunit.client.0.smithi157.stdout:removed './135/17' 2023-10-06T04:02:48.287 INFO:tasks.workunit.client.0.smithi157.stdout:removed './135/35' 2023-10-06T04:02:48.287 INFO:tasks.workunit.client.0.smithi157.stdout:removed './135/37' 2023-10-06T04:02:48.287 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './135' 2023-10-06T04:02:48.287 INFO:tasks.workunit.client.0.smithi157.stdout:removed './136/19' 2023-10-06T04:02:48.287 INFO:tasks.workunit.client.0.smithi157.stdout:removed './136/15' 2023-10-06T04:02:48.287 INFO:tasks.workunit.client.0.smithi157.stdout:removed './136/31' 2023-10-06T04:02:48.288 INFO:tasks.workunit.client.0.smithi157.stdout:removed './136/36' 2023-10-06T04:02:48.288 INFO:tasks.workunit.client.0.smithi157.stdout:removed './136/20' 2023-10-06T04:02:48.288 INFO:tasks.workunit.client.0.smithi157.stdout:removed './136/12' 2023-10-06T04:02:48.288 INFO:tasks.workunit.client.0.smithi157.stdout:removed './136/39' 2023-10-06T04:02:48.288 INFO:tasks.workunit.client.0.smithi157.stdout:removed './136/10' 2023-10-06T04:02:48.288 INFO:tasks.workunit.client.0.smithi157.stdout:removed './136/34' 2023-10-06T04:02:48.288 INFO:tasks.workunit.client.0.smithi157.stdout:removed './136/38' 2023-10-06T04:02:48.288 INFO:tasks.workunit.client.0.smithi157.stdout:removed './136/11' 2023-10-06T04:02:48.289 INFO:tasks.workunit.client.0.smithi157.stdout:removed './136/16' 2023-10-06T04:02:48.289 INFO:tasks.workunit.client.0.smithi157.stdout:removed './136/32' 2023-10-06T04:02:48.289 INFO:tasks.workunit.client.0.smithi157.stdout:removed './136/30' 2023-10-06T04:02:48.289 INFO:tasks.workunit.client.0.smithi157.stdout:removed './136/14' 2023-10-06T04:02:48.289 INFO:tasks.workunit.client.0.smithi157.stdout:removed './136/18' 2023-10-06T04:02:48.289 INFO:tasks.workunit.client.0.smithi157.stdout:removed './136/13' 2023-10-06T04:02:48.289 INFO:tasks.workunit.client.0.smithi157.stdout:removed './136/33' 2023-10-06T04:02:48.289 INFO:tasks.workunit.client.0.smithi157.stdout:removed './136/17' 2023-10-06T04:02:48.289 INFO:tasks.workunit.client.0.smithi157.stdout:removed './136/35' 2023-10-06T04:02:48.290 INFO:tasks.workunit.client.0.smithi157.stdout:removed './136/37' 2023-10-06T04:02:48.290 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './136' 2023-10-06T04:02:48.290 INFO:tasks.workunit.client.0.smithi157.stdout:removed './137/19' 2023-10-06T04:02:48.290 INFO:tasks.workunit.client.0.smithi157.stdout:removed './137/15' 2023-10-06T04:02:48.290 INFO:tasks.workunit.client.0.smithi157.stdout:removed './137/31' 2023-10-06T04:02:48.290 INFO:tasks.workunit.client.0.smithi157.stdout:removed './137/36' 2023-10-06T04:02:48.291 INFO:tasks.workunit.client.0.smithi157.stdout:removed './137/20' 2023-10-06T04:02:48.291 INFO:tasks.workunit.client.0.smithi157.stdout:removed './137/12' 2023-10-06T04:02:48.291 INFO:tasks.workunit.client.0.smithi157.stdout:removed './137/39' 2023-10-06T04:02:48.291 INFO:tasks.workunit.client.0.smithi157.stdout:removed './137/10' 2023-10-06T04:02:48.291 INFO:tasks.workunit.client.0.smithi157.stdout:removed './137/34' 2023-10-06T04:02:48.291 INFO:tasks.workunit.client.0.smithi157.stdout:removed './137/38' 2023-10-06T04:02:48.291 INFO:tasks.workunit.client.0.smithi157.stdout:removed './137/11' 2023-10-06T04:02:48.291 INFO:tasks.workunit.client.0.smithi157.stdout:removed './137/16' 2023-10-06T04:02:48.292 INFO:tasks.workunit.client.0.smithi157.stdout:removed './137/32' 2023-10-06T04:02:48.292 INFO:tasks.workunit.client.0.smithi157.stdout:removed './137/30' 2023-10-06T04:02:48.292 INFO:tasks.workunit.client.0.smithi157.stdout:removed './137/14' 2023-10-06T04:02:48.292 INFO:tasks.workunit.client.0.smithi157.stdout:removed './137/18' 2023-10-06T04:02:48.292 INFO:tasks.workunit.client.0.smithi157.stdout:removed './137/13' 2023-10-06T04:02:48.292 INFO:tasks.workunit.client.0.smithi157.stdout:removed './137/33' 2023-10-06T04:02:48.292 INFO:tasks.workunit.client.0.smithi157.stdout:removed './137/17' 2023-10-06T04:02:48.292 INFO:tasks.workunit.client.0.smithi157.stdout:removed './137/35' 2023-10-06T04:02:48.292 INFO:tasks.workunit.client.0.smithi157.stdout:removed './137/37' 2023-10-06T04:02:48.292 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './137' 2023-10-06T04:02:48.292 INFO:tasks.workunit.client.0.smithi157.stdout:removed './138/19' 2023-10-06T04:02:48.293 INFO:tasks.workunit.client.0.smithi157.stdout:removed './138/15' 2023-10-06T04:02:48.293 INFO:tasks.workunit.client.0.smithi157.stdout:removed './138/31' 2023-10-06T04:02:48.293 INFO:tasks.workunit.client.0.smithi157.stdout:removed './138/36' 2023-10-06T04:02:48.293 INFO:tasks.workunit.client.0.smithi157.stdout:removed './138/20' 2023-10-06T04:02:48.293 INFO:tasks.workunit.client.0.smithi157.stdout:removed './138/12' 2023-10-06T04:02:48.293 INFO:tasks.workunit.client.0.smithi157.stdout:removed './138/39' 2023-10-06T04:02:48.293 INFO:tasks.workunit.client.0.smithi157.stdout:removed './138/10' 2023-10-06T04:02:48.293 INFO:tasks.workunit.client.0.smithi157.stdout:removed './138/34' 2023-10-06T04:02:48.293 INFO:tasks.workunit.client.0.smithi157.stdout:removed './138/38' 2023-10-06T04:02:48.293 INFO:tasks.workunit.client.0.smithi157.stdout:removed './138/11' 2023-10-06T04:02:48.293 INFO:tasks.workunit.client.0.smithi157.stdout:removed './138/16' 2023-10-06T04:02:50.228 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:02:50.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.353+0000 7f3cb61dd700 1 Processor -- start 2023-10-06T04:02:50.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.353+0000 7f3cb61dd700 1 -- start start 2023-10-06T04:02:50.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.353+0000 7f3cb61dd700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cb006cbb0 0x7f3cb006d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:50.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.354+0000 7f3cb61dd700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3cb006c1f0 0x7f3cb006da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:50.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.354+0000 7f3cb61dd700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cb00c9780 con 0x7f3cb006cf80 2023-10-06T04:02:50.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.354+0000 7f3cb61dd700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb00c8c40 con 0x7f3cb006c1f0 2023-10-06T04:02:50.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.354+0000 7f3cb61dd700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb006e0c0 con 0x7f3cb006cbb0 2023-10-06T04:02:50.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.354+0000 7f3caf7fe700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cb006cbb0 0x7f3cb006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:50.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.354+0000 7f3caeffd700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3cb006c1f0 0x7f3cb006da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:50.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.354+0000 7f3caeffd700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3cb006c1f0 0x7f3cb006da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:56760/0 (socket says 172.21.15.157:56760) 2023-10-06T04:02:50.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.354+0000 7f3caeffd700 1 -- 172.21.15.157:0/3638779365 learned_addr learned my addr 172.21.15.157:0/3638779365 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:50.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.354+0000 7f3caffff700 1 --1- 172.21.15.157:0/3638779365 >> v1:172.21.15.175:6789/0 conn(0x7f3cb006cf80 0x7f3cb014fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.157:41542/0 (socket says 172.21.15.157:41542) 2023-10-06T04:02:50.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.354+0000 7f3caeffd700 1 -- 172.21.15.157:0/3638779365 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cb006cbb0 msgr2=0x7f3cb006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:50.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.354+0000 7f3caeffd700 1 --2- 172.21.15.157:0/3638779365 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cb006cbb0 0x7f3cb006d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:50.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.354+0000 7f3caeffd700 1 -- 172.21.15.157:0/3638779365 >> v1:172.21.15.175:6789/0 conn(0x7f3cb006cf80 legacy=0x7f3cb014fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:50.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.354+0000 7f3cae7fc700 1 -- 172.21.15.157:0/3638779365 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 289020150 0 0) 0x7f3cb00c9780 con 0x7f3cb006cf80 2023-10-06T04:02:50.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.354+0000 7f3caeffd700 1 -- 172.21.15.157:0/3638779365 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb00c8e50 con 0x7f3cb006c1f0 2023-10-06T04:02:50.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.354+0000 7f3caf7fe700 1 --2- 172.21.15.157:0/3638779365 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cb006cbb0 0x7f3cb006d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:02:50.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.355+0000 7f3caeffd700 1 --2- 172.21.15.157:0/3638779365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cb006c1f0 0x7f3cb006da70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3c980099e0 tx=0x7f3c98009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=516796f90a346913 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:50.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.355+0000 7f3cae7fc700 1 -- 172.21.15.157:0/3638779365 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9800b8b0 con 0x7f3cb006c1f0 2023-10-06T04:02:50.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.355+0000 7f3cae7fc700 1 -- 172.21.15.157:0/3638779365 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c9800ba10 con 0x7f3cb006c1f0 2023-10-06T04:02:50.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.355+0000 7f3cae7fc700 1 -- 172.21.15.157:0/3638779365 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9800bdb0 con 0x7f3cb006c1f0 2023-10-06T04:02:50.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.355+0000 7f3cb61dd700 1 -- 172.21.15.157:0/3638779365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cb006c1f0 msgr2=0x7f3cb006da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:50.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.355+0000 7f3cb61dd700 1 --2- 172.21.15.157:0/3638779365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cb006c1f0 0x7f3cb006da70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3c980099e0 tx=0x7f3c98009250 comp rx=0 tx=0).stop 2023-10-06T04:02:50.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.355+0000 7f3cb61dd700 1 -- 172.21.15.157:0/3638779365 shutdown_connections 2023-10-06T04:02:50.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.355+0000 7f3cb61dd700 1 --2- 172.21.15.157:0/3638779365 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cb006cbb0 0x7f3cb006d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:50.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.355+0000 7f3cb61dd700 1 --2- 172.21.15.157:0/3638779365 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cb006c1f0 0x7f3cb006da70 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:50.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.355+0000 7f3cb61dd700 1 -- 172.21.15.157:0/3638779365 >> 172.21.15.157:0/3638779365 conn(0x7f3cb006f230 msgr2=0x7f3cb006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:50.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.355+0000 7f3cb61dd700 1 -- 172.21.15.157:0/3638779365 shutdown_connections 2023-10-06T04:02:50.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.355+0000 7f3cb61dd700 1 -- 172.21.15.157:0/3638779365 wait complete. 2023-10-06T04:02:50.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.357+0000 7f3cb61dd700 1 Processor -- start 2023-10-06T04:02:50.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.357+0000 7f3cb61dd700 1 -- start start 2023-10-06T04:02:50.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.357+0000 7f3cb61dd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cb006c1f0 0x7f3cb00c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:50.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.357+0000 7f3cb61dd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cb006cbb0 0x7f3cb014fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:50.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.357+0000 7f3cb61dd700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cb006cf80 0x7f3cb0156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:50.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.357+0000 7f3cb61dd700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb00c01e0 con 0x7f3cb006c1f0 2023-10-06T04:02:50.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.357+0000 7f3cb61dd700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb01547a0 con 0x7f3cb006cbb0 2023-10-06T04:02:50.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.357+0000 7f3cb61dd700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb00c5020 con 0x7f3cb006cf80 2023-10-06T04:02:50.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.357+0000 7f3caf7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cb006c1f0 0x7f3cb00c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:50.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.358+0000 7f3caf7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cb006c1f0 0x7f3cb00c54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44638/0 (socket says 172.21.15.157:44638) 2023-10-06T04:02:50.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.358+0000 7f3caf7fe700 1 -- 172.21.15.157:0/2573171093 learned_addr learned my addr 172.21.15.157:0/2573171093 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:50.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.358+0000 7f3caeffd700 1 --2- 172.21.15.157:0/2573171093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cb006cbb0 0x7f3cb014fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:50.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.358+0000 7f3caffff700 1 --2- 172.21.15.157:0/2573171093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cb006cf80 0x7f3cb0156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:50.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.358+0000 7f3caf7fe700 1 -- 172.21.15.157:0/2573171093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cb006cf80 msgr2=0x7f3cb0156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:50.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.358+0000 7f3caf7fe700 1 --2- 172.21.15.157:0/2573171093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cb006cf80 0x7f3cb0156380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:50.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.358+0000 7f3caf7fe700 1 -- 172.21.15.157:0/2573171093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cb006cbb0 msgr2=0x7f3cb014fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:50.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.358+0000 7f3caf7fe700 1 --2- 172.21.15.157:0/2573171093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cb006cbb0 0x7f3cb014fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:50.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.358+0000 7f3caf7fe700 1 -- 172.21.15.157:0/2573171093 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca0009420 con 0x7f3cb006c1f0 2023-10-06T04:02:50.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.358+0000 7f3caf7fe700 1 --2- 172.21.15.157:0/2573171093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cb006c1f0 0x7f3cb00c54b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000ec80 tx=0x7f3ca00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:50.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.358+0000 7f3ca77fe700 1 -- 172.21.15.157:0/2573171093 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca000c920 con 0x7f3cb006c1f0 2023-10-06T04:02:50.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.358+0000 7f3ca77fe700 1 -- 172.21.15.157:0/2573171093 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca000ca80 con 0x7f3cb006c1f0 2023-10-06T04:02:50.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.358+0000 7f3cb61dd700 1 -- 172.21.15.157:0/2573171093 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c980094f0 con 0x7f3cb006c1f0 2023-10-06T04:02:50.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.358+0000 7f3ca77fe700 1 -- 172.21.15.157:0/2573171093 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca000ce00 con 0x7f3cb006c1f0 2023-10-06T04:02:50.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.358+0000 7f3cb61dd700 1 -- 172.21.15.157:0/2573171093 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb01541e0 con 0x7f3cb006c1f0 2023-10-06T04:02:50.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.359+0000 7f3cb61dd700 1 -- 172.21.15.157:0/2573171093 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb0089ca0 con 0x7f3cb006c1f0 2023-10-06T04:02:50.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.360+0000 7f3ca77fe700 1 -- 172.21.15.157:0/2573171093 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3ca0013680 con 0x7f3cb006c1f0 2023-10-06T04:02:50.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.360+0000 7f3ca77fe700 1 --2- 172.21.15.157:0/2573171093 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c90066f30 0x7f3c900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:50.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.360+0000 7f3ca77fe700 1 -- 172.21.15.157:0/2573171093 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7f3ca0084c70 con 0x7f3cb006c1f0 2023-10-06T04:02:50.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.360+0000 7f3caeffd700 1 --2- 172.21.15.157:0/2573171093 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c90066f30 0x7f3c900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:50.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.361+0000 7f3caeffd700 1 --2- 172.21.15.157:0/2573171093 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c90066f30 0x7f3c900693e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3c980099e0 tx=0x7f3c98017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:50.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.364+0000 7f3ca77fe700 1 -- 172.21.15.157:0/2573171093 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ca0051b30 con 0x7f3cb006c1f0 2023-10-06T04:02:50.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.587+0000 7f3cb61dd700 1 -- 172.21.15.157:0/2573171093 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3cb0154580 con 0x7f3cb006c1f0 2023-10-06T04:02:50.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.587+0000 7f3ca77fe700 1 -- 172.21.15.157:0/2573171093 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+6178 (secure 0 0 0) 0x7f3ca0055150 con 0x7f3cb006c1f0 2023-10-06T04:02:50.590 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:02:50.590 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:02:46.949420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2,3],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270},"failed":[],"damaged":[],"stopped":[4],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:stopping","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:02:50.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.589+0000 7f3cb61dd700 1 -- 172.21.15.157:0/2573171093 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c90066f30 msgr2=0x7f3c900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:50.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.589+0000 7f3cb61dd700 1 --2- 172.21.15.157:0/2573171093 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c90066f30 0x7f3c900693e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3c980099e0 tx=0x7f3c98017000 comp rx=0 tx=0).stop 2023-10-06T04:02:50.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.589+0000 7f3cb61dd700 1 -- 172.21.15.157:0/2573171093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cb006c1f0 msgr2=0x7f3cb00c54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:50.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.589+0000 7f3cb61dd700 1 --2- 172.21.15.157:0/2573171093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cb006c1f0 0x7f3cb00c54b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000ec80 tx=0x7f3ca00091a0 comp rx=0 tx=0).stop 2023-10-06T04:02:50.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.589+0000 7f3cb61dd700 1 -- 172.21.15.157:0/2573171093 shutdown_connections 2023-10-06T04:02:50.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.589+0000 7f3cb61dd700 1 --2- 172.21.15.157:0/2573171093 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c90066f30 0x7f3c900693e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:50.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.589+0000 7f3cb61dd700 1 --2- 172.21.15.157:0/2573171093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cb006cf80 0x7f3cb0156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:50.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.589+0000 7f3cb61dd700 1 --2- 172.21.15.157:0/2573171093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cb006c1f0 0x7f3cb00c54b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:50.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.589+0000 7f3cb61dd700 1 --2- 172.21.15.157:0/2573171093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cb006cbb0 0x7f3cb014fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:50.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.589+0000 7f3cb61dd700 1 -- 172.21.15.157:0/2573171093 >> 172.21.15.157:0/2573171093 conn(0x7f3cb006f230 msgr2=0x7f3cb00af240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:50.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.589+0000 7f3cb61dd700 1 -- 172.21.15.157:0/2573171093 shutdown_connections 2023-10-06T04:02:50.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:50.589+0000 7f3cb61dd700 1 -- 172.21.15.157:0/2573171093 wait complete. 2023-10-06T04:02:50.593 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 45 2023-10-06T04:02:50.604 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:02:52.605 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:02:52.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.726+0000 7f5ef7170700 1 Processor -- start 2023-10-06T04:02:52.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.726+0000 7f5ef7170700 1 -- start start 2023-10-06T04:02:52.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.727+0000 7f5ef7170700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ef006dc40 0x7f5ef014fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:52.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.727+0000 7f5ef7170700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ef00c9780 con 0x7f5ef006cbb0 2023-10-06T04:02:52.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.727+0000 7f5ef7170700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ef006e0c0 con 0x7f5ef006c1f0 2023-10-06T04:02:52.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.727+0000 7f5ef7170700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef00c8c40 con 0x7f5ef006dc40 2023-10-06T04:02:52.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.727+0000 7f5ef4f0c700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f5ef006cbb0 0x7f5ef006d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:54962/0 (socket says 172.21.15.157:54962) 2023-10-06T04:02:52.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.727+0000 7f5ef4f0c700 1 -- 172.21.15.157:0/567291582 learned_addr learned my addr 172.21.15.157:0/567291582 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:02:52.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.727+0000 7f5eef7fe700 1 -- 172.21.15.157:0/567291582 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 988862434 0 0) 0x7f5ef00c9780 con 0x7f5ef006cbb0 2023-10-06T04:02:52.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.727+0000 7f5eeffff700 1 --2- 172.21.15.157:0/567291582 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ef006dc40 0x7f5ef014fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:52.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.727+0000 7f5eef7fe700 1 -- 172.21.15.157:0/567291582 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5edc003570 con 0x7f5ef006cbb0 2023-10-06T04:02:52.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.727+0000 7f5eef7fe700 1 -- 172.21.15.157:0/567291582 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1413581352 0 0) 0x7f5edc003570 con 0x7f5ef006cbb0 2023-10-06T04:02:52.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.727+0000 7f5eef7fe700 1 -- 172.21.15.157:0/567291582 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ef006dc40 msgr2=0x7f5ef014fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:52.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.727+0000 7f5eef7fe700 1 --2- 172.21.15.157:0/567291582 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ef006dc40 0x7f5ef014fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:52.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.727+0000 7f5eef7fe700 1 -- 172.21.15.157:0/567291582 >> v1:172.21.15.175:6789/0 conn(0x7f5ef006c1f0 legacy=0x7f5ef0155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:52.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.727+0000 7f5eef7fe700 1 -- 172.21.15.157:0/567291582 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef00c8e50 con 0x7f5ef006cbb0 2023-10-06T04:02:52.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.727+0000 7f5eef7fe700 1 -- 172.21.15.157:0/567291582 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f5ee80017f0 con 0x7f5ef006cbb0 2023-10-06T04:02:52.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.727+0000 7f5eef7fe700 1 -- 172.21.15.157:0/567291582 >> v1:172.21.15.157:6789/0 conn(0x7f5ef006cbb0 legacy=0x7f5ef006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:52.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.727+0000 7f5eef7fe700 1 --2- 172.21.15.157:0/567291582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5edc003a90 0x7f5edc003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:52.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.727+0000 7f5eef7fe700 1 --2- 172.21.15.157:0/567291582 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5edc004810 0x7f5edc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:52.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.727+0000 7f5eef7fe700 1 --2- 172.21.15.157:0/567291582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5edc005210 0x7f5edc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:52.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.727+0000 7f5eef7fe700 1 -- 172.21.15.157:0/567291582 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef00c8c40 con 0x7f5edc003a90 2023-10-06T04:02:52.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.727+0000 7f5eef7fe700 1 -- 172.21.15.157:0/567291582 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5edc00c0e0 con 0x7f5edc005210 2023-10-06T04:02:52.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.727+0000 7f5eef7fe700 1 -- 172.21.15.157:0/567291582 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5edc00c2a0 con 0x7f5edc004810 2023-10-06T04:02:52.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.727+0000 7f5eeffff700 1 --2- 172.21.15.157:0/567291582 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ef006dc40 0x7f5ef014fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:02:52.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.727+0000 7f5ef4f0c700 1 --2- 172.21.15.157:0/567291582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5edc003a90 0x7f5edc003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:52.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.728+0000 7f5ef570d700 1 --2- 172.21.15.157:0/567291582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5edc005210 0x7f5edc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:52.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.728+0000 7f5eeffff700 1 --2- 172.21.15.157:0/567291582 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5edc004810 0x7f5edc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:52.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.728+0000 7f5ef4f0c700 1 -- 172.21.15.157:0/567291582 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5edc004810 msgr2=0x7f5edc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:52.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.728+0000 7f5ef4f0c700 1 --2- 172.21.15.157:0/567291582 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5edc004810 0x7f5edc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:52.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.728+0000 7f5ef4f0c700 1 -- 172.21.15.157:0/567291582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5edc005210 msgr2=0x7f5edc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:52.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.728+0000 7f5ef4f0c700 1 --2- 172.21.15.157:0/567291582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5edc005210 0x7f5edc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:52.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.728+0000 7f5ef4f0c700 1 -- 172.21.15.157:0/567291582 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5edc003570 con 0x7f5edc003a90 2023-10-06T04:02:52.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.728+0000 7f5ef570d700 1 --2- 172.21.15.157:0/567291582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5edc005210 0x7f5edc00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:02:52.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.728+0000 7f5eeffff700 1 --2- 172.21.15.157:0/567291582 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5edc004810 0x7f5edc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:02:52.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.728+0000 7f5ef4f0c700 1 --2- 172.21.15.157:0/567291582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5edc003a90 0x7f5edc003ea0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800a900 tx=0x7f5ef00b1020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e8cbf7e1ac3a5b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:52.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.728+0000 7f5eef7fe700 1 -- 172.21.15.157:0/567291582 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee80070d0 con 0x7f5edc003a90 2023-10-06T04:02:52.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.728+0000 7f5eef7fe700 1 -- 172.21.15.157:0/567291582 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee8007230 con 0x7f5edc003a90 2023-10-06T04:02:52.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.728+0000 7f5eef7fe700 1 -- 172.21.15.157:0/567291582 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee80075f0 con 0x7f5edc003a90 2023-10-06T04:02:52.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.729+0000 7f5ef7170700 1 -- 172.21.15.157:0/567291582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5edc003a90 msgr2=0x7f5edc003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:52.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.729+0000 7f5ef7170700 1 --2- 172.21.15.157:0/567291582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5edc003a90 0x7f5edc003ea0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800a900 tx=0x7f5ef00b1020 comp rx=0 tx=0).stop 2023-10-06T04:02:52.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.729+0000 7f5ef570d700 1 -- 172.21.15.157:0/567291582 reap_dead start 2023-10-06T04:02:52.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.729+0000 7f5ef7170700 1 -- 172.21.15.157:0/567291582 shutdown_connections 2023-10-06T04:02:52.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.729+0000 7f5ef7170700 1 -- 172.21.15.157:0/567291582 >> 172.21.15.157:0/567291582 conn(0x7f5ef006f230 msgr2=0x7f5ef006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:52.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.729+0000 7f5ef7170700 1 -- 172.21.15.157:0/567291582 shutdown_connections 2023-10-06T04:02:52.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.729+0000 7f5ef7170700 1 -- 172.21.15.157:0/567291582 wait complete. 2023-10-06T04:02:52.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.730+0000 7f5ef7170700 1 Processor -- start 2023-10-06T04:02:52.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.730+0000 7f5ef7170700 1 -- start start 2023-10-06T04:02:52.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.731+0000 7f5ef7170700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5edc003a90 0x7f5ef0167340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:52.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.731+0000 7f5ef7170700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ef00c58c0 0x7f5ef01647d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:52.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.731+0000 7f5ef7170700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ef016a750 0x7f5ef00c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:52.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.731+0000 7f5ef7170700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef016abd0 con 0x7f5edc003a90 2023-10-06T04:02:52.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.731+0000 7f5ef7170700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef0156e30 con 0x7f5ef00c58c0 2023-10-06T04:02:52.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.731+0000 7f5ef7170700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef00c5dc0 con 0x7f5ef016a750 2023-10-06T04:02:52.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.731+0000 7f5ef4f0c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5edc003a90 0x7f5ef0167340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:52.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.731+0000 7f5ef4f0c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5edc003a90 0x7f5ef0167340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44646/0 (socket says 172.21.15.157:44646) 2023-10-06T04:02:52.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.731+0000 7f5ef4f0c700 1 -- 172.21.15.157:0/561696423 learned_addr learned my addr 172.21.15.157:0/561696423 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:52.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.731+0000 7f5ef570d700 1 --2- 172.21.15.157:0/561696423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ef016a750 0x7f5ef00c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:52.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.731+0000 7f5eeffff700 1 --2- 172.21.15.157:0/561696423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ef00c58c0 0x7f5ef01647d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:52.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.731+0000 7f5ef4f0c700 1 -- 172.21.15.157:0/561696423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ef016a750 msgr2=0x7f5ef00c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:52.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.731+0000 7f5ef4f0c700 1 --2- 172.21.15.157:0/561696423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ef016a750 0x7f5ef00c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:52.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.731+0000 7f5ef4f0c700 1 -- 172.21.15.157:0/561696423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ef00c58c0 msgr2=0x7f5ef01647d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:52.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.731+0000 7f5ef4f0c700 1 --2- 172.21.15.157:0/561696423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ef00c58c0 0x7f5ef01647d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:52.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.731+0000 7f5ef4f0c700 1 -- 172.21.15.157:0/561696423 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef00c60a0 con 0x7f5edc003a90 2023-10-06T04:02:52.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.731+0000 7f5eeffff700 1 --2- 172.21.15.157:0/561696423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ef00c58c0 0x7f5ef01647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:02:52.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.731+0000 7f5ef4f0c700 1 --2- 172.21.15.157:0/561696423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5edc003a90 0x7f5ef0167340 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8001cc0 tx=0x7f5ee8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:52.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.731+0000 7f5eecff9700 1 -- 172.21.15.157:0/561696423 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8027470 con 0x7f5edc003a90 2023-10-06T04:02:52.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.731+0000 7f5eecff9700 1 -- 172.21.15.157:0/561696423 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee80275d0 con 0x7f5edc003a90 2023-10-06T04:02:52.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.731+0000 7f5eecff9700 1 -- 172.21.15.157:0/561696423 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8027990 con 0x7f5edc003a90 2023-10-06T04:02:52.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.731+0000 7f5ef7170700 1 -- 172.21.15.157:0/561696423 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef00c6380 con 0x7f5edc003a90 2023-10-06T04:02:52.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.731+0000 7f5ef7170700 1 -- 172.21.15.157:0/561696423 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef0166e20 con 0x7f5edc003a90 2023-10-06T04:02:52.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.732+0000 7f5ef7170700 1 -- 172.21.15.157:0/561696423 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef0167010 con 0x7f5edc003a90 2023-10-06T04:02:52.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.733+0000 7f5eecff9700 1 -- 172.21.15.157:0/561696423 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5ee8024070 con 0x7f5edc003a90 2023-10-06T04:02:52.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.733+0000 7f5eecff9700 1 --2- 172.21.15.157:0/561696423 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5ed0066f60 0x7f5ed0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:52.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.733+0000 7f5eecff9700 1 -- 172.21.15.157:0/561696423 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7f5ee8086e20 con 0x7f5edc003a90 2023-10-06T04:02:52.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.733+0000 7f5eeffff700 1 --2- 172.21.15.157:0/561696423 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5ed0066f60 0x7f5ed0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:52.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.734+0000 7f5eeffff700 1 --2- 172.21.15.157:0/561696423 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5ed0066f60 0x7f5ed0069410 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0006d60 tx=0x7f5ee0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:52.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.737+0000 7f5eecff9700 1 -- 172.21.15.157:0/561696423 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ee8053ce0 con 0x7f5edc003a90 2023-10-06T04:02:52.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.965+0000 7f5ef7170700 1 -- 172.21.15.157:0/561696423 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ef00c9130 con 0x7f5edc003a90 2023-10-06T04:02:52.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.966+0000 7f5eecff9700 1 -- 172.21.15.157:0/561696423 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+6178 (secure 0 0 0) 0x7f5ee8057300 con 0x7f5edc003a90 2023-10-06T04:02:52.970 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:02:52.970 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:02:46.949420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2,3],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270},"failed":[],"damaged":[],"stopped":[4],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:stopping","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:02:52.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.968+0000 7f5ef7170700 1 -- 172.21.15.157:0/561696423 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5ed0066f60 msgr2=0x7f5ed0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:52.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.968+0000 7f5ef7170700 1 --2- 172.21.15.157:0/561696423 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5ed0066f60 0x7f5ed0069410 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0006d60 tx=0x7f5ee0008040 comp rx=0 tx=0).stop 2023-10-06T04:02:52.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.968+0000 7f5ef7170700 1 -- 172.21.15.157:0/561696423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5edc003a90 msgr2=0x7f5ef0167340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:52.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.968+0000 7f5ef7170700 1 --2- 172.21.15.157:0/561696423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5edc003a90 0x7f5ef0167340 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8001cc0 tx=0x7f5ee8016000 comp rx=0 tx=0).stop 2023-10-06T04:02:52.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.968+0000 7f5ef7170700 1 -- 172.21.15.157:0/561696423 shutdown_connections 2023-10-06T04:02:52.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.968+0000 7f5ef7170700 1 --2- 172.21.15.157:0/561696423 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5ed0066f60 0x7f5ed0069410 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:52.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.968+0000 7f5ef7170700 1 --2- 172.21.15.157:0/561696423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ef016a750 0x7f5ef00c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:52.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.968+0000 7f5ef7170700 1 --2- 172.21.15.157:0/561696423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5edc003a90 0x7f5ef0167340 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:52.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.968+0000 7f5ef7170700 1 --2- 172.21.15.157:0/561696423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ef00c58c0 0x7f5ef01647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:52.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.968+0000 7f5ef7170700 1 -- 172.21.15.157:0/561696423 >> 172.21.15.157:0/561696423 conn(0x7f5ef006f230 msgr2=0x7f5ef014d780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:52.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.968+0000 7f5ef7170700 1 -- 172.21.15.157:0/561696423 shutdown_connections 2023-10-06T04:02:52.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:52.968+0000 7f5ef7170700 1 -- 172.21.15.157:0/561696423 wait complete. 2023-10-06T04:02:52.972 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 45 2023-10-06T04:02:52.983 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2023-10-06T04:02:52.984 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 45, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-06T03:57:10.718782+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 45, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0, 1, 2, 3 ], "info": { "gid_4270": { "addr": "172.21.15.175:6835/1596042861", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6834", "nonce": 1596042861, "type": "v2" }, { "addr": "172.21.15.175:6835", "nonce": 1596042861, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1, 2 ], "features": 4540138320759226367, "flags": 0, "gid": 4270, "incarnation": 13, "join_fscid": -1, "name": "b", "rank": 3, "state": "up:stopping", "state_seq": 8 }, "gid_4279": { "addr": "172.21.15.175:6837/1002404870", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6836", "nonce": 1002404870, "type": "v2" }, { "addr": "172.21.15.175:6837", "nonce": 1002404870, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 3 ], "features": 4540138320759226367, "flags": 0, "gid": 4279, "incarnation": 11, "join_fscid": -1, "name": "d", "rank": 2, "state": "up:active", "state_seq": 7 }, "gid_4287": { "addr": "172.21.15.157:6834/1179753185", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6832", "nonce": 1179753185, "type": "v2" }, { "addr": "172.21.15.157:6834", "nonce": 1179753185, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4287, "incarnation": 9, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4296": { "addr": "172.21.15.157:6837/4188652695", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6836", "nonce": 4188652695, "type": "v2" }, { "addr": "172.21.15.157:6837", "nonce": 4188652695, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 2 ], "features": 4540138320759226367, "flags": 0, "gid": 4296, "incarnation": 4, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 27, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2023-10-06T04:02:46.949420+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 4 ], "tableserver": 0, "up": { "mds_0": 4296, "mds_1": 4287, "mds_2": 4279, "mds_3": 4270 } } } ], "standbys": [ { "addr": "172.21.15.175:6839/745804271", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6838", "nonce": 745804271, "type": "v2" }, { "addr": "172.21.15.175:6839", "nonce": 745804271, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 42, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5164, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.157:6835/2461751318", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6833", "nonce": 2461751318, "type": "v2" }, { "addr": "172.21.15.157:6835", "nonce": 2461751318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 34, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5268, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-06T04:02:53.372 INFO:tasks.workunit.client.0.smithi157.stdout:removed './138/32' 2023-10-06T04:02:53.372 INFO:tasks.workunit.client.0.smithi157.stdout:removed './138/30' 2023-10-06T04:02:53.372 INFO:tasks.workunit.client.0.smithi157.stdout:removed './138/14' 2023-10-06T04:02:53.372 INFO:tasks.workunit.client.0.smithi157.stdout:removed './138/18' 2023-10-06T04:02:53.372 INFO:tasks.workunit.client.0.smithi157.stdout:removed './138/13' 2023-10-06T04:02:53.372 INFO:tasks.workunit.client.0.smithi157.stdout:removed './138/33' 2023-10-06T04:02:53.372 INFO:tasks.workunit.client.0.smithi157.stdout:removed './138/17' 2023-10-06T04:02:53.372 INFO:tasks.workunit.client.0.smithi157.stdout:removed './138/35' 2023-10-06T04:02:53.372 INFO:tasks.workunit.client.0.smithi157.stdout:removed './138/37' 2023-10-06T04:02:53.373 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './138' 2023-10-06T04:02:53.373 INFO:tasks.workunit.client.0.smithi157.stdout:removed './139/19' 2023-10-06T04:02:53.373 INFO:tasks.workunit.client.0.smithi157.stdout:removed './139/15' 2023-10-06T04:02:53.373 INFO:tasks.workunit.client.0.smithi157.stdout:removed './139/31' 2023-10-06T04:02:53.373 INFO:tasks.workunit.client.0.smithi157.stdout:removed './139/36' 2023-10-06T04:02:53.373 INFO:tasks.workunit.client.0.smithi157.stdout:removed './139/20' 2023-10-06T04:02:53.373 INFO:tasks.workunit.client.0.smithi157.stdout:removed './139/12' 2023-10-06T04:02:53.373 INFO:tasks.workunit.client.0.smithi157.stdout:removed './139/39' 2023-10-06T04:02:53.373 INFO:tasks.workunit.client.0.smithi157.stdout:removed './139/10' 2023-10-06T04:02:53.373 INFO:tasks.workunit.client.0.smithi157.stdout:removed './139/34' 2023-10-06T04:02:53.373 INFO:tasks.workunit.client.0.smithi157.stdout:removed './139/38' 2023-10-06T04:02:53.373 INFO:tasks.workunit.client.0.smithi157.stdout:removed './139/11' 2023-10-06T04:02:53.374 INFO:tasks.workunit.client.0.smithi157.stdout:removed './139/16' 2023-10-06T04:02:53.374 INFO:tasks.workunit.client.0.smithi157.stdout:removed './139/32' 2023-10-06T04:02:53.374 INFO:tasks.workunit.client.0.smithi157.stdout:removed './139/30' 2023-10-06T04:02:53.374 INFO:tasks.workunit.client.0.smithi157.stdout:removed './139/14' 2023-10-06T04:02:53.374 INFO:tasks.workunit.client.0.smithi157.stdout:removed './139/18' 2023-10-06T04:02:53.374 INFO:tasks.workunit.client.0.smithi157.stdout:removed './139/13' 2023-10-06T04:02:53.374 INFO:tasks.workunit.client.0.smithi157.stdout:removed './139/33' 2023-10-06T04:02:53.374 INFO:tasks.workunit.client.0.smithi157.stdout:removed './139/17' 2023-10-06T04:02:53.374 INFO:tasks.workunit.client.0.smithi157.stdout:removed './139/35' 2023-10-06T04:02:53.374 INFO:tasks.workunit.client.0.smithi157.stdout:removed './139/37' 2023-10-06T04:02:53.374 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './139' 2023-10-06T04:02:53.374 INFO:tasks.workunit.client.0.smithi157.stdout:removed './140/19' 2023-10-06T04:02:53.375 INFO:tasks.workunit.client.0.smithi157.stdout:removed './140/15' 2023-10-06T04:02:53.375 INFO:tasks.workunit.client.0.smithi157.stdout:removed './140/31' 2023-10-06T04:02:53.375 INFO:tasks.workunit.client.0.smithi157.stdout:removed './140/36' 2023-10-06T04:02:53.375 INFO:tasks.workunit.client.0.smithi157.stdout:removed './140/20' 2023-10-06T04:02:53.375 INFO:tasks.workunit.client.0.smithi157.stdout:removed './140/12' 2023-10-06T04:02:53.375 INFO:tasks.workunit.client.0.smithi157.stdout:removed './140/39' 2023-10-06T04:02:53.375 INFO:tasks.workunit.client.0.smithi157.stdout:removed './140/10' 2023-10-06T04:02:53.375 INFO:tasks.workunit.client.0.smithi157.stdout:removed './140/34' 2023-10-06T04:02:53.375 INFO:tasks.workunit.client.0.smithi157.stdout:removed './140/38' 2023-10-06T04:02:53.375 INFO:tasks.workunit.client.0.smithi157.stdout:removed './140/11' 2023-10-06T04:02:53.375 INFO:tasks.workunit.client.0.smithi157.stdout:removed './140/16' 2023-10-06T04:02:53.375 INFO:tasks.workunit.client.0.smithi157.stdout:removed './140/32' 2023-10-06T04:02:53.376 INFO:tasks.workunit.client.0.smithi157.stdout:removed './140/30' 2023-10-06T04:02:53.376 INFO:tasks.workunit.client.0.smithi157.stdout:removed './140/14' 2023-10-06T04:02:53.376 INFO:tasks.workunit.client.0.smithi157.stdout:removed './140/18' 2023-10-06T04:02:53.376 INFO:tasks.workunit.client.0.smithi157.stdout:removed './140/13' 2023-10-06T04:02:53.376 INFO:tasks.workunit.client.0.smithi157.stdout:removed './140/33' 2023-10-06T04:02:53.376 INFO:tasks.workunit.client.0.smithi157.stdout:removed './140/17' 2023-10-06T04:02:53.376 INFO:tasks.workunit.client.0.smithi157.stdout:removed './140/35' 2023-10-06T04:02:53.376 INFO:tasks.workunit.client.0.smithi157.stdout:removed './140/37' 2023-10-06T04:02:53.376 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './140' 2023-10-06T04:02:53.376 INFO:tasks.workunit.client.0.smithi157.stdout:removed './141/19' 2023-10-06T04:02:53.376 INFO:tasks.workunit.client.0.smithi157.stdout:removed './141/15' 2023-10-06T04:02:53.376 INFO:tasks.workunit.client.0.smithi157.stdout:removed './141/31' 2023-10-06T04:02:53.376 INFO:tasks.workunit.client.0.smithi157.stdout:removed './141/36' 2023-10-06T04:02:53.377 INFO:tasks.workunit.client.0.smithi157.stdout:removed './141/20' 2023-10-06T04:02:53.377 INFO:tasks.workunit.client.0.smithi157.stdout:removed './141/12' 2023-10-06T04:02:53.377 INFO:tasks.workunit.client.0.smithi157.stdout:removed './141/39' 2023-10-06T04:02:53.377 INFO:tasks.workunit.client.0.smithi157.stdout:removed './141/10' 2023-10-06T04:02:53.377 INFO:tasks.workunit.client.0.smithi157.stdout:removed './141/34' 2023-10-06T04:02:53.377 INFO:tasks.workunit.client.0.smithi157.stdout:removed './141/38' 2023-10-06T04:02:53.377 INFO:tasks.workunit.client.0.smithi157.stdout:removed './141/11' 2023-10-06T04:02:53.377 INFO:tasks.workunit.client.0.smithi157.stdout:removed './141/16' 2023-10-06T04:02:53.377 INFO:tasks.workunit.client.0.smithi157.stdout:removed './141/32' 2023-10-06T04:02:53.377 INFO:tasks.workunit.client.0.smithi157.stdout:removed './141/30' 2023-10-06T04:02:53.377 INFO:tasks.workunit.client.0.smithi157.stdout:removed './141/14' 2023-10-06T04:02:53.377 INFO:tasks.workunit.client.0.smithi157.stdout:removed './141/18' 2023-10-06T04:02:53.377 INFO:tasks.workunit.client.0.smithi157.stdout:removed './141/13' 2023-10-06T04:02:53.378 INFO:tasks.workunit.client.0.smithi157.stdout:removed './141/33' 2023-10-06T04:02:53.378 INFO:tasks.workunit.client.0.smithi157.stdout:removed './141/17' 2023-10-06T04:02:53.378 INFO:tasks.workunit.client.0.smithi157.stdout:removed './141/35' 2023-10-06T04:02:53.378 INFO:tasks.workunit.client.0.smithi157.stdout:removed './141/37' 2023-10-06T04:02:53.378 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './141' 2023-10-06T04:02:53.378 INFO:tasks.workunit.client.0.smithi157.stdout:removed './142/19' 2023-10-06T04:02:53.378 INFO:tasks.workunit.client.0.smithi157.stdout:removed './142/15' 2023-10-06T04:02:53.378 INFO:tasks.workunit.client.0.smithi157.stdout:removed './142/31' 2023-10-06T04:02:53.378 INFO:tasks.workunit.client.0.smithi157.stdout:removed './142/36' 2023-10-06T04:02:53.378 INFO:tasks.workunit.client.0.smithi157.stdout:removed './142/20' 2023-10-06T04:02:53.378 INFO:tasks.workunit.client.0.smithi157.stdout:removed './142/12' 2023-10-06T04:02:53.378 INFO:tasks.workunit.client.0.smithi157.stdout:removed './142/39' 2023-10-06T04:02:53.379 INFO:tasks.workunit.client.0.smithi157.stdout:removed './142/10' 2023-10-06T04:02:53.379 INFO:tasks.workunit.client.0.smithi157.stdout:removed './142/34' 2023-10-06T04:02:53.379 INFO:tasks.workunit.client.0.smithi157.stdout:removed './142/38' 2023-10-06T04:02:53.379 INFO:tasks.workunit.client.0.smithi157.stdout:removed './142/11' 2023-10-06T04:02:53.379 INFO:tasks.workunit.client.0.smithi157.stdout:removed './142/16' 2023-10-06T04:02:53.379 INFO:tasks.workunit.client.0.smithi157.stdout:removed './142/32' 2023-10-06T04:02:53.379 INFO:tasks.workunit.client.0.smithi157.stdout:removed './142/30' 2023-10-06T04:02:53.379 INFO:tasks.workunit.client.0.smithi157.stdout:removed './142/14' 2023-10-06T04:02:53.379 INFO:tasks.workunit.client.0.smithi157.stdout:removed './142/18' 2023-10-06T04:02:53.379 INFO:tasks.workunit.client.0.smithi157.stdout:removed './142/13' 2023-10-06T04:02:53.379 INFO:tasks.workunit.client.0.smithi157.stdout:removed './142/33' 2023-10-06T04:02:53.379 INFO:tasks.workunit.client.0.smithi157.stdout:removed './142/17' 2023-10-06T04:02:53.380 INFO:tasks.workunit.client.0.smithi157.stdout:removed './142/35' 2023-10-06T04:02:53.380 INFO:tasks.workunit.client.0.smithi157.stdout:removed './142/37' 2023-10-06T04:02:53.380 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './142' 2023-10-06T04:02:53.380 INFO:tasks.workunit.client.0.smithi157.stdout:removed './143/19' 2023-10-06T04:02:53.380 INFO:tasks.workunit.client.0.smithi157.stdout:removed './143/15' 2023-10-06T04:02:53.380 INFO:tasks.workunit.client.0.smithi157.stdout:removed './143/31' 2023-10-06T04:02:53.380 INFO:tasks.workunit.client.0.smithi157.stdout:removed './143/36' 2023-10-06T04:02:53.380 INFO:tasks.workunit.client.0.smithi157.stdout:removed './143/20' 2023-10-06T04:02:53.380 INFO:tasks.workunit.client.0.smithi157.stdout:removed './143/12' 2023-10-06T04:02:53.380 INFO:tasks.workunit.client.0.smithi157.stdout:removed './143/39' 2023-10-06T04:02:53.380 INFO:tasks.workunit.client.0.smithi157.stdout:removed './143/10' 2023-10-06T04:02:53.380 INFO:tasks.workunit.client.0.smithi157.stdout:removed './143/34' 2023-10-06T04:02:53.380 INFO:tasks.workunit.client.0.smithi157.stdout:removed './143/38' 2023-10-06T04:02:53.381 INFO:tasks.workunit.client.0.smithi157.stdout:removed './143/11' 2023-10-06T04:02:53.381 INFO:tasks.workunit.client.0.smithi157.stdout:removed './143/16' 2023-10-06T04:02:53.381 INFO:tasks.workunit.client.0.smithi157.stdout:removed './143/32' 2023-10-06T04:02:53.381 INFO:tasks.workunit.client.0.smithi157.stdout:removed './143/30' 2023-10-06T04:02:53.381 INFO:tasks.workunit.client.0.smithi157.stdout:removed './143/14' 2023-10-06T04:02:53.381 INFO:tasks.workunit.client.0.smithi157.stdout:removed './143/18' 2023-10-06T04:02:53.381 INFO:tasks.workunit.client.0.smithi157.stdout:removed './143/13' 2023-10-06T04:02:53.381 INFO:tasks.workunit.client.0.smithi157.stdout:removed './143/33' 2023-10-06T04:02:53.381 INFO:tasks.workunit.client.0.smithi157.stdout:removed './143/17' 2023-10-06T04:02:53.381 INFO:tasks.workunit.client.0.smithi157.stdout:removed './143/35' 2023-10-06T04:02:53.381 INFO:tasks.workunit.client.0.smithi157.stdout:removed './143/37' 2023-10-06T04:02:53.381 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './143' 2023-10-06T04:02:53.382 INFO:tasks.workunit.client.0.smithi157.stdout:removed './144/19' 2023-10-06T04:02:53.382 INFO:tasks.workunit.client.0.smithi157.stdout:removed './144/15' 2023-10-06T04:02:53.382 INFO:tasks.workunit.client.0.smithi157.stdout:removed './144/31' 2023-10-06T04:02:53.382 INFO:tasks.workunit.client.0.smithi157.stdout:removed './144/36' 2023-10-06T04:02:53.382 INFO:tasks.workunit.client.0.smithi157.stdout:removed './144/20' 2023-10-06T04:02:53.382 INFO:tasks.workunit.client.0.smithi157.stdout:removed './144/12' 2023-10-06T04:02:53.382 INFO:tasks.workunit.client.0.smithi157.stdout:removed './144/39' 2023-10-06T04:02:53.382 INFO:tasks.workunit.client.0.smithi157.stdout:removed './144/10' 2023-10-06T04:02:53.382 INFO:tasks.workunit.client.0.smithi157.stdout:removed './144/34' 2023-10-06T04:02:53.382 INFO:tasks.workunit.client.0.smithi157.stdout:removed './144/38' 2023-10-06T04:02:53.382 INFO:tasks.workunit.client.0.smithi157.stdout:removed './144/11' 2023-10-06T04:02:53.382 INFO:tasks.workunit.client.0.smithi157.stdout:removed './144/16' 2023-10-06T04:02:53.383 INFO:tasks.workunit.client.0.smithi157.stdout:removed './144/32' 2023-10-06T04:02:53.383 INFO:tasks.workunit.client.0.smithi157.stdout:removed './144/30' 2023-10-06T04:02:53.383 INFO:tasks.workunit.client.0.smithi157.stdout:removed './144/14' 2023-10-06T04:02:53.383 INFO:tasks.workunit.client.0.smithi157.stdout:removed './144/18' 2023-10-06T04:02:53.383 INFO:tasks.workunit.client.0.smithi157.stdout:removed './144/13' 2023-10-06T04:02:53.383 INFO:tasks.workunit.client.0.smithi157.stdout:removed './144/33' 2023-10-06T04:02:53.383 INFO:tasks.workunit.client.0.smithi157.stdout:removed './144/17' 2023-10-06T04:02:53.383 INFO:tasks.workunit.client.0.smithi157.stdout:removed './144/35' 2023-10-06T04:02:53.383 INFO:tasks.workunit.client.0.smithi157.stdout:removed './144/37' 2023-10-06T04:02:53.383 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './144' 2023-10-06T04:02:53.383 INFO:tasks.workunit.client.0.smithi157.stdout:removed './145/19' 2023-10-06T04:02:53.383 INFO:tasks.workunit.client.0.smithi157.stdout:removed './145/15' 2023-10-06T04:02:53.383 INFO:tasks.workunit.client.0.smithi157.stdout:removed './145/31' 2023-10-06T04:02:53.384 INFO:tasks.workunit.client.0.smithi157.stdout:removed './145/36' 2023-10-06T04:02:53.384 INFO:tasks.workunit.client.0.smithi157.stdout:removed './145/20' 2023-10-06T04:02:53.384 INFO:tasks.workunit.client.0.smithi157.stdout:removed './145/12' 2023-10-06T04:02:53.384 INFO:tasks.workunit.client.0.smithi157.stdout:removed './145/39' 2023-10-06T04:02:53.384 INFO:tasks.workunit.client.0.smithi157.stdout:removed './145/10' 2023-10-06T04:02:53.384 INFO:tasks.workunit.client.0.smithi157.stdout:removed './145/34' 2023-10-06T04:02:53.384 INFO:tasks.workunit.client.0.smithi157.stdout:removed './145/38' 2023-10-06T04:02:53.384 INFO:tasks.workunit.client.0.smithi157.stdout:removed './145/11' 2023-10-06T04:02:53.384 INFO:tasks.workunit.client.0.smithi157.stdout:removed './145/16' 2023-10-06T04:02:53.384 INFO:tasks.workunit.client.0.smithi157.stdout:removed './145/32' 2023-10-06T04:02:53.384 INFO:tasks.workunit.client.0.smithi157.stdout:removed './145/30' 2023-10-06T04:02:53.384 INFO:tasks.workunit.client.0.smithi157.stdout:removed './145/14' 2023-10-06T04:02:53.385 INFO:tasks.workunit.client.0.smithi157.stdout:removed './145/18' 2023-10-06T04:02:53.385 INFO:tasks.workunit.client.0.smithi157.stdout:removed './145/13' 2023-10-06T04:02:53.385 INFO:tasks.workunit.client.0.smithi157.stdout:removed './145/33' 2023-10-06T04:02:53.385 INFO:tasks.workunit.client.0.smithi157.stdout:removed './145/17' 2023-10-06T04:02:53.385 INFO:tasks.workunit.client.0.smithi157.stdout:removed './145/35' 2023-10-06T04:02:53.385 INFO:tasks.workunit.client.0.smithi157.stdout:removed './145/37' 2023-10-06T04:02:53.385 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './145' 2023-10-06T04:02:53.385 INFO:tasks.workunit.client.0.smithi157.stdout:removed './146/19' 2023-10-06T04:02:53.385 INFO:tasks.workunit.client.0.smithi157.stdout:removed './146/15' 2023-10-06T04:02:53.385 INFO:tasks.workunit.client.0.smithi157.stdout:removed './146/31' 2023-10-06T04:02:53.385 INFO:tasks.workunit.client.0.smithi157.stdout:removed './146/36' 2023-10-06T04:02:53.385 INFO:tasks.workunit.client.0.smithi157.stdout:removed './146/20' 2023-10-06T04:02:53.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './146/12' 2023-10-06T04:02:53.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './146/39' 2023-10-06T04:02:53.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './146/10' 2023-10-06T04:02:53.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './146/34' 2023-10-06T04:02:53.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './146/38' 2023-10-06T04:02:53.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './146/11' 2023-10-06T04:02:53.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './146/16' 2023-10-06T04:02:53.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './146/32' 2023-10-06T04:02:53.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './146/30' 2023-10-06T04:02:53.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './146/14' 2023-10-06T04:02:53.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './146/18' 2023-10-06T04:02:53.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './146/13' 2023-10-06T04:02:53.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './146/33' 2023-10-06T04:02:53.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './146/17' 2023-10-06T04:02:53.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './146/35' 2023-10-06T04:02:53.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './146/37' 2023-10-06T04:02:53.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './146' 2023-10-06T04:02:53.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './147/19' 2023-10-06T04:02:53.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './147/15' 2023-10-06T04:02:53.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './147/31' 2023-10-06T04:02:53.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './147/36' 2023-10-06T04:02:53.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './147/20' 2023-10-06T04:02:53.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './147/12' 2023-10-06T04:02:53.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './147/39' 2023-10-06T04:02:53.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './147/10' 2023-10-06T04:02:53.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed './147/34' 2023-10-06T04:02:53.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed './147/38' 2023-10-06T04:02:53.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed './147/11' 2023-10-06T04:02:53.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed './147/16' 2023-10-06T04:02:53.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed './147/32' 2023-10-06T04:02:53.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed './147/30' 2023-10-06T04:02:53.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed './147/14' 2023-10-06T04:02:53.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed './147/18' 2023-10-06T04:02:53.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed './147/13' 2023-10-06T04:02:53.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed './147/33' 2023-10-06T04:02:53.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed './147/17' 2023-10-06T04:02:53.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed './147/35' 2023-10-06T04:02:53.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed './147/37' 2023-10-06T04:02:53.389 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './147' 2023-10-06T04:02:53.389 INFO:tasks.workunit.client.0.smithi157.stdout:removed './148/19' 2023-10-06T04:02:53.389 INFO:tasks.workunit.client.0.smithi157.stdout:removed './148/15' 2023-10-06T04:02:53.389 INFO:tasks.workunit.client.0.smithi157.stdout:removed './148/31' 2023-10-06T04:02:53.389 INFO:tasks.workunit.client.0.smithi157.stdout:removed './148/36' 2023-10-06T04:02:54.080 INFO:tasks.workunit.client.0.smithi157.stdout:removed './148/20' 2023-10-06T04:02:54.080 INFO:tasks.workunit.client.0.smithi157.stdout:removed './148/12' 2023-10-06T04:02:54.080 INFO:tasks.workunit.client.0.smithi157.stdout:removed './148/39' 2023-10-06T04:02:54.080 INFO:tasks.workunit.client.0.smithi157.stdout:removed './148/10' 2023-10-06T04:02:54.080 INFO:tasks.workunit.client.0.smithi157.stdout:removed './148/34' 2023-10-06T04:02:54.080 INFO:tasks.workunit.client.0.smithi157.stdout:removed './148/38' 2023-10-06T04:02:54.080 INFO:tasks.workunit.client.0.smithi157.stdout:removed './148/11' 2023-10-06T04:02:54.081 INFO:tasks.workunit.client.0.smithi157.stdout:removed './148/16' 2023-10-06T04:02:54.081 INFO:tasks.workunit.client.0.smithi157.stdout:removed './148/32' 2023-10-06T04:02:54.081 INFO:tasks.workunit.client.0.smithi157.stdout:removed './148/30' 2023-10-06T04:02:54.081 INFO:tasks.workunit.client.0.smithi157.stdout:removed './148/14' 2023-10-06T04:02:54.081 INFO:tasks.workunit.client.0.smithi157.stdout:removed './148/18' 2023-10-06T04:02:54.081 INFO:tasks.workunit.client.0.smithi157.stdout:removed './148/13' 2023-10-06T04:02:54.082 INFO:tasks.workunit.client.0.smithi157.stdout:removed './148/33' 2023-10-06T04:02:54.082 INFO:tasks.workunit.client.0.smithi157.stdout:removed './148/17' 2023-10-06T04:02:54.082 INFO:tasks.workunit.client.0.smithi157.stdout:removed './148/35' 2023-10-06T04:02:54.082 INFO:tasks.workunit.client.0.smithi157.stdout:removed './148/37' 2023-10-06T04:02:54.082 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './148' 2023-10-06T04:02:54.083 INFO:tasks.workunit.client.0.smithi157.stdout:removed './149/19' 2023-10-06T04:02:54.083 INFO:tasks.workunit.client.0.smithi157.stdout:removed './149/15' 2023-10-06T04:02:54.083 INFO:tasks.workunit.client.0.smithi157.stdout:removed './149/31' 2023-10-06T04:02:54.083 INFO:tasks.workunit.client.0.smithi157.stdout:removed './149/36' 2023-10-06T04:02:54.083 INFO:tasks.workunit.client.0.smithi157.stdout:removed './149/20' 2023-10-06T04:02:54.083 INFO:tasks.workunit.client.0.smithi157.stdout:removed './149/12' 2023-10-06T04:02:54.084 INFO:tasks.workunit.client.0.smithi157.stdout:removed './149/39' 2023-10-06T04:02:54.084 INFO:tasks.workunit.client.0.smithi157.stdout:removed './149/10' 2023-10-06T04:02:54.084 INFO:tasks.workunit.client.0.smithi157.stdout:removed './149/34' 2023-10-06T04:02:54.084 INFO:tasks.workunit.client.0.smithi157.stdout:removed './149/38' 2023-10-06T04:02:54.084 INFO:tasks.workunit.client.0.smithi157.stdout:removed './149/11' 2023-10-06T04:02:54.084 INFO:tasks.workunit.client.0.smithi157.stdout:removed './149/16' 2023-10-06T04:02:54.084 INFO:tasks.workunit.client.0.smithi157.stdout:removed './149/32' 2023-10-06T04:02:54.084 INFO:tasks.workunit.client.0.smithi157.stdout:removed './149/30' 2023-10-06T04:02:54.085 INFO:tasks.workunit.client.0.smithi157.stdout:removed './149/14' 2023-10-06T04:02:54.085 INFO:tasks.workunit.client.0.smithi157.stdout:removed './149/18' 2023-10-06T04:02:54.085 INFO:tasks.workunit.client.0.smithi157.stdout:removed './149/13' 2023-10-06T04:02:54.085 INFO:tasks.workunit.client.0.smithi157.stdout:removed './149/33' 2023-10-06T04:02:54.085 INFO:tasks.workunit.client.0.smithi157.stdout:removed './149/17' 2023-10-06T04:02:54.085 INFO:tasks.workunit.client.0.smithi157.stdout:removed './149/35' 2023-10-06T04:02:54.085 INFO:tasks.workunit.client.0.smithi157.stdout:removed './149/37' 2023-10-06T04:02:54.085 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './149' 2023-10-06T04:02:54.086 INFO:tasks.workunit.client.0.smithi157.stdout:removed './150/19' 2023-10-06T04:02:54.086 INFO:tasks.workunit.client.0.smithi157.stdout:removed './150/15' 2023-10-06T04:02:54.086 INFO:tasks.workunit.client.0.smithi157.stdout:removed './150/31' 2023-10-06T04:02:54.086 INFO:tasks.workunit.client.0.smithi157.stdout:removed './150/36' 2023-10-06T04:02:54.086 INFO:tasks.workunit.client.0.smithi157.stdout:removed './150/20' 2023-10-06T04:02:54.086 INFO:tasks.workunit.client.0.smithi157.stdout:removed './150/12' 2023-10-06T04:02:54.086 INFO:tasks.workunit.client.0.smithi157.stdout:removed './150/39' 2023-10-06T04:02:54.087 INFO:tasks.workunit.client.0.smithi157.stdout:removed './150/10' 2023-10-06T04:02:54.087 INFO:tasks.workunit.client.0.smithi157.stdout:removed './150/34' 2023-10-06T04:02:54.087 INFO:tasks.workunit.client.0.smithi157.stdout:removed './150/38' 2023-10-06T04:02:54.087 INFO:tasks.workunit.client.0.smithi157.stdout:removed './150/11' 2023-10-06T04:02:54.087 INFO:tasks.workunit.client.0.smithi157.stdout:removed './150/16' 2023-10-06T04:02:54.087 INFO:tasks.workunit.client.0.smithi157.stdout:removed './150/32' 2023-10-06T04:02:54.088 INFO:tasks.workunit.client.0.smithi157.stdout:removed './150/30' 2023-10-06T04:02:54.088 INFO:tasks.workunit.client.0.smithi157.stdout:removed './150/14' 2023-10-06T04:02:54.088 INFO:tasks.workunit.client.0.smithi157.stdout:removed './150/18' 2023-10-06T04:02:54.088 INFO:tasks.workunit.client.0.smithi157.stdout:removed './150/13' 2023-10-06T04:02:54.088 INFO:tasks.workunit.client.0.smithi157.stdout:removed './150/33' 2023-10-06T04:02:54.088 INFO:tasks.workunit.client.0.smithi157.stdout:removed './150/17' 2023-10-06T04:02:54.088 INFO:tasks.workunit.client.0.smithi157.stdout:removed './150/35' 2023-10-06T04:02:54.089 INFO:tasks.workunit.client.0.smithi157.stdout:removed './150/37' 2023-10-06T04:02:54.089 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './150' 2023-10-06T04:02:54.089 INFO:tasks.workunit.client.0.smithi157.stdout:removed './151/19' 2023-10-06T04:02:54.089 INFO:tasks.workunit.client.0.smithi157.stdout:removed './151/15' 2023-10-06T04:02:54.089 INFO:tasks.workunit.client.0.smithi157.stdout:removed './151/31' 2023-10-06T04:02:54.090 INFO:tasks.workunit.client.0.smithi157.stdout:removed './151/36' 2023-10-06T04:02:54.090 INFO:tasks.workunit.client.0.smithi157.stdout:removed './151/20' 2023-10-06T04:02:54.090 INFO:tasks.workunit.client.0.smithi157.stdout:removed './151/12' 2023-10-06T04:02:54.090 INFO:tasks.workunit.client.0.smithi157.stdout:removed './151/39' 2023-10-06T04:02:54.090 INFO:tasks.workunit.client.0.smithi157.stdout:removed './151/10' 2023-10-06T04:02:54.090 INFO:tasks.workunit.client.0.smithi157.stdout:removed './151/34' 2023-10-06T04:02:54.090 INFO:tasks.workunit.client.0.smithi157.stdout:removed './151/38' 2023-10-06T04:02:54.091 INFO:tasks.workunit.client.0.smithi157.stdout:removed './151/11' 2023-10-06T04:02:54.091 INFO:tasks.workunit.client.0.smithi157.stdout:removed './151/16' 2023-10-06T04:02:54.091 INFO:tasks.workunit.client.0.smithi157.stdout:removed './151/32' 2023-10-06T04:02:54.091 INFO:tasks.workunit.client.0.smithi157.stdout:removed './151/30' 2023-10-06T04:02:54.091 INFO:tasks.workunit.client.0.smithi157.stdout:removed './151/14' 2023-10-06T04:02:54.092 INFO:tasks.workunit.client.0.smithi157.stdout:removed './151/18' 2023-10-06T04:02:54.092 INFO:tasks.workunit.client.0.smithi157.stdout:removed './151/13' 2023-10-06T04:02:54.092 INFO:tasks.workunit.client.0.smithi157.stdout:removed './151/33' 2023-10-06T04:02:54.092 INFO:tasks.workunit.client.0.smithi157.stdout:removed './151/17' 2023-10-06T04:02:54.092 INFO:tasks.workunit.client.0.smithi157.stdout:removed './151/35' 2023-10-06T04:02:54.093 INFO:tasks.workunit.client.0.smithi157.stdout:removed './151/37' 2023-10-06T04:02:54.093 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './151' 2023-10-06T04:02:54.093 INFO:tasks.workunit.client.0.smithi157.stdout:removed './152/19' 2023-10-06T04:02:54.093 INFO:tasks.workunit.client.0.smithi157.stdout:removed './152/15' 2023-10-06T04:02:54.093 INFO:tasks.workunit.client.0.smithi157.stdout:removed './152/31' 2023-10-06T04:02:54.093 INFO:tasks.workunit.client.0.smithi157.stdout:removed './152/36' 2023-10-06T04:02:54.094 INFO:tasks.workunit.client.0.smithi157.stdout:removed './152/20' 2023-10-06T04:02:54.094 INFO:tasks.workunit.client.0.smithi157.stdout:removed './152/12' 2023-10-06T04:02:54.094 INFO:tasks.workunit.client.0.smithi157.stdout:removed './152/39' 2023-10-06T04:02:54.094 INFO:tasks.workunit.client.0.smithi157.stdout:removed './152/10' 2023-10-06T04:02:54.094 INFO:tasks.workunit.client.0.smithi157.stdout:removed './152/34' 2023-10-06T04:02:54.094 INFO:tasks.workunit.client.0.smithi157.stdout:removed './152/38' 2023-10-06T04:02:54.095 INFO:tasks.workunit.client.0.smithi157.stdout:removed './152/11' 2023-10-06T04:02:54.095 INFO:tasks.workunit.client.0.smithi157.stdout:removed './152/16' 2023-10-06T04:02:54.095 INFO:tasks.workunit.client.0.smithi157.stdout:removed './152/32' 2023-10-06T04:02:54.095 INFO:tasks.workunit.client.0.smithi157.stdout:removed './152/30' 2023-10-06T04:02:54.095 INFO:tasks.workunit.client.0.smithi157.stdout:removed './152/14' 2023-10-06T04:02:54.095 INFO:tasks.workunit.client.0.smithi157.stdout:removed './152/18' 2023-10-06T04:02:54.095 INFO:tasks.workunit.client.0.smithi157.stdout:removed './152/13' 2023-10-06T04:02:54.096 INFO:tasks.workunit.client.0.smithi157.stdout:removed './152/33' 2023-10-06T04:02:54.096 INFO:tasks.workunit.client.0.smithi157.stdout:removed './152/17' 2023-10-06T04:02:54.096 INFO:tasks.workunit.client.0.smithi157.stdout:removed './152/35' 2023-10-06T04:02:54.096 INFO:tasks.workunit.client.0.smithi157.stdout:removed './152/37' 2023-10-06T04:02:54.096 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './152' 2023-10-06T04:02:54.096 INFO:tasks.workunit.client.0.smithi157.stdout:removed './153/19' 2023-10-06T04:02:54.097 INFO:tasks.workunit.client.0.smithi157.stdout:removed './153/15' 2023-10-06T04:02:54.097 INFO:tasks.workunit.client.0.smithi157.stdout:removed './153/31' 2023-10-06T04:02:54.097 INFO:tasks.workunit.client.0.smithi157.stdout:removed './153/36' 2023-10-06T04:02:54.097 INFO:tasks.workunit.client.0.smithi157.stdout:removed './153/20' 2023-10-06T04:02:54.097 INFO:tasks.workunit.client.0.smithi157.stdout:removed './153/12' 2023-10-06T04:02:54.097 INFO:tasks.workunit.client.0.smithi157.stdout:removed './153/39' 2023-10-06T04:02:54.097 INFO:tasks.workunit.client.0.smithi157.stdout:removed './153/10' 2023-10-06T04:02:54.098 INFO:tasks.workunit.client.0.smithi157.stdout:removed './153/34' 2023-10-06T04:02:54.098 INFO:tasks.workunit.client.0.smithi157.stdout:removed './153/38' 2023-10-06T04:02:54.098 INFO:tasks.workunit.client.0.smithi157.stdout:removed './153/11' 2023-10-06T04:02:54.098 INFO:tasks.workunit.client.0.smithi157.stdout:removed './153/16' 2023-10-06T04:02:54.098 INFO:tasks.workunit.client.0.smithi157.stdout:removed './153/32' 2023-10-06T04:02:54.098 INFO:tasks.workunit.client.0.smithi157.stdout:removed './153/30' 2023-10-06T04:02:54.099 INFO:tasks.workunit.client.0.smithi157.stdout:removed './153/14' 2023-10-06T04:02:54.099 INFO:tasks.workunit.client.0.smithi157.stdout:removed './153/18' 2023-10-06T04:02:54.099 INFO:tasks.workunit.client.0.smithi157.stdout:removed './153/13' 2023-10-06T04:02:54.099 INFO:tasks.workunit.client.0.smithi157.stdout:removed './153/33' 2023-10-06T04:02:54.099 INFO:tasks.workunit.client.0.smithi157.stdout:removed './153/17' 2023-10-06T04:02:54.099 INFO:tasks.workunit.client.0.smithi157.stdout:removed './153/35' 2023-10-06T04:02:54.099 INFO:tasks.workunit.client.0.smithi157.stdout:removed './153/37' 2023-10-06T04:02:54.100 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './153' 2023-10-06T04:02:54.100 INFO:tasks.workunit.client.0.smithi157.stdout:removed './154/19' 2023-10-06T04:02:54.100 INFO:tasks.workunit.client.0.smithi157.stdout:removed './154/15' 2023-10-06T04:02:54.100 INFO:tasks.workunit.client.0.smithi157.stdout:removed './154/31' 2023-10-06T04:02:54.100 INFO:tasks.workunit.client.0.smithi157.stdout:removed './154/36' 2023-10-06T04:02:54.100 INFO:tasks.workunit.client.0.smithi157.stdout:removed './154/20' 2023-10-06T04:02:54.100 INFO:tasks.workunit.client.0.smithi157.stdout:removed './154/12' 2023-10-06T04:02:54.101 INFO:tasks.workunit.client.0.smithi157.stdout:removed './154/39' 2023-10-06T04:02:54.101 INFO:tasks.workunit.client.0.smithi157.stdout:removed './154/10' 2023-10-06T04:02:54.101 INFO:tasks.workunit.client.0.smithi157.stdout:removed './154/34' 2023-10-06T04:02:54.101 INFO:tasks.workunit.client.0.smithi157.stdout:removed './154/38' 2023-10-06T04:02:54.101 INFO:tasks.workunit.client.0.smithi157.stdout:removed './154/11' 2023-10-06T04:02:54.101 INFO:tasks.workunit.client.0.smithi157.stdout:removed './154/16' 2023-10-06T04:02:54.102 INFO:tasks.workunit.client.0.smithi157.stdout:removed './154/32' 2023-10-06T04:02:54.102 INFO:tasks.workunit.client.0.smithi157.stdout:removed './154/30' 2023-10-06T04:02:54.102 INFO:tasks.workunit.client.0.smithi157.stdout:removed './154/14' 2023-10-06T04:02:54.102 INFO:tasks.workunit.client.0.smithi157.stdout:removed './154/18' 2023-10-06T04:02:54.102 INFO:tasks.workunit.client.0.smithi157.stdout:removed './154/13' 2023-10-06T04:02:54.102 INFO:tasks.workunit.client.0.smithi157.stdout:removed './154/33' 2023-10-06T04:02:54.102 INFO:tasks.workunit.client.0.smithi157.stdout:removed './154/17' 2023-10-06T04:02:54.103 INFO:tasks.workunit.client.0.smithi157.stdout:removed './154/35' 2023-10-06T04:02:54.103 INFO:tasks.workunit.client.0.smithi157.stdout:removed './154/37' 2023-10-06T04:02:54.103 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './154' 2023-10-06T04:02:54.103 INFO:tasks.workunit.client.0.smithi157.stdout:removed './155/19' 2023-10-06T04:02:54.103 INFO:tasks.workunit.client.0.smithi157.stdout:removed './155/15' 2023-10-06T04:02:54.103 INFO:tasks.workunit.client.0.smithi157.stdout:removed './155/31' 2023-10-06T04:02:54.103 INFO:tasks.workunit.client.0.smithi157.stdout:removed './155/36' 2023-10-06T04:02:54.104 INFO:tasks.workunit.client.0.smithi157.stdout:removed './155/20' 2023-10-06T04:02:54.104 INFO:tasks.workunit.client.0.smithi157.stdout:removed './155/12' 2023-10-06T04:02:54.104 INFO:tasks.workunit.client.0.smithi157.stdout:removed './155/39' 2023-10-06T04:02:54.104 INFO:tasks.workunit.client.0.smithi157.stdout:removed './155/10' 2023-10-06T04:02:54.104 INFO:tasks.workunit.client.0.smithi157.stdout:removed './155/34' 2023-10-06T04:02:54.104 INFO:tasks.workunit.client.0.smithi157.stdout:removed './155/38' 2023-10-06T04:02:54.105 INFO:tasks.workunit.client.0.smithi157.stdout:removed './155/11' 2023-10-06T04:02:54.105 INFO:tasks.workunit.client.0.smithi157.stdout:removed './155/16' 2023-10-06T04:02:54.105 INFO:tasks.workunit.client.0.smithi157.stdout:removed './155/32' 2023-10-06T04:02:54.105 INFO:tasks.workunit.client.0.smithi157.stdout:removed './155/30' 2023-10-06T04:02:54.105 INFO:tasks.workunit.client.0.smithi157.stdout:removed './155/14' 2023-10-06T04:02:54.105 INFO:tasks.workunit.client.0.smithi157.stdout:removed './155/18' 2023-10-06T04:02:54.105 INFO:tasks.workunit.client.0.smithi157.stdout:removed './155/13' 2023-10-06T04:02:54.106 INFO:tasks.workunit.client.0.smithi157.stdout:removed './155/33' 2023-10-06T04:02:54.106 INFO:tasks.workunit.client.0.smithi157.stdout:removed './155/17' 2023-10-06T04:02:54.106 INFO:tasks.workunit.client.0.smithi157.stdout:removed './155/35' 2023-10-06T04:02:54.106 INFO:tasks.workunit.client.0.smithi157.stdout:removed './155/37' 2023-10-06T04:02:54.106 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './155' 2023-10-06T04:02:54.106 INFO:tasks.workunit.client.0.smithi157.stdout:removed './156/19' 2023-10-06T04:02:54.106 INFO:tasks.workunit.client.0.smithi157.stdout:removed './156/15' 2023-10-06T04:02:54.107 INFO:tasks.workunit.client.0.smithi157.stdout:removed './156/31' 2023-10-06T04:02:54.107 INFO:tasks.workunit.client.0.smithi157.stdout:removed './156/36' 2023-10-06T04:02:54.107 INFO:tasks.workunit.client.0.smithi157.stdout:removed './156/20' 2023-10-06T04:02:54.107 INFO:tasks.workunit.client.0.smithi157.stdout:removed './156/12' 2023-10-06T04:02:54.107 INFO:tasks.workunit.client.0.smithi157.stdout:removed './156/39' 2023-10-06T04:02:54.107 INFO:tasks.workunit.client.0.smithi157.stdout:removed './156/10' 2023-10-06T04:02:54.107 INFO:tasks.workunit.client.0.smithi157.stdout:removed './156/34' 2023-10-06T04:02:54.108 INFO:tasks.workunit.client.0.smithi157.stdout:removed './156/38' 2023-10-06T04:02:54.108 INFO:tasks.workunit.client.0.smithi157.stdout:removed './156/11' 2023-10-06T04:02:54.108 INFO:tasks.workunit.client.0.smithi157.stdout:removed './156/16' 2023-10-06T04:02:54.108 INFO:tasks.workunit.client.0.smithi157.stdout:removed './156/32' 2023-10-06T04:02:54.108 INFO:tasks.workunit.client.0.smithi157.stdout:removed './156/30' 2023-10-06T04:02:54.108 INFO:tasks.workunit.client.0.smithi157.stdout:removed './156/14' 2023-10-06T04:02:54.108 INFO:tasks.workunit.client.0.smithi157.stdout:removed './156/18' 2023-10-06T04:02:54.109 INFO:tasks.workunit.client.0.smithi157.stdout:removed './156/13' 2023-10-06T04:02:54.109 INFO:tasks.workunit.client.0.smithi157.stdout:removed './156/33' 2023-10-06T04:02:54.109 INFO:tasks.workunit.client.0.smithi157.stdout:removed './156/17' 2023-10-06T04:02:54.109 INFO:tasks.workunit.client.0.smithi157.stdout:removed './156/35' 2023-10-06T04:02:54.109 INFO:tasks.workunit.client.0.smithi157.stdout:removed './156/37' 2023-10-06T04:02:54.109 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './156' 2023-10-06T04:02:54.109 INFO:tasks.workunit.client.0.smithi157.stdout:removed './157/19' 2023-10-06T04:02:54.110 INFO:tasks.workunit.client.0.smithi157.stdout:removed './157/15' 2023-10-06T04:02:54.110 INFO:tasks.workunit.client.0.smithi157.stdout:removed './157/31' 2023-10-06T04:02:54.110 INFO:tasks.workunit.client.0.smithi157.stdout:removed './157/36' 2023-10-06T04:02:54.110 INFO:tasks.workunit.client.0.smithi157.stdout:removed './157/20' 2023-10-06T04:02:54.110 INFO:tasks.workunit.client.0.smithi157.stdout:removed './157/12' 2023-10-06T04:02:54.110 INFO:tasks.workunit.client.0.smithi157.stdout:removed './157/39' 2023-10-06T04:02:54.110 INFO:tasks.workunit.client.0.smithi157.stdout:removed './157/10' 2023-10-06T04:02:54.111 INFO:tasks.workunit.client.0.smithi157.stdout:removed './157/34' 2023-10-06T04:02:54.111 INFO:tasks.workunit.client.0.smithi157.stdout:removed './157/38' 2023-10-06T04:02:54.111 INFO:tasks.workunit.client.0.smithi157.stdout:removed './157/11' 2023-10-06T04:02:54.111 INFO:tasks.workunit.client.0.smithi157.stdout:removed './157/16' 2023-10-06T04:02:54.111 INFO:tasks.workunit.client.0.smithi157.stdout:removed './157/32' 2023-10-06T04:02:54.111 INFO:tasks.workunit.client.0.smithi157.stdout:removed './157/30' 2023-10-06T04:02:54.111 INFO:tasks.workunit.client.0.smithi157.stdout:removed './157/14' 2023-10-06T04:02:54.111 INFO:tasks.workunit.client.0.smithi157.stdout:removed './157/18' 2023-10-06T04:02:54.111 INFO:tasks.workunit.client.0.smithi157.stdout:removed './157/13' 2023-10-06T04:02:54.111 INFO:tasks.workunit.client.0.smithi157.stdout:removed './157/33' 2023-10-06T04:02:54.984 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:02:55.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.103+0000 7f217b8bf700 1 Processor -- start 2023-10-06T04:02:55.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.103+0000 7f217b8bf700 1 -- start start 2023-10-06T04:02:55.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.104+0000 7f217b8bf700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f217406cbb0 0x7f217406d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:55.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.104+0000 7f217b8bf700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f217406c1f0 0x7f217406da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:55.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.104+0000 7f217b8bf700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21740c9780 con 0x7f217406cf80 2023-10-06T04:02:55.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.104+0000 7f217b8bf700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21740c8c40 con 0x7f217406c1f0 2023-10-06T04:02:55.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.104+0000 7f217b8bf700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217406e0c0 con 0x7f217406cbb0 2023-10-06T04:02:55.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.105+0000 7f217965b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f217406cbb0 0x7f217406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:55.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.105+0000 7f2178e5a700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f217406c1f0 0x7f217406da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:55.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.105+0000 7f2178e5a700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f217406c1f0 0x7f217406da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:50422/0 (socket says 172.21.15.157:50422) 2023-10-06T04:02:55.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.105+0000 7f2179e5c700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f217406cf80 0x7f217414fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.157:41022/0 (socket says 172.21.15.157:41022) 2023-10-06T04:02:55.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.105+0000 7f2178e5a700 1 -- 172.21.15.157:0/4136963723 learned_addr learned my addr 172.21.15.157:0/4136963723 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:55.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.105+0000 7f217965b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f217406cbb0 0x7f217406d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:33380/0 (socket says 172.21.15.157:33380) 2023-10-06T04:02:55.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.105+0000 7f2178e5a700 1 -- 172.21.15.157:0/4136963723 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f217406cbb0 msgr2=0x7f217406d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:55.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.105+0000 7f2178e5a700 1 --2- 172.21.15.157:0/4136963723 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f217406cbb0 0x7f217406d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:55.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.105+0000 7f2178e5a700 1 -- 172.21.15.157:0/4136963723 >> v1:172.21.15.175:6789/0 conn(0x7f217406cf80 legacy=0x7f217414fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:55.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.105+0000 7f216bfff700 1 -- 172.21.15.157:0/4136963723 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 848927523 0 0) 0x7f21740c9780 con 0x7f217406cf80 2023-10-06T04:02:55.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.105+0000 7f2178e5a700 1 -- 172.21.15.157:0/4136963723 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21740c8e50 con 0x7f217406c1f0 2023-10-06T04:02:55.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.105+0000 7f217965b700 1 --2- 172.21.15.157:0/4136963723 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f217406cbb0 0x7f217406d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:02:55.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.105+0000 7f2178e5a700 1 --2- 172.21.15.157:0/4136963723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f217406c1f0 0x7f217406da70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f21640099e0 tx=0x7f2164009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0fd1d7d0f442f75 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:55.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.105+0000 7f216bfff700 1 -- 172.21.15.157:0/4136963723 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216400b8b0 con 0x7f217406c1f0 2023-10-06T04:02:55.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.105+0000 7f216bfff700 1 -- 172.21.15.157:0/4136963723 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216400ba10 con 0x7f217406c1f0 2023-10-06T04:02:55.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.105+0000 7f216bfff700 1 -- 172.21.15.157:0/4136963723 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216400bdb0 con 0x7f217406c1f0 2023-10-06T04:02:55.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.106+0000 7f217b8bf700 1 -- 172.21.15.157:0/4136963723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f217406c1f0 msgr2=0x7f217406da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:55.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.106+0000 7f217b8bf700 1 --2- 172.21.15.157:0/4136963723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f217406c1f0 0x7f217406da70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f21640099e0 tx=0x7f2164009250 comp rx=0 tx=0).stop 2023-10-06T04:02:55.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.106+0000 7f217b8bf700 1 -- 172.21.15.157:0/4136963723 shutdown_connections 2023-10-06T04:02:55.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.106+0000 7f217b8bf700 1 --2- 172.21.15.157:0/4136963723 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f217406cbb0 0x7f217406d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:55.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.106+0000 7f217b8bf700 1 --2- 172.21.15.157:0/4136963723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f217406c1f0 0x7f217406da70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:55.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.106+0000 7f217b8bf700 1 -- 172.21.15.157:0/4136963723 >> 172.21.15.157:0/4136963723 conn(0x7f217406f230 msgr2=0x7f217406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:55.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.106+0000 7f217b8bf700 1 -- 172.21.15.157:0/4136963723 shutdown_connections 2023-10-06T04:02:55.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.106+0000 7f217b8bf700 1 -- 172.21.15.157:0/4136963723 wait complete. 2023-10-06T04:02:55.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.107+0000 7f217b8bf700 1 Processor -- start 2023-10-06T04:02:55.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.107+0000 7f217b8bf700 1 -- start start 2023-10-06T04:02:55.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.108+0000 7f217b8bf700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f217406c1f0 0x7f2174153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:55.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.108+0000 7f217b8bf700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f217406cbb0 0x7f2174155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:55.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.108+0000 7f217b8bf700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f217406cf80 0x7f2174148db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:55.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.108+0000 7f217b8bf700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21740cbe50 con 0x7f217406cf80 2023-10-06T04:02:55.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.108+0000 7f217b8bf700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2174150b80 con 0x7f217406c1f0 2023-10-06T04:02:55.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.108+0000 7f217b8bf700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217416a100 con 0x7f217406cbb0 2023-10-06T04:02:55.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.108+0000 7f2179e5c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f217406cf80 0x7f2174148db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:55.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.108+0000 7f2179e5c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f217406cf80 0x7f2174148db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47148/0 (socket says 172.21.15.157:47148) 2023-10-06T04:02:55.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.108+0000 7f2179e5c700 1 -- 172.21.15.157:0/1202082382 learned_addr learned my addr 172.21.15.157:0/1202082382 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:55.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.108+0000 7f217965b700 1 --2- 172.21.15.157:0/1202082382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f217406c1f0 0x7f2174153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:55.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.108+0000 7f2178e5a700 1 --2- 172.21.15.157:0/1202082382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f217406cbb0 0x7f2174155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:55.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.108+0000 7f2179e5c700 1 -- 172.21.15.157:0/1202082382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f217406cbb0 msgr2=0x7f2174155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:55.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.108+0000 7f2179e5c700 1 --2- 172.21.15.157:0/1202082382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f217406cbb0 0x7f2174155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:55.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.108+0000 7f2179e5c700 1 -- 172.21.15.157:0/1202082382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f217406c1f0 msgr2=0x7f2174153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:55.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.108+0000 7f2179e5c700 1 --2- 172.21.15.157:0/1202082382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f217406c1f0 0x7f2174153bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:55.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.108+0000 7f2179e5c700 1 -- 172.21.15.157:0/1202082382 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2170009420 con 0x7f217406cf80 2023-10-06T04:02:55.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.109+0000 7f2179e5c700 1 --2- 172.21.15.157:0/1202082382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f217406cf80 0x7f2174148db0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f216c00ccc0 tx=0x7f216c001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:55.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.109+0000 7f21697fa700 1 -- 172.21.15.157:0/1202082382 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c005df0 con 0x7f217406cf80 2023-10-06T04:02:55.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.109+0000 7f21697fa700 1 -- 172.21.15.157:0/1202082382 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216c00a700 con 0x7f217406cf80 2023-10-06T04:02:55.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.109+0000 7f217b8bf700 1 -- 172.21.15.157:0/1202082382 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21640094f0 con 0x7f217406cf80 2023-10-06T04:02:55.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.109+0000 7f21697fa700 1 -- 172.21.15.157:0/1202082382 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c00aac0 con 0x7f217406cf80 2023-10-06T04:02:55.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.109+0000 7f217b8bf700 1 -- 172.21.15.157:0/1202082382 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21740cb800 con 0x7f217406cf80 2023-10-06T04:02:55.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.110+0000 7f217b8bf700 1 -- 172.21.15.157:0/1202082382 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21740cbfc0 con 0x7f217406cf80 2023-10-06T04:02:55.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.110+0000 7f21697fa700 1 -- 172.21.15.157:0/1202082382 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f216c0070f0 con 0x7f217406cf80 2023-10-06T04:02:55.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.111+0000 7f21697fa700 1 --2- 172.21.15.157:0/1202082382 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2158066f30 0x7f21580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:55.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.111+0000 7f21697fa700 1 -- 172.21.15.157:0/1202082382 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7f216c084900 con 0x7f217406cf80 2023-10-06T04:02:55.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.111+0000 7f217965b700 1 --2- 172.21.15.157:0/1202082382 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2158066f30 0x7f21580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:55.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.111+0000 7f217965b700 1 --2- 172.21.15.157:0/1202082382 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2158066f30 0x7f21580693e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2170009f40 tx=0x7f21700091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:55.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.115+0000 7f21697fa700 1 -- 172.21.15.157:0/1202082382 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f216c051840 con 0x7f217406cf80 2023-10-06T04:02:55.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.342+0000 7f217b8bf700 1 -- 172.21.15.157:0/1202082382 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21740c9130 con 0x7f217406cf80 2023-10-06T04:02:55.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.343+0000 7f21697fa700 1 -- 172.21.15.157:0/1202082382 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+6178 (secure 0 0 0) 0x7f216c054e60 con 0x7f217406cf80 2023-10-06T04:02:55.346 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:02:55.346 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:02:46.949420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2,3],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270},"failed":[],"damaged":[],"stopped":[4],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:stopping","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:02:55.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.345+0000 7f217b8bf700 1 -- 172.21.15.157:0/1202082382 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2158066f30 msgr2=0x7f21580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:55.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.345+0000 7f217b8bf700 1 --2- 172.21.15.157:0/1202082382 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2158066f30 0x7f21580693e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2170009f40 tx=0x7f21700091a0 comp rx=0 tx=0).stop 2023-10-06T04:02:55.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.345+0000 7f217b8bf700 1 -- 172.21.15.157:0/1202082382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f217406cf80 msgr2=0x7f2174148db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:55.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.345+0000 7f217b8bf700 1 --2- 172.21.15.157:0/1202082382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f217406cf80 0x7f2174148db0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f216c00ccc0 tx=0x7f216c001a20 comp rx=0 tx=0).stop 2023-10-06T04:02:55.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.345+0000 7f217b8bf700 1 -- 172.21.15.157:0/1202082382 shutdown_connections 2023-10-06T04:02:55.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.345+0000 7f217b8bf700 1 --2- 172.21.15.157:0/1202082382 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2158066f30 0x7f21580693e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:55.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.345+0000 7f217b8bf700 1 --2- 172.21.15.157:0/1202082382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f217406c1f0 0x7f2174153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:55.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.345+0000 7f217b8bf700 1 --2- 172.21.15.157:0/1202082382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f217406cf80 0x7f2174148db0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:55.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.345+0000 7f217b8bf700 1 --2- 172.21.15.157:0/1202082382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f217406cbb0 0x7f2174155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:55.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.345+0000 7f217b8bf700 1 -- 172.21.15.157:0/1202082382 >> 172.21.15.157:0/1202082382 conn(0x7f217406f230 msgr2=0x7f2174150f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:55.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.345+0000 7f217b8bf700 1 -- 172.21.15.157:0/1202082382 shutdown_connections 2023-10-06T04:02:55.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:55.345+0000 7f217b8bf700 1 -- 172.21.15.157:0/1202082382 wait complete. 2023-10-06T04:02:55.349 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 45 2023-10-06T04:02:55.360 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:02:57.361 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:02:57.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.487+0000 7f8db4b64700 1 Processor -- start 2023-10-06T04:02:57.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.487+0000 7f8db4b64700 1 -- start start 2023-10-06T04:02:57.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.488+0000 7f8db4b64700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8db014dcd0 0x7f8db0153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:57.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.488+0000 7f8db4b64700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8db00c5770 con 0x7f8db014e0a0 2023-10-06T04:02:57.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.488+0000 7f8db4b64700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8db01543e0 con 0x7f8db00c5c00 2023-10-06T04:02:57.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.488+0000 7f8db4b64700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db0154c80 con 0x7f8db014dcd0 2023-10-06T04:02:57.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.488+0000 7f8dae59c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8db014dcd0 0x7f8db0153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:57.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.488+0000 7f8da5d9b700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f8db014e0a0 0x7f8db00c18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51370/0 (socket says 172.21.15.157:51370) 2023-10-06T04:02:57.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.488+0000 7f8da5d9b700 1 -- 172.21.15.157:0/2344412278 learned_addr learned my addr 172.21.15.157:0/2344412278 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:02:57.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.489+0000 7f8dae59c700 1 -- 172.21.15.157:0/2344412278 >> v1:172.21.15.175:6789/0 conn(0x7f8db00c5c00 legacy=0x7f8db00ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:57.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.489+0000 7f8dae59c700 1 -- 172.21.15.157:0/2344412278 >> v1:172.21.15.157:6789/0 conn(0x7f8db014e0a0 legacy=0x7f8db00c18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:57.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.489+0000 7f8dae59c700 1 -- 172.21.15.157:0/2344412278 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db01546c0 con 0x7f8db014dcd0 2023-10-06T04:02:57.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.489+0000 7f8dae59c700 1 --2- 172.21.15.157:0/2344412278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8db014dcd0 0x7f8db0153cc0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8da00098e0 tx=0x7f8da00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5928e843b4af51b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:57.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.489+0000 7f8dadd9b700 1 -- 172.21.15.157:0/2344412278 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da0009e80 con 0x7f8db014dcd0 2023-10-06T04:02:57.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.489+0000 7f8dadd9b700 1 -- 172.21.15.157:0/2344412278 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da000b8f0 con 0x7f8db014dcd0 2023-10-06T04:02:57.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.489+0000 7f8dadd9b700 1 -- 172.21.15.157:0/2344412278 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da000bc70 con 0x7f8db014dcd0 2023-10-06T04:02:57.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.489+0000 7f8db4b64700 1 -- 172.21.15.157:0/2344412278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8db014dcd0 msgr2=0x7f8db0153cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:57.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.489+0000 7f8db4b64700 1 --2- 172.21.15.157:0/2344412278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8db014dcd0 0x7f8db0153cc0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8da00098e0 tx=0x7f8da00091a0 comp rx=0 tx=0).stop 2023-10-06T04:02:57.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.489+0000 7f8db4b64700 1 -- 172.21.15.157:0/2344412278 shutdown_connections 2023-10-06T04:02:57.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.489+0000 7f8db4b64700 1 --2- 172.21.15.157:0/2344412278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8db014dcd0 0x7f8db0153cc0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:57.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.489+0000 7f8db4b64700 1 -- 172.21.15.157:0/2344412278 >> 172.21.15.157:0/2344412278 conn(0x7f8db00692b0 msgr2=0x7f8db00696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:57.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.489+0000 7f8db4b64700 1 -- 172.21.15.157:0/2344412278 shutdown_connections 2023-10-06T04:02:57.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.489+0000 7f8db4b64700 1 -- 172.21.15.157:0/2344412278 wait complete. 2023-10-06T04:02:57.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.491+0000 7f8db4b64700 1 Processor -- start 2023-10-06T04:02:57.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.491+0000 7f8db4b64700 1 -- start start 2023-10-06T04:02:57.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.491+0000 7f8db4b64700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8db00c5c00 0x7f8db00cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:57.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.491+0000 7f8db4b64700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8db014dcd0 0x7f8db01676f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:57.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.491+0000 7f8db4b64700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8db014e0a0 0x7f8db0146500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:57.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.491+0000 7f8db4b64700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db0154200 con 0x7f8db014e0a0 2023-10-06T04:02:57.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.491+0000 7f8db4b64700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db01682c0 con 0x7f8db014dcd0 2023-10-06T04:02:57.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.492+0000 7f8db4b64700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db00ca160 con 0x7f8db00c5c00 2023-10-06T04:02:57.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.492+0000 7f8daed9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8db014e0a0 0x7f8db0146500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:57.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.492+0000 7f8daed9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8db014e0a0 0x7f8db0146500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47172/0 (socket says 172.21.15.157:47172) 2023-10-06T04:02:57.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.492+0000 7f8daed9d700 1 -- 172.21.15.157:0/734658400 learned_addr learned my addr 172.21.15.157:0/734658400 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:57.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.492+0000 7f8dae59c700 1 --2- 172.21.15.157:0/734658400 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8db00c5c00 0x7f8db00cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:57.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.492+0000 7f8da5d9b700 1 --2- 172.21.15.157:0/734658400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8db014dcd0 0x7f8db01676f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:57.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.492+0000 7f8daed9d700 1 -- 172.21.15.157:0/734658400 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8db00c5c00 msgr2=0x7f8db00cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:57.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.492+0000 7f8daed9d700 1 --2- 172.21.15.157:0/734658400 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8db00c5c00 0x7f8db00cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:57.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.492+0000 7f8daed9d700 1 -- 172.21.15.157:0/734658400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8db014dcd0 msgr2=0x7f8db01676f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:57.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.492+0000 7f8daed9d700 1 --2- 172.21.15.157:0/734658400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8db014dcd0 0x7f8db01676f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:57.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.492+0000 7f8daed9d700 1 -- 172.21.15.157:0/734658400 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da0009420 con 0x7f8db014e0a0 2023-10-06T04:02:57.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.492+0000 7f8daed9d700 1 --2- 172.21.15.157:0/734658400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8db014e0a0 0x7f8db0146500 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8da800c8e0 tx=0x7f8da8001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:57.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.492+0000 7f8da6ffd700 1 -- 172.21.15.157:0/734658400 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da8012630 con 0x7f8db014e0a0 2023-10-06T04:02:57.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.492+0000 7f8da6ffd700 1 -- 172.21.15.157:0/734658400 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da8012790 con 0x7f8db014e0a0 2023-10-06T04:02:57.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.492+0000 7f8db4b64700 1 -- 172.21.15.157:0/734658400 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db00ca6c0 con 0x7f8db014e0a0 2023-10-06T04:02:57.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.492+0000 7f8da6ffd700 1 -- 172.21.15.157:0/734658400 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da8012b50 con 0x7f8db014e0a0 2023-10-06T04:02:57.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.492+0000 7f8db4b64700 1 -- 172.21.15.157:0/734658400 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db0167fe0 con 0x7f8db014e0a0 2023-10-06T04:02:57.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.493+0000 7f8db4b64700 1 -- 172.21.15.157:0/734658400 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db00ca850 con 0x7f8db014e0a0 2023-10-06T04:02:57.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.494+0000 7f8da6ffd700 1 -- 172.21.15.157:0/734658400 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8da801a3a0 con 0x7f8db014e0a0 2023-10-06T04:02:57.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.494+0000 7f8da6ffd700 1 --2- 172.21.15.157:0/734658400 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8d90066f30 0x7f8d900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:57.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.494+0000 7f8da6ffd700 1 -- 172.21.15.157:0/734658400 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7f8da8084af0 con 0x7f8db014e0a0 2023-10-06T04:02:57.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.494+0000 7f8dae59c700 1 --2- 172.21.15.157:0/734658400 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8d90066f30 0x7f8d900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:57.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.495+0000 7f8dae59c700 1 --2- 172.21.15.157:0/734658400 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8d90066f30 0x7f8d900693e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8da00093f0 tx=0x7f8da0017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:57.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.499+0000 7f8da6ffd700 1 -- 172.21.15.157:0/734658400 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8da8055440 con 0x7f8db014e0a0 2023-10-06T04:02:57.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.725+0000 7f8db4b64700 1 -- 172.21.15.157:0/734658400 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8db004bb40 con 0x7f8db014e0a0 2023-10-06T04:02:57.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.726+0000 7f8da6ffd700 1 -- 172.21.15.157:0/734658400 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+6178 (secure 0 0 0) 0x7f8da8054fd0 con 0x7f8db014e0a0 2023-10-06T04:02:57.729 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:02:57.729 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:02:46.949420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2,3],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279,"mds_3":4270},"failed":[],"damaged":[],"stopped":[4],"info":{"gid_4270":{"gid":4270,"name":"b","rank":3,"incarnation":13,"state":"up:stopping","state_seq":8,"addr":"172.21.15.175:6835/1596042861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1596042861},{"type":"v1","addr":"172.21.15.175:6835","nonce":1596042861}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:02:57.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.728+0000 7f8db4b64700 1 -- 172.21.15.157:0/734658400 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8d90066f30 msgr2=0x7f8d900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:57.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.728+0000 7f8db4b64700 1 --2- 172.21.15.157:0/734658400 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8d90066f30 0x7f8d900693e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8da00093f0 tx=0x7f8da0017000 comp rx=0 tx=0).stop 2023-10-06T04:02:57.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.728+0000 7f8db4b64700 1 -- 172.21.15.157:0/734658400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8db014e0a0 msgr2=0x7f8db0146500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:57.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.728+0000 7f8db4b64700 1 --2- 172.21.15.157:0/734658400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8db014e0a0 0x7f8db0146500 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8da800c8e0 tx=0x7f8da8001710 comp rx=0 tx=0).stop 2023-10-06T04:02:57.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.728+0000 7f8db4b64700 1 -- 172.21.15.157:0/734658400 shutdown_connections 2023-10-06T04:02:57.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.728+0000 7f8db4b64700 1 --2- 172.21.15.157:0/734658400 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8d90066f30 0x7f8d900693e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:57.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.728+0000 7f8db4b64700 1 --2- 172.21.15.157:0/734658400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8db014e0a0 0x7f8db0146500 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:57.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.728+0000 7f8db4b64700 1 --2- 172.21.15.157:0/734658400 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8db00c5c00 0x7f8db00cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:57.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.728+0000 7f8db4b64700 1 --2- 172.21.15.157:0/734658400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8db014dcd0 0x7f8db01676f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:57.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.728+0000 7f8db4b64700 1 -- 172.21.15.157:0/734658400 >> 172.21.15.157:0/734658400 conn(0x7f8db00692b0 msgr2=0x7f8db00c6060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:57.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.728+0000 7f8db4b64700 1 -- 172.21.15.157:0/734658400 shutdown_connections 2023-10-06T04:02:57.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:57.728+0000 7f8db4b64700 1 -- 172.21.15.157:0/734658400 wait complete. 2023-10-06T04:02:57.732 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 45 2023-10-06T04:02:57.742 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:02:58.023 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2023-10-06T04:02:59.743 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:02:59.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.863+0000 7f255809b700 1 Processor -- start 2023-10-06T04:02:59.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.863+0000 7f255809b700 1 -- start start 2023-10-06T04:02:59.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.864+0000 7f255809b700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f25500c9dc0 0x7f25500c3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:59.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.864+0000 7f255809b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2550159b30 0x7f2550159f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:59.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.864+0000 7f255809b700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25500c4290 con 0x7f255013f0e0 2023-10-06T04:02:59.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.864+0000 7f255809b700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f255015a4b0 con 0x7f25500c9dc0 2023-10-06T04:02:59.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.864+0000 7f255809b700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25500c6980 con 0x7f2550159b30 2023-10-06T04:02:59.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.864+0000 7f2555e37700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f255013f0e0 0x7f255006a5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51380/0 (socket says 172.21.15.157:51380) 2023-10-06T04:02:59.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.864+0000 7f2555e37700 1 -- 172.21.15.157:0/473768143 learned_addr learned my addr 172.21.15.157:0/473768143 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:02:59.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.864+0000 7f2555636700 1 --2- 172.21.15.157:0/473768143 >> v2:172.21.15.175:3300/0 conn(0x7f25500c9dc0 0x7f25500c3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:59.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.865+0000 7f2556638700 1 --2- 172.21.15.157:0/473768143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2550159b30 0x7f2550159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:59.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.865+0000 7f2554e35700 1 -- 172.21.15.157:0/473768143 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 35135988 0 0) 0x7f25500c4290 con 0x7f255013f0e0 2023-10-06T04:02:59.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.865+0000 7f2554e35700 1 -- 172.21.15.157:0/473768143 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f253c003570 con 0x7f255013f0e0 2023-10-06T04:02:59.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.865+0000 7f2554e35700 1 -- 172.21.15.157:0/473768143 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1514278507 0 0) 0x7f253c003570 con 0x7f255013f0e0 2023-10-06T04:02:59.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.865+0000 7f2554e35700 1 -- 172.21.15.157:0/473768143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2550159b30 msgr2=0x7f2550159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:59.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.865+0000 7f2554e35700 1 --2- 172.21.15.157:0/473768143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2550159b30 0x7f2550159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:59.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.865+0000 7f2554e35700 1 -- 172.21.15.157:0/473768143 >> v2:172.21.15.175:3300/0 conn(0x7f25500c9dc0 msgr2=0x7f25500c3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:59.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.865+0000 7f2554e35700 1 --2- 172.21.15.157:0/473768143 >> v2:172.21.15.175:3300/0 conn(0x7f25500c9dc0 0x7f25500c3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:59.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.865+0000 7f2554e35700 1 -- 172.21.15.157:0/473768143 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25501596a0 con 0x7f255013f0e0 2023-10-06T04:02:59.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.865+0000 7f2554e35700 1 -- 172.21.15.157:0/473768143 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f254c0017f0 con 0x7f255013f0e0 2023-10-06T04:02:59.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.865+0000 7f2554e35700 1 -- 172.21.15.157:0/473768143 >> v1:172.21.15.157:6789/0 conn(0x7f255013f0e0 legacy=0x7f255006a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:59.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.865+0000 7f2554e35700 1 --2- 172.21.15.157:0/473768143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f253c003a60 0x7f253c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:59.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.865+0000 7f2554e35700 1 --2- 172.21.15.157:0/473768143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f253c004810 0x7f253c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:59.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.865+0000 7f2554e35700 1 --2- 172.21.15.157:0/473768143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f253c0051c0 0x7f253c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:59.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.865+0000 7f2554e35700 1 -- 172.21.15.157:0/473768143 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255015a4b0 con 0x7f253c0051c0 2023-10-06T04:02:59.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.865+0000 7f2554e35700 1 -- 172.21.15.157:0/473768143 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25500c6980 con 0x7f253c003a60 2023-10-06T04:02:59.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.865+0000 7f2554e35700 1 -- 172.21.15.157:0/473768143 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253c00bfa0 con 0x7f253c004810 2023-10-06T04:02:59.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.865+0000 7f2555636700 1 --2- 172.21.15.157:0/473768143 >> v2:172.21.15.175:3300/0 conn(0x7f25500c9dc0 0x7f25500c3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:02:59.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.865+0000 7f2556638700 1 --2- 172.21.15.157:0/473768143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2550159b30 0x7f2550159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:02:59.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.865+0000 7f2556638700 1 --2- 172.21.15.157:0/473768143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f253c0051c0 0x7f253c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:59.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.865+0000 7f2555e37700 1 --2- 172.21.15.157:0/473768143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f253c003a60 0x7f253c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:59.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.865+0000 7f2555636700 1 --2- 172.21.15.157:0/473768143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f253c004810 0x7f253c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:59.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.865+0000 7f2556638700 1 -- 172.21.15.157:0/473768143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f253c004810 msgr2=0x7f253c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:59.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.865+0000 7f2556638700 1 --2- 172.21.15.157:0/473768143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f253c004810 0x7f253c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:59.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.865+0000 7f2556638700 1 -- 172.21.15.157:0/473768143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f253c003a60 msgr2=0x7f253c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:02:59.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.865+0000 7f2556638700 1 --2- 172.21.15.157:0/473768143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f253c003a60 0x7f253c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:59.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.865+0000 7f2556638700 1 -- 172.21.15.157:0/473768143 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c003570 con 0x7f253c0051c0 2023-10-06T04:02:59.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.866+0000 7f2556638700 1 --2- 172.21.15.157:0/473768143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f253c0051c0 0x7f253c00ba60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2550159f00 tx=0x7f25480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec19e5ed802b540 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:59.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.866+0000 7f2554e35700 1 -- 172.21.15.157:0/473768143 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254800e710 con 0x7f253c0051c0 2023-10-06T04:02:59.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.866+0000 7f2554e35700 1 -- 172.21.15.157:0/473768143 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254800e870 con 0x7f253c0051c0 2023-10-06T04:02:59.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.866+0000 7f2554e35700 1 -- 172.21.15.157:0/473768143 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254800ec10 con 0x7f253c0051c0 2023-10-06T04:02:59.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.866+0000 7f255809b700 1 -- 172.21.15.157:0/473768143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f253c0051c0 msgr2=0x7f253c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:59.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.866+0000 7f255809b700 1 --2- 172.21.15.157:0/473768143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f253c0051c0 0x7f253c00ba60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2550159f00 tx=0x7f25480091a0 comp rx=0 tx=0).stop 2023-10-06T04:02:59.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.866+0000 7f2556638700 1 -- 172.21.15.157:0/473768143 reap_dead start 2023-10-06T04:02:59.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.866+0000 7f255809b700 1 -- 172.21.15.157:0/473768143 shutdown_connections 2023-10-06T04:02:59.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.866+0000 7f255809b700 1 -- 172.21.15.157:0/473768143 >> 172.21.15.157:0/473768143 conn(0x7f255006b6a0 msgr2=0x7f25500631f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:02:59.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.866+0000 7f255809b700 1 -- 172.21.15.157:0/473768143 shutdown_connections 2023-10-06T04:02:59.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.866+0000 7f255809b700 1 -- 172.21.15.157:0/473768143 wait complete. 2023-10-06T04:02:59.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.867+0000 7f255809b700 1 Processor -- start 2023-10-06T04:02:59.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.867+0000 7f255809b700 1 -- start start 2023-10-06T04:02:59.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.868+0000 7f255809b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2550158a20 0x7f2550158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:59.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.868+0000 7f255809b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2550152cf0 0x7f2550167570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:59.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.868+0000 7f255809b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f255013bb30 0x7f255013bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:59.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.868+0000 7f255809b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25501524c0 con 0x7f255013bb30 2023-10-06T04:02:59.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.868+0000 7f255809b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255013d160 con 0x7f2550152cf0 2023-10-06T04:02:59.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.868+0000 7f255809b700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2550151930 con 0x7f2550158a20 2023-10-06T04:02:59.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.868+0000 7f2556638700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f255013bb30 0x7f255013bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:59.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.868+0000 7f2555e37700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2550158a20 0x7f2550158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:59.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.868+0000 7f2555e37700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2550158a20 0x7f2550158df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:33428/0 (socket says 172.21.15.157:33428) 2023-10-06T04:02:59.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.868+0000 7f2555636700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2550152cf0 0x7f2550167570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:59.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.868+0000 7f2556638700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f255013bb30 0x7f255013bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47198/0 (socket says 172.21.15.157:47198) 2023-10-06T04:02:59.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.868+0000 7f2555e37700 1 -- 172.21.15.157:0/842524759 learned_addr learned my addr 172.21.15.157:0/842524759 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:02:59.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.869+0000 7f2556638700 1 -- 172.21.15.157:0/842524759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2550158a20 msgr2=0x7f2550158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:59.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.869+0000 7f2556638700 1 --2- 172.21.15.157:0/842524759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2550158a20 0x7f2550158df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:59.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.869+0000 7f2556638700 1 -- 172.21.15.157:0/842524759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2550152cf0 msgr2=0x7f2550167570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:02:59.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.869+0000 7f2556638700 1 --2- 172.21.15.157:0/842524759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2550152cf0 0x7f2550167570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:02:59.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.869+0000 7f2556638700 1 -- 172.21.15.157:0/842524759 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2550151bd0 con 0x7f255013bb30 2023-10-06T04:02:59.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.869+0000 7f2556638700 1 --2- 172.21.15.157:0/842524759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f255013bb30 0x7f255013bf80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2548018000 tx=0x7f2548019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:59.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.869+0000 7f2545ffb700 1 -- 172.21.15.157:0/842524759 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2548022070 con 0x7f255013bb30 2023-10-06T04:02:59.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.869+0000 7f2545ffb700 1 -- 172.21.15.157:0/842524759 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254800e9e0 con 0x7f255013bb30 2023-10-06T04:02:59.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.869+0000 7f2545ffb700 1 -- 172.21.15.157:0/842524759 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254800e710 con 0x7f255013bb30 2023-10-06T04:02:59.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.869+0000 7f255809b700 1 -- 172.21.15.157:0/842524759 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2550151e70 con 0x7f255013bb30 2023-10-06T04:02:59.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.869+0000 7f255809b700 1 -- 172.21.15.157:0/842524759 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f255013da20 con 0x7f255013bb30 2023-10-06T04:02:59.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.870+0000 7f255809b700 1 -- 172.21.15.157:0/842524759 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2550166450 con 0x7f255013bb30 2023-10-06T04:02:59.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.870+0000 7f2545ffb700 1 -- 172.21.15.157:0/842524759 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2548027670 con 0x7f255013bb30 2023-10-06T04:02:59.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.871+0000 7f2545ffb700 1 --2- 172.21.15.157:0/842524759 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2534066f30 0x7f25340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:02:59.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.871+0000 7f2545ffb700 1 -- 172.21.15.157:0/842524759 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7f2548085910 con 0x7f255013bb30 2023-10-06T04:02:59.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.871+0000 7f2555e37700 1 --2- 172.21.15.157:0/842524759 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2534066f30 0x7f25340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:02:59.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.871+0000 7f2555e37700 1 --2- 172.21.15.157:0/842524759 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2534066f30 0x7f25340693e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f254c001460 tx=0x7f254c006040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:02:59.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:02:59.875+0000 7f2545ffb700 1 -- 172.21.15.157:0/842524759 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2550166450 con 0x7f255013bb30 2023-10-06T04:03:00.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:00.103+0000 7f255809b700 1 -- 172.21.15.157:0/842524759 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2550166450 con 0x7f255013bb30 2023-10-06T04:03:00.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:00.104+0000 7f2545ffb700 1 -- 172.21.15.157:0/842524759 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+6161 (secure 0 0 0) 0x7f2550166450 con 0x7f255013bb30 2023-10-06T04:03:00.105 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:03:00.106 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3192541484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3192541484},{"type":"v1","addr":"172.21.15.175:6835","nonce":3192541484}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:02:57.983451+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279},"failed":[],"damaged":[],"stopped":[3,4],"info":{"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:stopping","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[2],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:03:00.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:00.106+0000 7f255809b700 1 -- 172.21.15.157:0/842524759 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2534066f30 msgr2=0x7f25340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:00.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:00.106+0000 7f255809b700 1 --2- 172.21.15.157:0/842524759 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2534066f30 0x7f25340693e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f254c001460 tx=0x7f254c006040 comp rx=0 tx=0).stop 2023-10-06T04:03:00.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:00.106+0000 7f255809b700 1 -- 172.21.15.157:0/842524759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f255013bb30 msgr2=0x7f255013bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:00.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:00.106+0000 7f255809b700 1 --2- 172.21.15.157:0/842524759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f255013bb30 0x7f255013bf80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2548018000 tx=0x7f2548019000 comp rx=0 tx=0).stop 2023-10-06T04:03:00.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:00.106+0000 7f255809b700 1 -- 172.21.15.157:0/842524759 shutdown_connections 2023-10-06T04:03:00.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:00.106+0000 7f255809b700 1 --2- 172.21.15.157:0/842524759 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2534066f30 0x7f25340693e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:00.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:00.106+0000 7f255809b700 1 --2- 172.21.15.157:0/842524759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f255013bb30 0x7f255013bf80 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:00.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:00.106+0000 7f255809b700 1 --2- 172.21.15.157:0/842524759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2550158a20 0x7f2550158df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:00.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:00.106+0000 7f255809b700 1 --2- 172.21.15.157:0/842524759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2550152cf0 0x7f2550167570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:00.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:00.106+0000 7f255809b700 1 -- 172.21.15.157:0/842524759 >> 172.21.15.157:0/842524759 conn(0x7f255006b6a0 msgr2=0x7f255013c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:00.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:00.106+0000 7f255809b700 1 -- 172.21.15.157:0/842524759 shutdown_connections 2023-10-06T04:03:00.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:00.106+0000 7f255809b700 1 -- 172.21.15.157:0/842524759 wait complete. 2023-10-06T04:03:00.109 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 48 2023-10-06T04:03:00.120 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:03:02.121 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:03:02.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.246+0000 7f17fcfac700 1 Processor -- start 2023-10-06T04:03:02.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.247+0000 7f17fcfac700 1 -- start start 2023-10-06T04:03:02.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.247+0000 7f17fcfac700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f17f80c23c0 0x7f17f8155f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:02.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.247+0000 7f17fcfac700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17f806d5d0 con 0x7f17f80c2790 2023-10-06T04:03:02.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.247+0000 7f17fcfac700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17f806c6e0 con 0x7f17f806c1f0 2023-10-06T04:03:02.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.247+0000 7f17fcfac700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17f8157b80 con 0x7f17f80c23c0 2023-10-06T04:03:02.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.247+0000 7f17f659c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f17f80c23c0 0x7f17f8155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:02.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.247+0000 7f17f6d9d700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f17f80c2790 0x7f17f8069c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51394/0 (socket says 172.21.15.157:51394) 2023-10-06T04:03:02.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.247+0000 7f17f6d9d700 1 -- 172.21.15.157:0/1176811577 learned_addr learned my addr 172.21.15.157:0/1176811577 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:03:02.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.248+0000 7f17f659c700 1 -- 172.21.15.157:0/1176811577 >> v1:172.21.15.175:6789/0 conn(0x7f17f806c1f0 legacy=0x7f17f8157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:02.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.248+0000 7f17f659c700 1 -- 172.21.15.157:0/1176811577 >> v1:172.21.15.157:6789/0 conn(0x7f17f80c2790 legacy=0x7f17f8069c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:02.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.248+0000 7f17f659c700 1 -- 172.21.15.157:0/1176811577 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f806c9c0 con 0x7f17f80c23c0 2023-10-06T04:03:02.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.248+0000 7f17f5d9b700 1 -- 172.21.15.157:0/1176811577 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1972126419 0 0) 0x7f17f806d5d0 con 0x7f17f80c2790 2023-10-06T04:03:02.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.248+0000 7f17f659c700 1 --2- 172.21.15.157:0/1176811577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17f80c23c0 0x7f17f8155f70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f17e80098e0 tx=0x7f17e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab51a3944c50c1b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:02.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.248+0000 7f17f5d9b700 1 -- 172.21.15.157:0/1176811577 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e8009e80 con 0x7f17f80c23c0 2023-10-06T04:03:02.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.248+0000 7f17f5d9b700 1 -- 172.21.15.157:0/1176811577 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e800b8b0 con 0x7f17f80c23c0 2023-10-06T04:03:02.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.248+0000 7f17f5d9b700 1 -- 172.21.15.157:0/1176811577 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e800bc30 con 0x7f17f80c23c0 2023-10-06T04:03:02.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.249+0000 7f17fcfac700 1 -- 172.21.15.157:0/1176811577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17f80c23c0 msgr2=0x7f17f8155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:02.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.249+0000 7f17fcfac700 1 --2- 172.21.15.157:0/1176811577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17f80c23c0 0x7f17f8155f70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f17e80098e0 tx=0x7f17e80091a0 comp rx=0 tx=0).stop 2023-10-06T04:03:02.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.249+0000 7f17fcfac700 1 -- 172.21.15.157:0/1176811577 shutdown_connections 2023-10-06T04:03:02.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.249+0000 7f17fcfac700 1 --2- 172.21.15.157:0/1176811577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17f80c23c0 0x7f17f8155f70 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:02.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.249+0000 7f17fcfac700 1 -- 172.21.15.157:0/1176811577 >> 172.21.15.157:0/1176811577 conn(0x7f17f806f290 msgr2=0x7f17f806f690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:02.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.249+0000 7f17fcfac700 1 -- 172.21.15.157:0/1176811577 shutdown_connections 2023-10-06T04:03:02.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.249+0000 7f17fcfac700 1 -- 172.21.15.157:0/1176811577 wait complete. 2023-10-06T04:03:02.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.250+0000 7f17fcfac700 1 Processor -- start 2023-10-06T04:03:02.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.250+0000 7f17fcfac700 1 -- start start 2023-10-06T04:03:02.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.250+0000 7f17fcfac700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17f806c1f0 0x7f17f80bc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:02.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.250+0000 7f17fcfac700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17f80c23c0 0x7f17f80bd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:02.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.250+0000 7f17fcfac700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17f80c2790 0x7f17f8169b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:02.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.250+0000 7f17fcfac700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f80bc270 con 0x7f17f80c2790 2023-10-06T04:03:02.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.251+0000 7f17fcfac700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f80bf3b0 con 0x7f17f806c1f0 2023-10-06T04:03:02.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.251+0000 7f17fcfac700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f816a810 con 0x7f17f80c23c0 2023-10-06T04:03:02.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.251+0000 7f17f6d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17f80c2790 0x7f17f8169b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:02.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.251+0000 7f17f6d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17f80c2790 0x7f17f8169b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47210/0 (socket says 172.21.15.157:47210) 2023-10-06T04:03:02.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.251+0000 7f17f6d9d700 1 -- 172.21.15.157:0/2952993738 learned_addr learned my addr 172.21.15.157:0/2952993738 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:02.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.251+0000 7f17effff700 1 --2- 172.21.15.157:0/2952993738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17f80c23c0 0x7f17f80bd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:02.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.251+0000 7f17f6d9d700 1 -- 172.21.15.157:0/2952993738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17f80c23c0 msgr2=0x7f17f80bd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:02.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.251+0000 7f17f6d9d700 1 --2- 172.21.15.157:0/2952993738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17f80c23c0 0x7f17f80bd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:02.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.251+0000 7f17f6d9d700 1 -- 172.21.15.157:0/2952993738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17f806c1f0 msgr2=0x7f17f80bc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:02.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.251+0000 7f17f6d9d700 1 --2- 172.21.15.157:0/2952993738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17f806c1f0 0x7f17f80bc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:02.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.251+0000 7f17f6d9d700 1 -- 172.21.15.157:0/2952993738 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e8009420 con 0x7f17f80c2790 2023-10-06T04:03:02.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.251+0000 7f17effff700 1 --2- 172.21.15.157:0/2952993738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17f80c23c0 0x7f17f80bd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:03:02.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.251+0000 7f17f6d9d700 1 --2- 172.21.15.157:0/2952993738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17f80c2790 0x7f17f8169b40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f17f000cbc0 tx=0x7f17f0001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:02.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.251+0000 7f17ee7fc700 1 -- 172.21.15.157:0/2952993738 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f000a940 con 0x7f17f80c2790 2023-10-06T04:03:02.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.251+0000 7f17ee7fc700 1 -- 172.21.15.157:0/2952993738 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f000aaa0 con 0x7f17f80c2790 2023-10-06T04:03:02.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.251+0000 7f17fcfac700 1 -- 172.21.15.157:0/2952993738 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17f80c51b0 con 0x7f17f80c2790 2023-10-06T04:03:02.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.251+0000 7f17ee7fc700 1 -- 172.21.15.157:0/2952993738 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f000ae60 con 0x7f17f80c2790 2023-10-06T04:03:02.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.251+0000 7f17fcfac700 1 -- 172.21.15.157:0/2952993738 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17f8150830 con 0x7f17f80c2790 2023-10-06T04:03:02.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.252+0000 7f17fcfac700 1 -- 172.21.15.157:0/2952993738 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17f8068c80 con 0x7f17f80c2790 2023-10-06T04:03:02.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.253+0000 7f17ee7fc700 1 -- 172.21.15.157:0/2952993738 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f17f0007300 con 0x7f17f80c2790 2023-10-06T04:03:02.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.253+0000 7f17ee7fc700 1 --2- 172.21.15.157:0/2952993738 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f17d8066f30 0x7f17d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:02.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.253+0000 7f17ee7fc700 1 -- 172.21.15.157:0/2952993738 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7f17f0084cc0 con 0x7f17f80c2790 2023-10-06T04:03:02.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.253+0000 7f17f659c700 1 --2- 172.21.15.157:0/2952993738 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f17d8066f30 0x7f17d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:02.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.254+0000 7f17f659c700 1 --2- 172.21.15.157:0/2952993738 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f17d8066f30 0x7f17d80693e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f17e8006da0 tx=0x7f17e8016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:02.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.257+0000 7f17ee7fc700 1 -- 172.21.15.157:0/2952993738 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f17f0051c00 con 0x7f17f80c2790 2023-10-06T04:03:02.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.483+0000 7f17fcfac700 1 -- 172.21.15.157:0/2952993738 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17f80bf520 con 0x7f17f80c2790 2023-10-06T04:03:02.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.484+0000 7f17ee7fc700 1 -- 172.21.15.157:0/2952993738 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+6160 (secure 0 0 0) 0x7f17f80bf520 con 0x7f17f80c2790 2023-10-06T04:03:02.485 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:03:02.486 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3192541484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3192541484},{"type":"v1","addr":"172.21.15.175:6835","nonce":3192541484}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:01.994829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279},"failed":[],"damaged":[],"stopped":[3,4],"info":{"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:stopping","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:03:02.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.486+0000 7f17fcfac700 1 -- 172.21.15.157:0/2952993738 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f17d8066f30 msgr2=0x7f17d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:02.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.486+0000 7f17fcfac700 1 --2- 172.21.15.157:0/2952993738 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f17d8066f30 0x7f17d80693e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f17e8006da0 tx=0x7f17e8016000 comp rx=0 tx=0).stop 2023-10-06T04:03:02.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.486+0000 7f17fcfac700 1 -- 172.21.15.157:0/2952993738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17f80c2790 msgr2=0x7f17f8169b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:02.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.486+0000 7f17fcfac700 1 --2- 172.21.15.157:0/2952993738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17f80c2790 0x7f17f8169b40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f17f000cbc0 tx=0x7f17f0001980 comp rx=0 tx=0).stop 2023-10-06T04:03:02.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.486+0000 7f17fcfac700 1 -- 172.21.15.157:0/2952993738 shutdown_connections 2023-10-06T04:03:02.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.486+0000 7f17fcfac700 1 --2- 172.21.15.157:0/2952993738 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f17d8066f30 0x7f17d80693e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:02.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.486+0000 7f17fcfac700 1 --2- 172.21.15.157:0/2952993738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17f806c1f0 0x7f17f80bc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:02.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.486+0000 7f17fcfac700 1 --2- 172.21.15.157:0/2952993738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17f80c2790 0x7f17f8169b40 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:02.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.486+0000 7f17fcfac700 1 --2- 172.21.15.157:0/2952993738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17f80c23c0 0x7f17f80bd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:02.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.486+0000 7f17fcfac700 1 -- 172.21.15.157:0/2952993738 >> 172.21.15.157:0/2952993738 conn(0x7f17f806f290 msgr2=0x7f17f8157cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:02.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.486+0000 7f17fcfac700 1 -- 172.21.15.157:0/2952993738 shutdown_connections 2023-10-06T04:03:02.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:02.486+0000 7f17fcfac700 1 -- 172.21.15.157:0/2952993738 wait complete. 2023-10-06T04:03:02.489 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 49 2023-10-06T04:03:02.499 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:03:04.500 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:03:04.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.614+0000 7f44025ba700 1 Processor -- start 2023-10-06T04:03:04.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.614+0000 7f44025ba700 1 -- start start 2023-10-06T04:03:04.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.615+0000 7f44025ba700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43fc068d10 0x7f43fc06a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:04.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.615+0000 7f44025ba700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f43fc06d460 0x7f43fc06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:04.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.615+0000 7f44025ba700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43fc0bed40 con 0x7f43fc0be970 2023-10-06T04:03:04.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.615+0000 7f44025ba700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43fc157010 con 0x7f43fc06d460 2023-10-06T04:03:04.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.615+0000 7f44025ba700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc06f9f0 con 0x7f43fc068d10 2023-10-06T04:03:04.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.615+0000 7f43fbfff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f43fc0be970 0x7f43fc06df10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49028/0 (socket says 172.21.15.157:49028) 2023-10-06T04:03:04.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.615+0000 7f43fbfff700 1 -- 172.21.15.157:0/3849723123 learned_addr learned my addr 172.21.15.157:0/3849723123 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:03:04.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.615+0000 7f43fb7fe700 1 --2- 172.21.15.157:0/3849723123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43fc068d10 0x7f43fc06a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:04.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.615+0000 7f4400b57700 1 --2- 172.21.15.157:0/3849723123 >> v2:172.21.15.175:3300/0 conn(0x7f43fc06d460 0x7f43fc06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:04.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.616+0000 7f43faffd700 1 -- 172.21.15.157:0/3849723123 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1394368042 0 0) 0x7f43fc0bed40 con 0x7f43fc0be970 2023-10-06T04:03:04.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.616+0000 7f43faffd700 1 -- 172.21.15.157:0/3849723123 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43e4003570 con 0x7f43fc0be970 2023-10-06T04:03:04.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.616+0000 7f4400b57700 1 -- 172.21.15.157:0/3849723123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43fc068d10 msgr2=0x7f43fc06a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:04.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.616+0000 7f4400b57700 1 --2- 172.21.15.157:0/3849723123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43fc068d10 0x7f43fc06a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:04.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.616+0000 7f4400b57700 1 -- 172.21.15.157:0/3849723123 >> v1:172.21.15.157:6789/0 conn(0x7f43fc0be970 legacy=0x7f43fc06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:04.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.616+0000 7f4400b57700 1 -- 172.21.15.157:0/3849723123 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43fc1574a0 con 0x7f43fc06d460 2023-10-06T04:03:04.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.616+0000 7f43fb7fe700 1 --2- 172.21.15.157:0/3849723123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43fc068d10 0x7f43fc06a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:03:04.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.616+0000 7f4400b57700 1 --2- 172.21.15.157:0/3849723123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43fc06d460 0x7f43fc06ab40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f43f4009e60 tx=0x7f43f4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acbec7c78f65e37 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:04.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.616+0000 7f43faffd700 1 -- 172.21.15.157:0/3849723123 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f400e070 con 0x7f43fc06d460 2023-10-06T04:03:04.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.616+0000 7f43faffd700 1 -- 172.21.15.157:0/3849723123 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f400bab0 con 0x7f43fc06d460 2023-10-06T04:03:04.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.616+0000 7f43faffd700 1 -- 172.21.15.157:0/3849723123 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f400be70 con 0x7f43fc06d460 2023-10-06T04:03:04.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.617+0000 7f44025ba700 1 -- 172.21.15.157:0/3849723123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43fc06d460 msgr2=0x7f43fc06ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:04.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.617+0000 7f44025ba700 1 --2- 172.21.15.157:0/3849723123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43fc06d460 0x7f43fc06ab40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f43f4009e60 tx=0x7f43f4009780 comp rx=0 tx=0).stop 2023-10-06T04:03:04.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.617+0000 7f44025ba700 1 -- 172.21.15.157:0/3849723123 shutdown_connections 2023-10-06T04:03:04.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.617+0000 7f44025ba700 1 --2- 172.21.15.157:0/3849723123 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43fc06d460 0x7f43fc06ab40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:04.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.617+0000 7f44025ba700 1 --2- 172.21.15.157:0/3849723123 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43fc068d10 0x7f43fc06a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:04.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.617+0000 7f44025ba700 1 -- 172.21.15.157:0/3849723123 >> 172.21.15.157:0/3849723123 conn(0x7f43fc0c8d70 msgr2=0x7f43fc0c4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:04.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.617+0000 7f44025ba700 1 -- 172.21.15.157:0/3849723123 shutdown_connections 2023-10-06T04:03:04.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.617+0000 7f44025ba700 1 -- 172.21.15.157:0/3849723123 wait complete. 2023-10-06T04:03:04.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.618+0000 7f44025ba700 1 Processor -- start 2023-10-06T04:03:04.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.618+0000 7f44025ba700 1 -- start start 2023-10-06T04:03:04.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.619+0000 7f44025ba700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43fc068d10 0x7f43fc154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:04.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.619+0000 7f44025ba700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43fc06d460 0x7f43fc166960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:04.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.619+0000 7f44025ba700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43fc0be970 0x7f43fc069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:04.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.619+0000 7f44025ba700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc145df0 con 0x7f43fc06d460 2023-10-06T04:03:04.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.619+0000 7f44025ba700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc1664d0 con 0x7f43fc068d10 2023-10-06T04:03:04.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.619+0000 7f44025ba700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc1545c0 con 0x7f43fc0be970 2023-10-06T04:03:04.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.619+0000 7f43fb7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43fc06d460 0x7f43fc166960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:04.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.619+0000 7f43fb7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43fc06d460 0x7f43fc166960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47838/0 (socket says 172.21.15.157:47838) 2023-10-06T04:03:04.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.619+0000 7f43fb7fe700 1 -- 172.21.15.157:0/916647556 learned_addr learned my addr 172.21.15.157:0/916647556 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:04.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.619+0000 7f43fbfff700 1 --2- 172.21.15.157:0/916647556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43fc068d10 0x7f43fc154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:04.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.619+0000 7f4400b57700 1 --2- 172.21.15.157:0/916647556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43fc0be970 0x7f43fc069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:04.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.619+0000 7f43fb7fe700 1 -- 172.21.15.157:0/916647556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43fc0be970 msgr2=0x7f43fc069740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:04.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.619+0000 7f43fb7fe700 1 --2- 172.21.15.157:0/916647556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43fc0be970 0x7f43fc069740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:04.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.619+0000 7f43fb7fe700 1 -- 172.21.15.157:0/916647556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43fc068d10 msgr2=0x7f43fc154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:04.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.619+0000 7f43fb7fe700 1 --2- 172.21.15.157:0/916647556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43fc068d10 0x7f43fc154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:04.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.619+0000 7f43fb7fe700 1 -- 172.21.15.157:0/916647556 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f0009420 con 0x7f43fc06d460 2023-10-06T04:03:04.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.619+0000 7f43fb7fe700 1 --2- 172.21.15.157:0/916647556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43fc06d460 0x7f43fc166960 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f43f000ec80 tx=0x7f43f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:04.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.619+0000 7f43e3fff700 1 -- 172.21.15.157:0/916647556 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f000c9b0 con 0x7f43fc06d460 2023-10-06T04:03:04.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.619+0000 7f43e3fff700 1 -- 172.21.15.157:0/916647556 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f000cb10 con 0x7f43fc06d460 2023-10-06T04:03:04.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.619+0000 7f44025ba700 1 -- 172.21.15.157:0/916647556 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f4009a20 con 0x7f43fc06d460 2023-10-06T04:03:04.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.619+0000 7f43e3fff700 1 -- 172.21.15.157:0/916647556 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f000ce90 con 0x7f43fc06d460 2023-10-06T04:03:04.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.619+0000 7f44025ba700 1 -- 172.21.15.157:0/916647556 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43fc166160 con 0x7f43fc06d460 2023-10-06T04:03:04.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.620+0000 7f44025ba700 1 -- 172.21.15.157:0/916647556 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43fc0716b0 con 0x7f43fc06d460 2023-10-06T04:03:04.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.621+0000 7f43e3fff700 1 -- 172.21.15.157:0/916647556 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f43f0013680 con 0x7f43fc06d460 2023-10-06T04:03:04.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.621+0000 7f43e3fff700 1 --2- 172.21.15.157:0/916647556 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f43e8066f30 0x7f43e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:04.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.621+0000 7f43e3fff700 1 -- 172.21.15.157:0/916647556 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7f43f0084cd0 con 0x7f43fc06d460 2023-10-06T04:03:04.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.621+0000 7f43fbfff700 1 --2- 172.21.15.157:0/916647556 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f43e8066f30 0x7f43e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:04.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.622+0000 7f43fbfff700 1 --2- 172.21.15.157:0/916647556 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f43e8066f30 0x7f43e80693e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f43ec007fd0 tx=0x7f43ec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:04.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.625+0000 7f43e3fff700 1 -- 172.21.15.157:0/916647556 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43f0055620 con 0x7f43fc06d460 2023-10-06T04:03:04.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.868+0000 7f44025ba700 1 -- 172.21.15.157:0/916647556 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43fc0c9170 con 0x7f43fc06d460 2023-10-06T04:03:04.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.869+0000 7f43e3fff700 1 -- 172.21.15.157:0/916647556 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+6160 (secure 0 0 0) 0x7f43f00551b0 con 0x7f43fc06d460 2023-10-06T04:03:04.870 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:03:04.870 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3192541484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3192541484},{"type":"v1","addr":"172.21.15.175:6835","nonce":3192541484}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:01.994829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279},"failed":[],"damaged":[],"stopped":[3,4],"info":{"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:stopping","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:03:04.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.871+0000 7f44025ba700 1 -- 172.21.15.157:0/916647556 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f43e8066f30 msgr2=0x7f43e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:04.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.871+0000 7f44025ba700 1 --2- 172.21.15.157:0/916647556 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f43e8066f30 0x7f43e80693e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f43ec007fd0 tx=0x7f43ec009040 comp rx=0 tx=0).stop 2023-10-06T04:03:04.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.871+0000 7f44025ba700 1 -- 172.21.15.157:0/916647556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43fc06d460 msgr2=0x7f43fc166960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:04.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.871+0000 7f44025ba700 1 --2- 172.21.15.157:0/916647556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43fc06d460 0x7f43fc166960 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f43f000ec80 tx=0x7f43f00091a0 comp rx=0 tx=0).stop 2023-10-06T04:03:04.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.871+0000 7f44025ba700 1 -- 172.21.15.157:0/916647556 shutdown_connections 2023-10-06T04:03:04.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.871+0000 7f44025ba700 1 --2- 172.21.15.157:0/916647556 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f43e8066f30 0x7f43e80693e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:04.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.871+0000 7f44025ba700 1 --2- 172.21.15.157:0/916647556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43fc068d10 0x7f43fc154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:04.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.871+0000 7f44025ba700 1 --2- 172.21.15.157:0/916647556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43fc0be970 0x7f43fc069740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:04.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.871+0000 7f44025ba700 1 --2- 172.21.15.157:0/916647556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43fc06d460 0x7f43fc166960 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:04.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.871+0000 7f44025ba700 1 -- 172.21.15.157:0/916647556 >> 172.21.15.157:0/916647556 conn(0x7f43fc0c8d70 msgr2=0x7f43fc144e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:04.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.871+0000 7f44025ba700 1 -- 172.21.15.157:0/916647556 shutdown_connections 2023-10-06T04:03:04.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:04.871+0000 7f44025ba700 1 -- 172.21.15.157:0/916647556 wait complete. 2023-10-06T04:03:04.876 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 49 2023-10-06T04:03:04.884 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2023-10-06T04:03:04.885 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 49, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-06T03:57:10.718782+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 49, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4279": { "addr": "172.21.15.175:6837/1002404870", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6836", "nonce": 1002404870, "type": "v2" }, { "addr": "172.21.15.175:6837", "nonce": 1002404870, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 3 ], "features": 4540138320759226367, "flags": 0, "gid": 4279, "incarnation": 11, "join_fscid": -1, "name": "d", "rank": 2, "state": "up:stopping", "state_seq": 7 }, "gid_4287": { "addr": "172.21.15.157:6834/1179753185", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6832", "nonce": 1179753185, "type": "v2" }, { "addr": "172.21.15.157:6834", "nonce": 1179753185, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4287, "incarnation": 9, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4296": { "addr": "172.21.15.157:6837/4188652695", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6836", "nonce": 4188652695, "type": "v2" }, { "addr": "172.21.15.157:6837", "nonce": 4188652695, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4296, "incarnation": 4, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 27, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2023-10-06T04:03:01.994829+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 3, 4 ], "tableserver": 0, "up": { "mds_0": 4296, "mds_1": 4287, "mds_2": 4279 } } } ], "standbys": [ { "addr": "172.21.15.175:6835/3192541484", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6834", "nonce": 3192541484, "type": "v2" }, { "addr": "172.21.15.175:6835", "nonce": 3192541484, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 48, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5009, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.175:6839/745804271", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6838", "nonce": 745804271, "type": "v2" }, { "addr": "172.21.15.175:6839", "nonce": 745804271, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 42, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5164, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.157:6835/2461751318", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6833", "nonce": 2461751318, "type": "v2" }, { "addr": "172.21.15.157:6835", "nonce": 2461751318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 34, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5268, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-06T04:03:05.344 INFO:tasks.workunit.client.0.smithi157.stdout:removed './157/17' 2023-10-06T04:03:05.344 INFO:tasks.workunit.client.0.smithi157.stdout:removed './157/35' 2023-10-06T04:03:05.345 INFO:tasks.workunit.client.0.smithi157.stdout:removed './157/37' 2023-10-06T04:03:05.345 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './157' 2023-10-06T04:03:05.345 INFO:tasks.workunit.client.0.smithi157.stdout:removed './158/19' 2023-10-06T04:03:05.345 INFO:tasks.workunit.client.0.smithi157.stdout:removed './158/15' 2023-10-06T04:03:05.345 INFO:tasks.workunit.client.0.smithi157.stdout:removed './158/31' 2023-10-06T04:03:05.345 INFO:tasks.workunit.client.0.smithi157.stdout:removed './158/36' 2023-10-06T04:03:05.345 INFO:tasks.workunit.client.0.smithi157.stdout:removed './158/20' 2023-10-06T04:03:05.345 INFO:tasks.workunit.client.0.smithi157.stdout:removed './158/12' 2023-10-06T04:03:05.345 INFO:tasks.workunit.client.0.smithi157.stdout:removed './158/39' 2023-10-06T04:03:05.345 INFO:tasks.workunit.client.0.smithi157.stdout:removed './158/10' 2023-10-06T04:03:05.345 INFO:tasks.workunit.client.0.smithi157.stdout:removed './158/34' 2023-10-06T04:03:05.346 INFO:tasks.workunit.client.0.smithi157.stdout:removed './158/38' 2023-10-06T04:03:05.346 INFO:tasks.workunit.client.0.smithi157.stdout:removed './158/11' 2023-10-06T04:03:05.346 INFO:tasks.workunit.client.0.smithi157.stdout:removed './158/16' 2023-10-06T04:03:05.346 INFO:tasks.workunit.client.0.smithi157.stdout:removed './158/32' 2023-10-06T04:03:05.346 INFO:tasks.workunit.client.0.smithi157.stdout:removed './158/30' 2023-10-06T04:03:05.346 INFO:tasks.workunit.client.0.smithi157.stdout:removed './158/14' 2023-10-06T04:03:05.346 INFO:tasks.workunit.client.0.smithi157.stdout:removed './158/18' 2023-10-06T04:03:05.346 INFO:tasks.workunit.client.0.smithi157.stdout:removed './158/13' 2023-10-06T04:03:05.346 INFO:tasks.workunit.client.0.smithi157.stdout:removed './158/33' 2023-10-06T04:03:05.346 INFO:tasks.workunit.client.0.smithi157.stdout:removed './158/17' 2023-10-06T04:03:05.346 INFO:tasks.workunit.client.0.smithi157.stdout:removed './158/35' 2023-10-06T04:03:05.346 INFO:tasks.workunit.client.0.smithi157.stdout:removed './158/37' 2023-10-06T04:03:05.347 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './158' 2023-10-06T04:03:05.347 INFO:tasks.workunit.client.0.smithi157.stdout:removed './159/19' 2023-10-06T04:03:05.347 INFO:tasks.workunit.client.0.smithi157.stdout:removed './159/15' 2023-10-06T04:03:05.347 INFO:tasks.workunit.client.0.smithi157.stdout:removed './159/31' 2023-10-06T04:03:05.347 INFO:tasks.workunit.client.0.smithi157.stdout:removed './159/36' 2023-10-06T04:03:05.347 INFO:tasks.workunit.client.0.smithi157.stdout:removed './159/20' 2023-10-06T04:03:05.347 INFO:tasks.workunit.client.0.smithi157.stdout:removed './159/12' 2023-10-06T04:03:05.347 INFO:tasks.workunit.client.0.smithi157.stdout:removed './159/39' 2023-10-06T04:03:05.347 INFO:tasks.workunit.client.0.smithi157.stdout:removed './159/10' 2023-10-06T04:03:05.347 INFO:tasks.workunit.client.0.smithi157.stdout:removed './159/34' 2023-10-06T04:03:05.347 INFO:tasks.workunit.client.0.smithi157.stdout:removed './159/38' 2023-10-06T04:03:05.347 INFO:tasks.workunit.client.0.smithi157.stdout:removed './159/11' 2023-10-06T04:03:05.348 INFO:tasks.workunit.client.0.smithi157.stdout:removed './159/16' 2023-10-06T04:03:05.348 INFO:tasks.workunit.client.0.smithi157.stdout:removed './159/32' 2023-10-06T04:03:05.348 INFO:tasks.workunit.client.0.smithi157.stdout:removed './159/30' 2023-10-06T04:03:05.348 INFO:tasks.workunit.client.0.smithi157.stdout:removed './159/14' 2023-10-06T04:03:05.348 INFO:tasks.workunit.client.0.smithi157.stdout:removed './159/18' 2023-10-06T04:03:05.348 INFO:tasks.workunit.client.0.smithi157.stdout:removed './159/13' 2023-10-06T04:03:05.348 INFO:tasks.workunit.client.0.smithi157.stdout:removed './159/33' 2023-10-06T04:03:05.348 INFO:tasks.workunit.client.0.smithi157.stdout:removed './159/17' 2023-10-06T04:03:05.348 INFO:tasks.workunit.client.0.smithi157.stdout:removed './159/35' 2023-10-06T04:03:05.348 INFO:tasks.workunit.client.0.smithi157.stdout:removed './159/37' 2023-10-06T04:03:05.348 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './159' 2023-10-06T04:03:05.348 INFO:tasks.workunit.client.0.smithi157.stdout:removed './160/19' 2023-10-06T04:03:05.349 INFO:tasks.workunit.client.0.smithi157.stdout:removed './160/15' 2023-10-06T04:03:05.349 INFO:tasks.workunit.client.0.smithi157.stdout:removed './160/31' 2023-10-06T04:03:05.349 INFO:tasks.workunit.client.0.smithi157.stdout:removed './160/36' 2023-10-06T04:03:05.349 INFO:tasks.workunit.client.0.smithi157.stdout:removed './160/20' 2023-10-06T04:03:05.349 INFO:tasks.workunit.client.0.smithi157.stdout:removed './160/12' 2023-10-06T04:03:05.349 INFO:tasks.workunit.client.0.smithi157.stdout:removed './160/39' 2023-10-06T04:03:05.349 INFO:tasks.workunit.client.0.smithi157.stdout:removed './160/10' 2023-10-06T04:03:05.349 INFO:tasks.workunit.client.0.smithi157.stdout:removed './160/34' 2023-10-06T04:03:05.349 INFO:tasks.workunit.client.0.smithi157.stdout:removed './160/38' 2023-10-06T04:03:05.349 INFO:tasks.workunit.client.0.smithi157.stdout:removed './160/11' 2023-10-06T04:03:05.349 INFO:tasks.workunit.client.0.smithi157.stdout:removed './160/16' 2023-10-06T04:03:05.349 INFO:tasks.workunit.client.0.smithi157.stdout:removed './160/32' 2023-10-06T04:03:05.350 INFO:tasks.workunit.client.0.smithi157.stdout:removed './160/30' 2023-10-06T04:03:05.350 INFO:tasks.workunit.client.0.smithi157.stdout:removed './160/14' 2023-10-06T04:03:05.350 INFO:tasks.workunit.client.0.smithi157.stdout:removed './160/18' 2023-10-06T04:03:05.350 INFO:tasks.workunit.client.0.smithi157.stdout:removed './160/13' 2023-10-06T04:03:05.350 INFO:tasks.workunit.client.0.smithi157.stdout:removed './160/33' 2023-10-06T04:03:05.350 INFO:tasks.workunit.client.0.smithi157.stdout:removed './160/17' 2023-10-06T04:03:05.350 INFO:tasks.workunit.client.0.smithi157.stdout:removed './160/35' 2023-10-06T04:03:05.350 INFO:tasks.workunit.client.0.smithi157.stdout:removed './160/37' 2023-10-06T04:03:05.350 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './160' 2023-10-06T04:03:05.350 INFO:tasks.workunit.client.0.smithi157.stdout:removed './161/19' 2023-10-06T04:03:05.350 INFO:tasks.workunit.client.0.smithi157.stdout:removed './161/15' 2023-10-06T04:03:05.351 INFO:tasks.workunit.client.0.smithi157.stdout:removed './161/31' 2023-10-06T04:03:05.351 INFO:tasks.workunit.client.0.smithi157.stdout:removed './161/36' 2023-10-06T04:03:05.351 INFO:tasks.workunit.client.0.smithi157.stdout:removed './161/20' 2023-10-06T04:03:05.351 INFO:tasks.workunit.client.0.smithi157.stdout:removed './161/12' 2023-10-06T04:03:05.351 INFO:tasks.workunit.client.0.smithi157.stdout:removed './161/39' 2023-10-06T04:03:05.351 INFO:tasks.workunit.client.0.smithi157.stdout:removed './161/10' 2023-10-06T04:03:05.351 INFO:tasks.workunit.client.0.smithi157.stdout:removed './161/34' 2023-10-06T04:03:05.351 INFO:tasks.workunit.client.0.smithi157.stdout:removed './161/38' 2023-10-06T04:03:05.352 INFO:tasks.workunit.client.0.smithi157.stdout:removed './161/11' 2023-10-06T04:03:05.352 INFO:tasks.workunit.client.0.smithi157.stdout:removed './161/16' 2023-10-06T04:03:05.352 INFO:tasks.workunit.client.0.smithi157.stdout:removed './161/32' 2023-10-06T04:03:05.352 INFO:tasks.workunit.client.0.smithi157.stdout:removed './161/30' 2023-10-06T04:03:05.352 INFO:tasks.workunit.client.0.smithi157.stdout:removed './161/14' 2023-10-06T04:03:05.352 INFO:tasks.workunit.client.0.smithi157.stdout:removed './161/18' 2023-10-06T04:03:05.352 INFO:tasks.workunit.client.0.smithi157.stdout:removed './161/13' 2023-10-06T04:03:05.352 INFO:tasks.workunit.client.0.smithi157.stdout:removed './161/33' 2023-10-06T04:03:05.352 INFO:tasks.workunit.client.0.smithi157.stdout:removed './161/17' 2023-10-06T04:03:05.352 INFO:tasks.workunit.client.0.smithi157.stdout:removed './161/35' 2023-10-06T04:03:05.352 INFO:tasks.workunit.client.0.smithi157.stdout:removed './161/37' 2023-10-06T04:03:05.352 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './161' 2023-10-06T04:03:05.353 INFO:tasks.workunit.client.0.smithi157.stdout:removed './162/19' 2023-10-06T04:03:05.353 INFO:tasks.workunit.client.0.smithi157.stdout:removed './162/15' 2023-10-06T04:03:05.353 INFO:tasks.workunit.client.0.smithi157.stdout:removed './162/31' 2023-10-06T04:03:05.353 INFO:tasks.workunit.client.0.smithi157.stdout:removed './162/36' 2023-10-06T04:03:05.353 INFO:tasks.workunit.client.0.smithi157.stdout:removed './162/20' 2023-10-06T04:03:05.353 INFO:tasks.workunit.client.0.smithi157.stdout:removed './162/12' 2023-10-06T04:03:05.353 INFO:tasks.workunit.client.0.smithi157.stdout:removed './162/39' 2023-10-06T04:03:05.353 INFO:tasks.workunit.client.0.smithi157.stdout:removed './162/10' 2023-10-06T04:03:05.353 INFO:tasks.workunit.client.0.smithi157.stdout:removed './162/34' 2023-10-06T04:03:05.353 INFO:tasks.workunit.client.0.smithi157.stdout:removed './162/38' 2023-10-06T04:03:05.353 INFO:tasks.workunit.client.0.smithi157.stdout:removed './162/11' 2023-10-06T04:03:05.353 INFO:tasks.workunit.client.0.smithi157.stdout:removed './162/16' 2023-10-06T04:03:05.354 INFO:tasks.workunit.client.0.smithi157.stdout:removed './162/32' 2023-10-06T04:03:05.354 INFO:tasks.workunit.client.0.smithi157.stdout:removed './162/30' 2023-10-06T04:03:05.354 INFO:tasks.workunit.client.0.smithi157.stdout:removed './162/14' 2023-10-06T04:03:05.354 INFO:tasks.workunit.client.0.smithi157.stdout:removed './162/18' 2023-10-06T04:03:05.354 INFO:tasks.workunit.client.0.smithi157.stdout:removed './162/13' 2023-10-06T04:03:05.354 INFO:tasks.workunit.client.0.smithi157.stdout:removed './162/33' 2023-10-06T04:03:05.354 INFO:tasks.workunit.client.0.smithi157.stdout:removed './162/17' 2023-10-06T04:03:05.354 INFO:tasks.workunit.client.0.smithi157.stdout:removed './162/35' 2023-10-06T04:03:05.354 INFO:tasks.workunit.client.0.smithi157.stdout:removed './162/37' 2023-10-06T04:03:05.354 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './162' 2023-10-06T04:03:05.354 INFO:tasks.workunit.client.0.smithi157.stdout:removed './163/19' 2023-10-06T04:03:05.354 INFO:tasks.workunit.client.0.smithi157.stdout:removed './163/15' 2023-10-06T04:03:05.355 INFO:tasks.workunit.client.0.smithi157.stdout:removed './163/31' 2023-10-06T04:03:05.355 INFO:tasks.workunit.client.0.smithi157.stdout:removed './163/36' 2023-10-06T04:03:05.355 INFO:tasks.workunit.client.0.smithi157.stdout:removed './163/20' 2023-10-06T04:03:05.355 INFO:tasks.workunit.client.0.smithi157.stdout:removed './163/12' 2023-10-06T04:03:05.355 INFO:tasks.workunit.client.0.smithi157.stdout:removed './163/39' 2023-10-06T04:03:05.355 INFO:tasks.workunit.client.0.smithi157.stdout:removed './163/10' 2023-10-06T04:03:05.355 INFO:tasks.workunit.client.0.smithi157.stdout:removed './163/34' 2023-10-06T04:03:05.355 INFO:tasks.workunit.client.0.smithi157.stdout:removed './163/38' 2023-10-06T04:03:05.355 INFO:tasks.workunit.client.0.smithi157.stdout:removed './163/11' 2023-10-06T04:03:05.355 INFO:tasks.workunit.client.0.smithi157.stdout:removed './163/16' 2023-10-06T04:03:05.355 INFO:tasks.workunit.client.0.smithi157.stdout:removed './163/32' 2023-10-06T04:03:05.355 INFO:tasks.workunit.client.0.smithi157.stdout:removed './163/30' 2023-10-06T04:03:05.356 INFO:tasks.workunit.client.0.smithi157.stdout:removed './163/14' 2023-10-06T04:03:05.356 INFO:tasks.workunit.client.0.smithi157.stdout:removed './163/18' 2023-10-06T04:03:05.356 INFO:tasks.workunit.client.0.smithi157.stdout:removed './163/13' 2023-10-06T04:03:05.356 INFO:tasks.workunit.client.0.smithi157.stdout:removed './163/33' 2023-10-06T04:03:05.356 INFO:tasks.workunit.client.0.smithi157.stdout:removed './163/17' 2023-10-06T04:03:05.356 INFO:tasks.workunit.client.0.smithi157.stdout:removed './163/35' 2023-10-06T04:03:05.356 INFO:tasks.workunit.client.0.smithi157.stdout:removed './163/37' 2023-10-06T04:03:05.356 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './163' 2023-10-06T04:03:05.356 INFO:tasks.workunit.client.0.smithi157.stdout:removed './164/19' 2023-10-06T04:03:05.356 INFO:tasks.workunit.client.0.smithi157.stdout:removed './164/15' 2023-10-06T04:03:05.356 INFO:tasks.workunit.client.0.smithi157.stdout:removed './164/31' 2023-10-06T04:03:05.356 INFO:tasks.workunit.client.0.smithi157.stdout:removed './164/36' 2023-10-06T04:03:05.357 INFO:tasks.workunit.client.0.smithi157.stdout:removed './164/20' 2023-10-06T04:03:05.357 INFO:tasks.workunit.client.0.smithi157.stdout:removed './164/12' 2023-10-06T04:03:05.357 INFO:tasks.workunit.client.0.smithi157.stdout:removed './164/39' 2023-10-06T04:03:05.357 INFO:tasks.workunit.client.0.smithi157.stdout:removed './164/10' 2023-10-06T04:03:05.357 INFO:tasks.workunit.client.0.smithi157.stdout:removed './164/34' 2023-10-06T04:03:05.357 INFO:tasks.workunit.client.0.smithi157.stdout:removed './164/38' 2023-10-06T04:03:05.357 INFO:tasks.workunit.client.0.smithi157.stdout:removed './164/11' 2023-10-06T04:03:05.357 INFO:tasks.workunit.client.0.smithi157.stdout:removed './164/16' 2023-10-06T04:03:05.357 INFO:tasks.workunit.client.0.smithi157.stdout:removed './164/32' 2023-10-06T04:03:05.357 INFO:tasks.workunit.client.0.smithi157.stdout:removed './164/30' 2023-10-06T04:03:05.357 INFO:tasks.workunit.client.0.smithi157.stdout:removed './164/14' 2023-10-06T04:03:05.357 INFO:tasks.workunit.client.0.smithi157.stdout:removed './164/18' 2023-10-06T04:03:05.357 INFO:tasks.workunit.client.0.smithi157.stdout:removed './164/13' 2023-10-06T04:03:05.358 INFO:tasks.workunit.client.0.smithi157.stdout:removed './164/33' 2023-10-06T04:03:05.358 INFO:tasks.workunit.client.0.smithi157.stdout:removed './164/17' 2023-10-06T04:03:05.358 INFO:tasks.workunit.client.0.smithi157.stdout:removed './164/35' 2023-10-06T04:03:05.358 INFO:tasks.workunit.client.0.smithi157.stdout:removed './164/37' 2023-10-06T04:03:05.358 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './164' 2023-10-06T04:03:05.358 INFO:tasks.workunit.client.0.smithi157.stdout:removed './165/19' 2023-10-06T04:03:05.358 INFO:tasks.workunit.client.0.smithi157.stdout:removed './165/15' 2023-10-06T04:03:05.358 INFO:tasks.workunit.client.0.smithi157.stdout:removed './165/31' 2023-10-06T04:03:05.358 INFO:tasks.workunit.client.0.smithi157.stdout:removed './165/36' 2023-10-06T04:03:05.358 INFO:tasks.workunit.client.0.smithi157.stdout:removed './165/20' 2023-10-06T04:03:05.358 INFO:tasks.workunit.client.0.smithi157.stdout:removed './165/12' 2023-10-06T04:03:05.358 INFO:tasks.workunit.client.0.smithi157.stdout:removed './165/39' 2023-10-06T04:03:05.359 INFO:tasks.workunit.client.0.smithi157.stdout:removed './165/10' 2023-10-06T04:03:05.359 INFO:tasks.workunit.client.0.smithi157.stdout:removed './165/34' 2023-10-06T04:03:05.359 INFO:tasks.workunit.client.0.smithi157.stdout:removed './165/38' 2023-10-06T04:03:05.359 INFO:tasks.workunit.client.0.smithi157.stdout:removed './165/11' 2023-10-06T04:03:05.359 INFO:tasks.workunit.client.0.smithi157.stdout:removed './165/16' 2023-10-06T04:03:05.359 INFO:tasks.workunit.client.0.smithi157.stdout:removed './165/32' 2023-10-06T04:03:05.359 INFO:tasks.workunit.client.0.smithi157.stdout:removed './165/30' 2023-10-06T04:03:05.359 INFO:tasks.workunit.client.0.smithi157.stdout:removed './165/14' 2023-10-06T04:03:05.359 INFO:tasks.workunit.client.0.smithi157.stdout:removed './165/18' 2023-10-06T04:03:05.359 INFO:tasks.workunit.client.0.smithi157.stdout:removed './165/13' 2023-10-06T04:03:05.359 INFO:tasks.workunit.client.0.smithi157.stdout:removed './165/33' 2023-10-06T04:03:05.359 INFO:tasks.workunit.client.0.smithi157.stdout:removed './165/17' 2023-10-06T04:03:05.360 INFO:tasks.workunit.client.0.smithi157.stdout:removed './165/35' 2023-10-06T04:03:05.360 INFO:tasks.workunit.client.0.smithi157.stdout:removed './165/37' 2023-10-06T04:03:05.360 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './165' 2023-10-06T04:03:05.360 INFO:tasks.workunit.client.0.smithi157.stdout:removed './166/19' 2023-10-06T04:03:05.360 INFO:tasks.workunit.client.0.smithi157.stdout:removed './166/15' 2023-10-06T04:03:05.360 INFO:tasks.workunit.client.0.smithi157.stdout:removed './166/31' 2023-10-06T04:03:05.360 INFO:tasks.workunit.client.0.smithi157.stdout:removed './166/36' 2023-10-06T04:03:05.360 INFO:tasks.workunit.client.0.smithi157.stdout:removed './166/20' 2023-10-06T04:03:05.360 INFO:tasks.workunit.client.0.smithi157.stdout:removed './166/12' 2023-10-06T04:03:05.360 INFO:tasks.workunit.client.0.smithi157.stdout:removed './166/39' 2023-10-06T04:03:05.360 INFO:tasks.workunit.client.0.smithi157.stdout:removed './166/10' 2023-10-06T04:03:05.360 INFO:tasks.workunit.client.0.smithi157.stdout:removed './166/34' 2023-10-06T04:03:05.361 INFO:tasks.workunit.client.0.smithi157.stdout:removed './166/38' 2023-10-06T04:03:05.361 INFO:tasks.workunit.client.0.smithi157.stdout:removed './166/11' 2023-10-06T04:03:05.361 INFO:tasks.workunit.client.0.smithi157.stdout:removed './166/16' 2023-10-06T04:03:05.361 INFO:tasks.workunit.client.0.smithi157.stdout:removed './166/32' 2023-10-06T04:03:05.361 INFO:tasks.workunit.client.0.smithi157.stdout:removed './166/30' 2023-10-06T04:03:05.361 INFO:tasks.workunit.client.0.smithi157.stdout:removed './166/14' 2023-10-06T04:03:05.361 INFO:tasks.workunit.client.0.smithi157.stdout:removed './166/18' 2023-10-06T04:03:05.361 INFO:tasks.workunit.client.0.smithi157.stdout:removed './166/13' 2023-10-06T04:03:05.361 INFO:tasks.workunit.client.0.smithi157.stdout:removed './166/33' 2023-10-06T04:03:05.361 INFO:tasks.workunit.client.0.smithi157.stdout:removed './166/17' 2023-10-06T04:03:05.361 INFO:tasks.workunit.client.0.smithi157.stdout:removed './166/35' 2023-10-06T04:03:05.361 INFO:tasks.workunit.client.0.smithi157.stdout:removed './166/37' 2023-10-06T04:03:05.362 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './166' 2023-10-06T04:03:05.362 INFO:tasks.workunit.client.0.smithi157.stdout:removed './167/19' 2023-10-06T04:03:05.362 INFO:tasks.workunit.client.0.smithi157.stdout:removed './167/15' 2023-10-06T04:03:05.362 INFO:tasks.workunit.client.0.smithi157.stdout:removed './167/31' 2023-10-06T04:03:05.362 INFO:tasks.workunit.client.0.smithi157.stdout:removed './167/36' 2023-10-06T04:03:05.362 INFO:tasks.workunit.client.0.smithi157.stdout:removed './167/20' 2023-10-06T04:03:05.362 INFO:tasks.workunit.client.0.smithi157.stdout:removed './167/12' 2023-10-06T04:03:05.362 INFO:tasks.workunit.client.0.smithi157.stdout:removed './167/39' 2023-10-06T04:03:05.362 INFO:tasks.workunit.client.0.smithi157.stdout:removed './167/10' 2023-10-06T04:03:05.362 INFO:tasks.workunit.client.0.smithi157.stdout:removed './167/34' 2023-10-06T04:03:05.362 INFO:tasks.workunit.client.0.smithi157.stdout:removed './167/38' 2023-10-06T04:03:06.887 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:03:07.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.009+0000 7fb4d9f4b700 1 Processor -- start 2023-10-06T04:03:07.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.010+0000 7fb4d9f4b700 1 -- start start 2023-10-06T04:03:07.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.010+0000 7fb4d9f4b700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb4d40b1e40 0x7fb4d4157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:07.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.010+0000 7fb4d9f4b700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d406d8d0 con 0x7fb4d406c2d0 2023-10-06T04:03:07.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.010+0000 7fb4d9f4b700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d40af8f0 con 0x7fb4d40b04d0 2023-10-06T04:03:07.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.010+0000 7fb4d9f4b700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4d415a690 con 0x7fb4d40b1e40 2023-10-06T04:03:07.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.010+0000 7fb4d8f49700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb4d40b1e40 0x7fb4d4157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:07.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.010+0000 7fb4d974a700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb4d406c2d0 0x7fb4d4156ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49042/0 (socket says 172.21.15.157:49042) 2023-10-06T04:03:07.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.010+0000 7fb4d974a700 1 -- 172.21.15.157:0/3022756073 learned_addr learned my addr 172.21.15.157:0/3022756073 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:03:07.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.011+0000 7fb4d8f49700 1 -- 172.21.15.157:0/3022756073 >> v1:172.21.15.175:6789/0 conn(0x7fb4d40b04d0 legacy=0x7fb4d41563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:07.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.011+0000 7fb4d8f49700 1 -- 172.21.15.157:0/3022756073 >> v1:172.21.15.157:6789/0 conn(0x7fb4d406c2d0 legacy=0x7fb4d4156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:07.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.011+0000 7fb4d8f49700 1 -- 172.21.15.157:0/3022756073 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d41571d0 con 0x7fb4d40b1e40 2023-10-06T04:03:07.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.011+0000 7fb4cb7fe700 1 -- 172.21.15.157:0/3022756073 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2117542526 0 0) 0x7fb4d406d8d0 con 0x7fb4d406c2d0 2023-10-06T04:03:07.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.011+0000 7fb4d8f49700 1 --2- 172.21.15.157:0/3022756073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4d40b1e40 0x7fb4d4157750 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb4d00098e0 tx=0x7fb4d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5f339816eebff2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:07.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.011+0000 7fb4cb7fe700 1 -- 172.21.15.157:0/3022756073 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0009e80 con 0x7fb4d40b1e40 2023-10-06T04:03:07.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.011+0000 7fb4cb7fe700 1 -- 172.21.15.157:0/3022756073 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d000b860 con 0x7fb4d40b1e40 2023-10-06T04:03:07.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.011+0000 7fb4cb7fe700 1 -- 172.21.15.157:0/3022756073 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d000bbe0 con 0x7fb4d40b1e40 2023-10-06T04:03:07.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.012+0000 7fb4d9f4b700 1 -- 172.21.15.157:0/3022756073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4d40b1e40 msgr2=0x7fb4d4157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:07.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.012+0000 7fb4d9f4b700 1 --2- 172.21.15.157:0/3022756073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4d40b1e40 0x7fb4d4157750 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb4d00098e0 tx=0x7fb4d00091a0 comp rx=0 tx=0).stop 2023-10-06T04:03:07.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.012+0000 7fb4d9f4b700 1 -- 172.21.15.157:0/3022756073 shutdown_connections 2023-10-06T04:03:07.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.012+0000 7fb4d9f4b700 1 --2- 172.21.15.157:0/3022756073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4d40b1e40 0x7fb4d4157750 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:07.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.012+0000 7fb4d9f4b700 1 -- 172.21.15.157:0/3022756073 >> 172.21.15.157:0/3022756073 conn(0x7fb4d40cc540 msgr2=0x7fb4d40cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:07.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.012+0000 7fb4d9f4b700 1 -- 172.21.15.157:0/3022756073 shutdown_connections 2023-10-06T04:03:07.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.012+0000 7fb4d9f4b700 1 -- 172.21.15.157:0/3022756073 wait complete. 2023-10-06T04:03:07.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.013+0000 7fb4d9f4b700 1 Processor -- start 2023-10-06T04:03:07.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.013+0000 7fb4d9f4b700 1 -- start start 2023-10-06T04:03:07.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.014+0000 7fb4d9f4b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4d406c2d0 0x7fb4d4167370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:07.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.014+0000 7fb4d9f4b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4d40b04d0 0x7fb4d41669b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:07.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.014+0000 7fb4d9f4b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4d40b1e40 0x7fb4d40c2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:07.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.014+0000 7fb4d9f4b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d40ab7f0 con 0x7fb4d40b1e40 2023-10-06T04:03:07.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.014+0000 7fb4d9f4b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d416a370 con 0x7fb4d40b04d0 2023-10-06T04:03:07.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.014+0000 7fb4d9f4b700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d4166520 con 0x7fb4d406c2d0 2023-10-06T04:03:07.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.014+0000 7fb4d974a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4d40b1e40 0x7fb4d40c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:07.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.014+0000 7fb4d974a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4d40b1e40 0x7fb4d40c2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47852/0 (socket says 172.21.15.157:47852) 2023-10-06T04:03:07.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.014+0000 7fb4d974a700 1 -- 172.21.15.157:0/2988349683 learned_addr learned my addr 172.21.15.157:0/2988349683 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:07.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.014+0000 7fb4d8f49700 1 --2- 172.21.15.157:0/2988349683 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4d406c2d0 0x7fb4d4167370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:07.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.014+0000 7fb4cbfff700 1 --2- 172.21.15.157:0/2988349683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4d40b04d0 0x7fb4d41669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:07.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.014+0000 7fb4d974a700 1 -- 172.21.15.157:0/2988349683 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4d406c2d0 msgr2=0x7fb4d4167370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:07.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.014+0000 7fb4d974a700 1 --2- 172.21.15.157:0/2988349683 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4d406c2d0 0x7fb4d4167370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:07.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.014+0000 7fb4d974a700 1 -- 172.21.15.157:0/2988349683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4d40b04d0 msgr2=0x7fb4d41669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:07.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.014+0000 7fb4d974a700 1 --2- 172.21.15.157:0/2988349683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4d40b04d0 0x7fb4d41669b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:07.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.014+0000 7fb4d974a700 1 -- 172.21.15.157:0/2988349683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d0009420 con 0x7fb4d40b1e40 2023-10-06T04:03:07.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.014+0000 7fb4d974a700 1 --2- 172.21.15.157:0/2988349683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4d40b1e40 0x7fb4d40c2110 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00ccc0 tx=0x7fb4cc001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:07.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.014+0000 7fb4c8ff9700 1 -- 172.21.15.157:0/2988349683 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc00a800 con 0x7fb4d40b1e40 2023-10-06T04:03:07.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.015+0000 7fb4c8ff9700 1 -- 172.21.15.157:0/2988349683 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4cc00a960 con 0x7fb4d40b1e40 2023-10-06T04:03:07.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.015+0000 7fb4d9f4b700 1 -- 172.21.15.157:0/2988349683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d40c4fc0 con 0x7fb4d40b1e40 2023-10-06T04:03:07.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.015+0000 7fb4c8ff9700 1 -- 172.21.15.157:0/2988349683 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc00ad20 con 0x7fb4d40b1e40 2023-10-06T04:03:07.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.015+0000 7fb4d9f4b700 1 -- 172.21.15.157:0/2988349683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d40c5260 con 0x7fb4d40b1e40 2023-10-06T04:03:07.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.016+0000 7fb4d9f4b700 1 -- 172.21.15.157:0/2988349683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4d4088f90 con 0x7fb4d40b1e40 2023-10-06T04:03:07.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.016+0000 7fb4c8ff9700 1 -- 172.21.15.157:0/2988349683 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb4cc007300 con 0x7fb4d40b1e40 2023-10-06T04:03:07.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.016+0000 7fb4c8ff9700 1 --2- 172.21.15.157:0/2988349683 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb4c0066f60 0x7fb4c0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:07.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.016+0000 7fb4c8ff9700 1 -- 172.21.15.157:0/2988349683 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7fb4cc084b70 con 0x7fb4d40b1e40 2023-10-06T04:03:07.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.016+0000 7fb4d8f49700 1 --2- 172.21.15.157:0/2988349683 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb4c0066f60 0x7fb4c0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:07.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.017+0000 7fb4d8f49700 1 --2- 172.21.15.157:0/2988349683 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb4c0066f60 0x7fb4c0069410 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb4d00093f0 tx=0x7fb4d0017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:07.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.021+0000 7fb4c8ff9700 1 -- 172.21.15.157:0/2988349683 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4cc051ab0 con 0x7fb4d40b1e40 2023-10-06T04:03:07.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.245+0000 7fb4d9f4b700 1 -- 172.21.15.157:0/2988349683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4d4166660 con 0x7fb4d40b1e40 2023-10-06T04:03:07.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.245+0000 7fb4c8ff9700 1 -- 172.21.15.157:0/2988349683 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+6160 (secure 0 0 0) 0x7fb4cc0550d0 con 0x7fb4d40b1e40 2023-10-06T04:03:07.247 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:03:07.247 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3192541484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3192541484},{"type":"v1","addr":"172.21.15.175:6835","nonce":3192541484}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:01.994829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279},"failed":[],"damaged":[],"stopped":[3,4],"info":{"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:stopping","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:03:07.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.247+0000 7fb4d9f4b700 1 -- 172.21.15.157:0/2988349683 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb4c0066f60 msgr2=0x7fb4c0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:07.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.247+0000 7fb4d9f4b700 1 --2- 172.21.15.157:0/2988349683 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb4c0066f60 0x7fb4c0069410 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb4d00093f0 tx=0x7fb4d0017000 comp rx=0 tx=0).stop 2023-10-06T04:03:07.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.247+0000 7fb4d9f4b700 1 -- 172.21.15.157:0/2988349683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4d40b1e40 msgr2=0x7fb4d40c2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:07.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.247+0000 7fb4d9f4b700 1 --2- 172.21.15.157:0/2988349683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4d40b1e40 0x7fb4d40c2110 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00ccc0 tx=0x7fb4cc001a20 comp rx=0 tx=0).stop 2023-10-06T04:03:07.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.247+0000 7fb4d9f4b700 1 -- 172.21.15.157:0/2988349683 shutdown_connections 2023-10-06T04:03:07.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.247+0000 7fb4d9f4b700 1 --2- 172.21.15.157:0/2988349683 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb4c0066f60 0x7fb4c0069410 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:07.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.247+0000 7fb4d9f4b700 1 --2- 172.21.15.157:0/2988349683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4d40b1e40 0x7fb4d40c2110 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:07.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.247+0000 7fb4d9f4b700 1 --2- 172.21.15.157:0/2988349683 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4d406c2d0 0x7fb4d4167370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:07.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.247+0000 7fb4d9f4b700 1 --2- 172.21.15.157:0/2988349683 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4d40b04d0 0x7fb4d41669b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:07.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.247+0000 7fb4d9f4b700 1 -- 172.21.15.157:0/2988349683 >> 172.21.15.157:0/2988349683 conn(0x7fb4d40cc540 msgr2=0x7fb4d4157440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:07.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.248+0000 7fb4d9f4b700 1 -- 172.21.15.157:0/2988349683 shutdown_connections 2023-10-06T04:03:07.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:07.248+0000 7fb4d9f4b700 1 -- 172.21.15.157:0/2988349683 wait complete. 2023-10-06T04:03:07.250 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 49 2023-10-06T04:03:07.261 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:03:09.262 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:03:09.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.377+0000 7f6eb3a9f700 1 Processor -- start 2023-10-06T04:03:09.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.377+0000 7f6eb3a9f700 1 -- start start 2023-10-06T04:03:09.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.378+0000 7f6eb3a9f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6eac0b2110 0x7f6eac159df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:09.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.378+0000 7f6eb3a9f700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eac06ced0 con 0x7f6eac06c1f0 2023-10-06T04:03:09.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.378+0000 7f6eb3a9f700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eac0afb10 con 0x7f6eac0b1d40 2023-10-06T04:03:09.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.378+0000 7f6eb3a9f700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac06cc10 con 0x7f6eac0b2110 2023-10-06T04:03:09.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.378+0000 7f6eb203c700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f6eac06c1f0 0x7f6eac14fbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49048/0 (socket says 172.21.15.157:49048) 2023-10-06T04:03:09.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.378+0000 7f6eb203c700 1 -- 172.21.15.157:0/849459333 learned_addr learned my addr 172.21.15.157:0/849459333 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:03:09.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.378+0000 7f6eb103a700 1 --2- 172.21.15.157:0/849459333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6eac0b2110 0x7f6eac159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:09.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.378+0000 7f6eb0839700 1 -- 172.21.15.157:0/849459333 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3547236659 0 0) 0x7f6eac06ced0 con 0x7f6eac06c1f0 2023-10-06T04:03:09.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.378+0000 7f6eb0839700 1 -- 172.21.15.157:0/849459333 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e98003570 con 0x7f6eac06c1f0 2023-10-06T04:03:09.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.379+0000 7f6eb0839700 1 -- 172.21.15.157:0/849459333 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 703362989 0 0) 0x7f6e98003570 con 0x7f6eac06c1f0 2023-10-06T04:03:09.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.379+0000 7f6eb0839700 1 -- 172.21.15.157:0/849459333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6eac0b2110 msgr2=0x7f6eac159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:09.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.379+0000 7f6eb0839700 1 --2- 172.21.15.157:0/849459333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6eac0b2110 0x7f6eac159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:09.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.379+0000 7f6eb0839700 1 -- 172.21.15.157:0/849459333 >> v1:172.21.15.175:6789/0 conn(0x7f6eac0b1d40 legacy=0x7f6eac0af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:09.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.379+0000 7f6eb0839700 1 -- 172.21.15.157:0/849459333 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eac06e570 con 0x7f6eac06c1f0 2023-10-06T04:03:09.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.379+0000 7f6eb0839700 1 -- 172.21.15.157:0/849459333 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f6ea40017f0 con 0x7f6eac06c1f0 2023-10-06T04:03:09.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.379+0000 7f6eb0839700 1 -- 172.21.15.157:0/849459333 >> v1:172.21.15.157:6789/0 conn(0x7f6eac06c1f0 legacy=0x7f6eac14fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:09.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.379+0000 7f6eb0839700 1 --2- 172.21.15.157:0/849459333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e98003a90 0x7f6e98003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:09.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.379+0000 7f6eb0839700 1 --2- 172.21.15.157:0/849459333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e98004810 0x7f6e98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:09.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.379+0000 7f6eb0839700 1 --2- 172.21.15.157:0/849459333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6eac0b2110 0x7f6e980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:09.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.379+0000 7f6eb0839700 1 -- 172.21.15.157:0/849459333 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac06cc10 con 0x7f6e98003a90 2023-10-06T04:03:09.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.379+0000 7f6eb0839700 1 -- 172.21.15.157:0/849459333 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9800bb70 con 0x7f6e98004810 2023-10-06T04:03:09.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.379+0000 7f6eb0839700 1 -- 172.21.15.157:0/849459333 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9800bd30 con 0x7f6eac0b2110 2023-10-06T04:03:09.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.379+0000 7f6eb183b700 1 --2- 172.21.15.157:0/849459333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e98003a90 0x7f6e98003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:09.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.379+0000 7f6eb103a700 1 --2- 172.21.15.157:0/849459333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e98004810 0x7f6e98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:09.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.379+0000 7f6eb203c700 1 --2- 172.21.15.157:0/849459333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6eac0b2110 0x7f6e980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:09.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.379+0000 7f6eb183b700 1 -- 172.21.15.157:0/849459333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6eac0b2110 msgr2=0x7f6e980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:09.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.379+0000 7f6eb183b700 1 --2- 172.21.15.157:0/849459333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6eac0b2110 0x7f6e980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:09.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.379+0000 7f6eb183b700 1 -- 172.21.15.157:0/849459333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e98004810 msgr2=0x7f6e98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:09.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.379+0000 7f6eb183b700 1 --2- 172.21.15.157:0/849459333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e98004810 0x7f6e98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:09.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.379+0000 7f6eb183b700 1 -- 172.21.15.157:0/849459333 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e98003570 con 0x7f6e98003a90 2023-10-06T04:03:09.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.379+0000 7f6eb183b700 1 --2- 172.21.15.157:0/849459333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e98003a90 0x7f6e98003e60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6ea80019a0 tx=0x7f6ea8007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98bae31f48412dd0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:09.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.379+0000 7f6eb0839700 1 -- 172.21.15.157:0/849459333 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea800c5d0 con 0x7f6e98003a90 2023-10-06T04:03:09.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.380+0000 7f6eb0839700 1 -- 172.21.15.157:0/849459333 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea800c730 con 0x7f6e98003a90 2023-10-06T04:03:09.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.380+0000 7f6eb0839700 1 -- 172.21.15.157:0/849459333 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea800cad0 con 0x7f6e98003a90 2023-10-06T04:03:09.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.380+0000 7f6eb3a9f700 1 -- 172.21.15.157:0/849459333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e98003a90 msgr2=0x7f6e98003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:09.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.380+0000 7f6eb3a9f700 1 --2- 172.21.15.157:0/849459333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e98003a90 0x7f6e98003e60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6ea80019a0 tx=0x7f6ea8007d10 comp rx=0 tx=0).stop 2023-10-06T04:03:09.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.380+0000 7f6eb203c700 1 -- 172.21.15.157:0/849459333 reap_dead start 2023-10-06T04:03:09.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.380+0000 7f6eb3a9f700 1 -- 172.21.15.157:0/849459333 shutdown_connections 2023-10-06T04:03:09.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.380+0000 7f6eb3a9f700 1 -- 172.21.15.157:0/849459333 >> 172.21.15.157:0/849459333 conn(0x7f6eac0cc6e0 msgr2=0x7f6eac0ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:09.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.380+0000 7f6eb3a9f700 1 -- 172.21.15.157:0/849459333 shutdown_connections 2023-10-06T04:03:09.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.380+0000 7f6eb3a9f700 1 -- 172.21.15.157:0/849459333 wait complete. 2023-10-06T04:03:09.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.381+0000 7f6eb3a9f700 1 Processor -- start 2023-10-06T04:03:09.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.381+0000 7f6eb3a9f700 1 -- start start 2023-10-06T04:03:09.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.382+0000 7f6eb3a9f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e98003a90 0x7f6eac1669a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:09.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.382+0000 7f6eb3a9f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6eac151830 0x7f6eac157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:09.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.382+0000 7f6eb3a9f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6eac0c6510 0x7f6eac0c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:09.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.382+0000 7f6eb3a9f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac166ee0 con 0x7f6eac0c6510 2023-10-06T04:03:09.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.382+0000 7f6eb3a9f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac1699b0 con 0x7f6e98003a90 2023-10-06T04:03:09.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.382+0000 7f6eb3a9f700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac165c70 con 0x7f6eac151830 2023-10-06T04:03:09.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.382+0000 7f6eb203c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6eac0c6510 0x7f6eac0c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:09.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.382+0000 7f6eb203c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6eac0c6510 0x7f6eac0c94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47860/0 (socket says 172.21.15.157:47860) 2023-10-06T04:03:09.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.382+0000 7f6eb103a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6eac151830 0x7f6eac157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:09.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.382+0000 7f6eb203c700 1 -- 172.21.15.157:0/959903294 learned_addr learned my addr 172.21.15.157:0/959903294 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:09.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.382+0000 7f6eb183b700 1 --2- 172.21.15.157:0/959903294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e98003a90 0x7f6eac1669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:09.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.382+0000 7f6eb203c700 1 -- 172.21.15.157:0/959903294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6eac151830 msgr2=0x7f6eac157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:09.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.382+0000 7f6eb203c700 1 --2- 172.21.15.157:0/959903294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6eac151830 0x7f6eac157700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:09.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.382+0000 7f6eb203c700 1 -- 172.21.15.157:0/959903294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e98003a90 msgr2=0x7f6eac1669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:09.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.382+0000 7f6eb203c700 1 --2- 172.21.15.157:0/959903294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e98003a90 0x7f6eac1669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:09.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.382+0000 7f6eb203c700 1 -- 172.21.15.157:0/959903294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eac169710 con 0x7f6eac0c6510 2023-10-06T04:03:09.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.383+0000 7f6eb203c700 1 --2- 172.21.15.157:0/959903294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6eac0c6510 0x7f6eac0c94b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6eac064460 tx=0x7f6ea4002120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:09.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.383+0000 7f6ea1ffb700 1 -- 172.21.15.157:0/959903294 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea400ace0 con 0x7f6eac0c6510 2023-10-06T04:03:09.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.383+0000 7f6ea1ffb700 1 -- 172.21.15.157:0/959903294 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea400ae60 con 0x7f6eac0c6510 2023-10-06T04:03:09.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.383+0000 7f6eb3a9f700 1 -- 172.21.15.157:0/959903294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eac165620 con 0x7f6eac0c6510 2023-10-06T04:03:09.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.383+0000 7f6ea1ffb700 1 -- 172.21.15.157:0/959903294 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4013640 con 0x7f6eac0c6510 2023-10-06T04:03:09.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.383+0000 7f6eb3a9f700 1 -- 172.21.15.157:0/959903294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eac1693f0 con 0x7f6eac0c6510 2023-10-06T04:03:09.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.384+0000 7f6eb3a9f700 1 -- 172.21.15.157:0/959903294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eac165ac0 con 0x7f6eac0c6510 2023-10-06T04:03:09.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.384+0000 7f6ea1ffb700 1 -- 172.21.15.157:0/959903294 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ea4013a70 con 0x7f6eac0c6510 2023-10-06T04:03:09.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.384+0000 7f6ea1ffb700 1 --2- 172.21.15.157:0/959903294 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6e90066f30 0x7f6e900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:09.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.385+0000 7f6ea1ffb700 1 -- 172.21.15.157:0/959903294 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7f6ea4085220 con 0x7f6eac0c6510 2023-10-06T04:03:09.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.385+0000 7f6eb183b700 1 --2- 172.21.15.157:0/959903294 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6e90066f30 0x7f6e900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:09.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.385+0000 7f6eb183b700 1 --2- 172.21.15.157:0/959903294 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6e90066f30 0x7f6e900693e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8008fd0 tx=0x7f6ea8015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:09.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.389+0000 7f6ea1ffb700 1 -- 172.21.15.157:0/959903294 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ea40520e0 con 0x7f6eac0c6510 2023-10-06T04:03:09.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.614+0000 7f6eb3a9f700 1 -- 172.21.15.157:0/959903294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6eac165db0 con 0x7f6eac0c6510 2023-10-06T04:03:09.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.614+0000 7f6ea1ffb700 1 -- 172.21.15.157:0/959903294 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+6162 (secure 0 0 0) 0x7f6ea4055700 con 0x7f6eac0c6510 2023-10-06T04:03:09.616 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:03:09.616 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3192541484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3192541484},{"type":"v1","addr":"172.21.15.175:6835","nonce":3192541484}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:08.700610+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279},"failed":[],"damaged":[],"stopped":[3,4],"info":{"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:stopping","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:03:09.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.616+0000 7f6eb3a9f700 1 -- 172.21.15.157:0/959903294 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6e90066f30 msgr2=0x7f6e900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:09.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.616+0000 7f6eb3a9f700 1 --2- 172.21.15.157:0/959903294 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6e90066f30 0x7f6e900693e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8008fd0 tx=0x7f6ea8015000 comp rx=0 tx=0).stop 2023-10-06T04:03:09.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.616+0000 7f6eb3a9f700 1 -- 172.21.15.157:0/959903294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6eac0c6510 msgr2=0x7f6eac0c94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:09.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.616+0000 7f6eb3a9f700 1 --2- 172.21.15.157:0/959903294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6eac0c6510 0x7f6eac0c94b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6eac064460 tx=0x7f6ea4002120 comp rx=0 tx=0).stop 2023-10-06T04:03:09.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.616+0000 7f6eb3a9f700 1 -- 172.21.15.157:0/959903294 shutdown_connections 2023-10-06T04:03:09.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.616+0000 7f6eb3a9f700 1 --2- 172.21.15.157:0/959903294 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6e90066f30 0x7f6e900693e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:09.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.616+0000 7f6eb3a9f700 1 --2- 172.21.15.157:0/959903294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e98003a90 0x7f6eac1669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:09.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.616+0000 7f6eb3a9f700 1 --2- 172.21.15.157:0/959903294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6eac0c6510 0x7f6eac0c94b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:09.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.616+0000 7f6eb3a9f700 1 --2- 172.21.15.157:0/959903294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6eac151830 0x7f6eac157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:09.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.616+0000 7f6eb3a9f700 1 -- 172.21.15.157:0/959903294 >> 172.21.15.157:0/959903294 conn(0x7f6eac0cc6e0 msgr2=0x7f6eac152fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:09.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.616+0000 7f6eb3a9f700 1 -- 172.21.15.157:0/959903294 shutdown_connections 2023-10-06T04:03:09.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:09.616+0000 7f6eb3a9f700 1 -- 172.21.15.157:0/959903294 wait complete. 2023-10-06T04:03:09.620 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 50 2023-10-06T04:03:09.630 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:03:11.630 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:03:11.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.746+0000 7f15da8e3700 1 Processor -- start 2023-10-06T04:03:11.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.746+0000 7f15da8e3700 1 -- start start 2023-10-06T04:03:11.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.747+0000 7f15da8e3700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15d40b2110 0x7f15d4159df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:11.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.747+0000 7f15da8e3700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f15d406c1f0 0x7f15d414fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:11.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.747+0000 7f15da8e3700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d406ced0 con 0x7f15d40b1d40 2023-10-06T04:03:11.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.747+0000 7f15da8e3700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d40afb10 con 0x7f15d406c1f0 2023-10-06T04:03:11.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.747+0000 7f15da8e3700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d406cc10 con 0x7f15d40b2110 2023-10-06T04:03:11.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.747+0000 7f15d3fff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f15d40b1d40 0x7f15d40af400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49052/0 (socket says 172.21.15.157:49052) 2023-10-06T04:03:11.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.747+0000 7f15d3fff700 1 -- 172.21.15.157:0/140987743 learned_addr learned my addr 172.21.15.157:0/140987743 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:03:11.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.747+0000 7f15d37fe700 1 --2- 172.21.15.157:0/140987743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15d40b2110 0x7f15d4159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:11.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.747+0000 7f15d8e80700 1 --2- 172.21.15.157:0/140987743 >> v2:172.21.15.175:3300/0 conn(0x7f15d406c1f0 0x7f15d414fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:11.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.748+0000 7f15d2ffd700 1 -- 172.21.15.157:0/140987743 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2232809991 0 0) 0x7f15d406ced0 con 0x7f15d40b1d40 2023-10-06T04:03:11.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.748+0000 7f15d2ffd700 1 -- 172.21.15.157:0/140987743 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15bc003570 con 0x7f15d40b1d40 2023-10-06T04:03:11.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.748+0000 7f15d2ffd700 1 -- 172.21.15.157:0/140987743 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2655492575 0 0) 0x7f15bc003570 con 0x7f15d40b1d40 2023-10-06T04:03:11.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.748+0000 7f15d2ffd700 1 -- 172.21.15.157:0/140987743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15d40b2110 msgr2=0x7f15d4159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:11.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.748+0000 7f15d2ffd700 1 --2- 172.21.15.157:0/140987743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15d40b2110 0x7f15d4159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:11.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.748+0000 7f15d2ffd700 1 -- 172.21.15.157:0/140987743 >> v2:172.21.15.175:3300/0 conn(0x7f15d406c1f0 msgr2=0x7f15d414fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:11.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.748+0000 7f15d2ffd700 1 --2- 172.21.15.157:0/140987743 >> v2:172.21.15.175:3300/0 conn(0x7f15d406c1f0 0x7f15d414fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:11.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.748+0000 7f15d2ffd700 1 -- 172.21.15.157:0/140987743 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d406e570 con 0x7f15d40b1d40 2023-10-06T04:03:11.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.748+0000 7f15d2ffd700 1 -- 172.21.15.157:0/140987743 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f15c4001470 con 0x7f15d40b1d40 2023-10-06T04:03:11.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.748+0000 7f15d2ffd700 1 -- 172.21.15.157:0/140987743 >> v1:172.21.15.157:6789/0 conn(0x7f15d40b1d40 legacy=0x7f15d40af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:11.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.748+0000 7f15d2ffd700 1 --2- 172.21.15.157:0/140987743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15bc003a60 0x7f15bc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:11.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.748+0000 7f15d2ffd700 1 --2- 172.21.15.157:0/140987743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15bc004810 0x7f15bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:11.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.748+0000 7f15d2ffd700 1 --2- 172.21.15.157:0/140987743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15d40b2110 0x7f15bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:11.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.748+0000 7f15d2ffd700 1 -- 172.21.15.157:0/140987743 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d40afb10 con 0x7f15bc003a60 2023-10-06T04:03:11.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.748+0000 7f15d2ffd700 1 -- 172.21.15.157:0/140987743 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d406cc10 con 0x7f15bc004810 2023-10-06T04:03:11.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.748+0000 7f15d2ffd700 1 -- 172.21.15.157:0/140987743 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc00ba60 con 0x7f15d40b2110 2023-10-06T04:03:11.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.748+0000 7f15d3fff700 1 --2- 172.21.15.157:0/140987743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15bc003a60 0x7f15bc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:11.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.748+0000 7f15d37fe700 1 --2- 172.21.15.157:0/140987743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15bc004810 0x7f15bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:11.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.748+0000 7f15d8e80700 1 --2- 172.21.15.157:0/140987743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15d40b2110 0x7f15bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:11.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.748+0000 7f15d3fff700 1 -- 172.21.15.157:0/140987743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15d40b2110 msgr2=0x7f15bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:11.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.748+0000 7f15d3fff700 1 --2- 172.21.15.157:0/140987743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15d40b2110 0x7f15bc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:11.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.748+0000 7f15d3fff700 1 -- 172.21.15.157:0/140987743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15bc004810 msgr2=0x7f15bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:11.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.748+0000 7f15d3fff700 1 --2- 172.21.15.157:0/140987743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15bc004810 0x7f15bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:11.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.748+0000 7f15d3fff700 1 -- 172.21.15.157:0/140987743 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15bc003570 con 0x7f15bc003a60 2023-10-06T04:03:11.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.748+0000 7f15d8e80700 1 --2- 172.21.15.157:0/140987743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15d40b2110 0x7f15bc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:03:11.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.748+0000 7f15d3fff700 1 --2- 172.21.15.157:0/140987743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15bc003a60 0x7f15bc003e70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f15c400c3f0 tx=0x7f15c4001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e459bdb3ee076cc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:11.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.749+0000 7f15d2ffd700 1 -- 172.21.15.157:0/140987743 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c400cc80 con 0x7f15bc003a60 2023-10-06T04:03:11.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.749+0000 7f15d2ffd700 1 -- 172.21.15.157:0/140987743 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c400ce00 con 0x7f15bc003a60 2023-10-06T04:03:11.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.749+0000 7f15d2ffd700 1 -- 172.21.15.157:0/140987743 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c4005b70 con 0x7f15bc003a60 2023-10-06T04:03:11.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.749+0000 7f15da8e3700 1 -- 172.21.15.157:0/140987743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15bc003a60 msgr2=0x7f15bc003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:11.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.749+0000 7f15da8e3700 1 --2- 172.21.15.157:0/140987743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15bc003a60 0x7f15bc003e70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f15c400c3f0 tx=0x7f15c4001e90 comp rx=0 tx=0).stop 2023-10-06T04:03:11.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.749+0000 7f15d8e80700 1 -- 172.21.15.157:0/140987743 reap_dead start 2023-10-06T04:03:11.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.749+0000 7f15da8e3700 1 -- 172.21.15.157:0/140987743 shutdown_connections 2023-10-06T04:03:11.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.749+0000 7f15da8e3700 1 -- 172.21.15.157:0/140987743 >> 172.21.15.157:0/140987743 conn(0x7f15d40cc6e0 msgr2=0x7f15d40ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:11.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.749+0000 7f15da8e3700 1 -- 172.21.15.157:0/140987743 shutdown_connections 2023-10-06T04:03:11.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.749+0000 7f15da8e3700 1 -- 172.21.15.157:0/140987743 wait complete. 2023-10-06T04:03:11.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.750+0000 7f15da8e3700 1 Processor -- start 2023-10-06T04:03:11.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.751+0000 7f15da8e3700 1 -- start start 2023-10-06T04:03:11.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.751+0000 7f15da8e3700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15bc003a60 0x7f15d4166250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:11.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.751+0000 7f15da8e3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15d40c63b0 0x7f15d40c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:11.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.751+0000 7f15da8e3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15d4156050 0x7f15d41564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:11.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.751+0000 7f15da8e3700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d40c9ef0 con 0x7f15d4156050 2023-10-06T04:03:11.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.751+0000 7f15da8e3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d40c6c20 con 0x7f15d40c63b0 2023-10-06T04:03:11.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.751+0000 7f15da8e3700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d40c4750 con 0x7f15bc003a60 2023-10-06T04:03:11.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.751+0000 7f15d8e80700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15d4156050 0x7f15d41564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:11.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.751+0000 7f15d8e80700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15d4156050 0x7f15d41564c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47876/0 (socket says 172.21.15.157:47876) 2023-10-06T04:03:11.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.751+0000 7f15d8e80700 1 -- 172.21.15.157:0/423137893 learned_addr learned my addr 172.21.15.157:0/423137893 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:11.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.751+0000 7f15d3fff700 1 --2- 172.21.15.157:0/423137893 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15bc003a60 0x7f15d4166250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:11.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.751+0000 7f15d37fe700 1 --2- 172.21.15.157:0/423137893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15d40c63b0 0x7f15d40c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:11.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.752+0000 7f15d8e80700 1 -- 172.21.15.157:0/423137893 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15bc003a60 msgr2=0x7f15d4166250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:11.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.752+0000 7f15d8e80700 1 --2- 172.21.15.157:0/423137893 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15bc003a60 0x7f15d4166250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:11.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.752+0000 7f15d8e80700 1 -- 172.21.15.157:0/423137893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15d40c63b0 msgr2=0x7f15d40c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:11.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.752+0000 7f15d8e80700 1 --2- 172.21.15.157:0/423137893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15d40c63b0 0x7f15d40c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:11.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.752+0000 7f15d8e80700 1 -- 172.21.15.157:0/423137893 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d40c49c0 con 0x7f15d4156050 2023-10-06T04:03:11.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.752+0000 7f15d37fe700 1 --2- 172.21.15.157:0/423137893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15d40c63b0 0x7f15d40c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:03:11.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.752+0000 7f15d8e80700 1 --2- 172.21.15.157:0/423137893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15d4156050 0x7f15d41564c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f15cc009f80 tx=0x7f15cc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:11.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.752+0000 7f15bbfff700 1 -- 172.21.15.157:0/423137893 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc01b070 con 0x7f15d4156050 2023-10-06T04:03:11.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.752+0000 7f15bbfff700 1 -- 172.21.15.157:0/423137893 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15cc00f040 con 0x7f15d4156050 2023-10-06T04:03:11.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.752+0000 7f15bbfff700 1 -- 172.21.15.157:0/423137893 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc00e050 con 0x7f15d4156050 2023-10-06T04:03:11.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.752+0000 7f15da8e3700 1 -- 172.21.15.157:0/423137893 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d40c6e70 con 0x7f15d4156050 2023-10-06T04:03:11.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.752+0000 7f15da8e3700 1 -- 172.21.15.157:0/423137893 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d4153590 con 0x7f15d4156050 2023-10-06T04:03:11.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.753+0000 7f15da8e3700 1 -- 172.21.15.157:0/423137893 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d40896c0 con 0x7f15d4156050 2023-10-06T04:03:11.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.753+0000 7f15bbfff700 1 -- 172.21.15.157:0/423137893 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f15cc003020 con 0x7f15d4156050 2023-10-06T04:03:11.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.754+0000 7f15bbfff700 1 --2- 172.21.15.157:0/423137893 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f15c0066f30 0x7f15c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:11.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.754+0000 7f15bbfff700 1 -- 172.21.15.157:0/423137893 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7f15cc084d40 con 0x7f15d4156050 2023-10-06T04:03:11.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.754+0000 7f15d3fff700 1 --2- 172.21.15.157:0/423137893 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f15c0066f30 0x7f15c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:11.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.754+0000 7f15d3fff700 1 --2- 172.21.15.157:0/423137893 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f15c0066f30 0x7f15c00693e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f15c4001ed0 tx=0x7f15c4015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:11.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.758+0000 7f15bbfff700 1 -- 172.21.15.157:0/423137893 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f15cc051c00 con 0x7f15d4156050 2023-10-06T04:03:11.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.981+0000 7f15da8e3700 1 -- 172.21.15.157:0/423137893 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15d406d7b0 con 0x7f15d4156050 2023-10-06T04:03:11.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.982+0000 7f15bbfff700 1 -- 172.21.15.157:0/423137893 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+6162 (secure 0 0 0) 0x7f15cc010ea0 con 0x7f15d4156050 2023-10-06T04:03:11.984 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:03:11.984 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3192541484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3192541484},{"type":"v1","addr":"172.21.15.175:6835","nonce":3192541484}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:08.700610+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279},"failed":[],"damaged":[],"stopped":[3,4],"info":{"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:stopping","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,3],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:03:11.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.983+0000 7f15da8e3700 1 -- 172.21.15.157:0/423137893 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f15c0066f30 msgr2=0x7f15c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:11.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.984+0000 7f15da8e3700 1 --2- 172.21.15.157:0/423137893 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f15c0066f30 0x7f15c00693e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f15c4001ed0 tx=0x7f15c4015000 comp rx=0 tx=0).stop 2023-10-06T04:03:11.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.984+0000 7f15da8e3700 1 -- 172.21.15.157:0/423137893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15d4156050 msgr2=0x7f15d41564c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:11.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.984+0000 7f15da8e3700 1 --2- 172.21.15.157:0/423137893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15d4156050 0x7f15d41564c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f15cc009f80 tx=0x7f15cc00b040 comp rx=0 tx=0).stop 2023-10-06T04:03:11.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.984+0000 7f15da8e3700 1 -- 172.21.15.157:0/423137893 shutdown_connections 2023-10-06T04:03:11.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.984+0000 7f15da8e3700 1 --2- 172.21.15.157:0/423137893 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f15c0066f30 0x7f15c00693e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:11.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.984+0000 7f15da8e3700 1 --2- 172.21.15.157:0/423137893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15d4156050 0x7f15d41564c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:11.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.984+0000 7f15da8e3700 1 --2- 172.21.15.157:0/423137893 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15bc003a60 0x7f15d4166250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:11.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.984+0000 7f15da8e3700 1 --2- 172.21.15.157:0/423137893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15d40c63b0 0x7f15d40c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:11.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.984+0000 7f15da8e3700 1 -- 172.21.15.157:0/423137893 >> 172.21.15.157:0/423137893 conn(0x7f15d40cc6e0 msgr2=0x7f15d40ca4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:11.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.984+0000 7f15da8e3700 1 -- 172.21.15.157:0/423137893 shutdown_connections 2023-10-06T04:03:11.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:11.984+0000 7f15da8e3700 1 -- 172.21.15.157:0/423137893 wait complete. 2023-10-06T04:03:11.987 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 50 2023-10-06T04:03:11.997 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:03:13.998 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:03:14.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.122+0000 7fed71a09700 1 Processor -- start 2023-10-06T04:03:14.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.122+0000 7fed71a09700 1 -- start start 2023-10-06T04:03:14.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.122+0000 7fed71a09700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed6c06cbb0 0x7fed6c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:14.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.123+0000 7fed71a09700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fed6c150410 0x7fed6c06da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:14.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.123+0000 7fed71a09700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed6c0c9780 con 0x7fed6c06c1f0 2023-10-06T04:03:14.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.123+0000 7fed71a09700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed6c0c8c40 con 0x7fed6c150410 2023-10-06T04:03:14.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.123+0000 7fed71a09700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c06e0c0 con 0x7fed6c06cbb0 2023-10-06T04:03:14.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.123+0000 7fed6b7fe700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fed6c150410 0x7fed6c06da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:14.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.123+0000 7fed6affd700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed6c06cbb0 0x7fed6c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:14.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.123+0000 7fed6b7fe700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fed6c150410 0x7fed6c06da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:32866/0 (socket says 172.21.15.157:32866) 2023-10-06T04:03:14.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.123+0000 7fed6affd700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed6c06cbb0 0x7fed6c06d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:54684/0 (socket says 172.21.15.157:54684) 2023-10-06T04:03:14.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.123+0000 7fed6b7fe700 1 -- 172.21.15.157:0/276908395 learned_addr learned my addr 172.21.15.157:0/276908395 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:14.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.123+0000 7fed6a7fc700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fed6c06c1f0 0x7fed6c14fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.157:44612/0 (socket says 172.21.15.157:44612) 2023-10-06T04:03:14.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.123+0000 7fed6affd700 1 -- 172.21.15.157:0/276908395 >> v2:172.21.15.175:3300/0 conn(0x7fed6c150410 msgr2=0x7fed6c06da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:14.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.123+0000 7fed6affd700 1 --2- 172.21.15.157:0/276908395 >> v2:172.21.15.175:3300/0 conn(0x7fed6c150410 0x7fed6c06da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:14.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.123+0000 7fed6affd700 1 -- 172.21.15.157:0/276908395 >> v1:172.21.15.175:6789/0 conn(0x7fed6c06c1f0 legacy=0x7fed6c14fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:14.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.123+0000 7fed6affd700 1 -- 172.21.15.157:0/276908395 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6c0c8e50 con 0x7fed6c06cbb0 2023-10-06T04:03:14.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.123+0000 7fed6b7fe700 1 --2- 172.21.15.157:0/276908395 >> v2:172.21.15.175:3300/0 conn(0x7fed6c150410 0x7fed6c06da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:03:14.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.123+0000 7fed6affd700 1 --2- 172.21.15.157:0/276908395 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed6c06cbb0 0x7fed6c06d530 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fed5c0098e0 tx=0x7fed5c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48878436cfd56c3f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:14.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.124+0000 7fed69ffb700 1 -- 172.21.15.157:0/276908395 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c00b6d0 con 0x7fed6c06cbb0 2023-10-06T04:03:14.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.124+0000 7fed69ffb700 1 -- 172.21.15.157:0/276908395 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5c00b830 con 0x7fed6c06cbb0 2023-10-06T04:03:14.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.124+0000 7fed69ffb700 1 -- 172.21.15.157:0/276908395 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c00bbd0 con 0x7fed6c06cbb0 2023-10-06T04:03:14.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.124+0000 7fed71a09700 1 -- 172.21.15.157:0/276908395 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed6c06cbb0 msgr2=0x7fed6c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:14.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.124+0000 7fed71a09700 1 --2- 172.21.15.157:0/276908395 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed6c06cbb0 0x7fed6c06d530 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fed5c0098e0 tx=0x7fed5c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:03:14.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.124+0000 7fed71a09700 1 -- 172.21.15.157:0/276908395 shutdown_connections 2023-10-06T04:03:14.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.124+0000 7fed71a09700 1 --2- 172.21.15.157:0/276908395 >> v2:172.21.15.175:3300/0 conn(0x7fed6c150410 0x7fed6c06da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:14.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.124+0000 7fed71a09700 1 --2- 172.21.15.157:0/276908395 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed6c06cbb0 0x7fed6c06d530 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:14.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.124+0000 7fed71a09700 1 -- 172.21.15.157:0/276908395 >> 172.21.15.157:0/276908395 conn(0x7fed6c06f230 msgr2=0x7fed6c06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:14.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.124+0000 7fed71a09700 1 -- 172.21.15.157:0/276908395 shutdown_connections 2023-10-06T04:03:14.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.124+0000 7fed71a09700 1 -- 172.21.15.157:0/276908395 wait complete. 2023-10-06T04:03:14.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.125+0000 7fed71a09700 1 Processor -- start 2023-10-06T04:03:14.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.125+0000 7fed71a09700 1 -- start start 2023-10-06T04:03:14.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.126+0000 7fed71a09700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed6c06c1f0 0x7fed6c158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:14.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.126+0000 7fed71a09700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed6c06cbb0 0x7fed6c169bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:14.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.126+0000 7fed71a09700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed6c150410 0x7fed6c16a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:14.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.126+0000 7fed71a09700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c15a190 con 0x7fed6c150410 2023-10-06T04:03:14.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.126+0000 7fed71a09700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c159c60 con 0x7fed6c06c1f0 2023-10-06T04:03:14.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.126+0000 7fed71a09700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c157ff0 con 0x7fed6c06cbb0 2023-10-06T04:03:14.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.126+0000 7fed6b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed6c150410 0x7fed6c16a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:14.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.126+0000 7fed6a7fc700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed6c06cbb0 0x7fed6c169bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:14.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.126+0000 7fed6b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed6c150410 0x7fed6c16a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45842/0 (socket says 172.21.15.157:45842) 2023-10-06T04:03:14.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.126+0000 7fed6b7fe700 1 -- 172.21.15.157:0/3100987905 learned_addr learned my addr 172.21.15.157:0/3100987905 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:14.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.126+0000 7fed6affd700 1 --2- 172.21.15.157:0/3100987905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed6c06c1f0 0x7fed6c158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:14.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.127+0000 7fed6b7fe700 1 -- 172.21.15.157:0/3100987905 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed6c06cbb0 msgr2=0x7fed6c169bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:14.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.127+0000 7fed6b7fe700 1 --2- 172.21.15.157:0/3100987905 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed6c06cbb0 0x7fed6c169bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:14.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.127+0000 7fed6b7fe700 1 -- 172.21.15.157:0/3100987905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed6c06c1f0 msgr2=0x7fed6c158480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:14.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.127+0000 7fed6b7fe700 1 --2- 172.21.15.157:0/3100987905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed6c06c1f0 0x7fed6c158480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:14.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.127+0000 7fed6b7fe700 1 -- 172.21.15.157:0/3100987905 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed64009ae0 con 0x7fed6c150410 2023-10-06T04:03:14.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.127+0000 7fed6a7fc700 1 --2- 172.21.15.157:0/3100987905 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed6c06cbb0 0x7fed6c169bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:03:14.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.127+0000 7fed6b7fe700 1 --2- 172.21.15.157:0/3100987905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed6c150410 0x7fed6c16a130 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fed64009f00 tx=0x7fed6400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:14.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.127+0000 7fed52ffd700 1 -- 172.21.15.157:0/3100987905 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed64018070 con 0x7fed6c150410 2023-10-06T04:03:14.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.127+0000 7fed52ffd700 1 -- 172.21.15.157:0/3100987905 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed6400f040 con 0x7fed6c150410 2023-10-06T04:03:14.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.127+0000 7fed71a09700 1 -- 172.21.15.157:0/3100987905 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed5c009420 con 0x7fed6c150410 2023-10-06T04:03:14.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.127+0000 7fed52ffd700 1 -- 172.21.15.157:0/3100987905 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed640092e0 con 0x7fed6c150410 2023-10-06T04:03:14.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.127+0000 7fed71a09700 1 -- 172.21.15.157:0/3100987905 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6c16a800 con 0x7fed6c150410 2023-10-06T04:03:14.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.128+0000 7fed71a09700 1 -- 172.21.15.157:0/3100987905 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed6c158160 con 0x7fed6c150410 2023-10-06T04:03:14.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.128+0000 7fed52ffd700 1 -- 172.21.15.157:0/3100987905 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fed64003020 con 0x7fed6c150410 2023-10-06T04:03:14.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.129+0000 7fed52ffd700 1 --2- 172.21.15.157:0/3100987905 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fed58066f30 0x7fed580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:14.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.129+0000 7fed52ffd700 1 -- 172.21.15.157:0/3100987905 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7fed64084b90 con 0x7fed6c150410 2023-10-06T04:03:14.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.129+0000 7fed6affd700 1 --2- 172.21.15.157:0/3100987905 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fed58066f30 0x7fed580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:14.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.129+0000 7fed6affd700 1 --2- 172.21.15.157:0/3100987905 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fed58066f30 0x7fed580693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fed5c0093f0 tx=0x7fed5c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:14.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.133+0000 7fed52ffd700 1 -- 172.21.15.157:0/3100987905 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fed64051a50 con 0x7fed6c150410 2023-10-06T04:03:14.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.356+0000 7fed71a09700 1 -- 172.21.15.157:0/3100987905 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed6c0c9130 con 0x7fed6c150410 2023-10-06T04:03:14.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.357+0000 7fed52ffd700 1 -- 172.21.15.157:0/3100987905 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+6162 (secure 0 0 0) 0x7fed64055070 con 0x7fed6c150410 2023-10-06T04:03:14.358 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:03:14.359 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3192541484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3192541484},{"type":"v1","addr":"172.21.15.175:6835","nonce":3192541484}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:13.713593+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279},"failed":[],"damaged":[],"stopped":[3,4],"info":{"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:stopping","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:03:14.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.359+0000 7fed71a09700 1 -- 172.21.15.157:0/3100987905 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fed58066f30 msgr2=0x7fed580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:14.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.359+0000 7fed71a09700 1 --2- 172.21.15.157:0/3100987905 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fed58066f30 0x7fed580693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fed5c0093f0 tx=0x7fed5c017000 comp rx=0 tx=0).stop 2023-10-06T04:03:14.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.359+0000 7fed71a09700 1 -- 172.21.15.157:0/3100987905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed6c150410 msgr2=0x7fed6c16a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:14.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.359+0000 7fed71a09700 1 --2- 172.21.15.157:0/3100987905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed6c150410 0x7fed6c16a130 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fed64009f00 tx=0x7fed6400b040 comp rx=0 tx=0).stop 2023-10-06T04:03:14.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.359+0000 7fed71a09700 1 -- 172.21.15.157:0/3100987905 shutdown_connections 2023-10-06T04:03:14.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.359+0000 7fed71a09700 1 --2- 172.21.15.157:0/3100987905 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fed58066f30 0x7fed580693e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:14.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.359+0000 7fed71a09700 1 --2- 172.21.15.157:0/3100987905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed6c06c1f0 0x7fed6c158480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:14.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.359+0000 7fed71a09700 1 --2- 172.21.15.157:0/3100987905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed6c150410 0x7fed6c16a130 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:14.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.359+0000 7fed71a09700 1 --2- 172.21.15.157:0/3100987905 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed6c06cbb0 0x7fed6c169bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:14.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.359+0000 7fed71a09700 1 -- 172.21.15.157:0/3100987905 >> 172.21.15.157:0/3100987905 conn(0x7fed6c06f230 msgr2=0x7fed6c14e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:14.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.359+0000 7fed71a09700 1 -- 172.21.15.157:0/3100987905 shutdown_connections 2023-10-06T04:03:14.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:14.359+0000 7fed71a09700 1 -- 172.21.15.157:0/3100987905 wait complete. 2023-10-06T04:03:14.362 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 51 2023-10-06T04:03:14.373 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:03:14.891 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:03:14.893 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:03:16.373 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:03:16.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.489+0000 7fda036eb700 1 Processor -- start 2023-10-06T04:03:16.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.489+0000 7fda036eb700 1 -- start start 2023-10-06T04:03:16.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.490+0000 7fda036eb700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9fc06a780 0x7fd9fc159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:16.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.490+0000 7fda036eb700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd9fc06ab50 0x7fd9fc06c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:16.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.490+0000 7fda036eb700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd9fc0bc750 0x7fd9fc06c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:16.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.490+0000 7fda036eb700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9fc155e90 con 0x7fd9fc0bc750 2023-10-06T04:03:16.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.490+0000 7fda036eb700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9fc15a310 con 0x7fd9fc06ab50 2023-10-06T04:03:16.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.490+0000 7fda036eb700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc1569b0 con 0x7fd9fc06a780 2023-10-06T04:03:16.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.490+0000 7fda01c88700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd9fc0bc750 0x7fd9fc06c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:16.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.490+0000 7fda01c88700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd9fc0bc750 0x7fd9fc06c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45850/0 (socket says 172.21.15.157:45850) 2023-10-06T04:03:16.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.490+0000 7fda01c88700 1 -- 172.21.15.157:0/2826733294 learned_addr learned my addr 172.21.15.157:0/2826733294 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:16.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.490+0000 7fda01487700 1 --2- 172.21.15.157:0/2826733294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9fc06a780 0x7fd9fc159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:16.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.490+0000 7fda00c86700 1 --2- 172.21.15.157:0/2826733294 >> v2:172.21.15.175:3300/0 conn(0x7fd9fc06ab50 0x7fd9fc06c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:16.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.491+0000 7fda01c88700 1 -- 172.21.15.157:0/2826733294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9fc06a780 msgr2=0x7fd9fc159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:16.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.491+0000 7fda01c88700 1 --2- 172.21.15.157:0/2826733294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9fc06a780 0x7fd9fc159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:16.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.491+0000 7fda01c88700 1 -- 172.21.15.157:0/2826733294 >> v2:172.21.15.175:3300/0 conn(0x7fd9fc06ab50 msgr2=0x7fd9fc06c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:16.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.491+0000 7fda01c88700 1 --2- 172.21.15.157:0/2826733294 >> v2:172.21.15.175:3300/0 conn(0x7fd9fc06ab50 0x7fd9fc06c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:16.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.491+0000 7fda01c88700 1 -- 172.21.15.157:0/2826733294 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9fc06ea70 con 0x7fd9fc0bc750 2023-10-06T04:03:16.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.491+0000 7fda01c88700 1 --2- 172.21.15.157:0/2826733294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9fc0bc750 0x7fd9fc06c730 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc067c80 tx=0x7fd9f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1cce2abbd3faff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:16.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.491+0000 7fd9f3fff700 1 -- 172.21.15.157:0/2826733294 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f400b6d0 con 0x7fd9fc0bc750 2023-10-06T04:03:16.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.491+0000 7fd9f3fff700 1 -- 172.21.15.157:0/2826733294 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f400b830 con 0x7fd9fc0bc750 2023-10-06T04:03:16.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.491+0000 7fd9f3fff700 1 -- 172.21.15.157:0/2826733294 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f400bbd0 con 0x7fd9fc0bc750 2023-10-06T04:03:16.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.491+0000 7fda036eb700 1 -- 172.21.15.157:0/2826733294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9fc0bc750 msgr2=0x7fd9fc06c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:16.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.491+0000 7fda036eb700 1 --2- 172.21.15.157:0/2826733294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9fc0bc750 0x7fd9fc06c730 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc067c80 tx=0x7fd9f4009250 comp rx=0 tx=0).stop 2023-10-06T04:03:16.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.492+0000 7fda036eb700 1 -- 172.21.15.157:0/2826733294 shutdown_connections 2023-10-06T04:03:16.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.492+0000 7fda036eb700 1 --2- 172.21.15.157:0/2826733294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9fc0bc750 0x7fd9fc06c730 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:16.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.492+0000 7fda036eb700 1 --2- 172.21.15.157:0/2826733294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9fc06a780 0x7fd9fc159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:16.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.492+0000 7fda036eb700 1 --2- 172.21.15.157:0/2826733294 >> v2:172.21.15.175:3300/0 conn(0x7fd9fc06ab50 0x7fd9fc06c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:16.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.492+0000 7fda036eb700 1 -- 172.21.15.157:0/2826733294 >> 172.21.15.157:0/2826733294 conn(0x7fd9fc06b510 msgr2=0x7fd9fc06b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:16.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.492+0000 7fda036eb700 1 -- 172.21.15.157:0/2826733294 shutdown_connections 2023-10-06T04:03:16.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.492+0000 7fda036eb700 1 -- 172.21.15.157:0/2826733294 wait complete. 2023-10-06T04:03:16.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.493+0000 7fda036eb700 1 Processor -- start 2023-10-06T04:03:16.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.493+0000 7fda036eb700 1 -- start start 2023-10-06T04:03:16.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.493+0000 7fda036eb700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9fc06a780 0x7fd9fc0c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:16.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.494+0000 7fda036eb700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9fc06ab50 0x7fd9fc0bd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:16.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.494+0000 7fda036eb700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9fc0bc750 0x7fd9fc154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:16.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.494+0000 7fda036eb700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc0bd0b0 con 0x7fd9fc06ab50 2023-10-06T04:03:16.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.494+0000 7fda036eb700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc0c8db0 con 0x7fd9fc0bc750 2023-10-06T04:03:16.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.494+0000 7fda036eb700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc0bdd60 con 0x7fd9fc06a780 2023-10-06T04:03:16.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.494+0000 7fda00c86700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9fc06ab50 0x7fd9fc0bd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:16.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.494+0000 7fda00c86700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9fc06ab50 0x7fd9fc0bd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45856/0 (socket says 172.21.15.157:45856) 2023-10-06T04:03:16.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.494+0000 7fda00c86700 1 -- 172.21.15.157:0/2229555659 learned_addr learned my addr 172.21.15.157:0/2229555659 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:16.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.494+0000 7fda01487700 1 --2- 172.21.15.157:0/2229555659 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9fc06a780 0x7fd9fc0c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:16.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.494+0000 7fda01c88700 1 --2- 172.21.15.157:0/2229555659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9fc0bc750 0x7fd9fc154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:16.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.494+0000 7fda00c86700 1 -- 172.21.15.157:0/2229555659 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9fc06a780 msgr2=0x7fd9fc0c64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:16.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.494+0000 7fda00c86700 1 --2- 172.21.15.157:0/2229555659 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9fc06a780 0x7fd9fc0c64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:16.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.494+0000 7fda00c86700 1 -- 172.21.15.157:0/2229555659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9fc0bc750 msgr2=0x7fd9fc154aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:16.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.494+0000 7fda00c86700 1 --2- 172.21.15.157:0/2229555659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9fc0bc750 0x7fd9fc154aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:16.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.494+0000 7fda00c86700 1 -- 172.21.15.157:0/2229555659 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f40094f0 con 0x7fd9fc06ab50 2023-10-06T04:03:16.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.494+0000 7fda00c86700 1 --2- 172.21.15.157:0/2229555659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9fc06ab50 0x7fd9fc0bd820 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00ecb0 tx=0x7fd9ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:16.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.494+0000 7fd9f17fa700 1 -- 172.21.15.157:0/2229555659 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec00c850 con 0x7fd9fc06ab50 2023-10-06T04:03:16.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.494+0000 7fd9f17fa700 1 -- 172.21.15.157:0/2229555659 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9ec00c9b0 con 0x7fd9fc06ab50 2023-10-06T04:03:16.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.494+0000 7fda036eb700 1 -- 172.21.15.157:0/2229555659 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9fc0c81e0 con 0x7fd9fc06ab50 2023-10-06T04:03:16.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.494+0000 7fd9f17fa700 1 -- 172.21.15.157:0/2229555659 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec00cd50 con 0x7fd9fc06ab50 2023-10-06T04:03:16.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.494+0000 7fda036eb700 1 -- 172.21.15.157:0/2229555659 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9fc0c8480 con 0x7fd9fc06ab50 2023-10-06T04:03:16.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.495+0000 7fda036eb700 1 -- 172.21.15.157:0/2229555659 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9fc0692b0 con 0x7fd9fc06ab50 2023-10-06T04:03:16.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.496+0000 7fd9f17fa700 1 -- 172.21.15.157:0/2229555659 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd9ec013680 con 0x7fd9fc06ab50 2023-10-06T04:03:16.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.496+0000 7fd9f17fa700 1 --2- 172.21.15.157:0/2229555659 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd9e0066f30 0x7fd9e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:16.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.496+0000 7fd9f17fa700 1 -- 172.21.15.157:0/2229555659 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7fd9ec084e70 con 0x7fd9fc06ab50 2023-10-06T04:03:16.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.496+0000 7fda01487700 1 --2- 172.21.15.157:0/2229555659 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd9e0066f30 0x7fd9e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:16.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.497+0000 7fda01487700 1 --2- 172.21.15.157:0/2229555659 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd9e0066f30 0x7fd9e00693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800f570 tx=0x7fd9f800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:16.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.500+0000 7fd9f17fa700 1 -- 172.21.15.157:0/2229555659 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9ec051d30 con 0x7fd9fc06ab50 2023-10-06T04:03:16.727 INFO:tasks.workunit.client.0.smithi157.stdout:removed './167/11' 2023-10-06T04:03:16.727 INFO:tasks.workunit.client.0.smithi157.stdout:removed './167/16' 2023-10-06T04:03:16.727 INFO:tasks.workunit.client.0.smithi157.stdout:removed './167/32' 2023-10-06T04:03:16.728 INFO:tasks.workunit.client.0.smithi157.stdout:removed './167/30' 2023-10-06T04:03:16.728 INFO:tasks.workunit.client.0.smithi157.stdout:removed './167/14' 2023-10-06T04:03:16.728 INFO:tasks.workunit.client.0.smithi157.stdout:removed './167/18' 2023-10-06T04:03:16.728 INFO:tasks.workunit.client.0.smithi157.stdout:removed './167/13' 2023-10-06T04:03:16.728 INFO:tasks.workunit.client.0.smithi157.stdout:removed './167/33' 2023-10-06T04:03:16.728 INFO:tasks.workunit.client.0.smithi157.stdout:removed './167/17' 2023-10-06T04:03:16.728 INFO:tasks.workunit.client.0.smithi157.stdout:removed './167/35' 2023-10-06T04:03:16.728 INFO:tasks.workunit.client.0.smithi157.stdout:removed './167/37' 2023-10-06T04:03:16.728 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './167' 2023-10-06T04:03:16.728 INFO:tasks.workunit.client.0.smithi157.stdout:removed './168/19' 2023-10-06T04:03:16.728 INFO:tasks.workunit.client.0.smithi157.stdout:removed './168/15' 2023-10-06T04:03:16.729 INFO:tasks.workunit.client.0.smithi157.stdout:removed './168/31' 2023-10-06T04:03:16.729 INFO:tasks.workunit.client.0.smithi157.stdout:removed './168/36' 2023-10-06T04:03:16.729 INFO:tasks.workunit.client.0.smithi157.stdout:removed './168/20' 2023-10-06T04:03:16.729 INFO:tasks.workunit.client.0.smithi157.stdout:removed './168/12' 2023-10-06T04:03:16.729 INFO:tasks.workunit.client.0.smithi157.stdout:removed './168/39' 2023-10-06T04:03:16.729 INFO:tasks.workunit.client.0.smithi157.stdout:removed './168/10' 2023-10-06T04:03:16.729 INFO:tasks.workunit.client.0.smithi157.stdout:removed './168/34' 2023-10-06T04:03:16.729 INFO:tasks.workunit.client.0.smithi157.stdout:removed './168/38' 2023-10-06T04:03:16.729 INFO:tasks.workunit.client.0.smithi157.stdout:removed './168/11' 2023-10-06T04:03:16.729 INFO:tasks.workunit.client.0.smithi157.stdout:removed './168/16' 2023-10-06T04:03:16.729 INFO:tasks.workunit.client.0.smithi157.stdout:removed './168/32' 2023-10-06T04:03:16.730 INFO:tasks.workunit.client.0.smithi157.stdout:removed './168/30' 2023-10-06T04:03:16.730 INFO:tasks.workunit.client.0.smithi157.stdout:removed './168/14' 2023-10-06T04:03:16.730 INFO:tasks.workunit.client.0.smithi157.stdout:removed './168/18' 2023-10-06T04:03:16.730 INFO:tasks.workunit.client.0.smithi157.stdout:removed './168/13' 2023-10-06T04:03:16.730 INFO:tasks.workunit.client.0.smithi157.stdout:removed './168/33' 2023-10-06T04:03:16.730 INFO:tasks.workunit.client.0.smithi157.stdout:removed './168/17' 2023-10-06T04:03:16.730 INFO:tasks.workunit.client.0.smithi157.stdout:removed './168/35' 2023-10-06T04:03:16.731 INFO:tasks.workunit.client.0.smithi157.stdout:removed './168/37' 2023-10-06T04:03:16.731 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './168' 2023-10-06T04:03:16.731 INFO:tasks.workunit.client.0.smithi157.stdout:removed './169/19' 2023-10-06T04:03:16.731 INFO:tasks.workunit.client.0.smithi157.stdout:removed './169/15' 2023-10-06T04:03:16.731 INFO:tasks.workunit.client.0.smithi157.stdout:removed './169/31' 2023-10-06T04:03:16.731 INFO:tasks.workunit.client.0.smithi157.stdout:removed './169/36' 2023-10-06T04:03:16.731 INFO:tasks.workunit.client.0.smithi157.stdout:removed './169/20' 2023-10-06T04:03:16.731 INFO:tasks.workunit.client.0.smithi157.stdout:removed './169/12' 2023-10-06T04:03:16.731 INFO:tasks.workunit.client.0.smithi157.stdout:removed './169/39' 2023-10-06T04:03:16.731 INFO:tasks.workunit.client.0.smithi157.stdout:removed './169/10' 2023-10-06T04:03:16.731 INFO:tasks.workunit.client.0.smithi157.stdout:removed './169/34' 2023-10-06T04:03:16.732 INFO:tasks.workunit.client.0.smithi157.stdout:removed './169/38' 2023-10-06T04:03:16.732 INFO:tasks.workunit.client.0.smithi157.stdout:removed './169/11' 2023-10-06T04:03:16.732 INFO:tasks.workunit.client.0.smithi157.stdout:removed './169/16' 2023-10-06T04:03:16.732 INFO:tasks.workunit.client.0.smithi157.stdout:removed './169/32' 2023-10-06T04:03:16.732 INFO:tasks.workunit.client.0.smithi157.stdout:removed './169/30' 2023-10-06T04:03:16.732 INFO:tasks.workunit.client.0.smithi157.stdout:removed './169/14' 2023-10-06T04:03:16.732 INFO:tasks.workunit.client.0.smithi157.stdout:removed './169/18' 2023-10-06T04:03:16.732 INFO:tasks.workunit.client.0.smithi157.stdout:removed './169/13' 2023-10-06T04:03:16.732 INFO:tasks.workunit.client.0.smithi157.stdout:removed './169/33' 2023-10-06T04:03:16.732 INFO:tasks.workunit.client.0.smithi157.stdout:removed './169/17' 2023-10-06T04:03:16.732 INFO:tasks.workunit.client.0.smithi157.stdout:removed './169/35' 2023-10-06T04:03:16.733 INFO:tasks.workunit.client.0.smithi157.stdout:removed './169/37' 2023-10-06T04:03:16.733 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './169' 2023-10-06T04:03:16.733 INFO:tasks.workunit.client.0.smithi157.stdout:removed './170/19' 2023-10-06T04:03:16.733 INFO:tasks.workunit.client.0.smithi157.stdout:removed './170/15' 2023-10-06T04:03:16.733 INFO:tasks.workunit.client.0.smithi157.stdout:removed './170/31' 2023-10-06T04:03:16.733 INFO:tasks.workunit.client.0.smithi157.stdout:removed './170/36' 2023-10-06T04:03:16.733 INFO:tasks.workunit.client.0.smithi157.stdout:removed './170/20' 2023-10-06T04:03:16.733 INFO:tasks.workunit.client.0.smithi157.stdout:removed './170/12' 2023-10-06T04:03:16.733 INFO:tasks.workunit.client.0.smithi157.stdout:removed './170/39' 2023-10-06T04:03:16.733 INFO:tasks.workunit.client.0.smithi157.stdout:removed './170/10' 2023-10-06T04:03:16.733 INFO:tasks.workunit.client.0.smithi157.stdout:removed './170/34' 2023-10-06T04:03:16.733 INFO:tasks.workunit.client.0.smithi157.stdout:removed './170/38' 2023-10-06T04:03:16.734 INFO:tasks.workunit.client.0.smithi157.stdout:removed './170/11' 2023-10-06T04:03:16.734 INFO:tasks.workunit.client.0.smithi157.stdout:removed './170/16' 2023-10-06T04:03:16.734 INFO:tasks.workunit.client.0.smithi157.stdout:removed './170/32' 2023-10-06T04:03:16.734 INFO:tasks.workunit.client.0.smithi157.stdout:removed './170/30' 2023-10-06T04:03:16.734 INFO:tasks.workunit.client.0.smithi157.stdout:removed './170/14' 2023-10-06T04:03:16.734 INFO:tasks.workunit.client.0.smithi157.stdout:removed './170/18' 2023-10-06T04:03:16.734 INFO:tasks.workunit.client.0.smithi157.stdout:removed './170/13' 2023-10-06T04:03:16.734 INFO:tasks.workunit.client.0.smithi157.stdout:removed './170/33' 2023-10-06T04:03:16.734 INFO:tasks.workunit.client.0.smithi157.stdout:removed './170/17' 2023-10-06T04:03:16.734 INFO:tasks.workunit.client.0.smithi157.stdout:removed './170/35' 2023-10-06T04:03:16.734 INFO:tasks.workunit.client.0.smithi157.stdout:removed './170/37' 2023-10-06T04:03:16.735 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './170' 2023-10-06T04:03:16.735 INFO:tasks.workunit.client.0.smithi157.stdout:removed './171/19' 2023-10-06T04:03:16.735 INFO:tasks.workunit.client.0.smithi157.stdout:removed './171/15' 2023-10-06T04:03:16.735 INFO:tasks.workunit.client.0.smithi157.stdout:removed './171/31' 2023-10-06T04:03:16.735 INFO:tasks.workunit.client.0.smithi157.stdout:removed './171/36' 2023-10-06T04:03:16.735 INFO:tasks.workunit.client.0.smithi157.stdout:removed './171/20' 2023-10-06T04:03:16.735 INFO:tasks.workunit.client.0.smithi157.stdout:removed './171/12' 2023-10-06T04:03:16.735 INFO:tasks.workunit.client.0.smithi157.stdout:removed './171/39' 2023-10-06T04:03:16.735 INFO:tasks.workunit.client.0.smithi157.stdout:removed './171/10' 2023-10-06T04:03:16.735 INFO:tasks.workunit.client.0.smithi157.stdout:removed './171/34' 2023-10-06T04:03:16.735 INFO:tasks.workunit.client.0.smithi157.stdout:removed './171/38' 2023-10-06T04:03:16.736 INFO:tasks.workunit.client.0.smithi157.stdout:removed './171/11' 2023-10-06T04:03:16.736 INFO:tasks.workunit.client.0.smithi157.stdout:removed './171/16' 2023-10-06T04:03:16.736 INFO:tasks.workunit.client.0.smithi157.stdout:removed './171/32' 2023-10-06T04:03:16.736 INFO:tasks.workunit.client.0.smithi157.stdout:removed './171/30' 2023-10-06T04:03:16.736 INFO:tasks.workunit.client.0.smithi157.stdout:removed './171/14' 2023-10-06T04:03:16.736 INFO:tasks.workunit.client.0.smithi157.stdout:removed './171/18' 2023-10-06T04:03:16.736 INFO:tasks.workunit.client.0.smithi157.stdout:removed './171/13' 2023-10-06T04:03:16.736 INFO:tasks.workunit.client.0.smithi157.stdout:removed './171/33' 2023-10-06T04:03:16.736 INFO:tasks.workunit.client.0.smithi157.stdout:removed './171/17' 2023-10-06T04:03:16.736 INFO:tasks.workunit.client.0.smithi157.stdout:removed './171/35' 2023-10-06T04:03:16.736 INFO:tasks.workunit.client.0.smithi157.stdout:removed './171/37' 2023-10-06T04:03:16.736 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './171' 2023-10-06T04:03:16.737 INFO:tasks.workunit.client.0.smithi157.stdout:removed './172/19' 2023-10-06T04:03:16.737 INFO:tasks.workunit.client.0.smithi157.stdout:removed './172/15' 2023-10-06T04:03:16.737 INFO:tasks.workunit.client.0.smithi157.stdout:removed './172/31' 2023-10-06T04:03:16.737 INFO:tasks.workunit.client.0.smithi157.stdout:removed './172/36' 2023-10-06T04:03:16.737 INFO:tasks.workunit.client.0.smithi157.stdout:removed './172/20' 2023-10-06T04:03:16.737 INFO:tasks.workunit.client.0.smithi157.stdout:removed './172/12' 2023-10-06T04:03:16.737 INFO:tasks.workunit.client.0.smithi157.stdout:removed './172/39' 2023-10-06T04:03:16.737 INFO:tasks.workunit.client.0.smithi157.stdout:removed './172/10' 2023-10-06T04:03:16.737 INFO:tasks.workunit.client.0.smithi157.stdout:removed './172/34' 2023-10-06T04:03:16.737 INFO:tasks.workunit.client.0.smithi157.stdout:removed './172/38' 2023-10-06T04:03:16.737 INFO:tasks.workunit.client.0.smithi157.stdout:removed './172/11' 2023-10-06T04:03:16.738 INFO:tasks.workunit.client.0.smithi157.stdout:removed './172/16' 2023-10-06T04:03:16.738 INFO:tasks.workunit.client.0.smithi157.stdout:removed './172/32' 2023-10-06T04:03:16.738 INFO:tasks.workunit.client.0.smithi157.stdout:removed './172/30' 2023-10-06T04:03:16.738 INFO:tasks.workunit.client.0.smithi157.stdout:removed './172/14' 2023-10-06T04:03:16.738 INFO:tasks.workunit.client.0.smithi157.stdout:removed './172/18' 2023-10-06T04:03:16.738 INFO:tasks.workunit.client.0.smithi157.stdout:removed './172/13' 2023-10-06T04:03:16.738 INFO:tasks.workunit.client.0.smithi157.stdout:removed './172/33' 2023-10-06T04:03:16.738 INFO:tasks.workunit.client.0.smithi157.stdout:removed './172/17' 2023-10-06T04:03:16.738 INFO:tasks.workunit.client.0.smithi157.stdout:removed './172/35' 2023-10-06T04:03:16.738 INFO:tasks.workunit.client.0.smithi157.stdout:removed './172/37' 2023-10-06T04:03:16.738 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './172' 2023-10-06T04:03:16.738 INFO:tasks.workunit.client.0.smithi157.stdout:removed './173/19' 2023-10-06T04:03:16.739 INFO:tasks.workunit.client.0.smithi157.stdout:removed './173/15' 2023-10-06T04:03:16.739 INFO:tasks.workunit.client.0.smithi157.stdout:removed './173/31' 2023-10-06T04:03:16.739 INFO:tasks.workunit.client.0.smithi157.stdout:removed './173/36' 2023-10-06T04:03:16.739 INFO:tasks.workunit.client.0.smithi157.stdout:removed './173/20' 2023-10-06T04:03:16.739 INFO:tasks.workunit.client.0.smithi157.stdout:removed './173/12' 2023-10-06T04:03:16.739 INFO:tasks.workunit.client.0.smithi157.stdout:removed './173/39' 2023-10-06T04:03:16.739 INFO:tasks.workunit.client.0.smithi157.stdout:removed './173/10' 2023-10-06T04:03:16.739 INFO:tasks.workunit.client.0.smithi157.stdout:removed './173/34' 2023-10-06T04:03:16.739 INFO:tasks.workunit.client.0.smithi157.stdout:removed './173/38' 2023-10-06T04:03:16.739 INFO:tasks.workunit.client.0.smithi157.stdout:removed './173/11' 2023-10-06T04:03:16.739 INFO:tasks.workunit.client.0.smithi157.stdout:removed './173/16' 2023-10-06T04:03:16.740 INFO:tasks.workunit.client.0.smithi157.stdout:removed './173/32' 2023-10-06T04:03:16.740 INFO:tasks.workunit.client.0.smithi157.stdout:removed './173/30' 2023-10-06T04:03:16.740 INFO:tasks.workunit.client.0.smithi157.stdout:removed './173/14' 2023-10-06T04:03:16.740 INFO:tasks.workunit.client.0.smithi157.stdout:removed './173/18' 2023-10-06T04:03:16.740 INFO:tasks.workunit.client.0.smithi157.stdout:removed './173/13' 2023-10-06T04:03:16.740 INFO:tasks.workunit.client.0.smithi157.stdout:removed './173/33' 2023-10-06T04:03:16.740 INFO:tasks.workunit.client.0.smithi157.stdout:removed './173/17' 2023-10-06T04:03:16.740 INFO:tasks.workunit.client.0.smithi157.stdout:removed './173/35' 2023-10-06T04:03:16.740 INFO:tasks.workunit.client.0.smithi157.stdout:removed './173/37' 2023-10-06T04:03:16.741 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './173' 2023-10-06T04:03:16.741 INFO:tasks.workunit.client.0.smithi157.stdout:removed './174/19' 2023-10-06T04:03:16.741 INFO:tasks.workunit.client.0.smithi157.stdout:removed './174/15' 2023-10-06T04:03:16.741 INFO:tasks.workunit.client.0.smithi157.stdout:removed './174/31' 2023-10-06T04:03:16.741 INFO:tasks.workunit.client.0.smithi157.stdout:removed './174/36' 2023-10-06T04:03:16.741 INFO:tasks.workunit.client.0.smithi157.stdout:removed './174/20' 2023-10-06T04:03:16.741 INFO:tasks.workunit.client.0.smithi157.stdout:removed './174/12' 2023-10-06T04:03:16.741 INFO:tasks.workunit.client.0.smithi157.stdout:removed './174/39' 2023-10-06T04:03:16.741 INFO:tasks.workunit.client.0.smithi157.stdout:removed './174/10' 2023-10-06T04:03:16.741 INFO:tasks.workunit.client.0.smithi157.stdout:removed './174/34' 2023-10-06T04:03:16.742 INFO:tasks.workunit.client.0.smithi157.stdout:removed './174/38' 2023-10-06T04:03:16.742 INFO:tasks.workunit.client.0.smithi157.stdout:removed './174/11' 2023-10-06T04:03:16.742 INFO:tasks.workunit.client.0.smithi157.stdout:removed './174/16' 2023-10-06T04:03:16.742 INFO:tasks.workunit.client.0.smithi157.stdout:removed './174/32' 2023-10-06T04:03:16.742 INFO:tasks.workunit.client.0.smithi157.stdout:removed './174/30' 2023-10-06T04:03:16.742 INFO:tasks.workunit.client.0.smithi157.stdout:removed './174/14' 2023-10-06T04:03:16.742 INFO:tasks.workunit.client.0.smithi157.stdout:removed './174/18' 2023-10-06T04:03:16.742 INFO:tasks.workunit.client.0.smithi157.stdout:removed './174/13' 2023-10-06T04:03:16.742 INFO:tasks.workunit.client.0.smithi157.stdout:removed './174/33' 2023-10-06T04:03:16.742 INFO:tasks.workunit.client.0.smithi157.stdout:removed './174/17' 2023-10-06T04:03:16.742 INFO:tasks.workunit.client.0.smithi157.stdout:removed './174/35' 2023-10-06T04:03:16.743 INFO:tasks.workunit.client.0.smithi157.stdout:removed './174/37' 2023-10-06T04:03:16.743 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './174' 2023-10-06T04:03:16.743 INFO:tasks.workunit.client.0.smithi157.stdout:removed './175/19' 2023-10-06T04:03:16.743 INFO:tasks.workunit.client.0.smithi157.stdout:removed './175/15' 2023-10-06T04:03:16.743 INFO:tasks.workunit.client.0.smithi157.stdout:removed './175/31' 2023-10-06T04:03:16.743 INFO:tasks.workunit.client.0.smithi157.stdout:removed './175/36' 2023-10-06T04:03:16.743 INFO:tasks.workunit.client.0.smithi157.stdout:removed './175/20' 2023-10-06T04:03:16.743 INFO:tasks.workunit.client.0.smithi157.stdout:removed './175/12' 2023-10-06T04:03:16.743 INFO:tasks.workunit.client.0.smithi157.stdout:removed './175/39' 2023-10-06T04:03:16.743 INFO:tasks.workunit.client.0.smithi157.stdout:removed './175/10' 2023-10-06T04:03:16.743 INFO:tasks.workunit.client.0.smithi157.stdout:removed './175/34' 2023-10-06T04:03:16.744 INFO:tasks.workunit.client.0.smithi157.stdout:removed './175/38' 2023-10-06T04:03:16.744 INFO:tasks.workunit.client.0.smithi157.stdout:removed './175/11' 2023-10-06T04:03:16.744 INFO:tasks.workunit.client.0.smithi157.stdout:removed './175/16' 2023-10-06T04:03:16.744 INFO:tasks.workunit.client.0.smithi157.stdout:removed './175/32' 2023-10-06T04:03:16.744 INFO:tasks.workunit.client.0.smithi157.stdout:removed './175/30' 2023-10-06T04:03:16.744 INFO:tasks.workunit.client.0.smithi157.stdout:removed './175/14' 2023-10-06T04:03:16.744 INFO:tasks.workunit.client.0.smithi157.stdout:removed './175/18' 2023-10-06T04:03:16.744 INFO:tasks.workunit.client.0.smithi157.stdout:removed './175/13' 2023-10-06T04:03:16.744 INFO:tasks.workunit.client.0.smithi157.stdout:removed './175/33' 2023-10-06T04:03:16.744 INFO:tasks.workunit.client.0.smithi157.stdout:removed './175/17' 2023-10-06T04:03:16.744 INFO:tasks.workunit.client.0.smithi157.stdout:removed './175/35' 2023-10-06T04:03:16.744 INFO:tasks.workunit.client.0.smithi157.stdout:removed './175/37' 2023-10-06T04:03:16.745 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './175' 2023-10-06T04:03:16.745 INFO:tasks.workunit.client.0.smithi157.stdout:removed './176/19' 2023-10-06T04:03:16.745 INFO:tasks.workunit.client.0.smithi157.stdout:removed './176/15' 2023-10-06T04:03:16.745 INFO:tasks.workunit.client.0.smithi157.stdout:removed './176/31' 2023-10-06T04:03:16.745 INFO:tasks.workunit.client.0.smithi157.stdout:removed './176/36' 2023-10-06T04:03:16.745 INFO:tasks.workunit.client.0.smithi157.stdout:removed './176/20' 2023-10-06T04:03:16.745 INFO:tasks.workunit.client.0.smithi157.stdout:removed './176/12' 2023-10-06T04:03:16.745 INFO:tasks.workunit.client.0.smithi157.stdout:removed './176/39' 2023-10-06T04:03:16.745 INFO:tasks.workunit.client.0.smithi157.stdout:removed './176/10' 2023-10-06T04:03:16.745 INFO:tasks.workunit.client.0.smithi157.stdout:removed './176/34' 2023-10-06T04:03:16.745 INFO:tasks.workunit.client.0.smithi157.stdout:removed './176/38' 2023-10-06T04:03:16.745 INFO:tasks.workunit.client.0.smithi157.stdout:removed './176/11' 2023-10-06T04:03:16.746 INFO:tasks.workunit.client.0.smithi157.stdout:removed './176/16' 2023-10-06T04:03:16.746 INFO:tasks.workunit.client.0.smithi157.stdout:removed './176/32' 2023-10-06T04:03:16.746 INFO:tasks.workunit.client.0.smithi157.stdout:removed './176/30' 2023-10-06T04:03:16.746 INFO:tasks.workunit.client.0.smithi157.stdout:removed './176/14' 2023-10-06T04:03:16.746 INFO:tasks.workunit.client.0.smithi157.stdout:removed './176/18' 2023-10-06T04:03:16.746 INFO:tasks.workunit.client.0.smithi157.stdout:removed './176/13' 2023-10-06T04:03:16.746 INFO:tasks.workunit.client.0.smithi157.stdout:removed './176/33' 2023-10-06T04:03:16.746 INFO:tasks.workunit.client.0.smithi157.stdout:removed './176/17' 2023-10-06T04:03:16.746 INFO:tasks.workunit.client.0.smithi157.stdout:removed './176/35' 2023-10-06T04:03:16.746 INFO:tasks.workunit.client.0.smithi157.stdout:removed './176/37' 2023-10-06T04:03:16.746 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './176' 2023-10-06T04:03:16.746 INFO:tasks.workunit.client.0.smithi157.stdout:removed './177/19' 2023-10-06T04:03:16.747 INFO:tasks.workunit.client.0.smithi157.stdout:removed './177/15' 2023-10-06T04:03:16.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.756+0000 7fda036eb700 1 -- 172.21.15.157:0/2229555659 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9fc0c8ef0 con 0x7fd9fc06ab50 2023-10-06T04:03:16.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.757+0000 7fd9f17fa700 1 -- 172.21.15.157:0/2229555659 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+6162 (secure 0 0 0) 0x7fd9ec017090 con 0x7fd9fc06ab50 2023-10-06T04:03:16.759 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:03:16.759 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3192541484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3192541484},{"type":"v1","addr":"172.21.15.175:6835","nonce":3192541484}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:13.713593+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279},"failed":[],"damaged":[],"stopped":[3,4],"info":{"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:stopping","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:03:16.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.759+0000 7fda036eb700 1 -- 172.21.15.157:0/2229555659 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd9e0066f30 msgr2=0x7fd9e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:16.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.759+0000 7fda036eb700 1 --2- 172.21.15.157:0/2229555659 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd9e0066f30 0x7fd9e00693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800f570 tx=0x7fd9f800b040 comp rx=0 tx=0).stop 2023-10-06T04:03:16.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.759+0000 7fda036eb700 1 -- 172.21.15.157:0/2229555659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9fc06ab50 msgr2=0x7fd9fc0bd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:16.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.759+0000 7fda036eb700 1 --2- 172.21.15.157:0/2229555659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9fc06ab50 0x7fd9fc0bd820 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00ecb0 tx=0x7fd9ec0091a0 comp rx=0 tx=0).stop 2023-10-06T04:03:16.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.759+0000 7fda036eb700 1 -- 172.21.15.157:0/2229555659 shutdown_connections 2023-10-06T04:03:16.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.759+0000 7fda036eb700 1 --2- 172.21.15.157:0/2229555659 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd9e0066f30 0x7fd9e00693e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:16.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.759+0000 7fda036eb700 1 --2- 172.21.15.157:0/2229555659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9fc0bc750 0x7fd9fc154aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:16.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.759+0000 7fda036eb700 1 --2- 172.21.15.157:0/2229555659 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9fc06a780 0x7fd9fc0c64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:16.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.759+0000 7fda036eb700 1 --2- 172.21.15.157:0/2229555659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9fc06ab50 0x7fd9fc0bd820 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:16.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.759+0000 7fda036eb700 1 -- 172.21.15.157:0/2229555659 >> 172.21.15.157:0/2229555659 conn(0x7fd9fc06b510 msgr2=0x7fd9fc0c0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:16.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.759+0000 7fda036eb700 1 -- 172.21.15.157:0/2229555659 shutdown_connections 2023-10-06T04:03:16.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:16.760+0000 7fda036eb700 1 -- 172.21.15.157:0/2229555659 wait complete. 2023-10-06T04:03:16.763 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 51 2023-10-06T04:03:16.773 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2023-10-06T04:03:16.774 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 51, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-06T03:57:10.718782+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 51, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4279": { "addr": "172.21.15.175:6837/1002404870", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6836", "nonce": 1002404870, "type": "v2" }, { "addr": "172.21.15.175:6837", "nonce": 1002404870, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1 ], "features": 4540138320759226367, "flags": 0, "gid": 4279, "incarnation": 11, "join_fscid": -1, "name": "d", "rank": 2, "state": "up:stopping", "state_seq": 7 }, "gid_4287": { "addr": "172.21.15.157:6834/1179753185", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6832", "nonce": 1179753185, "type": "v2" }, { "addr": "172.21.15.157:6834", "nonce": 1179753185, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4287, "incarnation": 9, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4296": { "addr": "172.21.15.157:6837/4188652695", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6836", "nonce": 4188652695, "type": "v2" }, { "addr": "172.21.15.157:6837", "nonce": 4188652695, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4296, "incarnation": 4, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 27, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2023-10-06T04:03:13.713593+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 3, 4 ], "tableserver": 0, "up": { "mds_0": 4296, "mds_1": 4287, "mds_2": 4279 } } } ], "standbys": [ { "addr": "172.21.15.175:6835/3192541484", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6834", "nonce": 3192541484, "type": "v2" }, { "addr": "172.21.15.175:6835", "nonce": 3192541484, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 48, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5009, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.175:6839/745804271", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6838", "nonce": 745804271, "type": "v2" }, { "addr": "172.21.15.175:6839", "nonce": 745804271, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 42, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5164, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.157:6835/2461751318", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6833", "nonce": 2461751318, "type": "v2" }, { "addr": "172.21.15.157:6835", "nonce": 2461751318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 34, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5268, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-06T04:03:18.492 INFO:tasks.workunit.client.0.smithi157.stdout:removed './177/31' 2023-10-06T04:03:18.565 INFO:tasks.workunit.client.0.smithi157.stdout:removed './177/36' 2023-10-06T04:03:18.565 INFO:tasks.workunit.client.0.smithi157.stdout:removed './177/20' 2023-10-06T04:03:18.565 INFO:tasks.workunit.client.0.smithi157.stdout:removed './177/12' 2023-10-06T04:03:18.566 INFO:tasks.workunit.client.0.smithi157.stdout:removed './177/39' 2023-10-06T04:03:18.566 INFO:tasks.workunit.client.0.smithi157.stdout:removed './177/10' 2023-10-06T04:03:18.566 INFO:tasks.workunit.client.0.smithi157.stdout:removed './177/34' 2023-10-06T04:03:18.566 INFO:tasks.workunit.client.0.smithi157.stdout:removed './177/38' 2023-10-06T04:03:18.566 INFO:tasks.workunit.client.0.smithi157.stdout:removed './177/11' 2023-10-06T04:03:18.567 INFO:tasks.workunit.client.0.smithi157.stdout:removed './177/16' 2023-10-06T04:03:18.567 INFO:tasks.workunit.client.0.smithi157.stdout:removed './177/32' 2023-10-06T04:03:18.567 INFO:tasks.workunit.client.0.smithi157.stdout:removed './177/30' 2023-10-06T04:03:18.567 INFO:tasks.workunit.client.0.smithi157.stdout:removed './177/14' 2023-10-06T04:03:18.567 INFO:tasks.workunit.client.0.smithi157.stdout:removed './177/18' 2023-10-06T04:03:18.567 INFO:tasks.workunit.client.0.smithi157.stdout:removed './177/13' 2023-10-06T04:03:18.567 INFO:tasks.workunit.client.0.smithi157.stdout:removed './177/33' 2023-10-06T04:03:18.567 INFO:tasks.workunit.client.0.smithi157.stdout:removed './177/17' 2023-10-06T04:03:18.567 INFO:tasks.workunit.client.0.smithi157.stdout:removed './177/35' 2023-10-06T04:03:18.567 INFO:tasks.workunit.client.0.smithi157.stdout:removed './177/37' 2023-10-06T04:03:18.567 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './177' 2023-10-06T04:03:18.568 INFO:tasks.workunit.client.0.smithi157.stdout:removed './178/19' 2023-10-06T04:03:18.568 INFO:tasks.workunit.client.0.smithi157.stdout:removed './178/15' 2023-10-06T04:03:18.568 INFO:tasks.workunit.client.0.smithi157.stdout:removed './178/31' 2023-10-06T04:03:18.568 INFO:tasks.workunit.client.0.smithi157.stdout:removed './178/36' 2023-10-06T04:03:18.568 INFO:tasks.workunit.client.0.smithi157.stdout:removed './178/20' 2023-10-06T04:03:18.568 INFO:tasks.workunit.client.0.smithi157.stdout:removed './178/12' 2023-10-06T04:03:18.568 INFO:tasks.workunit.client.0.smithi157.stdout:removed './178/39' 2023-10-06T04:03:18.568 INFO:tasks.workunit.client.0.smithi157.stdout:removed './178/10' 2023-10-06T04:03:18.568 INFO:tasks.workunit.client.0.smithi157.stdout:removed './178/34' 2023-10-06T04:03:18.568 INFO:tasks.workunit.client.0.smithi157.stdout:removed './178/38' 2023-10-06T04:03:18.569 INFO:tasks.workunit.client.0.smithi157.stdout:removed './178/11' 2023-10-06T04:03:18.569 INFO:tasks.workunit.client.0.smithi157.stdout:removed './178/16' 2023-10-06T04:03:18.569 INFO:tasks.workunit.client.0.smithi157.stdout:removed './178/32' 2023-10-06T04:03:18.569 INFO:tasks.workunit.client.0.smithi157.stdout:removed './178/30' 2023-10-06T04:03:18.569 INFO:tasks.workunit.client.0.smithi157.stdout:removed './178/14' 2023-10-06T04:03:18.569 INFO:tasks.workunit.client.0.smithi157.stdout:removed './178/18' 2023-10-06T04:03:18.569 INFO:tasks.workunit.client.0.smithi157.stdout:removed './178/13' 2023-10-06T04:03:18.569 INFO:tasks.workunit.client.0.smithi157.stdout:removed './178/33' 2023-10-06T04:03:18.572 INFO:tasks.workunit.client.0.smithi157.stdout:removed './178/17' 2023-10-06T04:03:18.572 INFO:tasks.workunit.client.0.smithi157.stdout:removed './178/35' 2023-10-06T04:03:18.573 INFO:tasks.workunit.client.0.smithi157.stdout:removed './178/37' 2023-10-06T04:03:18.573 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './178' 2023-10-06T04:03:18.573 INFO:tasks.workunit.client.0.smithi157.stdout:removed './179/19' 2023-10-06T04:03:18.573 INFO:tasks.workunit.client.0.smithi157.stdout:removed './179/15' 2023-10-06T04:03:18.573 INFO:tasks.workunit.client.0.smithi157.stdout:removed './179/31' 2023-10-06T04:03:18.573 INFO:tasks.workunit.client.0.smithi157.stdout:removed './179/36' 2023-10-06T04:03:18.573 INFO:tasks.workunit.client.0.smithi157.stdout:removed './179/20' 2023-10-06T04:03:18.573 INFO:tasks.workunit.client.0.smithi157.stdout:removed './179/12' 2023-10-06T04:03:18.573 INFO:tasks.workunit.client.0.smithi157.stdout:removed './179/39' 2023-10-06T04:03:18.573 INFO:tasks.workunit.client.0.smithi157.stdout:removed './179/10' 2023-10-06T04:03:18.573 INFO:tasks.workunit.client.0.smithi157.stdout:removed './179/34' 2023-10-06T04:03:18.573 INFO:tasks.workunit.client.0.smithi157.stdout:removed './179/38' 2023-10-06T04:03:18.574 INFO:tasks.workunit.client.0.smithi157.stdout:removed './179/11' 2023-10-06T04:03:18.574 INFO:tasks.workunit.client.0.smithi157.stdout:removed './179/16' 2023-10-06T04:03:18.574 INFO:tasks.workunit.client.0.smithi157.stdout:removed './179/32' 2023-10-06T04:03:18.574 INFO:tasks.workunit.client.0.smithi157.stdout:removed './179/30' 2023-10-06T04:03:18.574 INFO:tasks.workunit.client.0.smithi157.stdout:removed './179/14' 2023-10-06T04:03:18.574 INFO:tasks.workunit.client.0.smithi157.stdout:removed './179/18' 2023-10-06T04:03:18.574 INFO:tasks.workunit.client.0.smithi157.stdout:removed './179/13' 2023-10-06T04:03:18.574 INFO:tasks.workunit.client.0.smithi157.stdout:removed './179/33' 2023-10-06T04:03:18.574 INFO:tasks.workunit.client.0.smithi157.stdout:removed './179/17' 2023-10-06T04:03:18.574 INFO:tasks.workunit.client.0.smithi157.stdout:removed './179/35' 2023-10-06T04:03:18.574 INFO:tasks.workunit.client.0.smithi157.stdout:removed './179/37' 2023-10-06T04:03:18.574 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './179' 2023-10-06T04:03:18.575 INFO:tasks.workunit.client.0.smithi157.stdout:removed './180/19' 2023-10-06T04:03:18.575 INFO:tasks.workunit.client.0.smithi157.stdout:removed './180/15' 2023-10-06T04:03:18.575 INFO:tasks.workunit.client.0.smithi157.stdout:removed './180/31' 2023-10-06T04:03:18.575 INFO:tasks.workunit.client.0.smithi157.stdout:removed './180/36' 2023-10-06T04:03:18.575 INFO:tasks.workunit.client.0.smithi157.stdout:removed './180/20' 2023-10-06T04:03:18.575 INFO:tasks.workunit.client.0.smithi157.stdout:removed './180/12' 2023-10-06T04:03:18.575 INFO:tasks.workunit.client.0.smithi157.stdout:removed './180/39' 2023-10-06T04:03:18.575 INFO:tasks.workunit.client.0.smithi157.stdout:removed './180/10' 2023-10-06T04:03:18.575 INFO:tasks.workunit.client.0.smithi157.stdout:removed './180/34' 2023-10-06T04:03:18.575 INFO:tasks.workunit.client.0.smithi157.stdout:removed './180/38' 2023-10-06T04:03:18.575 INFO:tasks.workunit.client.0.smithi157.stdout:removed './180/11' 2023-10-06T04:03:18.576 INFO:tasks.workunit.client.0.smithi157.stdout:removed './180/16' 2023-10-06T04:03:18.576 INFO:tasks.workunit.client.0.smithi157.stdout:removed './180/32' 2023-10-06T04:03:18.576 INFO:tasks.workunit.client.0.smithi157.stdout:removed './180/30' 2023-10-06T04:03:18.576 INFO:tasks.workunit.client.0.smithi157.stdout:removed './180/14' 2023-10-06T04:03:18.576 INFO:tasks.workunit.client.0.smithi157.stdout:removed './180/18' 2023-10-06T04:03:18.576 INFO:tasks.workunit.client.0.smithi157.stdout:removed './180/13' 2023-10-06T04:03:18.576 INFO:tasks.workunit.client.0.smithi157.stdout:removed './180/33' 2023-10-06T04:03:18.576 INFO:tasks.workunit.client.0.smithi157.stdout:removed './180/17' 2023-10-06T04:03:18.576 INFO:tasks.workunit.client.0.smithi157.stdout:removed './180/35' 2023-10-06T04:03:18.576 INFO:tasks.workunit.client.0.smithi157.stdout:removed './180/37' 2023-10-06T04:03:18.576 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './180' 2023-10-06T04:03:18.576 INFO:tasks.workunit.client.0.smithi157.stdout:removed './181/19' 2023-10-06T04:03:18.576 INFO:tasks.workunit.client.0.smithi157.stdout:removed './181/15' 2023-10-06T04:03:18.577 INFO:tasks.workunit.client.0.smithi157.stdout:removed './181/31' 2023-10-06T04:03:18.577 INFO:tasks.workunit.client.0.smithi157.stdout:removed './181/36' 2023-10-06T04:03:18.577 INFO:tasks.workunit.client.0.smithi157.stdout:removed './181/20' 2023-10-06T04:03:18.577 INFO:tasks.workunit.client.0.smithi157.stdout:removed './181/12' 2023-10-06T04:03:18.577 INFO:tasks.workunit.client.0.smithi157.stdout:removed './181/39' 2023-10-06T04:03:18.577 INFO:tasks.workunit.client.0.smithi157.stdout:removed './181/10' 2023-10-06T04:03:18.577 INFO:tasks.workunit.client.0.smithi157.stdout:removed './181/34' 2023-10-06T04:03:18.577 INFO:tasks.workunit.client.0.smithi157.stdout:removed './181/38' 2023-10-06T04:03:18.577 INFO:tasks.workunit.client.0.smithi157.stdout:removed './181/11' 2023-10-06T04:03:18.577 INFO:tasks.workunit.client.0.smithi157.stdout:removed './181/16' 2023-10-06T04:03:18.577 INFO:tasks.workunit.client.0.smithi157.stdout:removed './181/32' 2023-10-06T04:03:18.577 INFO:tasks.workunit.client.0.smithi157.stdout:removed './181/30' 2023-10-06T04:03:18.578 INFO:tasks.workunit.client.0.smithi157.stdout:removed './181/14' 2023-10-06T04:03:18.578 INFO:tasks.workunit.client.0.smithi157.stdout:removed './181/18' 2023-10-06T04:03:18.578 INFO:tasks.workunit.client.0.smithi157.stdout:removed './181/13' 2023-10-06T04:03:18.578 INFO:tasks.workunit.client.0.smithi157.stdout:removed './181/33' 2023-10-06T04:03:18.578 INFO:tasks.workunit.client.0.smithi157.stdout:removed './181/17' 2023-10-06T04:03:18.578 INFO:tasks.workunit.client.0.smithi157.stdout:removed './181/35' 2023-10-06T04:03:18.578 INFO:tasks.workunit.client.0.smithi157.stdout:removed './181/37' 2023-10-06T04:03:18.578 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './181' 2023-10-06T04:03:18.578 INFO:tasks.workunit.client.0.smithi157.stdout:removed './182/19' 2023-10-06T04:03:18.578 INFO:tasks.workunit.client.0.smithi157.stdout:removed './182/15' 2023-10-06T04:03:18.578 INFO:tasks.workunit.client.0.smithi157.stdout:removed './182/31' 2023-10-06T04:03:18.579 INFO:tasks.workunit.client.0.smithi157.stdout:removed './182/36' 2023-10-06T04:03:18.579 INFO:tasks.workunit.client.0.smithi157.stdout:removed './182/20' 2023-10-06T04:03:18.579 INFO:tasks.workunit.client.0.smithi157.stdout:removed './182/12' 2023-10-06T04:03:18.579 INFO:tasks.workunit.client.0.smithi157.stdout:removed './182/39' 2023-10-06T04:03:18.579 INFO:tasks.workunit.client.0.smithi157.stdout:removed './182/10' 2023-10-06T04:03:18.579 INFO:tasks.workunit.client.0.smithi157.stdout:removed './182/34' 2023-10-06T04:03:18.579 INFO:tasks.workunit.client.0.smithi157.stdout:removed './182/38' 2023-10-06T04:03:18.579 INFO:tasks.workunit.client.0.smithi157.stdout:removed './182/11' 2023-10-06T04:03:18.579 INFO:tasks.workunit.client.0.smithi157.stdout:removed './182/16' 2023-10-06T04:03:18.579 INFO:tasks.workunit.client.0.smithi157.stdout:removed './182/32' 2023-10-06T04:03:18.579 INFO:tasks.workunit.client.0.smithi157.stdout:removed './182/30' 2023-10-06T04:03:18.580 INFO:tasks.workunit.client.0.smithi157.stdout:removed './182/14' 2023-10-06T04:03:18.580 INFO:tasks.workunit.client.0.smithi157.stdout:removed './182/18' 2023-10-06T04:03:18.580 INFO:tasks.workunit.client.0.smithi157.stdout:removed './182/13' 2023-10-06T04:03:18.580 INFO:tasks.workunit.client.0.smithi157.stdout:removed './182/33' 2023-10-06T04:03:18.580 INFO:tasks.workunit.client.0.smithi157.stdout:removed './182/17' 2023-10-06T04:03:18.580 INFO:tasks.workunit.client.0.smithi157.stdout:removed './182/35' 2023-10-06T04:03:18.580 INFO:tasks.workunit.client.0.smithi157.stdout:removed './182/37' 2023-10-06T04:03:18.580 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './182' 2023-10-06T04:03:18.580 INFO:tasks.workunit.client.0.smithi157.stdout:removed './183/19' 2023-10-06T04:03:18.580 INFO:tasks.workunit.client.0.smithi157.stdout:removed './183/15' 2023-10-06T04:03:18.580 INFO:tasks.workunit.client.0.smithi157.stdout:removed './183/31' 2023-10-06T04:03:18.580 INFO:tasks.workunit.client.0.smithi157.stdout:removed './183/36' 2023-10-06T04:03:18.581 INFO:tasks.workunit.client.0.smithi157.stdout:removed './183/20' 2023-10-06T04:03:18.581 INFO:tasks.workunit.client.0.smithi157.stdout:removed './183/12' 2023-10-06T04:03:18.581 INFO:tasks.workunit.client.0.smithi157.stdout:removed './183/39' 2023-10-06T04:03:18.581 INFO:tasks.workunit.client.0.smithi157.stdout:removed './183/10' 2023-10-06T04:03:18.581 INFO:tasks.workunit.client.0.smithi157.stdout:removed './183/34' 2023-10-06T04:03:18.581 INFO:tasks.workunit.client.0.smithi157.stdout:removed './183/38' 2023-10-06T04:03:18.581 INFO:tasks.workunit.client.0.smithi157.stdout:removed './183/11' 2023-10-06T04:03:18.581 INFO:tasks.workunit.client.0.smithi157.stdout:removed './183/16' 2023-10-06T04:03:18.581 INFO:tasks.workunit.client.0.smithi157.stdout:removed './183/32' 2023-10-06T04:03:18.581 INFO:tasks.workunit.client.0.smithi157.stdout:removed './183/30' 2023-10-06T04:03:18.581 INFO:tasks.workunit.client.0.smithi157.stdout:removed './183/14' 2023-10-06T04:03:18.582 INFO:tasks.workunit.client.0.smithi157.stdout:removed './183/18' 2023-10-06T04:03:18.582 INFO:tasks.workunit.client.0.smithi157.stdout:removed './183/13' 2023-10-06T04:03:18.582 INFO:tasks.workunit.client.0.smithi157.stdout:removed './183/33' 2023-10-06T04:03:18.582 INFO:tasks.workunit.client.0.smithi157.stdout:removed './183/17' 2023-10-06T04:03:18.582 INFO:tasks.workunit.client.0.smithi157.stdout:removed './183/35' 2023-10-06T04:03:18.582 INFO:tasks.workunit.client.0.smithi157.stdout:removed './183/37' 2023-10-06T04:03:18.582 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './183' 2023-10-06T04:03:18.582 INFO:tasks.workunit.client.0.smithi157.stdout:removed './184/19' 2023-10-06T04:03:18.582 INFO:tasks.workunit.client.0.smithi157.stdout:removed './184/15' 2023-10-06T04:03:18.582 INFO:tasks.workunit.client.0.smithi157.stdout:removed './184/31' 2023-10-06T04:03:18.582 INFO:tasks.workunit.client.0.smithi157.stdout:removed './184/36' 2023-10-06T04:03:18.582 INFO:tasks.workunit.client.0.smithi157.stdout:removed './184/20' 2023-10-06T04:03:18.583 INFO:tasks.workunit.client.0.smithi157.stdout:removed './184/12' 2023-10-06T04:03:18.583 INFO:tasks.workunit.client.0.smithi157.stdout:removed './184/39' 2023-10-06T04:03:18.583 INFO:tasks.workunit.client.0.smithi157.stdout:removed './184/10' 2023-10-06T04:03:18.583 INFO:tasks.workunit.client.0.smithi157.stdout:removed './184/34' 2023-10-06T04:03:18.583 INFO:tasks.workunit.client.0.smithi157.stdout:removed './184/38' 2023-10-06T04:03:18.583 INFO:tasks.workunit.client.0.smithi157.stdout:removed './184/11' 2023-10-06T04:03:18.583 INFO:tasks.workunit.client.0.smithi157.stdout:removed './184/16' 2023-10-06T04:03:18.583 INFO:tasks.workunit.client.0.smithi157.stdout:removed './184/32' 2023-10-06T04:03:18.583 INFO:tasks.workunit.client.0.smithi157.stdout:removed './184/30' 2023-10-06T04:03:18.583 INFO:tasks.workunit.client.0.smithi157.stdout:removed './184/14' 2023-10-06T04:03:18.584 INFO:tasks.workunit.client.0.smithi157.stdout:removed './184/18' 2023-10-06T04:03:18.584 INFO:tasks.workunit.client.0.smithi157.stdout:removed './184/13' 2023-10-06T04:03:18.584 INFO:tasks.workunit.client.0.smithi157.stdout:removed './184/33' 2023-10-06T04:03:18.584 INFO:tasks.workunit.client.0.smithi157.stdout:removed './184/17' 2023-10-06T04:03:18.584 INFO:tasks.workunit.client.0.smithi157.stdout:removed './184/35' 2023-10-06T04:03:18.584 INFO:tasks.workunit.client.0.smithi157.stdout:removed './184/37' 2023-10-06T04:03:18.584 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './184' 2023-10-06T04:03:18.584 INFO:tasks.workunit.client.0.smithi157.stdout:removed './185/19' 2023-10-06T04:03:18.584 INFO:tasks.workunit.client.0.smithi157.stdout:removed './185/15' 2023-10-06T04:03:18.585 INFO:tasks.workunit.client.0.smithi157.stdout:removed './185/31' 2023-10-06T04:03:18.585 INFO:tasks.workunit.client.0.smithi157.stdout:removed './185/36' 2023-10-06T04:03:18.585 INFO:tasks.workunit.client.0.smithi157.stdout:removed './185/20' 2023-10-06T04:03:18.585 INFO:tasks.workunit.client.0.smithi157.stdout:removed './185/12' 2023-10-06T04:03:18.585 INFO:tasks.workunit.client.0.smithi157.stdout:removed './185/39' 2023-10-06T04:03:18.585 INFO:tasks.workunit.client.0.smithi157.stdout:removed './185/10' 2023-10-06T04:03:18.585 INFO:tasks.workunit.client.0.smithi157.stdout:removed './185/34' 2023-10-06T04:03:18.585 INFO:tasks.workunit.client.0.smithi157.stdout:removed './185/38' 2023-10-06T04:03:18.585 INFO:tasks.workunit.client.0.smithi157.stdout:removed './185/11' 2023-10-06T04:03:18.585 INFO:tasks.workunit.client.0.smithi157.stdout:removed './185/16' 2023-10-06T04:03:18.585 INFO:tasks.workunit.client.0.smithi157.stdout:removed './185/32' 2023-10-06T04:03:18.585 INFO:tasks.workunit.client.0.smithi157.stdout:removed './185/30' 2023-10-06T04:03:18.586 INFO:tasks.workunit.client.0.smithi157.stdout:removed './185/14' 2023-10-06T04:03:18.586 INFO:tasks.workunit.client.0.smithi157.stdout:removed './185/18' 2023-10-06T04:03:18.586 INFO:tasks.workunit.client.0.smithi157.stdout:removed './185/13' 2023-10-06T04:03:18.586 INFO:tasks.workunit.client.0.smithi157.stdout:removed './185/33' 2023-10-06T04:03:18.586 INFO:tasks.workunit.client.0.smithi157.stdout:removed './185/17' 2023-10-06T04:03:18.586 INFO:tasks.workunit.client.0.smithi157.stdout:removed './185/35' 2023-10-06T04:03:18.586 INFO:tasks.workunit.client.0.smithi157.stdout:removed './185/37' 2023-10-06T04:03:18.586 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './185' 2023-10-06T04:03:18.587 INFO:tasks.workunit.client.0.smithi157.stdout:removed './186/19' 2023-10-06T04:03:18.587 INFO:tasks.workunit.client.0.smithi157.stdout:removed './186/15' 2023-10-06T04:03:18.587 INFO:tasks.workunit.client.0.smithi157.stdout:removed './186/31' 2023-10-06T04:03:18.587 INFO:tasks.workunit.client.0.smithi157.stdout:removed './186/36' 2023-10-06T04:03:18.587 INFO:tasks.workunit.client.0.smithi157.stdout:removed './186/20' 2023-10-06T04:03:18.587 INFO:tasks.workunit.client.0.smithi157.stdout:removed './186/12' 2023-10-06T04:03:18.587 INFO:tasks.workunit.client.0.smithi157.stdout:removed './186/39' 2023-10-06T04:03:18.587 INFO:tasks.workunit.client.0.smithi157.stdout:removed './186/10' 2023-10-06T04:03:18.587 INFO:tasks.workunit.client.0.smithi157.stdout:removed './186/34' 2023-10-06T04:03:18.587 INFO:tasks.workunit.client.0.smithi157.stdout:removed './186/38' 2023-10-06T04:03:18.587 INFO:tasks.workunit.client.0.smithi157.stdout:removed './186/11' 2023-10-06T04:03:18.587 INFO:tasks.workunit.client.0.smithi157.stdout:removed './186/16' 2023-10-06T04:03:18.587 INFO:tasks.workunit.client.0.smithi157.stdout:removed './186/32' 2023-10-06T04:03:18.588 INFO:tasks.workunit.client.0.smithi157.stdout:removed './186/30' 2023-10-06T04:03:18.588 INFO:tasks.workunit.client.0.smithi157.stdout:removed './186/14' 2023-10-06T04:03:18.588 INFO:tasks.workunit.client.0.smithi157.stdout:removed './186/18' 2023-10-06T04:03:18.775 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:03:18.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.889+0000 7fbfb0e82700 1 Processor -- start 2023-10-06T04:03:18.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.889+0000 7fbfb0e82700 1 -- start start 2023-10-06T04:03:18.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.890+0000 7fbfb0e82700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbfac0b1e40 0x7fbfac157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:18.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.890+0000 7fbfb0e82700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbfac0b04d0 0x7fbfac06dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:18.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.890+0000 7fbfb0e82700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfac06d8d0 con 0x7fbfac06c2d0 2023-10-06T04:03:18.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.890+0000 7fbfb0e82700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfac0af8f0 con 0x7fbfac0b04d0 2023-10-06T04:03:18.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.890+0000 7fbfb0e82700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfac15a690 con 0x7fbfac0b1e40 2023-10-06T04:03:18.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.890+0000 7fbfaaffd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbfac0b04d0 0x7fbfac06dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:18.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.890+0000 7fbfabfff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fbfac06c2d0 0x7fbfac1563b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:52786/0 (socket says 172.21.15.157:52786) 2023-10-06T04:03:18.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.890+0000 7fbfabfff700 1 -- 172.21.15.157:0/2703568576 learned_addr learned my addr 172.21.15.157:0/2703568576 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:03:18.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.890+0000 7fbfab7fe700 1 --2- 172.21.15.157:0/2703568576 >> v2:172.21.15.175:3300/0 conn(0x7fbfac0b1e40 0x7fbfac157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:18.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.890+0000 7fbfaaffd700 1 -- 172.21.15.157:0/2703568576 >> v2:172.21.15.175:3300/0 conn(0x7fbfac0b1e40 msgr2=0x7fbfac157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:18.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.890+0000 7fbfaaffd700 1 --2- 172.21.15.157:0/2703568576 >> v2:172.21.15.175:3300/0 conn(0x7fbfac0b1e40 0x7fbfac157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:18.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.890+0000 7fbfaaffd700 1 -- 172.21.15.157:0/2703568576 >> v1:172.21.15.157:6789/0 conn(0x7fbfac06c2d0 legacy=0x7fbfac1563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:18.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.890+0000 7fbfaaffd700 1 -- 172.21.15.157:0/2703568576 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfac06ccc0 con 0x7fbfac0b04d0 2023-10-06T04:03:18.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.890+0000 7fbfaa7fc700 1 -- 172.21.15.157:0/2703568576 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3522206781 0 0) 0x7fbfac06d8d0 con 0x7fbfac06c2d0 2023-10-06T04:03:18.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.891+0000 7fbfaaffd700 1 --2- 172.21.15.157:0/2703568576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfac0b04d0 0x7fbfac06dd60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fbfac0cc450 tx=0x7fbfa00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c798bedbdbd4db2c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:18.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.891+0000 7fbfaa7fc700 1 -- 172.21.15.157:0/2703568576 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa000b6b0 con 0x7fbfac0b04d0 2023-10-06T04:03:18.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.891+0000 7fbfaa7fc700 1 -- 172.21.15.157:0/2703568576 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa000b810 con 0x7fbfac0b04d0 2023-10-06T04:03:18.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.891+0000 7fbfaa7fc700 1 -- 172.21.15.157:0/2703568576 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa000bbb0 con 0x7fbfac0b04d0 2023-10-06T04:03:18.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.891+0000 7fbfb0e82700 1 -- 172.21.15.157:0/2703568576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfac0b04d0 msgr2=0x7fbfac06dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:18.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.891+0000 7fbfb0e82700 1 --2- 172.21.15.157:0/2703568576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfac0b04d0 0x7fbfac06dd60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fbfac0cc450 tx=0x7fbfa00091a0 comp rx=0 tx=0).stop 2023-10-06T04:03:18.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.891+0000 7fbfb0e82700 1 -- 172.21.15.157:0/2703568576 shutdown_connections 2023-10-06T04:03:18.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.891+0000 7fbfb0e82700 1 --2- 172.21.15.157:0/2703568576 >> v2:172.21.15.175:3300/0 conn(0x7fbfac0b1e40 0x7fbfac157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:18.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.891+0000 7fbfb0e82700 1 --2- 172.21.15.157:0/2703568576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfac0b04d0 0x7fbfac06dd60 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:18.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.891+0000 7fbfb0e82700 1 -- 172.21.15.157:0/2703568576 >> 172.21.15.157:0/2703568576 conn(0x7fbfac0cc540 msgr2=0x7fbfac0cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:18.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.891+0000 7fbfb0e82700 1 -- 172.21.15.157:0/2703568576 shutdown_connections 2023-10-06T04:03:18.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.891+0000 7fbfb0e82700 1 -- 172.21.15.157:0/2703568576 wait complete. 2023-10-06T04:03:18.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.892+0000 7fbfb0e82700 1 Processor -- start 2023-10-06T04:03:18.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.892+0000 7fbfb0e82700 1 -- start start 2023-10-06T04:03:18.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.893+0000 7fbfb0e82700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfac06c2d0 0x7fbfac166250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:18.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.893+0000 7fbfb0e82700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfac0b04d0 0x7fbfac166790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:18.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.893+0000 7fbfb0e82700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfac0b1e40 0x7fbfac14f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:18.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.893+0000 7fbfb0e82700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfac151500 con 0x7fbfac0b1e40 2023-10-06T04:03:18.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.893+0000 7fbfb0e82700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfac14ec00 con 0x7fbfac06c2d0 2023-10-06T04:03:18.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.893+0000 7fbfb0e82700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfac166de0 con 0x7fbfac0b04d0 2023-10-06T04:03:18.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.893+0000 7fbfabfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfac0b1e40 0x7fbfac14f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:18.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.893+0000 7fbfabfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfac0b1e40 0x7fbfac14f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45872/0 (socket says 172.21.15.157:45872) 2023-10-06T04:03:18.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.893+0000 7fbfabfff700 1 -- 172.21.15.157:0/3555395245 learned_addr learned my addr 172.21.15.157:0/3555395245 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:18.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.893+0000 7fbfaaffd700 1 --2- 172.21.15.157:0/3555395245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfac0b04d0 0x7fbfac166790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:18.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.893+0000 7fbfab7fe700 1 --2- 172.21.15.157:0/3555395245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfac06c2d0 0x7fbfac166250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:18.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.894+0000 7fbfabfff700 1 -- 172.21.15.157:0/3555395245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfac0b04d0 msgr2=0x7fbfac166790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:18.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.894+0000 7fbfabfff700 1 --2- 172.21.15.157:0/3555395245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfac0b04d0 0x7fbfac166790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:18.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.894+0000 7fbfabfff700 1 -- 172.21.15.157:0/3555395245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfac06c2d0 msgr2=0x7fbfac166250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:18.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.894+0000 7fbfabfff700 1 --2- 172.21.15.157:0/3555395245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfac06c2d0 0x7fbfac166250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:18.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.894+0000 7fbfabfff700 1 -- 172.21.15.157:0/3555395245 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa0009420 con 0x7fbfac0b1e40 2023-10-06T04:03:18.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.894+0000 7fbfabfff700 1 --2- 172.21.15.157:0/3555395245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfac0b1e40 0x7fbfac14f090 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fbfa400c530 tx=0x7fbfa4007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:18.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.894+0000 7fbf937fe700 1 -- 172.21.15.157:0/3555395245 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4005b30 con 0x7fbfac0b1e40 2023-10-06T04:03:18.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.894+0000 7fbf937fe700 1 -- 172.21.15.157:0/3555395245 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa4005c90 con 0x7fbfac0b1e40 2023-10-06T04:03:18.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.894+0000 7fbfb0e82700 1 -- 172.21.15.157:0/3555395245 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfac150f20 con 0x7fbfac0b1e40 2023-10-06T04:03:18.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.894+0000 7fbf937fe700 1 -- 172.21.15.157:0/3555395245 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4007320 con 0x7fbfac0b1e40 2023-10-06T04:03:18.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.894+0000 7fbfb0e82700 1 -- 172.21.15.157:0/3555395245 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfac14e890 con 0x7fbfac0b1e40 2023-10-06T04:03:18.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.895+0000 7fbfb0e82700 1 -- 172.21.15.157:0/3555395245 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfac14ed70 con 0x7fbfac0b1e40 2023-10-06T04:03:18.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.895+0000 7fbf937fe700 1 -- 172.21.15.157:0/3555395245 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbfa401b3b0 con 0x7fbfac0b1e40 2023-10-06T04:03:18.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.896+0000 7fbf937fe700 1 --2- 172.21.15.157:0/3555395245 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbf98066f60 0x7fbf98069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:18.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.896+0000 7fbf937fe700 1 -- 172.21.15.157:0/3555395245 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7fbfa4083c60 con 0x7fbfac0b1e40 2023-10-06T04:03:18.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.896+0000 7fbfab7fe700 1 --2- 172.21.15.157:0/3555395245 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbf98066f60 0x7fbf98069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:18.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.896+0000 7fbfab7fe700 1 --2- 172.21.15.157:0/3555395245 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbf98066f60 0x7fbf98069410 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c009fd0 tx=0x7fbf9c009880 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:18.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:18.900+0000 7fbf937fe700 1 -- 172.21.15.157:0/3555395245 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbfa4050b20 con 0x7fbfac0b1e40 2023-10-06T04:03:19.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:19.136+0000 7fbfb0e82700 1 -- 172.21.15.157:0/3555395245 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbfac0afe00 con 0x7fbfac0b1e40 2023-10-06T04:03:19.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:19.136+0000 7fbf937fe700 1 -- 172.21.15.157:0/3555395245 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+6162 (secure 0 0 0) 0x7fbfa4017020 con 0x7fbfac0b1e40 2023-10-06T04:03:19.138 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:03:19.138 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3192541484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3192541484},{"type":"v1","addr":"172.21.15.175:6835","nonce":3192541484}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:13.713593+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279},"failed":[],"damaged":[],"stopped":[3,4],"info":{"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:stopping","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:03:19.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:19.138+0000 7fbfb0e82700 1 -- 172.21.15.157:0/3555395245 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbf98066f60 msgr2=0x7fbf98069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:19.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:19.138+0000 7fbfb0e82700 1 --2- 172.21.15.157:0/3555395245 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbf98066f60 0x7fbf98069410 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c009fd0 tx=0x7fbf9c009880 comp rx=0 tx=0).stop 2023-10-06T04:03:19.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:19.138+0000 7fbfb0e82700 1 -- 172.21.15.157:0/3555395245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfac0b1e40 msgr2=0x7fbfac14f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:19.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:19.138+0000 7fbfb0e82700 1 --2- 172.21.15.157:0/3555395245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfac0b1e40 0x7fbfac14f090 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fbfa400c530 tx=0x7fbfa4007f90 comp rx=0 tx=0).stop 2023-10-06T04:03:19.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:19.138+0000 7fbfb0e82700 1 -- 172.21.15.157:0/3555395245 shutdown_connections 2023-10-06T04:03:19.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:19.138+0000 7fbfb0e82700 1 --2- 172.21.15.157:0/3555395245 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbf98066f60 0x7fbf98069410 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:19.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:19.138+0000 7fbfb0e82700 1 --2- 172.21.15.157:0/3555395245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfac06c2d0 0x7fbfac166250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:19.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:19.138+0000 7fbfb0e82700 1 --2- 172.21.15.157:0/3555395245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfac0b1e40 0x7fbfac14f090 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:19.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:19.138+0000 7fbfb0e82700 1 --2- 172.21.15.157:0/3555395245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfac0b04d0 0x7fbfac166790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:19.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:19.138+0000 7fbfb0e82700 1 -- 172.21.15.157:0/3555395245 >> 172.21.15.157:0/3555395245 conn(0x7fbfac0cc540 msgr2=0x7fbfac157440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:19.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:19.138+0000 7fbfb0e82700 1 -- 172.21.15.157:0/3555395245 shutdown_connections 2023-10-06T04:03:19.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:19.138+0000 7fbfb0e82700 1 -- 172.21.15.157:0/3555395245 wait complete. 2023-10-06T04:03:19.141 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 51 2023-10-06T04:03:19.152 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:03:20.213 INFO:tasks.workunit.client.0.smithi157.stdout:removed './186/13' 2023-10-06T04:03:20.214 INFO:tasks.workunit.client.0.smithi157.stdout:removed './186/33' 2023-10-06T04:03:20.214 INFO:tasks.workunit.client.0.smithi157.stdout:removed './186/17' 2023-10-06T04:03:20.214 INFO:tasks.workunit.client.0.smithi157.stdout:removed './186/35' 2023-10-06T04:03:20.214 INFO:tasks.workunit.client.0.smithi157.stdout:removed './186/37' 2023-10-06T04:03:20.214 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './186' 2023-10-06T04:03:20.214 INFO:tasks.workunit.client.0.smithi157.stdout:removed './187/19' 2023-10-06T04:03:20.214 INFO:tasks.workunit.client.0.smithi157.stdout:removed './187/15' 2023-10-06T04:03:20.214 INFO:tasks.workunit.client.0.smithi157.stdout:removed './187/31' 2023-10-06T04:03:20.214 INFO:tasks.workunit.client.0.smithi157.stdout:removed './187/36' 2023-10-06T04:03:20.214 INFO:tasks.workunit.client.0.smithi157.stdout:removed './187/20' 2023-10-06T04:03:20.215 INFO:tasks.workunit.client.0.smithi157.stdout:removed './187/12' 2023-10-06T04:03:20.215 INFO:tasks.workunit.client.0.smithi157.stdout:removed './187/39' 2023-10-06T04:03:20.215 INFO:tasks.workunit.client.0.smithi157.stdout:removed './187/10' 2023-10-06T04:03:20.215 INFO:tasks.workunit.client.0.smithi157.stdout:removed './187/34' 2023-10-06T04:03:20.215 INFO:tasks.workunit.client.0.smithi157.stdout:removed './187/38' 2023-10-06T04:03:20.215 INFO:tasks.workunit.client.0.smithi157.stdout:removed './187/11' 2023-10-06T04:03:20.215 INFO:tasks.workunit.client.0.smithi157.stdout:removed './187/16' 2023-10-06T04:03:20.215 INFO:tasks.workunit.client.0.smithi157.stdout:removed './187/32' 2023-10-06T04:03:20.215 INFO:tasks.workunit.client.0.smithi157.stdout:removed './187/30' 2023-10-06T04:03:20.215 INFO:tasks.workunit.client.0.smithi157.stdout:removed './187/14' 2023-10-06T04:03:20.215 INFO:tasks.workunit.client.0.smithi157.stdout:removed './187/18' 2023-10-06T04:03:20.215 INFO:tasks.workunit.client.0.smithi157.stdout:removed './187/13' 2023-10-06T04:03:20.216 INFO:tasks.workunit.client.0.smithi157.stdout:removed './187/33' 2023-10-06T04:03:20.216 INFO:tasks.workunit.client.0.smithi157.stdout:removed './187/17' 2023-10-06T04:03:20.216 INFO:tasks.workunit.client.0.smithi157.stdout:removed './187/35' 2023-10-06T04:03:20.216 INFO:tasks.workunit.client.0.smithi157.stdout:removed './187/37' 2023-10-06T04:03:20.216 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './187' 2023-10-06T04:03:20.216 INFO:tasks.workunit.client.0.smithi157.stdout:removed './188/19' 2023-10-06T04:03:20.216 INFO:tasks.workunit.client.0.smithi157.stdout:removed './188/15' 2023-10-06T04:03:20.216 INFO:tasks.workunit.client.0.smithi157.stdout:removed './188/31' 2023-10-06T04:03:20.216 INFO:tasks.workunit.client.0.smithi157.stdout:removed './188/36' 2023-10-06T04:03:20.216 INFO:tasks.workunit.client.0.smithi157.stdout:removed './188/20' 2023-10-06T04:03:20.216 INFO:tasks.workunit.client.0.smithi157.stdout:removed './188/12' 2023-10-06T04:03:20.216 INFO:tasks.workunit.client.0.smithi157.stdout:removed './188/39' 2023-10-06T04:03:20.217 INFO:tasks.workunit.client.0.smithi157.stdout:removed './188/10' 2023-10-06T04:03:20.217 INFO:tasks.workunit.client.0.smithi157.stdout:removed './188/34' 2023-10-06T04:03:20.217 INFO:tasks.workunit.client.0.smithi157.stdout:removed './188/38' 2023-10-06T04:03:20.217 INFO:tasks.workunit.client.0.smithi157.stdout:removed './188/11' 2023-10-06T04:03:20.217 INFO:tasks.workunit.client.0.smithi157.stdout:removed './188/16' 2023-10-06T04:03:20.217 INFO:tasks.workunit.client.0.smithi157.stdout:removed './188/32' 2023-10-06T04:03:20.217 INFO:tasks.workunit.client.0.smithi157.stdout:removed './188/30' 2023-10-06T04:03:20.217 INFO:tasks.workunit.client.0.smithi157.stdout:removed './188/14' 2023-10-06T04:03:20.217 INFO:tasks.workunit.client.0.smithi157.stdout:removed './188/18' 2023-10-06T04:03:20.217 INFO:tasks.workunit.client.0.smithi157.stdout:removed './188/13' 2023-10-06T04:03:20.217 INFO:tasks.workunit.client.0.smithi157.stdout:removed './188/33' 2023-10-06T04:03:20.218 INFO:tasks.workunit.client.0.smithi157.stdout:removed './188/17' 2023-10-06T04:03:20.218 INFO:tasks.workunit.client.0.smithi157.stdout:removed './188/35' 2023-10-06T04:03:20.218 INFO:tasks.workunit.client.0.smithi157.stdout:removed './188/37' 2023-10-06T04:03:20.218 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './188' 2023-10-06T04:03:20.218 INFO:tasks.workunit.client.0.smithi157.stdout:removed './189/19' 2023-10-06T04:03:20.218 INFO:tasks.workunit.client.0.smithi157.stdout:removed './189/15' 2023-10-06T04:03:20.218 INFO:tasks.workunit.client.0.smithi157.stdout:removed './189/31' 2023-10-06T04:03:20.218 INFO:tasks.workunit.client.0.smithi157.stdout:removed './189/36' 2023-10-06T04:03:20.218 INFO:tasks.workunit.client.0.smithi157.stdout:removed './189/20' 2023-10-06T04:03:20.218 INFO:tasks.workunit.client.0.smithi157.stdout:removed './189/12' 2023-10-06T04:03:20.218 INFO:tasks.workunit.client.0.smithi157.stdout:removed './189/39' 2023-10-06T04:03:20.218 INFO:tasks.workunit.client.0.smithi157.stdout:removed './189/10' 2023-10-06T04:03:20.219 INFO:tasks.workunit.client.0.smithi157.stdout:removed './189/34' 2023-10-06T04:03:20.219 INFO:tasks.workunit.client.0.smithi157.stdout:removed './189/38' 2023-10-06T04:03:20.219 INFO:tasks.workunit.client.0.smithi157.stdout:removed './189/11' 2023-10-06T04:03:20.219 INFO:tasks.workunit.client.0.smithi157.stdout:removed './189/16' 2023-10-06T04:03:20.219 INFO:tasks.workunit.client.0.smithi157.stdout:removed './189/32' 2023-10-06T04:03:20.219 INFO:tasks.workunit.client.0.smithi157.stdout:removed './189/30' 2023-10-06T04:03:20.219 INFO:tasks.workunit.client.0.smithi157.stdout:removed './189/14' 2023-10-06T04:03:20.219 INFO:tasks.workunit.client.0.smithi157.stdout:removed './189/18' 2023-10-06T04:03:20.219 INFO:tasks.workunit.client.0.smithi157.stdout:removed './189/13' 2023-10-06T04:03:20.219 INFO:tasks.workunit.client.0.smithi157.stdout:removed './189/33' 2023-10-06T04:03:20.219 INFO:tasks.workunit.client.0.smithi157.stdout:removed './189/17' 2023-10-06T04:03:20.219 INFO:tasks.workunit.client.0.smithi157.stdout:removed './189/35' 2023-10-06T04:03:20.220 INFO:tasks.workunit.client.0.smithi157.stdout:removed './189/37' 2023-10-06T04:03:20.220 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './189' 2023-10-06T04:03:20.220 INFO:tasks.workunit.client.0.smithi157.stdout:removed './190/19' 2023-10-06T04:03:20.220 INFO:tasks.workunit.client.0.smithi157.stdout:removed './190/15' 2023-10-06T04:03:20.220 INFO:tasks.workunit.client.0.smithi157.stdout:removed './190/31' 2023-10-06T04:03:20.220 INFO:tasks.workunit.client.0.smithi157.stdout:removed './190/36' 2023-10-06T04:03:20.220 INFO:tasks.workunit.client.0.smithi157.stdout:removed './190/20' 2023-10-06T04:03:20.220 INFO:tasks.workunit.client.0.smithi157.stdout:removed './190/12' 2023-10-06T04:03:20.220 INFO:tasks.workunit.client.0.smithi157.stdout:removed './190/39' 2023-10-06T04:03:20.220 INFO:tasks.workunit.client.0.smithi157.stdout:removed './190/10' 2023-10-06T04:03:20.220 INFO:tasks.workunit.client.0.smithi157.stdout:removed './190/34' 2023-10-06T04:03:20.221 INFO:tasks.workunit.client.0.smithi157.stdout:removed './190/38' 2023-10-06T04:03:20.221 INFO:tasks.workunit.client.0.smithi157.stdout:removed './190/11' 2023-10-06T04:03:20.221 INFO:tasks.workunit.client.0.smithi157.stdout:removed './190/16' 2023-10-06T04:03:20.221 INFO:tasks.workunit.client.0.smithi157.stdout:removed './190/32' 2023-10-06T04:03:20.221 INFO:tasks.workunit.client.0.smithi157.stdout:removed './190/30' 2023-10-06T04:03:20.221 INFO:tasks.workunit.client.0.smithi157.stdout:removed './190/14' 2023-10-06T04:03:20.221 INFO:tasks.workunit.client.0.smithi157.stdout:removed './190/18' 2023-10-06T04:03:20.221 INFO:tasks.workunit.client.0.smithi157.stdout:removed './190/13' 2023-10-06T04:03:20.221 INFO:tasks.workunit.client.0.smithi157.stdout:removed './190/33' 2023-10-06T04:03:20.221 INFO:tasks.workunit.client.0.smithi157.stdout:removed './190/17' 2023-10-06T04:03:20.221 INFO:tasks.workunit.client.0.smithi157.stdout:removed './190/35' 2023-10-06T04:03:20.222 INFO:tasks.workunit.client.0.smithi157.stdout:removed './190/37' 2023-10-06T04:03:20.222 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './190' 2023-10-06T04:03:20.222 INFO:tasks.workunit.client.0.smithi157.stdout:removed './191/19' 2023-10-06T04:03:20.222 INFO:tasks.workunit.client.0.smithi157.stdout:removed './191/15' 2023-10-06T04:03:20.222 INFO:tasks.workunit.client.0.smithi157.stdout:removed './191/31' 2023-10-06T04:03:20.222 INFO:tasks.workunit.client.0.smithi157.stdout:removed './191/36' 2023-10-06T04:03:20.222 INFO:tasks.workunit.client.0.smithi157.stdout:removed './191/20' 2023-10-06T04:03:20.222 INFO:tasks.workunit.client.0.smithi157.stdout:removed './191/12' 2023-10-06T04:03:20.222 INFO:tasks.workunit.client.0.smithi157.stdout:removed './191/39' 2023-10-06T04:03:20.222 INFO:tasks.workunit.client.0.smithi157.stdout:removed './191/10' 2023-10-06T04:03:20.222 INFO:tasks.workunit.client.0.smithi157.stdout:removed './191/34' 2023-10-06T04:03:20.223 INFO:tasks.workunit.client.0.smithi157.stdout:removed './191/38' 2023-10-06T04:03:20.223 INFO:tasks.workunit.client.0.smithi157.stdout:removed './191/11' 2023-10-06T04:03:20.223 INFO:tasks.workunit.client.0.smithi157.stdout:removed './191/16' 2023-10-06T04:03:20.223 INFO:tasks.workunit.client.0.smithi157.stdout:removed './191/32' 2023-10-06T04:03:20.223 INFO:tasks.workunit.client.0.smithi157.stdout:removed './191/30' 2023-10-06T04:03:20.223 INFO:tasks.workunit.client.0.smithi157.stdout:removed './191/14' 2023-10-06T04:03:20.223 INFO:tasks.workunit.client.0.smithi157.stdout:removed './191/18' 2023-10-06T04:03:20.223 INFO:tasks.workunit.client.0.smithi157.stdout:removed './191/13' 2023-10-06T04:03:20.223 INFO:tasks.workunit.client.0.smithi157.stdout:removed './191/33' 2023-10-06T04:03:20.223 INFO:tasks.workunit.client.0.smithi157.stdout:removed './191/17' 2023-10-06T04:03:20.223 INFO:tasks.workunit.client.0.smithi157.stdout:removed './191/35' 2023-10-06T04:03:20.224 INFO:tasks.workunit.client.0.smithi157.stdout:removed './191/37' 2023-10-06T04:03:20.224 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './191' 2023-10-06T04:03:20.224 INFO:tasks.workunit.client.0.smithi157.stdout:removed './192/19' 2023-10-06T04:03:20.224 INFO:tasks.workunit.client.0.smithi157.stdout:removed './192/15' 2023-10-06T04:03:20.224 INFO:tasks.workunit.client.0.smithi157.stdout:removed './192/31' 2023-10-06T04:03:20.224 INFO:tasks.workunit.client.0.smithi157.stdout:removed './192/36' 2023-10-06T04:03:20.224 INFO:tasks.workunit.client.0.smithi157.stdout:removed './192/20' 2023-10-06T04:03:20.224 INFO:tasks.workunit.client.0.smithi157.stdout:removed './192/12' 2023-10-06T04:03:20.224 INFO:tasks.workunit.client.0.smithi157.stdout:removed './192/39' 2023-10-06T04:03:20.224 INFO:tasks.workunit.client.0.smithi157.stdout:removed './192/10' 2023-10-06T04:03:20.224 INFO:tasks.workunit.client.0.smithi157.stdout:removed './192/34' 2023-10-06T04:03:20.224 INFO:tasks.workunit.client.0.smithi157.stdout:removed './192/38' 2023-10-06T04:03:20.225 INFO:tasks.workunit.client.0.smithi157.stdout:removed './192/11' 2023-10-06T04:03:20.225 INFO:tasks.workunit.client.0.smithi157.stdout:removed './192/16' 2023-10-06T04:03:20.225 INFO:tasks.workunit.client.0.smithi157.stdout:removed './192/32' 2023-10-06T04:03:20.225 INFO:tasks.workunit.client.0.smithi157.stdout:removed './192/30' 2023-10-06T04:03:20.225 INFO:tasks.workunit.client.0.smithi157.stdout:removed './192/14' 2023-10-06T04:03:20.225 INFO:tasks.workunit.client.0.smithi157.stdout:removed './192/18' 2023-10-06T04:03:20.225 INFO:tasks.workunit.client.0.smithi157.stdout:removed './192/13' 2023-10-06T04:03:20.225 INFO:tasks.workunit.client.0.smithi157.stdout:removed './192/33' 2023-10-06T04:03:20.225 INFO:tasks.workunit.client.0.smithi157.stdout:removed './192/17' 2023-10-06T04:03:20.225 INFO:tasks.workunit.client.0.smithi157.stdout:removed './192/35' 2023-10-06T04:03:20.225 INFO:tasks.workunit.client.0.smithi157.stdout:removed './192/37' 2023-10-06T04:03:20.226 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './192' 2023-10-06T04:03:20.226 INFO:tasks.workunit.client.0.smithi157.stdout:removed './193/19' 2023-10-06T04:03:20.226 INFO:tasks.workunit.client.0.smithi157.stdout:removed './193/15' 2023-10-06T04:03:20.226 INFO:tasks.workunit.client.0.smithi157.stdout:removed './193/31' 2023-10-06T04:03:20.226 INFO:tasks.workunit.client.0.smithi157.stdout:removed './193/36' 2023-10-06T04:03:20.226 INFO:tasks.workunit.client.0.smithi157.stdout:removed './193/20' 2023-10-06T04:03:20.226 INFO:tasks.workunit.client.0.smithi157.stdout:removed './193/12' 2023-10-06T04:03:20.226 INFO:tasks.workunit.client.0.smithi157.stdout:removed './193/39' 2023-10-06T04:03:20.226 INFO:tasks.workunit.client.0.smithi157.stdout:removed './193/10' 2023-10-06T04:03:20.226 INFO:tasks.workunit.client.0.smithi157.stdout:removed './193/34' 2023-10-06T04:03:20.226 INFO:tasks.workunit.client.0.smithi157.stdout:removed './193/38' 2023-10-06T04:03:20.226 INFO:tasks.workunit.client.0.smithi157.stdout:removed './193/11' 2023-10-06T04:03:20.227 INFO:tasks.workunit.client.0.smithi157.stdout:removed './193/16' 2023-10-06T04:03:20.227 INFO:tasks.workunit.client.0.smithi157.stdout:removed './193/32' 2023-10-06T04:03:20.227 INFO:tasks.workunit.client.0.smithi157.stdout:removed './193/30' 2023-10-06T04:03:20.227 INFO:tasks.workunit.client.0.smithi157.stdout:removed './193/14' 2023-10-06T04:03:20.227 INFO:tasks.workunit.client.0.smithi157.stdout:removed './193/18' 2023-10-06T04:03:20.227 INFO:tasks.workunit.client.0.smithi157.stdout:removed './193/13' 2023-10-06T04:03:20.227 INFO:tasks.workunit.client.0.smithi157.stdout:removed './193/33' 2023-10-06T04:03:20.227 INFO:tasks.workunit.client.0.smithi157.stdout:removed './193/17' 2023-10-06T04:03:20.227 INFO:tasks.workunit.client.0.smithi157.stdout:removed './193/35' 2023-10-06T04:03:20.227 INFO:tasks.workunit.client.0.smithi157.stdout:removed './193/37' 2023-10-06T04:03:20.227 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './193' 2023-10-06T04:03:20.227 INFO:tasks.workunit.client.0.smithi157.stdout:removed './194/19' 2023-10-06T04:03:20.228 INFO:tasks.workunit.client.0.smithi157.stdout:removed './194/15' 2023-10-06T04:03:20.228 INFO:tasks.workunit.client.0.smithi157.stdout:removed './194/31' 2023-10-06T04:03:20.228 INFO:tasks.workunit.client.0.smithi157.stdout:removed './194/36' 2023-10-06T04:03:20.228 INFO:tasks.workunit.client.0.smithi157.stdout:removed './194/20' 2023-10-06T04:03:20.228 INFO:tasks.workunit.client.0.smithi157.stdout:removed './194/12' 2023-10-06T04:03:20.228 INFO:tasks.workunit.client.0.smithi157.stdout:removed './194/39' 2023-10-06T04:03:20.228 INFO:tasks.workunit.client.0.smithi157.stdout:removed './194/10' 2023-10-06T04:03:20.228 INFO:tasks.workunit.client.0.smithi157.stdout:removed './194/34' 2023-10-06T04:03:20.228 INFO:tasks.workunit.client.0.smithi157.stdout:removed './194/38' 2023-10-06T04:03:20.228 INFO:tasks.workunit.client.0.smithi157.stdout:removed './194/11' 2023-10-06T04:03:20.228 INFO:tasks.workunit.client.0.smithi157.stdout:removed './194/16' 2023-10-06T04:03:20.228 INFO:tasks.workunit.client.0.smithi157.stdout:removed './194/32' 2023-10-06T04:03:20.229 INFO:tasks.workunit.client.0.smithi157.stdout:removed './194/30' 2023-10-06T04:03:20.229 INFO:tasks.workunit.client.0.smithi157.stdout:removed './194/14' 2023-10-06T04:03:20.229 INFO:tasks.workunit.client.0.smithi157.stdout:removed './194/18' 2023-10-06T04:03:20.229 INFO:tasks.workunit.client.0.smithi157.stdout:removed './194/13' 2023-10-06T04:03:20.229 INFO:tasks.workunit.client.0.smithi157.stdout:removed './194/33' 2023-10-06T04:03:20.229 INFO:tasks.workunit.client.0.smithi157.stdout:removed './194/17' 2023-10-06T04:03:20.229 INFO:tasks.workunit.client.0.smithi157.stdout:removed './194/35' 2023-10-06T04:03:20.229 INFO:tasks.workunit.client.0.smithi157.stdout:removed './194/37' 2023-10-06T04:03:20.229 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './194' 2023-10-06T04:03:20.229 INFO:tasks.workunit.client.0.smithi157.stdout:removed './195/19' 2023-10-06T04:03:20.229 INFO:tasks.workunit.client.0.smithi157.stdout:removed './195/15' 2023-10-06T04:03:20.229 INFO:tasks.workunit.client.0.smithi157.stdout:removed './195/31' 2023-10-06T04:03:20.230 INFO:tasks.workunit.client.0.smithi157.stdout:removed './195/36' 2023-10-06T04:03:20.230 INFO:tasks.workunit.client.0.smithi157.stdout:removed './195/20' 2023-10-06T04:03:20.230 INFO:tasks.workunit.client.0.smithi157.stdout:removed './195/12' 2023-10-06T04:03:20.230 INFO:tasks.workunit.client.0.smithi157.stdout:removed './195/39' 2023-10-06T04:03:20.230 INFO:tasks.workunit.client.0.smithi157.stdout:removed './195/10' 2023-10-06T04:03:20.230 INFO:tasks.workunit.client.0.smithi157.stdout:removed './195/34' 2023-10-06T04:03:20.230 INFO:tasks.workunit.client.0.smithi157.stdout:removed './195/38' 2023-10-06T04:03:20.230 INFO:tasks.workunit.client.0.smithi157.stdout:removed './195/11' 2023-10-06T04:03:20.230 INFO:tasks.workunit.client.0.smithi157.stdout:removed './195/16' 2023-10-06T04:03:20.230 INFO:tasks.workunit.client.0.smithi157.stdout:removed './195/32' 2023-10-06T04:03:20.230 INFO:tasks.workunit.client.0.smithi157.stdout:removed './195/30' 2023-10-06T04:03:20.230 INFO:tasks.workunit.client.0.smithi157.stdout:removed './195/14' 2023-10-06T04:03:20.230 INFO:tasks.workunit.client.0.smithi157.stdout:removed './195/18' 2023-10-06T04:03:20.231 INFO:tasks.workunit.client.0.smithi157.stdout:removed './195/13' 2023-10-06T04:03:20.231 INFO:tasks.workunit.client.0.smithi157.stdout:removed './195/33' 2023-10-06T04:03:20.231 INFO:tasks.workunit.client.0.smithi157.stdout:removed './195/17' 2023-10-06T04:03:20.231 INFO:tasks.workunit.client.0.smithi157.stdout:removed './195/35' 2023-10-06T04:03:20.231 INFO:tasks.workunit.client.0.smithi157.stdout:removed './195/37' 2023-10-06T04:03:20.231 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './195' 2023-10-06T04:03:20.231 INFO:tasks.workunit.client.0.smithi157.stdout:removed './196/19' 2023-10-06T04:03:20.231 INFO:tasks.workunit.client.0.smithi157.stdout:removed './196/15' 2023-10-06T04:03:20.231 INFO:tasks.workunit.client.0.smithi157.stdout:removed './196/31' 2023-10-06T04:03:20.231 INFO:tasks.workunit.client.0.smithi157.stdout:removed './196/36' 2023-10-06T04:03:20.231 INFO:tasks.workunit.client.0.smithi157.stdout:removed './196/20' 2023-10-06T04:03:20.231 INFO:tasks.workunit.client.0.smithi157.stdout:removed './196/12' 2023-10-06T04:03:20.232 INFO:tasks.workunit.client.0.smithi157.stdout:removed './196/39' 2023-10-06T04:03:20.232 INFO:tasks.workunit.client.0.smithi157.stdout:removed './196/10' 2023-10-06T04:03:21.153 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:03:21.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.278+0000 7f468c08e700 1 Processor -- start 2023-10-06T04:03:21.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.279+0000 7f468c08e700 1 -- start start 2023-10-06T04:03:21.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.279+0000 7f468c08e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f468414dcd0 0x7f4684153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:21.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.279+0000 7f468c08e700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46840c5770 con 0x7f46840c5c00 2023-10-06T04:03:21.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.279+0000 7f468c08e700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46841543e0 con 0x7f468414e0a0 2023-10-06T04:03:21.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.279+0000 7f468c08e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4684154c80 con 0x7f468414dcd0 2023-10-06T04:03:21.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.280+0000 7f468a62b700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f46840c5c00 0x7f46840ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:52798/0 (socket says 172.21.15.157:52798) 2023-10-06T04:03:21.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.280+0000 7f468a62b700 1 -- 172.21.15.157:0/1285002891 learned_addr learned my addr 172.21.15.157:0/1285002891 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:03:21.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.280+0000 7f4689e2a700 1 --2- 172.21.15.157:0/1285002891 >> v2:172.21.15.175:3300/0 conn(0x7f468414dcd0 0x7f4684153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:21.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.280+0000 7f4688e28700 1 -- 172.21.15.157:0/1285002891 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2572321945 0 0) 0x7f46840c5770 con 0x7f46840c5c00 2023-10-06T04:03:21.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.280+0000 7f4688e28700 1 -- 172.21.15.157:0/1285002891 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4670003570 con 0x7f46840c5c00 2023-10-06T04:03:21.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.280+0000 7f4688e28700 1 -- 172.21.15.157:0/1285002891 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2947596660 0 0) 0x7f4670003570 con 0x7f46840c5c00 2023-10-06T04:03:21.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.280+0000 7f4688e28700 1 -- 172.21.15.157:0/1285002891 >> v2:172.21.15.175:3300/0 conn(0x7f468414dcd0 msgr2=0x7f4684153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:21.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.280+0000 7f4688e28700 1 --2- 172.21.15.157:0/1285002891 >> v2:172.21.15.175:3300/0 conn(0x7f468414dcd0 0x7f4684153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:21.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.280+0000 7f4688e28700 1 -- 172.21.15.157:0/1285002891 >> v1:172.21.15.175:6789/0 conn(0x7f468414e0a0 legacy=0x7f46840c18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:21.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.280+0000 7f4688e28700 1 -- 172.21.15.157:0/1285002891 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46841546c0 con 0x7f46840c5c00 2023-10-06T04:03:21.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.280+0000 7f4688e28700 1 -- 172.21.15.157:0/1285002891 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f467c0017f0 con 0x7f46840c5c00 2023-10-06T04:03:21.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.280+0000 7f4688e28700 1 -- 172.21.15.157:0/1285002891 >> v1:172.21.15.157:6789/0 conn(0x7f46840c5c00 legacy=0x7f46840ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:21.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.280+0000 7f4688e28700 1 --2- 172.21.15.157:0/1285002891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4670003a60 0x7f4670003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:21.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.280+0000 7f4688e28700 1 --2- 172.21.15.157:0/1285002891 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4670004810 0x7f4670004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:21.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.280+0000 7f4688e28700 1 --2- 172.21.15.157:0/1285002891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46700051c0 0x7f467000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:21.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.280+0000 7f4688e28700 1 -- 172.21.15.157:0/1285002891 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4684154c80 con 0x7f4670003a60 2023-10-06T04:03:21.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.280+0000 7f4688e28700 1 -- 172.21.15.157:0/1285002891 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467000c0d0 con 0x7f46700051c0 2023-10-06T04:03:21.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.280+0000 7f4688e28700 1 -- 172.21.15.157:0/1285002891 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467000c290 con 0x7f4670004810 2023-10-06T04:03:21.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.280+0000 7f4689e2a700 1 --2- 172.21.15.157:0/1285002891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4670003a60 0x7f4670003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:21.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.280+0000 7f4689629700 1 --2- 172.21.15.157:0/1285002891 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4670004810 0x7f4670004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:21.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.281+0000 7f468a62b700 1 --2- 172.21.15.157:0/1285002891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46700051c0 0x7f467000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:21.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.281+0000 7f4689e2a700 1 -- 172.21.15.157:0/1285002891 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4670004810 msgr2=0x7f4670004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:21.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.281+0000 7f4689e2a700 1 --2- 172.21.15.157:0/1285002891 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4670004810 0x7f4670004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:21.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.281+0000 7f4689e2a700 1 -- 172.21.15.157:0/1285002891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46700051c0 msgr2=0x7f467000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:21.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.281+0000 7f4689e2a700 1 --2- 172.21.15.157:0/1285002891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46700051c0 0x7f467000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:21.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.281+0000 7f4689e2a700 1 -- 172.21.15.157:0/1285002891 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4670003570 con 0x7f4670003a60 2023-10-06T04:03:21.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.281+0000 7f4689629700 1 --2- 172.21.15.157:0/1285002891 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4670004810 0x7f4670004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:03:21.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.281+0000 7f468a62b700 1 --2- 172.21.15.157:0/1285002891 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46700051c0 0x7f467000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:03:21.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.281+0000 7f468a62b700 1 -- 172.21.15.157:0/1285002891 reap_dead start 2023-10-06T04:03:21.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.281+0000 7f4689e2a700 1 --2- 172.21.15.157:0/1285002891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4670003a60 0x7f4670003e70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4680009c00 tx=0x7f46800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a322fe92107670e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:21.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.281+0000 7f4688e28700 1 -- 172.21.15.157:0/1285002891 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468000e710 con 0x7f4670003a60 2023-10-06T04:03:21.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.281+0000 7f4688e28700 1 -- 172.21.15.157:0/1285002891 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468000e870 con 0x7f4670003a60 2023-10-06T04:03:21.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.281+0000 7f4688e28700 1 -- 172.21.15.157:0/1285002891 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468000ec30 con 0x7f4670003a60 2023-10-06T04:03:21.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.282+0000 7f468c08e700 1 -- 172.21.15.157:0/1285002891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4670003a60 msgr2=0x7f4670003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:21.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.282+0000 7f468c08e700 1 --2- 172.21.15.157:0/1285002891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4670003a60 0x7f4670003e70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4680009c00 tx=0x7f46800091a0 comp rx=0 tx=0).stop 2023-10-06T04:03:21.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.282+0000 7f468c08e700 1 -- 172.21.15.157:0/1285002891 shutdown_connections 2023-10-06T04:03:21.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.282+0000 7f468c08e700 1 --2- 172.21.15.157:0/1285002891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4670003a60 0x7f4670003e70 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:21.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.282+0000 7f468c08e700 1 -- 172.21.15.157:0/1285002891 >> 172.21.15.157:0/1285002891 conn(0x7f46840692b0 msgr2=0x7f46840696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:21.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.282+0000 7f468c08e700 1 -- 172.21.15.157:0/1285002891 shutdown_connections 2023-10-06T04:03:21.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.282+0000 7f468c08e700 1 -- 172.21.15.157:0/1285002891 wait complete. 2023-10-06T04:03:21.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.283+0000 7f468c08e700 1 Processor -- start 2023-10-06T04:03:21.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.283+0000 7f468c08e700 1 -- start start 2023-10-06T04:03:21.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.285+0000 7f468c08e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4670003a60 0x7f46840be250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:21.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.285+0000 7f468c08e700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46840c1070 0x7f4684167320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:21.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.285+0000 7f468c08e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46840bf1c0 0x7f46840bf610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:21.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.285+0000 7f468c08e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4684154200 con 0x7f46840bf1c0 2023-10-06T04:03:21.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.285+0000 7f468c08e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4684167f60 con 0x7f4670003a60 2023-10-06T04:03:21.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.285+0000 7f468c08e700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4684151250 con 0x7f46840c1070 2023-10-06T04:03:21.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.285+0000 7f468a62b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46840bf1c0 0x7f46840bf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:21.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.285+0000 7f468a62b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46840bf1c0 0x7f46840bf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45884/0 (socket says 172.21.15.157:45884) 2023-10-06T04:03:21.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.285+0000 7f468a62b700 1 -- 172.21.15.157:0/1926571267 learned_addr learned my addr 172.21.15.157:0/1926571267 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:21.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.285+0000 7f4689e2a700 1 --2- 172.21.15.157:0/1926571267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4670003a60 0x7f46840be250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:21.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.285+0000 7f4689629700 1 --2- 172.21.15.157:0/1926571267 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46840c1070 0x7f4684167320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:21.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.285+0000 7f468a62b700 1 -- 172.21.15.157:0/1926571267 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46840c1070 msgr2=0x7f4684167320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:21.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.285+0000 7f468a62b700 1 --2- 172.21.15.157:0/1926571267 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46840c1070 0x7f4684167320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:21.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.285+0000 7f468a62b700 1 -- 172.21.15.157:0/1926571267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4670003a60 msgr2=0x7f46840be250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:21.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.285+0000 7f468a62b700 1 --2- 172.21.15.157:0/1926571267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4670003a60 0x7f46840be250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:21.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.285+0000 7f468a62b700 1 -- 172.21.15.157:0/1926571267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46840c3cf0 con 0x7f46840bf1c0 2023-10-06T04:03:21.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.285+0000 7f468a62b700 1 --2- 172.21.15.157:0/1926571267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46840bf1c0 0x7f46840bf610 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f467c001350 tx=0x7f467c0017f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:21.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.286+0000 7f4679ffb700 1 -- 172.21.15.157:0/1926571267 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c017070 con 0x7f46840bf1c0 2023-10-06T04:03:21.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.286+0000 7f4679ffb700 1 -- 172.21.15.157:0/1926571267 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467c00eab0 con 0x7f46840bf1c0 2023-10-06T04:03:21.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.286+0000 7f4679ffb700 1 -- 172.21.15.157:0/1926571267 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c00ee70 con 0x7f46840bf1c0 2023-10-06T04:03:21.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.286+0000 7f468c08e700 1 -- 172.21.15.157:0/1926571267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46840c0970 con 0x7f46840bf1c0 2023-10-06T04:03:21.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.286+0000 7f468c08e700 1 -- 172.21.15.157:0/1926571267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46840bdc90 con 0x7f46840bf1c0 2023-10-06T04:03:21.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.287+0000 7f468c08e700 1 -- 172.21.15.157:0/1926571267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f468406a8f0 con 0x7f46840bf1c0 2023-10-06T04:03:21.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.287+0000 7f4679ffb700 1 -- 172.21.15.157:0/1926571267 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f467c013630 con 0x7f46840bf1c0 2023-10-06T04:03:21.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.288+0000 7f4679ffb700 1 --2- 172.21.15.157:0/1926571267 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4668066f30 0x7f46680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:21.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.288+0000 7f4679ffb700 1 -- 172.21.15.157:0/1926571267 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7f467c0846a0 con 0x7f46840bf1c0 2023-10-06T04:03:21.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.288+0000 7f4689e2a700 1 --2- 172.21.15.157:0/1926571267 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4668066f30 0x7f46680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:21.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.288+0000 7f4689e2a700 1 --2- 172.21.15.157:0/1926571267 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4668066f30 0x7f46680693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f46800091e0 tx=0x7f468001a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:21.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.292+0000 7f4679ffb700 1 -- 172.21.15.157:0/1926571267 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f467c051560 con 0x7f46840bf1c0 2023-10-06T04:03:21.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.543+0000 7f468c08e700 1 -- 172.21.15.157:0/1926571267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46841515e0 con 0x7f46840bf1c0 2023-10-06T04:03:21.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.544+0000 7f4679ffb700 1 -- 172.21.15.157:0/1926571267 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+6162 (secure 0 0 0) 0x7f467c054b80 con 0x7f46840bf1c0 2023-10-06T04:03:21.546 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:03:21.546 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3192541484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3192541484},{"type":"v1","addr":"172.21.15.175:6835","nonce":3192541484}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:13.713593+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279},"failed":[],"damaged":[],"stopped":[3,4],"info":{"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:stopping","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:03:21.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.546+0000 7f468c08e700 1 -- 172.21.15.157:0/1926571267 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4668066f30 msgr2=0x7f46680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:21.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.546+0000 7f468c08e700 1 --2- 172.21.15.157:0/1926571267 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4668066f30 0x7f46680693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f46800091e0 tx=0x7f468001a000 comp rx=0 tx=0).stop 2023-10-06T04:03:21.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.546+0000 7f468c08e700 1 -- 172.21.15.157:0/1926571267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46840bf1c0 msgr2=0x7f46840bf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:21.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.546+0000 7f468c08e700 1 --2- 172.21.15.157:0/1926571267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46840bf1c0 0x7f46840bf610 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f467c001350 tx=0x7f467c0017f0 comp rx=0 tx=0).stop 2023-10-06T04:03:21.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.546+0000 7f468c08e700 1 -- 172.21.15.157:0/1926571267 shutdown_connections 2023-10-06T04:03:21.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.546+0000 7f468c08e700 1 --2- 172.21.15.157:0/1926571267 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4668066f30 0x7f46680693e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:21.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.546+0000 7f468c08e700 1 --2- 172.21.15.157:0/1926571267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4670003a60 0x7f46840be250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:21.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.546+0000 7f468c08e700 1 --2- 172.21.15.157:0/1926571267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46840bf1c0 0x7f46840bf610 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:21.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.546+0000 7f468c08e700 1 --2- 172.21.15.157:0/1926571267 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46840c1070 0x7f4684167320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:21.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.546+0000 7f468c08e700 1 -- 172.21.15.157:0/1926571267 >> 172.21.15.157:0/1926571267 conn(0x7f46840692b0 msgr2=0x7f46840c6510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:21.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.546+0000 7f468c08e700 1 -- 172.21.15.157:0/1926571267 shutdown_connections 2023-10-06T04:03:21.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:21.546+0000 7f468c08e700 1 -- 172.21.15.157:0/1926571267 wait complete. 2023-10-06T04:03:21.550 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 51 2023-10-06T04:03:21.559 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:03:22.774 INFO:tasks.workunit.client.0.smithi157.stdout:removed './196/34' 2023-10-06T04:03:22.774 INFO:tasks.workunit.client.0.smithi157.stdout:removed './196/38' 2023-10-06T04:03:22.774 INFO:tasks.workunit.client.0.smithi157.stdout:removed './196/11' 2023-10-06T04:03:22.774 INFO:tasks.workunit.client.0.smithi157.stdout:removed './196/16' 2023-10-06T04:03:22.774 INFO:tasks.workunit.client.0.smithi157.stdout:removed './196/32' 2023-10-06T04:03:22.774 INFO:tasks.workunit.client.0.smithi157.stdout:removed './196/30' 2023-10-06T04:03:22.775 INFO:tasks.workunit.client.0.smithi157.stdout:removed './196/14' 2023-10-06T04:03:22.775 INFO:tasks.workunit.client.0.smithi157.stdout:removed './196/18' 2023-10-06T04:03:22.775 INFO:tasks.workunit.client.0.smithi157.stdout:removed './196/13' 2023-10-06T04:03:22.775 INFO:tasks.workunit.client.0.smithi157.stdout:removed './196/33' 2023-10-06T04:03:22.775 INFO:tasks.workunit.client.0.smithi157.stdout:removed './196/17' 2023-10-06T04:03:22.775 INFO:tasks.workunit.client.0.smithi157.stdout:removed './196/35' 2023-10-06T04:03:22.775 INFO:tasks.workunit.client.0.smithi157.stdout:removed './196/37' 2023-10-06T04:03:22.775 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './196' 2023-10-06T04:03:22.775 INFO:tasks.workunit.client.0.smithi157.stdout:removed './197/19' 2023-10-06T04:03:22.775 INFO:tasks.workunit.client.0.smithi157.stdout:removed './197/15' 2023-10-06T04:03:22.775 INFO:tasks.workunit.client.0.smithi157.stdout:removed './197/31' 2023-10-06T04:03:22.775 INFO:tasks.workunit.client.0.smithi157.stdout:removed './197/36' 2023-10-06T04:03:22.776 INFO:tasks.workunit.client.0.smithi157.stdout:removed './197/20' 2023-10-06T04:03:22.776 INFO:tasks.workunit.client.0.smithi157.stdout:removed './197/12' 2023-10-06T04:03:22.776 INFO:tasks.workunit.client.0.smithi157.stdout:removed './197/39' 2023-10-06T04:03:22.776 INFO:tasks.workunit.client.0.smithi157.stdout:removed './197/10' 2023-10-06T04:03:22.776 INFO:tasks.workunit.client.0.smithi157.stdout:removed './197/34' 2023-10-06T04:03:22.776 INFO:tasks.workunit.client.0.smithi157.stdout:removed './197/38' 2023-10-06T04:03:22.776 INFO:tasks.workunit.client.0.smithi157.stdout:removed './197/11' 2023-10-06T04:03:22.776 INFO:tasks.workunit.client.0.smithi157.stdout:removed './197/16' 2023-10-06T04:03:22.776 INFO:tasks.workunit.client.0.smithi157.stdout:removed './197/32' 2023-10-06T04:03:22.776 INFO:tasks.workunit.client.0.smithi157.stdout:removed './197/30' 2023-10-06T04:03:22.776 INFO:tasks.workunit.client.0.smithi157.stdout:removed './197/14' 2023-10-06T04:03:22.776 INFO:tasks.workunit.client.0.smithi157.stdout:removed './197/18' 2023-10-06T04:03:22.777 INFO:tasks.workunit.client.0.smithi157.stdout:removed './197/13' 2023-10-06T04:03:22.777 INFO:tasks.workunit.client.0.smithi157.stdout:removed './197/33' 2023-10-06T04:03:22.777 INFO:tasks.workunit.client.0.smithi157.stdout:removed './197/17' 2023-10-06T04:03:22.777 INFO:tasks.workunit.client.0.smithi157.stdout:removed './197/35' 2023-10-06T04:03:22.777 INFO:tasks.workunit.client.0.smithi157.stdout:removed './197/37' 2023-10-06T04:03:22.777 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './197' 2023-10-06T04:03:22.777 INFO:tasks.workunit.client.0.smithi157.stdout:removed './198/19' 2023-10-06T04:03:22.777 INFO:tasks.workunit.client.0.smithi157.stdout:removed './198/15' 2023-10-06T04:03:22.777 INFO:tasks.workunit.client.0.smithi157.stdout:removed './198/31' 2023-10-06T04:03:22.777 INFO:tasks.workunit.client.0.smithi157.stdout:removed './198/36' 2023-10-06T04:03:22.777 INFO:tasks.workunit.client.0.smithi157.stdout:removed './198/20' 2023-10-06T04:03:22.777 INFO:tasks.workunit.client.0.smithi157.stdout:removed './198/12' 2023-10-06T04:03:22.777 INFO:tasks.workunit.client.0.smithi157.stdout:removed './198/39' 2023-10-06T04:03:22.778 INFO:tasks.workunit.client.0.smithi157.stdout:removed './198/10' 2023-10-06T04:03:22.778 INFO:tasks.workunit.client.0.smithi157.stdout:removed './198/34' 2023-10-06T04:03:22.778 INFO:tasks.workunit.client.0.smithi157.stdout:removed './198/38' 2023-10-06T04:03:22.778 INFO:tasks.workunit.client.0.smithi157.stdout:removed './198/11' 2023-10-06T04:03:22.778 INFO:tasks.workunit.client.0.smithi157.stdout:removed './198/16' 2023-10-06T04:03:22.778 INFO:tasks.workunit.client.0.smithi157.stdout:removed './198/32' 2023-10-06T04:03:22.778 INFO:tasks.workunit.client.0.smithi157.stdout:removed './198/30' 2023-10-06T04:03:22.778 INFO:tasks.workunit.client.0.smithi157.stdout:removed './198/14' 2023-10-06T04:03:22.778 INFO:tasks.workunit.client.0.smithi157.stdout:removed './198/18' 2023-10-06T04:03:22.778 INFO:tasks.workunit.client.0.smithi157.stdout:removed './198/13' 2023-10-06T04:03:22.778 INFO:tasks.workunit.client.0.smithi157.stdout:removed './198/33' 2023-10-06T04:03:22.778 INFO:tasks.workunit.client.0.smithi157.stdout:removed './198/17' 2023-10-06T04:03:22.779 INFO:tasks.workunit.client.0.smithi157.stdout:removed './198/35' 2023-10-06T04:03:22.779 INFO:tasks.workunit.client.0.smithi157.stdout:removed './198/37' 2023-10-06T04:03:22.779 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './198' 2023-10-06T04:03:22.779 INFO:tasks.workunit.client.0.smithi157.stdout:removed './199/19' 2023-10-06T04:03:22.779 INFO:tasks.workunit.client.0.smithi157.stdout:removed './199/15' 2023-10-06T04:03:22.779 INFO:tasks.workunit.client.0.smithi157.stdout:removed './199/31' 2023-10-06T04:03:22.779 INFO:tasks.workunit.client.0.smithi157.stdout:removed './199/36' 2023-10-06T04:03:22.779 INFO:tasks.workunit.client.0.smithi157.stdout:removed './199/20' 2023-10-06T04:03:22.779 INFO:tasks.workunit.client.0.smithi157.stdout:removed './199/12' 2023-10-06T04:03:22.779 INFO:tasks.workunit.client.0.smithi157.stdout:removed './199/39' 2023-10-06T04:03:22.779 INFO:tasks.workunit.client.0.smithi157.stdout:removed './199/10' 2023-10-06T04:03:22.780 INFO:tasks.workunit.client.0.smithi157.stdout:removed './199/34' 2023-10-06T04:03:22.780 INFO:tasks.workunit.client.0.smithi157.stdout:removed './199/38' 2023-10-06T04:03:22.780 INFO:tasks.workunit.client.0.smithi157.stdout:removed './199/11' 2023-10-06T04:03:22.780 INFO:tasks.workunit.client.0.smithi157.stdout:removed './199/16' 2023-10-06T04:03:22.780 INFO:tasks.workunit.client.0.smithi157.stdout:removed './199/32' 2023-10-06T04:03:22.780 INFO:tasks.workunit.client.0.smithi157.stdout:removed './199/30' 2023-10-06T04:03:22.780 INFO:tasks.workunit.client.0.smithi157.stdout:removed './199/14' 2023-10-06T04:03:22.780 INFO:tasks.workunit.client.0.smithi157.stdout:removed './199/18' 2023-10-06T04:03:22.780 INFO:tasks.workunit.client.0.smithi157.stdout:removed './199/13' 2023-10-06T04:03:22.781 INFO:tasks.workunit.client.0.smithi157.stdout:removed './199/33' 2023-10-06T04:03:22.781 INFO:tasks.workunit.client.0.smithi157.stdout:removed './199/17' 2023-10-06T04:03:22.781 INFO:tasks.workunit.client.0.smithi157.stdout:removed './199/35' 2023-10-06T04:03:22.781 INFO:tasks.workunit.client.0.smithi157.stdout:removed './199/37' 2023-10-06T04:03:22.781 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './199' 2023-10-06T04:03:22.781 INFO:tasks.workunit.client.0.smithi157.stdout:removed './200/21' 2023-10-06T04:03:22.781 INFO:tasks.workunit.client.0.smithi157.stdout:removed './200/27' 2023-10-06T04:03:22.781 INFO:tasks.workunit.client.0.smithi157.stdout:removed './200/31' 2023-10-06T04:03:22.781 INFO:tasks.workunit.client.0.smithi157.stdout:removed './200/36' 2023-10-06T04:03:22.781 INFO:tasks.workunit.client.0.smithi157.stdout:removed './200/20' 2023-10-06T04:03:22.781 INFO:tasks.workunit.client.0.smithi157.stdout:removed './200/39' 2023-10-06T04:03:22.781 INFO:tasks.workunit.client.0.smithi157.stdout:removed './200/34' 2023-10-06T04:03:22.782 INFO:tasks.workunit.client.0.smithi157.stdout:removed './200/38' 2023-10-06T04:03:22.782 INFO:tasks.workunit.client.0.smithi157.stdout:removed './200/28' 2023-10-06T04:03:22.782 INFO:tasks.workunit.client.0.smithi157.stdout:removed './200/24' 2023-10-06T04:03:22.782 INFO:tasks.workunit.client.0.smithi157.stdout:removed './200/29' 2023-10-06T04:03:22.782 INFO:tasks.workunit.client.0.smithi157.stdout:removed './200/32' 2023-10-06T04:03:22.782 INFO:tasks.workunit.client.0.smithi157.stdout:removed './200/22' 2023-10-06T04:03:22.782 INFO:tasks.workunit.client.0.smithi157.stdout:removed './200/25' 2023-10-06T04:03:22.782 INFO:tasks.workunit.client.0.smithi157.stdout:removed './200/30' 2023-10-06T04:03:22.782 INFO:tasks.workunit.client.0.smithi157.stdout:removed './200/33' 2023-10-06T04:03:22.782 INFO:tasks.workunit.client.0.smithi157.stdout:removed './200/23' 2023-10-06T04:03:22.782 INFO:tasks.workunit.client.0.smithi157.stdout:removed './200/35' 2023-10-06T04:03:22.782 INFO:tasks.workunit.client.0.smithi157.stdout:removed './200/26' 2023-10-06T04:03:22.783 INFO:tasks.workunit.client.0.smithi157.stdout:removed './200/37' 2023-10-06T04:03:22.783 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './200' 2023-10-06T04:03:22.783 INFO:tasks.workunit.client.0.smithi157.stdout:removed './201/21' 2023-10-06T04:03:22.783 INFO:tasks.workunit.client.0.smithi157.stdout:removed './201/27' 2023-10-06T04:03:22.783 INFO:tasks.workunit.client.0.smithi157.stdout:removed './201/31' 2023-10-06T04:03:22.783 INFO:tasks.workunit.client.0.smithi157.stdout:removed './201/36' 2023-10-06T04:03:22.783 INFO:tasks.workunit.client.0.smithi157.stdout:removed './201/20' 2023-10-06T04:03:22.783 INFO:tasks.workunit.client.0.smithi157.stdout:removed './201/39' 2023-10-06T04:03:22.783 INFO:tasks.workunit.client.0.smithi157.stdout:removed './201/34' 2023-10-06T04:03:22.783 INFO:tasks.workunit.client.0.smithi157.stdout:removed './201/38' 2023-10-06T04:03:22.783 INFO:tasks.workunit.client.0.smithi157.stdout:removed './201/28' 2023-10-06T04:03:22.784 INFO:tasks.workunit.client.0.smithi157.stdout:removed './201/24' 2023-10-06T04:03:22.784 INFO:tasks.workunit.client.0.smithi157.stdout:removed './201/29' 2023-10-06T04:03:22.784 INFO:tasks.workunit.client.0.smithi157.stdout:removed './201/32' 2023-10-06T04:03:22.784 INFO:tasks.workunit.client.0.smithi157.stdout:removed './201/22' 2023-10-06T04:03:22.784 INFO:tasks.workunit.client.0.smithi157.stdout:removed './201/25' 2023-10-06T04:03:22.784 INFO:tasks.workunit.client.0.smithi157.stdout:removed './201/30' 2023-10-06T04:03:22.784 INFO:tasks.workunit.client.0.smithi157.stdout:removed './201/33' 2023-10-06T04:03:22.784 INFO:tasks.workunit.client.0.smithi157.stdout:removed './201/23' 2023-10-06T04:03:22.784 INFO:tasks.workunit.client.0.smithi157.stdout:removed './201/35' 2023-10-06T04:03:22.784 INFO:tasks.workunit.client.0.smithi157.stdout:removed './201/26' 2023-10-06T04:03:22.784 INFO:tasks.workunit.client.0.smithi157.stdout:removed './201/37' 2023-10-06T04:03:22.784 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './201' 2023-10-06T04:03:22.784 INFO:tasks.workunit.client.0.smithi157.stdout:removed './202/21' 2023-10-06T04:03:22.785 INFO:tasks.workunit.client.0.smithi157.stdout:removed './202/27' 2023-10-06T04:03:22.785 INFO:tasks.workunit.client.0.smithi157.stdout:removed './202/31' 2023-10-06T04:03:22.785 INFO:tasks.workunit.client.0.smithi157.stdout:removed './202/36' 2023-10-06T04:03:22.785 INFO:tasks.workunit.client.0.smithi157.stdout:removed './202/20' 2023-10-06T04:03:22.785 INFO:tasks.workunit.client.0.smithi157.stdout:removed './202/39' 2023-10-06T04:03:22.785 INFO:tasks.workunit.client.0.smithi157.stdout:removed './202/34' 2023-10-06T04:03:22.785 INFO:tasks.workunit.client.0.smithi157.stdout:removed './202/38' 2023-10-06T04:03:22.785 INFO:tasks.workunit.client.0.smithi157.stdout:removed './202/28' 2023-10-06T04:03:22.785 INFO:tasks.workunit.client.0.smithi157.stdout:removed './202/24' 2023-10-06T04:03:22.785 INFO:tasks.workunit.client.0.smithi157.stdout:removed './202/29' 2023-10-06T04:03:22.786 INFO:tasks.workunit.client.0.smithi157.stdout:removed './202/32' 2023-10-06T04:03:22.786 INFO:tasks.workunit.client.0.smithi157.stdout:removed './202/22' 2023-10-06T04:03:22.786 INFO:tasks.workunit.client.0.smithi157.stdout:removed './202/25' 2023-10-06T04:03:22.786 INFO:tasks.workunit.client.0.smithi157.stdout:removed './202/30' 2023-10-06T04:03:22.786 INFO:tasks.workunit.client.0.smithi157.stdout:removed './202/33' 2023-10-06T04:03:22.786 INFO:tasks.workunit.client.0.smithi157.stdout:removed './202/23' 2023-10-06T04:03:22.786 INFO:tasks.workunit.client.0.smithi157.stdout:removed './202/35' 2023-10-06T04:03:22.786 INFO:tasks.workunit.client.0.smithi157.stdout:removed './202/26' 2023-10-06T04:03:22.786 INFO:tasks.workunit.client.0.smithi157.stdout:removed './202/37' 2023-10-06T04:03:22.787 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './202' 2023-10-06T04:03:22.787 INFO:tasks.workunit.client.0.smithi157.stdout:removed './203/21' 2023-10-06T04:03:22.787 INFO:tasks.workunit.client.0.smithi157.stdout:removed './203/27' 2023-10-06T04:03:22.787 INFO:tasks.workunit.client.0.smithi157.stdout:removed './203/31' 2023-10-06T04:03:22.787 INFO:tasks.workunit.client.0.smithi157.stdout:removed './203/36' 2023-10-06T04:03:22.787 INFO:tasks.workunit.client.0.smithi157.stdout:removed './203/20' 2023-10-06T04:03:22.787 INFO:tasks.workunit.client.0.smithi157.stdout:removed './203/39' 2023-10-06T04:03:22.787 INFO:tasks.workunit.client.0.smithi157.stdout:removed './203/34' 2023-10-06T04:03:22.787 INFO:tasks.workunit.client.0.smithi157.stdout:removed './203/38' 2023-10-06T04:03:22.787 INFO:tasks.workunit.client.0.smithi157.stdout:removed './203/28' 2023-10-06T04:03:22.787 INFO:tasks.workunit.client.0.smithi157.stdout:removed './203/24' 2023-10-06T04:03:22.787 INFO:tasks.workunit.client.0.smithi157.stdout:removed './203/29' 2023-10-06T04:03:22.788 INFO:tasks.workunit.client.0.smithi157.stdout:removed './203/32' 2023-10-06T04:03:22.788 INFO:tasks.workunit.client.0.smithi157.stdout:removed './203/22' 2023-10-06T04:03:22.788 INFO:tasks.workunit.client.0.smithi157.stdout:removed './203/25' 2023-10-06T04:03:22.788 INFO:tasks.workunit.client.0.smithi157.stdout:removed './203/30' 2023-10-06T04:03:22.788 INFO:tasks.workunit.client.0.smithi157.stdout:removed './203/33' 2023-10-06T04:03:22.788 INFO:tasks.workunit.client.0.smithi157.stdout:removed './203/23' 2023-10-06T04:03:22.788 INFO:tasks.workunit.client.0.smithi157.stdout:removed './203/35' 2023-10-06T04:03:22.788 INFO:tasks.workunit.client.0.smithi157.stdout:removed './203/26' 2023-10-06T04:03:22.788 INFO:tasks.workunit.client.0.smithi157.stdout:removed './203/37' 2023-10-06T04:03:22.788 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './203' 2023-10-06T04:03:22.788 INFO:tasks.workunit.client.0.smithi157.stdout:removed './204/21' 2023-10-06T04:03:22.788 INFO:tasks.workunit.client.0.smithi157.stdout:removed './204/27' 2023-10-06T04:03:22.789 INFO:tasks.workunit.client.0.smithi157.stdout:removed './204/31' 2023-10-06T04:03:22.789 INFO:tasks.workunit.client.0.smithi157.stdout:removed './204/36' 2023-10-06T04:03:22.789 INFO:tasks.workunit.client.0.smithi157.stdout:removed './204/20' 2023-10-06T04:03:22.789 INFO:tasks.workunit.client.0.smithi157.stdout:removed './204/39' 2023-10-06T04:03:22.789 INFO:tasks.workunit.client.0.smithi157.stdout:removed './204/34' 2023-10-06T04:03:22.789 INFO:tasks.workunit.client.0.smithi157.stdout:removed './204/38' 2023-10-06T04:03:22.789 INFO:tasks.workunit.client.0.smithi157.stdout:removed './204/28' 2023-10-06T04:03:22.789 INFO:tasks.workunit.client.0.smithi157.stdout:removed './204/24' 2023-10-06T04:03:22.789 INFO:tasks.workunit.client.0.smithi157.stdout:removed './204/29' 2023-10-06T04:03:22.789 INFO:tasks.workunit.client.0.smithi157.stdout:removed './204/32' 2023-10-06T04:03:22.789 INFO:tasks.workunit.client.0.smithi157.stdout:removed './204/22' 2023-10-06T04:03:22.789 INFO:tasks.workunit.client.0.smithi157.stdout:removed './204/25' 2023-10-06T04:03:22.790 INFO:tasks.workunit.client.0.smithi157.stdout:removed './204/30' 2023-10-06T04:03:22.790 INFO:tasks.workunit.client.0.smithi157.stdout:removed './204/33' 2023-10-06T04:03:22.790 INFO:tasks.workunit.client.0.smithi157.stdout:removed './204/23' 2023-10-06T04:03:22.790 INFO:tasks.workunit.client.0.smithi157.stdout:removed './204/35' 2023-10-06T04:03:22.790 INFO:tasks.workunit.client.0.smithi157.stdout:removed './204/26' 2023-10-06T04:03:22.790 INFO:tasks.workunit.client.0.smithi157.stdout:removed './204/37' 2023-10-06T04:03:22.790 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './204' 2023-10-06T04:03:22.790 INFO:tasks.workunit.client.0.smithi157.stdout:removed './205/21' 2023-10-06T04:03:22.790 INFO:tasks.workunit.client.0.smithi157.stdout:removed './205/27' 2023-10-06T04:03:22.790 INFO:tasks.workunit.client.0.smithi157.stdout:removed './205/31' 2023-10-06T04:03:22.790 INFO:tasks.workunit.client.0.smithi157.stdout:removed './205/36' 2023-10-06T04:03:22.790 INFO:tasks.workunit.client.0.smithi157.stdout:removed './205/20' 2023-10-06T04:03:22.791 INFO:tasks.workunit.client.0.smithi157.stdout:removed './205/39' 2023-10-06T04:03:22.791 INFO:tasks.workunit.client.0.smithi157.stdout:removed './205/34' 2023-10-06T04:03:22.791 INFO:tasks.workunit.client.0.smithi157.stdout:removed './205/38' 2023-10-06T04:03:22.791 INFO:tasks.workunit.client.0.smithi157.stdout:removed './205/28' 2023-10-06T04:03:22.791 INFO:tasks.workunit.client.0.smithi157.stdout:removed './205/24' 2023-10-06T04:03:22.791 INFO:tasks.workunit.client.0.smithi157.stdout:removed './205/29' 2023-10-06T04:03:22.791 INFO:tasks.workunit.client.0.smithi157.stdout:removed './205/32' 2023-10-06T04:03:22.791 INFO:tasks.workunit.client.0.smithi157.stdout:removed './205/22' 2023-10-06T04:03:22.791 INFO:tasks.workunit.client.0.smithi157.stdout:removed './205/25' 2023-10-06T04:03:22.791 INFO:tasks.workunit.client.0.smithi157.stdout:removed './205/30' 2023-10-06T04:03:22.791 INFO:tasks.workunit.client.0.smithi157.stdout:removed './205/33' 2023-10-06T04:03:22.791 INFO:tasks.workunit.client.0.smithi157.stdout:removed './205/23' 2023-10-06T04:03:22.791 INFO:tasks.workunit.client.0.smithi157.stdout:removed './205/35' 2023-10-06T04:03:22.792 INFO:tasks.workunit.client.0.smithi157.stdout:removed './205/26' 2023-10-06T04:03:22.792 INFO:tasks.workunit.client.0.smithi157.stdout:removed './205/37' 2023-10-06T04:03:22.792 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './205' 2023-10-06T04:03:22.792 INFO:tasks.workunit.client.0.smithi157.stdout:removed './206/21' 2023-10-06T04:03:22.792 INFO:tasks.workunit.client.0.smithi157.stdout:removed './206/27' 2023-10-06T04:03:22.792 INFO:tasks.workunit.client.0.smithi157.stdout:removed './206/31' 2023-10-06T04:03:22.792 INFO:tasks.workunit.client.0.smithi157.stdout:removed './206/36' 2023-10-06T04:03:22.792 INFO:tasks.workunit.client.0.smithi157.stdout:removed './206/20' 2023-10-06T04:03:22.793 INFO:tasks.workunit.client.0.smithi157.stdout:removed './206/39' 2023-10-06T04:03:23.560 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:03:23.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.681+0000 7f6fb2676700 1 Processor -- start 2023-10-06T04:03:23.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.681+0000 7f6fb2676700 1 -- start start 2023-10-06T04:03:23.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.682+0000 7f6fb2676700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6fac0ca630 0x7f6fac153e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:23.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.682+0000 7f6fb2676700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fac0caa00 0x7f6fac0c18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:23.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.682+0000 7f6fb2676700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6fac0c3e50 0x7f6fac0c4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:23.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.682+0000 7f6fb2676700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fac154340 con 0x7f6fac0ca630 2023-10-06T04:03:23.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.682+0000 7f6fb2676700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fac0c39c0 con 0x7f6fac0c3e50 2023-10-06T04:03:23.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.682+0000 7f6fb2676700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac068b80 con 0x7f6fac0caa00 2023-10-06T04:03:23.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.682+0000 7f6fabfff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6fac0ca630 0x7f6fac153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:23.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.682+0000 7f6fabfff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6fac0ca630 0x7f6fac153e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37446/0 (socket says 172.21.15.157:37446) 2023-10-06T04:03:23.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.682+0000 7f6fabfff700 1 -- 172.21.15.157:0/3272185044 learned_addr learned my addr 172.21.15.157:0/3272185044 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:23.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.683+0000 7f6fb0c13700 1 --2- 172.21.15.157:0/3272185044 >> v2:172.21.15.175:3300/0 conn(0x7f6fac0c3e50 0x7f6fac0c4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:23.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.683+0000 7f6fab7fe700 1 --2- 172.21.15.157:0/3272185044 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fac0caa00 0x7f6fac0c18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:23.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.683+0000 7f6fabfff700 1 -- 172.21.15.157:0/3272185044 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fac0caa00 msgr2=0x7f6fac0c18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:23.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.683+0000 7f6fabfff700 1 --2- 172.21.15.157:0/3272185044 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fac0caa00 0x7f6fac0c18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:23.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.683+0000 7f6fabfff700 1 -- 172.21.15.157:0/3272185044 >> v2:172.21.15.175:3300/0 conn(0x7f6fac0c3e50 msgr2=0x7f6fac0c4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:23.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.683+0000 7f6fabfff700 1 --2- 172.21.15.157:0/3272185044 >> v2:172.21.15.175:3300/0 conn(0x7f6fac0c3e50 0x7f6fac0c4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:23.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.683+0000 7f6fabfff700 1 -- 172.21.15.157:0/3272185044 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fac0c1e00 con 0x7f6fac0ca630 2023-10-06T04:03:23.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.683+0000 7f6fabfff700 1 --2- 172.21.15.157:0/3272185044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fac0ca630 0x7f6fac153e00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6fac0c1480 tx=0x7f6f9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d244d6c081cdfd51 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:23.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.683+0000 7f6faaffd700 1 -- 172.21.15.157:0/3272185044 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c00b860 con 0x7f6fac0ca630 2023-10-06T04:03:23.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.684+0000 7f6faaffd700 1 -- 172.21.15.157:0/3272185044 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9c00b9c0 con 0x7f6fac0ca630 2023-10-06T04:03:23.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.684+0000 7f6faaffd700 1 -- 172.21.15.157:0/3272185044 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c00bd60 con 0x7f6fac0ca630 2023-10-06T04:03:23.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.684+0000 7f6fb2676700 1 -- 172.21.15.157:0/3272185044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fac0ca630 msgr2=0x7f6fac153e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:23.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.684+0000 7f6fb2676700 1 --2- 172.21.15.157:0/3272185044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fac0ca630 0x7f6fac153e00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6fac0c1480 tx=0x7f6f9c009250 comp rx=0 tx=0).stop 2023-10-06T04:03:23.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.684+0000 7f6fb2676700 1 -- 172.21.15.157:0/3272185044 shutdown_connections 2023-10-06T04:03:23.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.684+0000 7f6fb2676700 1 --2- 172.21.15.157:0/3272185044 >> v2:172.21.15.175:3300/0 conn(0x7f6fac0c3e50 0x7f6fac0c4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:23.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.684+0000 7f6fb2676700 1 --2- 172.21.15.157:0/3272185044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fac0ca630 0x7f6fac153e00 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:23.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.684+0000 7f6fb2676700 1 --2- 172.21.15.157:0/3272185044 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fac0caa00 0x7f6fac0c18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:23.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.684+0000 7f6fb2676700 1 -- 172.21.15.157:0/3272185044 >> 172.21.15.157:0/3272185044 conn(0x7f6fac0692b0 msgr2=0x7f6fac0696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:23.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.685+0000 7f6fb2676700 1 -- 172.21.15.157:0/3272185044 shutdown_connections 2023-10-06T04:03:23.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.685+0000 7f6fb2676700 1 -- 172.21.15.157:0/3272185044 wait complete. 2023-10-06T04:03:23.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.687+0000 7f6fb2676700 1 Processor -- start 2023-10-06T04:03:23.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.687+0000 7f6fb2676700 1 -- start start 2023-10-06T04:03:23.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.688+0000 7f6fb2676700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fac0c3e50 0x7f6fac1657d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:23.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.688+0000 7f6fb2676700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fac0caa00 0x7f6fac165d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:23.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.688+0000 7f6fb2676700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fac148ef0 0x7f6fac149370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:23.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.688+0000 7f6fb2676700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac15c060 con 0x7f6fac148ef0 2023-10-06T04:03:23.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.688+0000 7f6fb2676700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac0c0680 con 0x7f6fac0caa00 2023-10-06T04:03:23.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.688+0000 7f6fb2676700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac0c0be0 con 0x7f6fac0c3e50 2023-10-06T04:03:23.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.689+0000 7f6fb0c13700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fac148ef0 0x7f6fac149370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:23.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.689+0000 7f6fb0c13700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fac148ef0 0x7f6fac149370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37450/0 (socket says 172.21.15.157:37450) 2023-10-06T04:03:23.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.689+0000 7f6fb0c13700 1 -- 172.21.15.157:0/133634554 learned_addr learned my addr 172.21.15.157:0/133634554 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:23.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.689+0000 7f6fab7fe700 1 --2- 172.21.15.157:0/133634554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fac0caa00 0x7f6fac165d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:23.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.689+0000 7f6fb0c13700 1 -- 172.21.15.157:0/133634554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fac0c3e50 msgr2=0x7f6fac1657d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:23.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.689+0000 7f6fabfff700 1 --2- 172.21.15.157:0/133634554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fac0c3e50 0x7f6fac1657d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:23.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.689+0000 7f6fb0c13700 1 --2- 172.21.15.157:0/133634554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fac0c3e50 0x7f6fac1657d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:23.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.689+0000 7f6fb0c13700 1 -- 172.21.15.157:0/133634554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fac0caa00 msgr2=0x7f6fac165d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:23.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.689+0000 7f6fb0c13700 1 --2- 172.21.15.157:0/133634554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fac0caa00 0x7f6fac165d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:23.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.689+0000 7f6fb0c13700 1 -- 172.21.15.157:0/133634554 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c0094f0 con 0x7f6fac148ef0 2023-10-06T04:03:23.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.689+0000 7f6fabfff700 1 --2- 172.21.15.157:0/133634554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fac0c3e50 0x7f6fac1657d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:03:23.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.689+0000 7f6fb0c13700 1 --2- 172.21.15.157:0/133634554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fac148ef0 0x7f6fac149370 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4007f50 tx=0x7f6fa4009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:23.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.690+0000 7f6f93fff700 1 -- 172.21.15.157:0/133634554 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa400ecb0 con 0x7f6fac148ef0 2023-10-06T04:03:23.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.690+0000 7f6f93fff700 1 -- 172.21.15.157:0/133634554 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa400ee10 con 0x7f6fac148ef0 2023-10-06T04:03:23.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.690+0000 7f6fb2676700 1 -- 172.21.15.157:0/133634554 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fac0c29c0 con 0x7f6fac148ef0 2023-10-06T04:03:23.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.690+0000 7f6f93fff700 1 -- 172.21.15.157:0/133634554 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4014650 con 0x7f6fac148ef0 2023-10-06T04:03:23.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.690+0000 7f6fb2676700 1 -- 172.21.15.157:0/133634554 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fac0c11a0 con 0x7f6fac148ef0 2023-10-06T04:03:23.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.692+0000 7f6fb2676700 1 -- 172.21.15.157:0/133634554 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fac071d00 con 0x7f6fac148ef0 2023-10-06T04:03:23.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.693+0000 7f6f93fff700 1 -- 172.21.15.157:0/133634554 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6fa40149c0 con 0x7f6fac148ef0 2023-10-06T04:03:23.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.693+0000 7f6f93fff700 1 --2- 172.21.15.157:0/133634554 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6f98066f30 0x7f6f980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:23.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.693+0000 7f6f93fff700 1 -- 172.21.15.157:0/133634554 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7f6fa4084fc0 con 0x7f6fac148ef0 2023-10-06T04:03:23.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.694+0000 7f6fabfff700 1 --2- 172.21.15.157:0/133634554 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6f98066f30 0x7f6f980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:23.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.695+0000 7f6fabfff700 1 --2- 172.21.15.157:0/133634554 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6f98066f30 0x7f6f980693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6fac0c1480 tx=0x7f6f9c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:23.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.701+0000 7f6f93fff700 1 -- 172.21.15.157:0/133634554 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6fa4051e80 con 0x7f6fac148ef0 2023-10-06T04:03:23.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.935+0000 7f6fb2676700 1 -- 172.21.15.157:0/133634554 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6fac155960 con 0x7f6fac148ef0 2023-10-06T04:03:23.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.936+0000 7f6f93fff700 1 -- 172.21.15.157:0/133634554 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+6162 (secure 0 0 0) 0x7f6fa40554a0 con 0x7f6fac148ef0 2023-10-06T04:03:23.938 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:03:23.938 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3192541484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3192541484},{"type":"v1","addr":"172.21.15.175:6835","nonce":3192541484}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:13.713593+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279},"failed":[],"damaged":[],"stopped":[3,4],"info":{"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:stopping","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:03:23.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.938+0000 7f6fb2676700 1 -- 172.21.15.157:0/133634554 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6f98066f30 msgr2=0x7f6f980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:23.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.938+0000 7f6fb2676700 1 --2- 172.21.15.157:0/133634554 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6f98066f30 0x7f6f980693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6fac0c1480 tx=0x7f6f9c017000 comp rx=0 tx=0).stop 2023-10-06T04:03:23.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.938+0000 7f6fb2676700 1 -- 172.21.15.157:0/133634554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fac148ef0 msgr2=0x7f6fac149370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:23.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.938+0000 7f6fb2676700 1 --2- 172.21.15.157:0/133634554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fac148ef0 0x7f6fac149370 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4007f50 tx=0x7f6fa4009860 comp rx=0 tx=0).stop 2023-10-06T04:03:23.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.938+0000 7f6fb2676700 1 -- 172.21.15.157:0/133634554 shutdown_connections 2023-10-06T04:03:23.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.938+0000 7f6fb2676700 1 --2- 172.21.15.157:0/133634554 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6f98066f30 0x7f6f980693e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:23.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.938+0000 7f6fb2676700 1 --2- 172.21.15.157:0/133634554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fac148ef0 0x7f6fac149370 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:23.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.938+0000 7f6fb2676700 1 --2- 172.21.15.157:0/133634554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fac0c3e50 0x7f6fac1657d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:23.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.938+0000 7f6fb2676700 1 --2- 172.21.15.157:0/133634554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fac0caa00 0x7f6fac165d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:23.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.938+0000 7f6fb2676700 1 -- 172.21.15.157:0/133634554 >> 172.21.15.157:0/133634554 conn(0x7f6fac0692b0 msgr2=0x7f6fac067530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:23.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.938+0000 7f6fb2676700 1 -- 172.21.15.157:0/133634554 shutdown_connections 2023-10-06T04:03:23.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:23.938+0000 7f6fb2676700 1 -- 172.21.15.157:0/133634554 wait complete. 2023-10-06T04:03:23.942 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 51 2023-10-06T04:03:23.952 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:03:24.979 INFO:tasks.workunit.client.0.smithi157.stdout:removed './206/34' 2023-10-06T04:03:24.979 INFO:tasks.workunit.client.0.smithi157.stdout:removed './206/38' 2023-10-06T04:03:24.979 INFO:tasks.workunit.client.0.smithi157.stdout:removed './206/28' 2023-10-06T04:03:24.979 INFO:tasks.workunit.client.0.smithi157.stdout:removed './206/24' 2023-10-06T04:03:24.979 INFO:tasks.workunit.client.0.smithi157.stdout:removed './206/29' 2023-10-06T04:03:24.980 INFO:tasks.workunit.client.0.smithi157.stdout:removed './206/32' 2023-10-06T04:03:24.980 INFO:tasks.workunit.client.0.smithi157.stdout:removed './206/22' 2023-10-06T04:03:24.980 INFO:tasks.workunit.client.0.smithi157.stdout:removed './206/25' 2023-10-06T04:03:24.980 INFO:tasks.workunit.client.0.smithi157.stdout:removed './206/30' 2023-10-06T04:03:24.980 INFO:tasks.workunit.client.0.smithi157.stdout:removed './206/33' 2023-10-06T04:03:24.980 INFO:tasks.workunit.client.0.smithi157.stdout:removed './206/23' 2023-10-06T04:03:24.980 INFO:tasks.workunit.client.0.smithi157.stdout:removed './206/35' 2023-10-06T04:03:24.980 INFO:tasks.workunit.client.0.smithi157.stdout:removed './206/26' 2023-10-06T04:03:24.980 INFO:tasks.workunit.client.0.smithi157.stdout:removed './206/37' 2023-10-06T04:03:24.980 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './206' 2023-10-06T04:03:24.980 INFO:tasks.workunit.client.0.smithi157.stdout:removed './207/21' 2023-10-06T04:03:24.980 INFO:tasks.workunit.client.0.smithi157.stdout:removed './207/27' 2023-10-06T04:03:24.981 INFO:tasks.workunit.client.0.smithi157.stdout:removed './207/31' 2023-10-06T04:03:24.981 INFO:tasks.workunit.client.0.smithi157.stdout:removed './207/36' 2023-10-06T04:03:24.981 INFO:tasks.workunit.client.0.smithi157.stdout:removed './207/20' 2023-10-06T04:03:24.981 INFO:tasks.workunit.client.0.smithi157.stdout:removed './207/39' 2023-10-06T04:03:24.981 INFO:tasks.workunit.client.0.smithi157.stdout:removed './207/34' 2023-10-06T04:03:24.981 INFO:tasks.workunit.client.0.smithi157.stdout:removed './207/38' 2023-10-06T04:03:24.981 INFO:tasks.workunit.client.0.smithi157.stdout:removed './207/28' 2023-10-06T04:03:24.981 INFO:tasks.workunit.client.0.smithi157.stdout:removed './207/24' 2023-10-06T04:03:24.981 INFO:tasks.workunit.client.0.smithi157.stdout:removed './207/29' 2023-10-06T04:03:24.981 INFO:tasks.workunit.client.0.smithi157.stdout:removed './207/32' 2023-10-06T04:03:24.981 INFO:tasks.workunit.client.0.smithi157.stdout:removed './207/22' 2023-10-06T04:03:24.981 INFO:tasks.workunit.client.0.smithi157.stdout:removed './207/25' 2023-10-06T04:03:24.982 INFO:tasks.workunit.client.0.smithi157.stdout:removed './207/30' 2023-10-06T04:03:24.982 INFO:tasks.workunit.client.0.smithi157.stdout:removed './207/33' 2023-10-06T04:03:24.982 INFO:tasks.workunit.client.0.smithi157.stdout:removed './207/23' 2023-10-06T04:03:24.982 INFO:tasks.workunit.client.0.smithi157.stdout:removed './207/35' 2023-10-06T04:03:24.982 INFO:tasks.workunit.client.0.smithi157.stdout:removed './207/26' 2023-10-06T04:03:24.982 INFO:tasks.workunit.client.0.smithi157.stdout:removed './207/37' 2023-10-06T04:03:24.982 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './207' 2023-10-06T04:03:24.982 INFO:tasks.workunit.client.0.smithi157.stdout:removed './208/21' 2023-10-06T04:03:24.982 INFO:tasks.workunit.client.0.smithi157.stdout:removed './208/27' 2023-10-06T04:03:24.982 INFO:tasks.workunit.client.0.smithi157.stdout:removed './208/31' 2023-10-06T04:03:24.982 INFO:tasks.workunit.client.0.smithi157.stdout:removed './208/36' 2023-10-06T04:03:24.982 INFO:tasks.workunit.client.0.smithi157.stdout:removed './208/20' 2023-10-06T04:03:24.983 INFO:tasks.workunit.client.0.smithi157.stdout:removed './208/39' 2023-10-06T04:03:24.983 INFO:tasks.workunit.client.0.smithi157.stdout:removed './208/34' 2023-10-06T04:03:24.983 INFO:tasks.workunit.client.0.smithi157.stdout:removed './208/38' 2023-10-06T04:03:24.983 INFO:tasks.workunit.client.0.smithi157.stdout:removed './208/28' 2023-10-06T04:03:24.983 INFO:tasks.workunit.client.0.smithi157.stdout:removed './208/24' 2023-10-06T04:03:24.983 INFO:tasks.workunit.client.0.smithi157.stdout:removed './208/29' 2023-10-06T04:03:24.983 INFO:tasks.workunit.client.0.smithi157.stdout:removed './208/32' 2023-10-06T04:03:24.983 INFO:tasks.workunit.client.0.smithi157.stdout:removed './208/22' 2023-10-06T04:03:24.983 INFO:tasks.workunit.client.0.smithi157.stdout:removed './208/25' 2023-10-06T04:03:24.983 INFO:tasks.workunit.client.0.smithi157.stdout:removed './208/30' 2023-10-06T04:03:24.983 INFO:tasks.workunit.client.0.smithi157.stdout:removed './208/33' 2023-10-06T04:03:24.983 INFO:tasks.workunit.client.0.smithi157.stdout:removed './208/23' 2023-10-06T04:03:24.984 INFO:tasks.workunit.client.0.smithi157.stdout:removed './208/35' 2023-10-06T04:03:24.984 INFO:tasks.workunit.client.0.smithi157.stdout:removed './208/26' 2023-10-06T04:03:24.984 INFO:tasks.workunit.client.0.smithi157.stdout:removed './208/37' 2023-10-06T04:03:24.984 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './208' 2023-10-06T04:03:24.984 INFO:tasks.workunit.client.0.smithi157.stdout:removed './209/21' 2023-10-06T04:03:24.984 INFO:tasks.workunit.client.0.smithi157.stdout:removed './209/27' 2023-10-06T04:03:24.984 INFO:tasks.workunit.client.0.smithi157.stdout:removed './209/31' 2023-10-06T04:03:24.984 INFO:tasks.workunit.client.0.smithi157.stdout:removed './209/36' 2023-10-06T04:03:24.984 INFO:tasks.workunit.client.0.smithi157.stdout:removed './209/20' 2023-10-06T04:03:24.984 INFO:tasks.workunit.client.0.smithi157.stdout:removed './209/39' 2023-10-06T04:03:24.984 INFO:tasks.workunit.client.0.smithi157.stdout:removed './209/34' 2023-10-06T04:03:24.984 INFO:tasks.workunit.client.0.smithi157.stdout:removed './209/38' 2023-10-06T04:03:24.984 INFO:tasks.workunit.client.0.smithi157.stdout:removed './209/28' 2023-10-06T04:03:24.985 INFO:tasks.workunit.client.0.smithi157.stdout:removed './209/24' 2023-10-06T04:03:24.985 INFO:tasks.workunit.client.0.smithi157.stdout:removed './209/29' 2023-10-06T04:03:24.985 INFO:tasks.workunit.client.0.smithi157.stdout:removed './209/32' 2023-10-06T04:03:24.985 INFO:tasks.workunit.client.0.smithi157.stdout:removed './209/22' 2023-10-06T04:03:24.985 INFO:tasks.workunit.client.0.smithi157.stdout:removed './209/25' 2023-10-06T04:03:24.985 INFO:tasks.workunit.client.0.smithi157.stdout:removed './209/30' 2023-10-06T04:03:24.985 INFO:tasks.workunit.client.0.smithi157.stdout:removed './209/33' 2023-10-06T04:03:24.985 INFO:tasks.workunit.client.0.smithi157.stdout:removed './209/23' 2023-10-06T04:03:24.985 INFO:tasks.workunit.client.0.smithi157.stdout:removed './209/35' 2023-10-06T04:03:24.985 INFO:tasks.workunit.client.0.smithi157.stdout:removed './209/26' 2023-10-06T04:03:24.985 INFO:tasks.workunit.client.0.smithi157.stdout:removed './209/37' 2023-10-06T04:03:24.985 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './209' 2023-10-06T04:03:24.986 INFO:tasks.workunit.client.0.smithi157.stdout:removed './210/21' 2023-10-06T04:03:24.986 INFO:tasks.workunit.client.0.smithi157.stdout:removed './210/27' 2023-10-06T04:03:24.986 INFO:tasks.workunit.client.0.smithi157.stdout:removed './210/31' 2023-10-06T04:03:24.986 INFO:tasks.workunit.client.0.smithi157.stdout:removed './210/36' 2023-10-06T04:03:24.986 INFO:tasks.workunit.client.0.smithi157.stdout:removed './210/20' 2023-10-06T04:03:24.986 INFO:tasks.workunit.client.0.smithi157.stdout:removed './210/39' 2023-10-06T04:03:24.986 INFO:tasks.workunit.client.0.smithi157.stdout:removed './210/34' 2023-10-06T04:03:24.986 INFO:tasks.workunit.client.0.smithi157.stdout:removed './210/38' 2023-10-06T04:03:24.986 INFO:tasks.workunit.client.0.smithi157.stdout:removed './210/28' 2023-10-06T04:03:24.986 INFO:tasks.workunit.client.0.smithi157.stdout:removed './210/24' 2023-10-06T04:03:24.986 INFO:tasks.workunit.client.0.smithi157.stdout:removed './210/29' 2023-10-06T04:03:24.986 INFO:tasks.workunit.client.0.smithi157.stdout:removed './210/32' 2023-10-06T04:03:24.987 INFO:tasks.workunit.client.0.smithi157.stdout:removed './210/22' 2023-10-06T04:03:24.987 INFO:tasks.workunit.client.0.smithi157.stdout:removed './210/25' 2023-10-06T04:03:24.987 INFO:tasks.workunit.client.0.smithi157.stdout:removed './210/30' 2023-10-06T04:03:24.987 INFO:tasks.workunit.client.0.smithi157.stdout:removed './210/33' 2023-10-06T04:03:24.987 INFO:tasks.workunit.client.0.smithi157.stdout:removed './210/23' 2023-10-06T04:03:24.987 INFO:tasks.workunit.client.0.smithi157.stdout:removed './210/35' 2023-10-06T04:03:24.987 INFO:tasks.workunit.client.0.smithi157.stdout:removed './210/26' 2023-10-06T04:03:24.987 INFO:tasks.workunit.client.0.smithi157.stdout:removed './210/37' 2023-10-06T04:03:24.987 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './210' 2023-10-06T04:03:24.987 INFO:tasks.workunit.client.0.smithi157.stdout:removed './211/21' 2023-10-06T04:03:24.987 INFO:tasks.workunit.client.0.smithi157.stdout:removed './211/27' 2023-10-06T04:03:24.987 INFO:tasks.workunit.client.0.smithi157.stdout:removed './211/31' 2023-10-06T04:03:24.988 INFO:tasks.workunit.client.0.smithi157.stdout:removed './211/36' 2023-10-06T04:03:24.988 INFO:tasks.workunit.client.0.smithi157.stdout:removed './211/20' 2023-10-06T04:03:24.988 INFO:tasks.workunit.client.0.smithi157.stdout:removed './211/39' 2023-10-06T04:03:24.988 INFO:tasks.workunit.client.0.smithi157.stdout:removed './211/34' 2023-10-06T04:03:24.988 INFO:tasks.workunit.client.0.smithi157.stdout:removed './211/38' 2023-10-06T04:03:24.988 INFO:tasks.workunit.client.0.smithi157.stdout:removed './211/28' 2023-10-06T04:03:24.988 INFO:tasks.workunit.client.0.smithi157.stdout:removed './211/24' 2023-10-06T04:03:24.988 INFO:tasks.workunit.client.0.smithi157.stdout:removed './211/29' 2023-10-06T04:03:24.988 INFO:tasks.workunit.client.0.smithi157.stdout:removed './211/32' 2023-10-06T04:03:24.988 INFO:tasks.workunit.client.0.smithi157.stdout:removed './211/22' 2023-10-06T04:03:24.988 INFO:tasks.workunit.client.0.smithi157.stdout:removed './211/25' 2023-10-06T04:03:24.988 INFO:tasks.workunit.client.0.smithi157.stdout:removed './211/30' 2023-10-06T04:03:24.988 INFO:tasks.workunit.client.0.smithi157.stdout:removed './211/33' 2023-10-06T04:03:24.989 INFO:tasks.workunit.client.0.smithi157.stdout:removed './211/23' 2023-10-06T04:03:24.989 INFO:tasks.workunit.client.0.smithi157.stdout:removed './211/35' 2023-10-06T04:03:24.989 INFO:tasks.workunit.client.0.smithi157.stdout:removed './211/26' 2023-10-06T04:03:24.989 INFO:tasks.workunit.client.0.smithi157.stdout:removed './211/37' 2023-10-06T04:03:24.989 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './211' 2023-10-06T04:03:24.989 INFO:tasks.workunit.client.0.smithi157.stdout:removed './212/21' 2023-10-06T04:03:24.989 INFO:tasks.workunit.client.0.smithi157.stdout:removed './212/27' 2023-10-06T04:03:24.989 INFO:tasks.workunit.client.0.smithi157.stdout:removed './212/31' 2023-10-06T04:03:24.989 INFO:tasks.workunit.client.0.smithi157.stdout:removed './212/36' 2023-10-06T04:03:24.989 INFO:tasks.workunit.client.0.smithi157.stdout:removed './212/20' 2023-10-06T04:03:24.989 INFO:tasks.workunit.client.0.smithi157.stdout:removed './212/39' 2023-10-06T04:03:24.989 INFO:tasks.workunit.client.0.smithi157.stdout:removed './212/34' 2023-10-06T04:03:24.990 INFO:tasks.workunit.client.0.smithi157.stdout:removed './212/38' 2023-10-06T04:03:24.990 INFO:tasks.workunit.client.0.smithi157.stdout:removed './212/28' 2023-10-06T04:03:24.990 INFO:tasks.workunit.client.0.smithi157.stdout:removed './212/24' 2023-10-06T04:03:24.990 INFO:tasks.workunit.client.0.smithi157.stdout:removed './212/29' 2023-10-06T04:03:24.990 INFO:tasks.workunit.client.0.smithi157.stdout:removed './212/32' 2023-10-06T04:03:24.990 INFO:tasks.workunit.client.0.smithi157.stdout:removed './212/22' 2023-10-06T04:03:24.990 INFO:tasks.workunit.client.0.smithi157.stdout:removed './212/25' 2023-10-06T04:03:24.990 INFO:tasks.workunit.client.0.smithi157.stdout:removed './212/30' 2023-10-06T04:03:24.990 INFO:tasks.workunit.client.0.smithi157.stdout:removed './212/33' 2023-10-06T04:03:24.990 INFO:tasks.workunit.client.0.smithi157.stdout:removed './212/23' 2023-10-06T04:03:24.990 INFO:tasks.workunit.client.0.smithi157.stdout:removed './212/35' 2023-10-06T04:03:24.990 INFO:tasks.workunit.client.0.smithi157.stdout:removed './212/26' 2023-10-06T04:03:24.991 INFO:tasks.workunit.client.0.smithi157.stdout:removed './212/37' 2023-10-06T04:03:24.991 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './212' 2023-10-06T04:03:24.991 INFO:tasks.workunit.client.0.smithi157.stdout:removed './213/21' 2023-10-06T04:03:24.991 INFO:tasks.workunit.client.0.smithi157.stdout:removed './213/27' 2023-10-06T04:03:24.991 INFO:tasks.workunit.client.0.smithi157.stdout:removed './213/31' 2023-10-06T04:03:24.991 INFO:tasks.workunit.client.0.smithi157.stdout:removed './213/36' 2023-10-06T04:03:24.991 INFO:tasks.workunit.client.0.smithi157.stdout:removed './213/20' 2023-10-06T04:03:24.991 INFO:tasks.workunit.client.0.smithi157.stdout:removed './213/39' 2023-10-06T04:03:24.991 INFO:tasks.workunit.client.0.smithi157.stdout:removed './213/34' 2023-10-06T04:03:24.991 INFO:tasks.workunit.client.0.smithi157.stdout:removed './213/38' 2023-10-06T04:03:24.991 INFO:tasks.workunit.client.0.smithi157.stdout:removed './213/28' 2023-10-06T04:03:24.991 INFO:tasks.workunit.client.0.smithi157.stdout:removed './213/24' 2023-10-06T04:03:24.991 INFO:tasks.workunit.client.0.smithi157.stdout:removed './213/29' 2023-10-06T04:03:24.992 INFO:tasks.workunit.client.0.smithi157.stdout:removed './213/32' 2023-10-06T04:03:24.992 INFO:tasks.workunit.client.0.smithi157.stdout:removed './213/22' 2023-10-06T04:03:24.992 INFO:tasks.workunit.client.0.smithi157.stdout:removed './213/25' 2023-10-06T04:03:24.992 INFO:tasks.workunit.client.0.smithi157.stdout:removed './213/30' 2023-10-06T04:03:24.992 INFO:tasks.workunit.client.0.smithi157.stdout:removed './213/33' 2023-10-06T04:03:24.992 INFO:tasks.workunit.client.0.smithi157.stdout:removed './213/23' 2023-10-06T04:03:24.992 INFO:tasks.workunit.client.0.smithi157.stdout:removed './213/35' 2023-10-06T04:03:24.992 INFO:tasks.workunit.client.0.smithi157.stdout:removed './213/26' 2023-10-06T04:03:24.992 INFO:tasks.workunit.client.0.smithi157.stdout:removed './213/37' 2023-10-06T04:03:24.992 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './213' 2023-10-06T04:03:24.992 INFO:tasks.workunit.client.0.smithi157.stdout:removed './214/21' 2023-10-06T04:03:24.992 INFO:tasks.workunit.client.0.smithi157.stdout:removed './214/27' 2023-10-06T04:03:24.993 INFO:tasks.workunit.client.0.smithi157.stdout:removed './214/31' 2023-10-06T04:03:24.993 INFO:tasks.workunit.client.0.smithi157.stdout:removed './214/36' 2023-10-06T04:03:24.993 INFO:tasks.workunit.client.0.smithi157.stdout:removed './214/20' 2023-10-06T04:03:24.993 INFO:tasks.workunit.client.0.smithi157.stdout:removed './214/39' 2023-10-06T04:03:24.993 INFO:tasks.workunit.client.0.smithi157.stdout:removed './214/34' 2023-10-06T04:03:24.993 INFO:tasks.workunit.client.0.smithi157.stdout:removed './214/38' 2023-10-06T04:03:24.993 INFO:tasks.workunit.client.0.smithi157.stdout:removed './214/28' 2023-10-06T04:03:24.993 INFO:tasks.workunit.client.0.smithi157.stdout:removed './214/24' 2023-10-06T04:03:24.993 INFO:tasks.workunit.client.0.smithi157.stdout:removed './214/29' 2023-10-06T04:03:24.993 INFO:tasks.workunit.client.0.smithi157.stdout:removed './214/32' 2023-10-06T04:03:24.993 INFO:tasks.workunit.client.0.smithi157.stdout:removed './214/22' 2023-10-06T04:03:24.993 INFO:tasks.workunit.client.0.smithi157.stdout:removed './214/25' 2023-10-06T04:03:24.994 INFO:tasks.workunit.client.0.smithi157.stdout:removed './214/30' 2023-10-06T04:03:24.994 INFO:tasks.workunit.client.0.smithi157.stdout:removed './214/33' 2023-10-06T04:03:24.994 INFO:tasks.workunit.client.0.smithi157.stdout:removed './214/23' 2023-10-06T04:03:24.994 INFO:tasks.workunit.client.0.smithi157.stdout:removed './214/35' 2023-10-06T04:03:24.994 INFO:tasks.workunit.client.0.smithi157.stdout:removed './214/26' 2023-10-06T04:03:24.994 INFO:tasks.workunit.client.0.smithi157.stdout:removed './214/37' 2023-10-06T04:03:24.994 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './214' 2023-10-06T04:03:24.994 INFO:tasks.workunit.client.0.smithi157.stdout:removed './215/21' 2023-10-06T04:03:24.994 INFO:tasks.workunit.client.0.smithi157.stdout:removed './215/27' 2023-10-06T04:03:24.994 INFO:tasks.workunit.client.0.smithi157.stdout:removed './215/31' 2023-10-06T04:03:24.994 INFO:tasks.workunit.client.0.smithi157.stdout:removed './215/36' 2023-10-06T04:03:24.994 INFO:tasks.workunit.client.0.smithi157.stdout:removed './215/20' 2023-10-06T04:03:24.995 INFO:tasks.workunit.client.0.smithi157.stdout:removed './215/39' 2023-10-06T04:03:24.995 INFO:tasks.workunit.client.0.smithi157.stdout:removed './215/34' 2023-10-06T04:03:24.995 INFO:tasks.workunit.client.0.smithi157.stdout:removed './215/38' 2023-10-06T04:03:24.995 INFO:tasks.workunit.client.0.smithi157.stdout:removed './215/28' 2023-10-06T04:03:24.995 INFO:tasks.workunit.client.0.smithi157.stdout:removed './215/24' 2023-10-06T04:03:24.995 INFO:tasks.workunit.client.0.smithi157.stdout:removed './215/29' 2023-10-06T04:03:24.995 INFO:tasks.workunit.client.0.smithi157.stdout:removed './215/32' 2023-10-06T04:03:24.995 INFO:tasks.workunit.client.0.smithi157.stdout:removed './215/22' 2023-10-06T04:03:24.995 INFO:tasks.workunit.client.0.smithi157.stdout:removed './215/25' 2023-10-06T04:03:24.995 INFO:tasks.workunit.client.0.smithi157.stdout:removed './215/30' 2023-10-06T04:03:24.995 INFO:tasks.workunit.client.0.smithi157.stdout:removed './215/33' 2023-10-06T04:03:24.995 INFO:tasks.workunit.client.0.smithi157.stdout:removed './215/23' 2023-10-06T04:03:24.995 INFO:tasks.workunit.client.0.smithi157.stdout:removed './215/35' 2023-10-06T04:03:24.996 INFO:tasks.workunit.client.0.smithi157.stdout:removed './215/26' 2023-10-06T04:03:24.996 INFO:tasks.workunit.client.0.smithi157.stdout:removed './215/37' 2023-10-06T04:03:24.996 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './215' 2023-10-06T04:03:24.996 INFO:tasks.workunit.client.0.smithi157.stdout:removed './216/21' 2023-10-06T04:03:24.996 INFO:tasks.workunit.client.0.smithi157.stdout:removed './216/27' 2023-10-06T04:03:24.996 INFO:tasks.workunit.client.0.smithi157.stdout:removed './216/31' 2023-10-06T04:03:24.996 INFO:tasks.workunit.client.0.smithi157.stdout:removed './216/36' 2023-10-06T04:03:24.996 INFO:tasks.workunit.client.0.smithi157.stdout:removed './216/20' 2023-10-06T04:03:24.996 INFO:tasks.workunit.client.0.smithi157.stdout:removed './216/39' 2023-10-06T04:03:24.996 INFO:tasks.workunit.client.0.smithi157.stdout:removed './216/34' 2023-10-06T04:03:24.996 INFO:tasks.workunit.client.0.smithi157.stdout:removed './216/38' 2023-10-06T04:03:25.953 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:03:26.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.070+0000 7f9dfe223700 1 Processor -- start 2023-10-06T04:03:26.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.070+0000 7f9dfe223700 1 -- start start 2023-10-06T04:03:26.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.070+0000 7f9dfe223700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9df8069ea0 0x7f9df80631f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:26.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.070+0000 7f9dfe223700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df80c48f0 con 0x7f9df806a270 2023-10-06T04:03:26.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.070+0000 7f9dfe223700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df80c3d20 con 0x7f9df806a860 2023-10-06T04:03:26.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.070+0000 7f9dfe223700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df8063840 con 0x7f9df8069ea0 2023-10-06T04:03:26.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.071+0000 7f9df7fff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f9df806a270 0x7f9df814faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59690/0 (socket says 172.21.15.157:59690) 2023-10-06T04:03:26.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.071+0000 7f9df7fff700 1 -- 172.21.15.157:0/4137411781 learned_addr learned my addr 172.21.15.157:0/4137411781 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:03:26.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.071+0000 7f9df77fe700 1 --2- 172.21.15.157:0/4137411781 >> v2:172.21.15.175:3300/0 conn(0x7f9df8069ea0 0x7f9df80631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:26.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.071+0000 7f9df67fc700 1 -- 172.21.15.157:0/4137411781 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4230689720 0 0) 0x7f9df80c48f0 con 0x7f9df806a270 2023-10-06T04:03:26.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.071+0000 7f9df67fc700 1 -- 172.21.15.157:0/4137411781 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9de0003570 con 0x7f9df806a270 2023-10-06T04:03:26.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.071+0000 7f9df67fc700 1 -- 172.21.15.157:0/4137411781 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 63369727 0 0) 0x7f9de0003570 con 0x7f9df806a270 2023-10-06T04:03:26.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.071+0000 7f9df67fc700 1 -- 172.21.15.157:0/4137411781 >> v2:172.21.15.175:3300/0 conn(0x7f9df8069ea0 msgr2=0x7f9df80631f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:26.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.071+0000 7f9df67fc700 1 --2- 172.21.15.157:0/4137411781 >> v2:172.21.15.175:3300/0 conn(0x7f9df8069ea0 0x7f9df80631f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:26.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.071+0000 7f9df67fc700 1 -- 172.21.15.157:0/4137411781 >> v1:172.21.15.175:6789/0 conn(0x7f9df806a860 legacy=0x7f9df80c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:26.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.071+0000 7f9df67fc700 1 -- 172.21.15.157:0/4137411781 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df80c3fc0 con 0x7f9df806a270 2023-10-06T04:03:26.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.071+0000 7f9df67fc700 1 -- 172.21.15.157:0/4137411781 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f9df00017f0 con 0x7f9df806a270 2023-10-06T04:03:26.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.071+0000 7f9df67fc700 1 -- 172.21.15.157:0/4137411781 >> v1:172.21.15.157:6789/0 conn(0x7f9df806a270 legacy=0x7f9df814faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:26.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.071+0000 7f9df67fc700 1 --2- 172.21.15.157:0/4137411781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9de0003a60 0x7f9de0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:26.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.071+0000 7f9df67fc700 1 --2- 172.21.15.157:0/4137411781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9de0004810 0x7f9de0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:26.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.071+0000 7f9df67fc700 1 --2- 172.21.15.157:0/4137411781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9de00051c0 0x7f9de000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:26.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.071+0000 7f9df67fc700 1 -- 172.21.15.157:0/4137411781 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df8063840 con 0x7f9de0004810 2023-10-06T04:03:26.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.071+0000 7f9df67fc700 1 -- 172.21.15.157:0/4137411781 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de000c0d0 con 0x7f9de0003a60 2023-10-06T04:03:26.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.071+0000 7f9df67fc700 1 -- 172.21.15.157:0/4137411781 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de000c290 con 0x7f9de00051c0 2023-10-06T04:03:26.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.071+0000 7f9df6ffd700 1 --2- 172.21.15.157:0/4137411781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9de0004810 0x7f9de0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:26.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.071+0000 7f9df77fe700 1 --2- 172.21.15.157:0/4137411781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9de0003a60 0x7f9de0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:26.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.071+0000 7f9df7fff700 1 --2- 172.21.15.157:0/4137411781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9de00051c0 0x7f9de000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:26.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.072+0000 7f9df6ffd700 1 -- 172.21.15.157:0/4137411781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9de00051c0 msgr2=0x7f9de000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:26.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.072+0000 7f9df6ffd700 1 --2- 172.21.15.157:0/4137411781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9de00051c0 0x7f9de000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:26.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.072+0000 7f9df6ffd700 1 -- 172.21.15.157:0/4137411781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9de0003a60 msgr2=0x7f9de0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:26.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.072+0000 7f9df6ffd700 1 --2- 172.21.15.157:0/4137411781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9de0003a60 0x7f9de0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:26.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.072+0000 7f9df6ffd700 1 -- 172.21.15.157:0/4137411781 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de0003570 con 0x7f9de0004810 2023-10-06T04:03:26.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.072+0000 7f9df7fff700 1 -- 172.21.15.157:0/4137411781 reap_dead start 2023-10-06T04:03:26.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.072+0000 7f9df6ffd700 1 --2- 172.21.15.157:0/4137411781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9de0004810 0x7f9de0004c80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9dec0018b0 tx=0x7f9dec007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e94158be5d50e3f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:26.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.072+0000 7f9df67fc700 1 -- 172.21.15.157:0/4137411781 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec012420 con 0x7f9de0004810 2023-10-06T04:03:26.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.072+0000 7f9df67fc700 1 -- 172.21.15.157:0/4137411781 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dec012580 con 0x7f9de0004810 2023-10-06T04:03:26.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.072+0000 7f9df67fc700 1 -- 172.21.15.157:0/4137411781 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec012920 con 0x7f9de0004810 2023-10-06T04:03:26.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.072+0000 7f9dfe223700 1 -- 172.21.15.157:0/4137411781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9de0004810 msgr2=0x7f9de0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:26.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.072+0000 7f9dfe223700 1 --2- 172.21.15.157:0/4137411781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9de0004810 0x7f9de0004c80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9dec0018b0 tx=0x7f9dec007d30 comp rx=0 tx=0).stop 2023-10-06T04:03:26.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.073+0000 7f9dfe223700 1 -- 172.21.15.157:0/4137411781 shutdown_connections 2023-10-06T04:03:26.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.073+0000 7f9dfe223700 1 --2- 172.21.15.157:0/4137411781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9de0004810 0x7f9de0004c80 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:26.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.073+0000 7f9dfe223700 1 -- 172.21.15.157:0/4137411781 >> 172.21.15.157:0/4137411781 conn(0x7f9df80c8b20 msgr2=0x7f9df80c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:26.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.073+0000 7f9dfe223700 1 -- 172.21.15.157:0/4137411781 shutdown_connections 2023-10-06T04:03:26.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.073+0000 7f9dfe223700 1 -- 172.21.15.157:0/4137411781 wait complete. 2023-10-06T04:03:26.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.074+0000 7f9dfe223700 1 Processor -- start 2023-10-06T04:03:26.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.074+0000 7f9dfe223700 1 -- start start 2023-10-06T04:03:26.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.075+0000 7f9dfe223700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9de0004810 0x7f9df81459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:26.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.075+0000 7f9dfe223700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9df814eeb0 0x7f9df814c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:26.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.075+0000 7f9dfe223700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9df8165f00 0x7f9df8166370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:26.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.075+0000 7f9dfe223700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df814adc0 con 0x7f9df814eeb0 2023-10-06T04:03:26.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.075+0000 7f9dfe223700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df814a930 con 0x7f9df8165f00 2023-10-06T04:03:26.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.075+0000 7f9dfe223700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df81668b0 con 0x7f9de0004810 2023-10-06T04:03:26.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.075+0000 7f9df6ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9df814eeb0 0x7f9df814c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:26.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.075+0000 7f9df6ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9df814eeb0 0x7f9df814c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37472/0 (socket says 172.21.15.157:37472) 2023-10-06T04:03:26.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.075+0000 7f9df6ffd700 1 -- 172.21.15.157:0/3273565480 learned_addr learned my addr 172.21.15.157:0/3273565480 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:26.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.075+0000 7f9df77fe700 1 --2- 172.21.15.157:0/3273565480 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9de0004810 0x7f9df81459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:26.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.075+0000 7f9df7fff700 1 --2- 172.21.15.157:0/3273565480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9df8165f00 0x7f9df8166370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:26.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.075+0000 7f9df6ffd700 1 -- 172.21.15.157:0/3273565480 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9de0004810 msgr2=0x7f9df81459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:26.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.075+0000 7f9df6ffd700 1 --2- 172.21.15.157:0/3273565480 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9de0004810 0x7f9df81459e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:26.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.075+0000 7f9df6ffd700 1 -- 172.21.15.157:0/3273565480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9df8165f00 msgr2=0x7f9df8166370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:26.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.075+0000 7f9df6ffd700 1 --2- 172.21.15.157:0/3273565480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9df8165f00 0x7f9df8166370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:26.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.075+0000 7f9df6ffd700 1 -- 172.21.15.157:0/3273565480 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df80c84c0 con 0x7f9df814eeb0 2023-10-06T04:03:26.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.075+0000 7f9df6ffd700 1 --2- 172.21.15.157:0/3273565480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9df814eeb0 0x7f9df814c4e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9dec00c3f0 tx=0x7f9dec015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:26.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.075+0000 7f9ddf7fe700 1 -- 172.21.15.157:0/3273565480 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec006b00 con 0x7f9df814eeb0 2023-10-06T04:03:26.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.075+0000 7f9ddf7fe700 1 -- 172.21.15.157:0/3273565480 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dec006c80 con 0x7f9df814eeb0 2023-10-06T04:03:26.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.075+0000 7f9dfe223700 1 -- 172.21.15.157:0/3273565480 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df80c8780 con 0x7f9df814eeb0 2023-10-06T04:03:26.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.075+0000 7f9ddf7fe700 1 -- 172.21.15.157:0/3273565480 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec007030 con 0x7f9df814eeb0 2023-10-06T04:03:26.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.075+0000 7f9dfe223700 1 -- 172.21.15.157:0/3273565480 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df814a2e0 con 0x7f9df814eeb0 2023-10-06T04:03:26.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.077+0000 7f9dfe223700 1 -- 172.21.15.157:0/3273565480 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df80642a0 con 0x7f9df814eeb0 2023-10-06T04:03:26.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.077+0000 7f9ddf7fe700 1 -- 172.21.15.157:0/3273565480 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9dec0073e0 con 0x7f9df814eeb0 2023-10-06T04:03:26.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.077+0000 7f9ddf7fe700 1 --2- 172.21.15.157:0/3273565480 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9de4066f30 0x7f9de40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:26.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.077+0000 7f9ddf7fe700 1 -- 172.21.15.157:0/3273565480 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7f9dec085e60 con 0x7f9df814eeb0 2023-10-06T04:03:26.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.077+0000 7f9df77fe700 1 --2- 172.21.15.157:0/3273565480 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9de4066f30 0x7f9de40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:26.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.078+0000 7f9df77fe700 1 --2- 172.21.15.157:0/3273565480 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9de4066f30 0x7f9de40693e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9df804d290 tx=0x7f9de80091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:26.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.086+0000 7f9ddf7fe700 1 -- 172.21.15.157:0/3273565480 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9dec052de0 con 0x7f9df814eeb0 2023-10-06T04:03:26.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.315+0000 7f9dfe223700 1 -- 172.21.15.157:0/3273565480 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9df80c42a0 con 0x7f9df814eeb0 2023-10-06T04:03:26.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.315+0000 7f9ddf7fe700 1 -- 172.21.15.157:0/3273565480 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+6162 (secure 0 0 0) 0x7f9dec017090 con 0x7f9df814eeb0 2023-10-06T04:03:26.317 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:03:26.317 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3192541484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3192541484},{"type":"v1","addr":"172.21.15.175:6835","nonce":3192541484}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:13.713593+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279},"failed":[],"damaged":[],"stopped":[3,4],"info":{"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:stopping","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:03:26.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.317+0000 7f9dfe223700 1 -- 172.21.15.157:0/3273565480 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9de4066f30 msgr2=0x7f9de40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:26.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.317+0000 7f9dfe223700 1 --2- 172.21.15.157:0/3273565480 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9de4066f30 0x7f9de40693e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9df804d290 tx=0x7f9de80091a0 comp rx=0 tx=0).stop 2023-10-06T04:03:26.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.317+0000 7f9dfe223700 1 -- 172.21.15.157:0/3273565480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9df814eeb0 msgr2=0x7f9df814c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:26.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.317+0000 7f9dfe223700 1 --2- 172.21.15.157:0/3273565480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9df814eeb0 0x7f9df814c4e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9dec00c3f0 tx=0x7f9dec015000 comp rx=0 tx=0).stop 2023-10-06T04:03:26.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.317+0000 7f9dfe223700 1 -- 172.21.15.157:0/3273565480 shutdown_connections 2023-10-06T04:03:26.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.317+0000 7f9dfe223700 1 --2- 172.21.15.157:0/3273565480 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9de4066f30 0x7f9de40693e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:26.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.317+0000 7f9dfe223700 1 --2- 172.21.15.157:0/3273565480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9df8165f00 0x7f9df8166370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:26.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.317+0000 7f9dfe223700 1 --2- 172.21.15.157:0/3273565480 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9de0004810 0x7f9df81459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:26.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.317+0000 7f9dfe223700 1 --2- 172.21.15.157:0/3273565480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9df814eeb0 0x7f9df814c4e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:26.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.317+0000 7f9dfe223700 1 -- 172.21.15.157:0/3273565480 >> 172.21.15.157:0/3273565480 conn(0x7f9df80c8b20 msgr2=0x7f9df806c280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:26.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.317+0000 7f9dfe223700 1 -- 172.21.15.157:0/3273565480 shutdown_connections 2023-10-06T04:03:26.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:26.317+0000 7f9dfe223700 1 -- 172.21.15.157:0/3273565480 wait complete. 2023-10-06T04:03:26.320 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 51 2023-10-06T04:03:26.331 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:03:28.332 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:03:28.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.458+0000 7fda1f59e700 1 Processor -- start 2023-10-06T04:03:28.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.458+0000 7fda1f59e700 1 -- start start 2023-10-06T04:03:28.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.459+0000 7fda1f59e700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fda200b1e40 0x7fda20157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:28.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.459+0000 7fda1f59e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fda200b04d0 0x7fda2006dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:28.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.459+0000 7fda1f59e700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda2006d8d0 con 0x7fda2006c2d0 2023-10-06T04:03:28.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.459+0000 7fda1f59e700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda200af8f0 con 0x7fda200b1e40 2023-10-06T04:03:28.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.459+0000 7fda1f59e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda2015a690 con 0x7fda200b04d0 2023-10-06T04:03:28.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.459+0000 7fda1e59c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fda200b1e40 0x7fda20157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:28.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.459+0000 7fda1ed9d700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fda2006c2d0 0x7fda201563b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59698/0 (socket says 172.21.15.157:59698) 2023-10-06T04:03:28.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.459+0000 7fda1ed9d700 1 -- 172.21.15.157:0/1821257571 learned_addr learned my addr 172.21.15.157:0/1821257571 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:03:28.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.459+0000 7fda1dd9b700 1 --2- 172.21.15.157:0/1821257571 >> v2:172.21.15.175:3300/0 conn(0x7fda200b04d0 0x7fda2006dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:28.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.459+0000 7fda1e59c700 1 -- 172.21.15.157:0/1821257571 >> v2:172.21.15.175:3300/0 conn(0x7fda200b04d0 msgr2=0x7fda2006dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:28.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.459+0000 7fda1e59c700 1 --2- 172.21.15.157:0/1821257571 >> v2:172.21.15.175:3300/0 conn(0x7fda200b04d0 0x7fda2006dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:28.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.459+0000 7fda1e59c700 1 -- 172.21.15.157:0/1821257571 >> v1:172.21.15.157:6789/0 conn(0x7fda2006c2d0 legacy=0x7fda201563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:28.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.459+0000 7fda1e59c700 1 -- 172.21.15.157:0/1821257571 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2006ccc0 con 0x7fda200b1e40 2023-10-06T04:03:28.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.459+0000 7fda1d59a700 1 -- 172.21.15.157:0/1821257571 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1574770360 0 0) 0x7fda2006d8d0 con 0x7fda2006c2d0 2023-10-06T04:03:28.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.460+0000 7fda1e59c700 1 --2- 172.21.15.157:0/1821257571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda200b1e40 0x7fda20157750 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fda200896d0 tx=0x7fda100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5ca47866f38dcd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:28.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.460+0000 7fda1d59a700 1 -- 172.21.15.157:0/1821257571 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1000b6b0 con 0x7fda200b1e40 2023-10-06T04:03:28.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.460+0000 7fda1d59a700 1 -- 172.21.15.157:0/1821257571 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda1000b810 con 0x7fda200b1e40 2023-10-06T04:03:28.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.460+0000 7fda1d59a700 1 -- 172.21.15.157:0/1821257571 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1000bbb0 con 0x7fda200b1e40 2023-10-06T04:03:28.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.460+0000 7fda1f59e700 1 -- 172.21.15.157:0/1821257571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda200b1e40 msgr2=0x7fda20157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:28.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.460+0000 7fda1f59e700 1 --2- 172.21.15.157:0/1821257571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda200b1e40 0x7fda20157750 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fda200896d0 tx=0x7fda100091a0 comp rx=0 tx=0).stop 2023-10-06T04:03:28.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.460+0000 7fda1f59e700 1 -- 172.21.15.157:0/1821257571 shutdown_connections 2023-10-06T04:03:28.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.460+0000 7fda1f59e700 1 --2- 172.21.15.157:0/1821257571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda200b1e40 0x7fda20157750 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:28.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.460+0000 7fda1f59e700 1 --2- 172.21.15.157:0/1821257571 >> v2:172.21.15.175:3300/0 conn(0x7fda200b04d0 0x7fda2006dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:28.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.460+0000 7fda1f59e700 1 -- 172.21.15.157:0/1821257571 >> 172.21.15.157:0/1821257571 conn(0x7fda200cc540 msgr2=0x7fda200cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:28.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.460+0000 7fda1f59e700 1 -- 172.21.15.157:0/1821257571 shutdown_connections 2023-10-06T04:03:28.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.460+0000 7fda1f59e700 1 -- 172.21.15.157:0/1821257571 wait complete. 2023-10-06T04:03:28.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.461+0000 7fda1f59e700 1 Processor -- start 2023-10-06T04:03:28.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.462+0000 7fda1f59e700 1 -- start start 2023-10-06T04:03:28.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.462+0000 7fda1f59e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda2006c2d0 0x7fda201533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:28.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.462+0000 7fda1f59e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda200b04d0 0x7fda200c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:28.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.462+0000 7fda1f59e700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda200b1e40 0x7fda20151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:28.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.462+0000 7fda1f59e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda200c8190 con 0x7fda2006c2d0 2023-10-06T04:03:28.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.462+0000 7fda1f59e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda20150d10 con 0x7fda200b04d0 2023-10-06T04:03:28.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.462+0000 7fda1f59e700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda20150ff0 con 0x7fda200b1e40 2023-10-06T04:03:28.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.462+0000 7fda1e59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda2006c2d0 0x7fda201533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:28.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.462+0000 7fda1e59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda2006c2d0 0x7fda201533e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37494/0 (socket says 172.21.15.157:37494) 2023-10-06T04:03:28.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.462+0000 7fda1e59c700 1 -- 172.21.15.157:0/958553702 learned_addr learned my addr 172.21.15.157:0/958553702 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:28.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.462+0000 7fda1dd9b700 1 --2- 172.21.15.157:0/958553702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda200b04d0 0x7fda200c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:28.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.462+0000 7fda1ed9d700 1 --2- 172.21.15.157:0/958553702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda200b1e40 0x7fda20151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:28.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.463+0000 7fda1e59c700 1 -- 172.21.15.157:0/958553702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda200b1e40 msgr2=0x7fda20151800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:28.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.463+0000 7fda1e59c700 1 --2- 172.21.15.157:0/958553702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda200b1e40 0x7fda20151800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:28.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.463+0000 7fda1e59c700 1 -- 172.21.15.157:0/958553702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda200b04d0 msgr2=0x7fda200c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:28.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.463+0000 7fda1e59c700 1 --2- 172.21.15.157:0/958553702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda200b04d0 0x7fda200c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:28.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.463+0000 7fda1e59c700 1 -- 172.21.15.157:0/958553702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda10009420 con 0x7fda2006c2d0 2023-10-06T04:03:28.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.463+0000 7fda1e59c700 1 --2- 172.21.15.157:0/958553702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda2006c2d0 0x7fda201533e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fda200899e0 tx=0x7fda10017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:28.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.463+0000 7fda0e7fc700 1 -- 172.21.15.157:0/958553702 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda10000d10 con 0x7fda2006c2d0 2023-10-06T04:03:28.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.463+0000 7fda0e7fc700 1 -- 172.21.15.157:0/958553702 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda1000bb90 con 0x7fda2006c2d0 2023-10-06T04:03:28.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.463+0000 7fda1f59e700 1 -- 172.21.15.157:0/958553702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda200c98f0 con 0x7fda2006c2d0 2023-10-06T04:03:28.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.463+0000 7fda0e7fc700 1 -- 172.21.15.157:0/958553702 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda10011670 con 0x7fda2006c2d0 2023-10-06T04:03:28.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.463+0000 7fda1f59e700 1 -- 172.21.15.157:0/958553702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda200c95c0 con 0x7fda2006c2d0 2023-10-06T04:03:28.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.464+0000 7fda1f59e700 1 -- 172.21.15.157:0/958553702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda20088f90 con 0x7fda2006c2d0 2023-10-06T04:03:28.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.464+0000 7fda0e7fc700 1 -- 172.21.15.157:0/958553702 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fda10011aa0 con 0x7fda2006c2d0 2023-10-06T04:03:28.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.465+0000 7fda0e7fc700 1 --2- 172.21.15.157:0/958553702 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fda00066f30 0x7fda000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:28.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.465+0000 7fda0e7fc700 1 -- 172.21.15.157:0/958553702 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7fda10086140 con 0x7fda2006c2d0 2023-10-06T04:03:28.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.465+0000 7fda1dd9b700 1 --2- 172.21.15.157:0/958553702 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fda00066f30 0x7fda000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:28.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.465+0000 7fda1dd9b700 1 --2- 172.21.15.157:0/958553702 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fda00066f30 0x7fda000693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fda2006df80 tx=0x7fda14008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:28.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.469+0000 7fda0e7fc700 1 -- 172.21.15.157:0/958553702 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda10053040 con 0x7fda2006c2d0 2023-10-06T04:03:28.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.693+0000 7fda1f59e700 1 -- 172.21.15.157:0/958553702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda20151420 con 0x7fda2006c2d0 2023-10-06T04:03:28.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.694+0000 7fda0e7fc700 1 -- 172.21.15.157:0/958553702 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+6162 (secure 0 0 0) 0x7fda10056620 con 0x7fda2006c2d0 2023-10-06T04:03:28.695 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:03:28.695 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3192541484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3192541484},{"type":"v1","addr":"172.21.15.175:6835","nonce":3192541484}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:13.713593+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279},"failed":[],"damaged":[],"stopped":[3,4],"info":{"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:stopping","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:03:28.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.695+0000 7fda1f59e700 1 -- 172.21.15.157:0/958553702 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fda00066f30 msgr2=0x7fda000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:28.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.695+0000 7fda1f59e700 1 --2- 172.21.15.157:0/958553702 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fda00066f30 0x7fda000693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fda2006df80 tx=0x7fda14008040 comp rx=0 tx=0).stop 2023-10-06T04:03:28.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.696+0000 7fda1f59e700 1 -- 172.21.15.157:0/958553702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda2006c2d0 msgr2=0x7fda201533e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:28.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.696+0000 7fda1f59e700 1 --2- 172.21.15.157:0/958553702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda2006c2d0 0x7fda201533e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fda200899e0 tx=0x7fda10017000 comp rx=0 tx=0).stop 2023-10-06T04:03:28.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.696+0000 7fda1f59e700 1 -- 172.21.15.157:0/958553702 shutdown_connections 2023-10-06T04:03:28.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.696+0000 7fda1f59e700 1 --2- 172.21.15.157:0/958553702 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fda00066f30 0x7fda000693e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:28.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.696+0000 7fda1f59e700 1 --2- 172.21.15.157:0/958553702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda200b1e40 0x7fda20151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:28.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.696+0000 7fda1f59e700 1 --2- 172.21.15.157:0/958553702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda2006c2d0 0x7fda201533e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:28.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.696+0000 7fda1f59e700 1 --2- 172.21.15.157:0/958553702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda200b04d0 0x7fda200c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:28.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.696+0000 7fda1f59e700 1 -- 172.21.15.157:0/958553702 >> 172.21.15.157:0/958553702 conn(0x7fda200cc540 msgr2=0x7fda201573c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:28.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.696+0000 7fda1f59e700 1 -- 172.21.15.157:0/958553702 shutdown_connections 2023-10-06T04:03:28.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:28.696+0000 7fda1f59e700 1 -- 172.21.15.157:0/958553702 wait complete. 2023-10-06T04:03:28.699 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 51 2023-10-06T04:03:28.709 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2023-10-06T04:03:28.710 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 51, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-06T03:57:10.718782+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 51, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4279": { "addr": "172.21.15.175:6837/1002404870", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6836", "nonce": 1002404870, "type": "v2" }, { "addr": "172.21.15.175:6837", "nonce": 1002404870, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1 ], "features": 4540138320759226367, "flags": 0, "gid": 4279, "incarnation": 11, "join_fscid": -1, "name": "d", "rank": 2, "state": "up:stopping", "state_seq": 7 }, "gid_4287": { "addr": "172.21.15.157:6834/1179753185", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6832", "nonce": 1179753185, "type": "v2" }, { "addr": "172.21.15.157:6834", "nonce": 1179753185, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4287, "incarnation": 9, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4296": { "addr": "172.21.15.157:6837/4188652695", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6836", "nonce": 4188652695, "type": "v2" }, { "addr": "172.21.15.157:6837", "nonce": 4188652695, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4296, "incarnation": 4, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 27, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2023-10-06T04:03:13.713593+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 3, 4 ], "tableserver": 0, "up": { "mds_0": 4296, "mds_1": 4287, "mds_2": 4279 } } } ], "standbys": [ { "addr": "172.21.15.175:6835/3192541484", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6834", "nonce": 3192541484, "type": "v2" }, { "addr": "172.21.15.175:6835", "nonce": 3192541484, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 48, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5009, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.175:6839/745804271", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6838", "nonce": 745804271, "type": "v2" }, { "addr": "172.21.15.175:6839", "nonce": 745804271, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 42, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5164, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.157:6835/2461751318", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6833", "nonce": 2461751318, "type": "v2" }, { "addr": "172.21.15.157:6835", "nonce": 2461751318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 34, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5268, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-06T04:03:30.712 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:03:30.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.838+0000 7f11cdb3c700 1 Processor -- start 2023-10-06T04:03:30.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.838+0000 7f11cdb3c700 1 -- start start 2023-10-06T04:03:30.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.839+0000 7f11cdb3c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f11c806cbb0 0x7f11c806d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:30.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.839+0000 7f11cdb3c700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f11c806c1f0 0x7f11c806da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:30.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.839+0000 7f11cdb3c700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c80c9780 con 0x7f11c806cf80 2023-10-06T04:03:30.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.839+0000 7f11cdb3c700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c80c8c40 con 0x7f11c806cbb0 2023-10-06T04:03:30.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.839+0000 7f11cdb3c700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c806e0c0 con 0x7f11c806c1f0 2023-10-06T04:03:30.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.839+0000 7f11c77fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f11c806cbb0 0x7f11c806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:30.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.839+0000 7f11c7fff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f11c806cf80 0x7f11c814fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59712/0 (socket says 172.21.15.157:59712) 2023-10-06T04:03:30.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.839+0000 7f11c7fff700 1 -- 172.21.15.157:0/2138436238 learned_addr learned my addr 172.21.15.157:0/2138436238 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:03:30.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.839+0000 7f11c6ffd700 1 --2- 172.21.15.157:0/2138436238 >> v2:172.21.15.175:3300/0 conn(0x7f11c806c1f0 0x7f11c806da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:30.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.839+0000 7f11c77fe700 1 -- 172.21.15.157:0/2138436238 >> v2:172.21.15.175:3300/0 conn(0x7f11c806c1f0 msgr2=0x7f11c806da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:30.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.839+0000 7f11c77fe700 1 --2- 172.21.15.157:0/2138436238 >> v2:172.21.15.175:3300/0 conn(0x7f11c806c1f0 0x7f11c806da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:30.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.839+0000 7f11c77fe700 1 -- 172.21.15.157:0/2138436238 >> v1:172.21.15.157:6789/0 conn(0x7f11c806cf80 legacy=0x7f11c814fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:30.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.839+0000 7f11c77fe700 1 -- 172.21.15.157:0/2138436238 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c80c8e50 con 0x7f11c806cbb0 2023-10-06T04:03:30.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.839+0000 7f11c67fc700 1 -- 172.21.15.157:0/2138436238 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 830105822 0 0) 0x7f11c80c9780 con 0x7f11c806cf80 2023-10-06T04:03:30.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.839+0000 7f11c77fe700 1 --2- 172.21.15.157:0/2138436238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11c806cbb0 0x7f11c806d530 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f11c8157070 tx=0x7f11b8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca35c951f9a0537 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:30.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.839+0000 7f11c67fc700 1 -- 172.21.15.157:0/2138436238 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b800e070 con 0x7f11c806cbb0 2023-10-06T04:03:30.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.840+0000 7f11c67fc700 1 -- 172.21.15.157:0/2138436238 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b800bbf0 con 0x7f11c806cbb0 2023-10-06T04:03:30.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.840+0000 7f11c67fc700 1 -- 172.21.15.157:0/2138436238 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b8012430 con 0x7f11c806cbb0 2023-10-06T04:03:30.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.840+0000 7f11cdb3c700 1 -- 172.21.15.157:0/2138436238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11c806cbb0 msgr2=0x7f11c806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:30.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.840+0000 7f11cdb3c700 1 --2- 172.21.15.157:0/2138436238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11c806cbb0 0x7f11c806d530 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f11c8157070 tx=0x7f11b8009780 comp rx=0 tx=0).stop 2023-10-06T04:03:30.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.840+0000 7f11cdb3c700 1 -- 172.21.15.157:0/2138436238 shutdown_connections 2023-10-06T04:03:30.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.840+0000 7f11cdb3c700 1 --2- 172.21.15.157:0/2138436238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11c806cbb0 0x7f11c806d530 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:30.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.840+0000 7f11cdb3c700 1 --2- 172.21.15.157:0/2138436238 >> v2:172.21.15.175:3300/0 conn(0x7f11c806c1f0 0x7f11c806da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:30.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.840+0000 7f11cdb3c700 1 -- 172.21.15.157:0/2138436238 >> 172.21.15.157:0/2138436238 conn(0x7f11c806f230 msgr2=0x7f11c806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:30.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.840+0000 7f11cdb3c700 1 -- 172.21.15.157:0/2138436238 shutdown_connections 2023-10-06T04:03:30.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.840+0000 7f11cdb3c700 1 -- 172.21.15.157:0/2138436238 wait complete. 2023-10-06T04:03:30.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.841+0000 7f11cdb3c700 1 Processor -- start 2023-10-06T04:03:30.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.841+0000 7f11cdb3c700 1 -- start start 2023-10-06T04:03:30.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.842+0000 7f11cdb3c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11c806c1f0 0x7f11c81486a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:30.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.842+0000 7f11cdb3c700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11c806cbb0 0x7f11c8153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:30.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.842+0000 7f11cdb3c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11c806cf80 0x7f11c80c1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:30.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.842+0000 7f11cdb3c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c8166780 con 0x7f11c806c1f0 2023-10-06T04:03:30.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.842+0000 7f11cdb3c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c8166fc0 con 0x7f11c806cf80 2023-10-06T04:03:30.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.842+0000 7f11cdb3c700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c81569d0 con 0x7f11c806cbb0 2023-10-06T04:03:30.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.842+0000 7f11c77fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11c806c1f0 0x7f11c81486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:30.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.842+0000 7f11c77fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11c806c1f0 0x7f11c81486a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37508/0 (socket says 172.21.15.157:37508) 2023-10-06T04:03:30.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.842+0000 7f11c77fe700 1 -- 172.21.15.157:0/731666565 learned_addr learned my addr 172.21.15.157:0/731666565 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:30.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.842+0000 7f11c7fff700 1 --2- 172.21.15.157:0/731666565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11c806cf80 0x7f11c80c1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:30.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.842+0000 7f11c6ffd700 1 --2- 172.21.15.157:0/731666565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11c806cbb0 0x7f11c8153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:30.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.842+0000 7f11c77fe700 1 -- 172.21.15.157:0/731666565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11c806cbb0 msgr2=0x7f11c8153fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:30.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.842+0000 7f11c77fe700 1 --2- 172.21.15.157:0/731666565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11c806cbb0 0x7f11c8153fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:30.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.842+0000 7f11c77fe700 1 -- 172.21.15.157:0/731666565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11c806cf80 msgr2=0x7f11c80c1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:30.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.842+0000 7f11c77fe700 1 --2- 172.21.15.157:0/731666565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11c806cf80 0x7f11c80c1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:30.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.842+0000 7f11c77fe700 1 -- 172.21.15.157:0/731666565 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b8009a20 con 0x7f11c806c1f0 2023-10-06T04:03:30.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.843+0000 7f11c77fe700 1 --2- 172.21.15.157:0/731666565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11c806c1f0 0x7f11c81486a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f11c80c3960 tx=0x7f11b8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:30.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.843+0000 7f11af7fe700 1 -- 172.21.15.157:0/731666565 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b8001090 con 0x7f11c806c1f0 2023-10-06T04:03:30.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.843+0000 7f11af7fe700 1 -- 172.21.15.157:0/731666565 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b8012e60 con 0x7f11c806c1f0 2023-10-06T04:03:30.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.843+0000 7f11cdb3c700 1 -- 172.21.15.157:0/731666565 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c8148be0 con 0x7f11c806c1f0 2023-10-06T04:03:30.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.843+0000 7f11af7fe700 1 -- 172.21.15.157:0/731666565 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b80276c0 con 0x7f11c806c1f0 2023-10-06T04:03:30.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.843+0000 7f11cdb3c700 1 -- 172.21.15.157:0/731666565 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c8166970 con 0x7f11c806c1f0 2023-10-06T04:03:30.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.844+0000 7f11cdb3c700 1 -- 172.21.15.157:0/731666565 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c8089ca0 con 0x7f11c806c1f0 2023-10-06T04:03:30.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.844+0000 7f11af7fe700 1 -- 172.21.15.157:0/731666565 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f11b8024070 con 0x7f11c806c1f0 2023-10-06T04:03:30.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.844+0000 7f11af7fe700 1 --2- 172.21.15.157:0/731666565 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f11b4066f30 0x7f11b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:30.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.845+0000 7f11af7fe700 1 -- 172.21.15.157:0/731666565 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7f11b8086820 con 0x7f11c806c1f0 2023-10-06T04:03:30.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.845+0000 7f11c6ffd700 1 --2- 172.21.15.157:0/731666565 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f11b4066f30 0x7f11b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:30.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.845+0000 7f11c6ffd700 1 --2- 172.21.15.157:0/731666565 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f11b4066f30 0x7f11b40693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f11c8071d70 tx=0x7f11bc0091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:30.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:30.849+0000 7f11af7fe700 1 -- 172.21.15.157:0/731666565 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f11b80536e0 con 0x7f11c806c1f0 2023-10-06T04:03:31.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:31.078+0000 7f11cdb3c700 1 -- 172.21.15.157:0/731666565 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11c80c9130 con 0x7f11c806c1f0 2023-10-06T04:03:31.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:31.078+0000 7f11af7fe700 1 -- 172.21.15.157:0/731666565 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+6162 (secure 0 0 0) 0x7f11b8056d00 con 0x7f11c806c1f0 2023-10-06T04:03:31.080 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:03:31.080 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3192541484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3192541484},{"type":"v1","addr":"172.21.15.175:6835","nonce":3192541484}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:13.713593+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279},"failed":[],"damaged":[],"stopped":[3,4],"info":{"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:stopping","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:03:31.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:31.080+0000 7f11cdb3c700 1 -- 172.21.15.157:0/731666565 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f11b4066f30 msgr2=0x7f11b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:31.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:31.080+0000 7f11cdb3c700 1 --2- 172.21.15.157:0/731666565 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f11b4066f30 0x7f11b40693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f11c8071d70 tx=0x7f11bc0091a0 comp rx=0 tx=0).stop 2023-10-06T04:03:31.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:31.080+0000 7f11cdb3c700 1 -- 172.21.15.157:0/731666565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11c806c1f0 msgr2=0x7f11c81486a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:31.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:31.080+0000 7f11cdb3c700 1 --2- 172.21.15.157:0/731666565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11c806c1f0 0x7f11c81486a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f11c80c3960 tx=0x7f11b8019000 comp rx=0 tx=0).stop 2023-10-06T04:03:31.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:31.080+0000 7f11cdb3c700 1 -- 172.21.15.157:0/731666565 shutdown_connections 2023-10-06T04:03:31.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:31.080+0000 7f11cdb3c700 1 --2- 172.21.15.157:0/731666565 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f11b4066f30 0x7f11b40693e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:31.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:31.080+0000 7f11cdb3c700 1 --2- 172.21.15.157:0/731666565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11c806cf80 0x7f11c80c1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:31.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:31.080+0000 7f11cdb3c700 1 --2- 172.21.15.157:0/731666565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11c806c1f0 0x7f11c81486a0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:31.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:31.080+0000 7f11cdb3c700 1 --2- 172.21.15.157:0/731666565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11c806cbb0 0x7f11c8153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:31.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:31.080+0000 7f11cdb3c700 1 -- 172.21.15.157:0/731666565 >> 172.21.15.157:0/731666565 conn(0x7f11c806f230 msgr2=0x7f11c806d530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:31.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:31.080+0000 7f11cdb3c700 1 -- 172.21.15.157:0/731666565 shutdown_connections 2023-10-06T04:03:31.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:31.080+0000 7f11cdb3c700 1 -- 172.21.15.157:0/731666565 wait complete. 2023-10-06T04:03:31.083 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 51 2023-10-06T04:03:31.094 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:03:32.143 INFO:tasks.workunit.client.0.smithi157.stdout:removed './216/28' 2023-10-06T04:03:32.143 INFO:tasks.workunit.client.0.smithi157.stdout:removed './216/24' 2023-10-06T04:03:32.143 INFO:tasks.workunit.client.0.smithi157.stdout:removed './216/29' 2023-10-06T04:03:32.143 INFO:tasks.workunit.client.0.smithi157.stdout:removed './216/32' 2023-10-06T04:03:32.143 INFO:tasks.workunit.client.0.smithi157.stdout:removed './216/22' 2023-10-06T04:03:32.144 INFO:tasks.workunit.client.0.smithi157.stdout:removed './216/25' 2023-10-06T04:03:32.144 INFO:tasks.workunit.client.0.smithi157.stdout:removed './216/30' 2023-10-06T04:03:32.144 INFO:tasks.workunit.client.0.smithi157.stdout:removed './216/33' 2023-10-06T04:03:32.144 INFO:tasks.workunit.client.0.smithi157.stdout:removed './216/23' 2023-10-06T04:03:32.144 INFO:tasks.workunit.client.0.smithi157.stdout:removed './216/35' 2023-10-06T04:03:32.144 INFO:tasks.workunit.client.0.smithi157.stdout:removed './216/26' 2023-10-06T04:03:32.144 INFO:tasks.workunit.client.0.smithi157.stdout:removed './216/37' 2023-10-06T04:03:32.144 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './216' 2023-10-06T04:03:32.144 INFO:tasks.workunit.client.0.smithi157.stdout:removed './217/21' 2023-10-06T04:03:32.144 INFO:tasks.workunit.client.0.smithi157.stdout:removed './217/27' 2023-10-06T04:03:32.144 INFO:tasks.workunit.client.0.smithi157.stdout:removed './217/31' 2023-10-06T04:03:32.144 INFO:tasks.workunit.client.0.smithi157.stdout:removed './217/36' 2023-10-06T04:03:32.145 INFO:tasks.workunit.client.0.smithi157.stdout:removed './217/20' 2023-10-06T04:03:32.145 INFO:tasks.workunit.client.0.smithi157.stdout:removed './217/39' 2023-10-06T04:03:32.145 INFO:tasks.workunit.client.0.smithi157.stdout:removed './217/34' 2023-10-06T04:03:32.145 INFO:tasks.workunit.client.0.smithi157.stdout:removed './217/38' 2023-10-06T04:03:32.145 INFO:tasks.workunit.client.0.smithi157.stdout:removed './217/28' 2023-10-06T04:03:32.145 INFO:tasks.workunit.client.0.smithi157.stdout:removed './217/24' 2023-10-06T04:03:32.145 INFO:tasks.workunit.client.0.smithi157.stdout:removed './217/29' 2023-10-06T04:03:32.145 INFO:tasks.workunit.client.0.smithi157.stdout:removed './217/32' 2023-10-06T04:03:32.145 INFO:tasks.workunit.client.0.smithi157.stdout:removed './217/22' 2023-10-06T04:03:32.145 INFO:tasks.workunit.client.0.smithi157.stdout:removed './217/25' 2023-10-06T04:03:32.145 INFO:tasks.workunit.client.0.smithi157.stdout:removed './217/30' 2023-10-06T04:03:32.145 INFO:tasks.workunit.client.0.smithi157.stdout:removed './217/33' 2023-10-06T04:03:32.146 INFO:tasks.workunit.client.0.smithi157.stdout:removed './217/23' 2023-10-06T04:03:32.146 INFO:tasks.workunit.client.0.smithi157.stdout:removed './217/35' 2023-10-06T04:03:32.146 INFO:tasks.workunit.client.0.smithi157.stdout:removed './217/26' 2023-10-06T04:03:32.146 INFO:tasks.workunit.client.0.smithi157.stdout:removed './217/37' 2023-10-06T04:03:32.146 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './217' 2023-10-06T04:03:32.146 INFO:tasks.workunit.client.0.smithi157.stdout:removed './218/21' 2023-10-06T04:03:32.146 INFO:tasks.workunit.client.0.smithi157.stdout:removed './218/27' 2023-10-06T04:03:32.146 INFO:tasks.workunit.client.0.smithi157.stdout:removed './218/31' 2023-10-06T04:03:32.146 INFO:tasks.workunit.client.0.smithi157.stdout:removed './218/36' 2023-10-06T04:03:32.146 INFO:tasks.workunit.client.0.smithi157.stdout:removed './218/20' 2023-10-06T04:03:32.146 INFO:tasks.workunit.client.0.smithi157.stdout:removed './218/39' 2023-10-06T04:03:32.146 INFO:tasks.workunit.client.0.smithi157.stdout:removed './218/34' 2023-10-06T04:03:32.147 INFO:tasks.workunit.client.0.smithi157.stdout:removed './218/38' 2023-10-06T04:03:32.147 INFO:tasks.workunit.client.0.smithi157.stdout:removed './218/28' 2023-10-06T04:03:32.147 INFO:tasks.workunit.client.0.smithi157.stdout:removed './218/24' 2023-10-06T04:03:32.147 INFO:tasks.workunit.client.0.smithi157.stdout:removed './218/29' 2023-10-06T04:03:32.147 INFO:tasks.workunit.client.0.smithi157.stdout:removed './218/32' 2023-10-06T04:03:32.147 INFO:tasks.workunit.client.0.smithi157.stdout:removed './218/22' 2023-10-06T04:03:32.147 INFO:tasks.workunit.client.0.smithi157.stdout:removed './218/25' 2023-10-06T04:03:32.147 INFO:tasks.workunit.client.0.smithi157.stdout:removed './218/30' 2023-10-06T04:03:32.147 INFO:tasks.workunit.client.0.smithi157.stdout:removed './218/33' 2023-10-06T04:03:32.147 INFO:tasks.workunit.client.0.smithi157.stdout:removed './218/23' 2023-10-06T04:03:32.147 INFO:tasks.workunit.client.0.smithi157.stdout:removed './218/35' 2023-10-06T04:03:32.147 INFO:tasks.workunit.client.0.smithi157.stdout:removed './218/26' 2023-10-06T04:03:32.148 INFO:tasks.workunit.client.0.smithi157.stdout:removed './218/37' 2023-10-06T04:03:32.148 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './218' 2023-10-06T04:03:32.148 INFO:tasks.workunit.client.0.smithi157.stdout:removed './219/21' 2023-10-06T04:03:32.148 INFO:tasks.workunit.client.0.smithi157.stdout:removed './219/27' 2023-10-06T04:03:32.148 INFO:tasks.workunit.client.0.smithi157.stdout:removed './219/31' 2023-10-06T04:03:32.148 INFO:tasks.workunit.client.0.smithi157.stdout:removed './219/36' 2023-10-06T04:03:32.148 INFO:tasks.workunit.client.0.smithi157.stdout:removed './219/20' 2023-10-06T04:03:32.148 INFO:tasks.workunit.client.0.smithi157.stdout:removed './219/39' 2023-10-06T04:03:32.148 INFO:tasks.workunit.client.0.smithi157.stdout:removed './219/34' 2023-10-06T04:03:32.148 INFO:tasks.workunit.client.0.smithi157.stdout:removed './219/38' 2023-10-06T04:03:32.148 INFO:tasks.workunit.client.0.smithi157.stdout:removed './219/28' 2023-10-06T04:03:32.149 INFO:tasks.workunit.client.0.smithi157.stdout:removed './219/24' 2023-10-06T04:03:32.149 INFO:tasks.workunit.client.0.smithi157.stdout:removed './219/29' 2023-10-06T04:03:32.149 INFO:tasks.workunit.client.0.smithi157.stdout:removed './219/32' 2023-10-06T04:03:32.149 INFO:tasks.workunit.client.0.smithi157.stdout:removed './219/22' 2023-10-06T04:03:32.149 INFO:tasks.workunit.client.0.smithi157.stdout:removed './219/25' 2023-10-06T04:03:32.149 INFO:tasks.workunit.client.0.smithi157.stdout:removed './219/30' 2023-10-06T04:03:32.149 INFO:tasks.workunit.client.0.smithi157.stdout:removed './219/33' 2023-10-06T04:03:32.149 INFO:tasks.workunit.client.0.smithi157.stdout:removed './219/23' 2023-10-06T04:03:32.149 INFO:tasks.workunit.client.0.smithi157.stdout:removed './219/35' 2023-10-06T04:03:32.149 INFO:tasks.workunit.client.0.smithi157.stdout:removed './219/26' 2023-10-06T04:03:32.149 INFO:tasks.workunit.client.0.smithi157.stdout:removed './219/37' 2023-10-06T04:03:32.149 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './219' 2023-10-06T04:03:32.150 INFO:tasks.workunit.client.0.smithi157.stdout:removed './220/21' 2023-10-06T04:03:32.150 INFO:tasks.workunit.client.0.smithi157.stdout:removed './220/27' 2023-10-06T04:03:32.150 INFO:tasks.workunit.client.0.smithi157.stdout:removed './220/31' 2023-10-06T04:03:32.150 INFO:tasks.workunit.client.0.smithi157.stdout:removed './220/36' 2023-10-06T04:03:32.150 INFO:tasks.workunit.client.0.smithi157.stdout:removed './220/20' 2023-10-06T04:03:32.150 INFO:tasks.workunit.client.0.smithi157.stdout:removed './220/39' 2023-10-06T04:03:32.150 INFO:tasks.workunit.client.0.smithi157.stdout:removed './220/34' 2023-10-06T04:03:32.150 INFO:tasks.workunit.client.0.smithi157.stdout:removed './220/38' 2023-10-06T04:03:32.150 INFO:tasks.workunit.client.0.smithi157.stdout:removed './220/28' 2023-10-06T04:03:32.151 INFO:tasks.workunit.client.0.smithi157.stdout:removed './220/24' 2023-10-06T04:03:32.151 INFO:tasks.workunit.client.0.smithi157.stdout:removed './220/29' 2023-10-06T04:03:32.151 INFO:tasks.workunit.client.0.smithi157.stdout:removed './220/32' 2023-10-06T04:03:32.151 INFO:tasks.workunit.client.0.smithi157.stdout:removed './220/22' 2023-10-06T04:03:32.151 INFO:tasks.workunit.client.0.smithi157.stdout:removed './220/25' 2023-10-06T04:03:32.151 INFO:tasks.workunit.client.0.smithi157.stdout:removed './220/30' 2023-10-06T04:03:32.151 INFO:tasks.workunit.client.0.smithi157.stdout:removed './220/33' 2023-10-06T04:03:32.151 INFO:tasks.workunit.client.0.smithi157.stdout:removed './220/23' 2023-10-06T04:03:32.151 INFO:tasks.workunit.client.0.smithi157.stdout:removed './220/35' 2023-10-06T04:03:32.151 INFO:tasks.workunit.client.0.smithi157.stdout:removed './220/26' 2023-10-06T04:03:32.151 INFO:tasks.workunit.client.0.smithi157.stdout:removed './220/37' 2023-10-06T04:03:32.151 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './220' 2023-10-06T04:03:32.151 INFO:tasks.workunit.client.0.smithi157.stdout:removed './221/21' 2023-10-06T04:03:32.152 INFO:tasks.workunit.client.0.smithi157.stdout:removed './221/27' 2023-10-06T04:03:32.152 INFO:tasks.workunit.client.0.smithi157.stdout:removed './221/31' 2023-10-06T04:03:32.152 INFO:tasks.workunit.client.0.smithi157.stdout:removed './221/36' 2023-10-06T04:03:32.152 INFO:tasks.workunit.client.0.smithi157.stdout:removed './221/20' 2023-10-06T04:03:32.152 INFO:tasks.workunit.client.0.smithi157.stdout:removed './221/39' 2023-10-06T04:03:32.152 INFO:tasks.workunit.client.0.smithi157.stdout:removed './221/34' 2023-10-06T04:03:32.152 INFO:tasks.workunit.client.0.smithi157.stdout:removed './221/38' 2023-10-06T04:03:32.152 INFO:tasks.workunit.client.0.smithi157.stdout:removed './221/28' 2023-10-06T04:03:32.152 INFO:tasks.workunit.client.0.smithi157.stdout:removed './221/24' 2023-10-06T04:03:32.152 INFO:tasks.workunit.client.0.smithi157.stdout:removed './221/29' 2023-10-06T04:03:32.152 INFO:tasks.workunit.client.0.smithi157.stdout:removed './221/32' 2023-10-06T04:03:32.152 INFO:tasks.workunit.client.0.smithi157.stdout:removed './221/22' 2023-10-06T04:03:32.153 INFO:tasks.workunit.client.0.smithi157.stdout:removed './221/25' 2023-10-06T04:03:32.153 INFO:tasks.workunit.client.0.smithi157.stdout:removed './221/30' 2023-10-06T04:03:32.153 INFO:tasks.workunit.client.0.smithi157.stdout:removed './221/33' 2023-10-06T04:03:32.153 INFO:tasks.workunit.client.0.smithi157.stdout:removed './221/23' 2023-10-06T04:03:32.153 INFO:tasks.workunit.client.0.smithi157.stdout:removed './221/35' 2023-10-06T04:03:32.153 INFO:tasks.workunit.client.0.smithi157.stdout:removed './221/26' 2023-10-06T04:03:32.153 INFO:tasks.workunit.client.0.smithi157.stdout:removed './221/37' 2023-10-06T04:03:32.153 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './221' 2023-10-06T04:03:32.153 INFO:tasks.workunit.client.0.smithi157.stdout:removed './222/21' 2023-10-06T04:03:32.153 INFO:tasks.workunit.client.0.smithi157.stdout:removed './222/27' 2023-10-06T04:03:32.153 INFO:tasks.workunit.client.0.smithi157.stdout:removed './222/31' 2023-10-06T04:03:32.153 INFO:tasks.workunit.client.0.smithi157.stdout:removed './222/36' 2023-10-06T04:03:32.153 INFO:tasks.workunit.client.0.smithi157.stdout:removed './222/20' 2023-10-06T04:03:32.154 INFO:tasks.workunit.client.0.smithi157.stdout:removed './222/39' 2023-10-06T04:03:32.154 INFO:tasks.workunit.client.0.smithi157.stdout:removed './222/34' 2023-10-06T04:03:32.154 INFO:tasks.workunit.client.0.smithi157.stdout:removed './222/38' 2023-10-06T04:03:32.154 INFO:tasks.workunit.client.0.smithi157.stdout:removed './222/28' 2023-10-06T04:03:32.154 INFO:tasks.workunit.client.0.smithi157.stdout:removed './222/24' 2023-10-06T04:03:32.154 INFO:tasks.workunit.client.0.smithi157.stdout:removed './222/29' 2023-10-06T04:03:32.154 INFO:tasks.workunit.client.0.smithi157.stdout:removed './222/32' 2023-10-06T04:03:32.154 INFO:tasks.workunit.client.0.smithi157.stdout:removed './222/22' 2023-10-06T04:03:32.154 INFO:tasks.workunit.client.0.smithi157.stdout:removed './222/25' 2023-10-06T04:03:32.154 INFO:tasks.workunit.client.0.smithi157.stdout:removed './222/30' 2023-10-06T04:03:32.154 INFO:tasks.workunit.client.0.smithi157.stdout:removed './222/33' 2023-10-06T04:03:32.154 INFO:tasks.workunit.client.0.smithi157.stdout:removed './222/23' 2023-10-06T04:03:32.155 INFO:tasks.workunit.client.0.smithi157.stdout:removed './222/35' 2023-10-06T04:03:32.155 INFO:tasks.workunit.client.0.smithi157.stdout:removed './222/26' 2023-10-06T04:03:32.155 INFO:tasks.workunit.client.0.smithi157.stdout:removed './222/37' 2023-10-06T04:03:32.155 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './222' 2023-10-06T04:03:32.155 INFO:tasks.workunit.client.0.smithi157.stdout:removed './223/21' 2023-10-06T04:03:32.155 INFO:tasks.workunit.client.0.smithi157.stdout:removed './223/27' 2023-10-06T04:03:32.155 INFO:tasks.workunit.client.0.smithi157.stdout:removed './223/31' 2023-10-06T04:03:32.155 INFO:tasks.workunit.client.0.smithi157.stdout:removed './223/36' 2023-10-06T04:03:32.155 INFO:tasks.workunit.client.0.smithi157.stdout:removed './223/20' 2023-10-06T04:03:32.155 INFO:tasks.workunit.client.0.smithi157.stdout:removed './223/39' 2023-10-06T04:03:32.155 INFO:tasks.workunit.client.0.smithi157.stdout:removed './223/34' 2023-10-06T04:03:32.155 INFO:tasks.workunit.client.0.smithi157.stdout:removed './223/38' 2023-10-06T04:03:32.155 INFO:tasks.workunit.client.0.smithi157.stdout:removed './223/28' 2023-10-06T04:03:32.156 INFO:tasks.workunit.client.0.smithi157.stdout:removed './223/24' 2023-10-06T04:03:32.156 INFO:tasks.workunit.client.0.smithi157.stdout:removed './223/29' 2023-10-06T04:03:32.156 INFO:tasks.workunit.client.0.smithi157.stdout:removed './223/32' 2023-10-06T04:03:32.156 INFO:tasks.workunit.client.0.smithi157.stdout:removed './223/22' 2023-10-06T04:03:32.156 INFO:tasks.workunit.client.0.smithi157.stdout:removed './223/25' 2023-10-06T04:03:32.156 INFO:tasks.workunit.client.0.smithi157.stdout:removed './223/30' 2023-10-06T04:03:32.156 INFO:tasks.workunit.client.0.smithi157.stdout:removed './223/33' 2023-10-06T04:03:32.156 INFO:tasks.workunit.client.0.smithi157.stdout:removed './223/23' 2023-10-06T04:03:32.156 INFO:tasks.workunit.client.0.smithi157.stdout:removed './223/35' 2023-10-06T04:03:32.156 INFO:tasks.workunit.client.0.smithi157.stdout:removed './223/26' 2023-10-06T04:03:32.156 INFO:tasks.workunit.client.0.smithi157.stdout:removed './223/37' 2023-10-06T04:03:32.156 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './223' 2023-10-06T04:03:32.156 INFO:tasks.workunit.client.0.smithi157.stdout:removed './224/21' 2023-10-06T04:03:32.157 INFO:tasks.workunit.client.0.smithi157.stdout:removed './224/27' 2023-10-06T04:03:32.157 INFO:tasks.workunit.client.0.smithi157.stdout:removed './224/31' 2023-10-06T04:03:32.157 INFO:tasks.workunit.client.0.smithi157.stdout:removed './224/36' 2023-10-06T04:03:32.157 INFO:tasks.workunit.client.0.smithi157.stdout:removed './224/20' 2023-10-06T04:03:32.157 INFO:tasks.workunit.client.0.smithi157.stdout:removed './224/39' 2023-10-06T04:03:32.157 INFO:tasks.workunit.client.0.smithi157.stdout:removed './224/34' 2023-10-06T04:03:32.157 INFO:tasks.workunit.client.0.smithi157.stdout:removed './224/38' 2023-10-06T04:03:32.157 INFO:tasks.workunit.client.0.smithi157.stdout:removed './224/28' 2023-10-06T04:03:32.157 INFO:tasks.workunit.client.0.smithi157.stdout:removed './224/24' 2023-10-06T04:03:32.157 INFO:tasks.workunit.client.0.smithi157.stdout:removed './224/29' 2023-10-06T04:03:32.157 INFO:tasks.workunit.client.0.smithi157.stdout:removed './224/32' 2023-10-06T04:03:32.157 INFO:tasks.workunit.client.0.smithi157.stdout:removed './224/22' 2023-10-06T04:03:32.158 INFO:tasks.workunit.client.0.smithi157.stdout:removed './224/25' 2023-10-06T04:03:32.158 INFO:tasks.workunit.client.0.smithi157.stdout:removed './224/30' 2023-10-06T04:03:32.158 INFO:tasks.workunit.client.0.smithi157.stdout:removed './224/33' 2023-10-06T04:03:32.158 INFO:tasks.workunit.client.0.smithi157.stdout:removed './224/23' 2023-10-06T04:03:32.158 INFO:tasks.workunit.client.0.smithi157.stdout:removed './224/35' 2023-10-06T04:03:32.158 INFO:tasks.workunit.client.0.smithi157.stdout:removed './224/26' 2023-10-06T04:03:32.158 INFO:tasks.workunit.client.0.smithi157.stdout:removed './224/37' 2023-10-06T04:03:32.158 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './224' 2023-10-06T04:03:32.158 INFO:tasks.workunit.client.0.smithi157.stdout:removed './225/21' 2023-10-06T04:03:32.158 INFO:tasks.workunit.client.0.smithi157.stdout:removed './225/27' 2023-10-06T04:03:32.158 INFO:tasks.workunit.client.0.smithi157.stdout:removed './225/31' 2023-10-06T04:03:32.158 INFO:tasks.workunit.client.0.smithi157.stdout:removed './225/36' 2023-10-06T04:03:32.159 INFO:tasks.workunit.client.0.smithi157.stdout:removed './225/20' 2023-10-06T04:03:32.159 INFO:tasks.workunit.client.0.smithi157.stdout:removed './225/39' 2023-10-06T04:03:32.159 INFO:tasks.workunit.client.0.smithi157.stdout:removed './225/34' 2023-10-06T04:03:32.159 INFO:tasks.workunit.client.0.smithi157.stdout:removed './225/38' 2023-10-06T04:03:32.159 INFO:tasks.workunit.client.0.smithi157.stdout:removed './225/28' 2023-10-06T04:03:32.159 INFO:tasks.workunit.client.0.smithi157.stdout:removed './225/24' 2023-10-06T04:03:32.159 INFO:tasks.workunit.client.0.smithi157.stdout:removed './225/29' 2023-10-06T04:03:32.159 INFO:tasks.workunit.client.0.smithi157.stdout:removed './225/32' 2023-10-06T04:03:32.159 INFO:tasks.workunit.client.0.smithi157.stdout:removed './225/22' 2023-10-06T04:03:32.159 INFO:tasks.workunit.client.0.smithi157.stdout:removed './225/25' 2023-10-06T04:03:32.159 INFO:tasks.workunit.client.0.smithi157.stdout:removed './225/30' 2023-10-06T04:03:32.159 INFO:tasks.workunit.client.0.smithi157.stdout:removed './225/33' 2023-10-06T04:03:32.159 INFO:tasks.workunit.client.0.smithi157.stdout:removed './225/23' 2023-10-06T04:03:32.160 INFO:tasks.workunit.client.0.smithi157.stdout:removed './225/35' 2023-10-06T04:03:32.160 INFO:tasks.workunit.client.0.smithi157.stdout:removed './225/26' 2023-10-06T04:03:32.160 INFO:tasks.workunit.client.0.smithi157.stdout:removed './225/37' 2023-10-06T04:03:32.160 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './225' 2023-10-06T04:03:32.160 INFO:tasks.workunit.client.0.smithi157.stdout:removed './226/21' 2023-10-06T04:03:32.160 INFO:tasks.workunit.client.0.smithi157.stdout:removed './226/27' 2023-10-06T04:03:32.160 INFO:tasks.workunit.client.0.smithi157.stdout:removed './226/31' 2023-10-06T04:03:32.160 INFO:tasks.workunit.client.0.smithi157.stdout:removed './226/36' 2023-10-06T04:03:32.160 INFO:tasks.workunit.client.0.smithi157.stdout:removed './226/20' 2023-10-06T04:03:32.160 INFO:tasks.workunit.client.0.smithi157.stdout:removed './226/39' 2023-10-06T04:03:32.160 INFO:tasks.workunit.client.0.smithi157.stdout:removed './226/34' 2023-10-06T04:03:32.160 INFO:tasks.workunit.client.0.smithi157.stdout:removed './226/38' 2023-10-06T04:03:32.161 INFO:tasks.workunit.client.0.smithi157.stdout:removed './226/28' 2023-10-06T04:03:32.161 INFO:tasks.workunit.client.0.smithi157.stdout:removed './226/24' 2023-10-06T04:03:32.840 INFO:tasks.workunit.client.0.smithi157.stdout:removed './226/29' 2023-10-06T04:03:32.840 INFO:tasks.workunit.client.0.smithi157.stdout:removed './226/32' 2023-10-06T04:03:32.840 INFO:tasks.workunit.client.0.smithi157.stdout:removed './226/22' 2023-10-06T04:03:32.840 INFO:tasks.workunit.client.0.smithi157.stdout:removed './226/25' 2023-10-06T04:03:32.840 INFO:tasks.workunit.client.0.smithi157.stdout:removed './226/30' 2023-10-06T04:03:32.841 INFO:tasks.workunit.client.0.smithi157.stdout:removed './226/33' 2023-10-06T04:03:32.841 INFO:tasks.workunit.client.0.smithi157.stdout:removed './226/23' 2023-10-06T04:03:32.841 INFO:tasks.workunit.client.0.smithi157.stdout:removed './226/35' 2023-10-06T04:03:32.841 INFO:tasks.workunit.client.0.smithi157.stdout:removed './226/26' 2023-10-06T04:03:32.841 INFO:tasks.workunit.client.0.smithi157.stdout:removed './226/37' 2023-10-06T04:03:32.841 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './226' 2023-10-06T04:03:32.841 INFO:tasks.workunit.client.0.smithi157.stdout:removed './227/21' 2023-10-06T04:03:32.841 INFO:tasks.workunit.client.0.smithi157.stdout:removed './227/27' 2023-10-06T04:03:32.841 INFO:tasks.workunit.client.0.smithi157.stdout:removed './227/31' 2023-10-06T04:03:32.841 INFO:tasks.workunit.client.0.smithi157.stdout:removed './227/36' 2023-10-06T04:03:32.841 INFO:tasks.workunit.client.0.smithi157.stdout:removed './227/20' 2023-10-06T04:03:32.841 INFO:tasks.workunit.client.0.smithi157.stdout:removed './227/39' 2023-10-06T04:03:32.842 INFO:tasks.workunit.client.0.smithi157.stdout:removed './227/34' 2023-10-06T04:03:32.842 INFO:tasks.workunit.client.0.smithi157.stdout:removed './227/38' 2023-10-06T04:03:32.842 INFO:tasks.workunit.client.0.smithi157.stdout:removed './227/28' 2023-10-06T04:03:32.842 INFO:tasks.workunit.client.0.smithi157.stdout:removed './227/24' 2023-10-06T04:03:32.842 INFO:tasks.workunit.client.0.smithi157.stdout:removed './227/29' 2023-10-06T04:03:32.842 INFO:tasks.workunit.client.0.smithi157.stdout:removed './227/32' 2023-10-06T04:03:32.842 INFO:tasks.workunit.client.0.smithi157.stdout:removed './227/22' 2023-10-06T04:03:32.842 INFO:tasks.workunit.client.0.smithi157.stdout:removed './227/25' 2023-10-06T04:03:32.842 INFO:tasks.workunit.client.0.smithi157.stdout:removed './227/30' 2023-10-06T04:03:32.842 INFO:tasks.workunit.client.0.smithi157.stdout:removed './227/33' 2023-10-06T04:03:32.842 INFO:tasks.workunit.client.0.smithi157.stdout:removed './227/23' 2023-10-06T04:03:32.843 INFO:tasks.workunit.client.0.smithi157.stdout:removed './227/35' 2023-10-06T04:03:32.843 INFO:tasks.workunit.client.0.smithi157.stdout:removed './227/26' 2023-10-06T04:03:32.843 INFO:tasks.workunit.client.0.smithi157.stdout:removed './227/37' 2023-10-06T04:03:32.843 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './227' 2023-10-06T04:03:32.843 INFO:tasks.workunit.client.0.smithi157.stdout:removed './228/21' 2023-10-06T04:03:32.843 INFO:tasks.workunit.client.0.smithi157.stdout:removed './228/27' 2023-10-06T04:03:32.843 INFO:tasks.workunit.client.0.smithi157.stdout:removed './228/31' 2023-10-06T04:03:32.843 INFO:tasks.workunit.client.0.smithi157.stdout:removed './228/36' 2023-10-06T04:03:32.843 INFO:tasks.workunit.client.0.smithi157.stdout:removed './228/20' 2023-10-06T04:03:32.843 INFO:tasks.workunit.client.0.smithi157.stdout:removed './228/39' 2023-10-06T04:03:32.843 INFO:tasks.workunit.client.0.smithi157.stdout:removed './228/34' 2023-10-06T04:03:32.843 INFO:tasks.workunit.client.0.smithi157.stdout:removed './228/38' 2023-10-06T04:03:32.844 INFO:tasks.workunit.client.0.smithi157.stdout:removed './228/28' 2023-10-06T04:03:32.844 INFO:tasks.workunit.client.0.smithi157.stdout:removed './228/24' 2023-10-06T04:03:32.844 INFO:tasks.workunit.client.0.smithi157.stdout:removed './228/29' 2023-10-06T04:03:32.844 INFO:tasks.workunit.client.0.smithi157.stdout:removed './228/32' 2023-10-06T04:03:32.844 INFO:tasks.workunit.client.0.smithi157.stdout:removed './228/22' 2023-10-06T04:03:32.844 INFO:tasks.workunit.client.0.smithi157.stdout:removed './228/25' 2023-10-06T04:03:32.844 INFO:tasks.workunit.client.0.smithi157.stdout:removed './228/30' 2023-10-06T04:03:32.844 INFO:tasks.workunit.client.0.smithi157.stdout:removed './228/33' 2023-10-06T04:03:32.844 INFO:tasks.workunit.client.0.smithi157.stdout:removed './228/23' 2023-10-06T04:03:32.844 INFO:tasks.workunit.client.0.smithi157.stdout:removed './228/35' 2023-10-06T04:03:32.844 INFO:tasks.workunit.client.0.smithi157.stdout:removed './228/26' 2023-10-06T04:03:32.844 INFO:tasks.workunit.client.0.smithi157.stdout:removed './228/37' 2023-10-06T04:03:32.845 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './228' 2023-10-06T04:03:32.845 INFO:tasks.workunit.client.0.smithi157.stdout:removed './229/21' 2023-10-06T04:03:32.845 INFO:tasks.workunit.client.0.smithi157.stdout:removed './229/27' 2023-10-06T04:03:32.845 INFO:tasks.workunit.client.0.smithi157.stdout:removed './229/31' 2023-10-06T04:03:32.845 INFO:tasks.workunit.client.0.smithi157.stdout:removed './229/36' 2023-10-06T04:03:32.845 INFO:tasks.workunit.client.0.smithi157.stdout:removed './229/20' 2023-10-06T04:03:32.845 INFO:tasks.workunit.client.0.smithi157.stdout:removed './229/39' 2023-10-06T04:03:32.845 INFO:tasks.workunit.client.0.smithi157.stdout:removed './229/34' 2023-10-06T04:03:32.845 INFO:tasks.workunit.client.0.smithi157.stdout:removed './229/38' 2023-10-06T04:03:32.845 INFO:tasks.workunit.client.0.smithi157.stdout:removed './229/28' 2023-10-06T04:03:32.845 INFO:tasks.workunit.client.0.smithi157.stdout:removed './229/24' 2023-10-06T04:03:32.845 INFO:tasks.workunit.client.0.smithi157.stdout:removed './229/29' 2023-10-06T04:03:32.845 INFO:tasks.workunit.client.0.smithi157.stdout:removed './229/32' 2023-10-06T04:03:32.846 INFO:tasks.workunit.client.0.smithi157.stdout:removed './229/22' 2023-10-06T04:03:32.846 INFO:tasks.workunit.client.0.smithi157.stdout:removed './229/25' 2023-10-06T04:03:32.846 INFO:tasks.workunit.client.0.smithi157.stdout:removed './229/30' 2023-10-06T04:03:32.846 INFO:tasks.workunit.client.0.smithi157.stdout:removed './229/33' 2023-10-06T04:03:32.846 INFO:tasks.workunit.client.0.smithi157.stdout:removed './229/23' 2023-10-06T04:03:32.846 INFO:tasks.workunit.client.0.smithi157.stdout:removed './229/35' 2023-10-06T04:03:32.846 INFO:tasks.workunit.client.0.smithi157.stdout:removed './229/26' 2023-10-06T04:03:32.846 INFO:tasks.workunit.client.0.smithi157.stdout:removed './229/37' 2023-10-06T04:03:32.846 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './229' 2023-10-06T04:03:32.846 INFO:tasks.workunit.client.0.smithi157.stdout:removed './230/21' 2023-10-06T04:03:32.846 INFO:tasks.workunit.client.0.smithi157.stdout:removed './230/27' 2023-10-06T04:03:32.847 INFO:tasks.workunit.client.0.smithi157.stdout:removed './230/31' 2023-10-06T04:03:32.847 INFO:tasks.workunit.client.0.smithi157.stdout:removed './230/36' 2023-10-06T04:03:32.847 INFO:tasks.workunit.client.0.smithi157.stdout:removed './230/20' 2023-10-06T04:03:32.847 INFO:tasks.workunit.client.0.smithi157.stdout:removed './230/39' 2023-10-06T04:03:32.847 INFO:tasks.workunit.client.0.smithi157.stdout:removed './230/34' 2023-10-06T04:03:32.847 INFO:tasks.workunit.client.0.smithi157.stdout:removed './230/38' 2023-10-06T04:03:32.847 INFO:tasks.workunit.client.0.smithi157.stdout:removed './230/28' 2023-10-06T04:03:32.847 INFO:tasks.workunit.client.0.smithi157.stdout:removed './230/24' 2023-10-06T04:03:32.847 INFO:tasks.workunit.client.0.smithi157.stdout:removed './230/29' 2023-10-06T04:03:32.847 INFO:tasks.workunit.client.0.smithi157.stdout:removed './230/32' 2023-10-06T04:03:32.847 INFO:tasks.workunit.client.0.smithi157.stdout:removed './230/22' 2023-10-06T04:03:32.847 INFO:tasks.workunit.client.0.smithi157.stdout:removed './230/25' 2023-10-06T04:03:32.848 INFO:tasks.workunit.client.0.smithi157.stdout:removed './230/30' 2023-10-06T04:03:32.848 INFO:tasks.workunit.client.0.smithi157.stdout:removed './230/33' 2023-10-06T04:03:32.848 INFO:tasks.workunit.client.0.smithi157.stdout:removed './230/23' 2023-10-06T04:03:32.848 INFO:tasks.workunit.client.0.smithi157.stdout:removed './230/35' 2023-10-06T04:03:32.848 INFO:tasks.workunit.client.0.smithi157.stdout:removed './230/26' 2023-10-06T04:03:32.848 INFO:tasks.workunit.client.0.smithi157.stdout:removed './230/37' 2023-10-06T04:03:32.848 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './230' 2023-10-06T04:03:32.848 INFO:tasks.workunit.client.0.smithi157.stdout:removed './231/21' 2023-10-06T04:03:32.848 INFO:tasks.workunit.client.0.smithi157.stdout:removed './231/27' 2023-10-06T04:03:32.848 INFO:tasks.workunit.client.0.smithi157.stdout:removed './231/31' 2023-10-06T04:03:32.848 INFO:tasks.workunit.client.0.smithi157.stdout:removed './231/36' 2023-10-06T04:03:32.848 INFO:tasks.workunit.client.0.smithi157.stdout:removed './231/20' 2023-10-06T04:03:32.848 INFO:tasks.workunit.client.0.smithi157.stdout:removed './231/39' 2023-10-06T04:03:32.849 INFO:tasks.workunit.client.0.smithi157.stdout:removed './231/34' 2023-10-06T04:03:32.849 INFO:tasks.workunit.client.0.smithi157.stdout:removed './231/38' 2023-10-06T04:03:32.849 INFO:tasks.workunit.client.0.smithi157.stdout:removed './231/28' 2023-10-06T04:03:32.849 INFO:tasks.workunit.client.0.smithi157.stdout:removed './231/24' 2023-10-06T04:03:32.849 INFO:tasks.workunit.client.0.smithi157.stdout:removed './231/29' 2023-10-06T04:03:32.849 INFO:tasks.workunit.client.0.smithi157.stdout:removed './231/32' 2023-10-06T04:03:32.849 INFO:tasks.workunit.client.0.smithi157.stdout:removed './231/22' 2023-10-06T04:03:32.849 INFO:tasks.workunit.client.0.smithi157.stdout:removed './231/25' 2023-10-06T04:03:32.849 INFO:tasks.workunit.client.0.smithi157.stdout:removed './231/30' 2023-10-06T04:03:32.849 INFO:tasks.workunit.client.0.smithi157.stdout:removed './231/33' 2023-10-06T04:03:32.849 INFO:tasks.workunit.client.0.smithi157.stdout:removed './231/23' 2023-10-06T04:03:32.849 INFO:tasks.workunit.client.0.smithi157.stdout:removed './231/35' 2023-10-06T04:03:32.850 INFO:tasks.workunit.client.0.smithi157.stdout:removed './231/26' 2023-10-06T04:03:32.850 INFO:tasks.workunit.client.0.smithi157.stdout:removed './231/37' 2023-10-06T04:03:32.850 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './231' 2023-10-06T04:03:32.850 INFO:tasks.workunit.client.0.smithi157.stdout:removed './232/21' 2023-10-06T04:03:32.850 INFO:tasks.workunit.client.0.smithi157.stdout:removed './232/27' 2023-10-06T04:03:32.850 INFO:tasks.workunit.client.0.smithi157.stdout:removed './232/31' 2023-10-06T04:03:32.850 INFO:tasks.workunit.client.0.smithi157.stdout:removed './232/36' 2023-10-06T04:03:32.850 INFO:tasks.workunit.client.0.smithi157.stdout:removed './232/20' 2023-10-06T04:03:32.850 INFO:tasks.workunit.client.0.smithi157.stdout:removed './232/39' 2023-10-06T04:03:32.850 INFO:tasks.workunit.client.0.smithi157.stdout:removed './232/34' 2023-10-06T04:03:32.850 INFO:tasks.workunit.client.0.smithi157.stdout:removed './232/38' 2023-10-06T04:03:32.850 INFO:tasks.workunit.client.0.smithi157.stdout:removed './232/28' 2023-10-06T04:03:32.851 INFO:tasks.workunit.client.0.smithi157.stdout:removed './232/24' 2023-10-06T04:03:32.851 INFO:tasks.workunit.client.0.smithi157.stdout:removed './232/29' 2023-10-06T04:03:32.851 INFO:tasks.workunit.client.0.smithi157.stdout:removed './232/32' 2023-10-06T04:03:32.851 INFO:tasks.workunit.client.0.smithi157.stdout:removed './232/22' 2023-10-06T04:03:32.851 INFO:tasks.workunit.client.0.smithi157.stdout:removed './232/25' 2023-10-06T04:03:32.851 INFO:tasks.workunit.client.0.smithi157.stdout:removed './232/30' 2023-10-06T04:03:32.851 INFO:tasks.workunit.client.0.smithi157.stdout:removed './232/33' 2023-10-06T04:03:32.851 INFO:tasks.workunit.client.0.smithi157.stdout:removed './232/23' 2023-10-06T04:03:32.851 INFO:tasks.workunit.client.0.smithi157.stdout:removed './232/35' 2023-10-06T04:03:32.851 INFO:tasks.workunit.client.0.smithi157.stdout:removed './232/26' 2023-10-06T04:03:32.851 INFO:tasks.workunit.client.0.smithi157.stdout:removed './232/37' 2023-10-06T04:03:32.851 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './232' 2023-10-06T04:03:32.852 INFO:tasks.workunit.client.0.smithi157.stdout:removed './233/21' 2023-10-06T04:03:32.852 INFO:tasks.workunit.client.0.smithi157.stdout:removed './233/27' 2023-10-06T04:03:32.852 INFO:tasks.workunit.client.0.smithi157.stdout:removed './233/31' 2023-10-06T04:03:32.852 INFO:tasks.workunit.client.0.smithi157.stdout:removed './233/36' 2023-10-06T04:03:32.852 INFO:tasks.workunit.client.0.smithi157.stdout:removed './233/20' 2023-10-06T04:03:32.852 INFO:tasks.workunit.client.0.smithi157.stdout:removed './233/39' 2023-10-06T04:03:32.852 INFO:tasks.workunit.client.0.smithi157.stdout:removed './233/34' 2023-10-06T04:03:32.852 INFO:tasks.workunit.client.0.smithi157.stdout:removed './233/38' 2023-10-06T04:03:32.852 INFO:tasks.workunit.client.0.smithi157.stdout:removed './233/28' 2023-10-06T04:03:32.852 INFO:tasks.workunit.client.0.smithi157.stdout:removed './233/24' 2023-10-06T04:03:32.852 INFO:tasks.workunit.client.0.smithi157.stdout:removed './233/29' 2023-10-06T04:03:32.852 INFO:tasks.workunit.client.0.smithi157.stdout:removed './233/32' 2023-10-06T04:03:32.852 INFO:tasks.workunit.client.0.smithi157.stdout:removed './233/22' 2023-10-06T04:03:32.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './233/25' 2023-10-06T04:03:32.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './233/30' 2023-10-06T04:03:32.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './233/33' 2023-10-06T04:03:32.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './233/23' 2023-10-06T04:03:32.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './233/35' 2023-10-06T04:03:32.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './233/26' 2023-10-06T04:03:32.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './233/37' 2023-10-06T04:03:32.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './233' 2023-10-06T04:03:32.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './234/21' 2023-10-06T04:03:32.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './234/27' 2023-10-06T04:03:32.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './234/31' 2023-10-06T04:03:32.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './234/36' 2023-10-06T04:03:32.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './234/20' 2023-10-06T04:03:32.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './234/39' 2023-10-06T04:03:32.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './234/34' 2023-10-06T04:03:32.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './234/38' 2023-10-06T04:03:32.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './234/28' 2023-10-06T04:03:32.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './234/24' 2023-10-06T04:03:32.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './234/29' 2023-10-06T04:03:32.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './234/32' 2023-10-06T04:03:32.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './234/22' 2023-10-06T04:03:32.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './234/25' 2023-10-06T04:03:32.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './234/30' 2023-10-06T04:03:32.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './234/33' 2023-10-06T04:03:32.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './234/23' 2023-10-06T04:03:32.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './234/35' 2023-10-06T04:03:32.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './234/26' 2023-10-06T04:03:32.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './234/37' 2023-10-06T04:03:32.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './234' 2023-10-06T04:03:32.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './235/21' 2023-10-06T04:03:32.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './235/27' 2023-10-06T04:03:32.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './235/31' 2023-10-06T04:03:32.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './235/36' 2023-10-06T04:03:32.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './235/20' 2023-10-06T04:03:32.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './235/39' 2023-10-06T04:03:32.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './235/34' 2023-10-06T04:03:32.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './235/38' 2023-10-06T04:03:32.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './235/28' 2023-10-06T04:03:32.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './235/24' 2023-10-06T04:03:32.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './235/29' 2023-10-06T04:03:32.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './235/32' 2023-10-06T04:03:32.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './235/22' 2023-10-06T04:03:32.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './235/25' 2023-10-06T04:03:32.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './235/30' 2023-10-06T04:03:32.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './235/33' 2023-10-06T04:03:32.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './235/23' 2023-10-06T04:03:32.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './235/35' 2023-10-06T04:03:32.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './235/26' 2023-10-06T04:03:32.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './235/37' 2023-10-06T04:03:32.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './235' 2023-10-06T04:03:32.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './236/21' 2023-10-06T04:03:32.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './236/27' 2023-10-06T04:03:32.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './236/31' 2023-10-06T04:03:32.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './236/36' 2023-10-06T04:03:32.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './236/20' 2023-10-06T04:03:32.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './236/39' 2023-10-06T04:03:32.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './236/34' 2023-10-06T04:03:32.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './236/38' 2023-10-06T04:03:32.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './236/28' 2023-10-06T04:03:32.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './236/24' 2023-10-06T04:03:32.858 INFO:tasks.workunit.client.0.smithi157.stdout:removed './236/29' 2023-10-06T04:03:32.858 INFO:tasks.workunit.client.0.smithi157.stdout:removed './236/32' 2023-10-06T04:03:33.094 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:03:33.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.209+0000 7f99ca702700 1 Processor -- start 2023-10-06T04:03:33.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.210+0000 7f99ca702700 1 -- start start 2023-10-06T04:03:33.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.210+0000 7f99ca702700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99c4069ea0 0x7f99c40631f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:33.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.210+0000 7f99ca702700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f99c406a860 0x7f99c4156870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:33.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.210+0000 7f99ca702700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c40c48f0 con 0x7f99c406a270 2023-10-06T04:03:33.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.210+0000 7f99ca702700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c4156db0 con 0x7f99c406a860 2023-10-06T04:03:33.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.210+0000 7f99ca702700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c40c3d20 con 0x7f99c4069ea0 2023-10-06T04:03:33.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.210+0000 7f99c8c9f700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f99c406a270 0x7f99c40c4d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:48366/0 (socket says 172.21.15.157:48366) 2023-10-06T04:03:33.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.211+0000 7f99c8c9f700 1 -- 172.21.15.157:0/4012198530 learned_addr learned my addr 172.21.15.157:0/4012198530 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:03:33.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.211+0000 7f99c3fff700 1 --2- 172.21.15.157:0/4012198530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99c4069ea0 0x7f99c40631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:33.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.211+0000 7f99c37fe700 1 --2- 172.21.15.157:0/4012198530 >> v2:172.21.15.175:3300/0 conn(0x7f99c406a860 0x7f99c4156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:33.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.211+0000 7f99c2ffd700 1 -- 172.21.15.157:0/4012198530 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2165842846 0 0) 0x7f99c40c48f0 con 0x7f99c406a270 2023-10-06T04:03:33.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.211+0000 7f99c2ffd700 1 -- 172.21.15.157:0/4012198530 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99ac003570 con 0x7f99c406a270 2023-10-06T04:03:33.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.211+0000 7f99c3fff700 1 -- 172.21.15.157:0/4012198530 >> v2:172.21.15.175:3300/0 conn(0x7f99c406a860 msgr2=0x7f99c4156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:33.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.211+0000 7f99c3fff700 1 --2- 172.21.15.157:0/4012198530 >> v2:172.21.15.175:3300/0 conn(0x7f99c406a860 0x7f99c4156870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:33.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.211+0000 7f99c3fff700 1 -- 172.21.15.157:0/4012198530 >> v1:172.21.15.157:6789/0 conn(0x7f99c406a270 legacy=0x7f99c40c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:33.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.211+0000 7f99c3fff700 1 -- 172.21.15.157:0/4012198530 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c4063840 con 0x7f99c4069ea0 2023-10-06T04:03:33.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.211+0000 7f99c37fe700 1 --2- 172.21.15.157:0/4012198530 >> v2:172.21.15.175:3300/0 conn(0x7f99c406a860 0x7f99c4156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:03:33.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.211+0000 7f99c3fff700 1 --2- 172.21.15.157:0/4012198530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99c4069ea0 0x7f99c40631f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f99b40098e0 tx=0x7f99b40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85f4b8ab733f7906 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:33.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.211+0000 7f99c2ffd700 1 -- 172.21.15.157:0/4012198530 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b4011630 con 0x7f99c4069ea0 2023-10-06T04:03:33.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.211+0000 7f99c2ffd700 1 -- 172.21.15.157:0/4012198530 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b4011790 con 0x7f99c4069ea0 2023-10-06T04:03:33.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.211+0000 7f99c2ffd700 1 -- 172.21.15.157:0/4012198530 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b4011b30 con 0x7f99c4069ea0 2023-10-06T04:03:33.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.212+0000 7f99ca702700 1 -- 172.21.15.157:0/4012198530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99c4069ea0 msgr2=0x7f99c40631f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:33.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.212+0000 7f99ca702700 1 --2- 172.21.15.157:0/4012198530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99c4069ea0 0x7f99c40631f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f99b40098e0 tx=0x7f99b40091a0 comp rx=0 tx=0).stop 2023-10-06T04:03:33.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.212+0000 7f99ca702700 1 -- 172.21.15.157:0/4012198530 shutdown_connections 2023-10-06T04:03:33.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.212+0000 7f99ca702700 1 --2- 172.21.15.157:0/4012198530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99c4069ea0 0x7f99c40631f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:33.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.212+0000 7f99ca702700 1 --2- 172.21.15.157:0/4012198530 >> v2:172.21.15.175:3300/0 conn(0x7f99c406a860 0x7f99c4156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:33.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.212+0000 7f99ca702700 1 -- 172.21.15.157:0/4012198530 >> 172.21.15.157:0/4012198530 conn(0x7f99c40c8b20 msgr2=0x7f99c40c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:33.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.212+0000 7f99ca702700 1 -- 172.21.15.157:0/4012198530 shutdown_connections 2023-10-06T04:03:33.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.212+0000 7f99ca702700 1 -- 172.21.15.157:0/4012198530 wait complete. 2023-10-06T04:03:33.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.213+0000 7f99ca702700 1 Processor -- start 2023-10-06T04:03:33.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.213+0000 7f99ca702700 1 -- start start 2023-10-06T04:03:33.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.214+0000 7f99ca702700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99c4069ea0 0x7f99c4158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:33.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.214+0000 7f99ca702700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99c406a270 0x7f99c41663c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:33.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.214+0000 7f99ca702700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c406a860 0x7f99c4166900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:33.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.214+0000 7f99ca702700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c414b840 con 0x7f99c406a860 2023-10-06T04:03:33.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.214+0000 7f99ca702700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c414c220 con 0x7f99c4069ea0 2023-10-06T04:03:33.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.214+0000 7f99ca702700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c414a6f0 con 0x7f99c406a270 2023-10-06T04:03:33.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.214+0000 7f99c8c9f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c406a860 0x7f99c4166900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:33.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.214+0000 7f99c8c9f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c406a860 0x7f99c4166900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:48478/0 (socket says 172.21.15.157:48478) 2023-10-06T04:03:33.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.214+0000 7f99c8c9f700 1 -- 172.21.15.157:0/4088432677 learned_addr learned my addr 172.21.15.157:0/4088432677 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:33.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.214+0000 7f99c37fe700 1 --2- 172.21.15.157:0/4088432677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99c406a270 0x7f99c41663c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:33.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.214+0000 7f99c3fff700 1 --2- 172.21.15.157:0/4088432677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99c4069ea0 0x7f99c4158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:33.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.215+0000 7f99c8c9f700 1 -- 172.21.15.157:0/4088432677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99c406a270 msgr2=0x7f99c41663c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:33.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.215+0000 7f99c8c9f700 1 --2- 172.21.15.157:0/4088432677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99c406a270 0x7f99c41663c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:33.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.215+0000 7f99c8c9f700 1 -- 172.21.15.157:0/4088432677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99c4069ea0 msgr2=0x7f99c4158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:33.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.215+0000 7f99c8c9f700 1 --2- 172.21.15.157:0/4088432677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99c4069ea0 0x7f99c4158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:33.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.215+0000 7f99c8c9f700 1 -- 172.21.15.157:0/4088432677 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b8009420 con 0x7f99c406a860 2023-10-06T04:03:33.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.215+0000 7f99c37fe700 1 --2- 172.21.15.157:0/4088432677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99c406a270 0x7f99c41663c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:03:33.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.215+0000 7f99c8c9f700 1 --2- 172.21.15.157:0/4088432677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c406a860 0x7f99c4166900 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00cca0 tx=0x7f99bc001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:33.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.215+0000 7f99abfff700 1 -- 172.21.15.157:0/4088432677 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc00a8d0 con 0x7f99c406a860 2023-10-06T04:03:33.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.215+0000 7f99abfff700 1 -- 172.21.15.157:0/4088432677 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99bc00aa30 con 0x7f99c406a860 2023-10-06T04:03:33.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.215+0000 7f99ca702700 1 -- 172.21.15.157:0/4088432677 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b4009420 con 0x7f99c406a860 2023-10-06T04:03:33.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.215+0000 7f99abfff700 1 -- 172.21.15.157:0/4088432677 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc00adb0 con 0x7f99c406a860 2023-10-06T04:03:33.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.215+0000 7f99ca702700 1 -- 172.21.15.157:0/4088432677 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c414ca50 con 0x7f99c406a860 2023-10-06T04:03:33.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.216+0000 7f99ca702700 1 -- 172.21.15.157:0/4088432677 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c41661c0 con 0x7f99c406a860 2023-10-06T04:03:33.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.217+0000 7f99abfff700 1 -- 172.21.15.157:0/4088432677 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f99bc007300 con 0x7f99c406a860 2023-10-06T04:03:33.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.218+0000 7f99abfff700 1 --2- 172.21.15.157:0/4088432677 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f99b0066f30 0x7f99b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:33.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.218+0000 7f99abfff700 1 -- 172.21.15.157:0/4088432677 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7f99bc085760 con 0x7f99c406a860 2023-10-06T04:03:33.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.218+0000 7f99c3fff700 1 --2- 172.21.15.157:0/4088432677 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f99b0066f30 0x7f99b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:33.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.219+0000 7f99c3fff700 1 --2- 172.21.15.157:0/4088432677 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f99b0066f30 0x7f99b00693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f99b40091e0 tx=0x7f99b4017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:33.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.221+0000 7f99abfff700 1 -- 172.21.15.157:0/4088432677 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f99bc052610 con 0x7f99c406a860 2023-10-06T04:03:33.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.446+0000 7f99ca702700 1 -- 172.21.15.157:0/4088432677 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99c40c3fc0 con 0x7f99c406a860 2023-10-06T04:03:33.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.446+0000 7f99abfff700 1 -- 172.21.15.157:0/4088432677 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+6162 (secure 0 0 0) 0x7f99bc055c30 con 0x7f99c406a860 2023-10-06T04:03:33.448 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:03:33.448 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3192541484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3192541484},{"type":"v1","addr":"172.21.15.175:6835","nonce":3192541484}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:13.713593+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4296,"mds_1":4287,"mds_2":4279},"failed":[],"damaged":[],"stopped":[3,4],"info":{"gid_4279":{"gid":4279,"name":"d","rank":2,"incarnation":11,"state":"up:stopping","state_seq":7,"addr":"172.21.15.175:6837/1002404870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1002404870},{"type":"v1","addr":"172.21.15.175:6837","nonce":1002404870}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:03:33.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.448+0000 7f99ca702700 1 -- 172.21.15.157:0/4088432677 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f99b0066f30 msgr2=0x7f99b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:33.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.448+0000 7f99ca702700 1 --2- 172.21.15.157:0/4088432677 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f99b0066f30 0x7f99b00693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f99b40091e0 tx=0x7f99b4017000 comp rx=0 tx=0).stop 2023-10-06T04:03:33.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.448+0000 7f99ca702700 1 -- 172.21.15.157:0/4088432677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c406a860 msgr2=0x7f99c4166900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:33.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.448+0000 7f99ca702700 1 --2- 172.21.15.157:0/4088432677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c406a860 0x7f99c4166900 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00cca0 tx=0x7f99bc001ae0 comp rx=0 tx=0).stop 2023-10-06T04:03:33.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.448+0000 7f99ca702700 1 -- 172.21.15.157:0/4088432677 shutdown_connections 2023-10-06T04:03:33.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.448+0000 7f99ca702700 1 --2- 172.21.15.157:0/4088432677 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f99b0066f30 0x7f99b00693e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:33.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.448+0000 7f99ca702700 1 --2- 172.21.15.157:0/4088432677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99c4069ea0 0x7f99c4158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:33.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.448+0000 7f99ca702700 1 --2- 172.21.15.157:0/4088432677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c406a860 0x7f99c4166900 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:33.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.448+0000 7f99ca702700 1 --2- 172.21.15.157:0/4088432677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99c406a270 0x7f99c41663c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:33.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.448+0000 7f99ca702700 1 -- 172.21.15.157:0/4088432677 >> 172.21.15.157:0/4088432677 conn(0x7f99c40c8b20 msgr2=0x7f99c406c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:33.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.449+0000 7f99ca702700 1 -- 172.21.15.157:0/4088432677 shutdown_connections 2023-10-06T04:03:33.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:33.449+0000 7f99ca702700 1 -- 172.21.15.157:0/4088432677 wait complete. 2023-10-06T04:03:33.452 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 51 2023-10-06T04:03:33.462 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:03:33.798 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2023-10-06T04:03:34.710 INFO:tasks.workunit.client.0.smithi157.stdout:removed './236/22' 2023-10-06T04:03:34.710 INFO:tasks.workunit.client.0.smithi157.stdout:removed './236/25' 2023-10-06T04:03:34.710 INFO:tasks.workunit.client.0.smithi157.stdout:removed './236/30' 2023-10-06T04:03:34.710 INFO:tasks.workunit.client.0.smithi157.stdout:removed './236/33' 2023-10-06T04:03:34.710 INFO:tasks.workunit.client.0.smithi157.stdout:removed './236/23' 2023-10-06T04:03:34.710 INFO:tasks.workunit.client.0.smithi157.stdout:removed './236/35' 2023-10-06T04:03:34.710 INFO:tasks.workunit.client.0.smithi157.stdout:removed './236/26' 2023-10-06T04:03:34.711 INFO:tasks.workunit.client.0.smithi157.stdout:removed './236/37' 2023-10-06T04:03:34.711 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './236' 2023-10-06T04:03:34.711 INFO:tasks.workunit.client.0.smithi157.stdout:removed './237/21' 2023-10-06T04:03:34.711 INFO:tasks.workunit.client.0.smithi157.stdout:removed './237/27' 2023-10-06T04:03:34.711 INFO:tasks.workunit.client.0.smithi157.stdout:removed './237/31' 2023-10-06T04:03:34.711 INFO:tasks.workunit.client.0.smithi157.stdout:removed './237/36' 2023-10-06T04:03:34.711 INFO:tasks.workunit.client.0.smithi157.stdout:removed './237/20' 2023-10-06T04:03:34.711 INFO:tasks.workunit.client.0.smithi157.stdout:removed './237/39' 2023-10-06T04:03:34.711 INFO:tasks.workunit.client.0.smithi157.stdout:removed './237/34' 2023-10-06T04:03:34.711 INFO:tasks.workunit.client.0.smithi157.stdout:removed './237/38' 2023-10-06T04:03:34.711 INFO:tasks.workunit.client.0.smithi157.stdout:removed './237/28' 2023-10-06T04:03:34.711 INFO:tasks.workunit.client.0.smithi157.stdout:removed './237/24' 2023-10-06T04:03:34.712 INFO:tasks.workunit.client.0.smithi157.stdout:removed './237/29' 2023-10-06T04:03:34.712 INFO:tasks.workunit.client.0.smithi157.stdout:removed './237/32' 2023-10-06T04:03:34.712 INFO:tasks.workunit.client.0.smithi157.stdout:removed './237/22' 2023-10-06T04:03:34.712 INFO:tasks.workunit.client.0.smithi157.stdout:removed './237/25' 2023-10-06T04:03:34.712 INFO:tasks.workunit.client.0.smithi157.stdout:removed './237/30' 2023-10-06T04:03:34.712 INFO:tasks.workunit.client.0.smithi157.stdout:removed './237/33' 2023-10-06T04:03:34.712 INFO:tasks.workunit.client.0.smithi157.stdout:removed './237/23' 2023-10-06T04:03:34.712 INFO:tasks.workunit.client.0.smithi157.stdout:removed './237/35' 2023-10-06T04:03:34.712 INFO:tasks.workunit.client.0.smithi157.stdout:removed './237/26' 2023-10-06T04:03:34.712 INFO:tasks.workunit.client.0.smithi157.stdout:removed './237/37' 2023-10-06T04:03:34.712 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './237' 2023-10-06T04:03:34.712 INFO:tasks.workunit.client.0.smithi157.stdout:removed './238/21' 2023-10-06T04:03:34.713 INFO:tasks.workunit.client.0.smithi157.stdout:removed './238/27' 2023-10-06T04:03:34.713 INFO:tasks.workunit.client.0.smithi157.stdout:removed './238/31' 2023-10-06T04:03:34.713 INFO:tasks.workunit.client.0.smithi157.stdout:removed './238/36' 2023-10-06T04:03:34.713 INFO:tasks.workunit.client.0.smithi157.stdout:removed './238/20' 2023-10-06T04:03:34.713 INFO:tasks.workunit.client.0.smithi157.stdout:removed './238/39' 2023-10-06T04:03:34.713 INFO:tasks.workunit.client.0.smithi157.stdout:removed './238/34' 2023-10-06T04:03:34.713 INFO:tasks.workunit.client.0.smithi157.stdout:removed './238/38' 2023-10-06T04:03:34.713 INFO:tasks.workunit.client.0.smithi157.stdout:removed './238/28' 2023-10-06T04:03:34.713 INFO:tasks.workunit.client.0.smithi157.stdout:removed './238/24' 2023-10-06T04:03:34.713 INFO:tasks.workunit.client.0.smithi157.stdout:removed './238/29' 2023-10-06T04:03:34.713 INFO:tasks.workunit.client.0.smithi157.stdout:removed './238/32' 2023-10-06T04:03:34.713 INFO:tasks.workunit.client.0.smithi157.stdout:removed './238/22' 2023-10-06T04:03:34.713 INFO:tasks.workunit.client.0.smithi157.stdout:removed './238/25' 2023-10-06T04:03:34.714 INFO:tasks.workunit.client.0.smithi157.stdout:removed './238/30' 2023-10-06T04:03:34.714 INFO:tasks.workunit.client.0.smithi157.stdout:removed './238/33' 2023-10-06T04:03:34.714 INFO:tasks.workunit.client.0.smithi157.stdout:removed './238/23' 2023-10-06T04:03:34.714 INFO:tasks.workunit.client.0.smithi157.stdout:removed './238/35' 2023-10-06T04:03:34.714 INFO:tasks.workunit.client.0.smithi157.stdout:removed './238/26' 2023-10-06T04:03:34.714 INFO:tasks.workunit.client.0.smithi157.stdout:removed './238/37' 2023-10-06T04:03:34.714 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './238' 2023-10-06T04:03:34.714 INFO:tasks.workunit.client.0.smithi157.stdout:removed './239/21' 2023-10-06T04:03:34.714 INFO:tasks.workunit.client.0.smithi157.stdout:removed './239/27' 2023-10-06T04:03:34.714 INFO:tasks.workunit.client.0.smithi157.stdout:removed './239/31' 2023-10-06T04:03:34.714 INFO:tasks.workunit.client.0.smithi157.stdout:removed './239/36' 2023-10-06T04:03:34.715 INFO:tasks.workunit.client.0.smithi157.stdout:removed './239/20' 2023-10-06T04:03:34.715 INFO:tasks.workunit.client.0.smithi157.stdout:removed './239/39' 2023-10-06T04:03:34.715 INFO:tasks.workunit.client.0.smithi157.stdout:removed './239/34' 2023-10-06T04:03:34.715 INFO:tasks.workunit.client.0.smithi157.stdout:removed './239/38' 2023-10-06T04:03:34.715 INFO:tasks.workunit.client.0.smithi157.stdout:removed './239/28' 2023-10-06T04:03:34.715 INFO:tasks.workunit.client.0.smithi157.stdout:removed './239/24' 2023-10-06T04:03:34.715 INFO:tasks.workunit.client.0.smithi157.stdout:removed './239/29' 2023-10-06T04:03:34.715 INFO:tasks.workunit.client.0.smithi157.stdout:removed './239/32' 2023-10-06T04:03:34.715 INFO:tasks.workunit.client.0.smithi157.stdout:removed './239/22' 2023-10-06T04:03:34.715 INFO:tasks.workunit.client.0.smithi157.stdout:removed './239/25' 2023-10-06T04:03:34.715 INFO:tasks.workunit.client.0.smithi157.stdout:removed './239/30' 2023-10-06T04:03:34.715 INFO:tasks.workunit.client.0.smithi157.stdout:removed './239/33' 2023-10-06T04:03:34.715 INFO:tasks.workunit.client.0.smithi157.stdout:removed './239/23' 2023-10-06T04:03:34.716 INFO:tasks.workunit.client.0.smithi157.stdout:removed './239/35' 2023-10-06T04:03:34.716 INFO:tasks.workunit.client.0.smithi157.stdout:removed './239/26' 2023-10-06T04:03:34.716 INFO:tasks.workunit.client.0.smithi157.stdout:removed './239/37' 2023-10-06T04:03:34.716 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './239' 2023-10-06T04:03:34.716 INFO:tasks.workunit.client.0.smithi157.stdout:removed './240/21' 2023-10-06T04:03:34.716 INFO:tasks.workunit.client.0.smithi157.stdout:removed './240/27' 2023-10-06T04:03:34.716 INFO:tasks.workunit.client.0.smithi157.stdout:removed './240/31' 2023-10-06T04:03:34.716 INFO:tasks.workunit.client.0.smithi157.stdout:removed './240/36' 2023-10-06T04:03:34.716 INFO:tasks.workunit.client.0.smithi157.stdout:removed './240/20' 2023-10-06T04:03:34.716 INFO:tasks.workunit.client.0.smithi157.stdout:removed './240/39' 2023-10-06T04:03:34.716 INFO:tasks.workunit.client.0.smithi157.stdout:removed './240/34' 2023-10-06T04:03:34.716 INFO:tasks.workunit.client.0.smithi157.stdout:removed './240/38' 2023-10-06T04:03:34.717 INFO:tasks.workunit.client.0.smithi157.stdout:removed './240/28' 2023-10-06T04:03:34.717 INFO:tasks.workunit.client.0.smithi157.stdout:removed './240/24' 2023-10-06T04:03:34.717 INFO:tasks.workunit.client.0.smithi157.stdout:removed './240/29' 2023-10-06T04:03:34.717 INFO:tasks.workunit.client.0.smithi157.stdout:removed './240/32' 2023-10-06T04:03:34.717 INFO:tasks.workunit.client.0.smithi157.stdout:removed './240/22' 2023-10-06T04:03:34.717 INFO:tasks.workunit.client.0.smithi157.stdout:removed './240/25' 2023-10-06T04:03:34.717 INFO:tasks.workunit.client.0.smithi157.stdout:removed './240/30' 2023-10-06T04:03:34.717 INFO:tasks.workunit.client.0.smithi157.stdout:removed './240/33' 2023-10-06T04:03:34.717 INFO:tasks.workunit.client.0.smithi157.stdout:removed './240/23' 2023-10-06T04:03:34.717 INFO:tasks.workunit.client.0.smithi157.stdout:removed './240/35' 2023-10-06T04:03:34.717 INFO:tasks.workunit.client.0.smithi157.stdout:removed './240/26' 2023-10-06T04:03:34.717 INFO:tasks.workunit.client.0.smithi157.stdout:removed './240/37' 2023-10-06T04:03:34.717 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './240' 2023-10-06T04:03:34.718 INFO:tasks.workunit.client.0.smithi157.stdout:removed './241/21' 2023-10-06T04:03:34.718 INFO:tasks.workunit.client.0.smithi157.stdout:removed './241/27' 2023-10-06T04:03:34.718 INFO:tasks.workunit.client.0.smithi157.stdout:removed './241/31' 2023-10-06T04:03:34.718 INFO:tasks.workunit.client.0.smithi157.stdout:removed './241/36' 2023-10-06T04:03:34.718 INFO:tasks.workunit.client.0.smithi157.stdout:removed './241/20' 2023-10-06T04:03:34.718 INFO:tasks.workunit.client.0.smithi157.stdout:removed './241/39' 2023-10-06T04:03:34.718 INFO:tasks.workunit.client.0.smithi157.stdout:removed './241/34' 2023-10-06T04:03:34.718 INFO:tasks.workunit.client.0.smithi157.stdout:removed './241/38' 2023-10-06T04:03:34.718 INFO:tasks.workunit.client.0.smithi157.stdout:removed './241/28' 2023-10-06T04:03:34.718 INFO:tasks.workunit.client.0.smithi157.stdout:removed './241/24' 2023-10-06T04:03:34.718 INFO:tasks.workunit.client.0.smithi157.stdout:removed './241/29' 2023-10-06T04:03:34.719 INFO:tasks.workunit.client.0.smithi157.stdout:removed './241/32' 2023-10-06T04:03:34.719 INFO:tasks.workunit.client.0.smithi157.stdout:removed './241/22' 2023-10-06T04:03:34.719 INFO:tasks.workunit.client.0.smithi157.stdout:removed './241/25' 2023-10-06T04:03:34.719 INFO:tasks.workunit.client.0.smithi157.stdout:removed './241/30' 2023-10-06T04:03:34.719 INFO:tasks.workunit.client.0.smithi157.stdout:removed './241/33' 2023-10-06T04:03:34.719 INFO:tasks.workunit.client.0.smithi157.stdout:removed './241/23' 2023-10-06T04:03:34.719 INFO:tasks.workunit.client.0.smithi157.stdout:removed './241/35' 2023-10-06T04:03:34.719 INFO:tasks.workunit.client.0.smithi157.stdout:removed './241/26' 2023-10-06T04:03:34.719 INFO:tasks.workunit.client.0.smithi157.stdout:removed './241/37' 2023-10-06T04:03:34.719 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './241' 2023-10-06T04:03:34.719 INFO:tasks.workunit.client.0.smithi157.stdout:removed './242/21' 2023-10-06T04:03:34.719 INFO:tasks.workunit.client.0.smithi157.stdout:removed './242/27' 2023-10-06T04:03:34.720 INFO:tasks.workunit.client.0.smithi157.stdout:removed './242/31' 2023-10-06T04:03:34.720 INFO:tasks.workunit.client.0.smithi157.stdout:removed './242/36' 2023-10-06T04:03:34.720 INFO:tasks.workunit.client.0.smithi157.stdout:removed './242/20' 2023-10-06T04:03:34.720 INFO:tasks.workunit.client.0.smithi157.stdout:removed './242/39' 2023-10-06T04:03:34.720 INFO:tasks.workunit.client.0.smithi157.stdout:removed './242/34' 2023-10-06T04:03:34.720 INFO:tasks.workunit.client.0.smithi157.stdout:removed './242/38' 2023-10-06T04:03:34.720 INFO:tasks.workunit.client.0.smithi157.stdout:removed './242/28' 2023-10-06T04:03:34.720 INFO:tasks.workunit.client.0.smithi157.stdout:removed './242/24' 2023-10-06T04:03:34.720 INFO:tasks.workunit.client.0.smithi157.stdout:removed './242/29' 2023-10-06T04:03:34.720 INFO:tasks.workunit.client.0.smithi157.stdout:removed './242/32' 2023-10-06T04:03:34.720 INFO:tasks.workunit.client.0.smithi157.stdout:removed './242/22' 2023-10-06T04:03:34.720 INFO:tasks.workunit.client.0.smithi157.stdout:removed './242/25' 2023-10-06T04:03:34.720 INFO:tasks.workunit.client.0.smithi157.stdout:removed './242/30' 2023-10-06T04:03:34.721 INFO:tasks.workunit.client.0.smithi157.stdout:removed './242/33' 2023-10-06T04:03:34.721 INFO:tasks.workunit.client.0.smithi157.stdout:removed './242/23' 2023-10-06T04:03:34.721 INFO:tasks.workunit.client.0.smithi157.stdout:removed './242/35' 2023-10-06T04:03:34.721 INFO:tasks.workunit.client.0.smithi157.stdout:removed './242/26' 2023-10-06T04:03:34.721 INFO:tasks.workunit.client.0.smithi157.stdout:removed './242/37' 2023-10-06T04:03:34.721 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './242' 2023-10-06T04:03:34.721 INFO:tasks.workunit.client.0.smithi157.stdout:removed './243/21' 2023-10-06T04:03:34.722 INFO:tasks.workunit.client.0.smithi157.stdout:removed './243/27' 2023-10-06T04:03:34.722 INFO:tasks.workunit.client.0.smithi157.stdout:removed './243/31' 2023-10-06T04:03:34.722 INFO:tasks.workunit.client.0.smithi157.stdout:removed './243/36' 2023-10-06T04:03:34.722 INFO:tasks.workunit.client.0.smithi157.stdout:removed './243/20' 2023-10-06T04:03:34.722 INFO:tasks.workunit.client.0.smithi157.stdout:removed './243/39' 2023-10-06T04:03:34.722 INFO:tasks.workunit.client.0.smithi157.stdout:removed './243/34' 2023-10-06T04:03:34.722 INFO:tasks.workunit.client.0.smithi157.stdout:removed './243/38' 2023-10-06T04:03:34.722 INFO:tasks.workunit.client.0.smithi157.stdout:removed './243/28' 2023-10-06T04:03:34.722 INFO:tasks.workunit.client.0.smithi157.stdout:removed './243/24' 2023-10-06T04:03:34.722 INFO:tasks.workunit.client.0.smithi157.stdout:removed './243/29' 2023-10-06T04:03:34.722 INFO:tasks.workunit.client.0.smithi157.stdout:removed './243/32' 2023-10-06T04:03:34.723 INFO:tasks.workunit.client.0.smithi157.stdout:removed './243/22' 2023-10-06T04:03:34.723 INFO:tasks.workunit.client.0.smithi157.stdout:removed './243/25' 2023-10-06T04:03:34.723 INFO:tasks.workunit.client.0.smithi157.stdout:removed './243/30' 2023-10-06T04:03:34.723 INFO:tasks.workunit.client.0.smithi157.stdout:removed './243/33' 2023-10-06T04:03:34.723 INFO:tasks.workunit.client.0.smithi157.stdout:removed './243/23' 2023-10-06T04:03:34.723 INFO:tasks.workunit.client.0.smithi157.stdout:removed './243/35' 2023-10-06T04:03:34.723 INFO:tasks.workunit.client.0.smithi157.stdout:removed './243/26' 2023-10-06T04:03:34.723 INFO:tasks.workunit.client.0.smithi157.stdout:removed './243/37' 2023-10-06T04:03:34.723 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './243' 2023-10-06T04:03:34.723 INFO:tasks.workunit.client.0.smithi157.stdout:removed './244/21' 2023-10-06T04:03:34.724 INFO:tasks.workunit.client.0.smithi157.stdout:removed './244/27' 2023-10-06T04:03:34.724 INFO:tasks.workunit.client.0.smithi157.stdout:removed './244/31' 2023-10-06T04:03:34.724 INFO:tasks.workunit.client.0.smithi157.stdout:removed './244/36' 2023-10-06T04:03:34.724 INFO:tasks.workunit.client.0.smithi157.stdout:removed './244/20' 2023-10-06T04:03:34.724 INFO:tasks.workunit.client.0.smithi157.stdout:removed './244/39' 2023-10-06T04:03:34.724 INFO:tasks.workunit.client.0.smithi157.stdout:removed './244/34' 2023-10-06T04:03:34.724 INFO:tasks.workunit.client.0.smithi157.stdout:removed './244/38' 2023-10-06T04:03:34.724 INFO:tasks.workunit.client.0.smithi157.stdout:removed './244/28' 2023-10-06T04:03:34.724 INFO:tasks.workunit.client.0.smithi157.stdout:removed './244/24' 2023-10-06T04:03:34.724 INFO:tasks.workunit.client.0.smithi157.stdout:removed './244/29' 2023-10-06T04:03:34.724 INFO:tasks.workunit.client.0.smithi157.stdout:removed './244/32' 2023-10-06T04:03:34.725 INFO:tasks.workunit.client.0.smithi157.stdout:removed './244/22' 2023-10-06T04:03:34.725 INFO:tasks.workunit.client.0.smithi157.stdout:removed './244/25' 2023-10-06T04:03:34.725 INFO:tasks.workunit.client.0.smithi157.stdout:removed './244/30' 2023-10-06T04:03:34.725 INFO:tasks.workunit.client.0.smithi157.stdout:removed './244/33' 2023-10-06T04:03:34.725 INFO:tasks.workunit.client.0.smithi157.stdout:removed './244/23' 2023-10-06T04:03:34.725 INFO:tasks.workunit.client.0.smithi157.stdout:removed './244/35' 2023-10-06T04:03:34.725 INFO:tasks.workunit.client.0.smithi157.stdout:removed './244/26' 2023-10-06T04:03:34.725 INFO:tasks.workunit.client.0.smithi157.stdout:removed './244/37' 2023-10-06T04:03:34.725 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './244' 2023-10-06T04:03:34.725 INFO:tasks.workunit.client.0.smithi157.stdout:removed './245/21' 2023-10-06T04:03:34.725 INFO:tasks.workunit.client.0.smithi157.stdout:removed './245/27' 2023-10-06T04:03:34.726 INFO:tasks.workunit.client.0.smithi157.stdout:removed './245/31' 2023-10-06T04:03:34.726 INFO:tasks.workunit.client.0.smithi157.stdout:removed './245/36' 2023-10-06T04:03:34.726 INFO:tasks.workunit.client.0.smithi157.stdout:removed './245/20' 2023-10-06T04:03:34.726 INFO:tasks.workunit.client.0.smithi157.stdout:removed './245/39' 2023-10-06T04:03:34.726 INFO:tasks.workunit.client.0.smithi157.stdout:removed './245/34' 2023-10-06T04:03:34.726 INFO:tasks.workunit.client.0.smithi157.stdout:removed './245/38' 2023-10-06T04:03:34.726 INFO:tasks.workunit.client.0.smithi157.stdout:removed './245/28' 2023-10-06T04:03:34.726 INFO:tasks.workunit.client.0.smithi157.stdout:removed './245/24' 2023-10-06T04:03:34.726 INFO:tasks.workunit.client.0.smithi157.stdout:removed './245/29' 2023-10-06T04:03:34.726 INFO:tasks.workunit.client.0.smithi157.stdout:removed './245/32' 2023-10-06T04:03:34.726 INFO:tasks.workunit.client.0.smithi157.stdout:removed './245/22' 2023-10-06T04:03:34.727 INFO:tasks.workunit.client.0.smithi157.stdout:removed './245/25' 2023-10-06T04:03:34.727 INFO:tasks.workunit.client.0.smithi157.stdout:removed './245/30' 2023-10-06T04:03:34.727 INFO:tasks.workunit.client.0.smithi157.stdout:removed './245/33' 2023-10-06T04:03:34.727 INFO:tasks.workunit.client.0.smithi157.stdout:removed './245/23' 2023-10-06T04:03:34.727 INFO:tasks.workunit.client.0.smithi157.stdout:removed './245/35' 2023-10-06T04:03:34.727 INFO:tasks.workunit.client.0.smithi157.stdout:removed './245/26' 2023-10-06T04:03:34.727 INFO:tasks.workunit.client.0.smithi157.stdout:removed './245/37' 2023-10-06T04:03:34.727 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './245' 2023-10-06T04:03:34.727 INFO:tasks.workunit.client.0.smithi157.stdout:removed './246/21' 2023-10-06T04:03:34.727 INFO:tasks.workunit.client.0.smithi157.stdout:removed './246/27' 2023-10-06T04:03:34.728 INFO:tasks.workunit.client.0.smithi157.stdout:removed './246/31' 2023-10-06T04:03:34.728 INFO:tasks.workunit.client.0.smithi157.stdout:removed './246/36' 2023-10-06T04:03:34.728 INFO:tasks.workunit.client.0.smithi157.stdout:removed './246/20' 2023-10-06T04:03:34.728 INFO:tasks.workunit.client.0.smithi157.stdout:removed './246/39' 2023-10-06T04:03:34.728 INFO:tasks.workunit.client.0.smithi157.stdout:removed './246/34' 2023-10-06T04:03:34.728 INFO:tasks.workunit.client.0.smithi157.stdout:removed './246/38' 2023-10-06T04:03:34.728 INFO:tasks.workunit.client.0.smithi157.stdout:removed './246/28' 2023-10-06T04:03:34.728 INFO:tasks.workunit.client.0.smithi157.stdout:removed './246/24' 2023-10-06T04:03:34.728 INFO:tasks.workunit.client.0.smithi157.stdout:removed './246/29' 2023-10-06T04:03:34.728 INFO:tasks.workunit.client.0.smithi157.stdout:removed './246/32' 2023-10-06T04:03:34.729 INFO:tasks.workunit.client.0.smithi157.stdout:removed './246/22' 2023-10-06T04:03:35.463 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:03:35.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.576+0000 7f400ebde700 1 Processor -- start 2023-10-06T04:03:35.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.576+0000 7f400ebde700 1 -- start start 2023-10-06T04:03:35.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.577+0000 7f400ebde700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f400806cbb0 0x7f400806d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:35.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.577+0000 7f400ebde700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f400806c1f0 0x7f400806da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:35.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.577+0000 7f400ebde700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40080c9780 con 0x7f400806cf80 2023-10-06T04:03:35.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.577+0000 7f400ebde700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40080c8c40 con 0x7f400806cbb0 2023-10-06T04:03:35.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.577+0000 7f400ebde700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f400806e0c0 con 0x7f400806c1f0 2023-10-06T04:03:35.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.577+0000 7f400c97a700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f400806cbb0 0x7f400806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:35.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.577+0000 7f3ffffff700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f400806c1f0 0x7f400806da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:35.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.577+0000 7f400d17b700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f400806cf80 0x7f400814fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.157:40744/0 (socket says 172.21.15.157:40744) 2023-10-06T04:03:35.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.577+0000 7f400d17b700 1 -- 172.21.15.157:0/1529634037 learned_addr learned my addr 172.21.15.157:0/1529634037 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:03:35.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.578+0000 7f3ffffff700 1 -- 172.21.15.157:0/1529634037 >> v2:172.21.15.175:3300/0 conn(0x7f400806cbb0 msgr2=0x7f400806d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:35.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.578+0000 7f3ffffff700 1 --2- 172.21.15.157:0/1529634037 >> v2:172.21.15.175:3300/0 conn(0x7f400806cbb0 0x7f400806d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:35.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.578+0000 7f3ffffff700 1 -- 172.21.15.157:0/1529634037 >> v1:172.21.15.175:6789/0 conn(0x7f400806cf80 legacy=0x7f400814fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:35.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.578+0000 7f3ffffff700 1 -- 172.21.15.157:0/1529634037 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40080c8e50 con 0x7f400806c1f0 2023-10-06T04:03:35.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.578+0000 7f3fff7fe700 1 -- 172.21.15.157:0/1529634037 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1769979744 0 0) 0x7f40080c9780 con 0x7f400806cf80 2023-10-06T04:03:35.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.578+0000 7f400c97a700 1 --2- 172.21.15.157:0/1529634037 >> v2:172.21.15.175:3300/0 conn(0x7f400806cbb0 0x7f400806d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:03:35.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.578+0000 7f3ffffff700 1 --2- 172.21.15.157:0/1529634037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f400806c1f0 0x7f400806da70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009910 tx=0x7f3ff80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6029d45533e7c6b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:35.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.578+0000 7f3fff7fe700 1 -- 172.21.15.157:0/1529634037 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff800b6d0 con 0x7f400806c1f0 2023-10-06T04:03:35.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.578+0000 7f3fff7fe700 1 -- 172.21.15.157:0/1529634037 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff800b830 con 0x7f400806c1f0 2023-10-06T04:03:35.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.578+0000 7f3fff7fe700 1 -- 172.21.15.157:0/1529634037 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff800bbf0 con 0x7f400806c1f0 2023-10-06T04:03:35.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.579+0000 7f400ebde700 1 -- 172.21.15.157:0/1529634037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f400806c1f0 msgr2=0x7f400806da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:35.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.579+0000 7f400ebde700 1 --2- 172.21.15.157:0/1529634037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f400806c1f0 0x7f400806da70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009910 tx=0x7f3ff80091a0 comp rx=0 tx=0).stop 2023-10-06T04:03:35.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.579+0000 7f400ebde700 1 -- 172.21.15.157:0/1529634037 shutdown_connections 2023-10-06T04:03:35.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.579+0000 7f400ebde700 1 --2- 172.21.15.157:0/1529634037 >> v2:172.21.15.175:3300/0 conn(0x7f400806cbb0 0x7f400806d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:35.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.579+0000 7f400ebde700 1 --2- 172.21.15.157:0/1529634037 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f400806c1f0 0x7f400806da70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:35.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.579+0000 7f400ebde700 1 -- 172.21.15.157:0/1529634037 >> 172.21.15.157:0/1529634037 conn(0x7f400806f230 msgr2=0x7f400806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:35.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.579+0000 7f400ebde700 1 -- 172.21.15.157:0/1529634037 shutdown_connections 2023-10-06T04:03:35.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.579+0000 7f400ebde700 1 -- 172.21.15.157:0/1529634037 wait complete. 2023-10-06T04:03:35.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.580+0000 7f400ebde700 1 Processor -- start 2023-10-06T04:03:35.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.580+0000 7f400ebde700 1 -- start start 2023-10-06T04:03:35.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.581+0000 7f400ebde700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f400806c1f0 0x7f40080c3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:35.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.581+0000 7f400ebde700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f400806cbb0 0x7f40080c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:35.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.581+0000 7f400ebde700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f400806cf80 0x7f40080c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:35.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.581+0000 7f400ebde700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40080c6320 con 0x7f400806cbb0 2023-10-06T04:03:35.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.581+0000 7f400ebde700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40080c3510 con 0x7f400806c1f0 2023-10-06T04:03:35.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.581+0000 7f400ebde700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40081487b0 con 0x7f400806cf80 2023-10-06T04:03:35.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.581+0000 7f3ffffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f400806cbb0 0x7f40080c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:35.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.581+0000 7f3ffffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f400806cbb0 0x7f40080c58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:48492/0 (socket says 172.21.15.157:48492) 2023-10-06T04:03:35.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.581+0000 7f3ffffff700 1 -- 172.21.15.157:0/1471826281 learned_addr learned my addr 172.21.15.157:0/1471826281 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:35.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.581+0000 7f400d17b700 1 --2- 172.21.15.157:0/1471826281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f400806cf80 0x7f40080c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:35.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.581+0000 7f400c97a700 1 --2- 172.21.15.157:0/1471826281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f400806c1f0 0x7f40080c3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:35.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.581+0000 7f3ffffff700 1 -- 172.21.15.157:0/1471826281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f400806cf80 msgr2=0x7f40080c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:35.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.581+0000 7f3ffffff700 1 --2- 172.21.15.157:0/1471826281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f400806cf80 0x7f40080c5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:35.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.581+0000 7f3ffffff700 1 -- 172.21.15.157:0/1471826281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f400806c1f0 msgr2=0x7f40080c3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:35.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.581+0000 7f3ffffff700 1 --2- 172.21.15.157:0/1471826281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f400806c1f0 0x7f40080c3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:35.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.581+0000 7f3ffffff700 1 -- 172.21.15.157:0/1471826281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4004009420 con 0x7f400806cbb0 2023-10-06T04:03:35.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.581+0000 7f3ffffff700 1 --2- 172.21.15.157:0/1471826281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f400806cbb0 0x7f40080c58a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009fb0 tx=0x7f3ff8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:35.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.582+0000 7f3ffcff9700 1 -- 172.21.15.157:0/1471826281 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff801f070 con 0x7f400806cbb0 2023-10-06T04:03:35.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.582+0000 7f3ffcff9700 1 -- 172.21.15.157:0/1471826281 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff8000bc0 con 0x7f400806cbb0 2023-10-06T04:03:35.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.582+0000 7f3ffcff9700 1 -- 172.21.15.157:0/1471826281 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff800bbf0 con 0x7f400806cbb0 2023-10-06T04:03:35.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.582+0000 7f400ebde700 1 -- 172.21.15.157:0/1471826281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff8009420 con 0x7f400806cbb0 2023-10-06T04:03:35.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.582+0000 7f400ebde700 1 -- 172.21.15.157:0/1471826281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4008146160 con 0x7f400806cbb0 2023-10-06T04:03:35.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.583+0000 7f400ebde700 1 -- 172.21.15.157:0/1471826281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4008089ca0 con 0x7f400806cbb0 2023-10-06T04:03:35.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.583+0000 7f3ffcff9700 1 -- 172.21.15.157:0/1471826281 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3ff8023070 con 0x7f400806cbb0 2023-10-06T04:03:35.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.583+0000 7f3ffcff9700 1 --2- 172.21.15.157:0/1471826281 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3ff4066f60 0x7f3ff4069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:35.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.584+0000 7f3ffcff9700 1 -- 172.21.15.157:0/1471826281 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7f3ff8085cc0 con 0x7f400806cbb0 2023-10-06T04:03:35.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.584+0000 7f400c97a700 1 --2- 172.21.15.157:0/1471826281 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3ff4066f60 0x7f3ff4069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:35.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.584+0000 7f400c97a700 1 --2- 172.21.15.157:0/1471826281 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3ff4066f60 0x7f3ff4069410 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4004009f40 tx=0x7f40040091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:35.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.588+0000 7f3ffcff9700 1 -- 172.21.15.157:0/1471826281 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ff8052bc0 con 0x7f400806cbb0 2023-10-06T04:03:35.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.841+0000 7f400ebde700 1 -- 172.21.15.157:0/1471826281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40080c9130 con 0x7f400806cbb0 2023-10-06T04:03:35.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.841+0000 7f3ffcff9700 1 -- 172.21.15.157:0/1471826281 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+6145 (secure 0 0 0) 0x7f3ff80561a0 con 0x7f400806cbb0 2023-10-06T04:03:35.843 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:03:35.843 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3192541484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3192541484},{"type":"v1","addr":"172.21.15.175:6835","nonce":3192541484}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:33.754360+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":4287},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:03:35.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.844+0000 7f400ebde700 1 -- 172.21.15.157:0/1471826281 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3ff4066f60 msgr2=0x7f3ff4069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:35.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.844+0000 7f400ebde700 1 --2- 172.21.15.157:0/1471826281 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3ff4066f60 0x7f3ff4069410 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4004009f40 tx=0x7f40040091a0 comp rx=0 tx=0).stop 2023-10-06T04:03:35.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.844+0000 7f400ebde700 1 -- 172.21.15.157:0/1471826281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f400806cbb0 msgr2=0x7f40080c58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:35.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.844+0000 7f400ebde700 1 --2- 172.21.15.157:0/1471826281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f400806cbb0 0x7f40080c58a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009fb0 tx=0x7f3ff8017000 comp rx=0 tx=0).stop 2023-10-06T04:03:35.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.844+0000 7f400ebde700 1 -- 172.21.15.157:0/1471826281 shutdown_connections 2023-10-06T04:03:35.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.844+0000 7f400ebde700 1 --2- 172.21.15.157:0/1471826281 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3ff4066f60 0x7f3ff4069410 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:35.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.844+0000 7f400ebde700 1 --2- 172.21.15.157:0/1471826281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f400806c1f0 0x7f40080c3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:35.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.844+0000 7f400ebde700 1 --2- 172.21.15.157:0/1471826281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f400806cf80 0x7f40080c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:35.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.844+0000 7f400ebde700 1 --2- 172.21.15.157:0/1471826281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f400806cbb0 0x7f40080c58a0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:35.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.844+0000 7f400ebde700 1 -- 172.21.15.157:0/1471826281 >> 172.21.15.157:0/1471826281 conn(0x7f400806f230 msgr2=0x7f40080c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:35.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.844+0000 7f400ebde700 1 -- 172.21.15.157:0/1471826281 shutdown_connections 2023-10-06T04:03:35.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.844+0000 7f400ebde700 1 -- 172.21.15.157:0/1471826281 wait complete. 2023-10-06T04:03:35.847 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 53 2023-10-06T04:03:35.857 INFO:tasks.mds_thrash.fs.[cephfs]:mds cluster has 2 alive and active, now stable! 2023-10-06T04:03:35.857 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.b (rank=3) 2023-10-06T04:03:35.857 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T04:03:35.857 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:03:35.866 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T04:03:35.866 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:03:35.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.979+0000 7f509276b700 1 Processor -- start 2023-10-06T04:03:35.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.980+0000 7f509276b700 1 -- start start 2023-10-06T04:03:35.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.980+0000 7f509276b700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f508c14e0a0 0x7f508c153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:35.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.980+0000 7f509276b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f508c0c5c00 0x7f508c0ca440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:35.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.980+0000 7f509276b700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f508c0c5770 con 0x7f508c14dcd0 2023-10-06T04:03:35.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.980+0000 7f509276b700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f508c154c80 con 0x7f508c14e0a0 2023-10-06T04:03:35.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.980+0000 7f509276b700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508c154200 con 0x7f508c0c5c00 2023-10-06T04:03:35.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.980+0000 7f508b7fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f508c14e0a0 0x7f508c153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:35.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.980+0000 7f508b7fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f508c14e0a0 0x7f508c153cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:48494/0 (socket says 172.21.15.157:48494) 2023-10-06T04:03:35.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.980+0000 7f508b7fe700 1 -- 172.21.15.157:0/441749629 learned_addr learned my addr 172.21.15.157:0/441749629 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:35.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.981+0000 7f5090d08700 1 --2- 172.21.15.157:0/441749629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f508c0c5c00 0x7f508c0ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:35.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.981+0000 7f508b7fe700 1 -- 172.21.15.157:0/441749629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f508c0c5c00 msgr2=0x7f508c0ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:35.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.981+0000 7f508b7fe700 1 --2- 172.21.15.157:0/441749629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f508c0c5c00 0x7f508c0ca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:35.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.981+0000 7f508b7fe700 1 -- 172.21.15.157:0/441749629 >> v1:172.21.15.175:6789/0 conn(0x7f508c14dcd0 legacy=0x7f508c0c18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:35.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.981+0000 7f508b7fe700 1 -- 172.21.15.157:0/441749629 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c1543e0 con 0x7f508c14e0a0 2023-10-06T04:03:35.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.981+0000 7f508b7fe700 1 --2- 172.21.15.157:0/441749629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f508c14e0a0 0x7f508c153cc0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f50800099e0 tx=0x7f5080009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91bd9ddca899d846 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:35.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.981+0000 7f508affd700 1 -- 172.21.15.157:0/441749629 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508000b8b0 con 0x7f508c14e0a0 2023-10-06T04:03:35.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.981+0000 7f508affd700 1 -- 172.21.15.157:0/441749629 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508000ba10 con 0x7f508c14e0a0 2023-10-06T04:03:35.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.981+0000 7f508affd700 1 -- 172.21.15.157:0/441749629 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508000bdb0 con 0x7f508c14e0a0 2023-10-06T04:03:35.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.981+0000 7f509276b700 1 -- 172.21.15.157:0/441749629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f508c14e0a0 msgr2=0x7f508c153cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:35.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.981+0000 7f509276b700 1 --2- 172.21.15.157:0/441749629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f508c14e0a0 0x7f508c153cc0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f50800099e0 tx=0x7f5080009250 comp rx=0 tx=0).stop 2023-10-06T04:03:35.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.981+0000 7f509276b700 1 -- 172.21.15.157:0/441749629 shutdown_connections 2023-10-06T04:03:35.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.981+0000 7f509276b700 1 --2- 172.21.15.157:0/441749629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f508c0c5c00 0x7f508c0ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:35.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.981+0000 7f509276b700 1 --2- 172.21.15.157:0/441749629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f508c14e0a0 0x7f508c153cc0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:35.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.982+0000 7f509276b700 1 -- 172.21.15.157:0/441749629 >> 172.21.15.157:0/441749629 conn(0x7f508c0692b0 msgr2=0x7f508c0696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:35.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.982+0000 7f509276b700 1 -- 172.21.15.157:0/441749629 shutdown_connections 2023-10-06T04:03:35.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.982+0000 7f509276b700 1 -- 172.21.15.157:0/441749629 wait complete. 2023-10-06T04:03:35.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.983+0000 7f509276b700 1 Processor -- start 2023-10-06T04:03:35.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.983+0000 7f509276b700 1 -- start start 2023-10-06T04:03:35.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.983+0000 7f509276b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f508c0c5c00 0x7f508c165b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:35.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.983+0000 7f509276b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f508c14dcd0 0x7f508c166070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:35.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.983+0000 7f509276b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f508c14e0a0 0x7f508c164060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:35.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.983+0000 7f509276b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c153f00 con 0x7f508c0c5c00 2023-10-06T04:03:35.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.983+0000 7f509276b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c14fe80 con 0x7f508c14dcd0 2023-10-06T04:03:35.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.983+0000 7f509276b700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508c14f6d0 con 0x7f508c14e0a0 2023-10-06T04:03:35.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.984+0000 7f508bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f508c0c5c00 0x7f508c165b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:35.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.984+0000 7f508bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f508c0c5c00 0x7f508c165b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:48496/0 (socket says 172.21.15.157:48496) 2023-10-06T04:03:35.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.984+0000 7f508bfff700 1 -- 172.21.15.157:0/1450143182 learned_addr learned my addr 172.21.15.157:0/1450143182 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:35.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.984+0000 7f508b7fe700 1 --2- 172.21.15.157:0/1450143182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f508c14dcd0 0x7f508c166070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:35.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.984+0000 7f5090d08700 1 --2- 172.21.15.157:0/1450143182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f508c14e0a0 0x7f508c164060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:35.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.984+0000 7f508bfff700 1 -- 172.21.15.157:0/1450143182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f508c14e0a0 msgr2=0x7f508c164060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:35.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.984+0000 7f508bfff700 1 --2- 172.21.15.157:0/1450143182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f508c14e0a0 0x7f508c164060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:35.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.984+0000 7f508bfff700 1 -- 172.21.15.157:0/1450143182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f508c14dcd0 msgr2=0x7f508c166070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:35.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.984+0000 7f508bfff700 1 --2- 172.21.15.157:0/1450143182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f508c14dcd0 0x7f508c166070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:35.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.984+0000 7f508bfff700 1 -- 172.21.15.157:0/1450143182 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50800094f0 con 0x7f508c0c5c00 2023-10-06T04:03:35.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.984+0000 7f508bfff700 1 --2- 172.21.15.157:0/1450143182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f508c0c5c00 0x7f508c165b30 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f507c00c590 tx=0x7f507c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:35.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.984+0000 7f5073fff700 1 -- 172.21.15.157:0/1450143182 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c011590 con 0x7f508c0c5c00 2023-10-06T04:03:35.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.984+0000 7f5073fff700 1 -- 172.21.15.157:0/1450143182 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507c0116f0 con 0x7f508c0c5c00 2023-10-06T04:03:35.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.984+0000 7f5073fff700 1 -- 172.21.15.157:0/1450143182 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c011ab0 con 0x7f508c0c5c00 2023-10-06T04:03:35.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.984+0000 7f509276b700 1 -- 172.21.15.157:0/1450143182 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f508c1539d0 con 0x7f508c0c5c00 2023-10-06T04:03:35.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.984+0000 7f509276b700 1 -- 172.21.15.157:0/1450143182 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f508c14f8c0 con 0x7f508c0c5c00 2023-10-06T04:03:35.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.985+0000 7f509276b700 1 -- 172.21.15.157:0/1450143182 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f508c06a8f0 con 0x7f508c0c5c00 2023-10-06T04:03:35.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.986+0000 7f5073fff700 1 -- 172.21.15.157:0/1450143182 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f507c01b3b0 con 0x7f508c0c5c00 2023-10-06T04:03:35.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.986+0000 7f5073fff700 1 --2- 172.21.15.157:0/1450143182 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5078066f30 0x7f50780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:35.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.986+0000 7f5073fff700 1 -- 172.21.15.157:0/1450143182 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7f507c083e90 con 0x7f508c0c5c00 2023-10-06T04:03:35.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.986+0000 7f508b7fe700 1 --2- 172.21.15.157:0/1450143182 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5078066f30 0x7f50780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:35.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.987+0000 7f508b7fe700 1 --2- 172.21.15.157:0/1450143182 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5078066f30 0x7f50780693e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f50800099e0 tx=0x7f5080017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:35.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:35.990+0000 7f5073fff700 1 -- 172.21.15.157:0/1450143182 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f507c050d50 con 0x7f508c0c5c00 2023-10-06T04:03:36.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:36.254+0000 7f509276b700 1 -- 172.21.15.157:0/1450143182 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f508c14b700 con 0x7f508c0c5c00 2023-10-06T04:03:36.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:36.255+0000 7f5073fff700 1 -- 172.21.15.157:0/1450143182 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+6145 (secure 0 0 0) 0x7f507c017020 con 0x7f508c0c5c00 2023-10-06T04:03:36.256 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:03:36.257 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3192541484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3192541484},{"type":"v1","addr":"172.21.15.175:6835","nonce":3192541484}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:33.754360+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":4287},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:03:36.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:36.257+0000 7f509276b700 1 -- 172.21.15.157:0/1450143182 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5078066f30 msgr2=0x7f50780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:36.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:36.257+0000 7f509276b700 1 --2- 172.21.15.157:0/1450143182 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5078066f30 0x7f50780693e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f50800099e0 tx=0x7f5080017000 comp rx=0 tx=0).stop 2023-10-06T04:03:36.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:36.257+0000 7f509276b700 1 -- 172.21.15.157:0/1450143182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f508c0c5c00 msgr2=0x7f508c165b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:36.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:36.257+0000 7f509276b700 1 --2- 172.21.15.157:0/1450143182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f508c0c5c00 0x7f508c165b30 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f507c00c590 tx=0x7f507c00c950 comp rx=0 tx=0).stop 2023-10-06T04:03:36.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:36.257+0000 7f509276b700 1 -- 172.21.15.157:0/1450143182 shutdown_connections 2023-10-06T04:03:36.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:36.257+0000 7f509276b700 1 --2- 172.21.15.157:0/1450143182 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5078066f30 0x7f50780693e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:36.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:36.257+0000 7f509276b700 1 --2- 172.21.15.157:0/1450143182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f508c14e0a0 0x7f508c164060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:36.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:36.257+0000 7f509276b700 1 --2- 172.21.15.157:0/1450143182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f508c0c5c00 0x7f508c165b30 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:36.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:36.257+0000 7f509276b700 1 --2- 172.21.15.157:0/1450143182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f508c14dcd0 0x7f508c166070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:36.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:36.257+0000 7f509276b700 1 -- 172.21.15.157:0/1450143182 >> 172.21.15.157:0/1450143182 conn(0x7f508c0692b0 msgr2=0x7f508c159630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:36.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:36.257+0000 7f509276b700 1 -- 172.21.15.157:0/1450143182 shutdown_connections 2023-10-06T04:03:36.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:36.257+0000 7f509276b700 1 -- 172.21.15.157:0/1450143182 wait complete. 2023-10-06T04:03:36.260 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 53 2023-10-06T04:03:36.270 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-10-06T04:03:36.852 INFO:tasks.workunit.client.0.smithi157.stdout:removed './246/25' 2023-10-06T04:03:36.852 INFO:tasks.workunit.client.0.smithi157.stdout:removed './246/30' 2023-10-06T04:03:36.852 INFO:tasks.workunit.client.0.smithi157.stdout:removed './246/33' 2023-10-06T04:03:36.852 INFO:tasks.workunit.client.0.smithi157.stdout:removed './246/23' 2023-10-06T04:03:36.852 INFO:tasks.workunit.client.0.smithi157.stdout:removed './246/35' 2023-10-06T04:03:36.852 INFO:tasks.workunit.client.0.smithi157.stdout:removed './246/26' 2023-10-06T04:03:36.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './246/37' 2023-10-06T04:03:36.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './246' 2023-10-06T04:03:36.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './247/21' 2023-10-06T04:03:36.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './247/27' 2023-10-06T04:03:36.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './247/31' 2023-10-06T04:03:36.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './247/36' 2023-10-06T04:03:36.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './247/20' 2023-10-06T04:03:36.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './247/39' 2023-10-06T04:03:36.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './247/34' 2023-10-06T04:03:36.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './247/38' 2023-10-06T04:03:36.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './247/28' 2023-10-06T04:03:36.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './247/24' 2023-10-06T04:03:36.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './247/29' 2023-10-06T04:03:36.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './247/32' 2023-10-06T04:03:36.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './247/22' 2023-10-06T04:03:36.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './247/25' 2023-10-06T04:03:36.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './247/30' 2023-10-06T04:03:36.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './247/33' 2023-10-06T04:03:36.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './247/23' 2023-10-06T04:03:36.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './247/35' 2023-10-06T04:03:36.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './247/26' 2023-10-06T04:03:36.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './247/37' 2023-10-06T04:03:36.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './247' 2023-10-06T04:03:36.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './248/21' 2023-10-06T04:03:36.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './248/27' 2023-10-06T04:03:36.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './248/31' 2023-10-06T04:03:36.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './248/36' 2023-10-06T04:03:36.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './248/20' 2023-10-06T04:03:36.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './248/39' 2023-10-06T04:03:36.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './248/34' 2023-10-06T04:03:36.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './248/38' 2023-10-06T04:03:36.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './248/28' 2023-10-06T04:03:36.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './248/24' 2023-10-06T04:03:36.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './248/29' 2023-10-06T04:03:36.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './248/32' 2023-10-06T04:03:36.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './248/22' 2023-10-06T04:03:36.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './248/25' 2023-10-06T04:03:36.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './248/30' 2023-10-06T04:03:36.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './248/33' 2023-10-06T04:03:36.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './248/23' 2023-10-06T04:03:36.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './248/35' 2023-10-06T04:03:36.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './248/26' 2023-10-06T04:03:36.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './248/37' 2023-10-06T04:03:36.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './248' 2023-10-06T04:03:36.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './249/21' 2023-10-06T04:03:36.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './249/27' 2023-10-06T04:03:36.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './249/31' 2023-10-06T04:03:36.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './249/36' 2023-10-06T04:03:36.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './249/20' 2023-10-06T04:03:36.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './249/39' 2023-10-06T04:03:36.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './249/34' 2023-10-06T04:03:36.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './249/38' 2023-10-06T04:03:36.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './249/28' 2023-10-06T04:03:36.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './249/24' 2023-10-06T04:03:36.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './249/29' 2023-10-06T04:03:36.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './249/32' 2023-10-06T04:03:36.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './249/22' 2023-10-06T04:03:36.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './249/25' 2023-10-06T04:03:36.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './249/30' 2023-10-06T04:03:36.858 INFO:tasks.workunit.client.0.smithi157.stdout:removed './249/33' 2023-10-06T04:03:36.858 INFO:tasks.workunit.client.0.smithi157.stdout:removed './249/23' 2023-10-06T04:03:36.858 INFO:tasks.workunit.client.0.smithi157.stdout:removed './249/35' 2023-10-06T04:03:36.858 INFO:tasks.workunit.client.0.smithi157.stdout:removed './249/26' 2023-10-06T04:03:36.858 INFO:tasks.workunit.client.0.smithi157.stdout:removed './249/37' 2023-10-06T04:03:36.858 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './249' 2023-10-06T04:03:36.858 INFO:tasks.workunit.client.0.smithi157.stdout:removed './250/21' 2023-10-06T04:03:36.858 INFO:tasks.workunit.client.0.smithi157.stdout:removed './250/27' 2023-10-06T04:03:36.858 INFO:tasks.workunit.client.0.smithi157.stdout:removed './250/31' 2023-10-06T04:03:36.858 INFO:tasks.workunit.client.0.smithi157.stdout:removed './250/36' 2023-10-06T04:03:36.858 INFO:tasks.workunit.client.0.smithi157.stdout:removed './250/20' 2023-10-06T04:03:36.859 INFO:tasks.workunit.client.0.smithi157.stdout:removed './250/39' 2023-10-06T04:03:36.859 INFO:tasks.workunit.client.0.smithi157.stdout:removed './250/34' 2023-10-06T04:03:36.859 INFO:tasks.workunit.client.0.smithi157.stdout:removed './250/38' 2023-10-06T04:03:36.859 INFO:tasks.workunit.client.0.smithi157.stdout:removed './250/28' 2023-10-06T04:03:36.859 INFO:tasks.workunit.client.0.smithi157.stdout:removed './250/24' 2023-10-06T04:03:36.859 INFO:tasks.workunit.client.0.smithi157.stdout:removed './250/29' 2023-10-06T04:03:36.859 INFO:tasks.workunit.client.0.smithi157.stdout:removed './250/32' 2023-10-06T04:03:36.859 INFO:tasks.workunit.client.0.smithi157.stdout:removed './250/22' 2023-10-06T04:03:36.859 INFO:tasks.workunit.client.0.smithi157.stdout:removed './250/25' 2023-10-06T04:03:36.859 INFO:tasks.workunit.client.0.smithi157.stdout:removed './250/30' 2023-10-06T04:03:36.859 INFO:tasks.workunit.client.0.smithi157.stdout:removed './250/33' 2023-10-06T04:03:36.859 INFO:tasks.workunit.client.0.smithi157.stdout:removed './250/23' 2023-10-06T04:03:36.860 INFO:tasks.workunit.client.0.smithi157.stdout:removed './250/35' 2023-10-06T04:03:36.860 INFO:tasks.workunit.client.0.smithi157.stdout:removed './250/26' 2023-10-06T04:03:36.860 INFO:tasks.workunit.client.0.smithi157.stdout:removed './250/37' 2023-10-06T04:03:36.860 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './250' 2023-10-06T04:03:36.860 INFO:tasks.workunit.client.0.smithi157.stdout:removed './251/21' 2023-10-06T04:03:36.860 INFO:tasks.workunit.client.0.smithi157.stdout:removed './251/27' 2023-10-06T04:03:36.860 INFO:tasks.workunit.client.0.smithi157.stdout:removed './251/31' 2023-10-06T04:03:36.860 INFO:tasks.workunit.client.0.smithi157.stdout:removed './251/36' 2023-10-06T04:03:36.860 INFO:tasks.workunit.client.0.smithi157.stdout:removed './251/20' 2023-10-06T04:03:36.861 INFO:tasks.workunit.client.0.smithi157.stdout:removed './251/39' 2023-10-06T04:03:36.861 INFO:tasks.workunit.client.0.smithi157.stdout:removed './251/34' 2023-10-06T04:03:36.861 INFO:tasks.workunit.client.0.smithi157.stdout:removed './251/38' 2023-10-06T04:03:36.861 INFO:tasks.workunit.client.0.smithi157.stdout:removed './251/28' 2023-10-06T04:03:36.861 INFO:tasks.workunit.client.0.smithi157.stdout:removed './251/24' 2023-10-06T04:03:36.861 INFO:tasks.workunit.client.0.smithi157.stdout:removed './251/29' 2023-10-06T04:03:36.861 INFO:tasks.workunit.client.0.smithi157.stdout:removed './251/32' 2023-10-06T04:03:36.861 INFO:tasks.workunit.client.0.smithi157.stdout:removed './251/22' 2023-10-06T04:03:36.861 INFO:tasks.workunit.client.0.smithi157.stdout:removed './251/25' 2023-10-06T04:03:36.861 INFO:tasks.workunit.client.0.smithi157.stdout:removed './251/30' 2023-10-06T04:03:36.861 INFO:tasks.workunit.client.0.smithi157.stdout:removed './251/33' 2023-10-06T04:03:36.861 INFO:tasks.workunit.client.0.smithi157.stdout:removed './251/23' 2023-10-06T04:03:36.862 INFO:tasks.workunit.client.0.smithi157.stdout:removed './251/35' 2023-10-06T04:03:36.862 INFO:tasks.workunit.client.0.smithi157.stdout:removed './251/26' 2023-10-06T04:03:36.862 INFO:tasks.workunit.client.0.smithi157.stdout:removed './251/37' 2023-10-06T04:03:36.862 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './251' 2023-10-06T04:03:36.862 INFO:tasks.workunit.client.0.smithi157.stdout:removed './252/21' 2023-10-06T04:03:36.862 INFO:tasks.workunit.client.0.smithi157.stdout:removed './252/27' 2023-10-06T04:03:36.862 INFO:tasks.workunit.client.0.smithi157.stdout:removed './252/31' 2023-10-06T04:03:36.862 INFO:tasks.workunit.client.0.smithi157.stdout:removed './252/36' 2023-10-06T04:03:36.862 INFO:tasks.workunit.client.0.smithi157.stdout:removed './252/20' 2023-10-06T04:03:36.862 INFO:tasks.workunit.client.0.smithi157.stdout:removed './252/39' 2023-10-06T04:03:36.862 INFO:tasks.workunit.client.0.smithi157.stdout:removed './252/34' 2023-10-06T04:03:36.862 INFO:tasks.workunit.client.0.smithi157.stdout:removed './252/38' 2023-10-06T04:03:36.863 INFO:tasks.workunit.client.0.smithi157.stdout:removed './252/28' 2023-10-06T04:03:36.863 INFO:tasks.workunit.client.0.smithi157.stdout:removed './252/24' 2023-10-06T04:03:36.863 INFO:tasks.workunit.client.0.smithi157.stdout:removed './252/29' 2023-10-06T04:03:36.863 INFO:tasks.workunit.client.0.smithi157.stdout:removed './252/32' 2023-10-06T04:03:36.863 INFO:tasks.workunit.client.0.smithi157.stdout:removed './252/22' 2023-10-06T04:03:36.863 INFO:tasks.workunit.client.0.smithi157.stdout:removed './252/25' 2023-10-06T04:03:36.863 INFO:tasks.workunit.client.0.smithi157.stdout:removed './252/30' 2023-10-06T04:03:36.863 INFO:tasks.workunit.client.0.smithi157.stdout:removed './252/33' 2023-10-06T04:03:36.863 INFO:tasks.workunit.client.0.smithi157.stdout:removed './252/23' 2023-10-06T04:03:36.864 INFO:tasks.workunit.client.0.smithi157.stdout:removed './252/35' 2023-10-06T04:03:36.864 INFO:tasks.workunit.client.0.smithi157.stdout:removed './252/26' 2023-10-06T04:03:36.864 INFO:tasks.workunit.client.0.smithi157.stdout:removed './252/37' 2023-10-06T04:03:36.864 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './252' 2023-10-06T04:03:36.864 INFO:tasks.workunit.client.0.smithi157.stdout:removed './253/21' 2023-10-06T04:03:36.864 INFO:tasks.workunit.client.0.smithi157.stdout:removed './253/27' 2023-10-06T04:03:36.864 INFO:tasks.workunit.client.0.smithi157.stdout:removed './253/31' 2023-10-06T04:03:36.864 INFO:tasks.workunit.client.0.smithi157.stdout:removed './253/36' 2023-10-06T04:03:36.864 INFO:tasks.workunit.client.0.smithi157.stdout:removed './253/20' 2023-10-06T04:03:36.864 INFO:tasks.workunit.client.0.smithi157.stdout:removed './253/39' 2023-10-06T04:03:36.864 INFO:tasks.workunit.client.0.smithi157.stdout:removed './253/34' 2023-10-06T04:03:36.865 INFO:tasks.workunit.client.0.smithi157.stdout:removed './253/38' 2023-10-06T04:03:36.865 INFO:tasks.workunit.client.0.smithi157.stdout:removed './253/28' 2023-10-06T04:03:36.865 INFO:tasks.workunit.client.0.smithi157.stdout:removed './253/24' 2023-10-06T04:03:36.865 INFO:tasks.workunit.client.0.smithi157.stdout:removed './253/29' 2023-10-06T04:03:36.865 INFO:tasks.workunit.client.0.smithi157.stdout:removed './253/32' 2023-10-06T04:03:36.865 INFO:tasks.workunit.client.0.smithi157.stdout:removed './253/22' 2023-10-06T04:03:36.865 INFO:tasks.workunit.client.0.smithi157.stdout:removed './253/25' 2023-10-06T04:03:36.865 INFO:tasks.workunit.client.0.smithi157.stdout:removed './253/30' 2023-10-06T04:03:36.865 INFO:tasks.workunit.client.0.smithi157.stdout:removed './253/33' 2023-10-06T04:03:36.865 INFO:tasks.workunit.client.0.smithi157.stdout:removed './253/23' 2023-10-06T04:03:36.865 INFO:tasks.workunit.client.0.smithi157.stdout:removed './253/35' 2023-10-06T04:03:36.866 INFO:tasks.workunit.client.0.smithi157.stdout:removed './253/26' 2023-10-06T04:03:36.866 INFO:tasks.workunit.client.0.smithi157.stdout:removed './253/37' 2023-10-06T04:03:36.866 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './253' 2023-10-06T04:03:36.866 INFO:tasks.workunit.client.0.smithi157.stdout:removed './254/21' 2023-10-06T04:03:36.866 INFO:tasks.workunit.client.0.smithi157.stdout:removed './254/27' 2023-10-06T04:03:36.866 INFO:tasks.workunit.client.0.smithi157.stdout:removed './254/31' 2023-10-06T04:03:36.866 INFO:tasks.workunit.client.0.smithi157.stdout:removed './254/36' 2023-10-06T04:03:36.866 INFO:tasks.workunit.client.0.smithi157.stdout:removed './254/20' 2023-10-06T04:03:36.866 INFO:tasks.workunit.client.0.smithi157.stdout:removed './254/39' 2023-10-06T04:03:36.866 INFO:tasks.workunit.client.0.smithi157.stdout:removed './254/34' 2023-10-06T04:03:36.866 INFO:tasks.workunit.client.0.smithi157.stdout:removed './254/38' 2023-10-06T04:03:36.866 INFO:tasks.workunit.client.0.smithi157.stdout:removed './254/28' 2023-10-06T04:03:36.867 INFO:tasks.workunit.client.0.smithi157.stdout:removed './254/24' 2023-10-06T04:03:36.867 INFO:tasks.workunit.client.0.smithi157.stdout:removed './254/29' 2023-10-06T04:03:36.867 INFO:tasks.workunit.client.0.smithi157.stdout:removed './254/32' 2023-10-06T04:03:36.867 INFO:tasks.workunit.client.0.smithi157.stdout:removed './254/22' 2023-10-06T04:03:36.867 INFO:tasks.workunit.client.0.smithi157.stdout:removed './254/25' 2023-10-06T04:03:36.867 INFO:tasks.workunit.client.0.smithi157.stdout:removed './254/30' 2023-10-06T04:03:36.867 INFO:tasks.workunit.client.0.smithi157.stdout:removed './254/33' 2023-10-06T04:03:36.867 INFO:tasks.workunit.client.0.smithi157.stdout:removed './254/23' 2023-10-06T04:03:36.867 INFO:tasks.workunit.client.0.smithi157.stdout:removed './254/35' 2023-10-06T04:03:36.867 INFO:tasks.workunit.client.0.smithi157.stdout:removed './254/26' 2023-10-06T04:03:36.867 INFO:tasks.workunit.client.0.smithi157.stdout:removed './254/37' 2023-10-06T04:03:36.868 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './254' 2023-10-06T04:03:36.868 INFO:tasks.workunit.client.0.smithi157.stdout:removed './255/21' 2023-10-06T04:03:36.868 INFO:tasks.workunit.client.0.smithi157.stdout:removed './255/27' 2023-10-06T04:03:36.868 INFO:tasks.workunit.client.0.smithi157.stdout:removed './255/31' 2023-10-06T04:03:36.868 INFO:tasks.workunit.client.0.smithi157.stdout:removed './255/36' 2023-10-06T04:03:36.868 INFO:tasks.workunit.client.0.smithi157.stdout:removed './255/20' 2023-10-06T04:03:36.868 INFO:tasks.workunit.client.0.smithi157.stdout:removed './255/39' 2023-10-06T04:03:36.868 INFO:tasks.workunit.client.0.smithi157.stdout:removed './255/34' 2023-10-06T04:03:36.868 INFO:tasks.workunit.client.0.smithi157.stdout:removed './255/38' 2023-10-06T04:03:36.869 INFO:tasks.workunit.client.0.smithi157.stdout:removed './255/28' 2023-10-06T04:03:36.869 INFO:tasks.workunit.client.0.smithi157.stdout:removed './255/24' 2023-10-06T04:03:36.869 INFO:tasks.workunit.client.0.smithi157.stdout:removed './255/29' 2023-10-06T04:03:36.869 INFO:tasks.workunit.client.0.smithi157.stdout:removed './255/32' 2023-10-06T04:03:36.869 INFO:tasks.workunit.client.0.smithi157.stdout:removed './255/22' 2023-10-06T04:03:36.869 INFO:tasks.workunit.client.0.smithi157.stdout:removed './255/25' 2023-10-06T04:03:36.869 INFO:tasks.workunit.client.0.smithi157.stdout:removed './255/30' 2023-10-06T04:03:36.869 INFO:tasks.workunit.client.0.smithi157.stdout:removed './255/33' 2023-10-06T04:03:36.869 INFO:tasks.workunit.client.0.smithi157.stdout:removed './255/23' 2023-10-06T04:03:36.869 INFO:tasks.workunit.client.0.smithi157.stdout:removed './255/35' 2023-10-06T04:03:36.870 INFO:tasks.workunit.client.0.smithi157.stdout:removed './255/26' 2023-10-06T04:03:36.870 INFO:tasks.workunit.client.0.smithi157.stdout:removed './255/37' 2023-10-06T04:03:36.870 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './255' 2023-10-06T04:03:36.870 INFO:tasks.workunit.client.0.smithi157.stdout:removed './256/21' 2023-10-06T04:03:36.870 INFO:tasks.workunit.client.0.smithi157.stdout:removed './256/27' 2023-10-06T04:03:36.870 INFO:tasks.workunit.client.0.smithi157.stdout:removed './256/31' 2023-10-06T04:03:36.870 INFO:tasks.workunit.client.0.smithi157.stdout:removed './256/36' 2023-10-06T04:03:36.870 INFO:tasks.workunit.client.0.smithi157.stdout:removed './256/20' 2023-10-06T04:03:36.870 INFO:tasks.workunit.client.0.smithi157.stdout:removed './256/39' 2023-10-06T04:03:36.870 INFO:tasks.workunit.client.0.smithi157.stdout:removed './256/34' 2023-10-06T04:03:36.870 INFO:tasks.workunit.client.0.smithi157.stdout:removed './256/38' 2023-10-06T04:03:36.871 INFO:tasks.workunit.client.0.smithi157.stdout:removed './256/28' 2023-10-06T04:03:36.871 INFO:tasks.workunit.client.0.smithi157.stdout:removed './256/24' 2023-10-06T04:03:36.871 INFO:tasks.workunit.client.0.smithi157.stdout:removed './256/29' 2023-10-06T04:03:36.871 INFO:tasks.workunit.client.0.smithi157.stdout:removed './256/32' 2023-10-06T04:03:36.871 INFO:tasks.workunit.client.0.smithi157.stdout:removed './256/22' 2023-10-06T04:03:36.871 INFO:tasks.workunit.client.0.smithi157.stdout:removed './256/25' 2023-10-06T04:03:36.871 INFO:tasks.workunit.client.0.smithi157.stdout:removed './256/30' 2023-10-06T04:03:38.271 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:03:38.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.386+0000 7f32cef62700 1 Processor -- start 2023-10-06T04:03:38.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.386+0000 7f32cef62700 1 -- start start 2023-10-06T04:03:38.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.387+0000 7f32cef62700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f32c806a780 0x7f32c8159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:38.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.387+0000 7f32cef62700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f32c806ab50 0x7f32c806c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:38.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.387+0000 7f32cef62700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32c80bc750 0x7f32c806c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:38.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.387+0000 7f32cef62700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32c8155e90 con 0x7f32c806ab50 2023-10-06T04:03:38.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.387+0000 7f32cef62700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32c815a310 con 0x7f32c806a780 2023-10-06T04:03:38.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.387+0000 7f32cef62700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c81569b0 con 0x7f32c80bc750 2023-10-06T04:03:38.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.387+0000 7f32bffff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f32c806ab50 0x7f32c806c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:38.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.387+0000 7f32bffff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f32c806ab50 0x7f32c806c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:48498/0 (socket says 172.21.15.157:48498) 2023-10-06T04:03:38.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.387+0000 7f32bffff700 1 -- 172.21.15.157:0/721985181 learned_addr learned my addr 172.21.15.157:0/721985181 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:38.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.387+0000 7f32cd4ff700 1 --2- 172.21.15.157:0/721985181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32c80bc750 0x7f32c806c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:38.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.387+0000 7f32cccfe700 1 --2- 172.21.15.157:0/721985181 >> v2:172.21.15.175:3300/0 conn(0x7f32c806a780 0x7f32c8159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:38.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.388+0000 7f32bffff700 1 -- 172.21.15.157:0/721985181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32c80bc750 msgr2=0x7f32c806c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:38.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.388+0000 7f32bffff700 1 --2- 172.21.15.157:0/721985181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32c80bc750 0x7f32c806c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:38.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.388+0000 7f32bffff700 1 -- 172.21.15.157:0/721985181 >> v2:172.21.15.175:3300/0 conn(0x7f32c806a780 msgr2=0x7f32c8159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:38.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.388+0000 7f32bffff700 1 --2- 172.21.15.157:0/721985181 >> v2:172.21.15.175:3300/0 conn(0x7f32c806a780 0x7f32c8159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:38.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.388+0000 7f32bffff700 1 -- 172.21.15.157:0/721985181 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c806ea70 con 0x7f32c806ab50 2023-10-06T04:03:38.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.388+0000 7f32bffff700 1 --2- 172.21.15.157:0/721985181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c806ab50 0x7f32c806c1f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f32c80684e0 tx=0x7f32b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383ac9cfd644831a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:38.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.388+0000 7f32bf7fe700 1 -- 172.21.15.157:0/721985181 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b800b8b0 con 0x7f32c806ab50 2023-10-06T04:03:38.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.388+0000 7f32bf7fe700 1 -- 172.21.15.157:0/721985181 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b800ba10 con 0x7f32c806ab50 2023-10-06T04:03:38.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.388+0000 7f32bf7fe700 1 -- 172.21.15.157:0/721985181 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b800bdb0 con 0x7f32c806ab50 2023-10-06T04:03:38.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.388+0000 7f32cef62700 1 -- 172.21.15.157:0/721985181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c806ab50 msgr2=0x7f32c806c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:38.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.388+0000 7f32cef62700 1 --2- 172.21.15.157:0/721985181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c806ab50 0x7f32c806c1f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f32c80684e0 tx=0x7f32b8009250 comp rx=0 tx=0).stop 2023-10-06T04:03:38.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.388+0000 7f32cef62700 1 -- 172.21.15.157:0/721985181 shutdown_connections 2023-10-06T04:03:38.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.388+0000 7f32cef62700 1 --2- 172.21.15.157:0/721985181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32c80bc750 0x7f32c806c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:38.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.388+0000 7f32cef62700 1 --2- 172.21.15.157:0/721985181 >> v2:172.21.15.175:3300/0 conn(0x7f32c806a780 0x7f32c8159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:38.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.388+0000 7f32cef62700 1 --2- 172.21.15.157:0/721985181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c806ab50 0x7f32c806c1f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:38.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.388+0000 7f32cef62700 1 -- 172.21.15.157:0/721985181 >> 172.21.15.157:0/721985181 conn(0x7f32c806b510 msgr2=0x7f32c806b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:38.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.388+0000 7f32cef62700 1 -- 172.21.15.157:0/721985181 shutdown_connections 2023-10-06T04:03:38.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.388+0000 7f32cef62700 1 -- 172.21.15.157:0/721985181 wait complete. 2023-10-06T04:03:38.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.389+0000 7f32cef62700 1 Processor -- start 2023-10-06T04:03:38.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.390+0000 7f32cef62700 1 -- start start 2023-10-06T04:03:38.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.390+0000 7f32cef62700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32c806a780 0x7f32c80c3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:38.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.390+0000 7f32cef62700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32c806ab50 0x7f32c80c0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:38.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.390+0000 7f32cef62700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c80bc750 0x7f32c80c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:38.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.390+0000 7f32cef62700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c80c7b70 con 0x7f32c80bc750 2023-10-06T04:03:38.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.390+0000 7f32cef62700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c80b9690 con 0x7f32c806ab50 2023-10-06T04:03:38.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.390+0000 7f32cef62700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c80c5930 con 0x7f32c806a780 2023-10-06T04:03:38.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.390+0000 7f32cd4ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c80bc750 0x7f32c80c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:38.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.391+0000 7f32cd4ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c80bc750 0x7f32c80c4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:48512/0 (socket says 172.21.15.157:48512) 2023-10-06T04:03:38.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.391+0000 7f32cd4ff700 1 -- 172.21.15.157:0/1157471847 learned_addr learned my addr 172.21.15.157:0/1157471847 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:38.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.391+0000 7f32bffff700 1 --2- 172.21.15.157:0/1157471847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32c806ab50 0x7f32c80c0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:38.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.391+0000 7f32cccfe700 1 --2- 172.21.15.157:0/1157471847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32c806a780 0x7f32c80c3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:38.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.391+0000 7f32cd4ff700 1 -- 172.21.15.157:0/1157471847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32c806a780 msgr2=0x7f32c80c3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:38.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.391+0000 7f32cd4ff700 1 --2- 172.21.15.157:0/1157471847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32c806a780 0x7f32c80c3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:38.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.391+0000 7f32cd4ff700 1 -- 172.21.15.157:0/1157471847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32c806ab50 msgr2=0x7f32c80c0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:38.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.391+0000 7f32cd4ff700 1 --2- 172.21.15.157:0/1157471847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32c806ab50 0x7f32c80c0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:38.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.391+0000 7f32cd4ff700 1 -- 172.21.15.157:0/1157471847 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b80094f0 con 0x7f32c80bc750 2023-10-06T04:03:38.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.391+0000 7f32cccfe700 1 --2- 172.21.15.157:0/1157471847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32c806a780 0x7f32c80c3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:03:38.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.391+0000 7f32cd4ff700 1 --2- 172.21.15.157:0/1157471847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c80bc750 0x7f32c80c4c40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f32c0007f20 tx=0x7f32c0009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:38.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.391+0000 7f32bcff9700 1 -- 172.21.15.157:0/1157471847 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c000ebf0 con 0x7f32c80bc750 2023-10-06T04:03:38.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.391+0000 7f32bcff9700 1 -- 172.21.15.157:0/1157471847 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c000ed70 con 0x7f32c80bc750 2023-10-06T04:03:38.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.391+0000 7f32cef62700 1 -- 172.21.15.157:0/1157471847 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c80c0b90 con 0x7f32c80bc750 2023-10-06T04:03:38.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.391+0000 7f32bcff9700 1 -- 172.21.15.157:0/1157471847 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0014670 con 0x7f32c80bc750 2023-10-06T04:03:38.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.391+0000 7f32cef62700 1 -- 172.21.15.157:0/1157471847 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c80c25f0 con 0x7f32c80bc750 2023-10-06T04:03:38.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.392+0000 7f32cef62700 1 -- 172.21.15.157:0/1157471847 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c8166760 con 0x7f32c80bc750 2023-10-06T04:03:38.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.392+0000 7f32bcff9700 1 -- 172.21.15.157:0/1157471847 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f32c00149e0 con 0x7f32c80bc750 2023-10-06T04:03:38.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.393+0000 7f32bcff9700 1 --2- 172.21.15.157:0/1157471847 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f32b4066f30 0x7f32b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:38.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.393+0000 7f32bcff9700 1 -- 172.21.15.157:0/1157471847 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7f32c0084ee0 con 0x7f32c80bc750 2023-10-06T04:03:38.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.393+0000 7f32cccfe700 1 --2- 172.21.15.157:0/1157471847 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f32b4066f30 0x7f32b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:38.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.394+0000 7f32cccfe700 1 --2- 172.21.15.157:0/1157471847 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f32b4066f30 0x7f32b40693e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f32c4006fd0 tx=0x7f32c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:38.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.397+0000 7f32bcff9700 1 -- 172.21.15.157:0/1157471847 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32c8166760 con 0x7f32c80bc750 2023-10-06T04:03:38.581 INFO:tasks.workunit.client.0.smithi157.stdout:removed './256/33' 2023-10-06T04:03:38.581 INFO:tasks.workunit.client.0.smithi157.stdout:removed './256/23' 2023-10-06T04:03:38.581 INFO:tasks.workunit.client.0.smithi157.stdout:removed './256/35' 2023-10-06T04:03:38.581 INFO:tasks.workunit.client.0.smithi157.stdout:removed './256/26' 2023-10-06T04:03:38.582 INFO:tasks.workunit.client.0.smithi157.stdout:removed './256/37' 2023-10-06T04:03:38.582 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './256' 2023-10-06T04:03:38.582 INFO:tasks.workunit.client.0.smithi157.stdout:removed './257/21' 2023-10-06T04:03:38.582 INFO:tasks.workunit.client.0.smithi157.stdout:removed './257/27' 2023-10-06T04:03:38.582 INFO:tasks.workunit.client.0.smithi157.stdout:removed './257/31' 2023-10-06T04:03:38.582 INFO:tasks.workunit.client.0.smithi157.stdout:removed './257/36' 2023-10-06T04:03:38.582 INFO:tasks.workunit.client.0.smithi157.stdout:removed './257/20' 2023-10-06T04:03:38.582 INFO:tasks.workunit.client.0.smithi157.stdout:removed './257/39' 2023-10-06T04:03:38.582 INFO:tasks.workunit.client.0.smithi157.stdout:removed './257/34' 2023-10-06T04:03:38.582 INFO:tasks.workunit.client.0.smithi157.stdout:removed './257/38' 2023-10-06T04:03:38.582 INFO:tasks.workunit.client.0.smithi157.stdout:removed './257/28' 2023-10-06T04:03:38.583 INFO:tasks.workunit.client.0.smithi157.stdout:removed './257/24' 2023-10-06T04:03:38.583 INFO:tasks.workunit.client.0.smithi157.stdout:removed './257/29' 2023-10-06T04:03:38.583 INFO:tasks.workunit.client.0.smithi157.stdout:removed './257/32' 2023-10-06T04:03:38.583 INFO:tasks.workunit.client.0.smithi157.stdout:removed './257/22' 2023-10-06T04:03:38.583 INFO:tasks.workunit.client.0.smithi157.stdout:removed './257/25' 2023-10-06T04:03:38.583 INFO:tasks.workunit.client.0.smithi157.stdout:removed './257/30' 2023-10-06T04:03:38.583 INFO:tasks.workunit.client.0.smithi157.stdout:removed './257/33' 2023-10-06T04:03:38.583 INFO:tasks.workunit.client.0.smithi157.stdout:removed './257/23' 2023-10-06T04:03:38.583 INFO:tasks.workunit.client.0.smithi157.stdout:removed './257/35' 2023-10-06T04:03:38.583 INFO:tasks.workunit.client.0.smithi157.stdout:removed './257/26' 2023-10-06T04:03:38.583 INFO:tasks.workunit.client.0.smithi157.stdout:removed './257/37' 2023-10-06T04:03:38.583 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './257' 2023-10-06T04:03:38.583 INFO:tasks.workunit.client.0.smithi157.stdout:removed './258/21' 2023-10-06T04:03:38.584 INFO:tasks.workunit.client.0.smithi157.stdout:removed './258/27' 2023-10-06T04:03:38.584 INFO:tasks.workunit.client.0.smithi157.stdout:removed './258/31' 2023-10-06T04:03:38.584 INFO:tasks.workunit.client.0.smithi157.stdout:removed './258/36' 2023-10-06T04:03:38.584 INFO:tasks.workunit.client.0.smithi157.stdout:removed './258/20' 2023-10-06T04:03:38.584 INFO:tasks.workunit.client.0.smithi157.stdout:removed './258/39' 2023-10-06T04:03:38.584 INFO:tasks.workunit.client.0.smithi157.stdout:removed './258/34' 2023-10-06T04:03:38.584 INFO:tasks.workunit.client.0.smithi157.stdout:removed './258/38' 2023-10-06T04:03:38.584 INFO:tasks.workunit.client.0.smithi157.stdout:removed './258/28' 2023-10-06T04:03:38.584 INFO:tasks.workunit.client.0.smithi157.stdout:removed './258/24' 2023-10-06T04:03:38.585 INFO:tasks.workunit.client.0.smithi157.stdout:removed './258/29' 2023-10-06T04:03:38.585 INFO:tasks.workunit.client.0.smithi157.stdout:removed './258/32' 2023-10-06T04:03:38.585 INFO:tasks.workunit.client.0.smithi157.stdout:removed './258/22' 2023-10-06T04:03:38.585 INFO:tasks.workunit.client.0.smithi157.stdout:removed './258/25' 2023-10-06T04:03:38.585 INFO:tasks.workunit.client.0.smithi157.stdout:removed './258/30' 2023-10-06T04:03:38.585 INFO:tasks.workunit.client.0.smithi157.stdout:removed './258/33' 2023-10-06T04:03:38.585 INFO:tasks.workunit.client.0.smithi157.stdout:removed './258/23' 2023-10-06T04:03:38.585 INFO:tasks.workunit.client.0.smithi157.stdout:removed './258/35' 2023-10-06T04:03:38.585 INFO:tasks.workunit.client.0.smithi157.stdout:removed './258/26' 2023-10-06T04:03:38.585 INFO:tasks.workunit.client.0.smithi157.stdout:removed './258/37' 2023-10-06T04:03:38.585 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './258' 2023-10-06T04:03:38.585 INFO:tasks.workunit.client.0.smithi157.stdout:removed './259/21' 2023-10-06T04:03:38.586 INFO:tasks.workunit.client.0.smithi157.stdout:removed './259/27' 2023-10-06T04:03:38.586 INFO:tasks.workunit.client.0.smithi157.stdout:removed './259/31' 2023-10-06T04:03:38.586 INFO:tasks.workunit.client.0.smithi157.stdout:removed './259/36' 2023-10-06T04:03:38.586 INFO:tasks.workunit.client.0.smithi157.stdout:removed './259/20' 2023-10-06T04:03:38.586 INFO:tasks.workunit.client.0.smithi157.stdout:removed './259/39' 2023-10-06T04:03:38.586 INFO:tasks.workunit.client.0.smithi157.stdout:removed './259/34' 2023-10-06T04:03:38.586 INFO:tasks.workunit.client.0.smithi157.stdout:removed './259/38' 2023-10-06T04:03:38.586 INFO:tasks.workunit.client.0.smithi157.stdout:removed './259/28' 2023-10-06T04:03:38.586 INFO:tasks.workunit.client.0.smithi157.stdout:removed './259/24' 2023-10-06T04:03:38.586 INFO:tasks.workunit.client.0.smithi157.stdout:removed './259/29' 2023-10-06T04:03:38.586 INFO:tasks.workunit.client.0.smithi157.stdout:removed './259/32' 2023-10-06T04:03:38.587 INFO:tasks.workunit.client.0.smithi157.stdout:removed './259/22' 2023-10-06T04:03:38.587 INFO:tasks.workunit.client.0.smithi157.stdout:removed './259/25' 2023-10-06T04:03:38.587 INFO:tasks.workunit.client.0.smithi157.stdout:removed './259/30' 2023-10-06T04:03:38.587 INFO:tasks.workunit.client.0.smithi157.stdout:removed './259/33' 2023-10-06T04:03:38.587 INFO:tasks.workunit.client.0.smithi157.stdout:removed './259/23' 2023-10-06T04:03:38.587 INFO:tasks.workunit.client.0.smithi157.stdout:removed './259/35' 2023-10-06T04:03:38.587 INFO:tasks.workunit.client.0.smithi157.stdout:removed './259/26' 2023-10-06T04:03:38.587 INFO:tasks.workunit.client.0.smithi157.stdout:removed './259/37' 2023-10-06T04:03:38.587 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './259' 2023-10-06T04:03:38.587 INFO:tasks.workunit.client.0.smithi157.stdout:removed './260/21' 2023-10-06T04:03:38.587 INFO:tasks.workunit.client.0.smithi157.stdout:removed './260/27' 2023-10-06T04:03:38.587 INFO:tasks.workunit.client.0.smithi157.stdout:removed './260/31' 2023-10-06T04:03:38.587 INFO:tasks.workunit.client.0.smithi157.stdout:removed './260/36' 2023-10-06T04:03:38.588 INFO:tasks.workunit.client.0.smithi157.stdout:removed './260/20' 2023-10-06T04:03:38.588 INFO:tasks.workunit.client.0.smithi157.stdout:removed './260/39' 2023-10-06T04:03:38.588 INFO:tasks.workunit.client.0.smithi157.stdout:removed './260/34' 2023-10-06T04:03:38.588 INFO:tasks.workunit.client.0.smithi157.stdout:removed './260/38' 2023-10-06T04:03:38.588 INFO:tasks.workunit.client.0.smithi157.stdout:removed './260/28' 2023-10-06T04:03:38.588 INFO:tasks.workunit.client.0.smithi157.stdout:removed './260/24' 2023-10-06T04:03:38.588 INFO:tasks.workunit.client.0.smithi157.stdout:removed './260/29' 2023-10-06T04:03:38.588 INFO:tasks.workunit.client.0.smithi157.stdout:removed './260/32' 2023-10-06T04:03:38.588 INFO:tasks.workunit.client.0.smithi157.stdout:removed './260/22' 2023-10-06T04:03:38.588 INFO:tasks.workunit.client.0.smithi157.stdout:removed './260/25' 2023-10-06T04:03:38.588 INFO:tasks.workunit.client.0.smithi157.stdout:removed './260/30' 2023-10-06T04:03:38.588 INFO:tasks.workunit.client.0.smithi157.stdout:removed './260/33' 2023-10-06T04:03:38.589 INFO:tasks.workunit.client.0.smithi157.stdout:removed './260/23' 2023-10-06T04:03:38.589 INFO:tasks.workunit.client.0.smithi157.stdout:removed './260/35' 2023-10-06T04:03:38.589 INFO:tasks.workunit.client.0.smithi157.stdout:removed './260/26' 2023-10-06T04:03:38.589 INFO:tasks.workunit.client.0.smithi157.stdout:removed './260/37' 2023-10-06T04:03:38.589 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './260' 2023-10-06T04:03:38.589 INFO:tasks.workunit.client.0.smithi157.stdout:removed './261/21' 2023-10-06T04:03:38.589 INFO:tasks.workunit.client.0.smithi157.stdout:removed './261/27' 2023-10-06T04:03:38.589 INFO:tasks.workunit.client.0.smithi157.stdout:removed './261/31' 2023-10-06T04:03:38.589 INFO:tasks.workunit.client.0.smithi157.stdout:removed './261/36' 2023-10-06T04:03:38.589 INFO:tasks.workunit.client.0.smithi157.stdout:removed './261/20' 2023-10-06T04:03:38.589 INFO:tasks.workunit.client.0.smithi157.stdout:removed './261/39' 2023-10-06T04:03:38.589 INFO:tasks.workunit.client.0.smithi157.stdout:removed './261/34' 2023-10-06T04:03:38.590 INFO:tasks.workunit.client.0.smithi157.stdout:removed './261/38' 2023-10-06T04:03:38.590 INFO:tasks.workunit.client.0.smithi157.stdout:removed './261/28' 2023-10-06T04:03:38.590 INFO:tasks.workunit.client.0.smithi157.stdout:removed './261/24' 2023-10-06T04:03:38.590 INFO:tasks.workunit.client.0.smithi157.stdout:removed './261/29' 2023-10-06T04:03:38.590 INFO:tasks.workunit.client.0.smithi157.stdout:removed './261/32' 2023-10-06T04:03:38.590 INFO:tasks.workunit.client.0.smithi157.stdout:removed './261/22' 2023-10-06T04:03:38.590 INFO:tasks.workunit.client.0.smithi157.stdout:removed './261/25' 2023-10-06T04:03:38.590 INFO:tasks.workunit.client.0.smithi157.stdout:removed './261/30' 2023-10-06T04:03:38.590 INFO:tasks.workunit.client.0.smithi157.stdout:removed './261/33' 2023-10-06T04:03:38.590 INFO:tasks.workunit.client.0.smithi157.stdout:removed './261/23' 2023-10-06T04:03:38.590 INFO:tasks.workunit.client.0.smithi157.stdout:removed './261/35' 2023-10-06T04:03:38.590 INFO:tasks.workunit.client.0.smithi157.stdout:removed './261/26' 2023-10-06T04:03:38.591 INFO:tasks.workunit.client.0.smithi157.stdout:removed './261/37' 2023-10-06T04:03:38.591 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './261' 2023-10-06T04:03:38.591 INFO:tasks.workunit.client.0.smithi157.stdout:removed './262/21' 2023-10-06T04:03:38.591 INFO:tasks.workunit.client.0.smithi157.stdout:removed './262/27' 2023-10-06T04:03:38.591 INFO:tasks.workunit.client.0.smithi157.stdout:removed './262/31' 2023-10-06T04:03:38.591 INFO:tasks.workunit.client.0.smithi157.stdout:removed './262/36' 2023-10-06T04:03:38.591 INFO:tasks.workunit.client.0.smithi157.stdout:removed './262/20' 2023-10-06T04:03:38.591 INFO:tasks.workunit.client.0.smithi157.stdout:removed './262/39' 2023-10-06T04:03:38.591 INFO:tasks.workunit.client.0.smithi157.stdout:removed './262/34' 2023-10-06T04:03:38.591 INFO:tasks.workunit.client.0.smithi157.stdout:removed './262/38' 2023-10-06T04:03:38.591 INFO:tasks.workunit.client.0.smithi157.stdout:removed './262/28' 2023-10-06T04:03:38.591 INFO:tasks.workunit.client.0.smithi157.stdout:removed './262/24' 2023-10-06T04:03:38.592 INFO:tasks.workunit.client.0.smithi157.stdout:removed './262/29' 2023-10-06T04:03:38.592 INFO:tasks.workunit.client.0.smithi157.stdout:removed './262/32' 2023-10-06T04:03:38.592 INFO:tasks.workunit.client.0.smithi157.stdout:removed './262/22' 2023-10-06T04:03:38.592 INFO:tasks.workunit.client.0.smithi157.stdout:removed './262/25' 2023-10-06T04:03:38.592 INFO:tasks.workunit.client.0.smithi157.stdout:removed './262/30' 2023-10-06T04:03:38.592 INFO:tasks.workunit.client.0.smithi157.stdout:removed './262/33' 2023-10-06T04:03:38.592 INFO:tasks.workunit.client.0.smithi157.stdout:removed './262/23' 2023-10-06T04:03:38.592 INFO:tasks.workunit.client.0.smithi157.stdout:removed './262/35' 2023-10-06T04:03:38.592 INFO:tasks.workunit.client.0.smithi157.stdout:removed './262/26' 2023-10-06T04:03:38.592 INFO:tasks.workunit.client.0.smithi157.stdout:removed './262/37' 2023-10-06T04:03:38.592 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './262' 2023-10-06T04:03:38.592 INFO:tasks.workunit.client.0.smithi157.stdout:removed './263/21' 2023-10-06T04:03:38.593 INFO:tasks.workunit.client.0.smithi157.stdout:removed './263/27' 2023-10-06T04:03:38.593 INFO:tasks.workunit.client.0.smithi157.stdout:removed './263/31' 2023-10-06T04:03:38.593 INFO:tasks.workunit.client.0.smithi157.stdout:removed './263/36' 2023-10-06T04:03:38.593 INFO:tasks.workunit.client.0.smithi157.stdout:removed './263/20' 2023-10-06T04:03:38.593 INFO:tasks.workunit.client.0.smithi157.stdout:removed './263/39' 2023-10-06T04:03:38.593 INFO:tasks.workunit.client.0.smithi157.stdout:removed './263/34' 2023-10-06T04:03:38.593 INFO:tasks.workunit.client.0.smithi157.stdout:removed './263/38' 2023-10-06T04:03:38.593 INFO:tasks.workunit.client.0.smithi157.stdout:removed './263/28' 2023-10-06T04:03:38.593 INFO:tasks.workunit.client.0.smithi157.stdout:removed './263/24' 2023-10-06T04:03:38.593 INFO:tasks.workunit.client.0.smithi157.stdout:removed './263/29' 2023-10-06T04:03:38.593 INFO:tasks.workunit.client.0.smithi157.stdout:removed './263/32' 2023-10-06T04:03:38.593 INFO:tasks.workunit.client.0.smithi157.stdout:removed './263/22' 2023-10-06T04:03:38.594 INFO:tasks.workunit.client.0.smithi157.stdout:removed './263/25' 2023-10-06T04:03:38.594 INFO:tasks.workunit.client.0.smithi157.stdout:removed './263/30' 2023-10-06T04:03:38.594 INFO:tasks.workunit.client.0.smithi157.stdout:removed './263/33' 2023-10-06T04:03:38.594 INFO:tasks.workunit.client.0.smithi157.stdout:removed './263/23' 2023-10-06T04:03:38.594 INFO:tasks.workunit.client.0.smithi157.stdout:removed './263/35' 2023-10-06T04:03:38.594 INFO:tasks.workunit.client.0.smithi157.stdout:removed './263/26' 2023-10-06T04:03:38.594 INFO:tasks.workunit.client.0.smithi157.stdout:removed './263/37' 2023-10-06T04:03:38.594 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './263' 2023-10-06T04:03:38.594 INFO:tasks.workunit.client.0.smithi157.stdout:removed './264/21' 2023-10-06T04:03:38.594 INFO:tasks.workunit.client.0.smithi157.stdout:removed './264/27' 2023-10-06T04:03:38.594 INFO:tasks.workunit.client.0.smithi157.stdout:removed './264/31' 2023-10-06T04:03:38.595 INFO:tasks.workunit.client.0.smithi157.stdout:removed './264/36' 2023-10-06T04:03:38.595 INFO:tasks.workunit.client.0.smithi157.stdout:removed './264/20' 2023-10-06T04:03:38.595 INFO:tasks.workunit.client.0.smithi157.stdout:removed './264/39' 2023-10-06T04:03:38.595 INFO:tasks.workunit.client.0.smithi157.stdout:removed './264/34' 2023-10-06T04:03:38.595 INFO:tasks.workunit.client.0.smithi157.stdout:removed './264/38' 2023-10-06T04:03:38.595 INFO:tasks.workunit.client.0.smithi157.stdout:removed './264/28' 2023-10-06T04:03:38.595 INFO:tasks.workunit.client.0.smithi157.stdout:removed './264/24' 2023-10-06T04:03:38.595 INFO:tasks.workunit.client.0.smithi157.stdout:removed './264/29' 2023-10-06T04:03:38.595 INFO:tasks.workunit.client.0.smithi157.stdout:removed './264/32' 2023-10-06T04:03:38.595 INFO:tasks.workunit.client.0.smithi157.stdout:removed './264/22' 2023-10-06T04:03:38.595 INFO:tasks.workunit.client.0.smithi157.stdout:removed './264/25' 2023-10-06T04:03:38.595 INFO:tasks.workunit.client.0.smithi157.stdout:removed './264/30' 2023-10-06T04:03:38.596 INFO:tasks.workunit.client.0.smithi157.stdout:removed './264/33' 2023-10-06T04:03:38.596 INFO:tasks.workunit.client.0.smithi157.stdout:removed './264/23' 2023-10-06T04:03:38.596 INFO:tasks.workunit.client.0.smithi157.stdout:removed './264/35' 2023-10-06T04:03:38.596 INFO:tasks.workunit.client.0.smithi157.stdout:removed './264/26' 2023-10-06T04:03:38.596 INFO:tasks.workunit.client.0.smithi157.stdout:removed './264/37' 2023-10-06T04:03:38.596 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './264' 2023-10-06T04:03:38.596 INFO:tasks.workunit.client.0.smithi157.stdout:removed './265/21' 2023-10-06T04:03:38.596 INFO:tasks.workunit.client.0.smithi157.stdout:removed './265/27' 2023-10-06T04:03:38.596 INFO:tasks.workunit.client.0.smithi157.stdout:removed './265/31' 2023-10-06T04:03:38.596 INFO:tasks.workunit.client.0.smithi157.stdout:removed './265/36' 2023-10-06T04:03:38.596 INFO:tasks.workunit.client.0.smithi157.stdout:removed './265/20' 2023-10-06T04:03:38.596 INFO:tasks.workunit.client.0.smithi157.stdout:removed './265/39' 2023-10-06T04:03:38.596 INFO:tasks.workunit.client.0.smithi157.stdout:removed './265/34' 2023-10-06T04:03:38.597 INFO:tasks.workunit.client.0.smithi157.stdout:removed './265/38' 2023-10-06T04:03:38.597 INFO:tasks.workunit.client.0.smithi157.stdout:removed './265/28' 2023-10-06T04:03:38.597 INFO:tasks.workunit.client.0.smithi157.stdout:removed './265/24' 2023-10-06T04:03:38.597 INFO:tasks.workunit.client.0.smithi157.stdout:removed './265/29' 2023-10-06T04:03:38.597 INFO:tasks.workunit.client.0.smithi157.stdout:removed './265/32' 2023-10-06T04:03:38.597 INFO:tasks.workunit.client.0.smithi157.stdout:removed './265/22' 2023-10-06T04:03:38.597 INFO:tasks.workunit.client.0.smithi157.stdout:removed './265/25' 2023-10-06T04:03:38.597 INFO:tasks.workunit.client.0.smithi157.stdout:removed './265/30' 2023-10-06T04:03:38.597 INFO:tasks.workunit.client.0.smithi157.stdout:removed './265/33' 2023-10-06T04:03:38.597 INFO:tasks.workunit.client.0.smithi157.stdout:removed './265/23' 2023-10-06T04:03:38.597 INFO:tasks.workunit.client.0.smithi157.stdout:removed './265/35' 2023-10-06T04:03:38.598 INFO:tasks.workunit.client.0.smithi157.stdout:removed './265/26' 2023-10-06T04:03:38.598 INFO:tasks.workunit.client.0.smithi157.stdout:removed './265/37' 2023-10-06T04:03:38.598 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './265' 2023-10-06T04:03:38.598 INFO:tasks.workunit.client.0.smithi157.stdout:removed './266/21' 2023-10-06T04:03:38.598 INFO:tasks.workunit.client.0.smithi157.stdout:removed './266/27' 2023-10-06T04:03:38.598 INFO:tasks.workunit.client.0.smithi157.stdout:removed './266/31' 2023-10-06T04:03:38.598 INFO:tasks.workunit.client.0.smithi157.stdout:removed './266/36' 2023-10-06T04:03:38.598 INFO:tasks.workunit.client.0.smithi157.stdout:removed './266/20' 2023-10-06T04:03:38.598 INFO:tasks.workunit.client.0.smithi157.stdout:removed './266/39' 2023-10-06T04:03:38.598 INFO:tasks.workunit.client.0.smithi157.stdout:removed './266/34' 2023-10-06T04:03:38.598 INFO:tasks.workunit.client.0.smithi157.stdout:removed './266/38' 2023-10-06T04:03:38.598 INFO:tasks.workunit.client.0.smithi157.stdout:removed './266/28' 2023-10-06T04:03:38.599 INFO:tasks.workunit.client.0.smithi157.stdout:removed './266/24' 2023-10-06T04:03:38.599 INFO:tasks.workunit.client.0.smithi157.stdout:removed './266/29' 2023-10-06T04:03:38.599 INFO:tasks.workunit.client.0.smithi157.stdout:removed './266/32' 2023-10-06T04:03:38.599 INFO:tasks.workunit.client.0.smithi157.stdout:removed './266/22' 2023-10-06T04:03:38.599 INFO:tasks.workunit.client.0.smithi157.stdout:removed './266/25' 2023-10-06T04:03:38.599 INFO:tasks.workunit.client.0.smithi157.stdout:removed './266/30' 2023-10-06T04:03:38.599 INFO:tasks.workunit.client.0.smithi157.stdout:removed './266/33' 2023-10-06T04:03:38.599 INFO:tasks.workunit.client.0.smithi157.stdout:removed './266/23' 2023-10-06T04:03:38.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.670+0000 7f32cef62700 1 -- 172.21.15.157:0/1157471847 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32c8166760 con 0x7f32c80bc750 2023-10-06T04:03:38.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.670+0000 7f32bcff9700 1 -- 172.21.15.157:0/1157471847 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+6145 (secure 0 0 0) 0x7f32c8166760 con 0x7f32c80bc750 2023-10-06T04:03:38.672 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:03:38.672 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3192541484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3192541484},{"type":"v1","addr":"172.21.15.175:6835","nonce":3192541484}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:33.754360+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":4287},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:03:38.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.673+0000 7f32cef62700 1 -- 172.21.15.157:0/1157471847 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f32b4066f30 msgr2=0x7f32b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:38.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.673+0000 7f32cef62700 1 --2- 172.21.15.157:0/1157471847 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f32b4066f30 0x7f32b40693e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f32c4006fd0 tx=0x7f32c4008040 comp rx=0 tx=0).stop 2023-10-06T04:03:38.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.673+0000 7f32cef62700 1 -- 172.21.15.157:0/1157471847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c80bc750 msgr2=0x7f32c80c4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:38.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.673+0000 7f32cef62700 1 --2- 172.21.15.157:0/1157471847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c80bc750 0x7f32c80c4c40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f32c0007f20 tx=0x7f32c0009860 comp rx=0 tx=0).stop 2023-10-06T04:03:38.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.673+0000 7f32cef62700 1 -- 172.21.15.157:0/1157471847 shutdown_connections 2023-10-06T04:03:38.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.673+0000 7f32cef62700 1 --2- 172.21.15.157:0/1157471847 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f32b4066f30 0x7f32b40693e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:38.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.673+0000 7f32cef62700 1 --2- 172.21.15.157:0/1157471847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32c80bc750 0x7f32c80c4c40 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:38.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.673+0000 7f32cef62700 1 --2- 172.21.15.157:0/1157471847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32c806a780 0x7f32c80c3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:38.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.673+0000 7f32cef62700 1 --2- 172.21.15.157:0/1157471847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32c806ab50 0x7f32c80c0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:38.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.673+0000 7f32cef62700 1 -- 172.21.15.157:0/1157471847 >> 172.21.15.157:0/1157471847 conn(0x7f32c806b510 msgr2=0x7f32c8070940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:38.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.673+0000 7f32cef62700 1 -- 172.21.15.157:0/1157471847 shutdown_connections 2023-10-06T04:03:38.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:38.673+0000 7f32cef62700 1 -- 172.21.15.157:0/1157471847 wait complete. 2023-10-06T04:03:38.676 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 53 2023-10-06T04:03:38.686 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-10-06T04:03:40.559 INFO:tasks.workunit.client.0.smithi157.stdout:removed './266/35' 2023-10-06T04:03:40.559 INFO:tasks.workunit.client.0.smithi157.stdout:removed './266/26' 2023-10-06T04:03:40.559 INFO:tasks.workunit.client.0.smithi157.stdout:removed './266/37' 2023-10-06T04:03:40.559 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './266' 2023-10-06T04:03:40.559 INFO:tasks.workunit.client.0.smithi157.stdout:removed './267/21' 2023-10-06T04:03:40.559 INFO:tasks.workunit.client.0.smithi157.stdout:removed './267/27' 2023-10-06T04:03:40.560 INFO:tasks.workunit.client.0.smithi157.stdout:removed './267/31' 2023-10-06T04:03:40.560 INFO:tasks.workunit.client.0.smithi157.stdout:removed './267/36' 2023-10-06T04:03:40.560 INFO:tasks.workunit.client.0.smithi157.stdout:removed './267/20' 2023-10-06T04:03:40.560 INFO:tasks.workunit.client.0.smithi157.stdout:removed './267/39' 2023-10-06T04:03:40.560 INFO:tasks.workunit.client.0.smithi157.stdout:removed './267/34' 2023-10-06T04:03:40.560 INFO:tasks.workunit.client.0.smithi157.stdout:removed './267/38' 2023-10-06T04:03:40.560 INFO:tasks.workunit.client.0.smithi157.stdout:removed './267/28' 2023-10-06T04:03:40.560 INFO:tasks.workunit.client.0.smithi157.stdout:removed './267/24' 2023-10-06T04:03:40.560 INFO:tasks.workunit.client.0.smithi157.stdout:removed './267/29' 2023-10-06T04:03:40.560 INFO:tasks.workunit.client.0.smithi157.stdout:removed './267/32' 2023-10-06T04:03:40.561 INFO:tasks.workunit.client.0.smithi157.stdout:removed './267/22' 2023-10-06T04:03:40.561 INFO:tasks.workunit.client.0.smithi157.stdout:removed './267/25' 2023-10-06T04:03:40.561 INFO:tasks.workunit.client.0.smithi157.stdout:removed './267/30' 2023-10-06T04:03:40.561 INFO:tasks.workunit.client.0.smithi157.stdout:removed './267/33' 2023-10-06T04:03:40.561 INFO:tasks.workunit.client.0.smithi157.stdout:removed './267/23' 2023-10-06T04:03:40.561 INFO:tasks.workunit.client.0.smithi157.stdout:removed './267/35' 2023-10-06T04:03:40.561 INFO:tasks.workunit.client.0.smithi157.stdout:removed './267/26' 2023-10-06T04:03:40.561 INFO:tasks.workunit.client.0.smithi157.stdout:removed './267/37' 2023-10-06T04:03:40.561 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './267' 2023-10-06T04:03:40.561 INFO:tasks.workunit.client.0.smithi157.stdout:removed './268/21' 2023-10-06T04:03:40.562 INFO:tasks.workunit.client.0.smithi157.stdout:removed './268/27' 2023-10-06T04:03:40.562 INFO:tasks.workunit.client.0.smithi157.stdout:removed './268/31' 2023-10-06T04:03:40.562 INFO:tasks.workunit.client.0.smithi157.stdout:removed './268/36' 2023-10-06T04:03:40.562 INFO:tasks.workunit.client.0.smithi157.stdout:removed './268/20' 2023-10-06T04:03:40.562 INFO:tasks.workunit.client.0.smithi157.stdout:removed './268/39' 2023-10-06T04:03:40.562 INFO:tasks.workunit.client.0.smithi157.stdout:removed './268/34' 2023-10-06T04:03:40.562 INFO:tasks.workunit.client.0.smithi157.stdout:removed './268/38' 2023-10-06T04:03:40.562 INFO:tasks.workunit.client.0.smithi157.stdout:removed './268/28' 2023-10-06T04:03:40.562 INFO:tasks.workunit.client.0.smithi157.stdout:removed './268/24' 2023-10-06T04:03:40.562 INFO:tasks.workunit.client.0.smithi157.stdout:removed './268/29' 2023-10-06T04:03:40.563 INFO:tasks.workunit.client.0.smithi157.stdout:removed './268/32' 2023-10-06T04:03:40.563 INFO:tasks.workunit.client.0.smithi157.stdout:removed './268/22' 2023-10-06T04:03:40.563 INFO:tasks.workunit.client.0.smithi157.stdout:removed './268/25' 2023-10-06T04:03:40.563 INFO:tasks.workunit.client.0.smithi157.stdout:removed './268/30' 2023-10-06T04:03:40.563 INFO:tasks.workunit.client.0.smithi157.stdout:removed './268/33' 2023-10-06T04:03:40.563 INFO:tasks.workunit.client.0.smithi157.stdout:removed './268/23' 2023-10-06T04:03:40.563 INFO:tasks.workunit.client.0.smithi157.stdout:removed './268/35' 2023-10-06T04:03:40.563 INFO:tasks.workunit.client.0.smithi157.stdout:removed './268/26' 2023-10-06T04:03:40.563 INFO:tasks.workunit.client.0.smithi157.stdout:removed './268/37' 2023-10-06T04:03:40.563 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './268' 2023-10-06T04:03:40.564 INFO:tasks.workunit.client.0.smithi157.stdout:removed './269/21' 2023-10-06T04:03:40.564 INFO:tasks.workunit.client.0.smithi157.stdout:removed './269/27' 2023-10-06T04:03:40.564 INFO:tasks.workunit.client.0.smithi157.stdout:removed './269/31' 2023-10-06T04:03:40.564 INFO:tasks.workunit.client.0.smithi157.stdout:removed './269/36' 2023-10-06T04:03:40.564 INFO:tasks.workunit.client.0.smithi157.stdout:removed './269/20' 2023-10-06T04:03:40.564 INFO:tasks.workunit.client.0.smithi157.stdout:removed './269/39' 2023-10-06T04:03:40.564 INFO:tasks.workunit.client.0.smithi157.stdout:removed './269/34' 2023-10-06T04:03:40.564 INFO:tasks.workunit.client.0.smithi157.stdout:removed './269/38' 2023-10-06T04:03:40.564 INFO:tasks.workunit.client.0.smithi157.stdout:removed './269/28' 2023-10-06T04:03:40.564 INFO:tasks.workunit.client.0.smithi157.stdout:removed './269/24' 2023-10-06T04:03:40.565 INFO:tasks.workunit.client.0.smithi157.stdout:removed './269/29' 2023-10-06T04:03:40.565 INFO:tasks.workunit.client.0.smithi157.stdout:removed './269/32' 2023-10-06T04:03:40.565 INFO:tasks.workunit.client.0.smithi157.stdout:removed './269/22' 2023-10-06T04:03:40.565 INFO:tasks.workunit.client.0.smithi157.stdout:removed './269/25' 2023-10-06T04:03:40.565 INFO:tasks.workunit.client.0.smithi157.stdout:removed './269/30' 2023-10-06T04:03:40.565 INFO:tasks.workunit.client.0.smithi157.stdout:removed './269/33' 2023-10-06T04:03:40.565 INFO:tasks.workunit.client.0.smithi157.stdout:removed './269/23' 2023-10-06T04:03:40.565 INFO:tasks.workunit.client.0.smithi157.stdout:removed './269/35' 2023-10-06T04:03:40.565 INFO:tasks.workunit.client.0.smithi157.stdout:removed './269/26' 2023-10-06T04:03:40.565 INFO:tasks.workunit.client.0.smithi157.stdout:removed './269/37' 2023-10-06T04:03:40.566 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './269' 2023-10-06T04:03:40.566 INFO:tasks.workunit.client.0.smithi157.stdout:removed './270/21' 2023-10-06T04:03:40.566 INFO:tasks.workunit.client.0.smithi157.stdout:removed './270/27' 2023-10-06T04:03:40.566 INFO:tasks.workunit.client.0.smithi157.stdout:removed './270/31' 2023-10-06T04:03:40.566 INFO:tasks.workunit.client.0.smithi157.stdout:removed './270/36' 2023-10-06T04:03:40.566 INFO:tasks.workunit.client.0.smithi157.stdout:removed './270/20' 2023-10-06T04:03:40.566 INFO:tasks.workunit.client.0.smithi157.stdout:removed './270/39' 2023-10-06T04:03:40.566 INFO:tasks.workunit.client.0.smithi157.stdout:removed './270/34' 2023-10-06T04:03:40.566 INFO:tasks.workunit.client.0.smithi157.stdout:removed './270/38' 2023-10-06T04:03:40.566 INFO:tasks.workunit.client.0.smithi157.stdout:removed './270/28' 2023-10-06T04:03:40.567 INFO:tasks.workunit.client.0.smithi157.stdout:removed './270/24' 2023-10-06T04:03:40.567 INFO:tasks.workunit.client.0.smithi157.stdout:removed './270/29' 2023-10-06T04:03:40.567 INFO:tasks.workunit.client.0.smithi157.stdout:removed './270/32' 2023-10-06T04:03:40.567 INFO:tasks.workunit.client.0.smithi157.stdout:removed './270/22' 2023-10-06T04:03:40.567 INFO:tasks.workunit.client.0.smithi157.stdout:removed './270/25' 2023-10-06T04:03:40.567 INFO:tasks.workunit.client.0.smithi157.stdout:removed './270/30' 2023-10-06T04:03:40.567 INFO:tasks.workunit.client.0.smithi157.stdout:removed './270/33' 2023-10-06T04:03:40.567 INFO:tasks.workunit.client.0.smithi157.stdout:removed './270/23' 2023-10-06T04:03:40.567 INFO:tasks.workunit.client.0.smithi157.stdout:removed './270/35' 2023-10-06T04:03:40.567 INFO:tasks.workunit.client.0.smithi157.stdout:removed './270/26' 2023-10-06T04:03:40.568 INFO:tasks.workunit.client.0.smithi157.stdout:removed './270/37' 2023-10-06T04:03:40.568 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './270' 2023-10-06T04:03:40.568 INFO:tasks.workunit.client.0.smithi157.stdout:removed './271/21' 2023-10-06T04:03:40.568 INFO:tasks.workunit.client.0.smithi157.stdout:removed './271/27' 2023-10-06T04:03:40.568 INFO:tasks.workunit.client.0.smithi157.stdout:removed './271/31' 2023-10-06T04:03:40.568 INFO:tasks.workunit.client.0.smithi157.stdout:removed './271/36' 2023-10-06T04:03:40.568 INFO:tasks.workunit.client.0.smithi157.stdout:removed './271/20' 2023-10-06T04:03:40.568 INFO:tasks.workunit.client.0.smithi157.stdout:removed './271/39' 2023-10-06T04:03:40.568 INFO:tasks.workunit.client.0.smithi157.stdout:removed './271/34' 2023-10-06T04:03:40.568 INFO:tasks.workunit.client.0.smithi157.stdout:removed './271/38' 2023-10-06T04:03:40.569 INFO:tasks.workunit.client.0.smithi157.stdout:removed './271/28' 2023-10-06T04:03:40.569 INFO:tasks.workunit.client.0.smithi157.stdout:removed './271/24' 2023-10-06T04:03:40.569 INFO:tasks.workunit.client.0.smithi157.stdout:removed './271/29' 2023-10-06T04:03:40.569 INFO:tasks.workunit.client.0.smithi157.stdout:removed './271/32' 2023-10-06T04:03:40.569 INFO:tasks.workunit.client.0.smithi157.stdout:removed './271/22' 2023-10-06T04:03:40.569 INFO:tasks.workunit.client.0.smithi157.stdout:removed './271/25' 2023-10-06T04:03:40.569 INFO:tasks.workunit.client.0.smithi157.stdout:removed './271/30' 2023-10-06T04:03:40.569 INFO:tasks.workunit.client.0.smithi157.stdout:removed './271/33' 2023-10-06T04:03:40.569 INFO:tasks.workunit.client.0.smithi157.stdout:removed './271/23' 2023-10-06T04:03:40.569 INFO:tasks.workunit.client.0.smithi157.stdout:removed './271/35' 2023-10-06T04:03:40.570 INFO:tasks.workunit.client.0.smithi157.stdout:removed './271/26' 2023-10-06T04:03:40.570 INFO:tasks.workunit.client.0.smithi157.stdout:removed './271/37' 2023-10-06T04:03:40.570 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './271' 2023-10-06T04:03:40.570 INFO:tasks.workunit.client.0.smithi157.stdout:removed './272/21' 2023-10-06T04:03:40.570 INFO:tasks.workunit.client.0.smithi157.stdout:removed './272/27' 2023-10-06T04:03:40.570 INFO:tasks.workunit.client.0.smithi157.stdout:removed './272/31' 2023-10-06T04:03:40.570 INFO:tasks.workunit.client.0.smithi157.stdout:removed './272/36' 2023-10-06T04:03:40.570 INFO:tasks.workunit.client.0.smithi157.stdout:removed './272/20' 2023-10-06T04:03:40.570 INFO:tasks.workunit.client.0.smithi157.stdout:removed './272/39' 2023-10-06T04:03:40.571 INFO:tasks.workunit.client.0.smithi157.stdout:removed './272/34' 2023-10-06T04:03:40.571 INFO:tasks.workunit.client.0.smithi157.stdout:removed './272/38' 2023-10-06T04:03:40.571 INFO:tasks.workunit.client.0.smithi157.stdout:removed './272/28' 2023-10-06T04:03:40.571 INFO:tasks.workunit.client.0.smithi157.stdout:removed './272/24' 2023-10-06T04:03:40.571 INFO:tasks.workunit.client.0.smithi157.stdout:removed './272/29' 2023-10-06T04:03:40.571 INFO:tasks.workunit.client.0.smithi157.stdout:removed './272/32' 2023-10-06T04:03:40.571 INFO:tasks.workunit.client.0.smithi157.stdout:removed './272/22' 2023-10-06T04:03:40.571 INFO:tasks.workunit.client.0.smithi157.stdout:removed './272/25' 2023-10-06T04:03:40.571 INFO:tasks.workunit.client.0.smithi157.stdout:removed './272/30' 2023-10-06T04:03:40.571 INFO:tasks.workunit.client.0.smithi157.stdout:removed './272/33' 2023-10-06T04:03:40.571 INFO:tasks.workunit.client.0.smithi157.stdout:removed './272/23' 2023-10-06T04:03:40.572 INFO:tasks.workunit.client.0.smithi157.stdout:removed './272/35' 2023-10-06T04:03:40.572 INFO:tasks.workunit.client.0.smithi157.stdout:removed './272/26' 2023-10-06T04:03:40.572 INFO:tasks.workunit.client.0.smithi157.stdout:removed './272/37' 2023-10-06T04:03:40.572 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './272' 2023-10-06T04:03:40.572 INFO:tasks.workunit.client.0.smithi157.stdout:removed './273/21' 2023-10-06T04:03:40.572 INFO:tasks.workunit.client.0.smithi157.stdout:removed './273/27' 2023-10-06T04:03:40.572 INFO:tasks.workunit.client.0.smithi157.stdout:removed './273/31' 2023-10-06T04:03:40.573 INFO:tasks.workunit.client.0.smithi157.stdout:removed './273/36' 2023-10-06T04:03:40.573 INFO:tasks.workunit.client.0.smithi157.stdout:removed './273/20' 2023-10-06T04:03:40.573 INFO:tasks.workunit.client.0.smithi157.stdout:removed './273/39' 2023-10-06T04:03:40.573 INFO:tasks.workunit.client.0.smithi157.stdout:removed './273/34' 2023-10-06T04:03:40.573 INFO:tasks.workunit.client.0.smithi157.stdout:removed './273/38' 2023-10-06T04:03:40.573 INFO:tasks.workunit.client.0.smithi157.stdout:removed './273/28' 2023-10-06T04:03:40.573 INFO:tasks.workunit.client.0.smithi157.stdout:removed './273/24' 2023-10-06T04:03:40.573 INFO:tasks.workunit.client.0.smithi157.stdout:removed './273/29' 2023-10-06T04:03:40.573 INFO:tasks.workunit.client.0.smithi157.stdout:removed './273/32' 2023-10-06T04:03:40.573 INFO:tasks.workunit.client.0.smithi157.stdout:removed './273/22' 2023-10-06T04:03:40.574 INFO:tasks.workunit.client.0.smithi157.stdout:removed './273/25' 2023-10-06T04:03:40.574 INFO:tasks.workunit.client.0.smithi157.stdout:removed './273/30' 2023-10-06T04:03:40.574 INFO:tasks.workunit.client.0.smithi157.stdout:removed './273/33' 2023-10-06T04:03:40.574 INFO:tasks.workunit.client.0.smithi157.stdout:removed './273/23' 2023-10-06T04:03:40.574 INFO:tasks.workunit.client.0.smithi157.stdout:removed './273/35' 2023-10-06T04:03:40.574 INFO:tasks.workunit.client.0.smithi157.stdout:removed './273/26' 2023-10-06T04:03:40.574 INFO:tasks.workunit.client.0.smithi157.stdout:removed './273/37' 2023-10-06T04:03:40.574 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './273' 2023-10-06T04:03:40.574 INFO:tasks.workunit.client.0.smithi157.stdout:removed './274/21' 2023-10-06T04:03:40.575 INFO:tasks.workunit.client.0.smithi157.stdout:removed './274/27' 2023-10-06T04:03:40.575 INFO:tasks.workunit.client.0.smithi157.stdout:removed './274/31' 2023-10-06T04:03:40.575 INFO:tasks.workunit.client.0.smithi157.stdout:removed './274/36' 2023-10-06T04:03:40.575 INFO:tasks.workunit.client.0.smithi157.stdout:removed './274/20' 2023-10-06T04:03:40.575 INFO:tasks.workunit.client.0.smithi157.stdout:removed './274/39' 2023-10-06T04:03:40.575 INFO:tasks.workunit.client.0.smithi157.stdout:removed './274/34' 2023-10-06T04:03:40.575 INFO:tasks.workunit.client.0.smithi157.stdout:removed './274/38' 2023-10-06T04:03:40.575 INFO:tasks.workunit.client.0.smithi157.stdout:removed './274/28' 2023-10-06T04:03:40.575 INFO:tasks.workunit.client.0.smithi157.stdout:removed './274/24' 2023-10-06T04:03:40.575 INFO:tasks.workunit.client.0.smithi157.stdout:removed './274/29' 2023-10-06T04:03:40.575 INFO:tasks.workunit.client.0.smithi157.stdout:removed './274/32' 2023-10-06T04:03:40.576 INFO:tasks.workunit.client.0.smithi157.stdout:removed './274/22' 2023-10-06T04:03:40.576 INFO:tasks.workunit.client.0.smithi157.stdout:removed './274/25' 2023-10-06T04:03:40.576 INFO:tasks.workunit.client.0.smithi157.stdout:removed './274/30' 2023-10-06T04:03:40.576 INFO:tasks.workunit.client.0.smithi157.stdout:removed './274/33' 2023-10-06T04:03:40.576 INFO:tasks.workunit.client.0.smithi157.stdout:removed './274/23' 2023-10-06T04:03:40.576 INFO:tasks.workunit.client.0.smithi157.stdout:removed './274/35' 2023-10-06T04:03:40.576 INFO:tasks.workunit.client.0.smithi157.stdout:removed './274/26' 2023-10-06T04:03:40.576 INFO:tasks.workunit.client.0.smithi157.stdout:removed './274/37' 2023-10-06T04:03:40.576 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './274' 2023-10-06T04:03:40.576 INFO:tasks.workunit.client.0.smithi157.stdout:removed './275/21' 2023-10-06T04:03:40.577 INFO:tasks.workunit.client.0.smithi157.stdout:removed './275/27' 2023-10-06T04:03:40.577 INFO:tasks.workunit.client.0.smithi157.stdout:removed './275/31' 2023-10-06T04:03:40.577 INFO:tasks.workunit.client.0.smithi157.stdout:removed './275/36' 2023-10-06T04:03:40.577 INFO:tasks.workunit.client.0.smithi157.stdout:removed './275/20' 2023-10-06T04:03:40.577 INFO:tasks.workunit.client.0.smithi157.stdout:removed './275/39' 2023-10-06T04:03:40.577 INFO:tasks.workunit.client.0.smithi157.stdout:removed './275/34' 2023-10-06T04:03:40.577 INFO:tasks.workunit.client.0.smithi157.stdout:removed './275/38' 2023-10-06T04:03:40.577 INFO:tasks.workunit.client.0.smithi157.stdout:removed './275/28' 2023-10-06T04:03:40.577 INFO:tasks.workunit.client.0.smithi157.stdout:removed './275/24' 2023-10-06T04:03:40.577 INFO:tasks.workunit.client.0.smithi157.stdout:removed './275/29' 2023-10-06T04:03:40.578 INFO:tasks.workunit.client.0.smithi157.stdout:removed './275/32' 2023-10-06T04:03:40.578 INFO:tasks.workunit.client.0.smithi157.stdout:removed './275/22' 2023-10-06T04:03:40.578 INFO:tasks.workunit.client.0.smithi157.stdout:removed './275/25' 2023-10-06T04:03:40.578 INFO:tasks.workunit.client.0.smithi157.stdout:removed './275/30' 2023-10-06T04:03:40.578 INFO:tasks.workunit.client.0.smithi157.stdout:removed './275/33' 2023-10-06T04:03:40.578 INFO:tasks.workunit.client.0.smithi157.stdout:removed './275/23' 2023-10-06T04:03:40.578 INFO:tasks.workunit.client.0.smithi157.stdout:removed './275/35' 2023-10-06T04:03:40.578 INFO:tasks.workunit.client.0.smithi157.stdout:removed './275/26' 2023-10-06T04:03:40.578 INFO:tasks.workunit.client.0.smithi157.stdout:removed './275/37' 2023-10-06T04:03:40.578 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './275' 2023-10-06T04:03:40.579 INFO:tasks.workunit.client.0.smithi157.stdout:removed './276/21' 2023-10-06T04:03:40.579 INFO:tasks.workunit.client.0.smithi157.stdout:removed './276/27' 2023-10-06T04:03:40.579 INFO:tasks.workunit.client.0.smithi157.stdout:removed './276/31' 2023-10-06T04:03:40.579 INFO:tasks.workunit.client.0.smithi157.stdout:removed './276/36' 2023-10-06T04:03:40.579 INFO:tasks.workunit.client.0.smithi157.stdout:removed './276/20' 2023-10-06T04:03:40.579 INFO:tasks.workunit.client.0.smithi157.stdout:removed './276/39' 2023-10-06T04:03:40.579 INFO:tasks.workunit.client.0.smithi157.stdout:removed './276/34' 2023-10-06T04:03:40.579 INFO:tasks.workunit.client.0.smithi157.stdout:removed './276/38' 2023-10-06T04:03:40.579 INFO:tasks.workunit.client.0.smithi157.stdout:removed './276/28' 2023-10-06T04:03:40.579 INFO:tasks.workunit.client.0.smithi157.stdout:removed './276/24' 2023-10-06T04:03:40.580 INFO:tasks.workunit.client.0.smithi157.stdout:removed './276/29' 2023-10-06T04:03:40.580 INFO:tasks.workunit.client.0.smithi157.stdout:removed './276/32' 2023-10-06T04:03:40.580 INFO:tasks.workunit.client.0.smithi157.stdout:removed './276/22' 2023-10-06T04:03:40.580 INFO:tasks.workunit.client.0.smithi157.stdout:removed './276/25' 2023-10-06T04:03:40.580 INFO:tasks.workunit.client.0.smithi157.stdout:removed './276/30' 2023-10-06T04:03:40.580 INFO:tasks.workunit.client.0.smithi157.stdout:removed './276/33' 2023-10-06T04:03:40.580 INFO:tasks.workunit.client.0.smithi157.stdout:removed './276/23' 2023-10-06T04:03:40.580 INFO:tasks.workunit.client.0.smithi157.stdout:removed './276/35' 2023-10-06T04:03:40.580 INFO:tasks.workunit.client.0.smithi157.stdout:removed './276/26' 2023-10-06T04:03:40.687 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:03:40.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.801+0000 7f3396941700 1 Processor -- start 2023-10-06T04:03:40.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.801+0000 7f3396941700 1 -- start start 2023-10-06T04:03:40.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.802+0000 7f3396941700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f339006c440 0x7f3390069130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:40.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.802+0000 7f3396941700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f339006c810 0x7f33901569e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:40.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.802+0000 7f3396941700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3390155f20 con 0x7f33900c5fc0 2023-10-06T04:03:40.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.802+0000 7f3396941700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f339006f310 con 0x7f339006c440 2023-10-06T04:03:40.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.802+0000 7f3396941700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f339006d4a0 con 0x7f339006c810 2023-10-06T04:03:40.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.802+0000 7f338ffff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f339006c440 0x7f3390069130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:40.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.802+0000 7f338f7fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f33900c5fc0 0x7f3390152e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:48368/0 (socket says 172.21.15.157:48368) 2023-10-06T04:03:40.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.802+0000 7f338f7fe700 1 -- 172.21.15.157:0/363387819 learned_addr learned my addr 172.21.15.157:0/363387819 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:03:40.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.803+0000 7f3394ede700 1 --2- 172.21.15.157:0/363387819 >> v2:172.21.15.175:3300/0 conn(0x7f339006c810 0x7f33901569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:40.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.803+0000 7f338ffff700 1 -- 172.21.15.157:0/363387819 >> v2:172.21.15.175:3300/0 conn(0x7f339006c810 msgr2=0x7f33901569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:40.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.803+0000 7f338ffff700 1 --2- 172.21.15.157:0/363387819 >> v2:172.21.15.175:3300/0 conn(0x7f339006c810 0x7f33901569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:40.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.803+0000 7f338ffff700 1 -- 172.21.15.157:0/363387819 >> v1:172.21.15.157:6789/0 conn(0x7f33900c5fc0 legacy=0x7f3390152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:40.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.803+0000 7f338effd700 1 -- 172.21.15.157:0/363387819 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2189741103 0 0) 0x7f3390155f20 con 0x7f33900c5fc0 2023-10-06T04:03:40.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.803+0000 7f338ffff700 1 -- 172.21.15.157:0/363387819 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f339006d780 con 0x7f339006c440 2023-10-06T04:03:40.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.803+0000 7f3394ede700 1 --2- 172.21.15.157:0/363387819 >> v2:172.21.15.175:3300/0 conn(0x7f339006c810 0x7f33901569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:03:40.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.803+0000 7f338ffff700 1 --2- 172.21.15.157:0/363387819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f339006c440 0x7f3390069130 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f339006e230 tx=0x7f33800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746d1fbd7ecae50 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:40.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.803+0000 7f338effd700 1 -- 172.21.15.157:0/363387819 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338000b6b0 con 0x7f339006c440 2023-10-06T04:03:40.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.803+0000 7f338effd700 1 -- 172.21.15.157:0/363387819 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338000b810 con 0x7f339006c440 2023-10-06T04:03:40.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.803+0000 7f338effd700 1 -- 172.21.15.157:0/363387819 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338000bbb0 con 0x7f339006c440 2023-10-06T04:03:40.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.804+0000 7f3396941700 1 -- 172.21.15.157:0/363387819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f339006c440 msgr2=0x7f3390069130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:40.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.804+0000 7f3396941700 1 --2- 172.21.15.157:0/363387819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f339006c440 0x7f3390069130 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f339006e230 tx=0x7f33800091a0 comp rx=0 tx=0).stop 2023-10-06T04:03:40.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.804+0000 7f3396941700 1 -- 172.21.15.157:0/363387819 shutdown_connections 2023-10-06T04:03:40.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.804+0000 7f3396941700 1 --2- 172.21.15.157:0/363387819 >> v2:172.21.15.175:3300/0 conn(0x7f339006c810 0x7f33901569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:40.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.804+0000 7f3396941700 1 --2- 172.21.15.157:0/363387819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f339006c440 0x7f3390069130 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:40.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.804+0000 7f3396941700 1 -- 172.21.15.157:0/363387819 >> 172.21.15.157:0/363387819 conn(0x7f3390068370 msgr2=0x7f3390068770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:40.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.804+0000 7f3396941700 1 -- 172.21.15.157:0/363387819 shutdown_connections 2023-10-06T04:03:40.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.804+0000 7f3396941700 1 -- 172.21.15.157:0/363387819 wait complete. 2023-10-06T04:03:40.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.805+0000 7f3396941700 1 Processor -- start 2023-10-06T04:03:40.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.805+0000 7f3396941700 1 -- start start 2023-10-06T04:03:40.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.806+0000 7f3396941700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f339006c440 0x7f339014f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:40.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.806+0000 7f3396941700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f339006c810 0x7f33901670f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:40.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.806+0000 7f3396941700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33900c5fc0 0x7f3390167630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:40.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.806+0000 7f3396941700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339014ff10 con 0x7f339006c440 2023-10-06T04:03:40.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.806+0000 7f3396941700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33900afd90 con 0x7f339006c810 2023-10-06T04:03:40.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.806+0000 7f3396941700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f339015c640 con 0x7f33900c5fc0 2023-10-06T04:03:40.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.806+0000 7f338ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f339006c440 0x7f339014f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:40.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.806+0000 7f338ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f339006c440 0x7f339014f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:48518/0 (socket says 172.21.15.157:48518) 2023-10-06T04:03:40.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.806+0000 7f338ffff700 1 -- 172.21.15.157:0/2698952127 learned_addr learned my addr 172.21.15.157:0/2698952127 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:40.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.806+0000 7f338ffff700 1 -- 172.21.15.157:0/2698952127 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33900c5fc0 msgr2=0x7f3390167630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:40.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.806+0000 7f338ffff700 1 --2- 172.21.15.157:0/2698952127 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33900c5fc0 0x7f3390167630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:40.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.806+0000 7f338ffff700 1 -- 172.21.15.157:0/2698952127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f339006c810 msgr2=0x7f33901670f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:40.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.806+0000 7f338ffff700 1 --2- 172.21.15.157:0/2698952127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f339006c810 0x7f33901670f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:40.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.806+0000 7f338ffff700 1 -- 172.21.15.157:0/2698952127 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3380009420 con 0x7f339006c440 2023-10-06T04:03:40.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.806+0000 7f338ffff700 1 --2- 172.21.15.157:0/2698952127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f339006c440 0x7f339014f9d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f33800093f0 tx=0x7f3380017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:40.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.806+0000 7f3377fff700 1 -- 172.21.15.157:0/2698952127 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338000bb90 con 0x7f339006c440 2023-10-06T04:03:40.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.806+0000 7f3377fff700 1 -- 172.21.15.157:0/2698952127 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3380011420 con 0x7f339006c440 2023-10-06T04:03:40.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.806+0000 7f3396941700 1 -- 172.21.15.157:0/2698952127 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33900af460 con 0x7f339006c440 2023-10-06T04:03:40.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.806+0000 7f3377fff700 1 -- 172.21.15.157:0/2698952127 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33800117a0 con 0x7f339006c440 2023-10-06T04:03:40.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.806+0000 7f3396941700 1 -- 172.21.15.157:0/2698952127 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33900afa20 con 0x7f339006c440 2023-10-06T04:03:40.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.807+0000 7f3396941700 1 -- 172.21.15.157:0/2698952127 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f339006a8f0 con 0x7f339006c440 2023-10-06T04:03:40.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.808+0000 7f3377fff700 1 -- 172.21.15.157:0/2698952127 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3380011b50 con 0x7f339006c440 2023-10-06T04:03:40.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.808+0000 7f3377fff700 1 --2- 172.21.15.157:0/2698952127 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f337c066f30 0x7f337c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:40.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.808+0000 7f3377fff700 1 -- 172.21.15.157:0/2698952127 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7f33800861d0 con 0x7f339006c440 2023-10-06T04:03:40.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.808+0000 7f338f7fe700 1 --2- 172.21.15.157:0/2698952127 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f337c066f30 0x7f337c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:40.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.809+0000 7f338f7fe700 1 --2- 172.21.15.157:0/2698952127 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f337c066f30 0x7f337c0693e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f33840011e0 tx=0x7f33840079d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:40.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:40.812+0000 7f3377fff700 1 -- 172.21.15.157:0/2698952127 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3380053090 con 0x7f339006c440 2023-10-06T04:03:41.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:41.092+0000 7f3396941700 1 -- 172.21.15.157:0/2698952127 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33901561e0 con 0x7f339006c440 2023-10-06T04:03:41.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:41.092+0000 7f3377fff700 1 -- 172.21.15.157:0/2698952127 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+6145 (secure 0 0 0) 0x7f33800566b0 con 0x7f339006c440 2023-10-06T04:03:41.094 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:03:41.094 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3192541484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3192541484},{"type":"v1","addr":"172.21.15.175:6835","nonce":3192541484}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:33.754360+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":4287},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:03:41.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:41.094+0000 7f3396941700 1 -- 172.21.15.157:0/2698952127 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f337c066f30 msgr2=0x7f337c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:41.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:41.094+0000 7f3396941700 1 --2- 172.21.15.157:0/2698952127 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f337c066f30 0x7f337c0693e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f33840011e0 tx=0x7f33840079d0 comp rx=0 tx=0).stop 2023-10-06T04:03:41.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:41.094+0000 7f3396941700 1 -- 172.21.15.157:0/2698952127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f339006c440 msgr2=0x7f339014f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:41.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:41.094+0000 7f3396941700 1 --2- 172.21.15.157:0/2698952127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f339006c440 0x7f339014f9d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f33800093f0 tx=0x7f3380017000 comp rx=0 tx=0).stop 2023-10-06T04:03:41.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:41.094+0000 7f3396941700 1 -- 172.21.15.157:0/2698952127 shutdown_connections 2023-10-06T04:03:41.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:41.094+0000 7f3396941700 1 --2- 172.21.15.157:0/2698952127 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f337c066f30 0x7f337c0693e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:41.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:41.094+0000 7f3396941700 1 --2- 172.21.15.157:0/2698952127 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33900c5fc0 0x7f3390167630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:41.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:41.094+0000 7f3396941700 1 --2- 172.21.15.157:0/2698952127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f339006c440 0x7f339014f9d0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:41.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:41.094+0000 7f3396941700 1 --2- 172.21.15.157:0/2698952127 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f339006c810 0x7f33901670f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:41.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:41.094+0000 7f3396941700 1 -- 172.21.15.157:0/2698952127 >> 172.21.15.157:0/2698952127 conn(0x7f3390068370 msgr2=0x7f3390152070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:41.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:41.094+0000 7f3396941700 1 -- 172.21.15.157:0/2698952127 shutdown_connections 2023-10-06T04:03:41.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:41.094+0000 7f3396941700 1 -- 172.21.15.157:0/2698952127 wait complete. 2023-10-06T04:03:41.097 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 53 2023-10-06T04:03:41.112 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-10-06T04:03:41.279 INFO:tasks.workunit.client.0.smithi157.stdout:removed './276/37' 2023-10-06T04:03:41.280 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './276' 2023-10-06T04:03:41.280 INFO:tasks.workunit.client.0.smithi157.stdout:removed './277/21' 2023-10-06T04:03:41.280 INFO:tasks.workunit.client.0.smithi157.stdout:removed './277/27' 2023-10-06T04:03:41.280 INFO:tasks.workunit.client.0.smithi157.stdout:removed './277/31' 2023-10-06T04:03:41.280 INFO:tasks.workunit.client.0.smithi157.stdout:removed './277/36' 2023-10-06T04:03:41.280 INFO:tasks.workunit.client.0.smithi157.stdout:removed './277/20' 2023-10-06T04:03:41.280 INFO:tasks.workunit.client.0.smithi157.stdout:removed './277/39' 2023-10-06T04:03:41.280 INFO:tasks.workunit.client.0.smithi157.stdout:removed './277/34' 2023-10-06T04:03:41.281 INFO:tasks.workunit.client.0.smithi157.stdout:removed './277/38' 2023-10-06T04:03:41.281 INFO:tasks.workunit.client.0.smithi157.stdout:removed './277/28' 2023-10-06T04:03:41.281 INFO:tasks.workunit.client.0.smithi157.stdout:removed './277/24' 2023-10-06T04:03:41.281 INFO:tasks.workunit.client.0.smithi157.stdout:removed './277/29' 2023-10-06T04:03:41.281 INFO:tasks.workunit.client.0.smithi157.stdout:removed './277/32' 2023-10-06T04:03:41.281 INFO:tasks.workunit.client.0.smithi157.stdout:removed './277/22' 2023-10-06T04:03:41.281 INFO:tasks.workunit.client.0.smithi157.stdout:removed './277/25' 2023-10-06T04:03:41.281 INFO:tasks.workunit.client.0.smithi157.stdout:removed './277/30' 2023-10-06T04:03:41.281 INFO:tasks.workunit.client.0.smithi157.stdout:removed './277/33' 2023-10-06T04:03:41.281 INFO:tasks.workunit.client.0.smithi157.stdout:removed './277/23' 2023-10-06T04:03:41.281 INFO:tasks.workunit.client.0.smithi157.stdout:removed './277/35' 2023-10-06T04:03:41.282 INFO:tasks.workunit.client.0.smithi157.stdout:removed './277/26' 2023-10-06T04:03:41.282 INFO:tasks.workunit.client.0.smithi157.stdout:removed './277/37' 2023-10-06T04:03:41.282 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './277' 2023-10-06T04:03:41.282 INFO:tasks.workunit.client.0.smithi157.stdout:removed './278/21' 2023-10-06T04:03:41.282 INFO:tasks.workunit.client.0.smithi157.stdout:removed './278/27' 2023-10-06T04:03:41.282 INFO:tasks.workunit.client.0.smithi157.stdout:removed './278/31' 2023-10-06T04:03:41.282 INFO:tasks.workunit.client.0.smithi157.stdout:removed './278/36' 2023-10-06T04:03:41.282 INFO:tasks.workunit.client.0.smithi157.stdout:removed './278/20' 2023-10-06T04:03:41.282 INFO:tasks.workunit.client.0.smithi157.stdout:removed './278/39' 2023-10-06T04:03:41.282 INFO:tasks.workunit.client.0.smithi157.stdout:removed './278/34' 2023-10-06T04:03:41.282 INFO:tasks.workunit.client.0.smithi157.stdout:removed './278/38' 2023-10-06T04:03:41.283 INFO:tasks.workunit.client.0.smithi157.stdout:removed './278/28' 2023-10-06T04:03:41.283 INFO:tasks.workunit.client.0.smithi157.stdout:removed './278/24' 2023-10-06T04:03:41.283 INFO:tasks.workunit.client.0.smithi157.stdout:removed './278/29' 2023-10-06T04:03:41.283 INFO:tasks.workunit.client.0.smithi157.stdout:removed './278/32' 2023-10-06T04:03:41.283 INFO:tasks.workunit.client.0.smithi157.stdout:removed './278/22' 2023-10-06T04:03:41.283 INFO:tasks.workunit.client.0.smithi157.stdout:removed './278/25' 2023-10-06T04:03:41.283 INFO:tasks.workunit.client.0.smithi157.stdout:removed './278/30' 2023-10-06T04:03:41.283 INFO:tasks.workunit.client.0.smithi157.stdout:removed './278/33' 2023-10-06T04:03:41.283 INFO:tasks.workunit.client.0.smithi157.stdout:removed './278/23' 2023-10-06T04:03:41.283 INFO:tasks.workunit.client.0.smithi157.stdout:removed './278/35' 2023-10-06T04:03:41.283 INFO:tasks.workunit.client.0.smithi157.stdout:removed './278/26' 2023-10-06T04:03:41.283 INFO:tasks.workunit.client.0.smithi157.stdout:removed './278/37' 2023-10-06T04:03:41.284 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './278' 2023-10-06T04:03:41.284 INFO:tasks.workunit.client.0.smithi157.stdout:removed './279/21' 2023-10-06T04:03:41.284 INFO:tasks.workunit.client.0.smithi157.stdout:removed './279/27' 2023-10-06T04:03:41.284 INFO:tasks.workunit.client.0.smithi157.stdout:removed './279/31' 2023-10-06T04:03:41.284 INFO:tasks.workunit.client.0.smithi157.stdout:removed './279/36' 2023-10-06T04:03:41.284 INFO:tasks.workunit.client.0.smithi157.stdout:removed './279/20' 2023-10-06T04:03:41.284 INFO:tasks.workunit.client.0.smithi157.stdout:removed './279/39' 2023-10-06T04:03:41.284 INFO:tasks.workunit.client.0.smithi157.stdout:removed './279/34' 2023-10-06T04:03:41.284 INFO:tasks.workunit.client.0.smithi157.stdout:removed './279/38' 2023-10-06T04:03:41.284 INFO:tasks.workunit.client.0.smithi157.stdout:removed './279/28' 2023-10-06T04:03:41.284 INFO:tasks.workunit.client.0.smithi157.stdout:removed './279/24' 2023-10-06T04:03:41.284 INFO:tasks.workunit.client.0.smithi157.stdout:removed './279/29' 2023-10-06T04:03:41.285 INFO:tasks.workunit.client.0.smithi157.stdout:removed './279/32' 2023-10-06T04:03:41.285 INFO:tasks.workunit.client.0.smithi157.stdout:removed './279/22' 2023-10-06T04:03:41.285 INFO:tasks.workunit.client.0.smithi157.stdout:removed './279/25' 2023-10-06T04:03:41.285 INFO:tasks.workunit.client.0.smithi157.stdout:removed './279/30' 2023-10-06T04:03:41.285 INFO:tasks.workunit.client.0.smithi157.stdout:removed './279/33' 2023-10-06T04:03:41.285 INFO:tasks.workunit.client.0.smithi157.stdout:removed './279/23' 2023-10-06T04:03:41.285 INFO:tasks.workunit.client.0.smithi157.stdout:removed './279/35' 2023-10-06T04:03:41.285 INFO:tasks.workunit.client.0.smithi157.stdout:removed './279/26' 2023-10-06T04:03:41.285 INFO:tasks.workunit.client.0.smithi157.stdout:removed './279/37' 2023-10-06T04:03:41.285 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './279' 2023-10-06T04:03:41.285 INFO:tasks.workunit.client.0.smithi157.stdout:removed './280/21' 2023-10-06T04:03:41.285 INFO:tasks.workunit.client.0.smithi157.stdout:removed './280/27' 2023-10-06T04:03:41.286 INFO:tasks.workunit.client.0.smithi157.stdout:removed './280/31' 2023-10-06T04:03:41.286 INFO:tasks.workunit.client.0.smithi157.stdout:removed './280/36' 2023-10-06T04:03:41.286 INFO:tasks.workunit.client.0.smithi157.stdout:removed './280/20' 2023-10-06T04:03:41.286 INFO:tasks.workunit.client.0.smithi157.stdout:removed './280/39' 2023-10-06T04:03:41.286 INFO:tasks.workunit.client.0.smithi157.stdout:removed './280/34' 2023-10-06T04:03:41.286 INFO:tasks.workunit.client.0.smithi157.stdout:removed './280/38' 2023-10-06T04:03:41.286 INFO:tasks.workunit.client.0.smithi157.stdout:removed './280/28' 2023-10-06T04:03:41.286 INFO:tasks.workunit.client.0.smithi157.stdout:removed './280/24' 2023-10-06T04:03:41.286 INFO:tasks.workunit.client.0.smithi157.stdout:removed './280/29' 2023-10-06T04:03:41.286 INFO:tasks.workunit.client.0.smithi157.stdout:removed './280/32' 2023-10-06T04:03:41.286 INFO:tasks.workunit.client.0.smithi157.stdout:removed './280/22' 2023-10-06T04:03:41.286 INFO:tasks.workunit.client.0.smithi157.stdout:removed './280/25' 2023-10-06T04:03:41.287 INFO:tasks.workunit.client.0.smithi157.stdout:removed './280/30' 2023-10-06T04:03:41.287 INFO:tasks.workunit.client.0.smithi157.stdout:removed './280/33' 2023-10-06T04:03:41.287 INFO:tasks.workunit.client.0.smithi157.stdout:removed './280/23' 2023-10-06T04:03:41.287 INFO:tasks.workunit.client.0.smithi157.stdout:removed './280/35' 2023-10-06T04:03:41.287 INFO:tasks.workunit.client.0.smithi157.stdout:removed './280/26' 2023-10-06T04:03:41.287 INFO:tasks.workunit.client.0.smithi157.stdout:removed './280/37' 2023-10-06T04:03:41.287 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './280' 2023-10-06T04:03:41.287 INFO:tasks.workunit.client.0.smithi157.stdout:removed './281/21' 2023-10-06T04:03:41.287 INFO:tasks.workunit.client.0.smithi157.stdout:removed './281/27' 2023-10-06T04:03:41.287 INFO:tasks.workunit.client.0.smithi157.stdout:removed './281/31' 2023-10-06T04:03:41.287 INFO:tasks.workunit.client.0.smithi157.stdout:removed './281/36' 2023-10-06T04:03:41.288 INFO:tasks.workunit.client.0.smithi157.stdout:removed './281/20' 2023-10-06T04:03:41.288 INFO:tasks.workunit.client.0.smithi157.stdout:removed './281/39' 2023-10-06T04:03:41.288 INFO:tasks.workunit.client.0.smithi157.stdout:removed './281/34' 2023-10-06T04:03:41.288 INFO:tasks.workunit.client.0.smithi157.stdout:removed './281/38' 2023-10-06T04:03:41.288 INFO:tasks.workunit.client.0.smithi157.stdout:removed './281/28' 2023-10-06T04:03:41.288 INFO:tasks.workunit.client.0.smithi157.stdout:removed './281/24' 2023-10-06T04:03:41.288 INFO:tasks.workunit.client.0.smithi157.stdout:removed './281/29' 2023-10-06T04:03:41.288 INFO:tasks.workunit.client.0.smithi157.stdout:removed './281/32' 2023-10-06T04:03:41.288 INFO:tasks.workunit.client.0.smithi157.stdout:removed './281/22' 2023-10-06T04:03:41.288 INFO:tasks.workunit.client.0.smithi157.stdout:removed './281/25' 2023-10-06T04:03:41.288 INFO:tasks.workunit.client.0.smithi157.stdout:removed './281/30' 2023-10-06T04:03:41.288 INFO:tasks.workunit.client.0.smithi157.stdout:removed './281/33' 2023-10-06T04:03:41.289 INFO:tasks.workunit.client.0.smithi157.stdout:removed './281/23' 2023-10-06T04:03:41.289 INFO:tasks.workunit.client.0.smithi157.stdout:removed './281/35' 2023-10-06T04:03:41.289 INFO:tasks.workunit.client.0.smithi157.stdout:removed './281/26' 2023-10-06T04:03:41.289 INFO:tasks.workunit.client.0.smithi157.stdout:removed './281/37' 2023-10-06T04:03:41.289 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './281' 2023-10-06T04:03:41.289 INFO:tasks.workunit.client.0.smithi157.stdout:removed './282/21' 2023-10-06T04:03:41.289 INFO:tasks.workunit.client.0.smithi157.stdout:removed './282/27' 2023-10-06T04:03:41.289 INFO:tasks.workunit.client.0.smithi157.stdout:removed './282/31' 2023-10-06T04:03:41.289 INFO:tasks.workunit.client.0.smithi157.stdout:removed './282/36' 2023-10-06T04:03:41.289 INFO:tasks.workunit.client.0.smithi157.stdout:removed './282/20' 2023-10-06T04:03:41.289 INFO:tasks.workunit.client.0.smithi157.stdout:removed './282/39' 2023-10-06T04:03:41.289 INFO:tasks.workunit.client.0.smithi157.stdout:removed './282/34' 2023-10-06T04:03:41.290 INFO:tasks.workunit.client.0.smithi157.stdout:removed './282/38' 2023-10-06T04:03:41.290 INFO:tasks.workunit.client.0.smithi157.stdout:removed './282/28' 2023-10-06T04:03:41.290 INFO:tasks.workunit.client.0.smithi157.stdout:removed './282/24' 2023-10-06T04:03:41.290 INFO:tasks.workunit.client.0.smithi157.stdout:removed './282/29' 2023-10-06T04:03:41.290 INFO:tasks.workunit.client.0.smithi157.stdout:removed './282/32' 2023-10-06T04:03:41.290 INFO:tasks.workunit.client.0.smithi157.stdout:removed './282/22' 2023-10-06T04:03:41.290 INFO:tasks.workunit.client.0.smithi157.stdout:removed './282/25' 2023-10-06T04:03:41.290 INFO:tasks.workunit.client.0.smithi157.stdout:removed './282/30' 2023-10-06T04:03:41.290 INFO:tasks.workunit.client.0.smithi157.stdout:removed './282/33' 2023-10-06T04:03:41.290 INFO:tasks.workunit.client.0.smithi157.stdout:removed './282/23' 2023-10-06T04:03:41.290 INFO:tasks.workunit.client.0.smithi157.stdout:removed './282/35' 2023-10-06T04:03:41.290 INFO:tasks.workunit.client.0.smithi157.stdout:removed './282/26' 2023-10-06T04:03:41.291 INFO:tasks.workunit.client.0.smithi157.stdout:removed './282/37' 2023-10-06T04:03:41.291 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './282' 2023-10-06T04:03:41.291 INFO:tasks.workunit.client.0.smithi157.stdout:removed './283/21' 2023-10-06T04:03:41.291 INFO:tasks.workunit.client.0.smithi157.stdout:removed './283/27' 2023-10-06T04:03:41.291 INFO:tasks.workunit.client.0.smithi157.stdout:removed './283/31' 2023-10-06T04:03:41.291 INFO:tasks.workunit.client.0.smithi157.stdout:removed './283/36' 2023-10-06T04:03:41.291 INFO:tasks.workunit.client.0.smithi157.stdout:removed './283/20' 2023-10-06T04:03:41.291 INFO:tasks.workunit.client.0.smithi157.stdout:removed './283/39' 2023-10-06T04:03:41.291 INFO:tasks.workunit.client.0.smithi157.stdout:removed './283/34' 2023-10-06T04:03:41.291 INFO:tasks.workunit.client.0.smithi157.stdout:removed './283/38' 2023-10-06T04:03:41.291 INFO:tasks.workunit.client.0.smithi157.stdout:removed './283/28' 2023-10-06T04:03:41.291 INFO:tasks.workunit.client.0.smithi157.stdout:removed './283/24' 2023-10-06T04:03:41.292 INFO:tasks.workunit.client.0.smithi157.stdout:removed './283/29' 2023-10-06T04:03:41.292 INFO:tasks.workunit.client.0.smithi157.stdout:removed './283/32' 2023-10-06T04:03:41.292 INFO:tasks.workunit.client.0.smithi157.stdout:removed './283/22' 2023-10-06T04:03:41.292 INFO:tasks.workunit.client.0.smithi157.stdout:removed './283/25' 2023-10-06T04:03:41.292 INFO:tasks.workunit.client.0.smithi157.stdout:removed './283/30' 2023-10-06T04:03:41.292 INFO:tasks.workunit.client.0.smithi157.stdout:removed './283/33' 2023-10-06T04:03:41.292 INFO:tasks.workunit.client.0.smithi157.stdout:removed './283/23' 2023-10-06T04:03:41.292 INFO:tasks.workunit.client.0.smithi157.stdout:removed './283/35' 2023-10-06T04:03:41.292 INFO:tasks.workunit.client.0.smithi157.stdout:removed './283/26' 2023-10-06T04:03:41.292 INFO:tasks.workunit.client.0.smithi157.stdout:removed './283/37' 2023-10-06T04:03:41.292 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './283' 2023-10-06T04:03:41.292 INFO:tasks.workunit.client.0.smithi157.stdout:removed './284/21' 2023-10-06T04:03:41.293 INFO:tasks.workunit.client.0.smithi157.stdout:removed './284/27' 2023-10-06T04:03:41.293 INFO:tasks.workunit.client.0.smithi157.stdout:removed './284/31' 2023-10-06T04:03:41.293 INFO:tasks.workunit.client.0.smithi157.stdout:removed './284/36' 2023-10-06T04:03:41.293 INFO:tasks.workunit.client.0.smithi157.stdout:removed './284/20' 2023-10-06T04:03:41.293 INFO:tasks.workunit.client.0.smithi157.stdout:removed './284/39' 2023-10-06T04:03:41.293 INFO:tasks.workunit.client.0.smithi157.stdout:removed './284/34' 2023-10-06T04:03:41.293 INFO:tasks.workunit.client.0.smithi157.stdout:removed './284/38' 2023-10-06T04:03:41.293 INFO:tasks.workunit.client.0.smithi157.stdout:removed './284/28' 2023-10-06T04:03:41.293 INFO:tasks.workunit.client.0.smithi157.stdout:removed './284/24' 2023-10-06T04:03:41.293 INFO:tasks.workunit.client.0.smithi157.stdout:removed './284/29' 2023-10-06T04:03:41.293 INFO:tasks.workunit.client.0.smithi157.stdout:removed './284/32' 2023-10-06T04:03:41.293 INFO:tasks.workunit.client.0.smithi157.stdout:removed './284/22' 2023-10-06T04:03:41.294 INFO:tasks.workunit.client.0.smithi157.stdout:removed './284/25' 2023-10-06T04:03:41.294 INFO:tasks.workunit.client.0.smithi157.stdout:removed './284/30' 2023-10-06T04:03:41.294 INFO:tasks.workunit.client.0.smithi157.stdout:removed './284/33' 2023-10-06T04:03:41.294 INFO:tasks.workunit.client.0.smithi157.stdout:removed './284/23' 2023-10-06T04:03:41.294 INFO:tasks.workunit.client.0.smithi157.stdout:removed './284/35' 2023-10-06T04:03:41.294 INFO:tasks.workunit.client.0.smithi157.stdout:removed './284/26' 2023-10-06T04:03:41.294 INFO:tasks.workunit.client.0.smithi157.stdout:removed './284/37' 2023-10-06T04:03:41.294 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './284' 2023-10-06T04:03:41.294 INFO:tasks.workunit.client.0.smithi157.stdout:removed './285/21' 2023-10-06T04:03:41.294 INFO:tasks.workunit.client.0.smithi157.stdout:removed './285/27' 2023-10-06T04:03:41.294 INFO:tasks.workunit.client.0.smithi157.stdout:removed './285/31' 2023-10-06T04:03:41.294 INFO:tasks.workunit.client.0.smithi157.stdout:removed './285/36' 2023-10-06T04:03:41.295 INFO:tasks.workunit.client.0.smithi157.stdout:removed './285/20' 2023-10-06T04:03:41.295 INFO:tasks.workunit.client.0.smithi157.stdout:removed './285/39' 2023-10-06T04:03:41.295 INFO:tasks.workunit.client.0.smithi157.stdout:removed './285/34' 2023-10-06T04:03:41.295 INFO:tasks.workunit.client.0.smithi157.stdout:removed './285/38' 2023-10-06T04:03:41.295 INFO:tasks.workunit.client.0.smithi157.stdout:removed './285/28' 2023-10-06T04:03:41.295 INFO:tasks.workunit.client.0.smithi157.stdout:removed './285/24' 2023-10-06T04:03:41.295 INFO:tasks.workunit.client.0.smithi157.stdout:removed './285/29' 2023-10-06T04:03:41.295 INFO:tasks.workunit.client.0.smithi157.stdout:removed './285/32' 2023-10-06T04:03:41.295 INFO:tasks.workunit.client.0.smithi157.stdout:removed './285/22' 2023-10-06T04:03:41.295 INFO:tasks.workunit.client.0.smithi157.stdout:removed './285/25' 2023-10-06T04:03:41.295 INFO:tasks.workunit.client.0.smithi157.stdout:removed './285/30' 2023-10-06T04:03:41.295 INFO:tasks.workunit.client.0.smithi157.stdout:removed './285/33' 2023-10-06T04:03:41.296 INFO:tasks.workunit.client.0.smithi157.stdout:removed './285/23' 2023-10-06T04:03:41.296 INFO:tasks.workunit.client.0.smithi157.stdout:removed './285/35' 2023-10-06T04:03:41.296 INFO:tasks.workunit.client.0.smithi157.stdout:removed './285/26' 2023-10-06T04:03:41.296 INFO:tasks.workunit.client.0.smithi157.stdout:removed './285/37' 2023-10-06T04:03:41.296 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './285' 2023-10-06T04:03:41.296 INFO:tasks.workunit.client.0.smithi157.stdout:removed './286/21' 2023-10-06T04:03:41.296 INFO:tasks.workunit.client.0.smithi157.stdout:removed './286/27' 2023-10-06T04:03:41.296 INFO:tasks.workunit.client.0.smithi157.stdout:removed './286/31' 2023-10-06T04:03:41.296 INFO:tasks.workunit.client.0.smithi157.stdout:removed './286/36' 2023-10-06T04:03:41.296 INFO:tasks.workunit.client.0.smithi157.stdout:removed './286/20' 2023-10-06T04:03:41.296 INFO:tasks.workunit.client.0.smithi157.stdout:removed './286/39' 2023-10-06T04:03:41.296 INFO:tasks.workunit.client.0.smithi157.stdout:removed './286/34' 2023-10-06T04:03:41.297 INFO:tasks.workunit.client.0.smithi157.stdout:removed './286/38' 2023-10-06T04:03:41.297 INFO:tasks.workunit.client.0.smithi157.stdout:removed './286/28' 2023-10-06T04:03:41.297 INFO:tasks.workunit.client.0.smithi157.stdout:removed './286/24' 2023-10-06T04:03:41.297 INFO:tasks.workunit.client.0.smithi157.stdout:removed './286/29' 2023-10-06T04:03:41.297 INFO:tasks.workunit.client.0.smithi157.stdout:removed './286/32' 2023-10-06T04:03:41.297 INFO:tasks.workunit.client.0.smithi157.stdout:removed './286/22' 2023-10-06T04:03:41.297 INFO:tasks.workunit.client.0.smithi157.stdout:removed './286/25' 2023-10-06T04:03:41.297 INFO:tasks.workunit.client.0.smithi157.stdout:removed './286/30' 2023-10-06T04:03:41.297 INFO:tasks.workunit.client.0.smithi157.stdout:removed './286/33' 2023-10-06T04:03:41.297 INFO:tasks.workunit.client.0.smithi157.stdout:removed './286/23' 2023-10-06T04:03:41.297 INFO:tasks.workunit.client.0.smithi157.stdout:removed './286/35' 2023-10-06T04:03:41.297 INFO:tasks.workunit.client.0.smithi157.stdout:removed './286/26' 2023-10-06T04:03:41.298 INFO:tasks.workunit.client.0.smithi157.stdout:removed './286/37' 2023-10-06T04:03:41.298 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './286' 2023-10-06T04:03:41.946 INFO:tasks.workunit.client.0.smithi157.stdout:removed './287/21' 2023-10-06T04:03:41.946 INFO:tasks.workunit.client.0.smithi157.stdout:removed './287/27' 2023-10-06T04:03:41.946 INFO:tasks.workunit.client.0.smithi157.stdout:removed './287/31' 2023-10-06T04:03:41.947 INFO:tasks.workunit.client.0.smithi157.stdout:removed './287/36' 2023-10-06T04:03:41.947 INFO:tasks.workunit.client.0.smithi157.stdout:removed './287/20' 2023-10-06T04:03:41.947 INFO:tasks.workunit.client.0.smithi157.stdout:removed './287/39' 2023-10-06T04:03:41.947 INFO:tasks.workunit.client.0.smithi157.stdout:removed './287/34' 2023-10-06T04:03:41.947 INFO:tasks.workunit.client.0.smithi157.stdout:removed './287/38' 2023-10-06T04:03:41.947 INFO:tasks.workunit.client.0.smithi157.stdout:removed './287/28' 2023-10-06T04:03:41.947 INFO:tasks.workunit.client.0.smithi157.stdout:removed './287/24' 2023-10-06T04:03:41.947 INFO:tasks.workunit.client.0.smithi157.stdout:removed './287/29' 2023-10-06T04:03:41.947 INFO:tasks.workunit.client.0.smithi157.stdout:removed './287/32' 2023-10-06T04:03:41.947 INFO:tasks.workunit.client.0.smithi157.stdout:removed './287/22' 2023-10-06T04:03:41.947 INFO:tasks.workunit.client.0.smithi157.stdout:removed './287/25' 2023-10-06T04:03:41.948 INFO:tasks.workunit.client.0.smithi157.stdout:removed './287/30' 2023-10-06T04:03:41.948 INFO:tasks.workunit.client.0.smithi157.stdout:removed './287/33' 2023-10-06T04:03:41.948 INFO:tasks.workunit.client.0.smithi157.stdout:removed './287/23' 2023-10-06T04:03:41.948 INFO:tasks.workunit.client.0.smithi157.stdout:removed './287/35' 2023-10-06T04:03:41.948 INFO:tasks.workunit.client.0.smithi157.stdout:removed './287/26' 2023-10-06T04:03:41.948 INFO:tasks.workunit.client.0.smithi157.stdout:removed './287/37' 2023-10-06T04:03:41.948 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './287' 2023-10-06T04:03:41.948 INFO:tasks.workunit.client.0.smithi157.stdout:removed './288/21' 2023-10-06T04:03:41.948 INFO:tasks.workunit.client.0.smithi157.stdout:removed './288/27' 2023-10-06T04:03:41.948 INFO:tasks.workunit.client.0.smithi157.stdout:removed './288/31' 2023-10-06T04:03:41.948 INFO:tasks.workunit.client.0.smithi157.stdout:removed './288/36' 2023-10-06T04:03:41.948 INFO:tasks.workunit.client.0.smithi157.stdout:removed './288/20' 2023-10-06T04:03:41.949 INFO:tasks.workunit.client.0.smithi157.stdout:removed './288/39' 2023-10-06T04:03:41.949 INFO:tasks.workunit.client.0.smithi157.stdout:removed './288/34' 2023-10-06T04:03:41.949 INFO:tasks.workunit.client.0.smithi157.stdout:removed './288/38' 2023-10-06T04:03:41.949 INFO:tasks.workunit.client.0.smithi157.stdout:removed './288/28' 2023-10-06T04:03:41.949 INFO:tasks.workunit.client.0.smithi157.stdout:removed './288/24' 2023-10-06T04:03:41.949 INFO:tasks.workunit.client.0.smithi157.stdout:removed './288/29' 2023-10-06T04:03:41.949 INFO:tasks.workunit.client.0.smithi157.stdout:removed './288/32' 2023-10-06T04:03:41.949 INFO:tasks.workunit.client.0.smithi157.stdout:removed './288/22' 2023-10-06T04:03:41.949 INFO:tasks.workunit.client.0.smithi157.stdout:removed './288/25' 2023-10-06T04:03:41.949 INFO:tasks.workunit.client.0.smithi157.stdout:removed './288/30' 2023-10-06T04:03:41.949 INFO:tasks.workunit.client.0.smithi157.stdout:removed './288/33' 2023-10-06T04:03:41.950 INFO:tasks.workunit.client.0.smithi157.stdout:removed './288/23' 2023-10-06T04:03:41.950 INFO:tasks.workunit.client.0.smithi157.stdout:removed './288/35' 2023-10-06T04:03:41.950 INFO:tasks.workunit.client.0.smithi157.stdout:removed './288/26' 2023-10-06T04:03:41.950 INFO:tasks.workunit.client.0.smithi157.stdout:removed './288/37' 2023-10-06T04:03:41.950 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './288' 2023-10-06T04:03:41.950 INFO:tasks.workunit.client.0.smithi157.stdout:removed './289/21' 2023-10-06T04:03:41.950 INFO:tasks.workunit.client.0.smithi157.stdout:removed './289/27' 2023-10-06T04:03:41.950 INFO:tasks.workunit.client.0.smithi157.stdout:removed './289/31' 2023-10-06T04:03:41.950 INFO:tasks.workunit.client.0.smithi157.stdout:removed './289/36' 2023-10-06T04:03:41.950 INFO:tasks.workunit.client.0.smithi157.stdout:removed './289/20' 2023-10-06T04:03:41.951 INFO:tasks.workunit.client.0.smithi157.stdout:removed './289/39' 2023-10-06T04:03:41.951 INFO:tasks.workunit.client.0.smithi157.stdout:removed './289/34' 2023-10-06T04:03:41.951 INFO:tasks.workunit.client.0.smithi157.stdout:removed './289/38' 2023-10-06T04:03:41.951 INFO:tasks.workunit.client.0.smithi157.stdout:removed './289/28' 2023-10-06T04:03:41.951 INFO:tasks.workunit.client.0.smithi157.stdout:removed './289/24' 2023-10-06T04:03:41.951 INFO:tasks.workunit.client.0.smithi157.stdout:removed './289/29' 2023-10-06T04:03:41.951 INFO:tasks.workunit.client.0.smithi157.stdout:removed './289/32' 2023-10-06T04:03:41.951 INFO:tasks.workunit.client.0.smithi157.stdout:removed './289/22' 2023-10-06T04:03:41.951 INFO:tasks.workunit.client.0.smithi157.stdout:removed './289/25' 2023-10-06T04:03:41.951 INFO:tasks.workunit.client.0.smithi157.stdout:removed './289/30' 2023-10-06T04:03:41.951 INFO:tasks.workunit.client.0.smithi157.stdout:removed './289/33' 2023-10-06T04:03:41.951 INFO:tasks.workunit.client.0.smithi157.stdout:removed './289/23' 2023-10-06T04:03:41.952 INFO:tasks.workunit.client.0.smithi157.stdout:removed './289/35' 2023-10-06T04:03:41.952 INFO:tasks.workunit.client.0.smithi157.stdout:removed './289/26' 2023-10-06T04:03:41.952 INFO:tasks.workunit.client.0.smithi157.stdout:removed './289/37' 2023-10-06T04:03:41.952 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './289' 2023-10-06T04:03:41.952 INFO:tasks.workunit.client.0.smithi157.stdout:removed './290/21' 2023-10-06T04:03:41.952 INFO:tasks.workunit.client.0.smithi157.stdout:removed './290/27' 2023-10-06T04:03:41.952 INFO:tasks.workunit.client.0.smithi157.stdout:removed './290/31' 2023-10-06T04:03:41.952 INFO:tasks.workunit.client.0.smithi157.stdout:removed './290/36' 2023-10-06T04:03:41.952 INFO:tasks.workunit.client.0.smithi157.stdout:removed './290/20' 2023-10-06T04:03:41.952 INFO:tasks.workunit.client.0.smithi157.stdout:removed './290/39' 2023-10-06T04:03:41.952 INFO:tasks.workunit.client.0.smithi157.stdout:removed './290/34' 2023-10-06T04:03:41.952 INFO:tasks.workunit.client.0.smithi157.stdout:removed './290/38' 2023-10-06T04:03:41.953 INFO:tasks.workunit.client.0.smithi157.stdout:removed './290/28' 2023-10-06T04:03:41.953 INFO:tasks.workunit.client.0.smithi157.stdout:removed './290/24' 2023-10-06T04:03:41.953 INFO:tasks.workunit.client.0.smithi157.stdout:removed './290/29' 2023-10-06T04:03:41.953 INFO:tasks.workunit.client.0.smithi157.stdout:removed './290/32' 2023-10-06T04:03:41.953 INFO:tasks.workunit.client.0.smithi157.stdout:removed './290/22' 2023-10-06T04:03:41.953 INFO:tasks.workunit.client.0.smithi157.stdout:removed './290/25' 2023-10-06T04:03:41.953 INFO:tasks.workunit.client.0.smithi157.stdout:removed './290/30' 2023-10-06T04:03:41.953 INFO:tasks.workunit.client.0.smithi157.stdout:removed './290/33' 2023-10-06T04:03:41.953 INFO:tasks.workunit.client.0.smithi157.stdout:removed './290/23' 2023-10-06T04:03:41.953 INFO:tasks.workunit.client.0.smithi157.stdout:removed './290/35' 2023-10-06T04:03:41.953 INFO:tasks.workunit.client.0.smithi157.stdout:removed './290/26' 2023-10-06T04:03:41.953 INFO:tasks.workunit.client.0.smithi157.stdout:removed './290/37' 2023-10-06T04:03:41.954 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './290' 2023-10-06T04:03:41.954 INFO:tasks.workunit.client.0.smithi157.stdout:removed './291/21' 2023-10-06T04:03:41.954 INFO:tasks.workunit.client.0.smithi157.stdout:removed './291/27' 2023-10-06T04:03:41.954 INFO:tasks.workunit.client.0.smithi157.stdout:removed './291/31' 2023-10-06T04:03:41.954 INFO:tasks.workunit.client.0.smithi157.stdout:removed './291/36' 2023-10-06T04:03:41.954 INFO:tasks.workunit.client.0.smithi157.stdout:removed './291/20' 2023-10-06T04:03:41.954 INFO:tasks.workunit.client.0.smithi157.stdout:removed './291/39' 2023-10-06T04:03:41.954 INFO:tasks.workunit.client.0.smithi157.stdout:removed './291/34' 2023-10-06T04:03:41.954 INFO:tasks.workunit.client.0.smithi157.stdout:removed './291/38' 2023-10-06T04:03:41.954 INFO:tasks.workunit.client.0.smithi157.stdout:removed './291/28' 2023-10-06T04:03:41.954 INFO:tasks.workunit.client.0.smithi157.stdout:removed './291/24' 2023-10-06T04:03:41.954 INFO:tasks.workunit.client.0.smithi157.stdout:removed './291/29' 2023-10-06T04:03:41.955 INFO:tasks.workunit.client.0.smithi157.stdout:removed './291/32' 2023-10-06T04:03:41.955 INFO:tasks.workunit.client.0.smithi157.stdout:removed './291/22' 2023-10-06T04:03:41.955 INFO:tasks.workunit.client.0.smithi157.stdout:removed './291/25' 2023-10-06T04:03:41.955 INFO:tasks.workunit.client.0.smithi157.stdout:removed './291/30' 2023-10-06T04:03:41.955 INFO:tasks.workunit.client.0.smithi157.stdout:removed './291/33' 2023-10-06T04:03:41.955 INFO:tasks.workunit.client.0.smithi157.stdout:removed './291/23' 2023-10-06T04:03:41.955 INFO:tasks.workunit.client.0.smithi157.stdout:removed './291/35' 2023-10-06T04:03:41.955 INFO:tasks.workunit.client.0.smithi157.stdout:removed './291/26' 2023-10-06T04:03:41.955 INFO:tasks.workunit.client.0.smithi157.stdout:removed './291/37' 2023-10-06T04:03:41.955 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './291' 2023-10-06T04:03:41.955 INFO:tasks.workunit.client.0.smithi157.stdout:removed './292/21' 2023-10-06T04:03:41.956 INFO:tasks.workunit.client.0.smithi157.stdout:removed './292/27' 2023-10-06T04:03:41.956 INFO:tasks.workunit.client.0.smithi157.stdout:removed './292/31' 2023-10-06T04:03:41.956 INFO:tasks.workunit.client.0.smithi157.stdout:removed './292/36' 2023-10-06T04:03:41.956 INFO:tasks.workunit.client.0.smithi157.stdout:removed './292/20' 2023-10-06T04:03:41.956 INFO:tasks.workunit.client.0.smithi157.stdout:removed './292/39' 2023-10-06T04:03:41.956 INFO:tasks.workunit.client.0.smithi157.stdout:removed './292/34' 2023-10-06T04:03:41.956 INFO:tasks.workunit.client.0.smithi157.stdout:removed './292/38' 2023-10-06T04:03:41.956 INFO:tasks.workunit.client.0.smithi157.stdout:removed './292/28' 2023-10-06T04:03:41.956 INFO:tasks.workunit.client.0.smithi157.stdout:removed './292/24' 2023-10-06T04:03:41.956 INFO:tasks.workunit.client.0.smithi157.stdout:removed './292/29' 2023-10-06T04:03:41.956 INFO:tasks.workunit.client.0.smithi157.stdout:removed './292/32' 2023-10-06T04:03:41.956 INFO:tasks.workunit.client.0.smithi157.stdout:removed './292/22' 2023-10-06T04:03:41.957 INFO:tasks.workunit.client.0.smithi157.stdout:removed './292/25' 2023-10-06T04:03:41.957 INFO:tasks.workunit.client.0.smithi157.stdout:removed './292/30' 2023-10-06T04:03:41.957 INFO:tasks.workunit.client.0.smithi157.stdout:removed './292/33' 2023-10-06T04:03:41.957 INFO:tasks.workunit.client.0.smithi157.stdout:removed './292/23' 2023-10-06T04:03:41.957 INFO:tasks.workunit.client.0.smithi157.stdout:removed './292/35' 2023-10-06T04:03:41.957 INFO:tasks.workunit.client.0.smithi157.stdout:removed './292/26' 2023-10-06T04:03:41.957 INFO:tasks.workunit.client.0.smithi157.stdout:removed './292/37' 2023-10-06T04:03:41.957 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './292' 2023-10-06T04:03:41.957 INFO:tasks.workunit.client.0.smithi157.stdout:removed './293/21' 2023-10-06T04:03:41.957 INFO:tasks.workunit.client.0.smithi157.stdout:removed './293/27' 2023-10-06T04:03:41.957 INFO:tasks.workunit.client.0.smithi157.stdout:removed './293/31' 2023-10-06T04:03:41.957 INFO:tasks.workunit.client.0.smithi157.stdout:removed './293/36' 2023-10-06T04:03:41.958 INFO:tasks.workunit.client.0.smithi157.stdout:removed './293/20' 2023-10-06T04:03:41.958 INFO:tasks.workunit.client.0.smithi157.stdout:removed './293/39' 2023-10-06T04:03:41.958 INFO:tasks.workunit.client.0.smithi157.stdout:removed './293/34' 2023-10-06T04:03:41.958 INFO:tasks.workunit.client.0.smithi157.stdout:removed './293/38' 2023-10-06T04:03:41.958 INFO:tasks.workunit.client.0.smithi157.stdout:removed './293/28' 2023-10-06T04:03:41.958 INFO:tasks.workunit.client.0.smithi157.stdout:removed './293/24' 2023-10-06T04:03:41.958 INFO:tasks.workunit.client.0.smithi157.stdout:removed './293/29' 2023-10-06T04:03:41.959 INFO:tasks.workunit.client.0.smithi157.stdout:removed './293/32' 2023-10-06T04:03:41.959 INFO:tasks.workunit.client.0.smithi157.stdout:removed './293/22' 2023-10-06T04:03:41.959 INFO:tasks.workunit.client.0.smithi157.stdout:removed './293/25' 2023-10-06T04:03:41.959 INFO:tasks.workunit.client.0.smithi157.stdout:removed './293/30' 2023-10-06T04:03:41.959 INFO:tasks.workunit.client.0.smithi157.stdout:removed './293/33' 2023-10-06T04:03:41.959 INFO:tasks.workunit.client.0.smithi157.stdout:removed './293/23' 2023-10-06T04:03:41.959 INFO:tasks.workunit.client.0.smithi157.stdout:removed './293/35' 2023-10-06T04:03:41.959 INFO:tasks.workunit.client.0.smithi157.stdout:removed './293/26' 2023-10-06T04:03:41.959 INFO:tasks.workunit.client.0.smithi157.stdout:removed './293/37' 2023-10-06T04:03:41.959 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './293' 2023-10-06T04:03:41.960 INFO:tasks.workunit.client.0.smithi157.stdout:removed './294/21' 2023-10-06T04:03:41.960 INFO:tasks.workunit.client.0.smithi157.stdout:removed './294/27' 2023-10-06T04:03:41.960 INFO:tasks.workunit.client.0.smithi157.stdout:removed './294/31' 2023-10-06T04:03:41.960 INFO:tasks.workunit.client.0.smithi157.stdout:removed './294/36' 2023-10-06T04:03:41.960 INFO:tasks.workunit.client.0.smithi157.stdout:removed './294/20' 2023-10-06T04:03:41.960 INFO:tasks.workunit.client.0.smithi157.stdout:removed './294/39' 2023-10-06T04:03:41.960 INFO:tasks.workunit.client.0.smithi157.stdout:removed './294/34' 2023-10-06T04:03:41.960 INFO:tasks.workunit.client.0.smithi157.stdout:removed './294/38' 2023-10-06T04:03:41.960 INFO:tasks.workunit.client.0.smithi157.stdout:removed './294/28' 2023-10-06T04:03:41.960 INFO:tasks.workunit.client.0.smithi157.stdout:removed './294/24' 2023-10-06T04:03:41.961 INFO:tasks.workunit.client.0.smithi157.stdout:removed './294/29' 2023-10-06T04:03:41.961 INFO:tasks.workunit.client.0.smithi157.stdout:removed './294/32' 2023-10-06T04:03:41.961 INFO:tasks.workunit.client.0.smithi157.stdout:removed './294/22' 2023-10-06T04:03:41.961 INFO:tasks.workunit.client.0.smithi157.stdout:removed './294/25' 2023-10-06T04:03:41.961 INFO:tasks.workunit.client.0.smithi157.stdout:removed './294/30' 2023-10-06T04:03:41.961 INFO:tasks.workunit.client.0.smithi157.stdout:removed './294/33' 2023-10-06T04:03:41.961 INFO:tasks.workunit.client.0.smithi157.stdout:removed './294/23' 2023-10-06T04:03:41.961 INFO:tasks.workunit.client.0.smithi157.stdout:removed './294/35' 2023-10-06T04:03:41.961 INFO:tasks.workunit.client.0.smithi157.stdout:removed './294/26' 2023-10-06T04:03:41.961 INFO:tasks.workunit.client.0.smithi157.stdout:removed './294/37' 2023-10-06T04:03:41.961 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './294' 2023-10-06T04:03:41.961 INFO:tasks.workunit.client.0.smithi157.stdout:removed './295/21' 2023-10-06T04:03:41.962 INFO:tasks.workunit.client.0.smithi157.stdout:removed './295/27' 2023-10-06T04:03:41.962 INFO:tasks.workunit.client.0.smithi157.stdout:removed './295/31' 2023-10-06T04:03:41.962 INFO:tasks.workunit.client.0.smithi157.stdout:removed './295/36' 2023-10-06T04:03:41.962 INFO:tasks.workunit.client.0.smithi157.stdout:removed './295/20' 2023-10-06T04:03:41.962 INFO:tasks.workunit.client.0.smithi157.stdout:removed './295/39' 2023-10-06T04:03:41.962 INFO:tasks.workunit.client.0.smithi157.stdout:removed './295/34' 2023-10-06T04:03:41.962 INFO:tasks.workunit.client.0.smithi157.stdout:removed './295/38' 2023-10-06T04:03:41.962 INFO:tasks.workunit.client.0.smithi157.stdout:removed './295/28' 2023-10-06T04:03:41.962 INFO:tasks.workunit.client.0.smithi157.stdout:removed './295/24' 2023-10-06T04:03:41.962 INFO:tasks.workunit.client.0.smithi157.stdout:removed './295/29' 2023-10-06T04:03:41.962 INFO:tasks.workunit.client.0.smithi157.stdout:removed './295/32' 2023-10-06T04:03:41.963 INFO:tasks.workunit.client.0.smithi157.stdout:removed './295/22' 2023-10-06T04:03:41.963 INFO:tasks.workunit.client.0.smithi157.stdout:removed './295/25' 2023-10-06T04:03:41.963 INFO:tasks.workunit.client.0.smithi157.stdout:removed './295/30' 2023-10-06T04:03:41.963 INFO:tasks.workunit.client.0.smithi157.stdout:removed './295/33' 2023-10-06T04:03:41.963 INFO:tasks.workunit.client.0.smithi157.stdout:removed './295/23' 2023-10-06T04:03:41.963 INFO:tasks.workunit.client.0.smithi157.stdout:removed './295/35' 2023-10-06T04:03:41.963 INFO:tasks.workunit.client.0.smithi157.stdout:removed './295/26' 2023-10-06T04:03:41.963 INFO:tasks.workunit.client.0.smithi157.stdout:removed './295/37' 2023-10-06T04:03:41.963 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './295' 2023-10-06T04:03:41.963 INFO:tasks.workunit.client.0.smithi157.stdout:removed './296/21' 2023-10-06T04:03:41.963 INFO:tasks.workunit.client.0.smithi157.stdout:removed './296/27' 2023-10-06T04:03:41.963 INFO:tasks.workunit.client.0.smithi157.stdout:removed './296/31' 2023-10-06T04:03:41.964 INFO:tasks.workunit.client.0.smithi157.stdout:removed './296/36' 2023-10-06T04:03:41.964 INFO:tasks.workunit.client.0.smithi157.stdout:removed './296/20' 2023-10-06T04:03:41.964 INFO:tasks.workunit.client.0.smithi157.stdout:removed './296/39' 2023-10-06T04:03:41.964 INFO:tasks.workunit.client.0.smithi157.stdout:removed './296/34' 2023-10-06T04:03:41.964 INFO:tasks.workunit.client.0.smithi157.stdout:removed './296/38' 2023-10-06T04:03:41.964 INFO:tasks.workunit.client.0.smithi157.stdout:removed './296/28' 2023-10-06T04:03:41.964 INFO:tasks.workunit.client.0.smithi157.stdout:removed './296/24' 2023-10-06T04:03:41.964 INFO:tasks.workunit.client.0.smithi157.stdout:removed './296/29' 2023-10-06T04:03:41.964 INFO:tasks.workunit.client.0.smithi157.stdout:removed './296/32' 2023-10-06T04:03:41.964 INFO:tasks.workunit.client.0.smithi157.stdout:removed './296/22' 2023-10-06T04:03:41.964 INFO:tasks.workunit.client.0.smithi157.stdout:removed './296/25' 2023-10-06T04:03:41.964 INFO:tasks.workunit.client.0.smithi157.stdout:removed './296/30' 2023-10-06T04:03:41.965 INFO:tasks.workunit.client.0.smithi157.stdout:removed './296/33' 2023-10-06T04:03:41.965 INFO:tasks.workunit.client.0.smithi157.stdout:removed './296/23' 2023-10-06T04:03:41.965 INFO:tasks.workunit.client.0.smithi157.stdout:removed './296/35' 2023-10-06T04:03:41.965 INFO:tasks.workunit.client.0.smithi157.stdout:removed './296/26' 2023-10-06T04:03:41.965 INFO:tasks.workunit.client.0.smithi157.stdout:removed './296/37' 2023-10-06T04:03:41.965 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './296' 2023-10-06T04:03:41.965 INFO:tasks.workunit.client.0.smithi157.stdout:removed './297/21' 2023-10-06T04:03:41.965 INFO:tasks.workunit.client.0.smithi157.stdout:removed './297/27' 2023-10-06T04:03:42.365 INFO:tasks.workunit.client.0.smithi157.stdout:removed './297/31' 2023-10-06T04:03:42.365 INFO:tasks.workunit.client.0.smithi157.stdout:removed './297/36' 2023-10-06T04:03:42.365 INFO:tasks.workunit.client.0.smithi157.stdout:removed './297/20' 2023-10-06T04:03:42.365 INFO:tasks.workunit.client.0.smithi157.stdout:removed './297/39' 2023-10-06T04:03:42.365 INFO:tasks.workunit.client.0.smithi157.stdout:removed './297/34' 2023-10-06T04:03:42.365 INFO:tasks.workunit.client.0.smithi157.stdout:removed './297/38' 2023-10-06T04:03:42.365 INFO:tasks.workunit.client.0.smithi157.stdout:removed './297/28' 2023-10-06T04:03:42.365 INFO:tasks.workunit.client.0.smithi157.stdout:removed './297/24' 2023-10-06T04:03:42.366 INFO:tasks.workunit.client.0.smithi157.stdout:removed './297/29' 2023-10-06T04:03:42.366 INFO:tasks.workunit.client.0.smithi157.stdout:removed './297/32' 2023-10-06T04:03:42.366 INFO:tasks.workunit.client.0.smithi157.stdout:removed './297/22' 2023-10-06T04:03:42.366 INFO:tasks.workunit.client.0.smithi157.stdout:removed './297/25' 2023-10-06T04:03:42.366 INFO:tasks.workunit.client.0.smithi157.stdout:removed './297/30' 2023-10-06T04:03:42.366 INFO:tasks.workunit.client.0.smithi157.stdout:removed './297/33' 2023-10-06T04:03:42.366 INFO:tasks.workunit.client.0.smithi157.stdout:removed './297/23' 2023-10-06T04:03:42.366 INFO:tasks.workunit.client.0.smithi157.stdout:removed './297/35' 2023-10-06T04:03:42.366 INFO:tasks.workunit.client.0.smithi157.stdout:removed './297/26' 2023-10-06T04:03:42.367 INFO:tasks.workunit.client.0.smithi157.stdout:removed './297/37' 2023-10-06T04:03:42.367 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './297' 2023-10-06T04:03:42.367 INFO:tasks.workunit.client.0.smithi157.stdout:removed './298/21' 2023-10-06T04:03:42.367 INFO:tasks.workunit.client.0.smithi157.stdout:removed './298/27' 2023-10-06T04:03:42.367 INFO:tasks.workunit.client.0.smithi157.stdout:removed './298/31' 2023-10-06T04:03:42.367 INFO:tasks.workunit.client.0.smithi157.stdout:removed './298/36' 2023-10-06T04:03:42.367 INFO:tasks.workunit.client.0.smithi157.stdout:removed './298/20' 2023-10-06T04:03:42.367 INFO:tasks.workunit.client.0.smithi157.stdout:removed './298/39' 2023-10-06T04:03:42.367 INFO:tasks.workunit.client.0.smithi157.stdout:removed './298/34' 2023-10-06T04:03:42.368 INFO:tasks.workunit.client.0.smithi157.stdout:removed './298/38' 2023-10-06T04:03:42.368 INFO:tasks.workunit.client.0.smithi157.stdout:removed './298/28' 2023-10-06T04:03:42.368 INFO:tasks.workunit.client.0.smithi157.stdout:removed './298/24' 2023-10-06T04:03:42.368 INFO:tasks.workunit.client.0.smithi157.stdout:removed './298/29' 2023-10-06T04:03:42.368 INFO:tasks.workunit.client.0.smithi157.stdout:removed './298/32' 2023-10-06T04:03:42.368 INFO:tasks.workunit.client.0.smithi157.stdout:removed './298/22' 2023-10-06T04:03:42.368 INFO:tasks.workunit.client.0.smithi157.stdout:removed './298/25' 2023-10-06T04:03:42.368 INFO:tasks.workunit.client.0.smithi157.stdout:removed './298/30' 2023-10-06T04:03:42.368 INFO:tasks.workunit.client.0.smithi157.stdout:removed './298/33' 2023-10-06T04:03:42.369 INFO:tasks.workunit.client.0.smithi157.stdout:removed './298/23' 2023-10-06T04:03:42.369 INFO:tasks.workunit.client.0.smithi157.stdout:removed './298/35' 2023-10-06T04:03:42.369 INFO:tasks.workunit.client.0.smithi157.stdout:removed './298/26' 2023-10-06T04:03:42.369 INFO:tasks.workunit.client.0.smithi157.stdout:removed './298/37' 2023-10-06T04:03:42.369 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './298' 2023-10-06T04:03:42.369 INFO:tasks.workunit.client.0.smithi157.stdout:removed './299/21' 2023-10-06T04:03:42.369 INFO:tasks.workunit.client.0.smithi157.stdout:removed './299/27' 2023-10-06T04:03:42.369 INFO:tasks.workunit.client.0.smithi157.stdout:removed './299/31' 2023-10-06T04:03:42.369 INFO:tasks.workunit.client.0.smithi157.stdout:removed './299/36' 2023-10-06T04:03:42.370 INFO:tasks.workunit.client.0.smithi157.stdout:removed './299/20' 2023-10-06T04:03:42.370 INFO:tasks.workunit.client.0.smithi157.stdout:removed './299/39' 2023-10-06T04:03:42.370 INFO:tasks.workunit.client.0.smithi157.stdout:removed './299/34' 2023-10-06T04:03:42.370 INFO:tasks.workunit.client.0.smithi157.stdout:removed './299/38' 2023-10-06T04:03:42.370 INFO:tasks.workunit.client.0.smithi157.stdout:removed './299/28' 2023-10-06T04:03:42.370 INFO:tasks.workunit.client.0.smithi157.stdout:removed './299/24' 2023-10-06T04:03:42.370 INFO:tasks.workunit.client.0.smithi157.stdout:removed './299/29' 2023-10-06T04:03:42.370 INFO:tasks.workunit.client.0.smithi157.stdout:removed './299/32' 2023-10-06T04:03:42.371 INFO:tasks.workunit.client.0.smithi157.stdout:removed './299/22' 2023-10-06T04:03:42.371 INFO:tasks.workunit.client.0.smithi157.stdout:removed './299/25' 2023-10-06T04:03:42.371 INFO:tasks.workunit.client.0.smithi157.stdout:removed './299/30' 2023-10-06T04:03:42.371 INFO:tasks.workunit.client.0.smithi157.stdout:removed './299/33' 2023-10-06T04:03:42.371 INFO:tasks.workunit.client.0.smithi157.stdout:removed './299/23' 2023-10-06T04:03:42.371 INFO:tasks.workunit.client.0.smithi157.stdout:removed './299/35' 2023-10-06T04:03:42.371 INFO:tasks.workunit.client.0.smithi157.stdout:removed './299/26' 2023-10-06T04:03:42.371 INFO:tasks.workunit.client.0.smithi157.stdout:removed './299/37' 2023-10-06T04:03:42.372 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './299' 2023-10-06T04:03:42.372 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/166/19' 2023-10-06T04:03:42.372 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/166/15' 2023-10-06T04:03:42.372 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/166/20' 2023-10-06T04:03:42.372 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/166/12' 2023-10-06T04:03:42.372 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/166/10' 2023-10-06T04:03:42.372 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/166/11' 2023-10-06T04:03:42.372 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/166/16' 2023-10-06T04:03:42.372 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/166/14' 2023-10-06T04:03:42.373 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/166/18' 2023-10-06T04:03:42.373 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/166/13' 2023-10-06T04:03:42.373 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/166/17' 2023-10-06T04:03:42.373 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/166' 2023-10-06T04:03:42.373 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/147/19' 2023-10-06T04:03:42.373 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/147/15' 2023-10-06T04:03:42.373 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/147/20' 2023-10-06T04:03:42.373 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/147/12' 2023-10-06T04:03:42.374 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/147/10' 2023-10-06T04:03:42.374 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/147/11' 2023-10-06T04:03:42.374 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/147/16' 2023-10-06T04:03:42.374 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/147/14' 2023-10-06T04:03:42.374 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/147/18' 2023-10-06T04:03:42.374 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/147/13' 2023-10-06T04:03:42.374 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/147/17' 2023-10-06T04:03:42.374 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/147' 2023-10-06T04:03:42.375 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/151/19' 2023-10-06T04:03:42.375 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/151/15' 2023-10-06T04:03:42.375 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/151/20' 2023-10-06T04:03:42.375 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/151/12' 2023-10-06T04:03:42.375 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/151/10' 2023-10-06T04:03:42.375 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/151/11' 2023-10-06T04:03:42.375 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/151/16' 2023-10-06T04:03:42.376 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/151/14' 2023-10-06T04:03:42.376 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/151/18' 2023-10-06T04:03:42.376 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/151/13' 2023-10-06T04:03:42.376 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/151/17' 2023-10-06T04:03:42.376 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/151' 2023-10-06T04:03:42.376 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/170/19' 2023-10-06T04:03:42.376 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/170/15' 2023-10-06T04:03:42.376 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/170/20' 2023-10-06T04:03:42.377 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/170/12' 2023-10-06T04:03:42.377 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/170/10' 2023-10-06T04:03:42.377 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/170/11' 2023-10-06T04:03:42.377 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/170/16' 2023-10-06T04:03:42.377 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/170/14' 2023-10-06T04:03:42.377 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/170/18' 2023-10-06T04:03:42.377 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/170/13' 2023-10-06T04:03:42.378 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/170/17' 2023-10-06T04:03:42.378 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/170' 2023-10-06T04:03:42.378 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/167/19' 2023-10-06T04:03:42.378 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/167/15' 2023-10-06T04:03:42.378 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/167/20' 2023-10-06T04:03:42.378 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/167/12' 2023-10-06T04:03:42.378 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/167/10' 2023-10-06T04:03:42.378 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/167/11' 2023-10-06T04:03:42.379 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/167/16' 2023-10-06T04:03:42.379 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/167/14' 2023-10-06T04:03:42.379 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/167/18' 2023-10-06T04:03:42.379 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/167/13' 2023-10-06T04:03:42.379 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/167/17' 2023-10-06T04:03:42.379 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/167' 2023-10-06T04:03:42.380 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/140/19' 2023-10-06T04:03:42.380 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/140/15' 2023-10-06T04:03:42.380 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/140/20' 2023-10-06T04:03:42.380 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/140/12' 2023-10-06T04:03:42.380 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/140/10' 2023-10-06T04:03:42.380 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/140/11' 2023-10-06T04:03:42.380 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/140/16' 2023-10-06T04:03:42.380 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/140/14' 2023-10-06T04:03:42.381 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/140/18' 2023-10-06T04:03:42.381 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/140/13' 2023-10-06T04:03:42.381 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/140/17' 2023-10-06T04:03:42.381 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/140' 2023-10-06T04:03:42.381 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/142/19' 2023-10-06T04:03:42.381 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/142/15' 2023-10-06T04:03:42.381 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/142/20' 2023-10-06T04:03:42.381 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/142/12' 2023-10-06T04:03:42.382 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/142/10' 2023-10-06T04:03:43.113 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:03:43.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.268+0000 7f390ed14700 1 Processor -- start 2023-10-06T04:03:43.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.268+0000 7f390ed14700 1 -- start start 2023-10-06T04:03:43.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.268+0000 7f390ed14700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f390806c260 0x7f3908155db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:43.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.268+0000 7f390ed14700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39080b20f0 0x7f39081562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:43.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.268+0000 7f390ed14700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f390806e590 con 0x7f39080b1d20 2023-10-06T04:03:43.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.268+0000 7f390ed14700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3908156d20 con 0x7f390806c260 2023-10-06T04:03:43.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.268+0000 7f390ed14700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3908156830 con 0x7f39080b20f0 2023-10-06T04:03:43.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.269+0000 7f3907fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f390806c260 0x7f3908155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:43.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.269+0000 7f390cab0700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f39080b1d20 0x7f39080af370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43544/0 (socket says 172.21.15.157:43544) 2023-10-06T04:03:43.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.269+0000 7f390cab0700 1 -- 172.21.15.157:0/2810987555 learned_addr learned my addr 172.21.15.157:0/2810987555 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:03:43.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.269+0000 7f390d2b1700 1 --2- 172.21.15.157:0/2810987555 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39080b20f0 0x7f39081562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:43.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.269+0000 7f3907fff700 1 -- 172.21.15.157:0/2810987555 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39080b20f0 msgr2=0x7f39081562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:43.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.269+0000 7f3907fff700 1 --2- 172.21.15.157:0/2810987555 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39080b20f0 0x7f39081562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:43.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.269+0000 7f3907fff700 1 -- 172.21.15.157:0/2810987555 >> v1:172.21.15.157:6789/0 conn(0x7f39080b1d20 legacy=0x7f39080af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:43.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.269+0000 7f3907fff700 1 -- 172.21.15.157:0/2810987555 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3908156a40 con 0x7f390806c260 2023-10-06T04:03:43.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.269+0000 7f39077fe700 1 -- 172.21.15.157:0/2810987555 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2150283188 0 0) 0x7f390806e590 con 0x7f39080b1d20 2023-10-06T04:03:43.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.269+0000 7f3907fff700 1 --2- 172.21.15.157:0/2810987555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f390806c260 0x7f3908155db0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f38f40098e0 tx=0x7f38f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab295bfba440abb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:43.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.269+0000 7f39077fe700 1 -- 172.21.15.157:0/2810987555 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f400b640 con 0x7f390806c260 2023-10-06T04:03:43.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.269+0000 7f39077fe700 1 -- 172.21.15.157:0/2810987555 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f400b7a0 con 0x7f390806c260 2023-10-06T04:03:43.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.269+0000 7f39077fe700 1 -- 172.21.15.157:0/2810987555 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f400bb40 con 0x7f390806c260 2023-10-06T04:03:43.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.270+0000 7f390ed14700 1 -- 172.21.15.157:0/2810987555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f390806c260 msgr2=0x7f3908155db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:43.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.270+0000 7f390ed14700 1 --2- 172.21.15.157:0/2810987555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f390806c260 0x7f3908155db0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f38f40098e0 tx=0x7f38f40091a0 comp rx=0 tx=0).stop 2023-10-06T04:03:43.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.270+0000 7f390ed14700 1 -- 172.21.15.157:0/2810987555 shutdown_connections 2023-10-06T04:03:43.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.270+0000 7f390ed14700 1 --2- 172.21.15.157:0/2810987555 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39080b20f0 0x7f39081562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:43.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.270+0000 7f390ed14700 1 --2- 172.21.15.157:0/2810987555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f390806c260 0x7f3908155db0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:43.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.270+0000 7f390ed14700 1 -- 172.21.15.157:0/2810987555 >> 172.21.15.157:0/2810987555 conn(0x7f39080cc6e0 msgr2=0x7f39080ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:43.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.270+0000 7f390ed14700 1 -- 172.21.15.157:0/2810987555 shutdown_connections 2023-10-06T04:03:43.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.270+0000 7f390ed14700 1 -- 172.21.15.157:0/2810987555 wait complete. 2023-10-06T04:03:43.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.272+0000 7f390ed14700 1 Processor -- start 2023-10-06T04:03:43.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.272+0000 7f390ed14700 1 -- start start 2023-10-06T04:03:43.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.273+0000 7f390ed14700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f390806c260 0x7f39080c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:43.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.273+0000 7f390ed14700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39080b1d20 0x7f3908151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:43.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.273+0000 7f390ed14700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39080b20f0 0x7f3908152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:43.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.273+0000 7f390ed14700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39081514d0 con 0x7f390806c260 2023-10-06T04:03:43.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.273+0000 7f390ed14700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390816ad20 con 0x7f39080b1d20 2023-10-06T04:03:43.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.273+0000 7f390ed14700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f390816e450 con 0x7f39080b20f0 2023-10-06T04:03:43.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.273+0000 7f390cab0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f390806c260 0x7f39080c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:43.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.273+0000 7f390cab0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f390806c260 0x7f39080c61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50512/0 (socket says 172.21.15.157:50512) 2023-10-06T04:03:43.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.273+0000 7f390cab0700 1 -- 172.21.15.157:0/1965350668 learned_addr learned my addr 172.21.15.157:0/1965350668 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:43.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.273+0000 7f3907fff700 1 --2- 172.21.15.157:0/1965350668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39080b1d20 0x7f3908151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:43.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.273+0000 7f390d2b1700 1 --2- 172.21.15.157:0/1965350668 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39080b20f0 0x7f3908152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:43.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.274+0000 7f390cab0700 1 -- 172.21.15.157:0/1965350668 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39080b20f0 msgr2=0x7f3908152110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:43.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.274+0000 7f390cab0700 1 --2- 172.21.15.157:0/1965350668 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39080b20f0 0x7f3908152110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:43.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.274+0000 7f390cab0700 1 -- 172.21.15.157:0/1965350668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39080b1d20 msgr2=0x7f3908151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:43.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.274+0000 7f390cab0700 1 --2- 172.21.15.157:0/1965350668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39080b1d20 0x7f3908151bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:43.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.274+0000 7f390cab0700 1 -- 172.21.15.157:0/1965350668 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f4009420 con 0x7f390806c260 2023-10-06T04:03:43.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.274+0000 7f390cab0700 1 --2- 172.21.15.157:0/1965350668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f390806c260 0x7f39080c61a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00c7e0 tx=0x7f38fc007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:43.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.274+0000 7f3904ff9700 1 -- 172.21.15.157:0/1965350668 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc005b30 con 0x7f390806c260 2023-10-06T04:03:43.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.274+0000 7f3904ff9700 1 -- 172.21.15.157:0/1965350668 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38fc005c90 con 0x7f390806c260 2023-10-06T04:03:43.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.274+0000 7f390ed14700 1 -- 172.21.15.157:0/1965350668 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390816a110 con 0x7f390806c260 2023-10-06T04:03:43.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.274+0000 7f3904ff9700 1 -- 172.21.15.157:0/1965350668 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc007300 con 0x7f390806c260 2023-10-06T04:03:43.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.274+0000 7f390ed14700 1 -- 172.21.15.157:0/1965350668 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390816a6d0 con 0x7f390806c260 2023-10-06T04:03:43.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.275+0000 7f390ed14700 1 -- 172.21.15.157:0/1965350668 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39080896c0 con 0x7f390806c260 2023-10-06T04:03:43.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.276+0000 7f3904ff9700 1 -- 172.21.15.157:0/1965350668 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f38fc0076b0 con 0x7f390806c260 2023-10-06T04:03:43.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.277+0000 7f3904ff9700 1 --2- 172.21.15.157:0/1965350668 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f38ec066f30 0x7f38ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:43.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.277+0000 7f3904ff9700 1 -- 172.21.15.157:0/1965350668 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7f38fc0841b0 con 0x7f390806c260 2023-10-06T04:03:43.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.277+0000 7f3907fff700 1 --2- 172.21.15.157:0/1965350668 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f38ec066f30 0x7f38ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:43.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.277+0000 7f3907fff700 1 --2- 172.21.15.157:0/1965350668 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f38ec066f30 0x7f38ec0693e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f39081558f0 tx=0x7f38f4017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:43.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.283+0000 7f3904ff9700 1 -- 172.21.15.157:0/1965350668 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f38fc0510f0 con 0x7f390806c260 2023-10-06T04:03:43.385 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/142/11' 2023-10-06T04:03:43.385 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/142/16' 2023-10-06T04:03:43.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/142/14' 2023-10-06T04:03:43.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/142/18' 2023-10-06T04:03:43.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/142/13' 2023-10-06T04:03:43.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/142/17' 2023-10-06T04:03:43.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/142' 2023-10-06T04:03:43.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/150/19' 2023-10-06T04:03:43.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/150/15' 2023-10-06T04:03:43.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/150/20' 2023-10-06T04:03:43.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/150/12' 2023-10-06T04:03:43.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/150/10' 2023-10-06T04:03:43.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/150/11' 2023-10-06T04:03:43.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/150/16' 2023-10-06T04:03:43.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/150/14' 2023-10-06T04:03:43.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/150/18' 2023-10-06T04:03:43.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/150/13' 2023-10-06T04:03:43.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/150/17' 2023-10-06T04:03:43.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/150' 2023-10-06T04:03:43.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/105/19' 2023-10-06T04:03:43.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/105/15' 2023-10-06T04:03:43.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/105/20' 2023-10-06T04:03:43.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/105/12' 2023-10-06T04:03:43.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/105/10' 2023-10-06T04:03:43.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/105/11' 2023-10-06T04:03:43.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/105/16' 2023-10-06T04:03:43.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/105/14' 2023-10-06T04:03:43.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/105/18' 2023-10-06T04:03:43.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/105/13' 2023-10-06T04:03:43.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/105/17' 2023-10-06T04:03:43.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/105' 2023-10-06T04:03:43.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/149/19' 2023-10-06T04:03:43.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/149/15' 2023-10-06T04:03:43.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/149/20' 2023-10-06T04:03:43.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/149/12' 2023-10-06T04:03:43.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/149/10' 2023-10-06T04:03:43.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/149/11' 2023-10-06T04:03:43.389 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/149/16' 2023-10-06T04:03:43.389 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/149/14' 2023-10-06T04:03:43.389 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/149/18' 2023-10-06T04:03:43.389 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/149/13' 2023-10-06T04:03:43.389 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/149/17' 2023-10-06T04:03:43.389 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/149' 2023-10-06T04:03:43.389 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/109/19' 2023-10-06T04:03:43.389 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/109/15' 2023-10-06T04:03:43.389 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/109/20' 2023-10-06T04:03:43.389 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/109/12' 2023-10-06T04:03:43.389 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/109/10' 2023-10-06T04:03:43.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/109/11' 2023-10-06T04:03:43.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/109/16' 2023-10-06T04:03:43.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/109/14' 2023-10-06T04:03:43.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/109/18' 2023-10-06T04:03:43.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/109/13' 2023-10-06T04:03:43.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/109/17' 2023-10-06T04:03:43.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/109' 2023-10-06T04:03:43.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/158/19' 2023-10-06T04:03:43.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/158/15' 2023-10-06T04:03:43.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/158/20' 2023-10-06T04:03:43.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/158/12' 2023-10-06T04:03:43.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/158/10' 2023-10-06T04:03:43.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/158/11' 2023-10-06T04:03:43.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/158/16' 2023-10-06T04:03:43.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/158/14' 2023-10-06T04:03:43.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/158/18' 2023-10-06T04:03:43.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/158/13' 2023-10-06T04:03:43.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/158/17' 2023-10-06T04:03:43.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/158' 2023-10-06T04:03:43.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/106/19' 2023-10-06T04:03:43.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/106/15' 2023-10-06T04:03:43.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/106/20' 2023-10-06T04:03:43.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/106/12' 2023-10-06T04:03:43.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/106/10' 2023-10-06T04:03:43.392 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/106/11' 2023-10-06T04:03:43.392 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/106/16' 2023-10-06T04:03:43.392 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/106/14' 2023-10-06T04:03:43.392 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/106/18' 2023-10-06T04:03:43.392 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/106/13' 2023-10-06T04:03:43.392 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/106/17' 2023-10-06T04:03:43.392 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/106' 2023-10-06T04:03:43.392 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/189/19' 2023-10-06T04:03:43.392 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/189/15' 2023-10-06T04:03:43.392 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/189/20' 2023-10-06T04:03:43.392 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/189/12' 2023-10-06T04:03:43.393 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/189/10' 2023-10-06T04:03:43.393 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/189/11' 2023-10-06T04:03:43.393 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/189/16' 2023-10-06T04:03:43.393 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/189/14' 2023-10-06T04:03:43.393 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/189/18' 2023-10-06T04:03:43.393 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/189/13' 2023-10-06T04:03:43.393 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/189/17' 2023-10-06T04:03:43.393 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/189' 2023-10-06T04:03:43.393 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/133/19' 2023-10-06T04:03:43.393 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/133/15' 2023-10-06T04:03:43.393 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/133/20' 2023-10-06T04:03:43.393 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/133/12' 2023-10-06T04:03:43.394 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/133/10' 2023-10-06T04:03:43.394 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/133/11' 2023-10-06T04:03:43.394 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/133/16' 2023-10-06T04:03:43.394 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/133/14' 2023-10-06T04:03:43.394 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/133/18' 2023-10-06T04:03:43.394 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/133/13' 2023-10-06T04:03:43.394 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/133/17' 2023-10-06T04:03:43.394 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/133' 2023-10-06T04:03:43.394 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/173/19' 2023-10-06T04:03:43.394 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/173/15' 2023-10-06T04:03:43.394 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/173/20' 2023-10-06T04:03:43.394 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/173/12' 2023-10-06T04:03:43.395 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/173/10' 2023-10-06T04:03:43.395 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/173/11' 2023-10-06T04:03:43.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.552+0000 7f390ed14700 1 -- 172.21.15.157:0/1965350668 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f390806d7d0 con 0x7f390806c260 2023-10-06T04:03:43.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.553+0000 7f3904ff9700 1 -- 172.21.15.157:0/1965350668 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+6146 (secure 0 0 0) 0x7f38fc001d60 con 0x7f390806c260 2023-10-06T04:03:43.556 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:03:43.556 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3192541484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3192541484},{"type":"v1","addr":"172.21.15.175:6835","nonce":3192541484}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:41.773401+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":4287},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:03:43.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.555+0000 7f390ed14700 1 -- 172.21.15.157:0/1965350668 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f38ec066f30 msgr2=0x7f38ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:43.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.555+0000 7f390ed14700 1 --2- 172.21.15.157:0/1965350668 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f38ec066f30 0x7f38ec0693e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f39081558f0 tx=0x7f38f4017000 comp rx=0 tx=0).stop 2023-10-06T04:03:43.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.555+0000 7f390ed14700 1 -- 172.21.15.157:0/1965350668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f390806c260 msgr2=0x7f39080c61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:43.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.555+0000 7f390ed14700 1 --2- 172.21.15.157:0/1965350668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f390806c260 0x7f39080c61a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00c7e0 tx=0x7f38fc007df0 comp rx=0 tx=0).stop 2023-10-06T04:03:43.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.555+0000 7f390ed14700 1 -- 172.21.15.157:0/1965350668 shutdown_connections 2023-10-06T04:03:43.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.555+0000 7f390ed14700 1 --2- 172.21.15.157:0/1965350668 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f38ec066f30 0x7f38ec0693e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:43.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.555+0000 7f390ed14700 1 --2- 172.21.15.157:0/1965350668 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39080b20f0 0x7f3908152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:43.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.555+0000 7f390ed14700 1 --2- 172.21.15.157:0/1965350668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f390806c260 0x7f39080c61a0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:43.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.555+0000 7f390ed14700 1 --2- 172.21.15.157:0/1965350668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39080b1d20 0x7f3908151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:43.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.555+0000 7f390ed14700 1 -- 172.21.15.157:0/1965350668 >> 172.21.15.157:0/1965350668 conn(0x7f39080cc6e0 msgr2=0x7f39080af0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:43.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.555+0000 7f390ed14700 1 -- 172.21.15.157:0/1965350668 shutdown_connections 2023-10-06T04:03:43.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:43.555+0000 7f390ed14700 1 -- 172.21.15.157:0/1965350668 wait complete. 2023-10-06T04:03:43.559 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 54 2023-10-06T04:03:43.569 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-10-06T04:03:44.389 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/173/16' 2023-10-06T04:03:44.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/173/14' 2023-10-06T04:03:44.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/173/18' 2023-10-06T04:03:44.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/173/13' 2023-10-06T04:03:44.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/173/17' 2023-10-06T04:03:44.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/173' 2023-10-06T04:03:44.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/174/19' 2023-10-06T04:03:44.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/174/15' 2023-10-06T04:03:44.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/174/20' 2023-10-06T04:03:44.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/174/12' 2023-10-06T04:03:44.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/174/10' 2023-10-06T04:03:44.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/174/11' 2023-10-06T04:03:44.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/174/16' 2023-10-06T04:03:44.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/174/14' 2023-10-06T04:03:44.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/174/18' 2023-10-06T04:03:44.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/174/13' 2023-10-06T04:03:44.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/174/17' 2023-10-06T04:03:44.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/174' 2023-10-06T04:03:44.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/155/19' 2023-10-06T04:03:44.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/155/15' 2023-10-06T04:03:44.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/155/20' 2023-10-06T04:03:44.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/155/12' 2023-10-06T04:03:44.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/155/10' 2023-10-06T04:03:44.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/155/11' 2023-10-06T04:03:44.392 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/155/16' 2023-10-06T04:03:44.392 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/155/14' 2023-10-06T04:03:44.392 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/155/18' 2023-10-06T04:03:44.392 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/155/13' 2023-10-06T04:03:44.392 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/155/17' 2023-10-06T04:03:44.392 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/155' 2023-10-06T04:03:44.392 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/114/19' 2023-10-06T04:03:44.392 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/114/15' 2023-10-06T04:03:44.392 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/114/20' 2023-10-06T04:03:44.392 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/114/12' 2023-10-06T04:03:44.392 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/114/10' 2023-10-06T04:03:44.392 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/114/11' 2023-10-06T04:03:44.393 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/114/16' 2023-10-06T04:03:44.393 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/114/14' 2023-10-06T04:03:44.393 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/114/18' 2023-10-06T04:03:44.393 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/114/13' 2023-10-06T04:03:44.393 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/114/17' 2023-10-06T04:03:44.393 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/114' 2023-10-06T04:03:44.393 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/159/19' 2023-10-06T04:03:44.393 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/159/15' 2023-10-06T04:03:44.393 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/159/20' 2023-10-06T04:03:44.393 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/159/12' 2023-10-06T04:03:44.393 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/159/10' 2023-10-06T04:03:44.393 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/159/11' 2023-10-06T04:03:44.394 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/159/16' 2023-10-06T04:03:44.394 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/159/14' 2023-10-06T04:03:44.394 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/159/18' 2023-10-06T04:03:44.394 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/159/13' 2023-10-06T04:03:44.394 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/159/17' 2023-10-06T04:03:44.394 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/159' 2023-10-06T04:03:44.394 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/187/19' 2023-10-06T04:03:44.394 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/187/15' 2023-10-06T04:03:44.394 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/187/20' 2023-10-06T04:03:44.394 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/187/12' 2023-10-06T04:03:44.394 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/187/10' 2023-10-06T04:03:44.395 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/187/11' 2023-10-06T04:03:44.395 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/187/16' 2023-10-06T04:03:44.395 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/187/14' 2023-10-06T04:03:44.395 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/187/18' 2023-10-06T04:03:44.395 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/187/13' 2023-10-06T04:03:44.395 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/187/17' 2023-10-06T04:03:44.395 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/187' 2023-10-06T04:03:44.395 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/185/19' 2023-10-06T04:03:44.395 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/185/15' 2023-10-06T04:03:44.395 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/185/20' 2023-10-06T04:03:44.395 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/185/12' 2023-10-06T04:03:44.395 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/185/10' 2023-10-06T04:03:44.396 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/185/11' 2023-10-06T04:03:44.396 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/185/16' 2023-10-06T04:03:44.396 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/185/14' 2023-10-06T04:03:44.396 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/185/18' 2023-10-06T04:03:44.396 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/185/13' 2023-10-06T04:03:44.396 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/185/17' 2023-10-06T04:03:44.396 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/185' 2023-10-06T04:03:44.396 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/120/19' 2023-10-06T04:03:44.396 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/120/15' 2023-10-06T04:03:44.396 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/120/20' 2023-10-06T04:03:44.396 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/120/12' 2023-10-06T04:03:44.396 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/120/10' 2023-10-06T04:03:44.397 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/120/11' 2023-10-06T04:03:44.397 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/120/16' 2023-10-06T04:03:44.397 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/120/14' 2023-10-06T04:03:44.397 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/120/18' 2023-10-06T04:03:44.397 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/120/13' 2023-10-06T04:03:44.397 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/120/17' 2023-10-06T04:03:44.397 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/120' 2023-10-06T04:03:44.397 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/110/19' 2023-10-06T04:03:44.397 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/110/15' 2023-10-06T04:03:44.397 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/110/20' 2023-10-06T04:03:44.397 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/110/12' 2023-10-06T04:03:44.397 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/110/10' 2023-10-06T04:03:44.398 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/110/11' 2023-10-06T04:03:44.398 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/110/16' 2023-10-06T04:03:44.398 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/110/14' 2023-10-06T04:03:44.398 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/110/18' 2023-10-06T04:03:44.398 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/110/13' 2023-10-06T04:03:44.398 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/110/17' 2023-10-06T04:03:44.398 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/110' 2023-10-06T04:03:44.398 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/137/19' 2023-10-06T04:03:44.398 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/137/15' 2023-10-06T04:03:44.398 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/137/20' 2023-10-06T04:03:44.398 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/137/12' 2023-10-06T04:03:44.398 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/137/10' 2023-10-06T04:03:44.399 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/137/11' 2023-10-06T04:03:44.399 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/137/16' 2023-10-06T04:03:44.847 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/137/14' 2023-10-06T04:03:44.848 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/137/18' 2023-10-06T04:03:44.848 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/137/13' 2023-10-06T04:03:44.848 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/137/17' 2023-10-06T04:03:44.848 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/137' 2023-10-06T04:03:44.848 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/198/19' 2023-10-06T04:03:44.849 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/198/15' 2023-10-06T04:03:44.849 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/198/20' 2023-10-06T04:03:44.849 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/198/12' 2023-10-06T04:03:44.849 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/198/10' 2023-10-06T04:03:44.849 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/198/11' 2023-10-06T04:03:44.849 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/198/16' 2023-10-06T04:03:44.849 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/198/14' 2023-10-06T04:03:44.849 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/198/18' 2023-10-06T04:03:44.849 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/198/13' 2023-10-06T04:03:44.849 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/198/17' 2023-10-06T04:03:44.849 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/198' 2023-10-06T04:03:44.850 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/136/19' 2023-10-06T04:03:44.850 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/136/15' 2023-10-06T04:03:44.850 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/136/20' 2023-10-06T04:03:44.850 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/136/12' 2023-10-06T04:03:44.850 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/136/10' 2023-10-06T04:03:44.850 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/136/11' 2023-10-06T04:03:44.850 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/136/16' 2023-10-06T04:03:44.851 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/136/14' 2023-10-06T04:03:44.851 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/136/18' 2023-10-06T04:03:44.851 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/136/13' 2023-10-06T04:03:44.851 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/136/17' 2023-10-06T04:03:44.851 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/136' 2023-10-06T04:03:44.851 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/125/19' 2023-10-06T04:03:44.851 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/125/15' 2023-10-06T04:03:44.851 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/125/20' 2023-10-06T04:03:44.851 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/125/12' 2023-10-06T04:03:44.852 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/125/10' 2023-10-06T04:03:44.852 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/125/11' 2023-10-06T04:03:44.852 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/125/16' 2023-10-06T04:03:44.852 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/125/14' 2023-10-06T04:03:44.852 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/125/18' 2023-10-06T04:03:44.852 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/125/13' 2023-10-06T04:03:44.852 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/125/17' 2023-10-06T04:03:44.852 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/125' 2023-10-06T04:03:44.852 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/194/19' 2023-10-06T04:03:44.852 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/194/15' 2023-10-06T04:03:44.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/194/20' 2023-10-06T04:03:44.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/194/12' 2023-10-06T04:03:44.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/194/10' 2023-10-06T04:03:44.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/194/11' 2023-10-06T04:03:44.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/194/16' 2023-10-06T04:03:44.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/194/14' 2023-10-06T04:03:44.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/194/18' 2023-10-06T04:03:44.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/194/13' 2023-10-06T04:03:44.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/194/17' 2023-10-06T04:03:44.853 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/194' 2023-10-06T04:03:44.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/119/19' 2023-10-06T04:03:44.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/119/15' 2023-10-06T04:03:44.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/119/20' 2023-10-06T04:03:44.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/119/12' 2023-10-06T04:03:44.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/119/10' 2023-10-06T04:03:44.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/119/11' 2023-10-06T04:03:44.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/119/16' 2023-10-06T04:03:44.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/119/14' 2023-10-06T04:03:44.854 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/119/18' 2023-10-06T04:03:44.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/119/13' 2023-10-06T04:03:44.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/119/17' 2023-10-06T04:03:44.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/119' 2023-10-06T04:03:44.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/157/19' 2023-10-06T04:03:44.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/157/15' 2023-10-06T04:03:44.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/157/20' 2023-10-06T04:03:44.855 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/157/12' 2023-10-06T04:03:44.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/157/10' 2023-10-06T04:03:44.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/157/11' 2023-10-06T04:03:44.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/157/16' 2023-10-06T04:03:44.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/157/14' 2023-10-06T04:03:44.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/157/18' 2023-10-06T04:03:44.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/157/13' 2023-10-06T04:03:44.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/157/17' 2023-10-06T04:03:44.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/157' 2023-10-06T04:03:44.856 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/172/19' 2023-10-06T04:03:44.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/172/15' 2023-10-06T04:03:44.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/172/20' 2023-10-06T04:03:44.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/172/12' 2023-10-06T04:03:44.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/172/10' 2023-10-06T04:03:44.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/172/11' 2023-10-06T04:03:44.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/172/16' 2023-10-06T04:03:44.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/172/14' 2023-10-06T04:03:44.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/172/18' 2023-10-06T04:03:44.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/172/13' 2023-10-06T04:03:44.857 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/172/17' 2023-10-06T04:03:44.858 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/172' 2023-10-06T04:03:44.858 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/186/19' 2023-10-06T04:03:44.858 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/186/15' 2023-10-06T04:03:44.858 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/186/20' 2023-10-06T04:03:44.858 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/186/12' 2023-10-06T04:03:44.858 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/186/10' 2023-10-06T04:03:44.858 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/186/11' 2023-10-06T04:03:44.858 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/186/16' 2023-10-06T04:03:44.858 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/186/14' 2023-10-06T04:03:44.858 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/186/18' 2023-10-06T04:03:44.858 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/186/13' 2023-10-06T04:03:44.859 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/186/17' 2023-10-06T04:03:44.859 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/186' 2023-10-06T04:03:44.859 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/190/19' 2023-10-06T04:03:44.859 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/190/15' 2023-10-06T04:03:44.859 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/190/20' 2023-10-06T04:03:44.859 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/190/12' 2023-10-06T04:03:44.859 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/190/10' 2023-10-06T04:03:44.859 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/190/11' 2023-10-06T04:03:44.859 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/190/16' 2023-10-06T04:03:44.859 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/190/14' 2023-10-06T04:03:44.953 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:03:44.957 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:03:45.573 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:03:45.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.687+0000 7ff5fa477700 1 Processor -- start 2023-10-06T04:03:45.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.687+0000 7ff5fa477700 1 -- start start 2023-10-06T04:03:45.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.688+0000 7ff5fa477700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff5f406dfc0 0x7ff5f406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:45.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.688+0000 7ff5fa477700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5f40c6160 con 0x7ff5f406a780 2023-10-06T04:03:45.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.688+0000 7ff5fa477700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5f4069040 con 0x7ff5f406ab50 2023-10-06T04:03:45.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.688+0000 7ff5fa477700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5f40c5c00 con 0x7ff5f406dfc0 2023-10-06T04:03:45.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.688+0000 7ff5f8a14700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff5f406dfc0 0x7ff5f406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:45.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.688+0000 7ff5f3fff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff5f406a780 0x7ff5f40c65f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43546/0 (socket says 172.21.15.157:43546) 2023-10-06T04:03:45.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.688+0000 7ff5f3fff700 1 -- 172.21.15.157:0/3161721764 learned_addr learned my addr 172.21.15.157:0/3161721764 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:03:45.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.688+0000 7ff5f8a14700 1 -- 172.21.15.157:0/3161721764 >> v1:172.21.15.175:6789/0 conn(0x7ff5f406ab50 legacy=0x7ff5f406d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:45.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.688+0000 7ff5f8a14700 1 -- 172.21.15.157:0/3161721764 >> v1:172.21.15.157:6789/0 conn(0x7ff5f406a780 legacy=0x7ff5f40c65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:45.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.688+0000 7ff5f8a14700 1 -- 172.21.15.157:0/3161721764 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f4153930 con 0x7ff5f406dfc0 2023-10-06T04:03:45.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.688+0000 7ff5f2ffd700 1 -- 172.21.15.157:0/3161721764 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2955317036 0 0) 0x7ff5f40c6160 con 0x7ff5f406a780 2023-10-06T04:03:45.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.689+0000 7ff5f8a14700 1 --2- 172.21.15.157:0/3161721764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5f406dfc0 0x7ff5f406e3c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009e60 tx=0x7ff5ec009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ff99d80f9fd302 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:45.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.689+0000 7ff5f2ffd700 1 -- 172.21.15.157:0/3161721764 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec00e070 con 0x7ff5f406dfc0 2023-10-06T04:03:45.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.689+0000 7ff5f2ffd700 1 -- 172.21.15.157:0/3161721764 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ec00bc80 con 0x7ff5f406dfc0 2023-10-06T04:03:45.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.689+0000 7ff5f2ffd700 1 -- 172.21.15.157:0/3161721764 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec012430 con 0x7ff5f406dfc0 2023-10-06T04:03:45.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.689+0000 7ff5fa477700 1 -- 172.21.15.157:0/3161721764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5f406dfc0 msgr2=0x7ff5f406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:45.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.689+0000 7ff5fa477700 1 --2- 172.21.15.157:0/3161721764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5f406dfc0 0x7ff5f406e3c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009e60 tx=0x7ff5ec009780 comp rx=0 tx=0).stop 2023-10-06T04:03:45.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.689+0000 7ff5fa477700 1 -- 172.21.15.157:0/3161721764 shutdown_connections 2023-10-06T04:03:45.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.689+0000 7ff5fa477700 1 --2- 172.21.15.157:0/3161721764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5f406dfc0 0x7ff5f406e3c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:45.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.689+0000 7ff5fa477700 1 -- 172.21.15.157:0/3161721764 >> 172.21.15.157:0/3161721764 conn(0x7ff5f406b510 msgr2=0x7ff5f406b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:45.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.689+0000 7ff5fa477700 1 -- 172.21.15.157:0/3161721764 shutdown_connections 2023-10-06T04:03:45.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.689+0000 7ff5fa477700 1 -- 172.21.15.157:0/3161721764 wait complete. 2023-10-06T04:03:45.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.690+0000 7ff5fa477700 1 Processor -- start 2023-10-06T04:03:45.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.690+0000 7ff5fa477700 1 -- start start 2023-10-06T04:03:45.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.691+0000 7ff5fa477700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5f406a780 0x7ff5f40c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:45.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.691+0000 7ff5fa477700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5f406ab50 0x7ff5f4163110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:45.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.691+0000 7ff5fa477700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5f406dfc0 0x7ff5f414cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:45.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.691+0000 7ff5fa477700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f4156780 con 0x7ff5f406dfc0 2023-10-06T04:03:45.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.691+0000 7ff5fa477700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f414d240 con 0x7ff5f406ab50 2023-10-06T04:03:45.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.691+0000 7ff5fa477700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f414c870 con 0x7ff5f406a780 2023-10-06T04:03:45.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.691+0000 7ff5f8a14700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5f406dfc0 0x7ff5f414cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:45.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.691+0000 7ff5f8a14700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5f406dfc0 0x7ff5f414cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50516/0 (socket says 172.21.15.157:50516) 2023-10-06T04:03:45.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.691+0000 7ff5f8a14700 1 -- 172.21.15.157:0/3435532105 learned_addr learned my addr 172.21.15.157:0/3435532105 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:45.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.691+0000 7ff5f37fe700 1 --2- 172.21.15.157:0/3435532105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5f406ab50 0x7ff5f4163110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:45.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.691+0000 7ff5f3fff700 1 --2- 172.21.15.157:0/3435532105 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5f406a780 0x7ff5f40c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:45.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.692+0000 7ff5f8a14700 1 -- 172.21.15.157:0/3435532105 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5f406a780 msgr2=0x7ff5f40c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:45.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.692+0000 7ff5f8a14700 1 --2- 172.21.15.157:0/3435532105 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5f406a780 0x7ff5f40c39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:45.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.692+0000 7ff5f8a14700 1 -- 172.21.15.157:0/3435532105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5f406ab50 msgr2=0x7ff5f4163110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:45.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.692+0000 7ff5f8a14700 1 --2- 172.21.15.157:0/3435532105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5f406ab50 0x7ff5f4163110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:45.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.692+0000 7ff5f8a14700 1 -- 172.21.15.157:0/3435532105 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ec009a20 con 0x7ff5f406dfc0 2023-10-06T04:03:45.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.692+0000 7ff5f8a14700 1 --2- 172.21.15.157:0/3435532105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5f406dfc0 0x7ff5f414cd00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec00b6c0 tx=0x7ff5ec018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:45.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.692+0000 7ff5dbfff700 1 -- 172.21.15.157:0/3435532105 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec01e640 con 0x7ff5f406dfc0 2023-10-06T04:03:45.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.692+0000 7ff5dbfff700 1 -- 172.21.15.157:0/3435532105 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ec01e7a0 con 0x7ff5f406dfc0 2023-10-06T04:03:45.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.692+0000 7ff5fa477700 1 -- 172.21.15.157:0/3435532105 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5f40c3ff0 con 0x7ff5f406dfc0 2023-10-06T04:03:45.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.692+0000 7ff5dbfff700 1 -- 172.21.15.157:0/3435532105 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec01eb60 con 0x7ff5f406dfc0 2023-10-06T04:03:45.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.692+0000 7ff5fa477700 1 -- 172.21.15.157:0/3435532105 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5f414bf40 con 0x7ff5f406dfc0 2023-10-06T04:03:45.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.693+0000 7ff5fa477700 1 -- 172.21.15.157:0/3435532105 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5f40692b0 con 0x7ff5f406dfc0 2023-10-06T04:03:45.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.693+0000 7ff5dbfff700 1 -- 172.21.15.157:0/3435532105 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff5ec024070 con 0x7ff5f406dfc0 2023-10-06T04:03:45.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.693+0000 7ff5dbfff700 1 --2- 172.21.15.157:0/3435532105 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff5e0066f30 0x7ff5e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:45.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.694+0000 7ff5dbfff700 1 -- 172.21.15.157:0/3435532105 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7ff5ec0872e0 con 0x7ff5f406dfc0 2023-10-06T04:03:45.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.694+0000 7ff5f3fff700 1 --2- 172.21.15.157:0/3435532105 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff5e0066f30 0x7ff5e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:45.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.694+0000 7ff5f3fff700 1 --2- 172.21.15.157:0/3435532105 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff5e0066f30 0x7ff5e00693e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4007fd0 tx=0x7ff5e4007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:45.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.698+0000 7ff5dbfff700 1 -- 172.21.15.157:0/3435532105 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff5ec0541a0 con 0x7ff5f406dfc0 2023-10-06T04:03:45.823 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/190/18' 2023-10-06T04:03:45.823 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/190/13' 2023-10-06T04:03:45.823 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/190/17' 2023-10-06T04:03:45.823 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/190' 2023-10-06T04:03:45.823 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/104/19' 2023-10-06T04:03:45.824 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/104/15' 2023-10-06T04:03:45.824 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/104/20' 2023-10-06T04:03:45.824 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/104/12' 2023-10-06T04:03:45.824 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/104/10' 2023-10-06T04:03:45.824 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/104/11' 2023-10-06T04:03:45.824 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/104/16' 2023-10-06T04:03:45.824 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/104/14' 2023-10-06T04:03:45.824 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/104/18' 2023-10-06T04:03:45.824 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/104/13' 2023-10-06T04:03:45.824 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/104/17' 2023-10-06T04:03:45.824 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/104' 2023-10-06T04:03:45.824 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/164/19' 2023-10-06T04:03:45.825 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/164/15' 2023-10-06T04:03:45.825 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/164/20' 2023-10-06T04:03:45.825 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/164/12' 2023-10-06T04:03:45.825 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/164/10' 2023-10-06T04:03:45.825 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/164/11' 2023-10-06T04:03:45.825 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/164/16' 2023-10-06T04:03:45.825 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/164/14' 2023-10-06T04:03:45.825 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/164/18' 2023-10-06T04:03:45.825 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/164/13' 2023-10-06T04:03:45.825 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/164/17' 2023-10-06T04:03:45.825 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/164' 2023-10-06T04:03:45.825 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/112/19' 2023-10-06T04:03:45.826 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/112/15' 2023-10-06T04:03:45.826 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/112/20' 2023-10-06T04:03:45.826 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/112/12' 2023-10-06T04:03:45.826 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/112/10' 2023-10-06T04:03:45.826 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/112/11' 2023-10-06T04:03:45.826 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/112/16' 2023-10-06T04:03:45.826 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/112/14' 2023-10-06T04:03:45.826 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/112/18' 2023-10-06T04:03:45.826 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/112/13' 2023-10-06T04:03:45.826 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/112/17' 2023-10-06T04:03:45.826 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/112' 2023-10-06T04:03:45.826 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/100/19' 2023-10-06T04:03:45.827 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/100/15' 2023-10-06T04:03:45.827 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/100/20' 2023-10-06T04:03:45.827 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/100/12' 2023-10-06T04:03:45.827 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/100/10' 2023-10-06T04:03:45.827 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/100/11' 2023-10-06T04:03:45.827 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/100/16' 2023-10-06T04:03:45.827 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/100/14' 2023-10-06T04:03:45.827 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/100/18' 2023-10-06T04:03:45.827 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/100/13' 2023-10-06T04:03:45.827 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/100/17' 2023-10-06T04:03:45.827 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/100' 2023-10-06T04:03:45.827 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/134/19' 2023-10-06T04:03:45.828 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/134/15' 2023-10-06T04:03:45.828 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/134/20' 2023-10-06T04:03:45.828 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/134/12' 2023-10-06T04:03:45.828 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/134/10' 2023-10-06T04:03:45.828 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/134/11' 2023-10-06T04:03:45.828 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/134/16' 2023-10-06T04:03:45.828 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/134/14' 2023-10-06T04:03:45.828 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/134/18' 2023-10-06T04:03:45.828 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/134/13' 2023-10-06T04:03:45.828 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/134/17' 2023-10-06T04:03:45.828 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/134' 2023-10-06T04:03:45.828 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/115/19' 2023-10-06T04:03:45.829 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/115/15' 2023-10-06T04:03:45.829 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/115/20' 2023-10-06T04:03:45.829 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/115/12' 2023-10-06T04:03:45.829 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/115/10' 2023-10-06T04:03:45.829 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/115/11' 2023-10-06T04:03:45.829 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/115/16' 2023-10-06T04:03:45.829 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/115/14' 2023-10-06T04:03:45.829 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/115/18' 2023-10-06T04:03:45.829 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/115/13' 2023-10-06T04:03:45.829 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/115/17' 2023-10-06T04:03:45.829 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/115' 2023-10-06T04:03:45.829 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/116/19' 2023-10-06T04:03:45.830 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/116/15' 2023-10-06T04:03:45.830 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/116/20' 2023-10-06T04:03:45.830 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/116/12' 2023-10-06T04:03:45.830 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/116/10' 2023-10-06T04:03:45.830 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/116/11' 2023-10-06T04:03:45.830 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/116/16' 2023-10-06T04:03:45.830 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/116/14' 2023-10-06T04:03:45.830 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/116/18' 2023-10-06T04:03:45.830 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/116/13' 2023-10-06T04:03:45.830 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/116/17' 2023-10-06T04:03:45.830 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/116' 2023-10-06T04:03:45.830 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/195/19' 2023-10-06T04:03:45.831 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/195/15' 2023-10-06T04:03:45.831 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/195/20' 2023-10-06T04:03:45.831 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/195/12' 2023-10-06T04:03:45.831 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/195/10' 2023-10-06T04:03:45.831 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/195/11' 2023-10-06T04:03:45.831 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/195/16' 2023-10-06T04:03:45.831 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/195/14' 2023-10-06T04:03:45.831 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/195/18' 2023-10-06T04:03:45.831 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/195/13' 2023-10-06T04:03:45.831 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/195/17' 2023-10-06T04:03:45.831 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/195' 2023-10-06T04:03:45.831 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/138/19' 2023-10-06T04:03:45.832 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/138/15' 2023-10-06T04:03:45.832 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/138/20' 2023-10-06T04:03:45.832 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/138/12' 2023-10-06T04:03:45.832 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/138/10' 2023-10-06T04:03:45.832 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/138/11' 2023-10-06T04:03:45.832 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/138/16' 2023-10-06T04:03:45.832 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/138/14' 2023-10-06T04:03:45.832 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/138/18' 2023-10-06T04:03:45.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.955+0000 7ff5fa477700 1 -- 172.21.15.157:0/3435532105 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5f414c9b0 con 0x7ff5f406dfc0 2023-10-06T04:03:45.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.956+0000 7ff5dbfff700 1 -- 172.21.15.157:0/3435532105 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+6146 (secure 0 0 0) 0x7ff5ec0577c0 con 0x7ff5f406dfc0 2023-10-06T04:03:45.959 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:03:45.959 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3192541484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3192541484},{"type":"v1","addr":"172.21.15.175:6835","nonce":3192541484}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:41.773401+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":4287},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:03:45.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.958+0000 7ff5fa477700 1 -- 172.21.15.157:0/3435532105 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff5e0066f30 msgr2=0x7ff5e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:45.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.958+0000 7ff5fa477700 1 --2- 172.21.15.157:0/3435532105 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff5e0066f30 0x7ff5e00693e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4007fd0 tx=0x7ff5e4007d30 comp rx=0 tx=0).stop 2023-10-06T04:03:45.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.958+0000 7ff5fa477700 1 -- 172.21.15.157:0/3435532105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5f406dfc0 msgr2=0x7ff5f414cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:45.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.958+0000 7ff5fa477700 1 --2- 172.21.15.157:0/3435532105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5f406dfc0 0x7ff5f414cd00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec00b6c0 tx=0x7ff5ec018000 comp rx=0 tx=0).stop 2023-10-06T04:03:45.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.958+0000 7ff5fa477700 1 -- 172.21.15.157:0/3435532105 shutdown_connections 2023-10-06T04:03:45.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.958+0000 7ff5fa477700 1 --2- 172.21.15.157:0/3435532105 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff5e0066f30 0x7ff5e00693e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:45.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.958+0000 7ff5fa477700 1 --2- 172.21.15.157:0/3435532105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5f406dfc0 0x7ff5f414cd00 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:45.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.958+0000 7ff5fa477700 1 --2- 172.21.15.157:0/3435532105 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5f406a780 0x7ff5f40c39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:45.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.958+0000 7ff5fa477700 1 --2- 172.21.15.157:0/3435532105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5f406ab50 0x7ff5f4163110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:45.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.958+0000 7ff5fa477700 1 -- 172.21.15.157:0/3435532105 >> 172.21.15.157:0/3435532105 conn(0x7ff5f406b510 msgr2=0x7ff5f4150a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:45.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.959+0000 7ff5fa477700 1 -- 172.21.15.157:0/3435532105 shutdown_connections 2023-10-06T04:03:45.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:45.959+0000 7ff5fa477700 1 -- 172.21.15.157:0/3435532105 wait complete. 2023-10-06T04:03:45.963 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 54 2023-10-06T04:03:45.973 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-10-06T04:03:46.959 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/138/13' 2023-10-06T04:03:46.959 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/138/17' 2023-10-06T04:03:46.959 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/138' 2023-10-06T04:03:46.959 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/135/19' 2023-10-06T04:03:46.959 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/135/15' 2023-10-06T04:03:46.959 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/135/20' 2023-10-06T04:03:46.959 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/135/12' 2023-10-06T04:03:46.959 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/135/10' 2023-10-06T04:03:46.960 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/135/11' 2023-10-06T04:03:46.960 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/135/16' 2023-10-06T04:03:46.960 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/135/14' 2023-10-06T04:03:46.960 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/135/18' 2023-10-06T04:03:46.960 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/135/13' 2023-10-06T04:03:46.960 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/135/17' 2023-10-06T04:03:46.960 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/135' 2023-10-06T04:03:46.960 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/117/19' 2023-10-06T04:03:46.961 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/117/15' 2023-10-06T04:03:46.961 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/117/20' 2023-10-06T04:03:46.961 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/117/12' 2023-10-06T04:03:46.961 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/117/10' 2023-10-06T04:03:46.961 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/117/11' 2023-10-06T04:03:46.961 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/117/16' 2023-10-06T04:03:46.961 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/117/14' 2023-10-06T04:03:46.961 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/117/18' 2023-10-06T04:03:46.961 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/117/13' 2023-10-06T04:03:46.961 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/117/17' 2023-10-06T04:03:46.961 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/117' 2023-10-06T04:03:46.961 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/143/19' 2023-10-06T04:03:46.962 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/143/15' 2023-10-06T04:03:46.962 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/143/20' 2023-10-06T04:03:46.962 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/143/12' 2023-10-06T04:03:46.962 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/143/10' 2023-10-06T04:03:46.962 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/143/11' 2023-10-06T04:03:46.962 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/143/16' 2023-10-06T04:03:46.962 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/143/14' 2023-10-06T04:03:46.962 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/143/18' 2023-10-06T04:03:46.962 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/143/13' 2023-10-06T04:03:46.962 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/143/17' 2023-10-06T04:03:46.962 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/143' 2023-10-06T04:03:46.963 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/180/19' 2023-10-06T04:03:46.963 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/180/15' 2023-10-06T04:03:46.963 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/180/20' 2023-10-06T04:03:46.963 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/180/12' 2023-10-06T04:03:46.963 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/180/10' 2023-10-06T04:03:46.963 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/180/11' 2023-10-06T04:03:46.963 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/180/16' 2023-10-06T04:03:46.963 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/180/14' 2023-10-06T04:03:46.963 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/180/18' 2023-10-06T04:03:46.963 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/180/13' 2023-10-06T04:03:46.963 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/180/17' 2023-10-06T04:03:46.964 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/180' 2023-10-06T04:03:46.964 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/126/19' 2023-10-06T04:03:46.964 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/126/15' 2023-10-06T04:03:46.964 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/126/20' 2023-10-06T04:03:46.964 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/126/12' 2023-10-06T04:03:46.964 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/126/10' 2023-10-06T04:03:46.964 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/126/11' 2023-10-06T04:03:46.964 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/126/16' 2023-10-06T04:03:46.964 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/126/14' 2023-10-06T04:03:46.964 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/126/18' 2023-10-06T04:03:46.964 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/126/13' 2023-10-06T04:03:46.964 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/126/17' 2023-10-06T04:03:46.965 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/126' 2023-10-06T04:03:46.965 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/161/19' 2023-10-06T04:03:46.965 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/161/15' 2023-10-06T04:03:46.965 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/161/20' 2023-10-06T04:03:46.965 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/161/12' 2023-10-06T04:03:46.965 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/161/10' 2023-10-06T04:03:46.965 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/161/11' 2023-10-06T04:03:46.965 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/161/16' 2023-10-06T04:03:46.965 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/161/14' 2023-10-06T04:03:46.965 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/161/18' 2023-10-06T04:03:46.965 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/161/13' 2023-10-06T04:03:46.965 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/161/17' 2023-10-06T04:03:46.966 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/161' 2023-10-06T04:03:46.966 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/176/19' 2023-10-06T04:03:46.966 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/176/15' 2023-10-06T04:03:46.966 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/176/20' 2023-10-06T04:03:46.966 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/176/12' 2023-10-06T04:03:46.966 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/176/10' 2023-10-06T04:03:46.966 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/176/11' 2023-10-06T04:03:46.966 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/176/16' 2023-10-06T04:03:46.966 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/176/14' 2023-10-06T04:03:46.966 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/176/18' 2023-10-06T04:03:46.966 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/176/13' 2023-10-06T04:03:46.967 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/176/17' 2023-10-06T04:03:46.967 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/176' 2023-10-06T04:03:46.967 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/111/19' 2023-10-06T04:03:46.967 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/111/15' 2023-10-06T04:03:46.967 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/111/20' 2023-10-06T04:03:46.967 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/111/12' 2023-10-06T04:03:46.967 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/111/10' 2023-10-06T04:03:46.967 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/111/11' 2023-10-06T04:03:46.967 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/111/16' 2023-10-06T04:03:46.967 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/111/14' 2023-10-06T04:03:46.967 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/111/18' 2023-10-06T04:03:46.967 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/111/13' 2023-10-06T04:03:46.968 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/111/17' 2023-10-06T04:03:46.968 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/111' 2023-10-06T04:03:46.968 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/191/19' 2023-10-06T04:03:46.968 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/191/15' 2023-10-06T04:03:46.968 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/191/20' 2023-10-06T04:03:46.968 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/191/12' 2023-10-06T04:03:46.968 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/191/10' 2023-10-06T04:03:46.968 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/191/11' 2023-10-06T04:03:46.968 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/191/16' 2023-10-06T04:03:46.968 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/191/14' 2023-10-06T04:03:46.968 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/191/18' 2023-10-06T04:03:46.969 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/191/13' 2023-10-06T04:03:47.484 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/191/17' 2023-10-06T04:03:47.484 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/191' 2023-10-06T04:03:47.484 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/184/19' 2023-10-06T04:03:47.484 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/184/15' 2023-10-06T04:03:47.484 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/184/20' 2023-10-06T04:03:47.485 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/184/12' 2023-10-06T04:03:47.485 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/184/10' 2023-10-06T04:03:47.485 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/184/11' 2023-10-06T04:03:47.485 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/184/16' 2023-10-06T04:03:47.485 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/184/14' 2023-10-06T04:03:47.485 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/184/18' 2023-10-06T04:03:47.485 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/184/13' 2023-10-06T04:03:47.485 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/184/17' 2023-10-06T04:03:47.485 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/184' 2023-10-06T04:03:47.485 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/192/19' 2023-10-06T04:03:47.486 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/192/15' 2023-10-06T04:03:47.486 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/192/20' 2023-10-06T04:03:47.486 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/192/12' 2023-10-06T04:03:47.486 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/192/10' 2023-10-06T04:03:47.486 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/192/11' 2023-10-06T04:03:47.486 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/192/16' 2023-10-06T04:03:47.486 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/192/14' 2023-10-06T04:03:47.486 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/192/18' 2023-10-06T04:03:47.486 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/192/13' 2023-10-06T04:03:47.486 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/192/17' 2023-10-06T04:03:47.487 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/192' 2023-10-06T04:03:47.487 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/103/19' 2023-10-06T04:03:47.487 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/103/15' 2023-10-06T04:03:47.487 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/103/20' 2023-10-06T04:03:47.487 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/103/12' 2023-10-06T04:03:47.487 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/103/10' 2023-10-06T04:03:47.487 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/103/11' 2023-10-06T04:03:47.487 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/103/16' 2023-10-06T04:03:47.488 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/103/14' 2023-10-06T04:03:47.488 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/103/18' 2023-10-06T04:03:47.488 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/103/13' 2023-10-06T04:03:47.488 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/103/17' 2023-10-06T04:03:47.488 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/103' 2023-10-06T04:03:47.488 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/128/19' 2023-10-06T04:03:47.488 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/128/15' 2023-10-06T04:03:47.488 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/128/20' 2023-10-06T04:03:47.488 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/128/12' 2023-10-06T04:03:47.489 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/128/10' 2023-10-06T04:03:47.489 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/128/11' 2023-10-06T04:03:47.489 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/128/16' 2023-10-06T04:03:47.489 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/128/14' 2023-10-06T04:03:47.489 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/128/18' 2023-10-06T04:03:47.489 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/128/13' 2023-10-06T04:03:47.490 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/128/17' 2023-10-06T04:03:47.491 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/128' 2023-10-06T04:03:47.491 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/144/19' 2023-10-06T04:03:47.491 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/144/15' 2023-10-06T04:03:47.491 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/144/20' 2023-10-06T04:03:47.491 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/144/12' 2023-10-06T04:03:47.491 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/144/10' 2023-10-06T04:03:47.492 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/144/11' 2023-10-06T04:03:47.492 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/144/16' 2023-10-06T04:03:47.492 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/144/14' 2023-10-06T04:03:47.492 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/144/18' 2023-10-06T04:03:47.492 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/144/13' 2023-10-06T04:03:47.492 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/144/17' 2023-10-06T04:03:47.492 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/144' 2023-10-06T04:03:47.493 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/146/19' 2023-10-06T04:03:47.493 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/146/15' 2023-10-06T04:03:47.493 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/146/20' 2023-10-06T04:03:47.493 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/146/12' 2023-10-06T04:03:47.493 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/146/10' 2023-10-06T04:03:47.493 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/146/11' 2023-10-06T04:03:47.493 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/146/16' 2023-10-06T04:03:47.493 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/146/14' 2023-10-06T04:03:47.494 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/146/18' 2023-10-06T04:03:47.494 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/146/13' 2023-10-06T04:03:47.494 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/146/17' 2023-10-06T04:03:47.494 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/146' 2023-10-06T04:03:47.495 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/141/19' 2023-10-06T04:03:47.495 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/141/15' 2023-10-06T04:03:47.495 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/141/20' 2023-10-06T04:03:47.495 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/141/12' 2023-10-06T04:03:47.495 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/141/10' 2023-10-06T04:03:47.495 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/141/11' 2023-10-06T04:03:47.495 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/141/16' 2023-10-06T04:03:47.496 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/141/14' 2023-10-06T04:03:47.496 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/141/18' 2023-10-06T04:03:47.496 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/141/13' 2023-10-06T04:03:47.496 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/141/17' 2023-10-06T04:03:47.496 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/141' 2023-10-06T04:03:47.496 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/182/19' 2023-10-06T04:03:47.497 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/182/15' 2023-10-06T04:03:47.497 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/182/20' 2023-10-06T04:03:47.497 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/182/12' 2023-10-06T04:03:47.497 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/182/10' 2023-10-06T04:03:47.497 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/182/11' 2023-10-06T04:03:47.498 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/182/16' 2023-10-06T04:03:47.498 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/182/14' 2023-10-06T04:03:47.498 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/182/18' 2023-10-06T04:03:47.498 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/182/13' 2023-10-06T04:03:47.498 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/182/17' 2023-10-06T04:03:47.499 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/182' 2023-10-06T04:03:47.499 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/107/19' 2023-10-06T04:03:47.499 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/107/15' 2023-10-06T04:03:47.499 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/107/20' 2023-10-06T04:03:47.499 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/107/12' 2023-10-06T04:03:47.499 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/107/10' 2023-10-06T04:03:47.500 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/107/11' 2023-10-06T04:03:47.500 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/107/16' 2023-10-06T04:03:47.500 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/107/14' 2023-10-06T04:03:47.501 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/107/18' 2023-10-06T04:03:47.501 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/107/13' 2023-10-06T04:03:47.501 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/107/17' 2023-10-06T04:03:47.973 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:03:48.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.084+0000 7fdc4b2e6700 1 Processor -- start 2023-10-06T04:03:48.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.085+0000 7fdc4b2e6700 1 -- start start 2023-10-06T04:03:48.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.085+0000 7fdc4b2e6700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdc44156f50 0x7fdc441618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:48.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.085+0000 7fdc4b2e6700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc4406d530 con 0x7fdc440ca630 2023-10-06T04:03:48.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.085+0000 7fdc4b2e6700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc4406d9c0 con 0x7fdc44068920 2023-10-06T04:03:48.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.085+0000 7fdc4b2e6700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc4406f0c0 con 0x7fdc44156f50 2023-10-06T04:03:48.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.085+0000 7fdc49082700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fdc440ca630 0x7fdc440c6ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43558/0 (socket says 172.21.15.157:43558) 2023-10-06T04:03:48.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.085+0000 7fdc49883700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdc44156f50 0x7fdc441618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:48.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.085+0000 7fdc49082700 1 -- 172.21.15.157:0/3823540356 learned_addr learned my addr 172.21.15.157:0/3823540356 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:03:48.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.086+0000 7fdc3bfff700 1 -- 172.21.15.157:0/3823540356 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1060884460 0 0) 0x7fdc4406d530 con 0x7fdc440ca630 2023-10-06T04:03:48.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.086+0000 7fdc3bfff700 1 -- 172.21.15.157:0/3823540356 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc30003570 con 0x7fdc440ca630 2023-10-06T04:03:48.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.086+0000 7fdc49883700 1 -- 172.21.15.157:0/3823540356 >> v1:172.21.15.175:6789/0 conn(0x7fdc44068920 legacy=0x7fdc44156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:48.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.086+0000 7fdc49883700 1 -- 172.21.15.157:0/3823540356 >> v1:172.21.15.157:6789/0 conn(0x7fdc440ca630 legacy=0x7fdc440c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:48.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.086+0000 7fdc49883700 1 -- 172.21.15.157:0/3823540356 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc4406f3a0 con 0x7fdc44156f50 2023-10-06T04:03:48.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.086+0000 7fdc49883700 1 --2- 172.21.15.157:0/3823540356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc44156f50 0x7fdc441618d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c009e90 tx=0x7fdc3c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40823d894b5e21f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:48.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.086+0000 7fdc3bfff700 1 -- 172.21.15.157:0/3823540356 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c00e070 con 0x7fdc44156f50 2023-10-06T04:03:48.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.086+0000 7fdc3bfff700 1 -- 172.21.15.157:0/3823540356 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3c00bc40 con 0x7fdc44156f50 2023-10-06T04:03:48.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.086+0000 7fdc3bfff700 1 -- 172.21.15.157:0/3823540356 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c012430 con 0x7fdc44156f50 2023-10-06T04:03:48.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.086+0000 7fdc4b2e6700 1 -- 172.21.15.157:0/3823540356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc44156f50 msgr2=0x7fdc441618d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:48.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.086+0000 7fdc4b2e6700 1 --2- 172.21.15.157:0/3823540356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc44156f50 0x7fdc441618d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c009e90 tx=0x7fdc3c009780 comp rx=0 tx=0).stop 2023-10-06T04:03:48.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.087+0000 7fdc4b2e6700 1 -- 172.21.15.157:0/3823540356 shutdown_connections 2023-10-06T04:03:48.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.087+0000 7fdc4b2e6700 1 --2- 172.21.15.157:0/3823540356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc44156f50 0x7fdc441618d0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:48.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.087+0000 7fdc4b2e6700 1 -- 172.21.15.157:0/3823540356 >> 172.21.15.157:0/3823540356 conn(0x7fdc440692b0 msgr2=0x7fdc440696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:48.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.087+0000 7fdc4b2e6700 1 -- 172.21.15.157:0/3823540356 shutdown_connections 2023-10-06T04:03:48.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.087+0000 7fdc4b2e6700 1 -- 172.21.15.157:0/3823540356 wait complete. 2023-10-06T04:03:48.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.088+0000 7fdc4b2e6700 1 Processor -- start 2023-10-06T04:03:48.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.088+0000 7fdc4b2e6700 1 -- start start 2023-10-06T04:03:48.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.089+0000 7fdc4b2e6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc44068920 0x7fdc44159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:48.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.089+0000 7fdc4b2e6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc440ca630 0x7fdc44158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:48.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.089+0000 7fdc4b2e6700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc44156f50 0x7fdc44166520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:48.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.089+0000 7fdc4b2e6700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4415a4e0 con 0x7fdc44068920 2023-10-06T04:03:48.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.089+0000 7fdc4b2e6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4406e870 con 0x7fdc440ca630 2023-10-06T04:03:48.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.089+0000 7fdc4b2e6700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4415a840 con 0x7fdc44156f50 2023-10-06T04:03:48.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.089+0000 7fdc49082700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc44068920 0x7fdc44159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:48.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.089+0000 7fdc49082700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc44068920 0x7fdc44159e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50546/0 (socket says 172.21.15.157:50546) 2023-10-06T04:03:48.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.089+0000 7fdc49082700 1 -- 172.21.15.157:0/139669823 learned_addr learned my addr 172.21.15.157:0/139669823 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:48.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.089+0000 7fdc48881700 1 --2- 172.21.15.157:0/139669823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc440ca630 0x7fdc44158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:48.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.089+0000 7fdc49883700 1 --2- 172.21.15.157:0/139669823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc44156f50 0x7fdc44166520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:48.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.089+0000 7fdc49082700 1 -- 172.21.15.157:0/139669823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc44156f50 msgr2=0x7fdc44166520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:48.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.089+0000 7fdc49082700 1 --2- 172.21.15.157:0/139669823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc44156f50 0x7fdc44166520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:48.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.089+0000 7fdc49082700 1 -- 172.21.15.157:0/139669823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc440ca630 msgr2=0x7fdc44158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:48.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.089+0000 7fdc49082700 1 --2- 172.21.15.157:0/139669823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc440ca630 0x7fdc44158730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:48.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.089+0000 7fdc49082700 1 -- 172.21.15.157:0/139669823 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3c009a20 con 0x7fdc44068920 2023-10-06T04:03:48.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.089+0000 7fdc48881700 1 --2- 172.21.15.157:0/139669823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc440ca630 0x7fdc44158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:03:48.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.089+0000 7fdc49082700 1 --2- 172.21.15.157:0/139669823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc44068920 0x7fdc44159e50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000c7f0 tx=0x7fdc40007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:48.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.089+0000 7fdc397fa700 1 -- 172.21.15.157:0/139669823 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc40005b30 con 0x7fdc44068920 2023-10-06T04:03:48.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.089+0000 7fdc397fa700 1 -- 172.21.15.157:0/139669823 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc40005c90 con 0x7fdc44068920 2023-10-06T04:03:48.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.089+0000 7fdc397fa700 1 -- 172.21.15.157:0/139669823 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc40007340 con 0x7fdc44068920 2023-10-06T04:03:48.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.089+0000 7fdc4b2e6700 1 -- 172.21.15.157:0/139669823 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc4415ab20 con 0x7fdc44068920 2023-10-06T04:03:48.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.089+0000 7fdc4b2e6700 1 -- 172.21.15.157:0/139669823 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc4415b0e0 con 0x7fdc44068920 2023-10-06T04:03:48.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.090+0000 7fdc4b2e6700 1 -- 172.21.15.157:0/139669823 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc44166370 con 0x7fdc44068920 2023-10-06T04:03:48.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.091+0000 7fdc397fa700 1 -- 172.21.15.157:0/139669823 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdc4001b400 con 0x7fdc44068920 2023-10-06T04:03:48.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.091+0000 7fdc397fa700 1 --2- 172.21.15.157:0/139669823 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdc28066f60 0x7fdc28069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:48.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.091+0000 7fdc397fa700 1 -- 172.21.15.157:0/139669823 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7fdc400843e0 con 0x7fdc44068920 2023-10-06T04:03:48.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.091+0000 7fdc48881700 1 --2- 172.21.15.157:0/139669823 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdc28066f60 0x7fdc28069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:48.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.092+0000 7fdc48881700 1 --2- 172.21.15.157:0/139669823 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdc28066f60 0x7fdc28069410 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdc44163cf0 tx=0x7fdc34009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:48.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.095+0000 7fdc397fa700 1 -- 172.21.15.157:0/139669823 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdc400513d0 con 0x7fdc44068920 2023-10-06T04:03:48.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.325+0000 7fdc4b2e6700 1 -- 172.21.15.157:0/139669823 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc44166140 con 0x7fdc44068920 2023-10-06T04:03:48.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.326+0000 7fdc397fa700 1 -- 172.21.15.157:0/139669823 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+6147 (secure 0 0 0) 0x7fdc400076f0 con 0x7fdc44068920 2023-10-06T04:03:48.328 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:03:48.329 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3192541484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3192541484},{"type":"v1","addr":"172.21.15.175:6835","nonce":3192541484}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:46.794132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":4287},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:03:48.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.328+0000 7fdc4b2e6700 1 -- 172.21.15.157:0/139669823 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdc28066f60 msgr2=0x7fdc28069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:48.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.328+0000 7fdc4b2e6700 1 --2- 172.21.15.157:0/139669823 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdc28066f60 0x7fdc28069410 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdc44163cf0 tx=0x7fdc34009040 comp rx=0 tx=0).stop 2023-10-06T04:03:48.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.328+0000 7fdc4b2e6700 1 -- 172.21.15.157:0/139669823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc44068920 msgr2=0x7fdc44159e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:48.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.328+0000 7fdc4b2e6700 1 --2- 172.21.15.157:0/139669823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc44068920 0x7fdc44159e50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000c7f0 tx=0x7fdc40007e00 comp rx=0 tx=0).stop 2023-10-06T04:03:48.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.328+0000 7fdc4b2e6700 1 -- 172.21.15.157:0/139669823 shutdown_connections 2023-10-06T04:03:48.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.328+0000 7fdc4b2e6700 1 --2- 172.21.15.157:0/139669823 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdc28066f60 0x7fdc28069410 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:48.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.328+0000 7fdc4b2e6700 1 --2- 172.21.15.157:0/139669823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc44156f50 0x7fdc44166520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:48.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.328+0000 7fdc4b2e6700 1 --2- 172.21.15.157:0/139669823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc44068920 0x7fdc44159e50 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:48.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.328+0000 7fdc4b2e6700 1 --2- 172.21.15.157:0/139669823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc440ca630 0x7fdc44158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:48.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.328+0000 7fdc4b2e6700 1 -- 172.21.15.157:0/139669823 >> 172.21.15.157:0/139669823 conn(0x7fdc440692b0 msgr2=0x7fdc44153f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:48.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.328+0000 7fdc4b2e6700 1 -- 172.21.15.157:0/139669823 shutdown_connections 2023-10-06T04:03:48.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:48.328+0000 7fdc4b2e6700 1 -- 172.21.15.157:0/139669823 wait complete. 2023-10-06T04:03:48.332 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 55 2023-10-06T04:03:48.342 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-10-06T04:03:49.865 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/107' 2023-10-06T04:03:49.865 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/178/19' 2023-10-06T04:03:49.865 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/178/15' 2023-10-06T04:03:49.865 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/178/20' 2023-10-06T04:03:49.865 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/178/12' 2023-10-06T04:03:49.865 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/178/10' 2023-10-06T04:03:49.865 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/178/11' 2023-10-06T04:03:49.865 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/178/16' 2023-10-06T04:03:49.865 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/178/14' 2023-10-06T04:03:49.866 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/178/18' 2023-10-06T04:03:49.866 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/178/13' 2023-10-06T04:03:49.866 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/178/17' 2023-10-06T04:03:49.866 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/178' 2023-10-06T04:03:49.866 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/156/19' 2023-10-06T04:03:49.866 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/156/15' 2023-10-06T04:03:49.866 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/156/20' 2023-10-06T04:03:49.866 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/156/12' 2023-10-06T04:03:49.866 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/156/10' 2023-10-06T04:03:49.866 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/156/11' 2023-10-06T04:03:49.866 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/156/16' 2023-10-06T04:03:49.867 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/156/14' 2023-10-06T04:03:49.867 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/156/18' 2023-10-06T04:03:49.867 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/156/13' 2023-10-06T04:03:49.867 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/156/17' 2023-10-06T04:03:49.867 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/156' 2023-10-06T04:03:49.867 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/131/19' 2023-10-06T04:03:49.867 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/131/15' 2023-10-06T04:03:49.867 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/131/20' 2023-10-06T04:03:49.867 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/131/12' 2023-10-06T04:03:49.867 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/131/10' 2023-10-06T04:03:49.867 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/131/11' 2023-10-06T04:03:49.867 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/131/16' 2023-10-06T04:03:49.867 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/131/14' 2023-10-06T04:03:49.868 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/131/18' 2023-10-06T04:03:49.868 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/131/13' 2023-10-06T04:03:49.868 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/131/17' 2023-10-06T04:03:49.868 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/131' 2023-10-06T04:03:49.868 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/123/19' 2023-10-06T04:03:49.868 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/123/15' 2023-10-06T04:03:49.868 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/123/20' 2023-10-06T04:03:49.868 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/123/12' 2023-10-06T04:03:49.868 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/123/10' 2023-10-06T04:03:49.868 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/123/11' 2023-10-06T04:03:49.868 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/123/16' 2023-10-06T04:03:49.869 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/123/14' 2023-10-06T04:03:49.869 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/123/18' 2023-10-06T04:03:49.869 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/123/13' 2023-10-06T04:03:49.869 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/123/17' 2023-10-06T04:03:49.869 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/123' 2023-10-06T04:03:49.869 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/160/19' 2023-10-06T04:03:49.869 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/160/15' 2023-10-06T04:03:49.869 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/160/20' 2023-10-06T04:03:49.869 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/160/12' 2023-10-06T04:03:49.869 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/160/10' 2023-10-06T04:03:49.869 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/160/11' 2023-10-06T04:03:49.869 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/160/16' 2023-10-06T04:03:49.869 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/160/14' 2023-10-06T04:03:49.870 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/160/18' 2023-10-06T04:03:49.870 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/160/13' 2023-10-06T04:03:49.870 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/160/17' 2023-10-06T04:03:49.870 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/160' 2023-10-06T04:03:49.870 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/165/19' 2023-10-06T04:03:49.870 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/165/15' 2023-10-06T04:03:49.870 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/165/20' 2023-10-06T04:03:49.870 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/165/12' 2023-10-06T04:03:49.870 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/165/10' 2023-10-06T04:03:49.870 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/165/11' 2023-10-06T04:03:49.870 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/165/16' 2023-10-06T04:03:49.871 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/165/14' 2023-10-06T04:03:49.871 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/165/18' 2023-10-06T04:03:49.871 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/165/13' 2023-10-06T04:03:49.871 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/165/17' 2023-10-06T04:03:49.871 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/165' 2023-10-06T04:03:49.871 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/101/19' 2023-10-06T04:03:49.871 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/101/15' 2023-10-06T04:03:49.871 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/101/20' 2023-10-06T04:03:49.871 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/101/12' 2023-10-06T04:03:49.871 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/101/10' 2023-10-06T04:03:49.871 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/101/11' 2023-10-06T04:03:49.871 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/101/16' 2023-10-06T04:03:49.871 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/101/14' 2023-10-06T04:03:49.872 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/101/18' 2023-10-06T04:03:49.872 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/101/13' 2023-10-06T04:03:49.872 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/101/17' 2023-10-06T04:03:49.872 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/101' 2023-10-06T04:03:49.872 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/148/19' 2023-10-06T04:03:49.872 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/148/15' 2023-10-06T04:03:49.872 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/148/20' 2023-10-06T04:03:49.872 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/148/12' 2023-10-06T04:03:49.872 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/148/10' 2023-10-06T04:03:49.872 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/148/11' 2023-10-06T04:03:49.872 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/148/16' 2023-10-06T04:03:49.872 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/148/14' 2023-10-06T04:03:49.873 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/148/18' 2023-10-06T04:03:49.873 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/148/13' 2023-10-06T04:03:49.873 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/148/17' 2023-10-06T04:03:49.873 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/148' 2023-10-06T04:03:49.873 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/168/19' 2023-10-06T04:03:49.873 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/168/15' 2023-10-06T04:03:49.873 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/168/20' 2023-10-06T04:03:49.873 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/168/12' 2023-10-06T04:03:49.873 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/168/10' 2023-10-06T04:03:49.873 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/168/11' 2023-10-06T04:03:49.873 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/168/16' 2023-10-06T04:03:49.873 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/168/14' 2023-10-06T04:03:49.874 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/168/18' 2023-10-06T04:03:49.874 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/168/13' 2023-10-06T04:03:49.874 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/168/17' 2023-10-06T04:03:49.874 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/168' 2023-10-06T04:03:50.343 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:03:50.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.457+0000 7f3c55bd5700 1 Processor -- start 2023-10-06T04:03:50.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.457+0000 7f3c55bd5700 1 -- start start 2023-10-06T04:03:50.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.458+0000 7f3c55bd5700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3c5006ab50 0x7f3c5006d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:50.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.458+0000 7f3c55bd5700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c5006ddf0 0x7f3c5006e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:50.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.458+0000 7f3c55bd5700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c500c6160 con 0x7f3c5006a780 2023-10-06T04:03:50.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.458+0000 7f3c55bd5700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c500c5c00 con 0x7f3c5006ab50 2023-10-06T04:03:50.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.458+0000 7f3c55bd5700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c50069040 con 0x7f3c5006ddf0 2023-10-06T04:03:50.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.458+0000 7f3c46ffd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3c5006ab50 0x7f3c5006d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:50.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.458+0000 7f3c4f7fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f3c5006a780 0x7f3c500c65f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43572/0 (socket says 172.21.15.157:43572) 2023-10-06T04:03:50.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.458+0000 7f3c4f7fe700 1 -- 172.21.15.157:0/1966135085 learned_addr learned my addr 172.21.15.157:0/1966135085 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:03:50.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.458+0000 7f3c4ffff700 1 --2- 172.21.15.157:0/1966135085 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c5006ddf0 0x7f3c5006e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:50.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.459+0000 7f3c46ffd700 1 -- 172.21.15.157:0/1966135085 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c5006ddf0 msgr2=0x7f3c5006e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:50.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.459+0000 7f3c46ffd700 1 --2- 172.21.15.157:0/1966135085 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c5006ddf0 0x7f3c5006e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:50.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.459+0000 7f3c46ffd700 1 -- 172.21.15.157:0/1966135085 >> v1:172.21.15.157:6789/0 conn(0x7f3c5006a780 legacy=0x7f3c500c65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:50.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.459+0000 7f3c46ffd700 1 -- 172.21.15.157:0/1966135085 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c50153930 con 0x7f3c5006ab50 2023-10-06T04:03:50.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.459+0000 7f3c4effd700 1 -- 172.21.15.157:0/1966135085 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3708480848 0 0) 0x7f3c500c6160 con 0x7f3c5006a780 2023-10-06T04:03:50.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.459+0000 7f3c46ffd700 1 --2- 172.21.15.157:0/1966135085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c5006ab50 0x7f3c5006d8b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3c38009940 tx=0x7f3c380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110c43ddb44649aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:50.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.459+0000 7f3c4effd700 1 -- 172.21.15.157:0/1966135085 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3800b870 con 0x7f3c5006ab50 2023-10-06T04:03:50.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.459+0000 7f3c4effd700 1 -- 172.21.15.157:0/1966135085 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c3800b9d0 con 0x7f3c5006ab50 2023-10-06T04:03:50.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.459+0000 7f3c4effd700 1 -- 172.21.15.157:0/1966135085 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3800bd90 con 0x7f3c5006ab50 2023-10-06T04:03:50.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.459+0000 7f3c55bd5700 1 -- 172.21.15.157:0/1966135085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c5006ab50 msgr2=0x7f3c5006d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:50.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.459+0000 7f3c55bd5700 1 --2- 172.21.15.157:0/1966135085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c5006ab50 0x7f3c5006d8b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3c38009940 tx=0x7f3c380091a0 comp rx=0 tx=0).stop 2023-10-06T04:03:50.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.459+0000 7f3c55bd5700 1 -- 172.21.15.157:0/1966135085 shutdown_connections 2023-10-06T04:03:50.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.459+0000 7f3c55bd5700 1 --2- 172.21.15.157:0/1966135085 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c5006ddf0 0x7f3c5006e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:50.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.459+0000 7f3c55bd5700 1 --2- 172.21.15.157:0/1966135085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c5006ab50 0x7f3c5006d8b0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:50.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.459+0000 7f3c55bd5700 1 -- 172.21.15.157:0/1966135085 >> 172.21.15.157:0/1966135085 conn(0x7f3c5006b510 msgr2=0x7f3c5006b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:50.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.460+0000 7f3c55bd5700 1 -- 172.21.15.157:0/1966135085 shutdown_connections 2023-10-06T04:03:50.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.460+0000 7f3c55bd5700 1 -- 172.21.15.157:0/1966135085 wait complete. 2023-10-06T04:03:50.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.461+0000 7f3c55bd5700 1 Processor -- start 2023-10-06T04:03:50.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.461+0000 7f3c55bd5700 1 -- start start 2023-10-06T04:03:50.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.461+0000 7f3c55bd5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c5006a780 0x7f3c501489c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:50.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.461+0000 7f3c55bd5700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c5006ab50 0x7f3c50164f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:50.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.462+0000 7f3c55bd5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c5006ddf0 0x7f3c50157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:50.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.462+0000 7f3c55bd5700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5014ad00 con 0x7f3c5006ddf0 2023-10-06T04:03:50.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.462+0000 7f3c55bd5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c50148530 con 0x7f3c5006a780 2023-10-06T04:03:50.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.462+0000 7f3c55bd5700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5014a4a0 con 0x7f3c5006ab50 2023-10-06T04:03:50.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.462+0000 7f3c4ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c5006ddf0 0x7f3c50157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:50.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.462+0000 7f3c4ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c5006ddf0 0x7f3c50157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50564/0 (socket says 172.21.15.157:50564) 2023-10-06T04:03:50.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.462+0000 7f3c4ffff700 1 -- 172.21.15.157:0/3045104095 learned_addr learned my addr 172.21.15.157:0/3045104095 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:50.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.462+0000 7f3c46ffd700 1 --2- 172.21.15.157:0/3045104095 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c5006ab50 0x7f3c50164f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:50.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.462+0000 7f3c4f7fe700 1 --2- 172.21.15.157:0/3045104095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c5006a780 0x7f3c501489c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:50.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.462+0000 7f3c4ffff700 1 -- 172.21.15.157:0/3045104095 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c5006ab50 msgr2=0x7f3c50164f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:50.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.462+0000 7f3c4ffff700 1 --2- 172.21.15.157:0/3045104095 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c5006ab50 0x7f3c50164f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:50.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.462+0000 7f3c4ffff700 1 -- 172.21.15.157:0/3045104095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c5006a780 msgr2=0x7f3c501489c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:50.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.462+0000 7f3c4ffff700 1 --2- 172.21.15.157:0/3045104095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c5006a780 0x7f3c501489c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:50.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.462+0000 7f3c4ffff700 1 -- 172.21.15.157:0/3045104095 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c38009420 con 0x7f3c5006ddf0 2023-10-06T04:03:50.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.462+0000 7f3c4f7fe700 1 --2- 172.21.15.157:0/3045104095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c5006a780 0x7f3c501489c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:03:50.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.462+0000 7f3c4ffff700 1 --2- 172.21.15.157:0/3045104095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c5006ddf0 0x7f3c50157370 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3c48007f50 tx=0x7f3c48009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:50.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.462+0000 7f3c47fff700 1 -- 172.21.15.157:0/3045104095 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4800ec20 con 0x7f3c5006ddf0 2023-10-06T04:03:50.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.462+0000 7f3c47fff700 1 -- 172.21.15.157:0/3045104095 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c4800eda0 con 0x7f3c5006ddf0 2023-10-06T04:03:50.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.462+0000 7f3c47fff700 1 -- 172.21.15.157:0/3045104095 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c48014670 con 0x7f3c5006ddf0 2023-10-06T04:03:50.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.462+0000 7f3c55bd5700 1 -- 172.21.15.157:0/3045104095 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c5014a740 con 0x7f3c5006ddf0 2023-10-06T04:03:50.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.462+0000 7f3c55bd5700 1 -- 172.21.15.157:0/3045104095 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c501481c0 con 0x7f3c5006ddf0 2023-10-06T04:03:50.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.463+0000 7f3c55bd5700 1 -- 172.21.15.157:0/3045104095 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c500692b0 con 0x7f3c5006ddf0 2023-10-06T04:03:50.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.464+0000 7f3c47fff700 1 -- 172.21.15.157:0/3045104095 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3c480149e0 con 0x7f3c5006ddf0 2023-10-06T04:03:50.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.465+0000 7f3c47fff700 1 --2- 172.21.15.157:0/3045104095 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c30066f60 0x7f3c30069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:50.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.465+0000 7f3c47fff700 1 -- 172.21.15.157:0/3045104095 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7f3c48084f50 con 0x7f3c5006ddf0 2023-10-06T04:03:50.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.465+0000 7f3c4f7fe700 1 --2- 172.21.15.157:0/3045104095 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c30066f60 0x7f3c30069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:50.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.465+0000 7f3c4f7fe700 1 --2- 172.21.15.157:0/3045104095 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c30066f60 0x7f3c30069410 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3c400011e0 tx=0x7f3c40009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:50.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.469+0000 7f3c47fff700 1 -- 172.21.15.157:0/3045104095 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c48051e10 con 0x7f3c5006ddf0 2023-10-06T04:03:50.527 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/179/19' 2023-10-06T04:03:50.527 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/179/15' 2023-10-06T04:03:50.527 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/179/20' 2023-10-06T04:03:50.527 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/179/12' 2023-10-06T04:03:50.527 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/179/10' 2023-10-06T04:03:50.528 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/179/11' 2023-10-06T04:03:50.528 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/179/16' 2023-10-06T04:03:50.528 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/179/14' 2023-10-06T04:03:50.528 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/179/18' 2023-10-06T04:03:50.528 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/179/13' 2023-10-06T04:03:50.528 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/179/17' 2023-10-06T04:03:50.529 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/179' 2023-10-06T04:03:50.529 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/132/19' 2023-10-06T04:03:50.529 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/132/15' 2023-10-06T04:03:50.529 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/132/20' 2023-10-06T04:03:50.529 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/132/12' 2023-10-06T04:03:50.529 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/132/10' 2023-10-06T04:03:50.529 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/132/11' 2023-10-06T04:03:50.529 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/132/16' 2023-10-06T04:03:50.529 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/132/14' 2023-10-06T04:03:50.530 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/132/18' 2023-10-06T04:03:50.530 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/132/13' 2023-10-06T04:03:50.530 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/132/17' 2023-10-06T04:03:50.530 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/132' 2023-10-06T04:03:50.530 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/199/19' 2023-10-06T04:03:50.530 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/199/15' 2023-10-06T04:03:50.530 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/199/20' 2023-10-06T04:03:50.530 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/199/12' 2023-10-06T04:03:50.530 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/199/10' 2023-10-06T04:03:50.530 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/199/11' 2023-10-06T04:03:50.530 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/199/16' 2023-10-06T04:03:50.530 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/199/14' 2023-10-06T04:03:50.531 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/199/18' 2023-10-06T04:03:50.531 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/199/13' 2023-10-06T04:03:50.531 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/199/17' 2023-10-06T04:03:50.531 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/199' 2023-10-06T04:03:50.531 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/163/19' 2023-10-06T04:03:50.531 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/163/15' 2023-10-06T04:03:50.531 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/163/20' 2023-10-06T04:03:50.531 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/163/12' 2023-10-06T04:03:50.531 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/163/10' 2023-10-06T04:03:50.531 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/163/11' 2023-10-06T04:03:50.531 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/163/16' 2023-10-06T04:03:50.531 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/163/14' 2023-10-06T04:03:50.532 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/163/18' 2023-10-06T04:03:50.532 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/163/13' 2023-10-06T04:03:50.532 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/163/17' 2023-10-06T04:03:50.532 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/163' 2023-10-06T04:03:50.532 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/153/19' 2023-10-06T04:03:50.532 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/153/15' 2023-10-06T04:03:50.532 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/153/20' 2023-10-06T04:03:50.532 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/153/12' 2023-10-06T04:03:50.532 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/153/10' 2023-10-06T04:03:50.533 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/153/11' 2023-10-06T04:03:50.533 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/153/16' 2023-10-06T04:03:50.533 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/153/14' 2023-10-06T04:03:50.533 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/153/18' 2023-10-06T04:03:50.533 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/153/13' 2023-10-06T04:03:50.533 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/153/17' 2023-10-06T04:03:50.533 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/153' 2023-10-06T04:03:50.533 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/122/19' 2023-10-06T04:03:50.533 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/122/15' 2023-10-06T04:03:50.533 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/122/20' 2023-10-06T04:03:50.533 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/122/12' 2023-10-06T04:03:50.533 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/122/10' 2023-10-06T04:03:50.534 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/122/11' 2023-10-06T04:03:50.534 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/122/16' 2023-10-06T04:03:50.534 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/122/14' 2023-10-06T04:03:50.534 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/122/18' 2023-10-06T04:03:50.534 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/122/13' 2023-10-06T04:03:50.534 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/122/17' 2023-10-06T04:03:50.534 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/122' 2023-10-06T04:03:50.534 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/162/19' 2023-10-06T04:03:50.534 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/162/15' 2023-10-06T04:03:50.534 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/162/20' 2023-10-06T04:03:50.534 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/162/12' 2023-10-06T04:03:50.534 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/162/10' 2023-10-06T04:03:50.535 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/162/11' 2023-10-06T04:03:50.535 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/162/16' 2023-10-06T04:03:50.535 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/162/14' 2023-10-06T04:03:50.535 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/162/18' 2023-10-06T04:03:50.535 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/162/13' 2023-10-06T04:03:50.535 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/162/17' 2023-10-06T04:03:50.535 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/162' 2023-10-06T04:03:50.535 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/197/19' 2023-10-06T04:03:50.535 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/197/15' 2023-10-06T04:03:50.535 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/197/20' 2023-10-06T04:03:50.535 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/197/12' 2023-10-06T04:03:50.535 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/197/10' 2023-10-06T04:03:50.536 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/197/11' 2023-10-06T04:03:50.536 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/197/16' 2023-10-06T04:03:50.536 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/197/14' 2023-10-06T04:03:50.536 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/197/18' 2023-10-06T04:03:50.536 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/197/13' 2023-10-06T04:03:50.536 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/197/17' 2023-10-06T04:03:50.536 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/197' 2023-10-06T04:03:50.536 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/113/19' 2023-10-06T04:03:50.536 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/113/15' 2023-10-06T04:03:50.536 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/113/20' 2023-10-06T04:03:50.536 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/113/12' 2023-10-06T04:03:50.537 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/113/10' 2023-10-06T04:03:50.537 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/113/11' 2023-10-06T04:03:50.537 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/113/16' 2023-10-06T04:03:50.537 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/113/14' 2023-10-06T04:03:50.537 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/113/18' 2023-10-06T04:03:50.537 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/113/13' 2023-10-06T04:03:50.537 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/113/17' 2023-10-06T04:03:50.537 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/113' 2023-10-06T04:03:50.537 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/152/19' 2023-10-06T04:03:50.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.704+0000 7f3c55bd5700 1 -- 172.21.15.157:0/3045104095 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c50148670 con 0x7f3c5006ddf0 2023-10-06T04:03:50.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.705+0000 7f3c47fff700 1 -- 172.21.15.157:0/3045104095 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+6147 (secure 0 0 0) 0x7f3c48055430 con 0x7f3c5006ddf0 2023-10-06T04:03:50.710 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:03:50.710 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3192541484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3192541484},{"type":"v1","addr":"172.21.15.175:6835","nonce":3192541484}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:46.794132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":4287},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:03:50.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.708+0000 7f3c55bd5700 1 -- 172.21.15.157:0/3045104095 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c30066f60 msgr2=0x7f3c30069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:50.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.708+0000 7f3c55bd5700 1 --2- 172.21.15.157:0/3045104095 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c30066f60 0x7f3c30069410 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3c400011e0 tx=0x7f3c40009040 comp rx=0 tx=0).stop 2023-10-06T04:03:50.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.708+0000 7f3c55bd5700 1 -- 172.21.15.157:0/3045104095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c5006ddf0 msgr2=0x7f3c50157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:50.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.708+0000 7f3c55bd5700 1 --2- 172.21.15.157:0/3045104095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c5006ddf0 0x7f3c50157370 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3c48007f50 tx=0x7f3c48009860 comp rx=0 tx=0).stop 2023-10-06T04:03:50.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.708+0000 7f3c55bd5700 1 -- 172.21.15.157:0/3045104095 shutdown_connections 2023-10-06T04:03:50.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.708+0000 7f3c55bd5700 1 --2- 172.21.15.157:0/3045104095 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c30066f60 0x7f3c30069410 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:50.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.708+0000 7f3c55bd5700 1 --2- 172.21.15.157:0/3045104095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c5006a780 0x7f3c501489c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:50.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.708+0000 7f3c55bd5700 1 --2- 172.21.15.157:0/3045104095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c5006ddf0 0x7f3c50157370 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:50.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.708+0000 7f3c55bd5700 1 --2- 172.21.15.157:0/3045104095 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c5006ab50 0x7f3c50164f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:50.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.708+0000 7f3c55bd5700 1 -- 172.21.15.157:0/3045104095 >> 172.21.15.157:0/3045104095 conn(0x7f3c5006b510 msgr2=0x7f3c50150000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:50.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.708+0000 7f3c55bd5700 1 -- 172.21.15.157:0/3045104095 shutdown_connections 2023-10-06T04:03:50.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:50.708+0000 7f3c55bd5700 1 -- 172.21.15.157:0/3045104095 wait complete. 2023-10-06T04:03:50.712 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 55 2023-10-06T04:03:50.726 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-10-06T04:03:51.385 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/152/15' 2023-10-06T04:03:51.385 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/152/20' 2023-10-06T04:03:51.385 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/152/12' 2023-10-06T04:03:51.385 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/152/10' 2023-10-06T04:03:51.385 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/152/11' 2023-10-06T04:03:51.385 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/152/16' 2023-10-06T04:03:51.385 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/152/14' 2023-10-06T04:03:51.385 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/152/18' 2023-10-06T04:03:51.385 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/152/13' 2023-10-06T04:03:51.385 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/152/17' 2023-10-06T04:03:51.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/152' 2023-10-06T04:03:51.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/121/19' 2023-10-06T04:03:51.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/121/15' 2023-10-06T04:03:51.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/121/20' 2023-10-06T04:03:51.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/121/12' 2023-10-06T04:03:51.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/121/10' 2023-10-06T04:03:51.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/121/11' 2023-10-06T04:03:51.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/121/16' 2023-10-06T04:03:51.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/121/14' 2023-10-06T04:03:51.386 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/121/18' 2023-10-06T04:03:51.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/121/13' 2023-10-06T04:03:51.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/121/17' 2023-10-06T04:03:51.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/121' 2023-10-06T04:03:51.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/118/19' 2023-10-06T04:03:51.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/118/15' 2023-10-06T04:03:51.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/118/20' 2023-10-06T04:03:51.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/118/12' 2023-10-06T04:03:51.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/118/10' 2023-10-06T04:03:51.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/118/11' 2023-10-06T04:03:51.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/118/16' 2023-10-06T04:03:51.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/118/14' 2023-10-06T04:03:51.387 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/118/18' 2023-10-06T04:03:51.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/118/13' 2023-10-06T04:03:51.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/118/17' 2023-10-06T04:03:51.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/118' 2023-10-06T04:03:51.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/196/19' 2023-10-06T04:03:51.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/196/15' 2023-10-06T04:03:51.388 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/196/20' 2023-10-06T04:03:51.389 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/196/12' 2023-10-06T04:03:51.389 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/196/10' 2023-10-06T04:03:51.389 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/196/11' 2023-10-06T04:03:51.389 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/196/16' 2023-10-06T04:03:51.389 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/196/14' 2023-10-06T04:03:51.389 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/196/18' 2023-10-06T04:03:51.389 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/196/13' 2023-10-06T04:03:51.389 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/196/17' 2023-10-06T04:03:51.389 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/196' 2023-10-06T04:03:51.389 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/124/19' 2023-10-06T04:03:51.389 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/124/15' 2023-10-06T04:03:51.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/124/20' 2023-10-06T04:03:51.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/124/12' 2023-10-06T04:03:51.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/124/10' 2023-10-06T04:03:51.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/124/11' 2023-10-06T04:03:51.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/124/16' 2023-10-06T04:03:51.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/124/14' 2023-10-06T04:03:51.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/124/18' 2023-10-06T04:03:51.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/124/13' 2023-10-06T04:03:51.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/124/17' 2023-10-06T04:03:51.390 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/124' 2023-10-06T04:03:51.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/188/19' 2023-10-06T04:03:51.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/188/15' 2023-10-06T04:03:51.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/188/20' 2023-10-06T04:03:51.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/188/12' 2023-10-06T04:03:51.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/188/10' 2023-10-06T04:03:51.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/188/11' 2023-10-06T04:03:51.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/188/16' 2023-10-06T04:03:51.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/188/14' 2023-10-06T04:03:51.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/188/18' 2023-10-06T04:03:51.391 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/188/13' 2023-10-06T04:03:51.392 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/188/17' 2023-10-06T04:03:51.392 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/188' 2023-10-06T04:03:51.392 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/145/19' 2023-10-06T04:03:51.392 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/145/15' 2023-10-06T04:03:51.392 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/145/20' 2023-10-06T04:03:51.392 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/145/12' 2023-10-06T04:03:51.393 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/145/10' 2023-10-06T04:03:51.393 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/145/11' 2023-10-06T04:03:51.393 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/145/16' 2023-10-06T04:03:51.393 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/145/14' 2023-10-06T04:03:51.393 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/145/18' 2023-10-06T04:03:51.394 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/145/13' 2023-10-06T04:03:51.394 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/145/17' 2023-10-06T04:03:51.394 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/145' 2023-10-06T04:03:51.394 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/130/19' 2023-10-06T04:03:51.394 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/130/15' 2023-10-06T04:03:51.394 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/130/20' 2023-10-06T04:03:51.395 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/130/12' 2023-10-06T04:03:51.395 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/130/10' 2023-10-06T04:03:51.395 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/130/11' 2023-10-06T04:03:51.395 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/130/16' 2023-10-06T04:03:51.395 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/130/14' 2023-10-06T04:03:51.395 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/130/18' 2023-10-06T04:03:51.396 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/130/13' 2023-10-06T04:03:51.396 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/130/17' 2023-10-06T04:03:51.396 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/130' 2023-10-06T04:03:51.396 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/193/19' 2023-10-06T04:03:51.396 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/193/15' 2023-10-06T04:03:51.396 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/193/20' 2023-10-06T04:03:51.397 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/193/12' 2023-10-06T04:03:51.397 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/193/10' 2023-10-06T04:03:51.397 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/193/11' 2023-10-06T04:03:51.397 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/193/16' 2023-10-06T04:03:51.397 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/193/14' 2023-10-06T04:03:51.398 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/193/18' 2023-10-06T04:03:51.398 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/193/13' 2023-10-06T04:03:51.398 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/193/17' 2023-10-06T04:03:51.398 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/193' 2023-10-06T04:03:51.398 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/175/19' 2023-10-06T04:03:51.398 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/175/15' 2023-10-06T04:03:51.913 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/175/20' 2023-10-06T04:03:51.914 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/175/12' 2023-10-06T04:03:51.914 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/175/10' 2023-10-06T04:03:51.914 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/175/11' 2023-10-06T04:03:51.914 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/175/16' 2023-10-06T04:03:51.914 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/175/14' 2023-10-06T04:03:51.914 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/175/18' 2023-10-06T04:03:51.914 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/175/13' 2023-10-06T04:03:51.914 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/175/17' 2023-10-06T04:03:51.914 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/175' 2023-10-06T04:03:51.915 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/181/19' 2023-10-06T04:03:51.915 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/181/15' 2023-10-06T04:03:51.915 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/181/20' 2023-10-06T04:03:51.915 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/181/12' 2023-10-06T04:03:51.915 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/181/10' 2023-10-06T04:03:51.915 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/181/11' 2023-10-06T04:03:51.915 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/181/16' 2023-10-06T04:03:51.915 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/181/14' 2023-10-06T04:03:51.915 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/181/18' 2023-10-06T04:03:51.915 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/181/13' 2023-10-06T04:03:51.915 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/181/17' 2023-10-06T04:03:51.916 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/181' 2023-10-06T04:03:51.916 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/127/19' 2023-10-06T04:03:51.916 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/127/15' 2023-10-06T04:03:51.916 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/127/20' 2023-10-06T04:03:51.916 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/127/12' 2023-10-06T04:03:51.916 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/127/10' 2023-10-06T04:03:51.916 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/127/11' 2023-10-06T04:03:51.916 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/127/16' 2023-10-06T04:03:51.916 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/127/14' 2023-10-06T04:03:51.916 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/127/18' 2023-10-06T04:03:51.916 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/127/13' 2023-10-06T04:03:51.916 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/127/17' 2023-10-06T04:03:51.917 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/127' 2023-10-06T04:03:51.917 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/108/19' 2023-10-06T04:03:51.917 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/108/15' 2023-10-06T04:03:51.917 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/108/20' 2023-10-06T04:03:51.917 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/108/12' 2023-10-06T04:03:51.917 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/108/10' 2023-10-06T04:03:51.917 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/108/11' 2023-10-06T04:03:51.917 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/108/16' 2023-10-06T04:03:51.917 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/108/14' 2023-10-06T04:03:51.917 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/108/18' 2023-10-06T04:03:51.917 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/108/13' 2023-10-06T04:03:51.917 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/108/17' 2023-10-06T04:03:51.918 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/108' 2023-10-06T04:03:51.918 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/183/19' 2023-10-06T04:03:51.918 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/183/15' 2023-10-06T04:03:51.918 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/183/20' 2023-10-06T04:03:51.918 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/183/12' 2023-10-06T04:03:51.918 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/183/10' 2023-10-06T04:03:51.918 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/183/11' 2023-10-06T04:03:51.918 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/183/16' 2023-10-06T04:03:51.918 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/183/14' 2023-10-06T04:03:51.918 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/183/18' 2023-10-06T04:03:51.918 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/183/13' 2023-10-06T04:03:51.919 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/183/17' 2023-10-06T04:03:51.919 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/183' 2023-10-06T04:03:51.919 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/177/19' 2023-10-06T04:03:51.919 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/177/15' 2023-10-06T04:03:51.919 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/177/20' 2023-10-06T04:03:51.919 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/177/12' 2023-10-06T04:03:51.919 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/177/10' 2023-10-06T04:03:51.919 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/177/11' 2023-10-06T04:03:51.919 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/177/16' 2023-10-06T04:03:51.919 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/177/14' 2023-10-06T04:03:51.919 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/177/18' 2023-10-06T04:03:51.919 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/177/13' 2023-10-06T04:03:51.920 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/177/17' 2023-10-06T04:03:51.920 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/177' 2023-10-06T04:03:51.920 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/169/19' 2023-10-06T04:03:51.920 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/169/15' 2023-10-06T04:03:51.920 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/169/20' 2023-10-06T04:03:51.920 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/169/12' 2023-10-06T04:03:51.920 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/169/10' 2023-10-06T04:03:51.920 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/169/11' 2023-10-06T04:03:51.920 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/169/16' 2023-10-06T04:03:51.920 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/169/14' 2023-10-06T04:03:51.921 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/169/18' 2023-10-06T04:03:51.921 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/169/13' 2023-10-06T04:03:51.921 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/169/17' 2023-10-06T04:03:51.921 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/169' 2023-10-06T04:03:51.921 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/154/19' 2023-10-06T04:03:51.921 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/154/15' 2023-10-06T04:03:51.921 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/154/20' 2023-10-06T04:03:51.921 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/154/12' 2023-10-06T04:03:51.921 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/154/10' 2023-10-06T04:03:51.921 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/154/11' 2023-10-06T04:03:51.921 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/154/16' 2023-10-06T04:03:51.922 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/154/14' 2023-10-06T04:03:51.922 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/154/18' 2023-10-06T04:03:51.922 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/154/13' 2023-10-06T04:03:51.922 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/154/17' 2023-10-06T04:03:51.922 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/154' 2023-10-06T04:03:51.922 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/171/19' 2023-10-06T04:03:51.922 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/171/15' 2023-10-06T04:03:51.922 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/171/20' 2023-10-06T04:03:51.922 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/171/12' 2023-10-06T04:03:51.922 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/171/10' 2023-10-06T04:03:51.922 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/171/11' 2023-10-06T04:03:51.923 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/171/16' 2023-10-06T04:03:51.923 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/171/14' 2023-10-06T04:03:51.923 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/171/18' 2023-10-06T04:03:51.923 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/171/13' 2023-10-06T04:03:51.923 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/171/17' 2023-10-06T04:03:51.923 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/171' 2023-10-06T04:03:51.923 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/139/19' 2023-10-06T04:03:51.923 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/139/15' 2023-10-06T04:03:51.923 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/139/20' 2023-10-06T04:03:52.280 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/139/12' 2023-10-06T04:03:52.280 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/139/10' 2023-10-06T04:03:52.281 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/139/11' 2023-10-06T04:03:52.281 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/139/16' 2023-10-06T04:03:52.281 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/139/14' 2023-10-06T04:03:52.281 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/139/18' 2023-10-06T04:03:52.281 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/139/13' 2023-10-06T04:03:52.281 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/139/17' 2023-10-06T04:03:52.281 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/139' 2023-10-06T04:03:52.281 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/102/19' 2023-10-06T04:03:52.281 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/102/15' 2023-10-06T04:03:52.281 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/102/20' 2023-10-06T04:03:52.281 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/102/12' 2023-10-06T04:03:52.282 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/102/10' 2023-10-06T04:03:52.282 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/102/11' 2023-10-06T04:03:52.282 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/102/16' 2023-10-06T04:03:52.282 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/102/14' 2023-10-06T04:03:52.282 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/102/18' 2023-10-06T04:03:52.282 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/102/13' 2023-10-06T04:03:52.282 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/102/17' 2023-10-06T04:03:52.282 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/102' 2023-10-06T04:03:52.282 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/129/19' 2023-10-06T04:03:52.282 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/129/15' 2023-10-06T04:03:52.282 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/129/20' 2023-10-06T04:03:52.283 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/129/12' 2023-10-06T04:03:52.283 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/129/10' 2023-10-06T04:03:52.283 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/129/11' 2023-10-06T04:03:52.283 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/129/16' 2023-10-06T04:03:52.283 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/129/14' 2023-10-06T04:03:52.283 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/129/18' 2023-10-06T04:03:52.283 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/129/13' 2023-10-06T04:03:52.283 INFO:tasks.workunit.client.0.smithi157.stdout:removed './copyofsnap1/test1/129/17' 2023-10-06T04:03:52.283 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1/129' 2023-10-06T04:03:52.283 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1/test1' 2023-10-06T04:03:52.283 INFO:tasks.workunit.client.0.smithi157.stdout:removed directory './copyofsnap1' 2023-10-06T04:03:52.283 INFO:tasks.workunit.client.0.smithi157.stdout:OK 2023-10-06T04:03:52.284 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-06T04:03:52.284 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-06T04:03:52.316 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-authwb.sh... 2023-10-06T04:03:52.317 DEBUG:teuthology.orchestra.run.smithi157:workunit test fs/snaps/snaptest-authwb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-authwb.sh 2023-10-06T04:03:52.728 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:03:52.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.874+0000 7fd08cab3700 1 Processor -- start 2023-10-06T04:03:52.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.874+0000 7fd08cab3700 1 -- start start 2023-10-06T04:03:52.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.875+0000 7fd08cab3700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd08806a780 0x7fd088159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:52.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.875+0000 7fd08cab3700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd08806ab50 0x7fd08806c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:52.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.875+0000 7fd08cab3700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0881569b0 con 0x7fd0880bc750 2023-10-06T04:03:52.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.875+0000 7fd08cab3700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd088155e90 con 0x7fd08806a780 2023-10-06T04:03:52.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.875+0000 7fd08cab3700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd08815a310 con 0x7fd08806ab50 2023-10-06T04:03:52.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.875+0000 7fd086d9d700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd0880bc750 0x7fd0880c5d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43578/0 (socket says 172.21.15.157:43578) 2023-10-06T04:03:52.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.875+0000 7fd086d9d700 1 -- 172.21.15.157:0/1800781327 learned_addr learned my addr 172.21.15.157:0/1800781327 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:03:52.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.876+0000 7fd08659c700 1 --2- 172.21.15.157:0/1800781327 >> v2:172.21.15.175:3300/0 conn(0x7fd08806a780 0x7fd088159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:52.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.876+0000 7fd085d9b700 1 --2- 172.21.15.157:0/1800781327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd08806ab50 0x7fd08806c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:52.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.876+0000 7fd08559a700 1 -- 172.21.15.157:0/1800781327 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 814519186 0 0) 0x7fd0881569b0 con 0x7fd0880bc750 2023-10-06T04:03:52.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.876+0000 7fd08559a700 1 -- 172.21.15.157:0/1800781327 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd070003570 con 0x7fd0880bc750 2023-10-06T04:03:52.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.876+0000 7fd08559a700 1 -- 172.21.15.157:0/1800781327 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 820590088 0 0) 0x7fd070003570 con 0x7fd0880bc750 2023-10-06T04:03:52.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.876+0000 7fd08559a700 1 -- 172.21.15.157:0/1800781327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd08806ab50 msgr2=0x7fd08806c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:52.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.876+0000 7fd08559a700 1 --2- 172.21.15.157:0/1800781327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd08806ab50 0x7fd08806c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:52.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.876+0000 7fd08559a700 1 -- 172.21.15.157:0/1800781327 >> v2:172.21.15.175:3300/0 conn(0x7fd08806a780 msgr2=0x7fd088159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:52.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.876+0000 7fd08559a700 1 --2- 172.21.15.157:0/1800781327 >> v2:172.21.15.175:3300/0 conn(0x7fd08806a780 0x7fd088159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:52.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.876+0000 7fd08559a700 1 -- 172.21.15.157:0/1800781327 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08806ea70 con 0x7fd0880bc750 2023-10-06T04:03:52.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.876+0000 7fd08559a700 1 -- 172.21.15.157:0/1800781327 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7fd0800017f0 con 0x7fd0880bc750 2023-10-06T04:03:52.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.876+0000 7fd08559a700 1 -- 172.21.15.157:0/1800781327 >> v1:172.21.15.157:6789/0 conn(0x7fd0880bc750 legacy=0x7fd0880c5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:52.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.876+0000 7fd08559a700 1 --2- 172.21.15.157:0/1800781327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd08806ab50 0x7fd070003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:52.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.876+0000 7fd08559a700 1 --2- 172.21.15.157:0/1800781327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd070004010 0x7fd070004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:52.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.876+0000 7fd08559a700 1 --2- 172.21.15.157:0/1800781327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd070004d50 0x7fd0700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:52.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.876+0000 7fd08559a700 1 -- 172.21.15.157:0/1800781327 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08815a310 con 0x7fd070004010 2023-10-06T04:03:52.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.876+0000 7fd08559a700 1 -- 172.21.15.157:0/1800781327 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07000ba60 con 0x7fd070004d50 2023-10-06T04:03:52.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.876+0000 7fd08559a700 1 -- 172.21.15.157:0/1800781327 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd07000bbf0 con 0x7fd08806ab50 2023-10-06T04:03:52.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.876+0000 7fd085d9b700 1 --2- 172.21.15.157:0/1800781327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd070004010 0x7fd070004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:52.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.876+0000 7fd08659c700 1 --2- 172.21.15.157:0/1800781327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd08806ab50 0x7fd070003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:52.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.876+0000 7fd086d9d700 1 --2- 172.21.15.157:0/1800781327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd070004d50 0x7fd0700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:52.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.876+0000 7fd085d9b700 1 -- 172.21.15.157:0/1800781327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd08806ab50 msgr2=0x7fd070003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:52.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.876+0000 7fd085d9b700 1 --2- 172.21.15.157:0/1800781327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd08806ab50 0x7fd070003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:52.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.876+0000 7fd085d9b700 1 -- 172.21.15.157:0/1800781327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd070004d50 msgr2=0x7fd0700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:52.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.876+0000 7fd085d9b700 1 --2- 172.21.15.157:0/1800781327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd070004d50 0x7fd0700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:52.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.876+0000 7fd085d9b700 1 -- 172.21.15.157:0/1800781327 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd070003570 con 0x7fd070004010 2023-10-06T04:03:52.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.876+0000 7fd086d9d700 1 --2- 172.21.15.157:0/1800781327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd070004d50 0x7fd0700051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:03:52.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.877+0000 7fd085d9b700 1 --2- 172.21.15.157:0/1800781327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd070004010 0x7fd070004810 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009c00 tx=0x7fd07c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131859127a9e8c16 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:52.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.877+0000 7fd08559a700 1 -- 172.21.15.157:0/1800781327 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c00e710 con 0x7fd070004010 2023-10-06T04:03:52.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.877+0000 7fd08559a700 1 -- 172.21.15.157:0/1800781327 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07c00e870 con 0x7fd070004010 2023-10-06T04:03:52.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.877+0000 7fd08559a700 1 -- 172.21.15.157:0/1800781327 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c00ec30 con 0x7fd070004010 2023-10-06T04:03:52.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.877+0000 7fd08cab3700 1 -- 172.21.15.157:0/1800781327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd070004010 msgr2=0x7fd070004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:52.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.877+0000 7fd08cab3700 1 --2- 172.21.15.157:0/1800781327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd070004010 0x7fd070004810 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009c00 tx=0x7fd07c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:03:52.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.877+0000 7fd086d9d700 1 -- 172.21.15.157:0/1800781327 reap_dead start 2023-10-06T04:03:52.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.877+0000 7fd08cab3700 1 -- 172.21.15.157:0/1800781327 shutdown_connections 2023-10-06T04:03:52.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.877+0000 7fd08cab3700 1 -- 172.21.15.157:0/1800781327 >> 172.21.15.157:0/1800781327 conn(0x7fd08806b510 msgr2=0x7fd08806b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:52.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.877+0000 7fd08cab3700 1 -- 172.21.15.157:0/1800781327 shutdown_connections 2023-10-06T04:03:52.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.877+0000 7fd08cab3700 1 -- 172.21.15.157:0/1800781327 wait complete. 2023-10-06T04:03:52.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.878+0000 7fd08cab3700 1 Processor -- start 2023-10-06T04:03:52.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.878+0000 7fd08cab3700 1 -- start start 2023-10-06T04:03:52.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.879+0000 7fd08cab3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd070004010 0x7fd08815c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:52.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.879+0000 7fd08cab3700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd08815ca90 0x7fd0881669d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:52.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.879+0000 7fd08cab3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0880c9e90 0x7fd0881550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:52.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.879+0000 7fd08cab3700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08815b790 con 0x7fd0880c9e90 2023-10-06T04:03:52.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.879+0000 7fd08cab3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08815cfb0 con 0x7fd070004010 2023-10-06T04:03:52.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.879+0000 7fd08cab3700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd08815d290 con 0x7fd08815ca90 2023-10-06T04:03:52.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.879+0000 7fd086d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0880c9e90 0x7fd0881550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:52.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.879+0000 7fd086d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0880c9e90 0x7fd0881550d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50582/0 (socket says 172.21.15.157:50582) 2023-10-06T04:03:52.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.879+0000 7fd08659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd070004010 0x7fd08815c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:52.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.879+0000 7fd086d9d700 1 -- 172.21.15.157:0/3127927309 learned_addr learned my addr 172.21.15.157:0/3127927309 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:52.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.879+0000 7fd085d9b700 1 --2- 172.21.15.157:0/3127927309 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd08815ca90 0x7fd0881669d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:52.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.880+0000 7fd086d9d700 1 -- 172.21.15.157:0/3127927309 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd08815ca90 msgr2=0x7fd0881669d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:52.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.880+0000 7fd086d9d700 1 --2- 172.21.15.157:0/3127927309 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd08815ca90 0x7fd0881669d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:52.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.880+0000 7fd086d9d700 1 -- 172.21.15.157:0/3127927309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd070004010 msgr2=0x7fd08815c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:52.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.880+0000 7fd086d9d700 1 --2- 172.21.15.157:0/3127927309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd070004010 0x7fd08815c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:52.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.880+0000 7fd086d9d700 1 -- 172.21.15.157:0/3127927309 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08815ba30 con 0x7fd0880c9e90 2023-10-06T04:03:52.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.880+0000 7fd086d9d700 1 --2- 172.21.15.157:0/3127927309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0880c9e90 0x7fd0881550d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd080000f80 tx=0x7fd08806e750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:52.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.880+0000 7fd0767fc700 1 -- 172.21.15.157:0/3127927309 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08000eb10 con 0x7fd0880c9e90 2023-10-06T04:03:52.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.880+0000 7fd0767fc700 1 -- 172.21.15.157:0/3127927309 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08000ec70 con 0x7fd0880c9e90 2023-10-06T04:03:52.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.880+0000 7fd08cab3700 1 -- 172.21.15.157:0/3127927309 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd08815bd10 con 0x7fd0880c9e90 2023-10-06T04:03:52.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.880+0000 7fd0767fc700 1 -- 172.21.15.157:0/3127927309 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd080013420 con 0x7fd0880c9e90 2023-10-06T04:03:52.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.880+0000 7fd08cab3700 1 -- 172.21.15.157:0/3127927309 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd088155610 con 0x7fd0880c9e90 2023-10-06T04:03:52.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.881+0000 7fd08cab3700 1 -- 172.21.15.157:0/3127927309 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd088068a60 con 0x7fd0880c9e90 2023-10-06T04:03:52.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.881+0000 7fd0767fc700 1 -- 172.21.15.157:0/3127927309 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd080013790 con 0x7fd0880c9e90 2023-10-06T04:03:52.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.881+0000 7fd0767fc700 1 --2- 172.21.15.157:0/3127927309 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd068066f60 0x7fd068069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:52.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.882+0000 7fd0767fc700 1 -- 172.21.15.157:0/3127927309 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7fd080084da0 con 0x7fd0880c9e90 2023-10-06T04:03:52.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.882+0000 7fd08659c700 1 --2- 172.21.15.157:0/3127927309 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd068066f60 0x7fd068069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:52.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.882+0000 7fd08659c700 1 --2- 172.21.15.157:0/3127927309 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd068066f60 0x7fd068069410 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd078006f00 tx=0x7fd0780091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:52.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:52.886+0000 7fd0767fc700 1 -- 172.21.15.157:0/3127927309 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd080051c60 con 0x7fd0880c9e90 2023-10-06T04:03:53.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.115+0000 7fd08cab3700 1 -- 172.21.15.157:0/3127927309 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd08815bf00 con 0x7fd0880c9e90 2023-10-06T04:03:53.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.116+0000 7fd0767fc700 1 -- 172.21.15.157:0/3127927309 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+5387 (secure 0 0 0) 0x7fd080055280 con 0x7fd0880c9e90 2023-10-06T04:03:53.118 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:03:53.119 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:46.794132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":4287},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:03:53.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.118+0000 7fd08cab3700 1 -- 172.21.15.157:0/3127927309 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd068066f60 msgr2=0x7fd068069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:53.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.118+0000 7fd08cab3700 1 --2- 172.21.15.157:0/3127927309 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd068066f60 0x7fd068069410 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd078006f00 tx=0x7fd0780091a0 comp rx=0 tx=0).stop 2023-10-06T04:03:53.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.118+0000 7fd08cab3700 1 -- 172.21.15.157:0/3127927309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0880c9e90 msgr2=0x7fd0881550d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:53.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.118+0000 7fd08cab3700 1 --2- 172.21.15.157:0/3127927309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0880c9e90 0x7fd0881550d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd080000f80 tx=0x7fd08806e750 comp rx=0 tx=0).stop 2023-10-06T04:03:53.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.118+0000 7fd08cab3700 1 -- 172.21.15.157:0/3127927309 shutdown_connections 2023-10-06T04:03:53.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.118+0000 7fd08cab3700 1 --2- 172.21.15.157:0/3127927309 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd068066f60 0x7fd068069410 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:53.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.118+0000 7fd08cab3700 1 --2- 172.21.15.157:0/3127927309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd070004010 0x7fd08815c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:53.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.118+0000 7fd08cab3700 1 --2- 172.21.15.157:0/3127927309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0880c9e90 0x7fd0881550d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:53.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.118+0000 7fd08cab3700 1 --2- 172.21.15.157:0/3127927309 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd08815ca90 0x7fd0881669d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:53.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.118+0000 7fd08cab3700 1 -- 172.21.15.157:0/3127927309 >> 172.21.15.157:0/3127927309 conn(0x7fd08806b510 msgr2=0x7fd0880c1560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:53.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.118+0000 7fd08cab3700 1 -- 172.21.15.157:0/3127927309 shutdown_connections 2023-10-06T04:03:53.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.118+0000 7fd08cab3700 1 -- 172.21.15.157:0/3127927309 wait complete. 2023-10-06T04:03:53.122 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 56 2023-10-06T04:03:53.132 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b down, removed from mdsmap 2023-10-06T04:03:53.132 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-10-06T04:03:53.132 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:03:53.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.246+0000 7f6911ad4700 1 Processor -- start 2023-10-06T04:03:53.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.246+0000 7f6911ad4700 1 -- start start 2023-10-06T04:03:53.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.247+0000 7f6911ad4700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f690c06dc40 0x7f690c14fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:53.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.247+0000 7f6911ad4700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f690c06c1f0 0x7f690c150240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:53.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.247+0000 7f6911ad4700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f690c0c9780 con 0x7f690c06cbb0 2023-10-06T04:03:53.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.247+0000 7f6911ad4700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f690c0c8c40 con 0x7f690c06c1f0 2023-10-06T04:03:53.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.247+0000 7f6911ad4700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690c06e0c0 con 0x7f690c06dc40 2023-10-06T04:03:53.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.247+0000 7f690affd700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f690c06dc40 0x7f690c14fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:53.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.247+0000 7f690bfff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f690c06c1f0 0x7f690c150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:53.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.247+0000 7f690b7fe700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f690c06cbb0 0x7f690c06d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.157:47054/0 (socket says 172.21.15.157:47054) 2023-10-06T04:03:53.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.247+0000 7f690bfff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f690c06c1f0 0x7f690c150240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:41896/0 (socket says 172.21.15.157:41896) 2023-10-06T04:03:53.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.247+0000 7f690b7fe700 1 -- 172.21.15.157:0/3244282798 learned_addr learned my addr 172.21.15.157:0/3244282798 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:03:53.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.247+0000 7f690affd700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f690c06dc40 0x7f690c14fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:45190/0 (socket says 172.21.15.157:45190) 2023-10-06T04:03:53.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.248+0000 7f690a7fc700 1 -- 172.21.15.157:0/3244282798 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3485270860 0 0) 0x7f690c0c9780 con 0x7f690c06cbb0 2023-10-06T04:03:53.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.248+0000 7f690a7fc700 1 -- 172.21.15.157:0/3244282798 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68f4003570 con 0x7f690c06cbb0 2023-10-06T04:03:53.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.248+0000 7f690affd700 1 -- 172.21.15.157:0/3244282798 >> v2:172.21.15.175:3300/0 conn(0x7f690c06c1f0 msgr2=0x7f690c150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:53.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.248+0000 7f690affd700 1 --2- 172.21.15.157:0/3244282798 >> v2:172.21.15.175:3300/0 conn(0x7f690c06c1f0 0x7f690c150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:53.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.248+0000 7f690affd700 1 -- 172.21.15.157:0/3244282798 >> v1:172.21.15.175:6789/0 conn(0x7f690c06cbb0 legacy=0x7f690c06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:53.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.248+0000 7f690affd700 1 -- 172.21.15.157:0/3244282798 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c0c8e50 con 0x7f690c06dc40 2023-10-06T04:03:53.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.248+0000 7f690bfff700 1 --2- 172.21.15.157:0/3244282798 >> v2:172.21.15.175:3300/0 conn(0x7f690c06c1f0 0x7f690c150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:03:53.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.248+0000 7f690affd700 1 --2- 172.21.15.157:0/3244282798 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f690c06dc40 0x7f690c14fd00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f69000098e0 tx=0x7f69000091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d86b3ea2f010f299 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:53.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.248+0000 7f690a7fc700 1 -- 172.21.15.157:0/3244282798 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900011630 con 0x7f690c06dc40 2023-10-06T04:03:53.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.248+0000 7f690a7fc700 1 -- 172.21.15.157:0/3244282798 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6900011790 con 0x7f690c06dc40 2023-10-06T04:03:53.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.248+0000 7f690a7fc700 1 -- 172.21.15.157:0/3244282798 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900011b30 con 0x7f690c06dc40 2023-10-06T04:03:53.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.249+0000 7f6911ad4700 1 -- 172.21.15.157:0/3244282798 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f690c06dc40 msgr2=0x7f690c14fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:53.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.249+0000 7f6911ad4700 1 --2- 172.21.15.157:0/3244282798 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f690c06dc40 0x7f690c14fd00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f69000098e0 tx=0x7f69000091a0 comp rx=0 tx=0).stop 2023-10-06T04:03:53.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.249+0000 7f6911ad4700 1 -- 172.21.15.157:0/3244282798 shutdown_connections 2023-10-06T04:03:53.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.249+0000 7f6911ad4700 1 --2- 172.21.15.157:0/3244282798 >> v2:172.21.15.175:3300/0 conn(0x7f690c06c1f0 0x7f690c150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:53.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.249+0000 7f6911ad4700 1 --2- 172.21.15.157:0/3244282798 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f690c06dc40 0x7f690c14fd00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:53.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.249+0000 7f6911ad4700 1 -- 172.21.15.157:0/3244282798 >> 172.21.15.157:0/3244282798 conn(0x7f690c06f230 msgr2=0x7f690c06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:53.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.249+0000 7f6911ad4700 1 -- 172.21.15.157:0/3244282798 shutdown_connections 2023-10-06T04:03:53.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.249+0000 7f6911ad4700 1 -- 172.21.15.157:0/3244282798 wait complete. 2023-10-06T04:03:53.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.250+0000 7f6911ad4700 1 Processor -- start 2023-10-06T04:03:53.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.250+0000 7f6911ad4700 1 -- start start 2023-10-06T04:03:53.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.251+0000 7f6911ad4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690c06c1f0 0x7f690c153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:53.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.251+0000 7f6911ad4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f690c06cbb0 0x7f690c153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:53.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.251+0000 7f6911ad4700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f690c150930 0x7f690c151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:53.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.251+0000 7f6911ad4700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c153570 con 0x7f690c06cbb0 2023-10-06T04:03:53.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.251+0000 7f6911ad4700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c1537e0 con 0x7f690c06c1f0 2023-10-06T04:03:53.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.251+0000 7f6911ad4700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690c156550 con 0x7f690c150930 2023-10-06T04:03:53.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.251+0000 7f690affd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f690c06cbb0 0x7f690c153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:53.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.251+0000 7f690affd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f690c06cbb0 0x7f690c153e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54224/0 (socket says 172.21.15.157:54224) 2023-10-06T04:03:53.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.251+0000 7f690affd700 1 -- 172.21.15.157:0/3881211704 learned_addr learned my addr 172.21.15.157:0/3881211704 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:53.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.251+0000 7f690bfff700 1 --2- 172.21.15.157:0/3881211704 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f690c150930 0x7f690c151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:53.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.251+0000 7f690b7fe700 1 --2- 172.21.15.157:0/3881211704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690c06c1f0 0x7f690c153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:53.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.251+0000 7f690affd700 1 -- 172.21.15.157:0/3881211704 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f690c150930 msgr2=0x7f690c151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:53.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.251+0000 7f690affd700 1 --2- 172.21.15.157:0/3881211704 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f690c150930 0x7f690c151680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:53.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.251+0000 7f690affd700 1 -- 172.21.15.157:0/3881211704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690c06c1f0 msgr2=0x7f690c153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:53.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.251+0000 7f690affd700 1 --2- 172.21.15.157:0/3881211704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690c06c1f0 0x7f690c153030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:53.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.251+0000 7f690affd700 1 -- 172.21.15.157:0/3881211704 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6904009a20 con 0x7f690c06cbb0 2023-10-06T04:03:53.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.251+0000 7f690affd700 1 --2- 172.21.15.157:0/3881211704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f690c06cbb0 0x7f690c153e00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f690c0c8d80 tx=0x7f6900017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:53.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.251+0000 7f68f37fe700 1 -- 172.21.15.157:0/3881211704 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900011b10 con 0x7f690c06cbb0 2023-10-06T04:03:53.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.251+0000 7f68f37fe700 1 -- 172.21.15.157:0/3881211704 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6900000cc0 con 0x7f690c06cbb0 2023-10-06T04:03:53.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.251+0000 7f6911ad4700 1 -- 172.21.15.157:0/3881211704 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6900009420 con 0x7f690c06cbb0 2023-10-06T04:03:53.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.251+0000 7f68f37fe700 1 -- 172.21.15.157:0/3881211704 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900025620 con 0x7f690c06cbb0 2023-10-06T04:03:53.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.251+0000 7f6911ad4700 1 -- 172.21.15.157:0/3881211704 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f690c156b10 con 0x7f690c06cbb0 2023-10-06T04:03:53.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.252+0000 7f6911ad4700 1 -- 172.21.15.157:0/3881211704 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f690c089ca0 con 0x7f690c06cbb0 2023-10-06T04:03:53.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.253+0000 7f68f37fe700 1 -- 172.21.15.157:0/3881211704 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6900025a10 con 0x7f690c06cbb0 2023-10-06T04:03:53.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.253+0000 7f68f37fe700 1 --2- 172.21.15.157:0/3881211704 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f68f8066f30 0x7f68f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:53.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.253+0000 7f68f37fe700 1 -- 172.21.15.157:0/3881211704 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7f6900085fc0 con 0x7f690c06cbb0 2023-10-06T04:03:53.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.253+0000 7f690b7fe700 1 --2- 172.21.15.157:0/3881211704 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f68f8066f30 0x7f68f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:53.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.254+0000 7f690b7fe700 1 --2- 172.21.15.157:0/3881211704 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f68f8066f30 0x7f68f80693e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f68fc007fd0 tx=0x7f68fc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:53.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.258+0000 7f68f37fe700 1 -- 172.21.15.157:0/3881211704 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f69000b75a0 con 0x7f690c06cbb0 2023-10-06T04:03:53.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.488+0000 7f6911ad4700 1 -- 172.21.15.157:0/3881211704 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f690c0c9130 con 0x7f690c06cbb0 2023-10-06T04:03:53.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.488+0000 7f68f37fe700 1 -- 172.21.15.157:0/3881211704 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+5387 (secure 0 0 0) 0x7f6900052f40 con 0x7f690c06cbb0 2023-10-06T04:03:53.490 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:03:53.490 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:46.794132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":4287},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:03:53.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.491+0000 7f6911ad4700 1 -- 172.21.15.157:0/3881211704 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f68f8066f30 msgr2=0x7f68f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:53.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.491+0000 7f6911ad4700 1 --2- 172.21.15.157:0/3881211704 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f68f8066f30 0x7f68f80693e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f68fc007fd0 tx=0x7f68fc009040 comp rx=0 tx=0).stop 2023-10-06T04:03:53.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.491+0000 7f6911ad4700 1 -- 172.21.15.157:0/3881211704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f690c06cbb0 msgr2=0x7f690c153e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:53.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.491+0000 7f6911ad4700 1 --2- 172.21.15.157:0/3881211704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f690c06cbb0 0x7f690c153e00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f690c0c8d80 tx=0x7f6900017000 comp rx=0 tx=0).stop 2023-10-06T04:03:53.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.491+0000 7f6911ad4700 1 -- 172.21.15.157:0/3881211704 shutdown_connections 2023-10-06T04:03:53.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.491+0000 7f6911ad4700 1 --2- 172.21.15.157:0/3881211704 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f68f8066f30 0x7f68f80693e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:53.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.491+0000 7f6911ad4700 1 --2- 172.21.15.157:0/3881211704 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690c06c1f0 0x7f690c153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:53.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.491+0000 7f6911ad4700 1 --2- 172.21.15.157:0/3881211704 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f690c150930 0x7f690c151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:53.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.491+0000 7f6911ad4700 1 --2- 172.21.15.157:0/3881211704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f690c06cbb0 0x7f690c153e00 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:53.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.491+0000 7f6911ad4700 1 -- 172.21.15.157:0/3881211704 >> 172.21.15.157:0/3881211704 conn(0x7f690c06f230 msgr2=0x7f690c14caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:53.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.491+0000 7f6911ad4700 1 -- 172.21.15.157:0/3881211704 shutdown_connections 2023-10-06T04:03:53.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:53.491+0000 7f6911ad4700 1 -- 172.21.15.157:0/3881211704 wait complete. 2023-10-06T04:03:53.494 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 56 2023-10-06T04:03:53.504 INFO:tasks.mds_thrash.fs.[cephfs]:cluster has 2 actives (max_mds is 2), no MDS can replace rank 3 2023-10-06T04:03:53.504 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 5 secs before reviving mds.b 2023-10-06T04:03:56.444 INFO:tasks.workunit.client.0.smithi157.stderr:+ set -e 2023-10-06T04:03:56.444 INFO:tasks.workunit.client.0.smithi157.stderr:+ touch foo 2023-10-06T04:03:56.445 INFO:tasks.workunit.client.0.smithi157.stderr:+ chmod +x foo 2023-10-06T04:03:56.446 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir .snap/s 2023-10-06T04:03:56.471 INFO:tasks.workunit.client.0.smithi157.stderr:+ find .snap/s/foo -executable 2023-10-06T04:03:56.471 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep foo 2023-10-06T04:03:56.473 INFO:tasks.workunit.client.0.smithi157.stdout:.snap/s/foo 2023-10-06T04:03:56.474 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir .snap/s 2023-10-06T04:03:56.480 INFO:tasks.workunit.client.0.smithi157.stderr:+ rm foo 2023-10-06T04:03:56.483 INFO:tasks.workunit.client.0.smithi157.stderr:+ echo OK 2023-10-06T04:03:56.484 INFO:tasks.workunit.client.0.smithi157.stdout:OK 2023-10-06T04:03:56.485 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-06T04:03:56.485 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-06T04:03:57.286 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-capwb.sh... 2023-10-06T04:03:57.287 DEBUG:teuthology.orchestra.run.smithi157:workunit test fs/snaps/snaptest-capwb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-capwb.sh 2023-10-06T04:03:57.315 INFO:tasks.workunit.client.0.smithi157.stderr:+ set -e 2023-10-06T04:03:57.315 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir foo 2023-10-06T04:03:57.317 INFO:tasks.workunit.client.0.smithi157.stderr:+ echo x 2023-10-06T04:03:57.317 INFO:tasks.workunit.client.0.smithi157.stderr:+ sync 2023-10-06T04:03:57.525 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir foo/.snap/ss 2023-10-06T04:03:58.225 INFO:tasks.workunit.client.0.smithi157.stderr:+ ln foo/x foo/xx 2023-10-06T04:03:58.227 INFO:tasks.workunit.client.0.smithi157.stderr:+ cat foo/.snap/ss/x 2023-10-06T04:03:58.233 INFO:tasks.workunit.client.0.smithi157.stdout:x 2023-10-06T04:03:58.233 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir foo/.snap/ss 2023-10-06T04:03:58.239 INFO:tasks.workunit.client.0.smithi157.stderr:+ echo a 2023-10-06T04:03:58.239 INFO:tasks.workunit.client.0.smithi157.stderr:+ echo b 2023-10-06T04:03:58.240 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir foo/.snap/s 2023-10-06T04:03:58.265 INFO:tasks.workunit.client.0.smithi157.stderr:++ cat foo/.snap/s/a 2023-10-06T04:03:58.273 INFO:tasks.workunit.client.0.smithi157.stderr:+ r=a 2023-10-06T04:03:58.273 INFO:tasks.workunit.client.0.smithi157.stderr:+ '[' -z a ']' 2023-10-06T04:03:58.274 INFO:tasks.workunit.client.0.smithi157.stderr:+ ln foo/b foo/b2 2023-10-06T04:03:58.275 INFO:tasks.workunit.client.0.smithi157.stderr:+ cat foo/.snap/s/b 2023-10-06T04:03:58.505 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.b 2023-10-06T04:03:58.505 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T04:03:58.505 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T04:03:58.509 INFO:tasks.ceph.mds.b:Started 2023-10-06T04:03:58.509 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:03:58.586 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2023-10-06T04:03:58.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.632+0000 7f9227fff700 1 Processor -- start 2023-10-06T04:03:58.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.632+0000 7f9227fff700 1 -- start start 2023-10-06T04:03:58.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.632+0000 7f9227fff700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92280c8790 0x7f9228155b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:58.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.632+0000 7f9227fff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f92280c8b60 0x7f92280b1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:58.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.632+0000 7f9227fff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92280af480 con 0x7f92280cc050 2023-10-06T04:03:58.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.632+0000 7f9227fff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92280af760 con 0x7f92280c8b60 2023-10-06T04:03:58.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.632+0000 7f9227fff700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92280afa40 con 0x7f92280c8790 2023-10-06T04:03:58.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.633+0000 7f92267fc700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f92280c8b60 0x7f92280b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:58.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.633+0000 7f9226ffd700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92280c8790 0x7f9228155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:58.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.633+0000 7f92277fe700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f92280cc050 0x7f92280c75d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.157:47058/0 (socket says 172.21.15.157:47058) 2023-10-06T04:03:58.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.633+0000 7f92277fe700 1 -- 172.21.15.157:0/2277903453 learned_addr learned my addr 172.21.15.157:0/2277903453 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:03:58.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.633+0000 7f9225ffb700 1 -- 172.21.15.157:0/2277903453 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2624090645 0 0) 0x7f92280af480 con 0x7f92280cc050 2023-10-06T04:03:58.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.633+0000 7f9225ffb700 1 -- 172.21.15.157:0/2277903453 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9210003570 con 0x7f92280cc050 2023-10-06T04:03:58.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.633+0000 7f92267fc700 1 -- 172.21.15.157:0/2277903453 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92280c8790 msgr2=0x7f9228155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:03:58.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.633+0000 7f92267fc700 1 --2- 172.21.15.157:0/2277903453 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92280c8790 0x7f9228155b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:58.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.633+0000 7f92267fc700 1 -- 172.21.15.157:0/2277903453 >> v1:172.21.15.175:6789/0 conn(0x7f92280cc050 legacy=0x7f92280c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:58.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.633+0000 7f92267fc700 1 -- 172.21.15.157:0/2277903453 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92280afd20 con 0x7f92280c8b60 2023-10-06T04:03:58.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.633+0000 7f9226ffd700 1 --2- 172.21.15.157:0/2277903453 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92280c8790 0x7f9228155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:03:58.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.633+0000 7f92267fc700 1 --2- 172.21.15.157:0/2277903453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92280c8b60 0x7f92280b1920 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f921c0098e0 tx=0x7f921c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e833c8419e7527c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:58.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.633+0000 7f9225ffb700 1 -- 172.21.15.157:0/2277903453 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c011630 con 0x7f92280c8b60 2023-10-06T04:03:58.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.633+0000 7f9225ffb700 1 -- 172.21.15.157:0/2277903453 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921c011790 con 0x7f92280c8b60 2023-10-06T04:03:58.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.633+0000 7f9225ffb700 1 -- 172.21.15.157:0/2277903453 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c011b30 con 0x7f92280c8b60 2023-10-06T04:03:58.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.634+0000 7f9227fff700 1 -- 172.21.15.157:0/2277903453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92280c8b60 msgr2=0x7f92280b1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:58.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.634+0000 7f9227fff700 1 --2- 172.21.15.157:0/2277903453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92280c8b60 0x7f92280b1920 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f921c0098e0 tx=0x7f921c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:03:58.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.634+0000 7f9227fff700 1 -- 172.21.15.157:0/2277903453 shutdown_connections 2023-10-06T04:03:58.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.634+0000 7f9227fff700 1 --2- 172.21.15.157:0/2277903453 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92280c8790 0x7f9228155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:58.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.634+0000 7f9227fff700 1 --2- 172.21.15.157:0/2277903453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92280c8b60 0x7f92280b1920 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:58.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.634+0000 7f9227fff700 1 -- 172.21.15.157:0/2277903453 >> 172.21.15.157:0/2277903453 conn(0x7f92280cc540 msgr2=0x7f92280cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:58.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.634+0000 7f9227fff700 1 -- 172.21.15.157:0/2277903453 shutdown_connections 2023-10-06T04:03:58.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.634+0000 7f9227fff700 1 -- 172.21.15.157:0/2277903453 wait complete. 2023-10-06T04:03:58.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.635+0000 7f9227fff700 1 Processor -- start 2023-10-06T04:03:58.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.635+0000 7f9227fff700 1 -- start start 2023-10-06T04:03:58.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.636+0000 7f9227fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92280c8790 0x7f92280c5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:58.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.636+0000 7f9227fff700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92280c8b60 0x7f92280bfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:58.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.636+0000 7f9227fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92280cc050 0x7f922814fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:58.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.636+0000 7f9227fff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92280c2000 con 0x7f92280c8790 2023-10-06T04:03:58.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.636+0000 7f9227fff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9228166970 con 0x7f92280cc050 2023-10-06T04:03:58.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.636+0000 7f9227fff700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92280c3030 con 0x7f92280c8b60 2023-10-06T04:03:58.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.636+0000 7f9226ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92280c8790 0x7f92280c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:58.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.636+0000 7f9226ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92280c8790 0x7f92280c5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54234/0 (socket says 172.21.15.157:54234) 2023-10-06T04:03:58.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.636+0000 7f9226ffd700 1 -- 172.21.15.157:0/2623334220 learned_addr learned my addr 172.21.15.157:0/2623334220 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:03:58.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.636+0000 7f92267fc700 1 --2- 172.21.15.157:0/2623334220 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92280c8b60 0x7f92280bfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:58.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.636+0000 7f92277fe700 1 --2- 172.21.15.157:0/2623334220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92280cc050 0x7f922814fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:58.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.636+0000 7f9226ffd700 1 -- 172.21.15.157:0/2623334220 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92280c8b60 msgr2=0x7f92280bfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:58.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.636+0000 7f9226ffd700 1 --2- 172.21.15.157:0/2623334220 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92280c8b60 0x7f92280bfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:58.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.636+0000 7f9226ffd700 1 -- 172.21.15.157:0/2623334220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92280cc050 msgr2=0x7f922814fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:58.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.636+0000 7f9226ffd700 1 --2- 172.21.15.157:0/2623334220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92280cc050 0x7f922814fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:58.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.636+0000 7f9226ffd700 1 -- 172.21.15.157:0/2623334220 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9218009420 con 0x7f92280c8790 2023-10-06T04:03:58.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.637+0000 7f9226ffd700 1 --2- 172.21.15.157:0/2623334220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92280c8790 0x7f92280c5860 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f921800ec80 tx=0x7f92180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:58.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.637+0000 7f920effd700 1 -- 172.21.15.157:0/2623334220 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921800c920 con 0x7f92280c8790 2023-10-06T04:03:58.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.637+0000 7f920effd700 1 -- 172.21.15.157:0/2623334220 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921800ca80 con 0x7f92280c8790 2023-10-06T04:03:58.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.637+0000 7f9227fff700 1 -- 172.21.15.157:0/2623334220 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f921c009420 con 0x7f92280c8790 2023-10-06T04:03:58.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.637+0000 7f920effd700 1 -- 172.21.15.157:0/2623334220 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921800ce00 con 0x7f92280c8790 2023-10-06T04:03:58.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.637+0000 7f9227fff700 1 -- 172.21.15.157:0/2623334220 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92280c2550 con 0x7f92280c8790 2023-10-06T04:03:58.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.638+0000 7f9227fff700 1 -- 172.21.15.157:0/2623334220 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9228063d70 con 0x7f92280c8790 2023-10-06T04:03:58.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.638+0000 7f920effd700 1 -- 172.21.15.157:0/2623334220 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9218013680 con 0x7f92280c8790 2023-10-06T04:03:58.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.639+0000 7f920effd700 1 --2- 172.21.15.157:0/2623334220 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9214066f30 0x7f92140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:03:58.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.639+0000 7f920effd700 1 -- 172.21.15.157:0/2623334220 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6652+0+0 (secure 0 0 0) 0x7f9218084fd0 con 0x7f92280c8790 2023-10-06T04:03:58.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.639+0000 7f92267fc700 1 --2- 172.21.15.157:0/2623334220 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9214066f30 0x7f92140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:03:58.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.639+0000 7f92267fc700 1 --2- 172.21.15.157:0/2623334220 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9214066f30 0x7f92140693e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f921c0091e0 tx=0x7f921c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:03:58.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.643+0000 7f920effd700 1 -- 172.21.15.157:0/2623334220 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9218051e90 con 0x7f92280c8790 2023-10-06T04:03:58.732 INFO:tasks.workunit.client.0.smithi157.stdout:b 2023-10-06T04:03:58.732 INFO:tasks.workunit.client.0.smithi157.stderr:+ echo 'this used to hang:' 2023-10-06T04:03:58.732 INFO:tasks.workunit.client.0.smithi157.stdout:this used to hang: 2023-10-06T04:03:58.733 INFO:tasks.workunit.client.0.smithi157.stderr:+ echo more 2023-10-06T04:03:58.733 INFO:tasks.workunit.client.0.smithi157.stdout:oh, it didn't hang! good job. 2023-10-06T04:03:58.733 INFO:tasks.workunit.client.0.smithi157.stderr:+ echo 'oh, it didn'\''t hang! good job.' 2023-10-06T04:03:58.733 INFO:tasks.workunit.client.0.smithi157.stderr:+ cat foo/b 2023-10-06T04:03:58.734 INFO:tasks.workunit.client.0.smithi157.stdout:b 2023-10-06T04:03:58.734 INFO:tasks.workunit.client.0.smithi157.stdout:more 2023-10-06T04:03:58.734 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir foo/.snap/s 2023-10-06T04:03:58.739 INFO:tasks.workunit.client.0.smithi157.stderr:+ rm -r foo 2023-10-06T04:03:58.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.879+0000 7f9227fff700 1 -- 172.21.15.157:0/2623334220 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92280b1140 con 0x7f92280c8790 2023-10-06T04:03:58.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.880+0000 7f920effd700 1 -- 172.21.15.157:0/2623334220 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+6147 (secure 0 0 0) 0x7f92180554b0 con 0x7f92280c8790 2023-10-06T04:03:58.881 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:03:58.882 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:46.794132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":4287},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:03:58.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.882+0000 7f9227fff700 1 -- 172.21.15.157:0/2623334220 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9214066f30 msgr2=0x7f92140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:58.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.882+0000 7f9227fff700 1 --2- 172.21.15.157:0/2623334220 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9214066f30 0x7f92140693e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f921c0091e0 tx=0x7f921c017000 comp rx=0 tx=0).stop 2023-10-06T04:03:58.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.882+0000 7f9227fff700 1 -- 172.21.15.157:0/2623334220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92280c8790 msgr2=0x7f92280c5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:03:58.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.882+0000 7f9227fff700 1 --2- 172.21.15.157:0/2623334220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92280c8790 0x7f92280c5860 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f921800ec80 tx=0x7f92180091a0 comp rx=0 tx=0).stop 2023-10-06T04:03:58.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.882+0000 7f9227fff700 1 -- 172.21.15.157:0/2623334220 shutdown_connections 2023-10-06T04:03:58.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.882+0000 7f9227fff700 1 --2- 172.21.15.157:0/2623334220 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9214066f30 0x7f92140693e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:58.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.882+0000 7f9227fff700 1 --2- 172.21.15.157:0/2623334220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92280cc050 0x7f922814fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:58.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.882+0000 7f9227fff700 1 --2- 172.21.15.157:0/2623334220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92280c8790 0x7f92280c5860 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:58.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.882+0000 7f9227fff700 1 --2- 172.21.15.157:0/2623334220 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92280c8b60 0x7f92280bfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:03:58.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.882+0000 7f9227fff700 1 -- 172.21.15.157:0/2623334220 >> 172.21.15.157:0/2623334220 conn(0x7f92280cc540 msgr2=0x7f922806d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:03:58.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.882+0000 7f9227fff700 1 -- 172.21.15.157:0/2623334220 shutdown_connections 2023-10-06T04:03:58.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:03:58.882+0000 7f9227fff700 1 -- 172.21.15.157:0/2623334220 wait complete. 2023-10-06T04:03:58.885 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 57 2023-10-06T04:03:58.895 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b reported in up:standby state 2023-10-06T04:03:58.895 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 12 secs before thrashing 2023-10-06T04:03:59.153 INFO:tasks.workunit.client.0.smithi157.stderr:+ echo OK 2023-10-06T04:03:59.154 INFO:tasks.workunit.client.0.smithi157.stdout:OK 2023-10-06T04:03:59.154 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-06T04:03:59.154 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-06T04:03:59.794 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-dir-rename.sh... 2023-10-06T04:03:59.794 DEBUG:teuthology.orchestra.run.smithi157:workunit test fs/snaps/snaptest-dir-rename.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-dir-rename.sh 2023-10-06T04:04:01.433 INFO:tasks.workunit.client.0.smithi157.stderr:+ set -e 2023-10-06T04:04:01.433 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a 2023-10-06T04:04:01.435 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir .snap/bar 2023-10-06T04:04:02.432 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/foo 2023-10-06T04:04:02.902 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a/.snap/foo 2023-10-06T04:04:02.908 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a 2023-10-06T04:04:02.911 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/bar/a 2023-10-06T04:04:02.915 INFO:tasks.workunit.client.0.smithi157.stdout: File: .snap/bar/a 2023-10-06T04:04:02.915 INFO:tasks.workunit.client.0.smithi157.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-10-06T04:04:02.915 INFO:tasks.workunit.client.0.smithi157.stdout:Device: feh/254d Inode: 1099511632305 Links: 2 2023-10-06T04:04:02.915 INFO:tasks.workunit.client.0.smithi157.stdout:Access: (0775/drwxrwxr-x) Uid: ( 1000/ ubuntu) Gid: ( 1275/ ubuntu) 2023-10-06T04:04:02.915 INFO:tasks.workunit.client.0.smithi157.stdout:Context: unconfined_u:object_r:cephfs_t:s0 2023-10-06T04:04:02.915 INFO:tasks.workunit.client.0.smithi157.stdout:Access: 2023-10-06 04:04:01.432635372 +0000 2023-10-06T04:04:02.916 INFO:tasks.workunit.client.0.smithi157.stdout:Modify: 2023-10-06 04:04:01.432635372 +0000 2023-10-06T04:04:02.916 INFO:tasks.workunit.client.0.smithi157.stdout:Change: 2023-10-06 04:04:01.432635372 +0000 2023-10-06T04:04:02.916 INFO:tasks.workunit.client.0.smithi157.stdout: Birth: - 2023-10-06T04:04:02.916 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir .snap/bar 2023-10-06T04:04:02.923 INFO:tasks.workunit.client.0.smithi157.stderr:+ echo OK 2023-10-06T04:04:02.924 INFO:tasks.workunit.client.0.smithi157.stdout:OK 2023-10-06T04:04:02.924 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-06T04:04:02.925 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-06T04:04:03.221 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-double-null.sh... 2023-10-06T04:04:03.221 DEBUG:teuthology.orchestra.run.smithi157:workunit test fs/snaps/snaptest-double-null.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-double-null.sh 2023-10-06T04:04:03.251 INFO:tasks.workunit.client.0.smithi157.stderr:+ set -e 2023-10-06T04:04:03.252 INFO:tasks.workunit.client.0.smithi157.stderr:++ seq 1 20 2023-10-06T04:04:03.252 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 20` 2023-10-06T04:04:03.252 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a 2023-10-06T04:04:03.254 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/one 2023-10-06T04:04:03.254 INFO:tasks.workunit.client.0.smithi157.stderr:+ cat 2023-10-06T04:04:03.278 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/two 2023-10-06T04:04:03.299 INFO:tasks.workunit.client.0.smithi157.stderr:+ chmod 777 a/foo 2023-10-06T04:04:03.300 INFO:tasks.workunit.client.0.smithi157.stderr:+ sync 2023-10-06T04:04:03.832 INFO:tasks.workunit.client.0.smithi157.stderr:+ ps 2023-10-06T04:04:03.849 INFO:tasks.workunit.client.0.smithi157.stdout: PID TTY TIME CMD 2023-10-06T04:04:03.849 INFO:tasks.workunit.client.0.smithi157.stdout: 6131 ? 00:00:00 systemd 2023-10-06T04:04:03.849 INFO:tasks.workunit.client.0.smithi157.stdout: 6134 ? 00:00:00 (sd-pam) 2023-10-06T04:04:03.849 INFO:tasks.workunit.client.0.smithi157.stdout: 78246 ? 00:00:00 sshd 2023-10-06T04:04:03.849 INFO:tasks.workunit.client.0.smithi157.stdout: 106071 ? 00:00:00 timeout 2023-10-06T04:04:03.850 INFO:tasks.workunit.client.0.smithi157.stdout: 106088 ? 00:00:00 snaptest-double 2023-10-06T04:04:03.850 INFO:tasks.workunit.client.0.smithi157.stdout: 106096 ? 00:00:00 ps 2023-10-06T04:04:03.850 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-06T04:04:03.862 INFO:tasks.workunit.client.0.smithi157.stderr:+ rm a/foo 2023-10-06T04:04:03.865 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a 2023-10-06T04:04:03.872 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 20` 2023-10-06T04:04:03.872 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a 2023-10-06T04:04:03.874 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/one 2023-10-06T04:04:03.874 INFO:tasks.workunit.client.0.smithi157.stderr:+ cat 2023-10-06T04:04:03.900 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/two 2023-10-06T04:04:03.921 INFO:tasks.workunit.client.0.smithi157.stderr:+ chmod 777 a/foo 2023-10-06T04:04:03.922 INFO:tasks.workunit.client.0.smithi157.stderr:+ sync 2023-10-06T04:04:03.989 INFO:tasks.workunit.client.0.smithi157.stderr:+ ps 2023-10-06T04:04:03.998 INFO:tasks.workunit.client.0.smithi157.stdout: PID TTY TIME CMD 2023-10-06T04:04:03.999 INFO:tasks.workunit.client.0.smithi157.stdout: 6131 ? 00:00:00 systemd 2023-10-06T04:04:03.999 INFO:tasks.workunit.client.0.smithi157.stdout: 6134 ? 00:00:00 (sd-pam) 2023-10-06T04:04:03.999 INFO:tasks.workunit.client.0.smithi157.stdout: 78246 ? 00:00:00 sshd 2023-10-06T04:04:03.999 INFO:tasks.workunit.client.0.smithi157.stdout: 106071 ? 00:00:00 timeout 2023-10-06T04:04:03.999 INFO:tasks.workunit.client.0.smithi157.stdout: 106088 ? 00:00:00 snaptest-double 2023-10-06T04:04:03.999 INFO:tasks.workunit.client.0.smithi157.stdout: 106106 ? 00:00:00 ps 2023-10-06T04:04:03.999 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-06T04:04:04.010 INFO:tasks.workunit.client.0.smithi157.stderr:+ rm a/foo 2023-10-06T04:04:04.012 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a 2023-10-06T04:04:04.019 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 20` 2023-10-06T04:04:04.019 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a 2023-10-06T04:04:04.021 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/one 2023-10-06T04:04:04.021 INFO:tasks.workunit.client.0.smithi157.stderr:+ cat 2023-10-06T04:04:04.046 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/two 2023-10-06T04:04:04.066 INFO:tasks.workunit.client.0.smithi157.stderr:+ chmod 777 a/foo 2023-10-06T04:04:04.067 INFO:tasks.workunit.client.0.smithi157.stderr:+ sync 2023-10-06T04:04:04.120 INFO:tasks.workunit.client.0.smithi157.stderr:+ ps 2023-10-06T04:04:04.129 INFO:tasks.workunit.client.0.smithi157.stdout: PID TTY TIME CMD 2023-10-06T04:04:04.129 INFO:tasks.workunit.client.0.smithi157.stdout: 6131 ? 00:00:00 systemd 2023-10-06T04:04:04.129 INFO:tasks.workunit.client.0.smithi157.stdout: 6134 ? 00:00:00 (sd-pam) 2023-10-06T04:04:04.129 INFO:tasks.workunit.client.0.smithi157.stdout: 78246 ? 00:00:00 sshd 2023-10-06T04:04:04.129 INFO:tasks.workunit.client.0.smithi157.stdout: 106071 ? 00:00:00 timeout 2023-10-06T04:04:04.129 INFO:tasks.workunit.client.0.smithi157.stdout: 106088 ? 00:00:00 snaptest-double 2023-10-06T04:04:04.130 INFO:tasks.workunit.client.0.smithi157.stdout: 106116 ? 00:00:00 ps 2023-10-06T04:04:04.130 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-06T04:04:04.140 INFO:tasks.workunit.client.0.smithi157.stderr:+ rm a/foo 2023-10-06T04:04:04.142 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a 2023-10-06T04:04:04.148 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 20` 2023-10-06T04:04:04.148 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a 2023-10-06T04:04:04.150 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/one 2023-10-06T04:04:04.150 INFO:tasks.workunit.client.0.smithi157.stderr:+ cat 2023-10-06T04:04:04.577 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/two 2023-10-06T04:04:04.804 INFO:tasks.workunit.client.0.smithi157.stderr:+ chmod 777 a/foo 2023-10-06T04:04:04.805 INFO:tasks.workunit.client.0.smithi157.stderr:+ sync 2023-10-06T04:04:04.880 INFO:tasks.workunit.client.0.smithi157.stderr:+ ps 2023-10-06T04:04:04.889 INFO:tasks.workunit.client.0.smithi157.stdout: PID TTY TIME CMD 2023-10-06T04:04:04.889 INFO:tasks.workunit.client.0.smithi157.stdout: 6131 ? 00:00:00 systemd 2023-10-06T04:04:04.890 INFO:tasks.workunit.client.0.smithi157.stdout: 6134 ? 00:00:00 (sd-pam) 2023-10-06T04:04:04.890 INFO:tasks.workunit.client.0.smithi157.stdout: 78246 ? 00:00:00 sshd 2023-10-06T04:04:04.890 INFO:tasks.workunit.client.0.smithi157.stdout: 106071 ? 00:00:00 timeout 2023-10-06T04:04:04.890 INFO:tasks.workunit.client.0.smithi157.stdout: 106088 ? 00:00:00 snaptest-double 2023-10-06T04:04:04.890 INFO:tasks.workunit.client.0.smithi157.stdout: 106126 ? 00:00:00 ps 2023-10-06T04:04:04.890 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-06T04:04:04.901 INFO:tasks.workunit.client.0.smithi157.stderr:+ rm a/foo 2023-10-06T04:04:04.904 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a 2023-10-06T04:04:04.911 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 20` 2023-10-06T04:04:04.911 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a 2023-10-06T04:04:04.913 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/one 2023-10-06T04:04:04.913 INFO:tasks.workunit.client.0.smithi157.stderr:+ cat 2023-10-06T04:04:05.186 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/two 2023-10-06T04:04:05.201 INFO:tasks.workunit.client.0.smithi157.stderr:+ chmod 777 a/foo 2023-10-06T04:04:05.202 INFO:tasks.workunit.client.0.smithi157.stderr:+ sync 2023-10-06T04:04:05.290 INFO:tasks.workunit.client.0.smithi157.stderr:+ ps 2023-10-06T04:04:05.298 INFO:tasks.workunit.client.0.smithi157.stdout: PID TTY TIME CMD 2023-10-06T04:04:05.298 INFO:tasks.workunit.client.0.smithi157.stdout: 6131 ? 00:00:00 systemd 2023-10-06T04:04:05.298 INFO:tasks.workunit.client.0.smithi157.stdout: 6134 ? 00:00:00 (sd-pam) 2023-10-06T04:04:05.299 INFO:tasks.workunit.client.0.smithi157.stdout: 78246 ? 00:00:00 sshd 2023-10-06T04:04:05.299 INFO:tasks.workunit.client.0.smithi157.stdout: 106071 ? 00:00:00 timeout 2023-10-06T04:04:05.299 INFO:tasks.workunit.client.0.smithi157.stdout: 106088 ? 00:00:00 snaptest-double 2023-10-06T04:04:05.299 INFO:tasks.workunit.client.0.smithi157.stdout: 106136 ? 00:00:00 ps 2023-10-06T04:04:05.299 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-06T04:04:05.311 INFO:tasks.workunit.client.0.smithi157.stderr:+ rm a/foo 2023-10-06T04:04:05.313 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a 2023-10-06T04:04:05.321 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 20` 2023-10-06T04:04:05.321 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a 2023-10-06T04:04:05.323 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/one 2023-10-06T04:04:05.323 INFO:tasks.workunit.client.0.smithi157.stderr:+ cat 2023-10-06T04:04:05.341 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/two 2023-10-06T04:04:05.356 INFO:tasks.workunit.client.0.smithi157.stderr:+ chmod 777 a/foo 2023-10-06T04:04:05.357 INFO:tasks.workunit.client.0.smithi157.stderr:+ sync 2023-10-06T04:04:05.414 INFO:tasks.workunit.client.0.smithi157.stderr:+ ps 2023-10-06T04:04:05.423 INFO:tasks.workunit.client.0.smithi157.stdout: PID TTY TIME CMD 2023-10-06T04:04:05.423 INFO:tasks.workunit.client.0.smithi157.stdout: 6131 ? 00:00:00 systemd 2023-10-06T04:04:05.423 INFO:tasks.workunit.client.0.smithi157.stdout: 6134 ? 00:00:00 (sd-pam) 2023-10-06T04:04:05.423 INFO:tasks.workunit.client.0.smithi157.stdout: 78246 ? 00:00:00 sshd 2023-10-06T04:04:05.423 INFO:tasks.workunit.client.0.smithi157.stdout: 106071 ? 00:00:00 timeout 2023-10-06T04:04:05.423 INFO:tasks.workunit.client.0.smithi157.stdout: 106088 ? 00:00:00 snaptest-double 2023-10-06T04:04:05.423 INFO:tasks.workunit.client.0.smithi157.stdout: 106146 ? 00:00:00 ps 2023-10-06T04:04:05.424 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-06T04:04:05.434 INFO:tasks.workunit.client.0.smithi157.stderr:+ rm a/foo 2023-10-06T04:04:05.436 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a 2023-10-06T04:04:05.443 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 20` 2023-10-06T04:04:05.443 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a 2023-10-06T04:04:05.445 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/one 2023-10-06T04:04:05.445 INFO:tasks.workunit.client.0.smithi157.stderr:+ cat 2023-10-06T04:04:05.464 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/two 2023-10-06T04:04:05.478 INFO:tasks.workunit.client.0.smithi157.stderr:+ chmod 777 a/foo 2023-10-06T04:04:05.479 INFO:tasks.workunit.client.0.smithi157.stderr:+ sync 2023-10-06T04:04:05.549 INFO:tasks.workunit.client.0.smithi157.stderr:+ ps 2023-10-06T04:04:05.558 INFO:tasks.workunit.client.0.smithi157.stdout: PID TTY TIME CMD 2023-10-06T04:04:05.558 INFO:tasks.workunit.client.0.smithi157.stdout: 6131 ? 00:00:00 systemd 2023-10-06T04:04:05.558 INFO:tasks.workunit.client.0.smithi157.stdout: 6134 ? 00:00:00 (sd-pam) 2023-10-06T04:04:05.558 INFO:tasks.workunit.client.0.smithi157.stdout: 78246 ? 00:00:00 sshd 2023-10-06T04:04:05.558 INFO:tasks.workunit.client.0.smithi157.stdout: 106071 ? 00:00:00 timeout 2023-10-06T04:04:05.558 INFO:tasks.workunit.client.0.smithi157.stdout: 106088 ? 00:00:00 snaptest-double 2023-10-06T04:04:05.558 INFO:tasks.workunit.client.0.smithi157.stdout: 106156 ? 00:00:00 ps 2023-10-06T04:04:05.559 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-06T04:04:06.278 INFO:tasks.workunit.client.0.smithi157.stderr:+ rm a/foo 2023-10-06T04:04:06.281 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a 2023-10-06T04:04:06.290 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 20` 2023-10-06T04:04:06.290 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a 2023-10-06T04:04:06.292 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/one 2023-10-06T04:04:06.292 INFO:tasks.workunit.client.0.smithi157.stderr:+ cat 2023-10-06T04:04:06.311 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/two 2023-10-06T04:04:06.326 INFO:tasks.workunit.client.0.smithi157.stderr:+ chmod 777 a/foo 2023-10-06T04:04:06.327 INFO:tasks.workunit.client.0.smithi157.stderr:+ sync 2023-10-06T04:04:06.423 INFO:tasks.workunit.client.0.smithi157.stderr:+ ps 2023-10-06T04:04:06.431 INFO:tasks.workunit.client.0.smithi157.stdout: PID TTY TIME CMD 2023-10-06T04:04:06.432 INFO:tasks.workunit.client.0.smithi157.stdout: 6131 ? 00:00:00 systemd 2023-10-06T04:04:06.432 INFO:tasks.workunit.client.0.smithi157.stdout: 6134 ? 00:00:00 (sd-pam) 2023-10-06T04:04:06.432 INFO:tasks.workunit.client.0.smithi157.stdout: 78246 ? 00:00:00 sshd 2023-10-06T04:04:06.432 INFO:tasks.workunit.client.0.smithi157.stdout: 106071 ? 00:00:00 timeout 2023-10-06T04:04:06.432 INFO:tasks.workunit.client.0.smithi157.stdout: 106088 ? 00:00:00 snaptest-double 2023-10-06T04:04:06.432 INFO:tasks.workunit.client.0.smithi157.stdout: 106166 ? 00:00:00 ps 2023-10-06T04:04:06.432 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-06T04:04:06.443 INFO:tasks.workunit.client.0.smithi157.stderr:+ rm a/foo 2023-10-06T04:04:06.446 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a 2023-10-06T04:04:06.454 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 20` 2023-10-06T04:04:06.454 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a 2023-10-06T04:04:06.456 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/one 2023-10-06T04:04:06.457 INFO:tasks.workunit.client.0.smithi157.stderr:+ cat 2023-10-06T04:04:06.475 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/two 2023-10-06T04:04:06.490 INFO:tasks.workunit.client.0.smithi157.stderr:+ chmod 777 a/foo 2023-10-06T04:04:06.491 INFO:tasks.workunit.client.0.smithi157.stderr:+ sync 2023-10-06T04:04:06.565 INFO:tasks.workunit.client.0.smithi157.stderr:+ ps 2023-10-06T04:04:06.574 INFO:tasks.workunit.client.0.smithi157.stdout: PID TTY TIME CMD 2023-10-06T04:04:06.574 INFO:tasks.workunit.client.0.smithi157.stdout: 6131 ? 00:00:00 systemd 2023-10-06T04:04:06.574 INFO:tasks.workunit.client.0.smithi157.stdout: 6134 ? 00:00:00 (sd-pam) 2023-10-06T04:04:06.574 INFO:tasks.workunit.client.0.smithi157.stdout: 78246 ? 00:00:00 sshd 2023-10-06T04:04:06.574 INFO:tasks.workunit.client.0.smithi157.stdout: 106071 ? 00:00:00 timeout 2023-10-06T04:04:06.575 INFO:tasks.workunit.client.0.smithi157.stdout: 106088 ? 00:00:00 snaptest-double 2023-10-06T04:04:06.575 INFO:tasks.workunit.client.0.smithi157.stdout: 106176 ? 00:00:00 ps 2023-10-06T04:04:06.575 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-06T04:04:06.841 INFO:tasks.workunit.client.0.smithi157.stderr:+ rm a/foo 2023-10-06T04:04:06.843 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a 2023-10-06T04:04:06.850 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 20` 2023-10-06T04:04:06.851 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a 2023-10-06T04:04:06.853 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/one 2023-10-06T04:04:06.853 INFO:tasks.workunit.client.0.smithi157.stderr:+ cat 2023-10-06T04:04:06.872 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/two 2023-10-06T04:04:07.813 INFO:tasks.workunit.client.0.smithi157.stderr:+ chmod 777 a/foo 2023-10-06T04:04:07.814 INFO:tasks.workunit.client.0.smithi157.stderr:+ sync 2023-10-06T04:04:08.142 INFO:tasks.workunit.client.0.smithi157.stderr:+ ps 2023-10-06T04:04:08.152 INFO:tasks.workunit.client.0.smithi157.stdout: PID TTY TIME CMD 2023-10-06T04:04:08.152 INFO:tasks.workunit.client.0.smithi157.stdout: 6131 ? 00:00:00 systemd 2023-10-06T04:04:08.152 INFO:tasks.workunit.client.0.smithi157.stdout: 6134 ? 00:00:00 (sd-pam) 2023-10-06T04:04:08.152 INFO:tasks.workunit.client.0.smithi157.stdout: 78246 ? 00:00:00 sshd 2023-10-06T04:04:08.153 INFO:tasks.workunit.client.0.smithi157.stdout: 106071 ? 00:00:00 timeout 2023-10-06T04:04:08.153 INFO:tasks.workunit.client.0.smithi157.stdout: 106088 ? 00:00:00 snaptest-double 2023-10-06T04:04:08.153 INFO:tasks.workunit.client.0.smithi157.stdout: 106186 ? 00:00:00 ps 2023-10-06T04:04:08.153 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-06T04:04:08.167 INFO:tasks.workunit.client.0.smithi157.stderr:+ rm a/foo 2023-10-06T04:04:08.169 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a 2023-10-06T04:04:08.987 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 20` 2023-10-06T04:04:09.156 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a 2023-10-06T04:04:09.158 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/one 2023-10-06T04:04:09.158 INFO:tasks.workunit.client.0.smithi157.stderr:+ cat 2023-10-06T04:04:09.158 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/two 2023-10-06T04:04:09.158 INFO:tasks.workunit.client.0.smithi157.stderr:+ chmod 777 a/foo 2023-10-06T04:04:09.158 INFO:tasks.workunit.client.0.smithi157.stderr:+ sync 2023-10-06T04:04:09.165 INFO:tasks.workunit.client.0.smithi157.stderr:+ ps 2023-10-06T04:04:09.175 INFO:tasks.workunit.client.0.smithi157.stdout: PID TTY TIME CMD 2023-10-06T04:04:09.175 INFO:tasks.workunit.client.0.smithi157.stdout: 6131 ? 00:00:00 systemd 2023-10-06T04:04:09.175 INFO:tasks.workunit.client.0.smithi157.stdout: 6134 ? 00:00:00 (sd-pam) 2023-10-06T04:04:09.175 INFO:tasks.workunit.client.0.smithi157.stdout: 78246 ? 00:00:00 sshd 2023-10-06T04:04:09.176 INFO:tasks.workunit.client.0.smithi157.stdout: 106071 ? 00:00:00 timeout 2023-10-06T04:04:09.176 INFO:tasks.workunit.client.0.smithi157.stdout: 106088 ? 00:00:00 snaptest-double 2023-10-06T04:04:09.176 INFO:tasks.workunit.client.0.smithi157.stdout: 106196 ? 00:00:00 ps 2023-10-06T04:04:09.176 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-06T04:04:09.188 INFO:tasks.workunit.client.0.smithi157.stderr:+ rm a/foo 2023-10-06T04:04:09.191 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a 2023-10-06T04:04:09.197 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 20` 2023-10-06T04:04:09.197 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a 2023-10-06T04:04:09.199 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/one 2023-10-06T04:04:09.199 INFO:tasks.workunit.client.0.smithi157.stderr:+ cat 2023-10-06T04:04:09.223 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/two 2023-10-06T04:04:09.244 INFO:tasks.workunit.client.0.smithi157.stderr:+ chmod 777 a/foo 2023-10-06T04:04:09.245 INFO:tasks.workunit.client.0.smithi157.stderr:+ sync 2023-10-06T04:04:09.299 INFO:tasks.workunit.client.0.smithi157.stderr:+ ps 2023-10-06T04:04:09.308 INFO:tasks.workunit.client.0.smithi157.stdout: PID TTY TIME CMD 2023-10-06T04:04:09.308 INFO:tasks.workunit.client.0.smithi157.stdout: 6131 ? 00:00:00 systemd 2023-10-06T04:04:09.308 INFO:tasks.workunit.client.0.smithi157.stdout: 6134 ? 00:00:00 (sd-pam) 2023-10-06T04:04:09.308 INFO:tasks.workunit.client.0.smithi157.stdout: 78246 ? 00:00:00 sshd 2023-10-06T04:04:09.308 INFO:tasks.workunit.client.0.smithi157.stdout: 106071 ? 00:00:00 timeout 2023-10-06T04:04:09.308 INFO:tasks.workunit.client.0.smithi157.stdout: 106088 ? 00:00:00 snaptest-double 2023-10-06T04:04:09.308 INFO:tasks.workunit.client.0.smithi157.stdout: 106206 ? 00:00:00 ps 2023-10-06T04:04:09.309 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-06T04:04:09.817 INFO:tasks.workunit.client.0.smithi157.stderr:+ rm a/foo 2023-10-06T04:04:09.819 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a 2023-10-06T04:04:10.326 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 20` 2023-10-06T04:04:10.326 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a 2023-10-06T04:04:10.329 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/one 2023-10-06T04:04:10.329 INFO:tasks.workunit.client.0.smithi157.stderr:+ cat 2023-10-06T04:04:10.353 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/two 2023-10-06T04:04:10.375 INFO:tasks.workunit.client.0.smithi157.stderr:+ chmod 777 a/foo 2023-10-06T04:04:10.375 INFO:tasks.workunit.client.0.smithi157.stderr:+ sync 2023-10-06T04:04:10.518 INFO:tasks.workunit.client.0.smithi157.stderr:+ ps 2023-10-06T04:04:10.526 INFO:tasks.workunit.client.0.smithi157.stdout: PID TTY TIME CMD 2023-10-06T04:04:10.526 INFO:tasks.workunit.client.0.smithi157.stdout: 6131 ? 00:00:00 systemd 2023-10-06T04:04:10.526 INFO:tasks.workunit.client.0.smithi157.stdout: 6134 ? 00:00:00 (sd-pam) 2023-10-06T04:04:10.526 INFO:tasks.workunit.client.0.smithi157.stdout: 78246 ? 00:00:00 sshd 2023-10-06T04:04:10.526 INFO:tasks.workunit.client.0.smithi157.stdout: 106071 ? 00:00:00 timeout 2023-10-06T04:04:10.526 INFO:tasks.workunit.client.0.smithi157.stdout: 106088 ? 00:00:00 snaptest-double 2023-10-06T04:04:10.526 INFO:tasks.workunit.client.0.smithi157.stdout: 106217 ? 00:00:00 ps 2023-10-06T04:04:10.527 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-06T04:04:10.536 INFO:tasks.workunit.client.0.smithi157.stderr:+ rm a/foo 2023-10-06T04:04:10.539 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a 2023-10-06T04:04:10.545 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 20` 2023-10-06T04:04:10.545 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a 2023-10-06T04:04:10.547 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/one 2023-10-06T04:04:10.547 INFO:tasks.workunit.client.0.smithi157.stderr:+ cat 2023-10-06T04:04:10.572 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/two 2023-10-06T04:04:10.593 INFO:tasks.workunit.client.0.smithi157.stderr:+ chmod 777 a/foo 2023-10-06T04:04:10.594 INFO:tasks.workunit.client.0.smithi157.stderr:+ sync 2023-10-06T04:04:10.667 INFO:tasks.workunit.client.0.smithi157.stderr:+ ps 2023-10-06T04:04:10.674 INFO:tasks.workunit.client.0.smithi157.stdout: PID TTY TIME CMD 2023-10-06T04:04:10.674 INFO:tasks.workunit.client.0.smithi157.stdout: 6131 ? 00:00:00 systemd 2023-10-06T04:04:10.674 INFO:tasks.workunit.client.0.smithi157.stdout: 6134 ? 00:00:00 (sd-pam) 2023-10-06T04:04:10.674 INFO:tasks.workunit.client.0.smithi157.stdout: 78246 ? 00:00:00 sshd 2023-10-06T04:04:10.674 INFO:tasks.workunit.client.0.smithi157.stdout: 106071 ? 00:00:00 timeout 2023-10-06T04:04:10.675 INFO:tasks.workunit.client.0.smithi157.stdout: 106088 ? 00:00:00 snaptest-double 2023-10-06T04:04:10.675 INFO:tasks.workunit.client.0.smithi157.stdout: 106227 ? 00:00:00 ps 2023-10-06T04:04:10.675 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-06T04:04:10.685 INFO:tasks.workunit.client.0.smithi157.stderr:+ rm a/foo 2023-10-06T04:04:10.688 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a 2023-10-06T04:04:10.695 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 20` 2023-10-06T04:04:10.696 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a 2023-10-06T04:04:10.697 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/one 2023-10-06T04:04:10.698 INFO:tasks.workunit.client.0.smithi157.stderr:+ cat 2023-10-06T04:04:10.737 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/two 2023-10-06T04:04:10.752 INFO:tasks.workunit.client.0.smithi157.stderr:+ chmod 777 a/foo 2023-10-06T04:04:10.752 INFO:tasks.workunit.client.0.smithi157.stderr:+ sync 2023-10-06T04:04:10.868 INFO:tasks.workunit.client.0.smithi157.stderr:+ ps 2023-10-06T04:04:10.875 INFO:tasks.workunit.client.0.smithi157.stdout: PID TTY TIME CMD 2023-10-06T04:04:10.876 INFO:tasks.workunit.client.0.smithi157.stdout: 6131 ? 00:00:00 systemd 2023-10-06T04:04:10.876 INFO:tasks.workunit.client.0.smithi157.stdout: 6134 ? 00:00:00 (sd-pam) 2023-10-06T04:04:10.876 INFO:tasks.workunit.client.0.smithi157.stdout: 78246 ? 00:00:00 sshd 2023-10-06T04:04:10.876 INFO:tasks.workunit.client.0.smithi157.stdout: 106071 ? 00:00:00 timeout 2023-10-06T04:04:10.876 INFO:tasks.workunit.client.0.smithi157.stdout: 106088 ? 00:00:00 snaptest-double 2023-10-06T04:04:10.876 INFO:tasks.workunit.client.0.smithi157.stdout: 106237 ? 00:00:00 ps 2023-10-06T04:04:10.876 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-06T04:04:10.886 INFO:tasks.workunit.client.0.smithi157.stderr:+ rm a/foo 2023-10-06T04:04:10.889 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a 2023-10-06T04:04:10.895 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 20` 2023-10-06T04:04:10.895 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a 2023-10-06T04:04:10.896 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:04:10.897 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/one 2023-10-06T04:04:10.897 INFO:tasks.workunit.client.0.smithi157.stderr:+ cat 2023-10-06T04:04:11.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.011+0000 7fc2293ae700 1 Processor -- start 2023-10-06T04:04:11.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.012+0000 7fc2293ae700 1 -- start start 2023-10-06T04:04:11.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.012+0000 7fc2293ae700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc22406cbb0 0x7fc22406d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:11.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.012+0000 7fc2293ae700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc224150410 0x7fc22406da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:11.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.012+0000 7fc2293ae700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2240c9780 con 0x7fc22406c1f0 2023-10-06T04:04:11.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.012+0000 7fc2293ae700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2240c8c40 con 0x7fc22406cbb0 2023-10-06T04:04:11.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.012+0000 7fc2293ae700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22406e0c0 con 0x7fc224150410 2023-10-06T04:04:11.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.013+0000 7fc2237fe700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc224150410 0x7fc22406da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:11.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.013+0000 7fc222ffd700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc22406cbb0 0x7fc22406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:11.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.013+0000 7fc222ffd700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc22406cbb0 0x7fc22406d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:58362/0 (socket says 172.21.15.157:58362) 2023-10-06T04:04:11.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.013+0000 7fc2237fe700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc224150410 0x7fc22406da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:56160/0 (socket says 172.21.15.157:56160) 2023-10-06T04:04:11.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.013+0000 7fc222ffd700 1 -- 172.21.15.157:0/582996810 learned_addr learned my addr 172.21.15.157:0/582996810 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:11.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.013+0000 7fc222ffd700 1 -- 172.21.15.157:0/582996810 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc224150410 msgr2=0x7fc22406da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:04:11.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.013+0000 7fc222ffd700 1 --2- 172.21.15.157:0/582996810 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc224150410 0x7fc22406da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:11.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.013+0000 7fc222ffd700 1 -- 172.21.15.157:0/582996810 >> v1:172.21.15.175:6789/0 conn(0x7fc22406c1f0 legacy=0x7fc22414fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:11.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.013+0000 7fc2227fc700 1 -- 172.21.15.157:0/582996810 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 390838768 0 0) 0x7fc2240c9780 con 0x7fc22406c1f0 2023-10-06T04:04:11.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.013+0000 7fc222ffd700 1 -- 172.21.15.157:0/582996810 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2240c8e50 con 0x7fc22406cbb0 2023-10-06T04:04:11.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.013+0000 7fc2237fe700 1 --2- 172.21.15.157:0/582996810 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc224150410 0x7fc22406da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:04:11.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.013+0000 7fc222ffd700 1 --2- 172.21.15.157:0/582996810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc22406cbb0 0x7fc22406d530 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc2140099b0 tx=0x7fc214009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4736ba301719b185 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:11.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.013+0000 7fc2227fc700 1 -- 172.21.15.157:0/582996810 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc214009ea0 con 0x7fc22406cbb0 2023-10-06T04:04:11.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.013+0000 7fc2227fc700 1 -- 172.21.15.157:0/582996810 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21400b8f0 con 0x7fc22406cbb0 2023-10-06T04:04:11.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.013+0000 7fc2227fc700 1 -- 172.21.15.157:0/582996810 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21400bc90 con 0x7fc22406cbb0 2023-10-06T04:04:11.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.014+0000 7fc2293ae700 1 -- 172.21.15.157:0/582996810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc22406cbb0 msgr2=0x7fc22406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:11.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.014+0000 7fc2293ae700 1 --2- 172.21.15.157:0/582996810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc22406cbb0 0x7fc22406d530 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc2140099b0 tx=0x7fc214009250 comp rx=0 tx=0).stop 2023-10-06T04:04:11.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.014+0000 7fc2293ae700 1 -- 172.21.15.157:0/582996810 shutdown_connections 2023-10-06T04:04:11.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.014+0000 7fc2293ae700 1 --2- 172.21.15.157:0/582996810 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc224150410 0x7fc22406da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:11.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.014+0000 7fc2293ae700 1 --2- 172.21.15.157:0/582996810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc22406cbb0 0x7fc22406d530 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:11.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.014+0000 7fc2293ae700 1 -- 172.21.15.157:0/582996810 >> 172.21.15.157:0/582996810 conn(0x7fc22406f230 msgr2=0x7fc22406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:11.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.014+0000 7fc2293ae700 1 -- 172.21.15.157:0/582996810 shutdown_connections 2023-10-06T04:04:11.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.014+0000 7fc2293ae700 1 -- 172.21.15.157:0/582996810 wait complete. 2023-10-06T04:04:11.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.015+0000 7fc2293ae700 1 Processor -- start 2023-10-06T04:04:11.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.015+0000 7fc2293ae700 1 -- start start 2023-10-06T04:04:11.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.016+0000 7fc2293ae700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc22406c1f0 0x7fc2240cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:11.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.016+0000 7fc2293ae700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc22406cbb0 0x7fc224166cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:11.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.016+0000 7fc2293ae700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc224150410 0x7fc2241671f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:11.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.016+0000 7fc2293ae700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc224166860 con 0x7fc22406c1f0 2023-10-06T04:04:11.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.016+0000 7fc2293ae700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc224169ef0 con 0x7fc22406cbb0 2023-10-06T04:04:11.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.016+0000 7fc2293ae700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22416a1d0 con 0x7fc224150410 2023-10-06T04:04:11.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.016+0000 7fc222ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc22406c1f0 0x7fc2240cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:11.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.016+0000 7fc222ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc22406c1f0 0x7fc2240cb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53918/0 (socket says 172.21.15.157:53918) 2023-10-06T04:04:11.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.016+0000 7fc222ffd700 1 -- 172.21.15.157:0/1986625605 learned_addr learned my addr 172.21.15.157:0/1986625605 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:11.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.016+0000 7fc2237fe700 1 --2- 172.21.15.157:0/1986625605 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc224150410 0x7fc2241671f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:11.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.016+0000 7fc21bfff700 1 --2- 172.21.15.157:0/1986625605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc22406cbb0 0x7fc224166cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:11.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.016+0000 7fc222ffd700 1 -- 172.21.15.157:0/1986625605 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc224150410 msgr2=0x7fc2241671f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:11.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.016+0000 7fc222ffd700 1 --2- 172.21.15.157:0/1986625605 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc224150410 0x7fc2241671f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:11.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.016+0000 7fc222ffd700 1 -- 172.21.15.157:0/1986625605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc22406cbb0 msgr2=0x7fc224166cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:11.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.016+0000 7fc222ffd700 1 --2- 172.21.15.157:0/1986625605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc22406cbb0 0x7fc224166cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:11.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.016+0000 7fc222ffd700 1 -- 172.21.15.157:0/1986625605 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21c009a20 con 0x7fc22406c1f0 2023-10-06T04:04:11.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.016+0000 7fc21bfff700 1 --2- 172.21.15.157:0/1986625605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc22406cbb0 0x7fc224166cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:04:11.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.016+0000 7fc222ffd700 1 --2- 172.21.15.157:0/1986625605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc22406c1f0 0x7fc2240cb960 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc214006fd0 tx=0x7fc214017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:11.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.017+0000 7fc21affd700 1 -- 172.21.15.157:0/1986625605 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21400bb30 con 0x7fc22406c1f0 2023-10-06T04:04:11.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.017+0000 7fc21affd700 1 -- 172.21.15.157:0/1986625605 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc214026430 con 0x7fc22406c1f0 2023-10-06T04:04:11.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.017+0000 7fc21affd700 1 -- 172.21.15.157:0/1986625605 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21400bc90 con 0x7fc22406c1f0 2023-10-06T04:04:11.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.017+0000 7fc2293ae700 1 -- 172.21.15.157:0/1986625605 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2140094f0 con 0x7fc22406c1f0 2023-10-06T04:04:11.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.017+0000 7fc2293ae700 1 -- 172.21.15.157:0/1986625605 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc22416a790 con 0x7fc22406c1f0 2023-10-06T04:04:11.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.018+0000 7fc2293ae700 1 -- 172.21.15.157:0/1986625605 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc224089ca0 con 0x7fc22406c1f0 2023-10-06T04:04:11.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.018+0000 7fc21affd700 1 -- 172.21.15.157:0/1986625605 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc214023030 con 0x7fc22406c1f0 2023-10-06T04:04:11.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.018+0000 7fc21affd700 1 --2- 172.21.15.157:0/1986625605 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc204066f60 0x7fc204069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:11.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.018+0000 7fc21affd700 1 -- 172.21.15.157:0/1986625605 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6344+0+0 (secure 0 0 0) 0x7fc214085bf0 con 0x7fc22406c1f0 2023-10-06T04:04:11.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.019+0000 7fc21bfff700 1 --2- 172.21.15.157:0/1986625605 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc204066f60 0x7fc204069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:11.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.019+0000 7fc21bfff700 1 --2- 172.21.15.157:0/1986625605 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc204066f60 0x7fc204069410 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc20c0011e0 tx=0x7fc20c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:11.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.025+0000 7fc21affd700 1 -- 172.21.15.157:0/1986625605 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc214052be0 con 0x7fc22406c1f0 2023-10-06T04:04:11.191 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/two 2023-10-06T04:04:11.217 INFO:tasks.workunit.client.0.smithi157.stderr:+ chmod 777 a/foo 2023-10-06T04:04:11.217 INFO:tasks.workunit.client.0.smithi157.stderr:+ sync 2023-10-06T04:04:11.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.260+0000 7fc2293ae700 1 -- 172.21.15.157:0/1986625605 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc224166b00 con 0x7fc22406c1f0 2023-10-06T04:04:11.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.260+0000 7fc21affd700 1 -- 172.21.15.157:0/1986625605 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+6147 (secure 0 0 0) 0x7fc214056200 con 0x7fc22406c1f0 2023-10-06T04:04:11.262 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:04:11.262 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:46.794132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":4287},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:04:11.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.262+0000 7fc2293ae700 1 -- 172.21.15.157:0/1986625605 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc204066f60 msgr2=0x7fc204069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:11.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.262+0000 7fc2293ae700 1 --2- 172.21.15.157:0/1986625605 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc204066f60 0x7fc204069410 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc20c0011e0 tx=0x7fc20c009040 comp rx=0 tx=0).stop 2023-10-06T04:04:11.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.262+0000 7fc2293ae700 1 -- 172.21.15.157:0/1986625605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc22406c1f0 msgr2=0x7fc2240cb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:11.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.262+0000 7fc2293ae700 1 --2- 172.21.15.157:0/1986625605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc22406c1f0 0x7fc2240cb960 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc214006fd0 tx=0x7fc214017000 comp rx=0 tx=0).stop 2023-10-06T04:04:11.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.262+0000 7fc2293ae700 1 -- 172.21.15.157:0/1986625605 shutdown_connections 2023-10-06T04:04:11.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.262+0000 7fc2293ae700 1 --2- 172.21.15.157:0/1986625605 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc204066f60 0x7fc204069410 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:11.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.262+0000 7fc2293ae700 1 --2- 172.21.15.157:0/1986625605 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc224150410 0x7fc2241671f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:11.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.262+0000 7fc2293ae700 1 --2- 172.21.15.157:0/1986625605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc22406c1f0 0x7fc2240cb960 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:11.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.262+0000 7fc2293ae700 1 --2- 172.21.15.157:0/1986625605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc22406cbb0 0x7fc224166cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:11.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.262+0000 7fc2293ae700 1 -- 172.21.15.157:0/1986625605 >> 172.21.15.157:0/1986625605 conn(0x7fc22406f230 msgr2=0x7fc22414e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:11.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.262+0000 7fc2293ae700 1 -- 172.21.15.157:0/1986625605 shutdown_connections 2023-10-06T04:04:11.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.262+0000 7fc2293ae700 1 -- 172.21.15.157:0/1986625605 wait complete. 2023-10-06T04:04:11.265 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 57 2023-10-06T04:04:11.276 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=1) 2023-10-06T04:04:11.276 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T04:04:11.276 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:04:11.285 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T04:04:11.285 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:04:11.318 INFO:tasks.workunit.client.0.smithi157.stderr:+ ps 2023-10-06T04:04:11.326 INFO:tasks.workunit.client.0.smithi157.stdout: PID TTY TIME CMD 2023-10-06T04:04:11.326 INFO:tasks.workunit.client.0.smithi157.stdout: 6131 ? 00:00:00 systemd 2023-10-06T04:04:11.326 INFO:tasks.workunit.client.0.smithi157.stdout: 6134 ? 00:00:00 (sd-pam) 2023-10-06T04:04:11.326 INFO:tasks.workunit.client.0.smithi157.stdout: 78246 ? 00:00:00 sshd 2023-10-06T04:04:11.326 INFO:tasks.workunit.client.0.smithi157.stdout: 106071 ? 00:00:00 timeout 2023-10-06T04:04:11.327 INFO:tasks.workunit.client.0.smithi157.stdout: 106088 ? 00:00:00 snaptest-double 2023-10-06T04:04:11.327 INFO:tasks.workunit.client.0.smithi157.stdout: 106289 ? 00:00:00 ps 2023-10-06T04:04:11.327 INFO:tasks.workunit.client.0.smithi157.stdout: 106290 ? 00:00:00 bash 2023-10-06T04:04:11.327 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-06T04:04:11.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.433+0000 7f970db38700 1 Processor -- start 2023-10-06T04:04:11.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.433+0000 7f970db38700 1 -- start start 2023-10-06T04:04:11.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.433+0000 7f970db38700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f970806ab50 0x7f970806d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:11.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.433+0000 7f970db38700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f970806ddf0 0x7f970806e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:11.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.433+0000 7f970db38700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97080c6160 con 0x7f970806a780 2023-10-06T04:04:11.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.433+0000 7f970db38700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97080c5c00 con 0x7f970806ab50 2023-10-06T04:04:11.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.433+0000 7f970db38700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9708069040 con 0x7f970806ddf0 2023-10-06T04:04:11.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.434+0000 7f97077fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f970806a780 0x7f97080c65f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59458/0 (socket says 172.21.15.157:59458) 2023-10-06T04:04:11.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.434+0000 7f97077fe700 1 -- 172.21.15.157:0/339055403 learned_addr learned my addr 172.21.15.157:0/339055403 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:04:11.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.434+0000 7f9707fff700 1 --2- 172.21.15.157:0/339055403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f970806ddf0 0x7f970806e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:11.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.434+0000 7f9706ffd700 1 --2- 172.21.15.157:0/339055403 >> v2:172.21.15.175:3300/0 conn(0x7f970806ab50 0x7f970806d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:11.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.434+0000 7f97067fc700 1 -- 172.21.15.157:0/339055403 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3933649812 0 0) 0x7f97080c6160 con 0x7f970806a780 2023-10-06T04:04:11.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.434+0000 7f97067fc700 1 -- 172.21.15.157:0/339055403 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96f0003570 con 0x7f970806a780 2023-10-06T04:04:11.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.434+0000 7f97067fc700 1 -- 172.21.15.157:0/339055403 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1710099319 0 0) 0x7f96f0003570 con 0x7f970806a780 2023-10-06T04:04:11.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.434+0000 7f97067fc700 1 -- 172.21.15.157:0/339055403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f970806ddf0 msgr2=0x7f970806e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:04:11.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.434+0000 7f97067fc700 1 --2- 172.21.15.157:0/339055403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f970806ddf0 0x7f970806e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:11.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.434+0000 7f97067fc700 1 -- 172.21.15.157:0/339055403 >> v2:172.21.15.175:3300/0 conn(0x7f970806ab50 msgr2=0x7f970806d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:04:11.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.434+0000 7f97067fc700 1 --2- 172.21.15.157:0/339055403 >> v2:172.21.15.175:3300/0 conn(0x7f970806ab50 0x7f970806d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:11.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.434+0000 7f97067fc700 1 -- 172.21.15.157:0/339055403 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9708153930 con 0x7f970806a780 2023-10-06T04:04:11.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.434+0000 7f97067fc700 1 -- 172.21.15.157:0/339055403 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f96f8001470 con 0x7f970806a780 2023-10-06T04:04:11.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.434+0000 7f97067fc700 1 -- 172.21.15.157:0/339055403 >> v1:172.21.15.157:6789/0 conn(0x7f970806a780 legacy=0x7f97080c65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:11.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.434+0000 7f97067fc700 1 --2- 172.21.15.157:0/339055403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96f0003a60 0x7f96f0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:11.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.434+0000 7f97067fc700 1 --2- 172.21.15.157:0/339055403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f970806ddf0 0x7f96f0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:11.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.434+0000 7f97067fc700 1 --2- 172.21.15.157:0/339055403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96f0004d50 0x7f96f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:11.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.434+0000 7f97067fc700 1 -- 172.21.15.157:0/339055403 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9708069040 con 0x7f96f0004d50 2023-10-06T04:04:11.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.434+0000 7f97067fc700 1 -- 172.21.15.157:0/339055403 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f000ba60 con 0x7f96f0003a60 2023-10-06T04:04:11.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.434+0000 7f97067fc700 1 -- 172.21.15.157:0/339055403 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f000bba0 con 0x7f970806ddf0 2023-10-06T04:04:11.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.434+0000 7f9707fff700 1 --2- 172.21.15.157:0/339055403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96f0004d50 0x7f96f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:11.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.434+0000 7f97077fe700 1 --2- 172.21.15.157:0/339055403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96f0003a60 0x7f96f0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:11.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.434+0000 7f9706ffd700 1 --2- 172.21.15.157:0/339055403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f970806ddf0 0x7f96f0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:11.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.435+0000 7f9707fff700 1 -- 172.21.15.157:0/339055403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f970806ddf0 msgr2=0x7f96f0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:04:11.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.435+0000 7f9707fff700 1 --2- 172.21.15.157:0/339055403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f970806ddf0 0x7f96f0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:11.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.435+0000 7f9707fff700 1 -- 172.21.15.157:0/339055403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96f0003a60 msgr2=0x7f96f0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:04:11.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.435+0000 7f9707fff700 1 --2- 172.21.15.157:0/339055403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96f0003a60 0x7f96f0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:11.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.435+0000 7f9707fff700 1 -- 172.21.15.157:0/339055403 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f0003570 con 0x7f96f0004d50 2023-10-06T04:04:11.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.435+0000 7f9707fff700 1 --2- 172.21.15.157:0/339055403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96f0004d50 0x7f96f00051c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9700009fc0 tx=0x7f970000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32874df8d803ac1b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:11.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.435+0000 7f97067fc700 1 -- 172.21.15.157:0/339055403 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970000ea30 con 0x7f96f0004d50 2023-10-06T04:04:11.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.435+0000 7f97067fc700 1 -- 172.21.15.157:0/339055403 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f970000f040 con 0x7f96f0004d50 2023-10-06T04:04:11.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.435+0000 7f97067fc700 1 -- 172.21.15.157:0/339055403 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970000ede0 con 0x7f96f0004d50 2023-10-06T04:04:11.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.435+0000 7f970db38700 1 -- 172.21.15.157:0/339055403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96f0004d50 msgr2=0x7f96f00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:11.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.435+0000 7f970db38700 1 --2- 172.21.15.157:0/339055403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96f0004d50 0x7f96f00051c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9700009fc0 tx=0x7f970000b040 comp rx=0 tx=0).stop 2023-10-06T04:04:11.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.435+0000 7f9707fff700 1 -- 172.21.15.157:0/339055403 reap_dead start 2023-10-06T04:04:11.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.435+0000 7f970db38700 1 -- 172.21.15.157:0/339055403 shutdown_connections 2023-10-06T04:04:11.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.435+0000 7f970db38700 1 -- 172.21.15.157:0/339055403 >> 172.21.15.157:0/339055403 conn(0x7f970806b510 msgr2=0x7f970806b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:11.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.435+0000 7f970db38700 1 -- 172.21.15.157:0/339055403 shutdown_connections 2023-10-06T04:04:11.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.435+0000 7f970db38700 1 -- 172.21.15.157:0/339055403 wait complete. 2023-10-06T04:04:11.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.437+0000 7f970db38700 1 Processor -- start 2023-10-06T04:04:11.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.437+0000 7f970db38700 1 -- start start 2023-10-06T04:04:11.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.437+0000 7f970db38700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96f0004d50 0x7f97080b9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:11.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.437+0000 7f970db38700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97080c1900 0x7f9708166900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:11.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.437+0000 7f970db38700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9708166e40 0x7f9708147e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:11.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.437+0000 7f970db38700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97080bb220 con 0x7f9708166e40 2023-10-06T04:04:11.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.437+0000 7f970db38700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97080c1f90 con 0x7f97080c1900 2023-10-06T04:04:11.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.437+0000 7f970db38700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9708166240 con 0x7f96f0004d50 2023-10-06T04:04:11.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.437+0000 7f9707fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9708166e40 0x7f9708147e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:11.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.437+0000 7f9707fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9708166e40 0x7f9708147e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53942/0 (socket says 172.21.15.157:53942) 2023-10-06T04:04:11.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.437+0000 7f9707fff700 1 -- 172.21.15.157:0/2583235250 learned_addr learned my addr 172.21.15.157:0/2583235250 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:11.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.438+0000 7f97077fe700 1 --2- 172.21.15.157:0/2583235250 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96f0004d50 0x7f97080b9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:11.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.438+0000 7f9706ffd700 1 --2- 172.21.15.157:0/2583235250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97080c1900 0x7f9708166900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:11.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.438+0000 7f9707fff700 1 -- 172.21.15.157:0/2583235250 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96f0004d50 msgr2=0x7f97080b9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:11.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.438+0000 7f9707fff700 1 --2- 172.21.15.157:0/2583235250 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96f0004d50 0x7f97080b9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:11.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.438+0000 7f9707fff700 1 -- 172.21.15.157:0/2583235250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97080c1900 msgr2=0x7f9708166900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:11.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.438+0000 7f9707fff700 1 --2- 172.21.15.157:0/2583235250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97080c1900 0x7f9708166900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:11.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.438+0000 7f9707fff700 1 -- 172.21.15.157:0/2583235250 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97080ba610 con 0x7f9708166e40 2023-10-06T04:04:11.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.438+0000 7f9706ffd700 1 --2- 172.21.15.157:0/2583235250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97080c1900 0x7f9708166900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:04:11.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.438+0000 7f9707fff700 1 --2- 172.21.15.157:0/2583235250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9708166e40 0x7f9708147e60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f96f0004730 tx=0x7f970001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:11.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.438+0000 7f96ef7fe700 1 -- 172.21.15.157:0/2583235250 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9700012db0 con 0x7f9708166e40 2023-10-06T04:04:11.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.438+0000 7f96ef7fe700 1 -- 172.21.15.157:0/2583235250 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f970001a420 con 0x7f9708166e40 2023-10-06T04:04:11.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.438+0000 7f970db38700 1 -- 172.21.15.157:0/2583235250 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97080ba8f0 con 0x7f9708166e40 2023-10-06T04:04:11.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.438+0000 7f96ef7fe700 1 -- 172.21.15.157:0/2583235250 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970001a7e0 con 0x7f9708166e40 2023-10-06T04:04:11.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.438+0000 7f970db38700 1 -- 172.21.15.157:0/2583235250 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97080baeb0 con 0x7f9708166e40 2023-10-06T04:04:11.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.439+0000 7f970db38700 1 -- 172.21.15.157:0/2583235250 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97080692b0 con 0x7f9708166e40 2023-10-06T04:04:11.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.439+0000 7f96ef7fe700 1 -- 172.21.15.157:0/2583235250 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f970001ac10 con 0x7f9708166e40 2023-10-06T04:04:11.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.440+0000 7f96ef7fe700 1 --2- 172.21.15.157:0/2583235250 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f96f4066f60 0x7f96f4069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:11.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.440+0000 7f96ef7fe700 1 -- 172.21.15.157:0/2583235250 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6344+0+0 (secure 0 0 0) 0x7f9700016070 con 0x7f9708166e40 2023-10-06T04:04:11.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.440+0000 7f97077fe700 1 --2- 172.21.15.157:0/2583235250 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f96f4066f60 0x7f96f4069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:11.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.440+0000 7f97077fe700 1 --2- 172.21.15.157:0/2583235250 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f96f4066f60 0x7f96f4069410 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f96f80010e0 tx=0x7f96f8001e90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:11.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.444+0000 7f96ef7fe700 1 -- 172.21.15.157:0/2583235250 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97000530e0 con 0x7f9708166e40 2023-10-06T04:04:11.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.674+0000 7f970db38700 1 -- 172.21.15.157:0/2583235250 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f970806cd30 con 0x7f9708166e40 2023-10-06T04:04:11.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.675+0000 7f96ef7fe700 1 -- 172.21.15.157:0/2583235250 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+6147 (secure 0 0 0) 0x7f9700009880 con 0x7f9708166e40 2023-10-06T04:04:11.676 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:04:11.677 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:46.794132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":4287},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:04:11.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.677+0000 7f970db38700 1 -- 172.21.15.157:0/2583235250 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f96f4066f60 msgr2=0x7f96f4069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:11.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.677+0000 7f970db38700 1 --2- 172.21.15.157:0/2583235250 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f96f4066f60 0x7f96f4069410 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f96f80010e0 tx=0x7f96f8001e90 comp rx=0 tx=0).stop 2023-10-06T04:04:11.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.677+0000 7f970db38700 1 -- 172.21.15.157:0/2583235250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9708166e40 msgr2=0x7f9708147e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:11.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.677+0000 7f970db38700 1 --2- 172.21.15.157:0/2583235250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9708166e40 0x7f9708147e60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f96f0004730 tx=0x7f970001c000 comp rx=0 tx=0).stop 2023-10-06T04:04:11.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.677+0000 7f970db38700 1 -- 172.21.15.157:0/2583235250 shutdown_connections 2023-10-06T04:04:11.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.677+0000 7f970db38700 1 --2- 172.21.15.157:0/2583235250 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f96f4066f60 0x7f96f4069410 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:11.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.677+0000 7f970db38700 1 --2- 172.21.15.157:0/2583235250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9708166e40 0x7f9708147e60 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:11.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.677+0000 7f970db38700 1 --2- 172.21.15.157:0/2583235250 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96f0004d50 0x7f97080b9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:11.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.677+0000 7f970db38700 1 --2- 172.21.15.157:0/2583235250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97080c1900 0x7f9708166900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:11.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.677+0000 7f970db38700 1 -- 172.21.15.157:0/2583235250 >> 172.21.15.157:0/2583235250 conn(0x7f970806b510 msgr2=0x7f970806c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:11.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.677+0000 7f970db38700 1 -- 172.21.15.157:0/2583235250 shutdown_connections 2023-10-06T04:04:11.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:11.677+0000 7f970db38700 1 -- 172.21.15.157:0/2583235250 wait complete. 2023-10-06T04:04:11.680 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 57 2023-10-06T04:04:11.691 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-06T04:04:13.692 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:04:13.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.819+0000 7fca3b941700 1 Processor -- start 2023-10-06T04:04:13.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.819+0000 7fca3b941700 1 -- start start 2023-10-06T04:04:13.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.820+0000 7fca3b941700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca3414e0a0 0x7fca34153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:13.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.820+0000 7fca3b941700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fca340c5c00 0x7fca340ca440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:13.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.820+0000 7fca3b941700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca340c5770 con 0x7fca3414dcd0 2023-10-06T04:04:13.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.820+0000 7fca3b941700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca34154c80 con 0x7fca340c5c00 2023-10-06T04:04:13.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.820+0000 7fca3b941700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca34154200 con 0x7fca3414e0a0 2023-10-06T04:04:13.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.820+0000 7fca396dd700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fca3414dcd0 0x7fca340c18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:33632/0 (socket says 172.21.15.157:33632) 2023-10-06T04:04:13.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.820+0000 7fca396dd700 1 -- 172.21.15.157:0/3344646076 learned_addr learned my addr 172.21.15.157:0/3344646076 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:04:13.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.820+0000 7fca39ede700 1 --2- 172.21.15.157:0/3344646076 >> v2:172.21.15.157:3300/0 conn(0x7fca340c5c00 0x7fca340ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:13.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.821+0000 7fca38edc700 1 --2- 172.21.15.157:0/3344646076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca3414e0a0 0x7fca34153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:13.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.821+0000 7fca39ede700 1 -- 172.21.15.157:0/3344646076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca3414e0a0 msgr2=0x7fca34153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:04:13.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.821+0000 7fca39ede700 1 --2- 172.21.15.157:0/3344646076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca3414e0a0 0x7fca34153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:13.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.821+0000 7fca39ede700 1 -- 172.21.15.157:0/3344646076 >> v1:172.21.15.157:6789/0 conn(0x7fca3414dcd0 legacy=0x7fca340c18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:13.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.821+0000 7fca39ede700 1 -- 172.21.15.157:0/3344646076 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca341543e0 con 0x7fca340c5c00 2023-10-06T04:04:13.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.821+0000 7fca38edc700 1 --2- 172.21.15.157:0/3344646076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca3414e0a0 0x7fca34153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:04:13.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.821+0000 7fca39ede700 1 --2- 172.21.15.157:0/3344646076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca340c5c00 0x7fca340ca440 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009e60 tx=0x7fca2c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc402851bf58aed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:13.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.821+0000 7fca2bfff700 1 -- 172.21.15.157:0/3344646076 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c00e070 con 0x7fca340c5c00 2023-10-06T04:04:13.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.821+0000 7fca2bfff700 1 -- 172.21.15.157:0/3344646076 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2c00bc30 con 0x7fca340c5c00 2023-10-06T04:04:13.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.821+0000 7fca2bfff700 1 -- 172.21.15.157:0/3344646076 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c012430 con 0x7fca340c5c00 2023-10-06T04:04:13.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.821+0000 7fca3b941700 1 -- 172.21.15.157:0/3344646076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca340c5c00 msgr2=0x7fca340ca440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:13.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.821+0000 7fca3b941700 1 --2- 172.21.15.157:0/3344646076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca340c5c00 0x7fca340ca440 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009e60 tx=0x7fca2c009780 comp rx=0 tx=0).stop 2023-10-06T04:04:13.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.822+0000 7fca3b941700 1 -- 172.21.15.157:0/3344646076 shutdown_connections 2023-10-06T04:04:13.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.822+0000 7fca3b941700 1 --2- 172.21.15.157:0/3344646076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca340c5c00 0x7fca340ca440 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:13.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.822+0000 7fca3b941700 1 --2- 172.21.15.157:0/3344646076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca3414e0a0 0x7fca34153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:13.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.822+0000 7fca3b941700 1 -- 172.21.15.157:0/3344646076 >> 172.21.15.157:0/3344646076 conn(0x7fca340692b0 msgr2=0x7fca340696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:13.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.822+0000 7fca3b941700 1 -- 172.21.15.157:0/3344646076 shutdown_connections 2023-10-06T04:04:13.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.822+0000 7fca3b941700 1 -- 172.21.15.157:0/3344646076 wait complete. 2023-10-06T04:04:13.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.823+0000 7fca3b941700 1 Processor -- start 2023-10-06T04:04:13.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.823+0000 7fca3b941700 1 -- start start 2023-10-06T04:04:13.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.823+0000 7fca3b941700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca340c5c00 0x7fca34147600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:13.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.823+0000 7fca3b941700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca3414dcd0 0x7fca34147b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:13.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.824+0000 7fca3b941700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca3414e0a0 0x7fca341553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:13.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.824+0000 7fca3b941700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca34154f20 con 0x7fca3414dcd0 2023-10-06T04:04:13.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.824+0000 7fca3b941700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca34163f60 con 0x7fca3414e0a0 2023-10-06T04:04:13.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.824+0000 7fca3b941700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca341641c0 con 0x7fca340c5c00 2023-10-06T04:04:13.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.824+0000 7fca38edc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca3414dcd0 0x7fca34147b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:13.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.824+0000 7fca38edc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca3414dcd0 0x7fca34147b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59404/0 (socket says 172.21.15.157:59404) 2023-10-06T04:04:13.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.824+0000 7fca38edc700 1 -- 172.21.15.157:0/3377293954 learned_addr learned my addr 172.21.15.157:0/3377293954 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:13.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.824+0000 7fca396dd700 1 --2- 172.21.15.157:0/3377293954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca340c5c00 0x7fca34147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:13.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.824+0000 7fca38edc700 1 -- 172.21.15.157:0/3377293954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca340c5c00 msgr2=0x7fca34147600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:13.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.824+0000 7fca38edc700 1 --2- 172.21.15.157:0/3377293954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca340c5c00 0x7fca34147600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:13.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.824+0000 7fca38edc700 1 -- 172.21.15.157:0/3377293954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca3414e0a0 msgr2=0x7fca341553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:13.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.824+0000 7fca39ede700 1 --2- 172.21.15.157:0/3377293954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca3414e0a0 0x7fca341553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:13.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.824+0000 7fca38edc700 1 --2- 172.21.15.157:0/3377293954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca3414e0a0 0x7fca341553b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:13.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.824+0000 7fca38edc700 1 -- 172.21.15.157:0/3377293954 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2c009a20 con 0x7fca3414dcd0 2023-10-06T04:04:13.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.824+0000 7fca38edc700 1 --2- 172.21.15.157:0/3377293954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca3414dcd0 0x7fca34147b40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fca2400d860 tx=0x7fca24008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:13.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.824+0000 7fca297fa700 1 -- 172.21.15.157:0/3377293954 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2400b630 con 0x7fca3414dcd0 2023-10-06T04:04:13.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.824+0000 7fca297fa700 1 -- 172.21.15.157:0/3377293954 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2400b790 con 0x7fca3414dcd0 2023-10-06T04:04:13.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.824+0000 7fca3b941700 1 -- 172.21.15.157:0/3377293954 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca34164480 con 0x7fca3414dcd0 2023-10-06T04:04:13.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.824+0000 7fca297fa700 1 -- 172.21.15.157:0/3377293954 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2400bb10 con 0x7fca3414dcd0 2023-10-06T04:04:13.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.824+0000 7fca3b941700 1 -- 172.21.15.157:0/3377293954 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca34164a40 con 0x7fca3414dcd0 2023-10-06T04:04:13.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.825+0000 7fca3b941700 1 -- 172.21.15.157:0/3377293954 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca34155090 con 0x7fca3414dcd0 2023-10-06T04:04:13.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.826+0000 7fca297fa700 1 -- 172.21.15.157:0/3377293954 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fca24014430 con 0x7fca3414dcd0 2023-10-06T04:04:13.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.826+0000 7fca297fa700 1 --2- 172.21.15.157:0/3377293954 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fca18066f30 0x7fca180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:13.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.826+0000 7fca297fa700 1 -- 172.21.15.157:0/3377293954 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6372+0+0 (secure 0 0 0) 0x7fca24084d10 con 0x7fca3414dcd0 2023-10-06T04:04:13.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.826+0000 7fca396dd700 1 --2- 172.21.15.157:0/3377293954 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fca18066f30 0x7fca180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:13.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.827+0000 7fca396dd700 1 --2- 172.21.15.157:0/3377293954 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fca18066f30 0x7fca180693e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fca30001820 tx=0x7fca30009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:13.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:13.831+0000 7fca297fa700 1 -- 172.21.15.157:0/3377293954 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca24051d90 con 0x7fca3414dcd0 2023-10-06T04:04:14.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:14.056+0000 7fca3b941700 1 -- 172.21.15.157:0/3377293954 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca340c4340 con 0x7fca3414dcd0 2023-10-06T04:04:14.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:14.056+0000 7fca297fa700 1 -- 172.21.15.157:0/3377293954 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+6147 (secure 0 0 0) 0x7fca240553b0 con 0x7fca3414dcd0 2023-10-06T04:04:14.058 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:04:14.058 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:46.794132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":4287},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:04:14.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:14.058+0000 7fca3b941700 1 -- 172.21.15.157:0/3377293954 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fca18066f30 msgr2=0x7fca180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:14.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:14.058+0000 7fca3b941700 1 --2- 172.21.15.157:0/3377293954 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fca18066f30 0x7fca180693e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fca30001820 tx=0x7fca30009040 comp rx=0 tx=0).stop 2023-10-06T04:04:14.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:14.058+0000 7fca3b941700 1 -- 172.21.15.157:0/3377293954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca3414dcd0 msgr2=0x7fca34147b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:14.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:14.058+0000 7fca3b941700 1 --2- 172.21.15.157:0/3377293954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca3414dcd0 0x7fca34147b40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fca2400d860 tx=0x7fca24008040 comp rx=0 tx=0).stop 2023-10-06T04:04:14.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:14.058+0000 7fca3b941700 1 -- 172.21.15.157:0/3377293954 shutdown_connections 2023-10-06T04:04:14.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:14.058+0000 7fca3b941700 1 --2- 172.21.15.157:0/3377293954 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fca18066f30 0x7fca180693e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:14.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:14.058+0000 7fca3b941700 1 --2- 172.21.15.157:0/3377293954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca3414e0a0 0x7fca341553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:14.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:14.058+0000 7fca3b941700 1 --2- 172.21.15.157:0/3377293954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca340c5c00 0x7fca34147600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:14.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:14.058+0000 7fca3b941700 1 --2- 172.21.15.157:0/3377293954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca3414dcd0 0x7fca34147b40 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:14.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:14.058+0000 7fca3b941700 1 -- 172.21.15.157:0/3377293954 >> 172.21.15.157:0/3377293954 conn(0x7fca340692b0 msgr2=0x7fca3414b530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:14.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:14.058+0000 7fca3b941700 1 -- 172.21.15.157:0/3377293954 shutdown_connections 2023-10-06T04:04:14.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:14.058+0000 7fca3b941700 1 -- 172.21.15.157:0/3377293954 wait complete. 2023-10-06T04:04:14.062 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 57 2023-10-06T04:04:14.073 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-06T04:04:15.014 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:04:15.017 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:04:16.073 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:04:16.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.196+0000 7fd3c74c8700 1 Processor -- start 2023-10-06T04:04:16.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.197+0000 7fd3c74c8700 1 -- start start 2023-10-06T04:04:16.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.197+0000 7fd3c74c8700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3c013f0e0 0x7fd3c00c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:16.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.197+0000 7fd3c74c8700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd3c00c3d50 0x7fd3c006a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:16.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.197+0000 7fd3c74c8700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3c01596a0 con 0x7fd3c006ab00 2023-10-06T04:04:16.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.197+0000 7fd3c74c8700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3c00c6980 con 0x7fd3c00c3d50 2023-10-06T04:04:16.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.197+0000 7fd3c74c8700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c0155f70 con 0x7fd3c013f0e0 2023-10-06T04:04:16.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.198+0000 7fd3c5a65700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd3c006ab00 0x7fd3c0159b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:33646/0 (socket says 172.21.15.157:33646) 2023-10-06T04:04:16.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.198+0000 7fd3c5a65700 1 -- 172.21.15.157:0/2639497089 learned_addr learned my addr 172.21.15.157:0/2639497089 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:04:16.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.198+0000 7fd3c5264700 1 --2- 172.21.15.157:0/2639497089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3c013f0e0 0x7fd3c00c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:16.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.198+0000 7fd3c4a63700 1 --2- 172.21.15.157:0/2639497089 >> v2:172.21.15.175:3300/0 conn(0x7fd3c00c3d50 0x7fd3c006a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:16.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.198+0000 7fd3b7fff700 1 -- 172.21.15.157:0/2639497089 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 467096543 0 0) 0x7fd3c01596a0 con 0x7fd3c006ab00 2023-10-06T04:04:16.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.198+0000 7fd3b7fff700 1 -- 172.21.15.157:0/2639497089 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3ac003570 con 0x7fd3c006ab00 2023-10-06T04:04:16.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.198+0000 7fd3c4a63700 1 -- 172.21.15.157:0/2639497089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3c013f0e0 msgr2=0x7fd3c00c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:04:16.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.198+0000 7fd3c4a63700 1 --2- 172.21.15.157:0/2639497089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3c013f0e0 0x7fd3c00c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:16.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.198+0000 7fd3c4a63700 1 -- 172.21.15.157:0/2639497089 >> v1:172.21.15.157:6789/0 conn(0x7fd3c006ab00 legacy=0x7fd3c0159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:16.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.198+0000 7fd3c4a63700 1 -- 172.21.15.157:0/2639497089 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c0156230 con 0x7fd3c00c3d50 2023-10-06T04:04:16.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.198+0000 7fd3c5264700 1 --2- 172.21.15.157:0/2639497089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3c013f0e0 0x7fd3c00c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:04:16.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.198+0000 7fd3c4a63700 1 --2- 172.21.15.157:0/2639497089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3c00c3d50 0x7fd3c006a5c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd3b00098e0 tx=0x7fd3b00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab33dd600748da8d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:16.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.198+0000 7fd3b7fff700 1 -- 172.21.15.157:0/2639497089 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0011630 con 0x7fd3c00c3d50 2023-10-06T04:04:16.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.198+0000 7fd3b7fff700 1 -- 172.21.15.157:0/2639497089 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b0011790 con 0x7fd3c00c3d50 2023-10-06T04:04:16.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.198+0000 7fd3b7fff700 1 -- 172.21.15.157:0/2639497089 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0011b10 con 0x7fd3c00c3d50 2023-10-06T04:04:16.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.199+0000 7fd3c74c8700 1 -- 172.21.15.157:0/2639497089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3c00c3d50 msgr2=0x7fd3c006a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:16.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.199+0000 7fd3c74c8700 1 --2- 172.21.15.157:0/2639497089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3c00c3d50 0x7fd3c006a5c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd3b00098e0 tx=0x7fd3b00091a0 comp rx=0 tx=0).stop 2023-10-06T04:04:16.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.199+0000 7fd3c74c8700 1 -- 172.21.15.157:0/2639497089 shutdown_connections 2023-10-06T04:04:16.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.199+0000 7fd3c74c8700 1 --2- 172.21.15.157:0/2639497089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3c013f0e0 0x7fd3c00c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:16.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.199+0000 7fd3c74c8700 1 --2- 172.21.15.157:0/2639497089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3c00c3d50 0x7fd3c006a5c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:16.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.199+0000 7fd3c74c8700 1 -- 172.21.15.157:0/2639497089 >> 172.21.15.157:0/2639497089 conn(0x7fd3c006b6a0 msgr2=0x7fd3c00631f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:16.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.199+0000 7fd3c74c8700 1 -- 172.21.15.157:0/2639497089 shutdown_connections 2023-10-06T04:04:16.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.199+0000 7fd3c74c8700 1 -- 172.21.15.157:0/2639497089 wait complete. 2023-10-06T04:04:16.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.200+0000 7fd3c74c8700 1 Processor -- start 2023-10-06T04:04:16.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.200+0000 7fd3c74c8700 1 -- start start 2023-10-06T04:04:16.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.201+0000 7fd3c74c8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3c006ab00 0x7fd3c0152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:16.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.201+0000 7fd3c74c8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3c00c3d50 0x7fd3c00c5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:16.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.201+0000 7fd3c74c8700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3c013f0e0 0x7fd3c0157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:16.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.201+0000 7fd3c74c8700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c0166370 con 0x7fd3c00c3d50 2023-10-06T04:04:16.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.201+0000 7fd3c74c8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c0157920 con 0x7fd3c006ab00 2023-10-06T04:04:16.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.201+0000 7fd3c74c8700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c0166560 con 0x7fd3c013f0e0 2023-10-06T04:04:16.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.201+0000 7fd3c4a63700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3c00c3d50 0x7fd3c00c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:16.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.201+0000 7fd3c4a63700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3c00c3d50 0x7fd3c00c5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59418/0 (socket says 172.21.15.157:59418) 2023-10-06T04:04:16.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.201+0000 7fd3c4a63700 1 -- 172.21.15.157:0/3888328541 learned_addr learned my addr 172.21.15.157:0/3888328541 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:16.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.201+0000 7fd3c5264700 1 --2- 172.21.15.157:0/3888328541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3c006ab00 0x7fd3c0152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:16.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.201+0000 7fd3c5a65700 1 --2- 172.21.15.157:0/3888328541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3c013f0e0 0x7fd3c0157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:16.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.202+0000 7fd3c4a63700 1 -- 172.21.15.157:0/3888328541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3c013f0e0 msgr2=0x7fd3c0157db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:16.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.202+0000 7fd3c4a63700 1 --2- 172.21.15.157:0/3888328541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3c013f0e0 0x7fd3c0157db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:16.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.202+0000 7fd3c4a63700 1 -- 172.21.15.157:0/3888328541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3c006ab00 msgr2=0x7fd3c0152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:16.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.202+0000 7fd3c4a63700 1 --2- 172.21.15.157:0/3888328541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3c006ab00 0x7fd3c0152fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:16.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.202+0000 7fd3c4a63700 1 -- 172.21.15.157:0/3888328541 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3bc009a20 con 0x7fd3c00c3d50 2023-10-06T04:04:16.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.202+0000 7fd3c4a63700 1 --2- 172.21.15.157:0/3888328541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3c00c3d50 0x7fd3c00c5800 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd3c006b130 tx=0x7fd3b0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:16.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.202+0000 7fd3b57fa700 1 -- 172.21.15.157:0/3888328541 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0011900 con 0x7fd3c00c3d50 2023-10-06T04:04:16.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.202+0000 7fd3b57fa700 1 -- 172.21.15.157:0/3888328541 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b0011a60 con 0x7fd3c00c3d50 2023-10-06T04:04:16.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.202+0000 7fd3c74c8700 1 -- 172.21.15.157:0/3888328541 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3b0009420 con 0x7fd3c00c3d50 2023-10-06T04:04:16.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.202+0000 7fd3b57fa700 1 -- 172.21.15.157:0/3888328541 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b001a680 con 0x7fd3c00c3d50 2023-10-06T04:04:16.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.202+0000 7fd3c74c8700 1 -- 172.21.15.157:0/3888328541 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c0166b20 con 0x7fd3c00c3d50 2023-10-06T04:04:16.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.203+0000 7fd3c74c8700 1 -- 172.21.15.157:0/3888328541 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3c014c9c0 con 0x7fd3c00c3d50 2023-10-06T04:04:16.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.203+0000 7fd3b57fa700 1 -- 172.21.15.157:0/3888328541 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd3b001aa30 con 0x7fd3c00c3d50 2023-10-06T04:04:16.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.204+0000 7fd3b57fa700 1 --2- 172.21.15.157:0/3888328541 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd3a4066f30 0x7fd3a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:16.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.204+0000 7fd3b57fa700 1 -- 172.21.15.157:0/3888328541 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6372+0+0 (secure 0 0 0) 0x7fd3b00863a0 con 0x7fd3c00c3d50 2023-10-06T04:04:16.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.204+0000 7fd3c5264700 1 --2- 172.21.15.157:0/3888328541 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd3a4066f30 0x7fd3a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:16.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.204+0000 7fd3c5264700 1 --2- 172.21.15.157:0/3888328541 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd3a4066f30 0x7fd3a40693e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00f650 tx=0x7fd3bc00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:16.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.208+0000 7fd3b57fa700 1 -- 172.21.15.157:0/3888328541 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd3b0053370 con 0x7fd3c00c3d50 2023-10-06T04:04:16.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.432+0000 7fd3c74c8700 1 -- 172.21.15.157:0/3888328541 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3c00c9510 con 0x7fd3c00c3d50 2023-10-06T04:04:16.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.433+0000 7fd3b57fa700 1 -- 172.21.15.157:0/3888328541 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+6147 (secure 0 0 0) 0x7fd3b0056990 con 0x7fd3c00c3d50 2023-10-06T04:04:16.435 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:04:16.435 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:46.794132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":4287},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:04:16.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.435+0000 7fd3c74c8700 1 -- 172.21.15.157:0/3888328541 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd3a4066f30 msgr2=0x7fd3a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:16.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.435+0000 7fd3c74c8700 1 --2- 172.21.15.157:0/3888328541 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd3a4066f30 0x7fd3a40693e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00f650 tx=0x7fd3bc00b040 comp rx=0 tx=0).stop 2023-10-06T04:04:16.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.435+0000 7fd3c74c8700 1 -- 172.21.15.157:0/3888328541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3c00c3d50 msgr2=0x7fd3c00c5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:16.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.435+0000 7fd3c74c8700 1 --2- 172.21.15.157:0/3888328541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3c00c3d50 0x7fd3c00c5800 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd3c006b130 tx=0x7fd3b0017000 comp rx=0 tx=0).stop 2023-10-06T04:04:16.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.435+0000 7fd3c74c8700 1 -- 172.21.15.157:0/3888328541 shutdown_connections 2023-10-06T04:04:16.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.435+0000 7fd3c74c8700 1 --2- 172.21.15.157:0/3888328541 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd3a4066f30 0x7fd3a40693e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:16.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.435+0000 7fd3c74c8700 1 --2- 172.21.15.157:0/3888328541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3c006ab00 0x7fd3c0152fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:16.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.435+0000 7fd3c74c8700 1 --2- 172.21.15.157:0/3888328541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3c013f0e0 0x7fd3c0157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:16.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.435+0000 7fd3c74c8700 1 --2- 172.21.15.157:0/3888328541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3c00c3d50 0x7fd3c00c5800 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:16.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.435+0000 7fd3c74c8700 1 -- 172.21.15.157:0/3888328541 >> 172.21.15.157:0/3888328541 conn(0x7fd3c006b6a0 msgr2=0x7fd3c013e300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:16.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.435+0000 7fd3c74c8700 1 -- 172.21.15.157:0/3888328541 shutdown_connections 2023-10-06T04:04:16.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:16.435+0000 7fd3c74c8700 1 -- 172.21.15.157:0/3888328541 wait complete. 2023-10-06T04:04:16.439 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 57 2023-10-06T04:04:16.449 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-06T04:04:18.450 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:04:18.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.575+0000 7fe29e192700 1 Processor -- start 2023-10-06T04:04:18.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.575+0000 7fe29e192700 1 -- start start 2023-10-06T04:04:18.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.575+0000 7fe29e192700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe29814e0a0 0x7fe298153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:18.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.575+0000 7fe29e192700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2980c5770 con 0x7fe2980c5c00 2023-10-06T04:04:18.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.575+0000 7fe29e192700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2981543e0 con 0x7fe29814dcd0 2023-10-06T04:04:18.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.575+0000 7fe29e192700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe298154c80 con 0x7fe29814e0a0 2023-10-06T04:04:18.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.576+0000 7fe296ffd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe29814e0a0 0x7fe298153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:18.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.576+0000 7fe296ffd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe29814e0a0 0x7fe298153cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59422/0 (socket says 172.21.15.157:59422) 2023-10-06T04:04:18.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.576+0000 7fe296ffd700 1 -- 172.21.15.157:0/2275031846 learned_addr learned my addr 172.21.15.157:0/2275031846 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:18.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.576+0000 7fe297fff700 1 --1- 172.21.15.157:0/2275031846 >> v1:172.21.15.157:6789/0 conn(0x7fe2980c5c00 0x7fe2980ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:33656/0 (socket says 172.21.15.157:33656) 2023-10-06T04:04:18.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.576+0000 7fe296ffd700 1 -- 172.21.15.157:0/2275031846 >> v1:172.21.15.175:6789/0 conn(0x7fe29814dcd0 legacy=0x7fe2980c18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:04:18.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.576+0000 7fe296ffd700 1 -- 172.21.15.157:0/2275031846 >> v1:172.21.15.157:6789/0 conn(0x7fe2980c5c00 legacy=0x7fe2980ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:18.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.576+0000 7fe296ffd700 1 -- 172.21.15.157:0/2275031846 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2981546c0 con 0x7fe29814e0a0 2023-10-06T04:04:18.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.576+0000 7fe296ffd700 1 --2- 172.21.15.157:0/2275031846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29814e0a0 0x7fe298153cc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe2880099b0 tx=0x7fe288009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66dcf806ceccb722 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:18.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.576+0000 7fe2967fc700 1 -- 172.21.15.157:0/2275031846 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28800b8b0 con 0x7fe29814e0a0 2023-10-06T04:04:18.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.576+0000 7fe2967fc700 1 -- 172.21.15.157:0/2275031846 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28800ba10 con 0x7fe29814e0a0 2023-10-06T04:04:18.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.576+0000 7fe2967fc700 1 -- 172.21.15.157:0/2275031846 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28800bdb0 con 0x7fe29814e0a0 2023-10-06T04:04:18.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.577+0000 7fe29e192700 1 -- 172.21.15.157:0/2275031846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29814e0a0 msgr2=0x7fe298153cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:18.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.577+0000 7fe29e192700 1 --2- 172.21.15.157:0/2275031846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29814e0a0 0x7fe298153cc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe2880099b0 tx=0x7fe288009250 comp rx=0 tx=0).stop 2023-10-06T04:04:18.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.577+0000 7fe29e192700 1 -- 172.21.15.157:0/2275031846 shutdown_connections 2023-10-06T04:04:18.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.577+0000 7fe29e192700 1 --2- 172.21.15.157:0/2275031846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29814e0a0 0x7fe298153cc0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:18.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.577+0000 7fe29e192700 1 -- 172.21.15.157:0/2275031846 >> 172.21.15.157:0/2275031846 conn(0x7fe2980692b0 msgr2=0x7fe2980696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:18.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.577+0000 7fe29e192700 1 -- 172.21.15.157:0/2275031846 shutdown_connections 2023-10-06T04:04:18.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.577+0000 7fe29e192700 1 -- 172.21.15.157:0/2275031846 wait complete. 2023-10-06T04:04:18.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.578+0000 7fe29e192700 1 Processor -- start 2023-10-06T04:04:18.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.578+0000 7fe29e192700 1 -- start start 2023-10-06T04:04:18.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.579+0000 7fe29e192700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2980c5c00 0x7fe298164760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:18.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.579+0000 7fe29e192700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe29814dcd0 0x7fe29814ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:18.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.579+0000 7fe29e192700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29814e0a0 0x7fe298150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:18.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.579+0000 7fe29e192700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe298154200 con 0x7fe29814e0a0 2023-10-06T04:04:18.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.579+0000 7fe29e192700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe298150290 con 0x7fe2980c5c00 2023-10-06T04:04:18.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.579+0000 7fe29e192700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29814f400 con 0x7fe29814dcd0 2023-10-06T04:04:18.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.579+0000 7fe297fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29814e0a0 0x7fe298150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:18.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.579+0000 7fe297fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29814e0a0 0x7fe298150740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59430/0 (socket says 172.21.15.157:59430) 2023-10-06T04:04:18.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.579+0000 7fe297fff700 1 -- 172.21.15.157:0/1479901156 learned_addr learned my addr 172.21.15.157:0/1479901156 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:18.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.579+0000 7fe296ffd700 1 --2- 172.21.15.157:0/1479901156 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe29814dcd0 0x7fe29814ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:18.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.579+0000 7fe2977fe700 1 --2- 172.21.15.157:0/1479901156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2980c5c00 0x7fe298164760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:18.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.579+0000 7fe297fff700 1 -- 172.21.15.157:0/1479901156 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe29814dcd0 msgr2=0x7fe29814ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:18.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.579+0000 7fe297fff700 1 --2- 172.21.15.157:0/1479901156 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe29814dcd0 0x7fe29814ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:18.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.579+0000 7fe297fff700 1 -- 172.21.15.157:0/1479901156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2980c5c00 msgr2=0x7fe298164760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:18.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.579+0000 7fe297fff700 1 --2- 172.21.15.157:0/1479901156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2980c5c00 0x7fe298164760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:18.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.579+0000 7fe297fff700 1 -- 172.21.15.157:0/1479901156 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2880094f0 con 0x7fe29814e0a0 2023-10-06T04:04:18.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.579+0000 7fe297fff700 1 --2- 172.21.15.157:0/1479901156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29814e0a0 0x7fe298150740 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe29000cd30 tx=0x7fe290001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:18.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.579+0000 7fe27f7fe700 1 -- 172.21.15.157:0/1479901156 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe290005df0 con 0x7fe29814e0a0 2023-10-06T04:04:18.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.579+0000 7fe27f7fe700 1 -- 172.21.15.157:0/1479901156 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29000a920 con 0x7fe29814e0a0 2023-10-06T04:04:18.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.579+0000 7fe27f7fe700 1 -- 172.21.15.157:0/1479901156 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29000ace0 con 0x7fe29814e0a0 2023-10-06T04:04:18.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.580+0000 7fe29e192700 1 -- 172.21.15.157:0/1479901156 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29814f680 con 0x7fe29814e0a0 2023-10-06T04:04:18.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.580+0000 7fe29e192700 1 -- 172.21.15.157:0/1479901156 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29814fc40 con 0x7fe29814e0a0 2023-10-06T04:04:18.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.581+0000 7fe29e192700 1 -- 172.21.15.157:0/1479901156 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe29806a8f0 con 0x7fe29814e0a0 2023-10-06T04:04:18.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.581+0000 7fe27f7fe700 1 -- 172.21.15.157:0/1479901156 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe290007300 con 0x7fe29814e0a0 2023-10-06T04:04:18.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.581+0000 7fe27f7fe700 1 --2- 172.21.15.157:0/1479901156 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe284066f30 0x7fe2840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:18.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.581+0000 7fe27f7fe700 1 -- 172.21.15.157:0/1479901156 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fe290084920 con 0x7fe29814e0a0 2023-10-06T04:04:18.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.581+0000 7fe2977fe700 1 --2- 172.21.15.157:0/1479901156 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe284066f30 0x7fe2840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:18.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.582+0000 7fe2977fe700 1 --2- 172.21.15.157:0/1479901156 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe284066f30 0x7fe2840693e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe28c003870 tx=0x7fe28c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:18.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.586+0000 7fe27f7fe700 1 -- 172.21.15.157:0/1479901156 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2900519b0 con 0x7fe29814e0a0 2023-10-06T04:04:18.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.814+0000 7fe29e192700 1 -- 172.21.15.157:0/1479901156 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe29804bb40 con 0x7fe29814e0a0 2023-10-06T04:04:18.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.814+0000 7fe27f7fe700 1 -- 172.21.15.157:0/1479901156 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+6147 (secure 0 0 0) 0x7fe290054fd0 con 0x7fe29814e0a0 2023-10-06T04:04:18.816 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:04:18.816 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:46.794132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":4287},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:04:18.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.816+0000 7fe29e192700 1 -- 172.21.15.157:0/1479901156 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe284066f30 msgr2=0x7fe2840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:18.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.816+0000 7fe29e192700 1 --2- 172.21.15.157:0/1479901156 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe284066f30 0x7fe2840693e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe28c003870 tx=0x7fe28c009040 comp rx=0 tx=0).stop 2023-10-06T04:04:18.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.816+0000 7fe29e192700 1 -- 172.21.15.157:0/1479901156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29814e0a0 msgr2=0x7fe298150740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:18.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.816+0000 7fe29e192700 1 --2- 172.21.15.157:0/1479901156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29814e0a0 0x7fe298150740 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe29000cd30 tx=0x7fe290001b10 comp rx=0 tx=0).stop 2023-10-06T04:04:18.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.816+0000 7fe29e192700 1 -- 172.21.15.157:0/1479901156 shutdown_connections 2023-10-06T04:04:18.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.816+0000 7fe29e192700 1 --2- 172.21.15.157:0/1479901156 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe284066f30 0x7fe2840693e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:18.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.816+0000 7fe29e192700 1 --2- 172.21.15.157:0/1479901156 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe2980c5c00 0x7fe298164760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:18.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.816+0000 7fe29e192700 1 --2- 172.21.15.157:0/1479901156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29814e0a0 0x7fe298150740 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:18.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.816+0000 7fe29e192700 1 --2- 172.21.15.157:0/1479901156 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe29814dcd0 0x7fe29814ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:18.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.816+0000 7fe29e192700 1 -- 172.21.15.157:0/1479901156 >> 172.21.15.157:0/1479901156 conn(0x7fe2980692b0 msgr2=0x7fe2980c68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:18.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.816+0000 7fe29e192700 1 -- 172.21.15.157:0/1479901156 shutdown_connections 2023-10-06T04:04:18.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:18.816+0000 7fe29e192700 1 -- 172.21.15.157:0/1479901156 wait complete. 2023-10-06T04:04:18.819 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 57 2023-10-06T04:04:18.830 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-06T04:04:20.831 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:04:20.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.958+0000 7f32eb11a700 1 Processor -- start 2023-10-06T04:04:20.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.958+0000 7f32eb11a700 1 -- start start 2023-10-06T04:04:20.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.959+0000 7f32eb11a700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f32e406dc40 0x7f32e414fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:20.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.959+0000 7f32eb11a700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e40c9780 con 0x7f32e406c1f0 2023-10-06T04:04:20.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.959+0000 7f32eb11a700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e406e0c0 con 0x7f32e406cbb0 2023-10-06T04:04:20.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.959+0000 7f32eb11a700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32e40c8c40 con 0x7f32e406dc40 2023-10-06T04:04:20.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.959+0000 7f32e96b7700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f32e406c1f0 0x7f32e4155f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:33664/0 (socket says 172.21.15.157:33664) 2023-10-06T04:04:20.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.959+0000 7f32e96b7700 1 -- 172.21.15.157:0/353643326 learned_addr learned my addr 172.21.15.157:0/353643326 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:04:20.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.959+0000 7f32e3fff700 1 --2- 172.21.15.157:0/353643326 >> v2:172.21.15.175:3300/0 conn(0x7f32e406dc40 0x7f32e414fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:20.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.959+0000 7f32e37fe700 1 -- 172.21.15.157:0/353643326 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 381504262 0 0) 0x7f32e40c9780 con 0x7f32e406c1f0 2023-10-06T04:04:20.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.959+0000 7f32e37fe700 1 -- 172.21.15.157:0/353643326 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32d0003570 con 0x7f32e406c1f0 2023-10-06T04:04:20.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.959+0000 7f32e37fe700 1 -- 172.21.15.157:0/353643326 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 641044920 0 0) 0x7f32e406e0c0 con 0x7f32e406cbb0 2023-10-06T04:04:20.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.959+0000 7f32e37fe700 1 -- 172.21.15.157:0/353643326 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32e40c9780 con 0x7f32e406cbb0 2023-10-06T04:04:20.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.959+0000 7f32e3fff700 1 -- 172.21.15.157:0/353643326 >> v1:172.21.15.175:6789/0 conn(0x7f32e406cbb0 legacy=0x7f32e406d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:20.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.959+0000 7f32e37fe700 1 -- 172.21.15.157:0/353643326 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1494178907 0 0) 0x7f32d0003570 con 0x7f32e406c1f0 2023-10-06T04:04:20.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.959+0000 7f32e3fff700 1 -- 172.21.15.157:0/353643326 >> v1:172.21.15.157:6789/0 conn(0x7f32e406c1f0 legacy=0x7f32e4155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:20.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.959+0000 7f32e3fff700 1 -- 172.21.15.157:0/353643326 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e40c8e50 con 0x7f32e406dc40 2023-10-06T04:04:20.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.960+0000 7f32e3fff700 1 --2- 172.21.15.157:0/353643326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32e406dc40 0x7f32e414fd00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f32d40098e0 tx=0x7f32d40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ce22b035563fb43 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:20.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.960+0000 7f32e37fe700 1 -- 172.21.15.157:0/353643326 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc001470 con 0x7f32e406dc40 2023-10-06T04:04:20.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.960+0000 7f32e37fe700 1 -- 172.21.15.157:0/353643326 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d4011420 con 0x7f32e406dc40 2023-10-06T04:04:20.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.960+0000 7f32e37fe700 1 -- 172.21.15.157:0/353643326 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d40117c0 con 0x7f32e406dc40 2023-10-06T04:04:20.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.960+0000 7f32eb11a700 1 -- 172.21.15.157:0/353643326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32e406dc40 msgr2=0x7f32e414fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:20.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.960+0000 7f32eb11a700 1 --2- 172.21.15.157:0/353643326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32e406dc40 0x7f32e414fd00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f32d40098e0 tx=0x7f32d40091a0 comp rx=0 tx=0).stop 2023-10-06T04:04:20.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.960+0000 7f32eb11a700 1 -- 172.21.15.157:0/353643326 shutdown_connections 2023-10-06T04:04:20.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.960+0000 7f32eb11a700 1 --2- 172.21.15.157:0/353643326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32e406dc40 0x7f32e414fd00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:20.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.960+0000 7f32eb11a700 1 -- 172.21.15.157:0/353643326 >> 172.21.15.157:0/353643326 conn(0x7f32e406f230 msgr2=0x7f32e406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:20.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.960+0000 7f32eb11a700 1 -- 172.21.15.157:0/353643326 shutdown_connections 2023-10-06T04:04:20.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.960+0000 7f32eb11a700 1 -- 172.21.15.157:0/353643326 wait complete. 2023-10-06T04:04:20.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.961+0000 7f32eb11a700 1 Processor -- start 2023-10-06T04:04:20.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.962+0000 7f32eb11a700 1 -- start start 2023-10-06T04:04:20.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.962+0000 7f32eb11a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32e406c1f0 0x7f32e4147980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:20.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.962+0000 7f32eb11a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32e406cbb0 0x7f32e40c5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:20.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.962+0000 7f32eb11a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32e4155f80 0x7f32e4156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:20.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.962+0000 7f32eb11a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e4156c20 con 0x7f32e406cbb0 2023-10-06T04:04:20.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.962+0000 7f32eb11a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e41474f0 con 0x7f32e4155f80 2023-10-06T04:04:20.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.962+0000 7f32eb11a700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e40c3a30 con 0x7f32e406c1f0 2023-10-06T04:04:20.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.962+0000 7f32e3fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32e406cbb0 0x7f32e40c5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:20.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.963+0000 7f32e3fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32e406cbb0 0x7f32e40c5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:59434/0 (socket says 172.21.15.157:59434) 2023-10-06T04:04:20.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.963+0000 7f32e3fff700 1 -- 172.21.15.157:0/3186662308 learned_addr learned my addr 172.21.15.157:0/3186662308 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:20.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.963+0000 7f32e8eb6700 1 --2- 172.21.15.157:0/3186662308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32e406c1f0 0x7f32e4147980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:20.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.963+0000 7f32e96b7700 1 --2- 172.21.15.157:0/3186662308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32e4155f80 0x7f32e4156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:20.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.963+0000 7f32e3fff700 1 -- 172.21.15.157:0/3186662308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32e406c1f0 msgr2=0x7f32e4147980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:20.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.963+0000 7f32e3fff700 1 --2- 172.21.15.157:0/3186662308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32e406c1f0 0x7f32e4147980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:20.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.963+0000 7f32e3fff700 1 -- 172.21.15.157:0/3186662308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32e4155f80 msgr2=0x7f32e4156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:20.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.963+0000 7f32e3fff700 1 --2- 172.21.15.157:0/3186662308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32e4155f80 0x7f32e4156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:20.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.963+0000 7f32e3fff700 1 -- 172.21.15.157:0/3186662308 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d4009420 con 0x7f32e406cbb0 2023-10-06T04:04:20.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.963+0000 7f32e3fff700 1 --2- 172.21.15.157:0/3186662308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32e406cbb0 0x7f32e40c5450 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f32e40c3960 tx=0x7f32d4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:20.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.963+0000 7f32e0ff9700 1 -- 172.21.15.157:0/3186662308 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d401cb30 con 0x7f32e406cbb0 2023-10-06T04:04:20.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.963+0000 7f32e0ff9700 1 -- 172.21.15.157:0/3186662308 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d401cc90 con 0x7f32e406cbb0 2023-10-06T04:04:20.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.963+0000 7f32eb11a700 1 -- 172.21.15.157:0/3186662308 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32e40c3cb0 con 0x7f32e406cbb0 2023-10-06T04:04:20.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.963+0000 7f32e0ff9700 1 -- 172.21.15.157:0/3186662308 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d401a6a0 con 0x7f32e406cbb0 2023-10-06T04:04:20.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.963+0000 7f32eb11a700 1 -- 172.21.15.157:0/3186662308 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e4156e10 con 0x7f32e406cbb0 2023-10-06T04:04:20.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.964+0000 7f32eb11a700 1 -- 172.21.15.157:0/3186662308 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32e4089ca0 con 0x7f32e406cbb0 2023-10-06T04:04:20.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.965+0000 7f32e0ff9700 1 -- 172.21.15.157:0/3186662308 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f32d401aa50 con 0x7f32e406cbb0 2023-10-06T04:04:20.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.965+0000 7f32e0ff9700 1 --2- 172.21.15.157:0/3186662308 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f32c4066f30 0x7f32c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:20.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.965+0000 7f32e0ff9700 1 -- 172.21.15.157:0/3186662308 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f32d4086070 con 0x7f32e406cbb0 2023-10-06T04:04:20.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.965+0000 7f32e8eb6700 1 --2- 172.21.15.157:0/3186662308 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f32c4066f30 0x7f32c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:20.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.966+0000 7f32e8eb6700 1 --2- 172.21.15.157:0/3186662308 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f32c4066f30 0x7f32c40693e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f32dc004140 tx=0x7f32dc007f90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:20.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:20.970+0000 7f32e0ff9700 1 -- 172.21.15.157:0/3186662308 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32d4053080 con 0x7f32e406cbb0 2023-10-06T04:04:21.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:21.195+0000 7f32eb11a700 1 -- 172.21.15.157:0/3186662308 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32e4147630 con 0x7f32e406cbb0 2023-10-06T04:04:21.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:21.196+0000 7f32e0ff9700 1 -- 172.21.15.157:0/3186662308 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+6147 (secure 0 0 0) 0x7f32d40566a0 con 0x7f32e406cbb0 2023-10-06T04:04:21.197 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:04:21.198 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:46.794132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":4287},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:04:21.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:21.197+0000 7f32eb11a700 1 -- 172.21.15.157:0/3186662308 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f32c4066f30 msgr2=0x7f32c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:21.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:21.197+0000 7f32eb11a700 1 --2- 172.21.15.157:0/3186662308 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f32c4066f30 0x7f32c40693e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f32dc004140 tx=0x7f32dc007f90 comp rx=0 tx=0).stop 2023-10-06T04:04:21.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:21.197+0000 7f32eb11a700 1 -- 172.21.15.157:0/3186662308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32e406cbb0 msgr2=0x7f32e40c5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:21.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:21.197+0000 7f32eb11a700 1 --2- 172.21.15.157:0/3186662308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32e406cbb0 0x7f32e40c5450 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f32e40c3960 tx=0x7f32d4016000 comp rx=0 tx=0).stop 2023-10-06T04:04:21.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:21.197+0000 7f32eb11a700 1 -- 172.21.15.157:0/3186662308 shutdown_connections 2023-10-06T04:04:21.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:21.197+0000 7f32eb11a700 1 --2- 172.21.15.157:0/3186662308 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f32c4066f30 0x7f32c40693e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:21.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:21.197+0000 7f32eb11a700 1 --2- 172.21.15.157:0/3186662308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32e4155f80 0x7f32e4156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:21.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:21.197+0000 7f32eb11a700 1 --2- 172.21.15.157:0/3186662308 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32e406c1f0 0x7f32e4147980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:21.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:21.197+0000 7f32eb11a700 1 --2- 172.21.15.157:0/3186662308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32e406cbb0 0x7f32e40c5450 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:21.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:21.197+0000 7f32eb11a700 1 -- 172.21.15.157:0/3186662308 >> 172.21.15.157:0/3186662308 conn(0x7f32e406f230 msgr2=0x7f32e406d530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:21.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:21.198+0000 7f32eb11a700 1 -- 172.21.15.157:0/3186662308 shutdown_connections 2023-10-06T04:04:21.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:21.198+0000 7f32eb11a700 1 -- 172.21.15.157:0/3186662308 wait complete. 2023-10-06T04:04:21.201 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 57 2023-10-06T04:04:21.211 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-06T04:04:23.212 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:04:23.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.338+0000 7f7b7e3be700 1 Processor -- start 2023-10-06T04:04:23.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.338+0000 7f7b7e3be700 1 -- start start 2023-10-06T04:04:23.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.339+0000 7f7b7e3be700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7b780b04d0 0x7f7b78157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:23.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.339+0000 7f7b7e3be700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b7806d8d0 con 0x7f7b780b1e40 2023-10-06T04:04:23.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.339+0000 7f7b7e3be700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b780af8f0 con 0x7f7b7806c2d0 2023-10-06T04:04:23.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.339+0000 7f7b7e3be700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b7815a690 con 0x7f7b780b04d0 2023-10-06T04:04:23.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.339+0000 7f7b7d3bc700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7b780b1e40 0x7f7b781563b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:35290/0 (socket says 172.21.15.157:35290) 2023-10-06T04:04:23.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.339+0000 7f7b7d3bc700 1 -- 172.21.15.157:0/2470015075 learned_addr learned my addr 172.21.15.157:0/2470015075 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:04:23.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.339+0000 7f7b7cbbb700 1 --2- 172.21.15.157:0/2470015075 >> v2:172.21.15.175:3300/0 conn(0x7f7b780b04d0 0x7f7b78157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:23.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.339+0000 7f7b6ffff700 1 -- 172.21.15.157:0/2470015075 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1177237072 0 0) 0x7f7b7806d8d0 con 0x7f7b780b1e40 2023-10-06T04:04:23.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.339+0000 7f7b6ffff700 1 -- 172.21.15.157:0/2470015075 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b64003570 con 0x7f7b780b1e40 2023-10-06T04:04:23.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.340+0000 7f7b6ffff700 1 -- 172.21.15.157:0/2470015075 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1320962576 0 0) 0x7f7b780af8f0 con 0x7f7b7806c2d0 2023-10-06T04:04:23.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.340+0000 7f7b6ffff700 1 -- 172.21.15.157:0/2470015075 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b7806d8d0 con 0x7f7b7806c2d0 2023-10-06T04:04:23.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.340+0000 7f7b6ffff700 1 -- 172.21.15.157:0/2470015075 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4147861566 0 0) 0x7f7b64003570 con 0x7f7b780b1e40 2023-10-06T04:04:23.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.340+0000 7f7b6ffff700 1 -- 172.21.15.157:0/2470015075 >> v2:172.21.15.175:3300/0 conn(0x7f7b780b04d0 msgr2=0x7f7b78157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:04:23.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.340+0000 7f7b6ffff700 1 --2- 172.21.15.157:0/2470015075 >> v2:172.21.15.175:3300/0 conn(0x7f7b780b04d0 0x7f7b78157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:23.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.340+0000 7f7b6ffff700 1 -- 172.21.15.157:0/2470015075 >> v1:172.21.15.175:6789/0 conn(0x7f7b7806c2d0 legacy=0x7f7b78156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:23.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.340+0000 7f7b6ffff700 1 -- 172.21.15.157:0/2470015075 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b781571d0 con 0x7f7b780b1e40 2023-10-06T04:04:23.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.340+0000 7f7b6ffff700 1 -- 172.21.15.157:0/2470015075 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f7b74001470 con 0x7f7b780b1e40 2023-10-06T04:04:23.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.340+0000 7f7b6ffff700 1 -- 172.21.15.157:0/2470015075 >> v1:172.21.15.157:6789/0 conn(0x7f7b780b1e40 legacy=0x7f7b781563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:23.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.340+0000 7f7b6ffff700 1 --2- 172.21.15.157:0/2470015075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b64003c40 0x7f7b64004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:23.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.340+0000 7f7b6ffff700 1 --2- 172.21.15.157:0/2470015075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b64004810 0x7f7b64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:23.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.340+0000 7f7b6ffff700 1 --2- 172.21.15.157:0/2470015075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b64005210 0x7f7b6400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:23.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.340+0000 7f7b6ffff700 1 -- 172.21.15.157:0/2470015075 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7815a690 con 0x7f7b64003c40 2023-10-06T04:04:23.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.340+0000 7f7b6ffff700 1 -- 172.21.15.157:0/2470015075 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6400c1b0 con 0x7f7b64005210 2023-10-06T04:04:23.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.340+0000 7f7b6ffff700 1 -- 172.21.15.157:0/2470015075 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6400c370 con 0x7f7b64004810 2023-10-06T04:04:23.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.340+0000 7f7b7d3bc700 1 --2- 172.21.15.157:0/2470015075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b64003c40 0x7f7b64004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:23.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.340+0000 7f7b7dbbd700 1 --2- 172.21.15.157:0/2470015075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b64005210 0x7f7b6400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:23.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.340+0000 7f7b7cbbb700 1 --2- 172.21.15.157:0/2470015075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b64004810 0x7f7b64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:23.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.340+0000 7f7b7d3bc700 1 -- 172.21.15.157:0/2470015075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b64004810 msgr2=0x7f7b64004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:04:23.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.340+0000 7f7b7d3bc700 1 --2- 172.21.15.157:0/2470015075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b64004810 0x7f7b64004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:23.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.340+0000 7f7b7d3bc700 1 -- 172.21.15.157:0/2470015075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b64005210 msgr2=0x7f7b6400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:04:23.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.340+0000 7f7b7d3bc700 1 --2- 172.21.15.157:0/2470015075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b64005210 0x7f7b6400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:23.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.340+0000 7f7b7d3bc700 1 -- 172.21.15.157:0/2470015075 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b64003570 con 0x7f7b64003c40 2023-10-06T04:04:23.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.340+0000 7f7b7dbbd700 1 -- 172.21.15.157:0/2470015075 reap_dead start 2023-10-06T04:04:23.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.340+0000 7f7b7cbbb700 1 --2- 172.21.15.157:0/2470015075 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b64004810 0x7f7b64004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:04:23.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.340+0000 7f7b7d3bc700 1 --2- 172.21.15.157:0/2470015075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b64003c40 0x7f7b64004030 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7b74007fd0 tx=0x7f7b74001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f613549085e031d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:23.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.340+0000 7f7b6ffff700 1 -- 172.21.15.157:0/2470015075 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7400cae0 con 0x7f7b64003c40 2023-10-06T04:04:23.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.340+0000 7f7b6ffff700 1 -- 172.21.15.157:0/2470015075 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7400cc60 con 0x7f7b64003c40 2023-10-06T04:04:23.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.340+0000 7f7b6ffff700 1 -- 172.21.15.157:0/2470015075 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b74005920 con 0x7f7b64003c40 2023-10-06T04:04:23.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.341+0000 7f7b7e3be700 1 -- 172.21.15.157:0/2470015075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b64003c40 msgr2=0x7f7b64004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:23.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.341+0000 7f7b7e3be700 1 --2- 172.21.15.157:0/2470015075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b64003c40 0x7f7b64004030 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7b74007fd0 tx=0x7f7b74001d40 comp rx=0 tx=0).stop 2023-10-06T04:04:23.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.341+0000 7f7b7e3be700 1 -- 172.21.15.157:0/2470015075 shutdown_connections 2023-10-06T04:04:23.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.341+0000 7f7b7e3be700 1 --2- 172.21.15.157:0/2470015075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b64003c40 0x7f7b64004030 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:23.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.341+0000 7f7b7e3be700 1 -- 172.21.15.157:0/2470015075 >> 172.21.15.157:0/2470015075 conn(0x7f7b780cc540 msgr2=0x7f7b780cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:23.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.341+0000 7f7b7e3be700 1 -- 172.21.15.157:0/2470015075 shutdown_connections 2023-10-06T04:04:23.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.341+0000 7f7b7e3be700 1 -- 172.21.15.157:0/2470015075 wait complete. 2023-10-06T04:04:23.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.342+0000 7f7b7e3be700 1 Processor -- start 2023-10-06T04:04:23.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.342+0000 7f7b7e3be700 1 -- start start 2023-10-06T04:04:23.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.343+0000 7f7b7e3be700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b64003c40 0x7f7b78153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:23.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.343+0000 7f7b7e3be700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b781543e0 0x7f7b781660c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:23.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.343+0000 7f7b7e3be700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b78155950 0x7f7b7814ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:23.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.343+0000 7f7b7e3be700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b78152490 con 0x7f7b78155950 2023-10-06T04:04:23.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.343+0000 7f7b7e3be700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b78169cf0 con 0x7f7b64003c40 2023-10-06T04:04:23.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.343+0000 7f7b7e3be700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b781697c0 con 0x7f7b781543e0 2023-10-06T04:04:23.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.343+0000 7f7b7dbbd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b78155950 0x7f7b7814ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:23.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.343+0000 7f7b7dbbd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b78155950 0x7f7b7814ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33254/0 (socket says 172.21.15.157:33254) 2023-10-06T04:04:23.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.343+0000 7f7b7dbbd700 1 -- 172.21.15.157:0/2802784408 learned_addr learned my addr 172.21.15.157:0/2802784408 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:23.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.343+0000 7f7b7cbbb700 1 --2- 172.21.15.157:0/2802784408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b781543e0 0x7f7b781660c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:23.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.344+0000 7f7b7d3bc700 1 --2- 172.21.15.157:0/2802784408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b64003c40 0x7f7b78153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:23.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.344+0000 7f7b7dbbd700 1 -- 172.21.15.157:0/2802784408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b781543e0 msgr2=0x7f7b781660c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:23.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.344+0000 7f7b7dbbd700 1 --2- 172.21.15.157:0/2802784408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b781543e0 0x7f7b781660c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:23.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.344+0000 7f7b7dbbd700 1 -- 172.21.15.157:0/2802784408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b64003c40 msgr2=0x7f7b78153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:23.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.344+0000 7f7b7dbbd700 1 --2- 172.21.15.157:0/2802784408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b64003c40 0x7f7b78153bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:23.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.344+0000 7f7b7dbbd700 1 -- 172.21.15.157:0/2802784408 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b78165c30 con 0x7f7b78155950 2023-10-06T04:04:23.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.344+0000 7f7b7dbbd700 1 --2- 172.21.15.157:0/2802784408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b78155950 0x7f7b7814ed80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7b70004290 tx=0x7f7b700017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:23.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.344+0000 7f7b6d7fa700 1 -- 172.21.15.157:0/2802784408 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b70006db0 con 0x7f7b78155950 2023-10-06T04:04:23.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.344+0000 7f7b6d7fa700 1 -- 172.21.15.157:0/2802784408 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7000c670 con 0x7f7b78155950 2023-10-06T04:04:23.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.344+0000 7f7b7e3be700 1 -- 172.21.15.157:0/2802784408 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b78154ce0 con 0x7f7b78155950 2023-10-06T04:04:23.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.344+0000 7f7b6d7fa700 1 -- 172.21.15.157:0/2802784408 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7000ca30 con 0x7f7b78155950 2023-10-06T04:04:23.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.344+0000 7f7b7e3be700 1 -- 172.21.15.157:0/2802784408 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b78154f80 con 0x7f7b78155950 2023-10-06T04:04:23.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.345+0000 7f7b7e3be700 1 -- 172.21.15.157:0/2802784408 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b78165dc0 con 0x7f7b78155950 2023-10-06T04:04:23.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.345+0000 7f7b6d7fa700 1 -- 172.21.15.157:0/2802784408 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7b70007070 con 0x7f7b78155950 2023-10-06T04:04:23.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.346+0000 7f7b6d7fa700 1 --2- 172.21.15.157:0/2802784408 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7b5c066f60 0x7f7b5c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:23.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.346+0000 7f7b6d7fa700 1 -- 172.21.15.157:0/2802784408 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f7b700842c0 con 0x7f7b78155950 2023-10-06T04:04:23.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.346+0000 7f7b7d3bc700 1 --2- 172.21.15.157:0/2802784408 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7b5c066f60 0x7f7b5c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:23.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.346+0000 7f7b7d3bc700 1 --2- 172.21.15.157:0/2802784408 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7b5c066f60 0x7f7b5c069410 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7b74001870 tx=0x7f7b74015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:23.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.350+0000 7f7b6d7fa700 1 -- 172.21.15.157:0/2802784408 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b700512d0 con 0x7f7b78155950 2023-10-06T04:04:23.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.580+0000 7f7b7e3be700 1 -- 172.21.15.157:0/2802784408 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b78088f90 con 0x7f7b78155950 2023-10-06T04:04:23.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.581+0000 7f7b6d7fa700 1 -- 172.21.15.157:0/2802784408 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+6147 (secure 0 0 0) 0x7f7b700548f0 con 0x7f7b78155950 2023-10-06T04:04:23.582 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:04:23.582 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:46.794132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":4287},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:04:23.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.583+0000 7f7b7e3be700 1 -- 172.21.15.157:0/2802784408 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7b5c066f60 msgr2=0x7f7b5c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:23.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.583+0000 7f7b7e3be700 1 --2- 172.21.15.157:0/2802784408 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7b5c066f60 0x7f7b5c069410 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7b74001870 tx=0x7f7b74015000 comp rx=0 tx=0).stop 2023-10-06T04:04:23.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.583+0000 7f7b7e3be700 1 -- 172.21.15.157:0/2802784408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b78155950 msgr2=0x7f7b7814ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:23.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.583+0000 7f7b7e3be700 1 --2- 172.21.15.157:0/2802784408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b78155950 0x7f7b7814ed80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7b70004290 tx=0x7f7b700017a0 comp rx=0 tx=0).stop 2023-10-06T04:04:23.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.583+0000 7f7b7e3be700 1 -- 172.21.15.157:0/2802784408 shutdown_connections 2023-10-06T04:04:23.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.583+0000 7f7b7e3be700 1 --2- 172.21.15.157:0/2802784408 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7b5c066f60 0x7f7b5c069410 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:23.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.583+0000 7f7b7e3be700 1 --2- 172.21.15.157:0/2802784408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b64003c40 0x7f7b78153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:23.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.583+0000 7f7b7e3be700 1 --2- 172.21.15.157:0/2802784408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b78155950 0x7f7b7814ed80 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:23.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.583+0000 7f7b7e3be700 1 --2- 172.21.15.157:0/2802784408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b781543e0 0x7f7b781660c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:23.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.583+0000 7f7b7e3be700 1 -- 172.21.15.157:0/2802784408 >> 172.21.15.157:0/2802784408 conn(0x7f7b780cc540 msgr2=0x7f7b78157440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:23.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.583+0000 7f7b7e3be700 1 -- 172.21.15.157:0/2802784408 shutdown_connections 2023-10-06T04:04:23.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:23.583+0000 7f7b7e3be700 1 -- 172.21.15.157:0/2802784408 wait complete. 2023-10-06T04:04:23.586 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 57 2023-10-06T04:04:23.596 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-06T04:04:25.597 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:04:25.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.724+0000 7f2b6359e700 1 Processor -- start 2023-10-06T04:04:25.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.724+0000 7f2b6359e700 1 -- start start 2023-10-06T04:04:25.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.725+0000 7f2b6359e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2b640b1e40 0x7f2b64157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:25.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.725+0000 7f2b6359e700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b640b04d0 0x7f2b6406dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:25.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.725+0000 7f2b6359e700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2b6406c2d0 0x7f2b641563b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:25.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.725+0000 7f2b6359e700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b6406d8d0 con 0x7f2b6406c2d0 2023-10-06T04:04:25.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.725+0000 7f2b6359e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b640af8f0 con 0x7f2b640b1e40 2023-10-06T04:04:25.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.725+0000 7f2b6359e700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6406ccc0 con 0x7f2b640b04d0 2023-10-06T04:04:25.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.725+0000 7f2b62d9d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2b6406c2d0 0x7f2b641563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:25.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.725+0000 7f2b62d9d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2b6406c2d0 0x7f2b641563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33256/0 (socket says 172.21.15.157:33256) 2023-10-06T04:04:25.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.725+0000 7f2b62d9d700 1 -- 172.21.15.157:0/2528001559 learned_addr learned my addr 172.21.15.157:0/2528001559 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:25.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.725+0000 7f2b6259c700 1 --2- 172.21.15.157:0/2528001559 >> v2:172.21.15.175:3300/0 conn(0x7f2b640b1e40 0x7f2b64157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:25.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.725+0000 7f2b61d9b700 1 --2- 172.21.15.157:0/2528001559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b640b04d0 0x7f2b6406dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:25.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.725+0000 7f2b62d9d700 1 -- 172.21.15.157:0/2528001559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b640b04d0 msgr2=0x7f2b6406dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:04:25.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.725+0000 7f2b62d9d700 1 --2- 172.21.15.157:0/2528001559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b640b04d0 0x7f2b6406dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:25.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.725+0000 7f2b62d9d700 1 -- 172.21.15.157:0/2528001559 >> v2:172.21.15.175:3300/0 conn(0x7f2b640b1e40 msgr2=0x7f2b64157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:04:25.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.725+0000 7f2b62d9d700 1 --2- 172.21.15.157:0/2528001559 >> v2:172.21.15.175:3300/0 conn(0x7f2b640b1e40 0x7f2b64157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:25.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.725+0000 7f2b62d9d700 1 -- 172.21.15.157:0/2528001559 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6406cfa0 con 0x7f2b6406c2d0 2023-10-06T04:04:25.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.726+0000 7f2b62d9d700 1 --2- 172.21.15.157:0/2528001559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b6406c2d0 0x7f2b641563b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2b640cc450 tx=0x7f2b54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc25446a14fd266 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:25.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.726+0000 7f2b6159a700 1 -- 172.21.15.157:0/2528001559 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5400b860 con 0x7f2b6406c2d0 2023-10-06T04:04:25.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.726+0000 7f2b6159a700 1 -- 172.21.15.157:0/2528001559 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5400b9c0 con 0x7f2b6406c2d0 2023-10-06T04:04:25.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.726+0000 7f2b6159a700 1 -- 172.21.15.157:0/2528001559 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5400bd80 con 0x7f2b6406c2d0 2023-10-06T04:04:25.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.726+0000 7f2b6359e700 1 -- 172.21.15.157:0/2528001559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b6406c2d0 msgr2=0x7f2b641563b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:25.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.726+0000 7f2b6359e700 1 --2- 172.21.15.157:0/2528001559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b6406c2d0 0x7f2b641563b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2b640cc450 tx=0x7f2b54009250 comp rx=0 tx=0).stop 2023-10-06T04:04:25.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.726+0000 7f2b6359e700 1 -- 172.21.15.157:0/2528001559 shutdown_connections 2023-10-06T04:04:25.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.726+0000 7f2b6359e700 1 --2- 172.21.15.157:0/2528001559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b6406c2d0 0x7f2b641563b0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:25.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.726+0000 7f2b6359e700 1 --2- 172.21.15.157:0/2528001559 >> v2:172.21.15.175:3300/0 conn(0x7f2b640b1e40 0x7f2b64157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:25.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.726+0000 7f2b6359e700 1 --2- 172.21.15.157:0/2528001559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b640b04d0 0x7f2b6406dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:25.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.726+0000 7f2b6359e700 1 -- 172.21.15.157:0/2528001559 >> 172.21.15.157:0/2528001559 conn(0x7f2b640cc540 msgr2=0x7f2b640cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:25.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.726+0000 7f2b6359e700 1 -- 172.21.15.157:0/2528001559 shutdown_connections 2023-10-06T04:04:25.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.726+0000 7f2b6359e700 1 -- 172.21.15.157:0/2528001559 wait complete. 2023-10-06T04:04:25.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.727+0000 7f2b6359e700 1 Processor -- start 2023-10-06T04:04:25.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.728+0000 7f2b6359e700 1 -- start start 2023-10-06T04:04:25.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.728+0000 7f2b6359e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b6406c2d0 0x7f2b640c0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:25.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.728+0000 7f2b6359e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b640b04d0 0x7f2b6415ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:25.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.728+0000 7f2b6359e700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b640b1e40 0x7f2b6415bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:25.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.728+0000 7f2b6359e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b640bfea0 con 0x7f2b6406c2d0 2023-10-06T04:04:25.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.728+0000 7f2b6359e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b64166030 con 0x7f2b640b04d0 2023-10-06T04:04:25.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.728+0000 7f2b6359e700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b640c0140 con 0x7f2b640b1e40 2023-10-06T04:04:25.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.728+0000 7f2b6259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b6406c2d0 0x7f2b640c0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:25.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.728+0000 7f2b6259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b6406c2d0 0x7f2b640c0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33268/0 (socket says 172.21.15.157:33268) 2023-10-06T04:04:25.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.728+0000 7f2b6259c700 1 -- 172.21.15.157:0/351726059 learned_addr learned my addr 172.21.15.157:0/351726059 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:25.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.728+0000 7f2b62d9d700 1 --2- 172.21.15.157:0/351726059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b640b1e40 0x7f2b6415bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:25.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.728+0000 7f2b61d9b700 1 --2- 172.21.15.157:0/351726059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b640b04d0 0x7f2b6415ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:25.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.729+0000 7f2b6259c700 1 -- 172.21.15.157:0/351726059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b640b1e40 msgr2=0x7f2b6415bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:25.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.729+0000 7f2b6259c700 1 --2- 172.21.15.157:0/351726059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b640b1e40 0x7f2b6415bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:25.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.729+0000 7f2b6259c700 1 -- 172.21.15.157:0/351726059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b640b04d0 msgr2=0x7f2b6415ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:25.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.729+0000 7f2b6259c700 1 --2- 172.21.15.157:0/351726059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b640b04d0 0x7f2b6415ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:25.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.729+0000 7f2b6259c700 1 -- 172.21.15.157:0/351726059 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b540094f0 con 0x7f2b6406c2d0 2023-10-06T04:04:25.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.729+0000 7f2b6259c700 1 --2- 172.21.15.157:0/351726059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b6406c2d0 0x7f2b640c0c20 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00e190 tx=0x7f2b5c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:25.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.729+0000 7f2b5a7fc700 1 -- 172.21.15.157:0/351726059 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c00ecd0 con 0x7f2b6406c2d0 2023-10-06T04:04:25.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.729+0000 7f2b5a7fc700 1 -- 172.21.15.157:0/351726059 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5c00ee30 con 0x7f2b6406c2d0 2023-10-06T04:04:25.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.729+0000 7f2b6359e700 1 -- 172.21.15.157:0/351726059 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b640c0420 con 0x7f2b6406c2d0 2023-10-06T04:04:25.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.729+0000 7f2b5a7fc700 1 -- 172.21.15.157:0/351726059 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c014670 con 0x7f2b6406c2d0 2023-10-06T04:04:25.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.729+0000 7f2b6359e700 1 -- 172.21.15.157:0/351726059 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b64166310 con 0x7f2b6406c2d0 2023-10-06T04:04:25.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.730+0000 7f2b6359e700 1 -- 172.21.15.157:0/351726059 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b640c08c0 con 0x7f2b6406c2d0 2023-10-06T04:04:25.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.730+0000 7f2b5a7fc700 1 -- 172.21.15.157:0/351726059 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2b5c014a20 con 0x7f2b6406c2d0 2023-10-06T04:04:25.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.731+0000 7f2b5a7fc700 1 --2- 172.21.15.157:0/351726059 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2b44066f30 0x7f2b440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:25.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.731+0000 7f2b5a7fc700 1 -- 172.21.15.157:0/351726059 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f2b5c084e50 con 0x7f2b6406c2d0 2023-10-06T04:04:25.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.731+0000 7f2b61d9b700 1 --2- 172.21.15.157:0/351726059 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2b44066f30 0x7f2b440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:25.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.732+0000 7f2b61d9b700 1 --2- 172.21.15.157:0/351726059 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2b44066f30 0x7f2b440693e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2b640cd560 tx=0x7f2b4c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:25.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.735+0000 7f2b5a7fc700 1 -- 172.21.15.157:0/351726059 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2b5c051e60 con 0x7f2b6406c2d0 2023-10-06T04:04:25.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.964+0000 7f2b6359e700 1 -- 172.21.15.157:0/351726059 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b640c0a70 con 0x7f2b6406c2d0 2023-10-06T04:04:25.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.965+0000 7f2b5a7fc700 1 -- 172.21.15.157:0/351726059 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+6147 (secure 0 0 0) 0x7f2b5c009a60 con 0x7f2b6406c2d0 2023-10-06T04:04:25.967 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:04:25.967 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:03:46.794132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":4287},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4287":{"gid":4287,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.157:6834/1179753185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1179753185},{"type":"v1","addr":"172.21.15.157:6834","nonce":1179753185}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:04:25.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.967+0000 7f2b6359e700 1 -- 172.21.15.157:0/351726059 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2b44066f30 msgr2=0x7f2b440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:25.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.967+0000 7f2b6359e700 1 --2- 172.21.15.157:0/351726059 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2b44066f30 0x7f2b440693e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2b640cd560 tx=0x7f2b4c008040 comp rx=0 tx=0).stop 2023-10-06T04:04:25.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.967+0000 7f2b6359e700 1 -- 172.21.15.157:0/351726059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b6406c2d0 msgr2=0x7f2b640c0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:25.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.967+0000 7f2b6359e700 1 --2- 172.21.15.157:0/351726059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b6406c2d0 0x7f2b640c0c20 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00e190 tx=0x7f2b5c009920 comp rx=0 tx=0).stop 2023-10-06T04:04:25.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.967+0000 7f2b6359e700 1 -- 172.21.15.157:0/351726059 shutdown_connections 2023-10-06T04:04:25.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.967+0000 7f2b6359e700 1 --2- 172.21.15.157:0/351726059 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2b44066f30 0x7f2b440693e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:25.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.967+0000 7f2b6359e700 1 --2- 172.21.15.157:0/351726059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b640b1e40 0x7f2b6415bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:25.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.967+0000 7f2b6359e700 1 --2- 172.21.15.157:0/351726059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b6406c2d0 0x7f2b640c0c20 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:25.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.967+0000 7f2b6359e700 1 --2- 172.21.15.157:0/351726059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b640b04d0 0x7f2b6415ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:25.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.967+0000 7f2b6359e700 1 -- 172.21.15.157:0/351726059 >> 172.21.15.157:0/351726059 conn(0x7f2b640cc540 msgr2=0x7f2b64157440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:25.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.967+0000 7f2b6359e700 1 -- 172.21.15.157:0/351726059 shutdown_connections 2023-10-06T04:04:25.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:25.967+0000 7f2b6359e700 1 -- 172.21.15.157:0/351726059 wait complete. 2023-10-06T04:04:25.970 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 57 2023-10-06T04:04:25.981 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-06T04:04:27.982 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:04:28.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.105+0000 7f5aa77d8700 1 Processor -- start 2023-10-06T04:04:28.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.105+0000 7f5aa77d8700 1 -- start start 2023-10-06T04:04:28.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.106+0000 7f5aa77d8700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5aa006dc40 0x7f5aa014fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:28.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.106+0000 7f5aa77d8700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5aa006c1f0 0x7f5aa0150240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:28.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.106+0000 7f5aa77d8700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aa00c9780 con 0x7f5aa006cbb0 2023-10-06T04:04:28.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.106+0000 7f5aa77d8700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aa00c8c40 con 0x7f5aa006c1f0 2023-10-06T04:04:28.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.106+0000 7f5aa77d8700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa006e0c0 con 0x7f5aa006dc40 2023-10-06T04:04:28.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.106+0000 7f5aa4d73700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5aa006dc40 0x7f5aa014fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:28.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.106+0000 7f5aa5d75700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5aa006c1f0 0x7f5aa0150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:28.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.107+0000 7f5aa5574700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f5aa006cbb0 0x7f5aa006d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.157:40008/0 (socket says 172.21.15.157:40008) 2023-10-06T04:04:28.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.107+0000 7f5aa4d73700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5aa006dc40 0x7f5aa014fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:49686/0 (socket says 172.21.15.157:49686) 2023-10-06T04:04:28.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.107+0000 7f5aa5574700 1 -- 172.21.15.157:0/298675102 learned_addr learned my addr 172.21.15.157:0/298675102 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:04:28.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.107+0000 7f5aa5d75700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5aa006c1f0 0x7f5aa0150240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:40710/0 (socket says 172.21.15.157:40710) 2023-10-06T04:04:28.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.107+0000 7f5aa5d75700 1 -- 172.21.15.157:0/298675102 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5aa006dc40 msgr2=0x7f5aa014fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:04:28.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.107+0000 7f5aa5d75700 1 --2- 172.21.15.157:0/298675102 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5aa006dc40 0x7f5aa014fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:28.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.107+0000 7f5aa5d75700 1 -- 172.21.15.157:0/298675102 >> v1:172.21.15.175:6789/0 conn(0x7f5aa006cbb0 legacy=0x7f5aa006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:28.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.107+0000 7f5aa5d75700 1 -- 172.21.15.157:0/298675102 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa00c8e50 con 0x7f5aa006c1f0 2023-10-06T04:04:28.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.107+0000 7f5aa4d73700 1 --2- 172.21.15.157:0/298675102 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5aa006dc40 0x7f5aa014fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:04:28.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.107+0000 7f5aa5d75700 1 --2- 172.21.15.157:0/298675102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5aa006c1f0 0x7f5aa0150240 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5a98009e60 tx=0x7f5a98009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c01583468870d1d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:28.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.107+0000 7f5a97fff700 1 -- 172.21.15.157:0/298675102 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9800e070 con 0x7f5aa006c1f0 2023-10-06T04:04:28.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.107+0000 7f5a97fff700 1 -- 172.21.15.157:0/298675102 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9800bbf0 con 0x7f5aa006c1f0 2023-10-06T04:04:28.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.107+0000 7f5a97fff700 1 -- 172.21.15.157:0/298675102 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a98012430 con 0x7f5aa006c1f0 2023-10-06T04:04:28.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.108+0000 7f5aa77d8700 1 -- 172.21.15.157:0/298675102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5aa006c1f0 msgr2=0x7f5aa0150240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:28.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.108+0000 7f5aa77d8700 1 --2- 172.21.15.157:0/298675102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5aa006c1f0 0x7f5aa0150240 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5a98009e60 tx=0x7f5a98009780 comp rx=0 tx=0).stop 2023-10-06T04:04:28.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.108+0000 7f5aa77d8700 1 -- 172.21.15.157:0/298675102 shutdown_connections 2023-10-06T04:04:28.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.108+0000 7f5aa77d8700 1 --2- 172.21.15.157:0/298675102 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5aa006c1f0 0x7f5aa0150240 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:28.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.108+0000 7f5aa77d8700 1 --2- 172.21.15.157:0/298675102 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5aa006dc40 0x7f5aa014fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:28.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.108+0000 7f5aa77d8700 1 -- 172.21.15.157:0/298675102 >> 172.21.15.157:0/298675102 conn(0x7f5aa006f230 msgr2=0x7f5aa006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:28.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.108+0000 7f5aa77d8700 1 -- 172.21.15.157:0/298675102 shutdown_connections 2023-10-06T04:04:28.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.108+0000 7f5aa77d8700 1 -- 172.21.15.157:0/298675102 wait complete. 2023-10-06T04:04:28.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.109+0000 7f5aa77d8700 1 Processor -- start 2023-10-06T04:04:28.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.109+0000 7f5aa77d8700 1 -- start start 2023-10-06T04:04:28.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.110+0000 7f5aa77d8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aa006c1f0 0x7f5aa0143800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:28.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.110+0000 7f5aa77d8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5aa006cbb0 0x7f5aa0143d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:28.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.110+0000 7f5aa77d8700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5aa0151ed0 0x7f5aa0152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:28.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.110+0000 7f5aa77d8700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa0143370 con 0x7f5aa006c1f0 2023-10-06T04:04:28.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.110+0000 7f5aa77d8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa016a430 con 0x7f5aa006cbb0 2023-10-06T04:04:28.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.110+0000 7f5aa77d8700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa0142a60 con 0x7f5aa0151ed0 2023-10-06T04:04:28.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.110+0000 7f5aa5574700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aa006c1f0 0x7f5aa0143800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:28.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.110+0000 7f5aa5574700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aa006c1f0 0x7f5aa0143800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33272/0 (socket says 172.21.15.157:33272) 2023-10-06T04:04:28.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.110+0000 7f5aa5574700 1 -- 172.21.15.157:0/358917044 learned_addr learned my addr 172.21.15.157:0/358917044 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:28.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.110+0000 7f5aa5d75700 1 --2- 172.21.15.157:0/358917044 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5aa0151ed0 0x7f5aa0152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:28.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.110+0000 7f5aa4d73700 1 --2- 172.21.15.157:0/358917044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5aa006cbb0 0x7f5aa0143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:28.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.110+0000 7f5aa5574700 1 -- 172.21.15.157:0/358917044 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5aa0151ed0 msgr2=0x7f5aa0152340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:28.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.110+0000 7f5aa5574700 1 --2- 172.21.15.157:0/358917044 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5aa0151ed0 0x7f5aa0152340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:28.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.110+0000 7f5aa5574700 1 -- 172.21.15.157:0/358917044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5aa006cbb0 msgr2=0x7f5aa0143d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:28.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.110+0000 7f5aa5574700 1 --2- 172.21.15.157:0/358917044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5aa006cbb0 0x7f5aa0143d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:28.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.110+0000 7f5aa5574700 1 -- 172.21.15.157:0/358917044 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a90009420 con 0x7f5aa006c1f0 2023-10-06T04:04:28.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.110+0000 7f5aa5574700 1 --2- 172.21.15.157:0/358917044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aa006c1f0 0x7f5aa0143800 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00c7c0 tx=0x7f5a9c007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:28.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.110+0000 7f5a957fa700 1 -- 172.21.15.157:0/358917044 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c005b30 con 0x7f5aa006c1f0 2023-10-06T04:04:28.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.111+0000 7f5a957fa700 1 -- 172.21.15.157:0/358917044 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9c005c90 con 0x7f5aa006c1f0 2023-10-06T04:04:28.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.111+0000 7f5aa77d8700 1 -- 172.21.15.157:0/358917044 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a98009a20 con 0x7f5aa006c1f0 2023-10-06T04:04:28.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.111+0000 7f5a957fa700 1 -- 172.21.15.157:0/358917044 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c007320 con 0x7f5aa006c1f0 2023-10-06T04:04:28.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.111+0000 7f5aa77d8700 1 -- 172.21.15.157:0/358917044 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa0143000 con 0x7f5aa006c1f0 2023-10-06T04:04:28.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.111+0000 7f5aa77d8700 1 -- 172.21.15.157:0/358917044 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa0143650 con 0x7f5aa006c1f0 2023-10-06T04:04:28.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.112+0000 7f5a957fa700 1 -- 172.21.15.157:0/358917044 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5a9c007690 con 0x7f5aa006c1f0 2023-10-06T04:04:28.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.112+0000 7f5a957fa700 1 --2- 172.21.15.157:0/358917044 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5a84066f60 0x7f5a84069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:28.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.112+0000 7f5a957fa700 1 -- 172.21.15.157:0/358917044 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6374+0+0 (secure 0 0 0) 0x7f5a9c083d10 con 0x7f5aa006c1f0 2023-10-06T04:04:28.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.112+0000 7f5aa4d73700 1 --2- 172.21.15.157:0/358917044 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5a84066f60 0x7f5a84069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:28.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.113+0000 7f5aa4d73700 1 --2- 172.21.15.157:0/358917044 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5a84066f60 0x7f5a84069410 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5a900093f0 tx=0x7f5a90008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:28.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.117+0000 7f5a957fa700 1 -- 172.21.15.157:0/358917044 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5a9c050ce0 con 0x7f5aa006c1f0 2023-10-06T04:04:28.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.344+0000 7f5aa77d8700 1 -- 172.21.15.157:0/358917044 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5aa014e0d0 con 0x7f5aa006c1f0 2023-10-06T04:04:28.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.344+0000 7f5a957fa700 1 -- 172.21.15.157:0/358917044 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+5387 (secure 0 0 0) 0x7f5a9c017020 con 0x7f5aa006c1f0 2023-10-06T04:04:28.346 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:04:28.346 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:04:26.701282+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:replay","state_seq":1,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:04:28.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.346+0000 7f5aa77d8700 1 -- 172.21.15.157:0/358917044 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5a84066f60 msgr2=0x7f5a84069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:28.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.346+0000 7f5aa77d8700 1 --2- 172.21.15.157:0/358917044 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5a84066f60 0x7f5a84069410 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5a900093f0 tx=0x7f5a90008040 comp rx=0 tx=0).stop 2023-10-06T04:04:28.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.346+0000 7f5aa77d8700 1 -- 172.21.15.157:0/358917044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aa006c1f0 msgr2=0x7f5aa0143800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:28.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.346+0000 7f5aa77d8700 1 --2- 172.21.15.157:0/358917044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aa006c1f0 0x7f5aa0143800 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00c7c0 tx=0x7f5a9c007f30 comp rx=0 tx=0).stop 2023-10-06T04:04:28.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.347+0000 7f5aa77d8700 1 -- 172.21.15.157:0/358917044 shutdown_connections 2023-10-06T04:04:28.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.347+0000 7f5aa77d8700 1 --2- 172.21.15.157:0/358917044 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5a84066f60 0x7f5a84069410 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:28.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.347+0000 7f5aa77d8700 1 --2- 172.21.15.157:0/358917044 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5aa0151ed0 0x7f5aa0152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:28.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.347+0000 7f5aa77d8700 1 --2- 172.21.15.157:0/358917044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5aa006c1f0 0x7f5aa0143800 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:28.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.347+0000 7f5aa77d8700 1 --2- 172.21.15.157:0/358917044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5aa006cbb0 0x7f5aa0143d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:28.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.347+0000 7f5aa77d8700 1 -- 172.21.15.157:0/358917044 >> 172.21.15.157:0/358917044 conn(0x7f5aa006f230 msgr2=0x7f5aa014fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:28.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.347+0000 7f5aa77d8700 1 -- 172.21.15.157:0/358917044 shutdown_connections 2023-10-06T04:04:28.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.347+0000 7f5aa77d8700 1 -- 172.21.15.157:0/358917044 wait complete. 2023-10-06T04:04:28.350 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 58 2023-10-06T04:04:28.360 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2023-10-06T04:04:28.360 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-10-06T04:04:28.360 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:04:28.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.474+0000 7f9745be9700 1 Processor -- start 2023-10-06T04:04:28.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.474+0000 7f9745be9700 1 -- start start 2023-10-06T04:04:28.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.475+0000 7f9745be9700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9740068d10 0x7f974006a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:28.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.475+0000 7f9745be9700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97400bed40 con 0x7f97400be970 2023-10-06T04:04:28.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.475+0000 7f9745be9700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9740157010 con 0x7f974006d460 2023-10-06T04:04:28.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.475+0000 7f9745be9700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f974006f9f0 con 0x7f9740068d10 2023-10-06T04:04:28.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.475+0000 7f973f7fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f97400be970 0x7f974006df10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:35292/0 (socket says 172.21.15.157:35292) 2023-10-06T04:04:28.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.475+0000 7f973f7fe700 1 -- 172.21.15.157:0/3818909395 learned_addr learned my addr 172.21.15.157:0/3818909395 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:04:28.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.475+0000 7f973effd700 1 --2- 172.21.15.157:0/3818909395 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9740068d10 0x7f974006a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:28.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.475+0000 7f973e7fc700 1 -- 172.21.15.157:0/3818909395 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3973027591 0 0) 0x7f97400bed40 con 0x7f97400be970 2023-10-06T04:04:28.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.475+0000 7f973e7fc700 1 -- 172.21.15.157:0/3818909395 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9728003570 con 0x7f97400be970 2023-10-06T04:04:28.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.475+0000 7f973e7fc700 1 -- 172.21.15.157:0/3818909395 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3165967376 0 0) 0x7f9728003570 con 0x7f97400be970 2023-10-06T04:04:28.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.475+0000 7f973e7fc700 1 -- 172.21.15.157:0/3818909395 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9740068d10 msgr2=0x7f974006a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:04:28.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.475+0000 7f973e7fc700 1 --2- 172.21.15.157:0/3818909395 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9740068d10 0x7f974006a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:28.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.475+0000 7f973e7fc700 1 -- 172.21.15.157:0/3818909395 >> v1:172.21.15.175:6789/0 conn(0x7f974006d460 legacy=0x7f974015d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:28.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.475+0000 7f973e7fc700 1 -- 172.21.15.157:0/3818909395 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97401574a0 con 0x7f97400be970 2023-10-06T04:04:28.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.475+0000 7f973e7fc700 1 -- 172.21.15.157:0/3818909395 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f9730001470 con 0x7f97400be970 2023-10-06T04:04:28.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.475+0000 7f973e7fc700 1 -- 172.21.15.157:0/3818909395 >> v1:172.21.15.157:6789/0 conn(0x7f97400be970 legacy=0x7f974006df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:28.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.475+0000 7f973e7fc700 1 --2- 172.21.15.157:0/3818909395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9728003a90 0x7f9728003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:28.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.475+0000 7f973e7fc700 1 --2- 172.21.15.157:0/3818909395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9728004810 0x7f9728004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:28.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.475+0000 7f973e7fc700 1 --2- 172.21.15.157:0/3818909395 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9740068d10 0x7f97280051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:28.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.475+0000 7f973e7fc700 1 -- 172.21.15.157:0/3818909395 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974006f9f0 con 0x7f9728003a90 2023-10-06T04:04:28.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.475+0000 7f973e7fc700 1 -- 172.21.15.157:0/3818909395 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972800bb60 con 0x7f9728004810 2023-10-06T04:04:28.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.475+0000 7f973e7fc700 1 -- 172.21.15.157:0/3818909395 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f972800df30 con 0x7f9740068d10 2023-10-06T04:04:28.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.475+0000 7f973f7fe700 1 --2- 172.21.15.157:0/3818909395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9728003a90 0x7f9728003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:28.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.476+0000 7f973effd700 1 --2- 172.21.15.157:0/3818909395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9728004810 0x7f9728004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:28.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.476+0000 7f973ffff700 1 --2- 172.21.15.157:0/3818909395 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9740068d10 0x7f97280051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:28.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.476+0000 7f973f7fe700 1 -- 172.21.15.157:0/3818909395 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9740068d10 msgr2=0x7f97280051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:04:28.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.476+0000 7f973f7fe700 1 --2- 172.21.15.157:0/3818909395 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9740068d10 0x7f97280051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:28.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.476+0000 7f973f7fe700 1 -- 172.21.15.157:0/3818909395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9728004810 msgr2=0x7f9728004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:04:28.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.476+0000 7f973f7fe700 1 --2- 172.21.15.157:0/3818909395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9728004810 0x7f9728004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:28.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.476+0000 7f973f7fe700 1 -- 172.21.15.157:0/3818909395 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9728003570 con 0x7f9728003a90 2023-10-06T04:04:28.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.476+0000 7f973f7fe700 1 --2- 172.21.15.157:0/3818909395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9728003a90 0x7f9728003ea0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f973000c3c0 tx=0x7f9730001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38880260f0891053 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:28.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.476+0000 7f973e7fc700 1 -- 172.21.15.157:0/3818909395 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973000cc90 con 0x7f9728003a90 2023-10-06T04:04:28.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.476+0000 7f973e7fc700 1 -- 172.21.15.157:0/3818909395 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973000cdf0 con 0x7f9728003a90 2023-10-06T04:04:28.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.476+0000 7f973e7fc700 1 -- 172.21.15.157:0/3818909395 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9730005b50 con 0x7f9728003a90 2023-10-06T04:04:28.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.477+0000 7f9745be9700 1 -- 172.21.15.157:0/3818909395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9728003a90 msgr2=0x7f9728003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:28.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.477+0000 7f9745be9700 1 --2- 172.21.15.157:0/3818909395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9728003a90 0x7f9728003ea0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f973000c3c0 tx=0x7f9730001e50 comp rx=0 tx=0).stop 2023-10-06T04:04:28.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.477+0000 7f973ffff700 1 -- 172.21.15.157:0/3818909395 reap_dead start 2023-10-06T04:04:28.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.477+0000 7f9745be9700 1 -- 172.21.15.157:0/3818909395 shutdown_connections 2023-10-06T04:04:28.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.477+0000 7f9745be9700 1 -- 172.21.15.157:0/3818909395 >> 172.21.15.157:0/3818909395 conn(0x7f97400c8d70 msgr2=0x7f97400c4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:28.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.477+0000 7f9745be9700 1 -- 172.21.15.157:0/3818909395 shutdown_connections 2023-10-06T04:04:28.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.477+0000 7f9745be9700 1 -- 172.21.15.157:0/3818909395 wait complete. 2023-10-06T04:04:28.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.478+0000 7f9745be9700 1 Processor -- start 2023-10-06T04:04:28.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.478+0000 7f9745be9700 1 -- start start 2023-10-06T04:04:28.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.479+0000 7f9745be9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9728003a90 0x7f9740163540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:28.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.479+0000 7f9745be9700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97401669c0 0x7f9740163eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:28.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.479+0000 7f9745be9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f974006e0d0 0x7f974014b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:28.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.479+0000 7f9745be9700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97400ad190 con 0x7f974006e0d0 2023-10-06T04:04:28.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.479+0000 7f9745be9700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97401630b0 con 0x7f9728003a90 2023-10-06T04:04:28.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.479+0000 7f9745be9700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9740151760 con 0x7f97401669c0 2023-10-06T04:04:28.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.479+0000 7f973ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f974006e0d0 0x7f974014b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:28.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.479+0000 7f973ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f974006e0d0 0x7f974014b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33298/0 (socket says 172.21.15.157:33298) 2023-10-06T04:04:28.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.479+0000 7f973ffff700 1 -- 172.21.15.157:0/3287702700 learned_addr learned my addr 172.21.15.157:0/3287702700 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:28.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.479+0000 7f973effd700 1 --2- 172.21.15.157:0/3287702700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97401669c0 0x7f9740163eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:28.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.479+0000 7f973f7fe700 1 --2- 172.21.15.157:0/3287702700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9728003a90 0x7f9740163540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:28.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.479+0000 7f973ffff700 1 -- 172.21.15.157:0/3287702700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97401669c0 msgr2=0x7f9740163eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:28.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.479+0000 7f973ffff700 1 --2- 172.21.15.157:0/3287702700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97401669c0 0x7f9740163eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:28.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.479+0000 7f973ffff700 1 -- 172.21.15.157:0/3287702700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9728003a90 msgr2=0x7f9740163540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:28.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.479+0000 7f973ffff700 1 --2- 172.21.15.157:0/3287702700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9728003a90 0x7f9740163540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:28.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.479+0000 7f973ffff700 1 -- 172.21.15.157:0/3287702700 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9740151a40 con 0x7f974006e0d0 2023-10-06T04:04:28.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.479+0000 7f973ffff700 1 --2- 172.21.15.157:0/3287702700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f974006e0d0 0x7f974014b770 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9740064920 tx=0x7f9738001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:28.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.479+0000 7f97277fe700 1 -- 172.21.15.157:0/3287702700 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9738018070 con 0x7f974006e0d0 2023-10-06T04:04:28.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.479+0000 7f97277fe700 1 -- 172.21.15.157:0/3287702700 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973800ee70 con 0x7f974006e0d0 2023-10-06T04:04:28.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.479+0000 7f97277fe700 1 -- 172.21.15.157:0/3287702700 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9738006dd0 con 0x7f974006e0d0 2023-10-06T04:04:28.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.479+0000 7f9745be9700 1 -- 172.21.15.157:0/3287702700 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9740162d40 con 0x7f974006e0d0 2023-10-06T04:04:28.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.479+0000 7f9745be9700 1 -- 172.21.15.157:0/3287702700 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9740155d40 con 0x7f974006e0d0 2023-10-06T04:04:28.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.480+0000 7f9745be9700 1 -- 172.21.15.157:0/3287702700 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97400716b0 con 0x7f974006e0d0 2023-10-06T04:04:28.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.481+0000 7f97277fe700 1 -- 172.21.15.157:0/3287702700 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f973800c9c0 con 0x7f974006e0d0 2023-10-06T04:04:28.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.481+0000 7f97277fe700 1 --2- 172.21.15.157:0/3287702700 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f972c066f60 0x7f972c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:28.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.481+0000 7f97277fe700 1 -- 172.21.15.157:0/3287702700 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6374+0+0 (secure 0 0 0) 0x7f9738083f10 con 0x7f974006e0d0 2023-10-06T04:04:28.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.481+0000 7f973f7fe700 1 --2- 172.21.15.157:0/3287702700 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f972c066f60 0x7f972c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:28.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.482+0000 7f973f7fe700 1 --2- 172.21.15.157:0/3287702700 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f972c066f60 0x7f972c069410 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9730007cd0 tx=0x7f9730015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:28.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.485+0000 7f97277fe700 1 -- 172.21.15.157:0/3287702700 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f973804d5a0 con 0x7f974006e0d0 2023-10-06T04:04:28.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.711+0000 7f9745be9700 1 -- 172.21.15.157:0/3287702700 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97400c9170 con 0x7f974006e0d0 2023-10-06T04:04:28.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.712+0000 7f97277fe700 1 -- 172.21.15.157:0/3287702700 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+5387 (secure 0 0 0) 0x7f9738050ee0 con 0x7f974006e0d0 2023-10-06T04:04:28.714 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:04:28.714 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:04:26.701282+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:replay","state_seq":1,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:04:28.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.714+0000 7f9745be9700 1 -- 172.21.15.157:0/3287702700 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f972c066f60 msgr2=0x7f972c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:28.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.714+0000 7f9745be9700 1 --2- 172.21.15.157:0/3287702700 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f972c066f60 0x7f972c069410 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9730007cd0 tx=0x7f9730015000 comp rx=0 tx=0).stop 2023-10-06T04:04:28.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.714+0000 7f9745be9700 1 -- 172.21.15.157:0/3287702700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f974006e0d0 msgr2=0x7f974014b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:28.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.714+0000 7f9745be9700 1 --2- 172.21.15.157:0/3287702700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f974006e0d0 0x7f974014b770 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9740064920 tx=0x7f9738001640 comp rx=0 tx=0).stop 2023-10-06T04:04:28.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.714+0000 7f9745be9700 1 -- 172.21.15.157:0/3287702700 shutdown_connections 2023-10-06T04:04:28.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.714+0000 7f9745be9700 1 --2- 172.21.15.157:0/3287702700 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f972c066f60 0x7f972c069410 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:28.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.714+0000 7f9745be9700 1 --2- 172.21.15.157:0/3287702700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9728003a90 0x7f9740163540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:28.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.714+0000 7f9745be9700 1 --2- 172.21.15.157:0/3287702700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f974006e0d0 0x7f974014b770 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:28.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.714+0000 7f9745be9700 1 --2- 172.21.15.157:0/3287702700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97401669c0 0x7f9740163eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:28.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.714+0000 7f9745be9700 1 -- 172.21.15.157:0/3287702700 >> 172.21.15.157:0/3287702700 conn(0x7f97400c8d70 msgr2=0x7f974006a660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:28.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.714+0000 7f9745be9700 1 -- 172.21.15.157:0/3287702700 shutdown_connections 2023-10-06T04:04:28.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:28.714+0000 7f9745be9700 1 -- 172.21.15.157:0/3287702700 wait complete. 2023-10-06T04:04:28.717 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 58 2023-10-06T04:04:28.728 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 58, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-06T03:57:10.718782+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 58, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_4296": { "addr": "172.21.15.157:6837/4188652695", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6836", "nonce": 4188652695, "type": "v2" }, { "addr": "172.21.15.157:6837", "nonce": 4188652695, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1 ], "features": 4540138320759226367, "flags": 0, "gid": 4296, "incarnation": 4, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 4 }, "gid_5362": { "addr": "172.21.15.175:6837/2388164729", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6836", "nonce": 2388164729, "type": "v2" }, { "addr": "172.21.15.175:6837", "nonce": 2388164729, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5362, "incarnation": 58, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 44, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2023-10-06T04:04:26.701282+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2, 3, 4 ], "tableserver": 0, "up": { "mds_0": 4296, "mds_1": 5362 } } } ], "standbys": [ { "addr": "172.21.15.175:6835/1799968248", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6834", "nonce": 1799968248, "type": "v2" }, { "addr": "172.21.15.175:6835", "nonce": 1799968248, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 57, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5126, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.175:6839/745804271", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6838", "nonce": 745804271, "type": "v2" }, { "addr": "172.21.15.175:6839", "nonce": 745804271, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 42, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5164, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.157:6835/2461751318", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6833", "nonce": 2461751318, "type": "v2" }, { "addr": "172.21.15.157:6835", "nonce": 2461751318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 34, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5268, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-06T04:04:30.729 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:04:30.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.855+0000 7f61f0754700 1 Processor -- start 2023-10-06T04:04:30.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.855+0000 7f61f0754700 1 -- start start 2023-10-06T04:04:30.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.856+0000 7f61f0754700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f61e814dcd0 0x7f61e8153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:30.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.856+0000 7f61f0754700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61e814e0a0 0x7f61e80c18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:30.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.856+0000 7f61f0754700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f61e80c5c00 0x7f61e80ca440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:30.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.856+0000 7f61f0754700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61e8154c80 con 0x7f61e814dcd0 2023-10-06T04:04:30.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.856+0000 7f61f0754700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61e8154200 con 0x7f61e80c5c00 2023-10-06T04:04:30.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.856+0000 7f61f0754700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e8068b00 con 0x7f61e814e0a0 2023-10-06T04:04:30.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.857+0000 7f61ee4f0700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f61e814dcd0 0x7f61e8153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:30.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.857+0000 7f61ee4f0700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f61e814dcd0 0x7f61e8153cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33302/0 (socket says 172.21.15.157:33302) 2023-10-06T04:04:30.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.857+0000 7f61ee4f0700 1 -- 172.21.15.157:0/1764064343 learned_addr learned my addr 172.21.15.157:0/1764064343 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:30.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.857+0000 7f61edcef700 1 --2- 172.21.15.157:0/1764064343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61e814e0a0 0x7f61e80c18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:30.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.857+0000 7f61eecf1700 1 --2- 172.21.15.157:0/1764064343 >> v2:172.21.15.175:3300/0 conn(0x7f61e80c5c00 0x7f61e80ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:30.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.857+0000 7f61ee4f0700 1 -- 172.21.15.157:0/1764064343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61e814e0a0 msgr2=0x7f61e80c18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:04:30.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.857+0000 7f61ee4f0700 1 --2- 172.21.15.157:0/1764064343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61e814e0a0 0x7f61e80c18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:30.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.857+0000 7f61ee4f0700 1 -- 172.21.15.157:0/1764064343 >> v2:172.21.15.175:3300/0 conn(0x7f61e80c5c00 msgr2=0x7f61e80ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:04:30.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.857+0000 7f61ee4f0700 1 --2- 172.21.15.157:0/1764064343 >> v2:172.21.15.175:3300/0 conn(0x7f61e80c5c00 0x7f61e80ca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:30.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.857+0000 7f61ee4f0700 1 -- 172.21.15.157:0/1764064343 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e80c5770 con 0x7f61e814dcd0 2023-10-06T04:04:30.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.857+0000 7f61ee4f0700 1 --2- 172.21.15.157:0/1764064343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61e814dcd0 0x7f61e8153cc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f61e80c48f0 tx=0x7f61e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a112dedf378bf3f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:30.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.857+0000 7f61ed4ee700 1 -- 172.21.15.157:0/1764064343 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e400b820 con 0x7f61e814dcd0 2023-10-06T04:04:30.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.857+0000 7f61ed4ee700 1 -- 172.21.15.157:0/1764064343 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e400b980 con 0x7f61e814dcd0 2023-10-06T04:04:30.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.857+0000 7f61ed4ee700 1 -- 172.21.15.157:0/1764064343 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e400bd20 con 0x7f61e814dcd0 2023-10-06T04:04:30.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.858+0000 7f61f0754700 1 -- 172.21.15.157:0/1764064343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61e814dcd0 msgr2=0x7f61e8153cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:30.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.858+0000 7f61f0754700 1 --2- 172.21.15.157:0/1764064343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61e814dcd0 0x7f61e8153cc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f61e80c48f0 tx=0x7f61e4009250 comp rx=0 tx=0).stop 2023-10-06T04:04:30.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.858+0000 7f61f0754700 1 -- 172.21.15.157:0/1764064343 shutdown_connections 2023-10-06T04:04:30.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.858+0000 7f61f0754700 1 --2- 172.21.15.157:0/1764064343 >> v2:172.21.15.175:3300/0 conn(0x7f61e80c5c00 0x7f61e80ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:30.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.858+0000 7f61f0754700 1 --2- 172.21.15.157:0/1764064343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61e814dcd0 0x7f61e8153cc0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:30.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.858+0000 7f61f0754700 1 --2- 172.21.15.157:0/1764064343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61e814e0a0 0x7f61e80c18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:30.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.858+0000 7f61f0754700 1 -- 172.21.15.157:0/1764064343 >> 172.21.15.157:0/1764064343 conn(0x7f61e80692b0 msgr2=0x7f61e80696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:30.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.858+0000 7f61f0754700 1 -- 172.21.15.157:0/1764064343 shutdown_connections 2023-10-06T04:04:30.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.858+0000 7f61f0754700 1 -- 172.21.15.157:0/1764064343 wait complete. 2023-10-06T04:04:30.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.859+0000 7f61f0754700 1 Processor -- start 2023-10-06T04:04:30.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.859+0000 7f61f0754700 1 -- start start 2023-10-06T04:04:30.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.860+0000 7f61f0754700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61e80c5c00 0x7f61e80c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:30.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.860+0000 7f61f0754700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61e814dcd0 0x7f61e8156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:30.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.860+0000 7f61f0754700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61e814e0a0 0x7f61e80c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:30.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.860+0000 7f61f0754700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e81512a0 con 0x7f61e80c5c00 2023-10-06T04:04:30.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.860+0000 7f61f0754700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e8156220 con 0x7f61e814e0a0 2023-10-06T04:04:30.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.860+0000 7f61f0754700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e8156500 con 0x7f61e814dcd0 2023-10-06T04:04:30.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.860+0000 7f61ee4f0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61e80c5c00 0x7f61e80c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:30.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.860+0000 7f61ee4f0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61e80c5c00 0x7f61e80c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33318/0 (socket says 172.21.15.157:33318) 2023-10-06T04:04:30.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.860+0000 7f61ee4f0700 1 -- 172.21.15.157:0/1999747032 learned_addr learned my addr 172.21.15.157:0/1999747032 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:30.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.860+0000 7f61edcef700 1 --2- 172.21.15.157:0/1999747032 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61e814dcd0 0x7f61e8156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:30.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.860+0000 7f61eecf1700 1 --2- 172.21.15.157:0/1999747032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61e814e0a0 0x7f61e80c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:30.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.860+0000 7f61ee4f0700 1 -- 172.21.15.157:0/1999747032 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61e814dcd0 msgr2=0x7f61e8156b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:30.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.860+0000 7f61ee4f0700 1 --2- 172.21.15.157:0/1999747032 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61e814dcd0 0x7f61e8156b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:30.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.860+0000 7f61ee4f0700 1 -- 172.21.15.157:0/1999747032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61e814e0a0 msgr2=0x7f61e80c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:30.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.860+0000 7f61ee4f0700 1 --2- 172.21.15.157:0/1999747032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61e814e0a0 0x7f61e80c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:30.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.860+0000 7f61ee4f0700 1 -- 172.21.15.157:0/1999747032 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e40094f0 con 0x7f61e80c5c00 2023-10-06T04:04:30.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.860+0000 7f61ee4f0700 1 --2- 172.21.15.157:0/1999747032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61e80c5c00 0x7f61e80c82c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f61e40094c0 tx=0x7f61e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:30.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.860+0000 7f61de7fc700 1 -- 172.21.15.157:0/1999747032 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e4024440 con 0x7f61e80c5c00 2023-10-06T04:04:30.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.860+0000 7f61de7fc700 1 -- 172.21.15.157:0/1999747032 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e40245e0 con 0x7f61e80c5c00 2023-10-06T04:04:30.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.860+0000 7f61f0754700 1 -- 172.21.15.157:0/1999747032 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e8151560 con 0x7f61e80c5c00 2023-10-06T04:04:30.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.860+0000 7f61de7fc700 1 -- 172.21.15.157:0/1999747032 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e40249a0 con 0x7f61e80c5c00 2023-10-06T04:04:30.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.861+0000 7f61f0754700 1 -- 172.21.15.157:0/1999747032 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e81681d0 con 0x7f61e80c5c00 2023-10-06T04:04:30.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.861+0000 7f61f0754700 1 -- 172.21.15.157:0/1999747032 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e806a8f0 con 0x7f61e80c5c00 2023-10-06T04:04:30.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.862+0000 7f61de7fc700 1 -- 172.21.15.157:0/1999747032 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f61e4024d50 con 0x7f61e80c5c00 2023-10-06T04:04:30.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.862+0000 7f61de7fc700 1 --2- 172.21.15.157:0/1999747032 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f61cc066f30 0x7f61cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:30.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.862+0000 7f61de7fc700 1 -- 172.21.15.157:0/1999747032 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6374+0+0 (secure 0 0 0) 0x7f61e4086330 con 0x7f61e80c5c00 2023-10-06T04:04:30.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.862+0000 7f61edcef700 1 --2- 172.21.15.157:0/1999747032 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f61cc066f30 0x7f61cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:30.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.863+0000 7f61edcef700 1 --2- 172.21.15.157:0/1999747032 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f61cc066f30 0x7f61cc0693e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f61e81547d0 tx=0x7f61d8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:30.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:30.866+0000 7f61de7fc700 1 -- 172.21.15.157:0/1999747032 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61e4053300 con 0x7f61e80c5c00 2023-10-06T04:04:31.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:31.090+0000 7f61f0754700 1 -- 172.21.15.157:0/1999747032 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61e8156970 con 0x7f61e80c5c00 2023-10-06T04:04:31.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:31.091+0000 7f61de7fc700 1 -- 172.21.15.157:0/1999747032 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+5387 (secure 0 0 0) 0x7f61e4056920 con 0x7f61e80c5c00 2023-10-06T04:04:31.092 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:04:31.092 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:04:26.701282+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:replay","state_seq":1,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:04:31.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:31.093+0000 7f61f0754700 1 -- 172.21.15.157:0/1999747032 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f61cc066f30 msgr2=0x7f61cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:31.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:31.093+0000 7f61f0754700 1 --2- 172.21.15.157:0/1999747032 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f61cc066f30 0x7f61cc0693e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f61e81547d0 tx=0x7f61d8008040 comp rx=0 tx=0).stop 2023-10-06T04:04:31.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:31.093+0000 7f61f0754700 1 -- 172.21.15.157:0/1999747032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61e80c5c00 msgr2=0x7f61e80c82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:31.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:31.093+0000 7f61f0754700 1 --2- 172.21.15.157:0/1999747032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61e80c5c00 0x7f61e80c82c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f61e40094c0 tx=0x7f61e4017000 comp rx=0 tx=0).stop 2023-10-06T04:04:31.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:31.093+0000 7f61f0754700 1 -- 172.21.15.157:0/1999747032 shutdown_connections 2023-10-06T04:04:31.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:31.093+0000 7f61f0754700 1 --2- 172.21.15.157:0/1999747032 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f61cc066f30 0x7f61cc0693e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:31.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:31.093+0000 7f61f0754700 1 --2- 172.21.15.157:0/1999747032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61e814e0a0 0x7f61e80c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:31.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:31.093+0000 7f61f0754700 1 --2- 172.21.15.157:0/1999747032 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61e80c5c00 0x7f61e80c82c0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:31.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:31.093+0000 7f61f0754700 1 --2- 172.21.15.157:0/1999747032 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61e814dcd0 0x7f61e8156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:31.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:31.093+0000 7f61f0754700 1 -- 172.21.15.157:0/1999747032 >> 172.21.15.157:0/1999747032 conn(0x7f61e80692b0 msgr2=0x7f61e814a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:31.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:31.093+0000 7f61f0754700 1 -- 172.21.15.157:0/1999747032 shutdown_connections 2023-10-06T04:04:31.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:31.093+0000 7f61f0754700 1 -- 172.21.15.157:0/1999747032 wait complete. 2023-10-06T04:04:31.096 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 58 2023-10-06T04:04:31.106 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:04:33.107 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:04:33.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.234+0000 7fc27db7d700 1 Processor -- start 2023-10-06T04:04:33.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.234+0000 7fc27db7d700 1 -- start start 2023-10-06T04:04:33.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.234+0000 7fc27db7d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc27806cbb0 0x7fc27806d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:33.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.235+0000 7fc27db7d700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2780c9780 con 0x7fc278150410 2023-10-06T04:04:33.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.235+0000 7fc27db7d700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc27806e0c0 con 0x7fc27806c1f0 2023-10-06T04:04:33.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.235+0000 7fc27db7d700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2780c8c40 con 0x7fc27806cbb0 2023-10-06T04:04:33.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.235+0000 7fc2777fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc27806cbb0 0x7fc27806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:33.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.235+0000 7fc2777fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc27806cbb0 0x7fc27806d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60472/0 (socket says 172.21.15.157:60472) 2023-10-06T04:04:33.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.235+0000 7fc2777fe700 1 -- 172.21.15.157:0/3183196689 learned_addr learned my addr 172.21.15.157:0/3183196689 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:33.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.235+0000 7fc2777fe700 1 -- 172.21.15.157:0/3183196689 >> v1:172.21.15.175:6789/0 conn(0x7fc27806c1f0 legacy=0x7fc27814fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:33.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.235+0000 7fc2777fe700 1 -- 172.21.15.157:0/3183196689 >> v1:172.21.15.157:6789/0 conn(0x7fc278150410 legacy=0x7fc278155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:33.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.235+0000 7fc2777fe700 1 -- 172.21.15.157:0/3183196689 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2780c8e50 con 0x7fc27806cbb0 2023-10-06T04:04:33.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.235+0000 7fc2777fe700 1 --2- 172.21.15.157:0/3183196689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc27806cbb0 0x7fc27806d530 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc2680099b0 tx=0x7fc268009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32abca408e62e8f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:33.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.235+0000 7fc2767fc700 1 -- 172.21.15.157:0/3183196689 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26800b8f0 con 0x7fc27806cbb0 2023-10-06T04:04:33.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.235+0000 7fc2767fc700 1 -- 172.21.15.157:0/3183196689 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc26800ba50 con 0x7fc27806cbb0 2023-10-06T04:04:33.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.235+0000 7fc2767fc700 1 -- 172.21.15.157:0/3183196689 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26800bdf0 con 0x7fc27806cbb0 2023-10-06T04:04:33.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.236+0000 7fc27db7d700 1 -- 172.21.15.157:0/3183196689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc27806cbb0 msgr2=0x7fc27806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:33.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.236+0000 7fc27db7d700 1 --2- 172.21.15.157:0/3183196689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc27806cbb0 0x7fc27806d530 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc2680099b0 tx=0x7fc268009250 comp rx=0 tx=0).stop 2023-10-06T04:04:33.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.236+0000 7fc27db7d700 1 -- 172.21.15.157:0/3183196689 shutdown_connections 2023-10-06T04:04:33.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.236+0000 7fc27db7d700 1 --2- 172.21.15.157:0/3183196689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc27806cbb0 0x7fc27806d530 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:33.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.236+0000 7fc27db7d700 1 -- 172.21.15.157:0/3183196689 >> 172.21.15.157:0/3183196689 conn(0x7fc27806f230 msgr2=0x7fc27806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:33.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.236+0000 7fc27db7d700 1 -- 172.21.15.157:0/3183196689 shutdown_connections 2023-10-06T04:04:33.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.236+0000 7fc27db7d700 1 -- 172.21.15.157:0/3183196689 wait complete. 2023-10-06T04:04:33.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.237+0000 7fc27db7d700 1 Processor -- start 2023-10-06T04:04:33.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.237+0000 7fc27db7d700 1 -- start start 2023-10-06T04:04:33.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.238+0000 7fc27db7d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc27806c1f0 0x7fc27814e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:33.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.238+0000 7fc27db7d700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc27806cbb0 0x7fc278154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:33.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.238+0000 7fc27db7d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc278150410 0x7fc278155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:33.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.238+0000 7fc27db7d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc278154070 con 0x7fc27806c1f0 2023-10-06T04:04:33.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.238+0000 7fc27db7d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2780c4960 con 0x7fc278150410 2023-10-06T04:04:33.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.238+0000 7fc27db7d700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2780c54e0 con 0x7fc27806cbb0 2023-10-06T04:04:33.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.238+0000 7fc2777fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc27806c1f0 0x7fc27814e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:33.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.238+0000 7fc2777fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc27806c1f0 0x7fc27814e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60482/0 (socket says 172.21.15.157:60482) 2023-10-06T04:04:33.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.238+0000 7fc2777fe700 1 -- 172.21.15.157:0/1577850981 learned_addr learned my addr 172.21.15.157:0/1577850981 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:33.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.238+0000 7fc277fff700 1 --2- 172.21.15.157:0/1577850981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc278150410 0x7fc278155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:33.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.238+0000 7fc276ffd700 1 --2- 172.21.15.157:0/1577850981 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc27806cbb0 0x7fc278154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:33.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.238+0000 7fc2777fe700 1 -- 172.21.15.157:0/1577850981 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc27806cbb0 msgr2=0x7fc278154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:33.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.238+0000 7fc2777fe700 1 --2- 172.21.15.157:0/1577850981 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc27806cbb0 0x7fc278154ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:33.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.238+0000 7fc2777fe700 1 -- 172.21.15.157:0/1577850981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc278150410 msgr2=0x7fc278155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:33.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.238+0000 7fc2777fe700 1 --2- 172.21.15.157:0/1577850981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc278150410 0x7fc278155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:33.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.238+0000 7fc2777fe700 1 -- 172.21.15.157:0/1577850981 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2680094f0 con 0x7fc27806c1f0 2023-10-06T04:04:33.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.238+0000 7fc277fff700 1 --2- 172.21.15.157:0/1577850981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc278150410 0x7fc278155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:04:33.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.238+0000 7fc276ffd700 1 --2- 172.21.15.157:0/1577850981 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc27806cbb0 0x7fc278154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:04:33.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.238+0000 7fc2777fe700 1 --2- 172.21.15.157:0/1577850981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc27806c1f0 0x7fc27814e250 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc2780c3960 tx=0x7fc268017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:33.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.239+0000 7fc25f7fe700 1 -- 172.21.15.157:0/1577850981 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268011e60 con 0x7fc27806c1f0 2023-10-06T04:04:33.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.239+0000 7fc25f7fe700 1 -- 172.21.15.157:0/1577850981 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc26801a630 con 0x7fc27806c1f0 2023-10-06T04:04:33.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.239+0000 7fc27db7d700 1 -- 172.21.15.157:0/1577850981 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2780c67c0 con 0x7fc27806c1f0 2023-10-06T04:04:33.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.239+0000 7fc25f7fe700 1 -- 172.21.15.157:0/1577850981 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26801a9f0 con 0x7fc27806c1f0 2023-10-06T04:04:33.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.239+0000 7fc27db7d700 1 -- 172.21.15.157:0/1577850981 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2780c4bb0 con 0x7fc27806c1f0 2023-10-06T04:04:33.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.240+0000 7fc27db7d700 1 -- 172.21.15.157:0/1577850981 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2780c5870 con 0x7fc27806c1f0 2023-10-06T04:04:33.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.240+0000 7fc25f7fe700 1 -- 172.21.15.157:0/1577850981 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc2680233e0 con 0x7fc27806c1f0 2023-10-06T04:04:33.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.240+0000 7fc25f7fe700 1 --2- 172.21.15.157:0/1577850981 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc264066fb0 0x7fc264069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:33.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.240+0000 7fc25f7fe700 1 -- 172.21.15.157:0/1577850981 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6374+0+0 (secure 0 0 0) 0x7fc268085f90 con 0x7fc27806c1f0 2023-10-06T04:04:33.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.241+0000 7fc276ffd700 1 --2- 172.21.15.157:0/1577850981 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc264066fb0 0x7fc264069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:33.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.241+0000 7fc276ffd700 1 --2- 172.21.15.157:0/1577850981 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc264066fb0 0x7fc264069460 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc26c007fd0 tx=0x7fc26c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:33.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.245+0000 7fc25f7fe700 1 -- 172.21.15.157:0/1577850981 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc268052ec0 con 0x7fc27806c1f0 2023-10-06T04:04:33.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.471+0000 7fc27db7d700 1 -- 172.21.15.157:0/1577850981 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2780c9130 con 0x7fc27806c1f0 2023-10-06T04:04:33.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.471+0000 7fc25f7fe700 1 -- 172.21.15.157:0/1577850981 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+5389 (secure 0 0 0) 0x7fc268009350 con 0x7fc27806c1f0 2023-10-06T04:04:33.473 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:04:33.473 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:04:32.766257+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:resolve","state_seq":16,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:04:33.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.473+0000 7fc27db7d700 1 -- 172.21.15.157:0/1577850981 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc264066fb0 msgr2=0x7fc264069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:33.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.473+0000 7fc27db7d700 1 --2- 172.21.15.157:0/1577850981 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc264066fb0 0x7fc264069460 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc26c007fd0 tx=0x7fc26c009040 comp rx=0 tx=0).stop 2023-10-06T04:04:33.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.473+0000 7fc27db7d700 1 -- 172.21.15.157:0/1577850981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc27806c1f0 msgr2=0x7fc27814e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:33.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.473+0000 7fc27db7d700 1 --2- 172.21.15.157:0/1577850981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc27806c1f0 0x7fc27814e250 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc2780c3960 tx=0x7fc268017000 comp rx=0 tx=0).stop 2023-10-06T04:04:33.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.473+0000 7fc27db7d700 1 -- 172.21.15.157:0/1577850981 shutdown_connections 2023-10-06T04:04:33.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.473+0000 7fc27db7d700 1 --2- 172.21.15.157:0/1577850981 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc264066fb0 0x7fc264069460 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:33.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.473+0000 7fc27db7d700 1 --2- 172.21.15.157:0/1577850981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc278150410 0x7fc278155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:33.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.473+0000 7fc27db7d700 1 --2- 172.21.15.157:0/1577850981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc27806c1f0 0x7fc27814e250 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:33.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.473+0000 7fc27db7d700 1 --2- 172.21.15.157:0/1577850981 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc27806cbb0 0x7fc278154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:33.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.473+0000 7fc27db7d700 1 -- 172.21.15.157:0/1577850981 >> 172.21.15.157:0/1577850981 conn(0x7fc27806f230 msgr2=0x7fc2780c6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:33.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.474+0000 7fc27db7d700 1 -- 172.21.15.157:0/1577850981 shutdown_connections 2023-10-06T04:04:33.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:33.474+0000 7fc27db7d700 1 -- 172.21.15.157:0/1577850981 wait complete. 2023-10-06T04:04:33.476 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 59 2023-10-06T04:04:33.487 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:04:35.488 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:04:35.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.615+0000 7f4742f41700 1 Processor -- start 2023-10-06T04:04:35.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.615+0000 7f4742f41700 1 -- start start 2023-10-06T04:04:35.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.616+0000 7f4742f41700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f473c06d550 0x7f473c0c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:35.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.616+0000 7f4742f41700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f473c0c2b60 0x7f473c153410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:35.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.616+0000 7f4742f41700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f473c0c1480 con 0x7f473c06d920 2023-10-06T04:04:35.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.616+0000 7f4742f41700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f473c0c9990 con 0x7f473c0c2b60 2023-10-06T04:04:35.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.616+0000 7f4742f41700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473c0c0620 con 0x7f473c06d550 2023-10-06T04:04:35.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.616+0000 7f474173e700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f473c0c2b60 0x7f473c153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:35.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.616+0000 7f474173e700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f473c0c2b60 0x7f473c153410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60486/0 (socket says 172.21.15.157:60486) 2023-10-06T04:04:35.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.616+0000 7f474173e700 1 -- 172.21.15.157:0/214102831 learned_addr learned my addr 172.21.15.157:0/214102831 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:35.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.616+0000 7f4741f3f700 1 --2- 172.21.15.157:0/214102831 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f473c06d550 0x7f473c0c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:35.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.616+0000 7f474173e700 1 -- 172.21.15.157:0/214102831 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f473c06d550 msgr2=0x7f473c0c90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:04:35.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.616+0000 7f474173e700 1 --2- 172.21.15.157:0/214102831 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f473c06d550 0x7f473c0c90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:35.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.616+0000 7f474173e700 1 -- 172.21.15.157:0/214102831 >> v1:172.21.15.175:6789/0 conn(0x7f473c06d920 legacy=0x7f473c0c1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:04:35.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.616+0000 7f474173e700 1 -- 172.21.15.157:0/214102831 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473c0c96b0 con 0x7f473c0c2b60 2023-10-06T04:04:35.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.617+0000 7f474173e700 1 --2- 172.21.15.157:0/214102831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f473c0c2b60 0x7f473c153410 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f472c0099e0 tx=0x7f472c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88db4899c5bfadd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:35.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.617+0000 7f4740f3d700 1 -- 172.21.15.157:0/214102831 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c00b870 con 0x7f473c0c2b60 2023-10-06T04:04:35.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.617+0000 7f4740f3d700 1 -- 172.21.15.157:0/214102831 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f472c00b9d0 con 0x7f473c0c2b60 2023-10-06T04:04:35.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.617+0000 7f4740f3d700 1 -- 172.21.15.157:0/214102831 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c00bd70 con 0x7f473c0c2b60 2023-10-06T04:04:35.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.617+0000 7f4742f41700 1 -- 172.21.15.157:0/214102831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f473c0c2b60 msgr2=0x7f473c153410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:35.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.617+0000 7f4742f41700 1 --2- 172.21.15.157:0/214102831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f473c0c2b60 0x7f473c153410 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f472c0099e0 tx=0x7f472c009250 comp rx=0 tx=0).stop 2023-10-06T04:04:35.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.617+0000 7f4742f41700 1 -- 172.21.15.157:0/214102831 shutdown_connections 2023-10-06T04:04:35.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.617+0000 7f4742f41700 1 --2- 172.21.15.157:0/214102831 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f473c06d550 0x7f473c0c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:35.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.617+0000 7f4742f41700 1 --2- 172.21.15.157:0/214102831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f473c0c2b60 0x7f473c153410 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:35.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.617+0000 7f4742f41700 1 -- 172.21.15.157:0/214102831 >> 172.21.15.157:0/214102831 conn(0x7f473c06f890 msgr2=0x7f473c0c89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:35.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.617+0000 7f4742f41700 1 -- 172.21.15.157:0/214102831 shutdown_connections 2023-10-06T04:04:35.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.618+0000 7f4742f41700 1 -- 172.21.15.157:0/214102831 wait complete. 2023-10-06T04:04:35.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.619+0000 7f4742f41700 1 Processor -- start 2023-10-06T04:04:35.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.619+0000 7f4742f41700 1 -- start start 2023-10-06T04:04:35.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.619+0000 7f4742f41700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f473c06d550 0x7f473c0c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:35.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.619+0000 7f4742f41700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f473c06d920 0x7f473c14dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:35.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.619+0000 7f4742f41700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f473c0c2b60 0x7f473c165be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:35.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.619+0000 7f4742f41700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c14f6c0 con 0x7f473c06d550 2023-10-06T04:04:35.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.619+0000 7f4742f41700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c164810 con 0x7f473c0c2b60 2023-10-06T04:04:35.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.619+0000 7f4742f41700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473c14d7e0 con 0x7f473c06d920 2023-10-06T04:04:35.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.619+0000 7f4741f3f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f473c06d550 0x7f473c0c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:35.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.619+0000 7f4741f3f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f473c06d550 0x7f473c0c5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60502/0 (socket says 172.21.15.157:60502) 2023-10-06T04:04:35.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.619+0000 7f4741f3f700 1 -- 172.21.15.157:0/3913997558 learned_addr learned my addr 172.21.15.157:0/3913997558 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:35.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.620+0000 7f4742740700 1 --2- 172.21.15.157:0/3913997558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f473c0c2b60 0x7f473c165be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:35.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.620+0000 7f474173e700 1 --2- 172.21.15.157:0/3913997558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f473c06d920 0x7f473c14dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:35.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.620+0000 7f4741f3f700 1 -- 172.21.15.157:0/3913997558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f473c06d920 msgr2=0x7f473c14dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:35.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.620+0000 7f4741f3f700 1 --2- 172.21.15.157:0/3913997558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f473c06d920 0x7f473c14dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:35.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.620+0000 7f4741f3f700 1 -- 172.21.15.157:0/3913997558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f473c0c2b60 msgr2=0x7f473c165be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:35.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.620+0000 7f4741f3f700 1 --2- 172.21.15.157:0/3913997558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f473c0c2b60 0x7f473c165be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:35.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.620+0000 7f4741f3f700 1 -- 172.21.15.157:0/3913997558 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f472c0094f0 con 0x7f473c06d550 2023-10-06T04:04:35.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.620+0000 7f474173e700 1 --2- 172.21.15.157:0/3913997558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f473c06d920 0x7f473c14dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:04:35.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.620+0000 7f4742740700 1 --2- 172.21.15.157:0/3913997558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f473c0c2b60 0x7f473c165be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:04:35.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.620+0000 7f4741f3f700 1 --2- 172.21.15.157:0/3913997558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f473c06d550 0x7f473c0c5ab0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f473800fc10 tx=0x7f4738008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:35.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.620+0000 7f4731ffb700 1 -- 172.21.15.157:0/3913997558 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4738007a60 con 0x7f473c06d550 2023-10-06T04:04:35.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.620+0000 7f4731ffb700 1 -- 172.21.15.157:0/3913997558 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4738007bc0 con 0x7f473c06d550 2023-10-06T04:04:35.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.620+0000 7f4742f41700 1 -- 172.21.15.157:0/3913997558 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f473c167740 con 0x7f473c06d550 2023-10-06T04:04:35.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.620+0000 7f4731ffb700 1 -- 172.21.15.157:0/3913997558 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473801a460 con 0x7f473c06d550 2023-10-06T04:04:35.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.620+0000 7f4742f41700 1 -- 172.21.15.157:0/3913997558 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f473c14f3e0 con 0x7f473c06d550 2023-10-06T04:04:35.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.621+0000 7f4742f41700 1 -- 172.21.15.157:0/3913997558 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f473c14dac0 con 0x7f473c06d550 2023-10-06T04:04:35.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.622+0000 7f4731ffb700 1 -- 172.21.15.157:0/3913997558 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f473801a7d0 con 0x7f473c06d550 2023-10-06T04:04:35.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.622+0000 7f4731ffb700 1 --2- 172.21.15.157:0/3913997558 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4720066f30 0x7f47200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:35.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.622+0000 7f4731ffb700 1 -- 172.21.15.157:0/3913997558 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6374+0+0 (secure 0 0 0) 0x7f47380870e0 con 0x7f473c06d550 2023-10-06T04:04:35.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.622+0000 7f474173e700 1 --2- 172.21.15.157:0/3913997558 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4720066f30 0x7f47200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:35.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.623+0000 7f474173e700 1 --2- 172.21.15.157:0/3913997558 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4720066f30 0x7f47200693e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f472c0099e0 tx=0x7f472c016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:35.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.626+0000 7f4731ffb700 1 -- 172.21.15.157:0/3913997558 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4738057b40 con 0x7f473c06d550 2023-10-06T04:04:35.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.854+0000 7f4742f41700 1 -- 172.21.15.157:0/3913997558 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f473c0c46f0 con 0x7f473c06d550 2023-10-06T04:04:35.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.854+0000 7f4731ffb700 1 -- 172.21.15.157:0/3913997558 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+5389 (secure 0 0 0) 0x7f47380576d0 con 0x7f473c06d550 2023-10-06T04:04:35.857 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:04:35.857 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:04:32.766257+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:resolve","state_seq":16,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:04:35.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.856+0000 7f4742f41700 1 -- 172.21.15.157:0/3913997558 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4720066f30 msgr2=0x7f47200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:35.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.856+0000 7f4742f41700 1 --2- 172.21.15.157:0/3913997558 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4720066f30 0x7f47200693e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f472c0099e0 tx=0x7f472c016000 comp rx=0 tx=0).stop 2023-10-06T04:04:35.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.856+0000 7f4742f41700 1 -- 172.21.15.157:0/3913997558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f473c06d550 msgr2=0x7f473c0c5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:35.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.856+0000 7f4742f41700 1 --2- 172.21.15.157:0/3913997558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f473c06d550 0x7f473c0c5ab0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f473800fc10 tx=0x7f4738008040 comp rx=0 tx=0).stop 2023-10-06T04:04:35.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.857+0000 7f4742f41700 1 -- 172.21.15.157:0/3913997558 shutdown_connections 2023-10-06T04:04:35.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.857+0000 7f4742f41700 1 --2- 172.21.15.157:0/3913997558 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4720066f30 0x7f47200693e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:35.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.857+0000 7f4742f41700 1 --2- 172.21.15.157:0/3913997558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f473c0c2b60 0x7f473c165be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:35.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.857+0000 7f4742f41700 1 --2- 172.21.15.157:0/3913997558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f473c06d550 0x7f473c0c5ab0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:35.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.857+0000 7f4742f41700 1 --2- 172.21.15.157:0/3913997558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f473c06d920 0x7f473c14dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:35.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.857+0000 7f4742f41700 1 -- 172.21.15.157:0/3913997558 >> 172.21.15.157:0/3913997558 conn(0x7f473c06f890 msgr2=0x7f473c1496c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:35.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.857+0000 7f4742f41700 1 -- 172.21.15.157:0/3913997558 shutdown_connections 2023-10-06T04:04:35.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:35.857+0000 7f4742f41700 1 -- 172.21.15.157:0/3913997558 wait complete. 2023-10-06T04:04:35.861 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 59 2023-10-06T04:04:35.871 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:04:37.872 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:04:37.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.990+0000 7f5a51f19700 1 Processor -- start 2023-10-06T04:04:37.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.990+0000 7f5a51f19700 1 -- start start 2023-10-06T04:04:37.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.991+0000 7f5a51f19700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5a4c06c2d0 0x7f5a4c157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:37.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.991+0000 7f5a51f19700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a4c06d8d0 con 0x7f5a4c0b1e40 2023-10-06T04:04:37.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.991+0000 7f5a51f19700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a4c0af8f0 con 0x7f5a4c0b04d0 2023-10-06T04:04:37.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.991+0000 7f5a51f19700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a4c15a690 con 0x7f5a4c06c2d0 2023-10-06T04:04:37.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.991+0000 7f5a51718700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5a4c06c2d0 0x7f5a4c157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:37.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.991+0000 7f5a51718700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5a4c06c2d0 0x7f5a4c157750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60516/0 (socket says 172.21.15.157:60516) 2023-10-06T04:04:37.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.991+0000 7f5a51718700 1 -- 172.21.15.157:0/1277756347 learned_addr learned my addr 172.21.15.157:0/1277756347 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:37.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.991+0000 7f5a50f17700 1 --1- 172.21.15.157:0/1277756347 >> v1:172.21.15.157:6789/0 conn(0x7f5a4c0b1e40 0x7f5a4c1563b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:52056/0 (socket says 172.21.15.157:52056) 2023-10-06T04:04:37.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.991+0000 7f5a51718700 1 -- 172.21.15.157:0/1277756347 >> v1:172.21.15.175:6789/0 conn(0x7f5a4c0b04d0 legacy=0x7f5a4c156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:37.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.991+0000 7f5a51718700 1 -- 172.21.15.157:0/1277756347 >> v1:172.21.15.157:6789/0 conn(0x7f5a4c0b1e40 legacy=0x7f5a4c1563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:37.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.991+0000 7f5a51718700 1 -- 172.21.15.157:0/1277756347 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a4c1571d0 con 0x7f5a4c06c2d0 2023-10-06T04:04:37.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.991+0000 7f5a51718700 1 --2- 172.21.15.157:0/1277756347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a4c06c2d0 0x7f5a4c157750 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009f20 tx=0x7f5a44009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bdc7298fc61710 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:37.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.991+0000 7f5a4b7fe700 1 -- 172.21.15.157:0/1277756347 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4400e070 con 0x7f5a4c06c2d0 2023-10-06T04:04:37.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.992+0000 7f5a4b7fe700 1 -- 172.21.15.157:0/1277756347 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4400ba80 con 0x7f5a4c06c2d0 2023-10-06T04:04:37.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.992+0000 7f5a4b7fe700 1 -- 172.21.15.157:0/1277756347 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a440092e0 con 0x7f5a4c06c2d0 2023-10-06T04:04:37.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.992+0000 7f5a51f19700 1 -- 172.21.15.157:0/1277756347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a4c06c2d0 msgr2=0x7f5a4c157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:37.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.992+0000 7f5a51f19700 1 --2- 172.21.15.157:0/1277756347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a4c06c2d0 0x7f5a4c157750 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009f20 tx=0x7f5a44009840 comp rx=0 tx=0).stop 2023-10-06T04:04:37.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.992+0000 7f5a51f19700 1 -- 172.21.15.157:0/1277756347 shutdown_connections 2023-10-06T04:04:37.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.992+0000 7f5a51f19700 1 --2- 172.21.15.157:0/1277756347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a4c06c2d0 0x7f5a4c157750 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:37.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.992+0000 7f5a51f19700 1 -- 172.21.15.157:0/1277756347 >> 172.21.15.157:0/1277756347 conn(0x7f5a4c0cc540 msgr2=0x7f5a4c0cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:37.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.992+0000 7f5a51f19700 1 -- 172.21.15.157:0/1277756347 shutdown_connections 2023-10-06T04:04:37.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.992+0000 7f5a51f19700 1 -- 172.21.15.157:0/1277756347 wait complete. 2023-10-06T04:04:38.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.993+0000 7f5a51f19700 1 Processor -- start 2023-10-06T04:04:38.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.993+0000 7f5a51f19700 1 -- start start 2023-10-06T04:04:38.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.994+0000 7f5a51f19700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a4c06c2d0 0x7f5a4c169b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:38.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.994+0000 7f5a51f19700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a4c0b04d0 0x7f5a4c156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:38.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.994+0000 7f5a51f19700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a4c0b1e40 0x7f5a4c14f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:38.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.994+0000 7f5a51f19700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c1693a0 con 0x7f5a4c06c2d0 2023-10-06T04:04:38.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.994+0000 7f5a51f19700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c0c6c70 con 0x7f5a4c0b1e40 2023-10-06T04:04:38.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.994+0000 7f5a51f19700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c0c6f50 con 0x7f5a4c0b04d0 2023-10-06T04:04:38.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.994+0000 7f5a50f17700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a4c06c2d0 0x7f5a4c169b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:38.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.994+0000 7f5a50f17700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a4c06c2d0 0x7f5a4c169b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60530/0 (socket says 172.21.15.157:60530) 2023-10-06T04:04:38.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.994+0000 7f5a50f17700 1 -- 172.21.15.157:0/1563479225 learned_addr learned my addr 172.21.15.157:0/1563479225 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:38.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.994+0000 7f5a51718700 1 --2- 172.21.15.157:0/1563479225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a4c0b1e40 0x7f5a4c14f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:38.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.994+0000 7f5a4bfff700 1 --2- 172.21.15.157:0/1563479225 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a4c0b04d0 0x7f5a4c156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:38.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.994+0000 7f5a50f17700 1 -- 172.21.15.157:0/1563479225 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a4c0b04d0 msgr2=0x7f5a4c156530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:38.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.994+0000 7f5a50f17700 1 --2- 172.21.15.157:0/1563479225 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a4c0b04d0 0x7f5a4c156530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:38.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.994+0000 7f5a50f17700 1 -- 172.21.15.157:0/1563479225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a4c0b1e40 msgr2=0x7f5a4c14f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:38.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.994+0000 7f5a50f17700 1 --2- 172.21.15.157:0/1563479225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a4c0b1e40 0x7f5a4c14f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:38.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.994+0000 7f5a50f17700 1 -- 172.21.15.157:0/1563479225 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a44009ae0 con 0x7f5a4c06c2d0 2023-10-06T04:04:38.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.995+0000 7f5a50f17700 1 --2- 172.21.15.157:0/1563479225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a4c06c2d0 0x7f5a4c169b30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000c560 tx=0x7f5a40007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:38.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.995+0000 7f5a48ff9700 1 -- 172.21.15.157:0/1563479225 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a40005a70 con 0x7f5a4c06c2d0 2023-10-06T04:04:38.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.995+0000 7f5a48ff9700 1 -- 172.21.15.157:0/1563479225 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a40005bd0 con 0x7f5a4c06c2d0 2023-10-06T04:04:38.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.995+0000 7f5a48ff9700 1 -- 172.21.15.157:0/1563479225 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a400070d0 con 0x7f5a4c06c2d0 2023-10-06T04:04:38.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.995+0000 7f5a51f19700 1 -- 172.21.15.157:0/1563479225 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a4c0c7230 con 0x7f5a4c06c2d0 2023-10-06T04:04:38.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.995+0000 7f5a51f19700 1 -- 172.21.15.157:0/1563479225 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a4c0af0a0 con 0x7f5a4c06c2d0 2023-10-06T04:04:38.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.996+0000 7f5a51f19700 1 -- 172.21.15.157:0/1563479225 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a4c169850 con 0x7f5a4c06c2d0 2023-10-06T04:04:38.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.996+0000 7f5a48ff9700 1 -- 172.21.15.157:0/1563479225 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5a40007480 con 0x7f5a4c06c2d0 2023-10-06T04:04:38.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.997+0000 7f5a48ff9700 1 --2- 172.21.15.157:0/1563479225 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5a30066f30 0x7f5a300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:38.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.997+0000 7f5a48ff9700 1 -- 172.21.15.157:0/1563479225 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6374+0+0 (secure 0 0 0) 0x7f5a40083bc0 con 0x7f5a4c06c2d0 2023-10-06T04:04:38.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.997+0000 7f5a4bfff700 1 --2- 172.21.15.157:0/1563479225 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5a30066f30 0x7f5a300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:38.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:37.997+0000 7f5a4bfff700 1 --2- 172.21.15.157:0/1563479225 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5a30066f30 0x7f5a300693e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c0af7f0 tx=0x7f5a38007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:38.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:38.001+0000 7f5a48ff9700 1 -- 172.21.15.157:0/1563479225 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5a4c169850 con 0x7f5a4c06c2d0 2023-10-06T04:04:38.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:38.226+0000 7f5a51f19700 1 -- 172.21.15.157:0/1563479225 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a4c169850 con 0x7f5a4c06c2d0 2023-10-06T04:04:38.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:38.227+0000 7f5a48ff9700 1 -- 172.21.15.157:0/1563479225 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+5391 (secure 0 0 0) 0x7f5a40017020 con 0x7f5a4c06c2d0 2023-10-06T04:04:38.230 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:04:38.230 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:04:37.779094+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:reconnect","state_seq":18,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:04:38.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:38.229+0000 7f5a51f19700 1 -- 172.21.15.157:0/1563479225 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5a30066f30 msgr2=0x7f5a300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:38.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:38.229+0000 7f5a51f19700 1 --2- 172.21.15.157:0/1563479225 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5a30066f30 0x7f5a300693e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c0af7f0 tx=0x7f5a38007ec0 comp rx=0 tx=0).stop 2023-10-06T04:04:38.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:38.229+0000 7f5a51f19700 1 -- 172.21.15.157:0/1563479225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a4c06c2d0 msgr2=0x7f5a4c169b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:38.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:38.229+0000 7f5a51f19700 1 --2- 172.21.15.157:0/1563479225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a4c06c2d0 0x7f5a4c169b30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000c560 tx=0x7f5a40007f70 comp rx=0 tx=0).stop 2023-10-06T04:04:38.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:38.229+0000 7f5a51f19700 1 -- 172.21.15.157:0/1563479225 shutdown_connections 2023-10-06T04:04:38.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:38.229+0000 7f5a51f19700 1 --2- 172.21.15.157:0/1563479225 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5a30066f30 0x7f5a300693e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:38.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:38.229+0000 7f5a51f19700 1 --2- 172.21.15.157:0/1563479225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a4c0b1e40 0x7f5a4c14f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:38.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:38.229+0000 7f5a51f19700 1 --2- 172.21.15.157:0/1563479225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a4c06c2d0 0x7f5a4c169b30 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:38.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:38.229+0000 7f5a51f19700 1 --2- 172.21.15.157:0/1563479225 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a4c0b04d0 0x7f5a4c156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:38.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:38.229+0000 7f5a51f19700 1 -- 172.21.15.157:0/1563479225 >> 172.21.15.157:0/1563479225 conn(0x7f5a4c0cc540 msgr2=0x7f5a4c157420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:38.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:38.229+0000 7f5a51f19700 1 -- 172.21.15.157:0/1563479225 shutdown_connections 2023-10-06T04:04:38.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:38.229+0000 7f5a51f19700 1 -- 172.21.15.157:0/1563479225 wait complete. 2023-10-06T04:04:38.233 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 60 2023-10-06T04:04:38.244 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:04:40.245 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:04:40.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.370+0000 7f0543426700 1 Processor -- start 2023-10-06T04:04:40.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.370+0000 7f0543426700 1 -- start start 2023-10-06T04:04:40.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.371+0000 7f0543426700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f053c06c2d0 0x7f053c157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:40.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.371+0000 7f0543426700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f053c06d8d0 con 0x7f053c0b1e40 2023-10-06T04:04:40.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.371+0000 7f0543426700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f053c0af8f0 con 0x7f053c0b04d0 2023-10-06T04:04:40.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.371+0000 7f0543426700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f053c15a690 con 0x7f053c06c2d0 2023-10-06T04:04:40.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.371+0000 7f0542424700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f053c0b1e40 0x7f053c1563b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:52070/0 (socket says 172.21.15.157:52070) 2023-10-06T04:04:40.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.371+0000 7f0542424700 1 -- 172.21.15.157:0/1460964276 learned_addr learned my addr 172.21.15.157:0/1460964276 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:04:40.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.371+0000 7f0542c25700 1 --2- 172.21.15.157:0/1460964276 >> v2:172.21.15.175:3300/0 conn(0x7f053c06c2d0 0x7f053c157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:40.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.371+0000 7f0541422700 1 -- 172.21.15.157:0/1460964276 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3952572374 0 0) 0x7f053c06d8d0 con 0x7f053c0b1e40 2023-10-06T04:04:40.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.371+0000 7f0541422700 1 -- 172.21.15.157:0/1460964276 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0528003570 con 0x7f053c0b1e40 2023-10-06T04:04:40.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.372+0000 7f0541422700 1 -- 172.21.15.157:0/1460964276 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 463015893 0 0) 0x7f053c0af8f0 con 0x7f053c0b04d0 2023-10-06T04:04:40.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.372+0000 7f0541422700 1 -- 172.21.15.157:0/1460964276 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f053c06d8d0 con 0x7f053c0b04d0 2023-10-06T04:04:40.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.372+0000 7f0541422700 1 -- 172.21.15.157:0/1460964276 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1679422080 0 0) 0x7f0528003570 con 0x7f053c0b1e40 2023-10-06T04:04:40.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.372+0000 7f0542c25700 1 -- 172.21.15.157:0/1460964276 >> v1:172.21.15.175:6789/0 conn(0x7f053c0b04d0 legacy=0x7f053c156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:40.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.372+0000 7f0542c25700 1 -- 172.21.15.157:0/1460964276 >> v1:172.21.15.157:6789/0 conn(0x7f053c0b1e40 legacy=0x7f053c1563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:40.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.372+0000 7f0542c25700 1 -- 172.21.15.157:0/1460964276 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f053c1571d0 con 0x7f053c06c2d0 2023-10-06T04:04:40.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.372+0000 7f0542c25700 1 --2- 172.21.15.157:0/1460964276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f053c06c2d0 0x7f053c157750 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0534009e60 tx=0x7f0534009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c432aa1223afb8f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:40.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.372+0000 7f0541422700 1 -- 172.21.15.157:0/1460964276 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053400e070 con 0x7f053c06c2d0 2023-10-06T04:04:40.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.372+0000 7f0541422700 1 -- 172.21.15.157:0/1460964276 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053400b7a0 con 0x7f053c06c2d0 2023-10-06T04:04:40.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.372+0000 7f0541422700 1 -- 172.21.15.157:0/1460964276 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0538001470 con 0x7f053c06c2d0 2023-10-06T04:04:40.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.373+0000 7f0543426700 1 -- 172.21.15.157:0/1460964276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f053c06c2d0 msgr2=0x7f053c157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:40.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.373+0000 7f0543426700 1 --2- 172.21.15.157:0/1460964276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f053c06c2d0 0x7f053c157750 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0534009e60 tx=0x7f0534009780 comp rx=0 tx=0).stop 2023-10-06T04:04:40.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.373+0000 7f0543426700 1 -- 172.21.15.157:0/1460964276 shutdown_connections 2023-10-06T04:04:40.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.373+0000 7f0543426700 1 --2- 172.21.15.157:0/1460964276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f053c06c2d0 0x7f053c157750 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:40.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.373+0000 7f0543426700 1 -- 172.21.15.157:0/1460964276 >> 172.21.15.157:0/1460964276 conn(0x7f053c0cc540 msgr2=0x7f053c0cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:40.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.373+0000 7f0543426700 1 -- 172.21.15.157:0/1460964276 shutdown_connections 2023-10-06T04:04:40.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.373+0000 7f0543426700 1 -- 172.21.15.157:0/1460964276 wait complete. 2023-10-06T04:04:40.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.374+0000 7f0543426700 1 Processor -- start 2023-10-06T04:04:40.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.374+0000 7f0543426700 1 -- start start 2023-10-06T04:04:40.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.375+0000 7f0543426700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f053c06c2d0 0x7f053c0c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:40.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.375+0000 7f0543426700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f053c0b04d0 0x7f053c154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:40.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.375+0000 7f0543426700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f053c0b1e40 0x7f053c14f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:40.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.375+0000 7f0543426700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053c151e30 con 0x7f053c0b1e40 2023-10-06T04:04:40.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.375+0000 7f0543426700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053c153680 con 0x7f053c06c2d0 2023-10-06T04:04:40.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.375+0000 7f0543426700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f053c151b90 con 0x7f053c0b04d0 2023-10-06T04:04:40.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.375+0000 7f0542c25700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f053c0b1e40 0x7f053c14f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:40.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.375+0000 7f0542c25700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f053c0b1e40 0x7f053c14f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60546/0 (socket says 172.21.15.157:60546) 2023-10-06T04:04:40.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.375+0000 7f0542c25700 1 -- 172.21.15.157:0/318544021 learned_addr learned my addr 172.21.15.157:0/318544021 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:40.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.375+0000 7f0542424700 1 --2- 172.21.15.157:0/318544021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f053c06c2d0 0x7f053c0c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:40.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.375+0000 7f0541c23700 1 --2- 172.21.15.157:0/318544021 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f053c0b04d0 0x7f053c154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:40.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.375+0000 7f0542c25700 1 -- 172.21.15.157:0/318544021 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f053c0b04d0 msgr2=0x7f053c154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:40.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.375+0000 7f0542c25700 1 --2- 172.21.15.157:0/318544021 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f053c0b04d0 0x7f053c154fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:40.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.375+0000 7f0542c25700 1 -- 172.21.15.157:0/318544021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f053c06c2d0 msgr2=0x7f053c0c6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:40.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.375+0000 7f0542c25700 1 --2- 172.21.15.157:0/318544021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f053c06c2d0 0x7f053c0c6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:40.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.375+0000 7f0542c25700 1 -- 172.21.15.157:0/318544021 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0534009a20 con 0x7f053c0b1e40 2023-10-06T04:04:40.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.375+0000 7f0542c25700 1 --2- 172.21.15.157:0/318544021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f053c0b1e40 0x7f053c14f9d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f053400b610 tx=0x7f0534017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:40.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.375+0000 7f05327fc700 1 -- 172.21.15.157:0/318544021 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0534012c30 con 0x7f053c0b1e40 2023-10-06T04:04:40.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.375+0000 7f05327fc700 1 -- 172.21.15.157:0/318544021 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0534012d90 con 0x7f053c0b1e40 2023-10-06T04:04:40.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.375+0000 7f0543426700 1 -- 172.21.15.157:0/318544021 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f053c0ccd20 con 0x7f053c0b1e40 2023-10-06T04:04:40.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.375+0000 7f05327fc700 1 -- 172.21.15.157:0/318544021 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053401e680 con 0x7f053c0b1e40 2023-10-06T04:04:40.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.375+0000 7f0543426700 1 -- 172.21.15.157:0/318544021 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f053c150d70 con 0x7f053c0b1e40 2023-10-06T04:04:40.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.376+0000 7f0543426700 1 -- 172.21.15.157:0/318544021 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f053c0c6b60 con 0x7f053c0b1e40 2023-10-06T04:04:40.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.377+0000 7f05327fc700 1 -- 172.21.15.157:0/318544021 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f053401e9f0 con 0x7f053c0b1e40 2023-10-06T04:04:40.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.377+0000 7f05327fc700 1 --2- 172.21.15.157:0/318544021 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0520066f30 0x7f05200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:40.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.377+0000 7f05327fc700 1 -- 172.21.15.157:0/318544021 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6374+0+0 (secure 0 0 0) 0x7f05340869e0 con 0x7f053c0b1e40 2023-10-06T04:04:40.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.377+0000 7f0542424700 1 --2- 172.21.15.157:0/318544021 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0520066f30 0x7f05200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:40.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.378+0000 7f0542424700 1 --2- 172.21.15.157:0/318544021 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0520066f30 0x7f05200693e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f05380015a0 tx=0x7f0538000bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:40.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.381+0000 7f05327fc700 1 -- 172.21.15.157:0/318544021 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f053c0c6b60 con 0x7f053c0b1e40 2023-10-06T04:04:40.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.607+0000 7f0543426700 1 -- 172.21.15.157:0/318544021 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f053c0c6b60 con 0x7f053c0b1e40 2023-10-06T04:04:40.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.608+0000 7f05327fc700 1 -- 172.21.15.157:0/318544021 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+5391 (secure 0 0 0) 0x7f053c0c6b60 con 0x7f053c0b1e40 2023-10-06T04:04:40.610 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:04:40.610 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:04:37.779094+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:reconnect","state_seq":18,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:04:40.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.610+0000 7f0543426700 1 -- 172.21.15.157:0/318544021 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0520066f30 msgr2=0x7f05200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:40.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.610+0000 7f0543426700 1 --2- 172.21.15.157:0/318544021 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0520066f30 0x7f05200693e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f05380015a0 tx=0x7f0538000bc0 comp rx=0 tx=0).stop 2023-10-06T04:04:40.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.610+0000 7f0543426700 1 -- 172.21.15.157:0/318544021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f053c0b1e40 msgr2=0x7f053c14f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:40.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.610+0000 7f0543426700 1 --2- 172.21.15.157:0/318544021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f053c0b1e40 0x7f053c14f9d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f053400b610 tx=0x7f0534017000 comp rx=0 tx=0).stop 2023-10-06T04:04:40.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.610+0000 7f0543426700 1 -- 172.21.15.157:0/318544021 shutdown_connections 2023-10-06T04:04:40.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.610+0000 7f0543426700 1 --2- 172.21.15.157:0/318544021 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0520066f30 0x7f05200693e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:40.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.610+0000 7f0543426700 1 --2- 172.21.15.157:0/318544021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f053c06c2d0 0x7f053c0c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:40.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.610+0000 7f0543426700 1 --2- 172.21.15.157:0/318544021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f053c0b1e40 0x7f053c14f9d0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:40.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.610+0000 7f0543426700 1 --2- 172.21.15.157:0/318544021 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f053c0b04d0 0x7f053c154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:40.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.610+0000 7f0543426700 1 -- 172.21.15.157:0/318544021 >> 172.21.15.157:0/318544021 conn(0x7f053c0cc540 msgr2=0x7f053c157440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:40.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.610+0000 7f0543426700 1 -- 172.21.15.157:0/318544021 shutdown_connections 2023-10-06T04:04:40.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:40.610+0000 7f0543426700 1 -- 172.21.15.157:0/318544021 wait complete. 2023-10-06T04:04:40.614 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 60 2023-10-06T04:04:40.625 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 60, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-06T03:57:10.718782+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 60, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_4296": { "addr": "172.21.15.157:6837/4188652695", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6836", "nonce": 4188652695, "type": "v2" }, { "addr": "172.21.15.157:6837", "nonce": 4188652695, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1 ], "features": 4540138320759226367, "flags": 0, "gid": 4296, "incarnation": 4, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 4 }, "gid_5362": { "addr": "172.21.15.175:6837/2388164729", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6836", "nonce": 2388164729, "type": "v2" }, { "addr": "172.21.15.175:6837", "nonce": 2388164729, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5362, "incarnation": 58, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:reconnect", "state_seq": 18 } }, "last_failure": 0, "last_failure_osd_epoch": 44, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2023-10-06T04:04:37.779094+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2, 3, 4 ], "tableserver": 0, "up": { "mds_0": 4296, "mds_1": 5362 } } } ], "standbys": [ { "addr": "172.21.15.175:6835/1799968248", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6834", "nonce": 1799968248, "type": "v2" }, { "addr": "172.21.15.175:6835", "nonce": 1799968248, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 57, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5126, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.175:6839/745804271", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6838", "nonce": 745804271, "type": "v2" }, { "addr": "172.21.15.175:6839", "nonce": 745804271, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 42, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5164, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.157:6835/2461751318", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6833", "nonce": 2461751318, "type": "v2" }, { "addr": "172.21.15.157:6835", "nonce": 2461751318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 34, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5268, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-06T04:04:42.626 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:04:42.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.742+0000 7f73ffdbc700 1 Processor -- start 2023-10-06T04:04:42.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.743+0000 7f73ffdbc700 1 -- start start 2023-10-06T04:04:42.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.743+0000 7f73ffdbc700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f73f814c370 0x7f73f806dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:42.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.743+0000 7f73ffdbc700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f73f814c740 0x7f73f8155780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:42.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.743+0000 7f73ffdbc700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73f80c9650 0x7f73f8155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:42.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.743+0000 7f73ffdbc700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73f806e2b0 con 0x7f73f814c370 2023-10-06T04:04:42.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.743+0000 7f73ffdbc700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73f806d8e0 con 0x7f73f814c740 2023-10-06T04:04:42.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.743+0000 7f73ffdbc700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f806c5e0 con 0x7f73f80c9650 2023-10-06T04:04:42.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.744+0000 7f73fdb58700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f73f814c370 0x7f73f806dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:42.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.744+0000 7f73fdb58700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f73f814c370 0x7f73f806dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60548/0 (socket says 172.21.15.157:60548) 2023-10-06T04:04:42.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.744+0000 7f73fdb58700 1 -- 172.21.15.157:0/4107194324 learned_addr learned my addr 172.21.15.157:0/4107194324 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:42.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.744+0000 7f73fe359700 1 --2- 172.21.15.157:0/4107194324 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73f80c9650 0x7f73f8155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:42.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.744+0000 7f73fd357700 1 --2- 172.21.15.157:0/4107194324 >> v2:172.21.15.175:3300/0 conn(0x7f73f814c740 0x7f73f8155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:42.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.744+0000 7f73fdb58700 1 -- 172.21.15.157:0/4107194324 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73f80c9650 msgr2=0x7f73f8155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:04:42.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.744+0000 7f73fdb58700 1 --2- 172.21.15.157:0/4107194324 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73f80c9650 0x7f73f8155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:42.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.744+0000 7f73fdb58700 1 -- 172.21.15.157:0/4107194324 >> v2:172.21.15.175:3300/0 conn(0x7f73f814c740 msgr2=0x7f73f8155780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:04:42.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.744+0000 7f73fdb58700 1 --2- 172.21.15.157:0/4107194324 >> v2:172.21.15.175:3300/0 conn(0x7f73f814c740 0x7f73f8155780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:42.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.744+0000 7f73fdb58700 1 -- 172.21.15.157:0/4107194324 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f806ab30 con 0x7f73f814c370 2023-10-06T04:04:42.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.744+0000 7f73fdb58700 1 --2- 172.21.15.157:0/4107194324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73f814c370 0x7f73f806dd70 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f73f806e740 tx=0x7f73f4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87138cbd8cadfd60 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:42.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.744+0000 7f73fcb56700 1 -- 172.21.15.157:0/4107194324 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f400e070 con 0x7f73f814c370 2023-10-06T04:04:42.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.744+0000 7f73fcb56700 1 -- 172.21.15.157:0/4107194324 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f400bae0 con 0x7f73f814c370 2023-10-06T04:04:42.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.744+0000 7f73fcb56700 1 -- 172.21.15.157:0/4107194324 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f40092e0 con 0x7f73f814c370 2023-10-06T04:04:42.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.745+0000 7f73ffdbc700 1 -- 172.21.15.157:0/4107194324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73f814c370 msgr2=0x7f73f806dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:42.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.745+0000 7f73ffdbc700 1 --2- 172.21.15.157:0/4107194324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73f814c370 0x7f73f806dd70 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f73f806e740 tx=0x7f73f4009840 comp rx=0 tx=0).stop 2023-10-06T04:04:42.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.745+0000 7f73ffdbc700 1 -- 172.21.15.157:0/4107194324 shutdown_connections 2023-10-06T04:04:42.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.745+0000 7f73ffdbc700 1 --2- 172.21.15.157:0/4107194324 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73f80c9650 0x7f73f8155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:42.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.745+0000 7f73ffdbc700 1 --2- 172.21.15.157:0/4107194324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73f814c370 0x7f73f806dd70 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:42.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.745+0000 7f73ffdbc700 1 --2- 172.21.15.157:0/4107194324 >> v2:172.21.15.175:3300/0 conn(0x7f73f814c740 0x7f73f8155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:42.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.745+0000 7f73ffdbc700 1 -- 172.21.15.157:0/4107194324 >> 172.21.15.157:0/4107194324 conn(0x7f73f806f9b0 msgr2=0x7f73f80c8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:42.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.745+0000 7f73ffdbc700 1 -- 172.21.15.157:0/4107194324 shutdown_connections 2023-10-06T04:04:42.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.745+0000 7f73ffdbc700 1 -- 172.21.15.157:0/4107194324 wait complete. 2023-10-06T04:04:42.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.746+0000 7f73ffdbc700 1 Processor -- start 2023-10-06T04:04:42.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.746+0000 7f73ffdbc700 1 -- start start 2023-10-06T04:04:42.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.747+0000 7f73ffdbc700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73f80c9650 0x7f73f8166a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:42.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.747+0000 7f73ffdbc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73f814c370 0x7f73f80beb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:42.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.747+0000 7f73ffdbc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73f814c740 0x7f73f814b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:42.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.747+0000 7f73ffdbc700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f8166fb0 con 0x7f73f814c740 2023-10-06T04:04:42.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.747+0000 7f73ffdbc700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f814aff0 con 0x7f73f814c370 2023-10-06T04:04:42.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.747+0000 7f73ffdbc700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f8149580 con 0x7f73f80c9650 2023-10-06T04:04:42.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.747+0000 7f73fe359700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73f814c740 0x7f73f814b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:42.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.747+0000 7f73fe359700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73f814c740 0x7f73f814b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60552/0 (socket says 172.21.15.157:60552) 2023-10-06T04:04:42.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.747+0000 7f73fe359700 1 -- 172.21.15.157:0/1986759924 learned_addr learned my addr 172.21.15.157:0/1986759924 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:42.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.747+0000 7f73fdb58700 1 --2- 172.21.15.157:0/1986759924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73f80c9650 0x7f73f8166a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:42.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.747+0000 7f73fd357700 1 --2- 172.21.15.157:0/1986759924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73f814c370 0x7f73f80beb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:42.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.747+0000 7f73fe359700 1 -- 172.21.15.157:0/1986759924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73f80c9650 msgr2=0x7f73f8166a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:42.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.747+0000 7f73fe359700 1 --2- 172.21.15.157:0/1986759924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73f80c9650 0x7f73f8166a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:42.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.747+0000 7f73fe359700 1 -- 172.21.15.157:0/1986759924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73f814c370 msgr2=0x7f73f80beb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:42.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.747+0000 7f73fe359700 1 --2- 172.21.15.157:0/1986759924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73f814c370 0x7f73f80beb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:42.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.747+0000 7f73fe359700 1 -- 172.21.15.157:0/1986759924 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f4009ae0 con 0x7f73f814c740 2023-10-06T04:04:42.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.747+0000 7f73fe359700 1 --2- 172.21.15.157:0/1986759924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73f814c740 0x7f73f814b480 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f73f000d8c0 tx=0x7f73f0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:42.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.747+0000 7f73edffb700 1 -- 172.21.15.157:0/1986759924 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f00125f0 con 0x7f73f814c740 2023-10-06T04:04:42.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.747+0000 7f73edffb700 1 -- 172.21.15.157:0/1986759924 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f0012770 con 0x7f73f814c740 2023-10-06T04:04:42.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.747+0000 7f73edffb700 1 -- 172.21.15.157:0/1986759924 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f0012b30 con 0x7f73f814c740 2023-10-06T04:04:42.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.747+0000 7f73ffdbc700 1 -- 172.21.15.157:0/1986759924 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73f8148a40 con 0x7f73f814c740 2023-10-06T04:04:42.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.748+0000 7f73ffdbc700 1 -- 172.21.15.157:0/1986759924 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73f8148fc0 con 0x7f73f814c740 2023-10-06T04:04:42.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.748+0000 7f73ffdbc700 1 -- 172.21.15.157:0/1986759924 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73f814b1a0 con 0x7f73f814c740 2023-10-06T04:04:42.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.749+0000 7f73edffb700 1 -- 172.21.15.157:0/1986759924 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f73f001e460 con 0x7f73f814c740 2023-10-06T04:04:42.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.749+0000 7f73edffb700 1 --2- 172.21.15.157:0/1986759924 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f73dc066f30 0x7f73dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:42.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.749+0000 7f73edffb700 1 -- 172.21.15.157:0/1986759924 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6374+0+0 (secure 0 0 0) 0x7f73f0084d00 con 0x7f73f814c740 2023-10-06T04:04:42.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.749+0000 7f73fdb58700 1 --2- 172.21.15.157:0/1986759924 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f73dc066f30 0x7f73dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:42.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.750+0000 7f73fdb58700 1 --2- 172.21.15.157:0/1986759924 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f73dc066f30 0x7f73dc0693e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f73f4009880 tx=0x7f73f4018000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:42.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.754+0000 7f73edffb700 1 -- 172.21.15.157:0/1986759924 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f73f0051cd0 con 0x7f73f814c740 2023-10-06T04:04:42.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.982+0000 7f73ffdbc700 1 -- 172.21.15.157:0/1986759924 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73f806ae10 con 0x7f73f814c740 2023-10-06T04:04:42.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.982+0000 7f73edffb700 1 -- 172.21.15.157:0/1986759924 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+5388 (secure 0 0 0) 0x7f73f00552f0 con 0x7f73f814c740 2023-10-06T04:04:42.985 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:04:42.985 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:04:42.790463+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:rejoin","state_seq":20,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:04:42.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.984+0000 7f73ffdbc700 1 -- 172.21.15.157:0/1986759924 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f73dc066f30 msgr2=0x7f73dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:42.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.984+0000 7f73ffdbc700 1 --2- 172.21.15.157:0/1986759924 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f73dc066f30 0x7f73dc0693e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f73f4009880 tx=0x7f73f4018000 comp rx=0 tx=0).stop 2023-10-06T04:04:42.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.984+0000 7f73ffdbc700 1 -- 172.21.15.157:0/1986759924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73f814c740 msgr2=0x7f73f814b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:42.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.984+0000 7f73ffdbc700 1 --2- 172.21.15.157:0/1986759924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73f814c740 0x7f73f814b480 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f73f000d8c0 tx=0x7f73f0008040 comp rx=0 tx=0).stop 2023-10-06T04:04:42.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.984+0000 7f73ffdbc700 1 -- 172.21.15.157:0/1986759924 shutdown_connections 2023-10-06T04:04:42.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.984+0000 7f73ffdbc700 1 --2- 172.21.15.157:0/1986759924 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f73dc066f30 0x7f73dc0693e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:42.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.984+0000 7f73ffdbc700 1 --2- 172.21.15.157:0/1986759924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73f814c740 0x7f73f814b480 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:42.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.984+0000 7f73ffdbc700 1 --2- 172.21.15.157:0/1986759924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73f80c9650 0x7f73f8166a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:42.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.984+0000 7f73ffdbc700 1 --2- 172.21.15.157:0/1986759924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73f814c370 0x7f73f80beb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:42.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.984+0000 7f73ffdbc700 1 -- 172.21.15.157:0/1986759924 >> 172.21.15.157:0/1986759924 conn(0x7f73f806f9b0 msgr2=0x7f73f80c4700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:42.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.984+0000 7f73ffdbc700 1 -- 172.21.15.157:0/1986759924 shutdown_connections 2023-10-06T04:04:42.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:42.984+0000 7f73ffdbc700 1 -- 172.21.15.157:0/1986759924 wait complete. 2023-10-06T04:04:42.989 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 61 2023-10-06T04:04:42.999 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:04:45.000 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:04:45.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.118+0000 7f76f8655700 1 Processor -- start 2023-10-06T04:04:45.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.118+0000 7f76f8655700 1 -- start start 2023-10-06T04:04:45.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.118+0000 7f76f8655700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f76f006cbb0 0x7f76f006d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:45.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.118+0000 7f76f8655700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f76f0150410 0x7f76f006da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:45.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.118+0000 7f76f8655700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76f00c9780 con 0x7f76f006c1f0 2023-10-06T04:04:45.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.118+0000 7f76f8655700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76f00c8c40 con 0x7f76f006cbb0 2023-10-06T04:04:45.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.118+0000 7f76f8655700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76f006e0c0 con 0x7f76f0150410 2023-10-06T04:04:45.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.119+0000 7f76f63f1700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f76f006cbb0 0x7f76f006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:45.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.119+0000 7f76f63f1700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f76f006cbb0 0x7f76f006d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42096/0 (socket says 172.21.15.157:42096) 2023-10-06T04:04:45.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.119+0000 7f76f5bf0700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f76f006c1f0 0x7f76f014fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:52500/0 (socket says 172.21.15.157:52500) 2023-10-06T04:04:45.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.119+0000 7f76f63f1700 1 -- 172.21.15.157:0/1727499893 learned_addr learned my addr 172.21.15.157:0/1727499893 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:45.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.119+0000 7f76f6bf2700 1 --2- 172.21.15.157:0/1727499893 >> v2:172.21.15.175:3300/0 conn(0x7f76f0150410 0x7f76f006da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:45.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.119+0000 7f76f63f1700 1 -- 172.21.15.157:0/1727499893 >> v2:172.21.15.175:3300/0 conn(0x7f76f0150410 msgr2=0x7f76f006da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:04:45.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.119+0000 7f76f63f1700 1 --2- 172.21.15.157:0/1727499893 >> v2:172.21.15.175:3300/0 conn(0x7f76f0150410 0x7f76f006da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:45.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.119+0000 7f76f63f1700 1 -- 172.21.15.157:0/1727499893 >> v1:172.21.15.157:6789/0 conn(0x7f76f006c1f0 legacy=0x7f76f014fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:45.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.119+0000 7f76f63f1700 1 -- 172.21.15.157:0/1727499893 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f00c8e50 con 0x7f76f006cbb0 2023-10-06T04:04:45.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.119+0000 7f76f63f1700 1 --2- 172.21.15.157:0/1727499893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76f006cbb0 0x7f76f006d530 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f76f00b1b50 tx=0x7f76ec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59bec3be0f503193 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:45.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.119+0000 7f76f53ef700 1 -- 172.21.15.157:0/1727499893 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec00e070 con 0x7f76f006cbb0 2023-10-06T04:04:45.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.119+0000 7f76f53ef700 1 -- 172.21.15.157:0/1727499893 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76ec00bab0 con 0x7f76f006cbb0 2023-10-06T04:04:45.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.119+0000 7f76f53ef700 1 -- 172.21.15.157:0/1727499893 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec0092e0 con 0x7f76f006cbb0 2023-10-06T04:04:45.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.120+0000 7f76f8655700 1 -- 172.21.15.157:0/1727499893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76f006cbb0 msgr2=0x7f76f006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:45.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.120+0000 7f76f8655700 1 --2- 172.21.15.157:0/1727499893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76f006cbb0 0x7f76f006d530 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f76f00b1b50 tx=0x7f76ec009840 comp rx=0 tx=0).stop 2023-10-06T04:04:45.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.120+0000 7f76f8655700 1 -- 172.21.15.157:0/1727499893 shutdown_connections 2023-10-06T04:04:45.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.120+0000 7f76f8655700 1 --2- 172.21.15.157:0/1727499893 >> v2:172.21.15.175:3300/0 conn(0x7f76f0150410 0x7f76f006da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:45.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.120+0000 7f76f8655700 1 --2- 172.21.15.157:0/1727499893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76f006cbb0 0x7f76f006d530 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:45.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.120+0000 7f76f8655700 1 -- 172.21.15.157:0/1727499893 >> 172.21.15.157:0/1727499893 conn(0x7f76f006f230 msgr2=0x7f76f006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:45.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.120+0000 7f76f8655700 1 -- 172.21.15.157:0/1727499893 shutdown_connections 2023-10-06T04:04:45.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.120+0000 7f76f8655700 1 -- 172.21.15.157:0/1727499893 wait complete. 2023-10-06T04:04:45.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.121+0000 7f76f8655700 1 Processor -- start 2023-10-06T04:04:45.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.121+0000 7f76f8655700 1 -- start start 2023-10-06T04:04:45.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.122+0000 7f76f8655700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76f006c1f0 0x7f76f0169750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:45.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.122+0000 7f76f8655700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76f006cbb0 0x7f76f0157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:45.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.122+0000 7f76f8655700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76f0150410 0x7f76f0152b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:45.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.122+0000 7f76f8655700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f016a630 con 0x7f76f006cbb0 2023-10-06T04:04:45.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.122+0000 7f76f8655700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f015a1b0 con 0x7f76f006c1f0 2023-10-06T04:04:45.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.122+0000 7f76f8655700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f0169d90 con 0x7f76f0150410 2023-10-06T04:04:45.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.122+0000 7f76f5bf0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76f006cbb0 0x7f76f0157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:45.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.122+0000 7f76f5bf0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76f006cbb0 0x7f76f0157a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42104/0 (socket says 172.21.15.157:42104) 2023-10-06T04:04:45.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.122+0000 7f76f5bf0700 1 -- 172.21.15.157:0/3064082293 learned_addr learned my addr 172.21.15.157:0/3064082293 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:45.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.122+0000 7f76f63f1700 1 --2- 172.21.15.157:0/3064082293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76f006c1f0 0x7f76f0169750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:45.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.122+0000 7f76f6bf2700 1 --2- 172.21.15.157:0/3064082293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76f0150410 0x7f76f0152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:45.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.122+0000 7f76f5bf0700 1 -- 172.21.15.157:0/3064082293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76f0150410 msgr2=0x7f76f0152b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:45.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.122+0000 7f76f5bf0700 1 --2- 172.21.15.157:0/3064082293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76f0150410 0x7f76f0152b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:45.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.122+0000 7f76f5bf0700 1 -- 172.21.15.157:0/3064082293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76f006c1f0 msgr2=0x7f76f0169750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:45.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.122+0000 7f76f5bf0700 1 --2- 172.21.15.157:0/3064082293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76f006c1f0 0x7f76f0169750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:45.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.122+0000 7f76f5bf0700 1 -- 172.21.15.157:0/3064082293 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ec009ae0 con 0x7f76f006cbb0 2023-10-06T04:04:45.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.122+0000 7f76f5bf0700 1 --2- 172.21.15.157:0/3064082293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76f006cbb0 0x7f76f0157a70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f76e000c7c0 tx=0x7f76e0007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:45.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.122+0000 7f76e67fc700 1 -- 172.21.15.157:0/3064082293 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e0005a70 con 0x7f76f006cbb0 2023-10-06T04:04:45.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.122+0000 7f76e67fc700 1 -- 172.21.15.157:0/3064082293 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e0005bf0 con 0x7f76f006cbb0 2023-10-06T04:04:45.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.123+0000 7f76f8655700 1 -- 172.21.15.157:0/3064082293 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f016a070 con 0x7f76f006cbb0 2023-10-06T04:04:45.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.123+0000 7f76e67fc700 1 -- 172.21.15.157:0/3064082293 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e00070d0 con 0x7f76f006cbb0 2023-10-06T04:04:45.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.123+0000 7f76f8655700 1 -- 172.21.15.157:0/3064082293 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f0157270 con 0x7f76f006cbb0 2023-10-06T04:04:45.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.123+0000 7f76f8655700 1 -- 172.21.15.157:0/3064082293 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f0089ca0 con 0x7f76f006cbb0 2023-10-06T04:04:45.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.124+0000 7f76e67fc700 1 -- 172.21.15.157:0/3064082293 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f76e0007480 con 0x7f76f006cbb0 2023-10-06T04:04:45.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.124+0000 7f76e67fc700 1 --2- 172.21.15.157:0/3064082293 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f76d4066f30 0x7f76d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:45.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.125+0000 7f76e67fc700 1 -- 172.21.15.157:0/3064082293 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6374+0+0 (secure 0 0 0) 0x7f76e0083b80 con 0x7f76f006cbb0 2023-10-06T04:04:45.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.125+0000 7f76f63f1700 1 --2- 172.21.15.157:0/3064082293 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f76d4066f30 0x7f76d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:45.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.125+0000 7f76f63f1700 1 --2- 172.21.15.157:0/3064082293 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f76d4066f30 0x7f76d40693e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009880 tx=0x7f76ec018000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:45.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.130+0000 7f76e67fc700 1 -- 172.21.15.157:0/3064082293 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f76e0050b50 con 0x7f76f006cbb0 2023-10-06T04:04:45.292 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:04:45.295 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:04:45.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.360+0000 7f76f8655700 1 -- 172.21.15.157:0/3064082293 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76f00c9130 con 0x7f76f006cbb0 2023-10-06T04:04:45.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.361+0000 7f76e67fc700 1 -- 172.21.15.157:0/3064082293 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+5388 (secure 0 0 0) 0x7f76e0017020 con 0x7f76f006cbb0 2023-10-06T04:04:45.363 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:04:45.363 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:04:42.790463+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:rejoin","state_seq":20,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:04:45.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.364+0000 7f76f8655700 1 -- 172.21.15.157:0/3064082293 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f76d4066f30 msgr2=0x7f76d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:45.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.364+0000 7f76f8655700 1 --2- 172.21.15.157:0/3064082293 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f76d4066f30 0x7f76d40693e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009880 tx=0x7f76ec018000 comp rx=0 tx=0).stop 2023-10-06T04:04:45.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.364+0000 7f76f8655700 1 -- 172.21.15.157:0/3064082293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76f006cbb0 msgr2=0x7f76f0157a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:45.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.364+0000 7f76f8655700 1 --2- 172.21.15.157:0/3064082293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76f006cbb0 0x7f76f0157a70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f76e000c7c0 tx=0x7f76e0007f70 comp rx=0 tx=0).stop 2023-10-06T04:04:45.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.364+0000 7f76f8655700 1 -- 172.21.15.157:0/3064082293 shutdown_connections 2023-10-06T04:04:45.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.364+0000 7f76f8655700 1 --2- 172.21.15.157:0/3064082293 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f76d4066f30 0x7f76d40693e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:45.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.364+0000 7f76f8655700 1 --2- 172.21.15.157:0/3064082293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76f006c1f0 0x7f76f0169750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:45.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.364+0000 7f76f8655700 1 --2- 172.21.15.157:0/3064082293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76f0150410 0x7f76f0152b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:45.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.364+0000 7f76f8655700 1 --2- 172.21.15.157:0/3064082293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76f006cbb0 0x7f76f0157a70 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:45.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.364+0000 7f76f8655700 1 -- 172.21.15.157:0/3064082293 >> 172.21.15.157:0/3064082293 conn(0x7f76f006f230 msgr2=0x7f76f014b690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:45.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.364+0000 7f76f8655700 1 -- 172.21.15.157:0/3064082293 shutdown_connections 2023-10-06T04:04:45.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:45.364+0000 7f76f8655700 1 -- 172.21.15.157:0/3064082293 wait complete. 2023-10-06T04:04:45.367 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 61 2023-10-06T04:04:45.377 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:04:47.379 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:04:47.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.505+0000 7f3cfdcad700 1 Processor -- start 2023-10-06T04:04:47.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.505+0000 7f3cfdcad700 1 -- start start 2023-10-06T04:04:47.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.506+0000 7f3cfdcad700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3cf80b1e40 0x7f3cf8157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:47.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.506+0000 7f3cfdcad700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cf806d8d0 con 0x7f3cf806c2d0 2023-10-06T04:04:47.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.506+0000 7f3cfdcad700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cf80af8f0 con 0x7f3cf80b04d0 2023-10-06T04:04:47.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.506+0000 7f3cfdcad700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cf815a690 con 0x7f3cf80b1e40 2023-10-06T04:04:47.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.506+0000 7f3cfd4ac700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f3cf806c2d0 0x7f3cf8156ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:52512/0 (socket says 172.21.15.157:52512) 2023-10-06T04:04:47.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.506+0000 7f3cfd4ac700 1 -- 172.21.15.157:0/2747090602 learned_addr learned my addr 172.21.15.157:0/2747090602 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:04:47.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.506+0000 7f3cfccab700 1 --2- 172.21.15.157:0/2747090602 >> v2:172.21.15.175:3300/0 conn(0x7f3cf80b1e40 0x7f3cf8157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:47.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.506+0000 7f3cf77fe700 1 -- 172.21.15.157:0/2747090602 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614690714 0 0) 0x7f3cf806d8d0 con 0x7f3cf806c2d0 2023-10-06T04:04:47.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.506+0000 7f3cf77fe700 1 -- 172.21.15.157:0/2747090602 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ce8003570 con 0x7f3cf806c2d0 2023-10-06T04:04:47.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.507+0000 7f3cf77fe700 1 -- 172.21.15.157:0/2747090602 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 477296114 0 0) 0x7f3ce8003570 con 0x7f3cf806c2d0 2023-10-06T04:04:47.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.507+0000 7f3cf77fe700 1 -- 172.21.15.157:0/2747090602 >> v2:172.21.15.175:3300/0 conn(0x7f3cf80b1e40 msgr2=0x7f3cf8157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:04:47.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.507+0000 7f3cf77fe700 1 --2- 172.21.15.157:0/2747090602 >> v2:172.21.15.175:3300/0 conn(0x7f3cf80b1e40 0x7f3cf8157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:47.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.507+0000 7f3cf77fe700 1 -- 172.21.15.157:0/2747090602 >> v1:172.21.15.175:6789/0 conn(0x7f3cf80b04d0 legacy=0x7f3cf81563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:47.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.507+0000 7f3cf77fe700 1 -- 172.21.15.157:0/2747090602 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf81571d0 con 0x7f3cf806c2d0 2023-10-06T04:04:47.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.507+0000 7f3cf77fe700 1 -- 172.21.15.157:0/2747090602 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f3cf00017f0 con 0x7f3cf806c2d0 2023-10-06T04:04:47.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.507+0000 7f3cf77fe700 1 -- 172.21.15.157:0/2747090602 >> v1:172.21.15.157:6789/0 conn(0x7f3cf806c2d0 legacy=0x7f3cf8156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:47.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.507+0000 7f3cf77fe700 1 --2- 172.21.15.157:0/2747090602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ce8003a60 0x7f3ce8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:47.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.507+0000 7f3cf77fe700 1 --2- 172.21.15.157:0/2747090602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ce8004810 0x7f3ce8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:47.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.507+0000 7f3cf77fe700 1 --2- 172.21.15.157:0/2747090602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ce80051c0 0x7f3ce800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:47.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.507+0000 7f3cf77fe700 1 -- 172.21.15.157:0/2747090602 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf815a690 con 0x7f3ce8003a60 2023-10-06T04:04:47.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.507+0000 7f3cf77fe700 1 -- 172.21.15.157:0/2747090602 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce800c160 con 0x7f3ce8004810 2023-10-06T04:04:47.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.507+0000 7f3cf77fe700 1 -- 172.21.15.157:0/2747090602 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce800c320 con 0x7f3ce80051c0 2023-10-06T04:04:47.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.507+0000 7f3cfccab700 1 --2- 172.21.15.157:0/2747090602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ce8003a60 0x7f3ce8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:47.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.507+0000 7f3cf7fff700 1 --2- 172.21.15.157:0/2747090602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ce8004810 0x7f3ce8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:47.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.507+0000 7f3cfd4ac700 1 --2- 172.21.15.157:0/2747090602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ce80051c0 0x7f3ce800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:47.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.507+0000 7f3cfccab700 1 -- 172.21.15.157:0/2747090602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ce80051c0 msgr2=0x7f3ce800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:04:47.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.507+0000 7f3cfccab700 1 --2- 172.21.15.157:0/2747090602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ce80051c0 0x7f3ce800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:47.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.507+0000 7f3cfccab700 1 -- 172.21.15.157:0/2747090602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ce8004810 msgr2=0x7f3ce8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:04:47.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.507+0000 7f3cfccab700 1 --2- 172.21.15.157:0/2747090602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ce8004810 0x7f3ce8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:47.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.507+0000 7f3cfccab700 1 -- 172.21.15.157:0/2747090602 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce8003570 con 0x7f3ce8003a60 2023-10-06T04:04:47.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.507+0000 7f3cfd4ac700 1 --2- 172.21.15.157:0/2747090602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ce80051c0 0x7f3ce800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:04:47.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.507+0000 7f3cf7fff700 1 --2- 172.21.15.157:0/2747090602 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ce8004810 0x7f3ce8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:04:47.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.507+0000 7f3cfd4ac700 1 -- 172.21.15.157:0/2747090602 reap_dead start 2023-10-06T04:04:47.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.507+0000 7f3cfccab700 1 --2- 172.21.15.157:0/2747090602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ce8003a60 0x7f3ce8003e70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3cec009c00 tx=0x7f3cec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a00862068d498e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:47.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.507+0000 7f3cf77fe700 1 -- 172.21.15.157:0/2747090602 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec00e710 con 0x7f3ce8003a60 2023-10-06T04:04:47.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.507+0000 7f3cf77fe700 1 -- 172.21.15.157:0/2747090602 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cec00e870 con 0x7f3ce8003a60 2023-10-06T04:04:47.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.507+0000 7f3cf77fe700 1 -- 172.21.15.157:0/2747090602 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec00ec30 con 0x7f3ce8003a60 2023-10-06T04:04:47.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.508+0000 7f3cfdcad700 1 -- 172.21.15.157:0/2747090602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ce8003a60 msgr2=0x7f3ce8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:47.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.508+0000 7f3cfdcad700 1 --2- 172.21.15.157:0/2747090602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ce8003a60 0x7f3ce8003e70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3cec009c00 tx=0x7f3cec0091a0 comp rx=0 tx=0).stop 2023-10-06T04:04:47.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.508+0000 7f3cfdcad700 1 -- 172.21.15.157:0/2747090602 shutdown_connections 2023-10-06T04:04:47.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.508+0000 7f3cfdcad700 1 --2- 172.21.15.157:0/2747090602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ce8003a60 0x7f3ce8003e70 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:47.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.508+0000 7f3cfdcad700 1 -- 172.21.15.157:0/2747090602 >> 172.21.15.157:0/2747090602 conn(0x7f3cf80cc540 msgr2=0x7f3cf80cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:47.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.508+0000 7f3cfdcad700 1 -- 172.21.15.157:0/2747090602 shutdown_connections 2023-10-06T04:04:47.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.508+0000 7f3cfdcad700 1 -- 172.21.15.157:0/2747090602 wait complete. 2023-10-06T04:04:47.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.509+0000 7f3cfdcad700 1 Processor -- start 2023-10-06T04:04:47.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.509+0000 7f3cfdcad700 1 -- start start 2023-10-06T04:04:47.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.510+0000 7f3cfdcad700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ce8003a60 0x7f3cf80c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:47.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.510+0000 7f3cfdcad700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cf80c7fe0 0x7f3cf814f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:47.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.510+0000 7f3cfdcad700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cf81566a0 0x7f3cf814fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:47.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.510+0000 7f3cfdcad700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf8169650 con 0x7f3cf81566a0 2023-10-06T04:04:47.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.510+0000 7f3cfdcad700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf8150320 con 0x7f3cf80c7fe0 2023-10-06T04:04:47.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.510+0000 7f3cfdcad700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf8169ae0 con 0x7f3ce8003a60 2023-10-06T04:04:47.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.510+0000 7f3cfd4ac700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cf81566a0 0x7f3cf814fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:47.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.510+0000 7f3cfd4ac700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cf81566a0 0x7f3cf814fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42128/0 (socket says 172.21.15.157:42128) 2023-10-06T04:04:47.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.510+0000 7f3cfd4ac700 1 -- 172.21.15.157:0/1958391591 learned_addr learned my addr 172.21.15.157:0/1958391591 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:47.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.510+0000 7f3cfccab700 1 --2- 172.21.15.157:0/1958391591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ce8003a60 0x7f3cf80c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:47.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.510+0000 7f3cf7fff700 1 --2- 172.21.15.157:0/1958391591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cf80c7fe0 0x7f3cf814f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:47.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.510+0000 7f3cfd4ac700 1 -- 172.21.15.157:0/1958391591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ce8003a60 msgr2=0x7f3cf80c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:47.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.510+0000 7f3cfd4ac700 1 --2- 172.21.15.157:0/1958391591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ce8003a60 0x7f3cf80c7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:47.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.511+0000 7f3cfd4ac700 1 -- 172.21.15.157:0/1958391591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cf80c7fe0 msgr2=0x7f3cf814f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:47.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.511+0000 7f3cfd4ac700 1 --2- 172.21.15.157:0/1958391591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cf80c7fe0 0x7f3cf814f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:47.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.511+0000 7f3cfd4ac700 1 -- 172.21.15.157:0/1958391591 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf81665b0 con 0x7f3cf81566a0 2023-10-06T04:04:47.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.511+0000 7f3cf7fff700 1 --2- 172.21.15.157:0/1958391591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cf80c7fe0 0x7f3cf814f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:04:47.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.511+0000 7f3cfd4ac700 1 --2- 172.21.15.157:0/1958391591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cf81566a0 0x7f3cf814fde0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3cf80ca020 tx=0x7f3cf0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:47.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.511+0000 7f3cf4ff9700 1 -- 172.21.15.157:0/1958391591 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0004dc0 con 0x7f3cf81566a0 2023-10-06T04:04:47.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.511+0000 7f3cf4ff9700 1 -- 172.21.15.157:0/1958391591 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf000b9d0 con 0x7f3cf81566a0 2023-10-06T04:04:47.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.511+0000 7f3cfdcad700 1 -- 172.21.15.157:0/1958391591 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf8168d60 con 0x7f3cf81566a0 2023-10-06T04:04:47.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.511+0000 7f3cf4ff9700 1 -- 172.21.15.157:0/1958391591 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf000bd50 con 0x7f3cf81566a0 2023-10-06T04:04:47.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.511+0000 7f3cfdcad700 1 -- 172.21.15.157:0/1958391591 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf8169000 con 0x7f3cf81566a0 2023-10-06T04:04:47.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.512+0000 7f3cf4ff9700 1 -- 172.21.15.157:0/1958391591 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3cf00073a0 con 0x7f3cf81566a0 2023-10-06T04:04:47.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.512+0000 7f3cfdcad700 1 -- 172.21.15.157:0/1958391591 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cf81697c0 con 0x7f3cf81566a0 2023-10-06T04:04:47.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.513+0000 7f3cf4ff9700 1 --2- 172.21.15.157:0/1958391591 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3cdc066f30 0x7f3cdc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:47.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.513+0000 7f3cf4ff9700 1 -- 172.21.15.157:0/1958391591 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6374+0+0 (secure 0 0 0) 0x7f3cf00845d0 con 0x7f3cf81566a0 2023-10-06T04:04:47.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.513+0000 7f3cfccab700 1 --2- 172.21.15.157:0/1958391591 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3cdc066f30 0x7f3cdc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:47.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.513+0000 7f3cfccab700 1 --2- 172.21.15.157:0/1958391591 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3cdc066f30 0x7f3cdc0693e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3cec0091e0 tx=0x7f3cec01a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:47.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.517+0000 7f3cf4ff9700 1 -- 172.21.15.157:0/1958391591 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3cf004c5a0 con 0x7f3cf81566a0 2023-10-06T04:04:47.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.740+0000 7f3cfdcad700 1 -- 172.21.15.157:0/1958391591 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3cf8088f90 con 0x7f3cf81566a0 2023-10-06T04:04:47.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.741+0000 7f3cf4ff9700 1 -- 172.21.15.157:0/1958391591 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+5388 (secure 0 0 0) 0x7f3cf0051620 con 0x7f3cf81566a0 2023-10-06T04:04:47.742 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:04:47.742 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:04:42.790463+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:rejoin","state_seq":20,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:04:47.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.742+0000 7f3cfdcad700 1 -- 172.21.15.157:0/1958391591 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3cdc066f30 msgr2=0x7f3cdc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:47.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.742+0000 7f3cfdcad700 1 --2- 172.21.15.157:0/1958391591 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3cdc066f30 0x7f3cdc0693e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3cec0091e0 tx=0x7f3cec01a000 comp rx=0 tx=0).stop 2023-10-06T04:04:47.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.742+0000 7f3cfdcad700 1 -- 172.21.15.157:0/1958391591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cf81566a0 msgr2=0x7f3cf814fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:47.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.742+0000 7f3cfdcad700 1 --2- 172.21.15.157:0/1958391591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cf81566a0 0x7f3cf814fde0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3cf80ca020 tx=0x7f3cf0009040 comp rx=0 tx=0).stop 2023-10-06T04:04:47.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.743+0000 7f3cfdcad700 1 -- 172.21.15.157:0/1958391591 shutdown_connections 2023-10-06T04:04:47.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.743+0000 7f3cfdcad700 1 --2- 172.21.15.157:0/1958391591 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3cdc066f30 0x7f3cdc0693e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:47.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.743+0000 7f3cfdcad700 1 --2- 172.21.15.157:0/1958391591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cf81566a0 0x7f3cf814fde0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:47.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.743+0000 7f3cfdcad700 1 --2- 172.21.15.157:0/1958391591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ce8003a60 0x7f3cf80c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:47.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.743+0000 7f3cfdcad700 1 --2- 172.21.15.157:0/1958391591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cf80c7fe0 0x7f3cf814f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:47.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.743+0000 7f3cfdcad700 1 -- 172.21.15.157:0/1958391591 >> 172.21.15.157:0/1958391591 conn(0x7f3cf80cc540 msgr2=0x7f3cf8157440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:47.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.743+0000 7f3cfdcad700 1 -- 172.21.15.157:0/1958391591 shutdown_connections 2023-10-06T04:04:47.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:47.743+0000 7f3cfdcad700 1 -- 172.21.15.157:0/1958391591 wait complete. 2023-10-06T04:04:47.746 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 61 2023-10-06T04:04:47.756 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:04:48.014 INFO:tasks.workunit.client.0.smithi157.stderr:+ rm a/foo 2023-10-06T04:04:48.016 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a 2023-10-06T04:04:48.021 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 20` 2023-10-06T04:04:48.022 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a 2023-10-06T04:04:48.023 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/one 2023-10-06T04:04:48.023 INFO:tasks.workunit.client.0.smithi157.stderr:+ cat 2023-10-06T04:04:48.052 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/two 2023-10-06T04:04:48.073 INFO:tasks.workunit.client.0.smithi157.stderr:+ chmod 777 a/foo 2023-10-06T04:04:48.074 INFO:tasks.workunit.client.0.smithi157.stderr:+ sync 2023-10-06T04:04:48.228 INFO:tasks.workunit.client.0.smithi157.stderr:+ ps 2023-10-06T04:04:48.235 INFO:tasks.workunit.client.0.smithi157.stdout: PID TTY TIME CMD 2023-10-06T04:04:48.235 INFO:tasks.workunit.client.0.smithi157.stdout: 6131 ? 00:00:00 systemd 2023-10-06T04:04:48.235 INFO:tasks.workunit.client.0.smithi157.stdout: 6134 ? 00:00:00 (sd-pam) 2023-10-06T04:04:48.236 INFO:tasks.workunit.client.0.smithi157.stdout: 78246 ? 00:00:00 sshd 2023-10-06T04:04:48.236 INFO:tasks.workunit.client.0.smithi157.stdout: 106071 ? 00:00:00 timeout 2023-10-06T04:04:48.236 INFO:tasks.workunit.client.0.smithi157.stdout: 106088 ? 00:00:00 snaptest-double 2023-10-06T04:04:48.236 INFO:tasks.workunit.client.0.smithi157.stdout: 107056 ? 00:00:00 ps 2023-10-06T04:04:48.236 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-06T04:04:49.757 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:04:49.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.884+0000 7f33737bf700 1 Processor -- start 2023-10-06T04:04:49.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.885+0000 7f33737bf700 1 -- start start 2023-10-06T04:04:49.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.885+0000 7f33737bf700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f336c0c5fc0 0x7f336c069130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:49.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.885+0000 7f33737bf700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f336c06c810 0x7f336c1569e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:49.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.885+0000 7f33737bf700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f336c155f20 con 0x7f336c06c440 2023-10-06T04:04:49.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.885+0000 7f33737bf700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f336c06f310 con 0x7f336c0c5fc0 2023-10-06T04:04:49.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.885+0000 7f33737bf700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f336c06d4a0 con 0x7f336c06c810 2023-10-06T04:04:49.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.886+0000 7f3370d5a700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f336c0c5fc0 0x7f336c069130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:49.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.886+0000 7f3370d5a700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f336c0c5fc0 0x7f336c069130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42144/0 (socket says 172.21.15.157:42144) 2023-10-06T04:04:49.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.886+0000 7f3370d5a700 1 -- 172.21.15.157:0/2334560547 learned_addr learned my addr 172.21.15.157:0/2334560547 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:49.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.886+0000 7f337155b700 1 --1- 172.21.15.157:0/2334560547 >> v1:172.21.15.157:6789/0 conn(0x7f336c06c440 0x7f336c152e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:52522/0 (socket says 172.21.15.157:52522) 2023-10-06T04:04:49.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.886+0000 7f3371d5c700 1 --2- 172.21.15.157:0/2334560547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f336c06c810 0x7f336c1569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:49.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.886+0000 7f3370d5a700 1 -- 172.21.15.157:0/2334560547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f336c06c810 msgr2=0x7f336c1569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:04:49.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.886+0000 7f3370d5a700 1 --2- 172.21.15.157:0/2334560547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f336c06c810 0x7f336c1569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:49.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.886+0000 7f3370d5a700 1 -- 172.21.15.157:0/2334560547 >> v1:172.21.15.157:6789/0 conn(0x7f336c06c440 legacy=0x7f336c152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:49.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.886+0000 7f3370d5a700 1 -- 172.21.15.157:0/2334560547 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f336c06d780 con 0x7f336c0c5fc0 2023-10-06T04:04:49.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.886+0000 7f3370d5a700 1 --2- 172.21.15.157:0/2334560547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c0c5fc0 0x7f336c069130 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f335c0099b0 tx=0x7f335c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b8c7ba5d9f7da1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:49.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.886+0000 7f3363fff700 1 -- 172.21.15.157:0/2334560547 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c00b860 con 0x7f336c0c5fc0 2023-10-06T04:04:49.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.886+0000 7f3363fff700 1 -- 172.21.15.157:0/2334560547 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f335c00b9c0 con 0x7f336c0c5fc0 2023-10-06T04:04:49.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.886+0000 7f3363fff700 1 -- 172.21.15.157:0/2334560547 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c00bd60 con 0x7f336c0c5fc0 2023-10-06T04:04:49.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.887+0000 7f33737bf700 1 -- 172.21.15.157:0/2334560547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c0c5fc0 msgr2=0x7f336c069130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:49.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.887+0000 7f33737bf700 1 --2- 172.21.15.157:0/2334560547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c0c5fc0 0x7f336c069130 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f335c0099b0 tx=0x7f335c009250 comp rx=0 tx=0).stop 2023-10-06T04:04:49.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.887+0000 7f33737bf700 1 -- 172.21.15.157:0/2334560547 shutdown_connections 2023-10-06T04:04:49.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.887+0000 7f33737bf700 1 --2- 172.21.15.157:0/2334560547 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f336c06c810 0x7f336c1569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:49.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.887+0000 7f33737bf700 1 --2- 172.21.15.157:0/2334560547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c0c5fc0 0x7f336c069130 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:49.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.887+0000 7f33737bf700 1 -- 172.21.15.157:0/2334560547 >> 172.21.15.157:0/2334560547 conn(0x7f336c068370 msgr2=0x7f336c068770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:49.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.887+0000 7f33737bf700 1 -- 172.21.15.157:0/2334560547 shutdown_connections 2023-10-06T04:04:49.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.887+0000 7f33737bf700 1 -- 172.21.15.157:0/2334560547 wait complete. 2023-10-06T04:04:49.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.888+0000 7f33737bf700 1 Processor -- start 2023-10-06T04:04:49.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.888+0000 7f33737bf700 1 -- start start 2023-10-06T04:04:49.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.889+0000 7f33737bf700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f336c06c440 0x7f336c1697f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:49.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.889+0000 7f33737bf700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f336c06c810 0x7f336c169d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:49.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.889+0000 7f33737bf700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c0c5fc0 0x7f336c0c3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:49.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.889+0000 7f33737bf700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336c0c33d0 con 0x7f336c0c5fc0 2023-10-06T04:04:49.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.889+0000 7f33737bf700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336c0c28c0 con 0x7f336c06c440 2023-10-06T04:04:49.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.889+0000 7f33737bf700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f336c0c0bf0 con 0x7f336c06c810 2023-10-06T04:04:49.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.889+0000 7f3371d5c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c0c5fc0 0x7f336c0c3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:49.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.889+0000 7f3371d5c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c0c5fc0 0x7f336c0c3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42148/0 (socket says 172.21.15.157:42148) 2023-10-06T04:04:49.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.889+0000 7f3371d5c700 1 -- 172.21.15.157:0/4221240276 learned_addr learned my addr 172.21.15.157:0/4221240276 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:49.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.889+0000 7f3370d5a700 1 --2- 172.21.15.157:0/4221240276 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f336c06c810 0x7f336c169d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:49.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.889+0000 7f337155b700 1 --2- 172.21.15.157:0/4221240276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f336c06c440 0x7f336c1697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:49.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.889+0000 7f3371d5c700 1 -- 172.21.15.157:0/4221240276 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f336c06c810 msgr2=0x7f336c169d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:49.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.889+0000 7f3371d5c700 1 --2- 172.21.15.157:0/4221240276 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f336c06c810 0x7f336c169d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:49.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.889+0000 7f3371d5c700 1 -- 172.21.15.157:0/4221240276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f336c06c440 msgr2=0x7f336c1697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:49.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.889+0000 7f3371d5c700 1 --2- 172.21.15.157:0/4221240276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f336c06c440 0x7f336c1697f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:49.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.889+0000 7f3371d5c700 1 -- 172.21.15.157:0/4221240276 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f335c0094f0 con 0x7f336c0c5fc0 2023-10-06T04:04:49.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.890+0000 7f3371d5c700 1 --2- 172.21.15.157:0/4221240276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c0c5fc0 0x7f336c0c3920 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3364007f50 tx=0x7f3364009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:49.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.890+0000 7f33617fa700 1 -- 172.21.15.157:0/4221240276 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336400eb70 con 0x7f336c0c5fc0 2023-10-06T04:04:49.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.890+0000 7f33617fa700 1 -- 172.21.15.157:0/4221240276 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f336400ecf0 con 0x7f336c0c5fc0 2023-10-06T04:04:49.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.890+0000 7f33617fa700 1 -- 172.21.15.157:0/4221240276 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3364014670 con 0x7f336c0c5fc0 2023-10-06T04:04:49.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.890+0000 7f33737bf700 1 -- 172.21.15.157:0/4221240276 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f336c0c9cf0 con 0x7f336c0c5fc0 2023-10-06T04:04:49.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.890+0000 7f33737bf700 1 -- 172.21.15.157:0/4221240276 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f336c0c9a80 con 0x7f336c0c5fc0 2023-10-06T04:04:49.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.891+0000 7f33737bf700 1 -- 172.21.15.157:0/4221240276 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f336c06a8f0 con 0x7f336c0c5fc0 2023-10-06T04:04:49.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.891+0000 7f33617fa700 1 -- 172.21.15.157:0/4221240276 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f33640149e0 con 0x7f336c0c5fc0 2023-10-06T04:04:49.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.892+0000 7f33617fa700 1 --2- 172.21.15.157:0/4221240276 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3350066f30 0x7f33500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:49.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.892+0000 7f33617fa700 1 -- 172.21.15.157:0/4221240276 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6374+0+0 (secure 0 0 0) 0x7f3364084d50 con 0x7f336c0c5fc0 2023-10-06T04:04:49.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.892+0000 7f337155b700 1 --2- 172.21.15.157:0/4221240276 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3350066f30 0x7f33500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:49.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.892+0000 7f337155b700 1 --2- 172.21.15.157:0/4221240276 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3350066f30 0x7f33500693e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3368004fd0 tx=0x7f3368007eb0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:49.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:49.896+0000 7f33617fa700 1 -- 172.21.15.157:0/4221240276 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3364088030 con 0x7f336c0c5fc0 2023-10-06T04:04:50.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:50.122+0000 7f33737bf700 1 -- 172.21.15.157:0/4221240276 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f336c1561e0 con 0x7f336c0c5fc0 2023-10-06T04:04:50.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:50.123+0000 7f33617fa700 1 -- 172.21.15.157:0/4221240276 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+5388 (secure 0 0 0) 0x7f3364051d20 con 0x7f336c0c5fc0 2023-10-06T04:04:50.125 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:04:50.125 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:04:48.805873+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:04:50.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:50.125+0000 7f33737bf700 1 -- 172.21.15.157:0/4221240276 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3350066f30 msgr2=0x7f33500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:50.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:50.125+0000 7f33737bf700 1 --2- 172.21.15.157:0/4221240276 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3350066f30 0x7f33500693e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3368004fd0 tx=0x7f3368007eb0 comp rx=0 tx=0).stop 2023-10-06T04:04:50.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:50.125+0000 7f33737bf700 1 -- 172.21.15.157:0/4221240276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c0c5fc0 msgr2=0x7f336c0c3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:50.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:50.125+0000 7f33737bf700 1 --2- 172.21.15.157:0/4221240276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c0c5fc0 0x7f336c0c3920 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3364007f50 tx=0x7f3364009860 comp rx=0 tx=0).stop 2023-10-06T04:04:50.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:50.125+0000 7f33737bf700 1 -- 172.21.15.157:0/4221240276 shutdown_connections 2023-10-06T04:04:50.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:50.125+0000 7f33737bf700 1 --2- 172.21.15.157:0/4221240276 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3350066f30 0x7f33500693e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:50.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:50.125+0000 7f33737bf700 1 --2- 172.21.15.157:0/4221240276 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f336c06c440 0x7f336c1697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:50.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:50.125+0000 7f33737bf700 1 --2- 172.21.15.157:0/4221240276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c0c5fc0 0x7f336c0c3920 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:50.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:50.125+0000 7f33737bf700 1 --2- 172.21.15.157:0/4221240276 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f336c06c810 0x7f336c169d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:50.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:50.125+0000 7f33737bf700 1 -- 172.21.15.157:0/4221240276 >> 172.21.15.157:0/4221240276 conn(0x7f336c068370 msgr2=0x7f336c153d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:50.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:50.125+0000 7f33737bf700 1 -- 172.21.15.157:0/4221240276 shutdown_connections 2023-10-06T04:04:50.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:50.125+0000 7f33737bf700 1 -- 172.21.15.157:0/4221240276 wait complete. 2023-10-06T04:04:50.128 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 62 2023-10-06T04:04:50.138 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d has gained rank=1, replacing gid=4287 2023-10-06T04:04:50.139 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 7 secs before reviving mds.c 2023-10-06T04:04:54.089 INFO:tasks.workunit.client.0.smithi157.stderr:+ rm a/foo 2023-10-06T04:04:54.092 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a 2023-10-06T04:04:54.101 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 20` 2023-10-06T04:04:54.101 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a 2023-10-06T04:04:54.103 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/one 2023-10-06T04:04:54.104 INFO:tasks.workunit.client.0.smithi157.stderr:+ cat 2023-10-06T04:04:54.653 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/two 2023-10-06T04:04:54.676 INFO:tasks.workunit.client.0.smithi157.stderr:+ chmod 777 a/foo 2023-10-06T04:04:54.677 INFO:tasks.workunit.client.0.smithi157.stderr:+ sync 2023-10-06T04:04:54.867 INFO:tasks.workunit.client.0.smithi157.stderr:+ ps 2023-10-06T04:04:54.875 INFO:tasks.workunit.client.0.smithi157.stdout: PID TTY TIME CMD 2023-10-06T04:04:54.875 INFO:tasks.workunit.client.0.smithi157.stdout: 6131 ? 00:00:00 systemd 2023-10-06T04:04:54.875 INFO:tasks.workunit.client.0.smithi157.stdout: 6134 ? 00:00:00 (sd-pam) 2023-10-06T04:04:54.875 INFO:tasks.workunit.client.0.smithi157.stdout: 78246 ? 00:00:00 sshd 2023-10-06T04:04:54.875 INFO:tasks.workunit.client.0.smithi157.stdout: 106071 ? 00:00:00 timeout 2023-10-06T04:04:54.875 INFO:tasks.workunit.client.0.smithi157.stdout: 106088 ? 00:00:00 snaptest-double 2023-10-06T04:04:54.875 INFO:tasks.workunit.client.0.smithi157.stdout: 107108 ? 00:00:00 ps 2023-10-06T04:04:54.876 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-06T04:04:54.891 INFO:tasks.workunit.client.0.smithi157.stderr:+ rm a/foo 2023-10-06T04:04:54.893 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a 2023-10-06T04:04:54.901 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 20` 2023-10-06T04:04:54.901 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a 2023-10-06T04:04:54.904 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/one 2023-10-06T04:04:54.904 INFO:tasks.workunit.client.0.smithi157.stderr:+ cat 2023-10-06T04:04:54.930 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/two 2023-10-06T04:04:54.952 INFO:tasks.workunit.client.0.smithi157.stderr:+ chmod 777 a/foo 2023-10-06T04:04:54.952 INFO:tasks.workunit.client.0.smithi157.stderr:+ sync 2023-10-06T04:04:55.025 INFO:tasks.workunit.client.0.smithi157.stderr:+ ps 2023-10-06T04:04:55.033 INFO:tasks.workunit.client.0.smithi157.stdout: PID TTY TIME CMD 2023-10-06T04:04:55.033 INFO:tasks.workunit.client.0.smithi157.stdout: 6131 ? 00:00:00 systemd 2023-10-06T04:04:55.033 INFO:tasks.workunit.client.0.smithi157.stdout: 6134 ? 00:00:00 (sd-pam) 2023-10-06T04:04:55.033 INFO:tasks.workunit.client.0.smithi157.stdout: 78246 ? 00:00:00 sshd 2023-10-06T04:04:55.033 INFO:tasks.workunit.client.0.smithi157.stdout: 106071 ? 00:00:00 timeout 2023-10-06T04:04:55.033 INFO:tasks.workunit.client.0.smithi157.stdout: 106088 ? 00:00:00 snaptest-double 2023-10-06T04:04:55.033 INFO:tasks.workunit.client.0.smithi157.stdout: 107118 ? 00:00:00 ps 2023-10-06T04:04:55.034 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-06T04:04:55.045 INFO:tasks.workunit.client.0.smithi157.stderr:+ rm a/foo 2023-10-06T04:04:55.048 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a 2023-10-06T04:04:55.055 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 20` 2023-10-06T04:04:55.055 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a 2023-10-06T04:04:55.057 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/one 2023-10-06T04:04:55.057 INFO:tasks.workunit.client.0.smithi157.stderr:+ cat 2023-10-06T04:04:55.082 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir a/.snap/two 2023-10-06T04:04:55.104 INFO:tasks.workunit.client.0.smithi157.stderr:+ chmod 777 a/foo 2023-10-06T04:04:55.105 INFO:tasks.workunit.client.0.smithi157.stderr:+ sync 2023-10-06T04:04:55.182 INFO:tasks.workunit.client.0.smithi157.stderr:+ ps 2023-10-06T04:04:55.189 INFO:tasks.workunit.client.0.smithi157.stdout: PID TTY TIME CMD 2023-10-06T04:04:55.190 INFO:tasks.workunit.client.0.smithi157.stdout: 6131 ? 00:00:00 systemd 2023-10-06T04:04:55.190 INFO:tasks.workunit.client.0.smithi157.stdout: 6134 ? 00:00:00 (sd-pam) 2023-10-06T04:04:55.190 INFO:tasks.workunit.client.0.smithi157.stdout: 78246 ? 00:00:00 sshd 2023-10-06T04:04:55.190 INFO:tasks.workunit.client.0.smithi157.stdout: 106071 ? 00:00:00 timeout 2023-10-06T04:04:55.191 INFO:tasks.workunit.client.0.smithi157.stdout: 106088 ? 00:00:00 snaptest-double 2023-10-06T04:04:55.191 INFO:tasks.workunit.client.0.smithi157.stdout: 107128 ? 00:00:00 ps 2023-10-06T04:04:55.191 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a/.snap/one a/.snap/two 2023-10-06T04:04:55.201 INFO:tasks.workunit.client.0.smithi157.stderr:+ rm a/foo 2023-10-06T04:04:55.204 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir a 2023-10-06T04:04:55.211 INFO:tasks.workunit.client.0.smithi157.stderr:+ echo OK 2023-10-06T04:04:55.212 INFO:tasks.workunit.client.0.smithi157.stdout:OK 2023-10-06T04:04:55.212 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-06T04:04:55.212 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-06T04:04:55.247 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-estale.sh... 2023-10-06T04:04:55.248 DEBUG:teuthology.orchestra.run.smithi157:workunit test fs/snaps/snaptest-estale.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-estale.sh 2023-10-06T04:04:56.429 INFO:tasks.workunit.client.0.smithi157.stderr:+ mkdir .snap/foo 2023-10-06T04:04:56.452 INFO:tasks.workunit.client.0.smithi157.stderr:+ echo 'We want ENOENT, not ESTALE, here.' 2023-10-06T04:04:56.452 INFO:tasks.workunit.client.0.smithi157.stdout:We want ENOENT, not ESTALE, here. 2023-10-06T04:04:56.453 INFO:tasks.workunit.client.0.smithi157.stderr:++ seq 1 100 2023-10-06T04:04:56.453 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.453 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/1 2023-10-06T04:04:56.453 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.455 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/1': No such file or directory 2023-10-06T04:04:56.455 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.456 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/2 2023-10-06T04:04:56.456 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.456 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/2': No such file or directory 2023-10-06T04:04:56.457 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.457 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/3 2023-10-06T04:04:56.457 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.457 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/3': No such file or directory 2023-10-06T04:04:56.458 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.458 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/4 2023-10-06T04:04:56.458 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.459 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/4': No such file or directory 2023-10-06T04:04:56.459 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.460 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/5 2023-10-06T04:04:56.460 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.460 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/5': No such file or directory 2023-10-06T04:04:56.460 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.461 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/6 2023-10-06T04:04:56.461 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.461 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/6': No such file or directory 2023-10-06T04:04:56.462 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.462 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/7 2023-10-06T04:04:56.462 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.462 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/7': No such file or directory 2023-10-06T04:04:56.463 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.463 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/8 2023-10-06T04:04:56.463 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.464 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/8': No such file or directory 2023-10-06T04:04:56.464 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.464 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/9 2023-10-06T04:04:56.464 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.465 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/9': No such file or directory 2023-10-06T04:04:56.466 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.466 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/10 2023-10-06T04:04:56.466 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.467 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/10': No such file or directory 2023-10-06T04:04:56.467 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.467 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/11 2023-10-06T04:04:56.467 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.468 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/11': No such file or directory 2023-10-06T04:04:56.468 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.468 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/12 2023-10-06T04:04:56.468 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.469 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/12': No such file or directory 2023-10-06T04:04:56.470 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.470 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/13 2023-10-06T04:04:56.470 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.470 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/13': No such file or directory 2023-10-06T04:04:56.471 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.471 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/14 2023-10-06T04:04:56.471 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.471 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/14': No such file or directory 2023-10-06T04:04:56.472 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.472 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/15 2023-10-06T04:04:56.472 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.473 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/15': No such file or directory 2023-10-06T04:04:56.473 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.473 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/16 2023-10-06T04:04:56.473 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.474 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/16': No such file or directory 2023-10-06T04:04:56.474 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.474 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/17 2023-10-06T04:04:56.475 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.475 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/17': No such file or directory 2023-10-06T04:04:56.475 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.476 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/18 2023-10-06T04:04:56.476 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.476 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/18': No such file or directory 2023-10-06T04:04:56.477 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.477 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/19 2023-10-06T04:04:56.477 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.477 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/19': No such file or directory 2023-10-06T04:04:56.478 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.478 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/20 2023-10-06T04:04:56.478 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.479 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/20': No such file or directory 2023-10-06T04:04:56.479 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.479 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/21 2023-10-06T04:04:56.479 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.480 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/21': No such file or directory 2023-10-06T04:04:56.481 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.481 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/22 2023-10-06T04:04:56.481 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.481 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/22': No such file or directory 2023-10-06T04:04:56.482 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.482 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/23 2023-10-06T04:04:56.482 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.482 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/23': No such file or directory 2023-10-06T04:04:56.483 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.483 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/24 2023-10-06T04:04:56.483 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.484 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/24': No such file or directory 2023-10-06T04:04:56.484 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.484 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/25 2023-10-06T04:04:56.484 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.485 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/25': No such file or directory 2023-10-06T04:04:56.485 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.485 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/26 2023-10-06T04:04:56.485 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.486 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/26': No such file or directory 2023-10-06T04:04:56.487 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.487 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/27 2023-10-06T04:04:56.487 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.487 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/27': No such file or directory 2023-10-06T04:04:56.488 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.488 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/28 2023-10-06T04:04:56.488 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.488 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/28': No such file or directory 2023-10-06T04:04:56.489 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.491 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/29 2023-10-06T04:04:56.491 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.492 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/29': No such file or directory 2023-10-06T04:04:56.492 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.492 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/30 2023-10-06T04:04:56.492 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.493 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/30': No such file or directory 2023-10-06T04:04:56.493 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.493 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/31 2023-10-06T04:04:56.493 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.494 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/31': No such file or directory 2023-10-06T04:04:56.494 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.494 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/32 2023-10-06T04:04:56.494 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.495 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/32': No such file or directory 2023-10-06T04:04:56.495 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.495 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/33 2023-10-06T04:04:56.495 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.496 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/33': No such file or directory 2023-10-06T04:04:56.496 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.496 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/34 2023-10-06T04:04:56.496 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.497 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/34': No such file or directory 2023-10-06T04:04:56.497 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.497 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/35 2023-10-06T04:04:56.498 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.498 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/35': No such file or directory 2023-10-06T04:04:56.499 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.499 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/36 2023-10-06T04:04:56.499 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.499 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/36': No such file or directory 2023-10-06T04:04:56.500 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.500 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/37 2023-10-06T04:04:56.500 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.500 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/37': No such file or directory 2023-10-06T04:04:56.501 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.501 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/38 2023-10-06T04:04:56.501 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.501 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/38': No such file or directory 2023-10-06T04:04:56.502 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.502 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/39 2023-10-06T04:04:56.502 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.503 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/39': No such file or directory 2023-10-06T04:04:56.503 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.503 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/40 2023-10-06T04:04:56.503 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.505 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/40': No such file or directory 2023-10-06T04:04:56.506 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.506 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/41 2023-10-06T04:04:56.506 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.506 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/41': No such file or directory 2023-10-06T04:04:56.507 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.507 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/42 2023-10-06T04:04:56.507 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.507 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/42': No such file or directory 2023-10-06T04:04:56.508 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.509 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/43 2023-10-06T04:04:56.509 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.509 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/43': No such file or directory 2023-10-06T04:04:56.510 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.510 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/44 2023-10-06T04:04:56.510 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.511 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/44': No such file or directory 2023-10-06T04:04:56.511 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.511 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/45 2023-10-06T04:04:56.511 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.512 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/45': No such file or directory 2023-10-06T04:04:56.512 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.513 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/46 2023-10-06T04:04:56.513 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.513 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/46': No such file or directory 2023-10-06T04:04:56.513 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.514 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/47 2023-10-06T04:04:56.514 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.514 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/47': No such file or directory 2023-10-06T04:04:56.515 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.515 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/48 2023-10-06T04:04:56.515 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.515 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/48': No such file or directory 2023-10-06T04:04:56.516 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.516 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/49 2023-10-06T04:04:56.516 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.516 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/49': No such file or directory 2023-10-06T04:04:56.517 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.517 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/50 2023-10-06T04:04:56.517 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.517 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/50': No such file or directory 2023-10-06T04:04:56.518 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.518 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/51 2023-10-06T04:04:56.518 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.519 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/51': No such file or directory 2023-10-06T04:04:56.519 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.519 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/52 2023-10-06T04:04:56.519 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.520 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/52': No such file or directory 2023-10-06T04:04:56.520 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.520 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/53 2023-10-06T04:04:56.520 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.521 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/53': No such file or directory 2023-10-06T04:04:56.521 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.521 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/54 2023-10-06T04:04:56.521 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.521 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/54': No such file or directory 2023-10-06T04:04:56.522 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.522 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/55 2023-10-06T04:04:56.522 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.523 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/55': No such file or directory 2023-10-06T04:04:56.523 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.523 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/56 2023-10-06T04:04:56.524 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.524 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/56': No such file or directory 2023-10-06T04:04:56.524 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.525 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/57 2023-10-06T04:04:56.525 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.525 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/57': No such file or directory 2023-10-06T04:04:56.525 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.525 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/58 2023-10-06T04:04:56.526 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.526 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/58': No such file or directory 2023-10-06T04:04:56.526 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.526 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/59 2023-10-06T04:04:56.527 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.527 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/59': No such file or directory 2023-10-06T04:04:56.527 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.527 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/60 2023-10-06T04:04:56.528 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.528 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/60': No such file or directory 2023-10-06T04:04:56.528 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.528 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/61 2023-10-06T04:04:56.528 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.529 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/61': No such file or directory 2023-10-06T04:04:56.530 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.530 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/62 2023-10-06T04:04:56.530 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.530 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/62': No such file or directory 2023-10-06T04:04:56.531 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.531 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/63 2023-10-06T04:04:56.531 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.531 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/63': No such file or directory 2023-10-06T04:04:56.532 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.532 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/64 2023-10-06T04:04:56.532 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.532 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/64': No such file or directory 2023-10-06T04:04:56.533 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.533 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/65 2023-10-06T04:04:56.533 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.533 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/65': No such file or directory 2023-10-06T04:04:56.534 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.534 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/66 2023-10-06T04:04:56.534 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.534 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/66': No such file or directory 2023-10-06T04:04:56.535 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.535 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/67 2023-10-06T04:04:56.535 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.535 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/67': No such file or directory 2023-10-06T04:04:56.536 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.536 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/68 2023-10-06T04:04:56.536 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.537 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/68': No such file or directory 2023-10-06T04:04:56.537 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.538 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/69 2023-10-06T04:04:56.538 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.538 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/69': No such file or directory 2023-10-06T04:04:56.539 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.539 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/70 2023-10-06T04:04:56.539 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.540 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/70': No such file or directory 2023-10-06T04:04:56.540 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.540 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/71 2023-10-06T04:04:56.541 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.541 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/71': No such file or directory 2023-10-06T04:04:56.542 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.542 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/72 2023-10-06T04:04:56.542 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.542 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/72': No such file or directory 2023-10-06T04:04:56.543 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.544 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/73 2023-10-06T04:04:56.544 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.544 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/73': No such file or directory 2023-10-06T04:04:56.545 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.545 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/74 2023-10-06T04:04:56.545 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.546 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/74': No such file or directory 2023-10-06T04:04:56.546 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.546 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/75 2023-10-06T04:04:56.547 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.547 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/75': No such file or directory 2023-10-06T04:04:56.548 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.548 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/76 2023-10-06T04:04:56.548 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.548 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/76': No such file or directory 2023-10-06T04:04:56.549 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.549 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/77 2023-10-06T04:04:56.549 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.550 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/77': No such file or directory 2023-10-06T04:04:56.550 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.551 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/78 2023-10-06T04:04:56.551 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.551 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/78': No such file or directory 2023-10-06T04:04:56.551 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.552 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/79 2023-10-06T04:04:56.552 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.552 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/79': No such file or directory 2023-10-06T04:04:56.553 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.553 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/80 2023-10-06T04:04:56.553 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.553 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/80': No such file or directory 2023-10-06T04:04:56.554 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.554 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/81 2023-10-06T04:04:56.554 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.554 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/81': No such file or directory 2023-10-06T04:04:56.555 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.555 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/82 2023-10-06T04:04:56.555 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.556 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/82': No such file or directory 2023-10-06T04:04:56.557 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.557 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/83 2023-10-06T04:04:56.557 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.557 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/83': No such file or directory 2023-10-06T04:04:56.558 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.559 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/84 2023-10-06T04:04:56.559 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.559 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/84': No such file or directory 2023-10-06T04:04:56.559 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.559 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/85 2023-10-06T04:04:56.560 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.560 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/85': No such file or directory 2023-10-06T04:04:56.560 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.560 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/86 2023-10-06T04:04:56.560 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.561 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/86': No such file or directory 2023-10-06T04:04:56.562 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.562 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/87 2023-10-06T04:04:56.562 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.562 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/87': No such file or directory 2023-10-06T04:04:56.563 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.563 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/88 2023-10-06T04:04:56.563 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.563 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/88': No such file or directory 2023-10-06T04:04:56.564 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.564 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/89 2023-10-06T04:04:56.564 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.564 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/89': No such file or directory 2023-10-06T04:04:56.565 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.565 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/90 2023-10-06T04:04:56.565 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.565 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/90': No such file or directory 2023-10-06T04:04:56.566 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.566 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/91 2023-10-06T04:04:56.566 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.566 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/91': No such file or directory 2023-10-06T04:04:56.567 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.567 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/92 2023-10-06T04:04:56.567 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.567 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/92': No such file or directory 2023-10-06T04:04:56.568 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.568 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/93 2023-10-06T04:04:56.568 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.568 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/93': No such file or directory 2023-10-06T04:04:56.569 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.569 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/94 2023-10-06T04:04:56.569 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.569 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/94': No such file or directory 2023-10-06T04:04:56.570 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.570 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/95 2023-10-06T04:04:56.570 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.570 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/95': No such file or directory 2023-10-06T04:04:56.571 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.571 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/96 2023-10-06T04:04:56.571 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.572 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/96': No such file or directory 2023-10-06T04:04:56.572 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.572 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/97 2023-10-06T04:04:56.572 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.573 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/97': No such file or directory 2023-10-06T04:04:56.573 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.573 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/98 2023-10-06T04:04:56.573 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.573 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/98': No such file or directory 2023-10-06T04:04:56.574 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.574 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/99 2023-10-06T04:04:56.574 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.575 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/99': No such file or directory 2023-10-06T04:04:56.575 INFO:tasks.workunit.client.0.smithi157.stderr:+ for f in `seq 1 100` 2023-10-06T04:04:56.575 INFO:tasks.workunit.client.0.smithi157.stderr:+ stat .snap/foo/100 2023-10-06T04:04:56.575 INFO:tasks.workunit.client.0.smithi157.stderr:+ grep 'No such file' 2023-10-06T04:04:56.576 INFO:tasks.workunit.client.0.smithi157.stdout:stat: cannot stat '.snap/foo/100': No such file or directory 2023-10-06T04:04:56.576 INFO:tasks.workunit.client.0.smithi157.stderr:+ rmdir .snap/foo 2023-10-06T04:04:56.576 INFO:tasks.workunit.client.0.smithi157.stdout:OK 2023-10-06T04:04:56.577 INFO:tasks.workunit.client.0.smithi157.stderr:+ echo OK 2023-10-06T04:04:56.578 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-06T04:04:56.578 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-06T04:04:56.606 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-git-ceph.sh... 2023-10-06T04:04:56.607 DEBUG:teuthology.orchestra.run.smithi157:workunit test fs/snaps/snaptest-git-ceph.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh 2023-10-06T04:04:57.140 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2023-10-06T04:04:57.140 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T04:04:57.140 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T04:04:57.143 INFO:tasks.ceph.mds.c:Started 2023-10-06T04:04:57.143 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:04:57.215 INFO:tasks.ceph.mds.c.smithi157.stdout:starting mds.c at 2023-10-06T04:04:57.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.259+0000 7f8c19d27700 1 Processor -- start 2023-10-06T04:04:57.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.259+0000 7f8c19d27700 1 -- start start 2023-10-06T04:04:57.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.260+0000 7f8c19d27700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8c1414e380 0x7f8c1406a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:57.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.260+0000 7f8c19d27700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1406b700 con 0x7f8c1406a580 2023-10-06T04:04:57.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.260+0000 7f8c19d27700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1406c630 con 0x7f8c1406c1f0 2023-10-06T04:04:57.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.260+0000 7f8c19d27700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c140c43d0 con 0x7f8c1414e380 2023-10-06T04:04:57.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.260+0000 7f8c19526700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8c1414e380 0x7f8c1406a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:57.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.260+0000 7f8c18d25700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f8c1406a580 0x7f8c140c3cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51592/0 (socket says 172.21.15.157:51592) 2023-10-06T04:04:57.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.260+0000 7f8c18d25700 1 -- 172.21.15.157:0/3193943564 learned_addr learned my addr 172.21.15.157:0/3193943564 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:04:57.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.260+0000 7f8c19526700 1 -- 172.21.15.157:0/3193943564 >> v1:172.21.15.175:6789/0 conn(0x7f8c1406c1f0 legacy=0x7f8c1414dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:57.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.260+0000 7f8c19526700 1 -- 172.21.15.157:0/3193943564 >> v1:172.21.15.157:6789/0 conn(0x7f8c1406a580 legacy=0x7f8c140c3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:57.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.260+0000 7f8c19526700 1 -- 172.21.15.157:0/3193943564 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1406b930 con 0x7f8c1414e380 2023-10-06T04:04:57.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.260+0000 7f8c137fe700 1 -- 172.21.15.157:0/3193943564 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1243074672 0 0) 0x7f8c1406b700 con 0x7f8c1406a580 2023-10-06T04:04:57.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.261+0000 7f8c19526700 1 --2- 172.21.15.157:0/3193943564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c1414e380 0x7f8c1406a950 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c009e60 tx=0x7f8c0c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3161531c13aa100b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:57.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.261+0000 7f8c137fe700 1 -- 172.21.15.157:0/3193943564 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c00e070 con 0x7f8c1414e380 2023-10-06T04:04:57.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.261+0000 7f8c137fe700 1 -- 172.21.15.157:0/3193943564 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0c00bc30 con 0x7f8c1414e380 2023-10-06T04:04:57.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.261+0000 7f8c137fe700 1 -- 172.21.15.157:0/3193943564 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c012430 con 0x7f8c1414e380 2023-10-06T04:04:57.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.261+0000 7f8c19d27700 1 -- 172.21.15.157:0/3193943564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c1414e380 msgr2=0x7f8c1406a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:57.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.261+0000 7f8c19d27700 1 --2- 172.21.15.157:0/3193943564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c1414e380 0x7f8c1406a950 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c009e60 tx=0x7f8c0c009780 comp rx=0 tx=0).stop 2023-10-06T04:04:57.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.261+0000 7f8c19d27700 1 -- 172.21.15.157:0/3193943564 shutdown_connections 2023-10-06T04:04:57.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.261+0000 7f8c19d27700 1 --2- 172.21.15.157:0/3193943564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c1414e380 0x7f8c1406a950 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:57.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.261+0000 7f8c19d27700 1 -- 172.21.15.157:0/3193943564 >> 172.21.15.157:0/3193943564 conn(0x7f8c14068020 msgr2=0x7f8c14068420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:57.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.261+0000 7f8c19d27700 1 -- 172.21.15.157:0/3193943564 shutdown_connections 2023-10-06T04:04:57.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.261+0000 7f8c19d27700 1 -- 172.21.15.157:0/3193943564 wait complete. 2023-10-06T04:04:57.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.262+0000 7f8c19d27700 1 Processor -- start 2023-10-06T04:04:57.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.263+0000 7f8c19d27700 1 -- start start 2023-10-06T04:04:57.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.263+0000 7f8c19d27700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c1406a580 0x7f8c140be980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:57.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.263+0000 7f8c19d27700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c1406c1f0 0x7f8c1414c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:57.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.263+0000 7f8c19d27700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c1414e380 0x7f8c140c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:57.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.263+0000 7f8c19d27700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c140c8540 con 0x7f8c1406c1f0 2023-10-06T04:04:57.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.263+0000 7f8c19d27700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c14169ac0 con 0x7f8c1414e380 2023-10-06T04:04:57.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.263+0000 7f8c19d27700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c14169da0 con 0x7f8c1406a580 2023-10-06T04:04:57.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.263+0000 7f8c13fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c1406c1f0 0x7f8c1414c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:57.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.263+0000 7f8c13fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c1406c1f0 0x7f8c1414c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:48940/0 (socket says 172.21.15.157:48940) 2023-10-06T04:04:57.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.263+0000 7f8c13fff700 1 -- 172.21.15.157:0/414291488 learned_addr learned my addr 172.21.15.157:0/414291488 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:57.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.264+0000 7f8c18d25700 1 --2- 172.21.15.157:0/414291488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c1406a580 0x7f8c140be980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:57.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.264+0000 7f8c19526700 1 --2- 172.21.15.157:0/414291488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c1414e380 0x7f8c140c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:57.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.264+0000 7f8c13fff700 1 -- 172.21.15.157:0/414291488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c1406a580 msgr2=0x7f8c140be980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:57.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.264+0000 7f8c13fff700 1 --2- 172.21.15.157:0/414291488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c1406a580 0x7f8c140be980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:57.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.264+0000 7f8c13fff700 1 -- 172.21.15.157:0/414291488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c1414e380 msgr2=0x7f8c140c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:57.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.264+0000 7f8c13fff700 1 --2- 172.21.15.157:0/414291488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c1414e380 0x7f8c140c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:57.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.264+0000 7f8c13fff700 1 -- 172.21.15.157:0/414291488 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c0c009a20 con 0x7f8c1406c1f0 2023-10-06T04:04:57.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.264+0000 7f8c13fff700 1 --2- 172.21.15.157:0/414291488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c1406c1f0 0x7f8c1414c290 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000c590 tx=0x7f8c0000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:57.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.264+0000 7f8c10ff9700 1 -- 172.21.15.157:0/414291488 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00011590 con 0x7f8c1406c1f0 2023-10-06T04:04:57.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.264+0000 7f8c10ff9700 1 -- 172.21.15.157:0/414291488 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c00011710 con 0x7f8c1406c1f0 2023-10-06T04:04:57.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.264+0000 7f8c19d27700 1 -- 172.21.15.157:0/414291488 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1416a080 con 0x7f8c1406c1f0 2023-10-06T04:04:57.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.264+0000 7f8c10ff9700 1 -- 172.21.15.157:0/414291488 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00011ad0 con 0x7f8c1406c1f0 2023-10-06T04:04:57.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.264+0000 7f8c19d27700 1 -- 172.21.15.157:0/414291488 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c140c81d0 con 0x7f8c1406c1f0 2023-10-06T04:04:57.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.265+0000 7f8c19d27700 1 -- 172.21.15.157:0/414291488 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c140c5950 con 0x7f8c1406c1f0 2023-10-06T04:04:57.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.265+0000 7f8c10ff9700 1 -- 172.21.15.157:0/414291488 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8c0001b3b0 con 0x7f8c1406c1f0 2023-10-06T04:04:57.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.266+0000 7f8c10ff9700 1 --2- 172.21.15.157:0/414291488 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8bf8066f30 0x7f8bf80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:57.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.266+0000 7f8c10ff9700 1 -- 172.21.15.157:0/414291488 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f8c00083ce0 con 0x7f8c1406c1f0 2023-10-06T04:04:57.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.266+0000 7f8c18d25700 1 --2- 172.21.15.157:0/414291488 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8bf8066f30 0x7f8bf80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:57.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.266+0000 7f8c18d25700 1 --2- 172.21.15.157:0/414291488 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8bf8066f30 0x7f8bf80693e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8c08007fd0 tx=0x7f8c08007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:57.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.270+0000 7f8c10ff9700 1 -- 172.21.15.157:0/414291488 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c00050c60 con 0x7f8c1406c1f0 2023-10-06T04:04:57.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.499+0000 7f8c19d27700 1 -- 172.21.15.157:0/414291488 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c1404bb40 con 0x7f8c1406c1f0 2023-10-06T04:04:57.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.500+0000 7f8c10ff9700 1 -- 172.21.15.157:0/414291488 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+5388 (secure 0 0 0) 0x7f8c00017070 con 0x7f8c1406c1f0 2023-10-06T04:04:57.501 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:04:57.501 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:04:48.805873+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:04:57.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.502+0000 7f8c19d27700 1 -- 172.21.15.157:0/414291488 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8bf8066f30 msgr2=0x7f8bf80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:57.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.502+0000 7f8c19d27700 1 --2- 172.21.15.157:0/414291488 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8bf8066f30 0x7f8bf80693e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8c08007fd0 tx=0x7f8c08007d30 comp rx=0 tx=0).stop 2023-10-06T04:04:57.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.502+0000 7f8c19d27700 1 -- 172.21.15.157:0/414291488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c1406c1f0 msgr2=0x7f8c1414c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:57.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.502+0000 7f8c19d27700 1 --2- 172.21.15.157:0/414291488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c1406c1f0 0x7f8c1414c290 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000c590 tx=0x7f8c0000c8a0 comp rx=0 tx=0).stop 2023-10-06T04:04:57.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.502+0000 7f8c19d27700 1 -- 172.21.15.157:0/414291488 shutdown_connections 2023-10-06T04:04:57.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.502+0000 7f8c19d27700 1 --2- 172.21.15.157:0/414291488 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8bf8066f30 0x7f8bf80693e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:57.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.502+0000 7f8c19d27700 1 --2- 172.21.15.157:0/414291488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c1414e380 0x7f8c140c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:57.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.502+0000 7f8c19d27700 1 --2- 172.21.15.157:0/414291488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c1406a580 0x7f8c140be980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:57.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.502+0000 7f8c19d27700 1 --2- 172.21.15.157:0/414291488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c1406c1f0 0x7f8c1414c290 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:57.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.502+0000 7f8c19d27700 1 -- 172.21.15.157:0/414291488 >> 172.21.15.157:0/414291488 conn(0x7f8c14068020 msgr2=0x7f8c1414d010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:57.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.502+0000 7f8c19d27700 1 -- 172.21.15.157:0/414291488 shutdown_connections 2023-10-06T04:04:57.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:57.502+0000 7f8c19d27700 1 -- 172.21.15.157:0/414291488 wait complete. 2023-10-06T04:04:57.505 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 62 2023-10-06T04:04:57.515 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2023-10-06T04:04:59.516 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:04:59.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.643+0000 7f949a698700 1 Processor -- start 2023-10-06T04:04:59.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.643+0000 7f949a698700 1 -- start start 2023-10-06T04:04:59.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.644+0000 7f949a698700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94940be970 0x7f949406df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:59.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.644+0000 7f949a698700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9494068d10 0x7f949415d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:59.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.644+0000 7f949a698700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94940bed40 con 0x7f949406e450 2023-10-06T04:04:59.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.644+0000 7f949a698700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9494157010 con 0x7f9494068d10 2023-10-06T04:04:59.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.644+0000 7f949a698700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949406f9f0 con 0x7f94940be970 2023-10-06T04:04:59.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.644+0000 7f94937fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f949406e450 0x7f949406a600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51594/0 (socket says 172.21.15.157:51594) 2023-10-06T04:04:59.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.644+0000 7f94937fe700 1 -- 172.21.15.157:0/263257970 learned_addr learned my addr 172.21.15.157:0/263257970 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:04:59.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.644+0000 7f9498c35700 1 --2- 172.21.15.157:0/263257970 >> v2:172.21.15.175:3300/0 conn(0x7f9494068d10 0x7f949415d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:59.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.644+0000 7f9493fff700 1 --2- 172.21.15.157:0/263257970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94940be970 0x7f949406df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:59.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.644+0000 7f9492ffd700 1 -- 172.21.15.157:0/263257970 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3426725494 0 0) 0x7f94940bed40 con 0x7f949406e450 2023-10-06T04:04:59.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.644+0000 7f9492ffd700 1 -- 172.21.15.157:0/263257970 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f947c003570 con 0x7f949406e450 2023-10-06T04:04:59.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.644+0000 7f9498c35700 1 -- 172.21.15.157:0/263257970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94940be970 msgr2=0x7f949406df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:04:59.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.644+0000 7f9498c35700 1 --2- 172.21.15.157:0/263257970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94940be970 0x7f949406df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:59.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.644+0000 7f9498c35700 1 -- 172.21.15.157:0/263257970 >> v1:172.21.15.157:6789/0 conn(0x7f949406e450 legacy=0x7f949406a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:59.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.644+0000 7f9498c35700 1 -- 172.21.15.157:0/263257970 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94941574a0 con 0x7f9494068d10 2023-10-06T04:04:59.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.644+0000 7f9493fff700 1 --2- 172.21.15.157:0/263257970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94940be970 0x7f949406df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:04:59.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.645+0000 7f9498c35700 1 --2- 172.21.15.157:0/263257970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9494068d10 0x7f949415d9e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f948c009e60 tx=0x7f948c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d881b846f6cf0818 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:59.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.645+0000 7f9492ffd700 1 -- 172.21.15.157:0/263257970 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c00e070 con 0x7f9494068d10 2023-10-06T04:04:59.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.645+0000 7f9492ffd700 1 -- 172.21.15.157:0/263257970 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948c00bab0 con 0x7f9494068d10 2023-10-06T04:04:59.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.645+0000 7f9492ffd700 1 -- 172.21.15.157:0/263257970 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c00be30 con 0x7f9494068d10 2023-10-06T04:04:59.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.645+0000 7f949a698700 1 -- 172.21.15.157:0/263257970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9494068d10 msgr2=0x7f949415d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:59.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.645+0000 7f949a698700 1 --2- 172.21.15.157:0/263257970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9494068d10 0x7f949415d9e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f948c009e60 tx=0x7f948c009780 comp rx=0 tx=0).stop 2023-10-06T04:04:59.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.645+0000 7f949a698700 1 -- 172.21.15.157:0/263257970 shutdown_connections 2023-10-06T04:04:59.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.645+0000 7f949a698700 1 --2- 172.21.15.157:0/263257970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9494068d10 0x7f949415d9e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:59.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.645+0000 7f949a698700 1 --2- 172.21.15.157:0/263257970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94940be970 0x7f949406df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:59.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.645+0000 7f949a698700 1 -- 172.21.15.157:0/263257970 >> 172.21.15.157:0/263257970 conn(0x7f94940c8d70 msgr2=0x7f94940c4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:59.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.645+0000 7f949a698700 1 -- 172.21.15.157:0/263257970 shutdown_connections 2023-10-06T04:04:59.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.645+0000 7f949a698700 1 -- 172.21.15.157:0/263257970 wait complete. 2023-10-06T04:04:59.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.646+0000 7f949a698700 1 Processor -- start 2023-10-06T04:04:59.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.647+0000 7f949a698700 1 -- start start 2023-10-06T04:04:59.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.647+0000 7f949a698700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9494068d10 0x7f94941674d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:59.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.647+0000 7f949a698700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f949406e450 0x7f949414fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:59.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.647+0000 7f949a698700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94940be970 0x7f94940694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:59.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.647+0000 7f949a698700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9494158f10 con 0x7f94940be970 2023-10-06T04:04:59.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.647+0000 7f949a698700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94940bfa80 con 0x7f949406e450 2023-10-06T04:04:59.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.647+0000 7f949a698700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9494151fa0 con 0x7f9494068d10 2023-10-06T04:04:59.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.647+0000 7f9498c35700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94940be970 0x7f94940694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:59.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.648+0000 7f9498c35700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94940be970 0x7f94940694f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:48952/0 (socket says 172.21.15.157:48952) 2023-10-06T04:04:59.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.648+0000 7f9498c35700 1 -- 172.21.15.157:0/3717401031 learned_addr learned my addr 172.21.15.157:0/3717401031 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:04:59.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.648+0000 7f9493fff700 1 --2- 172.21.15.157:0/3717401031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9494068d10 0x7f94941674d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:59.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.648+0000 7f94937fe700 1 --2- 172.21.15.157:0/3717401031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f949406e450 0x7f949414fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:59.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.648+0000 7f9498c35700 1 -- 172.21.15.157:0/3717401031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9494068d10 msgr2=0x7f94941674d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:59.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.648+0000 7f9498c35700 1 --2- 172.21.15.157:0/3717401031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9494068d10 0x7f94941674d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:59.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.648+0000 7f9498c35700 1 -- 172.21.15.157:0/3717401031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f949406e450 msgr2=0x7f949414fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:59.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.648+0000 7f9498c35700 1 --2- 172.21.15.157:0/3717401031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f949406e450 0x7f949414fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:59.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.648+0000 7f9498c35700 1 -- 172.21.15.157:0/3717401031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9484009420 con 0x7f94940be970 2023-10-06T04:04:59.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.648+0000 7f94937fe700 1 --2- 172.21.15.157:0/3717401031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f949406e450 0x7f949414fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:04:59.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.648+0000 7f9498c35700 1 --2- 172.21.15.157:0/3717401031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94940be970 0x7f94940694f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9494068050 tx=0x7f948c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:59.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.648+0000 7f947bfff700 1 -- 172.21.15.157:0/3717401031 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c012e60 con 0x7f94940be970 2023-10-06T04:04:59.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.648+0000 7f947bfff700 1 -- 172.21.15.157:0/3717401031 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948c027470 con 0x7f94940be970 2023-10-06T04:04:59.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.648+0000 7f949a698700 1 -- 172.21.15.157:0/3717401031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f948c009a20 con 0x7f94940be970 2023-10-06T04:04:59.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.648+0000 7f947bfff700 1 -- 172.21.15.157:0/3717401031 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c027830 con 0x7f94940be970 2023-10-06T04:04:59.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.648+0000 7f949a698700 1 -- 172.21.15.157:0/3717401031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94940bf430 con 0x7f94940be970 2023-10-06T04:04:59.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.649+0000 7f949a698700 1 -- 172.21.15.157:0/3717401031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94940716b0 con 0x7f94940be970 2023-10-06T04:04:59.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.650+0000 7f947bfff700 1 -- 172.21.15.157:0/3717401031 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f948c024070 con 0x7f94940be970 2023-10-06T04:04:59.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.650+0000 7f947bfff700 1 --2- 172.21.15.157:0/3717401031 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9480066f60 0x7f9480069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:04:59.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.650+0000 7f947bfff700 1 -- 172.21.15.157:0/3717401031 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f948c087370 con 0x7f94940be970 2023-10-06T04:04:59.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.650+0000 7f9493fff700 1 --2- 172.21.15.157:0/3717401031 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9480066f60 0x7f9480069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:04:59.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.651+0000 7f9493fff700 1 --2- 172.21.15.157:0/3717401031 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9480066f60 0x7f9480069410 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9484009f40 tx=0x7f94840091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:04:59.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.654+0000 7f947bfff700 1 -- 172.21.15.157:0/3717401031 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f948c0542f0 con 0x7f94940be970 2023-10-06T04:04:59.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.881+0000 7f949a698700 1 -- 172.21.15.157:0/3717401031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94940c9170 con 0x7f94940be970 2023-10-06T04:04:59.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.882+0000 7f947bfff700 1 -- 172.21.15.157:0/3717401031 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6145 (secure 0 0 0) 0x7f948c057910 con 0x7f94940be970 2023-10-06T04:04:59.883 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:04:59.883 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5799,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:04:48.805873+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:04:59.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.883+0000 7f949a698700 1 -- 172.21.15.157:0/3717401031 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9480066f60 msgr2=0x7f9480069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:59.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.883+0000 7f949a698700 1 --2- 172.21.15.157:0/3717401031 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9480066f60 0x7f9480069410 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9484009f40 tx=0x7f94840091a0 comp rx=0 tx=0).stop 2023-10-06T04:04:59.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.883+0000 7f949a698700 1 -- 172.21.15.157:0/3717401031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94940be970 msgr2=0x7f94940694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:04:59.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.883+0000 7f949a698700 1 --2- 172.21.15.157:0/3717401031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94940be970 0x7f94940694f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9494068050 tx=0x7f948c019000 comp rx=0 tx=0).stop 2023-10-06T04:04:59.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.884+0000 7f949a698700 1 -- 172.21.15.157:0/3717401031 shutdown_connections 2023-10-06T04:04:59.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.884+0000 7f949a698700 1 --2- 172.21.15.157:0/3717401031 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9480066f60 0x7f9480069410 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:59.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.884+0000 7f949a698700 1 --2- 172.21.15.157:0/3717401031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94940be970 0x7f94940694f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:59.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.884+0000 7f949a698700 1 --2- 172.21.15.157:0/3717401031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9494068d10 0x7f94941674d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:59.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.884+0000 7f949a698700 1 --2- 172.21.15.157:0/3717401031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f949406e450 0x7f949414fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:04:59.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.884+0000 7f949a698700 1 -- 172.21.15.157:0/3717401031 >> 172.21.15.157:0/3717401031 conn(0x7f94940c8d70 msgr2=0x7f94940c7210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:04:59.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.884+0000 7f949a698700 1 -- 172.21.15.157:0/3717401031 shutdown_connections 2023-10-06T04:04:59.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:04:59.884+0000 7f949a698700 1 -- 172.21.15.157:0/3717401031 wait complete. 2023-10-06T04:04:59.887 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 63 2023-10-06T04:04:59.897 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2023-10-06T04:04:59.897 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 9 secs before thrashing 2023-10-06T04:05:01.429 INFO:tasks.workunit.client.0.smithi157.stderr:+ set -e 2023-10-06T04:05:01.430 INFO:tasks.workunit.client.0.smithi157.stderr:+ retried=false 2023-10-06T04:05:01.430 INFO:tasks.workunit.client.0.smithi157.stderr:+ trap -- retry EXIT 2023-10-06T04:05:01.430 INFO:tasks.workunit.client.0.smithi157.stderr:+ rm -rf ceph 2023-10-06T04:05:01.430 INFO:tasks.workunit.client.0.smithi157.stderr:+ timeout 1800 git clone https://git.ceph.com/ceph.git 2023-10-06T04:05:01.432 INFO:tasks.workunit.client.0.smithi157.stderr:Cloning into 'ceph'... 2023-10-06T04:05:08.898 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:05:09.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.024+0000 7f69213f2700 1 Processor -- start 2023-10-06T04:05:09.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.024+0000 7f69213f2700 1 -- start start 2023-10-06T04:05:09.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.025+0000 7f69213f2700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f691c06f960 0x7f691c152f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:09.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.025+0000 7f69213f2700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f691c0c3560 con 0x7f691c0c5060 2023-10-06T04:05:09.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.025+0000 7f69213f2700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f691c0c55c0 con 0x7f691c0c4c90 2023-10-06T04:05:09.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.025+0000 7f69213f2700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f691c153490 con 0x7f691c06f960 2023-10-06T04:05:09.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.025+0000 7f691b7fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f691c0c5060 0x7f691c14bdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:56250/0 (socket says 172.21.15.157:56250) 2023-10-06T04:05:09.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.025+0000 7f691b7fe700 1 -- 172.21.15.157:0/3906412291 learned_addr learned my addr 172.21.15.157:0/3906412291 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:05:09.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.025+0000 7f69127fc700 1 --2- 172.21.15.157:0/3906412291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f691c06f960 0x7f691c152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:09.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.025+0000 7f691a7fc700 1 -- 172.21.15.157:0/3906412291 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3171936026 0 0) 0x7f691c0c3560 con 0x7f691c0c5060 2023-10-06T04:05:09.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.025+0000 7f691a7fc700 1 -- 172.21.15.157:0/3906412291 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6908003570 con 0x7f691c0c5060 2023-10-06T04:05:09.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.026+0000 7f691a7fc700 1 -- 172.21.15.157:0/3906412291 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3408654855 0 0) 0x7f691c0c55c0 con 0x7f691c0c4c90 2023-10-06T04:05:09.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.026+0000 7f691a7fc700 1 -- 172.21.15.157:0/3906412291 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f691c0c3560 con 0x7f691c0c4c90 2023-10-06T04:05:09.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.026+0000 7f691a7fc700 1 -- 172.21.15.157:0/3906412291 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3556559075 0 0) 0x7f6908003570 con 0x7f691c0c5060 2023-10-06T04:05:09.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.026+0000 7f691a7fc700 1 -- 172.21.15.157:0/3906412291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f691c06f960 msgr2=0x7f691c152f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:09.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.026+0000 7f691a7fc700 1 --2- 172.21.15.157:0/3906412291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f691c06f960 0x7f691c152f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:09.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.026+0000 7f691a7fc700 1 -- 172.21.15.157:0/3906412291 >> v1:172.21.15.175:6789/0 conn(0x7f691c0c4c90 legacy=0x7f691c0c39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:09.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.026+0000 7f691a7fc700 1 -- 172.21.15.157:0/3906412291 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f691c0ade20 con 0x7f691c0c5060 2023-10-06T04:05:09.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.026+0000 7f691a7fc700 1 -- 172.21.15.157:0/3906412291 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f69140017f0 con 0x7f691c0c5060 2023-10-06T04:05:09.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.026+0000 7f691a7fc700 1 -- 172.21.15.157:0/3906412291 >> v1:172.21.15.157:6789/0 conn(0x7f691c0c5060 legacy=0x7f691c14bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:09.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.026+0000 7f691a7fc700 1 --2- 172.21.15.157:0/3906412291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6908003c20 0x7f6908004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:09.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.026+0000 7f691a7fc700 1 --2- 172.21.15.157:0/3906412291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f691c06f960 0x7f6908004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:09.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.026+0000 7f691a7fc700 1 --2- 172.21.15.157:0/3906412291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6908004d50 0x7f69080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:09.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.026+0000 7f691a7fc700 1 -- 172.21.15.157:0/3906412291 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691c153490 con 0x7f6908004d50 2023-10-06T04:05:09.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.026+0000 7f691a7fc700 1 -- 172.21.15.157:0/3906412291 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690800bbf0 con 0x7f6908003c20 2023-10-06T04:05:09.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.026+0000 7f691a7fc700 1 -- 172.21.15.157:0/3906412291 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690800cec0 con 0x7f691c06f960 2023-10-06T04:05:09.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.026+0000 7f691b7fe700 1 --2- 172.21.15.157:0/3906412291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6908004d50 0x7f69080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:09.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.026+0000 7f691affd700 1 --2- 172.21.15.157:0/3906412291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6908003c20 0x7f6908004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:09.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.026+0000 7f69127fc700 1 --2- 172.21.15.157:0/3906412291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f691c06f960 0x7f6908004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:09.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.026+0000 7f691b7fe700 1 -- 172.21.15.157:0/3906412291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f691c06f960 msgr2=0x7f6908004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:09.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.026+0000 7f691b7fe700 1 --2- 172.21.15.157:0/3906412291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f691c06f960 0x7f6908004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:09.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.026+0000 7f691b7fe700 1 -- 172.21.15.157:0/3906412291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6908003c20 msgr2=0x7f6908004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:09.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.026+0000 7f691b7fe700 1 --2- 172.21.15.157:0/3906412291 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6908003c20 0x7f6908004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:09.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.026+0000 7f691b7fe700 1 -- 172.21.15.157:0/3906412291 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6908003570 con 0x7f6908004d50 2023-10-06T04:05:09.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.026+0000 7f691b7fe700 1 --2- 172.21.15.157:0/3906412291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6908004d50 0x7f69080051c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f691400a7d0 tx=0x7f69140020c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3508ef1e66f27404 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:09.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.026+0000 7f691a7fc700 1 -- 172.21.15.157:0/3906412291 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69140070d0 con 0x7f6908004d50 2023-10-06T04:05:09.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.026+0000 7f691a7fc700 1 -- 172.21.15.157:0/3906412291 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6914007230 con 0x7f6908004d50 2023-10-06T04:05:09.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.026+0000 7f691a7fc700 1 -- 172.21.15.157:0/3906412291 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69140075d0 con 0x7f6908004d50 2023-10-06T04:05:09.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.027+0000 7f69213f2700 1 -- 172.21.15.157:0/3906412291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6908004d50 msgr2=0x7f69080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:09.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.027+0000 7f69213f2700 1 --2- 172.21.15.157:0/3906412291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6908004d50 0x7f69080051c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f691400a7d0 tx=0x7f69140020c0 comp rx=0 tx=0).stop 2023-10-06T04:05:09.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.027+0000 7f691b7fe700 1 -- 172.21.15.157:0/3906412291 reap_dead start 2023-10-06T04:05:09.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.027+0000 7f69213f2700 1 -- 172.21.15.157:0/3906412291 shutdown_connections 2023-10-06T04:05:09.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.027+0000 7f69213f2700 1 -- 172.21.15.157:0/3906412291 >> 172.21.15.157:0/3906412291 conn(0x7f691c06eb10 msgr2=0x7f691c06c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:09.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.027+0000 7f69213f2700 1 -- 172.21.15.157:0/3906412291 shutdown_connections 2023-10-06T04:05:09.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.027+0000 7f69213f2700 1 -- 172.21.15.157:0/3906412291 wait complete. 2023-10-06T04:05:09.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.028+0000 7f69213f2700 1 Processor -- start 2023-10-06T04:05:09.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.028+0000 7f69213f2700 1 -- start start 2023-10-06T04:05:09.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.029+0000 7f69213f2700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f691c143420 0x7f691c1648e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:09.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.029+0000 7f69213f2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f691c150f00 0x7f691c0bdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:09.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.029+0000 7f69213f2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f691c151610 0x7f691c0be3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:09.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.029+0000 7f69213f2700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691c14ed20 con 0x7f691c150f00 2023-10-06T04:05:09.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.029+0000 7f69213f2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691c152110 con 0x7f691c151610 2023-10-06T04:05:09.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.029+0000 7f69213f2700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f691c14c800 con 0x7f691c143420 2023-10-06T04:05:09.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.029+0000 7f69127fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f691c150f00 0x7f691c0bdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:09.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.029+0000 7f69127fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f691c150f00 0x7f691c0bdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60424/0 (socket says 172.21.15.157:60424) 2023-10-06T04:05:09.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.029+0000 7f69127fc700 1 -- 172.21.15.157:0/931176968 learned_addr learned my addr 172.21.15.157:0/931176968 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:09.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.029+0000 7f691b7fe700 1 --2- 172.21.15.157:0/931176968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f691c151610 0x7f691c0be3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:09.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.029+0000 7f691affd700 1 --2- 172.21.15.157:0/931176968 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f691c143420 0x7f691c1648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:09.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.029+0000 7f69127fc700 1 -- 172.21.15.157:0/931176968 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f691c143420 msgr2=0x7f691c1648e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:09.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.029+0000 7f69127fc700 1 --2- 172.21.15.157:0/931176968 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f691c143420 0x7f691c1648e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:09.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.029+0000 7f69127fc700 1 -- 172.21.15.157:0/931176968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f691c151610 msgr2=0x7f691c0be3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:09.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.029+0000 7f69127fc700 1 --2- 172.21.15.157:0/931176968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f691c151610 0x7f691c0be3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:09.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.029+0000 7f69127fc700 1 -- 172.21.15.157:0/931176968 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f691c14fdd0 con 0x7f691c150f00 2023-10-06T04:05:09.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.030+0000 7f69127fc700 1 --2- 172.21.15.157:0/931176968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f691c150f00 0x7f691c0bdeb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6904006ac0 tx=0x7f69040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:09.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.030+0000 7f69137fe700 1 -- 172.21.15.157:0/931176968 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690400f9d0 con 0x7f691c150f00 2023-10-06T04:05:09.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.030+0000 7f69137fe700 1 -- 172.21.15.157:0/931176968 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690400fb30 con 0x7f691c150f00 2023-10-06T04:05:09.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.030+0000 7f69213f2700 1 -- 172.21.15.157:0/931176968 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f691c168160 con 0x7f691c150f00 2023-10-06T04:05:09.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.030+0000 7f69137fe700 1 -- 172.21.15.157:0/931176968 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690400feb0 con 0x7f691c150f00 2023-10-06T04:05:09.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.030+0000 7f69213f2700 1 -- 172.21.15.157:0/931176968 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f691c1683e0 con 0x7f691c150f00 2023-10-06T04:05:09.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.031+0000 7f69213f2700 1 -- 172.21.15.157:0/931176968 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f691c06f5d0 con 0x7f691c150f00 2023-10-06T04:05:09.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.031+0000 7f69137fe700 1 -- 172.21.15.157:0/931176968 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6904014650 con 0x7f691c150f00 2023-10-06T04:05:09.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.032+0000 7f69137fe700 1 --2- 172.21.15.157:0/931176968 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f68fc066f30 0x7f68fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:09.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.032+0000 7f69137fe700 1 -- 172.21.15.157:0/931176968 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6430+0+0 (secure 0 0 0) 0x7f6904018030 con 0x7f691c150f00 2023-10-06T04:05:09.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.032+0000 7f691affd700 1 --2- 172.21.15.157:0/931176968 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f68fc066f30 0x7f68fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:09.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.032+0000 7f691affd700 1 --2- 172.21.15.157:0/931176968 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f68fc066f30 0x7f68fc0693e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f690c003cb0 tx=0x7f690c007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:09.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.036+0000 7f69137fe700 1 -- 172.21.15.157:0/931176968 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f69040521f0 con 0x7f691c150f00 2023-10-06T04:05:09.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.262+0000 7f69213f2700 1 -- 172.21.15.157:0/931176968 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f691c04bb40 con 0x7f691c150f00 2023-10-06T04:05:09.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.263+0000 7f69137fe700 1 -- 172.21.15.157:0/931176968 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6145 (secure 0 0 0) 0x7f6904055810 con 0x7f691c150f00 2023-10-06T04:05:09.265 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:05:09.265 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5799,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:04:48.805873+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:05:09.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.265+0000 7f69213f2700 1 -- 172.21.15.157:0/931176968 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f68fc066f30 msgr2=0x7f68fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:09.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.265+0000 7f69213f2700 1 --2- 172.21.15.157:0/931176968 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f68fc066f30 0x7f68fc0693e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f690c003cb0 tx=0x7f690c007d30 comp rx=0 tx=0).stop 2023-10-06T04:05:09.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.265+0000 7f69213f2700 1 -- 172.21.15.157:0/931176968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f691c150f00 msgr2=0x7f691c0bdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:09.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.265+0000 7f69213f2700 1 --2- 172.21.15.157:0/931176968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f691c150f00 0x7f691c0bdeb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6904006ac0 tx=0x7f69040091a0 comp rx=0 tx=0).stop 2023-10-06T04:05:09.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.265+0000 7f69213f2700 1 -- 172.21.15.157:0/931176968 shutdown_connections 2023-10-06T04:05:09.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.265+0000 7f69213f2700 1 --2- 172.21.15.157:0/931176968 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f68fc066f30 0x7f68fc0693e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:09.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.265+0000 7f69213f2700 1 --2- 172.21.15.157:0/931176968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f691c151610 0x7f691c0be3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:09.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.265+0000 7f69213f2700 1 --2- 172.21.15.157:0/931176968 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f691c143420 0x7f691c1648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:09.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.265+0000 7f69213f2700 1 --2- 172.21.15.157:0/931176968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f691c150f00 0x7f691c0bdeb0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:09.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.265+0000 7f69213f2700 1 -- 172.21.15.157:0/931176968 >> 172.21.15.157:0/931176968 conn(0x7f691c06eb10 msgr2=0x7f691c146670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:09.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.265+0000 7f69213f2700 1 -- 172.21.15.157:0/931176968 shutdown_connections 2023-10-06T04:05:09.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.265+0000 7f69213f2700 1 -- 172.21.15.157:0/931176968 wait complete. 2023-10-06T04:05:09.268 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 63 2023-10-06T04:05:09.278 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.e (rank=0) 2023-10-06T04:05:09.279 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2023-10-06T04:05:09.279 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:05:09.290 INFO:tasks.ceph.mds.e:Stopped 2023-10-06T04:05:09.290 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:05:09.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.444+0000 7fa3d2cea700 1 Processor -- start 2023-10-06T04:05:09.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.444+0000 7fa3d2cea700 1 -- start start 2023-10-06T04:05:09.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.445+0000 7fa3d2cea700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3cc0b04d0 0x7fa3cc157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:09.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.445+0000 7fa3d2cea700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3cc06d8d0 con 0x7fa3cc0b1e40 2023-10-06T04:05:09.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.445+0000 7fa3d2cea700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3cc0af8f0 con 0x7fa3cc06c2d0 2023-10-06T04:05:09.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.445+0000 7fa3d2cea700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc15a690 con 0x7fa3cc0b04d0 2023-10-06T04:05:09.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.445+0000 7fa3d1ce8700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa3cc0b1e40 0x7fa3cc1563b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:56262/0 (socket says 172.21.15.157:56262) 2023-10-06T04:05:09.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.445+0000 7fa3d1ce8700 1 -- 172.21.15.157:0/1293640892 learned_addr learned my addr 172.21.15.157:0/1293640892 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:05:09.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.445+0000 7fa3d14e7700 1 --2- 172.21.15.157:0/1293640892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3cc0b04d0 0x7fa3cc157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:09.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.445+0000 7fa3d0ce6700 1 -- 172.21.15.157:0/1293640892 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2818313163 0 0) 0x7fa3cc06d8d0 con 0x7fa3cc0b1e40 2023-10-06T04:05:09.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.445+0000 7fa3d0ce6700 1 -- 172.21.15.157:0/1293640892 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3b4003570 con 0x7fa3cc0b1e40 2023-10-06T04:05:09.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.445+0000 7fa3d0ce6700 1 -- 172.21.15.157:0/1293640892 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 242135944 0 0) 0x7fa3cc0af8f0 con 0x7fa3cc06c2d0 2023-10-06T04:05:09.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.445+0000 7fa3d0ce6700 1 -- 172.21.15.157:0/1293640892 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3cc06d8d0 con 0x7fa3cc06c2d0 2023-10-06T04:05:09.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.445+0000 7fa3d0ce6700 1 -- 172.21.15.157:0/1293640892 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 743919112 0 0) 0x7fa3b4003570 con 0x7fa3cc0b1e40 2023-10-06T04:05:09.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.445+0000 7fa3d14e7700 1 -- 172.21.15.157:0/1293640892 >> v1:172.21.15.175:6789/0 conn(0x7fa3cc06c2d0 legacy=0x7fa3cc156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:09.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.445+0000 7fa3d14e7700 1 -- 172.21.15.157:0/1293640892 >> v1:172.21.15.157:6789/0 conn(0x7fa3cc0b1e40 legacy=0x7fa3cc1563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:09.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.445+0000 7fa3d14e7700 1 -- 172.21.15.157:0/1293640892 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc1571d0 con 0x7fa3cc0b04d0 2023-10-06T04:05:09.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.446+0000 7fa3d14e7700 1 --2- 172.21.15.157:0/1293640892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3cc0b04d0 0x7fa3cc157750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0009910 tx=0x7fa3c00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5996ce6e116252e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:09.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.446+0000 7fa3d0ce6700 1 -- 172.21.15.157:0/1293640892 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4001470 con 0x7fa3cc0b04d0 2023-10-06T04:05:09.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.446+0000 7fa3d0ce6700 1 -- 172.21.15.157:0/1293640892 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c0011420 con 0x7fa3cc0b04d0 2023-10-06T04:05:09.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.446+0000 7fa3d0ce6700 1 -- 172.21.15.157:0/1293640892 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c00117a0 con 0x7fa3cc0b04d0 2023-10-06T04:05:09.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.446+0000 7fa3d2cea700 1 -- 172.21.15.157:0/1293640892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3cc0b04d0 msgr2=0x7fa3cc157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:09.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.446+0000 7fa3d2cea700 1 --2- 172.21.15.157:0/1293640892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3cc0b04d0 0x7fa3cc157750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0009910 tx=0x7fa3c00091a0 comp rx=0 tx=0).stop 2023-10-06T04:05:09.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.446+0000 7fa3d2cea700 1 -- 172.21.15.157:0/1293640892 shutdown_connections 2023-10-06T04:05:09.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.446+0000 7fa3d2cea700 1 --2- 172.21.15.157:0/1293640892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3cc0b04d0 0x7fa3cc157750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:09.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.446+0000 7fa3d2cea700 1 -- 172.21.15.157:0/1293640892 >> 172.21.15.157:0/1293640892 conn(0x7fa3cc0cc540 msgr2=0x7fa3cc0cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:09.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.446+0000 7fa3d2cea700 1 -- 172.21.15.157:0/1293640892 shutdown_connections 2023-10-06T04:05:09.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.446+0000 7fa3d2cea700 1 -- 172.21.15.157:0/1293640892 wait complete. 2023-10-06T04:05:09.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.447+0000 7fa3d2cea700 1 Processor -- start 2023-10-06T04:05:09.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.447+0000 7fa3d2cea700 1 -- start start 2023-10-06T04:05:09.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.448+0000 7fa3d2cea700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3cc06c2d0 0x7fa3cc153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:09.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.448+0000 7fa3d2cea700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3cc0b04d0 0x7fa3cc154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:09.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.448+0000 7fa3d2cea700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3cc0b1e40 0x7fa3cc1550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:09.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.448+0000 7fa3d2cea700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc0c7040 con 0x7fa3cc0b1e40 2023-10-06T04:05:09.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.448+0000 7fa3d2cea700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc1665d0 con 0x7fa3cc06c2d0 2023-10-06T04:05:09.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.448+0000 7fa3d2cea700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc151a40 con 0x7fa3cc0b04d0 2023-10-06T04:05:09.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.448+0000 7fa3d24e9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3cc0b1e40 0x7fa3cc1550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:09.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.448+0000 7fa3d24e9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3cc0b1e40 0x7fa3cc1550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60430/0 (socket says 172.21.15.157:60430) 2023-10-06T04:05:09.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.448+0000 7fa3d24e9700 1 -- 172.21.15.157:0/3206449800 learned_addr learned my addr 172.21.15.157:0/3206449800 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:09.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.448+0000 7fa3d1ce8700 1 --2- 172.21.15.157:0/3206449800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3cc06c2d0 0x7fa3cc153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:09.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.448+0000 7fa3d14e7700 1 --2- 172.21.15.157:0/3206449800 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3cc0b04d0 0x7fa3cc154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:09.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.449+0000 7fa3d24e9700 1 -- 172.21.15.157:0/3206449800 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3cc0b04d0 msgr2=0x7fa3cc154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:09.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.449+0000 7fa3d24e9700 1 --2- 172.21.15.157:0/3206449800 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3cc0b04d0 0x7fa3cc154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:09.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.449+0000 7fa3d24e9700 1 -- 172.21.15.157:0/3206449800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3cc06c2d0 msgr2=0x7fa3cc153540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:09.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.449+0000 7fa3d24e9700 1 --2- 172.21.15.157:0/3206449800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3cc06c2d0 0x7fa3cc153540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:09.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.449+0000 7fa3d24e9700 1 -- 172.21.15.157:0/3206449800 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c0009420 con 0x7fa3cc0b1e40 2023-10-06T04:05:09.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.449+0000 7fa3d1ce8700 1 --2- 172.21.15.157:0/3206449800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3cc06c2d0 0x7fa3cc153540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:05:09.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.449+0000 7fa3d24e9700 1 --2- 172.21.15.157:0/3206449800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3cc0b1e40 0x7fa3cc1550e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00c940 tx=0x7fa3bc0027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:09.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.449+0000 7fa3c9ffb700 1 -- 172.21.15.157:0/3206449800 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc012630 con 0x7fa3cc0b1e40 2023-10-06T04:05:09.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.449+0000 7fa3c9ffb700 1 -- 172.21.15.157:0/3206449800 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3bc0127b0 con 0x7fa3cc0b1e40 2023-10-06T04:05:09.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.449+0000 7fa3d2cea700 1 -- 172.21.15.157:0/3206449800 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3cc0cbdb0 con 0x7fa3cc0b1e40 2023-10-06T04:05:09.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.449+0000 7fa3c9ffb700 1 -- 172.21.15.157:0/3206449800 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc012b70 con 0x7fa3cc0b1e40 2023-10-06T04:05:09.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.449+0000 7fa3d2cea700 1 -- 172.21.15.157:0/3206449800 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3cc1513f0 con 0x7fa3cc0b1e40 2023-10-06T04:05:09.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.450+0000 7fa3d2cea700 1 -- 172.21.15.157:0/3206449800 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3cc1548c0 con 0x7fa3cc0b1e40 2023-10-06T04:05:09.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.450+0000 7fa3c9ffb700 1 -- 172.21.15.157:0/3206449800 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa3bc01b400 con 0x7fa3cc0b1e40 2023-10-06T04:05:09.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.451+0000 7fa3c9ffb700 1 --2- 172.21.15.157:0/3206449800 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa3b8066f30 0x7fa3b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:09.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.451+0000 7fa3c9ffb700 1 -- 172.21.15.157:0/3206449800 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6430+0+0 (secure 0 0 0) 0x7fa3bc084b60 con 0x7fa3cc0b1e40 2023-10-06T04:05:09.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.451+0000 7fa3d1ce8700 1 --2- 172.21.15.157:0/3206449800 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa3b8066f30 0x7fa3b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:09.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.451+0000 7fa3d1ce8700 1 --2- 172.21.15.157:0/3206449800 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa3b8066f30 0x7fa3b80693e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4000ff0 tx=0x7fa3c4000bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:09.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.455+0000 7fa3c9ffb700 1 -- 172.21.15.157:0/3206449800 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa3cc1548c0 con 0x7fa3cc0b1e40 2023-10-06T04:05:09.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.679+0000 7fa3d2cea700 1 -- 172.21.15.157:0/3206449800 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3cc1548c0 con 0x7fa3cc0b1e40 2023-10-06T04:05:09.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.679+0000 7fa3c9ffb700 1 -- 172.21.15.157:0/3206449800 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6145 (secure 0 0 0) 0x7fa3cc1548c0 con 0x7fa3cc0b1e40 2023-10-06T04:05:09.681 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:05:09.681 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5799,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:04:48.805873+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:05:09.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.681+0000 7fa3d2cea700 1 -- 172.21.15.157:0/3206449800 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa3b8066f30 msgr2=0x7fa3b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:09.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.681+0000 7fa3d2cea700 1 --2- 172.21.15.157:0/3206449800 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa3b8066f30 0x7fa3b80693e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4000ff0 tx=0x7fa3c4000bc0 comp rx=0 tx=0).stop 2023-10-06T04:05:09.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.681+0000 7fa3d2cea700 1 -- 172.21.15.157:0/3206449800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3cc0b1e40 msgr2=0x7fa3cc1550e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:09.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.681+0000 7fa3d2cea700 1 --2- 172.21.15.157:0/3206449800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3cc0b1e40 0x7fa3cc1550e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00c940 tx=0x7fa3bc0027a0 comp rx=0 tx=0).stop 2023-10-06T04:05:09.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.681+0000 7fa3d2cea700 1 -- 172.21.15.157:0/3206449800 shutdown_connections 2023-10-06T04:05:09.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.681+0000 7fa3d2cea700 1 --2- 172.21.15.157:0/3206449800 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa3b8066f30 0x7fa3b80693e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:09.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.681+0000 7fa3d2cea700 1 --2- 172.21.15.157:0/3206449800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3cc06c2d0 0x7fa3cc153540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:09.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.681+0000 7fa3d2cea700 1 --2- 172.21.15.157:0/3206449800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3cc0b1e40 0x7fa3cc1550e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:09.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.681+0000 7fa3d2cea700 1 --2- 172.21.15.157:0/3206449800 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3cc0b04d0 0x7fa3cc154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:09.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.681+0000 7fa3d2cea700 1 -- 172.21.15.157:0/3206449800 >> 172.21.15.157:0/3206449800 conn(0x7fa3cc0cc540 msgr2=0x7fa3cc157440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:09.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.682+0000 7fa3d2cea700 1 -- 172.21.15.157:0/3206449800 shutdown_connections 2023-10-06T04:05:09.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:09.682+0000 7fa3d2cea700 1 -- 172.21.15.157:0/3206449800 wait complete. 2023-10-06T04:05:09.685 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 63 2023-10-06T04:05:09.695 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-06T04:05:11.696 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:05:11.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.819+0000 7f6be73db700 1 Processor -- start 2023-10-06T04:05:11.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.820+0000 7f6be73db700 1 -- start start 2023-10-06T04:05:11.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.820+0000 7f6be73db700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6be00ca630 0x7f6be00c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:11.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.820+0000 7f6be73db700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6be0068920 0x7f6be0156840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:11.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.820+0000 7f6be73db700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6be00c52e0 0x7f6be0156d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:11.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.820+0000 7f6be73db700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6be006d530 con 0x7f6be00ca630 2023-10-06T04:05:11.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.820+0000 7f6be73db700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6be006d9c0 con 0x7f6be00c52e0 2023-10-06T04:05:11.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.820+0000 7f6be73db700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be006f0c0 con 0x7f6be0068920 2023-10-06T04:05:11.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.820+0000 7f6be5177700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6be00ca630 0x7f6be00c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:11.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.821+0000 7f6be5177700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6be00ca630 0x7f6be00c6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60442/0 (socket says 172.21.15.157:60442) 2023-10-06T04:05:11.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.821+0000 7f6be5177700 1 -- 172.21.15.157:0/1180606529 learned_addr learned my addr 172.21.15.157:0/1180606529 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:11.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.821+0000 7f6be5978700 1 --2- 172.21.15.157:0/1180606529 >> v2:172.21.15.175:3300/0 conn(0x7f6be00c52e0 0x7f6be0156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:11.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.821+0000 7f6be4976700 1 --2- 172.21.15.157:0/1180606529 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6be0068920 0x7f6be0156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:11.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.821+0000 7f6be5177700 1 -- 172.21.15.157:0/1180606529 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6be0068920 msgr2=0x7f6be0156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:11.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.821+0000 7f6be5177700 1 --2- 172.21.15.157:0/1180606529 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6be0068920 0x7f6be0156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:11.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.821+0000 7f6be5177700 1 -- 172.21.15.157:0/1180606529 >> v2:172.21.15.175:3300/0 conn(0x7f6be00c52e0 msgr2=0x7f6be0156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:11.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.821+0000 7f6be5177700 1 --2- 172.21.15.157:0/1180606529 >> v2:172.21.15.175:3300/0 conn(0x7f6be00c52e0 0x7f6be0156d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:11.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.821+0000 7f6be5177700 1 -- 172.21.15.157:0/1180606529 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be006f3a0 con 0x7f6be00ca630 2023-10-06T04:05:11.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.821+0000 7f6be5978700 1 --2- 172.21.15.157:0/1180606529 >> v2:172.21.15.175:3300/0 conn(0x7f6be00c52e0 0x7f6be0156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:05:11.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.821+0000 7f6be5177700 1 --2- 172.21.15.157:0/1180606529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6be00ca630 0x7f6be00c6ef0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6be006ee60 tx=0x7f6bd8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9620b5a5a26a97e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:11.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.821+0000 7f6bdffff700 1 -- 172.21.15.157:0/1180606529 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd800e070 con 0x7f6be00ca630 2023-10-06T04:05:11.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.821+0000 7f6bdffff700 1 -- 172.21.15.157:0/1180606529 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd800ba50 con 0x7f6be00ca630 2023-10-06T04:05:11.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.821+0000 7f6bdffff700 1 -- 172.21.15.157:0/1180606529 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd80092e0 con 0x7f6be00ca630 2023-10-06T04:05:11.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.822+0000 7f6be73db700 1 -- 172.21.15.157:0/1180606529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6be00ca630 msgr2=0x7f6be00c6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:11.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.822+0000 7f6be73db700 1 --2- 172.21.15.157:0/1180606529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6be00ca630 0x7f6be00c6ef0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6be006ee60 tx=0x7f6bd8009840 comp rx=0 tx=0).stop 2023-10-06T04:05:11.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.822+0000 7f6be73db700 1 -- 172.21.15.157:0/1180606529 shutdown_connections 2023-10-06T04:05:11.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.822+0000 7f6be73db700 1 --2- 172.21.15.157:0/1180606529 >> v2:172.21.15.175:3300/0 conn(0x7f6be00c52e0 0x7f6be0156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:11.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.822+0000 7f6be73db700 1 --2- 172.21.15.157:0/1180606529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6be00ca630 0x7f6be00c6ef0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:11.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.822+0000 7f6be73db700 1 --2- 172.21.15.157:0/1180606529 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6be0068920 0x7f6be0156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:11.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.822+0000 7f6be73db700 1 -- 172.21.15.157:0/1180606529 >> 172.21.15.157:0/1180606529 conn(0x7f6be00692b0 msgr2=0x7f6be00696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:11.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.822+0000 7f6be73db700 1 -- 172.21.15.157:0/1180606529 shutdown_connections 2023-10-06T04:05:11.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.822+0000 7f6be73db700 1 -- 172.21.15.157:0/1180606529 wait complete. 2023-10-06T04:05:11.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.823+0000 7f6be73db700 1 Processor -- start 2023-10-06T04:05:11.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.823+0000 7f6be73db700 1 -- start start 2023-10-06T04:05:11.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.824+0000 7f6be73db700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6be0068920 0x7f6be00c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:11.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.824+0000 7f6be73db700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6be00c52e0 0x7f6be01699f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:11.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.824+0000 7f6be73db700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6be00ca630 0x7f6be014e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:11.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.824+0000 7f6be73db700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be01536d0 con 0x7f6be0068920 2023-10-06T04:05:11.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.824+0000 7f6be73db700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be00afef0 con 0x7f6be00ca630 2023-10-06T04:05:11.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.824+0000 7f6be73db700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be0165e20 con 0x7f6be00c52e0 2023-10-06T04:05:11.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.824+0000 7f6be5177700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6be0068920 0x7f6be00c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:11.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.824+0000 7f6be5177700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6be0068920 0x7f6be00c73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60448/0 (socket says 172.21.15.157:60448) 2023-10-06T04:05:11.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.824+0000 7f6be5177700 1 -- 172.21.15.157:0/995979027 learned_addr learned my addr 172.21.15.157:0/995979027 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:11.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.824+0000 7f6be5978700 1 --2- 172.21.15.157:0/995979027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6be00ca630 0x7f6be014e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:11.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.824+0000 7f6be4976700 1 --2- 172.21.15.157:0/995979027 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6be00c52e0 0x7f6be01699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:11.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.824+0000 7f6be5177700 1 -- 172.21.15.157:0/995979027 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6be00c52e0 msgr2=0x7f6be01699f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:11.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.824+0000 7f6be5177700 1 --2- 172.21.15.157:0/995979027 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6be00c52e0 0x7f6be01699f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:11.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.824+0000 7f6be5177700 1 -- 172.21.15.157:0/995979027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6be00ca630 msgr2=0x7f6be014e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:11.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.824+0000 7f6be5177700 1 --2- 172.21.15.157:0/995979027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6be00ca630 0x7f6be014e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:11.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.824+0000 7f6be5177700 1 -- 172.21.15.157:0/995979027 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd8009ae0 con 0x7f6be0068920 2023-10-06T04:05:11.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.824+0000 7f6be4976700 1 --2- 172.21.15.157:0/995979027 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6be00c52e0 0x7f6be01699f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:05:11.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.824+0000 7f6be5177700 1 --2- 172.21.15.157:0/995979027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6be0068920 0x7f6be00c73e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8009fd0 tx=0x7f6bd8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:11.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.825+0000 7f6bdd7fa700 1 -- 172.21.15.157:0/995979027 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd8008980 con 0x7f6be0068920 2023-10-06T04:05:11.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.825+0000 7f6bdd7fa700 1 -- 172.21.15.157:0/995979027 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd8008ae0 con 0x7f6be0068920 2023-10-06T04:05:11.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.825+0000 7f6be73db700 1 -- 172.21.15.157:0/995979027 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6be0159580 con 0x7f6be0068920 2023-10-06T04:05:11.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.825+0000 7f6bdd7fa700 1 -- 172.21.15.157:0/995979027 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd8008ea0 con 0x7f6be0068920 2023-10-06T04:05:11.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.825+0000 7f6be73db700 1 -- 172.21.15.157:0/995979027 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be01533f0 con 0x7f6be0068920 2023-10-06T04:05:11.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.826+0000 7f6be73db700 1 -- 172.21.15.157:0/995979027 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6be006a8f0 con 0x7f6be0068920 2023-10-06T04:05:11.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.826+0000 7f6bdd7fa700 1 -- 172.21.15.157:0/995979027 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6bd8003040 con 0x7f6be0068920 2023-10-06T04:05:11.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.826+0000 7f6bdd7fa700 1 --2- 172.21.15.157:0/995979027 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6bcc066f30 0x7f6bcc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:11.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.826+0000 7f6bdd7fa700 1 -- 172.21.15.157:0/995979027 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6402+0+0 (secure 0 0 0) 0x7f6bd8086160 con 0x7f6be0068920 2023-10-06T04:05:11.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.827+0000 7f6be4976700 1 --2- 172.21.15.157:0/995979027 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6bcc066f30 0x7f6bcc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:11.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.827+0000 7f6be4976700 1 --2- 172.21.15.157:0/995979027 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6bcc066f30 0x7f6bcc0693e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6be01549c0 tx=0x7f6bd4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:11.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:11.831+0000 7f6bdd7fa700 1 -- 172.21.15.157:0/995979027 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6bd8053110 con 0x7f6be0068920 2023-10-06T04:05:12.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:12.058+0000 7f6be73db700 1 -- 172.21.15.157:0/995979027 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6be004bb40 con 0x7f6be0068920 2023-10-06T04:05:12.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:12.058+0000 7f6bdd7fa700 1 -- 172.21.15.157:0/995979027 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6145 (secure 0 0 0) 0x7f6bd8056730 con 0x7f6be0068920 2023-10-06T04:05:12.060 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:05:12.060 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5799,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:04:48.805873+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:05:12.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:12.060+0000 7f6be73db700 1 -- 172.21.15.157:0/995979027 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6bcc066f30 msgr2=0x7f6bcc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:12.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:12.060+0000 7f6be73db700 1 --2- 172.21.15.157:0/995979027 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6bcc066f30 0x7f6bcc0693e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6be01549c0 tx=0x7f6bd4008040 comp rx=0 tx=0).stop 2023-10-06T04:05:12.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:12.060+0000 7f6be73db700 1 -- 172.21.15.157:0/995979027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6be0068920 msgr2=0x7f6be00c73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:12.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:12.060+0000 7f6be73db700 1 --2- 172.21.15.157:0/995979027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6be0068920 0x7f6be00c73e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8009fd0 tx=0x7f6bd8018000 comp rx=0 tx=0).stop 2023-10-06T04:05:12.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:12.060+0000 7f6be73db700 1 -- 172.21.15.157:0/995979027 shutdown_connections 2023-10-06T04:05:12.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:12.060+0000 7f6be73db700 1 --2- 172.21.15.157:0/995979027 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6bcc066f30 0x7f6bcc0693e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:12.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:12.060+0000 7f6be73db700 1 --2- 172.21.15.157:0/995979027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6be00ca630 0x7f6be014e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:12.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:12.060+0000 7f6be73db700 1 --2- 172.21.15.157:0/995979027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6be0068920 0x7f6be00c73e0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:12.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:12.060+0000 7f6be73db700 1 --2- 172.21.15.157:0/995979027 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6be00c52e0 0x7f6be01699f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:12.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:12.060+0000 7f6be73db700 1 -- 172.21.15.157:0/995979027 >> 172.21.15.157:0/995979027 conn(0x7f6be00692b0 msgr2=0x7f6be00c8900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:12.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:12.060+0000 7f6be73db700 1 -- 172.21.15.157:0/995979027 shutdown_connections 2023-10-06T04:05:12.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:12.060+0000 7f6be73db700 1 -- 172.21.15.157:0/995979027 wait complete. 2023-10-06T04:05:12.063 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 63 2023-10-06T04:05:12.074 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-06T04:05:14.076 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:05:14.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.203+0000 7f5c47ab0700 1 Processor -- start 2023-10-06T04:05:14.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.203+0000 7f5c47ab0700 1 -- start start 2023-10-06T04:05:14.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.204+0000 7f5c47ab0700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c4006cbb0 0x7f5c4006d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:14.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.204+0000 7f5c47ab0700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5c4006c1f0 0x7f5c4006da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:14.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.204+0000 7f5c47ab0700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5c4006cf80 0x7f5c4014fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:14.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.204+0000 7f5c47ab0700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c400c8c40 con 0x7f5c4006cf80 2023-10-06T04:05:14.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.204+0000 7f5c47ab0700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c400c9780 con 0x7f5c4006c1f0 2023-10-06T04:05:14.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.204+0000 7f5c47ab0700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4006e0c0 con 0x7f5c4006cbb0 2023-10-06T04:05:14.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.204+0000 7f5c4604d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5c4006cf80 0x7f5c4014fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:14.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.204+0000 7f5c4604d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5c4006cf80 0x7f5c4014fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49332/0 (socket says 172.21.15.157:49332) 2023-10-06T04:05:14.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.204+0000 7f5c4604d700 1 -- 172.21.15.157:0/1116662041 learned_addr learned my addr 172.21.15.157:0/1116662041 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:14.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.204+0000 7f5c4584c700 1 --2- 172.21.15.157:0/1116662041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c4006cbb0 0x7f5c4006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:14.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.204+0000 7f5c4504b700 1 --2- 172.21.15.157:0/1116662041 >> v2:172.21.15.175:3300/0 conn(0x7f5c4006c1f0 0x7f5c4006da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:14.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.205+0000 7f5c4604d700 1 -- 172.21.15.157:0/1116662041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c4006cbb0 msgr2=0x7f5c4006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:14.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.205+0000 7f5c4604d700 1 --2- 172.21.15.157:0/1116662041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c4006cbb0 0x7f5c4006d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:14.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.205+0000 7f5c4604d700 1 -- 172.21.15.157:0/1116662041 >> v2:172.21.15.175:3300/0 conn(0x7f5c4006c1f0 msgr2=0x7f5c4006da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:14.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.205+0000 7f5c4604d700 1 --2- 172.21.15.157:0/1116662041 >> v2:172.21.15.175:3300/0 conn(0x7f5c4006c1f0 0x7f5c4006da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:14.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.205+0000 7f5c4604d700 1 -- 172.21.15.157:0/1116662041 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c400c8e50 con 0x7f5c4006cf80 2023-10-06T04:05:14.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.205+0000 7f5c4504b700 1 --2- 172.21.15.157:0/1116662041 >> v2:172.21.15.175:3300/0 conn(0x7f5c4006c1f0 0x7f5c4006da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:05:14.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.205+0000 7f5c4604d700 1 --2- 172.21.15.157:0/1116662041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4006cf80 0x7f5c4014fd00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f5c380099b0 tx=0x7f5c38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd11d80468b621a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:14.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.205+0000 7f5c4484a700 1 -- 172.21.15.157:0/1116662041 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3800b8b0 con 0x7f5c4006cf80 2023-10-06T04:05:14.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.205+0000 7f5c4484a700 1 -- 172.21.15.157:0/1116662041 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3800ba10 con 0x7f5c4006cf80 2023-10-06T04:05:14.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.205+0000 7f5c4484a700 1 -- 172.21.15.157:0/1116662041 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3800bdb0 con 0x7f5c4006cf80 2023-10-06T04:05:14.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.205+0000 7f5c47ab0700 1 -- 172.21.15.157:0/1116662041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4006cf80 msgr2=0x7f5c4014fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:14.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.205+0000 7f5c47ab0700 1 --2- 172.21.15.157:0/1116662041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4006cf80 0x7f5c4014fd00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f5c380099b0 tx=0x7f5c38009250 comp rx=0 tx=0).stop 2023-10-06T04:05:14.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.206+0000 7f5c47ab0700 1 -- 172.21.15.157:0/1116662041 shutdown_connections 2023-10-06T04:05:14.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.206+0000 7f5c47ab0700 1 --2- 172.21.15.157:0/1116662041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4006cf80 0x7f5c4014fd00 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:14.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.206+0000 7f5c47ab0700 1 --2- 172.21.15.157:0/1116662041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c4006cbb0 0x7f5c4006d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:14.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.206+0000 7f5c47ab0700 1 --2- 172.21.15.157:0/1116662041 >> v2:172.21.15.175:3300/0 conn(0x7f5c4006c1f0 0x7f5c4006da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:14.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.206+0000 7f5c47ab0700 1 -- 172.21.15.157:0/1116662041 >> 172.21.15.157:0/1116662041 conn(0x7f5c4006f230 msgr2=0x7f5c4006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:14.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.206+0000 7f5c47ab0700 1 -- 172.21.15.157:0/1116662041 shutdown_connections 2023-10-06T04:05:14.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.206+0000 7f5c47ab0700 1 -- 172.21.15.157:0/1116662041 wait complete. 2023-10-06T04:05:14.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.207+0000 7f5c47ab0700 1 Processor -- start 2023-10-06T04:05:14.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.207+0000 7f5c47ab0700 1 -- start start 2023-10-06T04:05:14.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.207+0000 7f5c47ab0700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c4006c1f0 0x7f5c4014a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:14.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.207+0000 7f5c47ab0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c4006cbb0 0x7f5c40145e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:14.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.207+0000 7f5c47ab0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4006cf80 0x7f5c40146360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:14.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.207+0000 7f5c47ab0700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c40145990 con 0x7f5c4006cf80 2023-10-06T04:05:14.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.207+0000 7f5c47ab0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4014e510 con 0x7f5c4006cbb0 2023-10-06T04:05:14.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.207+0000 7f5c47ab0700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c401552b0 con 0x7f5c4006c1f0 2023-10-06T04:05:14.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.208+0000 7f5c4604d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4006cf80 0x7f5c40146360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:14.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.208+0000 7f5c4604d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4006cf80 0x7f5c40146360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49338/0 (socket says 172.21.15.157:49338) 2023-10-06T04:05:14.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.208+0000 7f5c4604d700 1 -- 172.21.15.157:0/1587283490 learned_addr learned my addr 172.21.15.157:0/1587283490 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:14.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.208+0000 7f5c4584c700 1 --2- 172.21.15.157:0/1587283490 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c4006c1f0 0x7f5c4014a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:14.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.208+0000 7f5c4504b700 1 --2- 172.21.15.157:0/1587283490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c4006cbb0 0x7f5c40145e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:14.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.208+0000 7f5c4604d700 1 -- 172.21.15.157:0/1587283490 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c4006c1f0 msgr2=0x7f5c4014a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:14.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.208+0000 7f5c4604d700 1 --2- 172.21.15.157:0/1587283490 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c4006c1f0 0x7f5c4014a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:14.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.208+0000 7f5c4604d700 1 -- 172.21.15.157:0/1587283490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c4006cbb0 msgr2=0x7f5c40145e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:14.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.208+0000 7f5c4604d700 1 --2- 172.21.15.157:0/1587283490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c4006cbb0 0x7f5c40145e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:14.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.208+0000 7f5c4604d700 1 -- 172.21.15.157:0/1587283490 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c380094f0 con 0x7f5c4006cf80 2023-10-06T04:05:14.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.208+0000 7f5c4604d700 1 --2- 172.21.15.157:0/1587283490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4006cf80 0x7f5c40146360 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5c380094c0 tx=0x7f5c38017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:14.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.208+0000 7f5c35ffb700 1 -- 172.21.15.157:0/1587283490 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3800b8b0 con 0x7f5c4006cf80 2023-10-06T04:05:14.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.208+0000 7f5c35ffb700 1 -- 172.21.15.157:0/1587283490 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3801f070 con 0x7f5c4006cf80 2023-10-06T04:05:14.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.208+0000 7f5c47ab0700 1 -- 172.21.15.157:0/1587283490 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c401479b0 con 0x7f5c4006cf80 2023-10-06T04:05:14.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.208+0000 7f5c35ffb700 1 -- 172.21.15.157:0/1587283490 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3801a890 con 0x7f5c4006cf80 2023-10-06T04:05:14.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.208+0000 7f5c47ab0700 1 -- 172.21.15.157:0/1587283490 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c40155850 con 0x7f5c4006cf80 2023-10-06T04:05:14.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.209+0000 7f5c47ab0700 1 -- 172.21.15.157:0/1587283490 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c40145b40 con 0x7f5c4006cf80 2023-10-06T04:05:14.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.210+0000 7f5c35ffb700 1 -- 172.21.15.157:0/1587283490 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5c3801acc0 con 0x7f5c4006cf80 2023-10-06T04:05:14.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.210+0000 7f5c35ffb700 1 --2- 172.21.15.157:0/1587283490 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5c24066f30 0x7f5c240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:14.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.210+0000 7f5c35ffb700 1 -- 172.21.15.157:0/1587283490 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6402+0+0 (secure 0 0 0) 0x7f5c38021070 con 0x7f5c4006cf80 2023-10-06T04:05:14.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.210+0000 7f5c4584c700 1 --2- 172.21.15.157:0/1587283490 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5c24066f30 0x7f5c240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:14.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.211+0000 7f5c4584c700 1 --2- 172.21.15.157:0/1587283490 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5c24066f30 0x7f5c240693e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c009fd0 tx=0x7f5c3c009880 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:14.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.215+0000 7f5c35ffb700 1 -- 172.21.15.157:0/1587283490 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c40145b40 con 0x7f5c4006cf80 2023-10-06T04:05:14.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.442+0000 7f5c47ab0700 1 -- 172.21.15.157:0/1587283490 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c40145ad0 con 0x7f5c4006cf80 2023-10-06T04:05:14.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.442+0000 7f5c35ffb700 1 -- 172.21.15.157:0/1587283490 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6145 (secure 0 0 0) 0x7f5c38011e60 con 0x7f5c4006cf80 2023-10-06T04:05:14.444 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:05:14.444 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5799,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:04:48.805873+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:05:14.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.445+0000 7f5c47ab0700 1 -- 172.21.15.157:0/1587283490 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5c24066f30 msgr2=0x7f5c240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:14.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.445+0000 7f5c47ab0700 1 --2- 172.21.15.157:0/1587283490 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5c24066f30 0x7f5c240693e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c009fd0 tx=0x7f5c3c009880 comp rx=0 tx=0).stop 2023-10-06T04:05:14.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.445+0000 7f5c47ab0700 1 -- 172.21.15.157:0/1587283490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4006cf80 msgr2=0x7f5c40146360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:14.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.445+0000 7f5c47ab0700 1 --2- 172.21.15.157:0/1587283490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4006cf80 0x7f5c40146360 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5c380094c0 tx=0x7f5c38017000 comp rx=0 tx=0).stop 2023-10-06T04:05:14.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.445+0000 7f5c47ab0700 1 -- 172.21.15.157:0/1587283490 shutdown_connections 2023-10-06T04:05:14.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.445+0000 7f5c47ab0700 1 --2- 172.21.15.157:0/1587283490 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5c24066f30 0x7f5c240693e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:14.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.445+0000 7f5c47ab0700 1 --2- 172.21.15.157:0/1587283490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c4006cf80 0x7f5c40146360 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:14.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.445+0000 7f5c47ab0700 1 --2- 172.21.15.157:0/1587283490 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c4006c1f0 0x7f5c4014a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:14.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.445+0000 7f5c47ab0700 1 --2- 172.21.15.157:0/1587283490 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c4006cbb0 0x7f5c40145e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:14.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.445+0000 7f5c47ab0700 1 -- 172.21.15.157:0/1587283490 >> 172.21.15.157:0/1587283490 conn(0x7f5c4006f230 msgr2=0x7f5c400aff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:14.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.445+0000 7f5c47ab0700 1 -- 172.21.15.157:0/1587283490 shutdown_connections 2023-10-06T04:05:14.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:14.445+0000 7f5c47ab0700 1 -- 172.21.15.157:0/1587283490 wait complete. 2023-10-06T04:05:14.448 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 63 2023-10-06T04:05:14.458 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-06T04:05:15.356 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:05:15.359 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:05:16.459 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:05:16.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.586+0000 7f636d350700 1 Processor -- start 2023-10-06T04:05:16.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.586+0000 7f636d350700 1 -- start start 2023-10-06T04:05:16.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.586+0000 7f636d350700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63680c97d0 0x7f6368156a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:16.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.586+0000 7f636d350700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6368157490 0x7f6368159c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:16.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.586+0000 7f636d350700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63681507c0 con 0x7f63680c81b0 2023-10-06T04:05:16.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.586+0000 7f636d350700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63680c60e0 con 0x7f6368157490 2023-10-06T04:05:16.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.586+0000 7f636d350700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63680c40b0 con 0x7f63680c97d0 2023-10-06T04:05:16.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.587+0000 7f63677fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6368157490 0x7f6368159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:16.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.587+0000 7f63677fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6368157490 0x7f6368159c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49354/0 (socket says 172.21.15.157:49354) 2023-10-06T04:05:16.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.587+0000 7f63677fe700 1 -- 172.21.15.157:0/1373531780 learned_addr learned my addr 172.21.15.157:0/1373531780 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:16.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.587+0000 7f635e7fc700 1 --2- 172.21.15.157:0/1373531780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63680c97d0 0x7f6368156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:16.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.587+0000 7f63677fe700 1 -- 172.21.15.157:0/1373531780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63680c97d0 msgr2=0x7f6368156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:16.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.587+0000 7f63677fe700 1 --2- 172.21.15.157:0/1373531780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63680c97d0 0x7f6368156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:16.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.587+0000 7f63677fe700 1 -- 172.21.15.157:0/1373531780 >> v1:172.21.15.175:6789/0 conn(0x7f63680c81b0 legacy=0x7f63680c4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:16.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.587+0000 7f63677fe700 1 -- 172.21.15.157:0/1373531780 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63680c6330 con 0x7f6368157490 2023-10-06T04:05:16.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.587+0000 7f63677fe700 1 --2- 172.21.15.157:0/1373531780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6368157490 0x7f6368159c30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6360009f20 tx=0x7f6360009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd94f71c07869857 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:16.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.587+0000 7f63667fc700 1 -- 172.21.15.157:0/1373531780 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636000e070 con 0x7f6368157490 2023-10-06T04:05:16.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.587+0000 7f63667fc700 1 -- 172.21.15.157:0/1373531780 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636000ba60 con 0x7f6368157490 2023-10-06T04:05:16.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.587+0000 7f63667fc700 1 -- 172.21.15.157:0/1373531780 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63600092e0 con 0x7f6368157490 2023-10-06T04:05:16.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.588+0000 7f636d350700 1 -- 172.21.15.157:0/1373531780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6368157490 msgr2=0x7f6368159c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:16.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.588+0000 7f636d350700 1 --2- 172.21.15.157:0/1373531780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6368157490 0x7f6368159c30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6360009f20 tx=0x7f6360009840 comp rx=0 tx=0).stop 2023-10-06T04:05:16.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.588+0000 7f636d350700 1 -- 172.21.15.157:0/1373531780 shutdown_connections 2023-10-06T04:05:16.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.588+0000 7f636d350700 1 --2- 172.21.15.157:0/1373531780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6368157490 0x7f6368159c30 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:16.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.588+0000 7f636d350700 1 --2- 172.21.15.157:0/1373531780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63680c97d0 0x7f6368156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:16.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.588+0000 7f636d350700 1 -- 172.21.15.157:0/1373531780 >> 172.21.15.157:0/1373531780 conn(0x7f63680cd0b0 msgr2=0x7f63680c9050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:16.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.588+0000 7f636d350700 1 -- 172.21.15.157:0/1373531780 shutdown_connections 2023-10-06T04:05:16.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.588+0000 7f636d350700 1 -- 172.21.15.157:0/1373531780 wait complete. 2023-10-06T04:05:16.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.589+0000 7f636d350700 1 Processor -- start 2023-10-06T04:05:16.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.589+0000 7f636d350700 1 -- start start 2023-10-06T04:05:16.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.590+0000 7f636d350700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63680c81b0 0x7f636815acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:16.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.590+0000 7f636d350700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63680c97d0 0x7f6368169750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:16.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.590+0000 7f636d350700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6368157490 0x7f636814ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:16.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.590+0000 7f636d350700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63680cba60 con 0x7f63680c81b0 2023-10-06T04:05:16.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.590+0000 7f636d350700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6368151eb0 con 0x7f63680c97d0 2023-10-06T04:05:16.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.590+0000 7f636d350700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f636816a0d0 con 0x7f6368157490 2023-10-06T04:05:16.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.590+0000 7f6366ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63680c81b0 0x7f636815acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:16.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.590+0000 7f6366ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63680c81b0 0x7f636815acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49364/0 (socket says 172.21.15.157:49364) 2023-10-06T04:05:16.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.590+0000 7f6366ffd700 1 -- 172.21.15.157:0/3566555569 learned_addr learned my addr 172.21.15.157:0/3566555569 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:16.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.590+0000 7f635e7fc700 1 --2- 172.21.15.157:0/3566555569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63680c97d0 0x7f6368169750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:16.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.590+0000 7f63677fe700 1 --2- 172.21.15.157:0/3566555569 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6368157490 0x7f636814ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:16.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.590+0000 7f6366ffd700 1 -- 172.21.15.157:0/3566555569 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6368157490 msgr2=0x7f636814ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:16.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.590+0000 7f6366ffd700 1 --2- 172.21.15.157:0/3566555569 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6368157490 0x7f636814ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:16.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.590+0000 7f6366ffd700 1 -- 172.21.15.157:0/3566555569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63680c97d0 msgr2=0x7f6368169750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:16.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.590+0000 7f6366ffd700 1 --2- 172.21.15.157:0/3566555569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63680c97d0 0x7f6368169750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:16.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.590+0000 7f6366ffd700 1 -- 172.21.15.157:0/3566555569 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6360009ae0 con 0x7f63680c81b0 2023-10-06T04:05:16.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.591+0000 7f6366ffd700 1 --2- 172.21.15.157:0/3566555569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63680c81b0 0x7f636815acc0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f635800c5c0 tx=0x7f635800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:16.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.591+0000 7f635f7fe700 1 -- 172.21.15.157:0/3566555569 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6358011590 con 0x7f63680c81b0 2023-10-06T04:05:16.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.591+0000 7f635f7fe700 1 -- 172.21.15.157:0/3566555569 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6358011710 con 0x7f63680c81b0 2023-10-06T04:05:16.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.591+0000 7f636d350700 1 -- 172.21.15.157:0/3566555569 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f636806c670 con 0x7f63680c81b0 2023-10-06T04:05:16.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.591+0000 7f635f7fe700 1 -- 172.21.15.157:0/3566555569 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6358011ad0 con 0x7f63680c81b0 2023-10-06T04:05:16.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.591+0000 7f636d350700 1 -- 172.21.15.157:0/3566555569 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6368151b40 con 0x7f63680c81b0 2023-10-06T04:05:16.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.592+0000 7f636d350700 1 -- 172.21.15.157:0/3566555569 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63680c9e30 con 0x7f63680c81b0 2023-10-06T04:05:16.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.592+0000 7f635f7fe700 1 -- 172.21.15.157:0/3566555569 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f635801b3b0 con 0x7f63680c81b0 2023-10-06T04:05:16.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.592+0000 7f635f7fe700 1 --2- 172.21.15.157:0/3566555569 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6348066f30 0x7f63480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:16.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.593+0000 7f635f7fe700 1 -- 172.21.15.157:0/3566555569 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6402+0+0 (secure 0 0 0) 0x7f6358083c50 con 0x7f63680c81b0 2023-10-06T04:05:16.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.593+0000 7f635e7fc700 1 --2- 172.21.15.157:0/3566555569 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6348066f30 0x7f63480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:16.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.593+0000 7f635e7fc700 1 --2- 172.21.15.157:0/3566555569 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6348066f30 0x7f63480693e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f63680c3310 tx=0x7f63500091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:16.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.597+0000 7f635f7fe700 1 -- 172.21.15.157:0/3566555569 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6358089020 con 0x7f63680c81b0 2023-10-06T04:05:16.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.821+0000 7f636d350700 1 -- 172.21.15.157:0/3566555569 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f636806fc70 con 0x7f63680c81b0 2023-10-06T04:05:16.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.822+0000 7f635f7fe700 1 -- 172.21.15.157:0/3566555569 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6145 (secure 0 0 0) 0x7f6358017070 con 0x7f63680c81b0 2023-10-06T04:05:16.823 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:05:16.823 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5799,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:04:48.805873+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:05:16.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.824+0000 7f636d350700 1 -- 172.21.15.157:0/3566555569 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6348066f30 msgr2=0x7f63480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:16.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.824+0000 7f636d350700 1 --2- 172.21.15.157:0/3566555569 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6348066f30 0x7f63480693e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f63680c3310 tx=0x7f63500091a0 comp rx=0 tx=0).stop 2023-10-06T04:05:16.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.824+0000 7f636d350700 1 -- 172.21.15.157:0/3566555569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63680c81b0 msgr2=0x7f636815acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:16.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.824+0000 7f636d350700 1 --2- 172.21.15.157:0/3566555569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63680c81b0 0x7f636815acc0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f635800c5c0 tx=0x7f635800c8d0 comp rx=0 tx=0).stop 2023-10-06T04:05:16.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.824+0000 7f636d350700 1 -- 172.21.15.157:0/3566555569 shutdown_connections 2023-10-06T04:05:16.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.824+0000 7f636d350700 1 --2- 172.21.15.157:0/3566555569 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6348066f30 0x7f63480693e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:16.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.824+0000 7f636d350700 1 --2- 172.21.15.157:0/3566555569 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6368157490 0x7f636814ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:16.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.824+0000 7f636d350700 1 --2- 172.21.15.157:0/3566555569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63680c81b0 0x7f636815acc0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:16.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.824+0000 7f636d350700 1 --2- 172.21.15.157:0/3566555569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63680c97d0 0x7f6368169750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:16.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.824+0000 7f636d350700 1 -- 172.21.15.157:0/3566555569 >> 172.21.15.157:0/3566555569 conn(0x7f63680cd0b0 msgr2=0x7f636806dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:16.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.824+0000 7f636d350700 1 -- 172.21.15.157:0/3566555569 shutdown_connections 2023-10-06T04:05:16.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:16.824+0000 7f636d350700 1 -- 172.21.15.157:0/3566555569 wait complete. 2023-10-06T04:05:16.827 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 63 2023-10-06T04:05:16.838 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-06T04:05:18.839 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:05:18.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.965+0000 7f63b131d700 1 Processor -- start 2023-10-06T04:05:18.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.965+0000 7f63b131d700 1 -- start start 2023-10-06T04:05:18.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.965+0000 7f63b131d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f63ac14dcd0 0x7f63ac153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:18.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.965+0000 7f63b131d700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ac0c5c00 0x7f63ac0ca440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:18.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.965+0000 7f63b131d700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63ac0c5770 con 0x7f63ac14e0a0 2023-10-06T04:05:18.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.965+0000 7f63b131d700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63ac154c80 con 0x7f63ac14dcd0 2023-10-06T04:05:18.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.965+0000 7f63b131d700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac154200 con 0x7f63ac0c5c00 2023-10-06T04:05:18.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.966+0000 7f63aaffd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f63ac14dcd0 0x7f63ac153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:18.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.966+0000 7f63a3fff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f63ac14e0a0 0x7f63ac0c18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38570/0 (socket says 172.21.15.157:38570) 2023-10-06T04:05:18.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.966+0000 7f63a3fff700 1 -- 172.21.15.157:0/367888992 learned_addr learned my addr 172.21.15.157:0/367888992 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:05:18.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.966+0000 7f63ab7fe700 1 --2- 172.21.15.157:0/367888992 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ac0c5c00 0x7f63ac0ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:18.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.966+0000 7f63aaffd700 1 -- 172.21.15.157:0/367888992 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ac0c5c00 msgr2=0x7f63ac0ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:18.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.966+0000 7f63aaffd700 1 --2- 172.21.15.157:0/367888992 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ac0c5c00 0x7f63ac0ca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:18.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.966+0000 7f63aaffd700 1 -- 172.21.15.157:0/367888992 >> v1:172.21.15.157:6789/0 conn(0x7f63ac14e0a0 legacy=0x7f63ac0c18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:18.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.966+0000 7f63aaffd700 1 -- 172.21.15.157:0/367888992 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ac1543e0 con 0x7f63ac14dcd0 2023-10-06T04:05:18.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.966+0000 7f63aa7fc700 1 -- 172.21.15.157:0/367888992 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2085823754 0 0) 0x7f63ac0c5770 con 0x7f63ac14e0a0 2023-10-06T04:05:18.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.966+0000 7f63ab7fe700 1 --2- 172.21.15.157:0/367888992 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ac0c5c00 0x7f63ac0ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:05:18.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.966+0000 7f63aaffd700 1 --2- 172.21.15.157:0/367888992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63ac14dcd0 0x7f63ac153cc0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f639c009910 tx=0x7f639c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab081f731873acde server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:18.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.966+0000 7f63aa7fc700 1 -- 172.21.15.157:0/367888992 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c00b8b0 con 0x7f63ac14dcd0 2023-10-06T04:05:18.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.966+0000 7f63aa7fc700 1 -- 172.21.15.157:0/367888992 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639c00ba10 con 0x7f63ac14dcd0 2023-10-06T04:05:18.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.966+0000 7f63aa7fc700 1 -- 172.21.15.157:0/367888992 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c00bdb0 con 0x7f63ac14dcd0 2023-10-06T04:05:18.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.967+0000 7f63b131d700 1 -- 172.21.15.157:0/367888992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63ac14dcd0 msgr2=0x7f63ac153cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:18.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.967+0000 7f63b131d700 1 --2- 172.21.15.157:0/367888992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63ac14dcd0 0x7f63ac153cc0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f639c009910 tx=0x7f639c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:05:18.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.967+0000 7f63b131d700 1 -- 172.21.15.157:0/367888992 shutdown_connections 2023-10-06T04:05:18.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.967+0000 7f63b131d700 1 --2- 172.21.15.157:0/367888992 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ac0c5c00 0x7f63ac0ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:18.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.967+0000 7f63b131d700 1 --2- 172.21.15.157:0/367888992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63ac14dcd0 0x7f63ac153cc0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:18.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.967+0000 7f63b131d700 1 -- 172.21.15.157:0/367888992 >> 172.21.15.157:0/367888992 conn(0x7f63ac0692b0 msgr2=0x7f63ac0696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:18.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.967+0000 7f63b131d700 1 -- 172.21.15.157:0/367888992 shutdown_connections 2023-10-06T04:05:18.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.967+0000 7f63b131d700 1 -- 172.21.15.157:0/367888992 wait complete. 2023-10-06T04:05:18.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.968+0000 7f63b131d700 1 Processor -- start 2023-10-06T04:05:18.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.968+0000 7f63b131d700 1 -- start start 2023-10-06T04:05:18.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.969+0000 7f63b131d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63ac0c5c00 0x7f63ac0c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:18.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.969+0000 7f63b131d700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ac14dcd0 0x7f63ac1595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:18.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.969+0000 7f63b131d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63ac14e0a0 0x7f63ac163c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:18.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.969+0000 7f63b131d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac164ba0 con 0x7f63ac0c5c00 2023-10-06T04:05:18.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.969+0000 7f63b131d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac0c8d80 con 0x7f63ac14e0a0 2023-10-06T04:05:18.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.969+0000 7f63b131d700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac1645e0 con 0x7f63ac14dcd0 2023-10-06T04:05:18.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.969+0000 7f63aaffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63ac0c5c00 0x7f63ac0c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:18.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.969+0000 7f63aaffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63ac0c5c00 0x7f63ac0c9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49388/0 (socket says 172.21.15.157:49388) 2023-10-06T04:05:18.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.969+0000 7f63aaffd700 1 -- 172.21.15.157:0/1190005294 learned_addr learned my addr 172.21.15.157:0/1190005294 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:18.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.969+0000 7f63ab7fe700 1 --2- 172.21.15.157:0/1190005294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63ac14e0a0 0x7f63ac163c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:18.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.969+0000 7f63a3fff700 1 --2- 172.21.15.157:0/1190005294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ac14dcd0 0x7f63ac1595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:18.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.969+0000 7f63aaffd700 1 -- 172.21.15.157:0/1190005294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ac14dcd0 msgr2=0x7f63ac1595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:18.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.969+0000 7f63aaffd700 1 --2- 172.21.15.157:0/1190005294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ac14dcd0 0x7f63ac1595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:18.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.969+0000 7f63aaffd700 1 -- 172.21.15.157:0/1190005294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63ac14e0a0 msgr2=0x7f63ac163c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:18.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.969+0000 7f63aaffd700 1 --2- 172.21.15.157:0/1190005294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63ac14e0a0 0x7f63ac163c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:18.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.969+0000 7f63aaffd700 1 -- 172.21.15.157:0/1190005294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639c009420 con 0x7f63ac0c5c00 2023-10-06T04:05:18.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.969+0000 7f63ab7fe700 1 --2- 172.21.15.157:0/1190005294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63ac14e0a0 0x7f63ac163c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:05:18.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.969+0000 7f63aaffd700 1 --2- 172.21.15.157:0/1190005294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63ac0c5c00 0x7f63ac0c9ae0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f639c00b3c0 tx=0x7f639c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:18.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.969+0000 7f63a2ffd700 1 -- 172.21.15.157:0/1190005294 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c01f070 con 0x7f63ac0c5c00 2023-10-06T04:05:18.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.969+0000 7f63a2ffd700 1 -- 172.21.15.157:0/1190005294 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639c002f70 con 0x7f63ac0c5c00 2023-10-06T04:05:18.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.969+0000 7f63b131d700 1 -- 172.21.15.157:0/1190005294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63ac1648c0 con 0x7f63ac0c5c00 2023-10-06T04:05:18.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.969+0000 7f63a2ffd700 1 -- 172.21.15.157:0/1190005294 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c006e50 con 0x7f63ac0c5c00 2023-10-06T04:05:18.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.969+0000 7f63b131d700 1 -- 172.21.15.157:0/1190005294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63ac0c8a10 con 0x7f63ac0c5c00 2023-10-06T04:05:18.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.970+0000 7f63b131d700 1 -- 172.21.15.157:0/1190005294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63ac06a8f0 con 0x7f63ac0c5c00 2023-10-06T04:05:18.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.971+0000 7f63a2ffd700 1 -- 172.21.15.157:0/1190005294 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f639c023070 con 0x7f63ac0c5c00 2023-10-06T04:05:18.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.971+0000 7f63a2ffd700 1 --2- 172.21.15.157:0/1190005294 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f638c066f60 0x7f638c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:18.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.971+0000 7f63a2ffd700 1 -- 172.21.15.157:0/1190005294 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6402+0+0 (secure 0 0 0) 0x7f639c085d90 con 0x7f63ac0c5c00 2023-10-06T04:05:18.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.971+0000 7f63a3fff700 1 --2- 172.21.15.157:0/1190005294 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f638c066f60 0x7f638c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:18.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.972+0000 7f63a3fff700 1 --2- 172.21.15.157:0/1190005294 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f638c066f60 0x7f638c069410 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f63ac156750 tx=0x7f6394009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:18.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:18.975+0000 7f63a2ffd700 1 -- 172.21.15.157:0/1190005294 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f639c052d40 con 0x7f63ac0c5c00 2023-10-06T04:05:19.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:19.200+0000 7f63b131d700 1 -- 172.21.15.157:0/1190005294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63ac0c2a60 con 0x7f63ac0c5c00 2023-10-06T04:05:19.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:19.200+0000 7f63a2ffd700 1 -- 172.21.15.157:0/1190005294 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6145 (secure 0 0 0) 0x7f639c015090 con 0x7f63ac0c5c00 2023-10-06T04:05:19.202 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:05:19.202 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5799,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:04:48.805873+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:05:19.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:19.202+0000 7f63b131d700 1 -- 172.21.15.157:0/1190005294 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f638c066f60 msgr2=0x7f638c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:19.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:19.202+0000 7f63b131d700 1 --2- 172.21.15.157:0/1190005294 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f638c066f60 0x7f638c069410 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f63ac156750 tx=0x7f6394009040 comp rx=0 tx=0).stop 2023-10-06T04:05:19.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:19.202+0000 7f63b131d700 1 -- 172.21.15.157:0/1190005294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63ac0c5c00 msgr2=0x7f63ac0c9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:19.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:19.202+0000 7f63b131d700 1 --2- 172.21.15.157:0/1190005294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63ac0c5c00 0x7f63ac0c9ae0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f639c00b3c0 tx=0x7f639c017000 comp rx=0 tx=0).stop 2023-10-06T04:05:19.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:19.202+0000 7f63b131d700 1 -- 172.21.15.157:0/1190005294 shutdown_connections 2023-10-06T04:05:19.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:19.202+0000 7f63b131d700 1 --2- 172.21.15.157:0/1190005294 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f638c066f60 0x7f638c069410 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:19.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:19.202+0000 7f63b131d700 1 --2- 172.21.15.157:0/1190005294 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63ac14e0a0 0x7f63ac163c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:19.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:19.202+0000 7f63b131d700 1 --2- 172.21.15.157:0/1190005294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63ac0c5c00 0x7f63ac0c9ae0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:19.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:19.202+0000 7f63b131d700 1 --2- 172.21.15.157:0/1190005294 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ac14dcd0 0x7f63ac1595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:19.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:19.203+0000 7f63b131d700 1 -- 172.21.15.157:0/1190005294 >> 172.21.15.157:0/1190005294 conn(0x7f63ac0692b0 msgr2=0x7f63ac14aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:19.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:19.203+0000 7f63b131d700 1 -- 172.21.15.157:0/1190005294 shutdown_connections 2023-10-06T04:05:19.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:19.203+0000 7f63b131d700 1 -- 172.21.15.157:0/1190005294 wait complete. 2023-10-06T04:05:19.205 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 63 2023-10-06T04:05:19.216 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-06T04:05:21.218 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:05:21.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.345+0000 7f2e87532700 1 Processor -- start 2023-10-06T04:05:21.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.345+0000 7f2e87532700 1 -- start start 2023-10-06T04:05:21.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.346+0000 7f2e87532700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2e800b1e40 0x7f2e80157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:21.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.346+0000 7f2e87532700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e800b04d0 0x7f2e8006dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:21.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.346+0000 7f2e87532700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e8006d8d0 con 0x7f2e8006c2d0 2023-10-06T04:05:21.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.346+0000 7f2e87532700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e800af8f0 con 0x7f2e800b1e40 2023-10-06T04:05:21.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.346+0000 7f2e87532700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8015a690 con 0x7f2e800b04d0 2023-10-06T04:05:21.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.346+0000 7f2e86530700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2e800b1e40 0x7f2e80157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:21.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.346+0000 7f2e85d2f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e800b04d0 0x7f2e8006dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:21.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.346+0000 7f2e86530700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2e800b1e40 0x7f2e80157750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:35248/0 (socket says 172.21.15.157:35248) 2023-10-06T04:05:21.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.346+0000 7f2e85d2f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e800b04d0 0x7f2e8006dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:48244/0 (socket says 172.21.15.157:48244) 2023-10-06T04:05:21.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.346+0000 7f2e86530700 1 -- 172.21.15.157:0/2020370452 learned_addr learned my addr 172.21.15.157:0/2020370452 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:21.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.346+0000 7f2e86d31700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f2e8006c2d0 0x7f2e801563b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.157:42598/0 (socket says 172.21.15.157:42598) 2023-10-06T04:05:21.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.347+0000 7f2e86530700 1 -- 172.21.15.157:0/2020370452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e800b04d0 msgr2=0x7f2e8006dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:21.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.347+0000 7f2e86530700 1 --2- 172.21.15.157:0/2020370452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e800b04d0 0x7f2e8006dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:21.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.347+0000 7f2e86530700 1 -- 172.21.15.157:0/2020370452 >> v1:172.21.15.175:6789/0 conn(0x7f2e8006c2d0 legacy=0x7f2e801563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:21.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.347+0000 7f2e86530700 1 -- 172.21.15.157:0/2020370452 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8006ccc0 con 0x7f2e800b1e40 2023-10-06T04:05:21.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.347+0000 7f2e85d2f700 1 --2- 172.21.15.157:0/2020370452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e800b04d0 0x7f2e8006dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:05:21.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.347+0000 7f2e86530700 1 --2- 172.21.15.157:0/2020370452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e800b1e40 0x7f2e80157750 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0099b0 tx=0x7f2e7c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2573a5a30130f3d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:21.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.347+0000 7f2e8552e700 1 -- 172.21.15.157:0/2020370452 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c009ea0 con 0x7f2e800b1e40 2023-10-06T04:05:21.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.347+0000 7f2e8552e700 1 -- 172.21.15.157:0/2020370452 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e7c00b860 con 0x7f2e800b1e40 2023-10-06T04:05:21.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.347+0000 7f2e8552e700 1 -- 172.21.15.157:0/2020370452 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c00bc20 con 0x7f2e800b1e40 2023-10-06T04:05:21.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.348+0000 7f2e87532700 1 -- 172.21.15.157:0/2020370452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e800b1e40 msgr2=0x7f2e80157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:21.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.348+0000 7f2e87532700 1 --2- 172.21.15.157:0/2020370452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e800b1e40 0x7f2e80157750 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0099b0 tx=0x7f2e7c009250 comp rx=0 tx=0).stop 2023-10-06T04:05:21.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.348+0000 7f2e87532700 1 -- 172.21.15.157:0/2020370452 shutdown_connections 2023-10-06T04:05:21.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.348+0000 7f2e87532700 1 --2- 172.21.15.157:0/2020370452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e800b1e40 0x7f2e80157750 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:21.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.348+0000 7f2e87532700 1 --2- 172.21.15.157:0/2020370452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e800b04d0 0x7f2e8006dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:21.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.348+0000 7f2e87532700 1 -- 172.21.15.157:0/2020370452 >> 172.21.15.157:0/2020370452 conn(0x7f2e800cc540 msgr2=0x7f2e800cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:21.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.348+0000 7f2e87532700 1 -- 172.21.15.157:0/2020370452 shutdown_connections 2023-10-06T04:05:21.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.348+0000 7f2e87532700 1 -- 172.21.15.157:0/2020370452 wait complete. 2023-10-06T04:05:21.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.349+0000 7f2e87532700 1 Processor -- start 2023-10-06T04:05:21.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.349+0000 7f2e87532700 1 -- start start 2023-10-06T04:05:21.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.349+0000 7f2e87532700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e8006c2d0 0x7f2e8014ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:21.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.349+0000 7f2e87532700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e800b04d0 0x7f2e801694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:21.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.349+0000 7f2e87532700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e800b1e40 0x7f2e801699e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:21.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.350+0000 7f2e87532700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e800cbc70 con 0x7f2e8006c2d0 2023-10-06T04:05:21.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.350+0000 7f2e87532700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e80166710 con 0x7f2e800b1e40 2023-10-06T04:05:21.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.350+0000 7f2e87532700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e800cb6b0 con 0x7f2e800b04d0 2023-10-06T04:05:21.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.350+0000 7f2e86530700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e8006c2d0 0x7f2e8014ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:21.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.350+0000 7f2e86530700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e8006c2d0 0x7f2e8014ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49392/0 (socket says 172.21.15.157:49392) 2023-10-06T04:05:21.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.350+0000 7f2e86530700 1 -- 172.21.15.157:0/3670323415 learned_addr learned my addr 172.21.15.157:0/3670323415 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:21.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.350+0000 7f2e85d2f700 1 --2- 172.21.15.157:0/3670323415 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e800b04d0 0x7f2e801694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:21.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.350+0000 7f2e86d31700 1 --2- 172.21.15.157:0/3670323415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e800b1e40 0x7f2e801699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:21.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.350+0000 7f2e86530700 1 -- 172.21.15.157:0/3670323415 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e800b04d0 msgr2=0x7f2e801694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:21.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.350+0000 7f2e86530700 1 --2- 172.21.15.157:0/3670323415 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e800b04d0 0x7f2e801694a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:21.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.350+0000 7f2e86530700 1 -- 172.21.15.157:0/3670323415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e800b1e40 msgr2=0x7f2e801699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:21.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.350+0000 7f2e86530700 1 --2- 172.21.15.157:0/3670323415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e800b1e40 0x7f2e801699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:21.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.350+0000 7f2e86530700 1 -- 172.21.15.157:0/3670323415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e70009420 con 0x7f2e8006c2d0 2023-10-06T04:05:21.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.350+0000 7f2e86530700 1 --2- 172.21.15.157:0/3670323415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e8006c2d0 0x7f2e8014ea50 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2e800899e0 tx=0x7f2e7c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:21.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.350+0000 7f2e767fc700 1 -- 172.21.15.157:0/3670323415 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c01f070 con 0x7f2e8006c2d0 2023-10-06T04:05:21.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.350+0000 7f2e767fc700 1 -- 172.21.15.157:0/3670323415 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e7c006ab0 con 0x7f2e8006c2d0 2023-10-06T04:05:21.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.350+0000 7f2e87532700 1 -- 172.21.15.157:0/3670323415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e7c0094f0 con 0x7f2e8006c2d0 2023-10-06T04:05:21.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.350+0000 7f2e767fc700 1 -- 172.21.15.157:0/3670323415 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c00b9d0 con 0x7f2e8006c2d0 2023-10-06T04:05:21.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.350+0000 7f2e87532700 1 -- 172.21.15.157:0/3670323415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e801660c0 con 0x7f2e8006c2d0 2023-10-06T04:05:21.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.351+0000 7f2e87532700 1 -- 172.21.15.157:0/3670323415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e80165f20 con 0x7f2e8006c2d0 2023-10-06T04:05:21.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.352+0000 7f2e767fc700 1 -- 172.21.15.157:0/3670323415 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2e7c023070 con 0x7f2e8006c2d0 2023-10-06T04:05:21.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.352+0000 7f2e767fc700 1 --2- 172.21.15.157:0/3670323415 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2e64066f60 0x7f2e64069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:21.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.352+0000 7f2e767fc700 1 -- 172.21.15.157:0/3670323415 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6402+0+0 (secure 0 0 0) 0x7f2e7c085a20 con 0x7f2e8006c2d0 2023-10-06T04:05:21.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.352+0000 7f2e85d2f700 1 --2- 172.21.15.157:0/3670323415 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2e64066f60 0x7f2e64069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:21.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.353+0000 7f2e85d2f700 1 --2- 172.21.15.157:0/3670323415 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2e64066f60 0x7f2e64069410 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2e700093f0 tx=0x7f2e70008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:21.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.356+0000 7f2e767fc700 1 -- 172.21.15.157:0/3670323415 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e80165f20 con 0x7f2e8006c2d0 2023-10-06T04:05:21.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.586+0000 7f2e87532700 1 -- 172.21.15.157:0/3670323415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e80165f20 con 0x7f2e8006c2d0 2023-10-06T04:05:21.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.586+0000 7f2e767fc700 1 -- 172.21.15.157:0/3670323415 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6145 (secure 0 0 0) 0x7f2e80165f20 con 0x7f2e8006c2d0 2023-10-06T04:05:21.588 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:05:21.588 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5799,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:04:48.805873+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:05:21.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.588+0000 7f2e87532700 1 -- 172.21.15.157:0/3670323415 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2e64066f60 msgr2=0x7f2e64069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:21.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.588+0000 7f2e87532700 1 --2- 172.21.15.157:0/3670323415 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2e64066f60 0x7f2e64069410 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2e700093f0 tx=0x7f2e70008040 comp rx=0 tx=0).stop 2023-10-06T04:05:21.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.588+0000 7f2e87532700 1 -- 172.21.15.157:0/3670323415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e8006c2d0 msgr2=0x7f2e8014ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:21.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.588+0000 7f2e87532700 1 --2- 172.21.15.157:0/3670323415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e8006c2d0 0x7f2e8014ea50 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2e800899e0 tx=0x7f2e7c017000 comp rx=0 tx=0).stop 2023-10-06T04:05:21.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.588+0000 7f2e87532700 1 -- 172.21.15.157:0/3670323415 shutdown_connections 2023-10-06T04:05:21.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.588+0000 7f2e87532700 1 --2- 172.21.15.157:0/3670323415 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2e64066f60 0x7f2e64069410 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:21.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.588+0000 7f2e87532700 1 --2- 172.21.15.157:0/3670323415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e800b1e40 0x7f2e801699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:21.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.588+0000 7f2e87532700 1 --2- 172.21.15.157:0/3670323415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e8006c2d0 0x7f2e8014ea50 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:21.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.588+0000 7f2e87532700 1 --2- 172.21.15.157:0/3670323415 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e800b04d0 0x7f2e801694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:21.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.589+0000 7f2e87532700 1 -- 172.21.15.157:0/3670323415 >> 172.21.15.157:0/3670323415 conn(0x7f2e800cc540 msgr2=0x7f2e80155320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:21.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.589+0000 7f2e87532700 1 -- 172.21.15.157:0/3670323415 shutdown_connections 2023-10-06T04:05:21.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:21.589+0000 7f2e87532700 1 -- 172.21.15.157:0/3670323415 wait complete. 2023-10-06T04:05:21.593 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 63 2023-10-06T04:05:21.602 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-06T04:05:23.603 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:05:23.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.730+0000 7f54f8d5b700 1 Processor -- start 2023-10-06T04:05:23.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.730+0000 7f54f8d5b700 1 -- start start 2023-10-06T04:05:23.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.731+0000 7f54f8d5b700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f54f4067da0 0x7f54f40c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:23.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.731+0000 7f54f8d5b700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f54f4068170 0x7f54f4153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:23.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.731+0000 7f54f8d5b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54f40c4cb0 0x7f54f4154310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:23.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.731+0000 7f54f8d5b700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54f40c1020 con 0x7f54f4067da0 2023-10-06T04:05:23.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.731+0000 7f54f8d5b700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54f4157b00 con 0x7f54f4068170 2023-10-06T04:05:23.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.731+0000 7f54f8d5b700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f40c0cf0 con 0x7f54f40c4cb0 2023-10-06T04:05:23.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.731+0000 7f54f37fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f54f4067da0 0x7f54f40c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:23.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.731+0000 7f54f37fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f54f4067da0 0x7f54f40c7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38322/0 (socket says 172.21.15.157:38322) 2023-10-06T04:05:23.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.731+0000 7f54f37fe700 1 -- 172.21.15.157:0/1190646864 learned_addr learned my addr 172.21.15.157:0/1190646864 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:23.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.731+0000 7f54f2ffd700 1 --2- 172.21.15.157:0/1190646864 >> v2:172.21.15.175:3300/0 conn(0x7f54f4068170 0x7f54f4153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:23.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.731+0000 7f54f3fff700 1 --2- 172.21.15.157:0/1190646864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54f40c4cb0 0x7f54f4154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:23.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.732+0000 7f54f37fe700 1 -- 172.21.15.157:0/1190646864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54f40c4cb0 msgr2=0x7f54f4154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:23.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.732+0000 7f54f37fe700 1 --2- 172.21.15.157:0/1190646864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54f40c4cb0 0x7f54f4154310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:23.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.732+0000 7f54f37fe700 1 -- 172.21.15.157:0/1190646864 >> v2:172.21.15.175:3300/0 conn(0x7f54f4068170 msgr2=0x7f54f4153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:23.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.732+0000 7f54f37fe700 1 --2- 172.21.15.157:0/1190646864 >> v2:172.21.15.175:3300/0 conn(0x7f54f4068170 0x7f54f4153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:23.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.732+0000 7f54f37fe700 1 -- 172.21.15.157:0/1190646864 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f4157da0 con 0x7f54f4067da0 2023-10-06T04:05:23.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.732+0000 7f54f2ffd700 1 --2- 172.21.15.157:0/1190646864 >> v2:172.21.15.175:3300/0 conn(0x7f54f4068170 0x7f54f4153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:05:23.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.732+0000 7f54f37fe700 1 --2- 172.21.15.157:0/1190646864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f4067da0 0x7f54f40c7930 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f54f406af40 tx=0x7f54e4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4843456b5021a72 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:23.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.732+0000 7f54f27fc700 1 -- 172.21.15.157:0/1190646864 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e400e070 con 0x7f54f4067da0 2023-10-06T04:05:23.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.732+0000 7f54f27fc700 1 -- 172.21.15.157:0/1190646864 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e400ba50 con 0x7f54f4067da0 2023-10-06T04:05:23.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.732+0000 7f54f27fc700 1 -- 172.21.15.157:0/1190646864 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e40092e0 con 0x7f54f4067da0 2023-10-06T04:05:23.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.732+0000 7f54f8d5b700 1 -- 172.21.15.157:0/1190646864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f4067da0 msgr2=0x7f54f40c7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:23.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.732+0000 7f54f8d5b700 1 --2- 172.21.15.157:0/1190646864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f4067da0 0x7f54f40c7930 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f54f406af40 tx=0x7f54e4009840 comp rx=0 tx=0).stop 2023-10-06T04:05:23.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.732+0000 7f54f8d5b700 1 -- 172.21.15.157:0/1190646864 shutdown_connections 2023-10-06T04:05:23.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.732+0000 7f54f8d5b700 1 --2- 172.21.15.157:0/1190646864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54f40c4cb0 0x7f54f4154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:23.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.732+0000 7f54f8d5b700 1 --2- 172.21.15.157:0/1190646864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f4067da0 0x7f54f40c7930 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:23.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.732+0000 7f54f8d5b700 1 --2- 172.21.15.157:0/1190646864 >> v2:172.21.15.175:3300/0 conn(0x7f54f4068170 0x7f54f4153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:23.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.732+0000 7f54f8d5b700 1 -- 172.21.15.157:0/1190646864 >> 172.21.15.157:0/1190646864 conn(0x7f54f406c1f0 msgr2=0x7f54f406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:23.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.732+0000 7f54f8d5b700 1 -- 172.21.15.157:0/1190646864 shutdown_connections 2023-10-06T04:05:23.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.732+0000 7f54f8d5b700 1 -- 172.21.15.157:0/1190646864 wait complete. 2023-10-06T04:05:23.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.734+0000 7f54f8d5b700 1 Processor -- start 2023-10-06T04:05:23.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.734+0000 7f54f8d5b700 1 -- start start 2023-10-06T04:05:23.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.734+0000 7f54f8d5b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f4067da0 0x7f54f40c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:23.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.734+0000 7f54f8d5b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54f4068170 0x7f54f414cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:23.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.734+0000 7f54f8d5b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f40c4cb0 0x7f54f414d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:23.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.734+0000 7f54f8d5b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f40c16c0 con 0x7f54f40c4cb0 2023-10-06T04:05:23.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.734+0000 7f54f8d5b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f4153600 con 0x7f54f4067da0 2023-10-06T04:05:23.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.734+0000 7f54f8d5b700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f414d5a0 con 0x7f54f4068170 2023-10-06T04:05:23.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.735+0000 7f54f3fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f40c4cb0 0x7f54f414d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:23.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.735+0000 7f54f3fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f40c4cb0 0x7f54f414d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38326/0 (socket says 172.21.15.157:38326) 2023-10-06T04:05:23.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.735+0000 7f54f3fff700 1 -- 172.21.15.157:0/1106382073 learned_addr learned my addr 172.21.15.157:0/1106382073 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:23.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.735+0000 7f54f2ffd700 1 --2- 172.21.15.157:0/1106382073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54f4068170 0x7f54f414cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:23.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.735+0000 7f54f37fe700 1 --2- 172.21.15.157:0/1106382073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f4067da0 0x7f54f40c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:23.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.735+0000 7f54f3fff700 1 -- 172.21.15.157:0/1106382073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54f4068170 msgr2=0x7f54f414cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:23.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.735+0000 7f54f3fff700 1 --2- 172.21.15.157:0/1106382073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54f4068170 0x7f54f414cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:23.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.735+0000 7f54f3fff700 1 -- 172.21.15.157:0/1106382073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f4067da0 msgr2=0x7f54f40c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:23.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.735+0000 7f54f3fff700 1 --2- 172.21.15.157:0/1106382073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f4067da0 0x7f54f40c92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:23.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.735+0000 7f54f3fff700 1 -- 172.21.15.157:0/1106382073 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e4009ae0 con 0x7f54f40c4cb0 2023-10-06T04:05:23.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.735+0000 7f54f37fe700 1 --2- 172.21.15.157:0/1106382073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f4067da0 0x7f54f40c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:05:23.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.735+0000 7f54f3fff700 1 --2- 172.21.15.157:0/1106382073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f40c4cb0 0x7f54f414d060 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00d860 tx=0x7f54ec008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:23.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.735+0000 7f54db7fe700 1 -- 172.21.15.157:0/1106382073 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec0125f0 con 0x7f54f40c4cb0 2023-10-06T04:05:23.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.735+0000 7f54db7fe700 1 -- 172.21.15.157:0/1106382073 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54ec012750 con 0x7f54f40c4cb0 2023-10-06T04:05:23.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.735+0000 7f54f8d5b700 1 -- 172.21.15.157:0/1106382073 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54f40c8760 con 0x7f54f40c4cb0 2023-10-06T04:05:23.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.735+0000 7f54db7fe700 1 -- 172.21.15.157:0/1106382073 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec012b10 con 0x7f54f40c4cb0 2023-10-06T04:05:23.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.735+0000 7f54f8d5b700 1 -- 172.21.15.157:0/1106382073 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f40af0a0 con 0x7f54f40c4cb0 2023-10-06T04:05:23.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.736+0000 7f54f8d5b700 1 -- 172.21.15.157:0/1106382073 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54f40c1870 con 0x7f54f40c4cb0 2023-10-06T04:05:23.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.737+0000 7f54db7fe700 1 -- 172.21.15.157:0/1106382073 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f54ec01e460 con 0x7f54f40c4cb0 2023-10-06T04:05:23.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.737+0000 7f54db7fe700 1 --2- 172.21.15.157:0/1106382073 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f54e0066f30 0x7f54e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:23.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.738+0000 7f54db7fe700 1 -- 172.21.15.157:0/1106382073 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6402+0+0 (secure 0 0 0) 0x7f54ec084eb0 con 0x7f54f40c4cb0 2023-10-06T04:05:23.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.738+0000 7f54f37fe700 1 --2- 172.21.15.157:0/1106382073 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f54e0066f30 0x7f54e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:23.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.738+0000 7f54f37fe700 1 --2- 172.21.15.157:0/1106382073 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f54e0066f30 0x7f54e00693e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009ab0 tx=0x7f54e4018000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:23.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.743+0000 7f54db7fe700 1 -- 172.21.15.157:0/1106382073 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f54f40c1870 con 0x7f54f40c4cb0 2023-10-06T04:05:23.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.971+0000 7f54f8d5b700 1 -- 172.21.15.157:0/1106382073 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54f40c1800 con 0x7f54f40c4cb0 2023-10-06T04:05:23.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.971+0000 7f54db7fe700 1 -- 172.21.15.157:0/1106382073 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6145 (secure 0 0 0) 0x7f54ec0558f0 con 0x7f54f40c4cb0 2023-10-06T04:05:23.973 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:05:23.973 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5799,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:04:48.805873+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:05:23.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.973+0000 7f54f8d5b700 1 -- 172.21.15.157:0/1106382073 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f54e0066f30 msgr2=0x7f54e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:23.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.973+0000 7f54f8d5b700 1 --2- 172.21.15.157:0/1106382073 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f54e0066f30 0x7f54e00693e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009ab0 tx=0x7f54e4018000 comp rx=0 tx=0).stop 2023-10-06T04:05:23.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.973+0000 7f54f8d5b700 1 -- 172.21.15.157:0/1106382073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f40c4cb0 msgr2=0x7f54f414d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:23.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.973+0000 7f54f8d5b700 1 --2- 172.21.15.157:0/1106382073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f40c4cb0 0x7f54f414d060 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00d860 tx=0x7f54ec008040 comp rx=0 tx=0).stop 2023-10-06T04:05:23.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.973+0000 7f54f8d5b700 1 -- 172.21.15.157:0/1106382073 shutdown_connections 2023-10-06T04:05:23.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.973+0000 7f54f8d5b700 1 --2- 172.21.15.157:0/1106382073 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f54e0066f30 0x7f54e00693e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:23.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.973+0000 7f54f8d5b700 1 --2- 172.21.15.157:0/1106382073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f4067da0 0x7f54f40c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:23.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.973+0000 7f54f8d5b700 1 --2- 172.21.15.157:0/1106382073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f40c4cb0 0x7f54f414d060 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:23.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.973+0000 7f54f8d5b700 1 --2- 172.21.15.157:0/1106382073 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54f4068170 0x7f54f414cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:23.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.973+0000 7f54f8d5b700 1 -- 172.21.15.157:0/1106382073 >> 172.21.15.157:0/1106382073 conn(0x7f54f406c1f0 msgr2=0x7f54f40c5150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:23.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.973+0000 7f54f8d5b700 1 -- 172.21.15.157:0/1106382073 shutdown_connections 2023-10-06T04:05:23.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:23.973+0000 7f54f8d5b700 1 -- 172.21.15.157:0/1106382073 wait complete. 2023-10-06T04:05:23.977 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 63 2023-10-06T04:05:23.987 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-06T04:05:25.988 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:05:26.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.115+0000 7f5b268b5700 1 Processor -- start 2023-10-06T04:05:26.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.115+0000 7f5b268b5700 1 -- start start 2023-10-06T04:05:26.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.116+0000 7f5b268b5700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b2006dc40 0x7f5b2014fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:26.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.116+0000 7f5b268b5700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5b2006c1f0 0x7f5b20150240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:26.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.116+0000 7f5b268b5700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b200c9780 con 0x7f5b2006cbb0 2023-10-06T04:05:26.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.116+0000 7f5b268b5700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b200c8c40 con 0x7f5b2006c1f0 2023-10-06T04:05:26.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.116+0000 7f5b268b5700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2006e0c0 con 0x7f5b2006dc40 2023-10-06T04:05:26.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.116+0000 7f5b24e52700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5b2006c1f0 0x7f5b20150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:26.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.116+0000 7f5b1f7fe700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b2006dc40 0x7f5b2014fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:26.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.116+0000 7f5b1ffff700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f5b2006cbb0 0x7f5b2006d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.157:58646/0 (socket says 172.21.15.157:58646) 2023-10-06T04:05:26.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.116+0000 7f5b1ffff700 1 -- 172.21.15.157:0/3105692810 learned_addr learned my addr 172.21.15.157:0/3105692810 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:05:26.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.117+0000 7f5b1f7fe700 1 -- 172.21.15.157:0/3105692810 >> v2:172.21.15.175:3300/0 conn(0x7f5b2006c1f0 msgr2=0x7f5b20150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:26.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.117+0000 7f5b1f7fe700 1 --2- 172.21.15.157:0/3105692810 >> v2:172.21.15.175:3300/0 conn(0x7f5b2006c1f0 0x7f5b20150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:26.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.117+0000 7f5b1f7fe700 1 -- 172.21.15.157:0/3105692810 >> v1:172.21.15.175:6789/0 conn(0x7f5b2006cbb0 legacy=0x7f5b2006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:26.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.117+0000 7f5b1f7fe700 1 -- 172.21.15.157:0/3105692810 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b200c8e50 con 0x7f5b2006dc40 2023-10-06T04:05:26.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.117+0000 7f5b1effd700 1 -- 172.21.15.157:0/3105692810 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3661366203 0 0) 0x7f5b200c9780 con 0x7f5b2006cbb0 2023-10-06T04:05:26.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.117+0000 7f5b24e52700 1 --2- 172.21.15.157:0/3105692810 >> v2:172.21.15.175:3300/0 conn(0x7f5b2006c1f0 0x7f5b20150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:05:26.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.117+0000 7f5b1f7fe700 1 --2- 172.21.15.157:0/3105692810 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b2006dc40 0x7f5b2014fd00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5b140098e0 tx=0x7f5b140091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35821d790cb53e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:26.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.117+0000 7f5b1effd700 1 -- 172.21.15.157:0/3105692810 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1400b6d0 con 0x7f5b2006dc40 2023-10-06T04:05:26.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.117+0000 7f5b1effd700 1 -- 172.21.15.157:0/3105692810 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1400b830 con 0x7f5b2006dc40 2023-10-06T04:05:26.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.117+0000 7f5b1effd700 1 -- 172.21.15.157:0/3105692810 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1400bbd0 con 0x7f5b2006dc40 2023-10-06T04:05:26.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.118+0000 7f5b268b5700 1 -- 172.21.15.157:0/3105692810 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b2006dc40 msgr2=0x7f5b2014fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:26.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.118+0000 7f5b268b5700 1 --2- 172.21.15.157:0/3105692810 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b2006dc40 0x7f5b2014fd00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5b140098e0 tx=0x7f5b140091a0 comp rx=0 tx=0).stop 2023-10-06T04:05:26.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.118+0000 7f5b268b5700 1 -- 172.21.15.157:0/3105692810 shutdown_connections 2023-10-06T04:05:26.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.118+0000 7f5b268b5700 1 --2- 172.21.15.157:0/3105692810 >> v2:172.21.15.175:3300/0 conn(0x7f5b2006c1f0 0x7f5b20150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:26.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.118+0000 7f5b268b5700 1 --2- 172.21.15.157:0/3105692810 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b2006dc40 0x7f5b2014fd00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:26.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.118+0000 7f5b268b5700 1 -- 172.21.15.157:0/3105692810 >> 172.21.15.157:0/3105692810 conn(0x7f5b2006f230 msgr2=0x7f5b2006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:26.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.118+0000 7f5b268b5700 1 -- 172.21.15.157:0/3105692810 shutdown_connections 2023-10-06T04:05:26.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.118+0000 7f5b268b5700 1 -- 172.21.15.157:0/3105692810 wait complete. 2023-10-06T04:05:26.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.119+0000 7f5b268b5700 1 Processor -- start 2023-10-06T04:05:26.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.119+0000 7f5b268b5700 1 -- start start 2023-10-06T04:05:26.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.120+0000 7f5b268b5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b2006c1f0 0x7f5b200c4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:26.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.120+0000 7f5b268b5700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b2006cbb0 0x7f5b200c4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:26.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.120+0000 7f5b268b5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b200cbde0 0x7f5b200c4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:26.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.120+0000 7f5b268b5700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b201542a0 con 0x7f5b2006c1f0 2023-10-06T04:05:26.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.120+0000 7f5b268b5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b200c3870 con 0x7f5b200cbde0 2023-10-06T04:05:26.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.120+0000 7f5b268b5700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b200c3da0 con 0x7f5b2006cbb0 2023-10-06T04:05:26.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.120+0000 7f5b1ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b2006c1f0 0x7f5b200c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:26.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.120+0000 7f5b1ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b2006c1f0 0x7f5b200c4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38342/0 (socket says 172.21.15.157:38342) 2023-10-06T04:05:26.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.120+0000 7f5b1ffff700 1 -- 172.21.15.157:0/2205872809 learned_addr learned my addr 172.21.15.157:0/2205872809 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:26.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.120+0000 7f5b1f7fe700 1 --2- 172.21.15.157:0/2205872809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b2006cbb0 0x7f5b200c4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:26.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.120+0000 7f5b24e52700 1 --2- 172.21.15.157:0/2205872809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b200cbde0 0x7f5b200c4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:26.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.120+0000 7f5b1ffff700 1 -- 172.21.15.157:0/2205872809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b2006cbb0 msgr2=0x7f5b200c4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:26.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.120+0000 7f5b1ffff700 1 --2- 172.21.15.157:0/2205872809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b2006cbb0 0x7f5b200c4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:26.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.120+0000 7f5b1ffff700 1 -- 172.21.15.157:0/2205872809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b200cbde0 msgr2=0x7f5b200c4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:26.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.120+0000 7f5b1ffff700 1 --2- 172.21.15.157:0/2205872809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b200cbde0 0x7f5b200c4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:26.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.120+0000 7f5b1ffff700 1 -- 172.21.15.157:0/2205872809 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b18009a20 con 0x7f5b2006c1f0 2023-10-06T04:05:26.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.120+0000 7f5b1ffff700 1 --2- 172.21.15.157:0/2205872809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b2006c1f0 0x7f5b200c4010 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000c7e0 tx=0x7f5b10007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:26.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.120+0000 7f5b07fff700 1 -- 172.21.15.157:0/2205872809 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b10005b30 con 0x7f5b2006c1f0 2023-10-06T04:05:26.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.121+0000 7f5b07fff700 1 -- 172.21.15.157:0/2205872809 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b10005c90 con 0x7f5b2006c1f0 2023-10-06T04:05:26.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.121+0000 7f5b268b5700 1 -- 172.21.15.157:0/2205872809 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b14009420 con 0x7f5b2006c1f0 2023-10-06T04:05:26.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.121+0000 7f5b07fff700 1 -- 172.21.15.157:0/2205872809 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b10007300 con 0x7f5b2006c1f0 2023-10-06T04:05:26.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.121+0000 7f5b268b5700 1 -- 172.21.15.157:0/2205872809 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b20153c50 con 0x7f5b2006c1f0 2023-10-06T04:05:26.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.121+0000 7f5b268b5700 1 -- 172.21.15.157:0/2205872809 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b20153aa0 con 0x7f5b2006c1f0 2023-10-06T04:05:26.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.122+0000 7f5b07fff700 1 -- 172.21.15.157:0/2205872809 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5b100076b0 con 0x7f5b2006c1f0 2023-10-06T04:05:26.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.122+0000 7f5b07fff700 1 --2- 172.21.15.157:0/2205872809 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5b0c066f60 0x7f5b0c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:26.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.122+0000 7f5b07fff700 1 -- 172.21.15.157:0/2205872809 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6402+0+0 (secure 0 0 0) 0x7f5b100842d0 con 0x7f5b2006c1f0 2023-10-06T04:05:26.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.122+0000 7f5b1f7fe700 1 --2- 172.21.15.157:0/2205872809 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5b0c066f60 0x7f5b0c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:26.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.123+0000 7f5b1f7fe700 1 --2- 172.21.15.157:0/2205872809 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5b0c066f60 0x7f5b0c069410 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5b200c3580 tx=0x7f5b14017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:26.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.126+0000 7f5b07fff700 1 -- 172.21.15.157:0/2205872809 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5b10051300 con 0x7f5b2006c1f0 2023-10-06T04:05:26.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.349+0000 7f5b268b5700 1 -- 172.21.15.157:0/2205872809 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b200c9130 con 0x7f5b2006c1f0 2023-10-06T04:05:26.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.350+0000 7f5b07fff700 1 -- 172.21.15.157:0/2205872809 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6145 (secure 0 0 0) 0x7f5b10054920 con 0x7f5b2006c1f0 2023-10-06T04:05:26.351 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:05:26.352 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5799,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:04:48.805873+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":4296,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_4296":{"gid":4296,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/4188652695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":4188652695},{"type":"v1","addr":"172.21.15.157:6837","nonce":4188652695}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:05:26.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.352+0000 7f5b268b5700 1 -- 172.21.15.157:0/2205872809 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5b0c066f60 msgr2=0x7f5b0c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:26.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.352+0000 7f5b268b5700 1 --2- 172.21.15.157:0/2205872809 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5b0c066f60 0x7f5b0c069410 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5b200c3580 tx=0x7f5b14017000 comp rx=0 tx=0).stop 2023-10-06T04:05:26.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.352+0000 7f5b268b5700 1 -- 172.21.15.157:0/2205872809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b2006c1f0 msgr2=0x7f5b200c4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:26.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.352+0000 7f5b268b5700 1 --2- 172.21.15.157:0/2205872809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b2006c1f0 0x7f5b200c4010 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000c7e0 tx=0x7f5b10007df0 comp rx=0 tx=0).stop 2023-10-06T04:05:26.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.352+0000 7f5b268b5700 1 -- 172.21.15.157:0/2205872809 shutdown_connections 2023-10-06T04:05:26.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.352+0000 7f5b268b5700 1 --2- 172.21.15.157:0/2205872809 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5b0c066f60 0x7f5b0c069410 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:26.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.352+0000 7f5b268b5700 1 --2- 172.21.15.157:0/2205872809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b200cbde0 0x7f5b200c4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:26.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.352+0000 7f5b268b5700 1 --2- 172.21.15.157:0/2205872809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b2006c1f0 0x7f5b200c4010 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:26.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.352+0000 7f5b268b5700 1 --2- 172.21.15.157:0/2205872809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b2006cbb0 0x7f5b200c4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:26.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.352+0000 7f5b268b5700 1 -- 172.21.15.157:0/2205872809 >> 172.21.15.157:0/2205872809 conn(0x7f5b2006f230 msgr2=0x7f5b2014fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:26.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.352+0000 7f5b268b5700 1 -- 172.21.15.157:0/2205872809 shutdown_connections 2023-10-06T04:05:26.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:26.352+0000 7f5b268b5700 1 -- 172.21.15.157:0/2205872809 wait complete. 2023-10-06T04:05:26.355 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 63 2023-10-06T04:05:26.366 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-06T04:05:28.368 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:05:28.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.494+0000 7ffa837cb700 1 Processor -- start 2023-10-06T04:05:28.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.495+0000 7ffa837cb700 1 -- start start 2023-10-06T04:05:28.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.495+0000 7ffa837cb700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ffa7c06cbb0 0x7ffa7c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:28.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.495+0000 7ffa837cb700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa7c150410 0x7ffa7c06da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:28.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.495+0000 7ffa837cb700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa7c0c9780 con 0x7ffa7c06c1f0 2023-10-06T04:05:28.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.495+0000 7ffa837cb700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa7c0c8c40 con 0x7ffa7c150410 2023-10-06T04:05:28.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.495+0000 7ffa837cb700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa7c06e0c0 con 0x7ffa7c06cbb0 2023-10-06T04:05:28.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.495+0000 7ffa81d68700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa7c150410 0x7ffa7c06da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:28.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.496+0000 7ffa80d66700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ffa7c06c1f0 0x7ffa7c14fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43916/0 (socket says 172.21.15.157:43916) 2023-10-06T04:05:28.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.496+0000 7ffa80d66700 1 -- 172.21.15.157:0/2994103998 learned_addr learned my addr 172.21.15.157:0/2994103998 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:05:28.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.496+0000 7ffa81567700 1 --2- 172.21.15.157:0/2994103998 >> v2:172.21.15.175:3300/0 conn(0x7ffa7c06cbb0 0x7ffa7c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:28.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.496+0000 7ffa81d68700 1 -- 172.21.15.157:0/2994103998 >> v2:172.21.15.175:3300/0 conn(0x7ffa7c06cbb0 msgr2=0x7ffa7c06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:28.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.496+0000 7ffa81d68700 1 --2- 172.21.15.157:0/2994103998 >> v2:172.21.15.175:3300/0 conn(0x7ffa7c06cbb0 0x7ffa7c06d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:28.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.496+0000 7ffa81d68700 1 -- 172.21.15.157:0/2994103998 >> v1:172.21.15.157:6789/0 conn(0x7ffa7c06c1f0 legacy=0x7ffa7c14fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:28.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.496+0000 7ffa81d68700 1 -- 172.21.15.157:0/2994103998 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa7c0c8e50 con 0x7ffa7c150410 2023-10-06T04:05:28.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.496+0000 7ffa81d68700 1 --2- 172.21.15.157:0/2994103998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa7c150410 0x7ffa7c06da70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c06fd70 tx=0x7ffa74009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfbe2a4e7ca1dc2f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:28.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.496+0000 7ffa73fff700 1 -- 172.21.15.157:0/2994103998 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7400e070 con 0x7ffa7c150410 2023-10-06T04:05:28.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.496+0000 7ffa73fff700 1 -- 172.21.15.157:0/2994103998 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa7400bcc0 con 0x7ffa7c150410 2023-10-06T04:05:28.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.496+0000 7ffa73fff700 1 -- 172.21.15.157:0/2994103998 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa74012680 con 0x7ffa7c150410 2023-10-06T04:05:28.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.497+0000 7ffa837cb700 1 -- 172.21.15.157:0/2994103998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa7c150410 msgr2=0x7ffa7c06da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:28.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.497+0000 7ffa837cb700 1 --2- 172.21.15.157:0/2994103998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa7c150410 0x7ffa7c06da70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c06fd70 tx=0x7ffa74009780 comp rx=0 tx=0).stop 2023-10-06T04:05:28.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.497+0000 7ffa837cb700 1 -- 172.21.15.157:0/2994103998 shutdown_connections 2023-10-06T04:05:28.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.497+0000 7ffa837cb700 1 --2- 172.21.15.157:0/2994103998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa7c150410 0x7ffa7c06da70 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:28.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.497+0000 7ffa837cb700 1 --2- 172.21.15.157:0/2994103998 >> v2:172.21.15.175:3300/0 conn(0x7ffa7c06cbb0 0x7ffa7c06d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:28.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.497+0000 7ffa837cb700 1 -- 172.21.15.157:0/2994103998 >> 172.21.15.157:0/2994103998 conn(0x7ffa7c06f230 msgr2=0x7ffa7c06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:28.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.497+0000 7ffa837cb700 1 -- 172.21.15.157:0/2994103998 shutdown_connections 2023-10-06T04:05:28.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.497+0000 7ffa837cb700 1 -- 172.21.15.157:0/2994103998 wait complete. 2023-10-06T04:05:28.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.498+0000 7ffa837cb700 1 Processor -- start 2023-10-06T04:05:28.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.498+0000 7ffa837cb700 1 -- start start 2023-10-06T04:05:28.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.499+0000 7ffa837cb700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa7c06c1f0 0x7ffa7c152b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:28.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.499+0000 7ffa837cb700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa7c06cbb0 0x7ffa7c146220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:28.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.499+0000 7ffa837cb700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa7c150410 0x7ffa7c1516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:28.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.499+0000 7ffa837cb700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c151220 con 0x7ffa7c06cbb0 2023-10-06T04:05:28.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.499+0000 7ffa837cb700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c148010 con 0x7ffa7c06c1f0 2023-10-06T04:05:28.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.499+0000 7ffa837cb700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c1508f0 con 0x7ffa7c150410 2023-10-06T04:05:28.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.499+0000 7ffa80d66700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa7c06cbb0 0x7ffa7c146220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:28.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.499+0000 7ffa80d66700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa7c06cbb0 0x7ffa7c146220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38360/0 (socket says 172.21.15.157:38360) 2023-10-06T04:05:28.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.499+0000 7ffa80d66700 1 -- 172.21.15.157:0/1203869010 learned_addr learned my addr 172.21.15.157:0/1203869010 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:28.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.499+0000 7ffa81567700 1 --2- 172.21.15.157:0/1203869010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa7c06c1f0 0x7ffa7c152b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:28.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.499+0000 7ffa81d68700 1 --2- 172.21.15.157:0/1203869010 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa7c150410 0x7ffa7c1516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:28.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.499+0000 7ffa80d66700 1 -- 172.21.15.157:0/1203869010 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa7c150410 msgr2=0x7ffa7c1516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:28.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.499+0000 7ffa80d66700 1 --2- 172.21.15.157:0/1203869010 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa7c150410 0x7ffa7c1516b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:28.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.499+0000 7ffa80d66700 1 -- 172.21.15.157:0/1203869010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa7c06c1f0 msgr2=0x7ffa7c152b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:28.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.499+0000 7ffa80d66700 1 --2- 172.21.15.157:0/1203869010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa7c06c1f0 0x7ffa7c152b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:28.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.499+0000 7ffa80d66700 1 -- 172.21.15.157:0/1203869010 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa74009a20 con 0x7ffa7c06cbb0 2023-10-06T04:05:28.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.499+0000 7ffa81567700 1 --2- 172.21.15.157:0/1203869010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa7c06c1f0 0x7ffa7c152b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:05:28.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.499+0000 7ffa80d66700 1 --2- 172.21.15.157:0/1203869010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa7c06cbb0 0x7ffa7c146220 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c00c7c0 tx=0x7ffa6c007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:28.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.500+0000 7ffa717fa700 1 -- 172.21.15.157:0/1203869010 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c005b30 con 0x7ffa7c06cbb0 2023-10-06T04:05:28.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.500+0000 7ffa717fa700 1 -- 172.21.15.157:0/1203869010 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6c005cb0 con 0x7ffa7c06cbb0 2023-10-06T04:05:28.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.500+0000 7ffa837cb700 1 -- 172.21.15.157:0/1203869010 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa7c150bd0 con 0x7ffa7c06cbb0 2023-10-06T04:05:28.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.500+0000 7ffa717fa700 1 -- 172.21.15.157:0/1203869010 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c004a80 con 0x7ffa7c06cbb0 2023-10-06T04:05:28.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.500+0000 7ffa837cb700 1 -- 172.21.15.157:0/1203869010 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa7c147d00 con 0x7ffa7c06cbb0 2023-10-06T04:05:28.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.501+0000 7ffa837cb700 1 -- 172.21.15.157:0/1203869010 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa7c151390 con 0x7ffa7c06cbb0 2023-10-06T04:05:28.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.501+0000 7ffa717fa700 1 -- 172.21.15.157:0/1203869010 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffa6c0070f0 con 0x7ffa7c06cbb0 2023-10-06T04:05:28.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.502+0000 7ffa717fa700 1 --2- 172.21.15.157:0/1203869010 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ffa60066f30 0x7ffa600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:28.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.502+0000 7ffa717fa700 1 -- 172.21.15.157:0/1203869010 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ffa6c083ca0 con 0x7ffa7c06cbb0 2023-10-06T04:05:28.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.502+0000 7ffa81567700 1 --2- 172.21.15.157:0/1203869010 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ffa60066f30 0x7ffa600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:28.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.502+0000 7ffa81567700 1 --2- 172.21.15.157:0/1203869010 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ffa60066f30 0x7ffa600693e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ffa78006fd0 tx=0x7ffa78008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:28.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.506+0000 7ffa717fa700 1 -- 172.21.15.157:0/1203869010 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa6c050ca0 con 0x7ffa7c06cbb0 2023-10-06T04:05:28.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.733+0000 7ffa837cb700 1 -- 172.21.15.157:0/1203869010 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa7c0c9130 con 0x7ffa7c06cbb0 2023-10-06T04:05:28.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.733+0000 7ffa717fa700 1 -- 172.21.15.157:0/1203869010 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+5385 (secure 0 0 0) 0x7ffa6c018070 con 0x7ffa7c06cbb0 2023-10-06T04:05:28.735 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:05:28.735 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:05:26.713720+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:replay","state_seq":1,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:05:28.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.735+0000 7ffa837cb700 1 -- 172.21.15.157:0/1203869010 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ffa60066f30 msgr2=0x7ffa600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:28.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.735+0000 7ffa837cb700 1 --2- 172.21.15.157:0/1203869010 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ffa60066f30 0x7ffa600693e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ffa78006fd0 tx=0x7ffa78008040 comp rx=0 tx=0).stop 2023-10-06T04:05:28.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.735+0000 7ffa837cb700 1 -- 172.21.15.157:0/1203869010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa7c06cbb0 msgr2=0x7ffa7c146220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:28.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.735+0000 7ffa837cb700 1 --2- 172.21.15.157:0/1203869010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa7c06cbb0 0x7ffa7c146220 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c00c7c0 tx=0x7ffa6c007f70 comp rx=0 tx=0).stop 2023-10-06T04:05:28.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.735+0000 7ffa837cb700 1 -- 172.21.15.157:0/1203869010 shutdown_connections 2023-10-06T04:05:28.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.735+0000 7ffa837cb700 1 --2- 172.21.15.157:0/1203869010 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ffa60066f30 0x7ffa600693e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:28.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.735+0000 7ffa837cb700 1 --2- 172.21.15.157:0/1203869010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa7c06c1f0 0x7ffa7c152b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:28.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.735+0000 7ffa837cb700 1 --2- 172.21.15.157:0/1203869010 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa7c150410 0x7ffa7c1516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:28.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.735+0000 7ffa837cb700 1 --2- 172.21.15.157:0/1203869010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa7c06cbb0 0x7ffa7c146220 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:28.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.735+0000 7ffa837cb700 1 -- 172.21.15.157:0/1203869010 >> 172.21.15.157:0/1203869010 conn(0x7ffa7c06f230 msgr2=0x7ffa7c14c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:28.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.735+0000 7ffa837cb700 1 -- 172.21.15.157:0/1203869010 shutdown_connections 2023-10-06T04:05:28.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.735+0000 7ffa837cb700 1 -- 172.21.15.157:0/1203869010 wait complete. 2023-10-06T04:05:28.738 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 64 2023-10-06T04:05:28.749 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e down, removed from mdsmap 2023-10-06T04:05:28.749 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-10-06T04:05:28.749 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:05:28.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.862+0000 7fba51602700 1 Processor -- start 2023-10-06T04:05:28.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.862+0000 7fba51602700 1 -- start start 2023-10-06T04:05:28.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.863+0000 7fba51602700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fba4c0c1db0 0x7fba4c1545c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:28.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.863+0000 7fba51602700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba4c0bfda0 con 0x7fba4c0c2180 2023-10-06T04:05:28.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.863+0000 7fba51602700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba4c06c420 con 0x7fba4c06cfc0 2023-10-06T04:05:28.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.863+0000 7fba51602700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba4c0c05b0 con 0x7fba4c0c1db0 2023-10-06T04:05:28.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.863+0000 7fba4b7fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fba4c0c2180 0x7fba4c06c8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43924/0 (socket says 172.21.15.157:43924) 2023-10-06T04:05:28.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.863+0000 7fba4b7fe700 1 -- 172.21.15.157:0/1934963713 learned_addr learned my addr 172.21.15.157:0/1934963713 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:05:28.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.863+0000 7fba4bfff700 1 --2- 172.21.15.157:0/1934963713 >> v2:172.21.15.175:3300/0 conn(0x7fba4c0c1db0 0x7fba4c1545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:28.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.864+0000 7fba4affd700 1 -- 172.21.15.157:0/1934963713 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1385615777 0 0) 0x7fba4c0bfda0 con 0x7fba4c0c2180 2023-10-06T04:05:28.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.864+0000 7fba4affd700 1 -- 172.21.15.157:0/1934963713 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba34003570 con 0x7fba4c0c2180 2023-10-06T04:05:28.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.864+0000 7fba4affd700 1 -- 172.21.15.157:0/1934963713 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1528508202 0 0) 0x7fba34003570 con 0x7fba4c0c2180 2023-10-06T04:05:28.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.864+0000 7fba4affd700 1 -- 172.21.15.157:0/1934963713 >> v2:172.21.15.175:3300/0 conn(0x7fba4c0c1db0 msgr2=0x7fba4c1545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:28.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.864+0000 7fba4affd700 1 --2- 172.21.15.157:0/1934963713 >> v2:172.21.15.175:3300/0 conn(0x7fba4c0c1db0 0x7fba4c1545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:28.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.864+0000 7fba4affd700 1 -- 172.21.15.157:0/1934963713 >> v1:172.21.15.175:6789/0 conn(0x7fba4c06cfc0 legacy=0x7fba4c14d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:28.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.864+0000 7fba4affd700 1 -- 172.21.15.157:0/1934963713 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba4c0c9900 con 0x7fba4c0c2180 2023-10-06T04:05:28.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.864+0000 7fba4affd700 1 -- 172.21.15.157:0/1934963713 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7fba40001470 con 0x7fba4c0c2180 2023-10-06T04:05:28.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.864+0000 7fba4bfff700 1 --2- 172.21.15.157:0/1934963713 >> v2:172.21.15.175:3300/0 conn(0x7fba4c0c1db0 0x7fba4c1545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:05:28.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.864+0000 7fba4affd700 1 -- 172.21.15.157:0/1934963713 >> v1:172.21.15.157:6789/0 conn(0x7fba4c0c2180 legacy=0x7fba4c06c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:28.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.864+0000 7fba4affd700 1 --2- 172.21.15.157:0/1934963713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba34003a60 0x7fba34003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:28.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.864+0000 7fba4affd700 1 --2- 172.21.15.157:0/1934963713 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba34004810 0x7fba34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:28.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.864+0000 7fba4affd700 1 --2- 172.21.15.157:0/1934963713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba340051c0 0x7fba3400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:28.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.864+0000 7fba4affd700 1 -- 172.21.15.157:0/1934963713 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba4c0c05b0 con 0x7fba34003a60 2023-10-06T04:05:28.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.864+0000 7fba4affd700 1 -- 172.21.15.157:0/1934963713 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3400c0d0 con 0x7fba340051c0 2023-10-06T04:05:28.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.864+0000 7fba4affd700 1 -- 172.21.15.157:0/1934963713 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba3400c290 con 0x7fba34004810 2023-10-06T04:05:28.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.864+0000 7fba4bfff700 1 --2- 172.21.15.157:0/1934963713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba34003a60 0x7fba34003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:28.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.864+0000 7fba4b7fe700 1 --2- 172.21.15.157:0/1934963713 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba34004810 0x7fba34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:28.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.864+0000 7fba50e01700 1 --2- 172.21.15.157:0/1934963713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba340051c0 0x7fba3400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:28.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.864+0000 7fba4bfff700 1 -- 172.21.15.157:0/1934963713 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba34004810 msgr2=0x7fba34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:28.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.864+0000 7fba4bfff700 1 --2- 172.21.15.157:0/1934963713 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba34004810 0x7fba34004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:28.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.864+0000 7fba4bfff700 1 -- 172.21.15.157:0/1934963713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba340051c0 msgr2=0x7fba3400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:28.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.864+0000 7fba4bfff700 1 --2- 172.21.15.157:0/1934963713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba340051c0 0x7fba3400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:28.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.864+0000 7fba4bfff700 1 -- 172.21.15.157:0/1934963713 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba34003570 con 0x7fba34003a60 2023-10-06T04:05:28.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.864+0000 7fba50e01700 1 -- 172.21.15.157:0/1934963713 reap_dead start 2023-10-06T04:05:28.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.864+0000 7fba4bfff700 1 --2- 172.21.15.157:0/1934963713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba34003a60 0x7fba34003e70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fba3c009df0 tx=0x7fba3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e03f9d355ff2abb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:28.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.865+0000 7fba4affd700 1 -- 172.21.15.157:0/1934963713 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c009740 con 0x7fba34003a60 2023-10-06T04:05:28.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.865+0000 7fba4affd700 1 -- 172.21.15.157:0/1934963713 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba3c00e920 con 0x7fba34003a60 2023-10-06T04:05:28.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.865+0000 7fba4affd700 1 -- 172.21.15.157:0/1934963713 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c00ece0 con 0x7fba34003a60 2023-10-06T04:05:28.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.865+0000 7fba51602700 1 -- 172.21.15.157:0/1934963713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba34003a60 msgr2=0x7fba34003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:28.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.865+0000 7fba51602700 1 --2- 172.21.15.157:0/1934963713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba34003a60 0x7fba34003e70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fba3c009df0 tx=0x7fba3c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:05:28.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.865+0000 7fba51602700 1 -- 172.21.15.157:0/1934963713 shutdown_connections 2023-10-06T04:05:28.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.865+0000 7fba51602700 1 --2- 172.21.15.157:0/1934963713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba34003a60 0x7fba34003e70 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:28.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.865+0000 7fba51602700 1 -- 172.21.15.157:0/1934963713 >> 172.21.15.157:0/1934963713 conn(0x7fba4c06f890 msgr2=0x7fba4c0c89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:28.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.865+0000 7fba51602700 1 -- 172.21.15.157:0/1934963713 shutdown_connections 2023-10-06T04:05:28.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.865+0000 7fba51602700 1 -- 172.21.15.157:0/1934963713 wait complete. 2023-10-06T04:05:28.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.866+0000 7fba51602700 1 Processor -- start 2023-10-06T04:05:28.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.867+0000 7fba51602700 1 -- start start 2023-10-06T04:05:28.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.867+0000 7fba51602700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba4c156650 0x7fba4c14e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:28.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.867+0000 7fba51602700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba4c156a20 0x7fba4c167a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:28.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.867+0000 7fba51602700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba4c156f00 0x7fba4c1552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:28.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.867+0000 7fba51602700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba4c166ff0 con 0x7fba4c156650 2023-10-06T04:05:28.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.867+0000 7fba51602700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba4c1675b0 con 0x7fba4c156a20 2023-10-06T04:05:28.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.867+0000 7fba51602700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba4c152b30 con 0x7fba4c156f00 2023-10-06T04:05:28.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.867+0000 7fba4bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba4c156650 0x7fba4c14e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:28.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.867+0000 7fba4bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba4c156650 0x7fba4c14e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38372/0 (socket says 172.21.15.157:38372) 2023-10-06T04:05:28.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.867+0000 7fba4bfff700 1 -- 172.21.15.157:0/517564137 learned_addr learned my addr 172.21.15.157:0/517564137 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:28.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.868+0000 7fba50e01700 1 --2- 172.21.15.157:0/517564137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba4c156f00 0x7fba4c1552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:28.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.868+0000 7fba4b7fe700 1 --2- 172.21.15.157:0/517564137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba4c156a20 0x7fba4c167a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:28.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.868+0000 7fba4bfff700 1 -- 172.21.15.157:0/517564137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba4c156f00 msgr2=0x7fba4c1552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:28.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.868+0000 7fba4bfff700 1 --2- 172.21.15.157:0/517564137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba4c156f00 0x7fba4c1552a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:28.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.868+0000 7fba4bfff700 1 -- 172.21.15.157:0/517564137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba4c156a20 msgr2=0x7fba4c167a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:28.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.868+0000 7fba4bfff700 1 --2- 172.21.15.157:0/517564137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba4c156a20 0x7fba4c167a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:28.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.868+0000 7fba4bfff700 1 -- 172.21.15.157:0/517564137 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba4c152e10 con 0x7fba4c156650 2023-10-06T04:05:28.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.868+0000 7fba4bfff700 1 --2- 172.21.15.157:0/517564137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba4c156650 0x7fba4c14e4f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fba3c016000 tx=0x7fba3c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:28.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.868+0000 7fba33fff700 1 -- 172.21.15.157:0/517564137 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c022070 con 0x7fba4c156650 2023-10-06T04:05:28.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.868+0000 7fba33fff700 1 -- 172.21.15.157:0/517564137 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba3c009740 con 0x7fba4c156650 2023-10-06T04:05:28.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.868+0000 7fba51602700 1 -- 172.21.15.157:0/517564137 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba4c1530f0 con 0x7fba4c156650 2023-10-06T04:05:28.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.868+0000 7fba33fff700 1 -- 172.21.15.157:0/517564137 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c013e30 con 0x7fba4c156650 2023-10-06T04:05:28.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.868+0000 7fba51602700 1 -- 172.21.15.157:0/517564137 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba4c0c6c00 con 0x7fba4c156650 2023-10-06T04:05:28.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.869+0000 7fba51602700 1 -- 172.21.15.157:0/517564137 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba4c167400 con 0x7fba4c156650 2023-10-06T04:05:28.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.869+0000 7fba33fff700 1 -- 172.21.15.157:0/517564137 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fba3c026070 con 0x7fba4c156650 2023-10-06T04:05:28.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.870+0000 7fba33fff700 1 --2- 172.21.15.157:0/517564137 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fba38066f60 0x7fba38069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:28.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.870+0000 7fba33fff700 1 -- 172.21.15.157:0/517564137 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fba3c086020 con 0x7fba4c156650 2023-10-06T04:05:28.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.870+0000 7fba4b7fe700 1 --2- 172.21.15.157:0/517564137 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fba38066f60 0x7fba38069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:28.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.870+0000 7fba4b7fe700 1 --2- 172.21.15.157:0/517564137 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fba38066f60 0x7fba38069410 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fba40001050 tx=0x7fba40009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:28.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:28.874+0000 7fba33fff700 1 -- 172.21.15.157:0/517564137 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba3c052fa0 con 0x7fba4c156650 2023-10-06T04:05:29.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:29.100+0000 7fba51602700 1 -- 172.21.15.157:0/517564137 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba4c0c9430 con 0x7fba4c156650 2023-10-06T04:05:29.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:29.100+0000 7fba33fff700 1 -- 172.21.15.157:0/517564137 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+5385 (secure 0 0 0) 0x7fba3c0565c0 con 0x7fba4c156650 2023-10-06T04:05:29.102 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:05:29.102 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:05:26.713720+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:replay","state_seq":1,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:05:29.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:29.102+0000 7fba51602700 1 -- 172.21.15.157:0/517564137 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fba38066f60 msgr2=0x7fba38069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:29.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:29.102+0000 7fba51602700 1 --2- 172.21.15.157:0/517564137 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fba38066f60 0x7fba38069410 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fba40001050 tx=0x7fba40009040 comp rx=0 tx=0).stop 2023-10-06T04:05:29.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:29.102+0000 7fba51602700 1 -- 172.21.15.157:0/517564137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba4c156650 msgr2=0x7fba4c14e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:29.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:29.102+0000 7fba51602700 1 --2- 172.21.15.157:0/517564137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba4c156650 0x7fba4c14e4f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fba3c016000 tx=0x7fba3c01a000 comp rx=0 tx=0).stop 2023-10-06T04:05:29.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:29.102+0000 7fba51602700 1 -- 172.21.15.157:0/517564137 shutdown_connections 2023-10-06T04:05:29.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:29.102+0000 7fba51602700 1 --2- 172.21.15.157:0/517564137 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fba38066f60 0x7fba38069410 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:29.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:29.102+0000 7fba51602700 1 --2- 172.21.15.157:0/517564137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba4c156f00 0x7fba4c1552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:29.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:29.102+0000 7fba51602700 1 --2- 172.21.15.157:0/517564137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba4c156650 0x7fba4c14e4f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:29.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:29.102+0000 7fba51602700 1 --2- 172.21.15.157:0/517564137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba4c156a20 0x7fba4c167a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:29.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:29.102+0000 7fba51602700 1 -- 172.21.15.157:0/517564137 >> 172.21.15.157:0/517564137 conn(0x7fba4c06f890 msgr2=0x7fba4c06d450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:29.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:29.102+0000 7fba51602700 1 -- 172.21.15.157:0/517564137 shutdown_connections 2023-10-06T04:05:29.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:29.102+0000 7fba51602700 1 -- 172.21.15.157:0/517564137 wait complete. 2023-10-06T04:05:29.105 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 64 2023-10-06T04:05:29.118 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 64, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-06T03:57:10.718782+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 64, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_5362": { "addr": "172.21.15.175:6837/2388164729", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6836", "nonce": 2388164729, "type": "v2" }, { "addr": "172.21.15.175:6837", "nonce": 2388164729, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5362, "incarnation": 58, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:active", "state_seq": 22 }, "gid_5799": { "addr": "172.21.15.157:6834/765157057", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6832", "nonce": 765157057, "type": "v2" }, { "addr": "172.21.15.157:6834", "nonce": 765157057, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5799, "incarnation": 64, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 50, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2023-10-06T04:05:26.713720+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2, 3, 4 ], "tableserver": 0, "up": { "mds_0": 5799, "mds_1": 5362 } } } ], "standbys": [ { "addr": "172.21.15.175:6835/1799968248", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6834", "nonce": 1799968248, "type": "v2" }, { "addr": "172.21.15.175:6835", "nonce": 1799968248, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 57, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5126, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.175:6839/745804271", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6838", "nonce": 745804271, "type": "v2" }, { "addr": "172.21.15.175:6839", "nonce": 745804271, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 42, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5164, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.157:6835/2461751318", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6833", "nonce": 2461751318, "type": "v2" }, { "addr": "172.21.15.157:6835", "nonce": 2461751318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 34, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5268, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-06T04:05:31.119 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:05:31.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.242+0000 7fd88d8d4700 1 Processor -- start 2023-10-06T04:05:31.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.242+0000 7fd88d8d4700 1 -- start start 2023-10-06T04:05:31.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.243+0000 7fd88d8d4700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd888150410 0x7fd888155f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:31.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.243+0000 7fd88d8d4700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8880c9780 con 0x7fd88806dc40 2023-10-06T04:05:31.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.243+0000 7fd88d8d4700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd88806e0c0 con 0x7fd88806cbb0 2023-10-06T04:05:31.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.243+0000 7fd88d8d4700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8880c8c40 con 0x7fd888150410 2023-10-06T04:05:31.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.243+0000 7fd8867fc700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd88806dc40 0x7fd88814fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43926/0 (socket says 172.21.15.157:43926) 2023-10-06T04:05:31.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.243+0000 7fd8867fc700 1 -- 172.21.15.157:0/2445409047 learned_addr learned my addr 172.21.15.157:0/2445409047 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:05:31.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.243+0000 7fd8877fe700 1 --2- 172.21.15.157:0/2445409047 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd888150410 0x7fd888155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:31.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.243+0000 7fd885ffb700 1 -- 172.21.15.157:0/2445409047 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1847327575 0 0) 0x7fd8880c9780 con 0x7fd88806dc40 2023-10-06T04:05:31.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.243+0000 7fd885ffb700 1 -- 172.21.15.157:0/2445409047 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd870003570 con 0x7fd88806dc40 2023-10-06T04:05:31.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.243+0000 7fd885ffb700 1 -- 172.21.15.157:0/2445409047 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1343487974 0 0) 0x7fd870003570 con 0x7fd88806dc40 2023-10-06T04:05:31.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.243+0000 7fd885ffb700 1 -- 172.21.15.157:0/2445409047 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd888150410 msgr2=0x7fd888155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:31.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.243+0000 7fd885ffb700 1 --2- 172.21.15.157:0/2445409047 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd888150410 0x7fd888155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:31.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.243+0000 7fd885ffb700 1 -- 172.21.15.157:0/2445409047 >> v1:172.21.15.175:6789/0 conn(0x7fd88806cbb0 legacy=0x7fd88806d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:31.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.243+0000 7fd885ffb700 1 -- 172.21.15.157:0/2445409047 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8880c8e50 con 0x7fd88806dc40 2023-10-06T04:05:31.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.243+0000 7fd885ffb700 1 -- 172.21.15.157:0/2445409047 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7fd87c001470 con 0x7fd88806dc40 2023-10-06T04:05:31.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.243+0000 7fd885ffb700 1 -- 172.21.15.157:0/2445409047 >> v1:172.21.15.157:6789/0 conn(0x7fd88806dc40 legacy=0x7fd88814fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:31.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.243+0000 7fd885ffb700 1 --2- 172.21.15.157:0/2445409047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd870003a90 0x7fd870003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:31.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.243+0000 7fd885ffb700 1 --2- 172.21.15.157:0/2445409047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd870004810 0x7fd870004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:31.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.243+0000 7fd885ffb700 1 --2- 172.21.15.157:0/2445409047 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd888150410 0x7fd8700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:31.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.243+0000 7fd885ffb700 1 -- 172.21.15.157:0/2445409047 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8880c8c40 con 0x7fd870004810 2023-10-06T04:05:31.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.243+0000 7fd885ffb700 1 -- 172.21.15.157:0/2445409047 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87000bb10 con 0x7fd870003a90 2023-10-06T04:05:31.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.243+0000 7fd885ffb700 1 -- 172.21.15.157:0/2445409047 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd87000bcd0 con 0x7fd888150410 2023-10-06T04:05:31.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.244+0000 7fd8867fc700 1 --2- 172.21.15.157:0/2445409047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd870004810 0x7fd870004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:31.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.244+0000 7fd886ffd700 1 --2- 172.21.15.157:0/2445409047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd870003a90 0x7fd870003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:31.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.244+0000 7fd8877fe700 1 --2- 172.21.15.157:0/2445409047 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd888150410 0x7fd8700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:31.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.244+0000 7fd8867fc700 1 -- 172.21.15.157:0/2445409047 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd888150410 msgr2=0x7fd8700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:31.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.244+0000 7fd8867fc700 1 --2- 172.21.15.157:0/2445409047 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd888150410 0x7fd8700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:31.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.244+0000 7fd8867fc700 1 -- 172.21.15.157:0/2445409047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd870003a90 msgr2=0x7fd870003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:31.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.244+0000 7fd8867fc700 1 --2- 172.21.15.157:0/2445409047 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd870003a90 0x7fd870003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:31.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.244+0000 7fd8867fc700 1 -- 172.21.15.157:0/2445409047 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd870003570 con 0x7fd870004810 2023-10-06T04:05:31.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.244+0000 7fd8867fc700 1 --2- 172.21.15.157:0/2445409047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd870004810 0x7fd870004c80 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00c390 tx=0x7fd8880c96f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb79f5617a129eb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:31.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.244+0000 7fd885ffb700 1 -- 172.21.15.157:0/2445409047 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c00cc60 con 0x7fd870004810 2023-10-06T04:05:31.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.244+0000 7fd885ffb700 1 -- 172.21.15.157:0/2445409047 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87c00ce00 con 0x7fd870004810 2023-10-06T04:05:31.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.244+0000 7fd885ffb700 1 -- 172.21.15.157:0/2445409047 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c005b70 con 0x7fd870004810 2023-10-06T04:05:31.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.245+0000 7fd88d8d4700 1 -- 172.21.15.157:0/2445409047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd870004810 msgr2=0x7fd870004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:31.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.245+0000 7fd88d8d4700 1 --2- 172.21.15.157:0/2445409047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd870004810 0x7fd870004c80 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00c390 tx=0x7fd8880c96f0 comp rx=0 tx=0).stop 2023-10-06T04:05:31.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.245+0000 7fd8877fe700 1 -- 172.21.15.157:0/2445409047 reap_dead start 2023-10-06T04:05:31.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.245+0000 7fd88d8d4700 1 -- 172.21.15.157:0/2445409047 shutdown_connections 2023-10-06T04:05:31.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.245+0000 7fd88d8d4700 1 -- 172.21.15.157:0/2445409047 >> 172.21.15.157:0/2445409047 conn(0x7fd88806f230 msgr2=0x7fd88806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:31.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.245+0000 7fd88d8d4700 1 -- 172.21.15.157:0/2445409047 shutdown_connections 2023-10-06T04:05:31.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.245+0000 7fd88d8d4700 1 -- 172.21.15.157:0/2445409047 wait complete. 2023-10-06T04:05:31.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.246+0000 7fd88d8d4700 1 Processor -- start 2023-10-06T04:05:31.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.246+0000 7fd88d8d4700 1 -- start start 2023-10-06T04:05:31.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.247+0000 7fd88d8d4700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd870004810 0x7fd888166b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:31.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.247+0000 7fd88d8d4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd888153bb0 0x7fd8880cce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:31.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.247+0000 7fd88d8d4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8880c0ed0 0x7fd8881696f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:31.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.247+0000 7fd88d8d4700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8880c5840 con 0x7fd888153bb0 2023-10-06T04:05:31.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.247+0000 7fd88d8d4700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8881541e0 con 0x7fd8880c0ed0 2023-10-06T04:05:31.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.247+0000 7fd88d8d4700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8880c53b0 con 0x7fd870004810 2023-10-06T04:05:31.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.247+0000 7fd8867fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd888153bb0 0x7fd8880cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:31.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.247+0000 7fd8867fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd888153bb0 0x7fd8880cce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38402/0 (socket says 172.21.15.157:38402) 2023-10-06T04:05:31.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.247+0000 7fd8867fc700 1 -- 172.21.15.157:0/2842092008 learned_addr learned my addr 172.21.15.157:0/2842092008 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:31.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.247+0000 7fd886ffd700 1 --2- 172.21.15.157:0/2842092008 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd870004810 0x7fd888166b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:31.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.247+0000 7fd8877fe700 1 --2- 172.21.15.157:0/2842092008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8880c0ed0 0x7fd8881696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:31.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.247+0000 7fd8867fc700 1 -- 172.21.15.157:0/2842092008 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd870004810 msgr2=0x7fd888166b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:31.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.247+0000 7fd8867fc700 1 --2- 172.21.15.157:0/2842092008 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd870004810 0x7fd888166b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:31.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.247+0000 7fd8867fc700 1 -- 172.21.15.157:0/2842092008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8880c0ed0 msgr2=0x7fd8881696f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:31.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.247+0000 7fd8867fc700 1 --2- 172.21.15.157:0/2842092008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8880c0ed0 0x7fd8881696f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:31.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.247+0000 7fd8867fc700 1 -- 172.21.15.157:0/2842092008 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd888167050 con 0x7fd888153bb0 2023-10-06T04:05:31.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.247+0000 7fd8877fe700 1 --2- 172.21.15.157:0/2842092008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8880c0ed0 0x7fd8881696f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:05:31.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.247+0000 7fd8867fc700 1 --2- 172.21.15.157:0/2842092008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd888153bb0 0x7fd8880cce10 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00c980 tx=0x7fd87c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:31.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.247+0000 7fd86effd700 1 -- 172.21.15.157:0/2842092008 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c00cc60 con 0x7fd888153bb0 2023-10-06T04:05:31.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.247+0000 7fd86effd700 1 -- 172.21.15.157:0/2842092008 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87c0072e0 con 0x7fd888153bb0 2023-10-06T04:05:31.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.247+0000 7fd88d8d4700 1 -- 172.21.15.157:0/2842092008 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8880c6820 con 0x7fd888153bb0 2023-10-06T04:05:31.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.247+0000 7fd86effd700 1 -- 172.21.15.157:0/2842092008 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c007660 con 0x7fd888153bb0 2023-10-06T04:05:31.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.247+0000 7fd88d8d4700 1 -- 172.21.15.157:0/2842092008 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8880c4d60 con 0x7fd888153bb0 2023-10-06T04:05:31.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.248+0000 7fd88d8d4700 1 -- 172.21.15.157:0/2842092008 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd888089ca0 con 0x7fd888153bb0 2023-10-06T04:05:31.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.249+0000 7fd86effd700 1 -- 172.21.15.157:0/2842092008 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd87c019070 con 0x7fd888153bb0 2023-10-06T04:05:31.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.249+0000 7fd86effd700 1 --2- 172.21.15.157:0/2842092008 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd874066f30 0x7fd8740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:31.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.249+0000 7fd86effd700 1 -- 172.21.15.157:0/2842092008 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd87c0858b0 con 0x7fd888153bb0 2023-10-06T04:05:31.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.249+0000 7fd886ffd700 1 --2- 172.21.15.157:0/2842092008 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd874066f30 0x7fd8740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:31.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.250+0000 7fd886ffd700 1 --2- 172.21.15.157:0/2842092008 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd874066f30 0x7fd8740693e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd8780010e0 tx=0x7fd878007d10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:31.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.253+0000 7fd86effd700 1 -- 172.21.15.157:0/2842092008 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd87c052830 con 0x7fd888153bb0 2023-10-06T04:05:31.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.479+0000 7fd88d8d4700 1 -- 172.21.15.157:0/2842092008 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8880c54f0 con 0x7fd888153bb0 2023-10-06T04:05:31.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.479+0000 7fd86effd700 1 -- 172.21.15.157:0/2842092008 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+5385 (secure 0 0 0) 0x7fd87c055e50 con 0x7fd888153bb0 2023-10-06T04:05:31.482 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:05:31.482 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:05:26.713720+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:replay","state_seq":1,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:05:31.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.481+0000 7fd88d8d4700 1 -- 172.21.15.157:0/2842092008 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd874066f30 msgr2=0x7fd8740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:31.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.481+0000 7fd88d8d4700 1 --2- 172.21.15.157:0/2842092008 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd874066f30 0x7fd8740693e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd8780010e0 tx=0x7fd878007d10 comp rx=0 tx=0).stop 2023-10-06T04:05:31.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.481+0000 7fd88d8d4700 1 -- 172.21.15.157:0/2842092008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd888153bb0 msgr2=0x7fd8880cce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:31.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.481+0000 7fd88d8d4700 1 --2- 172.21.15.157:0/2842092008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd888153bb0 0x7fd8880cce10 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00c980 tx=0x7fd87c015000 comp rx=0 tx=0).stop 2023-10-06T04:05:31.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.481+0000 7fd88d8d4700 1 -- 172.21.15.157:0/2842092008 shutdown_connections 2023-10-06T04:05:31.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.481+0000 7fd88d8d4700 1 --2- 172.21.15.157:0/2842092008 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd874066f30 0x7fd8740693e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:31.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.481+0000 7fd88d8d4700 1 --2- 172.21.15.157:0/2842092008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8880c0ed0 0x7fd8881696f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:31.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.481+0000 7fd88d8d4700 1 --2- 172.21.15.157:0/2842092008 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd870004810 0x7fd888166b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:31.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.481+0000 7fd88d8d4700 1 --2- 172.21.15.157:0/2842092008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd888153bb0 0x7fd8880cce10 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:31.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.481+0000 7fd88d8d4700 1 -- 172.21.15.157:0/2842092008 >> 172.21.15.157:0/2842092008 conn(0x7fd88806f230 msgr2=0x7fd8880ca830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:31.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.481+0000 7fd88d8d4700 1 -- 172.21.15.157:0/2842092008 shutdown_connections 2023-10-06T04:05:31.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:31.481+0000 7fd88d8d4700 1 -- 172.21.15.157:0/2842092008 wait complete. 2023-10-06T04:05:31.485 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 64 2023-10-06T04:05:31.496 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:05:33.497 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:05:33.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.610+0000 7f19f804a700 1 Processor -- start 2023-10-06T04:05:33.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.610+0000 7f19f804a700 1 -- start start 2023-10-06T04:05:33.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.611+0000 7f19f804a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19f006cbb0 0x7f19f006d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:33.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.611+0000 7f19f804a700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f19f006c1f0 0x7f19f006da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:33.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.611+0000 7f19f804a700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f19f006cf80 0x7f19f014fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:33.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.611+0000 7f19f804a700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19f00c8c40 con 0x7f19f006cf80 2023-10-06T04:05:33.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.611+0000 7f19f804a700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19f00c9780 con 0x7f19f006c1f0 2023-10-06T04:05:33.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.611+0000 7f19f804a700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f006e0c0 con 0x7f19f006cbb0 2023-10-06T04:05:33.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.611+0000 7f19f65e7700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f19f006cf80 0x7f19f014fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:33.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.611+0000 7f19f65e7700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f19f006cf80 0x7f19f014fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41454/0 (socket says 172.21.15.157:41454) 2023-10-06T04:05:33.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.611+0000 7f19f65e7700 1 -- 172.21.15.157:0/1968504575 learned_addr learned my addr 172.21.15.157:0/1968504575 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:33.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.611+0000 7f19f55e5700 1 --2- 172.21.15.157:0/1968504575 >> v2:172.21.15.175:3300/0 conn(0x7f19f006c1f0 0x7f19f006da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:33.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.611+0000 7f19f5de6700 1 --2- 172.21.15.157:0/1968504575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19f006cbb0 0x7f19f006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:33.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.611+0000 7f19f65e7700 1 -- 172.21.15.157:0/1968504575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19f006cbb0 msgr2=0x7f19f006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:33.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.611+0000 7f19f65e7700 1 --2- 172.21.15.157:0/1968504575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19f006cbb0 0x7f19f006d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:33.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.611+0000 7f19f65e7700 1 -- 172.21.15.157:0/1968504575 >> v2:172.21.15.175:3300/0 conn(0x7f19f006c1f0 msgr2=0x7f19f006da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:33.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.611+0000 7f19f65e7700 1 --2- 172.21.15.157:0/1968504575 >> v2:172.21.15.175:3300/0 conn(0x7f19f006c1f0 0x7f19f006da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:33.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.611+0000 7f19f65e7700 1 -- 172.21.15.157:0/1968504575 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f00c8e50 con 0x7f19f006cf80 2023-10-06T04:05:33.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.611+0000 7f19f5de6700 1 --2- 172.21.15.157:0/1968504575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19f006cbb0 0x7f19f006d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:05:33.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.611+0000 7f19f65e7700 1 --2- 172.21.15.157:0/1968504575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f006cf80 0x7f19f014fd00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f19e00099b0 tx=0x7f19e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1605b2bb481bf4f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:33.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.611+0000 7f19f4de4700 1 -- 172.21.15.157:0/1968504575 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0009ea0 con 0x7f19f006cf80 2023-10-06T04:05:33.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.611+0000 7f19f4de4700 1 -- 172.21.15.157:0/1968504575 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e000b8b0 con 0x7f19f006cf80 2023-10-06T04:05:33.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.611+0000 7f19f4de4700 1 -- 172.21.15.157:0/1968504575 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e000bc50 con 0x7f19f006cf80 2023-10-06T04:05:33.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.612+0000 7f19f804a700 1 -- 172.21.15.157:0/1968504575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f006cf80 msgr2=0x7f19f014fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:33.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.612+0000 7f19f804a700 1 --2- 172.21.15.157:0/1968504575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f006cf80 0x7f19f014fd00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f19e00099b0 tx=0x7f19e0009250 comp rx=0 tx=0).stop 2023-10-06T04:05:33.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.612+0000 7f19f804a700 1 -- 172.21.15.157:0/1968504575 shutdown_connections 2023-10-06T04:05:33.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.612+0000 7f19f804a700 1 --2- 172.21.15.157:0/1968504575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f006cf80 0x7f19f014fd00 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:33.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.612+0000 7f19f804a700 1 --2- 172.21.15.157:0/1968504575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19f006cbb0 0x7f19f006d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:33.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.612+0000 7f19f804a700 1 --2- 172.21.15.157:0/1968504575 >> v2:172.21.15.175:3300/0 conn(0x7f19f006c1f0 0x7f19f006da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:33.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.612+0000 7f19f804a700 1 -- 172.21.15.157:0/1968504575 >> 172.21.15.157:0/1968504575 conn(0x7f19f006f230 msgr2=0x7f19f006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:33.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.612+0000 7f19f804a700 1 -- 172.21.15.157:0/1968504575 shutdown_connections 2023-10-06T04:05:33.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.612+0000 7f19f804a700 1 -- 172.21.15.157:0/1968504575 wait complete. 2023-10-06T04:05:33.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.613+0000 7f19f804a700 1 Processor -- start 2023-10-06T04:05:33.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.613+0000 7f19f804a700 1 -- start start 2023-10-06T04:05:33.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.614+0000 7f19f804a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19f006c1f0 0x7f19f0166a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:33.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.614+0000 7f19f804a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f006cbb0 0x7f19f006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:33.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.614+0000 7f19f804a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19f006cf80 0x7f19f00c1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:33.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.614+0000 7f19f804a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f0166f90 con 0x7f19f006cbb0 2023-10-06T04:05:33.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.614+0000 7f19f804a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f01696d0 con 0x7f19f006c1f0 2023-10-06T04:05:33.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.614+0000 7f19f804a700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f0169970 con 0x7f19f006cf80 2023-10-06T04:05:33.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.614+0000 7f19f55e5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f006cbb0 0x7f19f006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:33.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.614+0000 7f19f55e5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f006cbb0 0x7f19f006d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41468/0 (socket says 172.21.15.157:41468) 2023-10-06T04:05:33.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.614+0000 7f19f55e5700 1 -- 172.21.15.157:0/1993495975 learned_addr learned my addr 172.21.15.157:0/1993495975 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:33.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.614+0000 7f19f65e7700 1 --2- 172.21.15.157:0/1993495975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19f006cf80 0x7f19f00c1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:33.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.614+0000 7f19f5de6700 1 --2- 172.21.15.157:0/1993495975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19f006c1f0 0x7f19f0166a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:33.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.614+0000 7f19f55e5700 1 -- 172.21.15.157:0/1993495975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19f006cf80 msgr2=0x7f19f00c1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:33.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.614+0000 7f19f55e5700 1 --2- 172.21.15.157:0/1993495975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19f006cf80 0x7f19f00c1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:33.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.614+0000 7f19f55e5700 1 -- 172.21.15.157:0/1993495975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19f006c1f0 msgr2=0x7f19f0166a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:33.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.614+0000 7f19f55e5700 1 --2- 172.21.15.157:0/1993495975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19f006c1f0 0x7f19f0166a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:33.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.614+0000 7f19f55e5700 1 -- 172.21.15.157:0/1993495975 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e00094f0 con 0x7f19f006cbb0 2023-10-06T04:05:33.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.614+0000 7f19f55e5700 1 --2- 172.21.15.157:0/1993495975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f006cbb0 0x7f19f006d780 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f19e4009ef0 tx=0x7f19e400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:33.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.614+0000 7f19edffb700 1 -- 172.21.15.157:0/1993495975 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e400cb70 con 0x7f19f006cbb0 2023-10-06T04:05:33.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.614+0000 7f19edffb700 1 -- 172.21.15.157:0/1993495975 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e400f040 con 0x7f19f006cbb0 2023-10-06T04:05:33.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.614+0000 7f19edffb700 1 -- 172.21.15.157:0/1993495975 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4014480 con 0x7f19f006cbb0 2023-10-06T04:05:33.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.615+0000 7f19f804a700 1 -- 172.21.15.157:0/1993495975 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f0169c50 con 0x7f19f006cbb0 2023-10-06T04:05:33.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.615+0000 7f19f804a700 1 -- 172.21.15.157:0/1993495975 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f016a210 con 0x7f19f006cbb0 2023-10-06T04:05:33.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.616+0000 7f19f804a700 1 -- 172.21.15.157:0/1993495975 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f0089ca0 con 0x7f19f006cbb0 2023-10-06T04:05:33.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.616+0000 7f19edffb700 1 -- 172.21.15.157:0/1993495975 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f19e40147f0 con 0x7f19f006cbb0 2023-10-06T04:05:33.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.616+0000 7f19edffb700 1 --2- 172.21.15.157:0/1993495975 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f19dc066f30 0x7f19dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:33.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.616+0000 7f19edffb700 1 -- 172.21.15.157:0/1993495975 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f19e4085500 con 0x7f19f006cbb0 2023-10-06T04:05:33.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.617+0000 7f19f5de6700 1 --2- 172.21.15.157:0/1993495975 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f19dc066f30 0x7f19dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:33.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.617+0000 7f19f5de6700 1 --2- 172.21.15.157:0/1993495975 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f19dc066f30 0x7f19dc0693e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f19e8006f00 tx=0x7f19e8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:33.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.621+0000 7f19edffb700 1 -- 172.21.15.157:0/1993495975 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f19e40523e0 con 0x7f19f006cbb0 2023-10-06T04:05:33.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.846+0000 7f19f804a700 1 -- 172.21.15.157:0/1993495975 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19f00c9130 con 0x7f19f006cbb0 2023-10-06T04:05:33.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.847+0000 7f19edffb700 1 -- 172.21.15.157:0/1993495975 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+5387 (secure 0 0 0) 0x7f19e4055a00 con 0x7f19f006cbb0 2023-10-06T04:05:33.849 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:05:33.849 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:05:33.755867+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:resolve","state_seq":11,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:05:33.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.848+0000 7f19f804a700 1 -- 172.21.15.157:0/1993495975 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f19dc066f30 msgr2=0x7f19dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:33.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.848+0000 7f19f804a700 1 --2- 172.21.15.157:0/1993495975 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f19dc066f30 0x7f19dc0693e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f19e8006f00 tx=0x7f19e8008040 comp rx=0 tx=0).stop 2023-10-06T04:05:33.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.848+0000 7f19f804a700 1 -- 172.21.15.157:0/1993495975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f006cbb0 msgr2=0x7f19f006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:33.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.848+0000 7f19f804a700 1 --2- 172.21.15.157:0/1993495975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f006cbb0 0x7f19f006d780 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f19e4009ef0 tx=0x7f19e400b040 comp rx=0 tx=0).stop 2023-10-06T04:05:33.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.849+0000 7f19f804a700 1 -- 172.21.15.157:0/1993495975 shutdown_connections 2023-10-06T04:05:33.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.849+0000 7f19f804a700 1 --2- 172.21.15.157:0/1993495975 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f19dc066f30 0x7f19dc0693e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:33.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.849+0000 7f19f804a700 1 --2- 172.21.15.157:0/1993495975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19f006c1f0 0x7f19f0166a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:33.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.849+0000 7f19f804a700 1 --2- 172.21.15.157:0/1993495975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19f006cf80 0x7f19f00c1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:33.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.849+0000 7f19f804a700 1 --2- 172.21.15.157:0/1993495975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f006cbb0 0x7f19f006d780 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:33.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.849+0000 7f19f804a700 1 -- 172.21.15.157:0/1993495975 >> 172.21.15.157:0/1993495975 conn(0x7f19f006f230 msgr2=0x7f19f014b740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:33.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.849+0000 7f19f804a700 1 -- 172.21.15.157:0/1993495975 shutdown_connections 2023-10-06T04:05:33.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:33.849+0000 7f19f804a700 1 -- 172.21.15.157:0/1993495975 wait complete. 2023-10-06T04:05:33.853 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 65 2023-10-06T04:05:33.863 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:05:35.864 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:05:35.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.989+0000 7f5e45ec2700 1 Processor -- start 2023-10-06T04:05:35.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.989+0000 7f5e45ec2700 1 -- start start 2023-10-06T04:05:35.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.990+0000 7f5e45ec2700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5e400b04d0 0x7f5e40157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:35.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.990+0000 7f5e45ec2700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e4006c2d0 0x7f5e4006dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:35.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.990+0000 7f5e45ec2700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e4006d8d0 con 0x7f5e400b1e40 2023-10-06T04:05:35.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.990+0000 7f5e45ec2700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e400af8f0 con 0x7f5e400b04d0 2023-10-06T04:05:35.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.990+0000 7f5e45ec2700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4015a690 con 0x7f5e4006c2d0 2023-10-06T04:05:35.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.990+0000 7f5e44ec0700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f5e400b1e40 0x7f5e401563b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:48934/0 (socket says 172.21.15.157:48934) 2023-10-06T04:05:35.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.990+0000 7f5e44ec0700 1 -- 172.21.15.157:0/3595882114 learned_addr learned my addr 172.21.15.157:0/3595882114 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:05:35.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.990+0000 7f5e3ffff700 1 --2- 172.21.15.157:0/3595882114 >> v2:172.21.15.175:3300/0 conn(0x7f5e400b04d0 0x7f5e40157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:35.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.990+0000 7f5e456c1700 1 --2- 172.21.15.157:0/3595882114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e4006c2d0 0x7f5e4006dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:35.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.990+0000 7f5e3f7fe700 1 -- 172.21.15.157:0/3595882114 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3908265069 0 0) 0x7f5e4006d8d0 con 0x7f5e400b1e40 2023-10-06T04:05:35.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.990+0000 7f5e3f7fe700 1 -- 172.21.15.157:0/3595882114 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e30003570 con 0x7f5e400b1e40 2023-10-06T04:05:35.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.991+0000 7f5e456c1700 1 -- 172.21.15.157:0/3595882114 >> v2:172.21.15.175:3300/0 conn(0x7f5e400b04d0 msgr2=0x7f5e40157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:35.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.991+0000 7f5e456c1700 1 --2- 172.21.15.157:0/3595882114 >> v2:172.21.15.175:3300/0 conn(0x7f5e400b04d0 0x7f5e40157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:35.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.991+0000 7f5e456c1700 1 -- 172.21.15.157:0/3595882114 >> v1:172.21.15.157:6789/0 conn(0x7f5e400b1e40 legacy=0x7f5e401563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:35.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.991+0000 7f5e3f7fe700 1 -- 172.21.15.157:0/3595882114 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2066574217 0 0) 0x7f5e30003570 con 0x7f5e400b1e40 2023-10-06T04:05:35.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.991+0000 7f5e456c1700 1 -- 172.21.15.157:0/3595882114 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4006ccc0 con 0x7f5e4006c2d0 2023-10-06T04:05:35.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.991+0000 7f5e3ffff700 1 --2- 172.21.15.157:0/3595882114 >> v2:172.21.15.175:3300/0 conn(0x7f5e400b04d0 0x7f5e40157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:05:35.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.991+0000 7f5e456c1700 1 --2- 172.21.15.157:0/3595882114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e4006c2d0 0x7f5e4006dd60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009e60 tx=0x7f5e38009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30b5f47330b03781 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:35.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.991+0000 7f5e3f7fe700 1 -- 172.21.15.157:0/3595882114 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3800e070 con 0x7f5e4006c2d0 2023-10-06T04:05:35.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.991+0000 7f5e3f7fe700 1 -- 172.21.15.157:0/3595882114 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3800b7a0 con 0x7f5e4006c2d0 2023-10-06T04:05:35.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.991+0000 7f5e3f7fe700 1 -- 172.21.15.157:0/3595882114 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e34001470 con 0x7f5e4006c2d0 2023-10-06T04:05:35.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.991+0000 7f5e45ec2700 1 -- 172.21.15.157:0/3595882114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e4006c2d0 msgr2=0x7f5e4006dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:35.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.991+0000 7f5e45ec2700 1 --2- 172.21.15.157:0/3595882114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e4006c2d0 0x7f5e4006dd60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009e60 tx=0x7f5e38009780 comp rx=0 tx=0).stop 2023-10-06T04:05:35.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.991+0000 7f5e45ec2700 1 -- 172.21.15.157:0/3595882114 shutdown_connections 2023-10-06T04:05:35.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.991+0000 7f5e45ec2700 1 --2- 172.21.15.157:0/3595882114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e4006c2d0 0x7f5e4006dd60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:35.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.991+0000 7f5e45ec2700 1 --2- 172.21.15.157:0/3595882114 >> v2:172.21.15.175:3300/0 conn(0x7f5e400b04d0 0x7f5e40157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:35.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.991+0000 7f5e45ec2700 1 -- 172.21.15.157:0/3595882114 >> 172.21.15.157:0/3595882114 conn(0x7f5e400cc540 msgr2=0x7f5e400cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:35.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.991+0000 7f5e45ec2700 1 -- 172.21.15.157:0/3595882114 shutdown_connections 2023-10-06T04:05:35.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.991+0000 7f5e45ec2700 1 -- 172.21.15.157:0/3595882114 wait complete. 2023-10-06T04:05:35.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.993+0000 7f5e45ec2700 1 Processor -- start 2023-10-06T04:05:35.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.993+0000 7f5e45ec2700 1 -- start start 2023-10-06T04:05:36.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.993+0000 7f5e45ec2700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e4006c2d0 0x7f5e40152c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:36.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.993+0000 7f5e45ec2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e400b04d0 0x7f5e40169210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:36.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.993+0000 7f5e45ec2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e400b1e40 0x7f5e40169750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:36.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.993+0000 7f5e45ec2700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4016a080 con 0x7f5e400b1e40 2023-10-06T04:05:36.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.993+0000 7f5e45ec2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e40159ef0 con 0x7f5e400b04d0 2023-10-06T04:05:36.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.993+0000 7f5e45ec2700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e40152280 con 0x7f5e4006c2d0 2023-10-06T04:05:36.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.994+0000 7f5e456c1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e400b1e40 0x7f5e40169750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:36.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.994+0000 7f5e456c1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e400b1e40 0x7f5e40169750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41482/0 (socket says 172.21.15.157:41482) 2023-10-06T04:05:36.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.994+0000 7f5e456c1700 1 -- 172.21.15.157:0/3030819164 learned_addr learned my addr 172.21.15.157:0/3030819164 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:36.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.994+0000 7f5e44ec0700 1 --2- 172.21.15.157:0/3030819164 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e4006c2d0 0x7f5e40152c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:36.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.994+0000 7f5e3ffff700 1 --2- 172.21.15.157:0/3030819164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e400b04d0 0x7f5e40169210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:36.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.994+0000 7f5e456c1700 1 -- 172.21.15.157:0/3030819164 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e4006c2d0 msgr2=0x7f5e40152c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:36.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.994+0000 7f5e456c1700 1 --2- 172.21.15.157:0/3030819164 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e4006c2d0 0x7f5e40152c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:36.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.994+0000 7f5e456c1700 1 -- 172.21.15.157:0/3030819164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e400b04d0 msgr2=0x7f5e40169210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:36.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.994+0000 7f5e456c1700 1 --2- 172.21.15.157:0/3030819164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e400b04d0 0x7f5e40169210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:36.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.994+0000 7f5e456c1700 1 -- 172.21.15.157:0/3030819164 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c009420 con 0x7f5e400b1e40 2023-10-06T04:05:36.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.994+0000 7f5e456c1700 1 --2- 172.21.15.157:0/3030819164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e400b1e40 0x7f5e40169750 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5e4015a7d0 tx=0x7f5e38018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:36.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.994+0000 7f5e3cff9700 1 -- 172.21.15.157:0/3030819164 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e38012ad0 con 0x7f5e400b1e40 2023-10-06T04:05:36.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.994+0000 7f5e3cff9700 1 -- 172.21.15.157:0/3030819164 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e38012c30 con 0x7f5e400b1e40 2023-10-06T04:05:36.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.994+0000 7f5e45ec2700 1 -- 172.21.15.157:0/3030819164 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e38009a20 con 0x7f5e400b1e40 2023-10-06T04:05:36.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.994+0000 7f5e3cff9700 1 -- 172.21.15.157:0/3030819164 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e38025460 con 0x7f5e400b1e40 2023-10-06T04:05:36.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.994+0000 7f5e45ec2700 1 -- 172.21.15.157:0/3030819164 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e4015a130 con 0x7f5e400b1e40 2023-10-06T04:05:36.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.995+0000 7f5e45ec2700 1 -- 172.21.15.157:0/3030819164 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e40088f90 con 0x7f5e400b1e40 2023-10-06T04:05:36.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.996+0000 7f5e3cff9700 1 -- 172.21.15.157:0/3030819164 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5e38025870 con 0x7f5e400b1e40 2023-10-06T04:05:36.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.996+0000 7f5e3cff9700 1 --2- 172.21.15.157:0/3030819164 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5e24066f30 0x7f5e240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:36.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.996+0000 7f5e3cff9700 1 -- 172.21.15.157:0/3030819164 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5e38086610 con 0x7f5e400b1e40 2023-10-06T04:05:36.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.996+0000 7f5e44ec0700 1 --2- 172.21.15.157:0/3030819164 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5e24066f30 0x7f5e240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:36.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:35.997+0000 7f5e44ec0700 1 --2- 172.21.15.157:0/3030819164 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5e24066f30 0x7f5e240693e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5e34004e60 tx=0x7f5e34001a10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:36.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:36.000+0000 7f5e3cff9700 1 -- 172.21.15.157:0/3030819164 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5e38053590 con 0x7f5e400b1e40 2023-10-06T04:05:36.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:36.231+0000 7f5e45ec2700 1 -- 172.21.15.157:0/3030819164 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e4006cfa0 con 0x7f5e400b1e40 2023-10-06T04:05:36.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:36.232+0000 7f5e3cff9700 1 -- 172.21.15.157:0/3030819164 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+5387 (secure 0 0 0) 0x7f5e38056bb0 con 0x7f5e400b1e40 2023-10-06T04:05:36.234 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:05:36.234 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:05:33.755867+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:resolve","state_seq":11,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:05:36.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:36.234+0000 7f5e45ec2700 1 -- 172.21.15.157:0/3030819164 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5e24066f30 msgr2=0x7f5e240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:36.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:36.234+0000 7f5e45ec2700 1 --2- 172.21.15.157:0/3030819164 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5e24066f30 0x7f5e240693e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5e34004e60 tx=0x7f5e34001a10 comp rx=0 tx=0).stop 2023-10-06T04:05:36.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:36.234+0000 7f5e45ec2700 1 -- 172.21.15.157:0/3030819164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e400b1e40 msgr2=0x7f5e40169750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:36.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:36.234+0000 7f5e45ec2700 1 --2- 172.21.15.157:0/3030819164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e400b1e40 0x7f5e40169750 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5e4015a7d0 tx=0x7f5e38018000 comp rx=0 tx=0).stop 2023-10-06T04:05:36.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:36.234+0000 7f5e45ec2700 1 -- 172.21.15.157:0/3030819164 shutdown_connections 2023-10-06T04:05:36.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:36.234+0000 7f5e45ec2700 1 --2- 172.21.15.157:0/3030819164 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5e24066f30 0x7f5e240693e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:36.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:36.234+0000 7f5e45ec2700 1 --2- 172.21.15.157:0/3030819164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e400b1e40 0x7f5e40169750 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:36.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:36.234+0000 7f5e45ec2700 1 --2- 172.21.15.157:0/3030819164 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e4006c2d0 0x7f5e40152c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:36.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:36.234+0000 7f5e45ec2700 1 --2- 172.21.15.157:0/3030819164 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e400b04d0 0x7f5e40169210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:36.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:36.234+0000 7f5e45ec2700 1 -- 172.21.15.157:0/3030819164 >> 172.21.15.157:0/3030819164 conn(0x7f5e400cc540 msgr2=0x7f5e400c52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:36.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:36.234+0000 7f5e45ec2700 1 -- 172.21.15.157:0/3030819164 shutdown_connections 2023-10-06T04:05:36.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:36.234+0000 7f5e45ec2700 1 -- 172.21.15.157:0/3030819164 wait complete. 2023-10-06T04:05:36.238 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 65 2023-10-06T04:05:36.248 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:05:38.249 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:05:38.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.374+0000 7fb3def46700 1 Processor -- start 2023-10-06T04:05:38.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.374+0000 7fb3def46700 1 -- start start 2023-10-06T04:05:38.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.375+0000 7fb3def46700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb3d80b1e40 0x7fb3d8157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:38.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.375+0000 7fb3def46700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3d80b04d0 0x7fb3d806dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:38.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.375+0000 7fb3def46700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3d806d8d0 con 0x7fb3d806c2d0 2023-10-06T04:05:38.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.375+0000 7fb3def46700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3d80af8f0 con 0x7fb3d80b1e40 2023-10-06T04:05:38.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.375+0000 7fb3def46700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d815a690 con 0x7fb3d80b04d0 2023-10-06T04:05:38.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.375+0000 7fb3ddf44700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb3d80b1e40 0x7fb3d8157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:38.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.375+0000 7fb3ddf44700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb3d80b1e40 0x7fb3d8157750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41498/0 (socket says 172.21.15.157:41498) 2023-10-06T04:05:38.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.375+0000 7fb3ddf44700 1 -- 172.21.15.157:0/1531562053 learned_addr learned my addr 172.21.15.157:0/1531562053 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:38.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.375+0000 7fb3dd743700 1 --2- 172.21.15.157:0/1531562053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3d80b04d0 0x7fb3d806dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:38.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.376+0000 7fb3ddf44700 1 -- 172.21.15.157:0/1531562053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3d80b04d0 msgr2=0x7fb3d806dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:38.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.376+0000 7fb3ddf44700 1 --2- 172.21.15.157:0/1531562053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3d80b04d0 0x7fb3d806dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:38.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.376+0000 7fb3ddf44700 1 -- 172.21.15.157:0/1531562053 >> v1:172.21.15.175:6789/0 conn(0x7fb3d806c2d0 legacy=0x7fb3d81563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:38.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.376+0000 7fb3ddf44700 1 -- 172.21.15.157:0/1531562053 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d806ccc0 con 0x7fb3d80b1e40 2023-10-06T04:05:38.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.376+0000 7fb3dd743700 1 --2- 172.21.15.157:0/1531562053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3d80b04d0 0x7fb3d806dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:05:38.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.376+0000 7fb3ddf44700 1 --2- 172.21.15.157:0/1531562053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3d80b1e40 0x7fb3d8157750 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40099b0 tx=0x7fb3d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e574eb69299de5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:38.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.376+0000 7fb3dcf42700 1 -- 172.21.15.157:0/1531562053 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d400b860 con 0x7fb3d80b1e40 2023-10-06T04:05:38.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.376+0000 7fb3dcf42700 1 -- 172.21.15.157:0/1531562053 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d400b9c0 con 0x7fb3d80b1e40 2023-10-06T04:05:38.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.376+0000 7fb3dcf42700 1 -- 172.21.15.157:0/1531562053 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d400bd80 con 0x7fb3d80b1e40 2023-10-06T04:05:38.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.376+0000 7fb3def46700 1 -- 172.21.15.157:0/1531562053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3d80b1e40 msgr2=0x7fb3d8157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:38.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.376+0000 7fb3def46700 1 --2- 172.21.15.157:0/1531562053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3d80b1e40 0x7fb3d8157750 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40099b0 tx=0x7fb3d4009250 comp rx=0 tx=0).stop 2023-10-06T04:05:38.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.377+0000 7fb3def46700 1 -- 172.21.15.157:0/1531562053 shutdown_connections 2023-10-06T04:05:38.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.377+0000 7fb3def46700 1 --2- 172.21.15.157:0/1531562053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3d80b1e40 0x7fb3d8157750 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:38.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.377+0000 7fb3def46700 1 --2- 172.21.15.157:0/1531562053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3d80b04d0 0x7fb3d806dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:38.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.377+0000 7fb3def46700 1 -- 172.21.15.157:0/1531562053 >> 172.21.15.157:0/1531562053 conn(0x7fb3d80cc540 msgr2=0x7fb3d80cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:38.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.377+0000 7fb3def46700 1 -- 172.21.15.157:0/1531562053 shutdown_connections 2023-10-06T04:05:38.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.377+0000 7fb3def46700 1 -- 172.21.15.157:0/1531562053 wait complete. 2023-10-06T04:05:38.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.378+0000 7fb3def46700 1 Processor -- start 2023-10-06T04:05:38.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.378+0000 7fb3def46700 1 -- start start 2023-10-06T04:05:38.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.378+0000 7fb3def46700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3d806c2d0 0x7fb3d8150c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:38.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.378+0000 7fb3def46700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3d80b04d0 0x7fb3d815b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:38.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.378+0000 7fb3def46700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3d80b1e40 0x7fb3d8151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:38.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.378+0000 7fb3def46700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d81507c0 con 0x7fb3d806c2d0 2023-10-06T04:05:38.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.378+0000 7fb3def46700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d8153500 con 0x7fb3d80b04d0 2023-10-06T04:05:38.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.378+0000 7fb3def46700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d81501b0 con 0x7fb3d80b1e40 2023-10-06T04:05:38.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.379+0000 7fb3ddf44700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3d806c2d0 0x7fb3d8150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:38.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.379+0000 7fb3ddf44700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3d806c2d0 0x7fb3d8150c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41506/0 (socket says 172.21.15.157:41506) 2023-10-06T04:05:38.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.379+0000 7fb3ddf44700 1 -- 172.21.15.157:0/652445448 learned_addr learned my addr 172.21.15.157:0/652445448 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:38.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.379+0000 7fb3dd743700 1 --2- 172.21.15.157:0/652445448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3d80b04d0 0x7fb3d815b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:38.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.379+0000 7fb3de745700 1 --2- 172.21.15.157:0/652445448 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3d80b1e40 0x7fb3d8151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:38.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.379+0000 7fb3ddf44700 1 -- 172.21.15.157:0/652445448 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3d80b1e40 msgr2=0x7fb3d8151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:38.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.379+0000 7fb3ddf44700 1 --2- 172.21.15.157:0/652445448 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3d80b1e40 0x7fb3d8151f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:38.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.379+0000 7fb3ddf44700 1 -- 172.21.15.157:0/652445448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3d80b04d0 msgr2=0x7fb3d815b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:38.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.379+0000 7fb3ddf44700 1 --2- 172.21.15.157:0/652445448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3d80b04d0 0x7fb3d815b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:38.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.379+0000 7fb3ddf44700 1 -- 172.21.15.157:0/652445448 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d40094f0 con 0x7fb3d806c2d0 2023-10-06T04:05:38.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.379+0000 7fb3ddf44700 1 --2- 172.21.15.157:0/652445448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3d806c2d0 0x7fb3d8150c50 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb3d80899e0 tx=0x7fb3d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:38.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.379+0000 7fb3cdffb700 1 -- 172.21.15.157:0/652445448 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4011e40 con 0x7fb3d806c2d0 2023-10-06T04:05:38.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.379+0000 7fb3cdffb700 1 -- 172.21.15.157:0/652445448 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d4026450 con 0x7fb3d806c2d0 2023-10-06T04:05:38.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.379+0000 7fb3def46700 1 -- 172.21.15.157:0/652445448 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d8153760 con 0x7fb3d806c2d0 2023-10-06T04:05:38.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.379+0000 7fb3cdffb700 1 -- 172.21.15.157:0/652445448 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4026810 con 0x7fb3d806c2d0 2023-10-06T04:05:38.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.380+0000 7fb3def46700 1 -- 172.21.15.157:0/652445448 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d8153a40 con 0x7fb3d806c2d0 2023-10-06T04:05:38.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.380+0000 7fb3def46700 1 -- 172.21.15.157:0/652445448 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3d8150aa0 con 0x7fb3d806c2d0 2023-10-06T04:05:38.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.381+0000 7fb3cdffb700 1 -- 172.21.15.157:0/652445448 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb3d401a420 con 0x7fb3d806c2d0 2023-10-06T04:05:38.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.381+0000 7fb3cdffb700 1 --2- 172.21.15.157:0/652445448 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb3bc066f30 0x7fb3bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:38.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.381+0000 7fb3cdffb700 1 -- 172.21.15.157:0/652445448 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb3d40862e0 con 0x7fb3d806c2d0 2023-10-06T04:05:38.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.381+0000 7fb3dd743700 1 --2- 172.21.15.157:0/652445448 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb3bc066f30 0x7fb3bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:38.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.382+0000 7fb3dd743700 1 --2- 172.21.15.157:0/652445448 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb3bc066f30 0x7fb3bc0693e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb3d815c6d0 tx=0x7fb3c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:38.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.386+0000 7fb3cdffb700 1 -- 172.21.15.157:0/652445448 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3d40532e0 con 0x7fb3d806c2d0 2023-10-06T04:05:38.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.609+0000 7fb3def46700 1 -- 172.21.15.157:0/652445448 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3d8088f90 con 0x7fb3d806c2d0 2023-10-06T04:05:38.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.609+0000 7fb3cdffb700 1 -- 172.21.15.157:0/652445448 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+5387 (secure 0 0 0) 0x7fb3d4056900 con 0x7fb3d806c2d0 2023-10-06T04:05:38.612 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:05:38.612 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:05:33.755867+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:resolve","state_seq":11,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:05:38.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.611+0000 7fb3def46700 1 -- 172.21.15.157:0/652445448 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb3bc066f30 msgr2=0x7fb3bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:38.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.611+0000 7fb3def46700 1 --2- 172.21.15.157:0/652445448 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb3bc066f30 0x7fb3bc0693e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb3d815c6d0 tx=0x7fb3c8008040 comp rx=0 tx=0).stop 2023-10-06T04:05:38.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.611+0000 7fb3def46700 1 -- 172.21.15.157:0/652445448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3d806c2d0 msgr2=0x7fb3d8150c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:38.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.611+0000 7fb3def46700 1 --2- 172.21.15.157:0/652445448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3d806c2d0 0x7fb3d8150c50 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb3d80899e0 tx=0x7fb3d4017000 comp rx=0 tx=0).stop 2023-10-06T04:05:38.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.611+0000 7fb3def46700 1 -- 172.21.15.157:0/652445448 shutdown_connections 2023-10-06T04:05:38.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.611+0000 7fb3def46700 1 --2- 172.21.15.157:0/652445448 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb3bc066f30 0x7fb3bc0693e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:38.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.611+0000 7fb3def46700 1 --2- 172.21.15.157:0/652445448 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3d80b1e40 0x7fb3d8151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:38.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.611+0000 7fb3def46700 1 --2- 172.21.15.157:0/652445448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3d806c2d0 0x7fb3d8150c50 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:38.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.611+0000 7fb3def46700 1 --2- 172.21.15.157:0/652445448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3d80b04d0 0x7fb3d815b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:38.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.611+0000 7fb3def46700 1 -- 172.21.15.157:0/652445448 >> 172.21.15.157:0/652445448 conn(0x7fb3d80cc540 msgr2=0x7fb3d8159e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:38.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.612+0000 7fb3def46700 1 -- 172.21.15.157:0/652445448 shutdown_connections 2023-10-06T04:05:38.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:38.612+0000 7fb3def46700 1 -- 172.21.15.157:0/652445448 wait complete. 2023-10-06T04:05:38.616 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 65 2023-10-06T04:05:38.626 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:05:40.626 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:05:40.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.752+0000 7f4dce8cf700 1 Processor -- start 2023-10-06T04:05:40.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.752+0000 7f4dce8cf700 1 -- start start 2023-10-06T04:05:40.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.753+0000 7f4dce8cf700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4dc806cbb0 0x7f4dc806d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:40.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.753+0000 7f4dce8cf700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dc806c1f0 0x7f4dc806da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:40.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.753+0000 7f4dce8cf700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dc80c9780 con 0x7f4dc806cf80 2023-10-06T04:05:40.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.753+0000 7f4dce8cf700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dc80c8c40 con 0x7f4dc806cbb0 2023-10-06T04:05:40.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.753+0000 7f4dce8cf700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc806e0c0 con 0x7f4dc806c1f0 2023-10-06T04:05:40.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.754+0000 7f4dc77fe700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dc806c1f0 0x7f4dc806da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:40.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.754+0000 7f4dc7fff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4dc806cbb0 0x7f4dc806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:40.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.754+0000 7f4dc77fe700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dc806c1f0 0x7f4dc806da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:39972/0 (socket says 172.21.15.157:39972) 2023-10-06T04:05:40.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.754+0000 7f4dc77fe700 1 -- 172.21.15.157:0/110611323 learned_addr learned my addr 172.21.15.157:0/110611323 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:40.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.754+0000 7f4dc7fff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4dc806cbb0 0x7f4dc806d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:34212/0 (socket says 172.21.15.157:34212) 2023-10-06T04:05:40.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.754+0000 7f4dcce6c700 1 --1- 172.21.15.157:0/110611323 >> v1:172.21.15.175:6789/0 conn(0x7f4dc806cf80 0x7f4dc814fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.157:41654/0 (socket says 172.21.15.157:41654) 2023-10-06T04:05:40.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.754+0000 7f4dc77fe700 1 -- 172.21.15.157:0/110611323 >> v2:172.21.15.175:3300/0 conn(0x7f4dc806cbb0 msgr2=0x7f4dc806d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:40.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.754+0000 7f4dc77fe700 1 --2- 172.21.15.157:0/110611323 >> v2:172.21.15.175:3300/0 conn(0x7f4dc806cbb0 0x7f4dc806d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:40.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.754+0000 7f4dc77fe700 1 -- 172.21.15.157:0/110611323 >> v1:172.21.15.175:6789/0 conn(0x7f4dc806cf80 legacy=0x7f4dc814fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:40.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.754+0000 7f4dc77fe700 1 -- 172.21.15.157:0/110611323 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc80c8e50 con 0x7f4dc806c1f0 2023-10-06T04:05:40.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.754+0000 7f4dc7fff700 1 --2- 172.21.15.157:0/110611323 >> v2:172.21.15.175:3300/0 conn(0x7f4dc806cbb0 0x7f4dc806d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:05:40.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.754+0000 7f4dc77fe700 1 --2- 172.21.15.157:0/110611323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dc806c1f0 0x7f4dc806da70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc0099e0 tx=0x7f4dbc009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60a70730e8f790e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:40.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.754+0000 7f4dc6ffd700 1 -- 172.21.15.157:0/110611323 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc00b8b0 con 0x7f4dc806c1f0 2023-10-06T04:05:40.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.754+0000 7f4dc6ffd700 1 -- 172.21.15.157:0/110611323 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dbc00ba10 con 0x7f4dc806c1f0 2023-10-06T04:05:40.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.754+0000 7f4dc6ffd700 1 -- 172.21.15.157:0/110611323 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc00bdb0 con 0x7f4dc806c1f0 2023-10-06T04:05:40.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.755+0000 7f4dce8cf700 1 -- 172.21.15.157:0/110611323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dc806c1f0 msgr2=0x7f4dc806da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:40.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.755+0000 7f4dce8cf700 1 --2- 172.21.15.157:0/110611323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dc806c1f0 0x7f4dc806da70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc0099e0 tx=0x7f4dbc009250 comp rx=0 tx=0).stop 2023-10-06T04:05:40.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.755+0000 7f4dce8cf700 1 -- 172.21.15.157:0/110611323 shutdown_connections 2023-10-06T04:05:40.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.755+0000 7f4dce8cf700 1 --2- 172.21.15.157:0/110611323 >> v2:172.21.15.175:3300/0 conn(0x7f4dc806cbb0 0x7f4dc806d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:40.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.755+0000 7f4dce8cf700 1 --2- 172.21.15.157:0/110611323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dc806c1f0 0x7f4dc806da70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:40.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.755+0000 7f4dce8cf700 1 -- 172.21.15.157:0/110611323 >> 172.21.15.157:0/110611323 conn(0x7f4dc806f230 msgr2=0x7f4dc806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:40.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.755+0000 7f4dce8cf700 1 -- 172.21.15.157:0/110611323 shutdown_connections 2023-10-06T04:05:40.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.755+0000 7f4dce8cf700 1 -- 172.21.15.157:0/110611323 wait complete. 2023-10-06T04:05:40.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.756+0000 7f4dce8cf700 1 Processor -- start 2023-10-06T04:05:40.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.756+0000 7f4dce8cf700 1 -- start start 2023-10-06T04:05:40.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.757+0000 7f4dce8cf700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dc806c1f0 0x7f4dc81584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:40.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.757+0000 7f4dce8cf700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dc806cbb0 0x7f4dc8158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:40.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.757+0000 7f4dce8cf700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dc806cf80 0x7f4dc8148930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:40.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.757+0000 7f4dce8cf700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc8157280 con 0x7f4dc806cf80 2023-10-06T04:05:40.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.757+0000 7f4dce8cf700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc8153100 con 0x7f4dc806cbb0 2023-10-06T04:05:40.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.757+0000 7f4dce8cf700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc8152480 con 0x7f4dc806c1f0 2023-10-06T04:05:40.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.757+0000 7f4dcce6c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dc806cf80 0x7f4dc8148930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:40.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.757+0000 7f4dcce6c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dc806cf80 0x7f4dc8148930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41522/0 (socket says 172.21.15.157:41522) 2023-10-06T04:05:40.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.757+0000 7f4dcce6c700 1 -- 172.21.15.157:0/3957003298 learned_addr learned my addr 172.21.15.157:0/3957003298 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:40.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.757+0000 7f4dc77fe700 1 --2- 172.21.15.157:0/3957003298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dc806cbb0 0x7f4dc8158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:40.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.757+0000 7f4dc7fff700 1 --2- 172.21.15.157:0/3957003298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dc806c1f0 0x7f4dc81584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:40.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.757+0000 7f4dcce6c700 1 -- 172.21.15.157:0/3957003298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dc806c1f0 msgr2=0x7f4dc81584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:40.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.757+0000 7f4dcce6c700 1 --2- 172.21.15.157:0/3957003298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dc806c1f0 0x7f4dc81584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:40.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.757+0000 7f4dcce6c700 1 -- 172.21.15.157:0/3957003298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dc806cbb0 msgr2=0x7f4dc8158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:40.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.757+0000 7f4dcce6c700 1 --2- 172.21.15.157:0/3957003298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dc806cbb0 0x7f4dc8158a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:40.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.757+0000 7f4dcce6c700 1 -- 172.21.15.157:0/3957003298 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db8009420 con 0x7f4dc806cf80 2023-10-06T04:05:40.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.757+0000 7f4dc77fe700 1 --2- 172.21.15.157:0/3957003298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dc806cbb0 0x7f4dc8158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:05:40.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.757+0000 7f4dcce6c700 1 --2- 172.21.15.157:0/3957003298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dc806cf80 0x7f4dc8148930 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f4dc000cc70 tx=0x7f4dc0001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:40.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.758+0000 7f4daffff700 1 -- 172.21.15.157:0/3957003298 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0005df0 con 0x7f4dc806cf80 2023-10-06T04:05:40.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.758+0000 7f4daffff700 1 -- 172.21.15.157:0/3957003298 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc000a5b0 con 0x7f4dc806cf80 2023-10-06T04:05:40.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.758+0000 7f4dce8cf700 1 -- 172.21.15.157:0/3957003298 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dbc0094f0 con 0x7f4dc806cf80 2023-10-06T04:05:40.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.758+0000 7f4daffff700 1 -- 172.21.15.157:0/3957003298 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc000a970 con 0x7f4dc806cf80 2023-10-06T04:05:40.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.758+0000 7f4dce8cf700 1 -- 172.21.15.157:0/3957003298 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc8152a40 con 0x7f4dc806cf80 2023-10-06T04:05:40.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.759+0000 7f4dce8cf700 1 -- 172.21.15.157:0/3957003298 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dc80b1c30 con 0x7f4dc806cf80 2023-10-06T04:05:40.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.759+0000 7f4daffff700 1 -- 172.21.15.157:0/3957003298 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4dc000ad20 con 0x7f4dc806cf80 2023-10-06T04:05:40.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.759+0000 7f4daffff700 1 --2- 172.21.15.157:0/3957003298 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4db4066f30 0x7f4db40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:40.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.759+0000 7f4daffff700 1 -- 172.21.15.157:0/3957003298 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4dc0084710 con 0x7f4dc806cf80 2023-10-06T04:05:40.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.760+0000 7f4dc7fff700 1 --2- 172.21.15.157:0/3957003298 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4db4066f30 0x7f4db40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:40.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.760+0000 7f4dc7fff700 1 --2- 172.21.15.157:0/3957003298 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4db4066f30 0x7f4db40693e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4db8009f40 tx=0x7f4db80091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:40.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.764+0000 7f4daffff700 1 -- 172.21.15.157:0/3957003298 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4dc0051710 con 0x7f4dc806cf80 2023-10-06T04:05:40.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.995+0000 7f4dce8cf700 1 -- 172.21.15.157:0/3957003298 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4dc80c9130 con 0x7f4dc806cf80 2023-10-06T04:05:40.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.995+0000 7f4daffff700 1 -- 172.21.15.157:0/3957003298 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+5389 (secure 0 0 0) 0x7f4dc0054d30 con 0x7f4dc806cf80 2023-10-06T04:05:40.997 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:05:40.997 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:05:39.766540+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:reconnect","state_seq":13,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:05:40.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.997+0000 7f4dce8cf700 1 -- 172.21.15.157:0/3957003298 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4db4066f30 msgr2=0x7f4db40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:40.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.997+0000 7f4dce8cf700 1 --2- 172.21.15.157:0/3957003298 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4db4066f30 0x7f4db40693e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4db8009f40 tx=0x7f4db80091a0 comp rx=0 tx=0).stop 2023-10-06T04:05:40.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.997+0000 7f4dce8cf700 1 -- 172.21.15.157:0/3957003298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dc806cf80 msgr2=0x7f4dc8148930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:40.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.997+0000 7f4dce8cf700 1 --2- 172.21.15.157:0/3957003298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dc806cf80 0x7f4dc8148930 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f4dc000cc70 tx=0x7f4dc0001a70 comp rx=0 tx=0).stop 2023-10-06T04:05:40.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.997+0000 7f4dce8cf700 1 -- 172.21.15.157:0/3957003298 shutdown_connections 2023-10-06T04:05:41.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.997+0000 7f4dce8cf700 1 --2- 172.21.15.157:0/3957003298 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4db4066f30 0x7f4db40693e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:41.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.997+0000 7f4dce8cf700 1 --2- 172.21.15.157:0/3957003298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dc806cf80 0x7f4dc8148930 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:41.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.997+0000 7f4dce8cf700 1 --2- 172.21.15.157:0/3957003298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4dc806c1f0 0x7f4dc81584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:41.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.997+0000 7f4dce8cf700 1 --2- 172.21.15.157:0/3957003298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4dc806cbb0 0x7f4dc8158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:41.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.997+0000 7f4dce8cf700 1 -- 172.21.15.157:0/3957003298 >> 172.21.15.157:0/3957003298 conn(0x7f4dc806f230 msgr2=0x7f4dc814d780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:41.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.997+0000 7f4dce8cf700 1 -- 172.21.15.157:0/3957003298 shutdown_connections 2023-10-06T04:05:41.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:40.997+0000 7f4dce8cf700 1 -- 172.21.15.157:0/3957003298 wait complete. 2023-10-06T04:05:41.000 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 66 2023-10-06T04:05:41.011 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 66, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-06T03:57:10.718782+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 66, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_5362": { "addr": "172.21.15.175:6837/2388164729", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6836", "nonce": 2388164729, "type": "v2" }, { "addr": "172.21.15.175:6837", "nonce": 2388164729, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5362, "incarnation": 58, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:active", "state_seq": 22 }, "gid_5799": { "addr": "172.21.15.157:6834/765157057", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6832", "nonce": 765157057, "type": "v2" }, { "addr": "172.21.15.157:6834", "nonce": 765157057, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5799, "incarnation": 64, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:reconnect", "state_seq": 13 } }, "last_failure": 0, "last_failure_osd_epoch": 50, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2023-10-06T04:05:39.766540+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2, 3, 4 ], "tableserver": 0, "up": { "mds_0": 5799, "mds_1": 5362 } } } ], "standbys": [ { "addr": "172.21.15.175:6835/1799968248", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6834", "nonce": 1799968248, "type": "v2" }, { "addr": "172.21.15.175:6835", "nonce": 1799968248, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 57, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5126, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.175:6839/745804271", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6838", "nonce": 745804271, "type": "v2" }, { "addr": "172.21.15.175:6839", "nonce": 745804271, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 42, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5164, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.157:6835/2461751318", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6833", "nonce": 2461751318, "type": "v2" }, { "addr": "172.21.15.157:6835", "nonce": 2461751318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 34, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5268, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-06T04:05:43.012 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:05:43.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.139+0000 7fc590f55700 1 Processor -- start 2023-10-06T04:05:43.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.139+0000 7fc590f55700 1 -- start start 2023-10-06T04:05:43.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.140+0000 7fc590f55700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc58c14dcd0 0x7fc58c153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:43.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.140+0000 7fc590f55700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc58c14e0a0 0x7fc58c0c18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:43.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.140+0000 7fc590f55700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc58c0c5770 con 0x7fc58c0c5c00 2023-10-06T04:05:43.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.140+0000 7fc590f55700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc58c154c80 con 0x7fc58c14e0a0 2023-10-06T04:05:43.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.140+0000 7fc590f55700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c154200 con 0x7fc58c14dcd0 2023-10-06T04:05:43.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.140+0000 7fc589d9b700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc58c14e0a0 0x7fc58c0c18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:43.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.140+0000 7fc58a59c700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc58c14dcd0 0x7fc58c153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:43.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.140+0000 7fc58a59c700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc58c14dcd0 0x7fc58c153cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:39988/0 (socket says 172.21.15.157:39988) 2023-10-06T04:05:43.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.140+0000 7fc58a59c700 1 -- 172.21.15.157:0/3958318299 learned_addr learned my addr 172.21.15.157:0/3958318299 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:43.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.140+0000 7fc589d9b700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc58c14e0a0 0x7fc58c0c18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:34226/0 (socket says 172.21.15.157:34226) 2023-10-06T04:05:43.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.141+0000 7fc58a59c700 1 -- 172.21.15.157:0/3958318299 >> v2:172.21.15.175:3300/0 conn(0x7fc58c14e0a0 msgr2=0x7fc58c0c18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:43.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.141+0000 7fc58a59c700 1 --2- 172.21.15.157:0/3958318299 >> v2:172.21.15.175:3300/0 conn(0x7fc58c14e0a0 0x7fc58c0c18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:43.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.141+0000 7fc58a59c700 1 -- 172.21.15.157:0/3958318299 >> v1:172.21.15.175:6789/0 conn(0x7fc58c0c5c00 legacy=0x7fc58c0ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:43.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.141+0000 7fc58a59c700 1 -- 172.21.15.157:0/3958318299 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc58c1543e0 con 0x7fc58c14dcd0 2023-10-06T04:05:43.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.141+0000 7fc589d9b700 1 --2- 172.21.15.157:0/3958318299 >> v2:172.21.15.175:3300/0 conn(0x7fc58c14e0a0 0x7fc58c0c18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:05:43.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.141+0000 7fc58a59c700 1 --2- 172.21.15.157:0/3958318299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc58c14dcd0 0x7fc58c153cc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc57c0099b0 tx=0x7fc57c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab4ea8c0c51a060d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:43.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.141+0000 7fc58959a700 1 -- 172.21.15.157:0/3958318299 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c009ea0 con 0x7fc58c14dcd0 2023-10-06T04:05:43.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.141+0000 7fc58959a700 1 -- 172.21.15.157:0/3958318299 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc57c00b860 con 0x7fc58c14dcd0 2023-10-06T04:05:43.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.141+0000 7fc58959a700 1 -- 172.21.15.157:0/3958318299 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c00bc00 con 0x7fc58c14dcd0 2023-10-06T04:05:43.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.141+0000 7fc590f55700 1 -- 172.21.15.157:0/3958318299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc58c14dcd0 msgr2=0x7fc58c153cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:43.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.142+0000 7fc590f55700 1 --2- 172.21.15.157:0/3958318299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc58c14dcd0 0x7fc58c153cc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc57c0099b0 tx=0x7fc57c009250 comp rx=0 tx=0).stop 2023-10-06T04:05:43.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.142+0000 7fc590f55700 1 -- 172.21.15.157:0/3958318299 shutdown_connections 2023-10-06T04:05:43.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.142+0000 7fc590f55700 1 --2- 172.21.15.157:0/3958318299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc58c14dcd0 0x7fc58c153cc0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:43.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.142+0000 7fc590f55700 1 --2- 172.21.15.157:0/3958318299 >> v2:172.21.15.175:3300/0 conn(0x7fc58c14e0a0 0x7fc58c0c18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:43.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.142+0000 7fc590f55700 1 -- 172.21.15.157:0/3958318299 >> 172.21.15.157:0/3958318299 conn(0x7fc58c0692b0 msgr2=0x7fc58c0696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:43.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.142+0000 7fc590f55700 1 -- 172.21.15.157:0/3958318299 shutdown_connections 2023-10-06T04:05:43.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.142+0000 7fc590f55700 1 -- 172.21.15.157:0/3958318299 wait complete. 2023-10-06T04:05:43.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.143+0000 7fc590f55700 1 Processor -- start 2023-10-06T04:05:43.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.143+0000 7fc590f55700 1 -- start start 2023-10-06T04:05:43.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.143+0000 7fc590f55700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc58c0c5c00 0x7fc58c164410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:43.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.143+0000 7fc590f55700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc58c14dcd0 0x7fc58c0ca740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:43.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.143+0000 7fc590f55700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc58c14e0a0 0x7fc58c0cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:43.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.143+0000 7fc590f55700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c158c50 con 0x7fc58c14e0a0 2023-10-06T04:05:43.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.143+0000 7fc590f55700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c156f50 con 0x7fc58c14dcd0 2023-10-06T04:05:43.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.143+0000 7fc590f55700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c0c8660 con 0x7fc58c0c5c00 2023-10-06T04:05:43.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.144+0000 7fc58ad9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc58c14e0a0 0x7fc58c0cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:43.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.144+0000 7fc58ad9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc58c14e0a0 0x7fc58c0cac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41534/0 (socket says 172.21.15.157:41534) 2023-10-06T04:05:43.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.144+0000 7fc58ad9d700 1 -- 172.21.15.157:0/2861116177 learned_addr learned my addr 172.21.15.157:0/2861116177 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:43.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.144+0000 7fc58a59c700 1 --2- 172.21.15.157:0/2861116177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc58c0c5c00 0x7fc58c164410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:43.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.144+0000 7fc589d9b700 1 --2- 172.21.15.157:0/2861116177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc58c14dcd0 0x7fc58c0ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:43.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.144+0000 7fc58ad9d700 1 -- 172.21.15.157:0/2861116177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc58c0c5c00 msgr2=0x7fc58c164410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:43.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.144+0000 7fc58ad9d700 1 --2- 172.21.15.157:0/2861116177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc58c0c5c00 0x7fc58c164410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:43.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.144+0000 7fc58ad9d700 1 -- 172.21.15.157:0/2861116177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc58c14dcd0 msgr2=0x7fc58c0ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:43.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.144+0000 7fc58ad9d700 1 --2- 172.21.15.157:0/2861116177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc58c14dcd0 0x7fc58c0ca740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:43.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.144+0000 7fc58ad9d700 1 -- 172.21.15.157:0/2861116177 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc580009420 con 0x7fc58c14e0a0 2023-10-06T04:05:43.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.144+0000 7fc589d9b700 1 --2- 172.21.15.157:0/2861116177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc58c14dcd0 0x7fc58c0ca740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:05:43.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.144+0000 7fc58ad9d700 1 --2- 172.21.15.157:0/2861116177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc58c14e0a0 0x7fc58c0cac80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc58400c8e0 tx=0x7fc584001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:43.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.144+0000 7fc57a7fc700 1 -- 172.21.15.157:0/2861116177 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58400a650 con 0x7fc58c14e0a0 2023-10-06T04:05:43.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.144+0000 7fc57a7fc700 1 -- 172.21.15.157:0/2861116177 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc58400a7b0 con 0x7fc58c14e0a0 2023-10-06T04:05:43.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.144+0000 7fc590f55700 1 -- 172.21.15.157:0/2861116177 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc57c0094f0 con 0x7fc58c14e0a0 2023-10-06T04:05:43.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.144+0000 7fc57a7fc700 1 -- 172.21.15.157:0/2861116177 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58400ab70 con 0x7fc58c14e0a0 2023-10-06T04:05:43.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.144+0000 7fc590f55700 1 -- 172.21.15.157:0/2861116177 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc58c156ac0 con 0x7fc58c14e0a0 2023-10-06T04:05:43.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.145+0000 7fc590f55700 1 -- 172.21.15.157:0/2861116177 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc58c164e40 con 0x7fc58c14e0a0 2023-10-06T04:05:43.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.146+0000 7fc57a7fc700 1 -- 172.21.15.157:0/2861116177 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc584012420 con 0x7fc58c14e0a0 2023-10-06T04:05:43.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.146+0000 7fc57a7fc700 1 --2- 172.21.15.157:0/2861116177 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc56c066f30 0x7fc56c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:43.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.146+0000 7fc57a7fc700 1 -- 172.21.15.157:0/2861116177 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc584084ae0 con 0x7fc58c14e0a0 2023-10-06T04:05:43.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.146+0000 7fc58a59c700 1 --2- 172.21.15.157:0/2861116177 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc56c066f30 0x7fc56c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:43.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.147+0000 7fc58a59c700 1 --2- 172.21.15.157:0/2861116177 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc56c066f30 0x7fc56c0693e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc57c0099b0 tx=0x7fc57c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:43.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.150+0000 7fc57a7fc700 1 -- 172.21.15.157:0/2861116177 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc584051a60 con 0x7fc58c14e0a0 2023-10-06T04:05:43.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.375+0000 7fc590f55700 1 -- 172.21.15.157:0/2861116177 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc58c156c50 con 0x7fc58c14e0a0 2023-10-06T04:05:43.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.376+0000 7fc57a7fc700 1 -- 172.21.15.157:0/2861116177 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+5389 (secure 0 0 0) 0x7fc584055080 con 0x7fc58c14e0a0 2023-10-06T04:05:43.378 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:05:43.378 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:05:39.766540+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:reconnect","state_seq":13,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:05:43.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.378+0000 7fc590f55700 1 -- 172.21.15.157:0/2861116177 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc56c066f30 msgr2=0x7fc56c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:43.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.378+0000 7fc590f55700 1 --2- 172.21.15.157:0/2861116177 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc56c066f30 0x7fc56c0693e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc57c0099b0 tx=0x7fc57c017000 comp rx=0 tx=0).stop 2023-10-06T04:05:43.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.378+0000 7fc590f55700 1 -- 172.21.15.157:0/2861116177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc58c14e0a0 msgr2=0x7fc58c0cac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:43.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.378+0000 7fc590f55700 1 --2- 172.21.15.157:0/2861116177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc58c14e0a0 0x7fc58c0cac80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc58400c8e0 tx=0x7fc584001710 comp rx=0 tx=0).stop 2023-10-06T04:05:43.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.378+0000 7fc590f55700 1 -- 172.21.15.157:0/2861116177 shutdown_connections 2023-10-06T04:05:43.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.378+0000 7fc590f55700 1 --2- 172.21.15.157:0/2861116177 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fc56c066f30 0x7fc56c0693e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:43.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.378+0000 7fc590f55700 1 --2- 172.21.15.157:0/2861116177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc58c14e0a0 0x7fc58c0cac80 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:43.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.378+0000 7fc590f55700 1 --2- 172.21.15.157:0/2861116177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc58c0c5c00 0x7fc58c164410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:43.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.378+0000 7fc590f55700 1 --2- 172.21.15.157:0/2861116177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc58c14dcd0 0x7fc58c0ca740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:43.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.378+0000 7fc590f55700 1 -- 172.21.15.157:0/2861116177 >> 172.21.15.157:0/2861116177 conn(0x7fc58c0692b0 msgr2=0x7fc58c14c540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:43.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.378+0000 7fc590f55700 1 -- 172.21.15.157:0/2861116177 shutdown_connections 2023-10-06T04:05:43.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:43.378+0000 7fc590f55700 1 -- 172.21.15.157:0/2861116177 wait complete. 2023-10-06T04:05:43.381 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 66 2023-10-06T04:05:43.392 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:05:45.393 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:05:45.419 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:05:45.440 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:05:45.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.522+0000 7f7b09607700 1 Processor -- start 2023-10-06T04:05:45.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.522+0000 7f7b09607700 1 -- start start 2023-10-06T04:05:45.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.523+0000 7f7b09607700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7b0406cbb0 0x7f7b0406d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:45.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.523+0000 7f7b09607700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b0406c1f0 0x7f7b0406da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:45.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.523+0000 7f7b09607700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b040c9780 con 0x7f7b0406cf80 2023-10-06T04:05:45.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.523+0000 7f7b09607700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b040c8c40 con 0x7f7b0406cbb0 2023-10-06T04:05:45.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.523+0000 7f7b09607700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0406e0c0 con 0x7f7b0406c1f0 2023-10-06T04:05:45.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.523+0000 7f7b037fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7b0406cf80 0x7f7b0414fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:56104/0 (socket says 172.21.15.157:56104) 2023-10-06T04:05:45.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.523+0000 7f7b037fe700 1 -- 172.21.15.157:0/3981246034 learned_addr learned my addr 172.21.15.157:0/3981246034 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:05:45.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.523+0000 7f7b027fc700 1 --2- 172.21.15.157:0/3981246034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b0406c1f0 0x7f7b0406da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:45.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.523+0000 7f7b02ffd700 1 --2- 172.21.15.157:0/3981246034 >> v2:172.21.15.175:3300/0 conn(0x7f7b0406cbb0 0x7f7b0406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:45.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.524+0000 7f7b01ffb700 1 -- 172.21.15.157:0/3981246034 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1948692451 0 0) 0x7f7b040c9780 con 0x7f7b0406cf80 2023-10-06T04:05:45.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.524+0000 7f7b01ffb700 1 -- 172.21.15.157:0/3981246034 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7aec003570 con 0x7f7b0406cf80 2023-10-06T04:05:45.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.524+0000 7f7b02ffd700 1 -- 172.21.15.157:0/3981246034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b0406c1f0 msgr2=0x7f7b0406da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:45.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.524+0000 7f7b02ffd700 1 --2- 172.21.15.157:0/3981246034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b0406c1f0 0x7f7b0406da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:45.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.524+0000 7f7b02ffd700 1 -- 172.21.15.157:0/3981246034 >> v1:172.21.15.157:6789/0 conn(0x7f7b0406cf80 legacy=0x7f7b0414fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:45.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.524+0000 7f7b02ffd700 1 -- 172.21.15.157:0/3981246034 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b040c8e50 con 0x7f7b0406cbb0 2023-10-06T04:05:45.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.524+0000 7f7b027fc700 1 --2- 172.21.15.157:0/3981246034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b0406c1f0 0x7f7b0406da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:05:45.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.524+0000 7f7b02ffd700 1 --2- 172.21.15.157:0/3981246034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b0406cbb0 0x7f7b0406d530 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009910 tx=0x7f7af40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12e898dc7d755a49 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:45.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.524+0000 7f7b01ffb700 1 -- 172.21.15.157:0/3981246034 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4011630 con 0x7f7b0406cbb0 2023-10-06T04:05:45.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.524+0000 7f7b01ffb700 1 -- 172.21.15.157:0/3981246034 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af4011790 con 0x7f7b0406cbb0 2023-10-06T04:05:45.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.524+0000 7f7b01ffb700 1 -- 172.21.15.157:0/3981246034 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4011b30 con 0x7f7b0406cbb0 2023-10-06T04:05:45.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.525+0000 7f7b09607700 1 -- 172.21.15.157:0/3981246034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b0406cbb0 msgr2=0x7f7b0406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:45.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.525+0000 7f7b09607700 1 --2- 172.21.15.157:0/3981246034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b0406cbb0 0x7f7b0406d530 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009910 tx=0x7f7af40091a0 comp rx=0 tx=0).stop 2023-10-06T04:05:45.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.525+0000 7f7b09607700 1 -- 172.21.15.157:0/3981246034 shutdown_connections 2023-10-06T04:05:45.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.525+0000 7f7b09607700 1 --2- 172.21.15.157:0/3981246034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b0406cbb0 0x7f7b0406d530 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:45.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.525+0000 7f7b09607700 1 --2- 172.21.15.157:0/3981246034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b0406c1f0 0x7f7b0406da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:45.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.525+0000 7f7b09607700 1 -- 172.21.15.157:0/3981246034 >> 172.21.15.157:0/3981246034 conn(0x7f7b0406f230 msgr2=0x7f7b0406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:45.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.525+0000 7f7b09607700 1 -- 172.21.15.157:0/3981246034 shutdown_connections 2023-10-06T04:05:45.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.525+0000 7f7b09607700 1 -- 172.21.15.157:0/3981246034 wait complete. 2023-10-06T04:05:45.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.526+0000 7f7b09607700 1 Processor -- start 2023-10-06T04:05:45.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.526+0000 7f7b09607700 1 -- start start 2023-10-06T04:05:45.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.527+0000 7f7b09607700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b0406c1f0 0x7f7b04169fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:45.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.527+0000 7f7b09607700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b0406cbb0 0x7f7b040c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:45.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.527+0000 7f7b09607700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b0406cf80 0x7f7b04166800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:45.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.527+0000 7f7b09607700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b04154680 con 0x7f7b0406cbb0 2023-10-06T04:05:45.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.527+0000 7f7b09607700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b04169b90 con 0x7f7b0406cf80 2023-10-06T04:05:45.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.527+0000 7f7b09607700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b04154f20 con 0x7f7b0406c1f0 2023-10-06T04:05:45.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.527+0000 7f7b027fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b0406cbb0 0x7f7b040c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:45.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.527+0000 7f7b027fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b0406cbb0 0x7f7b040c3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40010/0 (socket says 172.21.15.157:40010) 2023-10-06T04:05:45.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.527+0000 7f7b027fc700 1 -- 172.21.15.157:0/2032828792 learned_addr learned my addr 172.21.15.157:0/2032828792 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:45.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.527+0000 7f7b02ffd700 1 --2- 172.21.15.157:0/2032828792 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b0406c1f0 0x7f7b04169fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:45.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.527+0000 7f7b037fe700 1 --2- 172.21.15.157:0/2032828792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b0406cf80 0x7f7b04166800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:45.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.527+0000 7f7b027fc700 1 -- 172.21.15.157:0/2032828792 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b0406c1f0 msgr2=0x7f7b04169fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:45.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.527+0000 7f7b027fc700 1 --2- 172.21.15.157:0/2032828792 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b0406c1f0 0x7f7b04169fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:45.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.527+0000 7f7b027fc700 1 -- 172.21.15.157:0/2032828792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b0406cf80 msgr2=0x7f7b04166800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:45.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.527+0000 7f7b027fc700 1 --2- 172.21.15.157:0/2032828792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b0406cf80 0x7f7b04166800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:45.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.527+0000 7f7b027fc700 1 -- 172.21.15.157:0/2032828792 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af8009420 con 0x7f7b0406cbb0 2023-10-06T04:05:45.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.527+0000 7f7b037fe700 1 --2- 172.21.15.157:0/2032828792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b0406cf80 0x7f7b04166800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:05:45.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.527+0000 7f7b027fc700 1 --2- 172.21.15.157:0/2032828792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b0406cbb0 0x7f7b040c3be0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7af800ec80 tx=0x7f7af80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:45.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.527+0000 7f7aeaffd700 1 -- 172.21.15.157:0/2032828792 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af800c8f0 con 0x7f7b0406cbb0 2023-10-06T04:05:45.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.527+0000 7f7aeaffd700 1 -- 172.21.15.157:0/2032828792 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af800ca50 con 0x7f7b0406cbb0 2023-10-06T04:05:45.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.527+0000 7f7aeaffd700 1 -- 172.21.15.157:0/2032828792 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af800cdd0 con 0x7f7b0406cbb0 2023-10-06T04:05:45.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.527+0000 7f7b09607700 1 -- 172.21.15.157:0/2032828792 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af4009420 con 0x7f7b0406cbb0 2023-10-06T04:05:45.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.527+0000 7f7b09607700 1 -- 172.21.15.157:0/2032828792 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b040c4db0 con 0x7f7b0406cbb0 2023-10-06T04:05:45.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.528+0000 7f7b09607700 1 -- 172.21.15.157:0/2032828792 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b04169cd0 con 0x7f7b0406cbb0 2023-10-06T04:05:45.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.529+0000 7f7aeaffd700 1 -- 172.21.15.157:0/2032828792 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7af8013680 con 0x7f7b0406cbb0 2023-10-06T04:05:45.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.529+0000 7f7aeaffd700 1 --2- 172.21.15.157:0/2032828792 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7af0066f30 0x7f7af00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:45.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.529+0000 7f7aeaffd700 1 -- 172.21.15.157:0/2032828792 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7af8084bd0 con 0x7f7b0406cbb0 2023-10-06T04:05:45.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.529+0000 7f7b02ffd700 1 --2- 172.21.15.157:0/2032828792 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7af0066f30 0x7f7af00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:45.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.530+0000 7f7b02ffd700 1 --2- 172.21.15.157:0/2032828792 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7af0066f30 0x7f7af00693e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7af40091e0 tx=0x7f7af4017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:45.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.534+0000 7f7aeaffd700 1 -- 172.21.15.157:0/2032828792 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b04169cd0 con 0x7f7b0406cbb0 2023-10-06T04:05:45.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.762+0000 7f7b09607700 1 -- 172.21.15.157:0/2032828792 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b04169cd0 con 0x7f7b0406cbb0 2023-10-06T04:05:45.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.763+0000 7f7aeaffd700 1 -- 172.21.15.157:0/2032828792 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+5386 (secure 0 0 0) 0x7f7b04169cd0 con 0x7f7b0406cbb0 2023-10-06T04:05:45.764 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:05:45.765 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:05:44.778875+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:rejoin","state_seq":15,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:05:45.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.765+0000 7f7b09607700 1 -- 172.21.15.157:0/2032828792 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7af0066f30 msgr2=0x7f7af00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:45.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.765+0000 7f7b09607700 1 --2- 172.21.15.157:0/2032828792 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7af0066f30 0x7f7af00693e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7af40091e0 tx=0x7f7af4017000 comp rx=0 tx=0).stop 2023-10-06T04:05:45.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.765+0000 7f7b09607700 1 -- 172.21.15.157:0/2032828792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b0406cbb0 msgr2=0x7f7b040c3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:45.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.765+0000 7f7b09607700 1 --2- 172.21.15.157:0/2032828792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b0406cbb0 0x7f7b040c3be0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7af800ec80 tx=0x7f7af80091a0 comp rx=0 tx=0).stop 2023-10-06T04:05:45.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.765+0000 7f7b09607700 1 -- 172.21.15.157:0/2032828792 shutdown_connections 2023-10-06T04:05:45.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.765+0000 7f7b09607700 1 --2- 172.21.15.157:0/2032828792 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7af0066f30 0x7f7af00693e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:45.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.765+0000 7f7b09607700 1 --2- 172.21.15.157:0/2032828792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b0406cf80 0x7f7b04166800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:45.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.765+0000 7f7b09607700 1 --2- 172.21.15.157:0/2032828792 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b0406c1f0 0x7f7b04169fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:45.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.765+0000 7f7b09607700 1 --2- 172.21.15.157:0/2032828792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b0406cbb0 0x7f7b040c3be0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:45.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.765+0000 7f7b09607700 1 -- 172.21.15.157:0/2032828792 >> 172.21.15.157:0/2032828792 conn(0x7f7b0406f230 msgr2=0x7f7b040cbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:45.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.765+0000 7f7b09607700 1 -- 172.21.15.157:0/2032828792 shutdown_connections 2023-10-06T04:05:45.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:45.765+0000 7f7b09607700 1 -- 172.21.15.157:0/2032828792 wait complete. 2023-10-06T04:05:45.768 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 67 2023-10-06T04:05:45.779 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:05:47.780 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:05:47.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.906+0000 7f4e589bc700 1 Processor -- start 2023-10-06T04:05:47.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.906+0000 7f4e589bc700 1 -- start start 2023-10-06T04:05:47.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.907+0000 7f4e589bc700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4e5413f0e0 0x7f4e540c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:47.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.907+0000 7f4e589bc700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e540c3d50 0x7f4e5406a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:47.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.907+0000 7f4e589bc700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e541596a0 con 0x7f4e5406ab00 2023-10-06T04:05:47.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.907+0000 7f4e589bc700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e540c6980 con 0x7f4e5413f0e0 2023-10-06T04:05:47.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.907+0000 7f4e589bc700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e54155f70 con 0x7f4e540c3d50 2023-10-06T04:05:47.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.907+0000 7f4e52d9d700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f4e5406ab00 0x7f4e54159b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:56120/0 (socket says 172.21.15.157:56120) 2023-10-06T04:05:47.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.907+0000 7f4e52d9d700 1 -- 172.21.15.157:0/243464341 learned_addr learned my addr 172.21.15.157:0/243464341 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:05:47.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.907+0000 7f4e49d9b700 1 --2- 172.21.15.157:0/243464341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e540c3d50 0x7f4e5406a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:47.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.907+0000 7f4e5259c700 1 --2- 172.21.15.157:0/243464341 >> v2:172.21.15.175:3300/0 conn(0x7f4e5413f0e0 0x7f4e540c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:47.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.908+0000 7f4e51d9b700 1 -- 172.21.15.157:0/243464341 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 686794608 0 0) 0x7f4e541596a0 con 0x7f4e5406ab00 2023-10-06T04:05:47.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.908+0000 7f4e51d9b700 1 -- 172.21.15.157:0/243464341 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e40003570 con 0x7f4e5406ab00 2023-10-06T04:05:47.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.908+0000 7f4e51d9b700 1 -- 172.21.15.157:0/243464341 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3466502318 0 0) 0x7f4e40003570 con 0x7f4e5406ab00 2023-10-06T04:05:47.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.908+0000 7f4e49d9b700 1 -- 172.21.15.157:0/243464341 >> v2:172.21.15.175:3300/0 conn(0x7f4e5413f0e0 msgr2=0x7f4e540c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:47.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.908+0000 7f4e49d9b700 1 --2- 172.21.15.157:0/243464341 >> v2:172.21.15.175:3300/0 conn(0x7f4e5413f0e0 0x7f4e540c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:47.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.908+0000 7f4e49d9b700 1 -- 172.21.15.157:0/243464341 >> v1:172.21.15.157:6789/0 conn(0x7f4e5406ab00 legacy=0x7f4e54159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:47.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.908+0000 7f4e49d9b700 1 -- 172.21.15.157:0/243464341 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e54156230 con 0x7f4e540c3d50 2023-10-06T04:05:47.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.908+0000 7f4e5259c700 1 --2- 172.21.15.157:0/243464341 >> v2:172.21.15.175:3300/0 conn(0x7f4e5413f0e0 0x7f4e540c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:05:47.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.908+0000 7f4e49d9b700 1 --2- 172.21.15.157:0/243464341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e540c3d50 0x7f4e5406a5c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c0098e0 tx=0x7f4e3c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cff149ab947573dc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:47.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.908+0000 7f4e51d9b700 1 -- 172.21.15.157:0/243464341 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c0017f0 con 0x7f4e540c3d50 2023-10-06T04:05:47.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.908+0000 7f4e51d9b700 1 -- 172.21.15.157:0/243464341 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3c011420 con 0x7f4e540c3d50 2023-10-06T04:05:47.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.908+0000 7f4e51d9b700 1 -- 172.21.15.157:0/243464341 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c0117c0 con 0x7f4e540c3d50 2023-10-06T04:05:47.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.909+0000 7f4e589bc700 1 -- 172.21.15.157:0/243464341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e540c3d50 msgr2=0x7f4e5406a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:47.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.909+0000 7f4e589bc700 1 --2- 172.21.15.157:0/243464341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e540c3d50 0x7f4e5406a5c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c0098e0 tx=0x7f4e3c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:05:47.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.909+0000 7f4e589bc700 1 -- 172.21.15.157:0/243464341 shutdown_connections 2023-10-06T04:05:47.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.909+0000 7f4e589bc700 1 --2- 172.21.15.157:0/243464341 >> v2:172.21.15.175:3300/0 conn(0x7f4e5413f0e0 0x7f4e540c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:47.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.909+0000 7f4e589bc700 1 --2- 172.21.15.157:0/243464341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e540c3d50 0x7f4e5406a5c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:47.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.909+0000 7f4e589bc700 1 -- 172.21.15.157:0/243464341 >> 172.21.15.157:0/243464341 conn(0x7f4e5406b6a0 msgr2=0x7f4e540631f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:47.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.909+0000 7f4e589bc700 1 -- 172.21.15.157:0/243464341 shutdown_connections 2023-10-06T04:05:47.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.909+0000 7f4e589bc700 1 -- 172.21.15.157:0/243464341 wait complete. 2023-10-06T04:05:47.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.910+0000 7f4e589bc700 1 Processor -- start 2023-10-06T04:05:47.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.910+0000 7f4e589bc700 1 -- start start 2023-10-06T04:05:47.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.911+0000 7f4e589bc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e5406ab00 0x7f4e540c3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:47.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.911+0000 7f4e589bc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e540c3d50 0x7f4e540c1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:47.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.911+0000 7f4e589bc700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e5413f0e0 0x7f4e540c06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:47.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.911+0000 7f4e589bc700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e541669b0 con 0x7f4e5406ab00 2023-10-06T04:05:47.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.911+0000 7f4e589bc700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e54166e40 con 0x7f4e540c3d50 2023-10-06T04:05:47.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.911+0000 7f4e589bc700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5413b710 con 0x7f4e5413f0e0 2023-10-06T04:05:47.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.911+0000 7f4e5259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e5406ab00 0x7f4e540c3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:47.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.911+0000 7f4e5259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e5406ab00 0x7f4e540c3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40024/0 (socket says 172.21.15.157:40024) 2023-10-06T04:05:47.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.911+0000 7f4e5259c700 1 -- 172.21.15.157:0/2723553053 learned_addr learned my addr 172.21.15.157:0/2723553053 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:47.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.911+0000 7f4e49d9b700 1 --2- 172.21.15.157:0/2723553053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e540c3d50 0x7f4e540c1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:47.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.911+0000 7f4e52d9d700 1 --2- 172.21.15.157:0/2723553053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e5413f0e0 0x7f4e540c06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:47.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.911+0000 7f4e5259c700 1 -- 172.21.15.157:0/2723553053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e5413f0e0 msgr2=0x7f4e540c06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:47.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.911+0000 7f4e5259c700 1 --2- 172.21.15.157:0/2723553053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e5413f0e0 0x7f4e540c06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:47.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.911+0000 7f4e5259c700 1 -- 172.21.15.157:0/2723553053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e540c3d50 msgr2=0x7f4e540c1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:47.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.911+0000 7f4e5259c700 1 --2- 172.21.15.157:0/2723553053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e540c3d50 0x7f4e540c1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:47.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.911+0000 7f4e5259c700 1 -- 172.21.15.157:0/2723553053 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e44009420 con 0x7f4e5406ab00 2023-10-06T04:05:47.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.911+0000 7f4e5259c700 1 --2- 172.21.15.157:0/2723553053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e5406ab00 0x7f4e540c3240 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4e4400ec80 tx=0x7f4e440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:47.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.911+0000 7f4e4affd700 1 -- 172.21.15.157:0/2723553053 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4400c9d0 con 0x7f4e5406ab00 2023-10-06T04:05:47.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.911+0000 7f4e4affd700 1 -- 172.21.15.157:0/2723553053 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e4400cb30 con 0x7f4e5406ab00 2023-10-06T04:05:47.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.911+0000 7f4e589bc700 1 -- 172.21.15.157:0/2723553053 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e3c009420 con 0x7f4e5406ab00 2023-10-06T04:05:47.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.911+0000 7f4e4affd700 1 -- 172.21.15.157:0/2723553053 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4400ced0 con 0x7f4e5406ab00 2023-10-06T04:05:47.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.911+0000 7f4e589bc700 1 -- 172.21.15.157:0/2723553053 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e54166080 con 0x7f4e5406ab00 2023-10-06T04:05:47.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.912+0000 7f4e589bc700 1 -- 172.21.15.157:0/2723553053 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e54166c90 con 0x7f4e5406ab00 2023-10-06T04:05:47.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.913+0000 7f4e4affd700 1 -- 172.21.15.157:0/2723553053 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4e44013680 con 0x7f4e5406ab00 2023-10-06T04:05:47.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.913+0000 7f4e4affd700 1 --2- 172.21.15.157:0/2723553053 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4e34066f60 0x7f4e34069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:47.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.913+0000 7f4e4affd700 1 -- 172.21.15.157:0/2723553053 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4e44084b20 con 0x7f4e5406ab00 2023-10-06T04:05:47.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.913+0000 7f4e49d9b700 1 --2- 172.21.15.157:0/2723553053 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4e34066f60 0x7f4e34069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:47.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.914+0000 7f4e49d9b700 1 --2- 172.21.15.157:0/2723553053 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4e34066f60 0x7f4e34069410 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c0093f0 tx=0x7f4e3c016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:47.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:47.918+0000 7f4e4affd700 1 -- 172.21.15.157:0/2723553053 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4e44051aa0 con 0x7f4e5406ab00 2023-10-06T04:05:48.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:48.144+0000 7f4e589bc700 1 -- 172.21.15.157:0/2723553053 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e5404bb40 con 0x7f4e5406ab00 2023-10-06T04:05:48.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:48.144+0000 7f4e4affd700 1 -- 172.21.15.157:0/2723553053 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+5386 (secure 0 0 0) 0x7f4e44084020 con 0x7f4e5406ab00 2023-10-06T04:05:48.146 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:05:48.146 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:05:44.778875+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:rejoin","state_seq":15,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:05:48.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:48.146+0000 7f4e589bc700 1 -- 172.21.15.157:0/2723553053 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4e34066f60 msgr2=0x7f4e34069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:48.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:48.146+0000 7f4e589bc700 1 --2- 172.21.15.157:0/2723553053 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4e34066f60 0x7f4e34069410 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c0093f0 tx=0x7f4e3c016000 comp rx=0 tx=0).stop 2023-10-06T04:05:48.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:48.146+0000 7f4e589bc700 1 -- 172.21.15.157:0/2723553053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e5406ab00 msgr2=0x7f4e540c3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:48.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:48.146+0000 7f4e589bc700 1 --2- 172.21.15.157:0/2723553053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e5406ab00 0x7f4e540c3240 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4e4400ec80 tx=0x7f4e440091a0 comp rx=0 tx=0).stop 2023-10-06T04:05:48.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:48.146+0000 7f4e589bc700 1 -- 172.21.15.157:0/2723553053 shutdown_connections 2023-10-06T04:05:48.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:48.146+0000 7f4e589bc700 1 --2- 172.21.15.157:0/2723553053 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4e34066f60 0x7f4e34069410 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:48.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:48.146+0000 7f4e589bc700 1 --2- 172.21.15.157:0/2723553053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e5413f0e0 0x7f4e540c06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:48.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:48.146+0000 7f4e589bc700 1 --2- 172.21.15.157:0/2723553053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e5406ab00 0x7f4e540c3240 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:48.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:48.146+0000 7f4e589bc700 1 --2- 172.21.15.157:0/2723553053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e540c3d50 0x7f4e540c1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:48.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:48.146+0000 7f4e589bc700 1 -- 172.21.15.157:0/2723553053 >> 172.21.15.157:0/2723553053 conn(0x7f4e5406b6a0 msgr2=0x7f4e5413dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:48.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:48.146+0000 7f4e589bc700 1 -- 172.21.15.157:0/2723553053 shutdown_connections 2023-10-06T04:05:48.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:48.146+0000 7f4e589bc700 1 -- 172.21.15.157:0/2723553053 wait complete. 2023-10-06T04:05:48.149 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 67 2023-10-06T04:05:48.160 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:05:50.161 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:05:50.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.283+0000 7f2ed757f700 1 Processor -- start 2023-10-06T04:05:50.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.283+0000 7f2ed757f700 1 -- start start 2023-10-06T04:05:50.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.284+0000 7f2ed757f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2ed014dcd0 0x7f2ed0153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:50.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.284+0000 7f2ed757f700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2ed014e0a0 0x7f2ed00c18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:50.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.284+0000 7f2ed757f700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ed00c5770 con 0x7f2ed00c5c00 2023-10-06T04:05:50.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.284+0000 7f2ed757f700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ed0154c80 con 0x7f2ed014e0a0 2023-10-06T04:05:50.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.284+0000 7f2ed757f700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ed0154200 con 0x7f2ed014dcd0 2023-10-06T04:05:50.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.284+0000 7f2ed4b1a700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2ed014e0a0 0x7f2ed00c18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:50.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.284+0000 7f2ed4b1a700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2ed014e0a0 0x7f2ed00c18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40038/0 (socket says 172.21.15.157:40038) 2023-10-06T04:05:50.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.284+0000 7f2ed4b1a700 1 -- 172.21.15.157:0/3979509163 learned_addr learned my addr 172.21.15.157:0/3979509163 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:50.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.284+0000 7f2ed531b700 1 --2- 172.21.15.157:0/3979509163 >> v2:172.21.15.175:3300/0 conn(0x7f2ed014dcd0 0x7f2ed0153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:50.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.284+0000 7f2ed4b1a700 1 -- 172.21.15.157:0/3979509163 >> v2:172.21.15.175:3300/0 conn(0x7f2ed014dcd0 msgr2=0x7f2ed0153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:50.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.284+0000 7f2ed4b1a700 1 --2- 172.21.15.157:0/3979509163 >> v2:172.21.15.175:3300/0 conn(0x7f2ed014dcd0 0x7f2ed0153cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:50.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.284+0000 7f2ed4b1a700 1 -- 172.21.15.157:0/3979509163 >> v1:172.21.15.175:6789/0 conn(0x7f2ed00c5c00 legacy=0x7f2ed00ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:50.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.284+0000 7f2ed4b1a700 1 -- 172.21.15.157:0/3979509163 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed01543e0 con 0x7f2ed014e0a0 2023-10-06T04:05:50.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.284+0000 7f2ed4b1a700 1 --2- 172.21.15.157:0/3979509163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ed014e0a0 0x7f2ed00c18c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2ed006cd20 tx=0x7f2ec0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813123737d5aa9f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:50.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.284+0000 7f2ec7fff700 1 -- 172.21.15.157:0/3979509163 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec000b8b0 con 0x7f2ed014e0a0 2023-10-06T04:05:50.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.284+0000 7f2ec7fff700 1 -- 172.21.15.157:0/3979509163 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec000ba10 con 0x7f2ed014e0a0 2023-10-06T04:05:50.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.284+0000 7f2ec7fff700 1 -- 172.21.15.157:0/3979509163 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec000bdb0 con 0x7f2ed014e0a0 2023-10-06T04:05:50.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.285+0000 7f2ed757f700 1 -- 172.21.15.157:0/3979509163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ed014e0a0 msgr2=0x7f2ed00c18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:50.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.285+0000 7f2ed757f700 1 --2- 172.21.15.157:0/3979509163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ed014e0a0 0x7f2ed00c18c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2ed006cd20 tx=0x7f2ec0009250 comp rx=0 tx=0).stop 2023-10-06T04:05:50.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.285+0000 7f2ed757f700 1 -- 172.21.15.157:0/3979509163 shutdown_connections 2023-10-06T04:05:50.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.285+0000 7f2ed757f700 1 --2- 172.21.15.157:0/3979509163 >> v2:172.21.15.175:3300/0 conn(0x7f2ed014dcd0 0x7f2ed0153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:50.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.285+0000 7f2ed757f700 1 --2- 172.21.15.157:0/3979509163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ed014e0a0 0x7f2ed00c18c0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:50.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.285+0000 7f2ed757f700 1 -- 172.21.15.157:0/3979509163 >> 172.21.15.157:0/3979509163 conn(0x7f2ed00692b0 msgr2=0x7f2ed00696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:50.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.285+0000 7f2ed757f700 1 -- 172.21.15.157:0/3979509163 shutdown_connections 2023-10-06T04:05:50.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.285+0000 7f2ed757f700 1 -- 172.21.15.157:0/3979509163 wait complete. 2023-10-06T04:05:50.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.286+0000 7f2ed757f700 1 Processor -- start 2023-10-06T04:05:50.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.286+0000 7f2ed757f700 1 -- start start 2023-10-06T04:05:50.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.287+0000 7f2ed757f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ed00c5c00 0x7f2ed006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:50.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.287+0000 7f2ed757f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ed014dcd0 0x7f2ed01568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:50.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.287+0000 7f2ed757f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ed014e0a0 0x7f2ed0169ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:50.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.287+0000 7f2ed757f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed006e390 con 0x7f2ed00c5c00 2023-10-06T04:05:50.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.287+0000 7f2ed757f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed006e510 con 0x7f2ed014dcd0 2023-10-06T04:05:50.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.287+0000 7f2ed757f700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed006ead0 con 0x7f2ed014e0a0 2023-10-06T04:05:50.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.287+0000 7f2ed531b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ed00c5c00 0x7f2ed006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:50.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.287+0000 7f2ed531b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ed00c5c00 0x7f2ed006d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40044/0 (socket says 172.21.15.157:40044) 2023-10-06T04:05:50.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.287+0000 7f2ed531b700 1 -- 172.21.15.157:0/1785246960 learned_addr learned my addr 172.21.15.157:0/1785246960 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:50.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.287+0000 7f2ed4b1a700 1 --2- 172.21.15.157:0/1785246960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ed014dcd0 0x7f2ed01568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:50.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.287+0000 7f2ed5b1c700 1 --2- 172.21.15.157:0/1785246960 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ed014e0a0 0x7f2ed0169ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:50.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.287+0000 7f2ed531b700 1 -- 172.21.15.157:0/1785246960 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ed014e0a0 msgr2=0x7f2ed0169ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:50.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.287+0000 7f2ed531b700 1 --2- 172.21.15.157:0/1785246960 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ed014e0a0 0x7f2ed0169ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:50.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.287+0000 7f2ed531b700 1 -- 172.21.15.157:0/1785246960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ed014dcd0 msgr2=0x7f2ed01568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:50.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.287+0000 7f2ed531b700 1 --2- 172.21.15.157:0/1785246960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ed014dcd0 0x7f2ed01568a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:50.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.287+0000 7f2ed531b700 1 -- 172.21.15.157:0/1785246960 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec00094f0 con 0x7f2ed00c5c00 2023-10-06T04:05:50.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.287+0000 7f2ed4b1a700 1 --2- 172.21.15.157:0/1785246960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ed014dcd0 0x7f2ed01568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:05:50.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.287+0000 7f2ed531b700 1 --2- 172.21.15.157:0/1785246960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ed00c5c00 0x7f2ed006d300 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc00c800 tx=0x7f2ecc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:50.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.288+0000 7f2ec57fa700 1 -- 172.21.15.157:0/1785246960 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc011600 con 0x7f2ed00c5c00 2023-10-06T04:05:50.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.288+0000 7f2ec57fa700 1 -- 172.21.15.157:0/1785246960 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ecc011760 con 0x7f2ed00c5c00 2023-10-06T04:05:50.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.288+0000 7f2ed757f700 1 -- 172.21.15.157:0/1785246960 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed01555a0 con 0x7f2ed00c5c00 2023-10-06T04:05:50.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.288+0000 7f2ec57fa700 1 -- 172.21.15.157:0/1785246960 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc011b20 con 0x7f2ed00c5c00 2023-10-06T04:05:50.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.288+0000 7f2ed757f700 1 -- 172.21.15.157:0/1785246960 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed00c8640 con 0x7f2ed00c5c00 2023-10-06T04:05:50.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.289+0000 7f2ed757f700 1 -- 172.21.15.157:0/1785246960 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ed0151b30 con 0x7f2ed00c5c00 2023-10-06T04:05:50.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.289+0000 7f2ec57fa700 1 -- 172.21.15.157:0/1785246960 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2ecc0193f0 con 0x7f2ed00c5c00 2023-10-06T04:05:50.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.289+0000 7f2ec57fa700 1 --2- 172.21.15.157:0/1785246960 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2eb4066f30 0x7f2eb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:50.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.289+0000 7f2ec57fa700 1 -- 172.21.15.157:0/1785246960 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2ecc084c70 con 0x7f2ed00c5c00 2023-10-06T04:05:50.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.289+0000 7f2ed4b1a700 1 --2- 172.21.15.157:0/1785246960 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2eb4066f30 0x7f2eb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:50.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.290+0000 7f2ed4b1a700 1 --2- 172.21.15.157:0/1785246960 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2eb4066f30 0x7f2eb40693e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2ed006cd20 tx=0x7f2ec0016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:50.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.293+0000 7f2ec57fa700 1 -- 172.21.15.157:0/1785246960 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2ecc051bf0 con 0x7f2ed00c5c00 2023-10-06T04:05:50.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.524+0000 7f2ed757f700 1 -- 172.21.15.157:0/1785246960 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ed004bb40 con 0x7f2ed00c5c00 2023-10-06T04:05:50.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.524+0000 7f2ec57fa700 1 -- 172.21.15.157:0/1785246960 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+5386 (secure 0 0 0) 0x7f2ecc055210 con 0x7f2ed00c5c00 2023-10-06T04:05:50.526 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:05:50.526 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:05:44.778875+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:rejoin","state_seq":15,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:05:50.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.526+0000 7f2ed757f700 1 -- 172.21.15.157:0/1785246960 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2eb4066f30 msgr2=0x7f2eb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:50.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.526+0000 7f2ed757f700 1 --2- 172.21.15.157:0/1785246960 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2eb4066f30 0x7f2eb40693e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2ed006cd20 tx=0x7f2ec0016000 comp rx=0 tx=0).stop 2023-10-06T04:05:50.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.526+0000 7f2ed757f700 1 -- 172.21.15.157:0/1785246960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ed00c5c00 msgr2=0x7f2ed006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:50.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.526+0000 7f2ed757f700 1 --2- 172.21.15.157:0/1785246960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ed00c5c00 0x7f2ed006d300 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc00c800 tx=0x7f2ecc008040 comp rx=0 tx=0).stop 2023-10-06T04:05:50.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.527+0000 7f2ed757f700 1 -- 172.21.15.157:0/1785246960 shutdown_connections 2023-10-06T04:05:50.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.527+0000 7f2ed757f700 1 --2- 172.21.15.157:0/1785246960 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2eb4066f30 0x7f2eb40693e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:50.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.527+0000 7f2ed757f700 1 --2- 172.21.15.157:0/1785246960 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ed014e0a0 0x7f2ed0169ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:50.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.527+0000 7f2ed757f700 1 --2- 172.21.15.157:0/1785246960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ed00c5c00 0x7f2ed006d300 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:50.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.527+0000 7f2ed757f700 1 --2- 172.21.15.157:0/1785246960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ed014dcd0 0x7f2ed01568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:50.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.527+0000 7f2ed757f700 1 -- 172.21.15.157:0/1785246960 >> 172.21.15.157:0/1785246960 conn(0x7f2ed00692b0 msgr2=0x7f2ed014b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:50.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.527+0000 7f2ed757f700 1 -- 172.21.15.157:0/1785246960 shutdown_connections 2023-10-06T04:05:50.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:50.527+0000 7f2ed757f700 1 -- 172.21.15.157:0/1785246960 wait complete. 2023-10-06T04:05:50.530 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 67 2023-10-06T04:05:50.541 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:05:52.541 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:05:52.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.668+0000 7f6ce9782700 1 Processor -- start 2023-10-06T04:05:52.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.668+0000 7f6ce9782700 1 -- start start 2023-10-06T04:05:52.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.669+0000 7f6ce9782700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6ce40b1d40 0x7f6ce4159df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:52.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.669+0000 7f6ce9782700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ce40b2110 0x7f6ce40af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:52.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.669+0000 7f6ce9782700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6ce406c1f0 0x7f6ce414fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:52.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.669+0000 7f6ce9782700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ce406ced0 con 0x7f6ce40b1d40 2023-10-06T04:05:52.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.669+0000 7f6ce9782700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ce406cc10 con 0x7f6ce406c1f0 2023-10-06T04:05:52.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.669+0000 7f6ce9782700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce406e570 con 0x7f6ce40b2110 2023-10-06T04:05:52.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.669+0000 7f6ce2ffd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6ce40b1d40 0x7f6ce4159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:52.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.669+0000 7f6ce2ffd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6ce40b1d40 0x7f6ce4159df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40046/0 (socket says 172.21.15.157:40046) 2023-10-06T04:05:52.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.669+0000 7f6ce2ffd700 1 -- 172.21.15.157:0/2330231053 learned_addr learned my addr 172.21.15.157:0/2330231053 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:52.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.669+0000 7f6ce37fe700 1 --2- 172.21.15.157:0/2330231053 >> v2:172.21.15.175:3300/0 conn(0x7f6ce406c1f0 0x7f6ce414fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:52.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.669+0000 7f6ce27fc700 1 --2- 172.21.15.157:0/2330231053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ce40b2110 0x7f6ce40af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:52.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.669+0000 7f6ce2ffd700 1 -- 172.21.15.157:0/2330231053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ce40b2110 msgr2=0x7f6ce40af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:52.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.669+0000 7f6ce2ffd700 1 --2- 172.21.15.157:0/2330231053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ce40b2110 0x7f6ce40af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:52.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.669+0000 7f6ce2ffd700 1 -- 172.21.15.157:0/2330231053 >> v2:172.21.15.175:3300/0 conn(0x7f6ce406c1f0 msgr2=0x7f6ce414fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:52.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.669+0000 7f6ce2ffd700 1 --2- 172.21.15.157:0/2330231053 >> v2:172.21.15.175:3300/0 conn(0x7f6ce406c1f0 0x7f6ce414fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:52.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.669+0000 7f6ce2ffd700 1 -- 172.21.15.157:0/2330231053 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce406d7b0 con 0x7f6ce40b1d40 2023-10-06T04:05:52.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.669+0000 7f6ce37fe700 1 --2- 172.21.15.157:0/2330231053 >> v2:172.21.15.175:3300/0 conn(0x7f6ce406c1f0 0x7f6ce414fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:05:52.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.669+0000 7f6ce2ffd700 1 --2- 172.21.15.157:0/2330231053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ce40b1d40 0x7f6ce4159df0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40b2700 tx=0x7f6cdc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3a9fc506b6ae77 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:52.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.669+0000 7f6ce1ffb700 1 -- 172.21.15.157:0/2330231053 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc00b8f0 con 0x7f6ce40b1d40 2023-10-06T04:05:52.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.670+0000 7f6ce1ffb700 1 -- 172.21.15.157:0/2330231053 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cdc00ba50 con 0x7f6ce40b1d40 2023-10-06T04:05:52.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.670+0000 7f6ce1ffb700 1 -- 172.21.15.157:0/2330231053 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc00bdf0 con 0x7f6ce40b1d40 2023-10-06T04:05:52.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.670+0000 7f6ce9782700 1 -- 172.21.15.157:0/2330231053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ce40b1d40 msgr2=0x7f6ce4159df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:52.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.670+0000 7f6ce9782700 1 --2- 172.21.15.157:0/2330231053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ce40b1d40 0x7f6ce4159df0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40b2700 tx=0x7f6cdc009250 comp rx=0 tx=0).stop 2023-10-06T04:05:52.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.670+0000 7f6ce9782700 1 -- 172.21.15.157:0/2330231053 shutdown_connections 2023-10-06T04:05:52.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.670+0000 7f6ce9782700 1 --2- 172.21.15.157:0/2330231053 >> v2:172.21.15.175:3300/0 conn(0x7f6ce406c1f0 0x7f6ce414fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:52.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.670+0000 7f6ce9782700 1 --2- 172.21.15.157:0/2330231053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ce40b1d40 0x7f6ce4159df0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:52.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.670+0000 7f6ce9782700 1 --2- 172.21.15.157:0/2330231053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ce40b2110 0x7f6ce40af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:52.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.670+0000 7f6ce9782700 1 -- 172.21.15.157:0/2330231053 >> 172.21.15.157:0/2330231053 conn(0x7f6ce40cc6e0 msgr2=0x7f6ce40ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:52.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.670+0000 7f6ce9782700 1 -- 172.21.15.157:0/2330231053 shutdown_connections 2023-10-06T04:05:52.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.670+0000 7f6ce9782700 1 -- 172.21.15.157:0/2330231053 wait complete. 2023-10-06T04:05:52.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.671+0000 7f6ce9782700 1 Processor -- start 2023-10-06T04:05:52.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.671+0000 7f6ce9782700 1 -- start start 2023-10-06T04:05:52.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.672+0000 7f6ce9782700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ce406c1f0 0x7f6ce4166d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:52.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.672+0000 7f6ce9782700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ce40b1d40 0x7f6ce416a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:52.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.672+0000 7f6ce9782700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ce40b2110 0x7f6ce4169790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:52.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.672+0000 7f6ce9782700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce4166900 con 0x7f6ce406c1f0 2023-10-06T04:05:52.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.672+0000 7f6ce9782700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce40c6790 con 0x7f6ce40b2110 2023-10-06T04:05:52.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.672+0000 7f6ce9782700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce40c4fa0 con 0x7f6ce40b1d40 2023-10-06T04:05:52.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.672+0000 7f6ce2ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ce406c1f0 0x7f6ce4166d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:52.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.672+0000 7f6ce2ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ce406c1f0 0x7f6ce4166d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40054/0 (socket says 172.21.15.157:40054) 2023-10-06T04:05:52.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.672+0000 7f6ce2ffd700 1 -- 172.21.15.157:0/2270544382 learned_addr learned my addr 172.21.15.157:0/2270544382 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:52.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.672+0000 7f6ce27fc700 1 --2- 172.21.15.157:0/2270544382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ce40b1d40 0x7f6ce416a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:52.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.672+0000 7f6ce37fe700 1 --2- 172.21.15.157:0/2270544382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ce40b2110 0x7f6ce4169790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:52.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.673+0000 7f6ce2ffd700 1 -- 172.21.15.157:0/2270544382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ce40b1d40 msgr2=0x7f6ce416a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:52.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.673+0000 7f6ce2ffd700 1 --2- 172.21.15.157:0/2270544382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ce40b1d40 0x7f6ce416a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:52.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.673+0000 7f6ce2ffd700 1 -- 172.21.15.157:0/2270544382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ce40b2110 msgr2=0x7f6ce4169790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:52.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.673+0000 7f6ce2ffd700 1 --2- 172.21.15.157:0/2270544382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ce40b2110 0x7f6ce4169790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:52.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.673+0000 7f6ce2ffd700 1 -- 172.21.15.157:0/2270544382 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cdc0094f0 con 0x7f6ce406c1f0 2023-10-06T04:05:52.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.673+0000 7f6ce2ffd700 1 --2- 172.21.15.157:0/2270544382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ce406c1f0 0x7f6ce4166d20 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009290 tx=0x7f6cdc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:52.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.673+0000 7f6cdaffd700 1 -- 172.21.15.157:0/2270544382 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc026480 con 0x7f6ce406c1f0 2023-10-06T04:05:52.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.673+0000 7f6cdaffd700 1 -- 172.21.15.157:0/2270544382 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cdc01f070 con 0x7f6ce406c1f0 2023-10-06T04:05:52.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.673+0000 7f6cdaffd700 1 -- 172.21.15.157:0/2270544382 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc026830 con 0x7f6ce406c1f0 2023-10-06T04:05:52.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.673+0000 7f6ce9782700 1 -- 172.21.15.157:0/2270544382 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce40c5280 con 0x7f6ce406c1f0 2023-10-06T04:05:52.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.673+0000 7f6ce9782700 1 -- 172.21.15.157:0/2270544382 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce4166520 con 0x7f6ce406c1f0 2023-10-06T04:05:52.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.674+0000 7f6ce9782700 1 -- 172.21.15.157:0/2270544382 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce40896c0 con 0x7f6ce406c1f0 2023-10-06T04:05:52.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.674+0000 7f6cdaffd700 1 -- 172.21.15.157:0/2270544382 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6cdc026ba0 con 0x7f6ce406c1f0 2023-10-06T04:05:52.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.675+0000 7f6cdaffd700 1 --2- 172.21.15.157:0/2270544382 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6cc4066f60 0x7f6cc4069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:52.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.675+0000 7f6cdaffd700 1 -- 172.21.15.157:0/2270544382 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6cdc021070 con 0x7f6ce406c1f0 2023-10-06T04:05:52.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.675+0000 7f6ce27fc700 1 --2- 172.21.15.157:0/2270544382 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6cc4066f60 0x7f6cc4069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:52.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.675+0000 7f6ce27fc700 1 --2- 172.21.15.157:0/2270544382 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6cc4066f60 0x7f6cc4069410 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4157110 tx=0x7f6ccc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:52.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.679+0000 7f6cdaffd700 1 -- 172.21.15.157:0/2270544382 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6cdc053070 con 0x7f6ce406c1f0 2023-10-06T04:05:52.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.905+0000 7f6ce9782700 1 -- 172.21.15.157:0/2270544382 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ce406da90 con 0x7f6ce406c1f0 2023-10-06T04:05:52.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.906+0000 7f6cdaffd700 1 -- 172.21.15.157:0/2270544382 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+5386 (secure 0 0 0) 0x7f6cdc056690 con 0x7f6ce406c1f0 2023-10-06T04:05:52.907 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:05:52.908 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:05:50.793233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:05:52.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.908+0000 7f6ce9782700 1 -- 172.21.15.157:0/2270544382 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6cc4066f60 msgr2=0x7f6cc4069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:52.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.908+0000 7f6ce9782700 1 --2- 172.21.15.157:0/2270544382 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6cc4066f60 0x7f6cc4069410 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4157110 tx=0x7f6ccc008040 comp rx=0 tx=0).stop 2023-10-06T04:05:52.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.908+0000 7f6ce9782700 1 -- 172.21.15.157:0/2270544382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ce406c1f0 msgr2=0x7f6ce4166d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:52.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.908+0000 7f6ce9782700 1 --2- 172.21.15.157:0/2270544382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ce406c1f0 0x7f6ce4166d20 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009290 tx=0x7f6cdc017000 comp rx=0 tx=0).stop 2023-10-06T04:05:52.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.908+0000 7f6ce9782700 1 -- 172.21.15.157:0/2270544382 shutdown_connections 2023-10-06T04:05:52.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.908+0000 7f6ce9782700 1 --2- 172.21.15.157:0/2270544382 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6cc4066f60 0x7f6cc4069410 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:52.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.908+0000 7f6ce9782700 1 --2- 172.21.15.157:0/2270544382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ce40b2110 0x7f6ce4169790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:52.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.908+0000 7f6ce9782700 1 --2- 172.21.15.157:0/2270544382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ce406c1f0 0x7f6ce4166d20 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:52.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.908+0000 7f6ce9782700 1 --2- 172.21.15.157:0/2270544382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ce40b1d40 0x7f6ce416a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:52.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.908+0000 7f6ce9782700 1 -- 172.21.15.157:0/2270544382 >> 172.21.15.157:0/2270544382 conn(0x7f6ce40cc6e0 msgr2=0x7f6ce41593f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:52.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.908+0000 7f6ce9782700 1 -- 172.21.15.157:0/2270544382 shutdown_connections 2023-10-06T04:05:52.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:52.908+0000 7f6ce9782700 1 -- 172.21.15.157:0/2270544382 wait complete. 2023-10-06T04:05:52.912 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 68 2023-10-06T04:05:52.922 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c has gained rank=0, replacing gid=4296 2023-10-06T04:05:52.926 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 1 secs before reviving mds.e 2023-10-06T04:05:53.926 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.e 2023-10-06T04:05:53.926 INFO:tasks.ceph.mds.e:Restarting daemon 2023-10-06T04:05:53.927 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2023-10-06T04:05:53.929 INFO:tasks.ceph.mds.e:Started 2023-10-06T04:05:53.929 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:05:54.012 INFO:tasks.ceph.mds.e.smithi157.stdout:starting mds.e at 2023-10-06T04:05:54.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.059+0000 7f4bee7c7700 1 Processor -- start 2023-10-06T04:05:54.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.059+0000 7f4bee7c7700 1 -- start start 2023-10-06T04:05:54.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.060+0000 7f4bee7c7700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4be80c94c0 0x7f4be806a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:54.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.060+0000 7f4bee7c7700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4be80c9890 0x7f4be80c5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:54.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.060+0000 7f4bee7c7700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be8156a80 con 0x7f4be806c1f0 2023-10-06T04:05:54.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.060+0000 7f4bee7c7700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be8156190 con 0x7f4be80c9890 2023-10-06T04:05:54.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.060+0000 7f4bee7c7700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be806ac50 con 0x7f4be80c94c0 2023-10-06T04:05:54.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.060+0000 7f4becfc4700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4be80c9890 0x7f4be80c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:54.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.060+0000 7f4bed7c5700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4be80c94c0 0x7f4be806a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:54.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.060+0000 7f4becfc4700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4be80c9890 0x7f4be80c5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:40720/0 (socket says 172.21.15.157:40720) 2023-10-06T04:05:54.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.060+0000 7f4bedfc6700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f4be806c1f0 0x7f4be814fdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.157:59092/0 (socket says 172.21.15.157:59092) 2023-10-06T04:05:54.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.060+0000 7f4becfc4700 1 -- 172.21.15.157:0/2402372632 learned_addr learned my addr 172.21.15.157:0/2402372632 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:54.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.061+0000 7f4becfc4700 1 -- 172.21.15.157:0/2402372632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4be80c94c0 msgr2=0x7f4be806a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:54.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.061+0000 7f4becfc4700 1 --2- 172.21.15.157:0/2402372632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4be80c94c0 0x7f4be806a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:54.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.061+0000 7f4becfc4700 1 -- 172.21.15.157:0/2402372632 >> v1:172.21.15.175:6789/0 conn(0x7f4be806c1f0 legacy=0x7f4be814fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:54.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.061+0000 7f4becfc4700 1 -- 172.21.15.157:0/2402372632 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be8156430 con 0x7f4be80c9890 2023-10-06T04:05:54.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.061+0000 7f4bdffff700 1 -- 172.21.15.157:0/2402372632 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3060271085 0 0) 0x7f4be8156a80 con 0x7f4be806c1f0 2023-10-06T04:05:54.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.061+0000 7f4becfc4700 1 --2- 172.21.15.157:0/2402372632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4be80c9890 0x7f4be80c5180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4bd80099b0 tx=0x7f4bd8009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24c60013f34874c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:54.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.061+0000 7f4bdffff700 1 -- 172.21.15.157:0/2402372632 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8009ea0 con 0x7f4be80c9890 2023-10-06T04:05:54.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.061+0000 7f4bdffff700 1 -- 172.21.15.157:0/2402372632 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd800b8f0 con 0x7f4be80c9890 2023-10-06T04:05:54.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.061+0000 7f4bdffff700 1 -- 172.21.15.157:0/2402372632 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd800bc90 con 0x7f4be80c9890 2023-10-06T04:05:54.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.061+0000 7f4bee7c7700 1 -- 172.21.15.157:0/2402372632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4be80c9890 msgr2=0x7f4be80c5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:54.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.061+0000 7f4bee7c7700 1 --2- 172.21.15.157:0/2402372632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4be80c9890 0x7f4be80c5180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4bd80099b0 tx=0x7f4bd8009250 comp rx=0 tx=0).stop 2023-10-06T04:05:54.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.061+0000 7f4bee7c7700 1 -- 172.21.15.157:0/2402372632 shutdown_connections 2023-10-06T04:05:54.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.061+0000 7f4bee7c7700 1 --2- 172.21.15.157:0/2402372632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4be80c94c0 0x7f4be806a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:54.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.061+0000 7f4bee7c7700 1 --2- 172.21.15.157:0/2402372632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4be80c9890 0x7f4be80c5180 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:54.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.062+0000 7f4bee7c7700 1 -- 172.21.15.157:0/2402372632 >> 172.21.15.157:0/2402372632 conn(0x7f4be8068020 msgr2=0x7f4be8068420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:54.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.062+0000 7f4bee7c7700 1 -- 172.21.15.157:0/2402372632 shutdown_connections 2023-10-06T04:05:54.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.062+0000 7f4bee7c7700 1 -- 172.21.15.157:0/2402372632 wait complete. 2023-10-06T04:05:54.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.063+0000 7f4bee7c7700 1 Processor -- start 2023-10-06T04:05:54.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.063+0000 7f4bee7c7700 1 -- start start 2023-10-06T04:05:54.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.064+0000 7f4bee7c7700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4be806c1f0 0x7f4be8169cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:54.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.064+0000 7f4bee7c7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4be80c94c0 0x7f4be8165c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:54.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.064+0000 7f4bee7c7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4be80c9890 0x7f4be8153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:54.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.064+0000 7f4bee7c7700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be81661a0 con 0x7f4be80c94c0 2023-10-06T04:05:54.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.064+0000 7f4bee7c7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be814f3c0 con 0x7f4be80c9890 2023-10-06T04:05:54.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.064+0000 7f4bee7c7700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be814f680 con 0x7f4be806c1f0 2023-10-06T04:05:54.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.064+0000 7f4becfc4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4be80c94c0 0x7f4be8165c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:54.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.064+0000 7f4becfc4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4be80c94c0 0x7f4be8165c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38966/0 (socket says 172.21.15.157:38966) 2023-10-06T04:05:54.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.064+0000 7f4becfc4700 1 -- 172.21.15.157:0/2268634983 learned_addr learned my addr 172.21.15.157:0/2268634983 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:54.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.064+0000 7f4bedfc6700 1 --2- 172.21.15.157:0/2268634983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4be80c9890 0x7f4be8153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:54.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.064+0000 7f4bed7c5700 1 --2- 172.21.15.157:0/2268634983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4be806c1f0 0x7f4be8169cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:54.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.064+0000 7f4becfc4700 1 -- 172.21.15.157:0/2268634983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4be806c1f0 msgr2=0x7f4be8169cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:54.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.064+0000 7f4becfc4700 1 --2- 172.21.15.157:0/2268634983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4be806c1f0 0x7f4be8169cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:54.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.064+0000 7f4becfc4700 1 -- 172.21.15.157:0/2268634983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4be80c9890 msgr2=0x7f4be8153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:54.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.064+0000 7f4becfc4700 1 --2- 172.21.15.157:0/2268634983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4be80c9890 0x7f4be8153180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:54.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.064+0000 7f4becfc4700 1 -- 172.21.15.157:0/2268634983 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd80094f0 con 0x7f4be80c94c0 2023-10-06T04:05:54.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.064+0000 7f4becfc4700 1 --2- 172.21.15.157:0/2268634983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4be80c94c0 0x7f4be8165c60 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4be8066dc0 tx=0x7f4bd8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:54.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.064+0000 7f4bdd7fa700 1 -- 172.21.15.157:0/2268634983 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8000d10 con 0x7f4be80c94c0 2023-10-06T04:05:54.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.064+0000 7f4bdd7fa700 1 -- 172.21.15.157:0/2268634983 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd800bc70 con 0x7f4be80c94c0 2023-10-06T04:05:54.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.064+0000 7f4bdd7fa700 1 -- 172.21.15.157:0/2268634983 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd801a6c0 con 0x7f4be80c94c0 2023-10-06T04:05:54.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.064+0000 7f4bee7c7700 1 -- 172.21.15.157:0/2268634983 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be81572f0 con 0x7f4be80c94c0 2023-10-06T04:05:54.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.064+0000 7f4bee7c7700 1 -- 172.21.15.157:0/2268634983 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be81694c0 con 0x7f4be80c94c0 2023-10-06T04:05:54.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.065+0000 7f4bee7c7700 1 -- 172.21.15.157:0/2268634983 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be8156710 con 0x7f4be80c94c0 2023-10-06T04:05:54.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.066+0000 7f4bdd7fa700 1 -- 172.21.15.157:0/2268634983 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4bd801aa70 con 0x7f4be80c94c0 2023-10-06T04:05:54.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.066+0000 7f4bdd7fa700 1 --2- 172.21.15.157:0/2268634983 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4bcc066f60 0x7f4bcc069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:54.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.066+0000 7f4bdd7fa700 1 -- 172.21.15.157:0/2268634983 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4bd8086190 con 0x7f4be80c94c0 2023-10-06T04:05:54.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.066+0000 7f4bed7c5700 1 --2- 172.21.15.157:0/2268634983 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4bcc066f60 0x7f4bcc069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:54.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.067+0000 7f4bed7c5700 1 --2- 172.21.15.157:0/2268634983 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4bcc066f60 0x7f4bcc069410 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4be000f570 tx=0x7f4be000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:54.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.070+0000 7f4bdd7fa700 1 -- 172.21.15.157:0/2268634983 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4bd8053110 con 0x7f4be80c94c0 2023-10-06T04:05:54.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.296+0000 7f4bee7c7700 1 -- 172.21.15.157:0/2268634983 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4be804bb40 con 0x7f4be80c94c0 2023-10-06T04:05:54.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.296+0000 7f4bdd7fa700 1 -- 172.21.15.157:0/2268634983 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+5386 (secure 0 0 0) 0x7f4bd8056730 con 0x7f4be80c94c0 2023-10-06T04:05:54.298 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:05:54.298 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:05:50.793233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:05:54.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.298+0000 7f4bee7c7700 1 -- 172.21.15.157:0/2268634983 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4bcc066f60 msgr2=0x7f4bcc069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:54.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.298+0000 7f4bee7c7700 1 --2- 172.21.15.157:0/2268634983 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4bcc066f60 0x7f4bcc069410 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4be000f570 tx=0x7f4be000b040 comp rx=0 tx=0).stop 2023-10-06T04:05:54.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.298+0000 7f4bee7c7700 1 -- 172.21.15.157:0/2268634983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4be80c94c0 msgr2=0x7f4be8165c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:54.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.298+0000 7f4bee7c7700 1 --2- 172.21.15.157:0/2268634983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4be80c94c0 0x7f4be8165c60 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4be8066dc0 tx=0x7f4bd8017000 comp rx=0 tx=0).stop 2023-10-06T04:05:54.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.298+0000 7f4bee7c7700 1 -- 172.21.15.157:0/2268634983 shutdown_connections 2023-10-06T04:05:54.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.298+0000 7f4bee7c7700 1 --2- 172.21.15.157:0/2268634983 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4bcc066f60 0x7f4bcc069410 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:54.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.298+0000 7f4bee7c7700 1 --2- 172.21.15.157:0/2268634983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4be80c9890 0x7f4be8153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:54.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.298+0000 7f4bee7c7700 1 --2- 172.21.15.157:0/2268634983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4be806c1f0 0x7f4be8169cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:54.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.298+0000 7f4bee7c7700 1 --2- 172.21.15.157:0/2268634983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4be80c94c0 0x7f4be8165c60 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:54.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.298+0000 7f4bee7c7700 1 -- 172.21.15.157:0/2268634983 >> 172.21.15.157:0/2268634983 conn(0x7f4be8068020 msgr2=0x7f4be80beb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:54.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.298+0000 7f4bee7c7700 1 -- 172.21.15.157:0/2268634983 shutdown_connections 2023-10-06T04:05:54.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:54.298+0000 7f4bee7c7700 1 -- 172.21.15.157:0/2268634983 wait complete. 2023-10-06T04:05:54.301 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 68 2023-10-06T04:05:54.312 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is in active, standby or standby-replay 2023-10-06T04:05:56.313 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:05:56.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.426+0000 7efdeebc2700 1 Processor -- start 2023-10-06T04:05:56.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.426+0000 7efdeebc2700 1 -- start start 2023-10-06T04:05:56.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.427+0000 7efdeebc2700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7efde81547f0 0x7efde80c29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:56.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.427+0000 7efdeebc2700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efde8153be0 con 0x7efde80631f0 2023-10-06T04:05:56.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.427+0000 7efdeebc2700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efde80c3630 con 0x7efde80c00c0 2023-10-06T04:05:56.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.427+0000 7efdeebc2700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efde80bf550 con 0x7efde81547f0 2023-10-06T04:05:56.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.427+0000 7efded15f700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7efde80631f0 0x7efde814d650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43546/0 (socket says 172.21.15.157:43546) 2023-10-06T04:05:56.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.427+0000 7efded15f700 1 -- 172.21.15.157:0/3722765278 learned_addr learned my addr 172.21.15.157:0/3722765278 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:05:56.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.428+0000 7efddffff700 1 --2- 172.21.15.157:0/3722765278 >> v2:172.21.15.175:3300/0 conn(0x7efde81547f0 0x7efde80c29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:56.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.428+0000 7efddf7fe700 1 -- 172.21.15.157:0/3722765278 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1780232156 0 0) 0x7efde8153be0 con 0x7efde80631f0 2023-10-06T04:05:56.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.428+0000 7efddf7fe700 1 -- 172.21.15.157:0/3722765278 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdd0003570 con 0x7efde80631f0 2023-10-06T04:05:56.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.428+0000 7efddf7fe700 1 -- 172.21.15.157:0/3722765278 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2368515114 0 0) 0x7efdd0003570 con 0x7efde80631f0 2023-10-06T04:05:56.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.428+0000 7efddf7fe700 1 -- 172.21.15.157:0/3722765278 >> v2:172.21.15.175:3300/0 conn(0x7efde81547f0 msgr2=0x7efde80c29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:56.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.428+0000 7efddf7fe700 1 --2- 172.21.15.157:0/3722765278 >> v2:172.21.15.175:3300/0 conn(0x7efde81547f0 0x7efde80c29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:56.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.428+0000 7efddf7fe700 1 -- 172.21.15.157:0/3722765278 >> v1:172.21.15.175:6789/0 conn(0x7efde80c00c0 legacy=0x7efde81540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:56.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.428+0000 7efddf7fe700 1 -- 172.21.15.157:0/3722765278 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde80c3910 con 0x7efde80631f0 2023-10-06T04:05:56.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.428+0000 7efddf7fe700 1 -- 172.21.15.157:0/3722765278 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7efde00017f0 con 0x7efde80631f0 2023-10-06T04:05:56.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.428+0000 7efddf7fe700 1 -- 172.21.15.157:0/3722765278 >> v1:172.21.15.157:6789/0 conn(0x7efde80631f0 legacy=0x7efde814d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:56.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.428+0000 7efddf7fe700 1 --2- 172.21.15.157:0/3722765278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdd0003a60 0x7efdd0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:56.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.428+0000 7efddf7fe700 1 --2- 172.21.15.157:0/3722765278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efdd0004810 0x7efdd0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:56.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.428+0000 7efddf7fe700 1 --2- 172.21.15.157:0/3722765278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efdd00051c0 0x7efdd000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:56.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.428+0000 7efddf7fe700 1 -- 172.21.15.157:0/3722765278 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde80bf550 con 0x7efdd0003a60 2023-10-06T04:05:56.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.428+0000 7efddf7fe700 1 -- 172.21.15.157:0/3722765278 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd000c0d0 con 0x7efdd0004810 2023-10-06T04:05:56.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.428+0000 7efddf7fe700 1 -- 172.21.15.157:0/3722765278 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd000c290 con 0x7efdd00051c0 2023-10-06T04:05:56.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.428+0000 7efdec95e700 1 --2- 172.21.15.157:0/3722765278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdd0003a60 0x7efdd0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:56.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.428+0000 7efddffff700 1 --2- 172.21.15.157:0/3722765278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efdd0004810 0x7efdd0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:56.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.428+0000 7efded15f700 1 --2- 172.21.15.157:0/3722765278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efdd00051c0 0x7efdd000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:56.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.428+0000 7efdec95e700 1 -- 172.21.15.157:0/3722765278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efdd00051c0 msgr2=0x7efdd000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:56.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.428+0000 7efdec95e700 1 --2- 172.21.15.157:0/3722765278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efdd00051c0 0x7efdd000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:56.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.428+0000 7efdec95e700 1 -- 172.21.15.157:0/3722765278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efdd0004810 msgr2=0x7efdd0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:05:56.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.428+0000 7efdec95e700 1 --2- 172.21.15.157:0/3722765278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efdd0004810 0x7efdd0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:56.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.428+0000 7efded15f700 1 -- 172.21.15.157:0/3722765278 reap_dead start 2023-10-06T04:05:56.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.428+0000 7efdec95e700 1 -- 172.21.15.157:0/3722765278 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd0003570 con 0x7efdd0003a60 2023-10-06T04:05:56.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.429+0000 7efdec95e700 1 --2- 172.21.15.157:0/3722765278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdd0003a60 0x7efdd0003e70 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7efde40018b0 tx=0x7efde4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa2b54575cb1e389 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:56.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.429+0000 7efddf7fe700 1 -- 172.21.15.157:0/3722765278 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde4012420 con 0x7efdd0003a60 2023-10-06T04:05:56.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.429+0000 7efddf7fe700 1 -- 172.21.15.157:0/3722765278 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde40125a0 con 0x7efdd0003a60 2023-10-06T04:05:56.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.429+0000 7efddf7fe700 1 -- 172.21.15.157:0/3722765278 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde4012960 con 0x7efdd0003a60 2023-10-06T04:05:56.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.429+0000 7efdeebc2700 1 -- 172.21.15.157:0/3722765278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdd0003a60 msgr2=0x7efdd0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:56.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.429+0000 7efdeebc2700 1 --2- 172.21.15.157:0/3722765278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdd0003a60 0x7efdd0003e70 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7efde40018b0 tx=0x7efde4007d30 comp rx=0 tx=0).stop 2023-10-06T04:05:56.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.429+0000 7efdeebc2700 1 -- 172.21.15.157:0/3722765278 shutdown_connections 2023-10-06T04:05:56.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.429+0000 7efdeebc2700 1 --2- 172.21.15.157:0/3722765278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdd0003a60 0x7efdd0003e70 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:56.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.429+0000 7efdeebc2700 1 -- 172.21.15.157:0/3722765278 >> 172.21.15.157:0/3722765278 conn(0x7efde80c8b20 msgr2=0x7efde80c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:56.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.429+0000 7efdeebc2700 1 -- 172.21.15.157:0/3722765278 shutdown_connections 2023-10-06T04:05:56.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.429+0000 7efdeebc2700 1 -- 172.21.15.157:0/3722765278 wait complete. 2023-10-06T04:05:56.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.430+0000 7efdeebc2700 1 Processor -- start 2023-10-06T04:05:56.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.431+0000 7efdeebc2700 1 -- start start 2023-10-06T04:05:56.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.431+0000 7efdeebc2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdd0003a60 0x7efde8149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:56.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.431+0000 7efdeebc2700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efde81434c0 0x7efde8163da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:56.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.431+0000 7efdeebc2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efde8166910 0x7efde8152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:56.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.431+0000 7efdeebc2700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde8163390 con 0x7efdd0003a60 2023-10-06T04:05:56.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.431+0000 7efdeebc2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde81642e0 con 0x7efde8166910 2023-10-06T04:05:56.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.431+0000 7efdeebc2700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde8163910 con 0x7efde81434c0 2023-10-06T04:05:56.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.431+0000 7efdec95e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdd0003a60 0x7efde8149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:56.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.431+0000 7efdec95e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdd0003a60 0x7efde8149920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38978/0 (socket says 172.21.15.157:38978) 2023-10-06T04:05:56.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.431+0000 7efdec95e700 1 -- 172.21.15.157:0/2828775313 learned_addr learned my addr 172.21.15.157:0/2828775313 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:05:56.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.432+0000 7efddffff700 1 --2- 172.21.15.157:0/2828775313 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efde81434c0 0x7efde8163da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:56.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.432+0000 7efded15f700 1 --2- 172.21.15.157:0/2828775313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efde8166910 0x7efde8152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:56.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.432+0000 7efdec95e700 1 -- 172.21.15.157:0/2828775313 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efde81434c0 msgr2=0x7efde8163da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:56.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.432+0000 7efdec95e700 1 --2- 172.21.15.157:0/2828775313 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efde81434c0 0x7efde8163da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:56.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.432+0000 7efdec95e700 1 -- 172.21.15.157:0/2828775313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efde8166910 msgr2=0x7efde8152360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:56.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.432+0000 7efdec95e700 1 --2- 172.21.15.157:0/2828775313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efde8166910 0x7efde8152360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:56.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.432+0000 7efdec95e700 1 -- 172.21.15.157:0/2828775313 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde8149ff0 con 0x7efdd0003a60 2023-10-06T04:05:56.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.432+0000 7efdec95e700 1 --2- 172.21.15.157:0/2828775313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdd0003a60 0x7efde8149920 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7efde4012ad0 tx=0x7efde4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:56.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.432+0000 7efddcff9700 1 -- 172.21.15.157:0/2828775313 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde4012420 con 0x7efdd0003a60 2023-10-06T04:05:56.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.432+0000 7efddcff9700 1 -- 172.21.15.157:0/2828775313 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde4001200 con 0x7efdd0003a60 2023-10-06T04:05:56.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.432+0000 7efdeebc2700 1 -- 172.21.15.157:0/2828775313 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efde814a2d0 con 0x7efdd0003a60 2023-10-06T04:05:56.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.432+0000 7efddcff9700 1 -- 172.21.15.157:0/2828775313 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde400ce60 con 0x7efdd0003a60 2023-10-06T04:05:56.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.432+0000 7efdeebc2700 1 -- 172.21.15.157:0/2828775313 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde81635a0 con 0x7efdd0003a60 2023-10-06T04:05:56.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.433+0000 7efdeebc2700 1 -- 172.21.15.157:0/2828775313 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efde8163bf0 con 0x7efdd0003a60 2023-10-06T04:05:56.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.433+0000 7efddcff9700 1 -- 172.21.15.157:0/2828775313 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efde401f620 con 0x7efdd0003a60 2023-10-06T04:05:56.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.434+0000 7efddcff9700 1 --2- 172.21.15.157:0/2828775313 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7efdd4066f30 0x7efdd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:05:56.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.434+0000 7efddcff9700 1 -- 172.21.15.157:0/2828775313 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efde4086320 con 0x7efdd0003a60 2023-10-06T04:05:56.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.434+0000 7efddffff700 1 --2- 172.21.15.157:0/2828775313 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7efdd4066f30 0x7efdd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:05:56.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.434+0000 7efddffff700 1 --2- 172.21.15.157:0/2828775313 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7efdd4066f30 0x7efdd40693e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efde8143890 tx=0x7efdd8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:05:56.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.438+0000 7efddcff9700 1 -- 172.21.15.157:0/2828775313 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efde40532a0 con 0x7efdd0003a60 2023-10-06T04:05:56.560 INFO:tasks.workunit.client.0.smithi157.stderr:error: RPC failed; curl 18 transfer closed with outstanding read data remaining 2023-10-06T04:05:56.560 INFO:tasks.workunit.client.0.smithi157.stderr:fatal: the remote end hung up unexpectedly 2023-10-06T04:05:56.562 INFO:tasks.workunit.client.0.smithi157.stderr:fatal: early EOF 2023-10-06T04:05:56.563 INFO:tasks.workunit.client.0.smithi157.stderr:fatal: index-pack failed 2023-10-06T04:05:56.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.684+0000 7efdeebc2700 1 -- 172.21.15.157:0/2828775313 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efde814a7d0 con 0x7efdd0003a60 2023-10-06T04:05:56.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.684+0000 7efddcff9700 1 -- 172.21.15.157:0/2828775313 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+6146 (secure 0 0 0) 0x7efde40568c0 con 0x7efdd0003a60 2023-10-06T04:05:56.686 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:05:56.686 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":69,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5698,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:05:50.793233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:05:56.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.686+0000 7efdeebc2700 1 -- 172.21.15.157:0/2828775313 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7efdd4066f30 msgr2=0x7efdd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:56.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.686+0000 7efdeebc2700 1 --2- 172.21.15.157:0/2828775313 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7efdd4066f30 0x7efdd40693e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efde8143890 tx=0x7efdd8008040 comp rx=0 tx=0).stop 2023-10-06T04:05:56.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.686+0000 7efdeebc2700 1 -- 172.21.15.157:0/2828775313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdd0003a60 msgr2=0x7efde8149920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:05:56.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.686+0000 7efdeebc2700 1 --2- 172.21.15.157:0/2828775313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdd0003a60 0x7efde8149920 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7efde4012ad0 tx=0x7efde4015000 comp rx=0 tx=0).stop 2023-10-06T04:05:56.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.686+0000 7efdeebc2700 1 -- 172.21.15.157:0/2828775313 shutdown_connections 2023-10-06T04:05:56.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.686+0000 7efdeebc2700 1 --2- 172.21.15.157:0/2828775313 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7efdd4066f30 0x7efdd40693e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:56.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.686+0000 7efdeebc2700 1 --2- 172.21.15.157:0/2828775313 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efde8166910 0x7efde8152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:56.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.686+0000 7efdeebc2700 1 --2- 172.21.15.157:0/2828775313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdd0003a60 0x7efde8149920 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:56.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.686+0000 7efdeebc2700 1 --2- 172.21.15.157:0/2828775313 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efde81434c0 0x7efde8163da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:05:56.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.687+0000 7efdeebc2700 1 -- 172.21.15.157:0/2828775313 >> 172.21.15.157:0/2828775313 conn(0x7efde80c8b20 msgr2=0x7efde814bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:05:56.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.687+0000 7efdeebc2700 1 -- 172.21.15.157:0/2828775313 shutdown_connections 2023-10-06T04:05:56.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:05:56.687+0000 7efdeebc2700 1 -- 172.21.15.157:0/2828775313 wait complete. 2023-10-06T04:05:56.690 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 69 2023-10-06T04:05:56.700 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e reported in up:standby state 2023-10-06T04:05:56.701 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 78 secs before thrashing 2023-10-06T04:05:57.302 INFO:tasks.workunit.client.0.smithi157.stderr:+ retry 2023-10-06T04:05:57.303 INFO:tasks.workunit.client.0.smithi157.stderr:+ rm -rf ceph 2023-10-06T04:05:57.303 INFO:tasks.workunit.client.0.smithi157.stderr:+ timeout 3600 git clone https://git.ceph.com/ceph.git 2023-10-06T04:05:57.305 INFO:tasks.workunit.client.0.smithi157.stderr:Cloning into 'ceph'... 2023-10-06T04:05:57.837 INFO:tasks.ceph.mds.c.smithi157.stderr:2023-10-06T04:05:57.836+0000 7ff7d0710700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T04:05:57.837 INFO:tasks.ceph.mds.c.smithi157.stderr:2023-10-06T04:05:57.836+0000 7ff7d0710700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-10-06T04:06:15.498 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:06:15.502 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:06:45.586 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:06:45.588 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:07:14.701 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:07:14.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.837+0000 7f6202bfb700 1 Processor -- start 2023-10-06T04:07:14.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.837+0000 7f6202bfb700 1 -- start start 2023-10-06T04:07:14.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.838+0000 7f6202bfb700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f61fc0c8b60 0x7f61fc155b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:14.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.838+0000 7f6202bfb700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61fc0cc050 0x7f61fc0c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:14.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.838+0000 7f6202bfb700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61fc0af480 con 0x7f61fc0c8790 2023-10-06T04:07:14.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.838+0000 7f6202bfb700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61fc0af760 con 0x7f61fc0c8b60 2023-10-06T04:07:14.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.838+0000 7f6202bfb700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc0afa40 con 0x7f61fc0cc050 2023-10-06T04:07:14.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.838+0000 7f62013f8700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f61fc0c8b60 0x7f61fc155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:14.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.838+0000 7f62013f8700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f61fc0c8b60 0x7f61fc155b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42336/0 (socket says 172.21.15.157:42336) 2023-10-06T04:07:14.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.838+0000 7f62013f8700 1 -- 172.21.15.157:0/2557129531 learned_addr learned my addr 172.21.15.157:0/2557129531 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:07:14.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.838+0000 7f62023fa700 1 --2- 172.21.15.157:0/2557129531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61fc0cc050 0x7f61fc0c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:14.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.839+0000 7f62013f8700 1 -- 172.21.15.157:0/2557129531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61fc0cc050 msgr2=0x7f61fc0c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:14.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.839+0000 7f62013f8700 1 --2- 172.21.15.157:0/2557129531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61fc0cc050 0x7f61fc0c75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:14.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.839+0000 7f62013f8700 1 -- 172.21.15.157:0/2557129531 >> v1:172.21.15.175:6789/0 conn(0x7f61fc0c8790 legacy=0x7f61fc0b1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:14.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.839+0000 7f62013f8700 1 -- 172.21.15.157:0/2557129531 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61fc0afd20 con 0x7f61fc0c8b60 2023-10-06T04:07:14.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.839+0000 7f62013f8700 1 --2- 172.21.15.157:0/2557129531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61fc0c8b60 0x7f61fc155b60 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f61ec0099b0 tx=0x7f61ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ed70d68b610dc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:14.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.839+0000 7f6200bf7700 1 -- 172.21.15.157:0/2557129531 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec009ea0 con 0x7f61fc0c8b60 2023-10-06T04:07:14.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.839+0000 7f6200bf7700 1 -- 172.21.15.157:0/2557129531 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ec00b860 con 0x7f61fc0c8b60 2023-10-06T04:07:14.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.839+0000 7f6200bf7700 1 -- 172.21.15.157:0/2557129531 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec00bc00 con 0x7f61fc0c8b60 2023-10-06T04:07:14.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.840+0000 7f6202bfb700 1 -- 172.21.15.157:0/2557129531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61fc0c8b60 msgr2=0x7f61fc155b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:14.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.840+0000 7f6202bfb700 1 --2- 172.21.15.157:0/2557129531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61fc0c8b60 0x7f61fc155b60 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f61ec0099b0 tx=0x7f61ec009250 comp rx=0 tx=0).stop 2023-10-06T04:07:14.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.840+0000 7f6202bfb700 1 -- 172.21.15.157:0/2557129531 shutdown_connections 2023-10-06T04:07:14.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.840+0000 7f6202bfb700 1 --2- 172.21.15.157:0/2557129531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61fc0cc050 0x7f61fc0c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:14.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.840+0000 7f6202bfb700 1 --2- 172.21.15.157:0/2557129531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61fc0c8b60 0x7f61fc155b60 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:14.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.840+0000 7f6202bfb700 1 -- 172.21.15.157:0/2557129531 >> 172.21.15.157:0/2557129531 conn(0x7f61fc0cc540 msgr2=0x7f61fc0cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:14.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.840+0000 7f6202bfb700 1 -- 172.21.15.157:0/2557129531 shutdown_connections 2023-10-06T04:07:14.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.840+0000 7f6202bfb700 1 -- 172.21.15.157:0/2557129531 wait complete. 2023-10-06T04:07:14.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.842+0000 7f6202bfb700 1 Processor -- start 2023-10-06T04:07:14.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.842+0000 7f6202bfb700 1 -- start start 2023-10-06T04:07:14.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.843+0000 7f6202bfb700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61fc0c8790 0x7f61fc166900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:14.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.843+0000 7f6202bfb700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61fc0c8b60 0x7f61fc0c21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:14.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.843+0000 7f6202bfb700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61fc0cc050 0x7f61fc0c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:14.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.843+0000 7f6202bfb700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc15a920 con 0x7f61fc0c8b60 2023-10-06T04:07:14.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.843+0000 7f6202bfb700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc0c3710 con 0x7f61fc0cc050 2023-10-06T04:07:14.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.843+0000 7f6202bfb700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc0c3160 con 0x7f61fc0c8790 2023-10-06T04:07:14.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.843+0000 7f62013f8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61fc0c8b60 0x7f61fc0c21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:14.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.843+0000 7f62013f8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61fc0c8b60 0x7f61fc0c21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42342/0 (socket says 172.21.15.157:42342) 2023-10-06T04:07:14.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.843+0000 7f62013f8700 1 -- 172.21.15.157:0/4077799243 learned_addr learned my addr 172.21.15.157:0/4077799243 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:07:14.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.843+0000 7f62023fa700 1 --2- 172.21.15.157:0/4077799243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61fc0cc050 0x7f61fc0c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:14.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.843+0000 7f6201bf9700 1 --2- 172.21.15.157:0/4077799243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61fc0c8790 0x7f61fc166900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:14.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.843+0000 7f62013f8700 1 -- 172.21.15.157:0/4077799243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61fc0c8790 msgr2=0x7f61fc166900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:14.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.843+0000 7f62013f8700 1 --2- 172.21.15.157:0/4077799243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61fc0c8790 0x7f61fc166900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:14.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.843+0000 7f62013f8700 1 -- 172.21.15.157:0/4077799243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61fc0cc050 msgr2=0x7f61fc0c3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:14.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.843+0000 7f62013f8700 1 --2- 172.21.15.157:0/4077799243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61fc0cc050 0x7f61fc0c3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:14.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.843+0000 7f62013f8700 1 -- 172.21.15.157:0/4077799243 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ec0094f0 con 0x7f61fc0c8b60 2023-10-06T04:07:14.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.843+0000 7f62023fa700 1 --2- 172.21.15.157:0/4077799243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61fc0cc050 0x7f61fc0c3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:07:14.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.843+0000 7f6201bf9700 1 --2- 172.21.15.157:0/4077799243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61fc0c8790 0x7f61fc166900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:07:14.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.843+0000 7f62013f8700 1 --2- 172.21.15.157:0/4077799243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61fc0c8b60 0x7f61fc0c21c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f61ec006fd0 tx=0x7f61ec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:14.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.843+0000 7f61f1ffb700 1 -- 172.21.15.157:0/4077799243 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec00bbe0 con 0x7f61fc0c8b60 2023-10-06T04:07:14.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.844+0000 7f61f1ffb700 1 -- 172.21.15.157:0/4077799243 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ec011420 con 0x7f61fc0c8b60 2023-10-06T04:07:14.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.844+0000 7f6202bfb700 1 -- 172.21.15.157:0/4077799243 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61fc0c2bc0 con 0x7f61fc0c8b60 2023-10-06T04:07:14.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.844+0000 7f61f1ffb700 1 -- 172.21.15.157:0/4077799243 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec0117c0 con 0x7f61fc0c8b60 2023-10-06T04:07:14.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.844+0000 7f6202bfb700 1 -- 172.21.15.157:0/4077799243 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61fc15a360 con 0x7f61fc0c8b60 2023-10-06T04:07:14.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.845+0000 7f6202bfb700 1 -- 172.21.15.157:0/4077799243 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61fc0b1140 con 0x7f61fc0c8b60 2023-10-06T04:07:14.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.845+0000 7f61f1ffb700 1 -- 172.21.15.157:0/4077799243 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f61ec011bb0 con 0x7f61fc0c8b60 2023-10-06T04:07:14.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.845+0000 7f61f1ffb700 1 --2- 172.21.15.157:0/4077799243 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f61e0066f30 0x7f61e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:14.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.845+0000 7f61f1ffb700 1 -- 172.21.15.157:0/4077799243 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f61ec086170 con 0x7f61fc0c8b60 2023-10-06T04:07:14.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.846+0000 7f6201bf9700 1 --2- 172.21.15.157:0/4077799243 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f61e0066f30 0x7f61e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:14.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.846+0000 7f6201bf9700 1 --2- 172.21.15.157:0/4077799243 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f61e0066f30 0x7f61e00693e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f61f8001840 tx=0x7f61f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:14.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:14.850+0000 7f61f1ffb700 1 -- 172.21.15.157:0/4077799243 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61ec0530f0 con 0x7f61fc0c8b60 2023-10-06T04:07:15.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.080+0000 7f6202bfb700 1 -- 172.21.15.157:0/4077799243 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61fc04bb40 con 0x7f61fc0c8b60 2023-10-06T04:07:15.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.080+0000 7f61f1ffb700 1 -- 172.21.15.157:0/4077799243 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+6148 (secure 0 0 0) 0x7f61ec011e60 con 0x7f61fc0c8b60 2023-10-06T04:07:15.082 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:07:15.082 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5698,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:06:03.836756+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:07:15.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.082+0000 7f6202bfb700 1 -- 172.21.15.157:0/4077799243 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f61e0066f30 msgr2=0x7f61e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:15.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.082+0000 7f6202bfb700 1 --2- 172.21.15.157:0/4077799243 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f61e0066f30 0x7f61e00693e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f61f8001840 tx=0x7f61f8009040 comp rx=0 tx=0).stop 2023-10-06T04:07:15.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.082+0000 7f6202bfb700 1 -- 172.21.15.157:0/4077799243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61fc0c8b60 msgr2=0x7f61fc0c21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:15.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.082+0000 7f6202bfb700 1 --2- 172.21.15.157:0/4077799243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61fc0c8b60 0x7f61fc0c21c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f61ec006fd0 tx=0x7f61ec017000 comp rx=0 tx=0).stop 2023-10-06T04:07:15.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.083+0000 7f6202bfb700 1 -- 172.21.15.157:0/4077799243 shutdown_connections 2023-10-06T04:07:15.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.083+0000 7f6202bfb700 1 --2- 172.21.15.157:0/4077799243 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f61e0066f30 0x7f61e00693e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:15.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.083+0000 7f6202bfb700 1 --2- 172.21.15.157:0/4077799243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61fc0cc050 0x7f61fc0c3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:15.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.083+0000 7f6202bfb700 1 --2- 172.21.15.157:0/4077799243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61fc0c8790 0x7f61fc166900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:15.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.083+0000 7f6202bfb700 1 --2- 172.21.15.157:0/4077799243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61fc0c8b60 0x7f61fc0c21c0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:15.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.083+0000 7f6202bfb700 1 -- 172.21.15.157:0/4077799243 >> 172.21.15.157:0/4077799243 conn(0x7f61fc0cc540 msgr2=0x7f61fc06d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:15.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.083+0000 7f6202bfb700 1 -- 172.21.15.157:0/4077799243 shutdown_connections 2023-10-06T04:07:15.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.083+0000 7f6202bfb700 1 -- 172.21.15.157:0/4077799243 wait complete. 2023-10-06T04:07:15.086 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 71 2023-10-06T04:07:15.096 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.d (rank=1) 2023-10-06T04:07:15.097 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T04:07:15.097 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:07:15.106 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T04:07:15.106 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:07:15.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.220+0000 7fa2919da700 1 Processor -- start 2023-10-06T04:07:15.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.220+0000 7fa2919da700 1 -- start start 2023-10-06T04:07:15.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.221+0000 7fa2919da700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa28c0b1e40 0x7fa28c157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:15.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.221+0000 7fa2919da700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa28c06c2d0 0x7fa28c06dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:15.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.221+0000 7fa2919da700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa28c06d8d0 con 0x7fa28c0b04d0 2023-10-06T04:07:15.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.221+0000 7fa2919da700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa28c0af8f0 con 0x7fa28c06c2d0 2023-10-06T04:07:15.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.221+0000 7fa2919da700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c15a690 con 0x7fa28c0b1e40 2023-10-06T04:07:15.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.221+0000 7fa2911d9700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa28c06c2d0 0x7fa28c06dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:15.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.221+0000 7fa2911d9700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa28c06c2d0 0x7fa28c06dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42350/0 (socket says 172.21.15.157:42350) 2023-10-06T04:07:15.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.221+0000 7fa283fff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa28c0b04d0 0x7fa28c1563b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41522/0 (socket says 172.21.15.157:41522) 2023-10-06T04:07:15.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.221+0000 7fa2911d9700 1 -- 172.21.15.157:0/1947020768 learned_addr learned my addr 172.21.15.157:0/1947020768 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:07:15.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.221+0000 7fa2909d8700 1 --2- 172.21.15.157:0/1947020768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa28c0b1e40 0x7fa28c157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:15.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.221+0000 7fa2911d9700 1 -- 172.21.15.157:0/1947020768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa28c0b1e40 msgr2=0x7fa28c157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:15.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.221+0000 7fa2911d9700 1 --2- 172.21.15.157:0/1947020768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa28c0b1e40 0x7fa28c157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:15.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.221+0000 7fa2911d9700 1 -- 172.21.15.157:0/1947020768 >> v1:172.21.15.157:6789/0 conn(0x7fa28c0b04d0 legacy=0x7fa28c1563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:15.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.221+0000 7fa2911d9700 1 -- 172.21.15.157:0/1947020768 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28c06ccc0 con 0x7fa28c06c2d0 2023-10-06T04:07:15.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.221+0000 7fa2837fe700 1 -- 172.21.15.157:0/1947020768 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3013438877 0 0) 0x7fa28c06d8d0 con 0x7fa28c0b04d0 2023-10-06T04:07:15.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.222+0000 7fa2911d9700 1 --2- 172.21.15.157:0/1947020768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa28c06c2d0 0x7fa28c06dd60 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa284009f20 tx=0x7fa284009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=355278f09dbac103 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:15.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.222+0000 7fa2837fe700 1 -- 172.21.15.157:0/1947020768 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28400e070 con 0x7fa28c06c2d0 2023-10-06T04:07:15.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.222+0000 7fa2837fe700 1 -- 172.21.15.157:0/1947020768 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa28400b9d0 con 0x7fa28c06c2d0 2023-10-06T04:07:15.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.222+0000 7fa2837fe700 1 -- 172.21.15.157:0/1947020768 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2840092e0 con 0x7fa28c06c2d0 2023-10-06T04:07:15.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.222+0000 7fa2919da700 1 -- 172.21.15.157:0/1947020768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa28c06c2d0 msgr2=0x7fa28c06dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:15.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.222+0000 7fa2919da700 1 --2- 172.21.15.157:0/1947020768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa28c06c2d0 0x7fa28c06dd60 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa284009f20 tx=0x7fa284009840 comp rx=0 tx=0).stop 2023-10-06T04:07:15.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.222+0000 7fa2919da700 1 -- 172.21.15.157:0/1947020768 shutdown_connections 2023-10-06T04:07:15.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.222+0000 7fa2919da700 1 --2- 172.21.15.157:0/1947020768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa28c06c2d0 0x7fa28c06dd60 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:15.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.222+0000 7fa2919da700 1 --2- 172.21.15.157:0/1947020768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa28c0b1e40 0x7fa28c157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:15.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.222+0000 7fa2919da700 1 -- 172.21.15.157:0/1947020768 >> 172.21.15.157:0/1947020768 conn(0x7fa28c0cc540 msgr2=0x7fa28c0cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:15.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.222+0000 7fa2919da700 1 -- 172.21.15.157:0/1947020768 shutdown_connections 2023-10-06T04:07:15.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.222+0000 7fa2919da700 1 -- 172.21.15.157:0/1947020768 wait complete. 2023-10-06T04:07:15.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.223+0000 7fa2919da700 1 Processor -- start 2023-10-06T04:07:15.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.224+0000 7fa2919da700 1 -- start start 2023-10-06T04:07:15.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.224+0000 7fa2919da700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa28c06c2d0 0x7fa28c166600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:15.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.224+0000 7fa2919da700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa28c0b04d0 0x7fa28c166b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:15.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.224+0000 7fa2919da700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa28c0b1e40 0x7fa28c152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:15.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.224+0000 7fa2919da700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c151010 con 0x7fa28c0b04d0 2023-10-06T04:07:15.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.224+0000 7fa2919da700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c0c80c0 con 0x7fa28c06c2d0 2023-10-06T04:07:15.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.224+0000 7fa2919da700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c166100 con 0x7fa28c0b1e40 2023-10-06T04:07:15.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.224+0000 7fa283fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa28c0b04d0 0x7fa28c166b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:15.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.224+0000 7fa283fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa28c0b04d0 0x7fa28c166b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42362/0 (socket says 172.21.15.157:42362) 2023-10-06T04:07:15.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.224+0000 7fa283fff700 1 -- 172.21.15.157:0/3035258753 learned_addr learned my addr 172.21.15.157:0/3035258753 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:07:15.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.225+0000 7fa2909d8700 1 --2- 172.21.15.157:0/3035258753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa28c06c2d0 0x7fa28c166600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:15.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.225+0000 7fa2911d9700 1 --2- 172.21.15.157:0/3035258753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa28c0b1e40 0x7fa28c152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:15.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.225+0000 7fa283fff700 1 -- 172.21.15.157:0/3035258753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa28c0b1e40 msgr2=0x7fa28c152010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:15.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.225+0000 7fa283fff700 1 --2- 172.21.15.157:0/3035258753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa28c0b1e40 0x7fa28c152010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:15.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.225+0000 7fa283fff700 1 -- 172.21.15.157:0/3035258753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa28c06c2d0 msgr2=0x7fa28c166600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:15.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.225+0000 7fa283fff700 1 --2- 172.21.15.157:0/3035258753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa28c06c2d0 0x7fa28c166600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:15.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.225+0000 7fa283fff700 1 -- 172.21.15.157:0/3035258753 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa284009ae0 con 0x7fa28c0b04d0 2023-10-06T04:07:15.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.225+0000 7fa283fff700 1 --2- 172.21.15.157:0/3035258753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa28c0b04d0 0x7fa28c166b40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa27c0011e0 tx=0x7fa27c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:15.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.225+0000 7fa280ff9700 1 -- 172.21.15.157:0/3035258753 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c005a70 con 0x7fa28c0b04d0 2023-10-06T04:07:15.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.225+0000 7fa280ff9700 1 -- 172.21.15.157:0/3035258753 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27c005bd0 con 0x7fa28c0b04d0 2023-10-06T04:07:15.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.225+0000 7fa2919da700 1 -- 172.21.15.157:0/3035258753 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa28c0cccc0 con 0x7fa28c0b04d0 2023-10-06T04:07:15.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.225+0000 7fa280ff9700 1 -- 172.21.15.157:0/3035258753 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c0070d0 con 0x7fa28c0b04d0 2023-10-06T04:07:15.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.225+0000 7fa2919da700 1 -- 172.21.15.157:0/3035258753 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa28c150d30 con 0x7fa28c0b04d0 2023-10-06T04:07:15.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.226+0000 7fa2919da700 1 -- 172.21.15.157:0/3035258753 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa28c0b0310 con 0x7fa28c0b04d0 2023-10-06T04:07:15.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.226+0000 7fa280ff9700 1 -- 172.21.15.157:0/3035258753 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa27c007480 con 0x7fa28c0b04d0 2023-10-06T04:07:15.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.227+0000 7fa280ff9700 1 --2- 172.21.15.157:0/3035258753 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa278066f60 0x7fa278069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:15.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.227+0000 7fa280ff9700 1 -- 172.21.15.157:0/3035258753 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa27c083cf0 con 0x7fa28c0b04d0 2023-10-06T04:07:15.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.227+0000 7fa2909d8700 1 --2- 172.21.15.157:0/3035258753 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa278066f60 0x7fa278069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:15.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.227+0000 7fa2909d8700 1 --2- 172.21.15.157:0/3035258753 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa278066f60 0x7fa278069410 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa288009db0 tx=0x7fa2880091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:15.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.231+0000 7fa280ff9700 1 -- 172.21.15.157:0/3035258753 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa27c088070 con 0x7fa28c0b04d0 2023-10-06T04:07:15.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.464+0000 7fa2919da700 1 -- 172.21.15.157:0/3035258753 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa28c06cfa0 con 0x7fa28c0b04d0 2023-10-06T04:07:15.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.464+0000 7fa280ff9700 1 -- 172.21.15.157:0/3035258753 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+6148 (secure 0 0 0) 0x7fa27c017020 con 0x7fa28c0b04d0 2023-10-06T04:07:15.466 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:07:15.466 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5698,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:06:03.836756+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:07:15.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.466+0000 7fa2919da700 1 -- 172.21.15.157:0/3035258753 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa278066f60 msgr2=0x7fa278069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:15.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.466+0000 7fa2919da700 1 --2- 172.21.15.157:0/3035258753 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa278066f60 0x7fa278069410 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa288009db0 tx=0x7fa2880091a0 comp rx=0 tx=0).stop 2023-10-06T04:07:15.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.466+0000 7fa2919da700 1 -- 172.21.15.157:0/3035258753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa28c0b04d0 msgr2=0x7fa28c166b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:15.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.466+0000 7fa2919da700 1 --2- 172.21.15.157:0/3035258753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa28c0b04d0 0x7fa28c166b40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa27c0011e0 tx=0x7fa27c007e10 comp rx=0 tx=0).stop 2023-10-06T04:07:15.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.466+0000 7fa2919da700 1 -- 172.21.15.157:0/3035258753 shutdown_connections 2023-10-06T04:07:15.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.466+0000 7fa2919da700 1 --2- 172.21.15.157:0/3035258753 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa278066f60 0x7fa278069410 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:15.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.466+0000 7fa2919da700 1 --2- 172.21.15.157:0/3035258753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa28c06c2d0 0x7fa28c166600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:15.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.466+0000 7fa2919da700 1 --2- 172.21.15.157:0/3035258753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa28c0b1e40 0x7fa28c152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:15.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.466+0000 7fa2919da700 1 --2- 172.21.15.157:0/3035258753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa28c0b04d0 0x7fa28c166b40 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:15.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.466+0000 7fa2919da700 1 -- 172.21.15.157:0/3035258753 >> 172.21.15.157:0/3035258753 conn(0x7fa28c0cc540 msgr2=0x7fa28c157440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:15.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.466+0000 7fa2919da700 1 -- 172.21.15.157:0/3035258753 shutdown_connections 2023-10-06T04:07:15.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:15.466+0000 7fa2919da700 1 -- 172.21.15.157:0/3035258753 wait complete. 2023-10-06T04:07:15.470 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 71 2023-10-06T04:07:15.480 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-10-06T04:07:15.656 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:07:15.659 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:07:17.481 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:07:17.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.607+0000 7f35916b4700 1 Processor -- start 2023-10-06T04:07:17.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.607+0000 7f35916b4700 1 -- start start 2023-10-06T04:07:17.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.607+0000 7f35916b4700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f358c06cbb0 0x7f358c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:17.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.607+0000 7f35916b4700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f358c0c9780 con 0x7f358c150410 2023-10-06T04:07:17.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.607+0000 7f35916b4700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f358c06e0c0 con 0x7f358c06c1f0 2023-10-06T04:07:17.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.607+0000 7f35916b4700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f358c0c8c40 con 0x7f358c06cbb0 2023-10-06T04:07:17.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.608+0000 7f358b7fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f358c150410 0x7f358c155f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41532/0 (socket says 172.21.15.157:41532) 2023-10-06T04:07:17.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.608+0000 7f358b7fe700 1 -- 172.21.15.157:0/15695239 learned_addr learned my addr 172.21.15.157:0/15695239 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:07:17.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.608+0000 7f358affd700 1 --2- 172.21.15.157:0/15695239 >> v2:172.21.15.175:3300/0 conn(0x7f358c06cbb0 0x7f358c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:17.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.608+0000 7f358a7fc700 1 -- 172.21.15.157:0/15695239 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2621574082 0 0) 0x7f358c0c9780 con 0x7f358c150410 2023-10-06T04:07:17.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.608+0000 7f358a7fc700 1 -- 172.21.15.157:0/15695239 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3578003570 con 0x7f358c150410 2023-10-06T04:07:17.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.608+0000 7f358a7fc700 1 -- 172.21.15.157:0/15695239 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2145598083 0 0) 0x7f3578003570 con 0x7f358c150410 2023-10-06T04:07:17.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.608+0000 7f358a7fc700 1 -- 172.21.15.157:0/15695239 >> v2:172.21.15.175:3300/0 conn(0x7f358c06cbb0 msgr2=0x7f358c06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:17.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.608+0000 7f358a7fc700 1 --2- 172.21.15.157:0/15695239 >> v2:172.21.15.175:3300/0 conn(0x7f358c06cbb0 0x7f358c06d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:17.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.608+0000 7f358a7fc700 1 -- 172.21.15.157:0/15695239 >> v1:172.21.15.175:6789/0 conn(0x7f358c06c1f0 legacy=0x7f358c14fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:17.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.608+0000 7f358a7fc700 1 -- 172.21.15.157:0/15695239 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358c0c8e50 con 0x7f358c150410 2023-10-06T04:07:17.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.608+0000 7f358a7fc700 1 -- 172.21.15.157:0/15695239 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f35840017f0 con 0x7f358c150410 2023-10-06T04:07:17.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.608+0000 7f358a7fc700 1 -- 172.21.15.157:0/15695239 >> v1:172.21.15.157:6789/0 conn(0x7f358c150410 legacy=0x7f358c155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:17.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.608+0000 7f358a7fc700 1 --2- 172.21.15.157:0/15695239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3578003a60 0x7f3578003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:17.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.608+0000 7f358a7fc700 1 --2- 172.21.15.157:0/15695239 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3578004810 0x7f3578004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:17.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.608+0000 7f358a7fc700 1 --2- 172.21.15.157:0/15695239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35780051c0 0x7f357800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:17.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.608+0000 7f358a7fc700 1 -- 172.21.15.157:0/15695239 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c0c8c40 con 0x7f35780051c0 2023-10-06T04:07:17.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.608+0000 7f358a7fc700 1 -- 172.21.15.157:0/15695239 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357800c050 con 0x7f3578003a60 2023-10-06T04:07:17.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.608+0000 7f358a7fc700 1 -- 172.21.15.157:0/15695239 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f357800c210 con 0x7f3578004810 2023-10-06T04:07:17.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.608+0000 7f358b7fe700 1 --2- 172.21.15.157:0/15695239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35780051c0 0x7f357800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:17.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.608+0000 7f358affd700 1 --2- 172.21.15.157:0/15695239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3578003a60 0x7f3578003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:17.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.608+0000 7f35827fc700 1 --2- 172.21.15.157:0/15695239 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3578004810 0x7f3578004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:17.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.609+0000 7f358b7fe700 1 -- 172.21.15.157:0/15695239 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3578004810 msgr2=0x7f3578004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:17.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.609+0000 7f358b7fe700 1 --2- 172.21.15.157:0/15695239 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3578004810 0x7f3578004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:17.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.609+0000 7f358b7fe700 1 -- 172.21.15.157:0/15695239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3578003a60 msgr2=0x7f3578003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:17.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.609+0000 7f358b7fe700 1 --2- 172.21.15.157:0/15695239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3578003a60 0x7f3578003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:17.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.609+0000 7f358b7fe700 1 -- 172.21.15.157:0/15695239 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3578003570 con 0x7f35780051c0 2023-10-06T04:07:17.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.609+0000 7f358b7fe700 1 -- 172.21.15.157:0/15695239 reap_dead start 2023-10-06T04:07:17.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.609+0000 7f358b7fe700 1 --2- 172.21.15.157:0/15695239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35780051c0 0x7f357800ba60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f358400a8d0 tx=0x7f3584002170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b272f2d66a5acf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:17.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.609+0000 7f358a7fc700 1 -- 172.21.15.157:0/15695239 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35840070d0 con 0x7f35780051c0 2023-10-06T04:07:17.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.609+0000 7f358a7fc700 1 -- 172.21.15.157:0/15695239 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3584007270 con 0x7f35780051c0 2023-10-06T04:07:17.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.609+0000 7f358a7fc700 1 -- 172.21.15.157:0/15695239 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3584007630 con 0x7f35780051c0 2023-10-06T04:07:17.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.609+0000 7f35916b4700 1 -- 172.21.15.157:0/15695239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35780051c0 msgr2=0x7f357800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:17.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.609+0000 7f35916b4700 1 --2- 172.21.15.157:0/15695239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35780051c0 0x7f357800ba60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f358400a8d0 tx=0x7f3584002170 comp rx=0 tx=0).stop 2023-10-06T04:07:17.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.610+0000 7f35916b4700 1 -- 172.21.15.157:0/15695239 shutdown_connections 2023-10-06T04:07:17.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.610+0000 7f35916b4700 1 --2- 172.21.15.157:0/15695239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35780051c0 0x7f357800ba60 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:17.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.610+0000 7f35916b4700 1 -- 172.21.15.157:0/15695239 >> 172.21.15.157:0/15695239 conn(0x7f358c06f230 msgr2=0x7f358c06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:17.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.610+0000 7f35916b4700 1 -- 172.21.15.157:0/15695239 shutdown_connections 2023-10-06T04:07:17.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.610+0000 7f35916b4700 1 -- 172.21.15.157:0/15695239 wait complete. 2023-10-06T04:07:17.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.611+0000 7f35916b4700 1 Processor -- start 2023-10-06T04:07:17.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.611+0000 7f35916b4700 1 -- start start 2023-10-06T04:07:17.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.611+0000 7f35916b4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35780051c0 0x7f358c169f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:17.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.612+0000 7f35916b4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f358c148550 0x7f358c145ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:17.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.612+0000 7f35916b4700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f358c147160 0x7f358c147600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:17.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.612+0000 7f35916b4700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c1537f0 con 0x7f358c148550 2023-10-06T04:07:17.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.612+0000 7f35916b4700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c150850 con 0x7f35780051c0 2023-10-06T04:07:17.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.612+0000 7f35916b4700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f358c16a4c0 con 0x7f358c147160 2023-10-06T04:07:17.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.612+0000 7f35827fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f358c148550 0x7f358c145ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:17.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.612+0000 7f35827fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f358c148550 0x7f358c145ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42386/0 (socket says 172.21.15.157:42386) 2023-10-06T04:07:17.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.612+0000 7f35827fc700 1 -- 172.21.15.157:0/1262486572 learned_addr learned my addr 172.21.15.157:0/1262486572 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:07:17.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.612+0000 7f358b7fe700 1 --2- 172.21.15.157:0/1262486572 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f358c147160 0x7f358c147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:17.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.612+0000 7f358affd700 1 --2- 172.21.15.157:0/1262486572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35780051c0 0x7f358c169f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:17.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.612+0000 7f35827fc700 1 -- 172.21.15.157:0/1262486572 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f358c147160 msgr2=0x7f358c147600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:17.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.612+0000 7f35827fc700 1 --2- 172.21.15.157:0/1262486572 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f358c147160 0x7f358c147600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:17.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.612+0000 7f35827fc700 1 -- 172.21.15.157:0/1262486572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35780051c0 msgr2=0x7f358c169f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:17.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.612+0000 7f35827fc700 1 --2- 172.21.15.157:0/1262486572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35780051c0 0x7f358c169f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:17.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.612+0000 7f35827fc700 1 -- 172.21.15.157:0/1262486572 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358c146cd0 con 0x7f358c148550 2023-10-06T04:07:17.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.612+0000 7f358b7fe700 1 --2- 172.21.15.157:0/1262486572 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f358c147160 0x7f358c147600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:07:17.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.612+0000 7f35827fc700 1 --2- 172.21.15.157:0/1262486572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f358c148550 0x7f358c145ae0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f357400f900 tx=0x7f3574007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:17.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.612+0000 7f35837fe700 1 -- 172.21.15.157:0/1262486572 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357400fe40 con 0x7f358c148550 2023-10-06T04:07:17.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.612+0000 7f35837fe700 1 -- 172.21.15.157:0/1262486572 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357400d880 con 0x7f358c148550 2023-10-06T04:07:17.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.612+0000 7f35837fe700 1 -- 172.21.15.157:0/1262486572 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357400dc40 con 0x7f358c148550 2023-10-06T04:07:17.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.612+0000 7f35916b4700 1 -- 172.21.15.157:0/1262486572 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f358c153230 con 0x7f358c148550 2023-10-06T04:07:17.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.612+0000 7f35916b4700 1 -- 172.21.15.157:0/1262486572 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358c146680 con 0x7f358c148550 2023-10-06T04:07:17.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.613+0000 7f35916b4700 1 -- 172.21.15.157:0/1262486572 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f358c1464c0 con 0x7f358c148550 2023-10-06T04:07:17.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.614+0000 7f35837fe700 1 -- 172.21.15.157:0/1262486572 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f357401c400 con 0x7f358c148550 2023-10-06T04:07:17.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.614+0000 7f35837fe700 1 --2- 172.21.15.157:0/1262486572 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f356c066f30 0x7f356c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:17.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.614+0000 7f35837fe700 1 -- 172.21.15.157:0/1262486572 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f35740848d0 con 0x7f358c148550 2023-10-06T04:07:17.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.614+0000 7f358affd700 1 --2- 172.21.15.157:0/1262486572 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f356c066f30 0x7f356c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:17.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.615+0000 7f358affd700 1 --2- 172.21.15.157:0/1262486572 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f356c066f30 0x7f356c0693e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f357c006ed0 tx=0x7f357c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:17.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.618+0000 7f35837fe700 1 -- 172.21.15.157:0/1262486572 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3574051850 con 0x7f358c148550 2023-10-06T04:07:17.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.846+0000 7f35916b4700 1 -- 172.21.15.157:0/1262486572 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f358c06cf80 con 0x7f358c148550 2023-10-06T04:07:17.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.846+0000 7f35837fe700 1 -- 172.21.15.157:0/1262486572 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+6148 (secure 0 0 0) 0x7f3574054e70 con 0x7f358c148550 2023-10-06T04:07:17.849 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:07:17.849 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5698,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:06:03.836756+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:07:17.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.849+0000 7f35916b4700 1 -- 172.21.15.157:0/1262486572 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f356c066f30 msgr2=0x7f356c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:17.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.849+0000 7f35916b4700 1 --2- 172.21.15.157:0/1262486572 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f356c066f30 0x7f356c0693e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f357c006ed0 tx=0x7f357c008040 comp rx=0 tx=0).stop 2023-10-06T04:07:17.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.849+0000 7f35916b4700 1 -- 172.21.15.157:0/1262486572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f358c148550 msgr2=0x7f358c145ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:17.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.849+0000 7f35916b4700 1 --2- 172.21.15.157:0/1262486572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f358c148550 0x7f358c145ae0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f357400f900 tx=0x7f3574007d30 comp rx=0 tx=0).stop 2023-10-06T04:07:17.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.849+0000 7f35916b4700 1 -- 172.21.15.157:0/1262486572 shutdown_connections 2023-10-06T04:07:17.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.849+0000 7f35916b4700 1 --2- 172.21.15.157:0/1262486572 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f356c066f30 0x7f356c0693e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:17.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.849+0000 7f35916b4700 1 --2- 172.21.15.157:0/1262486572 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35780051c0 0x7f358c169f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:17.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.849+0000 7f35916b4700 1 --2- 172.21.15.157:0/1262486572 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f358c147160 0x7f358c147600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:17.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.849+0000 7f35916b4700 1 --2- 172.21.15.157:0/1262486572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f358c148550 0x7f358c145ae0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:17.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.849+0000 7f35916b4700 1 -- 172.21.15.157:0/1262486572 >> 172.21.15.157:0/1262486572 conn(0x7f358c06f230 msgr2=0x7f358c14cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:17.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.849+0000 7f35916b4700 1 -- 172.21.15.157:0/1262486572 shutdown_connections 2023-10-06T04:07:17.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:17.849+0000 7f35916b4700 1 -- 172.21.15.157:0/1262486572 wait complete. 2023-10-06T04:07:17.853 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 71 2023-10-06T04:07:17.863 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-10-06T04:07:19.864 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:07:19.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.983+0000 7ffada6c4700 1 Processor -- start 2023-10-06T04:07:19.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.983+0000 7ffada6c4700 1 -- start start 2023-10-06T04:07:19.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.984+0000 7ffada6c4700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ffad406d4c0 0x7ffad4156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:19.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.984+0000 7ffada6c4700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffad40c2c10 0x7ffad40c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:19.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.984+0000 7ffada6c4700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad40c4240 con 0x7ffad40c0fb0 2023-10-06T04:07:19.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.984+0000 7ffada6c4700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffad40c4c10 con 0x7ffad406d4c0 2023-10-06T04:07:19.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.984+0000 7ffada6c4700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad40c3630 con 0x7ffad40c2c10 2023-10-06T04:07:19.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.984+0000 7ffad8c61700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ffad40c0fb0 0x7ffad40bf350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41548/0 (socket says 172.21.15.157:41548) 2023-10-06T04:07:19.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.984+0000 7ffad8c61700 1 -- 172.21.15.157:0/2070759992 learned_addr learned my addr 172.21.15.157:0/2070759992 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:07:19.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.984+0000 7ffad3fff700 1 --2- 172.21.15.157:0/2070759992 >> v2:172.21.15.175:3300/0 conn(0x7ffad406d4c0 0x7ffad4156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:19.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.984+0000 7ffad37fe700 1 --2- 172.21.15.157:0/2070759992 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffad40c2c10 0x7ffad40c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:19.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.984+0000 7ffad2ffd700 1 -- 172.21.15.157:0/2070759992 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 951956431 0 0) 0x7ffad40c4240 con 0x7ffad40c0fb0 2023-10-06T04:07:19.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.984+0000 7ffad2ffd700 1 -- 172.21.15.157:0/2070759992 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffabc003570 con 0x7ffad40c0fb0 2023-10-06T04:07:19.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.984+0000 7ffad2ffd700 1 -- 172.21.15.157:0/2070759992 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1682234256 0 0) 0x7ffabc003570 con 0x7ffad40c0fb0 2023-10-06T04:07:19.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.984+0000 7ffad2ffd700 1 -- 172.21.15.157:0/2070759992 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffad40c2c10 msgr2=0x7ffad40c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:19.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.984+0000 7ffad2ffd700 1 --2- 172.21.15.157:0/2070759992 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffad40c2c10 0x7ffad40c46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:19.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.984+0000 7ffad2ffd700 1 -- 172.21.15.157:0/2070759992 >> v2:172.21.15.175:3300/0 conn(0x7ffad406d4c0 msgr2=0x7ffad4156ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:19.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.984+0000 7ffad2ffd700 1 --2- 172.21.15.157:0/2070759992 >> v2:172.21.15.175:3300/0 conn(0x7ffad406d4c0 0x7ffad4156ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:19.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.984+0000 7ffad2ffd700 1 -- 172.21.15.157:0/2070759992 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad40c3910 con 0x7ffad40c0fb0 2023-10-06T04:07:19.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.984+0000 7ffad2ffd700 1 -- 172.21.15.157:0/2070759992 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7ffacc0017f0 con 0x7ffad40c0fb0 2023-10-06T04:07:19.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.985+0000 7ffad2ffd700 1 -- 172.21.15.157:0/2070759992 >> v1:172.21.15.157:6789/0 conn(0x7ffad40c0fb0 legacy=0x7ffad40bf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:19.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.985+0000 7ffad2ffd700 1 --2- 172.21.15.157:0/2070759992 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffad40c2c10 0x7ffabc003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:19.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.985+0000 7ffad2ffd700 1 --2- 172.21.15.157:0/2070759992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffabc003fe0 0x7ffabc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:19.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.985+0000 7ffad2ffd700 1 --2- 172.21.15.157:0/2070759992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffabc004d50 0x7ffabc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:19.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.985+0000 7ffad2ffd700 1 -- 172.21.15.157:0/2070759992 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad40c4c10 con 0x7ffabc003fe0 2023-10-06T04:07:19.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.985+0000 7ffad2ffd700 1 -- 172.21.15.157:0/2070759992 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad40c3630 con 0x7ffabc004d50 2023-10-06T04:07:19.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.985+0000 7ffad2ffd700 1 -- 172.21.15.157:0/2070759992 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc00ba90 con 0x7ffad40c2c10 2023-10-06T04:07:19.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.985+0000 7ffad37fe700 1 --2- 172.21.15.157:0/2070759992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffabc003fe0 0x7ffabc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:19.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.985+0000 7ffad3fff700 1 --2- 172.21.15.157:0/2070759992 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffad40c2c10 0x7ffabc003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:19.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.985+0000 7ffad8c61700 1 --2- 172.21.15.157:0/2070759992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffabc004d50 0x7ffabc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:19.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.985+0000 7ffad37fe700 1 -- 172.21.15.157:0/2070759992 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffad40c2c10 msgr2=0x7ffabc003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:19.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.985+0000 7ffad37fe700 1 --2- 172.21.15.157:0/2070759992 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffad40c2c10 0x7ffabc003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:19.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.985+0000 7ffad37fe700 1 -- 172.21.15.157:0/2070759992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffabc004d50 msgr2=0x7ffabc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:19.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.985+0000 7ffad37fe700 1 --2- 172.21.15.157:0/2070759992 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffabc004d50 0x7ffabc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:19.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.985+0000 7ffad37fe700 1 -- 172.21.15.157:0/2070759992 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffabc003570 con 0x7ffabc003fe0 2023-10-06T04:07:19.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.985+0000 7ffad37fe700 1 --2- 172.21.15.157:0/2070759992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffabc003fe0 0x7ffabc004810 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009c00 tx=0x7ffac80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7cbea6cf9ee2d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:19.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.985+0000 7ffad2ffd700 1 -- 172.21.15.157:0/2070759992 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac800e6b0 con 0x7ffabc003fe0 2023-10-06T04:07:19.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.985+0000 7ffad2ffd700 1 -- 172.21.15.157:0/2070759992 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac800e810 con 0x7ffabc003fe0 2023-10-06T04:07:19.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.985+0000 7ffad2ffd700 1 -- 172.21.15.157:0/2070759992 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac800ebd0 con 0x7ffabc003fe0 2023-10-06T04:07:19.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.986+0000 7ffada6c4700 1 -- 172.21.15.157:0/2070759992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffabc003fe0 msgr2=0x7ffabc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:19.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.986+0000 7ffada6c4700 1 --2- 172.21.15.157:0/2070759992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffabc003fe0 0x7ffabc004810 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009c00 tx=0x7ffac80091a0 comp rx=0 tx=0).stop 2023-10-06T04:07:19.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.986+0000 7ffad8c61700 1 -- 172.21.15.157:0/2070759992 reap_dead start 2023-10-06T04:07:19.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.986+0000 7ffada6c4700 1 -- 172.21.15.157:0/2070759992 shutdown_connections 2023-10-06T04:07:19.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.986+0000 7ffada6c4700 1 -- 172.21.15.157:0/2070759992 >> 172.21.15.157:0/2070759992 conn(0x7ffad40c8b20 msgr2=0x7ffad40c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:19.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.986+0000 7ffada6c4700 1 -- 172.21.15.157:0/2070759992 shutdown_connections 2023-10-06T04:07:19.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.986+0000 7ffada6c4700 1 -- 172.21.15.157:0/2070759992 wait complete. 2023-10-06T04:07:19.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.987+0000 7ffada6c4700 1 Processor -- start 2023-10-06T04:07:19.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.987+0000 7ffada6c4700 1 -- start start 2023-10-06T04:07:19.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.988+0000 7ffada6c4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffabc003fe0 0x7ffad406a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:19.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.988+0000 7ffada6c4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffad41578f0 0x7ffad4142ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:19.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.988+0000 7ffada6c4700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffad414ad20 0x7ffad4141130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:19.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.988+0000 7ffada6c4700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad4162090 con 0x7ffabc003fe0 2023-10-06T04:07:19.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.988+0000 7ffada6c4700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad4142710 con 0x7ffad41578f0 2023-10-06T04:07:19.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.988+0000 7ffada6c4700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad414b300 con 0x7ffad414ad20 2023-10-06T04:07:19.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.988+0000 7ffad3fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffabc003fe0 0x7ffad406a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:19.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.988+0000 7ffad3fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffabc003fe0 0x7ffad406a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42408/0 (socket says 172.21.15.157:42408) 2023-10-06T04:07:19.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.988+0000 7ffad3fff700 1 -- 172.21.15.157:0/1405543460 learned_addr learned my addr 172.21.15.157:0/1405543460 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:07:19.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.988+0000 7ffad37fe700 1 --2- 172.21.15.157:0/1405543460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffad41578f0 0x7ffad4142ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:19.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.988+0000 7ffad8c61700 1 --2- 172.21.15.157:0/1405543460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffad414ad20 0x7ffad4141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:19.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.988+0000 7ffad3fff700 1 -- 172.21.15.157:0/1405543460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffad414ad20 msgr2=0x7ffad4141130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:19.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.988+0000 7ffad3fff700 1 --2- 172.21.15.157:0/1405543460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffad414ad20 0x7ffad4141130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:19.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.988+0000 7ffad3fff700 1 -- 172.21.15.157:0/1405543460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffad41578f0 msgr2=0x7ffad4142ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:19.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.988+0000 7ffad3fff700 1 --2- 172.21.15.157:0/1405543460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffad41578f0 0x7ffad4142ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:19.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.988+0000 7ffad3fff700 1 -- 172.21.15.157:0/1405543460 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad4161db0 con 0x7ffabc003fe0 2023-10-06T04:07:19.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.988+0000 7ffad8c61700 1 --2- 172.21.15.157:0/1405543460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffad414ad20 0x7ffad4141130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:07:19.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.988+0000 7ffad3fff700 1 --2- 172.21.15.157:0/1405543460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffabc003fe0 0x7ffad406a650 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ffac4011d70 tx=0x7ffac40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:19.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.988+0000 7ffabbfff700 1 -- 172.21.15.157:0/1405543460 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac400fb50 con 0x7ffabc003fe0 2023-10-06T04:07:19.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.988+0000 7ffabbfff700 1 -- 172.21.15.157:0/1405543460 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac400fcb0 con 0x7ffabc003fe0 2023-10-06T04:07:19.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.988+0000 7ffada6c4700 1 -- 172.21.15.157:0/1405543460 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad41423a0 con 0x7ffabc003fe0 2023-10-06T04:07:19.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.988+0000 7ffabbfff700 1 -- 172.21.15.157:0/1405543460 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac4014650 con 0x7ffabc003fe0 2023-10-06T04:07:19.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.988+0000 7ffada6c4700 1 -- 172.21.15.157:0/1405543460 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad406ab90 con 0x7ffabc003fe0 2023-10-06T04:07:19.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.989+0000 7ffada6c4700 1 -- 172.21.15.157:0/1405543460 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad4142530 con 0x7ffabc003fe0 2023-10-06T04:07:19.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.990+0000 7ffabbfff700 1 -- 172.21.15.157:0/1405543460 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffac4014a80 con 0x7ffabc003fe0 2023-10-06T04:07:19.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.990+0000 7ffabbfff700 1 --2- 172.21.15.157:0/1405543460 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ffac0066f60 0x7ffac0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:19.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.990+0000 7ffabbfff700 1 -- 172.21.15.157:0/1405543460 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ffac4018030 con 0x7ffabc003fe0 2023-10-06T04:07:19.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.990+0000 7ffad37fe700 1 --2- 172.21.15.157:0/1405543460 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ffac0066f60 0x7ffac0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:19.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.991+0000 7ffad37fe700 1 --2- 172.21.15.157:0/1405543460 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ffac0066f60 0x7ffac0069410 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ffac80091e0 tx=0x7ffac801a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:19.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:19.994+0000 7ffabbfff700 1 -- 172.21.15.157:0/1405543460 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffac40522f0 con 0x7ffabc003fe0 2023-10-06T04:07:20.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:20.224+0000 7ffada6c4700 1 -- 172.21.15.157:0/1405543460 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffad40642a0 con 0x7ffabc003fe0 2023-10-06T04:07:20.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:20.225+0000 7ffabbfff700 1 -- 172.21.15.157:0/1405543460 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+6148 (secure 0 0 0) 0x7ffac4055910 con 0x7ffabc003fe0 2023-10-06T04:07:20.227 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:07:20.228 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5698,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:06:03.836756+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:07:20.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:20.227+0000 7ffada6c4700 1 -- 172.21.15.157:0/1405543460 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ffac0066f60 msgr2=0x7ffac0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:20.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:20.227+0000 7ffada6c4700 1 --2- 172.21.15.157:0/1405543460 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ffac0066f60 0x7ffac0069410 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ffac80091e0 tx=0x7ffac801a000 comp rx=0 tx=0).stop 2023-10-06T04:07:20.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:20.227+0000 7ffada6c4700 1 -- 172.21.15.157:0/1405543460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffabc003fe0 msgr2=0x7ffad406a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:20.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:20.227+0000 7ffada6c4700 1 --2- 172.21.15.157:0/1405543460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffabc003fe0 0x7ffad406a650 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ffac4011d70 tx=0x7ffac40091a0 comp rx=0 tx=0).stop 2023-10-06T04:07:20.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:20.227+0000 7ffada6c4700 1 -- 172.21.15.157:0/1405543460 shutdown_connections 2023-10-06T04:07:20.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:20.227+0000 7ffada6c4700 1 --2- 172.21.15.157:0/1405543460 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ffac0066f60 0x7ffac0069410 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:20.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:20.227+0000 7ffada6c4700 1 --2- 172.21.15.157:0/1405543460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffad414ad20 0x7ffad4141130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:20.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:20.227+0000 7ffada6c4700 1 --2- 172.21.15.157:0/1405543460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffabc003fe0 0x7ffad406a650 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:20.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:20.227+0000 7ffada6c4700 1 --2- 172.21.15.157:0/1405543460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffad41578f0 0x7ffad4142ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:20.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:20.227+0000 7ffada6c4700 1 -- 172.21.15.157:0/1405543460 >> 172.21.15.157:0/1405543460 conn(0x7ffad40c8b20 msgr2=0x7ffad406d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:20.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:20.227+0000 7ffada6c4700 1 -- 172.21.15.157:0/1405543460 shutdown_connections 2023-10-06T04:07:20.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:20.227+0000 7ffada6c4700 1 -- 172.21.15.157:0/1405543460 wait complete. 2023-10-06T04:07:20.231 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 71 2023-10-06T04:07:20.242 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-10-06T04:07:22.243 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:07:22.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.366+0000 7fede2c09700 1 Processor -- start 2023-10-06T04:07:22.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.366+0000 7fede2c09700 1 -- start start 2023-10-06T04:07:22.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.367+0000 7fede2c09700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7feddc06dc40 0x7feddc14fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:22.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.367+0000 7fede2c09700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feddc06c1f0 0x7feddc150240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:22.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.367+0000 7fede2c09700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feddc0c9780 con 0x7feddc06cbb0 2023-10-06T04:07:22.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.367+0000 7fede2c09700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feddc0c8c40 con 0x7feddc06c1f0 2023-10-06T04:07:22.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.367+0000 7fede2c09700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feddc06e0c0 con 0x7feddc06dc40 2023-10-06T04:07:22.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.367+0000 7fede11a6700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feddc06c1f0 0x7feddc150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:22.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.367+0000 7fede09a5700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7feddc06cbb0 0x7feddc06d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41560/0 (socket says 172.21.15.157:41560) 2023-10-06T04:07:22.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.367+0000 7fede09a5700 1 -- 172.21.15.157:0/1332807551 learned_addr learned my addr 172.21.15.157:0/1332807551 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:07:22.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.367+0000 7feddbfff700 1 --2- 172.21.15.157:0/1332807551 >> v2:172.21.15.175:3300/0 conn(0x7feddc06dc40 0x7feddc14fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:22.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.367+0000 7fede11a6700 1 -- 172.21.15.157:0/1332807551 >> v2:172.21.15.175:3300/0 conn(0x7feddc06dc40 msgr2=0x7feddc14fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:22.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.367+0000 7fede11a6700 1 --2- 172.21.15.157:0/1332807551 >> v2:172.21.15.175:3300/0 conn(0x7feddc06dc40 0x7feddc14fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:22.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.367+0000 7fede11a6700 1 -- 172.21.15.157:0/1332807551 >> v1:172.21.15.157:6789/0 conn(0x7feddc06cbb0 legacy=0x7feddc06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:22.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.367+0000 7feddb7fe700 1 -- 172.21.15.157:0/1332807551 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3713229563 0 0) 0x7feddc0c9780 con 0x7feddc06cbb0 2023-10-06T04:07:22.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.367+0000 7fede11a6700 1 -- 172.21.15.157:0/1332807551 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feddc0c8e50 con 0x7feddc06c1f0 2023-10-06T04:07:22.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.367+0000 7fede11a6700 1 --2- 172.21.15.157:0/1332807551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feddc06c1f0 0x7feddc150240 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7feddc157070 tx=0x7fedd4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cef4ffcd31cfc0f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:22.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.367+0000 7feddb7fe700 1 -- 172.21.15.157:0/1332807551 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd400e070 con 0x7feddc06c1f0 2023-10-06T04:07:22.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.368+0000 7feddb7fe700 1 -- 172.21.15.157:0/1332807551 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd400bcc0 con 0x7feddc06c1f0 2023-10-06T04:07:22.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.368+0000 7feddb7fe700 1 -- 172.21.15.157:0/1332807551 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd4012680 con 0x7feddc06c1f0 2023-10-06T04:07:22.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.368+0000 7fede2c09700 1 -- 172.21.15.157:0/1332807551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feddc06c1f0 msgr2=0x7feddc150240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:22.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.368+0000 7fede2c09700 1 --2- 172.21.15.157:0/1332807551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feddc06c1f0 0x7feddc150240 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7feddc157070 tx=0x7fedd4009780 comp rx=0 tx=0).stop 2023-10-06T04:07:22.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.368+0000 7fede2c09700 1 -- 172.21.15.157:0/1332807551 shutdown_connections 2023-10-06T04:07:22.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.368+0000 7fede2c09700 1 --2- 172.21.15.157:0/1332807551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feddc06c1f0 0x7feddc150240 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:22.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.368+0000 7fede2c09700 1 --2- 172.21.15.157:0/1332807551 >> v2:172.21.15.175:3300/0 conn(0x7feddc06dc40 0x7feddc14fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:22.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.368+0000 7fede2c09700 1 -- 172.21.15.157:0/1332807551 >> 172.21.15.157:0/1332807551 conn(0x7feddc06f230 msgr2=0x7feddc06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:22.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.368+0000 7fede2c09700 1 -- 172.21.15.157:0/1332807551 shutdown_connections 2023-10-06T04:07:22.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.368+0000 7fede2c09700 1 -- 172.21.15.157:0/1332807551 wait complete. 2023-10-06T04:07:22.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.369+0000 7fede2c09700 1 Processor -- start 2023-10-06T04:07:22.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.369+0000 7fede2c09700 1 -- start start 2023-10-06T04:07:22.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.370+0000 7fede2c09700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feddc06c1f0 0x7feddc147a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:22.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.370+0000 7fede2c09700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feddc06cbb0 0x7feddc150850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:22.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.370+0000 7fede2c09700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feddc153590 0x7feddc145ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:22.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.370+0000 7fede2c09700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feddc147590 con 0x7feddc06cbb0 2023-10-06T04:07:22.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.370+0000 7fede2c09700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feddc166960 con 0x7feddc06c1f0 2023-10-06T04:07:22.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.370+0000 7fede2c09700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feddc152760 con 0x7feddc153590 2023-10-06T04:07:22.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.370+0000 7feddbfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feddc06cbb0 0x7feddc150850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:22.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.370+0000 7feddbfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feddc06cbb0 0x7feddc150850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42432/0 (socket says 172.21.15.157:42432) 2023-10-06T04:07:22.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.370+0000 7feddbfff700 1 -- 172.21.15.157:0/3716246686 learned_addr learned my addr 172.21.15.157:0/3716246686 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:07:22.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.370+0000 7fede09a5700 1 --2- 172.21.15.157:0/3716246686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feddc06c1f0 0x7feddc147a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:22.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.370+0000 7fede11a6700 1 --2- 172.21.15.157:0/3716246686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feddc153590 0x7feddc145ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:22.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.370+0000 7feddbfff700 1 -- 172.21.15.157:0/3716246686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feddc153590 msgr2=0x7feddc145ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:22.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.370+0000 7feddbfff700 1 --2- 172.21.15.157:0/3716246686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feddc153590 0x7feddc145ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:22.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.370+0000 7feddbfff700 1 -- 172.21.15.157:0/3716246686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feddc06c1f0 msgr2=0x7feddc147a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:22.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.371+0000 7feddbfff700 1 --2- 172.21.15.157:0/3716246686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feddc06c1f0 0x7feddc147a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:22.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.371+0000 7feddbfff700 1 -- 172.21.15.157:0/3716246686 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd4009a20 con 0x7feddc06cbb0 2023-10-06T04:07:22.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.371+0000 7feddbfff700 1 --2- 172.21.15.157:0/3716246686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feddc06cbb0 0x7feddc150850 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fedc800c8b0 tx=0x7fedc8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:22.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.371+0000 7fedd8ff9700 1 -- 172.21.15.157:0/3716246686 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc8011600 con 0x7feddc06cbb0 2023-10-06T04:07:22.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.371+0000 7fedd8ff9700 1 -- 172.21.15.157:0/3716246686 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc8011780 con 0x7feddc06cbb0 2023-10-06T04:07:22.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.371+0000 7fedd8ff9700 1 -- 172.21.15.157:0/3716246686 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc8011b40 con 0x7feddc06cbb0 2023-10-06T04:07:22.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.371+0000 7fede2c09700 1 -- 172.21.15.157:0/3716246686 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feddc1666c0 con 0x7feddc06cbb0 2023-10-06T04:07:22.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.371+0000 7fede2c09700 1 -- 172.21.15.157:0/3716246686 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feddc146f40 con 0x7feddc06cbb0 2023-10-06T04:07:22.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.372+0000 7fede2c09700 1 -- 172.21.15.157:0/3716246686 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feddc147740 con 0x7feddc06cbb0 2023-10-06T04:07:22.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.372+0000 7fedd8ff9700 1 -- 172.21.15.157:0/3716246686 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fedc8021350 con 0x7feddc06cbb0 2023-10-06T04:07:22.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.373+0000 7fedd8ff9700 1 --2- 172.21.15.157:0/3716246686 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fedc0066f30 0x7fedc00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:22.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.373+0000 7fedd8ff9700 1 -- 172.21.15.157:0/3716246686 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fedc80844f0 con 0x7feddc06cbb0 2023-10-06T04:07:22.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.373+0000 7fede09a5700 1 --2- 172.21.15.157:0/3716246686 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fedc0066f30 0x7fedc00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:22.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.373+0000 7fede09a5700 1 --2- 172.21.15.157:0/3716246686 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fedc0066f30 0x7fedc00693e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fedd0007fd0 tx=0x7fedd0007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:22.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.377+0000 7fedd8ff9700 1 -- 172.21.15.157:0/3716246686 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fedc8054e60 con 0x7feddc06cbb0 2023-10-06T04:07:22.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.600+0000 7fede2c09700 1 -- 172.21.15.157:0/3716246686 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feddc0c9130 con 0x7feddc06cbb0 2023-10-06T04:07:22.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.601+0000 7fedd8ff9700 1 -- 172.21.15.157:0/3716246686 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+6148 (secure 0 0 0) 0x7fedc80549f0 con 0x7feddc06cbb0 2023-10-06T04:07:22.604 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:07:22.604 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5698,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:06:03.836756+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:07:22.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.603+0000 7fede2c09700 1 -- 172.21.15.157:0/3716246686 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fedc0066f30 msgr2=0x7fedc00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:22.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.603+0000 7fede2c09700 1 --2- 172.21.15.157:0/3716246686 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fedc0066f30 0x7fedc00693e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fedd0007fd0 tx=0x7fedd0007d30 comp rx=0 tx=0).stop 2023-10-06T04:07:22.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.603+0000 7fede2c09700 1 -- 172.21.15.157:0/3716246686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feddc06cbb0 msgr2=0x7feddc150850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:22.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.603+0000 7fede2c09700 1 --2- 172.21.15.157:0/3716246686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feddc06cbb0 0x7feddc150850 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fedc800c8b0 tx=0x7fedc8008040 comp rx=0 tx=0).stop 2023-10-06T04:07:22.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.603+0000 7fede2c09700 1 -- 172.21.15.157:0/3716246686 shutdown_connections 2023-10-06T04:07:22.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.603+0000 7fede2c09700 1 --2- 172.21.15.157:0/3716246686 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fedc0066f30 0x7fedc00693e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:22.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.603+0000 7fede2c09700 1 --2- 172.21.15.157:0/3716246686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feddc06c1f0 0x7feddc147a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:22.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.603+0000 7fede2c09700 1 --2- 172.21.15.157:0/3716246686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feddc153590 0x7feddc145ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:22.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.603+0000 7fede2c09700 1 --2- 172.21.15.157:0/3716246686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feddc06cbb0 0x7feddc150850 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:22.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.603+0000 7fede2c09700 1 -- 172.21.15.157:0/3716246686 >> 172.21.15.157:0/3716246686 conn(0x7feddc06f230 msgr2=0x7feddc0c2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:22.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.603+0000 7fede2c09700 1 -- 172.21.15.157:0/3716246686 shutdown_connections 2023-10-06T04:07:22.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:22.603+0000 7fede2c09700 1 -- 172.21.15.157:0/3716246686 wait complete. 2023-10-06T04:07:22.607 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 71 2023-10-06T04:07:22.617 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-10-06T04:07:24.618 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:07:24.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.744+0000 7f9f302ac700 1 Processor -- start 2023-10-06T04:07:24.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.744+0000 7f9f302ac700 1 -- start start 2023-10-06T04:07:24.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.744+0000 7f9f302ac700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9f2806cbb0 0x7f9f2806d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:24.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.744+0000 7f9f302ac700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9f2806c1f0 0x7f9f2806da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:24.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.745+0000 7f9f302ac700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f280c9780 con 0x7f9f2806cf80 2023-10-06T04:07:24.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.745+0000 7f9f302ac700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f280c8c40 con 0x7f9f2806cbb0 2023-10-06T04:07:24.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.745+0000 7f9f302ac700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f2806e0c0 con 0x7f9f2806c1f0 2023-10-06T04:07:24.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.745+0000 7f9f2e048700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9f2806cbb0 0x7f9f2806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:24.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.745+0000 7f9f2e048700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9f2806cbb0 0x7f9f2806d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36380/0 (socket says 172.21.15.157:36380) 2023-10-06T04:07:24.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.745+0000 7f9f2e048700 1 -- 172.21.15.157:0/3200103905 learned_addr learned my addr 172.21.15.157:0/3200103905 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:07:24.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.745+0000 7f9f2d847700 1 --2- 172.21.15.157:0/3200103905 >> v2:172.21.15.175:3300/0 conn(0x7f9f2806c1f0 0x7f9f2806da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:24.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.745+0000 7f9f2e048700 1 -- 172.21.15.157:0/3200103905 >> v2:172.21.15.175:3300/0 conn(0x7f9f2806c1f0 msgr2=0x7f9f2806da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:24.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.745+0000 7f9f2e048700 1 --2- 172.21.15.157:0/3200103905 >> v2:172.21.15.175:3300/0 conn(0x7f9f2806c1f0 0x7f9f2806da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:24.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.745+0000 7f9f2e048700 1 -- 172.21.15.157:0/3200103905 >> v1:172.21.15.175:6789/0 conn(0x7f9f2806cf80 legacy=0x7f9f2814fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:24.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.745+0000 7f9f2e048700 1 -- 172.21.15.157:0/3200103905 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f280c8e50 con 0x7f9f2806cbb0 2023-10-06T04:07:24.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.745+0000 7f9f2d847700 1 --2- 172.21.15.157:0/3200103905 >> v2:172.21.15.175:3300/0 conn(0x7f9f2806c1f0 0x7f9f2806da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:07:24.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.745+0000 7f9f2e048700 1 --2- 172.21.15.157:0/3200103905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f2806cbb0 0x7f9f2806d530 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9f28157070 tx=0x7f9f18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3764c24c92b235 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:24.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.745+0000 7f9f2d046700 1 -- 172.21.15.157:0/3200103905 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1800b8f0 con 0x7f9f2806cbb0 2023-10-06T04:07:24.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.745+0000 7f9f2d046700 1 -- 172.21.15.157:0/3200103905 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1800ba50 con 0x7f9f2806cbb0 2023-10-06T04:07:24.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.745+0000 7f9f2d046700 1 -- 172.21.15.157:0/3200103905 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1800bdf0 con 0x7f9f2806cbb0 2023-10-06T04:07:24.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.746+0000 7f9f302ac700 1 -- 172.21.15.157:0/3200103905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f2806cbb0 msgr2=0x7f9f2806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:24.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.746+0000 7f9f302ac700 1 --2- 172.21.15.157:0/3200103905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f2806cbb0 0x7f9f2806d530 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9f28157070 tx=0x7f9f18009250 comp rx=0 tx=0).stop 2023-10-06T04:07:24.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.746+0000 7f9f302ac700 1 -- 172.21.15.157:0/3200103905 shutdown_connections 2023-10-06T04:07:24.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.746+0000 7f9f302ac700 1 --2- 172.21.15.157:0/3200103905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f2806cbb0 0x7f9f2806d530 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:24.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.746+0000 7f9f302ac700 1 --2- 172.21.15.157:0/3200103905 >> v2:172.21.15.175:3300/0 conn(0x7f9f2806c1f0 0x7f9f2806da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:24.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.746+0000 7f9f302ac700 1 -- 172.21.15.157:0/3200103905 >> 172.21.15.157:0/3200103905 conn(0x7f9f2806f230 msgr2=0x7f9f2806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:24.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.746+0000 7f9f302ac700 1 -- 172.21.15.157:0/3200103905 shutdown_connections 2023-10-06T04:07:24.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.746+0000 7f9f302ac700 1 -- 172.21.15.157:0/3200103905 wait complete. 2023-10-06T04:07:24.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.747+0000 7f9f302ac700 1 Processor -- start 2023-10-06T04:07:24.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.747+0000 7f9f302ac700 1 -- start start 2023-10-06T04:07:24.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.748+0000 7f9f302ac700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f2806c1f0 0x7f9f280bfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:24.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.748+0000 7f9f302ac700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f2806cbb0 0x7f9f281576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:24.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.748+0000 7f9f302ac700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f2806cf80 0x7f9f28158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:24.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.748+0000 7f9f302ac700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f280c18e0 con 0x7f9f2806c1f0 2023-10-06T04:07:24.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.748+0000 7f9f302ac700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f280c1bc0 con 0x7f9f2806cf80 2023-10-06T04:07:24.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.748+0000 7f9f302ac700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f280c1ea0 con 0x7f9f2806cbb0 2023-10-06T04:07:24.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.748+0000 7f9f2e048700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f2806c1f0 0x7f9f280bfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:24.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.748+0000 7f9f2e048700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f2806c1f0 0x7f9f280bfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36388/0 (socket says 172.21.15.157:36388) 2023-10-06T04:07:24.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.748+0000 7f9f2e048700 1 -- 172.21.15.157:0/3796055026 learned_addr learned my addr 172.21.15.157:0/3796055026 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:07:24.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.748+0000 7f9f2d847700 1 --2- 172.21.15.157:0/3796055026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f2806cbb0 0x7f9f281576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:24.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.748+0000 7f9f2e849700 1 --2- 172.21.15.157:0/3796055026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f2806cf80 0x7f9f28158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:24.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.748+0000 7f9f2e048700 1 -- 172.21.15.157:0/3796055026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f2806cbb0 msgr2=0x7f9f281576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:24.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.748+0000 7f9f2e048700 1 --2- 172.21.15.157:0/3796055026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f2806cbb0 0x7f9f281576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:24.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.748+0000 7f9f2e048700 1 -- 172.21.15.157:0/3796055026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f2806cf80 msgr2=0x7f9f28158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:24.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.748+0000 7f9f2e048700 1 --2- 172.21.15.157:0/3796055026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f2806cf80 0x7f9f28158f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:24.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.748+0000 7f9f2e048700 1 -- 172.21.15.157:0/3796055026 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f180094f0 con 0x7f9f2806c1f0 2023-10-06T04:07:24.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.748+0000 7f9f2e048700 1 --2- 172.21.15.157:0/3796055026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f2806c1f0 0x7f9f280bfc70 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009290 tx=0x7f9f18017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:24.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.749+0000 7f9f267fc700 1 -- 172.21.15.157:0/3796055026 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18011e60 con 0x7f9f2806c1f0 2023-10-06T04:07:24.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.749+0000 7f9f267fc700 1 -- 172.21.15.157:0/3796055026 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1801f070 con 0x7f9f2806c1f0 2023-10-06T04:07:24.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.749+0000 7f9f302ac700 1 -- 172.21.15.157:0/3796055026 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f280c2180 con 0x7f9f2806c1f0 2023-10-06T04:07:24.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.749+0000 7f9f267fc700 1 -- 172.21.15.157:0/3796055026 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1801a870 con 0x7f9f2806c1f0 2023-10-06T04:07:24.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.749+0000 7f9f302ac700 1 -- 172.21.15.157:0/3796055026 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f28158a40 con 0x7f9f2806c1f0 2023-10-06T04:07:24.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.750+0000 7f9f302ac700 1 -- 172.21.15.157:0/3796055026 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f28089ca0 con 0x7f9f2806c1f0 2023-10-06T04:07:24.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.750+0000 7f9f267fc700 1 -- 172.21.15.157:0/3796055026 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9f1801abe0 con 0x7f9f2806c1f0 2023-10-06T04:07:24.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.750+0000 7f9f267fc700 1 --2- 172.21.15.157:0/3796055026 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9f14066f30 0x7f9f140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:24.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.750+0000 7f9f267fc700 1 -- 172.21.15.157:0/3796055026 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9f18021070 con 0x7f9f2806c1f0 2023-10-06T04:07:24.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.751+0000 7f9f2d847700 1 --2- 172.21.15.157:0/3796055026 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9f14066f30 0x7f9f140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:24.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.751+0000 7f9f2d847700 1 --2- 172.21.15.157:0/3796055026 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9f14066f30 0x7f9f140693e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c006f00 tx=0x7f9f1c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:24.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.755+0000 7f9f267fc700 1 -- 172.21.15.157:0/3796055026 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f18053890 con 0x7f9f2806c1f0 2023-10-06T04:07:24.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.986+0000 7f9f302ac700 1 -- 172.21.15.157:0/3796055026 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f280c9130 con 0x7f9f2806c1f0 2023-10-06T04:07:24.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.987+0000 7f9f267fc700 1 -- 172.21.15.157:0/3796055026 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+6148 (secure 0 0 0) 0x7f9f18056eb0 con 0x7f9f2806c1f0 2023-10-06T04:07:24.989 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:07:24.990 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5698,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:06:03.836756+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:07:24.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.989+0000 7f9f302ac700 1 -- 172.21.15.157:0/3796055026 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9f14066f30 msgr2=0x7f9f140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:24.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.989+0000 7f9f302ac700 1 --2- 172.21.15.157:0/3796055026 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9f14066f30 0x7f9f140693e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c006f00 tx=0x7f9f1c008040 comp rx=0 tx=0).stop 2023-10-06T04:07:24.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.989+0000 7f9f302ac700 1 -- 172.21.15.157:0/3796055026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f2806c1f0 msgr2=0x7f9f280bfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:24.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.989+0000 7f9f302ac700 1 --2- 172.21.15.157:0/3796055026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f2806c1f0 0x7f9f280bfc70 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009290 tx=0x7f9f18017000 comp rx=0 tx=0).stop 2023-10-06T04:07:24.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.989+0000 7f9f302ac700 1 -- 172.21.15.157:0/3796055026 shutdown_connections 2023-10-06T04:07:24.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.989+0000 7f9f302ac700 1 --2- 172.21.15.157:0/3796055026 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9f14066f30 0x7f9f140693e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:24.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.989+0000 7f9f302ac700 1 --2- 172.21.15.157:0/3796055026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f2806cf80 0x7f9f28158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:24.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.989+0000 7f9f302ac700 1 --2- 172.21.15.157:0/3796055026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f2806c1f0 0x7f9f280bfc70 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:24.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.989+0000 7f9f302ac700 1 --2- 172.21.15.157:0/3796055026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f2806cbb0 0x7f9f281576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:24.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.989+0000 7f9f302ac700 1 -- 172.21.15.157:0/3796055026 >> 172.21.15.157:0/3796055026 conn(0x7f9f2806f230 msgr2=0x7f9f280af1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:24.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.989+0000 7f9f302ac700 1 -- 172.21.15.157:0/3796055026 shutdown_connections 2023-10-06T04:07:24.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:24.989+0000 7f9f302ac700 1 -- 172.21.15.157:0/3796055026 wait complete. 2023-10-06T04:07:24.993 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 71 2023-10-06T04:07:25.004 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-10-06T04:07:27.004 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:07:27.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.117+0000 7f3a96355700 1 Processor -- start 2023-10-06T04:07:27.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.117+0000 7f3a96355700 1 -- start start 2023-10-06T04:07:27.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.118+0000 7f3a96355700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3a901547f0 0x7f3a900c29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:27.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.118+0000 7f3a96355700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3a900631f0 0x7f3a9014d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:27.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.118+0000 7f3a96355700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a90153be0 con 0x7f3a900c00c0 2023-10-06T04:07:27.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.118+0000 7f3a96355700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a900bf550 con 0x7f3a901547f0 2023-10-06T04:07:27.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.118+0000 7f3a96355700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a900c3630 con 0x7f3a900631f0 2023-10-06T04:07:27.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.118+0000 7f3a8f7fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3a901547f0 0x7f3a900c29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:27.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.118+0000 7f3a8f7fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3a901547f0 0x7f3a900c29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36390/0 (socket says 172.21.15.157:36390) 2023-10-06T04:07:27.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.118+0000 7f3a8f7fe700 1 -- 172.21.15.157:0/2375339131 learned_addr learned my addr 172.21.15.157:0/2375339131 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:07:27.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.118+0000 7f3a8ffff700 1 --1- 172.21.15.157:0/2375339131 >> v1:172.21.15.157:6789/0 conn(0x7f3a900c00c0 0x7f3a901540e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43288/0 (socket says 172.21.15.157:43288) 2023-10-06T04:07:27.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.119+0000 7f3a948f2700 1 --2- 172.21.15.157:0/2375339131 >> v2:172.21.15.175:3300/0 conn(0x7f3a900631f0 0x7f3a9014d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:27.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.119+0000 7f3a8f7fe700 1 -- 172.21.15.157:0/2375339131 >> v2:172.21.15.175:3300/0 conn(0x7f3a900631f0 msgr2=0x7f3a9014d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:27.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.119+0000 7f3a8f7fe700 1 --2- 172.21.15.157:0/2375339131 >> v2:172.21.15.175:3300/0 conn(0x7f3a900631f0 0x7f3a9014d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:27.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.119+0000 7f3a8f7fe700 1 -- 172.21.15.157:0/2375339131 >> v1:172.21.15.157:6789/0 conn(0x7f3a900c00c0 legacy=0x7f3a901540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:27.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.119+0000 7f3a8f7fe700 1 -- 172.21.15.157:0/2375339131 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a900c3910 con 0x7f3a901547f0 2023-10-06T04:07:27.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.119+0000 7f3a8effd700 1 -- 172.21.15.157:0/2375339131 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1303096756 0 0) 0x7f3a90153be0 con 0x7f3a900c00c0 2023-10-06T04:07:27.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.119+0000 7f3a8f7fe700 1 --2- 172.21.15.157:0/2375339131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a901547f0 0x7f3a900c29f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f3a900c5470 tx=0x7f3a84009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1965e440fb41997e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:27.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.119+0000 7f3a8effd700 1 -- 172.21.15.157:0/2375339131 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8400b860 con 0x7f3a901547f0 2023-10-06T04:07:27.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.119+0000 7f3a8effd700 1 -- 172.21.15.157:0/2375339131 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8400b9c0 con 0x7f3a901547f0 2023-10-06T04:07:27.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.119+0000 7f3a8effd700 1 -- 172.21.15.157:0/2375339131 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8400bd60 con 0x7f3a901547f0 2023-10-06T04:07:27.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.120+0000 7f3a96355700 1 -- 172.21.15.157:0/2375339131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a901547f0 msgr2=0x7f3a900c29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:27.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.120+0000 7f3a96355700 1 --2- 172.21.15.157:0/2375339131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a901547f0 0x7f3a900c29f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f3a900c5470 tx=0x7f3a84009250 comp rx=0 tx=0).stop 2023-10-06T04:07:27.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.120+0000 7f3a96355700 1 -- 172.21.15.157:0/2375339131 shutdown_connections 2023-10-06T04:07:27.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.120+0000 7f3a96355700 1 --2- 172.21.15.157:0/2375339131 >> v2:172.21.15.175:3300/0 conn(0x7f3a900631f0 0x7f3a9014d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:27.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.120+0000 7f3a96355700 1 --2- 172.21.15.157:0/2375339131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a901547f0 0x7f3a900c29f0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:27.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.120+0000 7f3a96355700 1 -- 172.21.15.157:0/2375339131 >> 172.21.15.157:0/2375339131 conn(0x7f3a900c8b20 msgr2=0x7f3a900c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:27.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.120+0000 7f3a96355700 1 -- 172.21.15.157:0/2375339131 shutdown_connections 2023-10-06T04:07:27.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.120+0000 7f3a96355700 1 -- 172.21.15.157:0/2375339131 wait complete. 2023-10-06T04:07:27.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.121+0000 7f3a96355700 1 Processor -- start 2023-10-06T04:07:27.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.121+0000 7f3a96355700 1 -- start start 2023-10-06T04:07:27.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.121+0000 7f3a96355700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a900631f0 0x7f3a900c7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:27.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.121+0000 7f3a96355700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a900c00c0 0x7f3a90152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:27.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.121+0000 7f3a96355700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a901547f0 0x7f3a90152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:27.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.121+0000 7f3a96355700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a90167ea0 con 0x7f3a900c00c0 2023-10-06T04:07:27.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.121+0000 7f3a96355700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a901639e0 con 0x7f3a901547f0 2023-10-06T04:07:27.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.121+0000 7f3a96355700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a90164520 con 0x7f3a900631f0 2023-10-06T04:07:27.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.122+0000 7f3a8f7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a900c00c0 0x7f3a90152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:27.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.122+0000 7f3a8f7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a900c00c0 0x7f3a90152810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36400/0 (socket says 172.21.15.157:36400) 2023-10-06T04:07:27.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.122+0000 7f3a8f7fe700 1 -- 172.21.15.157:0/2437199112 learned_addr learned my addr 172.21.15.157:0/2437199112 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:07:27.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.122+0000 7f3a8ffff700 1 --2- 172.21.15.157:0/2437199112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a900631f0 0x7f3a900c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:27.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.122+0000 7f3a948f2700 1 --2- 172.21.15.157:0/2437199112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a901547f0 0x7f3a90152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:27.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.122+0000 7f3a8f7fe700 1 -- 172.21.15.157:0/2437199112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a900631f0 msgr2=0x7f3a900c7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:27.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.122+0000 7f3a8f7fe700 1 --2- 172.21.15.157:0/2437199112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a900631f0 0x7f3a900c7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:27.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.122+0000 7f3a8f7fe700 1 -- 172.21.15.157:0/2437199112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a901547f0 msgr2=0x7f3a90152d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:27.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.122+0000 7f3a8f7fe700 1 --2- 172.21.15.157:0/2437199112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a901547f0 0x7f3a90152d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:27.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.122+0000 7f3a8f7fe700 1 -- 172.21.15.157:0/2437199112 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a840094f0 con 0x7f3a900c00c0 2023-10-06T04:07:27.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.122+0000 7f3a8f7fe700 1 --2- 172.21.15.157:0/2437199112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a900c00c0 0x7f3a90152810 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009290 tx=0x7f3a84017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:27.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.122+0000 7f3a77fff700 1 -- 172.21.15.157:0/2437199112 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84026450 con 0x7f3a900c00c0 2023-10-06T04:07:27.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.122+0000 7f3a77fff700 1 -- 172.21.15.157:0/2437199112 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a840265b0 con 0x7f3a900c00c0 2023-10-06T04:07:27.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.122+0000 7f3a96355700 1 -- 172.21.15.157:0/2437199112 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a90167920 con 0x7f3a900c00c0 2023-10-06T04:07:27.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.122+0000 7f3a77fff700 1 -- 172.21.15.157:0/2437199112 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84026950 con 0x7f3a900c00c0 2023-10-06T04:07:27.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.122+0000 7f3a96355700 1 -- 172.21.15.157:0/2437199112 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a90163bf0 con 0x7f3a900c00c0 2023-10-06T04:07:27.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.123+0000 7f3a96355700 1 -- 172.21.15.157:0/2437199112 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a900642a0 con 0x7f3a900c00c0 2023-10-06T04:07:27.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.124+0000 7f3a77fff700 1 -- 172.21.15.157:0/2437199112 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3a840193b0 con 0x7f3a900c00c0 2023-10-06T04:07:27.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.124+0000 7f3a77fff700 1 --2- 172.21.15.157:0/2437199112 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3a7c066f60 0x7f3a7c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:27.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.124+0000 7f3a77fff700 1 -- 172.21.15.157:0/2437199112 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3a84085f40 con 0x7f3a900c00c0 2023-10-06T04:07:27.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.124+0000 7f3a8ffff700 1 --2- 172.21.15.157:0/2437199112 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3a7c066f60 0x7f3a7c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:27.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.125+0000 7f3a8ffff700 1 --2- 172.21.15.157:0/2437199112 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3a7c066f60 0x7f3a7c069410 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3a80007fd0 tx=0x7f3a80007d50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:27.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.129+0000 7f3a77fff700 1 -- 172.21.15.157:0/2437199112 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3a84052f40 con 0x7f3a900c00c0 2023-10-06T04:07:27.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.356+0000 7f3a96355700 1 -- 172.21.15.157:0/2437199112 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a9014d060 con 0x7f3a900c00c0 2023-10-06T04:07:27.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.356+0000 7f3a77fff700 1 -- 172.21.15.157:0/2437199112 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+6148 (secure 0 0 0) 0x7f3a84056560 con 0x7f3a900c00c0 2023-10-06T04:07:27.358 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:07:27.358 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5698,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:06:03.836756+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:07:27.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.358+0000 7f3a96355700 1 -- 172.21.15.157:0/2437199112 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3a7c066f60 msgr2=0x7f3a7c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:27.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.358+0000 7f3a96355700 1 --2- 172.21.15.157:0/2437199112 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3a7c066f60 0x7f3a7c069410 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3a80007fd0 tx=0x7f3a80007d50 comp rx=0 tx=0).stop 2023-10-06T04:07:27.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.358+0000 7f3a96355700 1 -- 172.21.15.157:0/2437199112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a900c00c0 msgr2=0x7f3a90152810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:27.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.358+0000 7f3a96355700 1 --2- 172.21.15.157:0/2437199112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a900c00c0 0x7f3a90152810 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009290 tx=0x7f3a84017000 comp rx=0 tx=0).stop 2023-10-06T04:07:27.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.358+0000 7f3a96355700 1 -- 172.21.15.157:0/2437199112 shutdown_connections 2023-10-06T04:07:27.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.358+0000 7f3a96355700 1 --2- 172.21.15.157:0/2437199112 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3a7c066f60 0x7f3a7c069410 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:27.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.358+0000 7f3a96355700 1 --2- 172.21.15.157:0/2437199112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a901547f0 0x7f3a90152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:27.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.358+0000 7f3a96355700 1 --2- 172.21.15.157:0/2437199112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a900631f0 0x7f3a900c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:27.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.358+0000 7f3a96355700 1 --2- 172.21.15.157:0/2437199112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a900c00c0 0x7f3a90152810 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:27.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.358+0000 7f3a96355700 1 -- 172.21.15.157:0/2437199112 >> 172.21.15.157:0/2437199112 conn(0x7f3a900c8b20 msgr2=0x7f3a9014c750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:27.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.358+0000 7f3a96355700 1 -- 172.21.15.157:0/2437199112 shutdown_connections 2023-10-06T04:07:27.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:27.359+0000 7f3a96355700 1 -- 172.21.15.157:0/2437199112 wait complete. 2023-10-06T04:07:27.363 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 71 2023-10-06T04:07:27.372 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-10-06T04:07:29.373 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:07:29.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.486+0000 7f1fb12cb700 1 Processor -- start 2023-10-06T04:07:29.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.486+0000 7f1fb12cb700 1 -- start start 2023-10-06T04:07:29.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.487+0000 7f1fb12cb700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fac0b04d0 0x7f1fac157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:29.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.487+0000 7f1fb12cb700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1fac06c2d0 0x7f1fac06dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:29.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.487+0000 7f1fb12cb700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fac06d8d0 con 0x7f1fac0b1e40 2023-10-06T04:07:29.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.487+0000 7f1fb12cb700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fac0af8f0 con 0x7f1fac06c2d0 2023-10-06T04:07:29.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.487+0000 7f1fb12cb700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac15a690 con 0x7f1fac0b04d0 2023-10-06T04:07:29.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.487+0000 7f1fb0aca700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1fac06c2d0 0x7f1fac06dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:29.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.487+0000 7f1fabfff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f1fac0b1e40 0x7f1fac1563b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43304/0 (socket says 172.21.15.157:43304) 2023-10-06T04:07:29.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.487+0000 7f1fabfff700 1 -- 172.21.15.157:0/2709054743 learned_addr learned my addr 172.21.15.157:0/2709054743 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:07:29.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.487+0000 7f1fab7fe700 1 --2- 172.21.15.157:0/2709054743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fac0b04d0 0x7f1fac157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:29.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.487+0000 7f1fb0aca700 1 -- 172.21.15.157:0/2709054743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fac0b04d0 msgr2=0x7f1fac157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:29.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.487+0000 7f1fb0aca700 1 --2- 172.21.15.157:0/2709054743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fac0b04d0 0x7f1fac157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:29.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.488+0000 7f1fb0aca700 1 -- 172.21.15.157:0/2709054743 >> v1:172.21.15.157:6789/0 conn(0x7f1fac0b1e40 legacy=0x7f1fac1563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:29.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.488+0000 7f1fb0aca700 1 -- 172.21.15.157:0/2709054743 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fac06ccc0 con 0x7f1fac06c2d0 2023-10-06T04:07:29.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.488+0000 7f1faaffd700 1 -- 172.21.15.157:0/2709054743 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2547913822 0 0) 0x7f1fac06d8d0 con 0x7f1fac0b1e40 2023-10-06T04:07:29.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.488+0000 7f1fab7fe700 1 --2- 172.21.15.157:0/2709054743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fac0b04d0 0x7f1fac157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:07:29.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.488+0000 7f1fb0aca700 1 --2- 172.21.15.157:0/2709054743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fac06c2d0 0x7f1fac06dd60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4009e60 tx=0x7f1fa4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8943bf2ed86c6748 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:29.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.488+0000 7f1faaffd700 1 -- 172.21.15.157:0/2709054743 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa400e070 con 0x7f1fac06c2d0 2023-10-06T04:07:29.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.488+0000 7f1faaffd700 1 -- 172.21.15.157:0/2709054743 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa400bc30 con 0x7f1fac06c2d0 2023-10-06T04:07:29.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.488+0000 7f1faaffd700 1 -- 172.21.15.157:0/2709054743 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4012430 con 0x7f1fac06c2d0 2023-10-06T04:07:29.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.488+0000 7f1fb12cb700 1 -- 172.21.15.157:0/2709054743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fac06c2d0 msgr2=0x7f1fac06dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:29.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.488+0000 7f1fb12cb700 1 --2- 172.21.15.157:0/2709054743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fac06c2d0 0x7f1fac06dd60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4009e60 tx=0x7f1fa4009780 comp rx=0 tx=0).stop 2023-10-06T04:07:29.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.488+0000 7f1fb12cb700 1 -- 172.21.15.157:0/2709054743 shutdown_connections 2023-10-06T04:07:29.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.488+0000 7f1fb12cb700 1 --2- 172.21.15.157:0/2709054743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fac06c2d0 0x7f1fac06dd60 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:29.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.488+0000 7f1fb12cb700 1 --2- 172.21.15.157:0/2709054743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fac0b04d0 0x7f1fac157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:29.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.488+0000 7f1fb12cb700 1 -- 172.21.15.157:0/2709054743 >> 172.21.15.157:0/2709054743 conn(0x7f1fac0cc540 msgr2=0x7f1fac0cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:29.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.488+0000 7f1fb12cb700 1 -- 172.21.15.157:0/2709054743 shutdown_connections 2023-10-06T04:07:29.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.488+0000 7f1fb12cb700 1 -- 172.21.15.157:0/2709054743 wait complete. 2023-10-06T04:07:29.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.489+0000 7f1fb12cb700 1 Processor -- start 2023-10-06T04:07:29.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.490+0000 7f1fb12cb700 1 -- start start 2023-10-06T04:07:29.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.490+0000 7f1fb12cb700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fac06c2d0 0x7f1fac1531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:29.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.490+0000 7f1fb12cb700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fac0b04d0 0x7f1fac0c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:29.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.490+0000 7f1fb12cb700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fac0b1e40 0x7f1fac0c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:29.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.490+0000 7f1fb12cb700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac0c6fe0 con 0x7f1fac0b1e40 2023-10-06T04:07:29.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.490+0000 7f1fb12cb700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac0c8a40 con 0x7f1fac06c2d0 2023-10-06T04:07:29.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.490+0000 7f1fb12cb700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac150da0 con 0x7f1fac0b04d0 2023-10-06T04:07:29.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.490+0000 7f1fb0aca700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fac0b1e40 0x7f1fac0c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:29.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.491+0000 7f1fb0aca700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fac0b1e40 0x7f1fac0c9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36420/0 (socket says 172.21.15.157:36420) 2023-10-06T04:07:29.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.491+0000 7f1fb0aca700 1 -- 172.21.15.157:0/3520285511 learned_addr learned my addr 172.21.15.157:0/3520285511 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:07:29.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.491+0000 7f1fab7fe700 1 --2- 172.21.15.157:0/3520285511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fac0b04d0 0x7f1fac0c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:29.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.491+0000 7f1fabfff700 1 --2- 172.21.15.157:0/3520285511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fac06c2d0 0x7f1fac1531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:29.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.491+0000 7f1fb0aca700 1 -- 172.21.15.157:0/3520285511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fac0b04d0 msgr2=0x7f1fac0c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:29.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.491+0000 7f1fb0aca700 1 --2- 172.21.15.157:0/3520285511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fac0b04d0 0x7f1fac0c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:29.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.491+0000 7f1fb0aca700 1 -- 172.21.15.157:0/3520285511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fac06c2d0 msgr2=0x7f1fac1531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:29.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.491+0000 7f1fb0aca700 1 --2- 172.21.15.157:0/3520285511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fac06c2d0 0x7f1fac1531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:29.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.491+0000 7f1fb0aca700 1 -- 172.21.15.157:0/3520285511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa4009a20 con 0x7f1fac0b1e40 2023-10-06T04:07:29.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.491+0000 7f1fb0aca700 1 --2- 172.21.15.157:0/3520285511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fac0b1e40 0x7f1fac0c9410 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4012ad0 tx=0x7f1fa4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:29.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.491+0000 7f1f93fff700 1 -- 172.21.15.157:0/3520285511 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4025440 con 0x7f1fac0b1e40 2023-10-06T04:07:29.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.491+0000 7f1f93fff700 1 -- 172.21.15.157:0/3520285511 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa40255a0 con 0x7f1fac0b1e40 2023-10-06T04:07:29.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.491+0000 7f1fb12cb700 1 -- 172.21.15.157:0/3520285511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fac0c6d00 con 0x7f1fac0b1e40 2023-10-06T04:07:29.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.491+0000 7f1f93fff700 1 -- 172.21.15.157:0/3520285511 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4025960 con 0x7f1fac0b1e40 2023-10-06T04:07:29.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.491+0000 7f1fb12cb700 1 -- 172.21.15.157:0/3520285511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fac0c86d0 con 0x7f1fac0b1e40 2023-10-06T04:07:29.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.492+0000 7f1fb12cb700 1 -- 172.21.15.157:0/3520285511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fac0c8bf0 con 0x7f1fac0b1e40 2023-10-06T04:07:29.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.492+0000 7f1f93fff700 1 -- 172.21.15.157:0/3520285511 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1fa4003050 con 0x7f1fac0b1e40 2023-10-06T04:07:29.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.493+0000 7f1f93fff700 1 --2- 172.21.15.157:0/3520285511 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1f98066f60 0x7f1f98069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:29.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.493+0000 7f1f93fff700 1 -- 172.21.15.157:0/3520285511 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1fa40868d0 con 0x7f1fac0b1e40 2023-10-06T04:07:29.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.493+0000 7f1fabfff700 1 --2- 172.21.15.157:0/3520285511 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1f98066f60 0x7f1f98069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:29.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.493+0000 7f1fabfff700 1 --2- 172.21.15.157:0/3520285511 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1f98066f60 0x7f1f98069410 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c0011e0 tx=0x7f1f9c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:29.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.497+0000 7f1f93fff700 1 -- 172.21.15.157:0/3520285511 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1fac0c8bf0 con 0x7f1fac0b1e40 2023-10-06T04:07:29.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.744+0000 7f1fb12cb700 1 -- 172.21.15.157:0/3520285511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1fac0c8bf0 con 0x7f1fac0b1e40 2023-10-06T04:07:29.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.744+0000 7f1f93fff700 1 -- 172.21.15.157:0/3520285511 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+6148 (secure 0 0 0) 0x7f1fac0c8bf0 con 0x7f1fac0b1e40 2023-10-06T04:07:29.746 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:07:29.746 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5698,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:06:03.836756+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5362},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5362":{"gid":5362,"name":"d","rank":1,"incarnation":58,"state":"up:active","state_seq":22,"addr":"172.21.15.175:6837/2388164729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":2388164729},{"type":"v1","addr":"172.21.15.175:6837","nonce":2388164729}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:07:29.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.747+0000 7f1fb12cb700 1 -- 172.21.15.157:0/3520285511 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1f98066f60 msgr2=0x7f1f98069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:29.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.747+0000 7f1fb12cb700 1 --2- 172.21.15.157:0/3520285511 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1f98066f60 0x7f1f98069410 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c0011e0 tx=0x7f1f9c009040 comp rx=0 tx=0).stop 2023-10-06T04:07:29.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.747+0000 7f1fb12cb700 1 -- 172.21.15.157:0/3520285511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fac0b1e40 msgr2=0x7f1fac0c9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:29.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.747+0000 7f1fb12cb700 1 --2- 172.21.15.157:0/3520285511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fac0b1e40 0x7f1fac0c9410 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4012ad0 tx=0x7f1fa4019000 comp rx=0 tx=0).stop 2023-10-06T04:07:29.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.747+0000 7f1fb12cb700 1 -- 172.21.15.157:0/3520285511 shutdown_connections 2023-10-06T04:07:29.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.747+0000 7f1fb12cb700 1 --2- 172.21.15.157:0/3520285511 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1f98066f60 0x7f1f98069410 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:29.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.747+0000 7f1fb12cb700 1 --2- 172.21.15.157:0/3520285511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fac06c2d0 0x7f1fac1531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:29.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.747+0000 7f1fb12cb700 1 --2- 172.21.15.157:0/3520285511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fac0b1e40 0x7f1fac0c9410 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:29.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.747+0000 7f1fb12cb700 1 --2- 172.21.15.157:0/3520285511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fac0b04d0 0x7f1fac0c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:29.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.747+0000 7f1fb12cb700 1 -- 172.21.15.157:0/3520285511 >> 172.21.15.157:0/3520285511 conn(0x7f1fac0cc540 msgr2=0x7f1fac157440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:29.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.747+0000 7f1fb12cb700 1 -- 172.21.15.157:0/3520285511 shutdown_connections 2023-10-06T04:07:29.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:29.747+0000 7f1fb12cb700 1 -- 172.21.15.157:0/3520285511 wait complete. 2023-10-06T04:07:29.750 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 71 2023-10-06T04:07:29.760 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-10-06T04:07:31.761 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:07:31.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.883+0000 7f67ac30a700 1 Processor -- start 2023-10-06T04:07:31.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.883+0000 7f67ac30a700 1 -- start start 2023-10-06T04:07:31.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.884+0000 7f67ac30a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67a40c2cb0 0x7f67a406c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:31.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.884+0000 7f67ac30a700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f67a4068bc0 0x7f67a406a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:31.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.884+0000 7f67ac30a700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f67a40bc850 0x7f67a406aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:31.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.884+0000 7f67ac30a700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a415a3c0 con 0x7f67a40bc850 2023-10-06T04:07:31.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.884+0000 7f67ac30a700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a4155f00 con 0x7f67a4068bc0 2023-10-06T04:07:31.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.884+0000 7f67ac30a700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a4156760 con 0x7f67a40c2cb0 2023-10-06T04:07:31.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.884+0000 7f67aa8a7700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f67a40bc850 0x7f67a406aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:31.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.884+0000 7f67aa8a7700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f67a40bc850 0x7f67a406aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36434/0 (socket says 172.21.15.157:36434) 2023-10-06T04:07:31.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.884+0000 7f67aa8a7700 1 -- 172.21.15.157:0/2956762729 learned_addr learned my addr 172.21.15.157:0/2956762729 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:07:31.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.885+0000 7f67aa0a6700 1 --2- 172.21.15.157:0/2956762729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67a40c2cb0 0x7f67a406c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:31.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.885+0000 7f67a98a5700 1 --2- 172.21.15.157:0/2956762729 >> v2:172.21.15.175:3300/0 conn(0x7f67a4068bc0 0x7f67a406a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:31.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.885+0000 7f67aa8a7700 1 -- 172.21.15.157:0/2956762729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67a40c2cb0 msgr2=0x7f67a406c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:31.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.885+0000 7f67aa8a7700 1 --2- 172.21.15.157:0/2956762729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67a40c2cb0 0x7f67a406c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:31.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.885+0000 7f67aa8a7700 1 -- 172.21.15.157:0/2956762729 >> v2:172.21.15.175:3300/0 conn(0x7f67a4068bc0 msgr2=0x7f67a406a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:31.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.885+0000 7f67aa8a7700 1 --2- 172.21.15.157:0/2956762729 >> v2:172.21.15.175:3300/0 conn(0x7f67a4068bc0 0x7f67a406a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:31.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.885+0000 7f67aa8a7700 1 -- 172.21.15.157:0/2956762729 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a4159e40 con 0x7f67a40bc850 2023-10-06T04:07:31.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.885+0000 7f67a98a5700 1 --2- 172.21.15.157:0/2956762729 >> v2:172.21.15.175:3300/0 conn(0x7f67a4068bc0 0x7f67a406a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:07:31.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.885+0000 7f67aa8a7700 1 --2- 172.21.15.157:0/2956762729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67a40bc850 0x7f67a406aa70 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f67a4067c80 tx=0x7f679c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665c02c32c04469b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:31.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.885+0000 7f67a90a4700 1 -- 172.21.15.157:0/2956762729 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c00e070 con 0x7f67a40bc850 2023-10-06T04:07:31.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.885+0000 7f67a90a4700 1 -- 172.21.15.157:0/2956762729 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679c00ba50 con 0x7f67a40bc850 2023-10-06T04:07:31.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.885+0000 7f67a90a4700 1 -- 172.21.15.157:0/2956762729 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c0092e0 con 0x7f67a40bc850 2023-10-06T04:07:31.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.886+0000 7f67ac30a700 1 -- 172.21.15.157:0/2956762729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67a40bc850 msgr2=0x7f67a406aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:31.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.886+0000 7f67ac30a700 1 --2- 172.21.15.157:0/2956762729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67a40bc850 0x7f67a406aa70 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f67a4067c80 tx=0x7f679c009840 comp rx=0 tx=0).stop 2023-10-06T04:07:31.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.886+0000 7f67ac30a700 1 -- 172.21.15.157:0/2956762729 shutdown_connections 2023-10-06T04:07:31.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.886+0000 7f67ac30a700 1 --2- 172.21.15.157:0/2956762729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67a40bc850 0x7f67a406aa70 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:31.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.886+0000 7f67ac30a700 1 --2- 172.21.15.157:0/2956762729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67a40c2cb0 0x7f67a406c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:31.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.886+0000 7f67ac30a700 1 --2- 172.21.15.157:0/2956762729 >> v2:172.21.15.175:3300/0 conn(0x7f67a4068bc0 0x7f67a406a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:31.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.886+0000 7f67ac30a700 1 -- 172.21.15.157:0/2956762729 >> 172.21.15.157:0/2956762729 conn(0x7f67a406b510 msgr2=0x7f67a406b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:31.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.886+0000 7f67ac30a700 1 -- 172.21.15.157:0/2956762729 shutdown_connections 2023-10-06T04:07:31.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.886+0000 7f67ac30a700 1 -- 172.21.15.157:0/2956762729 wait complete. 2023-10-06T04:07:31.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.887+0000 7f67ac30a700 1 Processor -- start 2023-10-06T04:07:31.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.887+0000 7f67ac30a700 1 -- start start 2023-10-06T04:07:31.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.888+0000 7f67ac30a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67a4068bc0 0x7f67a40b92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:31.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.888+0000 7f67ac30a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a40bc850 0x7f67a41526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:31.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.888+0000 7f67ac30a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67a40c2cb0 0x7f67a4169210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:31.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.888+0000 7f67ac30a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a40b9820 con 0x7f67a4068bc0 2023-10-06T04:07:31.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.888+0000 7f67ac30a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a4151360 con 0x7f67a40bc850 2023-10-06T04:07:31.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.888+0000 7f67ac30a700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a41521c0 con 0x7f67a40c2cb0 2023-10-06T04:07:31.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.888+0000 7f67aa0a6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67a4068bc0 0x7f67a40b92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:31.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.888+0000 7f67aa0a6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67a4068bc0 0x7f67a40b92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36442/0 (socket says 172.21.15.157:36442) 2023-10-06T04:07:31.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.888+0000 7f67aa0a6700 1 -- 172.21.15.157:0/4127111825 learned_addr learned my addr 172.21.15.157:0/4127111825 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:07:31.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.888+0000 7f67aa8a7700 1 --2- 172.21.15.157:0/4127111825 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67a40c2cb0 0x7f67a4169210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:31.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.888+0000 7f67a98a5700 1 --2- 172.21.15.157:0/4127111825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a40bc850 0x7f67a41526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:31.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.888+0000 7f67aa0a6700 1 -- 172.21.15.157:0/4127111825 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67a40c2cb0 msgr2=0x7f67a4169210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:31.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.888+0000 7f67aa0a6700 1 --2- 172.21.15.157:0/4127111825 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67a40c2cb0 0x7f67a4169210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:31.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.888+0000 7f67aa0a6700 1 -- 172.21.15.157:0/4127111825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a40bc850 msgr2=0x7f67a41526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:31.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.888+0000 7f67aa0a6700 1 --2- 172.21.15.157:0/4127111825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a40bc850 0x7f67a41526c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:31.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.888+0000 7f67aa0a6700 1 -- 172.21.15.157:0/4127111825 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679c009ae0 con 0x7f67a4068bc0 2023-10-06T04:07:31.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.888+0000 7f67a98a5700 1 --2- 172.21.15.157:0/4127111825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a40bc850 0x7f67a41526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:07:31.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.888+0000 7f67aa0a6700 1 --2- 172.21.15.157:0/4127111825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67a4068bc0 0x7f67a40b92e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f67a000d860 tx=0x7f67a0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:31.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.889+0000 7f679a7fc700 1 -- 172.21.15.157:0/4127111825 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a00125f0 con 0x7f67a4068bc0 2023-10-06T04:07:31.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.889+0000 7f679a7fc700 1 -- 172.21.15.157:0/4127111825 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a0012750 con 0x7f67a4068bc0 2023-10-06T04:07:31.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.889+0000 7f67ac30a700 1 -- 172.21.15.157:0/4127111825 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a406bdd0 con 0x7f67a4068bc0 2023-10-06T04:07:31.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.889+0000 7f679a7fc700 1 -- 172.21.15.157:0/4127111825 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a0012b10 con 0x7f67a4068bc0 2023-10-06T04:07:31.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.889+0000 7f67ac30a700 1 -- 172.21.15.157:0/4127111825 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a4151890 con 0x7f67a4068bc0 2023-10-06T04:07:31.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.890+0000 7f679a7fc700 1 -- 172.21.15.157:0/4127111825 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f67a001e460 con 0x7f67a4068bc0 2023-10-06T04:07:31.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.890+0000 7f67ac30a700 1 -- 172.21.15.157:0/4127111825 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a40692b0 con 0x7f67a4068bc0 2023-10-06T04:07:31.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.890+0000 7f679a7fc700 1 --2- 172.21.15.157:0/4127111825 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6788066f30 0x7f67880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:31.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.891+0000 7f679a7fc700 1 -- 172.21.15.157:0/4127111825 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6546+0+0 (secure 0 0 0) 0x7f67a0084f10 con 0x7f67a4068bc0 2023-10-06T04:07:31.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.891+0000 7f67a98a5700 1 --2- 172.21.15.157:0/4127111825 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6788066f30 0x7f67880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:31.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.891+0000 7f67a98a5700 1 --2- 172.21.15.157:0/4127111825 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6788066f30 0x7f67880693e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f67a406d160 tx=0x7f6794008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:31.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:31.898+0000 7f679a7fc700 1 -- 172.21.15.157:0/4127111825 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f67a0051e30 con 0x7f67a4068bc0 2023-10-06T04:07:32.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.137+0000 7f67ac30a700 1 -- 172.21.15.157:0/4127111825 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67a404bb40 con 0x7f67a4068bc0 2023-10-06T04:07:32.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.137+0000 7f679a7fc700 1 -- 172.21.15.157:0/4127111825 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+5386 (secure 0 0 0) 0x7f67a0055450 con 0x7f67a4068bc0 2023-10-06T04:07:32.139 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:07:32.139 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:07:31.737106+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5698},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5698":{"gid":5698,"name":"e","rank":1,"incarnation":72,"state":"up:replay","state_seq":1,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:07:32.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.139+0000 7f67ac30a700 1 -- 172.21.15.157:0/4127111825 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6788066f30 msgr2=0x7f67880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:32.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.139+0000 7f67ac30a700 1 --2- 172.21.15.157:0/4127111825 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6788066f30 0x7f67880693e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f67a406d160 tx=0x7f6794008040 comp rx=0 tx=0).stop 2023-10-06T04:07:32.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.139+0000 7f67ac30a700 1 -- 172.21.15.157:0/4127111825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67a4068bc0 msgr2=0x7f67a40b92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:32.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.139+0000 7f67ac30a700 1 --2- 172.21.15.157:0/4127111825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67a4068bc0 0x7f67a40b92e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f67a000d860 tx=0x7f67a0008040 comp rx=0 tx=0).stop 2023-10-06T04:07:32.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.139+0000 7f67ac30a700 1 -- 172.21.15.157:0/4127111825 shutdown_connections 2023-10-06T04:07:32.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.139+0000 7f67ac30a700 1 --2- 172.21.15.157:0/4127111825 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6788066f30 0x7f67880693e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:32.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.139+0000 7f67ac30a700 1 --2- 172.21.15.157:0/4127111825 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67a40c2cb0 0x7f67a4169210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:32.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.139+0000 7f67ac30a700 1 --2- 172.21.15.157:0/4127111825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67a4068bc0 0x7f67a40b92e0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:32.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.139+0000 7f67ac30a700 1 --2- 172.21.15.157:0/4127111825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a40bc850 0x7f67a41526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:32.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.139+0000 7f67ac30a700 1 -- 172.21.15.157:0/4127111825 >> 172.21.15.157:0/4127111825 conn(0x7f67a406b510 msgr2=0x7f67a40c8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:32.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.139+0000 7f67ac30a700 1 -- 172.21.15.157:0/4127111825 shutdown_connections 2023-10-06T04:07:32.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.139+0000 7f67ac30a700 1 -- 172.21.15.157:0/4127111825 wait complete. 2023-10-06T04:07:32.142 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 72 2023-10-06T04:07:32.156 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d down, removed from mdsmap 2023-10-06T04:07:32.156 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-10-06T04:07:32.156 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:07:32.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.269+0000 7f56860e3700 1 Processor -- start 2023-10-06T04:07:32.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.269+0000 7f56860e3700 1 -- start start 2023-10-06T04:07:32.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.270+0000 7f56860e3700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56800ca630 0x7f56800c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:32.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.270+0000 7f56860e3700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5680068920 0x7f5680156840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:32.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.270+0000 7f56860e3700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f56800c52e0 0x7f5680156d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:32.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.270+0000 7f56860e3700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f568006d530 con 0x7f5680068920 2023-10-06T04:07:32.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.270+0000 7f56860e3700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f568006d9c0 con 0x7f56800c52e0 2023-10-06T04:07:32.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.270+0000 7f56860e3700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568006f0c0 con 0x7f56800ca630 2023-10-06T04:07:32.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.270+0000 7f567effd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5680068920 0x7f5680156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:32.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.270+0000 7f567effd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5680068920 0x7f5680156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36454/0 (socket says 172.21.15.157:36454) 2023-10-06T04:07:32.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.270+0000 7f567effd700 1 -- 172.21.15.157:0/1505183955 learned_addr learned my addr 172.21.15.157:0/1505183955 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:07:32.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.271+0000 7f567ffff700 1 --2- 172.21.15.157:0/1505183955 >> v2:172.21.15.175:3300/0 conn(0x7f56800c52e0 0x7f5680156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:32.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.271+0000 7f567f7fe700 1 --2- 172.21.15.157:0/1505183955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56800ca630 0x7f56800c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:32.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.271+0000 7f567effd700 1 -- 172.21.15.157:0/1505183955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56800ca630 msgr2=0x7f56800c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:32.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.271+0000 7f567effd700 1 --2- 172.21.15.157:0/1505183955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56800ca630 0x7f56800c6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:32.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.271+0000 7f567effd700 1 -- 172.21.15.157:0/1505183955 >> v2:172.21.15.175:3300/0 conn(0x7f56800c52e0 msgr2=0x7f5680156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.271+0000 7f567effd700 1 --2- 172.21.15.157:0/1505183955 >> v2:172.21.15.175:3300/0 conn(0x7f56800c52e0 0x7f5680156d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.271+0000 7f567effd700 1 -- 172.21.15.157:0/1505183955 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568006f3a0 con 0x7f5680068920 2023-10-06T04:07:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.271+0000 7f567ffff700 1 --2- 172.21.15.157:0/1505183955 >> v2:172.21.15.175:3300/0 conn(0x7f56800c52e0 0x7f5680156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:07:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.271+0000 7f567f7fe700 1 --2- 172.21.15.157:0/1505183955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56800ca630 0x7f56800c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:07:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.271+0000 7f567effd700 1 --2- 172.21.15.157:0/1505183955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5680068920 0x7f5680156840 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f568006ee60 tx=0x7f5674009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4c80c5f802c2eea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.271+0000 7f567e7fc700 1 -- 172.21.15.157:0/1505183955 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567400b860 con 0x7f5680068920 2023-10-06T04:07:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.271+0000 7f567e7fc700 1 -- 172.21.15.157:0/1505183955 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567400b9c0 con 0x7f5680068920 2023-10-06T04:07:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.271+0000 7f567e7fc700 1 -- 172.21.15.157:0/1505183955 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567400bd60 con 0x7f5680068920 2023-10-06T04:07:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.271+0000 7f56860e3700 1 -- 172.21.15.157:0/1505183955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5680068920 msgr2=0x7f5680156840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.271+0000 7f56860e3700 1 --2- 172.21.15.157:0/1505183955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5680068920 0x7f5680156840 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f568006ee60 tx=0x7f5674009250 comp rx=0 tx=0).stop 2023-10-06T04:07:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.271+0000 7f56860e3700 1 -- 172.21.15.157:0/1505183955 shutdown_connections 2023-10-06T04:07:32.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.271+0000 7f56860e3700 1 --2- 172.21.15.157:0/1505183955 >> v2:172.21.15.175:3300/0 conn(0x7f56800c52e0 0x7f5680156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:32.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.271+0000 7f56860e3700 1 --2- 172.21.15.157:0/1505183955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56800ca630 0x7f56800c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:32.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.271+0000 7f56860e3700 1 --2- 172.21.15.157:0/1505183955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5680068920 0x7f5680156840 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:32.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.271+0000 7f56860e3700 1 -- 172.21.15.157:0/1505183955 >> 172.21.15.157:0/1505183955 conn(0x7f56800692b0 msgr2=0x7f56800696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:32.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.272+0000 7f56860e3700 1 -- 172.21.15.157:0/1505183955 shutdown_connections 2023-10-06T04:07:32.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.272+0000 7f56860e3700 1 -- 172.21.15.157:0/1505183955 wait complete. 2023-10-06T04:07:32.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.273+0000 7f56860e3700 1 Processor -- start 2023-10-06T04:07:32.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.273+0000 7f56860e3700 1 -- start start 2023-10-06T04:07:32.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.273+0000 7f56860e3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5680068920 0x7f56800c7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:32.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.273+0000 7f56860e3700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56800c52e0 0x7f5680165c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:32.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.273+0000 7f56860e3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56800ca630 0x7f5680166160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:32.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.273+0000 7f56860e3700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56800c1c20 con 0x7f5680068920 2023-10-06T04:07:32.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.273+0000 7f56860e3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56800ca130 con 0x7f56800ca630 2023-10-06T04:07:32.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.273+0000 7f56860e3700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56800c88a0 con 0x7f56800c52e0 2023-10-06T04:07:32.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.274+0000 7f567f7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5680068920 0x7f56800c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:32.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.274+0000 7f567f7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5680068920 0x7f56800c7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36466/0 (socket says 172.21.15.157:36466) 2023-10-06T04:07:32.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.274+0000 7f567f7fe700 1 -- 172.21.15.157:0/4180054510 learned_addr learned my addr 172.21.15.157:0/4180054510 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:07:32.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.274+0000 7f567effd700 1 --2- 172.21.15.157:0/4180054510 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56800c52e0 0x7f5680165c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:32.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.274+0000 7f567ffff700 1 --2- 172.21.15.157:0/4180054510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56800ca630 0x7f5680166160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:32.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.274+0000 7f567f7fe700 1 -- 172.21.15.157:0/4180054510 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56800c52e0 msgr2=0x7f5680165c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:32.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.274+0000 7f567f7fe700 1 --2- 172.21.15.157:0/4180054510 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56800c52e0 0x7f5680165c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:32.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.274+0000 7f567f7fe700 1 -- 172.21.15.157:0/4180054510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56800ca630 msgr2=0x7f5680166160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:32.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.274+0000 7f567f7fe700 1 --2- 172.21.15.157:0/4180054510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56800ca630 0x7f5680166160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:32.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.274+0000 7f567f7fe700 1 -- 172.21.15.157:0/4180054510 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56740094f0 con 0x7f5680068920 2023-10-06T04:07:32.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.274+0000 7f567f7fe700 1 --2- 172.21.15.157:0/4180054510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5680068920 0x7f56800c7470 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f567000cfb0 tx=0x7f5670007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:32.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.274+0000 7f56677fe700 1 -- 172.21.15.157:0/4180054510 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567000ad80 con 0x7f5680068920 2023-10-06T04:07:32.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.274+0000 7f56677fe700 1 -- 172.21.15.157:0/4180054510 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5670016420 con 0x7f5680068920 2023-10-06T04:07:32.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.274+0000 7f56677fe700 1 -- 172.21.15.157:0/4180054510 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56700167e0 con 0x7f5680068920 2023-10-06T04:07:32.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.274+0000 7f56860e3700 1 -- 172.21.15.157:0/4180054510 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5680150bc0 con 0x7f5680068920 2023-10-06T04:07:32.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.274+0000 7f56860e3700 1 -- 172.21.15.157:0/4180054510 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5680150e60 con 0x7f5680068920 2023-10-06T04:07:32.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.275+0000 7f56860e3700 1 -- 172.21.15.157:0/4180054510 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5680159e80 con 0x7f5680068920 2023-10-06T04:07:32.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.276+0000 7f56677fe700 1 -- 172.21.15.157:0/4180054510 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5670016b90 con 0x7f5680068920 2023-10-06T04:07:32.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.276+0000 7f56677fe700 1 --2- 172.21.15.157:0/4180054510 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f566c066f30 0x7f566c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:32.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.276+0000 7f56677fe700 1 -- 172.21.15.157:0/4180054510 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6546+0+0 (secure 0 0 0) 0x7f5670012070 con 0x7f5680068920 2023-10-06T04:07:32.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.276+0000 7f567effd700 1 --2- 172.21.15.157:0/4180054510 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f566c066f30 0x7f566c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:32.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.276+0000 7f567effd700 1 --2- 172.21.15.157:0/4180054510 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f566c066f30 0x7f566c0693e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f568006ee60 tx=0x7f5674017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:32.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.280+0000 7f56677fe700 1 -- 172.21.15.157:0/4180054510 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f567004e5a0 con 0x7f5680068920 2023-10-06T04:07:32.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.507+0000 7f56860e3700 1 -- 172.21.15.157:0/4180054510 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f568004bb40 con 0x7f5680068920 2023-10-06T04:07:32.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.508+0000 7f56677fe700 1 -- 172.21.15.157:0/4180054510 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+5386 (secure 0 0 0) 0x7f5670051fa0 con 0x7f5680068920 2023-10-06T04:07:32.511 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:07:32.511 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:07:31.737106+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5698},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5698":{"gid":5698,"name":"e","rank":1,"incarnation":72,"state":"up:replay","state_seq":1,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:07:32.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.510+0000 7f56860e3700 1 -- 172.21.15.157:0/4180054510 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f566c066f30 msgr2=0x7f566c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:32.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.510+0000 7f56860e3700 1 --2- 172.21.15.157:0/4180054510 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f566c066f30 0x7f566c0693e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f568006ee60 tx=0x7f5674017000 comp rx=0 tx=0).stop 2023-10-06T04:07:32.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.510+0000 7f56860e3700 1 -- 172.21.15.157:0/4180054510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5680068920 msgr2=0x7f56800c7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:32.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.510+0000 7f56860e3700 1 --2- 172.21.15.157:0/4180054510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5680068920 0x7f56800c7470 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f567000cfb0 tx=0x7f5670007870 comp rx=0 tx=0).stop 2023-10-06T04:07:32.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.510+0000 7f56860e3700 1 -- 172.21.15.157:0/4180054510 shutdown_connections 2023-10-06T04:07:32.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.510+0000 7f56860e3700 1 --2- 172.21.15.157:0/4180054510 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f566c066f30 0x7f566c0693e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:32.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.510+0000 7f56860e3700 1 --2- 172.21.15.157:0/4180054510 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56800ca630 0x7f5680166160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:32.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.510+0000 7f56860e3700 1 --2- 172.21.15.157:0/4180054510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5680068920 0x7f56800c7470 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:32.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.510+0000 7f56860e3700 1 --2- 172.21.15.157:0/4180054510 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56800c52e0 0x7f5680165c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:32.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.510+0000 7f56860e3700 1 -- 172.21.15.157:0/4180054510 >> 172.21.15.157:0/4180054510 conn(0x7f56800692b0 msgr2=0x7f5680154030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:32.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.510+0000 7f56860e3700 1 -- 172.21.15.157:0/4180054510 shutdown_connections 2023-10-06T04:07:32.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:32.510+0000 7f56860e3700 1 -- 172.21.15.157:0/4180054510 wait complete. 2023-10-06T04:07:32.514 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 72 2023-10-06T04:07:32.524 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 72, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-06T03:57:10.718782+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 72, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_5698": { "addr": "172.21.15.157:6837/3065749287", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6836", "nonce": 3065749287, "type": "v2" }, { "addr": "172.21.15.157:6837", "nonce": 3065749287, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5698, "incarnation": 72, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:replay", "state_seq": 1 }, "gid_5799": { "addr": "172.21.15.157:6834/765157057", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6832", "nonce": 765157057, "type": "v2" }, { "addr": "172.21.15.157:6834", "nonce": 765157057, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1 ], "features": 4540138320759226367, "flags": 0, "gid": 5799, "incarnation": 64, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 17 } }, "last_failure": 0, "last_failure_osd_epoch": 51, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2023-10-06T04:07:31.737106+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2, 3, 4 ], "tableserver": 0, "up": { "mds_0": 5799, "mds_1": 5698 } } } ], "standbys": [ { "addr": "172.21.15.175:6835/1799968248", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6834", "nonce": 1799968248, "type": "v2" }, { "addr": "172.21.15.175:6835", "nonce": 1799968248, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 57, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5126, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.175:6839/745804271", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6838", "nonce": 745804271, "type": "v2" }, { "addr": "172.21.15.175:6839", "nonce": 745804271, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 42, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5164, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.157:6835/2461751318", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6833", "nonce": 2461751318, "type": "v2" }, { "addr": "172.21.15.157:6835", "nonce": 2461751318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 34, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5268, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-06T04:07:34.526 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:07:34.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.640+0000 7f1097554700 1 Processor -- start 2023-10-06T04:07:34.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.640+0000 7f1097554700 1 -- start start 2023-10-06T04:07:34.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.641+0000 7f1097554700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10900c50c0 0x7f109006ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:34.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.641+0000 7f1097554700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1090152f30 con 0x7f109014f830 2023-10-06T04:07:34.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.641+0000 7f1097554700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10901533c0 con 0x7f109014fc00 2023-10-06T04:07:34.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.641+0000 7f1097554700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1090152410 con 0x7f10900c50c0 2023-10-06T04:07:34.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.641+0000 7f10952f0700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f109014f830 0x7f109006c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39982/0 (socket says 172.21.15.157:39982) 2023-10-06T04:07:34.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.641+0000 7f10952f0700 1 -- 172.21.15.157:0/3432888520 learned_addr learned my addr 172.21.15.157:0/3432888520 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:07:34.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.642+0000 7f1095af1700 1 --2- 172.21.15.157:0/3432888520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10900c50c0 0x7f109006ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:34.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.642+0000 7f1087fff700 1 -- 172.21.15.157:0/3432888520 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 467745566 0 0) 0x7f1090152f30 con 0x7f109014f830 2023-10-06T04:07:34.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.642+0000 7f1087fff700 1 -- 172.21.15.157:0/3432888520 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f107c003570 con 0x7f109014f830 2023-10-06T04:07:34.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.642+0000 7f1087fff700 1 -- 172.21.15.157:0/3432888520 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3748146072 0 0) 0x7f107c003570 con 0x7f109014f830 2023-10-06T04:07:34.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.642+0000 7f1087fff700 1 -- 172.21.15.157:0/3432888520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10900c50c0 msgr2=0x7f109006ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:34.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.642+0000 7f1087fff700 1 --2- 172.21.15.157:0/3432888520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10900c50c0 0x7f109006ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:34.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.642+0000 7f1087fff700 1 -- 172.21.15.157:0/3432888520 >> v1:172.21.15.175:6789/0 conn(0x7f109014fc00 legacy=0x7f109006a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:34.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.642+0000 7f1087fff700 1 -- 172.21.15.157:0/3432888520 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10900c37e0 con 0x7f109014f830 2023-10-06T04:07:34.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.642+0000 7f1087fff700 1 -- 172.21.15.157:0/3432888520 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f108c0017f0 con 0x7f109014f830 2023-10-06T04:07:34.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.642+0000 7f1087fff700 1 -- 172.21.15.157:0/3432888520 >> v1:172.21.15.157:6789/0 conn(0x7f109014f830 legacy=0x7f109006c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:34.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.642+0000 7f1087fff700 1 --2- 172.21.15.157:0/3432888520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f107c003a90 0x7f107c003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:34.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.642+0000 7f1087fff700 1 --2- 172.21.15.157:0/3432888520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10900c50c0 0x7f107c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:34.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.642+0000 7f1087fff700 1 --2- 172.21.15.157:0/3432888520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f107c004d50 0x7f107c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:34.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.642+0000 7f1087fff700 1 -- 172.21.15.157:0/3432888520 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107c00bbf0 con 0x7f107c003a90 2023-10-06T04:07:34.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.642+0000 7f1087fff700 1 -- 172.21.15.157:0/3432888520 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107c00bd30 con 0x7f107c004d50 2023-10-06T04:07:34.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.642+0000 7f1087fff700 1 -- 172.21.15.157:0/3432888520 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f107c00bef0 con 0x7f10900c50c0 2023-10-06T04:07:34.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.642+0000 7f10952f0700 1 --2- 172.21.15.157:0/3432888520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f107c003a90 0x7f107c003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:34.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.642+0000 7f1094aef700 1 --2- 172.21.15.157:0/3432888520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10900c50c0 0x7f107c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:34.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.642+0000 7f1095af1700 1 --2- 172.21.15.157:0/3432888520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f107c004d50 0x7f107c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:34.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.642+0000 7f10952f0700 1 -- 172.21.15.157:0/3432888520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10900c50c0 msgr2=0x7f107c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:34.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.642+0000 7f10952f0700 1 --2- 172.21.15.157:0/3432888520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10900c50c0 0x7f107c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:34.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.642+0000 7f10952f0700 1 -- 172.21.15.157:0/3432888520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f107c004d50 msgr2=0x7f107c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:34.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.642+0000 7f10952f0700 1 --2- 172.21.15.157:0/3432888520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f107c004d50 0x7f107c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:34.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.642+0000 7f10952f0700 1 -- 172.21.15.157:0/3432888520 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107c003570 con 0x7f107c003a90 2023-10-06T04:07:34.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.642+0000 7f10952f0700 1 --2- 172.21.15.157:0/3432888520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f107c003a90 0x7f107c003ec0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f108c00a900 tx=0x7f1090069670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741b019ee1a4944d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:34.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.643+0000 7f1087fff700 1 -- 172.21.15.157:0/3432888520 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c0070d0 con 0x7f107c003a90 2023-10-06T04:07:34.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.643+0000 7f1087fff700 1 -- 172.21.15.157:0/3432888520 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108c007230 con 0x7f107c003a90 2023-10-06T04:07:34.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.643+0000 7f1087fff700 1 -- 172.21.15.157:0/3432888520 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c0075f0 con 0x7f107c003a90 2023-10-06T04:07:34.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.643+0000 7f1097554700 1 -- 172.21.15.157:0/3432888520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f107c003a90 msgr2=0x7f107c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:34.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.643+0000 7f1097554700 1 --2- 172.21.15.157:0/3432888520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f107c003a90 0x7f107c003ec0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f108c00a900 tx=0x7f1090069670 comp rx=0 tx=0).stop 2023-10-06T04:07:34.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.643+0000 7f1095af1700 1 -- 172.21.15.157:0/3432888520 reap_dead start 2023-10-06T04:07:34.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.643+0000 7f1097554700 1 -- 172.21.15.157:0/3432888520 shutdown_connections 2023-10-06T04:07:34.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.643+0000 7f1097554700 1 -- 172.21.15.157:0/3432888520 >> 172.21.15.157:0/3432888520 conn(0x7f10900c8b20 msgr2=0x7f10900c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:34.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.643+0000 7f1097554700 1 -- 172.21.15.157:0/3432888520 shutdown_connections 2023-10-06T04:07:34.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.643+0000 7f1097554700 1 -- 172.21.15.157:0/3432888520 wait complete. 2023-10-06T04:07:34.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.644+0000 7f1097554700 1 Processor -- start 2023-10-06T04:07:34.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.644+0000 7f1097554700 1 -- start start 2023-10-06T04:07:34.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.645+0000 7f1097554700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f107c003a90 0x7f109014e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:34.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.645+0000 7f1097554700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10900c60f0 0x7f109014a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:34.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.645+0000 7f1097554700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f109014af00 0x7f10901482d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:34.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.645+0000 7f1097554700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109016a570 con 0x7f10900c60f0 2023-10-06T04:07:34.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.645+0000 7f1097554700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109014b380 con 0x7f107c003a90 2023-10-06T04:07:34.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.645+0000 7f1097554700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1090169960 con 0x7f109014af00 2023-10-06T04:07:34.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.645+0000 7f1094aef700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10900c60f0 0x7f109014a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:34.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.645+0000 7f1094aef700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10900c60f0 0x7f109014a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52594/0 (socket says 172.21.15.157:52594) 2023-10-06T04:07:34.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.645+0000 7f1094aef700 1 -- 172.21.15.157:0/4085742739 learned_addr learned my addr 172.21.15.157:0/4085742739 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:07:34.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.645+0000 7f1095af1700 1 --2- 172.21.15.157:0/4085742739 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f109014af00 0x7f10901482d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:34.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.645+0000 7f10952f0700 1 --2- 172.21.15.157:0/4085742739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f107c003a90 0x7f109014e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:34.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.645+0000 7f1094aef700 1 -- 172.21.15.157:0/4085742739 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f109014af00 msgr2=0x7f10901482d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:34.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.645+0000 7f1094aef700 1 --2- 172.21.15.157:0/4085742739 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f109014af00 0x7f10901482d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:34.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.645+0000 7f1094aef700 1 -- 172.21.15.157:0/4085742739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f107c003a90 msgr2=0x7f109014e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:34.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.645+0000 7f1094aef700 1 --2- 172.21.15.157:0/4085742739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f107c003a90 0x7f109014e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:34.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.646+0000 7f1094aef700 1 -- 172.21.15.157:0/4085742739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1090169c40 con 0x7f10900c60f0 2023-10-06T04:07:34.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.646+0000 7f1094aef700 1 --2- 172.21.15.157:0/4085742739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10900c60f0 0x7f109014a9c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1080002920 tx=0x7f1080007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:34.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.646+0000 7f10857fa700 1 -- 172.21.15.157:0/4085742739 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108000f670 con 0x7f10900c60f0 2023-10-06T04:07:34.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.646+0000 7f10857fa700 1 -- 172.21.15.157:0/4085742739 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108000f7d0 con 0x7f10900c60f0 2023-10-06T04:07:34.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.646+0000 7f10857fa700 1 -- 172.21.15.157:0/4085742739 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108000fb90 con 0x7f10900c60f0 2023-10-06T04:07:34.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.646+0000 7f1097554700 1 -- 172.21.15.157:0/4085742739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1090169f20 con 0x7f10900c60f0 2023-10-06T04:07:34.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.646+0000 7f1097554700 1 -- 172.21.15.157:0/4085742739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f109014cf60 con 0x7f10900c60f0 2023-10-06T04:07:34.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.647+0000 7f1097554700 1 -- 172.21.15.157:0/4085742739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f109016a6b0 con 0x7f10900c60f0 2023-10-06T04:07:34.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.647+0000 7f10857fa700 1 -- 172.21.15.157:0/4085742739 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f108001c400 con 0x7f10900c60f0 2023-10-06T04:07:34.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.648+0000 7f10857fa700 1 --2- 172.21.15.157:0/4085742739 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1074066f30 0x7f10740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:34.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.648+0000 7f10857fa700 1 -- 172.21.15.157:0/4085742739 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6546+0+0 (secure 0 0 0) 0x7f1080084bf0 con 0x7f10900c60f0 2023-10-06T04:07:34.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.648+0000 7f10952f0700 1 --2- 172.21.15.157:0/4085742739 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1074066f30 0x7f10740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:34.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.648+0000 7f10952f0700 1 --2- 172.21.15.157:0/4085742739 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1074066f30 0x7f10740693e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f108c00a3f0 tx=0x7f108c015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:34.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.652+0000 7f10857fa700 1 -- 172.21.15.157:0/4085742739 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f109016a6b0 con 0x7f10900c60f0 2023-10-06T04:07:34.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.885+0000 7f1097554700 1 -- 172.21.15.157:0/4085742739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f109016a6b0 con 0x7f10900c60f0 2023-10-06T04:07:34.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.885+0000 7f10857fa700 1 -- 172.21.15.157:0/4085742739 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+5386 (secure 0 0 0) 0x7f109016a6b0 con 0x7f10900c60f0 2023-10-06T04:07:34.887 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:07:34.887 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:07:31.737106+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5698},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5698":{"gid":5698,"name":"e","rank":1,"incarnation":72,"state":"up:replay","state_seq":1,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:07:34.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.888+0000 7f1097554700 1 -- 172.21.15.157:0/4085742739 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1074066f30 msgr2=0x7f10740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:34.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.888+0000 7f1097554700 1 --2- 172.21.15.157:0/4085742739 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1074066f30 0x7f10740693e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f108c00a3f0 tx=0x7f108c015000 comp rx=0 tx=0).stop 2023-10-06T04:07:34.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.888+0000 7f1097554700 1 -- 172.21.15.157:0/4085742739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10900c60f0 msgr2=0x7f109014a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:34.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.888+0000 7f1097554700 1 --2- 172.21.15.157:0/4085742739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10900c60f0 0x7f109014a9c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1080002920 tx=0x7f1080007d30 comp rx=0 tx=0).stop 2023-10-06T04:07:34.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.888+0000 7f1097554700 1 -- 172.21.15.157:0/4085742739 shutdown_connections 2023-10-06T04:07:34.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.888+0000 7f1097554700 1 --2- 172.21.15.157:0/4085742739 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1074066f30 0x7f10740693e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:34.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.888+0000 7f1097554700 1 --2- 172.21.15.157:0/4085742739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f107c003a90 0x7f109014e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:34.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.888+0000 7f1097554700 1 --2- 172.21.15.157:0/4085742739 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f109014af00 0x7f10901482d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:34.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.888+0000 7f1097554700 1 --2- 172.21.15.157:0/4085742739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10900c60f0 0x7f109014a9c0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:34.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.888+0000 7f1097554700 1 -- 172.21.15.157:0/4085742739 >> 172.21.15.157:0/4085742739 conn(0x7f10900c8b20 msgr2=0x7f109006c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:34.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.888+0000 7f1097554700 1 -- 172.21.15.157:0/4085742739 shutdown_connections 2023-10-06T04:07:34.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:34.888+0000 7f1097554700 1 -- 172.21.15.157:0/4085742739 wait complete. 2023-10-06T04:07:34.891 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 72 2023-10-06T04:07:34.901 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:07:36.902 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:07:37.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.034+0000 7f9d9b11d700 1 Processor -- start 2023-10-06T04:07:37.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.034+0000 7f9d9b11d700 1 -- start start 2023-10-06T04:07:37.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.035+0000 7f9d9b11d700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9d94068170 0x7f9d940c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:37.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.035+0000 7f9d9b11d700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d940c1020 con 0x7f9d941544e0 2023-10-06T04:07:37.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.035+0000 7f9d9b11d700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d94157b00 con 0x7f9d94067da0 2023-10-06T04:07:37.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.035+0000 7f9d9b11d700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d940c0cf0 con 0x7f9d94068170 2023-10-06T04:07:37.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.036+0000 7f9d9991a700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f9d941544e0 0x7f9d94150bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39996/0 (socket says 172.21.15.157:39996) 2023-10-06T04:07:37.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.036+0000 7f9d9991a700 1 -- 172.21.15.157:0/481842511 learned_addr learned my addr 172.21.15.157:0/481842511 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:07:37.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.036+0000 7f9d9a91c700 1 --2- 172.21.15.157:0/481842511 >> v2:172.21.15.175:3300/0 conn(0x7f9d94068170 0x7f9d940c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:37.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.036+0000 7f9d99119700 1 -- 172.21.15.157:0/481842511 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 895270553 0 0) 0x7f9d940c1020 con 0x7f9d941544e0 2023-10-06T04:07:37.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.036+0000 7f9d99119700 1 -- 172.21.15.157:0/481842511 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d80003570 con 0x7f9d941544e0 2023-10-06T04:07:37.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.036+0000 7f9d99119700 1 -- 172.21.15.157:0/481842511 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3292677119 0 0) 0x7f9d94157b00 con 0x7f9d94067da0 2023-10-06T04:07:37.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.036+0000 7f9d99119700 1 -- 172.21.15.157:0/481842511 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d940c1020 con 0x7f9d94067da0 2023-10-06T04:07:37.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.036+0000 7f9d99119700 1 -- 172.21.15.157:0/481842511 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1173296513 0 0) 0x7f9d80003570 con 0x7f9d941544e0 2023-10-06T04:07:37.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.036+0000 7f9d99119700 1 -- 172.21.15.157:0/481842511 >> v2:172.21.15.175:3300/0 conn(0x7f9d94068170 msgr2=0x7f9d940c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:37.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.036+0000 7f9d99119700 1 --2- 172.21.15.157:0/481842511 >> v2:172.21.15.175:3300/0 conn(0x7f9d94068170 0x7f9d940c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:37.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.036+0000 7f9d99119700 1 -- 172.21.15.157:0/481842511 >> v1:172.21.15.175:6789/0 conn(0x7f9d94067da0 legacy=0x7f9d94153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:37.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.036+0000 7f9d99119700 1 -- 172.21.15.157:0/481842511 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d94157da0 con 0x7f9d941544e0 2023-10-06T04:07:37.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.036+0000 7f9d99119700 1 -- 172.21.15.157:0/481842511 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f9d84001470 con 0x7f9d941544e0 2023-10-06T04:07:37.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.036+0000 7f9d99119700 1 -- 172.21.15.157:0/481842511 >> v1:172.21.15.157:6789/0 conn(0x7f9d941544e0 legacy=0x7f9d94150bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:37.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.036+0000 7f9d99119700 1 --2- 172.21.15.157:0/481842511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d80003bf0 0x7f9d80003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:37.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.036+0000 7f9d99119700 1 --2- 172.21.15.157:0/481842511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d80004810 0x7f9d80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:37.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.036+0000 7f9d99119700 1 --2- 172.21.15.157:0/481842511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d800051c0 0x7f9d8000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:37.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.036+0000 7f9d99119700 1 -- 172.21.15.157:0/481842511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d940c0cf0 con 0x7f9d80004810 2023-10-06T04:07:37.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.036+0000 7f9d99119700 1 -- 172.21.15.157:0/481842511 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d800104c0 con 0x7f9d80003bf0 2023-10-06T04:07:37.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.036+0000 7f9d99119700 1 -- 172.21.15.157:0/481842511 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d80010680 con 0x7f9d800051c0 2023-10-06T04:07:37.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.036+0000 7f9d9991a700 1 --2- 172.21.15.157:0/481842511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d80004810 0x7f9d80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:37.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.036+0000 7f9d9a11b700 1 --2- 172.21.15.157:0/481842511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d80003bf0 0x7f9d80003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:37.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.036+0000 7f9d9a91c700 1 --2- 172.21.15.157:0/481842511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d800051c0 0x7f9d8000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:37.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.037+0000 7f9d9991a700 1 -- 172.21.15.157:0/481842511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d800051c0 msgr2=0x7f9d8000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:37.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.037+0000 7f9d9991a700 1 --2- 172.21.15.157:0/481842511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d800051c0 0x7f9d8000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:37.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.037+0000 7f9d9991a700 1 -- 172.21.15.157:0/481842511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d80003bf0 msgr2=0x7f9d80003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:37.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.037+0000 7f9d9991a700 1 --2- 172.21.15.157:0/481842511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d80003bf0 0x7f9d80003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:37.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.037+0000 7f9d9991a700 1 -- 172.21.15.157:0/481842511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d80003570 con 0x7f9d80004810 2023-10-06T04:07:37.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.037+0000 7f9d9a91c700 1 --2- 172.21.15.157:0/481842511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d800051c0 0x7f9d8000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:07:37.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.037+0000 7f9d9a91c700 1 -- 172.21.15.157:0/481842511 reap_dead start 2023-10-06T04:07:37.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.037+0000 7f9d9991a700 1 --2- 172.21.15.157:0/481842511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d80004810 0x7f9d80004c80 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400c2a0 tx=0x7f9d940c0f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625b1a4c8ccc6cbf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:37.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.037+0000 7f9d99119700 1 -- 172.21.15.157:0/481842511 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8400cbd0 con 0x7f9d80004810 2023-10-06T04:07:37.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.037+0000 7f9d99119700 1 -- 172.21.15.157:0/481842511 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8400cd30 con 0x7f9d80004810 2023-10-06T04:07:37.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.037+0000 7f9d99119700 1 -- 172.21.15.157:0/481842511 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8400cea0 con 0x7f9d80004810 2023-10-06T04:07:37.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.037+0000 7f9d9b11d700 1 -- 172.21.15.157:0/481842511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d80004810 msgr2=0x7f9d80004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:37.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.037+0000 7f9d9b11d700 1 --2- 172.21.15.157:0/481842511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d80004810 0x7f9d80004c80 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400c2a0 tx=0x7f9d940c0f90 comp rx=0 tx=0).stop 2023-10-06T04:07:37.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.038+0000 7f9d9b11d700 1 -- 172.21.15.157:0/481842511 shutdown_connections 2023-10-06T04:07:37.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.038+0000 7f9d9b11d700 1 --2- 172.21.15.157:0/481842511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d80004810 0x7f9d80004c80 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:37.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.038+0000 7f9d9b11d700 1 -- 172.21.15.157:0/481842511 >> 172.21.15.157:0/481842511 conn(0x7f9d9406c1f0 msgr2=0x7f9d9406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:37.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.038+0000 7f9d9b11d700 1 -- 172.21.15.157:0/481842511 shutdown_connections 2023-10-06T04:07:37.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.038+0000 7f9d9b11d700 1 -- 172.21.15.157:0/481842511 wait complete. 2023-10-06T04:07:37.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.039+0000 7f9d9b11d700 1 Processor -- start 2023-10-06T04:07:37.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.039+0000 7f9d9b11d700 1 -- start start 2023-10-06T04:07:37.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.039+0000 7f9d9b11d700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d80004810 0x7f9d940c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:37.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.039+0000 7f9d9b11d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d940a6460 0x7f9d941621b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:37.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.039+0000 7f9d9b11d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d941535f0 0x7f9d9406eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:37.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.039+0000 7f9d9b11d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d94161d20 con 0x7f9d940a6460 2023-10-06T04:07:37.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.039+0000 7f9d9b11d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d941616f0 con 0x7f9d941535f0 2023-10-06T04:07:37.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.039+0000 7f9d9b11d700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d940a69c0 con 0x7f9d80004810 2023-10-06T04:07:37.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.039+0000 7f9d9991a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d940a6460 0x7f9d941621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:37.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.039+0000 7f9d9991a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d940a6460 0x7f9d941621b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52612/0 (socket says 172.21.15.157:52612) 2023-10-06T04:07:37.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.039+0000 7f9d9991a700 1 -- 172.21.15.157:0/33959281 learned_addr learned my addr 172.21.15.157:0/33959281 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:07:37.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.040+0000 7f9d9a11b700 1 --2- 172.21.15.157:0/33959281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d80004810 0x7f9d940c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:37.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.040+0000 7f9d9a91c700 1 --2- 172.21.15.157:0/33959281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d941535f0 0x7f9d9406eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:37.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.040+0000 7f9d9991a700 1 -- 172.21.15.157:0/33959281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d80004810 msgr2=0x7f9d940c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:37.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.040+0000 7f9d9991a700 1 --2- 172.21.15.157:0/33959281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d80004810 0x7f9d940c5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:37.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.040+0000 7f9d9991a700 1 -- 172.21.15.157:0/33959281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d941535f0 msgr2=0x7f9d9406eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:37.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.040+0000 7f9d9991a700 1 --2- 172.21.15.157:0/33959281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d941535f0 0x7f9d9406eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:37.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.040+0000 7f9d9991a700 1 -- 172.21.15.157:0/33959281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d940a6ca0 con 0x7f9d940a6460 2023-10-06T04:07:37.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.040+0000 7f9d9991a700 1 --2- 172.21.15.157:0/33959281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d940a6460 0x7f9d941621b0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f9d84001ed0 tx=0x7f9d84015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:37.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.040+0000 7f9d8a7fc700 1 -- 172.21.15.157:0/33959281 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d84004a40 con 0x7f9d940a6460 2023-10-06T04:07:37.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.040+0000 7f9d8a7fc700 1 -- 172.21.15.157:0/33959281 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d84004be0 con 0x7f9d940a6460 2023-10-06T04:07:37.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.040+0000 7f9d8a7fc700 1 -- 172.21.15.157:0/33959281 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d840070d0 con 0x7f9d940a6460 2023-10-06T04:07:37.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.040+0000 7f9d9b11d700 1 -- 172.21.15.157:0/33959281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d940a6f80 con 0x7f9d940a6460 2023-10-06T04:07:37.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.040+0000 7f9d9b11d700 1 -- 172.21.15.157:0/33959281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d9406f0a0 con 0x7f9d940a6460 2023-10-06T04:07:37.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.041+0000 7f9d9b11d700 1 -- 172.21.15.157:0/33959281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d94161b70 con 0x7f9d940a6460 2023-10-06T04:07:37.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.042+0000 7f9d8a7fc700 1 -- 172.21.15.157:0/33959281 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9d840074c0 con 0x7f9d940a6460 2023-10-06T04:07:37.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.042+0000 7f9d8a7fc700 1 --2- 172.21.15.157:0/33959281 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9d78066f30 0x7f9d780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:37.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.042+0000 7f9d8a7fc700 1 -- 172.21.15.157:0/33959281 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6546+0+0 (secure 0 0 0) 0x7f9d84085d00 con 0x7f9d940a6460 2023-10-06T04:07:37.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.042+0000 7f9d9a11b700 1 --2- 172.21.15.157:0/33959281 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9d78066f30 0x7f9d780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:37.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.043+0000 7f9d9a11b700 1 --2- 172.21.15.157:0/33959281 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9d78066f30 0x7f9d780693e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9d90000c00 tx=0x7f9d90009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:37.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.050+0000 7f9d8a7fc700 1 -- 172.21.15.157:0/33959281 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d84052c20 con 0x7f9d940a6460 2023-10-06T04:07:37.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.326+0000 7f9d9b11d700 1 -- 172.21.15.157:0/33959281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d94161e60 con 0x7f9d940a6460 2023-10-06T04:07:37.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.327+0000 7f9d8a7fc700 1 -- 172.21.15.157:0/33959281 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+5386 (secure 0 0 0) 0x7f9d84056240 con 0x7f9d940a6460 2023-10-06T04:07:37.328 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:07:37.328 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:07:31.737106+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5698},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5698":{"gid":5698,"name":"e","rank":1,"incarnation":72,"state":"up:replay","state_seq":1,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:07:37.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.328+0000 7f9d9b11d700 1 -- 172.21.15.157:0/33959281 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9d78066f30 msgr2=0x7f9d780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:37.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.328+0000 7f9d9b11d700 1 --2- 172.21.15.157:0/33959281 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9d78066f30 0x7f9d780693e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9d90000c00 tx=0x7f9d90009040 comp rx=0 tx=0).stop 2023-10-06T04:07:37.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.329+0000 7f9d9b11d700 1 -- 172.21.15.157:0/33959281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d940a6460 msgr2=0x7f9d941621b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:37.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.329+0000 7f9d9b11d700 1 --2- 172.21.15.157:0/33959281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d940a6460 0x7f9d941621b0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f9d84001ed0 tx=0x7f9d84015000 comp rx=0 tx=0).stop 2023-10-06T04:07:37.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.329+0000 7f9d9b11d700 1 -- 172.21.15.157:0/33959281 shutdown_connections 2023-10-06T04:07:37.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.329+0000 7f9d9b11d700 1 --2- 172.21.15.157:0/33959281 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9d78066f30 0x7f9d780693e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:37.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.329+0000 7f9d9b11d700 1 --2- 172.21.15.157:0/33959281 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d941535f0 0x7f9d9406eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:37.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.329+0000 7f9d9b11d700 1 --2- 172.21.15.157:0/33959281 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d80004810 0x7f9d940c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:37.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.329+0000 7f9d9b11d700 1 --2- 172.21.15.157:0/33959281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d940a6460 0x7f9d941621b0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:37.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.329+0000 7f9d9b11d700 1 -- 172.21.15.157:0/33959281 >> 172.21.15.157:0/33959281 conn(0x7f9d9406c1f0 msgr2=0x7f9d940c3100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:37.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.329+0000 7f9d9b11d700 1 -- 172.21.15.157:0/33959281 shutdown_connections 2023-10-06T04:07:37.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:37.329+0000 7f9d9b11d700 1 -- 172.21.15.157:0/33959281 wait complete. 2023-10-06T04:07:37.331 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 72 2023-10-06T04:07:37.343 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:07:39.344 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:07:39.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.459+0000 7fdb114a8700 1 Processor -- start 2023-10-06T04:07:39.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.459+0000 7fdb114a8700 1 -- start start 2023-10-06T04:07:39.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.460+0000 7fdb114a8700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb0c06c440 0x7fdb0c069130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:39.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.460+0000 7fdb114a8700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdb0c06c810 0x7fdb0c1569e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:39.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.460+0000 7fdb114a8700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb0c155f20 con 0x7fdb0c0c5fc0 2023-10-06T04:07:39.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.460+0000 7fdb114a8700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb0c06f310 con 0x7fdb0c06c810 2023-10-06T04:07:39.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.460+0000 7fdb114a8700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c06d4a0 con 0x7fdb0c06c440 2023-10-06T04:07:39.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.460+0000 7fdb0a7fc700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fdb0c0c5fc0 0x7fdb0c152e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39998/0 (socket says 172.21.15.157:39998) 2023-10-06T04:07:39.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.460+0000 7fdb0a7fc700 1 -- 172.21.15.157:0/1715044455 learned_addr learned my addr 172.21.15.157:0/1715044455 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:07:39.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.460+0000 7fdb0affd700 1 --2- 172.21.15.157:0/1715044455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb0c06c440 0x7fdb0c069130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:39.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.460+0000 7fdb0b7fe700 1 --2- 172.21.15.157:0/1715044455 >> v2:172.21.15.175:3300/0 conn(0x7fdb0c06c810 0x7fdb0c1569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:39.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.460+0000 7fdb09ffb700 1 -- 172.21.15.157:0/1715044455 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 863291515 0 0) 0x7fdb0c155f20 con 0x7fdb0c0c5fc0 2023-10-06T04:07:39.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.460+0000 7fdb09ffb700 1 -- 172.21.15.157:0/1715044455 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdaf4003570 con 0x7fdb0c0c5fc0 2023-10-06T04:07:39.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.461+0000 7fdb09ffb700 1 -- 172.21.15.157:0/1715044455 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2258675157 0 0) 0x7fdaf4003570 con 0x7fdb0c0c5fc0 2023-10-06T04:07:39.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.461+0000 7fdb09ffb700 1 -- 172.21.15.157:0/1715044455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb0c06c440 msgr2=0x7fdb0c069130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:39.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.461+0000 7fdb09ffb700 1 --2- 172.21.15.157:0/1715044455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb0c06c440 0x7fdb0c069130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:39.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.461+0000 7fdb09ffb700 1 -- 172.21.15.157:0/1715044455 >> v2:172.21.15.175:3300/0 conn(0x7fdb0c06c810 msgr2=0x7fdb0c1569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:39.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.461+0000 7fdb09ffb700 1 --2- 172.21.15.157:0/1715044455 >> v2:172.21.15.175:3300/0 conn(0x7fdb0c06c810 0x7fdb0c1569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:39.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.461+0000 7fdb09ffb700 1 -- 172.21.15.157:0/1715044455 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0c06d780 con 0x7fdb0c0c5fc0 2023-10-06T04:07:39.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.461+0000 7fdb09ffb700 1 -- 172.21.15.157:0/1715044455 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7fdb00001470 con 0x7fdb0c0c5fc0 2023-10-06T04:07:39.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.461+0000 7fdb09ffb700 1 -- 172.21.15.157:0/1715044455 >> v1:172.21.15.157:6789/0 conn(0x7fdb0c0c5fc0 legacy=0x7fdb0c152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:39.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.461+0000 7fdb09ffb700 1 --2- 172.21.15.157:0/1715044455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdaf4003a60 0x7fdaf4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:39.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.461+0000 7fdb09ffb700 1 --2- 172.21.15.157:0/1715044455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdaf4004810 0x7fdaf4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:39.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.461+0000 7fdb09ffb700 1 --2- 172.21.15.157:0/1715044455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb0c06c440 0x7fdaf40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:39.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.461+0000 7fdb09ffb700 1 -- 172.21.15.157:0/1715044455 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c06f310 con 0x7fdaf4004810 2023-10-06T04:07:39.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.461+0000 7fdb09ffb700 1 -- 172.21.15.157:0/1715044455 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c06d4a0 con 0x7fdaf4003a60 2023-10-06T04:07:39.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.461+0000 7fdb09ffb700 1 -- 172.21.15.157:0/1715044455 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf400ba60 con 0x7fdb0c06c440 2023-10-06T04:07:39.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.461+0000 7fdb0affd700 1 --2- 172.21.15.157:0/1715044455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdaf4003a60 0x7fdaf4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:39.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.461+0000 7fdb0b7fe700 1 --2- 172.21.15.157:0/1715044455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb0c06c440 0x7fdaf40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:39.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.461+0000 7fdb0a7fc700 1 --2- 172.21.15.157:0/1715044455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdaf4004810 0x7fdaf4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:39.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.461+0000 7fdb0a7fc700 1 -- 172.21.15.157:0/1715044455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb0c06c440 msgr2=0x7fdaf40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:39.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.461+0000 7fdb0a7fc700 1 --2- 172.21.15.157:0/1715044455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb0c06c440 0x7fdaf40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:39.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.461+0000 7fdb0a7fc700 1 -- 172.21.15.157:0/1715044455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdaf4003a60 msgr2=0x7fdaf4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:39.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.461+0000 7fdb0a7fc700 1 --2- 172.21.15.157:0/1715044455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdaf4003a60 0x7fdaf4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:39.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.461+0000 7fdb0a7fc700 1 -- 172.21.15.157:0/1715044455 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf4003570 con 0x7fdaf4004810 2023-10-06T04:07:39.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.461+0000 7fdb0a7fc700 1 --2- 172.21.15.157:0/1715044455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdaf4004810 0x7fdaf4004c80 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000c3c0 tx=0x7fdb0c156f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2e361c090f5d19 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:39.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.461+0000 7fdb09ffb700 1 -- 172.21.15.157:0/1715044455 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0000cc50 con 0x7fdaf4004810 2023-10-06T04:07:39.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.462+0000 7fdb09ffb700 1 -- 172.21.15.157:0/1715044455 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0000cdb0 con 0x7fdaf4004810 2023-10-06T04:07:39.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.462+0000 7fdb09ffb700 1 -- 172.21.15.157:0/1715044455 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb00005b50 con 0x7fdaf4004810 2023-10-06T04:07:39.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.462+0000 7fdb114a8700 1 -- 172.21.15.157:0/1715044455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdaf4004810 msgr2=0x7fdaf4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:39.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.462+0000 7fdb114a8700 1 --2- 172.21.15.157:0/1715044455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdaf4004810 0x7fdaf4004c80 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000c3c0 tx=0x7fdb0c156f20 comp rx=0 tx=0).stop 2023-10-06T04:07:39.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.462+0000 7fdb0b7fe700 1 -- 172.21.15.157:0/1715044455 reap_dead start 2023-10-06T04:07:39.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.462+0000 7fdb114a8700 1 -- 172.21.15.157:0/1715044455 shutdown_connections 2023-10-06T04:07:39.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.462+0000 7fdb114a8700 1 -- 172.21.15.157:0/1715044455 >> 172.21.15.157:0/1715044455 conn(0x7fdb0c068370 msgr2=0x7fdb0c068770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:39.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.462+0000 7fdb114a8700 1 -- 172.21.15.157:0/1715044455 shutdown_connections 2023-10-06T04:07:39.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.462+0000 7fdb114a8700 1 -- 172.21.15.157:0/1715044455 wait complete. 2023-10-06T04:07:39.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.463+0000 7fdb114a8700 1 Processor -- start 2023-10-06T04:07:39.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.463+0000 7fdb114a8700 1 -- start start 2023-10-06T04:07:39.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.464+0000 7fdb114a8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdaf4004810 0x7fdb0c166910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:39.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.464+0000 7fdb114a8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb0c0ad5a0 0x7fdb0c158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:39.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.464+0000 7fdb114a8700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb0c157750 0x7fdb0c157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:39.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.464+0000 7fdb114a8700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c063ce0 con 0x7fdb0c0ad5a0 2023-10-06T04:07:39.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.464+0000 7fdb114a8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c0aee60 con 0x7fdaf4004810 2023-10-06T04:07:39.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.464+0000 7fdb114a8700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c1572c0 con 0x7fdb0c157750 2023-10-06T04:07:39.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.464+0000 7fdb0a7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb0c0ad5a0 0x7fdb0c158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:39.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.464+0000 7fdb0a7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb0c0ad5a0 0x7fdb0c158410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52630/0 (socket says 172.21.15.157:52630) 2023-10-06T04:07:39.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.464+0000 7fdb0a7fc700 1 -- 172.21.15.157:0/999542468 learned_addr learned my addr 172.21.15.157:0/999542468 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:07:39.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.464+0000 7fdb0b7fe700 1 --2- 172.21.15.157:0/999542468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb0c157750 0x7fdb0c157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:39.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.464+0000 7fdb0affd700 1 --2- 172.21.15.157:0/999542468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdaf4004810 0x7fdb0c166910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:39.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.464+0000 7fdb0a7fc700 1 -- 172.21.15.157:0/999542468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb0c157750 msgr2=0x7fdb0c157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:39.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.464+0000 7fdb0a7fc700 1 --2- 172.21.15.157:0/999542468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb0c157750 0x7fdb0c157bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:39.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.464+0000 7fdb0a7fc700 1 -- 172.21.15.157:0/999542468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdaf4004810 msgr2=0x7fdb0c166910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:39.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.464+0000 7fdb0a7fc700 1 --2- 172.21.15.157:0/999542468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdaf4004810 0x7fdb0c166910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:39.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.464+0000 7fdb0a7fc700 1 -- 172.21.15.157:0/999542468 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0c0631f0 con 0x7fdb0c0ad5a0 2023-10-06T04:07:39.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.465+0000 7fdb0a7fc700 1 --2- 172.21.15.157:0/999542468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb0c0ad5a0 0x7fdb0c158410 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40046d0 tx=0x7fdb00015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:39.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.465+0000 7fdaf2ffd700 1 -- 172.21.15.157:0/999542468 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0000cc50 con 0x7fdb0c0ad5a0 2023-10-06T04:07:39.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.465+0000 7fdaf2ffd700 1 -- 172.21.15.157:0/999542468 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb00001430 con 0x7fdb0c0ad5a0 2023-10-06T04:07:39.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.465+0000 7fdb114a8700 1 -- 172.21.15.157:0/999542468 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb0c063440 con 0x7fdb0c0ad5a0 2023-10-06T04:07:39.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.465+0000 7fdaf2ffd700 1 -- 172.21.15.157:0/999542468 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0001e430 con 0x7fdb0c0ad5a0 2023-10-06T04:07:39.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.465+0000 7fdb114a8700 1 -- 172.21.15.157:0/999542468 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb0c063a00 con 0x7fdb0c0ad5a0 2023-10-06T04:07:39.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.466+0000 7fdb114a8700 1 -- 172.21.15.157:0/999542468 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb0c157090 con 0x7fdb0c0ad5a0 2023-10-06T04:07:39.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.466+0000 7fdaf2ffd700 1 -- 172.21.15.157:0/999542468 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdb0001e7e0 con 0x7fdb0c0ad5a0 2023-10-06T04:07:39.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.467+0000 7fdaf2ffd700 1 --2- 172.21.15.157:0/999542468 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdaf8066f30 0x7fdaf80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:39.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.467+0000 7fdaf2ffd700 1 -- 172.21.15.157:0/999542468 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6546+0+0 (secure 0 0 0) 0x7fdb00086030 con 0x7fdb0c0ad5a0 2023-10-06T04:07:39.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.467+0000 7fdb0affd700 1 --2- 172.21.15.157:0/999542468 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdaf8066f30 0x7fdaf80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:39.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.467+0000 7fdb0affd700 1 --2- 172.21.15.157:0/999542468 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdaf8066f30 0x7fdaf80693e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdafc006f00 tx=0x7fdafc0091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:39.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.471+0000 7fdaf2ffd700 1 -- 172.21.15.157:0/999542468 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdb0c157090 con 0x7fdb0c0ad5a0 2023-10-06T04:07:39.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.867+0000 7fdb114a8700 1 -- 172.21.15.157:0/999542468 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb0c157470 con 0x7fdb0c0ad5a0 2023-10-06T04:07:39.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.868+0000 7fdaf2ffd700 1 -- 172.21.15.157:0/999542468 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+5388 (secure 0 0 0) 0x7fdb0c157470 con 0x7fdb0c0ad5a0 2023-10-06T04:07:39.870 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:07:39.870 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:07:37.976483+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5698},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5698":{"gid":5698,"name":"e","rank":1,"incarnation":72,"state":"up:resolve","state_seq":27,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:07:39.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.871+0000 7fdb114a8700 1 -- 172.21.15.157:0/999542468 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdaf8066f30 msgr2=0x7fdaf80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:39.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.871+0000 7fdb114a8700 1 --2- 172.21.15.157:0/999542468 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdaf8066f30 0x7fdaf80693e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdafc006f00 tx=0x7fdafc0091a0 comp rx=0 tx=0).stop 2023-10-06T04:07:39.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.871+0000 7fdb114a8700 1 -- 172.21.15.157:0/999542468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb0c0ad5a0 msgr2=0x7fdb0c158410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:39.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.871+0000 7fdb114a8700 1 --2- 172.21.15.157:0/999542468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb0c0ad5a0 0x7fdb0c158410 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40046d0 tx=0x7fdb00015000 comp rx=0 tx=0).stop 2023-10-06T04:07:39.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.871+0000 7fdb114a8700 1 -- 172.21.15.157:0/999542468 shutdown_connections 2023-10-06T04:07:39.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.871+0000 7fdb114a8700 1 --2- 172.21.15.157:0/999542468 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdaf8066f30 0x7fdaf80693e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:39.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.871+0000 7fdb114a8700 1 --2- 172.21.15.157:0/999542468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdaf4004810 0x7fdb0c166910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:39.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.871+0000 7fdb114a8700 1 --2- 172.21.15.157:0/999542468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb0c157750 0x7fdb0c157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:39.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.871+0000 7fdb114a8700 1 --2- 172.21.15.157:0/999542468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb0c0ad5a0 0x7fdb0c158410 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:39.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.871+0000 7fdb114a8700 1 -- 172.21.15.157:0/999542468 >> 172.21.15.157:0/999542468 conn(0x7fdb0c068370 msgr2=0x7fdb0c1521a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:39.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.871+0000 7fdb114a8700 1 -- 172.21.15.157:0/999542468 shutdown_connections 2023-10-06T04:07:39.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:39.871+0000 7fdb114a8700 1 -- 172.21.15.157:0/999542468 wait complete. 2023-10-06T04:07:39.874 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 73 2023-10-06T04:07:39.885 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:07:41.885 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:07:42.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.004+0000 7fe7d4cd1700 1 Processor -- start 2023-10-06T04:07:42.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.004+0000 7fe7d4cd1700 1 -- start start 2023-10-06T04:07:42.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.005+0000 7fe7d4cd1700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d0067e70 0x7fe7d00aac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:42.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.005+0000 7fe7d4cd1700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe7d006d300 0x7fe7d00ab150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:42.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.005+0000 7fe7d4cd1700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7d0068670 con 0x7fe7d0067aa0 2023-10-06T04:07:42.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.005+0000 7fe7d4cd1700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7d006e140 con 0x7fe7d006d300 2023-10-06T04:07:42.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.005+0000 7fe7d4cd1700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d006f3e0 con 0x7fe7d0067e70 2023-10-06T04:07:42.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.005+0000 7fe7cf7fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe7d0067aa0 0x7fe7d006ecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40010/0 (socket says 172.21.15.157:40010) 2023-10-06T04:07:42.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.005+0000 7fe7cf7fe700 1 -- 172.21.15.157:0/2466465596 learned_addr learned my addr 172.21.15.157:0/2466465596 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:07:42.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.005+0000 7fe7cffff700 1 --2- 172.21.15.157:0/2466465596 >> v2:172.21.15.157:3300/0 conn(0x7fe7d006d300 0x7fe7d00ab150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:42.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.005+0000 7fe7ceffd700 1 --2- 172.21.15.157:0/2466465596 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d0067e70 0x7fe7d00aac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:42.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.005+0000 7fe7ce7fc700 1 -- 172.21.15.157:0/2466465596 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1875914110 0 0) 0x7fe7d0068670 con 0x7fe7d0067aa0 2023-10-06T04:07:42.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.005+0000 7fe7ce7fc700 1 -- 172.21.15.157:0/2466465596 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7b8003570 con 0x7fe7d0067aa0 2023-10-06T04:07:42.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.005+0000 7fe7cffff700 1 -- 172.21.15.157:0/2466465596 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d0067e70 msgr2=0x7fe7d00aac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:42.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.005+0000 7fe7cffff700 1 --2- 172.21.15.157:0/2466465596 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d0067e70 0x7fe7d00aac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:42.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.005+0000 7fe7cffff700 1 -- 172.21.15.157:0/2466465596 >> v1:172.21.15.157:6789/0 conn(0x7fe7d0067aa0 legacy=0x7fe7d006ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:42.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.005+0000 7fe7cffff700 1 -- 172.21.15.157:0/2466465596 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d0155f00 con 0x7fe7d006d300 2023-10-06T04:07:42.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.006+0000 7fe7cffff700 1 --2- 172.21.15.157:0/2466465596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7d006d300 0x7fe7d00ab150 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009e60 tx=0x7fe7c8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978034b9e60cd2b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:42.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.006+0000 7fe7ce7fc700 1 -- 172.21.15.157:0/2466465596 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c800e070 con 0x7fe7d006d300 2023-10-06T04:07:42.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.006+0000 7fe7ce7fc700 1 -- 172.21.15.157:0/2466465596 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c800bb60 con 0x7fe7d006d300 2023-10-06T04:07:42.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.006+0000 7fe7ce7fc700 1 -- 172.21.15.157:0/2466465596 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8012430 con 0x7fe7d006d300 2023-10-06T04:07:42.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.006+0000 7fe7d4cd1700 1 -- 172.21.15.157:0/2466465596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7d006d300 msgr2=0x7fe7d00ab150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:42.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.006+0000 7fe7d4cd1700 1 --2- 172.21.15.157:0/2466465596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7d006d300 0x7fe7d00ab150 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009e60 tx=0x7fe7c8009780 comp rx=0 tx=0).stop 2023-10-06T04:07:42.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.006+0000 7fe7d4cd1700 1 -- 172.21.15.157:0/2466465596 shutdown_connections 2023-10-06T04:07:42.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.006+0000 7fe7d4cd1700 1 --2- 172.21.15.157:0/2466465596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7d006d300 0x7fe7d00ab150 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:42.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.006+0000 7fe7d4cd1700 1 --2- 172.21.15.157:0/2466465596 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d0067e70 0x7fe7d00aac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:42.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.006+0000 7fe7d4cd1700 1 -- 172.21.15.157:0/2466465596 >> 172.21.15.157:0/2466465596 conn(0x7fe7d0069070 msgr2=0x7fe7d0069470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:42.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.006+0000 7fe7d4cd1700 1 -- 172.21.15.157:0/2466465596 shutdown_connections 2023-10-06T04:07:42.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.006+0000 7fe7d4cd1700 1 -- 172.21.15.157:0/2466465596 wait complete. 2023-10-06T04:07:42.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.007+0000 7fe7d4cd1700 1 Processor -- start 2023-10-06T04:07:42.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.007+0000 7fe7d4cd1700 1 -- start start 2023-10-06T04:07:42.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.008+0000 7fe7d4cd1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d0067aa0 0x7fe7d0148b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:42.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.008+0000 7fe7d4cd1700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d0067e70 0x7fe7d00c25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:42.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.008+0000 7fe7d4cd1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7d006d300 0x7fe7d00c2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:42.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.008+0000 7fe7d4cd1700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d014f720 con 0x7fe7d006d300 2023-10-06T04:07:42.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.008+0000 7fe7d4cd1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d0147070 con 0x7fe7d0067aa0 2023-10-06T04:07:42.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.008+0000 7fe7d4cd1700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d0155ba0 con 0x7fe7d0067e70 2023-10-06T04:07:42.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.008+0000 7fe7cffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7d006d300 0x7fe7d00c2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:42.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.008+0000 7fe7cffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7d006d300 0x7fe7d00c2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52648/0 (socket says 172.21.15.157:52648) 2023-10-06T04:07:42.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.008+0000 7fe7cffff700 1 -- 172.21.15.157:0/604045908 learned_addr learned my addr 172.21.15.157:0/604045908 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:07:42.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.008+0000 7fe7cf7fe700 1 --2- 172.21.15.157:0/604045908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d0067aa0 0x7fe7d0148b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:42.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.008+0000 7fe7ceffd700 1 --2- 172.21.15.157:0/604045908 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d0067e70 0x7fe7d00c25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:42.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.009+0000 7fe7cffff700 1 -- 172.21.15.157:0/604045908 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d0067e70 msgr2=0x7fe7d00c25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:42.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.009+0000 7fe7cffff700 1 --2- 172.21.15.157:0/604045908 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d0067e70 0x7fe7d00c25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:42.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.009+0000 7fe7cffff700 1 -- 172.21.15.157:0/604045908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d0067aa0 msgr2=0x7fe7d0148b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:42.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.009+0000 7fe7cffff700 1 --2- 172.21.15.157:0/604045908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d0067aa0 0x7fe7d0148b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:42.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.009+0000 7fe7cffff700 1 -- 172.21.15.157:0/604045908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c8009a20 con 0x7fe7d006d300 2023-10-06T04:07:42.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.009+0000 7fe7cffff700 1 --2- 172.21.15.157:0/604045908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7d006d300 0x7fe7d00c2af0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe7d006aeb0 tx=0x7fe7c8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:42.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.009+0000 7fe7b77fe700 1 -- 172.21.15.157:0/604045908 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8012ea0 con 0x7fe7d006d300 2023-10-06T04:07:42.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.009+0000 7fe7b77fe700 1 -- 172.21.15.157:0/604045908 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c8027470 con 0x7fe7d006d300 2023-10-06T04:07:42.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.009+0000 7fe7d4cd1700 1 -- 172.21.15.157:0/604045908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7d0152350 con 0x7fe7d006d300 2023-10-06T04:07:42.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.009+0000 7fe7b77fe700 1 -- 172.21.15.157:0/604045908 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8027830 con 0x7fe7d006d300 2023-10-06T04:07:42.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.009+0000 7fe7d4cd1700 1 -- 172.21.15.157:0/604045908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7d0146d00 con 0x7fe7d006d300 2023-10-06T04:07:42.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.010+0000 7fe7d4cd1700 1 -- 172.21.15.157:0/604045908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7d006a400 con 0x7fe7d006d300 2023-10-06T04:07:42.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.010+0000 7fe7b77fe700 1 -- 172.21.15.157:0/604045908 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe7c8024070 con 0x7fe7d006d300 2023-10-06T04:07:42.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.011+0000 7fe7b77fe700 1 --2- 172.21.15.157:0/604045908 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe7bc066f60 0x7fe7bc069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:42.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.011+0000 7fe7b77fe700 1 -- 172.21.15.157:0/604045908 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6546+0+0 (secure 0 0 0) 0x7fe7c8087320 con 0x7fe7d006d300 2023-10-06T04:07:42.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.011+0000 7fe7cf7fe700 1 --2- 172.21.15.157:0/604045908 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe7bc066f60 0x7fe7bc069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:42.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.011+0000 7fe7cf7fe700 1 --2- 172.21.15.157:0/604045908 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe7bc066f60 0x7fe7bc069410 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0007fd0 tx=0x7fe7c0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:42.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.015+0000 7fe7b77fe700 1 -- 172.21.15.157:0/604045908 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe7c8054240 con 0x7fe7d006d300 2023-10-06T04:07:42.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.240+0000 7fe7d4cd1700 1 -- 172.21.15.157:0/604045908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7d004bb40 con 0x7fe7d006d300 2023-10-06T04:07:42.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.240+0000 7fe7b77fe700 1 -- 172.21.15.157:0/604045908 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+5388 (secure 0 0 0) 0x7fe7c8057860 con 0x7fe7d006d300 2023-10-06T04:07:42.242 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:07:42.242 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:07:37.976483+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5698},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5698":{"gid":5698,"name":"e","rank":1,"incarnation":72,"state":"up:resolve","state_seq":27,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:07:42.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.242+0000 7fe7d4cd1700 1 -- 172.21.15.157:0/604045908 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe7bc066f60 msgr2=0x7fe7bc069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:42.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.242+0000 7fe7d4cd1700 1 --2- 172.21.15.157:0/604045908 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe7bc066f60 0x7fe7bc069410 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0007fd0 tx=0x7fe7c0009040 comp rx=0 tx=0).stop 2023-10-06T04:07:42.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.242+0000 7fe7d4cd1700 1 -- 172.21.15.157:0/604045908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7d006d300 msgr2=0x7fe7d00c2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:42.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.242+0000 7fe7d4cd1700 1 --2- 172.21.15.157:0/604045908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7d006d300 0x7fe7d00c2af0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe7d006aeb0 tx=0x7fe7c8019000 comp rx=0 tx=0).stop 2023-10-06T04:07:42.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.242+0000 7fe7d4cd1700 1 -- 172.21.15.157:0/604045908 shutdown_connections 2023-10-06T04:07:42.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.242+0000 7fe7d4cd1700 1 --2- 172.21.15.157:0/604045908 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe7bc066f60 0x7fe7bc069410 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:42.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.242+0000 7fe7d4cd1700 1 --2- 172.21.15.157:0/604045908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d0067aa0 0x7fe7d0148b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:42.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.242+0000 7fe7d4cd1700 1 --2- 172.21.15.157:0/604045908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7d006d300 0x7fe7d00c2af0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:42.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.242+0000 7fe7d4cd1700 1 --2- 172.21.15.157:0/604045908 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d0067e70 0x7fe7d00c25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:42.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.242+0000 7fe7d4cd1700 1 -- 172.21.15.157:0/604045908 >> 172.21.15.157:0/604045908 conn(0x7fe7d0069070 msgr2=0x7fe7d00aaaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:42.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.242+0000 7fe7d4cd1700 1 -- 172.21.15.157:0/604045908 shutdown_connections 2023-10-06T04:07:42.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:42.242+0000 7fe7d4cd1700 1 -- 172.21.15.157:0/604045908 wait complete. 2023-10-06T04:07:42.246 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 73 2023-10-06T04:07:42.256 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:07:44.257 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:07:44.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.374+0000 7f493a283700 1 Processor -- start 2023-10-06T04:07:44.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.374+0000 7f493a283700 1 -- start start 2023-10-06T04:07:44.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.375+0000 7f493a283700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f493406cbb0 0x7f493406d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:44.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.375+0000 7f493a283700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f493406c1f0 0x7f493406da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:44.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.375+0000 7f493a283700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49340c9780 con 0x7f493406cf80 2023-10-06T04:07:44.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.375+0000 7f493a283700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49340c8c40 con 0x7f493406c1f0 2023-10-06T04:07:44.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.375+0000 7f493a283700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493406e0c0 con 0x7f493406cbb0 2023-10-06T04:07:44.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.375+0000 7f49337fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f493406c1f0 0x7f493406da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:44.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.375+0000 7f49337fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f493406c1f0 0x7f493406da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36900/0 (socket says 172.21.15.157:36900) 2023-10-06T04:07:44.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.375+0000 7f49337fe700 1 -- 172.21.15.157:0/1039396185 learned_addr learned my addr 172.21.15.157:0/1039396185 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:07:44.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.375+0000 7f4933fff700 1 --2- 172.21.15.157:0/1039396185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f493406cbb0 0x7f493406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:44.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.375+0000 7f49337fe700 1 -- 172.21.15.157:0/1039396185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f493406cbb0 msgr2=0x7f493406d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:44.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.375+0000 7f49337fe700 1 --2- 172.21.15.157:0/1039396185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f493406cbb0 0x7f493406d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:44.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.375+0000 7f49337fe700 1 -- 172.21.15.157:0/1039396185 >> v1:172.21.15.175:6789/0 conn(0x7f493406cf80 legacy=0x7f493414fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:44.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.375+0000 7f49337fe700 1 -- 172.21.15.157:0/1039396185 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49340c8e50 con 0x7f493406c1f0 2023-10-06T04:07:44.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.375+0000 7f49337fe700 1 --2- 172.21.15.157:0/1039396185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f493406c1f0 0x7f493406da70 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f49280099e0 tx=0x7f4928009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b384cd22d9649ec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:44.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.375+0000 7f4932ffd700 1 -- 172.21.15.157:0/1039396185 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492800b870 con 0x7f493406c1f0 2023-10-06T04:07:44.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.376+0000 7f4932ffd700 1 -- 172.21.15.157:0/1039396185 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492800b9d0 con 0x7f493406c1f0 2023-10-06T04:07:44.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.376+0000 7f4932ffd700 1 -- 172.21.15.157:0/1039396185 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492800bd70 con 0x7f493406c1f0 2023-10-06T04:07:44.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.376+0000 7f493a283700 1 -- 172.21.15.157:0/1039396185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f493406c1f0 msgr2=0x7f493406da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:44.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.376+0000 7f493a283700 1 --2- 172.21.15.157:0/1039396185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f493406c1f0 0x7f493406da70 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f49280099e0 tx=0x7f4928009250 comp rx=0 tx=0).stop 2023-10-06T04:07:44.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.376+0000 7f493a283700 1 -- 172.21.15.157:0/1039396185 shutdown_connections 2023-10-06T04:07:44.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.376+0000 7f493a283700 1 --2- 172.21.15.157:0/1039396185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f493406cbb0 0x7f493406d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:44.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.376+0000 7f493a283700 1 --2- 172.21.15.157:0/1039396185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f493406c1f0 0x7f493406da70 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:44.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.376+0000 7f493a283700 1 -- 172.21.15.157:0/1039396185 >> 172.21.15.157:0/1039396185 conn(0x7f493406f230 msgr2=0x7f493406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:44.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.376+0000 7f493a283700 1 -- 172.21.15.157:0/1039396185 shutdown_connections 2023-10-06T04:07:44.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.376+0000 7f493a283700 1 -- 172.21.15.157:0/1039396185 wait complete. 2023-10-06T04:07:44.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.378+0000 7f493a283700 1 Processor -- start 2023-10-06T04:07:44.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.378+0000 7f493a283700 1 -- start start 2023-10-06T04:07:44.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.379+0000 7f493a283700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f493406c1f0 0x7f49340c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:44.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.379+0000 7f493a283700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f493406cbb0 0x7f4934153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:44.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.379+0000 7f493a283700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f493406cf80 0x7f4934169ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:44.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.379+0000 7f493a283700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4934155460 con 0x7f493406cbb0 2023-10-06T04:07:44.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.379+0000 7f493a283700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4934167490 con 0x7f493406c1f0 2023-10-06T04:07:44.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.379+0000 7f493a283700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49341668e0 con 0x7f493406cf80 2023-10-06T04:07:44.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.380+0000 7f49337fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f493406cbb0 0x7f4934153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:44.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.380+0000 7f49337fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f493406cbb0 0x7f4934153d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36902/0 (socket says 172.21.15.157:36902) 2023-10-06T04:07:44.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.380+0000 7f49337fe700 1 -- 172.21.15.157:0/4125602841 learned_addr learned my addr 172.21.15.157:0/4125602841 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:07:44.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.380+0000 7f4933fff700 1 --2- 172.21.15.157:0/4125602841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f493406c1f0 0x7f49340c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:44.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.380+0000 7f4938820700 1 --2- 172.21.15.157:0/4125602841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f493406cf80 0x7f4934169ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:44.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.380+0000 7f49337fe700 1 -- 172.21.15.157:0/4125602841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f493406cf80 msgr2=0x7f4934169ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:44.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.380+0000 7f49337fe700 1 --2- 172.21.15.157:0/4125602841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f493406cf80 0x7f4934169ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:44.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.380+0000 7f49337fe700 1 -- 172.21.15.157:0/4125602841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f493406c1f0 msgr2=0x7f49340c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:44.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.380+0000 7f49337fe700 1 --2- 172.21.15.157:0/4125602841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f493406c1f0 0x7f49340c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:44.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.380+0000 7f49337fe700 1 -- 172.21.15.157:0/4125602841 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49280094f0 con 0x7f493406cbb0 2023-10-06T04:07:44.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.380+0000 7f49337fe700 1 --2- 172.21.15.157:0/4125602841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f493406cbb0 0x7f4934153d70 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f4928006fd0 tx=0x7f4928016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:44.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.380+0000 7f491bfff700 1 -- 172.21.15.157:0/4125602841 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4928011c70 con 0x7f493406cbb0 2023-10-06T04:07:44.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.380+0000 7f491bfff700 1 -- 172.21.15.157:0/4125602841 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4928020070 con 0x7f493406cbb0 2023-10-06T04:07:44.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.380+0000 7f493a283700 1 -- 172.21.15.157:0/4125602841 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49340c69b0 con 0x7f493406cbb0 2023-10-06T04:07:44.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.380+0000 7f491bfff700 1 -- 172.21.15.157:0/4125602841 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492801b450 con 0x7f493406cbb0 2023-10-06T04:07:44.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.380+0000 7f493a283700 1 -- 172.21.15.157:0/4125602841 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4934166b60 con 0x7f493406cbb0 2023-10-06T04:07:44.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.381+0000 7f493a283700 1 -- 172.21.15.157:0/4125602841 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4934089ca0 con 0x7f493406cbb0 2023-10-06T04:07:44.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.382+0000 7f491bfff700 1 -- 172.21.15.157:0/4125602841 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4928022030 con 0x7f493406cbb0 2023-10-06T04:07:44.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.383+0000 7f491bfff700 1 --2- 172.21.15.157:0/4125602841 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4920066f30 0x7f49200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:44.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.383+0000 7f491bfff700 1 -- 172.21.15.157:0/4125602841 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6546+0+0 (secure 0 0 0) 0x7f4928086690 con 0x7f493406cbb0 2023-10-06T04:07:44.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.383+0000 7f4933fff700 1 --2- 172.21.15.157:0/4125602841 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4920066f30 0x7f49200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:44.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.383+0000 7f4933fff700 1 --2- 172.21.15.157:0/4125602841 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4920066f30 0x7f49200693e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4924006fd0 tx=0x7f4924008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:44.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.387+0000 7f491bfff700 1 -- 172.21.15.157:0/4125602841 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f49280535b0 con 0x7f493406cbb0 2023-10-06T04:07:44.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.646+0000 7f493a283700 1 -- 172.21.15.157:0/4125602841 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4934089ca0 con 0x7f493406cbb0 2023-10-06T04:07:44.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.647+0000 7f491bfff700 1 -- 172.21.15.157:0/4125602841 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+5390 (secure 0 0 0) 0x7f4928056bd0 con 0x7f493406cbb0 2023-10-06T04:07:44.649 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:07:44.649 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:07:43.992831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5698},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5698":{"gid":5698,"name":"e","rank":1,"incarnation":72,"state":"up:reconnect","state_seq":29,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:07:44.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.649+0000 7f493a283700 1 -- 172.21.15.157:0/4125602841 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4920066f30 msgr2=0x7f49200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:44.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.649+0000 7f493a283700 1 --2- 172.21.15.157:0/4125602841 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4920066f30 0x7f49200693e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4924006fd0 tx=0x7f4924008040 comp rx=0 tx=0).stop 2023-10-06T04:07:44.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.649+0000 7f493a283700 1 -- 172.21.15.157:0/4125602841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f493406cbb0 msgr2=0x7f4934153d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:44.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.649+0000 7f493a283700 1 --2- 172.21.15.157:0/4125602841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f493406cbb0 0x7f4934153d70 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f4928006fd0 tx=0x7f4928016000 comp rx=0 tx=0).stop 2023-10-06T04:07:44.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.649+0000 7f493a283700 1 -- 172.21.15.157:0/4125602841 shutdown_connections 2023-10-06T04:07:44.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.649+0000 7f493a283700 1 --2- 172.21.15.157:0/4125602841 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4920066f30 0x7f49200693e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:44.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.649+0000 7f493a283700 1 --2- 172.21.15.157:0/4125602841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f493406c1f0 0x7f49340c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:44.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.649+0000 7f493a283700 1 --2- 172.21.15.157:0/4125602841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f493406cf80 0x7f4934169ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:44.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.649+0000 7f493a283700 1 --2- 172.21.15.157:0/4125602841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f493406cbb0 0x7f4934153d70 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:44.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.649+0000 7f493a283700 1 -- 172.21.15.157:0/4125602841 >> 172.21.15.157:0/4125602841 conn(0x7f493406f230 msgr2=0x7f493414ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:44.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.649+0000 7f493a283700 1 -- 172.21.15.157:0/4125602841 shutdown_connections 2023-10-06T04:07:44.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:44.649+0000 7f493a283700 1 -- 172.21.15.157:0/4125602841 wait complete. 2023-10-06T04:07:44.652 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 74 2023-10-06T04:07:44.663 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 74, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-06T03:57:10.718782+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 74, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_5698": { "addr": "172.21.15.157:6837/3065749287", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6836", "nonce": 3065749287, "type": "v2" }, { "addr": "172.21.15.157:6837", "nonce": 3065749287, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5698, "incarnation": 72, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:reconnect", "state_seq": 29 }, "gid_5799": { "addr": "172.21.15.157:6834/765157057", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6832", "nonce": 765157057, "type": "v2" }, { "addr": "172.21.15.157:6834", "nonce": 765157057, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1 ], "features": 4540138320759226367, "flags": 0, "gid": 5799, "incarnation": 64, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 17 } }, "last_failure": 0, "last_failure_osd_epoch": 51, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2023-10-06T04:07:43.992831+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2, 3, 4 ], "tableserver": 0, "up": { "mds_0": 5799, "mds_1": 5698 } } } ], "standbys": [ { "addr": "172.21.15.175:6835/1799968248", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6834", "nonce": 1799968248, "type": "v2" }, { "addr": "172.21.15.175:6835", "nonce": 1799968248, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 57, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5126, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.175:6839/745804271", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6838", "nonce": 745804271, "type": "v2" }, { "addr": "172.21.15.175:6839", "nonce": 745804271, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 42, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5164, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.157:6835/2461751318", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6833", "nonce": 2461751318, "type": "v2" }, { "addr": "172.21.15.157:6835", "nonce": 2461751318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 34, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5268, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-06T04:07:45.777 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:07:45.780 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:07:46.664 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:07:46.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.782+0000 7fba93ea8700 1 Processor -- start 2023-10-06T04:07:46.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.782+0000 7fba93ea8700 1 -- start start 2023-10-06T04:07:46.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.783+0000 7fba93ea8700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba8c06d460 0x7fba8c15e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:46.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.783+0000 7fba93ea8700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba8c0bed40 con 0x7fba8c068d10 2023-10-06T04:07:46.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.783+0000 7fba93ea8700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba8c157010 con 0x7fba8c0be970 2023-10-06T04:07:46.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.783+0000 7fba93ea8700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c06f9f0 con 0x7fba8c06d460 2023-10-06T04:07:46.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.783+0000 7fba91443700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fba8c068d10 0x7fba8c06a600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:57440/0 (socket says 172.21.15.157:57440) 2023-10-06T04:07:46.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.783+0000 7fba91443700 1 -- 172.21.15.157:0/2076916651 learned_addr learned my addr 172.21.15.157:0/2076916651 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:07:46.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.783+0000 7fba92445700 1 --2- 172.21.15.157:0/2076916651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba8c06d460 0x7fba8c15e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:46.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.784+0000 7fba90c42700 1 -- 172.21.15.157:0/2076916651 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3528681120 0 0) 0x7fba8c0bed40 con 0x7fba8c068d10 2023-10-06T04:07:46.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.784+0000 7fba90c42700 1 -- 172.21.15.157:0/2076916651 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba78003570 con 0x7fba8c068d10 2023-10-06T04:07:46.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.784+0000 7fba90c42700 1 -- 172.21.15.157:0/2076916651 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 74294084 0 0) 0x7fba8c157010 con 0x7fba8c0be970 2023-10-06T04:07:46.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.784+0000 7fba90c42700 1 -- 172.21.15.157:0/2076916651 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba8c0bed40 con 0x7fba8c0be970 2023-10-06T04:07:46.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.784+0000 7fba90c42700 1 -- 172.21.15.157:0/2076916651 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1036068555 0 0) 0x7fba78003570 con 0x7fba8c068d10 2023-10-06T04:07:46.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.784+0000 7fba90c42700 1 -- 172.21.15.157:0/2076916651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba8c06d460 msgr2=0x7fba8c15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:46.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.784+0000 7fba90c42700 1 --2- 172.21.15.157:0/2076916651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba8c06d460 0x7fba8c15e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:46.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.784+0000 7fba90c42700 1 -- 172.21.15.157:0/2076916651 >> v1:172.21.15.175:6789/0 conn(0x7fba8c0be970 legacy=0x7fba8c06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:46.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.784+0000 7fba90c42700 1 -- 172.21.15.157:0/2076916651 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba8c1574a0 con 0x7fba8c068d10 2023-10-06T04:07:46.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.784+0000 7fba90c42700 1 -- 172.21.15.157:0/2076916651 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7fba7c001470 con 0x7fba8c068d10 2023-10-06T04:07:46.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.784+0000 7fba90c42700 1 -- 172.21.15.157:0/2076916651 >> v1:172.21.15.157:6789/0 conn(0x7fba8c068d10 legacy=0x7fba8c06a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:46.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.784+0000 7fba90c42700 1 --2- 172.21.15.157:0/2076916651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba8c06d460 0x7fba78003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:46.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.784+0000 7fba90c42700 1 --2- 172.21.15.157:0/2076916651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba78004180 0x7fba78004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:46.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.784+0000 7fba90c42700 1 --2- 172.21.15.157:0/2076916651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba78004d50 0x7fba780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:46.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.784+0000 7fba90c42700 1 -- 172.21.15.157:0/2076916651 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c06f9f0 con 0x7fba78004d50 2023-10-06T04:07:46.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.784+0000 7fba90c42700 1 -- 172.21.15.157:0/2076916651 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7800bba0 con 0x7fba78004180 2023-10-06T04:07:46.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.784+0000 7fba90c42700 1 -- 172.21.15.157:0/2076916651 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba7800bd60 con 0x7fba8c06d460 2023-10-06T04:07:46.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.784+0000 7fba92445700 1 --2- 172.21.15.157:0/2076916651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba78004d50 0x7fba780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:46.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.784+0000 7fba91443700 1 --2- 172.21.15.157:0/2076916651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba78004180 0x7fba78004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:46.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.784+0000 7fba91c44700 1 --2- 172.21.15.157:0/2076916651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba8c06d460 0x7fba78003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:46.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.784+0000 7fba92445700 1 -- 172.21.15.157:0/2076916651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba8c06d460 msgr2=0x7fba78003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:46.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.784+0000 7fba92445700 1 --2- 172.21.15.157:0/2076916651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba8c06d460 0x7fba78003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:46.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.784+0000 7fba92445700 1 -- 172.21.15.157:0/2076916651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba78004180 msgr2=0x7fba78004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:46.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.784+0000 7fba92445700 1 --2- 172.21.15.157:0/2076916651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba78004180 0x7fba78004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:46.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.784+0000 7fba92445700 1 -- 172.21.15.157:0/2076916651 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba78003570 con 0x7fba78004d50 2023-10-06T04:07:46.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.785+0000 7fba92445700 1 --2- 172.21.15.157:0/2076916651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba78004d50 0x7fba780051c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fba8400e210 tx=0x7fba8400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa65e8210741cf85 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:46.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.785+0000 7fba90c42700 1 -- 172.21.15.157:0/2076916651 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8400eac0 con 0x7fba78004d50 2023-10-06T04:07:46.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.785+0000 7fba90c42700 1 -- 172.21.15.157:0/2076916651 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba8400f040 con 0x7fba78004d50 2023-10-06T04:07:46.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.785+0000 7fba90c42700 1 -- 172.21.15.157:0/2076916651 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8400ee70 con 0x7fba78004d50 2023-10-06T04:07:46.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.785+0000 7fba93ea8700 1 -- 172.21.15.157:0/2076916651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba78004d50 msgr2=0x7fba780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:46.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.785+0000 7fba93ea8700 1 --2- 172.21.15.157:0/2076916651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba78004d50 0x7fba780051c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fba8400e210 tx=0x7fba8400b040 comp rx=0 tx=0).stop 2023-10-06T04:07:46.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.785+0000 7fba92445700 1 -- 172.21.15.157:0/2076916651 reap_dead start 2023-10-06T04:07:46.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.785+0000 7fba93ea8700 1 -- 172.21.15.157:0/2076916651 shutdown_connections 2023-10-06T04:07:46.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.785+0000 7fba93ea8700 1 -- 172.21.15.157:0/2076916651 >> 172.21.15.157:0/2076916651 conn(0x7fba8c0c8d70 msgr2=0x7fba8c0c4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:46.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.785+0000 7fba93ea8700 1 -- 172.21.15.157:0/2076916651 shutdown_connections 2023-10-06T04:07:46.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.785+0000 7fba93ea8700 1 -- 172.21.15.157:0/2076916651 wait complete. 2023-10-06T04:07:46.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.786+0000 7fba93ea8700 1 Processor -- start 2023-10-06T04:07:46.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.786+0000 7fba93ea8700 1 -- start start 2023-10-06T04:07:46.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.787+0000 7fba93ea8700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba8c0696b0 0x7fba8c069a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:46.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.787+0000 7fba93ea8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba8c14f5c0 0x7fba8c14fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:46.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.787+0000 7fba93ea8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba8c0c1a30 0x7fba8c0c1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:46.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.787+0000 7fba93ea8700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c1679a0 con 0x7fba8c14f5c0 2023-10-06T04:07:46.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.787+0000 7fba93ea8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c0c7910 con 0x7fba8c0c1a30 2023-10-06T04:07:46.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.787+0000 7fba93ea8700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c167370 con 0x7fba8c0696b0 2023-10-06T04:07:46.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.787+0000 7fba91443700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba8c14f5c0 0x7fba8c14fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:46.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.787+0000 7fba91443700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba8c14f5c0 0x7fba8c14fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36926/0 (socket says 172.21.15.157:36926) 2023-10-06T04:07:46.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.787+0000 7fba91443700 1 -- 172.21.15.157:0/1630442057 learned_addr learned my addr 172.21.15.157:0/1630442057 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:07:46.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.787+0000 7fba91c44700 1 --2- 172.21.15.157:0/1630442057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba8c0696b0 0x7fba8c069a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:46.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.787+0000 7fba92445700 1 --2- 172.21.15.157:0/1630442057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba8c0c1a30 0x7fba8c0c1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:46.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.787+0000 7fba91443700 1 -- 172.21.15.157:0/1630442057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba8c0696b0 msgr2=0x7fba8c069a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:46.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.787+0000 7fba91443700 1 --2- 172.21.15.157:0/1630442057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba8c0696b0 0x7fba8c069a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:46.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.787+0000 7fba91443700 1 -- 172.21.15.157:0/1630442057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba8c0c1a30 msgr2=0x7fba8c0c1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:46.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.787+0000 7fba91443700 1 --2- 172.21.15.157:0/1630442057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba8c0c1a30 0x7fba8c0c1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:46.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.787+0000 7fba91443700 1 -- 172.21.15.157:0/1630442057 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba8c1502b0 con 0x7fba8c14f5c0 2023-10-06T04:07:46.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.788+0000 7fba91443700 1 --2- 172.21.15.157:0/1630442057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba8c14f5c0 0x7fba8c14fa10 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fba7c0073c0 tx=0x7fba7c001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:46.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.788+0000 7fba81ffb700 1 -- 172.21.15.157:0/1630442057 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c00f620 con 0x7fba8c14f5c0 2023-10-06T04:07:46.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.788+0000 7fba81ffb700 1 -- 172.21.15.157:0/1630442057 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7c00f7a0 con 0x7fba8c14f5c0 2023-10-06T04:07:46.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.788+0000 7fba93ea8700 1 -- 172.21.15.157:0/1630442057 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba8c150590 con 0x7fba8c14f5c0 2023-10-06T04:07:46.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.788+0000 7fba81ffb700 1 -- 172.21.15.157:0/1630442057 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c00fb60 con 0x7fba8c14f5c0 2023-10-06T04:07:46.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.788+0000 7fba93ea8700 1 -- 172.21.15.157:0/1630442057 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba8c14f0a0 con 0x7fba8c14f5c0 2023-10-06T04:07:46.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.789+0000 7fba93ea8700 1 -- 172.21.15.157:0/1630442057 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba8c14ef00 con 0x7fba8c14f5c0 2023-10-06T04:07:46.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.789+0000 7fba81ffb700 1 -- 172.21.15.157:0/1630442057 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fba7c01e3b0 con 0x7fba8c14f5c0 2023-10-06T04:07:46.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.790+0000 7fba81ffb700 1 --2- 172.21.15.157:0/1630442057 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fba70066f30 0x7fba700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:46.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.790+0000 7fba81ffb700 1 -- 172.21.15.157:0/1630442057 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6546+0+0 (secure 0 0 0) 0x7fba7c0844d0 con 0x7fba8c14f5c0 2023-10-06T04:07:46.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.790+0000 7fba91c44700 1 --2- 172.21.15.157:0/1630442057 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fba70066f30 0x7fba700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:46.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.790+0000 7fba91c44700 1 --2- 172.21.15.157:0/1630442057 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fba70066f30 0x7fba700693e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fba88002c40 tx=0x7fba88007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:46.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:46.794+0000 7fba81ffb700 1 -- 172.21.15.157:0/1630442057 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba8c14ef00 con 0x7fba8c14f5c0 2023-10-06T04:07:47.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:47.022+0000 7fba93ea8700 1 -- 172.21.15.157:0/1630442057 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba8c14ef00 con 0x7fba8c14f5c0 2023-10-06T04:07:47.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:47.023+0000 7fba81ffb700 1 -- 172.21.15.157:0/1630442057 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+5390 (secure 0 0 0) 0x7fba8c14ef00 con 0x7fba8c14f5c0 2023-10-06T04:07:47.026 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:07:47.026 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:07:43.992831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5698},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5698":{"gid":5698,"name":"e","rank":1,"incarnation":72,"state":"up:reconnect","state_seq":29,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:07:47.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:47.025+0000 7fba93ea8700 1 -- 172.21.15.157:0/1630442057 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fba70066f30 msgr2=0x7fba700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:47.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:47.025+0000 7fba93ea8700 1 --2- 172.21.15.157:0/1630442057 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fba70066f30 0x7fba700693e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fba88002c40 tx=0x7fba88007d30 comp rx=0 tx=0).stop 2023-10-06T04:07:47.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:47.025+0000 7fba93ea8700 1 -- 172.21.15.157:0/1630442057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba8c14f5c0 msgr2=0x7fba8c14fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:47.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:47.025+0000 7fba93ea8700 1 --2- 172.21.15.157:0/1630442057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba8c14f5c0 0x7fba8c14fa10 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fba7c0073c0 tx=0x7fba7c001e10 comp rx=0 tx=0).stop 2023-10-06T04:07:47.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:47.025+0000 7fba93ea8700 1 -- 172.21.15.157:0/1630442057 shutdown_connections 2023-10-06T04:07:47.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:47.025+0000 7fba93ea8700 1 --2- 172.21.15.157:0/1630442057 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fba70066f30 0x7fba700693e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:47.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:47.025+0000 7fba93ea8700 1 --2- 172.21.15.157:0/1630442057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba8c0c1a30 0x7fba8c0c1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:47.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:47.025+0000 7fba93ea8700 1 --2- 172.21.15.157:0/1630442057 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba8c0696b0 0x7fba8c069a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:47.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:47.025+0000 7fba93ea8700 1 --2- 172.21.15.157:0/1630442057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba8c14f5c0 0x7fba8c14fa10 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:47.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:47.025+0000 7fba93ea8700 1 -- 172.21.15.157:0/1630442057 >> 172.21.15.157:0/1630442057 conn(0x7fba8c0c8d70 msgr2=0x7fba8c06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:47.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:47.025+0000 7fba93ea8700 1 -- 172.21.15.157:0/1630442057 shutdown_connections 2023-10-06T04:07:47.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:47.025+0000 7fba93ea8700 1 -- 172.21.15.157:0/1630442057 wait complete. 2023-10-06T04:07:47.028 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 74 2023-10-06T04:07:47.039 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:07:49.040 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:07:49.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.167+0000 7f35b9654700 1 Processor -- start 2023-10-06T04:07:49.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.167+0000 7f35b9654700 1 -- start start 2023-10-06T04:07:49.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.168+0000 7f35b9654700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35b4150350 0x7f35b4159e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:49.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.168+0000 7f35b9654700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f35b40c8560 0x7f35b40c1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:49.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.168+0000 7f35b9654700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35b40c1500 con 0x7f35b40c8190 2023-10-06T04:07:49.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.168+0000 7f35b9654700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b40b25f0 con 0x7f35b40c8560 2023-10-06T04:07:49.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.168+0000 7f35b9654700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b415a360 con 0x7f35b4150350 2023-10-06T04:07:49.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.168+0000 7f35b37fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f35b40c8560 0x7f35b40c1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:49.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.168+0000 7f35b2ffd700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f35b40c8190 0x7f35b414fc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:57444/0 (socket says 172.21.15.157:57444) 2023-10-06T04:07:49.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.168+0000 7f35b2ffd700 1 -- 172.21.15.157:0/3895714416 learned_addr learned my addr 172.21.15.157:0/3895714416 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:07:49.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.168+0000 7f35abfff700 1 --2- 172.21.15.157:0/3895714416 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35b4150350 0x7f35b4159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:49.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.168+0000 7f35b37fe700 1 -- 172.21.15.157:0/3895714416 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35b4150350 msgr2=0x7f35b4159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:49.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.168+0000 7f35b37fe700 1 --2- 172.21.15.157:0/3895714416 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35b4150350 0x7f35b4159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:49.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.168+0000 7f35b37fe700 1 -- 172.21.15.157:0/3895714416 >> v1:172.21.15.157:6789/0 conn(0x7f35b40c8190 legacy=0x7f35b414fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:49.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.168+0000 7f35b27fc700 1 -- 172.21.15.157:0/3895714416 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1370323331 0 0) 0x7f35b40c1500 con 0x7f35b40c8190 2023-10-06T04:07:49.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.168+0000 7f35b37fe700 1 -- 172.21.15.157:0/3895714416 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b40c8c80 con 0x7f35b40c8560 2023-10-06T04:07:49.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.168+0000 7f35b37fe700 1 --2- 172.21.15.157:0/3895714416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b40c8560 0x7f35b40c1990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009e60 tx=0x7f35ac009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863fb037e597abca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:49.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.168+0000 7f35b27fc700 1 -- 172.21.15.157:0/3895714416 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ac00e070 con 0x7f35b40c8560 2023-10-06T04:07:49.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.168+0000 7f35b27fc700 1 -- 172.21.15.157:0/3895714416 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35ac00bc30 con 0x7f35b40c8560 2023-10-06T04:07:49.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.168+0000 7f35b27fc700 1 -- 172.21.15.157:0/3895714416 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ac012430 con 0x7f35b40c8560 2023-10-06T04:07:49.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.169+0000 7f35b9654700 1 -- 172.21.15.157:0/3895714416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b40c8560 msgr2=0x7f35b40c1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:49.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.169+0000 7f35b9654700 1 --2- 172.21.15.157:0/3895714416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b40c8560 0x7f35b40c1990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009e60 tx=0x7f35ac009780 comp rx=0 tx=0).stop 2023-10-06T04:07:49.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.169+0000 7f35b9654700 1 -- 172.21.15.157:0/3895714416 shutdown_connections 2023-10-06T04:07:49.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.169+0000 7f35b9654700 1 --2- 172.21.15.157:0/3895714416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b40c8560 0x7f35b40c1990 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:49.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.169+0000 7f35b9654700 1 --2- 172.21.15.157:0/3895714416 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35b4150350 0x7f35b4159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:49.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.169+0000 7f35b9654700 1 -- 172.21.15.157:0/3895714416 >> 172.21.15.157:0/3895714416 conn(0x7f35b40631f0 msgr2=0x7f35b40635f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:49.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.169+0000 7f35b9654700 1 -- 172.21.15.157:0/3895714416 shutdown_connections 2023-10-06T04:07:49.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.169+0000 7f35b9654700 1 -- 172.21.15.157:0/3895714416 wait complete. 2023-10-06T04:07:49.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.170+0000 7f35b9654700 1 Processor -- start 2023-10-06T04:07:49.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.170+0000 7f35b9654700 1 -- start start 2023-10-06T04:07:49.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.171+0000 7f35b9654700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b40c8190 0x7f35b4145460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:49.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.171+0000 7f35b9654700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35b40c8560 0x7f35b40b1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:49.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.171+0000 7f35b9654700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35b4150350 0x7f35b4144110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:49.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.171+0000 7f35b9654700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b40b01d0 con 0x7f35b40c8190 2023-10-06T04:07:49.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.171+0000 7f35b9654700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b40afe80 con 0x7f35b4150350 2023-10-06T04:07:49.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.171+0000 7f35b9654700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b4166ab0 con 0x7f35b40c8560 2023-10-06T04:07:49.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.171+0000 7f35b2ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b40c8190 0x7f35b4145460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:49.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.171+0000 7f35b2ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b40c8190 0x7f35b4145460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36934/0 (socket says 172.21.15.157:36934) 2023-10-06T04:07:49.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.171+0000 7f35b2ffd700 1 -- 172.21.15.157:0/461064187 learned_addr learned my addr 172.21.15.157:0/461064187 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:07:49.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.171+0000 7f35b2ffd700 1 -- 172.21.15.157:0/461064187 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35b40c8560 msgr2=0x7f35b40b1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:49.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.171+0000 7f35abfff700 1 --2- 172.21.15.157:0/461064187 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35b40c8560 0x7f35b40b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:49.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.171+0000 7f35b37fe700 1 --2- 172.21.15.157:0/461064187 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35b4150350 0x7f35b4144110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:49.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.171+0000 7f35b2ffd700 1 --2- 172.21.15.157:0/461064187 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35b40c8560 0x7f35b40b1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:49.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.171+0000 7f35b2ffd700 1 -- 172.21.15.157:0/461064187 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35b4150350 msgr2=0x7f35b4144110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:49.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.171+0000 7f35b2ffd700 1 --2- 172.21.15.157:0/461064187 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35b4150350 0x7f35b4144110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:49.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.171+0000 7f35b2ffd700 1 -- 172.21.15.157:0/461064187 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ac009a20 con 0x7f35b40c8190 2023-10-06T04:07:49.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.171+0000 7f35abfff700 1 --2- 172.21.15.157:0/461064187 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35b40c8560 0x7f35b40b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:07:49.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.171+0000 7f35b2ffd700 1 --2- 172.21.15.157:0/461064187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b40c8190 0x7f35b4145460 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f35a400c7e0 tx=0x7f35a4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:49.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.172+0000 7f35aaffd700 1 -- 172.21.15.157:0/461064187 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a4005b30 con 0x7f35b40c8190 2023-10-06T04:07:49.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.172+0000 7f35aaffd700 1 -- 172.21.15.157:0/461064187 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a4005c90 con 0x7f35b40c8190 2023-10-06T04:07:49.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.172+0000 7f35aaffd700 1 -- 172.21.15.157:0/461064187 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a4007300 con 0x7f35b40c8190 2023-10-06T04:07:49.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.172+0000 7f35b9654700 1 -- 172.21.15.157:0/461064187 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b41667d0 con 0x7f35b40c8190 2023-10-06T04:07:49.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.172+0000 7f35b9654700 1 -- 172.21.15.157:0/461064187 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b414cc30 con 0x7f35b40c8190 2023-10-06T04:07:49.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.172+0000 7f35b9654700 1 -- 172.21.15.157:0/461064187 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b4089370 con 0x7f35b40c8190 2023-10-06T04:07:49.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.173+0000 7f35aaffd700 1 -- 172.21.15.157:0/461064187 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f35a401b400 con 0x7f35b40c8190 2023-10-06T04:07:49.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.173+0000 7f35aaffd700 1 --2- 172.21.15.157:0/461064187 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3594066f60 0x7f3594069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:49.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.173+0000 7f35aaffd700 1 -- 172.21.15.157:0/461064187 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6546+0+0 (secure 0 0 0) 0x7f35a40842e0 con 0x7f35b40c8190 2023-10-06T04:07:49.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.173+0000 7f35abfff700 1 --2- 172.21.15.157:0/461064187 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3594066f60 0x7f3594069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:49.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.174+0000 7f35abfff700 1 --2- 172.21.15.157:0/461064187 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3594066f60 0x7f3594069410 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f35b414b9e0 tx=0x7f359c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:49.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.177+0000 7f35aaffd700 1 -- 172.21.15.157:0/461064187 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f35a4051280 con 0x7f35b40c8190 2023-10-06T04:07:49.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.404+0000 7f35b9654700 1 -- 172.21.15.157:0/461064187 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35b40c3630 con 0x7f35b40c8190 2023-10-06T04:07:49.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.404+0000 7f35aaffd700 1 -- 172.21.15.157:0/461064187 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+5387 (secure 0 0 0) 0x7f35a4007e60 con 0x7f35b40c8190 2023-10-06T04:07:49.406 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:07:49.406 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:07:49.007090+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5698},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5698":{"gid":5698,"name":"e","rank":1,"incarnation":72,"state":"up:rejoin","state_seq":31,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:07:49.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.406+0000 7f35b9654700 1 -- 172.21.15.157:0/461064187 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3594066f60 msgr2=0x7f3594069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:49.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.406+0000 7f35b9654700 1 --2- 172.21.15.157:0/461064187 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3594066f60 0x7f3594069410 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f35b414b9e0 tx=0x7f359c008040 comp rx=0 tx=0).stop 2023-10-06T04:07:49.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.406+0000 7f35b9654700 1 -- 172.21.15.157:0/461064187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b40c8190 msgr2=0x7f35b4145460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:49.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.406+0000 7f35b9654700 1 --2- 172.21.15.157:0/461064187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b40c8190 0x7f35b4145460 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f35a400c7e0 tx=0x7f35a4007df0 comp rx=0 tx=0).stop 2023-10-06T04:07:49.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.406+0000 7f35b9654700 1 -- 172.21.15.157:0/461064187 shutdown_connections 2023-10-06T04:07:49.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.406+0000 7f35b9654700 1 --2- 172.21.15.157:0/461064187 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3594066f60 0x7f3594069410 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:49.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.406+0000 7f35b9654700 1 --2- 172.21.15.157:0/461064187 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35b4150350 0x7f35b4144110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:49.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.406+0000 7f35b9654700 1 --2- 172.21.15.157:0/461064187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b40c8190 0x7f35b4145460 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:49.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.406+0000 7f35b9654700 1 --2- 172.21.15.157:0/461064187 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35b40c8560 0x7f35b40b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:49.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.406+0000 7f35b9654700 1 -- 172.21.15.157:0/461064187 >> 172.21.15.157:0/461064187 conn(0x7f35b40631f0 msgr2=0x7f35b40ca360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:49.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.406+0000 7f35b9654700 1 -- 172.21.15.157:0/461064187 shutdown_connections 2023-10-06T04:07:49.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:49.406+0000 7f35b9654700 1 -- 172.21.15.157:0/461064187 wait complete. 2023-10-06T04:07:49.410 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 75 2023-10-06T04:07:49.420 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:07:51.420 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:07:51.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.546+0000 7f2454923700 1 Processor -- start 2023-10-06T04:07:51.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.546+0000 7f2454923700 1 -- start start 2023-10-06T04:07:51.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.547+0000 7f2454923700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f245014dcd0 0x7f2450153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:51.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.547+0000 7f2454923700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f245014e0a0 0x7f24500c18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:51.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.547+0000 7f2454923700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24500c5770 con 0x7f24500c5c00 2023-10-06T04:07:51.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.547+0000 7f2454923700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2450154c80 con 0x7f245014e0a0 2023-10-06T04:07:51.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.547+0000 7f2454923700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2450154200 con 0x7f245014dcd0 2023-10-06T04:07:51.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.547+0000 7f244ed9d700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f24500c5c00 0x7f24500ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:57450/0 (socket says 172.21.15.157:57450) 2023-10-06T04:07:51.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.547+0000 7f244ed9d700 1 -- 172.21.15.157:0/1359670474 learned_addr learned my addr 172.21.15.157:0/1359670474 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:07:51.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.548+0000 7f2447fff700 1 --2- 172.21.15.157:0/1359670474 >> v2:172.21.15.175:3300/0 conn(0x7f245014e0a0 0x7f24500c18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:51.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.548+0000 7f244e59c700 1 --2- 172.21.15.157:0/1359670474 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f245014dcd0 0x7f2450153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:51.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.548+0000 7f244dd9b700 1 -- 172.21.15.157:0/1359670474 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483414098 0 0) 0x7f24500c5770 con 0x7f24500c5c00 2023-10-06T04:07:51.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.548+0000 7f244dd9b700 1 -- 172.21.15.157:0/1359670474 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f243c003570 con 0x7f24500c5c00 2023-10-06T04:07:51.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.548+0000 7f244dd9b700 1 -- 172.21.15.157:0/1359670474 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 959481527 0 0) 0x7f243c003570 con 0x7f24500c5c00 2023-10-06T04:07:51.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.548+0000 7f244dd9b700 1 -- 172.21.15.157:0/1359670474 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f245014dcd0 msgr2=0x7f2450153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:51.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.548+0000 7f244dd9b700 1 --2- 172.21.15.157:0/1359670474 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f245014dcd0 0x7f2450153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:51.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.548+0000 7f244dd9b700 1 -- 172.21.15.157:0/1359670474 >> v2:172.21.15.175:3300/0 conn(0x7f245014e0a0 msgr2=0x7f24500c18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:51.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.548+0000 7f244dd9b700 1 --2- 172.21.15.157:0/1359670474 >> v2:172.21.15.175:3300/0 conn(0x7f245014e0a0 0x7f24500c18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:51.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.548+0000 7f244dd9b700 1 -- 172.21.15.157:0/1359670474 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24501543e0 con 0x7f24500c5c00 2023-10-06T04:07:51.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.548+0000 7f244dd9b700 1 -- 172.21.15.157:0/1359670474 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f24480017f0 con 0x7f24500c5c00 2023-10-06T04:07:51.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.548+0000 7f244dd9b700 1 -- 172.21.15.157:0/1359670474 >> v1:172.21.15.157:6789/0 conn(0x7f24500c5c00 legacy=0x7f24500ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:51.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.548+0000 7f244dd9b700 1 --2- 172.21.15.157:0/1359670474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f243c003a60 0x7f243c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:51.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.548+0000 7f244dd9b700 1 --2- 172.21.15.157:0/1359670474 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f245014dcd0 0x7f243c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:51.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.548+0000 7f244dd9b700 1 --2- 172.21.15.157:0/1359670474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f243c004d50 0x7f243c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:51.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.548+0000 7f244dd9b700 1 -- 172.21.15.157:0/1359670474 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2450154c80 con 0x7f243c003a60 2023-10-06T04:07:51.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.548+0000 7f244dd9b700 1 -- 172.21.15.157:0/1359670474 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2450154200 con 0x7f243c004d50 2023-10-06T04:07:51.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.548+0000 7f244dd9b700 1 -- 172.21.15.157:0/1359670474 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f243c00ba60 con 0x7f245014dcd0 2023-10-06T04:07:51.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.548+0000 7f244e59c700 1 --2- 172.21.15.157:0/1359670474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f243c003a60 0x7f243c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:51.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.548+0000 7f2447fff700 1 --2- 172.21.15.157:0/1359670474 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f245014dcd0 0x7f243c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:51.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.548+0000 7f244ed9d700 1 --2- 172.21.15.157:0/1359670474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f243c004d50 0x7f243c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:51.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.548+0000 7f244e59c700 1 -- 172.21.15.157:0/1359670474 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f245014dcd0 msgr2=0x7f243c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:51.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.548+0000 7f244e59c700 1 --2- 172.21.15.157:0/1359670474 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f245014dcd0 0x7f243c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:51.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.548+0000 7f244e59c700 1 -- 172.21.15.157:0/1359670474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f243c004d50 msgr2=0x7f243c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:51.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.548+0000 7f244e59c700 1 --2- 172.21.15.157:0/1359670474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f243c004d50 0x7f243c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:51.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.548+0000 7f244e59c700 1 -- 172.21.15.157:0/1359670474 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243c003570 con 0x7f243c003a60 2023-10-06T04:07:51.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.549+0000 7f244e59c700 1 --2- 172.21.15.157:0/1359670474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f243c003a60 0x7f243c003e70 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2440009c00 tx=0x7f24400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed634921ed234f77 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:51.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.549+0000 7f244dd9b700 1 -- 172.21.15.157:0/1359670474 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244000e6b0 con 0x7f243c003a60 2023-10-06T04:07:51.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.549+0000 7f244dd9b700 1 -- 172.21.15.157:0/1359670474 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f244000e810 con 0x7f243c003a60 2023-10-06T04:07:51.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.549+0000 7f244dd9b700 1 -- 172.21.15.157:0/1359670474 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244000ebd0 con 0x7f243c003a60 2023-10-06T04:07:51.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.549+0000 7f2454923700 1 -- 172.21.15.157:0/1359670474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f243c003a60 msgr2=0x7f243c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:51.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.549+0000 7f2454923700 1 --2- 172.21.15.157:0/1359670474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f243c003a60 0x7f243c003e70 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2440009c00 tx=0x7f24400091a0 comp rx=0 tx=0).stop 2023-10-06T04:07:51.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.549+0000 7f244ed9d700 1 -- 172.21.15.157:0/1359670474 reap_dead start 2023-10-06T04:07:51.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.549+0000 7f2454923700 1 -- 172.21.15.157:0/1359670474 shutdown_connections 2023-10-06T04:07:51.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.549+0000 7f2454923700 1 -- 172.21.15.157:0/1359670474 >> 172.21.15.157:0/1359670474 conn(0x7f24500692b0 msgr2=0x7f24500696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:51.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.549+0000 7f2454923700 1 -- 172.21.15.157:0/1359670474 shutdown_connections 2023-10-06T04:07:51.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.549+0000 7f2454923700 1 -- 172.21.15.157:0/1359670474 wait complete. 2023-10-06T04:07:51.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.550+0000 7f2454923700 1 Processor -- start 2023-10-06T04:07:51.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.550+0000 7f2454923700 1 -- start start 2023-10-06T04:07:51.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.551+0000 7f2454923700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f243c003a60 0x7f2450151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:51.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.551+0000 7f2454923700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24501560c0 0x7f2450152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:51.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.551+0000 7f2454923700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24501484c0 0x7f2450148920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:51.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.551+0000 7f2454923700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2450153e30 con 0x7f243c003a60 2023-10-06T04:07:51.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.551+0000 7f2454923700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24500c8760 con 0x7f24501560c0 2023-10-06T04:07:51.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.551+0000 7f2454923700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2450148e60 con 0x7f24501484c0 2023-10-06T04:07:51.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.551+0000 7f244e59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f243c003a60 0x7f2450151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:51.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.551+0000 7f244e59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f243c003a60 0x7f2450151b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36942/0 (socket says 172.21.15.157:36942) 2023-10-06T04:07:51.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.551+0000 7f244e59c700 1 -- 172.21.15.157:0/588807822 learned_addr learned my addr 172.21.15.157:0/588807822 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:07:51.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.551+0000 7f2447fff700 1 --2- 172.21.15.157:0/588807822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24501560c0 0x7f2450152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:51.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.551+0000 7f244ed9d700 1 --2- 172.21.15.157:0/588807822 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24501484c0 0x7f2450148920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:51.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.551+0000 7f244e59c700 1 -- 172.21.15.157:0/588807822 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24501484c0 msgr2=0x7f2450148920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:51.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.551+0000 7f244e59c700 1 --2- 172.21.15.157:0/588807822 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24501484c0 0x7f2450148920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:51.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.551+0000 7f244e59c700 1 -- 172.21.15.157:0/588807822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24501560c0 msgr2=0x7f2450152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:51.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.551+0000 7f244e59c700 1 --2- 172.21.15.157:0/588807822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24501560c0 0x7f2450152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:51.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.551+0000 7f244e59c700 1 -- 172.21.15.157:0/588807822 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24501538d0 con 0x7f243c003a60 2023-10-06T04:07:51.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.552+0000 7f244e59c700 1 --2- 172.21.15.157:0/588807822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f243c003a60 0x7f2450151b30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2440013ef0 tx=0x7f244001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:51.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.552+0000 7f24467fc700 1 -- 172.21.15.157:0/588807822 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2440022070 con 0x7f243c003a60 2023-10-06T04:07:51.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.552+0000 7f24467fc700 1 -- 172.21.15.157:0/588807822 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f244000e980 con 0x7f243c003a60 2023-10-06T04:07:51.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.552+0000 7f2454923700 1 -- 172.21.15.157:0/588807822 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24500c0eb0 con 0x7f243c003a60 2023-10-06T04:07:51.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.552+0000 7f24467fc700 1 -- 172.21.15.157:0/588807822 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2440027470 con 0x7f243c003a60 2023-10-06T04:07:51.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.552+0000 7f2454923700 1 -- 172.21.15.157:0/588807822 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2450153b50 con 0x7f243c003a60 2023-10-06T04:07:51.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.553+0000 7f2454923700 1 -- 172.21.15.157:0/588807822 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2450153d70 con 0x7f243c003a60 2023-10-06T04:07:51.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.553+0000 7f24467fc700 1 -- 172.21.15.157:0/588807822 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f244001c450 con 0x7f243c003a60 2023-10-06T04:07:51.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.554+0000 7f24467fc700 1 --2- 172.21.15.157:0/588807822 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2430066f30 0x7f24300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:51.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.554+0000 7f24467fc700 1 -- 172.21.15.157:0/588807822 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6546+0+0 (secure 0 0 0) 0x7f24400864b0 con 0x7f243c003a60 2023-10-06T04:07:51.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.554+0000 7f2447fff700 1 --2- 172.21.15.157:0/588807822 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2430066f30 0x7f24300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:51.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.554+0000 7f2447fff700 1 --2- 172.21.15.157:0/588807822 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2430066f30 0x7f24300693e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2438006dc0 tx=0x7f2438008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:51.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.558+0000 7f24467fc700 1 -- 172.21.15.157:0/588807822 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2440053450 con 0x7f243c003a60 2023-10-06T04:07:51.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.786+0000 7f2454923700 1 -- 172.21.15.157:0/588807822 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2450067ce0 con 0x7f243c003a60 2023-10-06T04:07:51.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.786+0000 7f24467fc700 1 -- 172.21.15.157:0/588807822 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+5387 (secure 0 0 0) 0x7f24400569f0 con 0x7f243c003a60 2023-10-06T04:07:51.788 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:07:51.788 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:07:49.007090+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5698},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5698":{"gid":5698,"name":"e","rank":1,"incarnation":72,"state":"up:rejoin","state_seq":31,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:07:51.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.788+0000 7f2454923700 1 -- 172.21.15.157:0/588807822 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2430066f30 msgr2=0x7f24300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:51.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.788+0000 7f2454923700 1 --2- 172.21.15.157:0/588807822 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2430066f30 0x7f24300693e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2438006dc0 tx=0x7f2438008040 comp rx=0 tx=0).stop 2023-10-06T04:07:51.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.788+0000 7f2454923700 1 -- 172.21.15.157:0/588807822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f243c003a60 msgr2=0x7f2450151b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:51.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.788+0000 7f2454923700 1 --2- 172.21.15.157:0/588807822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f243c003a60 0x7f2450151b30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2440013ef0 tx=0x7f244001a000 comp rx=0 tx=0).stop 2023-10-06T04:07:51.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.788+0000 7f2454923700 1 -- 172.21.15.157:0/588807822 shutdown_connections 2023-10-06T04:07:51.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.788+0000 7f2454923700 1 --2- 172.21.15.157:0/588807822 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2430066f30 0x7f24300693e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:51.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.788+0000 7f2454923700 1 --2- 172.21.15.157:0/588807822 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f24501484c0 0x7f2450148920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:51.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.788+0000 7f2454923700 1 --2- 172.21.15.157:0/588807822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f243c003a60 0x7f2450151b30 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:51.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.788+0000 7f2454923700 1 --2- 172.21.15.157:0/588807822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24501560c0 0x7f2450152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:51.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.788+0000 7f2454923700 1 -- 172.21.15.157:0/588807822 >> 172.21.15.157:0/588807822 conn(0x7f24500692b0 msgr2=0x7f245014cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:51.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.788+0000 7f2454923700 1 -- 172.21.15.157:0/588807822 shutdown_connections 2023-10-06T04:07:51.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:51.788+0000 7f2454923700 1 -- 172.21.15.157:0/588807822 wait complete. 2023-10-06T04:07:51.792 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 75 2023-10-06T04:07:51.802 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:07:53.802 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:07:53.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.929+0000 7efc70d4c700 1 Processor -- start 2023-10-06T04:07:53.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.929+0000 7efc70d4c700 1 -- start start 2023-10-06T04:07:53.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.930+0000 7efc70d4c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efc6c06cbb0 0x7efc6c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:53.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.930+0000 7efc70d4c700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc6c0c9780 con 0x7efc6c06c1f0 2023-10-06T04:07:53.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.930+0000 7efc70d4c700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc6c06e0c0 con 0x7efc6c150410 2023-10-06T04:07:53.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.930+0000 7efc70d4c700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc6c0c8c40 con 0x7efc6c06cbb0 2023-10-06T04:07:53.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.930+0000 7efc6a59c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efc6c06cbb0 0x7efc6c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:53.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.930+0000 7efc6a59c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efc6c06cbb0 0x7efc6c06d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38474/0 (socket says 172.21.15.157:38474) 2023-10-06T04:07:53.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.930+0000 7efc69d9b700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7efc6c06c1f0 0x7efc6c14fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:48818/0 (socket says 172.21.15.157:48818) 2023-10-06T04:07:53.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.930+0000 7efc6a59c700 1 -- 172.21.15.157:0/1347905921 learned_addr learned my addr 172.21.15.157:0/1347905921 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:07:53.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.930+0000 7efc6a59c700 1 -- 172.21.15.157:0/1347905921 >> v1:172.21.15.175:6789/0 conn(0x7efc6c150410 legacy=0x7efc6c155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:53.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.930+0000 7efc6a59c700 1 -- 172.21.15.157:0/1347905921 >> v1:172.21.15.157:6789/0 conn(0x7efc6c06c1f0 legacy=0x7efc6c14fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:53.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.930+0000 7efc6a59c700 1 -- 172.21.15.157:0/1347905921 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc6c0c8e50 con 0x7efc6c06cbb0 2023-10-06T04:07:53.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.930+0000 7efc6959a700 1 -- 172.21.15.157:0/1347905921 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1334956594 0 0) 0x7efc6c0c9780 con 0x7efc6c06c1f0 2023-10-06T04:07:53.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.930+0000 7efc6a59c700 1 --2- 172.21.15.157:0/1347905921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc6c06cbb0 0x7efc6c06d530 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7efc5c0099b0 tx=0x7efc5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59585de80b9a3f4e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:53.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.930+0000 7efc6959a700 1 -- 172.21.15.157:0/1347905921 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5c00b8b0 con 0x7efc6c06cbb0 2023-10-06T04:07:53.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.931+0000 7efc6959a700 1 -- 172.21.15.157:0/1347905921 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc5c00ba10 con 0x7efc6c06cbb0 2023-10-06T04:07:53.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.931+0000 7efc6959a700 1 -- 172.21.15.157:0/1347905921 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5c00bdb0 con 0x7efc6c06cbb0 2023-10-06T04:07:53.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.931+0000 7efc70d4c700 1 -- 172.21.15.157:0/1347905921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc6c06cbb0 msgr2=0x7efc6c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:53.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.931+0000 7efc70d4c700 1 --2- 172.21.15.157:0/1347905921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc6c06cbb0 0x7efc6c06d530 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7efc5c0099b0 tx=0x7efc5c009250 comp rx=0 tx=0).stop 2023-10-06T04:07:53.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.931+0000 7efc70d4c700 1 -- 172.21.15.157:0/1347905921 shutdown_connections 2023-10-06T04:07:53.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.931+0000 7efc70d4c700 1 --2- 172.21.15.157:0/1347905921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc6c06cbb0 0x7efc6c06d530 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:53.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.931+0000 7efc70d4c700 1 -- 172.21.15.157:0/1347905921 >> 172.21.15.157:0/1347905921 conn(0x7efc6c06f230 msgr2=0x7efc6c06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:53.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.931+0000 7efc70d4c700 1 -- 172.21.15.157:0/1347905921 shutdown_connections 2023-10-06T04:07:53.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.931+0000 7efc70d4c700 1 -- 172.21.15.157:0/1347905921 wait complete. 2023-10-06T04:07:53.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.932+0000 7efc70d4c700 1 Processor -- start 2023-10-06T04:07:53.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.932+0000 7efc70d4c700 1 -- start start 2023-10-06T04:07:53.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.933+0000 7efc70d4c700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc6c06c1f0 0x7efc6c14ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:53.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.933+0000 7efc70d4c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc6c06cbb0 0x7efc6c146010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:53.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.933+0000 7efc70d4c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc6c150410 0x7efc6c146550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:53.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.933+0000 7efc70d4c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6c16aeb0 con 0x7efc6c06cbb0 2023-10-06T04:07:53.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.933+0000 7efc70d4c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6c166a50 con 0x7efc6c150410 2023-10-06T04:07:53.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.933+0000 7efc70d4c700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc6c166c60 con 0x7efc6c06c1f0 2023-10-06T04:07:53.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.933+0000 7efc69d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc6c06cbb0 0x7efc6c146010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:53.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.933+0000 7efc6a59c700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc6c06c1f0 0x7efc6c14ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:53.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.933+0000 7efc69d9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc6c06cbb0 0x7efc6c146010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38486/0 (socket says 172.21.15.157:38486) 2023-10-06T04:07:53.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.933+0000 7efc69d9b700 1 -- 172.21.15.157:0/3133140076 learned_addr learned my addr 172.21.15.157:0/3133140076 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:07:53.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.933+0000 7efc6ad9d700 1 --2- 172.21.15.157:0/3133140076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc6c150410 0x7efc6c146550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:53.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.934+0000 7efc69d9b700 1 -- 172.21.15.157:0/3133140076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc6c06c1f0 msgr2=0x7efc6c14ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:53.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.934+0000 7efc69d9b700 1 --2- 172.21.15.157:0/3133140076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc6c06c1f0 0x7efc6c14ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:53.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.934+0000 7efc69d9b700 1 -- 172.21.15.157:0/3133140076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc6c150410 msgr2=0x7efc6c146550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:53.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.934+0000 7efc69d9b700 1 --2- 172.21.15.157:0/3133140076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc6c150410 0x7efc6c146550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:53.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.934+0000 7efc69d9b700 1 -- 172.21.15.157:0/3133140076 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5c0094f0 con 0x7efc6c06cbb0 2023-10-06T04:07:53.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.934+0000 7efc69d9b700 1 --2- 172.21.15.157:0/3133140076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc6c06cbb0 0x7efc6c146010 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7efc600011e0 tx=0x7efc60007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:53.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.934+0000 7efc5a7fc700 1 -- 172.21.15.157:0/3133140076 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc60005a70 con 0x7efc6c06cbb0 2023-10-06T04:07:53.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.934+0000 7efc5a7fc700 1 -- 172.21.15.157:0/3133140076 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc60005bd0 con 0x7efc6c06cbb0 2023-10-06T04:07:53.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.934+0000 7efc70d4c700 1 -- 172.21.15.157:0/3133140076 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc6c166f40 con 0x7efc6c06cbb0 2023-10-06T04:07:53.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.934+0000 7efc5a7fc700 1 -- 172.21.15.157:0/3133140076 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc600070d0 con 0x7efc6c06cbb0 2023-10-06T04:07:53.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.934+0000 7efc70d4c700 1 -- 172.21.15.157:0/3133140076 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc6c167500 con 0x7efc6c06cbb0 2023-10-06T04:07:53.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.935+0000 7efc70d4c700 1 -- 172.21.15.157:0/3133140076 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc6c14d4b0 con 0x7efc6c06cbb0 2023-10-06T04:07:53.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.935+0000 7efc5a7fc700 1 -- 172.21.15.157:0/3133140076 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efc60007480 con 0x7efc6c06cbb0 2023-10-06T04:07:53.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.936+0000 7efc5a7fc700 1 --2- 172.21.15.157:0/3133140076 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7efc4c066f30 0x7efc4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:53.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.936+0000 7efc5a7fc700 1 -- 172.21.15.157:0/3133140076 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6546+0+0 (secure 0 0 0) 0x7efc60083da0 con 0x7efc6c06cbb0 2023-10-06T04:07:53.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.936+0000 7efc6a59c700 1 --2- 172.21.15.157:0/3133140076 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7efc4c066f30 0x7efc4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:53.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.936+0000 7efc6a59c700 1 --2- 172.21.15.157:0/3133140076 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7efc4c066f30 0x7efc4c0693e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efc5c0099b0 tx=0x7efc5c016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:53.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:53.940+0000 7efc5a7fc700 1 -- 172.21.15.157:0/3133140076 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efc60050cc0 con 0x7efc6c06cbb0 2023-10-06T04:07:54.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:54.179+0000 7efc70d4c700 1 -- 172.21.15.157:0/3133140076 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc6c167830 con 0x7efc6c06cbb0 2023-10-06T04:07:54.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:54.180+0000 7efc5a7fc700 1 -- 172.21.15.157:0/3133140076 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+5387 (secure 0 0 0) 0x7efc60017020 con 0x7efc6c06cbb0 2023-10-06T04:07:54.181 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:07:54.182 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:07:49.007090+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5698},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5698":{"gid":5698,"name":"e","rank":1,"incarnation":72,"state":"up:rejoin","state_seq":31,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:07:54.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:54.182+0000 7efc70d4c700 1 -- 172.21.15.157:0/3133140076 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7efc4c066f30 msgr2=0x7efc4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:54.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:54.182+0000 7efc70d4c700 1 --2- 172.21.15.157:0/3133140076 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7efc4c066f30 0x7efc4c0693e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efc5c0099b0 tx=0x7efc5c016000 comp rx=0 tx=0).stop 2023-10-06T04:07:54.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:54.182+0000 7efc70d4c700 1 -- 172.21.15.157:0/3133140076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc6c06cbb0 msgr2=0x7efc6c146010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:54.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:54.182+0000 7efc70d4c700 1 --2- 172.21.15.157:0/3133140076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc6c06cbb0 0x7efc6c146010 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7efc600011e0 tx=0x7efc60007e10 comp rx=0 tx=0).stop 2023-10-06T04:07:54.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:54.182+0000 7efc70d4c700 1 -- 172.21.15.157:0/3133140076 shutdown_connections 2023-10-06T04:07:54.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:54.182+0000 7efc70d4c700 1 --2- 172.21.15.157:0/3133140076 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7efc4c066f30 0x7efc4c0693e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:54.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:54.182+0000 7efc70d4c700 1 --2- 172.21.15.157:0/3133140076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc6c150410 0x7efc6c146550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:54.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:54.182+0000 7efc70d4c700 1 --2- 172.21.15.157:0/3133140076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc6c06c1f0 0x7efc6c14ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:54.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:54.182+0000 7efc70d4c700 1 --2- 172.21.15.157:0/3133140076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc6c06cbb0 0x7efc6c146010 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:54.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:54.182+0000 7efc70d4c700 1 -- 172.21.15.157:0/3133140076 >> 172.21.15.157:0/3133140076 conn(0x7efc6c06f230 msgr2=0x7efc6c06d530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:54.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:54.182+0000 7efc70d4c700 1 -- 172.21.15.157:0/3133140076 shutdown_connections 2023-10-06T04:07:54.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:54.182+0000 7efc70d4c700 1 -- 172.21.15.157:0/3133140076 wait complete. 2023-10-06T04:07:54.185 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 75 2023-10-06T04:07:54.196 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:07:56.198 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:07:56.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.316+0000 7f6883df0700 1 Processor -- start 2023-10-06T04:07:56.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.317+0000 7f6883df0700 1 -- start start 2023-10-06T04:07:56.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.317+0000 7f6883df0700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f687c06cbb0 0x7f687c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:56.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.317+0000 7f6883df0700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f687c06c1f0 0x7f687c06da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:56.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.317+0000 7f6883df0700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f687c0c9780 con 0x7f687c06cf80 2023-10-06T04:07:56.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.317+0000 7f6883df0700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f687c0c8c40 con 0x7f687c06cbb0 2023-10-06T04:07:56.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.317+0000 7f6883df0700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687c06e0c0 con 0x7f687c06c1f0 2023-10-06T04:07:56.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.318+0000 7f688238d700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f687c06cf80 0x7f687c14fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:48832/0 (socket says 172.21.15.157:48832) 2023-10-06T04:07:56.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.318+0000 7f688238d700 1 -- 172.21.15.157:0/4090925129 learned_addr learned my addr 172.21.15.157:0/4090925129 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:07:56.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.318+0000 7f688138b700 1 --2- 172.21.15.157:0/4090925129 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f687c06c1f0 0x7f687c06da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:56.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.318+0000 7f6881b8c700 1 --2- 172.21.15.157:0/4090925129 >> v2:172.21.15.175:3300/0 conn(0x7f687c06cbb0 0x7f687c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:56.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.318+0000 7f6880b8a700 1 -- 172.21.15.157:0/4090925129 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3201483742 0 0) 0x7f687c0c9780 con 0x7f687c06cf80 2023-10-06T04:07:56.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.318+0000 7f6880b8a700 1 -- 172.21.15.157:0/4090925129 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6868003570 con 0x7f687c06cf80 2023-10-06T04:07:56.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.318+0000 7f688138b700 1 -- 172.21.15.157:0/4090925129 >> v2:172.21.15.175:3300/0 conn(0x7f687c06cbb0 msgr2=0x7f687c06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:07:56.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.318+0000 7f6880b8a700 1 -- 172.21.15.157:0/4090925129 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1891029491 0 0) 0x7f6868003570 con 0x7f687c06cf80 2023-10-06T04:07:56.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.318+0000 7f688138b700 1 --2- 172.21.15.157:0/4090925129 >> v2:172.21.15.175:3300/0 conn(0x7f687c06cbb0 0x7f687c06d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:56.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.318+0000 7f688138b700 1 -- 172.21.15.157:0/4090925129 >> v1:172.21.15.157:6789/0 conn(0x7f687c06cf80 legacy=0x7f687c14fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:56.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.318+0000 7f688138b700 1 -- 172.21.15.157:0/4090925129 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687c0c8e50 con 0x7f687c06c1f0 2023-10-06T04:07:56.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.318+0000 7f688138b700 1 --2- 172.21.15.157:0/4090925129 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f687c06c1f0 0x7f687c06da70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f686c009910 tx=0x7f686c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc6da28911e026b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:56.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.318+0000 7f6880b8a700 1 -- 172.21.15.157:0/4090925129 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6874001470 con 0x7f687c06c1f0 2023-10-06T04:07:56.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.319+0000 7f6880b8a700 1 -- 172.21.15.157:0/4090925129 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686c011420 con 0x7f687c06c1f0 2023-10-06T04:07:56.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.319+0000 7f6880b8a700 1 -- 172.21.15.157:0/4090925129 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686c0117a0 con 0x7f687c06c1f0 2023-10-06T04:07:56.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.319+0000 7f6883df0700 1 -- 172.21.15.157:0/4090925129 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f687c06c1f0 msgr2=0x7f687c06da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:56.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.319+0000 7f6883df0700 1 --2- 172.21.15.157:0/4090925129 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f687c06c1f0 0x7f687c06da70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f686c009910 tx=0x7f686c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:07:56.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.319+0000 7f6883df0700 1 -- 172.21.15.157:0/4090925129 shutdown_connections 2023-10-06T04:07:56.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.319+0000 7f6883df0700 1 --2- 172.21.15.157:0/4090925129 >> v2:172.21.15.175:3300/0 conn(0x7f687c06cbb0 0x7f687c06d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:56.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.319+0000 7f6883df0700 1 --2- 172.21.15.157:0/4090925129 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f687c06c1f0 0x7f687c06da70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:56.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.319+0000 7f6883df0700 1 -- 172.21.15.157:0/4090925129 >> 172.21.15.157:0/4090925129 conn(0x7f687c06f230 msgr2=0x7f687c06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:56.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.319+0000 7f6883df0700 1 -- 172.21.15.157:0/4090925129 shutdown_connections 2023-10-06T04:07:56.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.319+0000 7f6883df0700 1 -- 172.21.15.157:0/4090925129 wait complete. 2023-10-06T04:07:56.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.320+0000 7f6883df0700 1 Processor -- start 2023-10-06T04:07:56.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.320+0000 7f6883df0700 1 -- start start 2023-10-06T04:07:56.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.321+0000 7f6883df0700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f687c06c1f0 0x7f687c0cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:56.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.321+0000 7f6883df0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f687c06cbb0 0x7f687c0c1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:56.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.321+0000 7f6883df0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f687c06cf80 0x7f687c0c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:56.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.321+0000 7f6883df0700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687c0c14c0 con 0x7f687c06cf80 2023-10-06T04:07:56.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.321+0000 7f6883df0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687c0c1170 con 0x7f687c06cbb0 2023-10-06T04:07:56.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.321+0000 7f6883df0700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687c155920 con 0x7f687c06c1f0 2023-10-06T04:07:56.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.321+0000 7f688238d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f687c06cf80 0x7f687c0c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:56.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.321+0000 7f688238d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f687c06cf80 0x7f687c0c1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38496/0 (socket says 172.21.15.157:38496) 2023-10-06T04:07:56.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.321+0000 7f688238d700 1 -- 172.21.15.157:0/2849044736 learned_addr learned my addr 172.21.15.157:0/2849044736 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:07:56.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.321+0000 7f6881b8c700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f687c06c1f0 0x7f687c0cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:56.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.321+0000 7f688138b700 1 --2- 172.21.15.157:0/2849044736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f687c06cbb0 0x7f687c0c1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:56.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.321+0000 7f688238d700 1 -- 172.21.15.157:0/2849044736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f687c06c1f0 msgr2=0x7f687c0cb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:56.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.321+0000 7f688238d700 1 --2- 172.21.15.157:0/2849044736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f687c06c1f0 0x7f687c0cb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:56.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.321+0000 7f688238d700 1 -- 172.21.15.157:0/2849044736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f687c06cbb0 msgr2=0x7f687c0c1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:56.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.321+0000 7f688238d700 1 --2- 172.21.15.157:0/2849044736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f687c06cbb0 0x7f687c0c1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:56.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.321+0000 7f688238d700 1 -- 172.21.15.157:0/2849044736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f686c009420 con 0x7f687c06cf80 2023-10-06T04:07:56.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.322+0000 7f688238d700 1 --2- 172.21.15.157:0/2849044736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f687c06cf80 0x7f687c0c1e90 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f687400c7a0 tx=0x7f6874001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:56.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.322+0000 7f6871ffb700 1 -- 172.21.15.157:0/2849044736 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6874007360 con 0x7f687c06cf80 2023-10-06T04:07:56.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.322+0000 7f6871ffb700 1 -- 172.21.15.157:0/2849044736 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68740074c0 con 0x7f687c06cf80 2023-10-06T04:07:56.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.322+0000 7f6883df0700 1 -- 172.21.15.157:0/2849044736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f687c155c00 con 0x7f687c06cf80 2023-10-06T04:07:56.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.322+0000 7f6871ffb700 1 -- 172.21.15.157:0/2849044736 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6874004830 con 0x7f687c06cf80 2023-10-06T04:07:56.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.322+0000 7f6883df0700 1 -- 172.21.15.157:0/2849044736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f687c0c08f0 con 0x7f687c06cf80 2023-10-06T04:07:56.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.323+0000 7f6883df0700 1 -- 172.21.15.157:0/2849044736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f687c089ca0 con 0x7f687c06cf80 2023-10-06T04:07:56.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.323+0000 7f6871ffb700 1 -- 172.21.15.157:0/2849044736 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6874004be0 con 0x7f687c06cf80 2023-10-06T04:07:56.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.323+0000 7f6871ffb700 1 --2- 172.21.15.157:0/2849044736 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6860066f30 0x7f68600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:07:56.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.323+0000 7f6871ffb700 1 -- 172.21.15.157:0/2849044736 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6546+0+0 (secure 0 0 0) 0x7f6874083e10 con 0x7f687c06cf80 2023-10-06T04:07:56.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.324+0000 7f6881b8c700 1 --2- 172.21.15.157:0/2849044736 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6860066f30 0x7f68600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:07:56.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.324+0000 7f6881b8c700 1 --2- 172.21.15.157:0/2849044736 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6860066f30 0x7f68600693e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f687800f570 tx=0x7f687800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:07:56.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.328+0000 7f6871ffb700 1 -- 172.21.15.157:0/2849044736 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6874050db0 con 0x7f687c06cf80 2023-10-06T04:07:56.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.561+0000 7f6883df0700 1 -- 172.21.15.157:0/2849044736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f687c0c9130 con 0x7f687c06cf80 2023-10-06T04:07:56.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.562+0000 7f6871ffb700 1 -- 172.21.15.157:0/2849044736 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+5387 (secure 0 0 0) 0x7f6874018020 con 0x7f687c06cf80 2023-10-06T04:07:56.563 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:07:56.564 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:07:54.745189+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5698},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5698":{"gid":5698,"name":"e","rank":1,"incarnation":72,"state":"up:active","state_seq":33,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:07:56.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.565+0000 7f6883df0700 1 -- 172.21.15.157:0/2849044736 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6860066f30 msgr2=0x7f68600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:56.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.565+0000 7f6883df0700 1 --2- 172.21.15.157:0/2849044736 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6860066f30 0x7f68600693e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f687800f570 tx=0x7f687800b040 comp rx=0 tx=0).stop 2023-10-06T04:07:56.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.565+0000 7f6883df0700 1 -- 172.21.15.157:0/2849044736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f687c06cf80 msgr2=0x7f687c0c1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:07:56.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.565+0000 7f6883df0700 1 --2- 172.21.15.157:0/2849044736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f687c06cf80 0x7f687c0c1e90 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f687400c7a0 tx=0x7f6874001a00 comp rx=0 tx=0).stop 2023-10-06T04:07:56.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.565+0000 7f6883df0700 1 -- 172.21.15.157:0/2849044736 shutdown_connections 2023-10-06T04:07:56.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.565+0000 7f6883df0700 1 --2- 172.21.15.157:0/2849044736 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6860066f30 0x7f68600693e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:56.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.565+0000 7f6883df0700 1 --2- 172.21.15.157:0/2849044736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f687c06cf80 0x7f687c0c1e90 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:56.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.565+0000 7f6883df0700 1 --2- 172.21.15.157:0/2849044736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f687c06c1f0 0x7f687c0cb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:56.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.565+0000 7f6883df0700 1 --2- 172.21.15.157:0/2849044736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f687c06cbb0 0x7f687c0c1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:07:56.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.565+0000 7f6883df0700 1 -- 172.21.15.157:0/2849044736 >> 172.21.15.157:0/2849044736 conn(0x7f687c06f230 msgr2=0x7f687c0c6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:07:56.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.565+0000 7f6883df0700 1 -- 172.21.15.157:0/2849044736 shutdown_connections 2023-10-06T04:07:56.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:07:56.565+0000 7f6883df0700 1 -- 172.21.15.157:0/2849044736 wait complete. 2023-10-06T04:07:56.568 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 76 2023-10-06T04:07:56.579 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e has gained rank=1, replacing gid=5362 2023-10-06T04:07:56.579 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 7 secs before reviving mds.d 2023-10-06T04:08:03.580 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.d 2023-10-06T04:08:03.581 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T04:08:03.581 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T04:08:03.584 INFO:tasks.ceph.mds.d:Started 2023-10-06T04:08:03.584 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:08:03.664 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2023-10-06T04:08:03.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.704+0000 7f9e2bce6700 1 Processor -- start 2023-10-06T04:08:03.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.704+0000 7f9e2bce6700 1 -- start start 2023-10-06T04:08:03.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.705+0000 7f9e2bce6700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9e2406cbb0 0x7f9e2406d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:03.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.705+0000 7f9e2bce6700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e24150410 0x7f9e2406da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:03.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.705+0000 7f9e2bce6700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e240c9780 con 0x7f9e2406c1f0 2023-10-06T04:08:03.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.705+0000 7f9e2bce6700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e240c8c40 con 0x7f9e2406cbb0 2023-10-06T04:08:03.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.705+0000 7f9e2bce6700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2406e0c0 con 0x7f9e24150410 2023-10-06T04:08:03.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.705+0000 7f9e29a82700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9e2406cbb0 0x7f9e2406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:03.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.705+0000 7f9e29a82700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9e2406cbb0 0x7f9e2406d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56848/0 (socket says 172.21.15.157:56848) 2023-10-06T04:08:03.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.705+0000 7f9e29a82700 1 -- 172.21.15.157:0/619288708 learned_addr learned my addr 172.21.15.157:0/619288708 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:08:03.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.705+0000 7f9e2a283700 1 --2- 172.21.15.157:0/619288708 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e24150410 0x7f9e2406da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:03.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.705+0000 7f9e29a82700 1 -- 172.21.15.157:0/619288708 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e24150410 msgr2=0x7f9e2406da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:03.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.705+0000 7f9e29a82700 1 --2- 172.21.15.157:0/619288708 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e24150410 0x7f9e2406da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:03.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.705+0000 7f9e29a82700 1 -- 172.21.15.157:0/619288708 >> v1:172.21.15.175:6789/0 conn(0x7f9e2406c1f0 legacy=0x7f9e2414fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:03.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.705+0000 7f9e29a82700 1 -- 172.21.15.157:0/619288708 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e240c8e50 con 0x7f9e2406cbb0 2023-10-06T04:08:03.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.705+0000 7f9e29a82700 1 --2- 172.21.15.157:0/619288708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e2406cbb0 0x7f9e2406d530 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f9e200099b0 tx=0x7f9e20009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4966f2e6e877b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:03.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.705+0000 7f9e28a80700 1 -- 172.21.15.157:0/619288708 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2000b8f0 con 0x7f9e2406cbb0 2023-10-06T04:08:03.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.705+0000 7f9e28a80700 1 -- 172.21.15.157:0/619288708 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2000ba50 con 0x7f9e2406cbb0 2023-10-06T04:08:03.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.705+0000 7f9e28a80700 1 -- 172.21.15.157:0/619288708 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2000bdf0 con 0x7f9e2406cbb0 2023-10-06T04:08:03.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.706+0000 7f9e2bce6700 1 -- 172.21.15.157:0/619288708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e2406cbb0 msgr2=0x7f9e2406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:03.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.706+0000 7f9e2bce6700 1 --2- 172.21.15.157:0/619288708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e2406cbb0 0x7f9e2406d530 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f9e200099b0 tx=0x7f9e20009250 comp rx=0 tx=0).stop 2023-10-06T04:08:03.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.706+0000 7f9e2bce6700 1 -- 172.21.15.157:0/619288708 shutdown_connections 2023-10-06T04:08:03.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.706+0000 7f9e2bce6700 1 --2- 172.21.15.157:0/619288708 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e24150410 0x7f9e2406da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:03.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.706+0000 7f9e2bce6700 1 --2- 172.21.15.157:0/619288708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e2406cbb0 0x7f9e2406d530 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:03.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.706+0000 7f9e2bce6700 1 -- 172.21.15.157:0/619288708 >> 172.21.15.157:0/619288708 conn(0x7f9e2406f230 msgr2=0x7f9e2406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:03.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.706+0000 7f9e2bce6700 1 -- 172.21.15.157:0/619288708 shutdown_connections 2023-10-06T04:08:03.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.706+0000 7f9e2bce6700 1 -- 172.21.15.157:0/619288708 wait complete. 2023-10-06T04:08:03.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.707+0000 7f9e2bce6700 1 Processor -- start 2023-10-06T04:08:03.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.707+0000 7f9e2bce6700 1 -- start start 2023-10-06T04:08:03.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.708+0000 7f9e2bce6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e2406c1f0 0x7f9e240c6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:03.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.708+0000 7f9e2bce6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e2406cbb0 0x7f9e24155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:03.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.708+0000 7f9e2bce6700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e24150410 0x7f9e24151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:03.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.708+0000 7f9e2bce6700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e240cb860 con 0x7f9e2406c1f0 2023-10-06T04:08:03.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.708+0000 7f9e2bce6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e24151a70 con 0x7f9e2406cbb0 2023-10-06T04:08:03.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.708+0000 7f9e2bce6700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e24166f50 con 0x7f9e24150410 2023-10-06T04:08:03.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.708+0000 7f9e29a82700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e2406c1f0 0x7f9e240c6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:03.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.708+0000 7f9e29a82700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e2406c1f0 0x7f9e240c6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56860/0 (socket says 172.21.15.157:56860) 2023-10-06T04:08:03.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.708+0000 7f9e29a82700 1 -- 172.21.15.157:0/4165340789 learned_addr learned my addr 172.21.15.157:0/4165340789 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:08:03.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.708+0000 7f9e2a283700 1 --2- 172.21.15.157:0/4165340789 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e24150410 0x7f9e24151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:03.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.708+0000 7f9e29281700 1 --2- 172.21.15.157:0/4165340789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e2406cbb0 0x7f9e24155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:03.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.708+0000 7f9e29a82700 1 -- 172.21.15.157:0/4165340789 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e24150410 msgr2=0x7f9e24151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:03.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.708+0000 7f9e29a82700 1 --2- 172.21.15.157:0/4165340789 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e24150410 0x7f9e24151f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:03.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.708+0000 7f9e29a82700 1 -- 172.21.15.157:0/4165340789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e2406cbb0 msgr2=0x7f9e24155e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:03.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.708+0000 7f9e29a82700 1 --2- 172.21.15.157:0/4165340789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e2406cbb0 0x7f9e24155e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:03.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.708+0000 7f9e29a82700 1 -- 172.21.15.157:0/4165340789 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e200094f0 con 0x7f9e2406c1f0 2023-10-06T04:08:03.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.708+0000 7f9e2a283700 1 --2- 172.21.15.157:0/4165340789 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e24150410 0x7f9e24151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:08:03.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.708+0000 7f9e29281700 1 --2- 172.21.15.157:0/4165340789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e2406cbb0 0x7f9e24155e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:08:03.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.708+0000 7f9e29a82700 1 --2- 172.21.15.157:0/4165340789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e2406c1f0 0x7f9e240c6400 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9e240c3960 tx=0x7f9e20017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:03.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.708+0000 7f9e19ffb700 1 -- 172.21.15.157:0/4165340789 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e20024440 con 0x7f9e2406c1f0 2023-10-06T04:08:03.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.708+0000 7f9e19ffb700 1 -- 172.21.15.157:0/4165340789 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e200245c0 con 0x7f9e2406c1f0 2023-10-06T04:08:03.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.708+0000 7f9e2bce6700 1 -- 172.21.15.157:0/4165340789 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e24167230 con 0x7f9e2406c1f0 2023-10-06T04:08:03.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.709+0000 7f9e19ffb700 1 -- 172.21.15.157:0/4165340789 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e20024980 con 0x7f9e2406c1f0 2023-10-06T04:08:03.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.709+0000 7f9e2bce6700 1 -- 172.21.15.157:0/4165340789 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e241677f0 con 0x7f9e2406c1f0 2023-10-06T04:08:03.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.710+0000 7f9e2bce6700 1 -- 172.21.15.157:0/4165340789 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e240c9ca0 con 0x7f9e2406c1f0 2023-10-06T04:08:03.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.710+0000 7f9e19ffb700 1 -- 172.21.15.157:0/4165340789 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9e2001a420 con 0x7f9e2406c1f0 2023-10-06T04:08:03.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.710+0000 7f9e19ffb700 1 --2- 172.21.15.157:0/4165340789 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9e08066f30 0x7f9e080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:03.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.710+0000 7f9e19ffb700 1 -- 172.21.15.157:0/4165340789 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6546+0+0 (secure 0 0 0) 0x7f9e200866c0 con 0x7f9e2406c1f0 2023-10-06T04:08:03.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.711+0000 7f9e29281700 1 --2- 172.21.15.157:0/4165340789 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9e08066f30 0x7f9e080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:03.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.711+0000 7f9e29281700 1 --2- 172.21.15.157:0/4165340789 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9e08066f30 0x7f9e080693e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9e14007fd0 tx=0x7f9e14009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:03.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.715+0000 7f9e19ffb700 1 -- 172.21.15.157:0/4165340789 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9e20053540 con 0x7f9e2406c1f0 2023-10-06T04:08:03.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.940+0000 7f9e2bce6700 1 -- 172.21.15.157:0/4165340789 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e240cc470 con 0x7f9e2406c1f0 2023-10-06T04:08:03.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.940+0000 7f9e19ffb700 1 -- 172.21.15.157:0/4165340789 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+6147 (secure 0 0 0) 0x7f9e20056b60 con 0x7f9e2406c1f0 2023-10-06T04:08:03.942 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:08:03.942 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5812,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:07:54.745189+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5698},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5698":{"gid":5698,"name":"e","rank":1,"incarnation":72,"state":"up:active","state_seq":33,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:08:03.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.942+0000 7f9e2bce6700 1 -- 172.21.15.157:0/4165340789 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9e08066f30 msgr2=0x7f9e080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:03.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.942+0000 7f9e2bce6700 1 --2- 172.21.15.157:0/4165340789 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9e08066f30 0x7f9e080693e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9e14007fd0 tx=0x7f9e14009040 comp rx=0 tx=0).stop 2023-10-06T04:08:03.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.942+0000 7f9e2bce6700 1 -- 172.21.15.157:0/4165340789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e2406c1f0 msgr2=0x7f9e240c6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:03.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.942+0000 7f9e2bce6700 1 --2- 172.21.15.157:0/4165340789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e2406c1f0 0x7f9e240c6400 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9e240c3960 tx=0x7f9e20017000 comp rx=0 tx=0).stop 2023-10-06T04:08:03.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.942+0000 7f9e2bce6700 1 -- 172.21.15.157:0/4165340789 shutdown_connections 2023-10-06T04:08:03.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.942+0000 7f9e2bce6700 1 --2- 172.21.15.157:0/4165340789 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9e08066f30 0x7f9e080693e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:03.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.942+0000 7f9e2bce6700 1 --2- 172.21.15.157:0/4165340789 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e24150410 0x7f9e24151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:03.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.942+0000 7f9e2bce6700 1 --2- 172.21.15.157:0/4165340789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e2406c1f0 0x7f9e240c6400 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:03.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.942+0000 7f9e2bce6700 1 --2- 172.21.15.157:0/4165340789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e2406cbb0 0x7f9e24155e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:03.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.942+0000 7f9e2bce6700 1 -- 172.21.15.157:0/4165340789 >> 172.21.15.157:0/4165340789 conn(0x7f9e2406f230 msgr2=0x7f9e240af1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:03.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.942+0000 7f9e2bce6700 1 -- 172.21.15.157:0/4165340789 shutdown_connections 2023-10-06T04:08:03.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:03.942+0000 7f9e2bce6700 1 -- 172.21.15.157:0/4165340789 wait complete. 2023-10-06T04:08:03.945 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 77 2023-10-06T04:08:03.956 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d reported in up:standby state 2023-10-06T04:08:03.956 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 16 secs before thrashing 2023-10-06T04:08:15.914 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:08:15.918 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:08:19.957 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:08:20.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.074+0000 7f4ff8a11700 1 Processor -- start 2023-10-06T04:08:20.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.074+0000 7f4ff8a11700 1 -- start start 2023-10-06T04:08:20.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.075+0000 7f4ff8a11700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4ff41544e0 0x7f4ff40c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:20.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.075+0000 7f4ff8a11700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ff4068170 0x7f4ff4150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:20.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.075+0000 7f4ff8a11700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff40c1020 con 0x7f4ff4067da0 2023-10-06T04:08:20.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.075+0000 7f4ff8a11700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ff4157b00 con 0x7f4ff41544e0 2023-10-06T04:08:20.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.075+0000 7f4ff8a11700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff40c0cf0 con 0x7f4ff4068170 2023-10-06T04:08:20.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.075+0000 7f4ff37fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f4ff4067da0 0x7f4ff4153dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:45352/0 (socket says 172.21.15.157:45352) 2023-10-06T04:08:20.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.075+0000 7f4ff2ffd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4ff41544e0 0x7f4ff40c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:20.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.075+0000 7f4ff37fe700 1 -- 172.21.15.157:0/1159506274 learned_addr learned my addr 172.21.15.157:0/1159506274 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:08:20.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.075+0000 7f4ff2ffd700 1 -- 172.21.15.157:0/1159506274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ff4068170 msgr2=0x7f4ff4150bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:20.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.075+0000 7f4ff3fff700 1 --2- 172.21.15.157:0/1159506274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ff4068170 0x7f4ff4150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:20.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.075+0000 7f4ff2ffd700 1 --2- 172.21.15.157:0/1159506274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ff4068170 0x7f4ff4150bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:20.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.075+0000 7f4ff2ffd700 1 -- 172.21.15.157:0/1159506274 >> v1:172.21.15.157:6789/0 conn(0x7f4ff4067da0 legacy=0x7f4ff4153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:20.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.075+0000 7f4ff27fc700 1 -- 172.21.15.157:0/1159506274 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3867345691 0 0) 0x7f4ff40c1020 con 0x7f4ff4067da0 2023-10-06T04:08:20.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.075+0000 7f4ff2ffd700 1 -- 172.21.15.157:0/1159506274 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff4157da0 con 0x7f4ff41544e0 2023-10-06T04:08:20.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.075+0000 7f4ff2ffd700 1 --2- 172.21.15.157:0/1159506274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ff41544e0 0x7f4ff40c7930 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80098e0 tx=0x7f4fe80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59658a6c4d30eb29 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:20.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.076+0000 7f4ff27fc700 1 -- 172.21.15.157:0/1159506274 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8009e80 con 0x7f4ff41544e0 2023-10-06T04:08:20.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.076+0000 7f4ff27fc700 1 -- 172.21.15.157:0/1159506274 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe800b860 con 0x7f4ff41544e0 2023-10-06T04:08:20.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.076+0000 7f4ff27fc700 1 -- 172.21.15.157:0/1159506274 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe800bbe0 con 0x7f4ff41544e0 2023-10-06T04:08:20.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.076+0000 7f4ff8a11700 1 -- 172.21.15.157:0/1159506274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ff41544e0 msgr2=0x7f4ff40c7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:20.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.076+0000 7f4ff8a11700 1 --2- 172.21.15.157:0/1159506274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ff41544e0 0x7f4ff40c7930 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80098e0 tx=0x7f4fe80091a0 comp rx=0 tx=0).stop 2023-10-06T04:08:20.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.076+0000 7f4ff8a11700 1 -- 172.21.15.157:0/1159506274 shutdown_connections 2023-10-06T04:08:20.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.076+0000 7f4ff8a11700 1 --2- 172.21.15.157:0/1159506274 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ff4068170 0x7f4ff4150bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:20.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.076+0000 7f4ff8a11700 1 --2- 172.21.15.157:0/1159506274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ff41544e0 0x7f4ff40c7930 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:20.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.076+0000 7f4ff8a11700 1 -- 172.21.15.157:0/1159506274 >> 172.21.15.157:0/1159506274 conn(0x7f4ff406c1f0 msgr2=0x7f4ff406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:20.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.076+0000 7f4ff8a11700 1 -- 172.21.15.157:0/1159506274 shutdown_connections 2023-10-06T04:08:20.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.076+0000 7f4ff8a11700 1 -- 172.21.15.157:0/1159506274 wait complete. 2023-10-06T04:08:20.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.077+0000 7f4ff8a11700 1 Processor -- start 2023-10-06T04:08:20.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.077+0000 7f4ff8a11700 1 -- start start 2023-10-06T04:08:20.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.078+0000 7f4ff8a11700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ff4067da0 0x7f4ff4166ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:20.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.078+0000 7f4ff8a11700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ff4068170 0x7f4ff40ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:20.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.078+0000 7f4ff8a11700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ff41544e0 0x7f4ff40ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:20.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.078+0000 7f4ff8a11700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff40641e0 con 0x7f4ff4067da0 2023-10-06T04:08:20.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.078+0000 7f4ff8a11700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff4166630 con 0x7f4ff41544e0 2023-10-06T04:08:20.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.078+0000 7f4ff8a11700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff40a6500 con 0x7f4ff4068170 2023-10-06T04:08:20.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.078+0000 7f4ff37fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ff4067da0 0x7f4ff4166ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:20.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.078+0000 7f4ff37fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ff4067da0 0x7f4ff4166ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52048/0 (socket says 172.21.15.157:52048) 2023-10-06T04:08:20.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.078+0000 7f4ff37fe700 1 -- 172.21.15.157:0/422794998 learned_addr learned my addr 172.21.15.157:0/422794998 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:08:20.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.078+0000 7f4ff2ffd700 1 --2- 172.21.15.157:0/422794998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ff4068170 0x7f4ff40ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:20.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.078+0000 7f4ff3fff700 1 --2- 172.21.15.157:0/422794998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ff41544e0 0x7f4ff40ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:20.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.078+0000 7f4ff37fe700 1 -- 172.21.15.157:0/422794998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ff4068170 msgr2=0x7f4ff40ae1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:20.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.078+0000 7f4ff37fe700 1 --2- 172.21.15.157:0/422794998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ff4068170 0x7f4ff40ae1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:20.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.079+0000 7f4ff37fe700 1 -- 172.21.15.157:0/422794998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ff41544e0 msgr2=0x7f4ff40ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:20.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.079+0000 7f4ff37fe700 1 --2- 172.21.15.157:0/422794998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ff41544e0 0x7f4ff40ae700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:20.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.079+0000 7f4ff37fe700 1 -- 172.21.15.157:0/422794998 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe8009420 con 0x7f4ff4067da0 2023-10-06T04:08:20.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.079+0000 7f4ff37fe700 1 --2- 172.21.15.157:0/422794998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ff4067da0 0x7f4ff4166ac0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400cd80 tx=0x7f4fe4001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:20.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.079+0000 7f4fdb7fe700 1 -- 172.21.15.157:0/422794998 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe400aaf0 con 0x7f4ff4067da0 2023-10-06T04:08:20.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.079+0000 7f4fdb7fe700 1 -- 172.21.15.157:0/422794998 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe400ac50 con 0x7f4ff4067da0 2023-10-06T04:08:20.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.079+0000 7f4ff8a11700 1 -- 172.21.15.157:0/422794998 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff40a67e0 con 0x7f4ff4067da0 2023-10-06T04:08:20.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.079+0000 7f4fdb7fe700 1 -- 172.21.15.157:0/422794998 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe40070f0 con 0x7f4ff4067da0 2023-10-06T04:08:20.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.079+0000 7f4ff8a11700 1 -- 172.21.15.157:0/422794998 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff4165fe0 con 0x7f4ff4067da0 2023-10-06T04:08:20.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.080+0000 7f4ff8a11700 1 -- 172.21.15.157:0/422794998 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff4166910 con 0x7f4ff4067da0 2023-10-06T04:08:20.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.080+0000 7f4fdb7fe700 1 -- 172.21.15.157:0/422794998 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4fe40074a0 con 0x7f4ff4067da0 2023-10-06T04:08:20.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.081+0000 7f4fdb7fe700 1 --2- 172.21.15.157:0/422794998 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4fe0066f60 0x7f4fe0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:20.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.081+0000 7f4fdb7fe700 1 -- 172.21.15.157:0/422794998 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6546+0+0 (secure 0 0 0) 0x7f4fe4084d90 con 0x7f4ff4067da0 2023-10-06T04:08:20.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.081+0000 7f4ff2ffd700 1 --2- 172.21.15.157:0/422794998 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4fe0066f60 0x7f4fe0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:20.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.081+0000 7f4ff2ffd700 1 --2- 172.21.15.157:0/422794998 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4fe0066f60 0x7f4fe0069410 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4154940 tx=0x7f4fe8017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:20.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.085+0000 7f4fdb7fe700 1 -- 172.21.15.157:0/422794998 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4fe4051cb0 con 0x7f4ff4067da0 2023-10-06T04:08:20.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.314+0000 7f4ff8a11700 1 -- 172.21.15.157:0/422794998 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ff406aae0 con 0x7f4ff4067da0 2023-10-06T04:08:20.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.315+0000 7f4fdb7fe700 1 -- 172.21.15.157:0/422794998 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+6147 (secure 0 0 0) 0x7f4fe40552d0 con 0x7f4ff4067da0 2023-10-06T04:08:20.316 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:08:20.316 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5812,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:07:54.745189+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5698},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5698":{"gid":5698,"name":"e","rank":1,"incarnation":72,"state":"up:active","state_seq":33,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:08:20.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.317+0000 7f4ff8a11700 1 -- 172.21.15.157:0/422794998 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4fe0066f60 msgr2=0x7f4fe0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:20.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.317+0000 7f4ff8a11700 1 --2- 172.21.15.157:0/422794998 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4fe0066f60 0x7f4fe0069410 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4154940 tx=0x7f4fe8017000 comp rx=0 tx=0).stop 2023-10-06T04:08:20.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.317+0000 7f4ff8a11700 1 -- 172.21.15.157:0/422794998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ff4067da0 msgr2=0x7f4ff4166ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:20.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.317+0000 7f4ff8a11700 1 --2- 172.21.15.157:0/422794998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ff4067da0 0x7f4ff4166ac0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400cd80 tx=0x7f4fe4001ae0 comp rx=0 tx=0).stop 2023-10-06T04:08:20.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.317+0000 7f4ff8a11700 1 -- 172.21.15.157:0/422794998 shutdown_connections 2023-10-06T04:08:20.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.317+0000 7f4ff8a11700 1 --2- 172.21.15.157:0/422794998 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4fe0066f60 0x7f4fe0069410 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:20.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.317+0000 7f4ff8a11700 1 --2- 172.21.15.157:0/422794998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ff41544e0 0x7f4ff40ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:20.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.317+0000 7f4ff8a11700 1 --2- 172.21.15.157:0/422794998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ff4067da0 0x7f4ff4166ac0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:20.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.317+0000 7f4ff8a11700 1 --2- 172.21.15.157:0/422794998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ff4068170 0x7f4ff40ae1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:20.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.317+0000 7f4ff8a11700 1 -- 172.21.15.157:0/422794998 >> 172.21.15.157:0/422794998 conn(0x7f4ff406c1f0 msgr2=0x7f4ff4153dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:20.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.317+0000 7f4ff8a11700 1 -- 172.21.15.157:0/422794998 shutdown_connections 2023-10-06T04:08:20.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.317+0000 7f4ff8a11700 1 -- 172.21.15.157:0/422794998 wait complete. 2023-10-06T04:08:20.320 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 77 2023-10-06T04:08:20.330 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.e (rank=1) 2023-10-06T04:08:20.331 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2023-10-06T04:08:20.331 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:08:20.340 INFO:tasks.ceph.mds.e:Stopped 2023-10-06T04:08:20.340 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:08:20.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.494+0000 7faa1a939700 1 Processor -- start 2023-10-06T04:08:20.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.495+0000 7faa1a939700 1 -- start start 2023-10-06T04:08:20.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.495+0000 7faa1a939700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faa1406dc40 0x7faa1414fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:20.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.495+0000 7faa1a939700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7faa1406c1f0 0x7faa14150240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:20.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.495+0000 7faa1a939700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa140c9780 con 0x7faa1406cbb0 2023-10-06T04:08:20.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.495+0000 7faa1a939700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa140c8c40 con 0x7faa1406dc40 2023-10-06T04:08:20.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.495+0000 7faa1a939700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa1406e0c0 con 0x7faa1406c1f0 2023-10-06T04:08:20.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.495+0000 7faa137fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faa1406dc40 0x7faa1414fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:20.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.495+0000 7faa13fff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7faa1406cbb0 0x7faa1406d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:45358/0 (socket says 172.21.15.157:45358) 2023-10-06T04:08:20.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.495+0000 7faa13fff700 1 -- 172.21.15.157:0/1954723030 learned_addr learned my addr 172.21.15.157:0/1954723030 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:08:20.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.496+0000 7faa18ed6700 1 --2- 172.21.15.157:0/1954723030 >> v2:172.21.15.175:3300/0 conn(0x7faa1406c1f0 0x7faa14150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:20.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.496+0000 7faa137fe700 1 -- 172.21.15.157:0/1954723030 >> v2:172.21.15.175:3300/0 conn(0x7faa1406c1f0 msgr2=0x7faa14150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:20.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.496+0000 7faa137fe700 1 --2- 172.21.15.157:0/1954723030 >> v2:172.21.15.175:3300/0 conn(0x7faa1406c1f0 0x7faa14150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:20.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.496+0000 7faa137fe700 1 -- 172.21.15.157:0/1954723030 >> v1:172.21.15.157:6789/0 conn(0x7faa1406cbb0 legacy=0x7faa1406d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:20.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.496+0000 7faa12ffd700 1 -- 172.21.15.157:0/1954723030 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2360327800 0 0) 0x7faa140c9780 con 0x7faa1406cbb0 2023-10-06T04:08:20.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.496+0000 7faa137fe700 1 -- 172.21.15.157:0/1954723030 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa140c8e50 con 0x7faa1406dc40 2023-10-06T04:08:20.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.496+0000 7faa18ed6700 1 --2- 172.21.15.157:0/1954723030 >> v2:172.21.15.175:3300/0 conn(0x7faa1406c1f0 0x7faa14150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:08:20.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.496+0000 7faa137fe700 1 --2- 172.21.15.157:0/1954723030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa1406dc40 0x7faa1414fd00 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7faa140c96f0 tx=0x7faa080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9c442f6813d76e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:20.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.496+0000 7faa12ffd700 1 -- 172.21.15.157:0/1954723030 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0800b740 con 0x7faa1406dc40 2023-10-06T04:08:20.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.496+0000 7faa12ffd700 1 -- 172.21.15.157:0/1954723030 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0800b8a0 con 0x7faa1406dc40 2023-10-06T04:08:20.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.496+0000 7faa12ffd700 1 -- 172.21.15.157:0/1954723030 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0800bc40 con 0x7faa1406dc40 2023-10-06T04:08:20.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.496+0000 7faa1a939700 1 -- 172.21.15.157:0/1954723030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa1406dc40 msgr2=0x7faa1414fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:20.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.496+0000 7faa1a939700 1 --2- 172.21.15.157:0/1954723030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa1406dc40 0x7faa1414fd00 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7faa140c96f0 tx=0x7faa080091a0 comp rx=0 tx=0).stop 2023-10-06T04:08:20.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.497+0000 7faa1a939700 1 -- 172.21.15.157:0/1954723030 shutdown_connections 2023-10-06T04:08:20.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.497+0000 7faa1a939700 1 --2- 172.21.15.157:0/1954723030 >> v2:172.21.15.175:3300/0 conn(0x7faa1406c1f0 0x7faa14150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:20.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.497+0000 7faa1a939700 1 --2- 172.21.15.157:0/1954723030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa1406dc40 0x7faa1414fd00 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:20.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.497+0000 7faa1a939700 1 -- 172.21.15.157:0/1954723030 >> 172.21.15.157:0/1954723030 conn(0x7faa1406f230 msgr2=0x7faa1406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:20.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.497+0000 7faa1a939700 1 -- 172.21.15.157:0/1954723030 shutdown_connections 2023-10-06T04:08:20.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.497+0000 7faa1a939700 1 -- 172.21.15.157:0/1954723030 wait complete. 2023-10-06T04:08:20.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.498+0000 7faa1a939700 1 Processor -- start 2023-10-06T04:08:20.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.498+0000 7faa1a939700 1 -- start start 2023-10-06T04:08:20.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.498+0000 7faa1a939700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa1406c1f0 0x7faa1414d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:20.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.498+0000 7faa1a939700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa1406cbb0 0x7faa140c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:20.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.498+0000 7faa1a939700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa140c5d70 0x7faa140c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:20.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.498+0000 7faa1a939700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa14152fc0 con 0x7faa1406cbb0 2023-10-06T04:08:20.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.498+0000 7faa1a939700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa14151230 con 0x7faa1406c1f0 2023-10-06T04:08:20.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.498+0000 7faa1a939700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa14152b80 con 0x7faa140c5d70 2023-10-06T04:08:20.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.499+0000 7faa137fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa1406cbb0 0x7faa140c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:20.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.499+0000 7faa137fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa1406cbb0 0x7faa140c48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52058/0 (socket says 172.21.15.157:52058) 2023-10-06T04:08:20.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.499+0000 7faa137fe700 1 -- 172.21.15.157:0/1279823692 learned_addr learned my addr 172.21.15.157:0/1279823692 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:08:20.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.499+0000 7faa18ed6700 1 --2- 172.21.15.157:0/1279823692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa140c5d70 0x7faa140c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:20.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.499+0000 7faa13fff700 1 --2- 172.21.15.157:0/1279823692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa1406c1f0 0x7faa1414d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:20.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.499+0000 7faa137fe700 1 -- 172.21.15.157:0/1279823692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa140c5d70 msgr2=0x7faa140c4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:20.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.499+0000 7faa137fe700 1 --2- 172.21.15.157:0/1279823692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa140c5d70 0x7faa140c4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:20.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.499+0000 7faa137fe700 1 -- 172.21.15.157:0/1279823692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa1406c1f0 msgr2=0x7faa1414d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:20.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.499+0000 7faa137fe700 1 --2- 172.21.15.157:0/1279823692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa1406c1f0 0x7faa1414d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:20.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.499+0000 7faa137fe700 1 -- 172.21.15.157:0/1279823692 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa08009420 con 0x7faa1406cbb0 2023-10-06T04:08:20.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.499+0000 7faa13fff700 1 --2- 172.21.15.157:0/1279823692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa1406c1f0 0x7faa1414d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:08:20.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.499+0000 7faa137fe700 1 --2- 172.21.15.157:0/1279823692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa1406cbb0 0x7faa140c48a0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7faa140c3960 tx=0x7faa08017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:20.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.499+0000 7fa9fbfff700 1 -- 172.21.15.157:0/1279823692 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0800bc20 con 0x7faa1406cbb0 2023-10-06T04:08:20.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.499+0000 7fa9fbfff700 1 -- 172.21.15.157:0/1279823692 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa08011420 con 0x7faa1406cbb0 2023-10-06T04:08:20.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.499+0000 7faa1a939700 1 -- 172.21.15.157:0/1279823692 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa141514b0 con 0x7faa1406cbb0 2023-10-06T04:08:20.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.499+0000 7fa9fbfff700 1 -- 172.21.15.157:0/1279823692 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa080117a0 con 0x7faa1406cbb0 2023-10-06T04:08:20.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.499+0000 7faa1a939700 1 -- 172.21.15.157:0/1279823692 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa140c40a0 con 0x7faa1406cbb0 2023-10-06T04:08:20.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.500+0000 7faa1a939700 1 -- 172.21.15.157:0/1279823692 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa14089ca0 con 0x7faa1406cbb0 2023-10-06T04:08:20.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.501+0000 7fa9fbfff700 1 -- 172.21.15.157:0/1279823692 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faa08011b10 con 0x7faa1406cbb0 2023-10-06T04:08:20.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.501+0000 7fa9fbfff700 1 --2- 172.21.15.157:0/1279823692 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faa00066f30 0x7faa000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:20.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.501+0000 7fa9fbfff700 1 -- 172.21.15.157:0/1279823692 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6546+0+0 (secure 0 0 0) 0x7faa080860f0 con 0x7faa1406cbb0 2023-10-06T04:08:20.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.501+0000 7faa13fff700 1 --2- 172.21.15.157:0/1279823692 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faa00066f30 0x7faa000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:20.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.502+0000 7faa13fff700 1 --2- 172.21.15.157:0/1279823692 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faa00066f30 0x7faa000693e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7faa040011e0 tx=0x7faa04009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:20.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.505+0000 7fa9fbfff700 1 -- 172.21.15.157:0/1279823692 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa08053010 con 0x7faa1406cbb0 2023-10-06T04:08:20.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.799+0000 7faa1a939700 1 -- 172.21.15.157:0/1279823692 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa140c9130 con 0x7faa1406cbb0 2023-10-06T04:08:20.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.800+0000 7fa9fbfff700 1 -- 172.21.15.157:0/1279823692 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+6147 (secure 0 0 0) 0x7faa08056630 con 0x7faa1406cbb0 2023-10-06T04:08:20.801 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:08:20.802 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5812,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:07:54.745189+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5698},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5698":{"gid":5698,"name":"e","rank":1,"incarnation":72,"state":"up:active","state_seq":33,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:08:20.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.802+0000 7faa1a939700 1 -- 172.21.15.157:0/1279823692 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faa00066f30 msgr2=0x7faa000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:20.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.802+0000 7faa1a939700 1 --2- 172.21.15.157:0/1279823692 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faa00066f30 0x7faa000693e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7faa040011e0 tx=0x7faa04009040 comp rx=0 tx=0).stop 2023-10-06T04:08:20.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.802+0000 7faa1a939700 1 -- 172.21.15.157:0/1279823692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa1406cbb0 msgr2=0x7faa140c48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:20.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.802+0000 7faa1a939700 1 --2- 172.21.15.157:0/1279823692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa1406cbb0 0x7faa140c48a0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7faa140c3960 tx=0x7faa08017000 comp rx=0 tx=0).stop 2023-10-06T04:08:20.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.802+0000 7faa1a939700 1 -- 172.21.15.157:0/1279823692 shutdown_connections 2023-10-06T04:08:20.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.802+0000 7faa1a939700 1 --2- 172.21.15.157:0/1279823692 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faa00066f30 0x7faa000693e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:20.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.802+0000 7faa1a939700 1 --2- 172.21.15.157:0/1279823692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa1406c1f0 0x7faa1414d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:20.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.802+0000 7faa1a939700 1 --2- 172.21.15.157:0/1279823692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa140c5d70 0x7faa140c4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:20.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.802+0000 7faa1a939700 1 --2- 172.21.15.157:0/1279823692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa1406cbb0 0x7faa140c48a0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:20.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.802+0000 7faa1a939700 1 -- 172.21.15.157:0/1279823692 >> 172.21.15.157:0/1279823692 conn(0x7faa1406f230 msgr2=0x7faa140aff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:20.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.802+0000 7faa1a939700 1 -- 172.21.15.157:0/1279823692 shutdown_connections 2023-10-06T04:08:20.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:20.802+0000 7faa1a939700 1 -- 172.21.15.157:0/1279823692 wait complete. 2023-10-06T04:08:20.805 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 77 2023-10-06T04:08:20.816 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-06T04:08:22.817 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:08:22.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.943+0000 7f1f6ac2a700 1 Processor -- start 2023-10-06T04:08:22.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.944+0000 7f1f6ac2a700 1 -- start start 2023-10-06T04:08:22.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.944+0000 7f1f6ac2a700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1f6406ade0 0x7f1f640694a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:22.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.944+0000 7f1f6ac2a700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f640c5680 con 0x7f1f640c4570 2023-10-06T04:08:22.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.944+0000 7f1f6ac2a700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f640c5920 con 0x7f1f640ad4c0 2023-10-06T04:08:22.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.944+0000 7f1f6ac2a700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f64158440 con 0x7f1f6406ade0 2023-10-06T04:08:22.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.944+0000 7f1f5bfff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1f6406ade0 0x7f1f640694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:22.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.944+0000 7f1f691c7700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f1f640c4570 0x7f1f6415eac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:45372/0 (socket says 172.21.15.157:45372) 2023-10-06T04:08:22.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.944+0000 7f1f691c7700 1 -- 172.21.15.157:0/1454504919 learned_addr learned my addr 172.21.15.157:0/1454504919 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:08:22.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.945+0000 7f1f5bfff700 1 -- 172.21.15.157:0/1454504919 >> v1:172.21.15.175:6789/0 conn(0x7f1f640ad4c0 legacy=0x7f1f640ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:22.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.945+0000 7f1f5bfff700 1 -- 172.21.15.157:0/1454504919 >> v1:172.21.15.157:6789/0 conn(0x7f1f640c4570 legacy=0x7f1f6415eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:22.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.945+0000 7f1f5bfff700 1 -- 172.21.15.157:0/1454504919 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f641577c0 con 0x7f1f6406ade0 2023-10-06T04:08:22.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.945+0000 7f1f5b7fe700 1 -- 172.21.15.157:0/1454504919 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2023429057 0 0) 0x7f1f640c5680 con 0x7f1f640c4570 2023-10-06T04:08:22.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.945+0000 7f1f5bfff700 1 --2- 172.21.15.157:0/1454504919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f6406ade0 0x7f1f640694a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1f540098e0 tx=0x7f1f540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb031b02a5e19c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:22.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.945+0000 7f1f5b7fe700 1 -- 172.21.15.157:0/1454504919 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f54009e80 con 0x7f1f6406ade0 2023-10-06T04:08:22.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.945+0000 7f1f5b7fe700 1 -- 172.21.15.157:0/1454504919 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5400b860 con 0x7f1f6406ade0 2023-10-06T04:08:22.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.945+0000 7f1f5b7fe700 1 -- 172.21.15.157:0/1454504919 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5400bbe0 con 0x7f1f6406ade0 2023-10-06T04:08:22.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.946+0000 7f1f6ac2a700 1 -- 172.21.15.157:0/1454504919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f6406ade0 msgr2=0x7f1f640694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:22.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.946+0000 7f1f6ac2a700 1 --2- 172.21.15.157:0/1454504919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f6406ade0 0x7f1f640694a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1f540098e0 tx=0x7f1f540091a0 comp rx=0 tx=0).stop 2023-10-06T04:08:22.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.946+0000 7f1f6ac2a700 1 -- 172.21.15.157:0/1454504919 shutdown_connections 2023-10-06T04:08:22.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.946+0000 7f1f6ac2a700 1 --2- 172.21.15.157:0/1454504919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f6406ade0 0x7f1f640694a0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:22.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.946+0000 7f1f6ac2a700 1 -- 172.21.15.157:0/1454504919 >> 172.21.15.157:0/1454504919 conn(0x7f1f640c8d70 msgr2=0x7f1f640c4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:22.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.946+0000 7f1f6ac2a700 1 -- 172.21.15.157:0/1454504919 shutdown_connections 2023-10-06T04:08:22.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.946+0000 7f1f6ac2a700 1 -- 172.21.15.157:0/1454504919 wait complete. 2023-10-06T04:08:22.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.947+0000 7f1f6ac2a700 1 Processor -- start 2023-10-06T04:08:22.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.947+0000 7f1f6ac2a700 1 -- start start 2023-10-06T04:08:22.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.948+0000 7f1f6ac2a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f6406ade0 0x7f1f641569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:22.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.948+0000 7f1f6ac2a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f640ad4c0 0x7f1f640bd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:22.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.948+0000 7f1f6ac2a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f640c4570 0x7f1f640ac7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:22.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.948+0000 7f1f689c6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f6406ade0 0x7f1f641569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:22.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.948+0000 7f1f6ac2a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f640ac310 con 0x7f1f6406ade0 2023-10-06T04:08:22.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.948+0000 7f1f6ac2a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f640acf10 con 0x7f1f640c4570 2023-10-06T04:08:22.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.948+0000 7f1f6ac2a700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f64149fb0 con 0x7f1f640ad4c0 2023-10-06T04:08:22.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.948+0000 7f1f689c6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f6406ade0 0x7f1f641569f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52078/0 (socket says 172.21.15.157:52078) 2023-10-06T04:08:22.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.948+0000 7f1f689c6700 1 -- 172.21.15.157:0/928800617 learned_addr learned my addr 172.21.15.157:0/928800617 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:08:22.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.948+0000 7f1f5bfff700 1 --2- 172.21.15.157:0/928800617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f640ad4c0 0x7f1f640bd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:22.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.948+0000 7f1f691c7700 1 --2- 172.21.15.157:0/928800617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f640c4570 0x7f1f640ac7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:22.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.948+0000 7f1f689c6700 1 -- 172.21.15.157:0/928800617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f640ad4c0 msgr2=0x7f1f640bd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:22.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.948+0000 7f1f689c6700 1 --2- 172.21.15.157:0/928800617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f640ad4c0 0x7f1f640bd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:22.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.948+0000 7f1f689c6700 1 -- 172.21.15.157:0/928800617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f640c4570 msgr2=0x7f1f640ac7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:22.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.948+0000 7f1f689c6700 1 --2- 172.21.15.157:0/928800617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f640c4570 0x7f1f640ac7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:22.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.948+0000 7f1f689c6700 1 -- 172.21.15.157:0/928800617 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f54009420 con 0x7f1f6406ade0 2023-10-06T04:08:22.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.948+0000 7f1f691c7700 1 --2- 172.21.15.157:0/928800617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f640c4570 0x7f1f640ac7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:08:22.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.948+0000 7f1f689c6700 1 --2- 172.21.15.157:0/928800617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f6406ade0 0x7f1f641569f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000cb40 tx=0x7f1f600017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:22.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.948+0000 7f1f58ff9700 1 -- 172.21.15.157:0/928800617 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6000a7a0 con 0x7f1f6406ade0 2023-10-06T04:08:22.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.949+0000 7f1f58ff9700 1 -- 172.21.15.157:0/928800617 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6000a920 con 0x7f1f6406ade0 2023-10-06T04:08:22.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.949+0000 7f1f6ac2a700 1 -- 172.21.15.157:0/928800617 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f640ad180 con 0x7f1f6406ade0 2023-10-06T04:08:22.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.949+0000 7f1f58ff9700 1 -- 172.21.15.157:0/928800617 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6000ace0 con 0x7f1f6406ade0 2023-10-06T04:08:22.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.949+0000 7f1f6ac2a700 1 -- 172.21.15.157:0/928800617 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f640abc60 con 0x7f1f6406ade0 2023-10-06T04:08:22.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.950+0000 7f1f6ac2a700 1 -- 172.21.15.157:0/928800617 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f640ac5f0 con 0x7f1f6406ade0 2023-10-06T04:08:22.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.950+0000 7f1f58ff9700 1 -- 172.21.15.157:0/928800617 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1f60003030 con 0x7f1f6406ade0 2023-10-06T04:08:22.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.950+0000 7f1f58ff9700 1 --2- 172.21.15.157:0/928800617 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1f50066f30 0x7f1f500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:22.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.950+0000 7f1f58ff9700 1 -- 172.21.15.157:0/928800617 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6546+0+0 (secure 0 0 0) 0x7f1f60084980 con 0x7f1f6406ade0 2023-10-06T04:08:22.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.951+0000 7f1f5bfff700 1 --2- 172.21.15.157:0/928800617 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1f50066f30 0x7f1f500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:22.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.951+0000 7f1f5bfff700 1 --2- 172.21.15.157:0/928800617 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1f50066f30 0x7f1f500693e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1f6406b200 tx=0x7f1f54017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:22.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:22.955+0000 7f1f58ff9700 1 -- 172.21.15.157:0/928800617 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f600518a0 con 0x7f1f6406ade0 2023-10-06T04:08:23.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:23.178+0000 7f1f6ac2a700 1 -- 172.21.15.157:0/928800617 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f640716b0 con 0x7f1f6406ade0 2023-10-06T04:08:23.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:23.178+0000 7f1f58ff9700 1 -- 172.21.15.157:0/928800617 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+6147 (secure 0 0 0) 0x7f1f60054ec0 con 0x7f1f6406ade0 2023-10-06T04:08:23.180 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:08:23.180 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5812,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:07:54.745189+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5698},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5698":{"gid":5698,"name":"e","rank":1,"incarnation":72,"state":"up:active","state_seq":33,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:08:23.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:23.181+0000 7f1f6ac2a700 1 -- 172.21.15.157:0/928800617 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1f50066f30 msgr2=0x7f1f500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:23.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:23.181+0000 7f1f6ac2a700 1 --2- 172.21.15.157:0/928800617 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1f50066f30 0x7f1f500693e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1f6406b200 tx=0x7f1f54017000 comp rx=0 tx=0).stop 2023-10-06T04:08:23.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:23.181+0000 7f1f6ac2a700 1 -- 172.21.15.157:0/928800617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f6406ade0 msgr2=0x7f1f641569f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:23.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:23.181+0000 7f1f6ac2a700 1 --2- 172.21.15.157:0/928800617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f6406ade0 0x7f1f641569f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000cb40 tx=0x7f1f600017b0 comp rx=0 tx=0).stop 2023-10-06T04:08:23.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:23.181+0000 7f1f6ac2a700 1 -- 172.21.15.157:0/928800617 shutdown_connections 2023-10-06T04:08:23.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:23.181+0000 7f1f6ac2a700 1 --2- 172.21.15.157:0/928800617 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1f50066f30 0x7f1f500693e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:23.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:23.181+0000 7f1f6ac2a700 1 --2- 172.21.15.157:0/928800617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f640c4570 0x7f1f640ac7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:23.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:23.181+0000 7f1f6ac2a700 1 --2- 172.21.15.157:0/928800617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f6406ade0 0x7f1f641569f0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:23.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:23.181+0000 7f1f6ac2a700 1 --2- 172.21.15.157:0/928800617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f640ad4c0 0x7f1f640bd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:23.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:23.181+0000 7f1f6ac2a700 1 -- 172.21.15.157:0/928800617 >> 172.21.15.157:0/928800617 conn(0x7f1f640c8d70 msgr2=0x7f1f6406f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:23.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:23.181+0000 7f1f6ac2a700 1 -- 172.21.15.157:0/928800617 shutdown_connections 2023-10-06T04:08:23.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:23.181+0000 7f1f6ac2a700 1 -- 172.21.15.157:0/928800617 wait complete. 2023-10-06T04:08:23.184 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 77 2023-10-06T04:08:23.195 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-06T04:08:25.197 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:08:25.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.323+0000 7f72ea5ee700 1 Processor -- start 2023-10-06T04:08:25.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.323+0000 7f72ea5ee700 1 -- start start 2023-10-06T04:08:25.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.324+0000 7f72ea5ee700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f72e406a780 0x7f72e4156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:25.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.324+0000 7f72ea5ee700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f72e406ab50 0x7f72e40c2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:25.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.324+0000 7f72ea5ee700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72e406e840 con 0x7f72e406e340 2023-10-06T04:08:25.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.324+0000 7f72ea5ee700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72e4153970 con 0x7f72e406ab50 2023-10-06T04:08:25.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.324+0000 7f72ea5ee700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72e4157100 con 0x7f72e406a780 2023-10-06T04:08:25.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.324+0000 7f72e8b8b700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f72e406ab50 0x7f72e40c2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:25.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.324+0000 7f72e37fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f72e406e340 0x7f72e40c5bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40016/0 (socket says 172.21.15.157:40016) 2023-10-06T04:08:25.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.324+0000 7f72e37fe700 1 -- 172.21.15.157:0/3921904396 learned_addr learned my addr 172.21.15.157:0/3921904396 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:08:25.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.325+0000 7f72e3fff700 1 --2- 172.21.15.157:0/3921904396 >> v2:172.21.15.175:3300/0 conn(0x7f72e406a780 0x7f72e4156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:25.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.325+0000 7f72e8b8b700 1 -- 172.21.15.157:0/3921904396 >> v2:172.21.15.175:3300/0 conn(0x7f72e406a780 msgr2=0x7f72e4156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:25.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.325+0000 7f72e8b8b700 1 --2- 172.21.15.157:0/3921904396 >> v2:172.21.15.175:3300/0 conn(0x7f72e406a780 0x7f72e4156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:25.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.325+0000 7f72e8b8b700 1 -- 172.21.15.157:0/3921904396 >> v1:172.21.15.157:6789/0 conn(0x7f72e406e340 legacy=0x7f72e40c5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:25.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.325+0000 7f72e8b8b700 1 -- 172.21.15.157:0/3921904396 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e4152fd0 con 0x7f72e406ab50 2023-10-06T04:08:25.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.325+0000 7f72e2ffd700 1 -- 172.21.15.157:0/3921904396 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 758755122 0 0) 0x7f72e406e840 con 0x7f72e406e340 2023-10-06T04:08:25.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.325+0000 7f72e3fff700 1 --2- 172.21.15.157:0/3921904396 >> v2:172.21.15.175:3300/0 conn(0x7f72e406a780 0x7f72e4156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:08:25.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.325+0000 7f72e8b8b700 1 --2- 172.21.15.157:0/3921904396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72e406ab50 0x7f72e40c2400 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f72e4069a80 tx=0x7f72dc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18628d867ae9170b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:25.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.325+0000 7f72e2ffd700 1 -- 172.21.15.157:0/3921904396 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc00e070 con 0x7f72e406ab50 2023-10-06T04:08:25.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.325+0000 7f72e2ffd700 1 -- 172.21.15.157:0/3921904396 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72dc00bc80 con 0x7f72e406ab50 2023-10-06T04:08:25.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.325+0000 7f72e2ffd700 1 -- 172.21.15.157:0/3921904396 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc012430 con 0x7f72e406ab50 2023-10-06T04:08:25.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.326+0000 7f72ea5ee700 1 -- 172.21.15.157:0/3921904396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72e406ab50 msgr2=0x7f72e40c2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:25.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.326+0000 7f72ea5ee700 1 --2- 172.21.15.157:0/3921904396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72e406ab50 0x7f72e40c2400 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f72e4069a80 tx=0x7f72dc009780 comp rx=0 tx=0).stop 2023-10-06T04:08:25.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.326+0000 7f72ea5ee700 1 -- 172.21.15.157:0/3921904396 shutdown_connections 2023-10-06T04:08:25.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.326+0000 7f72ea5ee700 1 --2- 172.21.15.157:0/3921904396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72e406ab50 0x7f72e40c2400 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:25.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.326+0000 7f72ea5ee700 1 --2- 172.21.15.157:0/3921904396 >> v2:172.21.15.175:3300/0 conn(0x7f72e406a780 0x7f72e4156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:25.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.326+0000 7f72ea5ee700 1 -- 172.21.15.157:0/3921904396 >> 172.21.15.157:0/3921904396 conn(0x7f72e406b510 msgr2=0x7f72e406b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:25.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.326+0000 7f72ea5ee700 1 -- 172.21.15.157:0/3921904396 shutdown_connections 2023-10-06T04:08:25.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.326+0000 7f72ea5ee700 1 -- 172.21.15.157:0/3921904396 wait complete. 2023-10-06T04:08:25.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.327+0000 7f72ea5ee700 1 Processor -- start 2023-10-06T04:08:25.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.327+0000 7f72ea5ee700 1 -- start start 2023-10-06T04:08:25.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.328+0000 7f72ea5ee700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72e406a780 0x7f72e40c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:25.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.328+0000 7f72ea5ee700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72e406ab50 0x7f72e4166620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:25.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.328+0000 7f72ea5ee700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72e406e340 0x7f72e414fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:25.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.328+0000 7f72ea5ee700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e4165e80 con 0x7f72e406e340 2023-10-06T04:08:25.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.328+0000 7f72ea5ee700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e4152310 con 0x7f72e406a780 2023-10-06T04:08:25.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.328+0000 7f72ea5ee700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e41525d0 con 0x7f72e406ab50 2023-10-06T04:08:25.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.328+0000 7f72e8b8b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72e406e340 0x7f72e414fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:25.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.328+0000 7f72e8b8b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72e406e340 0x7f72e414fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45724/0 (socket says 172.21.15.157:45724) 2023-10-06T04:08:25.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.328+0000 7f72e8b8b700 1 -- 172.21.15.157:0/1578992973 learned_addr learned my addr 172.21.15.157:0/1578992973 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:08:25.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.328+0000 7f72e3fff700 1 --2- 172.21.15.157:0/1578992973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72e406a780 0x7f72e40c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:25.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.328+0000 7f72e37fe700 1 --2- 172.21.15.157:0/1578992973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72e406ab50 0x7f72e4166620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:25.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.328+0000 7f72e8b8b700 1 -- 172.21.15.157:0/1578992973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72e406ab50 msgr2=0x7f72e4166620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:25.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.328+0000 7f72e8b8b700 1 --2- 172.21.15.157:0/1578992973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72e406ab50 0x7f72e4166620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:25.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.328+0000 7f72e8b8b700 1 -- 172.21.15.157:0/1578992973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72e406a780 msgr2=0x7f72e40c97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:25.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.328+0000 7f72e8b8b700 1 --2- 172.21.15.157:0/1578992973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72e406a780 0x7f72e40c97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:25.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.328+0000 7f72e8b8b700 1 -- 172.21.15.157:0/1578992973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72dc009a20 con 0x7f72e406e340 2023-10-06T04:08:25.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.328+0000 7f72e37fe700 1 --2- 172.21.15.157:0/1578992973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72e406ab50 0x7f72e4166620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:08:25.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.328+0000 7f72e8b8b700 1 --2- 172.21.15.157:0/1578992973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72e406e340 0x7f72e414fb20 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f72e406b1f0 tx=0x7f72dc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:25.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.328+0000 7f72cbfff700 1 -- 172.21.15.157:0/1578992973 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc01e700 con 0x7f72e406e340 2023-10-06T04:08:25.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.329+0000 7f72cbfff700 1 -- 172.21.15.157:0/1578992973 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72dc01e880 con 0x7f72e406e340 2023-10-06T04:08:25.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.329+0000 7f72ea5ee700 1 -- 172.21.15.157:0/1578992973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e41528b0 con 0x7f72e406e340 2023-10-06T04:08:25.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.329+0000 7f72cbfff700 1 -- 172.21.15.157:0/1578992973 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc01ec40 con 0x7f72e406e340 2023-10-06T04:08:25.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.329+0000 7f72ea5ee700 1 -- 172.21.15.157:0/1578992973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e4166100 con 0x7f72e406e340 2023-10-06T04:08:25.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.330+0000 7f72ea5ee700 1 -- 172.21.15.157:0/1578992973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72e41662f0 con 0x7f72e406e340 2023-10-06T04:08:25.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.330+0000 7f72cbfff700 1 -- 172.21.15.157:0/1578992973 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f72dc024070 con 0x7f72e406e340 2023-10-06T04:08:25.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.330+0000 7f72cbfff700 1 --2- 172.21.15.157:0/1578992973 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f72d0066f30 0x7f72d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:25.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.331+0000 7f72cbfff700 1 -- 172.21.15.157:0/1578992973 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6546+0+0 (secure 0 0 0) 0x7f72dc0873e0 con 0x7f72e406e340 2023-10-06T04:08:25.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.331+0000 7f72e3fff700 1 --2- 172.21.15.157:0/1578992973 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f72d0066f30 0x7f72d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:25.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.331+0000 7f72e3fff700 1 --2- 172.21.15.157:0/1578992973 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f72d0066f30 0x7f72d00693e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f72d400ac00 tx=0x7f72d4006f90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:25.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.335+0000 7f72cbfff700 1 -- 172.21.15.157:0/1578992973 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72dc0543b0 con 0x7f72e406e340 2023-10-06T04:08:25.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.561+0000 7f72ea5ee700 1 -- 172.21.15.157:0/1578992973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72e41532b0 con 0x7f72e406e340 2023-10-06T04:08:25.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.562+0000 7f72cbfff700 1 -- 172.21.15.157:0/1578992973 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+6147 (secure 0 0 0) 0x7f72dc0579d0 con 0x7f72e406e340 2023-10-06T04:08:25.564 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:08:25.564 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5812,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:07:54.745189+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5698},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5698":{"gid":5698,"name":"e","rank":1,"incarnation":72,"state":"up:active","state_seq":33,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:08:25.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.564+0000 7f72ea5ee700 1 -- 172.21.15.157:0/1578992973 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f72d0066f30 msgr2=0x7f72d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:25.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.564+0000 7f72ea5ee700 1 --2- 172.21.15.157:0/1578992973 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f72d0066f30 0x7f72d00693e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f72d400ac00 tx=0x7f72d4006f90 comp rx=0 tx=0).stop 2023-10-06T04:08:25.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.564+0000 7f72ea5ee700 1 -- 172.21.15.157:0/1578992973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72e406e340 msgr2=0x7f72e414fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:25.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.564+0000 7f72ea5ee700 1 --2- 172.21.15.157:0/1578992973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72e406e340 0x7f72e414fb20 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f72e406b1f0 tx=0x7f72dc019000 comp rx=0 tx=0).stop 2023-10-06T04:08:25.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.564+0000 7f72ea5ee700 1 -- 172.21.15.157:0/1578992973 shutdown_connections 2023-10-06T04:08:25.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.564+0000 7f72ea5ee700 1 --2- 172.21.15.157:0/1578992973 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f72d0066f30 0x7f72d00693e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:25.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.564+0000 7f72ea5ee700 1 --2- 172.21.15.157:0/1578992973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72e406a780 0x7f72e40c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:25.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.564+0000 7f72ea5ee700 1 --2- 172.21.15.157:0/1578992973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72e406e340 0x7f72e414fb20 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:25.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.564+0000 7f72ea5ee700 1 --2- 172.21.15.157:0/1578992973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72e406ab50 0x7f72e4166620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:25.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.564+0000 7f72ea5ee700 1 -- 172.21.15.157:0/1578992973 >> 172.21.15.157:0/1578992973 conn(0x7f72e406b510 msgr2=0x7f72e40c7170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:25.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.564+0000 7f72ea5ee700 1 -- 172.21.15.157:0/1578992973 shutdown_connections 2023-10-06T04:08:25.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:25.564+0000 7f72ea5ee700 1 -- 172.21.15.157:0/1578992973 wait complete. 2023-10-06T04:08:25.569 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 77 2023-10-06T04:08:25.578 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-06T04:08:27.579 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:08:27.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.700+0000 7fb0fb7d2700 1 Processor -- start 2023-10-06T04:08:27.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.701+0000 7fb0fb7d2700 1 -- start start 2023-10-06T04:08:27.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.701+0000 7fb0fb7d2700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb0f406dc40 0x7fb0f414fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:27.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.701+0000 7fb0fb7d2700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f40c9780 con 0x7fb0f406c1f0 2023-10-06T04:08:27.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.702+0000 7fb0fb7d2700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f406e0c0 con 0x7fb0f406cbb0 2023-10-06T04:08:27.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.702+0000 7fb0fb7d2700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0f40c8c40 con 0x7fb0f406dc40 2023-10-06T04:08:27.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.702+0000 7fb0f9d6f700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb0f406c1f0 0x7fb0f4155f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40018/0 (socket says 172.21.15.157:40018) 2023-10-06T04:08:27.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.702+0000 7fb0f9d6f700 1 -- 172.21.15.157:0/40807881 learned_addr learned my addr 172.21.15.157:0/40807881 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:08:27.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.702+0000 7fb0f8d6d700 1 --2- 172.21.15.157:0/40807881 >> v2:172.21.15.175:3300/0 conn(0x7fb0f406dc40 0x7fb0f414fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:27.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.702+0000 7fb0ebfff700 1 -- 172.21.15.157:0/40807881 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4037651319 0 0) 0x7fb0f40c9780 con 0x7fb0f406c1f0 2023-10-06T04:08:27.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.702+0000 7fb0ebfff700 1 -- 172.21.15.157:0/40807881 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0e0003570 con 0x7fb0f406c1f0 2023-10-06T04:08:27.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.702+0000 7fb0ebfff700 1 -- 172.21.15.157:0/40807881 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1404044731 0 0) 0x7fb0f406e0c0 con 0x7fb0f406cbb0 2023-10-06T04:08:27.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.702+0000 7fb0ebfff700 1 -- 172.21.15.157:0/40807881 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0f40c9780 con 0x7fb0f406cbb0 2023-10-06T04:08:27.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.702+0000 7fb0ebfff700 1 -- 172.21.15.157:0/40807881 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4015235380 0 0) 0x7fb0e0003570 con 0x7fb0f406c1f0 2023-10-06T04:08:27.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.702+0000 7fb0ebfff700 1 -- 172.21.15.157:0/40807881 >> v2:172.21.15.175:3300/0 conn(0x7fb0f406dc40 msgr2=0x7fb0f414fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:27.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.702+0000 7fb0ebfff700 1 --2- 172.21.15.157:0/40807881 >> v2:172.21.15.175:3300/0 conn(0x7fb0f406dc40 0x7fb0f414fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:27.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.702+0000 7fb0ebfff700 1 -- 172.21.15.157:0/40807881 >> v1:172.21.15.175:6789/0 conn(0x7fb0f406cbb0 legacy=0x7fb0f406d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:27.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.702+0000 7fb0ebfff700 1 -- 172.21.15.157:0/40807881 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f40c8e50 con 0x7fb0f406c1f0 2023-10-06T04:08:27.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.702+0000 7fb0ebfff700 1 -- 172.21.15.157:0/40807881 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7fb0ec0017f0 con 0x7fb0f406c1f0 2023-10-06T04:08:27.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.702+0000 7fb0f8d6d700 1 --2- 172.21.15.157:0/40807881 >> v2:172.21.15.175:3300/0 conn(0x7fb0f406dc40 0x7fb0f414fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:08:27.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.702+0000 7fb0ebfff700 1 -- 172.21.15.157:0/40807881 >> v1:172.21.15.157:6789/0 conn(0x7fb0f406c1f0 legacy=0x7fb0f4155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:27.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.702+0000 7fb0ebfff700 1 --2- 172.21.15.157:0/40807881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0e0003bf0 0x7fb0e0003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:27.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.702+0000 7fb0ebfff700 1 --2- 172.21.15.157:0/40807881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0e0004810 0x7fb0e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:27.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.702+0000 7fb0ebfff700 1 --2- 172.21.15.157:0/40807881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0e00051c0 0x7fb0e000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:27.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.702+0000 7fb0ebfff700 1 -- 172.21.15.157:0/40807881 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f40c8c40 con 0x7fb0e00051c0 2023-10-06T04:08:27.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.702+0000 7fb0ebfff700 1 -- 172.21.15.157:0/40807881 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e000c0b0 con 0x7fb0e0004810 2023-10-06T04:08:27.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.702+0000 7fb0ebfff700 1 -- 172.21.15.157:0/40807881 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e000c270 con 0x7fb0e0003bf0 2023-10-06T04:08:27.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.702+0000 7fb0f9d6f700 1 --2- 172.21.15.157:0/40807881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0e00051c0 0x7fb0e000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:27.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.702+0000 7fb0f8d6d700 1 --2- 172.21.15.157:0/40807881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0e0004810 0x7fb0e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:27.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.702+0000 7fb0f956e700 1 --2- 172.21.15.157:0/40807881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0e0003bf0 0x7fb0e0003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:27.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.703+0000 7fb0f9d6f700 1 -- 172.21.15.157:0/40807881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0e0003bf0 msgr2=0x7fb0e0003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:27.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.703+0000 7fb0f9d6f700 1 --2- 172.21.15.157:0/40807881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0e0003bf0 0x7fb0e0003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:27.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.703+0000 7fb0f9d6f700 1 -- 172.21.15.157:0/40807881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0e0004810 msgr2=0x7fb0e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:27.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.703+0000 7fb0f9d6f700 1 --2- 172.21.15.157:0/40807881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0e0004810 0x7fb0e0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:27.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.703+0000 7fb0f9d6f700 1 -- 172.21.15.157:0/40807881 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e0003570 con 0x7fb0e00051c0 2023-10-06T04:08:27.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.703+0000 7fb0f9d6f700 1 -- 172.21.15.157:0/40807881 reap_dead start 2023-10-06T04:08:27.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.703+0000 7fb0f9d6f700 1 --2- 172.21.15.157:0/40807881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0e00051c0 0x7fb0e000ba60 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00a8d0 tx=0x7fb0ec002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3adef4931bd315e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:27.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.703+0000 7fb0ebfff700 1 -- 172.21.15.157:0/40807881 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec0070d0 con 0x7fb0e00051c0 2023-10-06T04:08:27.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.703+0000 7fb0ebfff700 1 -- 172.21.15.157:0/40807881 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0ec007230 con 0x7fb0e00051c0 2023-10-06T04:08:27.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.703+0000 7fb0ebfff700 1 -- 172.21.15.157:0/40807881 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec0075f0 con 0x7fb0e00051c0 2023-10-06T04:08:27.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.704+0000 7fb0fb7d2700 1 -- 172.21.15.157:0/40807881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0e00051c0 msgr2=0x7fb0e000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:27.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.704+0000 7fb0fb7d2700 1 --2- 172.21.15.157:0/40807881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0e00051c0 0x7fb0e000ba60 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00a8d0 tx=0x7fb0ec002190 comp rx=0 tx=0).stop 2023-10-06T04:08:27.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.704+0000 7fb0fb7d2700 1 -- 172.21.15.157:0/40807881 shutdown_connections 2023-10-06T04:08:27.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.704+0000 7fb0fb7d2700 1 --2- 172.21.15.157:0/40807881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0e00051c0 0x7fb0e000ba60 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:27.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.704+0000 7fb0fb7d2700 1 -- 172.21.15.157:0/40807881 >> 172.21.15.157:0/40807881 conn(0x7fb0f406f230 msgr2=0x7fb0f406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:27.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.704+0000 7fb0fb7d2700 1 -- 172.21.15.157:0/40807881 shutdown_connections 2023-10-06T04:08:27.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.704+0000 7fb0fb7d2700 1 -- 172.21.15.157:0/40807881 wait complete. 2023-10-06T04:08:27.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.705+0000 7fb0fb7d2700 1 Processor -- start 2023-10-06T04:08:27.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.705+0000 7fb0fb7d2700 1 -- start start 2023-10-06T04:08:27.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.706+0000 7fb0fb7d2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0e00051c0 0x7fb0f40bff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:27.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.706+0000 7fb0fb7d2700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0f40c12d0 0x7fb0f40afd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:27.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.706+0000 7fb0fb7d2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0f4166f20 0x7fb0f415ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:27.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.706+0000 7fb0fb7d2700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f4145d10 con 0x7fb0e00051c0 2023-10-06T04:08:27.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.706+0000 7fb0fb7d2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f41488f0 con 0x7fb0f4166f20 2023-10-06T04:08:27.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.706+0000 7fb0fb7d2700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f40cbc50 con 0x7fb0f40c12d0 2023-10-06T04:08:27.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.706+0000 7fb0f956e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0e00051c0 0x7fb0f40bff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:27.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.706+0000 7fb0f956e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0e00051c0 0x7fb0f40bff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45744/0 (socket says 172.21.15.157:45744) 2023-10-06T04:08:27.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.706+0000 7fb0f956e700 1 -- 172.21.15.157:0/1202356624 learned_addr learned my addr 172.21.15.157:0/1202356624 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:08:27.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.706+0000 7fb0f9d6f700 1 --2- 172.21.15.157:0/1202356624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0f4166f20 0x7fb0f415ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:27.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.706+0000 7fb0f8d6d700 1 --2- 172.21.15.157:0/1202356624 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0f40c12d0 0x7fb0f40afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:27.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.706+0000 7fb0f956e700 1 -- 172.21.15.157:0/1202356624 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0f40c12d0 msgr2=0x7fb0f40afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:27.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.706+0000 7fb0f956e700 1 --2- 172.21.15.157:0/1202356624 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0f40c12d0 0x7fb0f40afd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:27.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.706+0000 7fb0f956e700 1 -- 172.21.15.157:0/1202356624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0f4166f20 msgr2=0x7fb0f415ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:27.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.706+0000 7fb0f956e700 1 --2- 172.21.15.157:0/1202356624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0f4166f20 0x7fb0f415ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:27.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.706+0000 7fb0f956e700 1 -- 172.21.15.157:0/1202356624 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f40cbed0 con 0x7fb0e00051c0 2023-10-06T04:08:27.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.706+0000 7fb0f956e700 1 --2- 172.21.15.157:0/1202356624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0e00051c0 0x7fb0f40bff80 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0002b20 tx=0x7fb0f0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:27.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.707+0000 7fb0e97fa700 1 -- 172.21.15.157:0/1202356624 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f000fa30 con 0x7fb0e00051c0 2023-10-06T04:08:27.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.707+0000 7fb0e97fa700 1 -- 172.21.15.157:0/1202356624 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f000fbb0 con 0x7fb0e00051c0 2023-10-06T04:08:27.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.707+0000 7fb0fb7d2700 1 -- 172.21.15.157:0/1202356624 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f40cc1b0 con 0x7fb0e00051c0 2023-10-06T04:08:27.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.707+0000 7fb0e97fa700 1 -- 172.21.15.157:0/1202356624 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f00133f0 con 0x7fb0e00051c0 2023-10-06T04:08:27.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.707+0000 7fb0fb7d2700 1 -- 172.21.15.157:0/1202356624 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f40af520 con 0x7fb0e00051c0 2023-10-06T04:08:27.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.708+0000 7fb0fb7d2700 1 -- 172.21.15.157:0/1202356624 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f4089ca0 con 0x7fb0e00051c0 2023-10-06T04:08:27.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.708+0000 7fb0e97fa700 1 -- 172.21.15.157:0/1202356624 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb0f0013760 con 0x7fb0e00051c0 2023-10-06T04:08:27.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.709+0000 7fb0e97fa700 1 --2- 172.21.15.157:0/1202356624 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb0d8066f60 0x7fb0d8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:27.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.709+0000 7fb0e97fa700 1 -- 172.21.15.157:0/1202356624 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6546+0+0 (secure 0 0 0) 0x7fb0f00851e0 con 0x7fb0e00051c0 2023-10-06T04:08:27.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.709+0000 7fb0f8d6d700 1 --2- 172.21.15.157:0/1202356624 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb0d8066f60 0x7fb0d8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:27.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.709+0000 7fb0f8d6d700 1 --2- 172.21.15.157:0/1202356624 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb0d8066f60 0x7fb0d8069410 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb0f40b06b0 tx=0x7fb0e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:27.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.713+0000 7fb0e97fa700 1 -- 172.21.15.157:0/1202356624 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0f0052180 con 0x7fb0e00051c0 2023-10-06T04:08:27.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.942+0000 7fb0fb7d2700 1 -- 172.21.15.157:0/1202356624 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0f40c9130 con 0x7fb0e00051c0 2023-10-06T04:08:27.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.942+0000 7fb0e97fa700 1 -- 172.21.15.157:0/1202356624 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+6147 (secure 0 0 0) 0x7fb0f00557a0 con 0x7fb0e00051c0 2023-10-06T04:08:27.944 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:08:27.944 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5812,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:07:54.745189+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5698},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5698":{"gid":5698,"name":"e","rank":1,"incarnation":72,"state":"up:active","state_seq":33,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:08:27.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.945+0000 7fb0fb7d2700 1 -- 172.21.15.157:0/1202356624 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb0d8066f60 msgr2=0x7fb0d8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:27.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.945+0000 7fb0fb7d2700 1 --2- 172.21.15.157:0/1202356624 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb0d8066f60 0x7fb0d8069410 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb0f40b06b0 tx=0x7fb0e4008040 comp rx=0 tx=0).stop 2023-10-06T04:08:27.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.945+0000 7fb0fb7d2700 1 -- 172.21.15.157:0/1202356624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0e00051c0 msgr2=0x7fb0f40bff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:27.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.945+0000 7fb0fb7d2700 1 --2- 172.21.15.157:0/1202356624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0e00051c0 0x7fb0f40bff80 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0002b20 tx=0x7fb0f0007d30 comp rx=0 tx=0).stop 2023-10-06T04:08:27.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.945+0000 7fb0fb7d2700 1 -- 172.21.15.157:0/1202356624 shutdown_connections 2023-10-06T04:08:27.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.945+0000 7fb0fb7d2700 1 --2- 172.21.15.157:0/1202356624 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb0d8066f60 0x7fb0d8069410 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:27.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.945+0000 7fb0fb7d2700 1 --2- 172.21.15.157:0/1202356624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb0f4166f20 0x7fb0f415ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:27.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.945+0000 7fb0fb7d2700 1 --2- 172.21.15.157:0/1202356624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0e00051c0 0x7fb0f40bff80 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:27.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.945+0000 7fb0fb7d2700 1 --2- 172.21.15.157:0/1202356624 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0f40c12d0 0x7fb0f40afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:27.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.945+0000 7fb0fb7d2700 1 -- 172.21.15.157:0/1202356624 >> 172.21.15.157:0/1202356624 conn(0x7fb0f406f230 msgr2=0x7fb0f414d950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:27.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.945+0000 7fb0fb7d2700 1 -- 172.21.15.157:0/1202356624 shutdown_connections 2023-10-06T04:08:27.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:27.945+0000 7fb0fb7d2700 1 -- 172.21.15.157:0/1202356624 wait complete. 2023-10-06T04:08:27.948 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 77 2023-10-06T04:08:27.958 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-06T04:08:29.959 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:08:30.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.085+0000 7f91a31df700 1 Processor -- start 2023-10-06T04:08:30.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.085+0000 7f91a31df700 1 -- start start 2023-10-06T04:08:30.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.086+0000 7f91a31df700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f919c0b1e40 0x7f919c157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:30.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.086+0000 7f91a31df700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f919c0b04d0 0x7f919c06dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:30.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.086+0000 7f91a31df700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f919c06d8d0 con 0x7f919c06c2d0 2023-10-06T04:08:30.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.086+0000 7f91a31df700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f919c0af8f0 con 0x7f919c0b04d0 2023-10-06T04:08:30.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.086+0000 7f91a31df700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f919c15a690 con 0x7f919c0b1e40 2023-10-06T04:08:30.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.086+0000 7f91a29de700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f919c06c2d0 0x7f919c1563b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40024/0 (socket says 172.21.15.157:40024) 2023-10-06T04:08:30.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.086+0000 7f91a29de700 1 -- 172.21.15.157:0/117445516 learned_addr learned my addr 172.21.15.157:0/117445516 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:08:30.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.086+0000 7f91a19dc700 1 --2- 172.21.15.157:0/117445516 >> v2:172.21.15.175:3300/0 conn(0x7f919c0b04d0 0x7f919c06dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:30.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.086+0000 7f91a21dd700 1 --2- 172.21.15.157:0/117445516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f919c0b1e40 0x7f919c157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:30.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.086+0000 7f91a11db700 1 -- 172.21.15.157:0/117445516 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 491747056 0 0) 0x7f919c06d8d0 con 0x7f919c06c2d0 2023-10-06T04:08:30.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.086+0000 7f91a11db700 1 -- 172.21.15.157:0/117445516 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9184003570 con 0x7f919c06c2d0 2023-10-06T04:08:30.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.087+0000 7f91a11db700 1 -- 172.21.15.157:0/117445516 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3127693421 0 0) 0x7f9184003570 con 0x7f919c06c2d0 2023-10-06T04:08:30.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.087+0000 7f91a11db700 1 -- 172.21.15.157:0/117445516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f919c0b1e40 msgr2=0x7f919c157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:30.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.087+0000 7f91a11db700 1 --2- 172.21.15.157:0/117445516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f919c0b1e40 0x7f919c157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:30.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.087+0000 7f91a11db700 1 -- 172.21.15.157:0/117445516 >> v2:172.21.15.175:3300/0 conn(0x7f919c0b04d0 msgr2=0x7f919c06dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:30.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.087+0000 7f91a11db700 1 --2- 172.21.15.157:0/117445516 >> v2:172.21.15.175:3300/0 conn(0x7f919c0b04d0 0x7f919c06dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:30.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.087+0000 7f91a11db700 1 -- 172.21.15.157:0/117445516 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f919c06ccc0 con 0x7f919c06c2d0 2023-10-06T04:08:30.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.087+0000 7f91a11db700 1 -- 172.21.15.157:0/117445516 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f9194001470 con 0x7f919c06c2d0 2023-10-06T04:08:30.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.087+0000 7f91a11db700 1 -- 172.21.15.157:0/117445516 >> v1:172.21.15.157:6789/0 conn(0x7f919c06c2d0 legacy=0x7f919c1563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:30.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.087+0000 7f91a11db700 1 --2- 172.21.15.157:0/117445516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f919c0b1e40 0x7f9184003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:30.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.087+0000 7f91a11db700 1 --2- 172.21.15.157:0/117445516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9184003fe0 0x7f9184004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:30.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.087+0000 7f91a11db700 1 --2- 172.21.15.157:0/117445516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9184004d50 0x7f91840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:30.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.087+0000 7f91a11db700 1 -- 172.21.15.157:0/117445516 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c0af8f0 con 0x7f9184003fe0 2023-10-06T04:08:30.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.087+0000 7f91a11db700 1 -- 172.21.15.157:0/117445516 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c15a690 con 0x7f9184004d50 2023-10-06T04:08:30.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.087+0000 7f91a11db700 1 -- 172.21.15.157:0/117445516 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9184014350 con 0x7f919c0b1e40 2023-10-06T04:08:30.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.087+0000 7f91a19dc700 1 --2- 172.21.15.157:0/117445516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9184003fe0 0x7f9184004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:30.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.087+0000 7f91a21dd700 1 --2- 172.21.15.157:0/117445516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f919c0b1e40 0x7f9184003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:30.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.087+0000 7f91a29de700 1 --2- 172.21.15.157:0/117445516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9184004d50 0x7f91840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:30.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.087+0000 7f91a19dc700 1 -- 172.21.15.157:0/117445516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f919c0b1e40 msgr2=0x7f9184003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:30.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.087+0000 7f91a19dc700 1 --2- 172.21.15.157:0/117445516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f919c0b1e40 0x7f9184003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:30.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.087+0000 7f91a19dc700 1 -- 172.21.15.157:0/117445516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9184004d50 msgr2=0x7f91840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:30.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.087+0000 7f91a19dc700 1 --2- 172.21.15.157:0/117445516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9184004d50 0x7f91840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:30.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.087+0000 7f91a19dc700 1 -- 172.21.15.157:0/117445516 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9184003570 con 0x7f9184003fe0 2023-10-06T04:08:30.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.087+0000 7f91a19dc700 1 --2- 172.21.15.157:0/117445516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9184003fe0 0x7f9184004810 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9190006f00 tx=0x7f91900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15f05bca56a853d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:30.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.087+0000 7f91a11db700 1 -- 172.21.15.157:0/117445516 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919000e710 con 0x7f9184003fe0 2023-10-06T04:08:30.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.088+0000 7f91a11db700 1 -- 172.21.15.157:0/117445516 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f919000e870 con 0x7f9184003fe0 2023-10-06T04:08:30.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.088+0000 7f91a11db700 1 -- 172.21.15.157:0/117445516 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919000ec30 con 0x7f9184003fe0 2023-10-06T04:08:30.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.088+0000 7f91a31df700 1 -- 172.21.15.157:0/117445516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9184003fe0 msgr2=0x7f9184004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:30.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.088+0000 7f91a31df700 1 --2- 172.21.15.157:0/117445516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9184003fe0 0x7f9184004810 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9190006f00 tx=0x7f91900091a0 comp rx=0 tx=0).stop 2023-10-06T04:08:30.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.088+0000 7f91a29de700 1 -- 172.21.15.157:0/117445516 reap_dead start 2023-10-06T04:08:30.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.088+0000 7f91a31df700 1 -- 172.21.15.157:0/117445516 shutdown_connections 2023-10-06T04:08:30.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.088+0000 7f91a31df700 1 -- 172.21.15.157:0/117445516 >> 172.21.15.157:0/117445516 conn(0x7f919c0cc540 msgr2=0x7f919c0cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:30.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.088+0000 7f91a31df700 1 -- 172.21.15.157:0/117445516 shutdown_connections 2023-10-06T04:08:30.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.088+0000 7f91a31df700 1 -- 172.21.15.157:0/117445516 wait complete. 2023-10-06T04:08:30.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.089+0000 7f91a31df700 1 Processor -- start 2023-10-06T04:08:30.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.089+0000 7f91a31df700 1 -- start start 2023-10-06T04:08:30.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.090+0000 7f91a31df700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9184003fe0 0x7f919c14ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:30.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.090+0000 7f91a31df700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f919c14db30 0x7f919c152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:30.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.090+0000 7f91a31df700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f919c14b860 0x7f919c14bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:30.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.090+0000 7f91a31df700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c1517d0 con 0x7f919c14db30 2023-10-06T04:08:30.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.090+0000 7f91a31df700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c1512c0 con 0x7f9184003fe0 2023-10-06T04:08:30.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.090+0000 7f91a31df700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f919c1514f0 con 0x7f919c14b860 2023-10-06T04:08:30.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.090+0000 7f91a19dc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f919c14db30 0x7f919c152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:30.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.090+0000 7f91a19dc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f919c14db30 0x7f919c152190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45760/0 (socket says 172.21.15.157:45760) 2023-10-06T04:08:30.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.090+0000 7f91a19dc700 1 -- 172.21.15.157:0/289207515 learned_addr learned my addr 172.21.15.157:0/289207515 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:08:30.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.090+0000 7f91a29de700 1 --2- 172.21.15.157:0/289207515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f919c14b860 0x7f919c14bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:30.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.090+0000 7f91a21dd700 1 --2- 172.21.15.157:0/289207515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9184003fe0 0x7f919c14ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:30.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.090+0000 7f91a19dc700 1 -- 172.21.15.157:0/289207515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f919c14b860 msgr2=0x7f919c14bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:30.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.090+0000 7f91a19dc700 1 --2- 172.21.15.157:0/289207515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f919c14b860 0x7f919c14bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:30.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.090+0000 7f91a19dc700 1 -- 172.21.15.157:0/289207515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9184003fe0 msgr2=0x7f919c14ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:30.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.090+0000 7f91a19dc700 1 --2- 172.21.15.157:0/289207515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9184003fe0 0x7f919c14ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:30.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.090+0000 7f91a19dc700 1 -- 172.21.15.157:0/289207515 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f919c14df80 con 0x7f919c14db30 2023-10-06T04:08:30.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.091+0000 7f91a19dc700 1 --2- 172.21.15.157:0/289207515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f919c14db30 0x7f919c152190 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f919000e4b0 tx=0x7f919001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:30.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.091+0000 7f919a7fc700 1 -- 172.21.15.157:0/289207515 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9190022070 con 0x7f919c14db30 2023-10-06T04:08:30.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.091+0000 7f919a7fc700 1 -- 172.21.15.157:0/289207515 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f919000e9e0 con 0x7f919c14db30 2023-10-06T04:08:30.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.091+0000 7f91a31df700 1 -- 172.21.15.157:0/289207515 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f919c14c820 con 0x7f919c14db30 2023-10-06T04:08:30.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.091+0000 7f919a7fc700 1 -- 172.21.15.157:0/289207515 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9190027470 con 0x7f919c14db30 2023-10-06T04:08:30.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.091+0000 7f91a31df700 1 -- 172.21.15.157:0/289207515 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f919c150f40 con 0x7f919c14db30 2023-10-06T04:08:30.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.092+0000 7f91a31df700 1 -- 172.21.15.157:0/289207515 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f919c14c9b0 con 0x7f919c14db30 2023-10-06T04:08:30.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.092+0000 7f919a7fc700 1 -- 172.21.15.157:0/289207515 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f919001c450 con 0x7f919c14db30 2023-10-06T04:08:30.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.092+0000 7f919a7fc700 1 --2- 172.21.15.157:0/289207515 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9188066f30 0x7f91880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:30.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.093+0000 7f919a7fc700 1 -- 172.21.15.157:0/289207515 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6546+0+0 (secure 0 0 0) 0x7f9190086260 con 0x7f919c14db30 2023-10-06T04:08:30.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.093+0000 7f91a21dd700 1 --2- 172.21.15.157:0/289207515 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9188066f30 0x7f91880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:30.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.093+0000 7f91a21dd700 1 --2- 172.21.15.157:0/289207515 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9188066f30 0x7f91880693e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f918c0094d0 tx=0x7f918c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:30.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.097+0000 7f919a7fc700 1 -- 172.21.15.157:0/289207515 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9190053240 con 0x7f919c14db30 2023-10-06T04:08:30.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.320+0000 7f91a31df700 1 -- 172.21.15.157:0/289207515 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f919c14df80 con 0x7f919c14db30 2023-10-06T04:08:30.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.321+0000 7f919a7fc700 1 -- 172.21.15.157:0/289207515 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+6147 (secure 0 0 0) 0x7f919c14df80 con 0x7f919c14db30 2023-10-06T04:08:30.323 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:08:30.323 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5812,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:07:54.745189+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5698},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5698":{"gid":5698,"name":"e","rank":1,"incarnation":72,"state":"up:active","state_seq":33,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:08:30.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.323+0000 7f91a31df700 1 -- 172.21.15.157:0/289207515 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9188066f30 msgr2=0x7f91880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:30.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.323+0000 7f91a31df700 1 --2- 172.21.15.157:0/289207515 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9188066f30 0x7f91880693e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f918c0094d0 tx=0x7f918c00b040 comp rx=0 tx=0).stop 2023-10-06T04:08:30.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.323+0000 7f91a31df700 1 -- 172.21.15.157:0/289207515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f919c14db30 msgr2=0x7f919c152190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:30.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.323+0000 7f91a31df700 1 --2- 172.21.15.157:0/289207515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f919c14db30 0x7f919c152190 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f919000e4b0 tx=0x7f919001a000 comp rx=0 tx=0).stop 2023-10-06T04:08:30.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.323+0000 7f91a31df700 1 -- 172.21.15.157:0/289207515 shutdown_connections 2023-10-06T04:08:30.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.323+0000 7f91a31df700 1 --2- 172.21.15.157:0/289207515 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9188066f30 0x7f91880693e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:30.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.323+0000 7f91a31df700 1 --2- 172.21.15.157:0/289207515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9184003fe0 0x7f919c14ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:30.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.323+0000 7f91a31df700 1 --2- 172.21.15.157:0/289207515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f919c14b860 0x7f919c14bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:30.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.323+0000 7f91a31df700 1 --2- 172.21.15.157:0/289207515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f919c14db30 0x7f919c152190 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:30.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.323+0000 7f91a31df700 1 -- 172.21.15.157:0/289207515 >> 172.21.15.157:0/289207515 conn(0x7f919c0cc540 msgr2=0x7f919c0c5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:30.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.323+0000 7f91a31df700 1 -- 172.21.15.157:0/289207515 shutdown_connections 2023-10-06T04:08:30.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:30.323+0000 7f91a31df700 1 -- 172.21.15.157:0/289207515 wait complete. 2023-10-06T04:08:30.326 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 77 2023-10-06T04:08:30.337 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-06T04:08:32.338 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:08:32.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.464+0000 7f7c2bfff700 1 Processor -- start 2023-10-06T04:08:32.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.464+0000 7f7c2bfff700 1 -- start start 2023-10-06T04:08:32.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.465+0000 7f7c2bfff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7c2c0b04d0 0x7f7c2c157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:32.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.465+0000 7f7c2bfff700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c2c06c2d0 0x7f7c2c06dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:32.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.465+0000 7f7c2bfff700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c2c06d8d0 con 0x7f7c2c0b1e40 2023-10-06T04:08:32.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.465+0000 7f7c2bfff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c2c0af8f0 con 0x7f7c2c0b04d0 2023-10-06T04:08:32.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.465+0000 7f7c2bfff700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c15a690 con 0x7f7c2c06c2d0 2023-10-06T04:08:32.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.465+0000 7f7c2a7fc700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7c2c0b04d0 0x7f7c2c157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:32.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.465+0000 7f7c2a7fc700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7c2c0b04d0 0x7f7c2c157750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45766/0 (socket says 172.21.15.157:45766) 2023-10-06T04:08:32.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.465+0000 7f7c2a7fc700 1 -- 172.21.15.157:0/2810565579 learned_addr learned my addr 172.21.15.157:0/2810565579 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:08:32.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.465+0000 7f7c2b7fe700 1 --2- 172.21.15.157:0/2810565579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c2c06c2d0 0x7f7c2c06dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:32.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.466+0000 7f7c2a7fc700 1 -- 172.21.15.157:0/2810565579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c2c06c2d0 msgr2=0x7f7c2c06dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:32.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.466+0000 7f7c2a7fc700 1 --2- 172.21.15.157:0/2810565579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c2c06c2d0 0x7f7c2c06dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:32.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.466+0000 7f7c2a7fc700 1 -- 172.21.15.157:0/2810565579 >> v1:172.21.15.175:6789/0 conn(0x7f7c2c0b1e40 legacy=0x7f7c2c1563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:32.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.466+0000 7f7c2a7fc700 1 -- 172.21.15.157:0/2810565579 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c2c06ccc0 con 0x7f7c2c0b04d0 2023-10-06T04:08:32.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.466+0000 7f7c2a7fc700 1 --2- 172.21.15.157:0/2810565579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c2c0b04d0 0x7f7c2c157750 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f7c200099b0 tx=0x7f7c20009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9703e6b4a631b5c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:32.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.466+0000 7f7c29ffb700 1 -- 172.21.15.157:0/2810565579 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2000b860 con 0x7f7c2c0b04d0 2023-10-06T04:08:32.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.466+0000 7f7c29ffb700 1 -- 172.21.15.157:0/2810565579 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2000b9c0 con 0x7f7c2c0b04d0 2023-10-06T04:08:32.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.466+0000 7f7c29ffb700 1 -- 172.21.15.157:0/2810565579 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2000bd80 con 0x7f7c2c0b04d0 2023-10-06T04:08:32.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.466+0000 7f7c2bfff700 1 -- 172.21.15.157:0/2810565579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c2c0b04d0 msgr2=0x7f7c2c157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:32.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.466+0000 7f7c2bfff700 1 --2- 172.21.15.157:0/2810565579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c2c0b04d0 0x7f7c2c157750 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f7c200099b0 tx=0x7f7c20009250 comp rx=0 tx=0).stop 2023-10-06T04:08:32.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.466+0000 7f7c2bfff700 1 -- 172.21.15.157:0/2810565579 shutdown_connections 2023-10-06T04:08:32.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.466+0000 7f7c2bfff700 1 --2- 172.21.15.157:0/2810565579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c2c06c2d0 0x7f7c2c06dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:32.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.466+0000 7f7c2bfff700 1 --2- 172.21.15.157:0/2810565579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c2c0b04d0 0x7f7c2c157750 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:32.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.466+0000 7f7c2bfff700 1 -- 172.21.15.157:0/2810565579 >> 172.21.15.157:0/2810565579 conn(0x7f7c2c0cc540 msgr2=0x7f7c2c0cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:32.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.467+0000 7f7c2bfff700 1 -- 172.21.15.157:0/2810565579 shutdown_connections 2023-10-06T04:08:32.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.467+0000 7f7c2bfff700 1 -- 172.21.15.157:0/2810565579 wait complete. 2023-10-06T04:08:32.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.468+0000 7f7c2bfff700 1 Processor -- start 2023-10-06T04:08:32.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.468+0000 7f7c2bfff700 1 -- start start 2023-10-06T04:08:32.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.468+0000 7f7c2bfff700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c2c06c2d0 0x7f7c2c14f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:32.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.468+0000 7f7c2bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c2c0b1e40 0x7f7c2c167ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:32.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.468+0000 7f7c2bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c2c168000 0x7f7c2c14dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:32.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.468+0000 7f7c2bfff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c150540 con 0x7f7c2c0b1e40 2023-10-06T04:08:32.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.468+0000 7f7c2bfff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c151940 con 0x7f7c2c168000 2023-10-06T04:08:32.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.468+0000 7f7c2bfff700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c159f70 con 0x7f7c2c06c2d0 2023-10-06T04:08:32.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.469+0000 7f7c2a7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c2c0b1e40 0x7f7c2c167ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:32.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.469+0000 7f7c2a7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c2c0b1e40 0x7f7c2c167ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45772/0 (socket says 172.21.15.157:45772) 2023-10-06T04:08:32.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.469+0000 7f7c2a7fc700 1 -- 172.21.15.157:0/2358055784 learned_addr learned my addr 172.21.15.157:0/2358055784 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:08:32.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.469+0000 7f7c2b7fe700 1 --2- 172.21.15.157:0/2358055784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c2c168000 0x7f7c2c14dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:32.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.469+0000 7f7c2affd700 1 --2- 172.21.15.157:0/2358055784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c2c06c2d0 0x7f7c2c14f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:32.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.469+0000 7f7c2a7fc700 1 -- 172.21.15.157:0/2358055784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c2c06c2d0 msgr2=0x7f7c2c14f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:32.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.469+0000 7f7c2a7fc700 1 --2- 172.21.15.157:0/2358055784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c2c06c2d0 0x7f7c2c14f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:32.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.469+0000 7f7c2a7fc700 1 -- 172.21.15.157:0/2358055784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c2c168000 msgr2=0x7f7c2c14dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:32.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.469+0000 7f7c2a7fc700 1 --2- 172.21.15.157:0/2358055784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c2c168000 0x7f7c2c14dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:32.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.469+0000 7f7c2a7fc700 1 -- 172.21.15.157:0/2358055784 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c200094f0 con 0x7f7c2c0b1e40 2023-10-06T04:08:32.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.469+0000 7f7c2a7fc700 1 --2- 172.21.15.157:0/2358055784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c2c0b1e40 0x7f7c2c167ac0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000b3c0 tx=0x7f7c20017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:32.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.469+0000 7f7c12ffd700 1 -- 172.21.15.157:0/2358055784 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2001f070 con 0x7f7c2c0b1e40 2023-10-06T04:08:32.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.469+0000 7f7c12ffd700 1 -- 172.21.15.157:0/2358055784 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c20011e80 con 0x7f7c2c0b1e40 2023-10-06T04:08:32.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.469+0000 7f7c2bfff700 1 -- 172.21.15.157:0/2358055784 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c2c167630 con 0x7f7c2c0b1e40 2023-10-06T04:08:32.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.469+0000 7f7c12ffd700 1 -- 172.21.15.157:0/2358055784 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2000b860 con 0x7f7c2c0b1e40 2023-10-06T04:08:32.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.469+0000 7f7c2bfff700 1 -- 172.21.15.157:0/2358055784 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c2c150260 con 0x7f7c2c0b1e40 2023-10-06T04:08:32.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.470+0000 7f7c2bfff700 1 -- 172.21.15.157:0/2358055784 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c2c1677c0 con 0x7f7c2c0b1e40 2023-10-06T04:08:32.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.471+0000 7f7c12ffd700 1 -- 172.21.15.157:0/2358055784 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7c2001a920 con 0x7f7c2c0b1e40 2023-10-06T04:08:32.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.471+0000 7f7c12ffd700 1 --2- 172.21.15.157:0/2358055784 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7c18066f30 0x7f7c180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:32.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.471+0000 7f7c12ffd700 1 -- 172.21.15.157:0/2358055784 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6546+0+0 (secure 0 0 0) 0x7f7c20085e60 con 0x7f7c2c0b1e40 2023-10-06T04:08:32.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.471+0000 7f7c2affd700 1 --2- 172.21.15.157:0/2358055784 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7c18066f30 0x7f7c180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:32.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.472+0000 7f7c2affd700 1 --2- 172.21.15.157:0/2358055784 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7c18066f30 0x7f7c180693e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c007fd0 tx=0x7f7c1c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:32.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.475+0000 7f7c12ffd700 1 -- 172.21.15.157:0/2358055784 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7c20052e00 con 0x7f7c2c0b1e40 2023-10-06T04:08:32.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.703+0000 7f7c2bfff700 1 -- 172.21.15.157:0/2358055784 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c2c088f90 con 0x7f7c2c0b1e40 2023-10-06T04:08:32.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.703+0000 7f7c12ffd700 1 -- 172.21.15.157:0/2358055784 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+6147 (secure 0 0 0) 0x7f7c20028090 con 0x7f7c2c0b1e40 2023-10-06T04:08:32.705 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:08:32.705 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5812,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:07:54.745189+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5698},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5698":{"gid":5698,"name":"e","rank":1,"incarnation":72,"state":"up:active","state_seq":33,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:08:32.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.705+0000 7f7c2bfff700 1 -- 172.21.15.157:0/2358055784 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7c18066f30 msgr2=0x7f7c180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:32.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.705+0000 7f7c2bfff700 1 --2- 172.21.15.157:0/2358055784 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7c18066f30 0x7f7c180693e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c007fd0 tx=0x7f7c1c009040 comp rx=0 tx=0).stop 2023-10-06T04:08:32.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.705+0000 7f7c2bfff700 1 -- 172.21.15.157:0/2358055784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c2c0b1e40 msgr2=0x7f7c2c167ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:32.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.705+0000 7f7c2bfff700 1 --2- 172.21.15.157:0/2358055784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c2c0b1e40 0x7f7c2c167ac0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000b3c0 tx=0x7f7c20017000 comp rx=0 tx=0).stop 2023-10-06T04:08:32.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.706+0000 7f7c2bfff700 1 -- 172.21.15.157:0/2358055784 shutdown_connections 2023-10-06T04:08:32.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.706+0000 7f7c2bfff700 1 --2- 172.21.15.157:0/2358055784 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7c18066f30 0x7f7c180693e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:32.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.706+0000 7f7c2bfff700 1 --2- 172.21.15.157:0/2358055784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c2c168000 0x7f7c2c14dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:32.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.706+0000 7f7c2bfff700 1 --2- 172.21.15.157:0/2358055784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c2c06c2d0 0x7f7c2c14f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:32.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.706+0000 7f7c2bfff700 1 --2- 172.21.15.157:0/2358055784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c2c0b1e40 0x7f7c2c167ac0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:32.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.706+0000 7f7c2bfff700 1 -- 172.21.15.157:0/2358055784 >> 172.21.15.157:0/2358055784 conn(0x7f7c2c0cc540 msgr2=0x7f7c2c156aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:32.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.706+0000 7f7c2bfff700 1 -- 172.21.15.157:0/2358055784 shutdown_connections 2023-10-06T04:08:32.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:32.706+0000 7f7c2bfff700 1 -- 172.21.15.157:0/2358055784 wait complete. 2023-10-06T04:08:32.709 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 77 2023-10-06T04:08:32.719 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-06T04:08:34.720 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:08:34.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.847+0000 7f4d6ffff700 1 Processor -- start 2023-10-06T04:08:34.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.847+0000 7f4d6ffff700 1 -- start start 2023-10-06T04:08:34.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.848+0000 7f4d6ffff700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4d70153420 0x7f4d700c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:34.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.848+0000 7f4d6ffff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4d7006c7f0 0x7f4d700c2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:34.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.848+0000 7f4d6ffff700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d7015a2f0 con 0x7f4d701537f0 2023-10-06T04:08:34.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.848+0000 7f4d6ffff700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d700c9c30 con 0x7f4d7006c7f0 2023-10-06T04:08:34.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.848+0000 7f4d6ffff700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d701596e0 con 0x7f4d70153420 2023-10-06T04:08:34.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.848+0000 7f4d667fc700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4d7006c7f0 0x7f4d700c2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:34.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.848+0000 7f4d6f7fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f4d701537f0 0x7f4d700c7df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49012/0 (socket says 172.21.15.157:49012) 2023-10-06T04:08:34.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.848+0000 7f4d6f7fe700 1 -- 172.21.15.157:0/1818262880 learned_addr learned my addr 172.21.15.157:0/1818262880 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:08:34.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.848+0000 7f4d6effd700 1 --2- 172.21.15.157:0/1818262880 >> v2:172.21.15.175:3300/0 conn(0x7f4d70153420 0x7f4d700c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:34.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.848+0000 7f4d667fc700 1 -- 172.21.15.157:0/1818262880 >> v2:172.21.15.175:3300/0 conn(0x7f4d70153420 msgr2=0x7f4d700c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:34.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.848+0000 7f4d667fc700 1 --2- 172.21.15.157:0/1818262880 >> v2:172.21.15.175:3300/0 conn(0x7f4d70153420 0x7f4d700c95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:34.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.848+0000 7f4d667fc700 1 -- 172.21.15.157:0/1818262880 >> v1:172.21.15.157:6789/0 conn(0x7f4d701537f0 legacy=0x7f4d700c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:34.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.848+0000 7f4d667fc700 1 -- 172.21.15.157:0/1818262880 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d701599c0 con 0x7f4d7006c7f0 2023-10-06T04:08:34.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.848+0000 7f4d667fc700 1 --2- 172.21.15.157:0/1818262880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d7006c7f0 0x7f4d700c2e00 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f4d700c8bb0 tx=0x7f4d580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c60f689b23a723 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:34.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.849+0000 7f4d6e7fc700 1 -- 172.21.15.157:0/1818262880 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5800b6b0 con 0x7f4d7006c7f0 2023-10-06T04:08:34.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.849+0000 7f4d6e7fc700 1 -- 172.21.15.157:0/1818262880 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5800b810 con 0x7f4d7006c7f0 2023-10-06T04:08:34.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.849+0000 7f4d6e7fc700 1 -- 172.21.15.157:0/1818262880 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5800bbb0 con 0x7f4d7006c7f0 2023-10-06T04:08:34.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.849+0000 7f4d6ffff700 1 -- 172.21.15.157:0/1818262880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d7006c7f0 msgr2=0x7f4d700c2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:34.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.849+0000 7f4d6ffff700 1 --2- 172.21.15.157:0/1818262880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d7006c7f0 0x7f4d700c2e00 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f4d700c8bb0 tx=0x7f4d580091a0 comp rx=0 tx=0).stop 2023-10-06T04:08:34.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.849+0000 7f4d6ffff700 1 -- 172.21.15.157:0/1818262880 shutdown_connections 2023-10-06T04:08:34.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.849+0000 7f4d6ffff700 1 --2- 172.21.15.157:0/1818262880 >> v2:172.21.15.175:3300/0 conn(0x7f4d70153420 0x7f4d700c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:34.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.849+0000 7f4d6ffff700 1 --2- 172.21.15.157:0/1818262880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d7006c7f0 0x7f4d700c2e00 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:34.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.849+0000 7f4d6ffff700 1 -- 172.21.15.157:0/1818262880 >> 172.21.15.157:0/1818262880 conn(0x7f4d700ccf40 msgr2=0x7f4d700c8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:34.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.849+0000 7f4d6ffff700 1 -- 172.21.15.157:0/1818262880 shutdown_connections 2023-10-06T04:08:34.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.849+0000 7f4d6ffff700 1 -- 172.21.15.157:0/1818262880 wait complete. 2023-10-06T04:08:34.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.851+0000 7f4d6ffff700 1 Processor -- start 2023-10-06T04:08:34.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.851+0000 7f4d6ffff700 1 -- start start 2023-10-06T04:08:34.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.851+0000 7f4d6ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d7006c7f0 0x7f4d70154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:34.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.851+0000 7f4d6ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d70153420 0x7f4d701578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:34.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.851+0000 7f4d6ffff700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d701537f0 0x7f4d70157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:34.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.851+0000 7f4d6ffff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7016a7a0 con 0x7f4d7006c7f0 2023-10-06T04:08:34.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.851+0000 7f4d6ffff700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d701592e0 con 0x7f4d70153420 2023-10-06T04:08:34.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.851+0000 7f4d6ffff700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d70169c40 con 0x7f4d701537f0 2023-10-06T04:08:34.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.851+0000 7f4d6effd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d7006c7f0 0x7f4d70154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:34.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.851+0000 7f4d6effd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d7006c7f0 0x7f4d70154da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40308/0 (socket says 172.21.15.157:40308) 2023-10-06T04:08:34.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.851+0000 7f4d6effd700 1 -- 172.21.15.157:0/2164563089 learned_addr learned my addr 172.21.15.157:0/2164563089 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:08:34.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.852+0000 7f4d667fc700 1 --2- 172.21.15.157:0/2164563089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d70153420 0x7f4d701578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:34.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.852+0000 7f4d6f7fe700 1 --2- 172.21.15.157:0/2164563089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d701537f0 0x7f4d70157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:34.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.852+0000 7f4d6effd700 1 -- 172.21.15.157:0/2164563089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d701537f0 msgr2=0x7f4d70157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:34.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.852+0000 7f4d6effd700 1 --2- 172.21.15.157:0/2164563089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d701537f0 0x7f4d70157e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:34.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.852+0000 7f4d6effd700 1 -- 172.21.15.157:0/2164563089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d70153420 msgr2=0x7f4d701578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:34.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.852+0000 7f4d6effd700 1 --2- 172.21.15.157:0/2164563089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d70153420 0x7f4d701578d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:34.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.852+0000 7f4d6effd700 1 -- 172.21.15.157:0/2164563089 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d58009420 con 0x7f4d7006c7f0 2023-10-06T04:08:34.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.852+0000 7f4d6f7fe700 1 --2- 172.21.15.157:0/2164563089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d701537f0 0x7f4d70157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:08:34.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.852+0000 7f4d6effd700 1 --2- 172.21.15.157:0/2164563089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d7006c7f0 0x7f4d70154da0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000d860 tx=0x7f4d60008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:34.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.852+0000 7f4d677fe700 1 -- 172.21.15.157:0/2164563089 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d600125f0 con 0x7f4d7006c7f0 2023-10-06T04:08:34.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.852+0000 7f4d677fe700 1 -- 172.21.15.157:0/2164563089 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d60012750 con 0x7f4d7006c7f0 2023-10-06T04:08:34.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.852+0000 7f4d6ffff700 1 -- 172.21.15.157:0/2164563089 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d70169f00 con 0x7f4d7006c7f0 2023-10-06T04:08:34.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.852+0000 7f4d677fe700 1 -- 172.21.15.157:0/2164563089 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d60012b10 con 0x7f4d7006c7f0 2023-10-06T04:08:34.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.852+0000 7f4d6ffff700 1 -- 172.21.15.157:0/2164563089 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d7016a4c0 con 0x7f4d7006c7f0 2023-10-06T04:08:34.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.853+0000 7f4d6ffff700 1 -- 172.21.15.157:0/2164563089 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d70064af0 con 0x7f4d7006c7f0 2023-10-06T04:08:34.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.853+0000 7f4d677fe700 1 -- 172.21.15.157:0/2164563089 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4d6001e460 con 0x7f4d7006c7f0 2023-10-06T04:08:34.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.854+0000 7f4d677fe700 1 --2- 172.21.15.157:0/2164563089 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4d50066f30 0x7f4d500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:34.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.854+0000 7f4d677fe700 1 -- 172.21.15.157:0/2164563089 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6546+0+0 (secure 0 0 0) 0x7f4d60084e20 con 0x7f4d7006c7f0 2023-10-06T04:08:34.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.854+0000 7f4d667fc700 1 --2- 172.21.15.157:0/2164563089 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4d50066f30 0x7f4d500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:34.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.855+0000 7f4d667fc700 1 --2- 172.21.15.157:0/2164563089 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4d50066f30 0x7f4d500693e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4d580093f0 tx=0x7f4d58017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:34.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:34.858+0000 7f4d677fe700 1 -- 172.21.15.157:0/2164563089 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d60051d40 con 0x7f4d7006c7f0 2023-10-06T04:08:35.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:35.084+0000 7f4d6ffff700 1 -- 172.21.15.157:0/2164563089 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d7005f230 con 0x7f4d7006c7f0 2023-10-06T04:08:35.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:35.085+0000 7f4d677fe700 1 -- 172.21.15.157:0/2164563089 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+6147 (secure 0 0 0) 0x7f4d60055360 con 0x7f4d7006c7f0 2023-10-06T04:08:35.087 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:08:35.087 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5812,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:07:54.745189+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5698},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5698":{"gid":5698,"name":"e","rank":1,"incarnation":72,"state":"up:active","state_seq":33,"addr":"172.21.15.157:6837/3065749287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":3065749287},{"type":"v1","addr":"172.21.15.157:6837","nonce":3065749287}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:08:35.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:35.087+0000 7f4d6ffff700 1 -- 172.21.15.157:0/2164563089 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4d50066f30 msgr2=0x7f4d500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:35.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:35.087+0000 7f4d6ffff700 1 --2- 172.21.15.157:0/2164563089 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4d50066f30 0x7f4d500693e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4d580093f0 tx=0x7f4d58017000 comp rx=0 tx=0).stop 2023-10-06T04:08:35.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:35.087+0000 7f4d6ffff700 1 -- 172.21.15.157:0/2164563089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d7006c7f0 msgr2=0x7f4d70154da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:35.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:35.087+0000 7f4d6ffff700 1 --2- 172.21.15.157:0/2164563089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d7006c7f0 0x7f4d70154da0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000d860 tx=0x7f4d60008040 comp rx=0 tx=0).stop 2023-10-06T04:08:35.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:35.087+0000 7f4d6ffff700 1 -- 172.21.15.157:0/2164563089 shutdown_connections 2023-10-06T04:08:35.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:35.087+0000 7f4d6ffff700 1 --2- 172.21.15.157:0/2164563089 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4d50066f30 0x7f4d500693e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:35.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:35.087+0000 7f4d6ffff700 1 --2- 172.21.15.157:0/2164563089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d701537f0 0x7f4d70157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:35.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:35.087+0000 7f4d6ffff700 1 --2- 172.21.15.157:0/2164563089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d7006c7f0 0x7f4d70154da0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:35.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:35.087+0000 7f4d6ffff700 1 --2- 172.21.15.157:0/2164563089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d70153420 0x7f4d701578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:35.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:35.087+0000 7f4d6ffff700 1 -- 172.21.15.157:0/2164563089 >> 172.21.15.157:0/2164563089 conn(0x7f4d700ccf40 msgr2=0x7f4d7006cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:35.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:35.087+0000 7f4d6ffff700 1 -- 172.21.15.157:0/2164563089 shutdown_connections 2023-10-06T04:08:35.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:35.087+0000 7f4d6ffff700 1 -- 172.21.15.157:0/2164563089 wait complete. 2023-10-06T04:08:35.090 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 77 2023-10-06T04:08:35.101 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-10-06T04:08:37.102 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:08:37.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.227+0000 7effbf29a700 1 Processor -- start 2023-10-06T04:08:37.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.228+0000 7effbf29a700 1 -- start start 2023-10-06T04:08:37.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.228+0000 7effbf29a700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7effb814dcd0 0x7effb8153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:37.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.228+0000 7effbf29a700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7effb814e0a0 0x7effb80c18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:37.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.228+0000 7effbf29a700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb80c5770 con 0x7effb80c5c00 2023-10-06T04:08:37.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.228+0000 7effbf29a700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb8154c80 con 0x7effb814e0a0 2023-10-06T04:08:37.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.228+0000 7effbf29a700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb8154200 con 0x7effb814dcd0 2023-10-06T04:08:37.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.228+0000 7effbc835700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7effb814e0a0 0x7effb80c18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:37.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.228+0000 7effbc835700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7effb814e0a0 0x7effb80c18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40316/0 (socket says 172.21.15.157:40316) 2023-10-06T04:08:37.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.228+0000 7effbc835700 1 -- 172.21.15.157:0/4079624418 learned_addr learned my addr 172.21.15.157:0/4079624418 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:08:37.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.229+0000 7effbd036700 1 --2- 172.21.15.157:0/4079624418 >> v2:172.21.15.175:3300/0 conn(0x7effb814dcd0 0x7effb8153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:37.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.229+0000 7effbc835700 1 -- 172.21.15.157:0/4079624418 >> v2:172.21.15.175:3300/0 conn(0x7effb814dcd0 msgr2=0x7effb8153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:37.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.229+0000 7effbc835700 1 --2- 172.21.15.157:0/4079624418 >> v2:172.21.15.175:3300/0 conn(0x7effb814dcd0 0x7effb8153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:37.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.229+0000 7effbc835700 1 -- 172.21.15.157:0/4079624418 >> v1:172.21.15.175:6789/0 conn(0x7effb80c5c00 legacy=0x7effb80ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:37.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.229+0000 7effbc835700 1 -- 172.21.15.157:0/4079624418 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb81543e0 con 0x7effb814e0a0 2023-10-06T04:08:37.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.229+0000 7effbc835700 1 --2- 172.21.15.157:0/4079624418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effb814e0a0 0x7effb80c18c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7effb806cd20 tx=0x7effa8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6a3d70b22a6dcb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:37.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.229+0000 7effaffff700 1 -- 172.21.15.157:0/4079624418 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa800b8f0 con 0x7effb814e0a0 2023-10-06T04:08:37.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.229+0000 7effaffff700 1 -- 172.21.15.157:0/4079624418 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa800ba50 con 0x7effb814e0a0 2023-10-06T04:08:37.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.229+0000 7effaffff700 1 -- 172.21.15.157:0/4079624418 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa800bdf0 con 0x7effb814e0a0 2023-10-06T04:08:37.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.230+0000 7effbf29a700 1 -- 172.21.15.157:0/4079624418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effb814e0a0 msgr2=0x7effb80c18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:37.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.230+0000 7effbf29a700 1 --2- 172.21.15.157:0/4079624418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effb814e0a0 0x7effb80c18c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7effb806cd20 tx=0x7effa8009250 comp rx=0 tx=0).stop 2023-10-06T04:08:37.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.230+0000 7effbf29a700 1 -- 172.21.15.157:0/4079624418 shutdown_connections 2023-10-06T04:08:37.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.230+0000 7effbf29a700 1 --2- 172.21.15.157:0/4079624418 >> v2:172.21.15.175:3300/0 conn(0x7effb814dcd0 0x7effb8153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:37.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.230+0000 7effbf29a700 1 --2- 172.21.15.157:0/4079624418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effb814e0a0 0x7effb80c18c0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:37.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.230+0000 7effbf29a700 1 -- 172.21.15.157:0/4079624418 >> 172.21.15.157:0/4079624418 conn(0x7effb80692b0 msgr2=0x7effb80696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:37.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.230+0000 7effbf29a700 1 -- 172.21.15.157:0/4079624418 shutdown_connections 2023-10-06T04:08:37.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.230+0000 7effbf29a700 1 -- 172.21.15.157:0/4079624418 wait complete. 2023-10-06T04:08:37.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.231+0000 7effbf29a700 1 Processor -- start 2023-10-06T04:08:37.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.231+0000 7effbf29a700 1 -- start start 2023-10-06T04:08:37.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.232+0000 7effbf29a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effb80c5c00 0x7effb80c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:37.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.232+0000 7effbf29a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effb814dcd0 0x7effb80ca390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:37.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.232+0000 7effbf29a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effb814e0a0 0x7effb8167420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:37.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.232+0000 7effbf29a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb8155510 con 0x7effb814e0a0 2023-10-06T04:08:37.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.232+0000 7effbf29a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb814e860 con 0x7effb80c5c00 2023-10-06T04:08:37.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.232+0000 7effbf29a700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb80c11e0 con 0x7effb814dcd0 2023-10-06T04:08:37.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.232+0000 7effbd837700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effb814e0a0 0x7effb8167420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:37.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.232+0000 7effbd837700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effb814e0a0 0x7effb8167420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40328/0 (socket says 172.21.15.157:40328) 2023-10-06T04:08:37.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.232+0000 7effbd837700 1 -- 172.21.15.157:0/1139882062 learned_addr learned my addr 172.21.15.157:0/1139882062 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:08:37.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.232+0000 7effbd036700 1 --2- 172.21.15.157:0/1139882062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effb80c5c00 0x7effb80c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:37.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.232+0000 7effbc835700 1 --2- 172.21.15.157:0/1139882062 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effb814dcd0 0x7effb80ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:37.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.232+0000 7effbd837700 1 -- 172.21.15.157:0/1139882062 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effb814dcd0 msgr2=0x7effb80ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:37.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.232+0000 7effbd837700 1 --2- 172.21.15.157:0/1139882062 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effb814dcd0 0x7effb80ca390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:37.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.232+0000 7effbd837700 1 -- 172.21.15.157:0/1139882062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effb80c5c00 msgr2=0x7effb80c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:37.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.232+0000 7effbd837700 1 --2- 172.21.15.157:0/1139882062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effb80c5c00 0x7effb80c9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:37.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.232+0000 7effbd837700 1 -- 172.21.15.157:0/1139882062 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa80094f0 con 0x7effb814e0a0 2023-10-06T04:08:37.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.232+0000 7effbc835700 1 --2- 172.21.15.157:0/1139882062 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effb814dcd0 0x7effb80ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:08:37.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.232+0000 7effbd036700 1 --2- 172.21.15.157:0/1139882062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effb80c5c00 0x7effb80c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:08:37.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.232+0000 7effbd837700 1 --2- 172.21.15.157:0/1139882062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effb814e0a0 0x7effb8167420 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7effb000c560 tx=0x7effb0007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:37.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.232+0000 7effad7fa700 1 -- 172.21.15.157:0/1139882062 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb0001120 con 0x7effb814e0a0 2023-10-06T04:08:37.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.232+0000 7effad7fa700 1 -- 172.21.15.157:0/1139882062 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb0011590 con 0x7effb814e0a0 2023-10-06T04:08:37.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.232+0000 7effbf29a700 1 -- 172.21.15.157:0/1139882062 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb80c0f00 con 0x7effb814e0a0 2023-10-06T04:08:37.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.232+0000 7effad7fa700 1 -- 172.21.15.157:0/1139882062 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb0011950 con 0x7effb814e0a0 2023-10-06T04:08:37.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.233+0000 7effbf29a700 1 -- 172.21.15.157:0/1139882062 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb814ea70 con 0x7effb814e0a0 2023-10-06T04:08:37.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.233+0000 7effbf29a700 1 -- 172.21.15.157:0/1139882062 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb806a8f0 con 0x7effb814e0a0 2023-10-06T04:08:37.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.234+0000 7effad7fa700 1 -- 172.21.15.157:0/1139882062 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7effb0011d00 con 0x7effb814e0a0 2023-10-06T04:08:37.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.234+0000 7effad7fa700 1 --2- 172.21.15.157:0/1139882062 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7eff9c066f30 0x7eff9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:37.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.234+0000 7effad7fa700 1 -- 172.21.15.157:0/1139882062 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6632+0+0 (secure 0 0 0) 0x7effb0083d80 con 0x7effb814e0a0 2023-10-06T04:08:37.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.235+0000 7effbd036700 1 --2- 172.21.15.157:0/1139882062 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7eff9c066f30 0x7eff9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:37.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.235+0000 7effbd036700 1 --2- 172.21.15.157:0/1139882062 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7eff9c066f30 0x7eff9c0693e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7effb4007540 tx=0x7effb400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:37.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.239+0000 7effad7fa700 1 -- 172.21.15.157:0/1139882062 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7effb0050c50 con 0x7effb814e0a0 2023-10-06T04:08:37.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.462+0000 7effbf29a700 1 -- 172.21.15.157:0/1139882062 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effb8153fe0 con 0x7effb814e0a0 2023-10-06T04:08:37.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.462+0000 7effad7fa700 1 -- 172.21.15.157:0/1139882062 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+5386 (secure 0 0 0) 0x7effb0017070 con 0x7effb814e0a0 2023-10-06T04:08:37.464 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:08:37.464 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":78,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:08:36.748721+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:replay","state_seq":1,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:08:37.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.465+0000 7effbf29a700 1 -- 172.21.15.157:0/1139882062 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7eff9c066f30 msgr2=0x7eff9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:37.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.465+0000 7effbf29a700 1 --2- 172.21.15.157:0/1139882062 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7eff9c066f30 0x7eff9c0693e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7effb4007540 tx=0x7effb400b040 comp rx=0 tx=0).stop 2023-10-06T04:08:37.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.465+0000 7effbf29a700 1 -- 172.21.15.157:0/1139882062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effb814e0a0 msgr2=0x7effb8167420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:37.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.465+0000 7effbf29a700 1 --2- 172.21.15.157:0/1139882062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effb814e0a0 0x7effb8167420 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7effb000c560 tx=0x7effb0007ef0 comp rx=0 tx=0).stop 2023-10-06T04:08:37.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.465+0000 7effbf29a700 1 -- 172.21.15.157:0/1139882062 shutdown_connections 2023-10-06T04:08:37.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.465+0000 7effbf29a700 1 --2- 172.21.15.157:0/1139882062 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7eff9c066f30 0x7eff9c0693e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:37.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.465+0000 7effbf29a700 1 --2- 172.21.15.157:0/1139882062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effb80c5c00 0x7effb80c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:37.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.465+0000 7effbf29a700 1 --2- 172.21.15.157:0/1139882062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effb814e0a0 0x7effb8167420 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:37.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.465+0000 7effbf29a700 1 --2- 172.21.15.157:0/1139882062 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effb814dcd0 0x7effb80ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:37.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.465+0000 7effbf29a700 1 -- 172.21.15.157:0/1139882062 >> 172.21.15.157:0/1139882062 conn(0x7effb80692b0 msgr2=0x7effb80c17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:37.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.465+0000 7effbf29a700 1 -- 172.21.15.157:0/1139882062 shutdown_connections 2023-10-06T04:08:37.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.465+0000 7effbf29a700 1 -- 172.21.15.157:0/1139882062 wait complete. 2023-10-06T04:08:37.468 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 78 2023-10-06T04:08:37.479 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e down, removed from mdsmap 2023-10-06T04:08:37.479 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-10-06T04:08:37.479 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:08:37.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.594+0000 7f35c5e0c700 1 Processor -- start 2023-10-06T04:08:37.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.594+0000 7f35c5e0c700 1 -- start start 2023-10-06T04:08:37.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.595+0000 7f35c5e0c700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35c006cbb0 0x7f35c006d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:37.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.595+0000 7f35c5e0c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f35c006c1f0 0x7f35c006da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:37.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.595+0000 7f35c5e0c700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35c00c9780 con 0x7f35c006cf80 2023-10-06T04:08:37.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.595+0000 7f35c5e0c700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35c00c8c40 con 0x7f35c006c1f0 2023-10-06T04:08:37.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.595+0000 7f35c5e0c700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c006e0c0 con 0x7f35c006cbb0 2023-10-06T04:08:37.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.595+0000 7f35beffd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f35c006c1f0 0x7f35c006da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:37.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.595+0000 7f35bffff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f35c006cf80 0x7f35c014fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49022/0 (socket says 172.21.15.157:49022) 2023-10-06T04:08:37.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.595+0000 7f35bffff700 1 -- 172.21.15.157:0/3472504338 learned_addr learned my addr 172.21.15.157:0/3472504338 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:08:37.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.595+0000 7f35bf7fe700 1 --2- 172.21.15.157:0/3472504338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35c006cbb0 0x7f35c006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:37.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.595+0000 7f35beffd700 1 -- 172.21.15.157:0/3472504338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35c006cbb0 msgr2=0x7f35c006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:37.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.595+0000 7f35beffd700 1 --2- 172.21.15.157:0/3472504338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35c006cbb0 0x7f35c006d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:37.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.595+0000 7f35beffd700 1 -- 172.21.15.157:0/3472504338 >> v1:172.21.15.157:6789/0 conn(0x7f35c006cf80 legacy=0x7f35c014fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:37.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.595+0000 7f35beffd700 1 -- 172.21.15.157:0/3472504338 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c00c8e50 con 0x7f35c006c1f0 2023-10-06T04:08:37.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.595+0000 7f35be7fc700 1 -- 172.21.15.157:0/3472504338 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 767771272 0 0) 0x7f35c00c9780 con 0x7f35c006cf80 2023-10-06T04:08:37.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.596+0000 7f35beffd700 1 --2- 172.21.15.157:0/3472504338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35c006c1f0 0x7f35c006da70 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f35b4009910 tx=0x7f35b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d4f3c61c4e8c65 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:37.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.596+0000 7f35be7fc700 1 -- 172.21.15.157:0/3472504338 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b400b6d0 con 0x7f35c006c1f0 2023-10-06T04:08:37.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.596+0000 7f35be7fc700 1 -- 172.21.15.157:0/3472504338 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b400b830 con 0x7f35c006c1f0 2023-10-06T04:08:37.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.596+0000 7f35be7fc700 1 -- 172.21.15.157:0/3472504338 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b400bbf0 con 0x7f35c006c1f0 2023-10-06T04:08:37.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.596+0000 7f35c5e0c700 1 -- 172.21.15.157:0/3472504338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35c006c1f0 msgr2=0x7f35c006da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:37.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.596+0000 7f35c5e0c700 1 --2- 172.21.15.157:0/3472504338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35c006c1f0 0x7f35c006da70 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f35b4009910 tx=0x7f35b40091a0 comp rx=0 tx=0).stop 2023-10-06T04:08:37.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.596+0000 7f35c5e0c700 1 -- 172.21.15.157:0/3472504338 shutdown_connections 2023-10-06T04:08:37.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.596+0000 7f35c5e0c700 1 --2- 172.21.15.157:0/3472504338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35c006cbb0 0x7f35c006d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:37.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.596+0000 7f35c5e0c700 1 --2- 172.21.15.157:0/3472504338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35c006c1f0 0x7f35c006da70 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:37.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.596+0000 7f35c5e0c700 1 -- 172.21.15.157:0/3472504338 >> 172.21.15.157:0/3472504338 conn(0x7f35c006f230 msgr2=0x7f35c006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:37.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.596+0000 7f35c5e0c700 1 -- 172.21.15.157:0/3472504338 shutdown_connections 2023-10-06T04:08:37.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.596+0000 7f35c5e0c700 1 -- 172.21.15.157:0/3472504338 wait complete. 2023-10-06T04:08:37.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.597+0000 7f35c5e0c700 1 Processor -- start 2023-10-06T04:08:37.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.598+0000 7f35c5e0c700 1 -- start start 2023-10-06T04:08:37.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.598+0000 7f35c5e0c700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35c006c1f0 0x7f35c0169cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:37.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.598+0000 7f35c5e0c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35c006cbb0 0x7f35c00c55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:37.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.598+0000 7f35c5e0c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35c006cf80 0x7f35c00c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:37.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.598+0000 7f35c5e0c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c0153bb0 con 0x7f35c006cf80 2023-10-06T04:08:37.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.598+0000 7f35c5e0c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c0154170 con 0x7f35c006cbb0 2023-10-06T04:08:37.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.598+0000 7f35c5e0c700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c0154450 con 0x7f35c006c1f0 2023-10-06T04:08:37.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.598+0000 7f35bffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35c006cf80 0x7f35c00c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:37.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.598+0000 7f35bffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35c006cf80 0x7f35c00c5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40356/0 (socket says 172.21.15.157:40356) 2023-10-06T04:08:37.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.598+0000 7f35bffff700 1 -- 172.21.15.157:0/2347839560 learned_addr learned my addr 172.21.15.157:0/2347839560 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:08:37.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.598+0000 7f35beffd700 1 --2- 172.21.15.157:0/2347839560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35c006cbb0 0x7f35c00c55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:37.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.598+0000 7f35bf7fe700 1 --2- 172.21.15.157:0/2347839560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35c006c1f0 0x7f35c0169cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:37.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.599+0000 7f35bffff700 1 -- 172.21.15.157:0/2347839560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35c006c1f0 msgr2=0x7f35c0169cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:37.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.599+0000 7f35bffff700 1 --2- 172.21.15.157:0/2347839560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35c006c1f0 0x7f35c0169cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:37.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.599+0000 7f35bffff700 1 -- 172.21.15.157:0/2347839560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35c006cbb0 msgr2=0x7f35c00c55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:37.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.599+0000 7f35bffff700 1 --2- 172.21.15.157:0/2347839560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35c006cbb0 0x7f35c00c55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:37.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.599+0000 7f35bffff700 1 -- 172.21.15.157:0/2347839560 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b4009420 con 0x7f35c006cf80 2023-10-06T04:08:37.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.599+0000 7f35bffff700 1 --2- 172.21.15.157:0/2347839560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35c006cf80 0x7f35c00c5af0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f35b800c530 tx=0x7f35b8007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:37.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.599+0000 7f35a77fe700 1 -- 172.21.15.157:0/2347839560 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b8005b30 con 0x7f35c006cf80 2023-10-06T04:08:37.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.599+0000 7f35a77fe700 1 -- 172.21.15.157:0/2347839560 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b8005c90 con 0x7f35c006cf80 2023-10-06T04:08:37.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.599+0000 7f35a77fe700 1 -- 172.21.15.157:0/2347839560 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b8007320 con 0x7f35c006cf80 2023-10-06T04:08:37.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.599+0000 7f35c5e0c700 1 -- 172.21.15.157:0/2347839560 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c01572c0 con 0x7f35c006cf80 2023-10-06T04:08:37.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.599+0000 7f35c5e0c700 1 -- 172.21.15.157:0/2347839560 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c0153e00 con 0x7f35c006cf80 2023-10-06T04:08:37.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.600+0000 7f35c5e0c700 1 -- 172.21.15.157:0/2347839560 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35c016a9e0 con 0x7f35c006cf80 2023-10-06T04:08:37.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.601+0000 7f35a77fe700 1 -- 172.21.15.157:0/2347839560 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f35b801b3b0 con 0x7f35c006cf80 2023-10-06T04:08:37.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.601+0000 7f35a77fe700 1 --2- 172.21.15.157:0/2347839560 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f35ac066f60 0x7f35ac069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:37.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.601+0000 7f35a77fe700 1 -- 172.21.15.157:0/2347839560 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6632+0+0 (secure 0 0 0) 0x7f35b8083c50 con 0x7f35c006cf80 2023-10-06T04:08:37.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.601+0000 7f35bf7fe700 1 --2- 172.21.15.157:0/2347839560 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f35ac066f60 0x7f35ac069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:37.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.602+0000 7f35bf7fe700 1 --2- 172.21.15.157:0/2347839560 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f35ac066f60 0x7f35ac069410 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f35b000f570 tx=0x7f35b000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:37.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.605+0000 7f35a77fe700 1 -- 172.21.15.157:0/2347839560 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f35b8089020 con 0x7f35c006cf80 2023-10-06T04:08:37.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.836+0000 7f35c5e0c700 1 -- 172.21.15.157:0/2347839560 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35c00c9130 con 0x7f35c006cf80 2023-10-06T04:08:37.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.836+0000 7f35a77fe700 1 -- 172.21.15.157:0/2347839560 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+5386 (secure 0 0 0) 0x7f35b8017020 con 0x7f35c006cf80 2023-10-06T04:08:37.838 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:08:37.838 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":78,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:08:36.748721+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:replay","state_seq":1,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:08:37.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.839+0000 7f35c5e0c700 1 -- 172.21.15.157:0/2347839560 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f35ac066f60 msgr2=0x7f35ac069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:37.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.839+0000 7f35c5e0c700 1 --2- 172.21.15.157:0/2347839560 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f35ac066f60 0x7f35ac069410 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f35b000f570 tx=0x7f35b000b040 comp rx=0 tx=0).stop 2023-10-06T04:08:37.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.839+0000 7f35c5e0c700 1 -- 172.21.15.157:0/2347839560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35c006cf80 msgr2=0x7f35c00c5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:37.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.839+0000 7f35c5e0c700 1 --2- 172.21.15.157:0/2347839560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35c006cf80 0x7f35c00c5af0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f35b800c530 tx=0x7f35b8007f90 comp rx=0 tx=0).stop 2023-10-06T04:08:37.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.839+0000 7f35c5e0c700 1 -- 172.21.15.157:0/2347839560 shutdown_connections 2023-10-06T04:08:37.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.839+0000 7f35c5e0c700 1 --2- 172.21.15.157:0/2347839560 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f35ac066f60 0x7f35ac069410 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:37.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.839+0000 7f35c5e0c700 1 --2- 172.21.15.157:0/2347839560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35c006cf80 0x7f35c00c5af0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:37.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.839+0000 7f35c5e0c700 1 --2- 172.21.15.157:0/2347839560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35c006c1f0 0x7f35c0169cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:37.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.839+0000 7f35c5e0c700 1 --2- 172.21.15.157:0/2347839560 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35c006cbb0 0x7f35c00c55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:37.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.839+0000 7f35c5e0c700 1 -- 172.21.15.157:0/2347839560 >> 172.21.15.157:0/2347839560 conn(0x7f35c006f230 msgr2=0x7f35c0150e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:37.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.839+0000 7f35c5e0c700 1 -- 172.21.15.157:0/2347839560 shutdown_connections 2023-10-06T04:08:37.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:37.840+0000 7f35c5e0c700 1 -- 172.21.15.157:0/2347839560 wait complete. 2023-10-06T04:08:37.842 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 78 2023-10-06T04:08:37.854 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 78, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-06T03:57:10.718782+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 78, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_5799": { "addr": "172.21.15.157:6834/765157057", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6832", "nonce": 765157057, "type": "v2" }, { "addr": "172.21.15.157:6834", "nonce": 765157057, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1 ], "features": 4540138320759226367, "flags": 0, "gid": 5799, "incarnation": 64, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 17 }, "gid_5812": { "addr": "172.21.15.175:6837/1436765859", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6836", "nonce": 1436765859, "type": "v2" }, { "addr": "172.21.15.175:6837", "nonce": 1436765859, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5812, "incarnation": 78, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 52, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2023-10-06T04:08:36.748721+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2, 3, 4 ], "tableserver": 0, "up": { "mds_0": 5799, "mds_1": 5812 } } } ], "standbys": [ { "addr": "172.21.15.175:6835/1799968248", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6834", "nonce": 1799968248, "type": "v2" }, { "addr": "172.21.15.175:6835", "nonce": 1799968248, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 57, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5126, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.175:6839/745804271", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6838", "nonce": 745804271, "type": "v2" }, { "addr": "172.21.15.175:6839", "nonce": 745804271, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 42, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5164, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.157:6835/2461751318", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6833", "nonce": 2461751318, "type": "v2" }, { "addr": "172.21.15.157:6835", "nonce": 2461751318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 34, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5268, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-06T04:08:39.855 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:08:39.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.982+0000 7faff5f57700 1 Processor -- start 2023-10-06T04:08:39.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.982+0000 7faff5f57700 1 -- start start 2023-10-06T04:08:39.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.982+0000 7faff5f57700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faff00b1d40 0x7faff0159df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:39.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.982+0000 7faff5f57700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faff006c1f0 0x7faff014fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:39.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.983+0000 7faff5f57700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faff006ced0 con 0x7faff00b2110 2023-10-06T04:08:39.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.983+0000 7faff5f57700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faff00afb10 con 0x7faff006c1f0 2023-10-06T04:08:39.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.983+0000 7faff5f57700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff006cc10 con 0x7faff00b1d40 2023-10-06T04:08:39.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.983+0000 7fafeffff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faff006c1f0 0x7faff014fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:39.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.983+0000 7fafeffff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faff006c1f0 0x7faff014fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40366/0 (socket says 172.21.15.157:40366) 2023-10-06T04:08:39.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.983+0000 7fafeffff700 1 -- 172.21.15.157:0/14451448 learned_addr learned my addr 172.21.15.157:0/14451448 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:08:39.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.983+0000 7fafef7fe700 1 --2- 172.21.15.157:0/14451448 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faff00b1d40 0x7faff0159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:39.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.983+0000 7fafeffff700 1 -- 172.21.15.157:0/14451448 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faff00b1d40 msgr2=0x7faff0159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:39.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.983+0000 7fafeffff700 1 --2- 172.21.15.157:0/14451448 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faff00b1d40 0x7faff0159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:39.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.983+0000 7fafeffff700 1 -- 172.21.15.157:0/14451448 >> v1:172.21.15.175:6789/0 conn(0x7faff00b2110 legacy=0x7faff00af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:39.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.983+0000 7fafeffff700 1 -- 172.21.15.157:0/14451448 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff006e570 con 0x7faff006c1f0 2023-10-06T04:08:39.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.983+0000 7fafeffff700 1 --2- 172.21.15.157:0/14451448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff006c1f0 0x7faff014fbe0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fafe8009f20 tx=0x7fafe8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc34b0ffddd3472 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:39.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.983+0000 7fafee7fc700 1 -- 172.21.15.157:0/14451448 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe800e070 con 0x7faff006c1f0 2023-10-06T04:08:39.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.983+0000 7fafee7fc700 1 -- 172.21.15.157:0/14451448 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe800bad0 con 0x7faff006c1f0 2023-10-06T04:08:39.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.983+0000 7fafee7fc700 1 -- 172.21.15.157:0/14451448 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe80092e0 con 0x7faff006c1f0 2023-10-06T04:08:39.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.984+0000 7faff5f57700 1 -- 172.21.15.157:0/14451448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff006c1f0 msgr2=0x7faff014fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:39.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.984+0000 7faff5f57700 1 --2- 172.21.15.157:0/14451448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff006c1f0 0x7faff014fbe0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fafe8009f20 tx=0x7fafe8009840 comp rx=0 tx=0).stop 2023-10-06T04:08:39.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.984+0000 7faff5f57700 1 -- 172.21.15.157:0/14451448 shutdown_connections 2023-10-06T04:08:39.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.984+0000 7faff5f57700 1 --2- 172.21.15.157:0/14451448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff006c1f0 0x7faff014fbe0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:39.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.984+0000 7faff5f57700 1 --2- 172.21.15.157:0/14451448 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faff00b1d40 0x7faff0159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:39.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.984+0000 7faff5f57700 1 -- 172.21.15.157:0/14451448 >> 172.21.15.157:0/14451448 conn(0x7faff00cc6e0 msgr2=0x7faff00ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:39.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.984+0000 7faff5f57700 1 -- 172.21.15.157:0/14451448 shutdown_connections 2023-10-06T04:08:39.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.984+0000 7faff5f57700 1 -- 172.21.15.157:0/14451448 wait complete. 2023-10-06T04:08:39.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.985+0000 7faff5f57700 1 Processor -- start 2023-10-06T04:08:39.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.985+0000 7faff5f57700 1 -- start start 2023-10-06T04:08:39.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.986+0000 7faff5f57700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faff006c1f0 0x7faff0156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:39.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.986+0000 7faff5f57700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff00b1d40 0x7faff00c0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:39.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.986+0000 7faff5f57700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faff00b2110 0x7faff0148d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:39.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.986+0000 7faff5f57700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff0155280 con 0x7faff00b1d40 2023-10-06T04:08:39.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.986+0000 7faff5f57700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff0166380 con 0x7faff006c1f0 2023-10-06T04:08:39.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.986+0000 7faff5f57700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff0157020 con 0x7faff00b2110 2023-10-06T04:08:39.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.986+0000 7fafeeffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff00b1d40 0x7faff00c0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:39.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.986+0000 7fafeeffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff00b1d40 0x7faff00c0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40382/0 (socket says 172.21.15.157:40382) 2023-10-06T04:08:39.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.986+0000 7fafeeffd700 1 -- 172.21.15.157:0/2065946158 learned_addr learned my addr 172.21.15.157:0/2065946158 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:08:39.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.986+0000 7fafef7fe700 1 --2- 172.21.15.157:0/2065946158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faff006c1f0 0x7faff0156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:39.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.986+0000 7fafeffff700 1 --2- 172.21.15.157:0/2065946158 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faff00b2110 0x7faff0148d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:39.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.986+0000 7fafeeffd700 1 -- 172.21.15.157:0/2065946158 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faff00b2110 msgr2=0x7faff0148d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:39.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.986+0000 7fafeeffd700 1 --2- 172.21.15.157:0/2065946158 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faff00b2110 0x7faff0148d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:39.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.986+0000 7fafeeffd700 1 -- 172.21.15.157:0/2065946158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faff006c1f0 msgr2=0x7faff0156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:39.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.986+0000 7fafeeffd700 1 --2- 172.21.15.157:0/2065946158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faff006c1f0 0x7faff0156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:39.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.986+0000 7fafeeffd700 1 -- 172.21.15.157:0/2065946158 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe8009ae0 con 0x7faff00b1d40 2023-10-06T04:08:39.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.987+0000 7fafeeffd700 1 --2- 172.21.15.157:0/2065946158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff00b1d40 0x7faff00c0990 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fafe000b610 tx=0x7fafe0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:39.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.987+0000 7fafd77fe700 1 -- 172.21.15.157:0/2065946158 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe000f510 con 0x7faff00b1d40 2023-10-06T04:08:39.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.987+0000 7fafd77fe700 1 -- 172.21.15.157:0/2065946158 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe000f670 con 0x7faff00b1d40 2023-10-06T04:08:39.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.987+0000 7faff5f57700 1 -- 172.21.15.157:0/2065946158 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faff00c13d0 con 0x7faff00b1d40 2023-10-06T04:08:39.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.987+0000 7fafd77fe700 1 -- 172.21.15.157:0/2065946158 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe000fa30 con 0x7faff00b1d40 2023-10-06T04:08:39.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.987+0000 7faff5f57700 1 -- 172.21.15.157:0/2065946158 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff00bfeb0 con 0x7faff00b1d40 2023-10-06T04:08:39.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.988+0000 7faff5f57700 1 -- 172.21.15.157:0/2065946158 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faff01665d0 con 0x7faff00b1d40 2023-10-06T04:08:39.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.988+0000 7fafd77fe700 1 -- 172.21.15.157:0/2065946158 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fafe001a400 con 0x7faff00b1d40 2023-10-06T04:08:39.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.988+0000 7fafd77fe700 1 --2- 172.21.15.157:0/2065946158 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fafdc066f60 0x7fafdc069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:39.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.988+0000 7fafd77fe700 1 -- 172.21.15.157:0/2065946158 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6632+0+0 (secure 0 0 0) 0x7fafe00842c0 con 0x7faff00b1d40 2023-10-06T04:08:39.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.989+0000 7fafef7fe700 1 --2- 172.21.15.157:0/2065946158 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fafdc066f60 0x7fafdc069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:39.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.989+0000 7fafef7fe700 1 --2- 172.21.15.157:0/2065946158 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fafdc066f60 0x7fafdc069410 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fafe4006fd0 tx=0x7fafe4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:39.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:39.993+0000 7fafd77fe700 1 -- 172.21.15.157:0/2065946158 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faff01665d0 con 0x7faff00b1d40 2023-10-06T04:08:40.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:40.217+0000 7faff5f57700 1 -- 172.21.15.157:0/2065946158 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faff01665d0 con 0x7faff00b1d40 2023-10-06T04:08:40.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:40.218+0000 7fafd77fe700 1 -- 172.21.15.157:0/2065946158 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+5386 (secure 0 0 0) 0x7faff01665d0 con 0x7faff00b1d40 2023-10-06T04:08:40.220 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:08:40.220 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":78,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:08:36.748721+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:replay","state_seq":1,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:08:40.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:40.220+0000 7faff5f57700 1 -- 172.21.15.157:0/2065946158 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fafdc066f60 msgr2=0x7fafdc069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:40.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:40.220+0000 7faff5f57700 1 --2- 172.21.15.157:0/2065946158 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fafdc066f60 0x7fafdc069410 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fafe4006fd0 tx=0x7fafe4008040 comp rx=0 tx=0).stop 2023-10-06T04:08:40.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:40.220+0000 7faff5f57700 1 -- 172.21.15.157:0/2065946158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff00b1d40 msgr2=0x7faff00c0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:40.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:40.220+0000 7faff5f57700 1 --2- 172.21.15.157:0/2065946158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff00b1d40 0x7faff00c0990 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fafe000b610 tx=0x7fafe0009120 comp rx=0 tx=0).stop 2023-10-06T04:08:40.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:40.220+0000 7faff5f57700 1 -- 172.21.15.157:0/2065946158 shutdown_connections 2023-10-06T04:08:40.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:40.220+0000 7faff5f57700 1 --2- 172.21.15.157:0/2065946158 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fafdc066f60 0x7fafdc069410 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:40.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:40.220+0000 7faff5f57700 1 --2- 172.21.15.157:0/2065946158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faff006c1f0 0x7faff0156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:40.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:40.220+0000 7faff5f57700 1 --2- 172.21.15.157:0/2065946158 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faff00b2110 0x7faff0148d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:40.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:40.220+0000 7faff5f57700 1 --2- 172.21.15.157:0/2065946158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff00b1d40 0x7faff00c0990 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:40.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:40.220+0000 7faff5f57700 1 -- 172.21.15.157:0/2065946158 >> 172.21.15.157:0/2065946158 conn(0x7faff00cc6e0 msgr2=0x7faff01576b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:40.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:40.220+0000 7faff5f57700 1 -- 172.21.15.157:0/2065946158 shutdown_connections 2023-10-06T04:08:40.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:40.220+0000 7faff5f57700 1 -- 172.21.15.157:0/2065946158 wait complete. 2023-10-06T04:08:40.223 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 78 2023-10-06T04:08:40.234 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:08:42.235 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:08:42.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.358+0000 7f6ec6ec1700 1 Processor -- start 2023-10-06T04:08:42.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.358+0000 7f6ec6ec1700 1 -- start start 2023-10-06T04:08:42.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.359+0000 7f6ec6ec1700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6ec00b1e40 0x7f6ec0157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:42.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.359+0000 7f6ec6ec1700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ec006c2d0 0x7f6ec006dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:42.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.359+0000 7f6ec6ec1700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ec006d8d0 con 0x7f6ec00b04d0 2023-10-06T04:08:42.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.359+0000 7f6ec6ec1700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ec00af8f0 con 0x7f6ec00b1e40 2023-10-06T04:08:42.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.359+0000 7f6ec6ec1700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec015a690 con 0x7f6ec006c2d0 2023-10-06T04:08:42.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.359+0000 7f6ec5ebf700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6ec00b1e40 0x7f6ec0157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:42.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.359+0000 7f6ec5ebf700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6ec00b1e40 0x7f6ec0157750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40392/0 (socket says 172.21.15.157:40392) 2023-10-06T04:08:42.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.359+0000 7f6ec5ebf700 1 -- 172.21.15.157:0/2320276040 learned_addr learned my addr 172.21.15.157:0/2320276040 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:08:42.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.359+0000 7f6ec66c0700 1 --2- 172.21.15.157:0/2320276040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ec006c2d0 0x7f6ec006dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:42.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.360+0000 7f6ec5ebf700 1 -- 172.21.15.157:0/2320276040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ec006c2d0 msgr2=0x7f6ec006dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:42.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.360+0000 7f6ec5ebf700 1 --2- 172.21.15.157:0/2320276040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ec006c2d0 0x7f6ec006dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:42.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.360+0000 7f6ec5ebf700 1 -- 172.21.15.157:0/2320276040 >> v1:172.21.15.175:6789/0 conn(0x7f6ec00b04d0 legacy=0x7f6ec01563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:42.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.360+0000 7f6ec5ebf700 1 -- 172.21.15.157:0/2320276040 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec006ccc0 con 0x7f6ec00b1e40 2023-10-06T04:08:42.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.360+0000 7f6ec5ebf700 1 --2- 172.21.15.157:0/2320276040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ec00b1e40 0x7f6ec0157750 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc009f20 tx=0x7f6ebc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52039cd18ffa0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:42.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.360+0000 7f6ec4ebd700 1 -- 172.21.15.157:0/2320276040 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc00e070 con 0x7f6ec00b1e40 2023-10-06T04:08:42.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.360+0000 7f6ec4ebd700 1 -- 172.21.15.157:0/2320276040 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ebc00b9d0 con 0x7f6ec00b1e40 2023-10-06T04:08:42.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.360+0000 7f6ec4ebd700 1 -- 172.21.15.157:0/2320276040 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc0092e0 con 0x7f6ec00b1e40 2023-10-06T04:08:42.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.360+0000 7f6ec6ec1700 1 -- 172.21.15.157:0/2320276040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ec00b1e40 msgr2=0x7f6ec0157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:42.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.360+0000 7f6ec6ec1700 1 --2- 172.21.15.157:0/2320276040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ec00b1e40 0x7f6ec0157750 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc009f20 tx=0x7f6ebc009840 comp rx=0 tx=0).stop 2023-10-06T04:08:42.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.361+0000 7f6ec6ec1700 1 -- 172.21.15.157:0/2320276040 shutdown_connections 2023-10-06T04:08:42.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.361+0000 7f6ec6ec1700 1 --2- 172.21.15.157:0/2320276040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ec006c2d0 0x7f6ec006dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:42.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.361+0000 7f6ec6ec1700 1 --2- 172.21.15.157:0/2320276040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ec00b1e40 0x7f6ec0157750 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:42.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.361+0000 7f6ec6ec1700 1 -- 172.21.15.157:0/2320276040 >> 172.21.15.157:0/2320276040 conn(0x7f6ec00cc540 msgr2=0x7f6ec00cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:42.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.361+0000 7f6ec6ec1700 1 -- 172.21.15.157:0/2320276040 shutdown_connections 2023-10-06T04:08:42.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.361+0000 7f6ec6ec1700 1 -- 172.21.15.157:0/2320276040 wait complete. 2023-10-06T04:08:42.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.362+0000 7f6ec6ec1700 1 Processor -- start 2023-10-06T04:08:42.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.362+0000 7f6ec6ec1700 1 -- start start 2023-10-06T04:08:42.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.362+0000 7f6ec6ec1700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ec006c2d0 0x7f6ec015aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:42.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.362+0000 7f6ec6ec1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ec00b04d0 0x7f6ec0169800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:42.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.362+0000 7f6ec6ec1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ec00b1e40 0x7f6ec0169d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:42.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.362+0000 7f6ec6ec1700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec015b5c0 con 0x7f6ec00b1e40 2023-10-06T04:08:42.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.362+0000 7f6ec6ec1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec01693b0 con 0x7f6ec00b04d0 2023-10-06T04:08:42.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.362+0000 7f6ec6ec1700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec01579e0 con 0x7f6ec006c2d0 2023-10-06T04:08:42.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.363+0000 7f6ec66c0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ec00b1e40 0x7f6ec0169d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:42.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.363+0000 7f6ec66c0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ec00b1e40 0x7f6ec0169d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40408/0 (socket says 172.21.15.157:40408) 2023-10-06T04:08:42.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.363+0000 7f6ec66c0700 1 -- 172.21.15.157:0/3581668626 learned_addr learned my addr 172.21.15.157:0/3581668626 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:08:42.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.363+0000 7f6ec5ebf700 1 --2- 172.21.15.157:0/3581668626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ec006c2d0 0x7f6ec015aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:42.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.363+0000 7f6ec56be700 1 --2- 172.21.15.157:0/3581668626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ec00b04d0 0x7f6ec0169800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:42.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.363+0000 7f6ec66c0700 1 -- 172.21.15.157:0/3581668626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ec006c2d0 msgr2=0x7f6ec015aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:42.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.363+0000 7f6ec66c0700 1 --2- 172.21.15.157:0/3581668626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ec006c2d0 0x7f6ec015aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:42.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.363+0000 7f6ec66c0700 1 -- 172.21.15.157:0/3581668626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ec00b04d0 msgr2=0x7f6ec0169800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:42.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.363+0000 7f6ec66c0700 1 --2- 172.21.15.157:0/3581668626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ec00b04d0 0x7f6ec0169800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:42.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.363+0000 7f6ec66c0700 1 -- 172.21.15.157:0/3581668626 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ebc009ae0 con 0x7f6ec00b1e40 2023-10-06T04:08:42.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.363+0000 7f6ec66c0700 1 --2- 172.21.15.157:0/3581668626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ec00b1e40 0x7f6ec0169d40 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6eb800e8d0 tx=0x7f6eb80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:42.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.363+0000 7f6eb5ffb700 1 -- 172.21.15.157:0/3581668626 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8013680 con 0x7f6ec00b1e40 2023-10-06T04:08:42.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.363+0000 7f6eb5ffb700 1 -- 172.21.15.157:0/3581668626 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb80137e0 con 0x7f6ec00b1e40 2023-10-06T04:08:42.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.363+0000 7f6ec6ec1700 1 -- 172.21.15.157:0/3581668626 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec0166b10 con 0x7f6ec00b1e40 2023-10-06T04:08:42.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.363+0000 7f6eb5ffb700 1 -- 172.21.15.157:0/3581668626 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8013ba0 con 0x7f6ec00b1e40 2023-10-06T04:08:42.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.363+0000 7f6ec6ec1700 1 -- 172.21.15.157:0/3581668626 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec0166830 con 0x7f6ec00b1e40 2023-10-06T04:08:42.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.364+0000 7f6ec6ec1700 1 -- 172.21.15.157:0/3581668626 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ec0166690 con 0x7f6ec00b1e40 2023-10-06T04:08:42.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.365+0000 7f6eb5ffb700 1 -- 172.21.15.157:0/3581668626 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6eb80123d0 con 0x7f6ec00b1e40 2023-10-06T04:08:42.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.365+0000 7f6eb5ffb700 1 --2- 172.21.15.157:0/3581668626 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6ea4066f30 0x7f6ea40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:42.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.365+0000 7f6eb5ffb700 1 -- 172.21.15.157:0/3581668626 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6632+0+0 (secure 0 0 0) 0x7f6eb8084a00 con 0x7f6ec00b1e40 2023-10-06T04:08:42.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.365+0000 7f6ec5ebf700 1 --2- 172.21.15.157:0/3581668626 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6ea4066f30 0x7f6ea40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:42.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.366+0000 7f6ec5ebf700 1 --2- 172.21.15.157:0/3581668626 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6ea4066f30 0x7f6ea40693e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc009f20 tx=0x7f6ebc018000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:42.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.369+0000 7f6eb5ffb700 1 -- 172.21.15.157:0/3581668626 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ec0166690 con 0x7f6ec00b1e40 2023-10-06T04:08:42.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.596+0000 7f6ec6ec1700 1 -- 172.21.15.157:0/3581668626 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ec01691c0 con 0x7f6ec00b1e40 2023-10-06T04:08:42.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.597+0000 7f6eb5ffb700 1 -- 172.21.15.157:0/3581668626 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+5386 (secure 0 0 0) 0x7f6eb80518d0 con 0x7f6ec00b1e40 2023-10-06T04:08:42.599 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:08:42.599 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":78,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:08:36.748721+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:replay","state_seq":1,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:08:42.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.599+0000 7f6ec6ec1700 1 -- 172.21.15.157:0/3581668626 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6ea4066f30 msgr2=0x7f6ea40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:42.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.599+0000 7f6ec6ec1700 1 --2- 172.21.15.157:0/3581668626 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6ea4066f30 0x7f6ea40693e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc009f20 tx=0x7f6ebc018000 comp rx=0 tx=0).stop 2023-10-06T04:08:42.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.599+0000 7f6ec6ec1700 1 -- 172.21.15.157:0/3581668626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ec00b1e40 msgr2=0x7f6ec0169d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:42.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.599+0000 7f6ec6ec1700 1 --2- 172.21.15.157:0/3581668626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ec00b1e40 0x7f6ec0169d40 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6eb800e8d0 tx=0x7f6eb80091a0 comp rx=0 tx=0).stop 2023-10-06T04:08:42.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.599+0000 7f6ec6ec1700 1 -- 172.21.15.157:0/3581668626 shutdown_connections 2023-10-06T04:08:42.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.599+0000 7f6ec6ec1700 1 --2- 172.21.15.157:0/3581668626 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6ea4066f30 0x7f6ea40693e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:42.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.599+0000 7f6ec6ec1700 1 --2- 172.21.15.157:0/3581668626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ec00b1e40 0x7f6ec0169d40 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:42.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.599+0000 7f6ec6ec1700 1 --2- 172.21.15.157:0/3581668626 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ec006c2d0 0x7f6ec015aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:42.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.599+0000 7f6ec6ec1700 1 --2- 172.21.15.157:0/3581668626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ec00b04d0 0x7f6ec0169800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:42.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.599+0000 7f6ec6ec1700 1 -- 172.21.15.157:0/3581668626 >> 172.21.15.157:0/3581668626 conn(0x7f6ec00cc540 msgr2=0x7f6ec00c4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:42.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.599+0000 7f6ec6ec1700 1 -- 172.21.15.157:0/3581668626 shutdown_connections 2023-10-06T04:08:42.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:42.599+0000 7f6ec6ec1700 1 -- 172.21.15.157:0/3581668626 wait complete. 2023-10-06T04:08:42.602 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 78 2023-10-06T04:08:42.613 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:08:44.614 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:08:44.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.742+0000 7f88e6b8f700 1 Processor -- start 2023-10-06T04:08:44.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.742+0000 7f88e6b8f700 1 -- start start 2023-10-06T04:08:44.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.742+0000 7f88e6b8f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f88e006cbb0 0x7f88e006d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:44.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.743+0000 7f88e6b8f700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f88e0150410 0x7f88e006da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:44.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.743+0000 7f88e6b8f700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88e00c9780 con 0x7f88e006c1f0 2023-10-06T04:08:44.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.743+0000 7f88e6b8f700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88e00c8c40 con 0x7f88e0150410 2023-10-06T04:08:44.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.743+0000 7f88e6b8f700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88e006e0c0 con 0x7f88e006cbb0 2023-10-06T04:08:44.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.743+0000 7f88e512c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f88e0150410 0x7f88e006da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:44.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.743+0000 7f88e512c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f88e0150410 0x7f88e006da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53714/0 (socket says 172.21.15.157:53714) 2023-10-06T04:08:44.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.743+0000 7f88e512c700 1 -- 172.21.15.157:0/428892715 learned_addr learned my addr 172.21.15.157:0/428892715 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:08:44.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.743+0000 7f88e492b700 1 --2- 172.21.15.157:0/428892715 >> v2:172.21.15.175:3300/0 conn(0x7f88e006cbb0 0x7f88e006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:44.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.743+0000 7f88e512c700 1 -- 172.21.15.157:0/428892715 >> v2:172.21.15.175:3300/0 conn(0x7f88e006cbb0 msgr2=0x7f88e006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:44.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.743+0000 7f88e512c700 1 --2- 172.21.15.157:0/428892715 >> v2:172.21.15.175:3300/0 conn(0x7f88e006cbb0 0x7f88e006d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:44.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.743+0000 7f88e512c700 1 -- 172.21.15.157:0/428892715 >> v1:172.21.15.175:6789/0 conn(0x7f88e006c1f0 legacy=0x7f88e014fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:44.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.743+0000 7f88e512c700 1 -- 172.21.15.157:0/428892715 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e00c8e50 con 0x7f88e0150410 2023-10-06T04:08:44.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.743+0000 7f88e512c700 1 --2- 172.21.15.157:0/428892715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88e0150410 0x7f88e006da70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f88e006fd70 tx=0x7f88d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce2da80cf658859 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:44.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.743+0000 7f88d77fe700 1 -- 172.21.15.157:0/428892715 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d8009e70 con 0x7f88e0150410 2023-10-06T04:08:44.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.743+0000 7f88d77fe700 1 -- 172.21.15.157:0/428892715 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d800b8f0 con 0x7f88e0150410 2023-10-06T04:08:44.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.743+0000 7f88d77fe700 1 -- 172.21.15.157:0/428892715 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d800bc90 con 0x7f88e0150410 2023-10-06T04:08:44.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.744+0000 7f88e6b8f700 1 -- 172.21.15.157:0/428892715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88e0150410 msgr2=0x7f88e006da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:44.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.744+0000 7f88e6b8f700 1 --2- 172.21.15.157:0/428892715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88e0150410 0x7f88e006da70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f88e006fd70 tx=0x7f88d8009250 comp rx=0 tx=0).stop 2023-10-06T04:08:44.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.744+0000 7f88e6b8f700 1 -- 172.21.15.157:0/428892715 shutdown_connections 2023-10-06T04:08:44.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.744+0000 7f88e6b8f700 1 --2- 172.21.15.157:0/428892715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88e0150410 0x7f88e006da70 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:44.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.744+0000 7f88e6b8f700 1 --2- 172.21.15.157:0/428892715 >> v2:172.21.15.175:3300/0 conn(0x7f88e006cbb0 0x7f88e006d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:44.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.744+0000 7f88e6b8f700 1 -- 172.21.15.157:0/428892715 >> 172.21.15.157:0/428892715 conn(0x7f88e006f230 msgr2=0x7f88e006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:44.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.744+0000 7f88e6b8f700 1 -- 172.21.15.157:0/428892715 shutdown_connections 2023-10-06T04:08:44.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.744+0000 7f88e6b8f700 1 -- 172.21.15.157:0/428892715 wait complete. 2023-10-06T04:08:44.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.745+0000 7f88e6b8f700 1 Processor -- start 2023-10-06T04:08:44.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.745+0000 7f88e6b8f700 1 -- start start 2023-10-06T04:08:44.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.746+0000 7f88e6b8f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88e006c1f0 0x7f88e01478c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:44.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.746+0000 7f88e6b8f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88e006cbb0 0x7f88e0147e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:44.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.746+0000 7f88e6b8f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88e0150410 0x7f88e00c5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:44.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.746+0000 7f88e6b8f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e0166230 con 0x7f88e006c1f0 2023-10-06T04:08:44.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.746+0000 7f88e6b8f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e0147430 con 0x7f88e006cbb0 2023-10-06T04:08:44.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.746+0000 7f88e6b8f700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e0166420 con 0x7f88e0150410 2023-10-06T04:08:44.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.746+0000 7f88e492b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88e006c1f0 0x7f88e01478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:44.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.746+0000 7f88e492b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88e006c1f0 0x7f88e01478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53726/0 (socket says 172.21.15.157:53726) 2023-10-06T04:08:44.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.746+0000 7f88e492b700 1 -- 172.21.15.157:0/2752662533 learned_addr learned my addr 172.21.15.157:0/2752662533 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:08:44.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.746+0000 7f88e512c700 1 --2- 172.21.15.157:0/2752662533 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88e0150410 0x7f88e00c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:44.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.746+0000 7f88d7fff700 1 --2- 172.21.15.157:0/2752662533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88e006cbb0 0x7f88e0147e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:44.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.747+0000 7f88e492b700 1 -- 172.21.15.157:0/2752662533 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88e0150410 msgr2=0x7f88e00c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:44.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.747+0000 7f88e492b700 1 --2- 172.21.15.157:0/2752662533 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88e0150410 0x7f88e00c5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:44.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.747+0000 7f88e492b700 1 -- 172.21.15.157:0/2752662533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88e006cbb0 msgr2=0x7f88e0147e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:44.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.747+0000 7f88e492b700 1 --2- 172.21.15.157:0/2752662533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88e006cbb0 0x7f88e0147e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:44.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.747+0000 7f88e492b700 1 -- 172.21.15.157:0/2752662533 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d80094f0 con 0x7f88e006c1f0 2023-10-06T04:08:44.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.747+0000 7f88e492b700 1 --2- 172.21.15.157:0/2752662533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88e006c1f0 0x7f88e01478c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009ef0 tx=0x7f88dc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:44.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.747+0000 7f88d4ff9700 1 -- 172.21.15.157:0/2752662533 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88dc00cb70 con 0x7f88e006c1f0 2023-10-06T04:08:44.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.747+0000 7f88d4ff9700 1 -- 172.21.15.157:0/2752662533 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88dc00f040 con 0x7f88e006c1f0 2023-10-06T04:08:44.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.747+0000 7f88e6b8f700 1 -- 172.21.15.157:0/2752662533 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88e0166700 con 0x7f88e006c1f0 2023-10-06T04:08:44.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.747+0000 7f88d4ff9700 1 -- 172.21.15.157:0/2752662533 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88dc014480 con 0x7f88e006c1f0 2023-10-06T04:08:44.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.747+0000 7f88e6b8f700 1 -- 172.21.15.157:0/2752662533 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e0166cc0 con 0x7f88e006c1f0 2023-10-06T04:08:44.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.748+0000 7f88e6b8f700 1 -- 172.21.15.157:0/2752662533 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88e0089ca0 con 0x7f88e006c1f0 2023-10-06T04:08:44.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.748+0000 7f88d4ff9700 1 -- 172.21.15.157:0/2752662533 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f88dc0147f0 con 0x7f88e006c1f0 2023-10-06T04:08:44.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.749+0000 7f88d4ff9700 1 --2- 172.21.15.157:0/2752662533 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f88cc066f30 0x7f88cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:44.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.749+0000 7f88d4ff9700 1 -- 172.21.15.157:0/2752662533 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6632+0+0 (secure 0 0 0) 0x7f88dc085200 con 0x7f88e006c1f0 2023-10-06T04:08:44.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.749+0000 7f88d7fff700 1 --2- 172.21.15.157:0/2752662533 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f88cc066f30 0x7f88cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:44.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.749+0000 7f88d7fff700 1 --2- 172.21.15.157:0/2752662533 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f88cc066f30 0x7f88cc0693e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f88e0152be0 tx=0x7f88d0007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:44.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.753+0000 7f88d4ff9700 1 -- 172.21.15.157:0/2752662533 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f88dc0520d0 con 0x7f88e006c1f0 2023-10-06T04:08:44.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.981+0000 7f88e6b8f700 1 -- 172.21.15.157:0/2752662533 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88e00c9130 con 0x7f88e006c1f0 2023-10-06T04:08:44.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.982+0000 7f88d4ff9700 1 -- 172.21.15.157:0/2752662533 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+5388 (secure 0 0 0) 0x7f88dc0556f0 con 0x7f88e006c1f0 2023-10-06T04:08:44.984 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:08:44.984 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:08:42.843235+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:resolve","state_seq":11,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:08:44.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.984+0000 7f88e6b8f700 1 -- 172.21.15.157:0/2752662533 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f88cc066f30 msgr2=0x7f88cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:44.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.984+0000 7f88e6b8f700 1 --2- 172.21.15.157:0/2752662533 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f88cc066f30 0x7f88cc0693e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f88e0152be0 tx=0x7f88d0007ec0 comp rx=0 tx=0).stop 2023-10-06T04:08:44.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.984+0000 7f88e6b8f700 1 -- 172.21.15.157:0/2752662533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88e006c1f0 msgr2=0x7f88e01478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:44.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.984+0000 7f88e6b8f700 1 --2- 172.21.15.157:0/2752662533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88e006c1f0 0x7f88e01478c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009ef0 tx=0x7f88dc00b040 comp rx=0 tx=0).stop 2023-10-06T04:08:44.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.984+0000 7f88e6b8f700 1 -- 172.21.15.157:0/2752662533 shutdown_connections 2023-10-06T04:08:44.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.984+0000 7f88e6b8f700 1 --2- 172.21.15.157:0/2752662533 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f88cc066f30 0x7f88cc0693e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:44.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.984+0000 7f88e6b8f700 1 --2- 172.21.15.157:0/2752662533 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88e0150410 0x7f88e00c5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:44.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.984+0000 7f88e6b8f700 1 --2- 172.21.15.157:0/2752662533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88e006c1f0 0x7f88e01478c0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:44.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.984+0000 7f88e6b8f700 1 --2- 172.21.15.157:0/2752662533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88e006cbb0 0x7f88e0147e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:44.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.984+0000 7f88e6b8f700 1 -- 172.21.15.157:0/2752662533 >> 172.21.15.157:0/2752662533 conn(0x7f88e006f230 msgr2=0x7f88e014d780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:44.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.984+0000 7f88e6b8f700 1 -- 172.21.15.157:0/2752662533 shutdown_connections 2023-10-06T04:08:44.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:44.984+0000 7f88e6b8f700 1 -- 172.21.15.157:0/2752662533 wait complete. 2023-10-06T04:08:44.987 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 79 2023-10-06T04:08:44.998 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:08:45.986 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:08:45.988 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:08:46.999 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:08:47.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.123+0000 7f6fa1fa8700 1 Processor -- start 2023-10-06T04:08:47.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.123+0000 7f6fa1fa8700 1 -- start start 2023-10-06T04:08:47.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.124+0000 7f6fa1fa8700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6f9c0c3d50 0x7f6f9c161920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:47.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.124+0000 7f6fa1fa8700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f9c1596a0 con 0x7f6f9c0c9dc0 2023-10-06T04:08:47.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.124+0000 7f6fa1fa8700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f9c155f70 con 0x7f6f9c13f0e0 2023-10-06T04:08:47.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.124+0000 7f6fa1fa8700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f9c0c6980 con 0x7f6f9c0c3d50 2023-10-06T04:08:47.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.124+0000 7f6f9affd700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f6f9c0c9dc0 0x7f6f9c159b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:46646/0 (socket says 172.21.15.157:46646) 2023-10-06T04:08:47.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.124+0000 7f6f9affd700 1 -- 172.21.15.157:0/2264826221 learned_addr learned my addr 172.21.15.157:0/2264826221 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:08:47.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.124+0000 7f6f9bfff700 1 --2- 172.21.15.157:0/2264826221 >> v2:172.21.15.175:3300/0 conn(0x7f6f9c0c3d50 0x7f6f9c161920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:47.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.124+0000 7f6f9a7fc700 1 -- 172.21.15.157:0/2264826221 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3369217247 0 0) 0x7f6f9c1596a0 con 0x7f6f9c0c9dc0 2023-10-06T04:08:47.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.124+0000 7f6f9a7fc700 1 -- 172.21.15.157:0/2264826221 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f84003570 con 0x7f6f9c0c9dc0 2023-10-06T04:08:47.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.124+0000 7f6f9a7fc700 1 -- 172.21.15.157:0/2264826221 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1227765438 0 0) 0x7f6f9c155f70 con 0x7f6f9c13f0e0 2023-10-06T04:08:47.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.124+0000 7f6f9a7fc700 1 -- 172.21.15.157:0/2264826221 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f9c1596a0 con 0x7f6f9c13f0e0 2023-10-06T04:08:47.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.124+0000 7f6f9a7fc700 1 -- 172.21.15.157:0/2264826221 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1112655394 0 0) 0x7f6f84003570 con 0x7f6f9c0c9dc0 2023-10-06T04:08:47.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.124+0000 7f6f9a7fc700 1 -- 172.21.15.157:0/2264826221 >> v2:172.21.15.175:3300/0 conn(0x7f6f9c0c3d50 msgr2=0x7f6f9c161920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:47.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.124+0000 7f6f9a7fc700 1 --2- 172.21.15.157:0/2264826221 >> v2:172.21.15.175:3300/0 conn(0x7f6f9c0c3d50 0x7f6f9c161920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:47.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.124+0000 7f6f9a7fc700 1 -- 172.21.15.157:0/2264826221 >> v1:172.21.15.175:6789/0 conn(0x7f6f9c13f0e0 legacy=0x7f6f9c06a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:47.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.124+0000 7f6f9a7fc700 1 -- 172.21.15.157:0/2264826221 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c156230 con 0x7f6f9c0c9dc0 2023-10-06T04:08:47.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.124+0000 7f6f9a7fc700 1 -- 172.21.15.157:0/2264826221 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f6f90001470 con 0x7f6f9c0c9dc0 2023-10-06T04:08:47.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.124+0000 7f6f9a7fc700 1 -- 172.21.15.157:0/2264826221 >> v1:172.21.15.157:6789/0 conn(0x7f6f9c0c9dc0 legacy=0x7f6f9c159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:47.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.124+0000 7f6f9a7fc700 1 --2- 172.21.15.157:0/2264826221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f84003bf0 0x7f6f84003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:47.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.124+0000 7f6f9a7fc700 1 --2- 172.21.15.157:0/2264826221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f84004810 0x7f6f84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:47.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.124+0000 7f6f9a7fc700 1 --2- 172.21.15.157:0/2264826221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f840051c0 0x7f6f8400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:47.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.125+0000 7f6f9a7fc700 1 -- 172.21.15.157:0/2264826221 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c0c6980 con 0x7f6f84003bf0 2023-10-06T04:08:47.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.125+0000 7f6f9a7fc700 1 -- 172.21.15.157:0/2264826221 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8400c0b0 con 0x7f6f840051c0 2023-10-06T04:08:47.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.125+0000 7f6f9a7fc700 1 -- 172.21.15.157:0/2264826221 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8400c270 con 0x7f6f84004810 2023-10-06T04:08:47.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.125+0000 7f6f9b7fe700 1 --2- 172.21.15.157:0/2264826221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f84003bf0 0x7f6f84003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:47.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.125+0000 7f6f9affd700 1 --2- 172.21.15.157:0/2264826221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f84004810 0x7f6f84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:47.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.125+0000 7f6f9bfff700 1 --2- 172.21.15.157:0/2264826221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f840051c0 0x7f6f8400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:47.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.125+0000 7f6f9b7fe700 1 -- 172.21.15.157:0/2264826221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f84004810 msgr2=0x7f6f84004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:47.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.125+0000 7f6f9b7fe700 1 --2- 172.21.15.157:0/2264826221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f84004810 0x7f6f84004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:47.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.125+0000 7f6f9b7fe700 1 -- 172.21.15.157:0/2264826221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f840051c0 msgr2=0x7f6f8400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:47.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.125+0000 7f6f9b7fe700 1 --2- 172.21.15.157:0/2264826221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f840051c0 0x7f6f8400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:47.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.125+0000 7f6f9bfff700 1 -- 172.21.15.157:0/2264826221 reap_dead start 2023-10-06T04:08:47.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.125+0000 7f6f9b7fe700 1 -- 172.21.15.157:0/2264826221 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f84003570 con 0x7f6f84003bf0 2023-10-06T04:08:47.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.125+0000 7f6f9b7fe700 1 --2- 172.21.15.157:0/2264826221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f84003bf0 0x7f6f84003fe0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c001a10 tx=0x7f6f8c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5973b20f0f9087b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:47.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.125+0000 7f6f9a7fc700 1 -- 172.21.15.157:0/2264826221 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c00c590 con 0x7f6f84003bf0 2023-10-06T04:08:47.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.125+0000 7f6f9a7fc700 1 -- 172.21.15.157:0/2264826221 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8c00c6f0 con 0x7f6f84003bf0 2023-10-06T04:08:47.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.125+0000 7f6f9a7fc700 1 -- 172.21.15.157:0/2264826221 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c00ca90 con 0x7f6f84003bf0 2023-10-06T04:08:47.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.126+0000 7f6fa1fa8700 1 -- 172.21.15.157:0/2264826221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f84003bf0 msgr2=0x7f6f84003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:47.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.126+0000 7f6fa1fa8700 1 --2- 172.21.15.157:0/2264826221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f84003bf0 0x7f6f84003fe0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c001a10 tx=0x7f6f8c007d30 comp rx=0 tx=0).stop 2023-10-06T04:08:47.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.126+0000 7f6fa1fa8700 1 -- 172.21.15.157:0/2264826221 shutdown_connections 2023-10-06T04:08:47.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.126+0000 7f6fa1fa8700 1 --2- 172.21.15.157:0/2264826221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f84003bf0 0x7f6f84003fe0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:47.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.126+0000 7f6fa1fa8700 1 -- 172.21.15.157:0/2264826221 >> 172.21.15.157:0/2264826221 conn(0x7f6f9c06b6a0 msgr2=0x7f6f9c0631f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:47.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.126+0000 7f6fa1fa8700 1 -- 172.21.15.157:0/2264826221 shutdown_connections 2023-10-06T04:08:47.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.126+0000 7f6fa1fa8700 1 -- 172.21.15.157:0/2264826221 wait complete. 2023-10-06T04:08:47.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.127+0000 7f6fa1fa8700 1 Processor -- start 2023-10-06T04:08:47.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.127+0000 7f6fa1fa8700 1 -- start start 2023-10-06T04:08:47.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.128+0000 7f6fa1fa8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f84003bf0 0x7f6f9c0c1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:47.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.128+0000 7f6fa1fa8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f9c14d300 0x7f6f9c169d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:47.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.128+0000 7f6fa1fa8700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f9c0c06c0 0x7f6f9c13df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:47.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.128+0000 7f6fa1fa8700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c153920 con 0x7f6f84003bf0 2023-10-06T04:08:47.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.128+0000 7f6fa1fa8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c16a260 con 0x7f6f9c14d300 2023-10-06T04:08:47.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.128+0000 7f6fa1fa8700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c169890 con 0x7f6f9c0c06c0 2023-10-06T04:08:47.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.128+0000 7f6f9b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f84003bf0 0x7f6f9c0c1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:47.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.128+0000 7f6f9b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f84003bf0 0x7f6f9c0c1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53736/0 (socket says 172.21.15.157:53736) 2023-10-06T04:08:47.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.128+0000 7f6f9b7fe700 1 -- 172.21.15.157:0/4130156270 learned_addr learned my addr 172.21.15.157:0/4130156270 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:08:47.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.128+0000 7f6f9bfff700 1 --2- 172.21.15.157:0/4130156270 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f9c0c06c0 0x7f6f9c13df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:47.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.128+0000 7f6f9affd700 1 --2- 172.21.15.157:0/4130156270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f9c14d300 0x7f6f9c169d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:47.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.128+0000 7f6f9b7fe700 1 -- 172.21.15.157:0/4130156270 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f9c0c06c0 msgr2=0x7f6f9c13df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:47.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.128+0000 7f6f9b7fe700 1 --2- 172.21.15.157:0/4130156270 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f9c0c06c0 0x7f6f9c13df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:47.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.128+0000 7f6f9b7fe700 1 -- 172.21.15.157:0/4130156270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f9c14d300 msgr2=0x7f6f9c169d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:47.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.128+0000 7f6f9b7fe700 1 --2- 172.21.15.157:0/4130156270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f9c14d300 0x7f6f9c169d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:47.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.128+0000 7f6f9b7fe700 1 -- 172.21.15.157:0/4130156270 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c153080 con 0x7f6f84003bf0 2023-10-06T04:08:47.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.128+0000 7f6f9b7fe700 1 --2- 172.21.15.157:0/4130156270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f84003bf0 0x7f6f9c0c1760 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c00c3c0 tx=0x7f6f8c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:47.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.128+0000 7f6f837fe700 1 -- 172.21.15.157:0/4130156270 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c00c590 con 0x7f6f84003bf0 2023-10-06T04:08:47.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.128+0000 7f6f837fe700 1 -- 172.21.15.157:0/4130156270 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8c00c860 con 0x7f6f84003bf0 2023-10-06T04:08:47.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.128+0000 7f6fa1fa8700 1 -- 172.21.15.157:0/4130156270 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f9c153360 con 0x7f6f84003bf0 2023-10-06T04:08:47.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.128+0000 7f6f837fe700 1 -- 172.21.15.157:0/4130156270 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c007b10 con 0x7f6f84003bf0 2023-10-06T04:08:47.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.129+0000 7f6fa1fa8700 1 -- 172.21.15.157:0/4130156270 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9c169520 con 0x7f6f84003bf0 2023-10-06T04:08:47.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.130+0000 7f6fa1fa8700 1 -- 172.21.15.157:0/4130156270 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f9c169380 con 0x7f6f84003bf0 2023-10-06T04:08:47.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.130+0000 7f6f837fe700 1 -- 172.21.15.157:0/4130156270 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6f8c019070 con 0x7f6f84003bf0 2023-10-06T04:08:47.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.130+0000 7f6f837fe700 1 --2- 172.21.15.157:0/4130156270 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6f88066f30 0x7f6f880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:47.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.130+0000 7f6f837fe700 1 -- 172.21.15.157:0/4130156270 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6632+0+0 (secure 0 0 0) 0x7f6f8c086020 con 0x7f6f84003bf0 2023-10-06T04:08:47.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.130+0000 7f6f9affd700 1 --2- 172.21.15.157:0/4130156270 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6f88066f30 0x7f6f880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:47.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.131+0000 7f6f9affd700 1 --2- 172.21.15.157:0/4130156270 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6f88066f30 0x7f6f880693e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c156090 tx=0x7f6f90001df0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:47.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.134+0000 7f6f837fe700 1 -- 172.21.15.157:0/4130156270 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f9c169380 con 0x7f6f84003bf0 2023-10-06T04:08:47.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.364+0000 7f6fa1fa8700 1 -- 172.21.15.157:0/4130156270 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f9c169380 con 0x7f6f84003bf0 2023-10-06T04:08:47.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.365+0000 7f6f837fe700 1 -- 172.21.15.157:0/4130156270 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+5388 (secure 0 0 0) 0x7f6f9c169380 con 0x7f6f84003bf0 2023-10-06T04:08:47.367 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:08:47.368 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:08:42.843235+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:resolve","state_seq":11,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:08:47.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.367+0000 7f6fa1fa8700 1 -- 172.21.15.157:0/4130156270 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6f88066f30 msgr2=0x7f6f880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:47.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.367+0000 7f6fa1fa8700 1 --2- 172.21.15.157:0/4130156270 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6f88066f30 0x7f6f880693e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c156090 tx=0x7f6f90001df0 comp rx=0 tx=0).stop 2023-10-06T04:08:47.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.367+0000 7f6fa1fa8700 1 -- 172.21.15.157:0/4130156270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f84003bf0 msgr2=0x7f6f9c0c1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:47.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.367+0000 7f6fa1fa8700 1 --2- 172.21.15.157:0/4130156270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f84003bf0 0x7f6f9c0c1760 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c00c3c0 tx=0x7f6f8c015000 comp rx=0 tx=0).stop 2023-10-06T04:08:47.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.367+0000 7f6fa1fa8700 1 -- 172.21.15.157:0/4130156270 shutdown_connections 2023-10-06T04:08:47.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.367+0000 7f6fa1fa8700 1 --2- 172.21.15.157:0/4130156270 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6f88066f30 0x7f6f880693e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:47.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.367+0000 7f6fa1fa8700 1 --2- 172.21.15.157:0/4130156270 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f9c0c06c0 0x7f6f9c13df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:47.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.367+0000 7f6fa1fa8700 1 --2- 172.21.15.157:0/4130156270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f84003bf0 0x7f6f9c0c1760 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:47.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.367+0000 7f6fa1fa8700 1 --2- 172.21.15.157:0/4130156270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f9c14d300 0x7f6f9c169d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:47.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.367+0000 7f6fa1fa8700 1 -- 172.21.15.157:0/4130156270 >> 172.21.15.157:0/4130156270 conn(0x7f6f9c06b6a0 msgr2=0x7f6f9c0c89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:47.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.367+0000 7f6fa1fa8700 1 -- 172.21.15.157:0/4130156270 shutdown_connections 2023-10-06T04:08:47.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:47.367+0000 7f6fa1fa8700 1 -- 172.21.15.157:0/4130156270 wait complete. 2023-10-06T04:08:47.372 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 79 2023-10-06T04:08:47.381 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:08:49.382 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:08:49.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.506+0000 7f6b72c19700 1 Processor -- start 2023-10-06T04:08:49.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.506+0000 7f6b72c19700 1 -- start start 2023-10-06T04:08:49.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.507+0000 7f6b72c19700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6b6c06cbb0 0x7f6b6c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:49.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.507+0000 7f6b72c19700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b6c0c9780 con 0x7f6b6c06c1f0 2023-10-06T04:08:49.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.507+0000 7f6b72c19700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b6c06e0c0 con 0x7f6b6c150410 2023-10-06T04:08:49.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.507+0000 7f6b72c19700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b6c0c8c40 con 0x7f6b6c06cbb0 2023-10-06T04:08:49.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.507+0000 7f6b63fff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f6b6c06c1f0 0x7f6b6c14fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:46650/0 (socket says 172.21.15.157:46650) 2023-10-06T04:08:49.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.507+0000 7f6b63fff700 1 -- 172.21.15.157:0/3007612280 learned_addr learned my addr 172.21.15.157:0/3007612280 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:08:49.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.507+0000 7f6b709b5700 1 --2- 172.21.15.157:0/3007612280 >> v2:172.21.15.175:3300/0 conn(0x7f6b6c06cbb0 0x7f6b6c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:49.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.507+0000 7f6b637fe700 1 -- 172.21.15.157:0/3007612280 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 456894671 0 0) 0x7f6b6c0c9780 con 0x7f6b6c06c1f0 2023-10-06T04:08:49.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.508+0000 7f6b637fe700 1 -- 172.21.15.157:0/3007612280 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b54003570 con 0x7f6b6c06c1f0 2023-10-06T04:08:49.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.508+0000 7f6b637fe700 1 -- 172.21.15.157:0/3007612280 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2653910682 0 0) 0x7f6b6c06e0c0 con 0x7f6b6c150410 2023-10-06T04:08:49.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.508+0000 7f6b637fe700 1 -- 172.21.15.157:0/3007612280 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b6c0c9780 con 0x7f6b6c150410 2023-10-06T04:08:49.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.508+0000 7f6b637fe700 1 -- 172.21.15.157:0/3007612280 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1044528700 0 0) 0x7f6b54003570 con 0x7f6b6c06c1f0 2023-10-06T04:08:49.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.508+0000 7f6b709b5700 1 -- 172.21.15.157:0/3007612280 >> v1:172.21.15.175:6789/0 conn(0x7f6b6c150410 legacy=0x7f6b6c155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:49.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.508+0000 7f6b709b5700 1 -- 172.21.15.157:0/3007612280 >> v1:172.21.15.157:6789/0 conn(0x7f6b6c06c1f0 legacy=0x7f6b6c14fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:49.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.508+0000 7f6b709b5700 1 -- 172.21.15.157:0/3007612280 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c0c8e50 con 0x7f6b6c06cbb0 2023-10-06T04:08:49.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.508+0000 7f6b709b5700 1 --2- 172.21.15.157:0/3007612280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b6c06cbb0 0x7f6b6c06d530 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6b68009e60 tx=0x7f6b68009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40e1ddb2d5bdb63f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:49.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.508+0000 7f6b637fe700 1 -- 172.21.15.157:0/3007612280 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6800e070 con 0x7f6b6c06cbb0 2023-10-06T04:08:49.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.508+0000 7f6b637fe700 1 -- 172.21.15.157:0/3007612280 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6800b830 con 0x7f6b6c06cbb0 2023-10-06T04:08:49.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.508+0000 7f6b637fe700 1 -- 172.21.15.157:0/3007612280 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c001470 con 0x7f6b6c06cbb0 2023-10-06T04:08:49.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.509+0000 7f6b72c19700 1 -- 172.21.15.157:0/3007612280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b6c06cbb0 msgr2=0x7f6b6c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:49.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.509+0000 7f6b72c19700 1 --2- 172.21.15.157:0/3007612280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b6c06cbb0 0x7f6b6c06d530 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6b68009e60 tx=0x7f6b68009780 comp rx=0 tx=0).stop 2023-10-06T04:08:49.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.509+0000 7f6b72c19700 1 -- 172.21.15.157:0/3007612280 shutdown_connections 2023-10-06T04:08:49.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.509+0000 7f6b72c19700 1 --2- 172.21.15.157:0/3007612280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b6c06cbb0 0x7f6b6c06d530 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:49.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.509+0000 7f6b72c19700 1 -- 172.21.15.157:0/3007612280 >> 172.21.15.157:0/3007612280 conn(0x7f6b6c06f230 msgr2=0x7f6b6c06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:49.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.509+0000 7f6b72c19700 1 -- 172.21.15.157:0/3007612280 shutdown_connections 2023-10-06T04:08:49.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.509+0000 7f6b72c19700 1 -- 172.21.15.157:0/3007612280 wait complete. 2023-10-06T04:08:49.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.510+0000 7f6b72c19700 1 Processor -- start 2023-10-06T04:08:49.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.510+0000 7f6b72c19700 1 -- start start 2023-10-06T04:08:49.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.510+0000 7f6b72c19700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b6c06c1f0 0x7f6b6c156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:49.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.511+0000 7f6b72c19700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b6c06cbb0 0x7f6b6c1545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:49.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.511+0000 7f6b72c19700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b6c150410 0x7f6b6c14a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:49.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.511+0000 7f6b72c19700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c154150 con 0x7f6b6c150410 2023-10-06T04:08:49.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.511+0000 7f6b72c19700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c0c3990 con 0x7f6b6c06c1f0 2023-10-06T04:08:49.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.511+0000 7f6b72c19700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c152dc0 con 0x7f6b6c06cbb0 2023-10-06T04:08:49.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.511+0000 7f6b711b6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b6c150410 0x7f6b6c14a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:49.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.511+0000 7f6b711b6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b6c150410 0x7f6b6c14a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53744/0 (socket says 172.21.15.157:53744) 2023-10-06T04:08:49.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.511+0000 7f6b711b6700 1 -- 172.21.15.157:0/1289182810 learned_addr learned my addr 172.21.15.157:0/1289182810 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:08:49.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.511+0000 7f6b63fff700 1 --2- 172.21.15.157:0/1289182810 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b6c06cbb0 0x7f6b6c1545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:49.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.511+0000 7f6b709b5700 1 --2- 172.21.15.157:0/1289182810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b6c06c1f0 0x7f6b6c156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:49.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.511+0000 7f6b711b6700 1 -- 172.21.15.157:0/1289182810 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b6c06cbb0 msgr2=0x7f6b6c1545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:49.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.511+0000 7f6b711b6700 1 --2- 172.21.15.157:0/1289182810 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b6c06cbb0 0x7f6b6c1545e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:49.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.511+0000 7f6b711b6700 1 -- 172.21.15.157:0/1289182810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b6c06c1f0 msgr2=0x7f6b6c156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:49.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.511+0000 7f6b711b6700 1 --2- 172.21.15.157:0/1289182810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b6c06c1f0 0x7f6b6c156590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:49.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.511+0000 7f6b711b6700 1 -- 172.21.15.157:0/1289182810 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b68009a20 con 0x7f6b6c150410 2023-10-06T04:08:49.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.511+0000 7f6b711b6700 1 --2- 172.21.15.157:0/1289182810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b6c150410 0x7f6b6c14a8e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6b6400c590 tx=0x7f6b64007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:49.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.511+0000 7f6b60ff9700 1 -- 172.21.15.157:0/1289182810 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b64016070 con 0x7f6b6c150410 2023-10-06T04:08:49.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.511+0000 7f6b60ff9700 1 -- 172.21.15.157:0/1289182810 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b64011590 con 0x7f6b6c150410 2023-10-06T04:08:49.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.511+0000 7f6b72c19700 1 -- 172.21.15.157:0/1289182810 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b6c1530a0 con 0x7f6b6c150410 2023-10-06T04:08:49.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.511+0000 7f6b60ff9700 1 -- 172.21.15.157:0/1289182810 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b64011970 con 0x7f6b6c150410 2023-10-06T04:08:49.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.512+0000 7f6b72c19700 1 -- 172.21.15.157:0/1289182810 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b6c153660 con 0x7f6b6c150410 2023-10-06T04:08:49.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.512+0000 7f6b72c19700 1 -- 172.21.15.157:0/1289182810 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b6c154300 con 0x7f6b6c150410 2023-10-06T04:08:49.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.513+0000 7f6b60ff9700 1 -- 172.21.15.157:0/1289182810 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6b64012420 con 0x7f6b6c150410 2023-10-06T04:08:49.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.513+0000 7f6b60ff9700 1 --2- 172.21.15.157:0/1289182810 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6b58066f30 0x7f6b580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:49.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.513+0000 7f6b60ff9700 1 -- 172.21.15.157:0/1289182810 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6632+0+0 (secure 0 0 0) 0x7f6b64084390 con 0x7f6b6c150410 2023-10-06T04:08:49.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.513+0000 7f6b709b5700 1 --2- 172.21.15.157:0/1289182810 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6b58066f30 0x7f6b580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:49.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.514+0000 7f6b709b5700 1 --2- 172.21.15.157:0/1289182810 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6b58066f30 0x7f6b580693e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6b680097c0 tx=0x7f6b68017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:49.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.517+0000 7f6b60ff9700 1 -- 172.21.15.157:0/1289182810 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6b6c154300 con 0x7f6b6c150410 2023-10-06T04:08:49.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.744+0000 7f6b72c19700 1 -- 172.21.15.157:0/1289182810 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b6c154290 con 0x7f6b6c150410 2023-10-06T04:08:49.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.744+0000 7f6b60ff9700 1 -- 172.21.15.157:0/1289182810 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+5390 (secure 0 0 0) 0x7f6b64051260 con 0x7f6b6c150410 2023-10-06T04:08:49.746 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:08:49.746 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:08:48.866849+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:reconnect","state_seq":13,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:08:49.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.746+0000 7f6b72c19700 1 -- 172.21.15.157:0/1289182810 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6b58066f30 msgr2=0x7f6b580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:49.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.746+0000 7f6b72c19700 1 --2- 172.21.15.157:0/1289182810 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6b58066f30 0x7f6b580693e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6b680097c0 tx=0x7f6b68017000 comp rx=0 tx=0).stop 2023-10-06T04:08:49.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.746+0000 7f6b72c19700 1 -- 172.21.15.157:0/1289182810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b6c150410 msgr2=0x7f6b6c14a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:49.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.746+0000 7f6b72c19700 1 --2- 172.21.15.157:0/1289182810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b6c150410 0x7f6b6c14a8e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6b6400c590 tx=0x7f6b64007d80 comp rx=0 tx=0).stop 2023-10-06T04:08:49.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.746+0000 7f6b72c19700 1 -- 172.21.15.157:0/1289182810 shutdown_connections 2023-10-06T04:08:49.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.746+0000 7f6b72c19700 1 --2- 172.21.15.157:0/1289182810 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6b58066f30 0x7f6b580693e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:49.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.746+0000 7f6b72c19700 1 --2- 172.21.15.157:0/1289182810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b6c06c1f0 0x7f6b6c156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:49.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.746+0000 7f6b72c19700 1 --2- 172.21.15.157:0/1289182810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b6c150410 0x7f6b6c14a8e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:49.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.746+0000 7f6b72c19700 1 --2- 172.21.15.157:0/1289182810 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b6c06cbb0 0x7f6b6c1545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:49.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.746+0000 7f6b72c19700 1 -- 172.21.15.157:0/1289182810 >> 172.21.15.157:0/1289182810 conn(0x7f6b6c06f230 msgr2=0x7f6b6c14e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:49.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.746+0000 7f6b72c19700 1 -- 172.21.15.157:0/1289182810 shutdown_connections 2023-10-06T04:08:49.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:49.746+0000 7f6b72c19700 1 -- 172.21.15.157:0/1289182810 wait complete. 2023-10-06T04:08:49.749 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 80 2023-10-06T04:08:49.760 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 80, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-06T03:57:10.718782+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 80, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_5799": { "addr": "172.21.15.157:6834/765157057", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6832", "nonce": 765157057, "type": "v2" }, { "addr": "172.21.15.157:6834", "nonce": 765157057, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1 ], "features": 4540138320759226367, "flags": 0, "gid": 5799, "incarnation": 64, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 17 }, "gid_5812": { "addr": "172.21.15.175:6837/1436765859", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6836", "nonce": 1436765859, "type": "v2" }, { "addr": "172.21.15.175:6837", "nonce": 1436765859, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5812, "incarnation": 78, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:reconnect", "state_seq": 13 } }, "last_failure": 0, "last_failure_osd_epoch": 52, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2023-10-06T04:08:48.866849+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2, 3, 4 ], "tableserver": 0, "up": { "mds_0": 5799, "mds_1": 5812 } } } ], "standbys": [ { "addr": "172.21.15.175:6835/1799968248", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6834", "nonce": 1799968248, "type": "v2" }, { "addr": "172.21.15.175:6835", "nonce": 1799968248, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 57, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5126, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.175:6839/745804271", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6838", "nonce": 745804271, "type": "v2" }, { "addr": "172.21.15.175:6839", "nonce": 745804271, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 42, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5164, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.157:6835/2461751318", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6833", "nonce": 2461751318, "type": "v2" }, { "addr": "172.21.15.157:6835", "nonce": 2461751318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 34, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5268, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-06T04:08:51.762 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:08:51.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.883+0000 7f19db72f700 1 Processor -- start 2023-10-06T04:08:51.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.884+0000 7f19db72f700 1 -- start start 2023-10-06T04:08:51.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.884+0000 7f19db72f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f19d406dc40 0x7f19d414fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:51.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.884+0000 7f19db72f700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f19d406c1f0 0x7f19d4150240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:51.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.884+0000 7f19db72f700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19d40c9780 con 0x7f19d406cbb0 2023-10-06T04:08:51.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.884+0000 7f19db72f700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19d40c8c40 con 0x7f19d406c1f0 2023-10-06T04:08:51.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.884+0000 7f19db72f700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19d406e0c0 con 0x7f19d406dc40 2023-10-06T04:08:51.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.885+0000 7f19d94cb700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f19d406cbb0 0x7f19d406d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:46662/0 (socket says 172.21.15.157:46662) 2023-10-06T04:08:51.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.885+0000 7f19d9ccc700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f19d406c1f0 0x7f19d4150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:51.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.885+0000 7f19d94cb700 1 -- 172.21.15.157:0/3262628584 learned_addr learned my addr 172.21.15.157:0/3262628584 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:08:51.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.885+0000 7f19cbfff700 1 -- 172.21.15.157:0/3262628584 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 123471201 0 0) 0x7f19d40c9780 con 0x7f19d406cbb0 2023-10-06T04:08:51.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.885+0000 7f19cbfff700 1 -- 172.21.15.157:0/3262628584 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19c0003570 con 0x7f19d406cbb0 2023-10-06T04:08:51.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.885+0000 7f19d8cca700 1 --2- 172.21.15.157:0/3262628584 >> v2:172.21.15.175:3300/0 conn(0x7f19d406dc40 0x7f19d414fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:51.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.885+0000 7f19d9ccc700 1 -- 172.21.15.157:0/3262628584 >> v2:172.21.15.175:3300/0 conn(0x7f19d406dc40 msgr2=0x7f19d414fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:51.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.885+0000 7f19d9ccc700 1 --2- 172.21.15.157:0/3262628584 >> v2:172.21.15.175:3300/0 conn(0x7f19d406dc40 0x7f19d414fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:51.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.885+0000 7f19d9ccc700 1 -- 172.21.15.157:0/3262628584 >> v1:172.21.15.157:6789/0 conn(0x7f19d406cbb0 legacy=0x7f19d406d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:51.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.885+0000 7f19d9ccc700 1 -- 172.21.15.157:0/3262628584 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d40c8e50 con 0x7f19d406c1f0 2023-10-06T04:08:51.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.885+0000 7f19d8cca700 1 --2- 172.21.15.157:0/3262628584 >> v2:172.21.15.175:3300/0 conn(0x7f19d406dc40 0x7f19d414fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:08:51.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.885+0000 7f19d9ccc700 1 --2- 172.21.15.157:0/3262628584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19d406c1f0 0x7f19d4150240 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f19d4157070 tx=0x7f19cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eea30a84698ab92 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:51.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.885+0000 7f19cbfff700 1 -- 172.21.15.157:0/3262628584 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc00b690 con 0x7f19d406c1f0 2023-10-06T04:08:51.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.885+0000 7f19cbfff700 1 -- 172.21.15.157:0/3262628584 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19cc00b7f0 con 0x7f19d406c1f0 2023-10-06T04:08:51.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.885+0000 7f19cbfff700 1 -- 172.21.15.157:0/3262628584 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc00bb90 con 0x7f19d406c1f0 2023-10-06T04:08:51.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.886+0000 7f19db72f700 1 -- 172.21.15.157:0/3262628584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19d406c1f0 msgr2=0x7f19d4150240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:51.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.886+0000 7f19db72f700 1 --2- 172.21.15.157:0/3262628584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19d406c1f0 0x7f19d4150240 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f19d4157070 tx=0x7f19cc0091a0 comp rx=0 tx=0).stop 2023-10-06T04:08:51.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.886+0000 7f19db72f700 1 -- 172.21.15.157:0/3262628584 shutdown_connections 2023-10-06T04:08:51.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.886+0000 7f19db72f700 1 --2- 172.21.15.157:0/3262628584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19d406c1f0 0x7f19d4150240 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:51.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.886+0000 7f19db72f700 1 --2- 172.21.15.157:0/3262628584 >> v2:172.21.15.175:3300/0 conn(0x7f19d406dc40 0x7f19d414fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:51.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.886+0000 7f19db72f700 1 -- 172.21.15.157:0/3262628584 >> 172.21.15.157:0/3262628584 conn(0x7f19d406f230 msgr2=0x7f19d406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:51.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.886+0000 7f19db72f700 1 -- 172.21.15.157:0/3262628584 shutdown_connections 2023-10-06T04:08:51.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.886+0000 7f19db72f700 1 -- 172.21.15.157:0/3262628584 wait complete. 2023-10-06T04:08:51.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.887+0000 7f19db72f700 1 Processor -- start 2023-10-06T04:08:51.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.887+0000 7f19db72f700 1 -- start start 2023-10-06T04:08:51.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.888+0000 7f19db72f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19d406c1f0 0x7f19d41531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:51.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.888+0000 7f19db72f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19d406cbb0 0x7f19d41476f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:51.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.888+0000 7f19db72f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19d4169a00 0x7f19d4169e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:51.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.888+0000 7f19db72f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d4146ce0 con 0x7f19d406cbb0 2023-10-06T04:08:51.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.888+0000 7f19db72f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d4145dc0 con 0x7f19d406c1f0 2023-10-06T04:08:51.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.888+0000 7f19db72f700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d4147260 con 0x7f19d4169a00 2023-10-06T04:08:51.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.888+0000 7f19d8cca700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19d406cbb0 0x7f19d41476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:51.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.888+0000 7f19d8cca700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19d406cbb0 0x7f19d41476f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53748/0 (socket says 172.21.15.157:53748) 2023-10-06T04:08:51.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.888+0000 7f19d8cca700 1 -- 172.21.15.157:0/845300548 learned_addr learned my addr 172.21.15.157:0/845300548 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:08:51.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.888+0000 7f19d94cb700 1 --2- 172.21.15.157:0/845300548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19d406c1f0 0x7f19d41531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:51.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.888+0000 7f19d9ccc700 1 --2- 172.21.15.157:0/845300548 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19d4169a00 0x7f19d4169e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:51.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.888+0000 7f19d8cca700 1 -- 172.21.15.157:0/845300548 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19d4169a00 msgr2=0x7f19d4169e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:51.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.888+0000 7f19d8cca700 1 --2- 172.21.15.157:0/845300548 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19d4169a00 0x7f19d4169e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:51.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.888+0000 7f19d8cca700 1 -- 172.21.15.157:0/845300548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19d406c1f0 msgr2=0x7f19d41531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:51.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.888+0000 7f19d8cca700 1 --2- 172.21.15.157:0/845300548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19d406c1f0 0x7f19d41531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:51.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.888+0000 7f19d8cca700 1 -- 172.21.15.157:0/845300548 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19cc009420 con 0x7f19d406cbb0 2023-10-06T04:08:51.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.888+0000 7f19d8cca700 1 --2- 172.21.15.157:0/845300548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19d406cbb0 0x7f19d41476f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f19c400c850 tx=0x7f19c4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:51.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.888+0000 7f19c97fa700 1 -- 172.21.15.157:0/845300548 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c4011600 con 0x7f19d406cbb0 2023-10-06T04:08:51.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.888+0000 7f19c97fa700 1 -- 172.21.15.157:0/845300548 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c4011760 con 0x7f19d406cbb0 2023-10-06T04:08:51.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.888+0000 7f19db72f700 1 -- 172.21.15.157:0/845300548 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d41536f0 con 0x7f19d406cbb0 2023-10-06T04:08:51.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.888+0000 7f19c97fa700 1 -- 172.21.15.157:0/845300548 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c4011b00 con 0x7f19d406cbb0 2023-10-06T04:08:51.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.889+0000 7f19db72f700 1 -- 172.21.15.157:0/845300548 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d41462f0 con 0x7f19d406cbb0 2023-10-06T04:08:51.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.890+0000 7f19db72f700 1 -- 172.21.15.157:0/845300548 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d41473d0 con 0x7f19d406cbb0 2023-10-06T04:08:51.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.890+0000 7f19c97fa700 1 -- 172.21.15.157:0/845300548 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f19c4018350 con 0x7f19d406cbb0 2023-10-06T04:08:51.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.890+0000 7f19c97fa700 1 --2- 172.21.15.157:0/845300548 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f19b8066f60 0x7f19b8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:51.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.890+0000 7f19c97fa700 1 -- 172.21.15.157:0/845300548 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6632+0+0 (secure 0 0 0) 0x7f19c4084720 con 0x7f19d406cbb0 2023-10-06T04:08:51.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.890+0000 7f19d94cb700 1 --2- 172.21.15.157:0/845300548 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f19b8066f60 0x7f19b8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:51.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.891+0000 7f19d94cb700 1 --2- 172.21.15.157:0/845300548 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f19b8066f60 0x7f19b8069410 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f19d0005c50 tx=0x7f19d0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:51.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:51.895+0000 7f19c97fa700 1 -- 172.21.15.157:0/845300548 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f19c4055080 con 0x7f19d406cbb0 2023-10-06T04:08:52.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:52.125+0000 7f19db72f700 1 -- 172.21.15.157:0/845300548 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19d40c9130 con 0x7f19d406cbb0 2023-10-06T04:08:52.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:52.125+0000 7f19c97fa700 1 -- 172.21.15.157:0/845300548 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+5390 (secure 0 0 0) 0x7f19c4054c10 con 0x7f19d406cbb0 2023-10-06T04:08:52.127 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:08:52.127 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:08:48.866849+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:reconnect","state_seq":13,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:08:52.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:52.127+0000 7f19db72f700 1 -- 172.21.15.157:0/845300548 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f19b8066f60 msgr2=0x7f19b8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:52.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:52.127+0000 7f19db72f700 1 --2- 172.21.15.157:0/845300548 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f19b8066f60 0x7f19b8069410 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f19d0005c50 tx=0x7f19d0009040 comp rx=0 tx=0).stop 2023-10-06T04:08:52.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:52.127+0000 7f19db72f700 1 -- 172.21.15.157:0/845300548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19d406cbb0 msgr2=0x7f19d41476f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:52.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:52.127+0000 7f19db72f700 1 --2- 172.21.15.157:0/845300548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19d406cbb0 0x7f19d41476f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f19c400c850 tx=0x7f19c4008040 comp rx=0 tx=0).stop 2023-10-06T04:08:52.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:52.127+0000 7f19db72f700 1 -- 172.21.15.157:0/845300548 shutdown_connections 2023-10-06T04:08:52.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:52.127+0000 7f19db72f700 1 --2- 172.21.15.157:0/845300548 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f19b8066f60 0x7f19b8069410 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:52.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:52.127+0000 7f19db72f700 1 --2- 172.21.15.157:0/845300548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19d406c1f0 0x7f19d41531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:52.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:52.128+0000 7f19db72f700 1 --2- 172.21.15.157:0/845300548 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19d4169a00 0x7f19d4169e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:52.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:52.128+0000 7f19db72f700 1 --2- 172.21.15.157:0/845300548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19d406cbb0 0x7f19d41476f0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:52.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:52.128+0000 7f19db72f700 1 -- 172.21.15.157:0/845300548 >> 172.21.15.157:0/845300548 conn(0x7f19d406f230 msgr2=0x7f19d40c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:52.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:52.128+0000 7f19db72f700 1 -- 172.21.15.157:0/845300548 shutdown_connections 2023-10-06T04:08:52.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:52.128+0000 7f19db72f700 1 -- 172.21.15.157:0/845300548 wait complete. 2023-10-06T04:08:52.131 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 80 2023-10-06T04:08:52.142 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:08:54.143 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:08:54.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.269+0000 7f11e8fb3700 1 Processor -- start 2023-10-06T04:08:54.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.269+0000 7f11e8fb3700 1 -- start start 2023-10-06T04:08:54.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.269+0000 7f11e8fb3700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f11e406cbb0 0x7f11e406d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:54.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.270+0000 7f11e8fb3700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e40c9780 con 0x7f11e4150410 2023-10-06T04:08:54.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.270+0000 7f11e8fb3700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e406e0c0 con 0x7f11e406c1f0 2023-10-06T04:08:54.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.270+0000 7f11e8fb3700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11e40c8c40 con 0x7f11e406cbb0 2023-10-06T04:08:54.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.270+0000 7f11e2d9d700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f11e4150410 0x7f11e4155f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37666/0 (socket says 172.21.15.157:37666) 2023-10-06T04:08:54.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.270+0000 7f11e2d9d700 1 -- 172.21.15.157:0/4007838336 learned_addr learned my addr 172.21.15.157:0/4007838336 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:08:54.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.270+0000 7f11e259c700 1 --2- 172.21.15.157:0/4007838336 >> v2:172.21.15.175:3300/0 conn(0x7f11e406cbb0 0x7f11e406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:54.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.270+0000 7f11e159a700 1 -- 172.21.15.157:0/4007838336 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4037861182 0 0) 0x7f11e40c9780 con 0x7f11e4150410 2023-10-06T04:08:54.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.270+0000 7f11e159a700 1 -- 172.21.15.157:0/4007838336 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11cc003570 con 0x7f11e4150410 2023-10-06T04:08:54.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.270+0000 7f11e159a700 1 -- 172.21.15.157:0/4007838336 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 396459685 0 0) 0x7f11cc003570 con 0x7f11e4150410 2023-10-06T04:08:54.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.270+0000 7f11e159a700 1 -- 172.21.15.157:0/4007838336 >> v2:172.21.15.175:3300/0 conn(0x7f11e406cbb0 msgr2=0x7f11e406d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:54.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.270+0000 7f11e159a700 1 --2- 172.21.15.157:0/4007838336 >> v2:172.21.15.175:3300/0 conn(0x7f11e406cbb0 0x7f11e406d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:54.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.270+0000 7f11e159a700 1 -- 172.21.15.157:0/4007838336 >> v1:172.21.15.175:6789/0 conn(0x7f11e406c1f0 legacy=0x7f11e414fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:54.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.270+0000 7f11e159a700 1 -- 172.21.15.157:0/4007838336 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e40c8e50 con 0x7f11e4150410 2023-10-06T04:08:54.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.270+0000 7f11e159a700 1 -- 172.21.15.157:0/4007838336 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f11dc001470 con 0x7f11e4150410 2023-10-06T04:08:54.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.270+0000 7f11e159a700 1 -- 172.21.15.157:0/4007838336 >> v1:172.21.15.157:6789/0 conn(0x7f11e4150410 legacy=0x7f11e4155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:54.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.270+0000 7f11e159a700 1 --2- 172.21.15.157:0/4007838336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11cc003a60 0x7f11cc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:54.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.270+0000 7f11e159a700 1 --2- 172.21.15.157:0/4007838336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11cc004810 0x7f11cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:54.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.270+0000 7f11e159a700 1 --2- 172.21.15.157:0/4007838336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11cc0051c0 0x7f11cc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:54.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.270+0000 7f11e159a700 1 -- 172.21.15.157:0/4007838336 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e40c8c40 con 0x7f11cc003a60 2023-10-06T04:08:54.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.270+0000 7f11e159a700 1 -- 172.21.15.157:0/4007838336 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc00c0d0 con 0x7f11cc004810 2023-10-06T04:08:54.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.270+0000 7f11e159a700 1 -- 172.21.15.157:0/4007838336 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc00c290 con 0x7f11cc0051c0 2023-10-06T04:08:54.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.270+0000 7f11e259c700 1 --2- 172.21.15.157:0/4007838336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11cc003a60 0x7f11cc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:54.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.271+0000 7f11e1d9b700 1 --2- 172.21.15.157:0/4007838336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11cc004810 0x7f11cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:54.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.271+0000 7f11e2d9d700 1 --2- 172.21.15.157:0/4007838336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11cc0051c0 0x7f11cc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:54.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.271+0000 7f11e259c700 1 -- 172.21.15.157:0/4007838336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11cc0051c0 msgr2=0x7f11cc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:54.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.271+0000 7f11e259c700 1 --2- 172.21.15.157:0/4007838336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11cc0051c0 0x7f11cc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:54.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.271+0000 7f11e259c700 1 -- 172.21.15.157:0/4007838336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11cc004810 msgr2=0x7f11cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:54.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.271+0000 7f11e259c700 1 --2- 172.21.15.157:0/4007838336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11cc004810 0x7f11cc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:54.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.271+0000 7f11e259c700 1 -- 172.21.15.157:0/4007838336 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11cc003570 con 0x7f11cc003a60 2023-10-06T04:08:54.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.271+0000 7f11e1d9b700 1 --2- 172.21.15.157:0/4007838336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11cc004810 0x7f11cc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:08:54.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.271+0000 7f11e2d9d700 1 -- 172.21.15.157:0/4007838336 reap_dead start 2023-10-06T04:08:54.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.271+0000 7f11e259c700 1 --2- 172.21.15.157:0/4007838336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11cc003a60 0x7f11cc003e70 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f11d400e210 tx=0x7f11d400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b75244cc19397 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:54.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.271+0000 7f11e159a700 1 -- 172.21.15.157:0/4007838336 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d400eac0 con 0x7f11cc003a60 2023-10-06T04:08:54.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.271+0000 7f11e159a700 1 -- 172.21.15.157:0/4007838336 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d400f040 con 0x7f11cc003a60 2023-10-06T04:08:54.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.271+0000 7f11e159a700 1 -- 172.21.15.157:0/4007838336 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d400ee70 con 0x7f11cc003a60 2023-10-06T04:08:54.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.271+0000 7f11e8fb3700 1 -- 172.21.15.157:0/4007838336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11cc003a60 msgr2=0x7f11cc003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:54.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.272+0000 7f11e8fb3700 1 --2- 172.21.15.157:0/4007838336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11cc003a60 0x7f11cc003e70 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f11d400e210 tx=0x7f11d400b040 comp rx=0 tx=0).stop 2023-10-06T04:08:54.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.272+0000 7f11e8fb3700 1 -- 172.21.15.157:0/4007838336 shutdown_connections 2023-10-06T04:08:54.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.272+0000 7f11e8fb3700 1 --2- 172.21.15.157:0/4007838336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11cc003a60 0x7f11cc003e70 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:54.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.272+0000 7f11e8fb3700 1 -- 172.21.15.157:0/4007838336 >> 172.21.15.157:0/4007838336 conn(0x7f11e406f230 msgr2=0x7f11e406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:54.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.272+0000 7f11e8fb3700 1 -- 172.21.15.157:0/4007838336 shutdown_connections 2023-10-06T04:08:54.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.272+0000 7f11e8fb3700 1 -- 172.21.15.157:0/4007838336 wait complete. 2023-10-06T04:08:54.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.273+0000 7f11e8fb3700 1 Processor -- start 2023-10-06T04:08:54.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.273+0000 7f11e8fb3700 1 -- start start 2023-10-06T04:08:54.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.273+0000 7f11e8fb3700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11cc003a60 0x7f11e41522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:54.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.273+0000 7f11e8fb3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11e41527e0 0x7f11e4169770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:54.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.273+0000 7f11e8fb3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11e40c1720 0x7f11e4169cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:54.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.273+0000 7f11e8fb3700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e4152e30 con 0x7f11e40c1720 2023-10-06T04:08:54.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.273+0000 7f11e8fb3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e4151e80 con 0x7f11e41527e0 2023-10-06T04:08:54.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.273+0000 7f11e8fb3700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e41517a0 con 0x7f11cc003a60 2023-10-06T04:08:54.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.274+0000 7f11e2d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11e40c1720 0x7f11e4169cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:54.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.274+0000 7f11e2d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11e40c1720 0x7f11e4169cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:48754/0 (socket says 172.21.15.157:48754) 2023-10-06T04:08:54.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.274+0000 7f11e2d9d700 1 -- 172.21.15.157:0/1872817025 learned_addr learned my addr 172.21.15.157:0/1872817025 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:08:54.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.274+0000 7f11e259c700 1 --2- 172.21.15.157:0/1872817025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11cc003a60 0x7f11e41522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:54.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.274+0000 7f11e1d9b700 1 --2- 172.21.15.157:0/1872817025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11e41527e0 0x7f11e4169770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:54.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.274+0000 7f11e2d9d700 1 -- 172.21.15.157:0/1872817025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11cc003a60 msgr2=0x7f11e41522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:54.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.274+0000 7f11e2d9d700 1 --2- 172.21.15.157:0/1872817025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11cc003a60 0x7f11e41522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:54.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.274+0000 7f11e2d9d700 1 -- 172.21.15.157:0/1872817025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11e41527e0 msgr2=0x7f11e4169770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:54.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.274+0000 7f11e2d9d700 1 --2- 172.21.15.157:0/1872817025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11e41527e0 0x7f11e4169770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:54.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.274+0000 7f11e2d9d700 1 -- 172.21.15.157:0/1872817025 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e40c2ab0 con 0x7f11e40c1720 2023-10-06T04:08:54.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.274+0000 7f11e2d9d700 1 --2- 172.21.15.157:0/1872817025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11e40c1720 0x7f11e4169cb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f11e40b1020 tx=0x7f11dc000fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:54.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.274+0000 7f11d27fc700 1 -- 172.21.15.157:0/1872817025 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc00c600 con 0x7f11e40c1720 2023-10-06T04:08:54.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.274+0000 7f11d27fc700 1 -- 172.21.15.157:0/1872817025 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11dc00c780 con 0x7f11e40c1720 2023-10-06T04:08:54.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.274+0000 7f11e8fb3700 1 -- 172.21.15.157:0/1872817025 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e4151ae0 con 0x7f11e40c1720 2023-10-06T04:08:54.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.274+0000 7f11d27fc700 1 -- 172.21.15.157:0/1872817025 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc00cb40 con 0x7f11e40c1720 2023-10-06T04:08:54.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.274+0000 7f11e8fb3700 1 -- 172.21.15.157:0/1872817025 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e40c27d0 con 0x7f11e40c1720 2023-10-06T04:08:54.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.275+0000 7f11e8fb3700 1 -- 172.21.15.157:0/1872817025 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e4151fc0 con 0x7f11e40c1720 2023-10-06T04:08:54.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.276+0000 7f11d27fc700 1 -- 172.21.15.157:0/1872817025 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f11dc013420 con 0x7f11e40c1720 2023-10-06T04:08:54.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.276+0000 7f11d27fc700 1 --2- 172.21.15.157:0/1872817025 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f11c4066f60 0x7f11c4069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:54.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.276+0000 7f11d27fc700 1 -- 172.21.15.157:0/1872817025 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6632+0+0 (secure 0 0 0) 0x7f11dc084300 con 0x7f11e40c1720 2023-10-06T04:08:54.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.276+0000 7f11e259c700 1 --2- 172.21.15.157:0/1872817025 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f11c4066f60 0x7f11c4069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:54.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.277+0000 7f11e259c700 1 --2- 172.21.15.157:0/1872817025 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f11c4066f60 0x7f11c4069410 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f11d400afd0 tx=0x7f11d401a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:54.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.280+0000 7f11d27fc700 1 -- 172.21.15.157:0/1872817025 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f11e4151fc0 con 0x7f11e40c1720 2023-10-06T04:08:54.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.507+0000 7f11e8fb3700 1 -- 172.21.15.157:0/1872817025 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11e4151fc0 con 0x7f11e40c1720 2023-10-06T04:08:54.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.507+0000 7f11d27fc700 1 -- 172.21.15.157:0/1872817025 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+5387 (secure 0 0 0) 0x7f11e4151fc0 con 0x7f11e40c1720 2023-10-06T04:08:54.509 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:08:54.509 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":81,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:08:53.880120+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:rejoin","state_seq":15,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:08:54.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.509+0000 7f11e8fb3700 1 -- 172.21.15.157:0/1872817025 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f11c4066f60 msgr2=0x7f11c4069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:54.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.509+0000 7f11e8fb3700 1 --2- 172.21.15.157:0/1872817025 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f11c4066f60 0x7f11c4069410 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f11d400afd0 tx=0x7f11d401a000 comp rx=0 tx=0).stop 2023-10-06T04:08:54.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.509+0000 7f11e8fb3700 1 -- 172.21.15.157:0/1872817025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11e40c1720 msgr2=0x7f11e4169cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:54.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.509+0000 7f11e8fb3700 1 --2- 172.21.15.157:0/1872817025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11e40c1720 0x7f11e4169cb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f11e40b1020 tx=0x7f11dc000fd0 comp rx=0 tx=0).stop 2023-10-06T04:08:54.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.509+0000 7f11e8fb3700 1 -- 172.21.15.157:0/1872817025 shutdown_connections 2023-10-06T04:08:54.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.509+0000 7f11e8fb3700 1 --2- 172.21.15.157:0/1872817025 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f11c4066f60 0x7f11c4069410 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:54.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.509+0000 7f11e8fb3700 1 --2- 172.21.15.157:0/1872817025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11e40c1720 0x7f11e4169cb0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:54.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.509+0000 7f11e8fb3700 1 --2- 172.21.15.157:0/1872817025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11cc003a60 0x7f11e41522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:54.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.509+0000 7f11e8fb3700 1 --2- 172.21.15.157:0/1872817025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11e41527e0 0x7f11e4169770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:54.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.509+0000 7f11e8fb3700 1 -- 172.21.15.157:0/1872817025 >> 172.21.15.157:0/1872817025 conn(0x7f11e406f230 msgr2=0x7f11e414e650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:54.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.509+0000 7f11e8fb3700 1 -- 172.21.15.157:0/1872817025 shutdown_connections 2023-10-06T04:08:54.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:54.509+0000 7f11e8fb3700 1 -- 172.21.15.157:0/1872817025 wait complete. 2023-10-06T04:08:54.512 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 81 2023-10-06T04:08:54.523 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:08:56.524 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:08:56.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.650+0000 7f3cad21a700 1 Processor -- start 2023-10-06T04:08:56.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.650+0000 7f3cad21a700 1 -- start start 2023-10-06T04:08:56.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.651+0000 7f3cad21a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ca80b2110 0x7f3ca8159df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:56.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.651+0000 7f3cad21a700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3ca806c1f0 0x7f3ca814fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:56.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.651+0000 7f3cad21a700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ca806ced0 con 0x7f3ca80b1d40 2023-10-06T04:08:56.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.651+0000 7f3cad21a700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ca80afb10 con 0x7f3ca806c1f0 2023-10-06T04:08:56.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.651+0000 7f3cad21a700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca806cc10 con 0x7f3ca80b2110 2023-10-06T04:08:56.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.651+0000 7f3ca6d9d700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f3ca80b1d40 0x7f3ca80af400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37668/0 (socket says 172.21.15.157:37668) 2023-10-06T04:08:56.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.651+0000 7f3ca6d9d700 1 -- 172.21.15.157:0/1682042980 learned_addr learned my addr 172.21.15.157:0/1682042980 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:08:56.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.651+0000 7f3ca759e700 1 --2- 172.21.15.157:0/1682042980 >> v2:172.21.15.175:3300/0 conn(0x7f3ca806c1f0 0x7f3ca814fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:56.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.651+0000 7f3c9e59c700 1 --2- 172.21.15.157:0/1682042980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ca80b2110 0x7f3ca8159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:56.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.651+0000 7f3ca659c700 1 -- 172.21.15.157:0/1682042980 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2865103641 0 0) 0x7f3ca806ced0 con 0x7f3ca80b1d40 2023-10-06T04:08:56.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.652+0000 7f3ca659c700 1 -- 172.21.15.157:0/1682042980 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c94003570 con 0x7f3ca80b1d40 2023-10-06T04:08:56.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.652+0000 7f3ca659c700 1 -- 172.21.15.157:0/1682042980 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4105366302 0 0) 0x7f3c94003570 con 0x7f3ca80b1d40 2023-10-06T04:08:56.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.652+0000 7f3ca659c700 1 -- 172.21.15.157:0/1682042980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ca80b2110 msgr2=0x7f3ca8159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:56.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.652+0000 7f3ca659c700 1 --2- 172.21.15.157:0/1682042980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ca80b2110 0x7f3ca8159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:56.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.652+0000 7f3ca659c700 1 -- 172.21.15.157:0/1682042980 >> v2:172.21.15.175:3300/0 conn(0x7f3ca806c1f0 msgr2=0x7f3ca814fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:56.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.652+0000 7f3ca659c700 1 --2- 172.21.15.157:0/1682042980 >> v2:172.21.15.175:3300/0 conn(0x7f3ca806c1f0 0x7f3ca814fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:56.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.652+0000 7f3ca659c700 1 -- 172.21.15.157:0/1682042980 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca806e570 con 0x7f3ca80b1d40 2023-10-06T04:08:56.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.652+0000 7f3ca659c700 1 -- 172.21.15.157:0/1682042980 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f3c98001470 con 0x7f3ca80b1d40 2023-10-06T04:08:56.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.652+0000 7f3ca759e700 1 --2- 172.21.15.157:0/1682042980 >> v2:172.21.15.175:3300/0 conn(0x7f3ca806c1f0 0x7f3ca814fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:08:56.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.652+0000 7f3ca659c700 1 -- 172.21.15.157:0/1682042980 >> v1:172.21.15.157:6789/0 conn(0x7f3ca80b1d40 legacy=0x7f3ca80af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:56.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.652+0000 7f3ca659c700 1 --2- 172.21.15.157:0/1682042980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c94003a60 0x7f3c94003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:56.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.652+0000 7f3ca659c700 1 --2- 172.21.15.157:0/1682042980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ca80b2110 0x7f3c94004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:56.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.652+0000 7f3ca659c700 1 --2- 172.21.15.157:0/1682042980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c94004d50 0x7f3c940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:56.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.652+0000 7f3ca659c700 1 -- 172.21.15.157:0/1682042980 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca80afb10 con 0x7f3c94004d50 2023-10-06T04:08:56.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.652+0000 7f3ca659c700 1 -- 172.21.15.157:0/1682042980 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca806cc10 con 0x7f3c94003a60 2023-10-06T04:08:56.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.652+0000 7f3ca659c700 1 -- 172.21.15.157:0/1682042980 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9400ba60 con 0x7f3ca80b2110 2023-10-06T04:08:56.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.652+0000 7f3ca759e700 1 --2- 172.21.15.157:0/1682042980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c94004d50 0x7f3c940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:56.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.652+0000 7f3ca6d9d700 1 --2- 172.21.15.157:0/1682042980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c94003a60 0x7f3c94003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:56.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.652+0000 7f3c9e59c700 1 --2- 172.21.15.157:0/1682042980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ca80b2110 0x7f3c94004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:56.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.652+0000 7f3ca759e700 1 -- 172.21.15.157:0/1682042980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ca80b2110 msgr2=0x7f3c94004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:56.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.652+0000 7f3ca759e700 1 --2- 172.21.15.157:0/1682042980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ca80b2110 0x7f3c94004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:56.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.652+0000 7f3ca759e700 1 -- 172.21.15.157:0/1682042980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c94003a60 msgr2=0x7f3c94003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:08:56.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.652+0000 7f3ca759e700 1 --2- 172.21.15.157:0/1682042980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c94003a60 0x7f3c94003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:56.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.652+0000 7f3ca759e700 1 -- 172.21.15.157:0/1682042980 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c94003570 con 0x7f3c94004d50 2023-10-06T04:08:56.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.652+0000 7f3ca759e700 1 --2- 172.21.15.157:0/1682042980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c94004d50 0x7f3c940051c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000e3c0 tx=0x7f3ca000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97b3df2ee56b0d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:56.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.653+0000 7f3ca659c700 1 -- 172.21.15.157:0/1682042980 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca000ec10 con 0x7f3c94004d50 2023-10-06T04:08:56.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.653+0000 7f3ca659c700 1 -- 172.21.15.157:0/1682042980 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca000f040 con 0x7f3c94004d50 2023-10-06T04:08:56.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.653+0000 7f3ca659c700 1 -- 172.21.15.157:0/1682042980 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0014480 con 0x7f3c94004d50 2023-10-06T04:08:56.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.653+0000 7f3cad21a700 1 -- 172.21.15.157:0/1682042980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c94004d50 msgr2=0x7f3c940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:56.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.653+0000 7f3cad21a700 1 --2- 172.21.15.157:0/1682042980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c94004d50 0x7f3c940051c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000e3c0 tx=0x7f3ca000b040 comp rx=0 tx=0).stop 2023-10-06T04:08:56.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.653+0000 7f3ca759e700 1 -- 172.21.15.157:0/1682042980 reap_dead start 2023-10-06T04:08:56.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.653+0000 7f3cad21a700 1 -- 172.21.15.157:0/1682042980 shutdown_connections 2023-10-06T04:08:56.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.653+0000 7f3cad21a700 1 -- 172.21.15.157:0/1682042980 >> 172.21.15.157:0/1682042980 conn(0x7f3ca80cc6e0 msgr2=0x7f3ca80ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:56.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.653+0000 7f3cad21a700 1 -- 172.21.15.157:0/1682042980 shutdown_connections 2023-10-06T04:08:56.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.653+0000 7f3cad21a700 1 -- 172.21.15.157:0/1682042980 wait complete. 2023-10-06T04:08:56.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.654+0000 7f3cad21a700 1 Processor -- start 2023-10-06T04:08:56.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.655+0000 7f3cad21a700 1 -- start start 2023-10-06T04:08:56.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.655+0000 7f3cad21a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c94004d50 0x7f3ca80c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:56.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.655+0000 7f3cad21a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ca80c3190 0x7f3ca8152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:56.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.655+0000 7f3cad21a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ca81694d0 0x7f3ca8169940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:56.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.655+0000 7f3cad21a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca80c9910 con 0x7f3ca81694d0 2023-10-06T04:08:56.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.655+0000 7f3cad21a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca8157020 con 0x7f3ca80c3190 2023-10-06T04:08:56.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.655+0000 7f3cad21a700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca8156730 con 0x7f3c94004d50 2023-10-06T04:08:56.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.655+0000 7f3ca759e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ca81694d0 0x7f3ca8169940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:56.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.655+0000 7f3ca6d9d700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c94004d50 0x7f3ca80c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:56.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.655+0000 7f3ca6d9d700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c94004d50 0x7f3ca80c7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:58310/0 (socket says 172.21.15.157:58310) 2023-10-06T04:08:56.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.655+0000 7f3ca759e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ca81694d0 0x7f3ca8169940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:48772/0 (socket says 172.21.15.157:48772) 2023-10-06T04:08:56.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.655+0000 7f3ca6d9d700 1 -- 172.21.15.157:0/2727989972 learned_addr learned my addr 172.21.15.157:0/2727989972 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:08:56.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.655+0000 7f3c9e59c700 1 --2- 172.21.15.157:0/2727989972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ca80c3190 0x7f3ca8152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:56.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.656+0000 7f3ca759e700 1 -- 172.21.15.157:0/2727989972 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c94004d50 msgr2=0x7f3ca80c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:56.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.656+0000 7f3ca759e700 1 --2- 172.21.15.157:0/2727989972 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c94004d50 0x7f3ca80c7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:56.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.656+0000 7f3ca759e700 1 -- 172.21.15.157:0/2727989972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ca80c3190 msgr2=0x7f3ca8152260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:56.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.656+0000 7f3ca759e700 1 --2- 172.21.15.157:0/2727989972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ca80c3190 0x7f3ca8152260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:56.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.656+0000 7f3ca759e700 1 -- 172.21.15.157:0/2727989972 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca80c9b40 con 0x7f3ca81694d0 2023-10-06T04:08:56.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.656+0000 7f3ca759e700 1 --2- 172.21.15.157:0/2727989972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ca81694d0 0x7f3ca8169940 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0019000 tx=0x7f3ca001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:56.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.656+0000 7f3c9f7fe700 1 -- 172.21.15.157:0/2727989972 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0014480 con 0x7f3ca81694d0 2023-10-06T04:08:56.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.656+0000 7f3c9f7fe700 1 -- 172.21.15.157:0/2727989972 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca001d630 con 0x7f3ca81694d0 2023-10-06T04:08:56.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.656+0000 7f3cad21a700 1 -- 172.21.15.157:0/2727989972 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca81569d0 con 0x7f3ca81694d0 2023-10-06T04:08:56.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.656+0000 7f3c9f7fe700 1 -- 172.21.15.157:0/2727989972 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca001d9d0 con 0x7f3ca81694d0 2023-10-06T04:08:56.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.656+0000 7f3cad21a700 1 -- 172.21.15.157:0/2727989972 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca80c9e20 con 0x7f3ca81694d0 2023-10-06T04:08:56.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.657+0000 7f3cad21a700 1 -- 172.21.15.157:0/2727989972 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ca80896c0 con 0x7f3ca81694d0 2023-10-06T04:08:56.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.657+0000 7f3c9f7fe700 1 -- 172.21.15.157:0/2727989972 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3ca0003040 con 0x7f3ca81694d0 2023-10-06T04:08:56.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.658+0000 7f3c9f7fe700 1 --2- 172.21.15.157:0/2727989972 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c88066f30 0x7f3c880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:56.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.658+0000 7f3c9f7fe700 1 -- 172.21.15.157:0/2727989972 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6632+0+0 (secure 0 0 0) 0x7f3ca0085710 con 0x7f3ca81694d0 2023-10-06T04:08:56.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.658+0000 7f3ca6d9d700 1 --2- 172.21.15.157:0/2727989972 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c88066f30 0x7f3c880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:56.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.658+0000 7f3ca6d9d700 1 --2- 172.21.15.157:0/2727989972 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c88066f30 0x7f3c880693e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3c980010e0 tx=0x7f3c98001e90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:56.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.662+0000 7f3c9f7fe700 1 -- 172.21.15.157:0/2727989972 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ca0052660 con 0x7f3ca81694d0 2023-10-06T04:08:56.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.888+0000 7f3cad21a700 1 -- 172.21.15.157:0/2727989972 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ca806d7b0 con 0x7f3ca81694d0 2023-10-06T04:08:56.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.889+0000 7f3c9f7fe700 1 -- 172.21.15.157:0/2727989972 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+5387 (secure 0 0 0) 0x7f3ca0055c80 con 0x7f3ca81694d0 2023-10-06T04:08:56.891 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:08:56.891 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":81,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:08:53.880120+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:rejoin","state_seq":15,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:08:56.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.891+0000 7f3cad21a700 1 -- 172.21.15.157:0/2727989972 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c88066f30 msgr2=0x7f3c880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:56.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.891+0000 7f3cad21a700 1 --2- 172.21.15.157:0/2727989972 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c88066f30 0x7f3c880693e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3c980010e0 tx=0x7f3c98001e90 comp rx=0 tx=0).stop 2023-10-06T04:08:56.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.891+0000 7f3cad21a700 1 -- 172.21.15.157:0/2727989972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ca81694d0 msgr2=0x7f3ca8169940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:56.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.891+0000 7f3cad21a700 1 --2- 172.21.15.157:0/2727989972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ca81694d0 0x7f3ca8169940 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0019000 tx=0x7f3ca001a000 comp rx=0 tx=0).stop 2023-10-06T04:08:56.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.891+0000 7f3cad21a700 1 -- 172.21.15.157:0/2727989972 shutdown_connections 2023-10-06T04:08:56.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.891+0000 7f3cad21a700 1 --2- 172.21.15.157:0/2727989972 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c88066f30 0x7f3c880693e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:56.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.891+0000 7f3cad21a700 1 --2- 172.21.15.157:0/2727989972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ca81694d0 0x7f3ca8169940 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:56.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.891+0000 7f3cad21a700 1 --2- 172.21.15.157:0/2727989972 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c94004d50 0x7f3ca80c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:56.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.891+0000 7f3cad21a700 1 --2- 172.21.15.157:0/2727989972 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ca80c3190 0x7f3ca8152260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:56.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.891+0000 7f3cad21a700 1 -- 172.21.15.157:0/2727989972 >> 172.21.15.157:0/2727989972 conn(0x7f3ca80cc6e0 msgr2=0x7f3ca814e680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:56.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.891+0000 7f3cad21a700 1 -- 172.21.15.157:0/2727989972 shutdown_connections 2023-10-06T04:08:56.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:56.892+0000 7f3cad21a700 1 -- 172.21.15.157:0/2727989972 wait complete. 2023-10-06T04:08:56.894 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 81 2023-10-06T04:08:56.905 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:08:58.907 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:08:59.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.035+0000 7faed54c5700 1 Processor -- start 2023-10-06T04:08:59.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.035+0000 7faed54c5700 1 -- start start 2023-10-06T04:08:59.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.036+0000 7faed54c5700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7faed006c2d0 0x7faed0157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:59.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.036+0000 7faed54c5700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed006d8d0 con 0x7faed00b1e40 2023-10-06T04:08:59.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.036+0000 7faed54c5700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed00af8f0 con 0x7faed00b04d0 2023-10-06T04:08:59.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.036+0000 7faed54c5700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed015a690 con 0x7faed006c2d0 2023-10-06T04:08:59.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.036+0000 7faecffff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7faed00b1e40 0x7faed01563b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37672/0 (socket says 172.21.15.157:37672) 2023-10-06T04:08:59.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.036+0000 7faecffff700 1 -- 172.21.15.157:0/1055108223 learned_addr learned my addr 172.21.15.157:0/1055108223 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:08:59.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.036+0000 7faed4cc4700 1 --2- 172.21.15.157:0/1055108223 >> v2:172.21.15.175:3300/0 conn(0x7faed006c2d0 0x7faed0157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:59.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.036+0000 7faeceffd700 1 -- 172.21.15.157:0/1055108223 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1888030828 0 0) 0x7faed006d8d0 con 0x7faed00b1e40 2023-10-06T04:08:59.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.036+0000 7faeceffd700 1 -- 172.21.15.157:0/1055108223 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faeb8003570 con 0x7faed00b1e40 2023-10-06T04:08:59.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.037+0000 7faed4cc4700 1 -- 172.21.15.157:0/1055108223 >> v1:172.21.15.175:6789/0 conn(0x7faed00b04d0 legacy=0x7faed0156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:59.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.037+0000 7faed4cc4700 1 -- 172.21.15.157:0/1055108223 >> v1:172.21.15.157:6789/0 conn(0x7faed00b1e40 legacy=0x7faed01563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:59.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.037+0000 7faeceffd700 1 -- 172.21.15.157:0/1055108223 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3095248154 0 0) 0x7faeb8003570 con 0x7faed00b1e40 2023-10-06T04:08:59.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.037+0000 7faed4cc4700 1 -- 172.21.15.157:0/1055108223 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed01571d0 con 0x7faed006c2d0 2023-10-06T04:08:59.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.037+0000 7faed4cc4700 1 --2- 172.21.15.157:0/1055108223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faed006c2d0 0x7faed0157750 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7faec8009e60 tx=0x7faec8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc33a2749312472f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:59.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.037+0000 7faeceffd700 1 -- 172.21.15.157:0/1055108223 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec800e070 con 0x7faed006c2d0 2023-10-06T04:08:59.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.037+0000 7faeceffd700 1 -- 172.21.15.157:0/1055108223 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec800b850 con 0x7faed006c2d0 2023-10-06T04:08:59.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.037+0000 7faeceffd700 1 -- 172.21.15.157:0/1055108223 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec0001470 con 0x7faed006c2d0 2023-10-06T04:08:59.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.037+0000 7faed54c5700 1 -- 172.21.15.157:0/1055108223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faed006c2d0 msgr2=0x7faed0157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:59.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.037+0000 7faed54c5700 1 --2- 172.21.15.157:0/1055108223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faed006c2d0 0x7faed0157750 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7faec8009e60 tx=0x7faec8009780 comp rx=0 tx=0).stop 2023-10-06T04:08:59.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.037+0000 7faed54c5700 1 -- 172.21.15.157:0/1055108223 shutdown_connections 2023-10-06T04:08:59.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.037+0000 7faed54c5700 1 --2- 172.21.15.157:0/1055108223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faed006c2d0 0x7faed0157750 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:59.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.037+0000 7faed54c5700 1 -- 172.21.15.157:0/1055108223 >> 172.21.15.157:0/1055108223 conn(0x7faed00cc540 msgr2=0x7faed00cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:59.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.037+0000 7faed54c5700 1 -- 172.21.15.157:0/1055108223 shutdown_connections 2023-10-06T04:08:59.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.038+0000 7faed54c5700 1 -- 172.21.15.157:0/1055108223 wait complete. 2023-10-06T04:08:59.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.039+0000 7faed54c5700 1 Processor -- start 2023-10-06T04:08:59.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.039+0000 7faed54c5700 1 -- start start 2023-10-06T04:08:59.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.039+0000 7faed54c5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faed006c2d0 0x7faed0152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:59.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.039+0000 7faed54c5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faed00b04d0 0x7faed0169800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:59.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.039+0000 7faed54c5700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faed00b1e40 0x7faed0154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:59.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.039+0000 7faed54c5700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed0153460 con 0x7faed006c2d0 2023-10-06T04:08:59.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.039+0000 7faed54c5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed0153660 con 0x7faed00b04d0 2023-10-06T04:08:59.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.039+0000 7faed54c5700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed0169370 con 0x7faed00b1e40 2023-10-06T04:08:59.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.040+0000 7faecffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faed006c2d0 0x7faed0152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:59.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.040+0000 7faecffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faed006c2d0 0x7faed0152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:48780/0 (socket says 172.21.15.157:48780) 2023-10-06T04:08:59.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.040+0000 7faecffff700 1 -- 172.21.15.157:0/3175882277 learned_addr learned my addr 172.21.15.157:0/3175882277 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:08:59.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.040+0000 7faecf7fe700 1 --2- 172.21.15.157:0/3175882277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faed00b04d0 0x7faed0169800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:59.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.040+0000 7faed4cc4700 1 --2- 172.21.15.157:0/3175882277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faed00b1e40 0x7faed0154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:59.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.040+0000 7faecffff700 1 -- 172.21.15.157:0/3175882277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faed00b1e40 msgr2=0x7faed0154f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:59.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.040+0000 7faecffff700 1 --2- 172.21.15.157:0/3175882277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faed00b1e40 0x7faed0154f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:59.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.040+0000 7faecffff700 1 -- 172.21.15.157:0/3175882277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faed00b04d0 msgr2=0x7faed0169800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:59.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.040+0000 7faecffff700 1 --2- 172.21.15.157:0/3175882277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faed00b04d0 0x7faed0169800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:59.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.040+0000 7faecffff700 1 -- 172.21.15.157:0/3175882277 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec8009a20 con 0x7faed006c2d0 2023-10-06T04:08:59.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.040+0000 7faecffff700 1 --2- 172.21.15.157:0/3175882277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faed006c2d0 0x7faed0152f20 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7faec000cbc0 tx=0x7faec0001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:59.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.040+0000 7faeb7fff700 1 -- 172.21.15.157:0/3175882277 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec0007150 con 0x7faed006c2d0 2023-10-06T04:08:59.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.040+0000 7faeb7fff700 1 -- 172.21.15.157:0/3175882277 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec00072d0 con 0x7faed006c2d0 2023-10-06T04:08:59.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.040+0000 7faed54c5700 1 -- 172.21.15.157:0/3175882277 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed0169040 con 0x7faed006c2d0 2023-10-06T04:08:59.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.040+0000 7faeb7fff700 1 -- 172.21.15.157:0/3175882277 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec0007690 con 0x7faed006c2d0 2023-10-06T04:08:59.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.040+0000 7faed54c5700 1 -- 172.21.15.157:0/3175882277 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed00c6f70 con 0x7faed006c2d0 2023-10-06T04:08:59.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.041+0000 7faed54c5700 1 -- 172.21.15.157:0/3175882277 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed0088f90 con 0x7faed006c2d0 2023-10-06T04:08:59.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.042+0000 7faeb7fff700 1 -- 172.21.15.157:0/3175882277 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faec0004a40 con 0x7faed006c2d0 2023-10-06T04:08:59.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.042+0000 7faeb7fff700 1 --2- 172.21.15.157:0/3175882277 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faebc066f30 0x7faebc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:08:59.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.042+0000 7faeb7fff700 1 -- 172.21.15.157:0/3175882277 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6632+0+0 (secure 0 0 0) 0x7faec0083f90 con 0x7faed006c2d0 2023-10-06T04:08:59.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.042+0000 7faecf7fe700 1 --2- 172.21.15.157:0/3175882277 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faebc066f30 0x7faebc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:08:59.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.043+0000 7faecf7fe700 1 --2- 172.21.15.157:0/3175882277 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faebc066f30 0x7faebc0693e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faed006cbf0 tx=0x7faec4007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:08:59.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.046+0000 7faeb7fff700 1 -- 172.21.15.157:0/3175882277 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faec0050ee0 con 0x7faed006c2d0 2023-10-06T04:08:59.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.273+0000 7faed54c5700 1 -- 172.21.15.157:0/3175882277 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faed006ccc0 con 0x7faed006c2d0 2023-10-06T04:08:59.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.273+0000 7faeb7fff700 1 -- 172.21.15.157:0/3175882277 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+5387 (secure 0 0 0) 0x7faec0018090 con 0x7faed006c2d0 2023-10-06T04:08:59.275 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:08:59.276 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":81,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:08:53.880120+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:rejoin","state_seq":15,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:08:59.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.276+0000 7faed54c5700 1 -- 172.21.15.157:0/3175882277 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faebc066f30 msgr2=0x7faebc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:59.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.276+0000 7faed54c5700 1 --2- 172.21.15.157:0/3175882277 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faebc066f30 0x7faebc0693e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faed006cbf0 tx=0x7faec4007ec0 comp rx=0 tx=0).stop 2023-10-06T04:08:59.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.276+0000 7faed54c5700 1 -- 172.21.15.157:0/3175882277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faed006c2d0 msgr2=0x7faed0152f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:08:59.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.276+0000 7faed54c5700 1 --2- 172.21.15.157:0/3175882277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faed006c2d0 0x7faed0152f20 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7faec000cbc0 tx=0x7faec0001a30 comp rx=0 tx=0).stop 2023-10-06T04:08:59.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.276+0000 7faed54c5700 1 -- 172.21.15.157:0/3175882277 shutdown_connections 2023-10-06T04:08:59.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.276+0000 7faed54c5700 1 --2- 172.21.15.157:0/3175882277 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7faebc066f30 0x7faebc0693e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:59.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.276+0000 7faed54c5700 1 --2- 172.21.15.157:0/3175882277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faed00b1e40 0x7faed0154f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:59.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.276+0000 7faed54c5700 1 --2- 172.21.15.157:0/3175882277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faed006c2d0 0x7faed0152f20 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:59.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.276+0000 7faed54c5700 1 --2- 172.21.15.157:0/3175882277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faed00b04d0 0x7faed0169800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:08:59.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.276+0000 7faed54c5700 1 -- 172.21.15.157:0/3175882277 >> 172.21.15.157:0/3175882277 conn(0x7faed00cc540 msgr2=0x7faed0157440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:08:59.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.276+0000 7faed54c5700 1 -- 172.21.15.157:0/3175882277 shutdown_connections 2023-10-06T04:08:59.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:08:59.276+0000 7faed54c5700 1 -- 172.21.15.157:0/3175882277 wait complete. 2023-10-06T04:08:59.279 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 81 2023-10-06T04:08:59.289 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:09:01.290 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:09:01.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.416+0000 7fe153bd2700 1 Processor -- start 2023-10-06T04:09:01.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.416+0000 7fe153bd2700 1 -- start start 2023-10-06T04:09:01.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.417+0000 7fe153bd2700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe14c06dc40 0x7fe14c14fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:01.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.417+0000 7fe153bd2700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe14c06c1f0 0x7fe14c150240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:01.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.417+0000 7fe153bd2700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe14c0c9780 con 0x7fe14c06cbb0 2023-10-06T04:09:01.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.417+0000 7fe153bd2700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe14c0c8c40 con 0x7fe14c06c1f0 2023-10-06T04:09:01.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.417+0000 7fe153bd2700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c06e0c0 con 0x7fe14c06dc40 2023-10-06T04:09:01.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.418+0000 7fe15216f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe14c06c1f0 0x7fe14c150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:01.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.418+0000 7fe15116d700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe14c06dc40 0x7fe14c14fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:01.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.418+0000 7fe15116d700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe14c06dc40 0x7fe14c14fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:58328/0 (socket says 172.21.15.157:58328) 2023-10-06T04:09:01.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.418+0000 7fe15216f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe14c06c1f0 0x7fe14c150240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:45002/0 (socket says 172.21.15.157:45002) 2023-10-06T04:09:01.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.418+0000 7fe15116d700 1 -- 172.21.15.157:0/4042281345 learned_addr learned my addr 172.21.15.157:0/4042281345 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:01.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.418+0000 7fe15196e700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fe14c06cbb0 0x7fe14c06d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.157:40406/0 (socket says 172.21.15.157:40406) 2023-10-06T04:09:01.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.418+0000 7fe15216f700 1 -- 172.21.15.157:0/4042281345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe14c06dc40 msgr2=0x7fe14c14fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:01.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.418+0000 7fe15216f700 1 --2- 172.21.15.157:0/4042281345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe14c06dc40 0x7fe14c14fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:01.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.418+0000 7fe15216f700 1 -- 172.21.15.157:0/4042281345 >> v1:172.21.15.175:6789/0 conn(0x7fe14c06cbb0 legacy=0x7fe14c06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:01.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.418+0000 7fe15216f700 1 -- 172.21.15.157:0/4042281345 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14c0c8e50 con 0x7fe14c06c1f0 2023-10-06T04:09:01.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.418+0000 7fe15116d700 1 --2- 172.21.15.157:0/4042281345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe14c06dc40 0x7fe14c14fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:09:01.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.418+0000 7fe15216f700 1 --2- 172.21.15.157:0/4042281345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe14c06c1f0 0x7fe14c150240 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe1440098e0 tx=0x7fe1440091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c25654ac5a50734 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:01.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.418+0000 7fe15096c700 1 -- 172.21.15.157:0/4042281345 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14400b6d0 con 0x7fe14c06c1f0 2023-10-06T04:09:01.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.419+0000 7fe15096c700 1 -- 172.21.15.157:0/4042281345 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14400b830 con 0x7fe14c06c1f0 2023-10-06T04:09:01.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.419+0000 7fe15096c700 1 -- 172.21.15.157:0/4042281345 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14400bbd0 con 0x7fe14c06c1f0 2023-10-06T04:09:01.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.419+0000 7fe153bd2700 1 -- 172.21.15.157:0/4042281345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe14c06c1f0 msgr2=0x7fe14c150240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:01.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.419+0000 7fe153bd2700 1 --2- 172.21.15.157:0/4042281345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe14c06c1f0 0x7fe14c150240 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe1440098e0 tx=0x7fe1440091a0 comp rx=0 tx=0).stop 2023-10-06T04:09:01.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.419+0000 7fe153bd2700 1 -- 172.21.15.157:0/4042281345 shutdown_connections 2023-10-06T04:09:01.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.419+0000 7fe153bd2700 1 --2- 172.21.15.157:0/4042281345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe14c06c1f0 0x7fe14c150240 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:01.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.419+0000 7fe153bd2700 1 --2- 172.21.15.157:0/4042281345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe14c06dc40 0x7fe14c14fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:01.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.419+0000 7fe153bd2700 1 -- 172.21.15.157:0/4042281345 >> 172.21.15.157:0/4042281345 conn(0x7fe14c06f230 msgr2=0x7fe14c06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:01.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.419+0000 7fe153bd2700 1 -- 172.21.15.157:0/4042281345 shutdown_connections 2023-10-06T04:09:01.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.419+0000 7fe153bd2700 1 -- 172.21.15.157:0/4042281345 wait complete. 2023-10-06T04:09:01.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.420+0000 7fe153bd2700 1 Processor -- start 2023-10-06T04:09:01.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.420+0000 7fe153bd2700 1 -- start start 2023-10-06T04:09:01.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.421+0000 7fe153bd2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe14c06c1f0 0x7fe14c1666a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:01.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.421+0000 7fe153bd2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe14c06cbb0 0x7fe14c14a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:01.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.421+0000 7fe153bd2700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe14c0c5880 0x7fe14c0c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:01.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.421+0000 7fe153bd2700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c0c3990 con 0x7fe14c06cbb0 2023-10-06T04:09:01.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.421+0000 7fe153bd2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c0c5e40 con 0x7fe14c06c1f0 2023-10-06T04:09:01.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.421+0000 7fe153bd2700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c0c6100 con 0x7fe14c0c5880 2023-10-06T04:09:01.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.421+0000 7fe15116d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe14c06cbb0 0x7fe14c14a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:01.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.421+0000 7fe15116d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe14c06cbb0 0x7fe14c14a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:48794/0 (socket says 172.21.15.157:48794) 2023-10-06T04:09:01.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.421+0000 7fe15116d700 1 -- 172.21.15.157:0/4271368624 learned_addr learned my addr 172.21.15.157:0/4271368624 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:01.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.421+0000 7fe15196e700 1 --2- 172.21.15.157:0/4271368624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe14c06c1f0 0x7fe14c1666a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:01.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.421+0000 7fe15216f700 1 --2- 172.21.15.157:0/4271368624 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe14c0c5880 0x7fe14c0c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:01.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.421+0000 7fe15116d700 1 -- 172.21.15.157:0/4271368624 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe14c0c5880 msgr2=0x7fe14c0c40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:01.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.421+0000 7fe15116d700 1 --2- 172.21.15.157:0/4271368624 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe14c0c5880 0x7fe14c0c40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:01.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.421+0000 7fe15116d700 1 -- 172.21.15.157:0/4271368624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe14c06c1f0 msgr2=0x7fe14c1666a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:01.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.421+0000 7fe15116d700 1 --2- 172.21.15.157:0/4271368624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe14c06c1f0 0x7fe14c1666a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:01.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.421+0000 7fe15116d700 1 -- 172.21.15.157:0/4271368624 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe13c0094f0 con 0x7fe14c06cbb0 2023-10-06T04:09:01.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.421+0000 7fe15116d700 1 --2- 172.21.15.157:0/4271368624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe14c06cbb0 0x7fe14c14a3f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00ec50 tx=0x7fe13c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:01.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.422+0000 7fe141ffb700 1 -- 172.21.15.157:0/4271368624 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c00c8e0 con 0x7fe14c06cbb0 2023-10-06T04:09:01.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.422+0000 7fe141ffb700 1 -- 172.21.15.157:0/4271368624 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13c00ca40 con 0x7fe14c06cbb0 2023-10-06T04:09:01.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.422+0000 7fe153bd2700 1 -- 172.21.15.157:0/4271368624 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe144009420 con 0x7fe14c06cbb0 2023-10-06T04:09:01.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.422+0000 7fe141ffb700 1 -- 172.21.15.157:0/4271368624 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c00cdc0 con 0x7fe14c06cbb0 2023-10-06T04:09:01.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.422+0000 7fe153bd2700 1 -- 172.21.15.157:0/4271368624 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe14c0c66c0 con 0x7fe14c06cbb0 2023-10-06T04:09:01.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.423+0000 7fe153bd2700 1 -- 172.21.15.157:0/4271368624 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe14c089ca0 con 0x7fe14c06cbb0 2023-10-06T04:09:01.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.423+0000 7fe141ffb700 1 -- 172.21.15.157:0/4271368624 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe13c013680 con 0x7fe14c06cbb0 2023-10-06T04:09:01.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.424+0000 7fe141ffb700 1 --2- 172.21.15.157:0/4271368624 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe130066f60 0x7fe130069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:01.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.424+0000 7fe141ffb700 1 -- 172.21.15.157:0/4271368624 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6632+0+0 (secure 0 0 0) 0x7fe13c084fb0 con 0x7fe14c06cbb0 2023-10-06T04:09:01.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.424+0000 7fe15196e700 1 --2- 172.21.15.157:0/4271368624 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe130066f60 0x7fe130069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:01.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.424+0000 7fe15196e700 1 --2- 172.21.15.157:0/4271368624 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe130066f60 0x7fe130069410 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe1480020c0 tx=0x7fe148001ad0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:01.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.428+0000 7fe141ffb700 1 -- 172.21.15.157:0/4271368624 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe13c0b65a0 con 0x7fe14c06cbb0 2023-10-06T04:09:01.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.655+0000 7fe153bd2700 1 -- 172.21.15.157:0/4271368624 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe14c04bb40 con 0x7fe14c06cbb0 2023-10-06T04:09:01.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.655+0000 7fe141ffb700 1 -- 172.21.15.157:0/4271368624 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+5393 (secure 0 0 0) 0x7fe13c017090 con 0x7fe14c06cbb0 2023-10-06T04:09:01.657 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:09:01.657 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":82,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:08:59.896620+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:clientreplay","state_seq":17,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:09:01.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.657+0000 7fe153bd2700 1 -- 172.21.15.157:0/4271368624 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe130066f60 msgr2=0x7fe130069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:01.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.657+0000 7fe153bd2700 1 --2- 172.21.15.157:0/4271368624 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe130066f60 0x7fe130069410 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe1480020c0 tx=0x7fe148001ad0 comp rx=0 tx=0).stop 2023-10-06T04:09:01.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.657+0000 7fe153bd2700 1 -- 172.21.15.157:0/4271368624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe14c06cbb0 msgr2=0x7fe14c14a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:01.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.657+0000 7fe153bd2700 1 --2- 172.21.15.157:0/4271368624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe14c06cbb0 0x7fe14c14a3f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00ec50 tx=0x7fe13c009250 comp rx=0 tx=0).stop 2023-10-06T04:09:01.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.657+0000 7fe153bd2700 1 -- 172.21.15.157:0/4271368624 shutdown_connections 2023-10-06T04:09:01.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.657+0000 7fe153bd2700 1 --2- 172.21.15.157:0/4271368624 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe130066f60 0x7fe130069410 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:01.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.657+0000 7fe153bd2700 1 --2- 172.21.15.157:0/4271368624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe14c06c1f0 0x7fe14c1666a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:01.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.657+0000 7fe153bd2700 1 --2- 172.21.15.157:0/4271368624 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe14c0c5880 0x7fe14c0c40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:01.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.657+0000 7fe153bd2700 1 --2- 172.21.15.157:0/4271368624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe14c06cbb0 0x7fe14c14a3f0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:01.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.657+0000 7fe153bd2700 1 -- 172.21.15.157:0/4271368624 >> 172.21.15.157:0/4271368624 conn(0x7fe14c06f230 msgr2=0x7fe14c0c7760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:01.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.657+0000 7fe153bd2700 1 -- 172.21.15.157:0/4271368624 shutdown_connections 2023-10-06T04:09:01.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:01.657+0000 7fe153bd2700 1 -- 172.21.15.157:0/4271368624 wait complete. 2023-10-06T04:09:01.660 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 82 2023-10-06T04:09:01.672 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 82, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-06T03:57:10.718782+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 82, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_5799": { "addr": "172.21.15.157:6834/765157057", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6832", "nonce": 765157057, "type": "v2" }, { "addr": "172.21.15.157:6834", "nonce": 765157057, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1 ], "features": 4540138320759226367, "flags": 0, "gid": 5799, "incarnation": 64, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 17 }, "gid_5812": { "addr": "172.21.15.175:6837/1436765859", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6836", "nonce": 1436765859, "type": "v2" }, { "addr": "172.21.15.175:6837", "nonce": 1436765859, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5812, "incarnation": 78, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:clientreplay", "state_seq": 17 } }, "last_failure": 0, "last_failure_osd_epoch": 52, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2023-10-06T04:08:59.896620+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2, 3, 4 ], "tableserver": 0, "up": { "mds_0": 5799, "mds_1": 5812 } } } ], "standbys": [ { "addr": "172.21.15.175:6835/1799968248", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6834", "nonce": 1799968248, "type": "v2" }, { "addr": "172.21.15.175:6835", "nonce": 1799968248, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 57, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5126, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.175:6839/745804271", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6838", "nonce": 745804271, "type": "v2" }, { "addr": "172.21.15.175:6839", "nonce": 745804271, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 42, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5164, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.157:6835/2461751318", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6833", "nonce": 2461751318, "type": "v2" }, { "addr": "172.21.15.157:6835", "nonce": 2461751318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 34, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5268, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-06T04:09:03.675 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:09:03.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.801+0000 7f22b4972700 1 Processor -- start 2023-10-06T04:09:03.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.801+0000 7f22b4972700 1 -- start start 2023-10-06T04:09:03.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.802+0000 7f22b4972700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22b00b1e40 0x7f22b0157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:03.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.802+0000 7f22b4972700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f22b006c2d0 0x7f22b006dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:03.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.802+0000 7f22b4972700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22b006d8d0 con 0x7f22b00b04d0 2023-10-06T04:09:03.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.802+0000 7f22b4972700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b00af8f0 con 0x7f22b006c2d0 2023-10-06T04:09:03.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.802+0000 7f22b4972700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b015a690 con 0x7f22b00b1e40 2023-10-06T04:09:03.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.802+0000 7f22ae59c700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f22b00b04d0 0x7f22b01563b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:53370/0 (socket says 172.21.15.157:53370) 2023-10-06T04:09:03.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.802+0000 7f22ae59c700 1 -- 172.21.15.157:0/362187207 learned_addr learned my addr 172.21.15.157:0/362187207 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:09:03.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.802+0000 7f22aed9d700 1 --2- 172.21.15.157:0/362187207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22b00b1e40 0x7f22b0157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:03.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.802+0000 7f22af59e700 1 --2- 172.21.15.157:0/362187207 >> v2:172.21.15.175:3300/0 conn(0x7f22b006c2d0 0x7f22b006dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:03.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.802+0000 7f22add9b700 1 -- 172.21.15.157:0/362187207 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2171569463 0 0) 0x7f22b006d8d0 con 0x7f22b00b04d0 2023-10-06T04:09:03.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.802+0000 7f22add9b700 1 -- 172.21.15.157:0/362187207 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2298003570 con 0x7f22b00b04d0 2023-10-06T04:09:03.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.803+0000 7f22add9b700 1 -- 172.21.15.157:0/362187207 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3082024552 0 0) 0x7f2298003570 con 0x7f22b00b04d0 2023-10-06T04:09:03.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.803+0000 7f22add9b700 1 -- 172.21.15.157:0/362187207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22b00b1e40 msgr2=0x7f22b0157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:03.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.803+0000 7f22add9b700 1 --2- 172.21.15.157:0/362187207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22b00b1e40 0x7f22b0157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:03.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.803+0000 7f22add9b700 1 -- 172.21.15.157:0/362187207 >> v2:172.21.15.175:3300/0 conn(0x7f22b006c2d0 msgr2=0x7f22b006dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:03.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.803+0000 7f22add9b700 1 --2- 172.21.15.157:0/362187207 >> v2:172.21.15.175:3300/0 conn(0x7f22b006c2d0 0x7f22b006dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:03.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.803+0000 7f22add9b700 1 -- 172.21.15.157:0/362187207 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b006ccc0 con 0x7f22b00b04d0 2023-10-06T04:09:03.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.803+0000 7f22add9b700 1 -- 172.21.15.157:0/362187207 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f22a4001470 con 0x7f22b00b04d0 2023-10-06T04:09:03.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.803+0000 7f22add9b700 1 -- 172.21.15.157:0/362187207 >> v1:172.21.15.157:6789/0 conn(0x7f22b00b04d0 legacy=0x7f22b01563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:03.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.803+0000 7f22add9b700 1 --2- 172.21.15.157:0/362187207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2298003a60 0x7f2298003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:03.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.803+0000 7f22add9b700 1 --2- 172.21.15.157:0/362187207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2298004810 0x7f2298004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:03.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.803+0000 7f22add9b700 1 --2- 172.21.15.157:0/362187207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22b00b1e40 0x7f22980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:03.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.803+0000 7f22add9b700 1 -- 172.21.15.157:0/362187207 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b00af8f0 con 0x7f2298003a60 2023-10-06T04:09:03.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.803+0000 7f22add9b700 1 -- 172.21.15.157:0/362187207 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b015a690 con 0x7f2298004810 2023-10-06T04:09:03.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.803+0000 7f22add9b700 1 -- 172.21.15.157:0/362187207 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f229800bae0 con 0x7f22b00b1e40 2023-10-06T04:09:03.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.803+0000 7f22aed9d700 1 --2- 172.21.15.157:0/362187207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2298003a60 0x7f2298003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:03.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.803+0000 7f22ae59c700 1 --2- 172.21.15.157:0/362187207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2298004810 0x7f2298004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:03.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.803+0000 7f22af59e700 1 --2- 172.21.15.157:0/362187207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22b00b1e40 0x7f22980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:03.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.803+0000 7f22aed9d700 1 -- 172.21.15.157:0/362187207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22b00b1e40 msgr2=0x7f22980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:03.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.803+0000 7f22aed9d700 1 --2- 172.21.15.157:0/362187207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22b00b1e40 0x7f22980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:03.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.803+0000 7f22aed9d700 1 -- 172.21.15.157:0/362187207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2298004810 msgr2=0x7f2298004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:03.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.803+0000 7f22aed9d700 1 --2- 172.21.15.157:0/362187207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2298004810 0x7f2298004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:03.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.803+0000 7f22aed9d700 1 -- 172.21.15.157:0/362187207 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2298003570 con 0x7f2298003a60 2023-10-06T04:09:03.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.803+0000 7f22aed9d700 1 --2- 172.21.15.157:0/362187207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2298003a60 0x7f2298003e70 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f22a0009c00 tx=0x7f22a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10ad5b1987fd62 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:03.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.803+0000 7f22add9b700 1 -- 172.21.15.157:0/362187207 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a000e710 con 0x7f2298003a60 2023-10-06T04:09:03.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.803+0000 7f22add9b700 1 -- 172.21.15.157:0/362187207 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a000e870 con 0x7f2298003a60 2023-10-06T04:09:03.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.803+0000 7f22add9b700 1 -- 172.21.15.157:0/362187207 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a000ec30 con 0x7f2298003a60 2023-10-06T04:09:03.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.804+0000 7f22b4972700 1 -- 172.21.15.157:0/362187207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2298003a60 msgr2=0x7f2298003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:03.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.804+0000 7f22b4972700 1 --2- 172.21.15.157:0/362187207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2298003a60 0x7f2298003e70 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f22a0009c00 tx=0x7f22a00091a0 comp rx=0 tx=0).stop 2023-10-06T04:09:03.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.804+0000 7f22af59e700 1 -- 172.21.15.157:0/362187207 reap_dead start 2023-10-06T04:09:03.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.804+0000 7f22b4972700 1 -- 172.21.15.157:0/362187207 shutdown_connections 2023-10-06T04:09:03.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.804+0000 7f22b4972700 1 -- 172.21.15.157:0/362187207 >> 172.21.15.157:0/362187207 conn(0x7f22b00cc540 msgr2=0x7f22b00cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:03.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.804+0000 7f22b4972700 1 -- 172.21.15.157:0/362187207 shutdown_connections 2023-10-06T04:09:03.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.804+0000 7f22b4972700 1 -- 172.21.15.157:0/362187207 wait complete. 2023-10-06T04:09:03.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.805+0000 7f22b4972700 1 Processor -- start 2023-10-06T04:09:03.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.805+0000 7f22b4972700 1 -- start start 2023-10-06T04:09:03.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.806+0000 7f22b4972700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2298003a60 0x7f22b0152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:03.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.806+0000 7f22b4972700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22b0152f90 0x7f22b014bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:03.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.806+0000 7f22b4972700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22b00cbae0 0x7f22b014c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:03.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.806+0000 7f22b4972700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b0169dd0 con 0x7f22b00cbae0 2023-10-06T04:09:03.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.806+0000 7f22b4972700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b014e700 con 0x7f22b0152f90 2023-10-06T04:09:03.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.806+0000 7f22b4972700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b0151470 con 0x7f2298003a60 2023-10-06T04:09:03.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.806+0000 7f22af59e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22b00cbae0 0x7f22b014c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:03.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.806+0000 7f22af59e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22b00cbae0 0x7f22b014c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33262/0 (socket says 172.21.15.157:33262) 2023-10-06T04:09:03.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.806+0000 7f22af59e700 1 -- 172.21.15.157:0/1040560787 learned_addr learned my addr 172.21.15.157:0/1040560787 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:03.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.806+0000 7f22ae59c700 1 --2- 172.21.15.157:0/1040560787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22b0152f90 0x7f22b014bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:03.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.806+0000 7f22aed9d700 1 --2- 172.21.15.157:0/1040560787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2298003a60 0x7f22b0152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:03.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.806+0000 7f22af59e700 1 -- 172.21.15.157:0/1040560787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2298003a60 msgr2=0x7f22b0152a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:03.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.806+0000 7f22af59e700 1 --2- 172.21.15.157:0/1040560787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2298003a60 0x7f22b0152a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:03.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.806+0000 7f22af59e700 1 -- 172.21.15.157:0/1040560787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22b0152f90 msgr2=0x7f22b014bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:03.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.806+0000 7f22af59e700 1 --2- 172.21.15.157:0/1040560787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22b0152f90 0x7f22b014bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:03.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.806+0000 7f22af59e700 1 -- 172.21.15.157:0/1040560787 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b01536f0 con 0x7f22b00cbae0 2023-10-06T04:09:03.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.807+0000 7f22af59e700 1 --2- 172.21.15.157:0/1040560787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22b00cbae0 0x7f22b014c070 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f22a80094d0 tx=0x7f22a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:03.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.807+0000 7f2296ffd700 1 -- 172.21.15.157:0/1040560787 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a800e040 con 0x7f22b00cbae0 2023-10-06T04:09:03.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.807+0000 7f2296ffd700 1 -- 172.21.15.157:0/1040560787 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a800f040 con 0x7f22b00cbae0 2023-10-06T04:09:03.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.807+0000 7f22b4972700 1 -- 172.21.15.157:0/1040560787 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b01539d0 con 0x7f22b00cbae0 2023-10-06T04:09:03.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.807+0000 7f2296ffd700 1 -- 172.21.15.157:0/1040560787 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a800fda0 con 0x7f22b00cbae0 2023-10-06T04:09:03.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.807+0000 7f22b4972700 1 -- 172.21.15.157:0/1040560787 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b0151c90 con 0x7f22b00cbae0 2023-10-06T04:09:03.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.808+0000 7f22b4972700 1 -- 172.21.15.157:0/1040560787 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b0151620 con 0x7f22b00cbae0 2023-10-06T04:09:03.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.808+0000 7f2296ffd700 1 -- 172.21.15.157:0/1040560787 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f22a8015640 con 0x7f22b00cbae0 2023-10-06T04:09:03.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.809+0000 7f2296ffd700 1 --2- 172.21.15.157:0/1040560787 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f229c066f30 0x7f229c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:03.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.809+0000 7f2296ffd700 1 -- 172.21.15.157:0/1040560787 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6632+0+0 (secure 0 0 0) 0x7f22a8084ed0 con 0x7f22b00cbae0 2023-10-06T04:09:03.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.809+0000 7f22aed9d700 1 --2- 172.21.15.157:0/1040560787 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f229c066f30 0x7f229c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:03.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.809+0000 7f22aed9d700 1 --2- 172.21.15.157:0/1040560787 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f229c066f30 0x7f229c0693e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f22a00091e0 tx=0x7f22a001a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:03.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:03.813+0000 7f2296ffd700 1 -- 172.21.15.157:0/1040560787 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f22b0151620 con 0x7f22b00cbae0 2023-10-06T04:09:04.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:04.038+0000 7f22b4972700 1 -- 172.21.15.157:0/1040560787 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22b0151620 con 0x7f22b00cbae0 2023-10-06T04:09:04.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:04.038+0000 7f2296ffd700 1 -- 172.21.15.157:0/1040560787 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+5393 (secure 0 0 0) 0x7f22b0151620 con 0x7f22b00cbae0 2023-10-06T04:09:04.041 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:09:04.041 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":82,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:08:59.896620+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:clientreplay","state_seq":17,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:09:04.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:04.041+0000 7f22b4972700 1 -- 172.21.15.157:0/1040560787 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f229c066f30 msgr2=0x7f229c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:04.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:04.041+0000 7f22b4972700 1 --2- 172.21.15.157:0/1040560787 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f229c066f30 0x7f229c0693e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f22a00091e0 tx=0x7f22a001a000 comp rx=0 tx=0).stop 2023-10-06T04:09:04.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:04.041+0000 7f22b4972700 1 -- 172.21.15.157:0/1040560787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22b00cbae0 msgr2=0x7f22b014c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:04.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:04.041+0000 7f22b4972700 1 --2- 172.21.15.157:0/1040560787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22b00cbae0 0x7f22b014c070 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f22a80094d0 tx=0x7f22a800b040 comp rx=0 tx=0).stop 2023-10-06T04:09:04.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:04.041+0000 7f22b4972700 1 -- 172.21.15.157:0/1040560787 shutdown_connections 2023-10-06T04:09:04.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:04.041+0000 7f22b4972700 1 --2- 172.21.15.157:0/1040560787 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f229c066f30 0x7f229c0693e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:04.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:04.041+0000 7f22b4972700 1 --2- 172.21.15.157:0/1040560787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22b00cbae0 0x7f22b014c070 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:04.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:04.041+0000 7f22b4972700 1 --2- 172.21.15.157:0/1040560787 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2298003a60 0x7f22b0152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:04.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:04.041+0000 7f22b4972700 1 --2- 172.21.15.157:0/1040560787 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22b0152f90 0x7f22b014bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:04.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:04.041+0000 7f22b4972700 1 -- 172.21.15.157:0/1040560787 >> 172.21.15.157:0/1040560787 conn(0x7f22b00cc540 msgr2=0x7f22b0159e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:04.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:04.041+0000 7f22b4972700 1 -- 172.21.15.157:0/1040560787 shutdown_connections 2023-10-06T04:09:04.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:04.041+0000 7f22b4972700 1 -- 172.21.15.157:0/1040560787 wait complete. 2023-10-06T04:09:04.045 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 82 2023-10-06T04:09:04.055 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:09:06.057 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:09:06.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.171+0000 7fe05e25c700 1 Processor -- start 2023-10-06T04:09:06.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.171+0000 7fe05e25c700 1 -- start start 2023-10-06T04:09:06.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.171+0000 7fe05e25c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe05806dc40 0x7fe05814fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:06.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.171+0000 7fe05e25c700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0580c9780 con 0x7fe05806cbb0 2023-10-06T04:09:06.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.172+0000 7fe05e25c700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe05806e0c0 con 0x7fe05806c1f0 2023-10-06T04:09:06.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.172+0000 7fe05e25c700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0580c8c40 con 0x7fe05806dc40 2023-10-06T04:09:06.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.172+0000 7fe056ffd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe05806dc40 0x7fe05814fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:06.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.172+0000 7fe0577fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe05806cbb0 0x7fe05806d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:53382/0 (socket says 172.21.15.157:53382) 2023-10-06T04:09:06.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.172+0000 7fe0577fe700 1 -- 172.21.15.157:0/757905881 learned_addr learned my addr 172.21.15.157:0/757905881 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:09:06.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.172+0000 7fe056ffd700 1 -- 172.21.15.157:0/757905881 >> v1:172.21.15.175:6789/0 conn(0x7fe05806c1f0 legacy=0x7fe058155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:06.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.172+0000 7fe056ffd700 1 -- 172.21.15.157:0/757905881 >> v1:172.21.15.157:6789/0 conn(0x7fe05806cbb0 legacy=0x7fe05806d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:06.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.172+0000 7fe056ffd700 1 -- 172.21.15.157:0/757905881 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0580c8e50 con 0x7fe05806dc40 2023-10-06T04:09:06.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.172+0000 7fe0567fc700 1 -- 172.21.15.157:0/757905881 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1449951498 0 0) 0x7fe0580c9780 con 0x7fe05806cbb0 2023-10-06T04:09:06.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.172+0000 7fe056ffd700 1 --2- 172.21.15.157:0/757905881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe05806dc40 0x7fe05814fd00 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe0400098e0 tx=0x7fe0400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a169be98c51bd457 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:06.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.172+0000 7fe0567fc700 1 -- 172.21.15.157:0/757905881 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040009e80 con 0x7fe05806dc40 2023-10-06T04:09:06.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.172+0000 7fe0567fc700 1 -- 172.21.15.157:0/757905881 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04000b8f0 con 0x7fe05806dc40 2023-10-06T04:09:06.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.172+0000 7fe0567fc700 1 -- 172.21.15.157:0/757905881 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04000bc70 con 0x7fe05806dc40 2023-10-06T04:09:06.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.173+0000 7fe05e25c700 1 -- 172.21.15.157:0/757905881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe05806dc40 msgr2=0x7fe05814fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:06.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.173+0000 7fe05e25c700 1 --2- 172.21.15.157:0/757905881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe05806dc40 0x7fe05814fd00 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe0400098e0 tx=0x7fe0400091a0 comp rx=0 tx=0).stop 2023-10-06T04:09:06.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.173+0000 7fe05e25c700 1 -- 172.21.15.157:0/757905881 shutdown_connections 2023-10-06T04:09:06.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.173+0000 7fe05e25c700 1 --2- 172.21.15.157:0/757905881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe05806dc40 0x7fe05814fd00 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:06.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.173+0000 7fe05e25c700 1 -- 172.21.15.157:0/757905881 >> 172.21.15.157:0/757905881 conn(0x7fe05806f230 msgr2=0x7fe05806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:06.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.173+0000 7fe05e25c700 1 -- 172.21.15.157:0/757905881 shutdown_connections 2023-10-06T04:09:06.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.173+0000 7fe05e25c700 1 -- 172.21.15.157:0/757905881 wait complete. 2023-10-06T04:09:06.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.174+0000 7fe05e25c700 1 Processor -- start 2023-10-06T04:09:06.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.175+0000 7fe05e25c700 1 -- start start 2023-10-06T04:09:06.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.175+0000 7fe05e25c700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe05806c1f0 0x7fe0581489e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:06.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.175+0000 7fe05e25c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe05806cbb0 0x7fe0580c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:06.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.175+0000 7fe05e25c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe058145990 0x7fe058145e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:06.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.175+0000 7fe05e25c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe058147130 con 0x7fe05806cbb0 2023-10-06T04:09:06.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.175+0000 7fe05e25c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0580c3e30 con 0x7fe058145990 2023-10-06T04:09:06.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.175+0000 7fe05e25c700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0580c3b90 con 0x7fe05806c1f0 2023-10-06T04:09:06.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.175+0000 7fe056ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe05806cbb0 0x7fe0580c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:06.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.175+0000 7fe056ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe05806cbb0 0x7fe0580c4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33280/0 (socket says 172.21.15.157:33280) 2023-10-06T04:09:06.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.175+0000 7fe056ffd700 1 -- 172.21.15.157:0/2182013673 learned_addr learned my addr 172.21.15.157:0/2182013673 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:06.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.176+0000 7fe057fff700 1 --2- 172.21.15.157:0/2182013673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe058145990 0x7fe058145e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:06.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.176+0000 7fe0577fe700 1 --2- 172.21.15.157:0/2182013673 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe05806c1f0 0x7fe0581489e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:06.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.176+0000 7fe056ffd700 1 -- 172.21.15.157:0/2182013673 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe05806c1f0 msgr2=0x7fe0581489e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:06.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.176+0000 7fe056ffd700 1 --2- 172.21.15.157:0/2182013673 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe05806c1f0 0x7fe0581489e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:06.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.176+0000 7fe056ffd700 1 -- 172.21.15.157:0/2182013673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe058145990 msgr2=0x7fe058145e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:06.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.176+0000 7fe056ffd700 1 --2- 172.21.15.157:0/2182013673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe058145990 0x7fe058145e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:06.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.176+0000 7fe056ffd700 1 -- 172.21.15.157:0/2182013673 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe040009420 con 0x7fe05806cbb0 2023-10-06T04:09:06.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.176+0000 7fe056ffd700 1 --2- 172.21.15.157:0/2182013673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe05806cbb0 0x7fe0580c4b20 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe0580c3960 tx=0x7fe040017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:06.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.176+0000 7fe04f7fe700 1 -- 172.21.15.157:0/2182013673 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04001f070 con 0x7fe05806cbb0 2023-10-06T04:09:06.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.176+0000 7fe04f7fe700 1 -- 172.21.15.157:0/2182013673 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04001a420 con 0x7fe05806cbb0 2023-10-06T04:09:06.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.176+0000 7fe05e25c700 1 -- 172.21.15.157:0/2182013673 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0580c46d0 con 0x7fe05806cbb0 2023-10-06T04:09:06.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.176+0000 7fe04f7fe700 1 -- 172.21.15.157:0/2182013673 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04000bc70 con 0x7fe05806cbb0 2023-10-06T04:09:06.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.176+0000 7fe05e25c700 1 -- 172.21.15.157:0/2182013673 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0580c6460 con 0x7fe05806cbb0 2023-10-06T04:09:06.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.177+0000 7fe05e25c700 1 -- 172.21.15.157:0/2182013673 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe058089ca0 con 0x7fe05806cbb0 2023-10-06T04:09:06.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.177+0000 7fe04f7fe700 1 -- 172.21.15.157:0/2182013673 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe04001aa30 con 0x7fe05806cbb0 2023-10-06T04:09:06.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.178+0000 7fe04f7fe700 1 --2- 172.21.15.157:0/2182013673 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe038066f30 0x7fe0380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:06.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.178+0000 7fe04f7fe700 1 -- 172.21.15.157:0/2182013673 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6632+0+0 (secure 0 0 0) 0x7fe040085e70 con 0x7fe05806cbb0 2023-10-06T04:09:06.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.178+0000 7fe0577fe700 1 --2- 172.21.15.157:0/2182013673 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe038066f30 0x7fe0380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:06.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.178+0000 7fe0577fe700 1 --2- 172.21.15.157:0/2182013673 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe038066f30 0x7fe0380693e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe050000f80 tx=0x7fe050009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:06.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.182+0000 7fe04f7fe700 1 -- 172.21.15.157:0/2182013673 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe040052d40 con 0x7fe05806cbb0 2023-10-06T04:09:06.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.413+0000 7fe05e25c700 1 -- 172.21.15.157:0/2182013673 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0580c9130 con 0x7fe05806cbb0 2023-10-06T04:09:06.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.414+0000 7fe04f7fe700 1 -- 172.21.15.157:0/2182013673 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+5387 (secure 0 0 0) 0x7fe040028090 con 0x7fe05806cbb0 2023-10-06T04:09:06.417 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:09:06.417 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":83,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:09:05.917135+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:active","state_seq":19,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:09:06.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.416+0000 7fe05e25c700 1 -- 172.21.15.157:0/2182013673 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe038066f30 msgr2=0x7fe0380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:06.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.416+0000 7fe05e25c700 1 --2- 172.21.15.157:0/2182013673 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe038066f30 0x7fe0380693e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe050000f80 tx=0x7fe050009040 comp rx=0 tx=0).stop 2023-10-06T04:09:06.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.416+0000 7fe05e25c700 1 -- 172.21.15.157:0/2182013673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe05806cbb0 msgr2=0x7fe0580c4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:06.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.416+0000 7fe05e25c700 1 --2- 172.21.15.157:0/2182013673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe05806cbb0 0x7fe0580c4b20 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe0580c3960 tx=0x7fe040017000 comp rx=0 tx=0).stop 2023-10-06T04:09:06.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.416+0000 7fe05e25c700 1 -- 172.21.15.157:0/2182013673 shutdown_connections 2023-10-06T04:09:06.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.416+0000 7fe05e25c700 1 --2- 172.21.15.157:0/2182013673 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe038066f30 0x7fe0380693e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:06.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.416+0000 7fe05e25c700 1 --2- 172.21.15.157:0/2182013673 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe058145990 0x7fe058145e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:06.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.416+0000 7fe05e25c700 1 --2- 172.21.15.157:0/2182013673 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe05806c1f0 0x7fe0581489e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:06.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.416+0000 7fe05e25c700 1 --2- 172.21.15.157:0/2182013673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe05806cbb0 0x7fe0580c4b20 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:06.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.416+0000 7fe05e25c700 1 -- 172.21.15.157:0/2182013673 >> 172.21.15.157:0/2182013673 conn(0x7fe05806f230 msgr2=0x7fe0580c8280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:06.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.416+0000 7fe05e25c700 1 -- 172.21.15.157:0/2182013673 shutdown_connections 2023-10-06T04:09:06.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:06.416+0000 7fe05e25c700 1 -- 172.21.15.157:0/2182013673 wait complete. 2023-10-06T04:09:06.420 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 83 2023-10-06T04:09:06.432 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d has gained rank=1, replacing gid=5698 2023-10-06T04:09:06.432 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 1 secs before reviving mds.e 2023-10-06T04:09:07.434 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.e 2023-10-06T04:09:07.434 INFO:tasks.ceph.mds.e:Restarting daemon 2023-10-06T04:09:07.434 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2023-10-06T04:09:07.437 INFO:tasks.ceph.mds.e:Started 2023-10-06T04:09:07.437 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:09:07.516 INFO:tasks.ceph.mds.e.smithi157.stdout:starting mds.e at 2023-10-06T04:09:07.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.559+0000 7f6094214700 1 Processor -- start 2023-10-06T04:09:07.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.559+0000 7f6094214700 1 -- start start 2023-10-06T04:09:07.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.560+0000 7f6094214700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f608c06cbb0 0x7f608c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:07.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.560+0000 7f6094214700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f608c06c1f0 0x7f608c06da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:07.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.560+0000 7f6094214700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f608c0c9780 con 0x7f608c06cf80 2023-10-06T04:09:07.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.560+0000 7f6094214700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f608c0c8c40 con 0x7f608c06c1f0 2023-10-06T04:09:07.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.560+0000 7f6094214700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f608c06e0c0 con 0x7f608c06cbb0 2023-10-06T04:09:07.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.560+0000 7f60917af700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f608c06c1f0 0x7f608c06da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:07.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.560+0000 7f60917af700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f608c06c1f0 0x7f608c06da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33306/0 (socket says 172.21.15.157:33306) 2023-10-06T04:09:07.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.560+0000 7f60917af700 1 -- 172.21.15.157:0/2973536450 learned_addr learned my addr 172.21.15.157:0/2973536450 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:07.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.560+0000 7f6091fb0700 1 --2- 172.21.15.157:0/2973536450 >> v2:172.21.15.175:3300/0 conn(0x7f608c06cbb0 0x7f608c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:07.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.560+0000 7f60917af700 1 -- 172.21.15.157:0/2973536450 >> v2:172.21.15.175:3300/0 conn(0x7f608c06cbb0 msgr2=0x7f608c06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:07.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.560+0000 7f60917af700 1 --2- 172.21.15.157:0/2973536450 >> v2:172.21.15.175:3300/0 conn(0x7f608c06cbb0 0x7f608c06d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:07.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.560+0000 7f60917af700 1 -- 172.21.15.157:0/2973536450 >> v1:172.21.15.175:6789/0 conn(0x7f608c06cf80 legacy=0x7f608c14fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:07.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.560+0000 7f60917af700 1 -- 172.21.15.157:0/2973536450 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c0c8e50 con 0x7f608c06c1f0 2023-10-06T04:09:07.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.560+0000 7f6091fb0700 1 --2- 172.21.15.157:0/2973536450 >> v2:172.21.15.175:3300/0 conn(0x7f608c06cbb0 0x7f608c06d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:09:07.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.560+0000 7f60917af700 1 --2- 172.21.15.157:0/2973536450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f608c06c1f0 0x7f608c06da70 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f608c06fd70 tx=0x7f607c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8052062510dd04dd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:07.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.560+0000 7f6090fae700 1 -- 172.21.15.157:0/2973536450 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c00b8b0 con 0x7f608c06c1f0 2023-10-06T04:09:07.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.560+0000 7f6090fae700 1 -- 172.21.15.157:0/2973536450 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607c00ba10 con 0x7f608c06c1f0 2023-10-06T04:09:07.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.560+0000 7f6090fae700 1 -- 172.21.15.157:0/2973536450 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c00bdb0 con 0x7f608c06c1f0 2023-10-06T04:09:07.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.561+0000 7f6094214700 1 -- 172.21.15.157:0/2973536450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f608c06c1f0 msgr2=0x7f608c06da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:07.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.561+0000 7f6094214700 1 --2- 172.21.15.157:0/2973536450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f608c06c1f0 0x7f608c06da70 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f608c06fd70 tx=0x7f607c009250 comp rx=0 tx=0).stop 2023-10-06T04:09:07.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.561+0000 7f6094214700 1 -- 172.21.15.157:0/2973536450 shutdown_connections 2023-10-06T04:09:07.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.561+0000 7f6094214700 1 --2- 172.21.15.157:0/2973536450 >> v2:172.21.15.175:3300/0 conn(0x7f608c06cbb0 0x7f608c06d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:07.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.561+0000 7f6094214700 1 --2- 172.21.15.157:0/2973536450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f608c06c1f0 0x7f608c06da70 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:07.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.561+0000 7f6094214700 1 -- 172.21.15.157:0/2973536450 >> 172.21.15.157:0/2973536450 conn(0x7f608c06f230 msgr2=0x7f608c06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:07.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.561+0000 7f6094214700 1 -- 172.21.15.157:0/2973536450 shutdown_connections 2023-10-06T04:09:07.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.561+0000 7f6094214700 1 -- 172.21.15.157:0/2973536450 wait complete. 2023-10-06T04:09:07.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.562+0000 7f6094214700 1 Processor -- start 2023-10-06T04:09:07.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.562+0000 7f6094214700 1 -- start start 2023-10-06T04:09:07.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.563+0000 7f6094214700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f608c06c1f0 0x7f608c156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:07.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.563+0000 7f6094214700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f608c06cbb0 0x7f608c156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:07.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.563+0000 7f6094214700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f608c06cf80 0x7f608c152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:07.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.563+0000 7f6094214700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c151470 con 0x7f608c06cf80 2023-10-06T04:09:07.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.563+0000 7f6094214700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c16a560 con 0x7f608c06c1f0 2023-10-06T04:09:07.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.563+0000 7f6094214700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608c155eb0 con 0x7f608c06cbb0 2023-10-06T04:09:07.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.563+0000 7f60927b1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f608c06cf80 0x7f608c152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:07.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.563+0000 7f60927b1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f608c06cf80 0x7f608c152dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33320/0 (socket says 172.21.15.157:33320) 2023-10-06T04:09:07.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.563+0000 7f60927b1700 1 -- 172.21.15.157:0/2253283231 learned_addr learned my addr 172.21.15.157:0/2253283231 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:07.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.563+0000 7f60917af700 1 --2- 172.21.15.157:0/2253283231 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f608c06cbb0 0x7f608c156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:07.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.563+0000 7f6091fb0700 1 --2- 172.21.15.157:0/2253283231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f608c06c1f0 0x7f608c156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:07.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.563+0000 7f60927b1700 1 -- 172.21.15.157:0/2253283231 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f608c06cbb0 msgr2=0x7f608c156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:07.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.563+0000 7f60927b1700 1 --2- 172.21.15.157:0/2253283231 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f608c06cbb0 0x7f608c156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:07.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.563+0000 7f60927b1700 1 -- 172.21.15.157:0/2253283231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f608c06c1f0 msgr2=0x7f608c156300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:07.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.563+0000 7f60927b1700 1 --2- 172.21.15.157:0/2253283231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f608c06c1f0 0x7f608c156300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:07.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.563+0000 7f60927b1700 1 -- 172.21.15.157:0/2253283231 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607c0094f0 con 0x7f608c06cf80 2023-10-06T04:09:07.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.563+0000 7f60927b1700 1 --2- 172.21.15.157:0/2253283231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f608c06cf80 0x7f608c152dd0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f608400bc30 tx=0x7f6084009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:07.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.563+0000 7f6081ffb700 1 -- 172.21.15.157:0/2253283231 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608400c8d0 con 0x7f608c06cf80 2023-10-06T04:09:07.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.563+0000 7f6081ffb700 1 -- 172.21.15.157:0/2253283231 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608400ca70 con 0x7f608c06cf80 2023-10-06T04:09:07.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.563+0000 7f6094214700 1 -- 172.21.15.157:0/2253283231 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f608c16aa40 con 0x7f608c06cf80 2023-10-06T04:09:07.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.563+0000 7f6081ffb700 1 -- 172.21.15.157:0/2253283231 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608400ce30 con 0x7f608c06cf80 2023-10-06T04:09:07.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.564+0000 7f6094214700 1 -- 172.21.15.157:0/2253283231 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f608c1545b0 con 0x7f608c06cf80 2023-10-06T04:09:07.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.564+0000 7f6094214700 1 -- 172.21.15.157:0/2253283231 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f608c156020 con 0x7f608c06cf80 2023-10-06T04:09:07.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.565+0000 7f6081ffb700 1 -- 172.21.15.157:0/2253283231 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6084014620 con 0x7f608c06cf80 2023-10-06T04:09:07.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.565+0000 7f6081ffb700 1 --2- 172.21.15.157:0/2253283231 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6070066f30 0x7f60700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:07.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.565+0000 7f6081ffb700 1 -- 172.21.15.157:0/2253283231 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6632+0+0 (secure 0 0 0) 0x7f6084084970 con 0x7f608c06cf80 2023-10-06T04:09:07.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.565+0000 7f6091fb0700 1 --2- 172.21.15.157:0/2253283231 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6070066f30 0x7f60700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:07.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.566+0000 7f6091fb0700 1 --2- 172.21.15.157:0/2253283231 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6070066f30 0x7f60700693e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f608800adc0 tx=0x7f6088006f90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:07.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.569+0000 7f6081ffb700 1 -- 172.21.15.157:0/2253283231 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f608c156020 con 0x7f608c06cf80 2023-10-06T04:09:07.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.793+0000 7f6094214700 1 -- 172.21.15.157:0/2253283231 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f608c156020 con 0x7f608c06cf80 2023-10-06T04:09:07.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.793+0000 7f6081ffb700 1 -- 172.21.15.157:0/2253283231 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+5387 (secure 0 0 0) 0x7f608c156020 con 0x7f608c06cf80 2023-10-06T04:09:07.796 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:09:07.796 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":83,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:09:05.917135+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:active","state_seq":19,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:09:07.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.795+0000 7f6094214700 1 -- 172.21.15.157:0/2253283231 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6070066f30 msgr2=0x7f60700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:07.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.795+0000 7f6094214700 1 --2- 172.21.15.157:0/2253283231 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6070066f30 0x7f60700693e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f608800adc0 tx=0x7f6088006f90 comp rx=0 tx=0).stop 2023-10-06T04:09:07.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.795+0000 7f6094214700 1 -- 172.21.15.157:0/2253283231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f608c06cf80 msgr2=0x7f608c152dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:07.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.795+0000 7f6094214700 1 --2- 172.21.15.157:0/2253283231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f608c06cf80 0x7f608c152dd0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f608400bc30 tx=0x7f6084009820 comp rx=0 tx=0).stop 2023-10-06T04:09:07.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.795+0000 7f6094214700 1 -- 172.21.15.157:0/2253283231 shutdown_connections 2023-10-06T04:09:07.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.795+0000 7f6094214700 1 --2- 172.21.15.157:0/2253283231 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6070066f30 0x7f60700693e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:07.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.795+0000 7f6094214700 1 --2- 172.21.15.157:0/2253283231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f608c06c1f0 0x7f608c156300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:07.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.795+0000 7f6094214700 1 --2- 172.21.15.157:0/2253283231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f608c06cf80 0x7f608c152dd0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:07.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.795+0000 7f6094214700 1 --2- 172.21.15.157:0/2253283231 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f608c06cbb0 0x7f608c156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:07.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.795+0000 7f6094214700 1 -- 172.21.15.157:0/2253283231 >> 172.21.15.157:0/2253283231 conn(0x7f608c06f230 msgr2=0x7f608c06d530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:07.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.795+0000 7f6094214700 1 -- 172.21.15.157:0/2253283231 shutdown_connections 2023-10-06T04:09:07.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:07.795+0000 7f6094214700 1 -- 172.21.15.157:0/2253283231 wait complete. 2023-10-06T04:09:07.799 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 83 2023-10-06T04:09:07.810 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is in active, standby or standby-replay 2023-10-06T04:09:07.847 INFO:tasks.ceph.mds.c.smithi157.stderr:2023-10-06T04:09:07.845+0000 7ff7d0710700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-10-06T04:09:09.810 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:09:09.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.938+0000 7f8dbc9ab700 1 Processor -- start 2023-10-06T04:09:09.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.939+0000 7f8dbc9ab700 1 -- start start 2023-10-06T04:09:09.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.939+0000 7f8dbc9ab700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8db80be970 0x7f8db806df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:09.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.939+0000 7f8dbc9ab700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8db8068d10 0x7f8db815d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:09.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.939+0000 7f8dbc9ab700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8db80bed40 con 0x7f8db806e450 2023-10-06T04:09:09.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.939+0000 7f8dbc9ab700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db8157010 con 0x7f8db80be970 2023-10-06T04:09:09.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.939+0000 7f8dbc9ab700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db806f9f0 con 0x7f8db8068d10 2023-10-06T04:09:09.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.939+0000 7f8db659c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8db80be970 0x7f8db806df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:09.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.940+0000 7f8db659c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8db80be970 0x7f8db806df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33324/0 (socket says 172.21.15.157:33324) 2023-10-06T04:09:09.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.940+0000 7f8db659c700 1 -- 172.21.15.157:0/3548220765 learned_addr learned my addr 172.21.15.157:0/3548220765 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:09.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.940+0000 7f8db6d9d700 1 --2- 172.21.15.157:0/3548220765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8db8068d10 0x7f8db815d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:09.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.940+0000 7f8db659c700 1 -- 172.21.15.157:0/3548220765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8db8068d10 msgr2=0x7f8db815d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:09.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.940+0000 7f8db659c700 1 --2- 172.21.15.157:0/3548220765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8db8068d10 0x7f8db815d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:09.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.940+0000 7f8db659c700 1 -- 172.21.15.157:0/3548220765 >> v1:172.21.15.175:6789/0 conn(0x7f8db806e450 legacy=0x7f8db806a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:09.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.940+0000 7f8db659c700 1 -- 172.21.15.157:0/3548220765 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db81574a0 con 0x7f8db80be970 2023-10-06T04:09:09.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.940+0000 7f8db6d9d700 1 --2- 172.21.15.157:0/3548220765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8db8068d10 0x7f8db815d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:09:09.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.940+0000 7f8db659c700 1 --2- 172.21.15.157:0/3548220765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8db80be970 0x7f8db806df10 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f8da80099b0 tx=0x7f8da8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b06cdd4cb4ce28a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:09.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.940+0000 7f8db5d9b700 1 -- 172.21.15.157:0/3548220765 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da800b820 con 0x7f8db80be970 2023-10-06T04:09:09.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.940+0000 7f8db5d9b700 1 -- 172.21.15.157:0/3548220765 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da800b980 con 0x7f8db80be970 2023-10-06T04:09:09.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.940+0000 7f8db5d9b700 1 -- 172.21.15.157:0/3548220765 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da800bd20 con 0x7f8db80be970 2023-10-06T04:09:09.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.941+0000 7f8dbc9ab700 1 -- 172.21.15.157:0/3548220765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8db80be970 msgr2=0x7f8db806df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:09.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.941+0000 7f8dbc9ab700 1 --2- 172.21.15.157:0/3548220765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8db80be970 0x7f8db806df10 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f8da80099b0 tx=0x7f8da8009250 comp rx=0 tx=0).stop 2023-10-06T04:09:09.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.941+0000 7f8dbc9ab700 1 -- 172.21.15.157:0/3548220765 shutdown_connections 2023-10-06T04:09:09.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.941+0000 7f8dbc9ab700 1 --2- 172.21.15.157:0/3548220765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8db8068d10 0x7f8db815d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:09.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.941+0000 7f8dbc9ab700 1 --2- 172.21.15.157:0/3548220765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8db80be970 0x7f8db806df10 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:09.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.941+0000 7f8dbc9ab700 1 -- 172.21.15.157:0/3548220765 >> 172.21.15.157:0/3548220765 conn(0x7f8db80c8d70 msgr2=0x7f8db80c4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:09.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.941+0000 7f8dbc9ab700 1 -- 172.21.15.157:0/3548220765 shutdown_connections 2023-10-06T04:09:09.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.941+0000 7f8dbc9ab700 1 -- 172.21.15.157:0/3548220765 wait complete. 2023-10-06T04:09:09.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.942+0000 7f8dbc9ab700 1 Processor -- start 2023-10-06T04:09:09.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.942+0000 7f8dbc9ab700 1 -- start start 2023-10-06T04:09:09.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.943+0000 7f8dbc9ab700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8db8068d10 0x7f8db816a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:09.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.943+0000 7f8dbc9ab700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8db806e450 0x7f8db8152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:09.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.943+0000 7f8dbc9ab700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8db80be970 0x7f8db8150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:09.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.943+0000 7f8dbc9ab700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db80ad7c0 con 0x7f8db8068d10 2023-10-06T04:09:09.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.943+0000 7f8dbc9ab700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db816a850 con 0x7f8db80be970 2023-10-06T04:09:09.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.943+0000 7f8dbc9ab700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db80c3600 con 0x7f8db806e450 2023-10-06T04:09:09.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.943+0000 7f8db659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8db8068d10 0x7f8db816a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:09.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.943+0000 7f8db659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8db8068d10 0x7f8db816a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33326/0 (socket says 172.21.15.157:33326) 2023-10-06T04:09:09.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.943+0000 7f8db659c700 1 -- 172.21.15.157:0/4177965734 learned_addr learned my addr 172.21.15.157:0/4177965734 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:09.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.943+0000 7f8daffff700 1 --2- 172.21.15.157:0/4177965734 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8db806e450 0x7f8db8152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:09.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.943+0000 7f8db6d9d700 1 --2- 172.21.15.157:0/4177965734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8db80be970 0x7f8db8150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:09.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.943+0000 7f8db659c700 1 -- 172.21.15.157:0/4177965734 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8db806e450 msgr2=0x7f8db8152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:09.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.943+0000 7f8db659c700 1 --2- 172.21.15.157:0/4177965734 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8db806e450 0x7f8db8152650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:09.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.943+0000 7f8db659c700 1 -- 172.21.15.157:0/4177965734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8db80be970 msgr2=0x7f8db8150d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:09.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.943+0000 7f8db659c700 1 --2- 172.21.15.157:0/4177965734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8db80be970 0x7f8db8150d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:09.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.943+0000 7f8db659c700 1 -- 172.21.15.157:0/4177965734 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da80094f0 con 0x7f8db8068d10 2023-10-06T04:09:09.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.943+0000 7f8db6d9d700 1 --2- 172.21.15.157:0/4177965734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8db80be970 0x7f8db8150d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:09:09.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.943+0000 7f8daffff700 1 --2- 172.21.15.157:0/4177965734 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8db806e450 0x7f8db8152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:09:09.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.943+0000 7f8db659c700 1 --2- 172.21.15.157:0/4177965734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8db8068d10 0x7f8db816a310 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f8db8068050 tx=0x7f8da8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:09.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.943+0000 7f8dae7fc700 1 -- 172.21.15.157:0/4177965734 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da8011cf0 con 0x7f8db8068d10 2023-10-06T04:09:09.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.943+0000 7f8dae7fc700 1 -- 172.21.15.157:0/4177965734 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da8020070 con 0x7f8db8068d10 2023-10-06T04:09:09.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.943+0000 7f8dbc9ab700 1 -- 172.21.15.157:0/4177965734 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db80ad270 con 0x7f8db8068d10 2023-10-06T04:09:09.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.943+0000 7f8dae7fc700 1 -- 172.21.15.157:0/4177965734 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da801b6a0 con 0x7f8db8068d10 2023-10-06T04:09:09.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.943+0000 7f8dbc9ab700 1 -- 172.21.15.157:0/4177965734 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db80ad4e0 con 0x7f8db8068d10 2023-10-06T04:09:09.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.944+0000 7f8dbc9ab700 1 -- 172.21.15.157:0/4177965734 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db80c38e0 con 0x7f8db8068d10 2023-10-06T04:09:09.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.945+0000 7f8dae7fc700 1 -- 172.21.15.157:0/4177965734 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8da8022030 con 0x7f8db8068d10 2023-10-06T04:09:09.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.945+0000 7f8dae7fc700 1 --2- 172.21.15.157:0/4177965734 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8d98066f30 0x7f8d980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:09.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.945+0000 7f8dae7fc700 1 -- 172.21.15.157:0/4177965734 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6632+0+0 (secure 0 0 0) 0x7f8da8086010 con 0x7f8db8068d10 2023-10-06T04:09:09.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.945+0000 7f8daffff700 1 --2- 172.21.15.157:0/4177965734 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8d98066f30 0x7f8d980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:09.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.946+0000 7f8daffff700 1 --2- 172.21.15.157:0/4177965734 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8d98066f30 0x7f8d980693e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8da0007fd0 tx=0x7f8da0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:09.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:09.949+0000 7f8dae7fc700 1 -- 172.21.15.157:0/4177965734 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8da8052ee0 con 0x7f8db8068d10 2023-10-06T04:09:10.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:10.174+0000 7f8dbc9ab700 1 -- 172.21.15.157:0/4177965734 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8db80716b0 con 0x7f8db8068d10 2023-10-06T04:09:10.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:10.175+0000 7f8dae7fc700 1 -- 172.21.15.157:0/4177965734 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+6147 (secure 0 0 0) 0x7f8da8056500 con 0x7f8db8068d10 2023-10-06T04:09:10.177 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:09:10.178 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":84,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6270,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1036901172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1036901172},{"type":"v1","addr":"172.21.15.157:6837","nonce":1036901172}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:09:05.917135+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:active","state_seq":19,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:09:10.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:10.177+0000 7f8dbc9ab700 1 -- 172.21.15.157:0/4177965734 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8d98066f30 msgr2=0x7f8d980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:10.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:10.177+0000 7f8dbc9ab700 1 --2- 172.21.15.157:0/4177965734 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8d98066f30 0x7f8d980693e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8da0007fd0 tx=0x7f8da0009040 comp rx=0 tx=0).stop 2023-10-06T04:09:10.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:10.177+0000 7f8dbc9ab700 1 -- 172.21.15.157:0/4177965734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8db8068d10 msgr2=0x7f8db816a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:10.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:10.177+0000 7f8dbc9ab700 1 --2- 172.21.15.157:0/4177965734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8db8068d10 0x7f8db816a310 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f8db8068050 tx=0x7f8da8016000 comp rx=0 tx=0).stop 2023-10-06T04:09:10.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:10.177+0000 7f8dbc9ab700 1 -- 172.21.15.157:0/4177965734 shutdown_connections 2023-10-06T04:09:10.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:10.177+0000 7f8dbc9ab700 1 --2- 172.21.15.157:0/4177965734 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8d98066f30 0x7f8d980693e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:10.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:10.177+0000 7f8dbc9ab700 1 --2- 172.21.15.157:0/4177965734 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8db80be970 0x7f8db8150d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:10.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:10.177+0000 7f8dbc9ab700 1 --2- 172.21.15.157:0/4177965734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8db8068d10 0x7f8db816a310 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:10.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:10.177+0000 7f8dbc9ab700 1 --2- 172.21.15.157:0/4177965734 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8db806e450 0x7f8db8152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:10.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:10.177+0000 7f8dbc9ab700 1 -- 172.21.15.157:0/4177965734 >> 172.21.15.157:0/4177965734 conn(0x7f8db80c8d70 msgr2=0x7f8db80c4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:10.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:10.177+0000 7f8dbc9ab700 1 -- 172.21.15.157:0/4177965734 shutdown_connections 2023-10-06T04:09:10.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:10.177+0000 7f8dbc9ab700 1 -- 172.21.15.157:0/4177965734 wait complete. 2023-10-06T04:09:10.181 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 84 2023-10-06T04:09:10.191 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e reported in up:standby state 2023-10-06T04:09:10.191 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 20 secs before thrashing 2023-10-06T04:09:16.059 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:09:16.062 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:09:30.192 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:09:30.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.317+0000 7f8fba305700 1 Processor -- start 2023-10-06T04:09:30.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.317+0000 7f8fba305700 1 -- start start 2023-10-06T04:09:30.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.318+0000 7f8fba305700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8fb406cbb0 0x7f8fb406d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:30.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.318+0000 7f8fba305700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb406c1f0 0x7f8fb406da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:30.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.318+0000 7f8fba305700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8fb406cf80 0x7f8fb414fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:30.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.318+0000 7f8fba305700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb40c8c40 con 0x7f8fb406cf80 2023-10-06T04:09:30.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.318+0000 7f8fba305700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb40c9780 con 0x7f8fb406cbb0 2023-10-06T04:09:30.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.318+0000 7f8fba305700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb406e0c0 con 0x7f8fb406c1f0 2023-10-06T04:09:30.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.318+0000 7f8fb88a2700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8fb406cf80 0x7f8fb414fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:30.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.318+0000 7f8fb88a2700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8fb406cf80 0x7f8fb414fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60616/0 (socket says 172.21.15.157:60616) 2023-10-06T04:09:30.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.318+0000 7f8fb88a2700 1 -- 172.21.15.157:0/570216333 learned_addr learned my addr 172.21.15.157:0/570216333 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:30.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.318+0000 7f8fabfff700 1 --2- 172.21.15.157:0/570216333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb406c1f0 0x7f8fb406da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:30.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.319+0000 7f8fb3fff700 1 --2- 172.21.15.157:0/570216333 >> v2:172.21.15.175:3300/0 conn(0x7f8fb406cbb0 0x7f8fb406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:30.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.319+0000 7f8fb88a2700 1 -- 172.21.15.157:0/570216333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb406c1f0 msgr2=0x7f8fb406da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:30.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.319+0000 7f8fb88a2700 1 --2- 172.21.15.157:0/570216333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb406c1f0 0x7f8fb406da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:30.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.319+0000 7f8fb88a2700 1 -- 172.21.15.157:0/570216333 >> v2:172.21.15.175:3300/0 conn(0x7f8fb406cbb0 msgr2=0x7f8fb406d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:30.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.319+0000 7f8fb88a2700 1 --2- 172.21.15.157:0/570216333 >> v2:172.21.15.175:3300/0 conn(0x7f8fb406cbb0 0x7f8fb406d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:30.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.319+0000 7f8fb88a2700 1 -- 172.21.15.157:0/570216333 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb40c8e50 con 0x7f8fb406cf80 2023-10-06T04:09:30.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.319+0000 7f8fb88a2700 1 --2- 172.21.15.157:0/570216333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb406cf80 0x7f8fb414fd00 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f8fac009f20 tx=0x7f8fac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5795a2e3e39d2258 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:30.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.319+0000 7f8fb37fe700 1 -- 172.21.15.157:0/570216333 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac00e070 con 0x7f8fb406cf80 2023-10-06T04:09:30.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.319+0000 7f8fb37fe700 1 -- 172.21.15.157:0/570216333 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fac00b9c0 con 0x7f8fb406cf80 2023-10-06T04:09:30.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.319+0000 7f8fb37fe700 1 -- 172.21.15.157:0/570216333 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac0092e0 con 0x7f8fb406cf80 2023-10-06T04:09:30.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.320+0000 7f8fba305700 1 -- 172.21.15.157:0/570216333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb406cf80 msgr2=0x7f8fb414fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:30.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.320+0000 7f8fba305700 1 --2- 172.21.15.157:0/570216333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb406cf80 0x7f8fb414fd00 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f8fac009f20 tx=0x7f8fac009840 comp rx=0 tx=0).stop 2023-10-06T04:09:30.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.320+0000 7f8fba305700 1 -- 172.21.15.157:0/570216333 shutdown_connections 2023-10-06T04:09:30.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.320+0000 7f8fba305700 1 --2- 172.21.15.157:0/570216333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb406cf80 0x7f8fb414fd00 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:30.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.320+0000 7f8fba305700 1 --2- 172.21.15.157:0/570216333 >> v2:172.21.15.175:3300/0 conn(0x7f8fb406cbb0 0x7f8fb406d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:30.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.320+0000 7f8fba305700 1 --2- 172.21.15.157:0/570216333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb406c1f0 0x7f8fb406da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:30.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.320+0000 7f8fba305700 1 -- 172.21.15.157:0/570216333 >> 172.21.15.157:0/570216333 conn(0x7f8fb406f230 msgr2=0x7f8fb406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:30.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.320+0000 7f8fba305700 1 -- 172.21.15.157:0/570216333 shutdown_connections 2023-10-06T04:09:30.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.320+0000 7f8fba305700 1 -- 172.21.15.157:0/570216333 wait complete. 2023-10-06T04:09:30.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.321+0000 7f8fba305700 1 Processor -- start 2023-10-06T04:09:30.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.321+0000 7f8fba305700 1 -- start start 2023-10-06T04:09:30.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.321+0000 7f8fba305700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fb406c1f0 0x7f8fb414d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:30.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.321+0000 7f8fba305700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb406cbb0 0x7f8fb414d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:30.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.321+0000 7f8fba305700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb406cf80 0x7f8fb41699e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:30.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.321+0000 7f8fba305700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb41472a0 con 0x7f8fb406cf80 2023-10-06T04:09:30.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.321+0000 7f8fba305700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb41515e0 con 0x7f8fb406c1f0 2023-10-06T04:09:30.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.322+0000 7f8fba305700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb41518c0 con 0x7f8fb406cbb0 2023-10-06T04:09:30.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.322+0000 7f8fb88a2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb406cf80 0x7f8fb41699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:30.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.322+0000 7f8fb88a2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb406cf80 0x7f8fb41699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60620/0 (socket says 172.21.15.157:60620) 2023-10-06T04:09:30.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.322+0000 7f8fb88a2700 1 -- 172.21.15.157:0/3138079497 learned_addr learned my addr 172.21.15.157:0/3138079497 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:30.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.322+0000 7f8fabfff700 1 --2- 172.21.15.157:0/3138079497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb406cbb0 0x7f8fb414d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:30.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.322+0000 7f8fb3fff700 1 --2- 172.21.15.157:0/3138079497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fb406c1f0 0x7f8fb414d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:30.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.322+0000 7f8fb88a2700 1 -- 172.21.15.157:0/3138079497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb406cbb0 msgr2=0x7f8fb414d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:30.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.322+0000 7f8fb88a2700 1 --2- 172.21.15.157:0/3138079497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb406cbb0 0x7f8fb414d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:30.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.322+0000 7f8fb88a2700 1 -- 172.21.15.157:0/3138079497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fb406c1f0 msgr2=0x7f8fb414d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:30.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.322+0000 7f8fb88a2700 1 --2- 172.21.15.157:0/3138079497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fb406c1f0 0x7f8fb414d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:30.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.322+0000 7f8fb88a2700 1 -- 172.21.15.157:0/3138079497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fac009ae0 con 0x7f8fb406cf80 2023-10-06T04:09:30.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.322+0000 7f8fb3fff700 1 --2- 172.21.15.157:0/3138079497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fb406c1f0 0x7f8fb414d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:09:30.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.322+0000 7f8fb88a2700 1 --2- 172.21.15.157:0/3138079497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb406cf80 0x7f8fb41699e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f8fac009880 tx=0x7f8fac018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:30.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.322+0000 7f8fb0ff9700 1 -- 172.21.15.157:0/3138079497 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac008c30 con 0x7f8fb406cf80 2023-10-06T04:09:30.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.322+0000 7f8fb0ff9700 1 -- 172.21.15.157:0/3138079497 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fac008db0 con 0x7f8fb406cf80 2023-10-06T04:09:30.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.322+0000 7f8fb0ff9700 1 -- 172.21.15.157:0/3138079497 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac01c660 con 0x7f8fb406cf80 2023-10-06T04:09:30.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.322+0000 7f8fba305700 1 -- 172.21.15.157:0/3138079497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb4147520 con 0x7f8fb406cf80 2023-10-06T04:09:30.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.322+0000 7f8fba305700 1 -- 172.21.15.157:0/3138079497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb4147ae0 con 0x7f8fb406cf80 2023-10-06T04:09:30.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.323+0000 7f8fba305700 1 -- 172.21.15.157:0/3138079497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fb4089ca0 con 0x7f8fb406cf80 2023-10-06T04:09:30.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.324+0000 7f8fb0ff9700 1 -- 172.21.15.157:0/3138079497 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8fac003040 con 0x7f8fb406cf80 2023-10-06T04:09:30.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.324+0000 7f8fb0ff9700 1 --2- 172.21.15.157:0/3138079497 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8f94066f60 0x7f8f94069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:30.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.324+0000 7f8fb0ff9700 1 -- 172.21.15.157:0/3138079497 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6632+0+0 (secure 0 0 0) 0x7f8fac086100 con 0x7f8fb406cf80 2023-10-06T04:09:30.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.324+0000 7f8fb3fff700 1 --2- 172.21.15.157:0/3138079497 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8f94066f60 0x7f8f94069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:30.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.325+0000 7f8fb3fff700 1 --2- 172.21.15.157:0/3138079497 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8f94066f60 0x7f8f94069410 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009d80 tx=0x7f8fa4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:30.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.329+0000 7f8fb0ff9700 1 -- 172.21.15.157:0/3138079497 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8fac052fd0 con 0x7f8fb406cf80 2023-10-06T04:09:30.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.554+0000 7f8fba305700 1 -- 172.21.15.157:0/3138079497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8fb4147cd0 con 0x7f8fb406cf80 2023-10-06T04:09:30.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.555+0000 7f8fb0ff9700 1 -- 172.21.15.157:0/3138079497 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+6148 (secure 0 0 0) 0x7f8fac0565f0 con 0x7f8fb406cf80 2023-10-06T04:09:30.557 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:09:30.557 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6270,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1036901172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1036901172},{"type":"v1","addr":"172.21.15.157:6837","nonce":1036901172}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:09:16.934246+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:active","state_seq":19,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:09:30.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.557+0000 7f8fba305700 1 -- 172.21.15.157:0/3138079497 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8f94066f60 msgr2=0x7f8f94069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:30.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.557+0000 7f8fba305700 1 --2- 172.21.15.157:0/3138079497 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8f94066f60 0x7f8f94069410 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009d80 tx=0x7f8fa4008040 comp rx=0 tx=0).stop 2023-10-06T04:09:30.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.557+0000 7f8fba305700 1 -- 172.21.15.157:0/3138079497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb406cf80 msgr2=0x7f8fb41699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:30.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.557+0000 7f8fba305700 1 --2- 172.21.15.157:0/3138079497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb406cf80 0x7f8fb41699e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f8fac009880 tx=0x7f8fac018000 comp rx=0 tx=0).stop 2023-10-06T04:09:30.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.557+0000 7f8fba305700 1 -- 172.21.15.157:0/3138079497 shutdown_connections 2023-10-06T04:09:30.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.557+0000 7f8fba305700 1 --2- 172.21.15.157:0/3138079497 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8f94066f60 0x7f8f94069410 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:30.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.557+0000 7f8fba305700 1 --2- 172.21.15.157:0/3138079497 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fb406c1f0 0x7f8fb414d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:30.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.557+0000 7f8fba305700 1 --2- 172.21.15.157:0/3138079497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb406cf80 0x7f8fb41699e0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:30.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.557+0000 7f8fba305700 1 --2- 172.21.15.157:0/3138079497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fb406cbb0 0x7f8fb414d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:30.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.557+0000 7f8fba305700 1 -- 172.21.15.157:0/3138079497 >> 172.21.15.157:0/3138079497 conn(0x7f8fb406f230 msgr2=0x7f8fb414ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:30.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.557+0000 7f8fba305700 1 -- 172.21.15.157:0/3138079497 shutdown_connections 2023-10-06T04:09:30.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.557+0000 7f8fba305700 1 -- 172.21.15.157:0/3138079497 wait complete. 2023-10-06T04:09:30.560 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 85 2023-10-06T04:09:30.571 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=0) 2023-10-06T04:09:30.571 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T04:09:30.571 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:09:30.583 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T04:09:30.583 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:09:30.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.738+0000 7fa422675700 1 Processor -- start 2023-10-06T04:09:30.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.738+0000 7fa422675700 1 -- start start 2023-10-06T04:09:30.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.739+0000 7fa422675700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa41c06ab50 0x7fa41c157190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:30.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.739+0000 7fa422675700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa41c0c46f0 con 0x7fa41c068cd0 2023-10-06T04:09:30.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.739+0000 7fa422675700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa41c0684c0 con 0x7fa41c06a780 2023-10-06T04:09:30.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.739+0000 7fa422675700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa41c0c3f20 con 0x7fa41c06ab50 2023-10-06T04:09:30.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.739+0000 7fa420c12700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa41c068cd0 0x7fa41c15e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:52116/0 (socket says 172.21.15.157:52116) 2023-10-06T04:09:30.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.739+0000 7fa420c12700 1 -- 172.21.15.157:0/109230083 learned_addr learned my addr 172.21.15.157:0/109230083 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:09:30.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.740+0000 7fa41b7fe700 1 --2- 172.21.15.157:0/109230083 >> v2:172.21.15.175:3300/0 conn(0x7fa41c06ab50 0x7fa41c157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:30.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.740+0000 7fa41affd700 1 -- 172.21.15.157:0/109230083 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3852285314 0 0) 0x7fa41c0c46f0 con 0x7fa41c068cd0 2023-10-06T04:09:30.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.740+0000 7fa41affd700 1 -- 172.21.15.157:0/109230083 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa404003570 con 0x7fa41c068cd0 2023-10-06T04:09:30.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.740+0000 7fa41b7fe700 1 -- 172.21.15.157:0/109230083 >> v1:172.21.15.175:6789/0 conn(0x7fa41c06a780 legacy=0x7fa41c0c6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:30.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.740+0000 7fa41b7fe700 1 -- 172.21.15.157:0/109230083 >> v1:172.21.15.157:6789/0 conn(0x7fa41c068cd0 legacy=0x7fa41c15e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:30.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.740+0000 7fa41affd700 1 -- 172.21.15.157:0/109230083 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 330037869 0 0) 0x7fa404003570 con 0x7fa41c068cd0 2023-10-06T04:09:30.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.740+0000 7fa41b7fe700 1 -- 172.21.15.157:0/109230083 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41c0c4130 con 0x7fa41c06ab50 2023-10-06T04:09:30.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.740+0000 7fa41b7fe700 1 --2- 172.21.15.157:0/109230083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa41c06ab50 0x7fa41c157190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa4100098e0 tx=0x7fa4100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d504ef4019de537e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:30.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.740+0000 7fa41affd700 1 -- 172.21.15.157:0/109230083 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4140017f0 con 0x7fa41c06ab50 2023-10-06T04:09:30.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.740+0000 7fa41affd700 1 -- 172.21.15.157:0/109230083 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa410011420 con 0x7fa41c06ab50 2023-10-06T04:09:30.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.740+0000 7fa41affd700 1 -- 172.21.15.157:0/109230083 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4100117c0 con 0x7fa41c06ab50 2023-10-06T04:09:30.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.741+0000 7fa422675700 1 -- 172.21.15.157:0/109230083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa41c06ab50 msgr2=0x7fa41c157190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:30.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.741+0000 7fa422675700 1 --2- 172.21.15.157:0/109230083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa41c06ab50 0x7fa41c157190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa4100098e0 tx=0x7fa4100091a0 comp rx=0 tx=0).stop 2023-10-06T04:09:30.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.741+0000 7fa422675700 1 -- 172.21.15.157:0/109230083 shutdown_connections 2023-10-06T04:09:30.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.741+0000 7fa422675700 1 --2- 172.21.15.157:0/109230083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa41c06ab50 0x7fa41c157190 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:30.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.741+0000 7fa422675700 1 -- 172.21.15.157:0/109230083 >> 172.21.15.157:0/109230083 conn(0x7fa41c06b510 msgr2=0x7fa41c06b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:30.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.741+0000 7fa422675700 1 -- 172.21.15.157:0/109230083 shutdown_connections 2023-10-06T04:09:30.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.741+0000 7fa422675700 1 -- 172.21.15.157:0/109230083 wait complete. 2023-10-06T04:09:30.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.742+0000 7fa422675700 1 Processor -- start 2023-10-06T04:09:30.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.742+0000 7fa422675700 1 -- start start 2023-10-06T04:09:30.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.743+0000 7fa422675700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa41c068cd0 0x7fa41c0bcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:30.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.743+0000 7fa422675700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41c06a780 0x7fa41c164450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:30.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.743+0000 7fa422675700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa41c06ab50 0x7fa41c0bf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:30.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.743+0000 7fa422675700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c163fc0 con 0x7fa41c06a780 2023-10-06T04:09:30.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.743+0000 7fa422675700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c1639d0 con 0x7fa41c06ab50 2023-10-06T04:09:30.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.743+0000 7fa422675700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c0689c0 con 0x7fa41c068cd0 2023-10-06T04:09:30.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.743+0000 7fa41b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41c06a780 0x7fa41c164450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:30.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.743+0000 7fa41b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41c06a780 0x7fa41c164450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60632/0 (socket says 172.21.15.157:60632) 2023-10-06T04:09:30.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.743+0000 7fa41b7fe700 1 -- 172.21.15.157:0/1127371555 learned_addr learned my addr 172.21.15.157:0/1127371555 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:30.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.743+0000 7fa420c12700 1 --2- 172.21.15.157:0/1127371555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa41c06ab50 0x7fa41c0bf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:30.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.743+0000 7fa41bfff700 1 --2- 172.21.15.157:0/1127371555 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa41c068cd0 0x7fa41c0bcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:30.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.743+0000 7fa41b7fe700 1 -- 172.21.15.157:0/1127371555 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa41c068cd0 msgr2=0x7fa41c0bcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:30.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.743+0000 7fa41b7fe700 1 --2- 172.21.15.157:0/1127371555 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa41c068cd0 0x7fa41c0bcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:30.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.743+0000 7fa41b7fe700 1 -- 172.21.15.157:0/1127371555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa41c06ab50 msgr2=0x7fa41c0bf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:30.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.743+0000 7fa41b7fe700 1 --2- 172.21.15.157:0/1127371555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa41c06ab50 0x7fa41c0bf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:30.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.743+0000 7fa41b7fe700 1 -- 172.21.15.157:0/1127371555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa410009420 con 0x7fa41c06a780 2023-10-06T04:09:30.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.743+0000 7fa420c12700 1 --2- 172.21.15.157:0/1127371555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa41c06ab50 0x7fa41c0bf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:09:30.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.743+0000 7fa41b7fe700 1 --2- 172.21.15.157:0/1127371555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41c06a780 0x7fa41c164450 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa41c06b1f0 tx=0x7fa410016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:30.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.743+0000 7fa403fff700 1 -- 172.21.15.157:0/1127371555 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41001cc50 con 0x7fa41c06a780 2023-10-06T04:09:30.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.744+0000 7fa403fff700 1 -- 172.21.15.157:0/1127371555 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41001cdb0 con 0x7fa41c06a780 2023-10-06T04:09:30.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.744+0000 7fa422675700 1 -- 172.21.15.157:0/1127371555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa41c157880 con 0x7fa41c06a780 2023-10-06T04:09:30.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.744+0000 7fa403fff700 1 -- 172.21.15.157:0/1127371555 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41001a6a0 con 0x7fa41c06a780 2023-10-06T04:09:30.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.744+0000 7fa422675700 1 -- 172.21.15.157:0/1127371555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa41c157e40 con 0x7fa41c06a780 2023-10-06T04:09:30.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.745+0000 7fa422675700 1 -- 172.21.15.157:0/1127371555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa41c164130 con 0x7fa41c06a780 2023-10-06T04:09:30.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.745+0000 7fa403fff700 1 -- 172.21.15.157:0/1127371555 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa41001aa50 con 0x7fa41c06a780 2023-10-06T04:09:30.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.745+0000 7fa403fff700 1 --2- 172.21.15.157:0/1127371555 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa408066f60 0x7fa408069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:30.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.745+0000 7fa403fff700 1 -- 172.21.15.157:0/1127371555 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6632+0+0 (secure 0 0 0) 0x7fa4100863e0 con 0x7fa41c06a780 2023-10-06T04:09:30.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.745+0000 7fa41bfff700 1 --2- 172.21.15.157:0/1127371555 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa408066f60 0x7fa408069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:30.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.746+0000 7fa41bfff700 1 --2- 172.21.15.157:0/1127371555 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa408066f60 0x7fa408069410 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0037c0 tx=0x7fa40c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:30.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.749+0000 7fa403fff700 1 -- 172.21.15.157:0/1127371555 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa4100532b0 con 0x7fa41c06a780 2023-10-06T04:09:30.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.981+0000 7fa422675700 1 -- 172.21.15.157:0/1127371555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa41c04bb40 con 0x7fa41c06a780 2023-10-06T04:09:30.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.982+0000 7fa403fff700 1 -- 172.21.15.157:0/1127371555 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+6148 (secure 0 0 0) 0x7fa4100568d0 con 0x7fa41c06a780 2023-10-06T04:09:30.984 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:09:30.984 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6270,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1036901172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1036901172},{"type":"v1","addr":"172.21.15.157:6837","nonce":1036901172}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:09:16.934246+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:active","state_seq":19,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:09:30.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.984+0000 7fa422675700 1 -- 172.21.15.157:0/1127371555 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa408066f60 msgr2=0x7fa408069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:30.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.984+0000 7fa422675700 1 --2- 172.21.15.157:0/1127371555 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa408066f60 0x7fa408069410 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0037c0 tx=0x7fa40c009040 comp rx=0 tx=0).stop 2023-10-06T04:09:30.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.984+0000 7fa422675700 1 -- 172.21.15.157:0/1127371555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41c06a780 msgr2=0x7fa41c164450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:30.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.984+0000 7fa422675700 1 --2- 172.21.15.157:0/1127371555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41c06a780 0x7fa41c164450 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa41c06b1f0 tx=0x7fa410016000 comp rx=0 tx=0).stop 2023-10-06T04:09:30.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.984+0000 7fa422675700 1 -- 172.21.15.157:0/1127371555 shutdown_connections 2023-10-06T04:09:30.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.984+0000 7fa422675700 1 --2- 172.21.15.157:0/1127371555 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa408066f60 0x7fa408069410 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:30.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.984+0000 7fa422675700 1 --2- 172.21.15.157:0/1127371555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa41c06ab50 0x7fa41c0bf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:30.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.984+0000 7fa422675700 1 --2- 172.21.15.157:0/1127371555 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa41c068cd0 0x7fa41c0bcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:30.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.984+0000 7fa422675700 1 --2- 172.21.15.157:0/1127371555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41c06a780 0x7fa41c164450 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:30.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.984+0000 7fa422675700 1 -- 172.21.15.157:0/1127371555 >> 172.21.15.157:0/1127371555 conn(0x7fa41c06b510 msgr2=0x7fa41c151a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:30.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.984+0000 7fa422675700 1 -- 172.21.15.157:0/1127371555 shutdown_connections 2023-10-06T04:09:30.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:30.984+0000 7fa422675700 1 -- 172.21.15.157:0/1127371555 wait complete. 2023-10-06T04:09:30.987 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 85 2023-10-06T04:09:30.998 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-06T04:09:32.999 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:09:33.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.123+0000 7f82669d1700 1 Processor -- start 2023-10-06T04:09:33.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.123+0000 7f82669d1700 1 -- start start 2023-10-06T04:09:33.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.124+0000 7f82669d1700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f826006cbb0 0x7f826006d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:33.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.124+0000 7f82669d1700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8260150410 0x7f826006da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:33.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.124+0000 7f82669d1700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82600c9780 con 0x7f826006c1f0 2023-10-06T04:09:33.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.124+0000 7f82669d1700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82600c8c40 con 0x7f8260150410 2023-10-06T04:09:33.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.124+0000 7f82669d1700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826006e0c0 con 0x7f826006cbb0 2023-10-06T04:09:33.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.124+0000 7f825f7fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f826006c1f0 0x7f826014fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:52128/0 (socket says 172.21.15.157:52128) 2023-10-06T04:09:33.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.124+0000 7f825f7fe700 1 -- 172.21.15.157:0/4120155686 learned_addr learned my addr 172.21.15.157:0/4120155686 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:09:33.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.124+0000 7f825ffff700 1 --2- 172.21.15.157:0/4120155686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f826006cbb0 0x7f826006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:33.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.124+0000 7f8264f6e700 1 --2- 172.21.15.157:0/4120155686 >> v2:172.21.15.175:3300/0 conn(0x7f8260150410 0x7f826006da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:33.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.125+0000 7f825effd700 1 -- 172.21.15.157:0/4120155686 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 322595029 0 0) 0x7f82600c9780 con 0x7f826006c1f0 2023-10-06T04:09:33.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.125+0000 7f825effd700 1 -- 172.21.15.157:0/4120155686 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8248003570 con 0x7f826006c1f0 2023-10-06T04:09:33.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.125+0000 7f825ffff700 1 -- 172.21.15.157:0/4120155686 >> v2:172.21.15.175:3300/0 conn(0x7f8260150410 msgr2=0x7f826006da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:33.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.125+0000 7f825ffff700 1 --2- 172.21.15.157:0/4120155686 >> v2:172.21.15.175:3300/0 conn(0x7f8260150410 0x7f826006da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:33.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.125+0000 7f825ffff700 1 -- 172.21.15.157:0/4120155686 >> v1:172.21.15.157:6789/0 conn(0x7f826006c1f0 legacy=0x7f826014fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:33.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.125+0000 7f825ffff700 1 -- 172.21.15.157:0/4120155686 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82600c8e50 con 0x7f826006cbb0 2023-10-06T04:09:33.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.125+0000 7f8264f6e700 1 --2- 172.21.15.157:0/4120155686 >> v2:172.21.15.175:3300/0 conn(0x7f8260150410 0x7f826006da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:09:33.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.125+0000 7f825ffff700 1 --2- 172.21.15.157:0/4120155686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f826006cbb0 0x7f826006d530 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f82500098e0 tx=0x7f82500091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae926e3564bbfc39 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:33.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.125+0000 7f825effd700 1 -- 172.21.15.157:0/4120155686 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8250011630 con 0x7f826006cbb0 2023-10-06T04:09:33.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.125+0000 7f825effd700 1 -- 172.21.15.157:0/4120155686 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8250011790 con 0x7f826006cbb0 2023-10-06T04:09:33.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.125+0000 7f825effd700 1 -- 172.21.15.157:0/4120155686 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8250011b30 con 0x7f826006cbb0 2023-10-06T04:09:33.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.126+0000 7f82669d1700 1 -- 172.21.15.157:0/4120155686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f826006cbb0 msgr2=0x7f826006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:33.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.126+0000 7f82669d1700 1 --2- 172.21.15.157:0/4120155686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f826006cbb0 0x7f826006d530 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f82500098e0 tx=0x7f82500091a0 comp rx=0 tx=0).stop 2023-10-06T04:09:33.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.126+0000 7f82669d1700 1 -- 172.21.15.157:0/4120155686 shutdown_connections 2023-10-06T04:09:33.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.126+0000 7f82669d1700 1 --2- 172.21.15.157:0/4120155686 >> v2:172.21.15.175:3300/0 conn(0x7f8260150410 0x7f826006da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:33.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.126+0000 7f82669d1700 1 --2- 172.21.15.157:0/4120155686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f826006cbb0 0x7f826006d530 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:33.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.126+0000 7f82669d1700 1 -- 172.21.15.157:0/4120155686 >> 172.21.15.157:0/4120155686 conn(0x7f826006f230 msgr2=0x7f826006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:33.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.126+0000 7f82669d1700 1 -- 172.21.15.157:0/4120155686 shutdown_connections 2023-10-06T04:09:33.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.126+0000 7f82669d1700 1 -- 172.21.15.157:0/4120155686 wait complete. 2023-10-06T04:09:33.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.127+0000 7f82669d1700 1 Processor -- start 2023-10-06T04:09:33.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.127+0000 7f82669d1700 1 -- start start 2023-10-06T04:09:33.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.128+0000 7f82669d1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f826006c1f0 0x7f8260147f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:33.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.128+0000 7f82669d1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f826006cbb0 0x7f8260166fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:33.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.128+0000 7f82669d1700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8260150410 0x7f8260154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:33.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.128+0000 7f82669d1700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82601487b0 con 0x7f826006cbb0 2023-10-06T04:09:33.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.128+0000 7f82669d1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8260156660 con 0x7f826006c1f0 2023-10-06T04:09:33.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.128+0000 7f82669d1700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8260156940 con 0x7f8260150410 2023-10-06T04:09:33.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.128+0000 7f825f7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f826006cbb0 0x7f8260166fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:33.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.128+0000 7f825f7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f826006cbb0 0x7f8260166fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60642/0 (socket says 172.21.15.157:60642) 2023-10-06T04:09:33.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.128+0000 7f825f7fe700 1 -- 172.21.15.157:0/1173523119 learned_addr learned my addr 172.21.15.157:0/1173523119 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:33.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.128+0000 7f825ffff700 1 --2- 172.21.15.157:0/1173523119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f826006c1f0 0x7f8260147f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:33.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.128+0000 7f8264f6e700 1 --2- 172.21.15.157:0/1173523119 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8260150410 0x7f8260154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:33.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.128+0000 7f825f7fe700 1 -- 172.21.15.157:0/1173523119 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8260150410 msgr2=0x7f8260154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:33.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.128+0000 7f825f7fe700 1 --2- 172.21.15.157:0/1173523119 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8260150410 0x7f8260154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:33.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.128+0000 7f825f7fe700 1 -- 172.21.15.157:0/1173523119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f826006c1f0 msgr2=0x7f8260147f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:33.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.128+0000 7f825f7fe700 1 --2- 172.21.15.157:0/1173523119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f826006c1f0 0x7f8260147f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:33.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.128+0000 7f825f7fe700 1 -- 172.21.15.157:0/1173523119 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8258009a20 con 0x7f826006cbb0 2023-10-06T04:09:33.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.128+0000 7f8264f6e700 1 --2- 172.21.15.157:0/1173523119 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8260150410 0x7f8260154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:09:33.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.128+0000 7f825f7fe700 1 --2- 172.21.15.157:0/1173523119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f826006cbb0 0x7f8260166fe0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f825400c7c0 tx=0x7f8254007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:33.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.128+0000 7f8247fff700 1 -- 172.21.15.157:0/1173523119 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8254005b30 con 0x7f826006cbb0 2023-10-06T04:09:33.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.129+0000 7f8247fff700 1 -- 172.21.15.157:0/1173523119 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8254005c90 con 0x7f826006cbb0 2023-10-06T04:09:33.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.129+0000 7f82669d1700 1 -- 172.21.15.157:0/1173523119 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8250009420 con 0x7f826006cbb0 2023-10-06T04:09:33.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.129+0000 7f8247fff700 1 -- 172.21.15.157:0/1173523119 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8254007340 con 0x7f826006cbb0 2023-10-06T04:09:33.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.129+0000 7f82669d1700 1 -- 172.21.15.157:0/1173523119 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8260147770 con 0x7f826006cbb0 2023-10-06T04:09:33.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.130+0000 7f82669d1700 1 -- 172.21.15.157:0/1173523119 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8260089ca0 con 0x7f826006cbb0 2023-10-06T04:09:33.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.130+0000 7f8247fff700 1 -- 172.21.15.157:0/1173523119 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f825401b400 con 0x7f826006cbb0 2023-10-06T04:09:33.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.130+0000 7f8247fff700 1 --2- 172.21.15.157:0/1173523119 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f824c066f30 0x7f824c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:33.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.130+0000 7f8247fff700 1 -- 172.21.15.157:0/1173523119 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6632+0+0 (secure 0 0 0) 0x7f8254084450 con 0x7f826006cbb0 2023-10-06T04:09:33.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.131+0000 7f825ffff700 1 --2- 172.21.15.157:0/1173523119 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f824c066f30 0x7f824c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:33.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.131+0000 7f825ffff700 1 --2- 172.21.15.157:0/1173523119 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f824c066f30 0x7f824c0693e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f82500091e0 tx=0x7f8250017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:33.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.135+0000 7f8247fff700 1 -- 172.21.15.157:0/1173523119 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82540513a0 con 0x7f826006cbb0 2023-10-06T04:09:33.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.359+0000 7f82669d1700 1 -- 172.21.15.157:0/1173523119 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82600c9130 con 0x7f826006cbb0 2023-10-06T04:09:33.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.360+0000 7f8247fff700 1 -- 172.21.15.157:0/1173523119 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+6148 (secure 0 0 0) 0x7f82540549c0 con 0x7f826006cbb0 2023-10-06T04:09:33.362 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:09:33.362 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6270,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1036901172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1036901172},{"type":"v1","addr":"172.21.15.157:6837","nonce":1036901172}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:09:16.934246+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:active","state_seq":19,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:09:33.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.362+0000 7f82669d1700 1 -- 172.21.15.157:0/1173523119 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f824c066f30 msgr2=0x7f824c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:33.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.362+0000 7f82669d1700 1 --2- 172.21.15.157:0/1173523119 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f824c066f30 0x7f824c0693e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f82500091e0 tx=0x7f8250017000 comp rx=0 tx=0).stop 2023-10-06T04:09:33.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.362+0000 7f82669d1700 1 -- 172.21.15.157:0/1173523119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f826006cbb0 msgr2=0x7f8260166fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:33.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.362+0000 7f82669d1700 1 --2- 172.21.15.157:0/1173523119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f826006cbb0 0x7f8260166fe0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f825400c7c0 tx=0x7f8254007e00 comp rx=0 tx=0).stop 2023-10-06T04:09:33.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.362+0000 7f82669d1700 1 -- 172.21.15.157:0/1173523119 shutdown_connections 2023-10-06T04:09:33.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.362+0000 7f82669d1700 1 --2- 172.21.15.157:0/1173523119 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f824c066f30 0x7f824c0693e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:33.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.362+0000 7f82669d1700 1 --2- 172.21.15.157:0/1173523119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f826006c1f0 0x7f8260147f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:33.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.362+0000 7f82669d1700 1 --2- 172.21.15.157:0/1173523119 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8260150410 0x7f8260154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:33.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.362+0000 7f82669d1700 1 --2- 172.21.15.157:0/1173523119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f826006cbb0 0x7f8260166fe0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:33.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.362+0000 7f82669d1700 1 -- 172.21.15.157:0/1173523119 >> 172.21.15.157:0/1173523119 conn(0x7f826006f230 msgr2=0x7f826006d530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:33.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.362+0000 7f82669d1700 1 -- 172.21.15.157:0/1173523119 shutdown_connections 2023-10-06T04:09:33.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:33.362+0000 7f82669d1700 1 -- 172.21.15.157:0/1173523119 wait complete. 2023-10-06T04:09:33.367 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 85 2023-10-06T04:09:33.377 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-06T04:09:35.377 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:09:35.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.503+0000 7f2e870b5700 1 Processor -- start 2023-10-06T04:09:35.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.503+0000 7f2e870b5700 1 -- start start 2023-10-06T04:09:35.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.504+0000 7f2e870b5700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2e800c00c0 0x7f2e800c29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:35.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.504+0000 7f2e870b5700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2e801547f0 0x7f2e8014d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:35.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.504+0000 7f2e870b5700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e80153be0 con 0x7f2e800631f0 2023-10-06T04:09:35.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.504+0000 7f2e870b5700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e800bf550 con 0x7f2e800c00c0 2023-10-06T04:09:35.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.504+0000 7f2e870b5700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e800c3630 con 0x7f2e801547f0 2023-10-06T04:09:35.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.504+0000 7f2e84e51700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2e800c00c0 0x7f2e800c29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:35.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.504+0000 7f2e84e51700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2e800c00c0 0x7f2e800c29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56536/0 (socket says 172.21.15.157:56536) 2023-10-06T04:09:35.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.504+0000 7f2e84e51700 1 -- 172.21.15.157:0/1469702237 learned_addr learned my addr 172.21.15.157:0/1469702237 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:35.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.504+0000 7f2e84e51700 1 -- 172.21.15.157:0/1469702237 >> v2:172.21.15.175:3300/0 conn(0x7f2e801547f0 msgr2=0x7f2e8014d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:35.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.504+0000 7f2e85652700 1 --2- 172.21.15.157:0/1469702237 >> v2:172.21.15.175:3300/0 conn(0x7f2e801547f0 0x7f2e8014d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:35.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.504+0000 7f2e84e51700 1 --2- 172.21.15.157:0/1469702237 >> v2:172.21.15.175:3300/0 conn(0x7f2e801547f0 0x7f2e8014d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:35.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.504+0000 7f2e84e51700 1 -- 172.21.15.157:0/1469702237 >> v1:172.21.15.175:6789/0 conn(0x7f2e800631f0 legacy=0x7f2e801540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:35.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.504+0000 7f2e84e51700 1 -- 172.21.15.157:0/1469702237 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e800c3910 con 0x7f2e800c00c0 2023-10-06T04:09:35.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.504+0000 7f2e84e51700 1 --2- 172.21.15.157:0/1469702237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e800c00c0 0x7f2e800c29f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2e8006e470 tx=0x7f2e74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae0b689c25fcc4d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:35.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.504+0000 7f2e7f7fe700 1 -- 172.21.15.157:0/1469702237 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7400b820 con 0x7f2e800c00c0 2023-10-06T04:09:35.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.505+0000 7f2e7f7fe700 1 -- 172.21.15.157:0/1469702237 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e7400b980 con 0x7f2e800c00c0 2023-10-06T04:09:35.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.505+0000 7f2e7f7fe700 1 -- 172.21.15.157:0/1469702237 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7400bd20 con 0x7f2e800c00c0 2023-10-06T04:09:35.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.505+0000 7f2e870b5700 1 -- 172.21.15.157:0/1469702237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e800c00c0 msgr2=0x7f2e800c29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:35.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.505+0000 7f2e870b5700 1 --2- 172.21.15.157:0/1469702237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e800c00c0 0x7f2e800c29f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2e8006e470 tx=0x7f2e74009250 comp rx=0 tx=0).stop 2023-10-06T04:09:35.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.505+0000 7f2e870b5700 1 -- 172.21.15.157:0/1469702237 shutdown_connections 2023-10-06T04:09:35.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.505+0000 7f2e870b5700 1 --2- 172.21.15.157:0/1469702237 >> v2:172.21.15.175:3300/0 conn(0x7f2e801547f0 0x7f2e8014d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:35.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.505+0000 7f2e870b5700 1 --2- 172.21.15.157:0/1469702237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e800c00c0 0x7f2e800c29f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:35.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.505+0000 7f2e870b5700 1 -- 172.21.15.157:0/1469702237 >> 172.21.15.157:0/1469702237 conn(0x7f2e800c8b20 msgr2=0x7f2e800c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:35.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.505+0000 7f2e870b5700 1 -- 172.21.15.157:0/1469702237 shutdown_connections 2023-10-06T04:09:35.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.505+0000 7f2e870b5700 1 -- 172.21.15.157:0/1469702237 wait complete. 2023-10-06T04:09:35.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.506+0000 7f2e870b5700 1 Processor -- start 2023-10-06T04:09:35.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.506+0000 7f2e870b5700 1 -- start start 2023-10-06T04:09:35.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.507+0000 7f2e870b5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e800631f0 0x7f2e80164b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:35.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.507+0000 7f2e870b5700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e800c00c0 0x7f2e800c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:35.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.507+0000 7f2e870b5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e801547f0 0x7f2e80155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:35.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.507+0000 7f2e870b5700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e801576e0 con 0x7f2e800631f0 2023-10-06T04:09:35.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.507+0000 7f2e870b5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e801680a0 con 0x7f2e801547f0 2023-10-06T04:09:35.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.507+0000 7f2e870b5700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e80168530 con 0x7f2e800c00c0 2023-10-06T04:09:35.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.507+0000 7f2e84e51700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e800631f0 0x7f2e80164b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:35.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.507+0000 7f2e84e51700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e800631f0 0x7f2e80164b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56540/0 (socket says 172.21.15.157:56540) 2023-10-06T04:09:35.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.507+0000 7f2e84e51700 1 -- 172.21.15.157:0/3309528874 learned_addr learned my addr 172.21.15.157:0/3309528874 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:35.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.507+0000 7f2e85652700 1 --2- 172.21.15.157:0/3309528874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e801547f0 0x7f2e80155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:35.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.507+0000 7f2e84e51700 1 -- 172.21.15.157:0/3309528874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e800c00c0 msgr2=0x7f2e800c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:35.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.507+0000 7f2e7ffff700 1 --2- 172.21.15.157:0/3309528874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e800c00c0 0x7f2e800c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:35.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.507+0000 7f2e84e51700 1 --2- 172.21.15.157:0/3309528874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e800c00c0 0x7f2e800c7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:35.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.507+0000 7f2e84e51700 1 -- 172.21.15.157:0/3309528874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e801547f0 msgr2=0x7f2e80155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:35.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.507+0000 7f2e84e51700 1 --2- 172.21.15.157:0/3309528874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e801547f0 0x7f2e80155750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:35.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.507+0000 7f2e84e51700 1 -- 172.21.15.157:0/3309528874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e740094f0 con 0x7f2e800631f0 2023-10-06T04:09:35.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.508+0000 7f2e84e51700 1 --2- 172.21.15.157:0/3309528874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e800631f0 0x7f2e80164b80 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2e80067b80 tx=0x7f2e74016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:35.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.508+0000 7f2e7cff9700 1 -- 172.21.15.157:0/3309528874 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e74011de0 con 0x7f2e800631f0 2023-10-06T04:09:35.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.508+0000 7f2e7cff9700 1 -- 172.21.15.157:0/3309528874 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e74020070 con 0x7f2e800631f0 2023-10-06T04:09:35.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.508+0000 7f2e870b5700 1 -- 172.21.15.157:0/3309528874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e8014fb30 con 0x7f2e800631f0 2023-10-06T04:09:35.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.508+0000 7f2e7cff9700 1 -- 172.21.15.157:0/3309528874 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7401b6a0 con 0x7f2e800631f0 2023-10-06T04:09:35.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.508+0000 7f2e870b5700 1 -- 172.21.15.157:0/3309528874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e80167770 con 0x7f2e800631f0 2023-10-06T04:09:35.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.509+0000 7f2e870b5700 1 -- 172.21.15.157:0/3309528874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e800642a0 con 0x7f2e800631f0 2023-10-06T04:09:35.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.509+0000 7f2e7cff9700 1 -- 172.21.15.157:0/3309528874 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2e74022030 con 0x7f2e800631f0 2023-10-06T04:09:35.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.509+0000 7f2e7cff9700 1 --2- 172.21.15.157:0/3309528874 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2e64066f60 0x7f2e64069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:35.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.510+0000 7f2e7cff9700 1 -- 172.21.15.157:0/3309528874 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6632+0+0 (secure 0 0 0) 0x7f2e74085ff0 con 0x7f2e800631f0 2023-10-06T04:09:35.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.510+0000 7f2e7ffff700 1 --2- 172.21.15.157:0/3309528874 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2e64066f60 0x7f2e64069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:35.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.510+0000 7f2e7ffff700 1 --2- 172.21.15.157:0/3309528874 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2e64066f60 0x7f2e64069410 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2e80161ce0 tx=0x7f2e6c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:35.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.514+0000 7f2e7cff9700 1 -- 172.21.15.157:0/3309528874 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e74052ec0 con 0x7f2e800631f0 2023-10-06T04:09:35.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.742+0000 7f2e870b5700 1 -- 172.21.15.157:0/3309528874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e80151cd0 con 0x7f2e800631f0 2023-10-06T04:09:35.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.742+0000 7f2e7cff9700 1 -- 172.21.15.157:0/3309528874 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+6148 (secure 0 0 0) 0x7f2e740564e0 con 0x7f2e800631f0 2023-10-06T04:09:35.744 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:09:35.744 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6270,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1036901172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1036901172},{"type":"v1","addr":"172.21.15.157:6837","nonce":1036901172}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:09:16.934246+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:active","state_seq":19,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:09:35.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.744+0000 7f2e870b5700 1 -- 172.21.15.157:0/3309528874 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2e64066f60 msgr2=0x7f2e64069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:35.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.744+0000 7f2e870b5700 1 --2- 172.21.15.157:0/3309528874 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2e64066f60 0x7f2e64069410 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2e80161ce0 tx=0x7f2e6c008040 comp rx=0 tx=0).stop 2023-10-06T04:09:35.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.744+0000 7f2e870b5700 1 -- 172.21.15.157:0/3309528874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e800631f0 msgr2=0x7f2e80164b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:35.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.744+0000 7f2e870b5700 1 --2- 172.21.15.157:0/3309528874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e800631f0 0x7f2e80164b80 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2e80067b80 tx=0x7f2e74016000 comp rx=0 tx=0).stop 2023-10-06T04:09:35.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.744+0000 7f2e870b5700 1 -- 172.21.15.157:0/3309528874 shutdown_connections 2023-10-06T04:09:35.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.744+0000 7f2e870b5700 1 --2- 172.21.15.157:0/3309528874 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2e64066f60 0x7f2e64069410 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:35.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.744+0000 7f2e870b5700 1 --2- 172.21.15.157:0/3309528874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e801547f0 0x7f2e80155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:35.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.744+0000 7f2e870b5700 1 --2- 172.21.15.157:0/3309528874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e800631f0 0x7f2e80164b80 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:35.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.744+0000 7f2e870b5700 1 --2- 172.21.15.157:0/3309528874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e800c00c0 0x7f2e800c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:35.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.744+0000 7f2e870b5700 1 -- 172.21.15.157:0/3309528874 >> 172.21.15.157:0/3309528874 conn(0x7f2e800c8b20 msgr2=0x7f2e800c9190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:35.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.745+0000 7f2e870b5700 1 -- 172.21.15.157:0/3309528874 shutdown_connections 2023-10-06T04:09:35.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:35.745+0000 7f2e870b5700 1 -- 172.21.15.157:0/3309528874 wait complete. 2023-10-06T04:09:35.747 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 85 2023-10-06T04:09:35.759 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-06T04:09:37.760 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:09:37.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.887+0000 7fb9e451b700 1 Processor -- start 2023-10-06T04:09:37.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.887+0000 7fb9e451b700 1 -- start start 2023-10-06T04:09:37.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.888+0000 7fb9e451b700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb9dc06dc40 0x7fb9dc14fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:37.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.888+0000 7fb9e451b700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9dc0c9780 con 0x7fb9dc06cbb0 2023-10-06T04:09:37.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.888+0000 7fb9e451b700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9dc06e0c0 con 0x7fb9dc06c1f0 2023-10-06T04:09:37.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.888+0000 7fb9e451b700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9dc0c8c40 con 0x7fb9dc06dc40 2023-10-06T04:09:37.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.888+0000 7fb9e22b7700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb9dc06cbb0 0x7fb9dc06d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:53134/0 (socket says 172.21.15.157:53134) 2023-10-06T04:09:37.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.888+0000 7fb9e22b7700 1 -- 172.21.15.157:0/2954937384 learned_addr learned my addr 172.21.15.157:0/2954937384 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:09:37.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.888+0000 7fb9e1ab6700 1 --2- 172.21.15.157:0/2954937384 >> v2:172.21.15.175:3300/0 conn(0x7fb9dc06dc40 0x7fb9dc14fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:37.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.888+0000 7fb9e12b5700 1 -- 172.21.15.157:0/2954937384 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3434764118 0 0) 0x7fb9dc0c9780 con 0x7fb9dc06cbb0 2023-10-06T04:09:37.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.888+0000 7fb9e12b5700 1 -- 172.21.15.157:0/2954937384 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9c4003570 con 0x7fb9dc06cbb0 2023-10-06T04:09:37.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.888+0000 7fb9e12b5700 1 -- 172.21.15.157:0/2954937384 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1389977000 0 0) 0x7fb9dc06e0c0 con 0x7fb9dc06c1f0 2023-10-06T04:09:37.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.888+0000 7fb9e12b5700 1 -- 172.21.15.157:0/2954937384 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9dc0c9780 con 0x7fb9dc06c1f0 2023-10-06T04:09:37.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.888+0000 7fb9e12b5700 1 -- 172.21.15.157:0/2954937384 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1935480138 0 0) 0x7fb9c4003570 con 0x7fb9dc06cbb0 2023-10-06T04:09:37.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.888+0000 7fb9e12b5700 1 -- 172.21.15.157:0/2954937384 >> v2:172.21.15.175:3300/0 conn(0x7fb9dc06dc40 msgr2=0x7fb9dc14fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:37.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.888+0000 7fb9e12b5700 1 --2- 172.21.15.157:0/2954937384 >> v2:172.21.15.175:3300/0 conn(0x7fb9dc06dc40 0x7fb9dc14fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:37.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.888+0000 7fb9e12b5700 1 -- 172.21.15.157:0/2954937384 >> v1:172.21.15.175:6789/0 conn(0x7fb9dc06c1f0 legacy=0x7fb9dc155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:37.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.888+0000 7fb9e12b5700 1 -- 172.21.15.157:0/2954937384 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9dc0c8e50 con 0x7fb9dc06cbb0 2023-10-06T04:09:37.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.888+0000 7fb9e12b5700 1 -- 172.21.15.157:0/2954937384 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7fb9cc0017f0 con 0x7fb9dc06cbb0 2023-10-06T04:09:37.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.888+0000 7fb9e12b5700 1 -- 172.21.15.157:0/2954937384 >> v1:172.21.15.157:6789/0 conn(0x7fb9dc06cbb0 legacy=0x7fb9dc06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:37.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.888+0000 7fb9e12b5700 1 --2- 172.21.15.157:0/2954937384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9c4003bf0 0x7fb9c4003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:37.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.888+0000 7fb9e12b5700 1 --2- 172.21.15.157:0/2954937384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9c4004810 0x7fb9c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:37.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.888+0000 7fb9e12b5700 1 --2- 172.21.15.157:0/2954937384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9c40051c0 0x7fb9c400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:37.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.888+0000 7fb9e12b5700 1 -- 172.21.15.157:0/2954937384 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9dc0c8c40 con 0x7fb9c4004810 2023-10-06T04:09:37.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.888+0000 7fb9e12b5700 1 -- 172.21.15.157:0/2954937384 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c400c0b0 con 0x7fb9c4003bf0 2023-10-06T04:09:37.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.888+0000 7fb9e12b5700 1 -- 172.21.15.157:0/2954937384 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c400c270 con 0x7fb9c40051c0 2023-10-06T04:09:37.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.889+0000 7fb9e1ab6700 1 --2- 172.21.15.157:0/2954937384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9c4004810 0x7fb9c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:37.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.889+0000 7fb9e22b7700 1 --2- 172.21.15.157:0/2954937384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9c4003bf0 0x7fb9c4003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:37.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.889+0000 7fb9e2ab8700 1 --2- 172.21.15.157:0/2954937384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9c40051c0 0x7fb9c400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:37.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.889+0000 7fb9e1ab6700 1 -- 172.21.15.157:0/2954937384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9c40051c0 msgr2=0x7fb9c400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:37.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.889+0000 7fb9e1ab6700 1 --2- 172.21.15.157:0/2954937384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9c40051c0 0x7fb9c400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:37.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.889+0000 7fb9e1ab6700 1 -- 172.21.15.157:0/2954937384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9c4003bf0 msgr2=0x7fb9c4003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:37.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.889+0000 7fb9e1ab6700 1 --2- 172.21.15.157:0/2954937384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9c4003bf0 0x7fb9c4003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:37.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.889+0000 7fb9e1ab6700 1 -- 172.21.15.157:0/2954937384 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c4003570 con 0x7fb9c4004810 2023-10-06T04:09:37.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.889+0000 7fb9e2ab8700 1 -- 172.21.15.157:0/2954937384 reap_dead start 2023-10-06T04:09:37.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.889+0000 7fb9e1ab6700 1 --2- 172.21.15.157:0/2954937384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9c4004810 0x7fb9c4004c80 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0009c00 tx=0x7fb9d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58089bc3da33b38e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:37.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.889+0000 7fb9e12b5700 1 -- 172.21.15.157:0/2954937384 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d000e710 con 0x7fb9c4004810 2023-10-06T04:09:37.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.889+0000 7fb9e12b5700 1 -- 172.21.15.157:0/2954937384 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d000e870 con 0x7fb9c4004810 2023-10-06T04:09:37.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.889+0000 7fb9e12b5700 1 -- 172.21.15.157:0/2954937384 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d000ec30 con 0x7fb9c4004810 2023-10-06T04:09:37.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.890+0000 7fb9e451b700 1 -- 172.21.15.157:0/2954937384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9c4004810 msgr2=0x7fb9c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:37.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.890+0000 7fb9e451b700 1 --2- 172.21.15.157:0/2954937384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9c4004810 0x7fb9c4004c80 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0009c00 tx=0x7fb9d00091a0 comp rx=0 tx=0).stop 2023-10-06T04:09:37.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.890+0000 7fb9e451b700 1 -- 172.21.15.157:0/2954937384 shutdown_connections 2023-10-06T04:09:37.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.890+0000 7fb9e451b700 1 --2- 172.21.15.157:0/2954937384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9c4004810 0x7fb9c4004c80 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:37.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.890+0000 7fb9e451b700 1 -- 172.21.15.157:0/2954937384 >> 172.21.15.157:0/2954937384 conn(0x7fb9dc06f230 msgr2=0x7fb9dc06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:37.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.890+0000 7fb9e451b700 1 -- 172.21.15.157:0/2954937384 shutdown_connections 2023-10-06T04:09:37.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.890+0000 7fb9e451b700 1 -- 172.21.15.157:0/2954937384 wait complete. 2023-10-06T04:09:37.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.891+0000 7fb9e451b700 1 Processor -- start 2023-10-06T04:09:37.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.891+0000 7fb9e451b700 1 -- start start 2023-10-06T04:09:37.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.891+0000 7fb9e451b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9c4004810 0x7fb9dc0bfe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:37.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.891+0000 7fb9e451b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9dc14c210 0x7fb9dc167000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:37.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.891+0000 7fb9e451b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9dc167540 0x7fb9dc14b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:37.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.892+0000 7fb9e451b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9dc0c1a10 con 0x7fb9dc14c210 2023-10-06T04:09:37.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.892+0000 7fb9e451b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9dc157a20 con 0x7fb9c4004810 2023-10-06T04:09:37.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.892+0000 7fb9e451b700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9dc166b70 con 0x7fb9dc167540 2023-10-06T04:09:37.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.892+0000 7fb9e1ab6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9dc14c210 0x7fb9dc167000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:37.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.892+0000 7fb9e1ab6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9dc14c210 0x7fb9dc167000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56566/0 (socket says 172.21.15.157:56566) 2023-10-06T04:09:37.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.892+0000 7fb9e1ab6700 1 -- 172.21.15.157:0/3419347975 learned_addr learned my addr 172.21.15.157:0/3419347975 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:37.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.892+0000 7fb9e22b7700 1 --2- 172.21.15.157:0/3419347975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9c4004810 0x7fb9dc0bfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:37.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.892+0000 7fb9e2ab8700 1 --2- 172.21.15.157:0/3419347975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9dc167540 0x7fb9dc14b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:37.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.892+0000 7fb9e1ab6700 1 -- 172.21.15.157:0/3419347975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9dc167540 msgr2=0x7fb9dc14b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:37.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.892+0000 7fb9e1ab6700 1 --2- 172.21.15.157:0/3419347975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9dc167540 0x7fb9dc14b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:37.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.892+0000 7fb9e1ab6700 1 -- 172.21.15.157:0/3419347975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9c4004810 msgr2=0x7fb9dc0bfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:37.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.892+0000 7fb9e1ab6700 1 --2- 172.21.15.157:0/3419347975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9c4004810 0x7fb9dc0bfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:37.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.892+0000 7fb9e1ab6700 1 -- 172.21.15.157:0/3419347975 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9dc167990 con 0x7fb9dc14c210 2023-10-06T04:09:37.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.892+0000 7fb9e22b7700 1 --2- 172.21.15.157:0/3419347975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9c4004810 0x7fb9dc0bfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:09:37.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.892+0000 7fb9e2ab8700 1 --2- 172.21.15.157:0/3419347975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9dc167540 0x7fb9dc14b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:09:37.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.892+0000 7fb9e1ab6700 1 --2- 172.21.15.157:0/3419347975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9dc14c210 0x7fb9dc167000 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000e4b0 tx=0x7fb9d001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:37.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.892+0000 7fb9da7fc700 1 -- 172.21.15.157:0/3419347975 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0022070 con 0x7fb9dc14c210 2023-10-06T04:09:37.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.892+0000 7fb9da7fc700 1 -- 172.21.15.157:0/3419347975 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d000e9e0 con 0x7fb9dc14c210 2023-10-06T04:09:37.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.892+0000 7fb9da7fc700 1 -- 172.21.15.157:0/3419347975 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0027470 con 0x7fb9dc14c210 2023-10-06T04:09:37.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.892+0000 7fb9e451b700 1 -- 172.21.15.157:0/3419347975 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9dc157c50 con 0x7fb9dc14c210 2023-10-06T04:09:37.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.892+0000 7fb9e451b700 1 -- 172.21.15.157:0/3419347975 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9dc153090 con 0x7fb9dc14c210 2023-10-06T04:09:37.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.893+0000 7fb9e451b700 1 -- 172.21.15.157:0/3419347975 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9dc089ca0 con 0x7fb9dc14c210 2023-10-06T04:09:37.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.894+0000 7fb9da7fc700 1 -- 172.21.15.157:0/3419347975 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb9d001c450 con 0x7fb9dc14c210 2023-10-06T04:09:37.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.894+0000 7fb9da7fc700 1 --2- 172.21.15.157:0/3419347975 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb9c8066f60 0x7fb9c8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:37.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.894+0000 7fb9da7fc700 1 -- 172.21.15.157:0/3419347975 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6632+0+0 (secure 0 0 0) 0x7fb9d0086300 con 0x7fb9dc14c210 2023-10-06T04:09:37.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.894+0000 7fb9e22b7700 1 --2- 172.21.15.157:0/3419347975 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb9c8066f60 0x7fb9c8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:37.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.895+0000 7fb9e22b7700 1 --2- 172.21.15.157:0/3419347975 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb9c8066f60 0x7fb9c8069410 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc004fd0 tx=0x7fb9cc006040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:37.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:37.898+0000 7fb9da7fc700 1 -- 172.21.15.157:0/3419347975 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb9d00531d0 con 0x7fb9dc14c210 2023-10-06T04:09:38.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:38.125+0000 7fb9e451b700 1 -- 172.21.15.157:0/3419347975 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9dc166cb0 con 0x7fb9dc14c210 2023-10-06T04:09:38.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:38.126+0000 7fb9da7fc700 1 -- 172.21.15.157:0/3419347975 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+6148 (secure 0 0 0) 0x7fb9d00567f0 con 0x7fb9dc14c210 2023-10-06T04:09:38.127 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:09:38.127 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6270,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1036901172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1036901172},{"type":"v1","addr":"172.21.15.157:6837","nonce":1036901172}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:09:16.934246+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:active","state_seq":19,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:09:38.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:38.128+0000 7fb9e451b700 1 -- 172.21.15.157:0/3419347975 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb9c8066f60 msgr2=0x7fb9c8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:38.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:38.128+0000 7fb9e451b700 1 --2- 172.21.15.157:0/3419347975 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb9c8066f60 0x7fb9c8069410 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc004fd0 tx=0x7fb9cc006040 comp rx=0 tx=0).stop 2023-10-06T04:09:38.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:38.128+0000 7fb9e451b700 1 -- 172.21.15.157:0/3419347975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9dc14c210 msgr2=0x7fb9dc167000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:38.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:38.128+0000 7fb9e451b700 1 --2- 172.21.15.157:0/3419347975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9dc14c210 0x7fb9dc167000 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000e4b0 tx=0x7fb9d001a000 comp rx=0 tx=0).stop 2023-10-06T04:09:38.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:38.128+0000 7fb9e451b700 1 -- 172.21.15.157:0/3419347975 shutdown_connections 2023-10-06T04:09:38.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:38.128+0000 7fb9e451b700 1 --2- 172.21.15.157:0/3419347975 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb9c8066f60 0x7fb9c8069410 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:38.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:38.128+0000 7fb9e451b700 1 --2- 172.21.15.157:0/3419347975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9c4004810 0x7fb9dc0bfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:38.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:38.128+0000 7fb9e451b700 1 --2- 172.21.15.157:0/3419347975 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9dc167540 0x7fb9dc14b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:38.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:38.128+0000 7fb9e451b700 1 --2- 172.21.15.157:0/3419347975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9dc14c210 0x7fb9dc167000 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:38.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:38.128+0000 7fb9e451b700 1 -- 172.21.15.157:0/3419347975 >> 172.21.15.157:0/3419347975 conn(0x7fb9dc06f230 msgr2=0x7fb9dc0af240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:38.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:38.128+0000 7fb9e451b700 1 -- 172.21.15.157:0/3419347975 shutdown_connections 2023-10-06T04:09:38.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:38.128+0000 7fb9e451b700 1 -- 172.21.15.157:0/3419347975 wait complete. 2023-10-06T04:09:38.131 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 85 2023-10-06T04:09:38.142 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-06T04:09:40.143 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:09:40.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.268+0000 7f870eb92700 1 Processor -- start 2023-10-06T04:09:40.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.268+0000 7f870eb92700 1 -- start start 2023-10-06T04:09:40.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.269+0000 7f870eb92700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f870806cbb0 0x7f870806d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:40.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.269+0000 7f870eb92700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8708150410 0x7f870806da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:40.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.269+0000 7f870eb92700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87080c9780 con 0x7f870806c1f0 2023-10-06T04:09:40.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.269+0000 7f870eb92700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87080c8c40 con 0x7f870806cbb0 2023-10-06T04:09:40.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.269+0000 7f870eb92700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f870806e0c0 con 0x7f8708150410 2023-10-06T04:09:40.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.269+0000 7f870c92e700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f870806cbb0 0x7f870806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:40.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.270+0000 7f86fffff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f870806c1f0 0x7f870814fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:53136/0 (socket says 172.21.15.157:53136) 2023-10-06T04:09:40.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.270+0000 7f86fffff700 1 -- 172.21.15.157:0/1774580030 learned_addr learned my addr 172.21.15.157:0/1774580030 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:09:40.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.270+0000 7f870c92e700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f870806cbb0 0x7f870806d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56580/0 (socket says 172.21.15.157:56580) 2023-10-06T04:09:40.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.270+0000 7f870d12f700 1 --2- 172.21.15.157:0/1774580030 >> v2:172.21.15.175:3300/0 conn(0x7f8708150410 0x7f870806da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:40.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.270+0000 7f870c92e700 1 -- 172.21.15.157:0/1774580030 >> v2:172.21.15.175:3300/0 conn(0x7f8708150410 msgr2=0x7f870806da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:40.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.270+0000 7f870c92e700 1 --2- 172.21.15.157:0/1774580030 >> v2:172.21.15.175:3300/0 conn(0x7f8708150410 0x7f870806da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:40.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.270+0000 7f870c92e700 1 -- 172.21.15.157:0/1774580030 >> v1:172.21.15.157:6789/0 conn(0x7f870806c1f0 legacy=0x7f870814fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:40.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.270+0000 7f870c92e700 1 -- 172.21.15.157:0/1774580030 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87080c8e50 con 0x7f870806cbb0 2023-10-06T04:09:40.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.270+0000 7f86ff7fe700 1 -- 172.21.15.157:0/1774580030 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1647638998 0 0) 0x7f87080c9780 con 0x7f870806c1f0 2023-10-06T04:09:40.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.270+0000 7f870c92e700 1 --2- 172.21.15.157:0/1774580030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f870806cbb0 0x7f870806d530 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f87080b1b50 tx=0x7f87040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f872a0dc799c13a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:40.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.270+0000 7f86ff7fe700 1 -- 172.21.15.157:0/1774580030 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870400b690 con 0x7f870806cbb0 2023-10-06T04:09:40.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.270+0000 7f86ff7fe700 1 -- 172.21.15.157:0/1774580030 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870400b7f0 con 0x7f870806cbb0 2023-10-06T04:09:40.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.270+0000 7f86ff7fe700 1 -- 172.21.15.157:0/1774580030 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870400bb90 con 0x7f870806cbb0 2023-10-06T04:09:40.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.271+0000 7f870eb92700 1 -- 172.21.15.157:0/1774580030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f870806cbb0 msgr2=0x7f870806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:40.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.271+0000 7f870eb92700 1 --2- 172.21.15.157:0/1774580030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f870806cbb0 0x7f870806d530 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f87080b1b50 tx=0x7f87040091a0 comp rx=0 tx=0).stop 2023-10-06T04:09:40.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.271+0000 7f870eb92700 1 -- 172.21.15.157:0/1774580030 shutdown_connections 2023-10-06T04:09:40.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.271+0000 7f870eb92700 1 --2- 172.21.15.157:0/1774580030 >> v2:172.21.15.175:3300/0 conn(0x7f8708150410 0x7f870806da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:40.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.271+0000 7f870eb92700 1 --2- 172.21.15.157:0/1774580030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f870806cbb0 0x7f870806d530 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:40.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.271+0000 7f870eb92700 1 -- 172.21.15.157:0/1774580030 >> 172.21.15.157:0/1774580030 conn(0x7f870806f230 msgr2=0x7f870806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:40.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.271+0000 7f870eb92700 1 -- 172.21.15.157:0/1774580030 shutdown_connections 2023-10-06T04:09:40.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.271+0000 7f870eb92700 1 -- 172.21.15.157:0/1774580030 wait complete. 2023-10-06T04:09:40.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.272+0000 7f870eb92700 1 Processor -- start 2023-10-06T04:09:40.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.272+0000 7f870eb92700 1 -- start start 2023-10-06T04:09:40.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.273+0000 7f870eb92700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f870806c1f0 0x7f8708146f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:40.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.273+0000 7f870eb92700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f870806cbb0 0x7f8708148070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:40.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.273+0000 7f870eb92700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8708150410 0x7f87081485b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:40.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.273+0000 7f870eb92700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8708151c30 con 0x7f8708150410 2023-10-06T04:09:40.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.273+0000 7f870eb92700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87081664b0 con 0x7f870806cbb0 2023-10-06T04:09:40.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.273+0000 7f870eb92700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87081666c0 con 0x7f870806c1f0 2023-10-06T04:09:40.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.273+0000 7f870d12f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8708150410 0x7f87081485b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:40.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.273+0000 7f870d12f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8708150410 0x7f87081485b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56590/0 (socket says 172.21.15.157:56590) 2023-10-06T04:09:40.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.273+0000 7f870d12f700 1 -- 172.21.15.157:0/2769104511 learned_addr learned my addr 172.21.15.157:0/2769104511 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:40.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.273+0000 7f86fffff700 1 --2- 172.21.15.157:0/2769104511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f870806cbb0 0x7f8708148070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:40.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.273+0000 7f870c92e700 1 --2- 172.21.15.157:0/2769104511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f870806c1f0 0x7f8708146f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:40.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.273+0000 7f870d12f700 1 -- 172.21.15.157:0/2769104511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f870806c1f0 msgr2=0x7f8708146f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:40.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.273+0000 7f870d12f700 1 --2- 172.21.15.157:0/2769104511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f870806c1f0 0x7f8708146f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:40.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.273+0000 7f870d12f700 1 -- 172.21.15.157:0/2769104511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f870806cbb0 msgr2=0x7f8708148070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:40.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.273+0000 7f870d12f700 1 --2- 172.21.15.157:0/2769104511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f870806cbb0 0x7f8708148070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:40.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.273+0000 7f870d12f700 1 -- 172.21.15.157:0/2769104511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8704009420 con 0x7f8708150410 2023-10-06T04:09:40.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.273+0000 7f870d12f700 1 --2- 172.21.15.157:0/2769104511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8708150410 0x7f87081485b0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f8700009df0 tx=0x7f870000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:40.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.274+0000 7f86fcff9700 1 -- 172.21.15.157:0/2769104511 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870000cb80 con 0x7f8708150410 2023-10-06T04:09:40.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.274+0000 7f86fcff9700 1 -- 172.21.15.157:0/2769104511 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870000f040 con 0x7f8708150410 2023-10-06T04:09:40.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.274+0000 7f870eb92700 1 -- 172.21.15.157:0/2769104511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87081669a0 con 0x7f8708150410 2023-10-06T04:09:40.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.274+0000 7f86fcff9700 1 -- 172.21.15.157:0/2769104511 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8700014480 con 0x7f8708150410 2023-10-06T04:09:40.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.274+0000 7f870eb92700 1 -- 172.21.15.157:0/2769104511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8708166f60 con 0x7f8708150410 2023-10-06T04:09:40.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.275+0000 7f870eb92700 1 -- 172.21.15.157:0/2769104511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8708089ca0 con 0x7f8708150410 2023-10-06T04:09:40.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.275+0000 7f86fcff9700 1 -- 172.21.15.157:0/2769104511 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8700014830 con 0x7f8708150410 2023-10-06T04:09:40.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.275+0000 7f86fcff9700 1 --2- 172.21.15.157:0/2769104511 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f86f4066f30 0x7f86f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:40.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.276+0000 7f86fcff9700 1 -- 172.21.15.157:0/2769104511 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6632+0+0 (secure 0 0 0) 0x7f87000852b0 con 0x7f8708150410 2023-10-06T04:09:40.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.276+0000 7f870c92e700 1 --2- 172.21.15.157:0/2769104511 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f86f4066f30 0x7f86f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:40.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.276+0000 7f870c92e700 1 --2- 172.21.15.157:0/2769104511 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f86f4066f30 0x7f86f40693e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f87040093f0 tx=0x7f8704017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:40.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.280+0000 7f86fcff9700 1 -- 172.21.15.157:0/2769104511 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8700052180 con 0x7f8708150410 2023-10-06T04:09:40.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.502+0000 7f870eb92700 1 -- 172.21.15.157:0/2769104511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87080c9130 con 0x7f8708150410 2023-10-06T04:09:40.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.503+0000 7f86fcff9700 1 -- 172.21.15.157:0/2769104511 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+6148 (secure 0 0 0) 0x7f87000557a0 con 0x7f8708150410 2023-10-06T04:09:40.505 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:09:40.505 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6270,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1036901172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1036901172},{"type":"v1","addr":"172.21.15.157:6837","nonce":1036901172}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:09:16.934246+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:active","state_seq":19,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:09:40.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.504+0000 7f870eb92700 1 -- 172.21.15.157:0/2769104511 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f86f4066f30 msgr2=0x7f86f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:40.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.504+0000 7f870eb92700 1 --2- 172.21.15.157:0/2769104511 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f86f4066f30 0x7f86f40693e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f87040093f0 tx=0x7f8704017000 comp rx=0 tx=0).stop 2023-10-06T04:09:40.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.504+0000 7f870eb92700 1 -- 172.21.15.157:0/2769104511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8708150410 msgr2=0x7f87081485b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:40.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.504+0000 7f870eb92700 1 --2- 172.21.15.157:0/2769104511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8708150410 0x7f87081485b0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f8700009df0 tx=0x7f870000b040 comp rx=0 tx=0).stop 2023-10-06T04:09:40.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.505+0000 7f870eb92700 1 -- 172.21.15.157:0/2769104511 shutdown_connections 2023-10-06T04:09:40.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.505+0000 7f870eb92700 1 --2- 172.21.15.157:0/2769104511 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f86f4066f30 0x7f86f40693e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:40.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.505+0000 7f870eb92700 1 --2- 172.21.15.157:0/2769104511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8708150410 0x7f87081485b0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:40.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.505+0000 7f870eb92700 1 --2- 172.21.15.157:0/2769104511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f870806c1f0 0x7f8708146f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:40.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.505+0000 7f870eb92700 1 --2- 172.21.15.157:0/2769104511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f870806cbb0 0x7f8708148070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:40.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.505+0000 7f870eb92700 1 -- 172.21.15.157:0/2769104511 >> 172.21.15.157:0/2769104511 conn(0x7f870806f230 msgr2=0x7f870814d780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:40.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.505+0000 7f870eb92700 1 -- 172.21.15.157:0/2769104511 shutdown_connections 2023-10-06T04:09:40.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:40.505+0000 7f870eb92700 1 -- 172.21.15.157:0/2769104511 wait complete. 2023-10-06T04:09:40.508 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 85 2023-10-06T04:09:40.520 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-06T04:09:42.520 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:09:42.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.647+0000 7fb23eefa700 1 Processor -- start 2023-10-06T04:09:42.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.647+0000 7fb23eefa700 1 -- start start 2023-10-06T04:09:42.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.648+0000 7fb23eefa700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb2380b04d0 0x7fb238157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:42.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.648+0000 7fb23eefa700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb23806c2d0 0x7fb23806dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:42.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.648+0000 7fb23eefa700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb23806d8d0 con 0x7fb2380b1e40 2023-10-06T04:09:42.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.648+0000 7fb23eefa700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2380af8f0 con 0x7fb2380b04d0 2023-10-06T04:09:42.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.648+0000 7fb23eefa700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb23815a690 con 0x7fb23806c2d0 2023-10-06T04:09:42.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.648+0000 7fb23d6f7700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb2380b04d0 0x7fb238157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:42.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.648+0000 7fb23d6f7700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb2380b04d0 0x7fb238157750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56596/0 (socket says 172.21.15.157:56596) 2023-10-06T04:09:42.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.648+0000 7fb23d6f7700 1 -- 172.21.15.157:0/732642992 learned_addr learned my addr 172.21.15.157:0/732642992 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:42.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.648+0000 7fb23e6f9700 1 --2- 172.21.15.157:0/732642992 >> v2:172.21.15.175:3300/0 conn(0x7fb23806c2d0 0x7fb23806dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:42.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.649+0000 7fb23d6f7700 1 -- 172.21.15.157:0/732642992 >> v2:172.21.15.175:3300/0 conn(0x7fb23806c2d0 msgr2=0x7fb23806dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:42.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.649+0000 7fb23d6f7700 1 --2- 172.21.15.157:0/732642992 >> v2:172.21.15.175:3300/0 conn(0x7fb23806c2d0 0x7fb23806dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:42.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.649+0000 7fb23d6f7700 1 -- 172.21.15.157:0/732642992 >> v1:172.21.15.175:6789/0 conn(0x7fb2380b1e40 legacy=0x7fb2381563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:42.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.649+0000 7fb23d6f7700 1 -- 172.21.15.157:0/732642992 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23806ccc0 con 0x7fb2380b04d0 2023-10-06T04:09:42.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.649+0000 7fb23e6f9700 1 --2- 172.21.15.157:0/732642992 >> v2:172.21.15.175:3300/0 conn(0x7fb23806c2d0 0x7fb23806dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:09:42.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.649+0000 7fb23d6f7700 1 --2- 172.21.15.157:0/732642992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2380b04d0 0x7fb238157750 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fb2380c34e0 tx=0x7fb228009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc78feffa5934a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:42.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.649+0000 7fb23cef6700 1 -- 172.21.15.157:0/732642992 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22800b860 con 0x7fb2380b04d0 2023-10-06T04:09:42.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.649+0000 7fb23cef6700 1 -- 172.21.15.157:0/732642992 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22800b9c0 con 0x7fb2380b04d0 2023-10-06T04:09:42.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.649+0000 7fb23cef6700 1 -- 172.21.15.157:0/732642992 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22800bd80 con 0x7fb2380b04d0 2023-10-06T04:09:42.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.649+0000 7fb23eefa700 1 -- 172.21.15.157:0/732642992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2380b04d0 msgr2=0x7fb238157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:42.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.649+0000 7fb23eefa700 1 --2- 172.21.15.157:0/732642992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2380b04d0 0x7fb238157750 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fb2380c34e0 tx=0x7fb228009250 comp rx=0 tx=0).stop 2023-10-06T04:09:42.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.649+0000 7fb23eefa700 1 -- 172.21.15.157:0/732642992 shutdown_connections 2023-10-06T04:09:42.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.649+0000 7fb23eefa700 1 --2- 172.21.15.157:0/732642992 >> v2:172.21.15.175:3300/0 conn(0x7fb23806c2d0 0x7fb23806dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:42.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.649+0000 7fb23eefa700 1 --2- 172.21.15.157:0/732642992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2380b04d0 0x7fb238157750 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:42.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.649+0000 7fb23eefa700 1 -- 172.21.15.157:0/732642992 >> 172.21.15.157:0/732642992 conn(0x7fb2380cc540 msgr2=0x7fb2380cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:42.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.649+0000 7fb23eefa700 1 -- 172.21.15.157:0/732642992 shutdown_connections 2023-10-06T04:09:42.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.649+0000 7fb23eefa700 1 -- 172.21.15.157:0/732642992 wait complete. 2023-10-06T04:09:42.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.650+0000 7fb23eefa700 1 Processor -- start 2023-10-06T04:09:42.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.651+0000 7fb23eefa700 1 -- start start 2023-10-06T04:09:42.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.651+0000 7fb23eefa700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb23806c2d0 0x7fb2381689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:42.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.651+0000 7fb23eefa700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2380b04d0 0x7fb238152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:42.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.651+0000 7fb23eefa700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2380b1e40 0x7fb238152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:42.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.651+0000 7fb23eefa700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb238151db0 con 0x7fb2380b1e40 2023-10-06T04:09:42.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.651+0000 7fb23eefa700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb238152090 con 0x7fb23806c2d0 2023-10-06T04:09:42.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.651+0000 7fb23eefa700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2380caef0 con 0x7fb2380b04d0 2023-10-06T04:09:42.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.651+0000 7fb23e6f9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2380b1e40 0x7fb238152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:42.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.651+0000 7fb23e6f9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2380b1e40 0x7fb238152a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56598/0 (socket says 172.21.15.157:56598) 2023-10-06T04:09:42.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.651+0000 7fb23e6f9700 1 -- 172.21.15.157:0/956958744 learned_addr learned my addr 172.21.15.157:0/956958744 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:42.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.651+0000 7fb23d6f7700 1 --2- 172.21.15.157:0/956958744 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2380b04d0 0x7fb238152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:42.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.651+0000 7fb23def8700 1 --2- 172.21.15.157:0/956958744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb23806c2d0 0x7fb2381689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:42.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.652+0000 7fb23e6f9700 1 -- 172.21.15.157:0/956958744 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2380b04d0 msgr2=0x7fb238152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:42.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.652+0000 7fb23e6f9700 1 --2- 172.21.15.157:0/956958744 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2380b04d0 0x7fb238152520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:42.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.652+0000 7fb23e6f9700 1 -- 172.21.15.157:0/956958744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb23806c2d0 msgr2=0x7fb2381689b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:42.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.652+0000 7fb23e6f9700 1 --2- 172.21.15.157:0/956958744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb23806c2d0 0x7fb2381689b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:42.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.652+0000 7fb23e6f9700 1 -- 172.21.15.157:0/956958744 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2280094f0 con 0x7fb2380b1e40 2023-10-06T04:09:42.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.652+0000 7fb23e6f9700 1 --2- 172.21.15.157:0/956958744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2380b1e40 0x7fb238152a60 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fb23000d100 tx=0x7fb2300099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:42.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.652+0000 7fb22dffb700 1 -- 172.21.15.157:0/956958744 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23000dc40 con 0x7fb2380b1e40 2023-10-06T04:09:42.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.652+0000 7fb22dffb700 1 -- 172.21.15.157:0/956958744 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb23000f040 con 0x7fb2380b1e40 2023-10-06T04:09:42.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.652+0000 7fb23eefa700 1 -- 172.21.15.157:0/956958744 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb238169d50 con 0x7fb2380b1e40 2023-10-06T04:09:42.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.652+0000 7fb22dffb700 1 -- 172.21.15.157:0/956958744 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb230015440 con 0x7fb2380b1e40 2023-10-06T04:09:42.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.652+0000 7fb23eefa700 1 -- 172.21.15.157:0/956958744 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb23816a030 con 0x7fb2380b1e40 2023-10-06T04:09:42.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.653+0000 7fb23eefa700 1 -- 172.21.15.157:0/956958744 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2380cb3d0 con 0x7fb2380b1e40 2023-10-06T04:09:42.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.653+0000 7fb22dffb700 1 -- 172.21.15.157:0/956958744 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb23001e3b0 con 0x7fb2380b1e40 2023-10-06T04:09:42.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.654+0000 7fb22dffb700 1 --2- 172.21.15.157:0/956958744 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb21c066f60 0x7fb21c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:42.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.654+0000 7fb22dffb700 1 -- 172.21.15.157:0/956958744 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6632+0+0 (secure 0 0 0) 0x7fb230084cd0 con 0x7fb2380b1e40 2023-10-06T04:09:42.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.654+0000 7fb23def8700 1 --2- 172.21.15.157:0/956958744 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb21c066f60 0x7fb21c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:42.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.654+0000 7fb23def8700 1 --2- 172.21.15.157:0/956958744 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb21c066f60 0x7fb21c069410 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb2340039f0 tx=0x7fb234007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:42.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.658+0000 7fb22dffb700 1 -- 172.21.15.157:0/956958744 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb230051ba0 con 0x7fb2380b1e40 2023-10-06T04:09:42.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.881+0000 7fb23eefa700 1 -- 172.21.15.157:0/956958744 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb238088f90 con 0x7fb2380b1e40 2023-10-06T04:09:42.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.881+0000 7fb22dffb700 1 -- 172.21.15.157:0/956958744 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+6148 (secure 0 0 0) 0x7fb23001a090 con 0x7fb2380b1e40 2023-10-06T04:09:42.883 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:09:42.883 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6270,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1036901172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1036901172},{"type":"v1","addr":"172.21.15.157:6837","nonce":1036901172}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:09:16.934246+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:active","state_seq":19,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:09:42.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.883+0000 7fb23eefa700 1 -- 172.21.15.157:0/956958744 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb21c066f60 msgr2=0x7fb21c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:42.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.883+0000 7fb23eefa700 1 --2- 172.21.15.157:0/956958744 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb21c066f60 0x7fb21c069410 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb2340039f0 tx=0x7fb234007ec0 comp rx=0 tx=0).stop 2023-10-06T04:09:42.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.883+0000 7fb23eefa700 1 -- 172.21.15.157:0/956958744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2380b1e40 msgr2=0x7fb238152a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:42.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.883+0000 7fb23eefa700 1 --2- 172.21.15.157:0/956958744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2380b1e40 0x7fb238152a60 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fb23000d100 tx=0x7fb2300099b0 comp rx=0 tx=0).stop 2023-10-06T04:09:42.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.883+0000 7fb23eefa700 1 -- 172.21.15.157:0/956958744 shutdown_connections 2023-10-06T04:09:42.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.883+0000 7fb23eefa700 1 --2- 172.21.15.157:0/956958744 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb21c066f60 0x7fb21c069410 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:42.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.883+0000 7fb23eefa700 1 --2- 172.21.15.157:0/956958744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb23806c2d0 0x7fb2381689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:42.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.883+0000 7fb23eefa700 1 --2- 172.21.15.157:0/956958744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2380b1e40 0x7fb238152a60 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:42.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.883+0000 7fb23eefa700 1 --2- 172.21.15.157:0/956958744 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2380b04d0 0x7fb238152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:42.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.883+0000 7fb23eefa700 1 -- 172.21.15.157:0/956958744 >> 172.21.15.157:0/956958744 conn(0x7fb2380cc540 msgr2=0x7fb238157440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:42.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.883+0000 7fb23eefa700 1 -- 172.21.15.157:0/956958744 shutdown_connections 2023-10-06T04:09:42.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:42.883+0000 7fb23eefa700 1 -- 172.21.15.157:0/956958744 wait complete. 2023-10-06T04:09:42.887 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 85 2023-10-06T04:09:42.898 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-06T04:09:44.899 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:09:45.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.025+0000 7f7439c0f700 1 Processor -- start 2023-10-06T04:09:45.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.025+0000 7f7439c0f700 1 -- start start 2023-10-06T04:09:45.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.026+0000 7f7439c0f700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7434068bc0 0x7f743406a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:45.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.026+0000 7f7439c0f700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7434156760 con 0x7f74340c2cb0 2023-10-06T04:09:45.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.026+0000 7f7439c0f700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7434159e40 con 0x7f74340bc850 2023-10-06T04:09:45.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.026+0000 7f7439c0f700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f743415a3c0 con 0x7f7434068bc0 2023-10-06T04:09:45.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.026+0000 7f7432ffd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7434068bc0 0x7f743406a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:45.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.026+0000 7f7432ffd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7434068bc0 0x7f743406a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56314/0 (socket says 172.21.15.157:56314) 2023-10-06T04:09:45.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.026+0000 7f74337fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f74340c2cb0 0x7f743406c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59734/0 (socket says 172.21.15.157:59734) 2023-10-06T04:09:45.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.026+0000 7f7432ffd700 1 -- 172.21.15.157:0/81097255 learned_addr learned my addr 172.21.15.157:0/81097255 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:45.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.027+0000 7f7432ffd700 1 -- 172.21.15.157:0/81097255 >> v1:172.21.15.175:6789/0 conn(0x7f74340bc850 legacy=0x7f74341607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:45.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.027+0000 7f7432ffd700 1 -- 172.21.15.157:0/81097255 >> v1:172.21.15.157:6789/0 conn(0x7f74340c2cb0 legacy=0x7f743406c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:45.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.027+0000 7f7432ffd700 1 -- 172.21.15.157:0/81097255 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743415a0e0 con 0x7f7434068bc0 2023-10-06T04:09:45.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.027+0000 7f7432ffd700 1 --2- 172.21.15.157:0/81097255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7434068bc0 0x7f743406a530 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f74280099b0 tx=0x7f7428009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa3f349e8e350d23 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:45.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.027+0000 7f74327fc700 1 -- 172.21.15.157:0/81097255 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742800b820 con 0x7f7434068bc0 2023-10-06T04:09:45.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.027+0000 7f74327fc700 1 -- 172.21.15.157:0/81097255 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742800b980 con 0x7f7434068bc0 2023-10-06T04:09:45.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.027+0000 7f74327fc700 1 -- 172.21.15.157:0/81097255 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742800bd20 con 0x7f7434068bc0 2023-10-06T04:09:45.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.027+0000 7f7439c0f700 1 -- 172.21.15.157:0/81097255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7434068bc0 msgr2=0x7f743406a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:45.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.028+0000 7f7439c0f700 1 --2- 172.21.15.157:0/81097255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7434068bc0 0x7f743406a530 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f74280099b0 tx=0x7f7428009250 comp rx=0 tx=0).stop 2023-10-06T04:09:45.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.028+0000 7f7439c0f700 1 -- 172.21.15.157:0/81097255 shutdown_connections 2023-10-06T04:09:45.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.028+0000 7f7439c0f700 1 --2- 172.21.15.157:0/81097255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7434068bc0 0x7f743406a530 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:45.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.028+0000 7f7439c0f700 1 -- 172.21.15.157:0/81097255 >> 172.21.15.157:0/81097255 conn(0x7f743406b510 msgr2=0x7f743406b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:45.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.028+0000 7f7439c0f700 1 -- 172.21.15.157:0/81097255 shutdown_connections 2023-10-06T04:09:45.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.028+0000 7f7439c0f700 1 -- 172.21.15.157:0/81097255 wait complete. 2023-10-06T04:09:45.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.029+0000 7f7439c0f700 1 Processor -- start 2023-10-06T04:09:45.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.029+0000 7f7439c0f700 1 -- start start 2023-10-06T04:09:45.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.030+0000 7f7439c0f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7434068bc0 0x7f7434169c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:45.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.030+0000 7f7439c0f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74340bc850 0x7f74340c1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:45.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.030+0000 7f7439c0f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74340c2cb0 0x7f7434157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:45.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.030+0000 7f7439c0f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74341697a0 con 0x7f7434068bc0 2023-10-06T04:09:45.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.030+0000 7f7439c0f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74340bd080 con 0x7f74340bc850 2023-10-06T04:09:45.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.030+0000 7f7439c0f700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74340bd360 con 0x7f74340c2cb0 2023-10-06T04:09:45.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.030+0000 7f74337fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7434068bc0 0x7f7434169c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:45.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.030+0000 7f74337fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7434068bc0 0x7f7434169c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56320/0 (socket says 172.21.15.157:56320) 2023-10-06T04:09:45.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.030+0000 7f74337fe700 1 -- 172.21.15.157:0/3587454296 learned_addr learned my addr 172.21.15.157:0/3587454296 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:45.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.030+0000 7f7432ffd700 1 --2- 172.21.15.157:0/3587454296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74340bc850 0x7f74340c1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:45.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.030+0000 7f7433fff700 1 --2- 172.21.15.157:0/3587454296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74340c2cb0 0x7f7434157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:45.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.030+0000 7f74337fe700 1 -- 172.21.15.157:0/3587454296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74340c2cb0 msgr2=0x7f7434157f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:45.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.030+0000 7f74337fe700 1 --2- 172.21.15.157:0/3587454296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74340c2cb0 0x7f7434157f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:45.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.030+0000 7f74337fe700 1 -- 172.21.15.157:0/3587454296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74340bc850 msgr2=0x7f74340c1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:45.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.030+0000 7f74337fe700 1 --2- 172.21.15.157:0/3587454296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74340bc850 0x7f74340c1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:45.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.030+0000 7f74337fe700 1 -- 172.21.15.157:0/3587454296 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74280094f0 con 0x7f7434068bc0 2023-10-06T04:09:45.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.030+0000 7f7432ffd700 1 --2- 172.21.15.157:0/3587454296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74340bc850 0x7f74340c1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:09:45.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.030+0000 7f74337fe700 1 --2- 172.21.15.157:0/3587454296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7434068bc0 0x7f7434169c30 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f742400c560 tx=0x7f7424007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:45.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.030+0000 7f741b7fe700 1 -- 172.21.15.157:0/3587454296 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7424005a70 con 0x7f7434068bc0 2023-10-06T04:09:45.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.030+0000 7f741b7fe700 1 -- 172.21.15.157:0/3587454296 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7424005bd0 con 0x7f7434068bc0 2023-10-06T04:09:45.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.030+0000 7f741b7fe700 1 -- 172.21.15.157:0/3587454296 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74240070d0 con 0x7f7434068bc0 2023-10-06T04:09:45.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.030+0000 7f7439c0f700 1 -- 172.21.15.157:0/3587454296 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74340bd640 con 0x7f7434068bc0 2023-10-06T04:09:45.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.030+0000 7f7439c0f700 1 -- 172.21.15.157:0/3587454296 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f743416a170 con 0x7f7434068bc0 2023-10-06T04:09:45.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.031+0000 7f7439c0f700 1 -- 172.21.15.157:0/3587454296 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74340bec10 con 0x7f7434068bc0 2023-10-06T04:09:45.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.032+0000 7f741b7fe700 1 -- 172.21.15.157:0/3587454296 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7424007480 con 0x7f7434068bc0 2023-10-06T04:09:45.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.032+0000 7f741b7fe700 1 --2- 172.21.15.157:0/3587454296 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7420066f60 0x7f7420069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:45.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.032+0000 7f741b7fe700 1 -- 172.21.15.157:0/3587454296 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6632+0+0 (secure 0 0 0) 0x7f7424083e90 con 0x7f7434068bc0 2023-10-06T04:09:45.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.032+0000 7f7432ffd700 1 --2- 172.21.15.157:0/3587454296 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7420066f60 0x7f7420069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:45.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.033+0000 7f7432ffd700 1 --2- 172.21.15.157:0/3587454296 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7420066f60 0x7f7420069410 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f74280099b0 tx=0x7f7428016000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:45.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.036+0000 7f741b7fe700 1 -- 172.21.15.157:0/3587454296 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f74340bec10 con 0x7f7434068bc0 2023-10-06T04:09:45.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.260+0000 7f7439c0f700 1 -- 172.21.15.157:0/3587454296 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74340bec10 con 0x7f7434068bc0 2023-10-06T04:09:45.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.261+0000 7f741b7fe700 1 -- 172.21.15.157:0/3587454296 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+6148 (secure 0 0 0) 0x7f74340bec10 con 0x7f7434068bc0 2023-10-06T04:09:45.263 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:09:45.263 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6270,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1036901172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1036901172},{"type":"v1","addr":"172.21.15.157:6837","nonce":1036901172}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:09:16.934246+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5799,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5799":{"gid":5799,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":17,"addr":"172.21.15.157:6834/765157057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":765157057},{"type":"v1","addr":"172.21.15.157:6834","nonce":765157057}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:active","state_seq":19,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:09:45.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.263+0000 7f7439c0f700 1 -- 172.21.15.157:0/3587454296 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7420066f60 msgr2=0x7f7420069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:45.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.263+0000 7f7439c0f700 1 --2- 172.21.15.157:0/3587454296 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7420066f60 0x7f7420069410 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f74280099b0 tx=0x7f7428016000 comp rx=0 tx=0).stop 2023-10-06T04:09:45.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.263+0000 7f7439c0f700 1 -- 172.21.15.157:0/3587454296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7434068bc0 msgr2=0x7f7434169c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:45.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.263+0000 7f7439c0f700 1 --2- 172.21.15.157:0/3587454296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7434068bc0 0x7f7434169c30 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f742400c560 tx=0x7f7424007f70 comp rx=0 tx=0).stop 2023-10-06T04:09:45.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.263+0000 7f7439c0f700 1 -- 172.21.15.157:0/3587454296 shutdown_connections 2023-10-06T04:09:45.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.263+0000 7f7439c0f700 1 --2- 172.21.15.157:0/3587454296 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7420066f60 0x7f7420069410 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:45.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.263+0000 7f7439c0f700 1 --2- 172.21.15.157:0/3587454296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74340c2cb0 0x7f7434157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:45.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.263+0000 7f7439c0f700 1 --2- 172.21.15.157:0/3587454296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7434068bc0 0x7f7434169c30 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:45.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.263+0000 7f7439c0f700 1 --2- 172.21.15.157:0/3587454296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74340bc850 0x7f74340c1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:45.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.263+0000 7f7439c0f700 1 -- 172.21.15.157:0/3587454296 >> 172.21.15.157:0/3587454296 conn(0x7f743406b510 msgr2=0x7f74340c9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:45.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.263+0000 7f7439c0f700 1 -- 172.21.15.157:0/3587454296 shutdown_connections 2023-10-06T04:09:45.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:45.263+0000 7f7439c0f700 1 -- 172.21.15.157:0/3587454296 wait complete. 2023-10-06T04:09:45.267 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 85 2023-10-06T04:09:45.277 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2023-10-06T04:09:46.133 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:09:46.136 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:09:47.278 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:09:47.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.405+0000 7f9a6ae9a700 1 Processor -- start 2023-10-06T04:09:47.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.405+0000 7f9a6ae9a700 1 -- start start 2023-10-06T04:09:47.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.407+0000 7f9a6ae9a700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9a64069ea0 0x7f9a640631f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:47.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.407+0000 7f9a6ae9a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a6406a860 0x7f9a64156870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:47.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.407+0000 7f9a6ae9a700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9a6406a270 0x7f9a640c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:47.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.407+0000 7f9a6ae9a700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a64156db0 con 0x7f9a64069ea0 2023-10-06T04:09:47.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.407+0000 7f9a6ae9a700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a640c48f0 con 0x7f9a6406a270 2023-10-06T04:09:47.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.407+0000 7f9a6ae9a700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a640c3d20 con 0x7f9a6406a860 2023-10-06T04:09:47.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.407+0000 7f9a68c36700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9a64069ea0 0x7f9a640631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:47.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.407+0000 7f9a69437700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9a6406a270 0x7f9a640c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:47.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.407+0000 7f9a63fff700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a6406a860 0x7f9a64156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:47.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.407+0000 7f9a68c36700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9a64069ea0 0x7f9a640631f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56332/0 (socket says 172.21.15.157:56332) 2023-10-06T04:09:47.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.407+0000 7f9a68c36700 1 -- 172.21.15.157:0/743449948 learned_addr learned my addr 172.21.15.157:0/743449948 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:47.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.407+0000 7f9a68c36700 1 -- 172.21.15.157:0/743449948 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a6406a860 msgr2=0x7f9a64156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:47.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.407+0000 7f9a68c36700 1 --2- 172.21.15.157:0/743449948 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a6406a860 0x7f9a64156870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:47.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.407+0000 7f9a68c36700 1 -- 172.21.15.157:0/743449948 >> v2:172.21.15.175:3300/0 conn(0x7f9a6406a270 msgr2=0x7f9a640c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:47.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.407+0000 7f9a68c36700 1 --2- 172.21.15.157:0/743449948 >> v2:172.21.15.175:3300/0 conn(0x7f9a6406a270 0x7f9a640c4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:47.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.407+0000 7f9a68c36700 1 -- 172.21.15.157:0/743449948 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a64063840 con 0x7f9a64069ea0 2023-10-06T04:09:47.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.407+0000 7f9a63fff700 1 --2- 172.21.15.157:0/743449948 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a6406a860 0x7f9a64156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:09:47.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.407+0000 7f9a69437700 1 --2- 172.21.15.157:0/743449948 >> v2:172.21.15.175:3300/0 conn(0x7f9a6406a270 0x7f9a640c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:09:47.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.407+0000 7f9a68c36700 1 --2- 172.21.15.157:0/743449948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a64069ea0 0x7f9a640631f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f9a6406bf30 tx=0x7f9a5c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79204a7a0973b9c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:47.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.408+0000 7f9a637fe700 1 -- 172.21.15.157:0/743449948 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c00e070 con 0x7f9a64069ea0 2023-10-06T04:09:47.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.408+0000 7f9a637fe700 1 -- 172.21.15.157:0/743449948 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a5c00ba00 con 0x7f9a64069ea0 2023-10-06T04:09:47.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.408+0000 7f9a637fe700 1 -- 172.21.15.157:0/743449948 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c0092e0 con 0x7f9a64069ea0 2023-10-06T04:09:47.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.408+0000 7f9a6ae9a700 1 -- 172.21.15.157:0/743449948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a64069ea0 msgr2=0x7f9a640631f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:47.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.408+0000 7f9a6ae9a700 1 --2- 172.21.15.157:0/743449948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a64069ea0 0x7f9a640631f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f9a6406bf30 tx=0x7f9a5c009840 comp rx=0 tx=0).stop 2023-10-06T04:09:47.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.408+0000 7f9a6ae9a700 1 -- 172.21.15.157:0/743449948 shutdown_connections 2023-10-06T04:09:47.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.408+0000 7f9a6ae9a700 1 --2- 172.21.15.157:0/743449948 >> v2:172.21.15.175:3300/0 conn(0x7f9a6406a270 0x7f9a640c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:47.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.408+0000 7f9a6ae9a700 1 --2- 172.21.15.157:0/743449948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a64069ea0 0x7f9a640631f0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:47.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.408+0000 7f9a6ae9a700 1 --2- 172.21.15.157:0/743449948 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a6406a860 0x7f9a64156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:47.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.408+0000 7f9a6ae9a700 1 -- 172.21.15.157:0/743449948 >> 172.21.15.157:0/743449948 conn(0x7f9a640c8b20 msgr2=0x7f9a640c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:47.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.408+0000 7f9a6ae9a700 1 -- 172.21.15.157:0/743449948 shutdown_connections 2023-10-06T04:09:47.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.408+0000 7f9a6ae9a700 1 -- 172.21.15.157:0/743449948 wait complete. 2023-10-06T04:09:47.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.409+0000 7f9a6ae9a700 1 Processor -- start 2023-10-06T04:09:47.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.410+0000 7f9a6ae9a700 1 -- start start 2023-10-06T04:09:47.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.410+0000 7f9a6ae9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a64069ea0 0x7f9a64158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:47.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.410+0000 7f9a6ae9a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a6406a270 0x7f9a64145560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:47.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.410+0000 7f9a6ae9a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a6406a860 0x7f9a64145aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:47.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.410+0000 7f9a6ae9a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a641673a0 con 0x7f9a6406a270 2023-10-06T04:09:47.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.410+0000 7f9a6ae9a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6414ccf0 con 0x7f9a64069ea0 2023-10-06T04:09:47.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.410+0000 7f9a6ae9a700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a641566e0 con 0x7f9a6406a860 2023-10-06T04:09:47.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.410+0000 7f9a63fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a6406a270 0x7f9a64145560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:47.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.411+0000 7f9a63fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a6406a270 0x7f9a64145560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56346/0 (socket says 172.21.15.157:56346) 2023-10-06T04:09:47.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.411+0000 7f9a63fff700 1 -- 172.21.15.157:0/4258911176 learned_addr learned my addr 172.21.15.157:0/4258911176 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:47.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.411+0000 7f9a68c36700 1 --2- 172.21.15.157:0/4258911176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a64069ea0 0x7f9a64158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:47.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.411+0000 7f9a69437700 1 --2- 172.21.15.157:0/4258911176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a6406a860 0x7f9a64145aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:47.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.411+0000 7f9a63fff700 1 -- 172.21.15.157:0/4258911176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a6406a860 msgr2=0x7f9a64145aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:47.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.411+0000 7f9a63fff700 1 --2- 172.21.15.157:0/4258911176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a6406a860 0x7f9a64145aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:47.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.411+0000 7f9a63fff700 1 -- 172.21.15.157:0/4258911176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a64069ea0 msgr2=0x7f9a64158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:47.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.411+0000 7f9a63fff700 1 --2- 172.21.15.157:0/4258911176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a64069ea0 0x7f9a64158390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:47.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.411+0000 7f9a63fff700 1 -- 172.21.15.157:0/4258911176 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a54009420 con 0x7f9a6406a270 2023-10-06T04:09:47.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.411+0000 7f9a63fff700 1 --2- 172.21.15.157:0/4258911176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a6406a270 0x7f9a64145560 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f9a5800ecb0 tx=0x7f9a580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:47.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.411+0000 7f9a60ff9700 1 -- 172.21.15.157:0/4258911176 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5800c9d0 con 0x7f9a6406a270 2023-10-06T04:09:47.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.411+0000 7f9a60ff9700 1 -- 172.21.15.157:0/4258911176 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a5800cb30 con 0x7f9a6406a270 2023-10-06T04:09:47.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.411+0000 7f9a6ae9a700 1 -- 172.21.15.157:0/4258911176 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a58009420 con 0x7f9a6406a270 2023-10-06T04:09:47.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.411+0000 7f9a60ff9700 1 -- 172.21.15.157:0/4258911176 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5800ceb0 con 0x7f9a6406a270 2023-10-06T04:09:47.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.411+0000 7f9a6ae9a700 1 -- 172.21.15.157:0/4258911176 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a5c009ae0 con 0x7f9a6406a270 2023-10-06T04:09:47.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.412+0000 7f9a6ae9a700 1 -- 172.21.15.157:0/4258911176 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a640642a0 con 0x7f9a6406a270 2023-10-06T04:09:47.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.413+0000 7f9a60ff9700 1 -- 172.21.15.157:0/4258911176 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9a58013680 con 0x7f9a6406a270 2023-10-06T04:09:47.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.413+0000 7f9a60ff9700 1 --2- 172.21.15.157:0/4258911176 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9a50066f60 0x7f9a50069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:47.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.413+0000 7f9a60ff9700 1 -- 172.21.15.157:0/4258911176 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f9a58084cb0 con 0x7f9a6406a270 2023-10-06T04:09:47.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.413+0000 7f9a68c36700 1 --2- 172.21.15.157:0/4258911176 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9a50066f60 0x7f9a50069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:47.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.414+0000 7f9a68c36700 1 --2- 172.21.15.157:0/4258911176 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9a50066f60 0x7f9a50069410 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c00c010 tx=0x7f9a5c018000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:47.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.417+0000 7f9a60ff9700 1 -- 172.21.15.157:0/4258911176 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a580555c0 con 0x7f9a6406a270 2023-10-06T04:09:47.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.641+0000 7f9a6ae9a700 1 -- 172.21.15.157:0/4258911176 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a6414ce30 con 0x7f9a6406a270 2023-10-06T04:09:47.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.642+0000 7f9a60ff9700 1 -- 172.21.15.157:0/4258911176 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+5389 (secure 0 0 0) 0x7f9a58055150 con 0x7f9a6406a270 2023-10-06T04:09:47.643 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:09:47.644 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":86,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:09:46.762378+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6270,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:active","state_seq":19,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6270":{"gid":6270,"name":"e","rank":0,"incarnation":86,"state":"up:replay","state_seq":1,"addr":"172.21.15.157:6837/1036901172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1036901172},{"type":"v1","addr":"172.21.15.157:6837","nonce":1036901172}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:09:47.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.644+0000 7f9a6ae9a700 1 -- 172.21.15.157:0/4258911176 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9a50066f60 msgr2=0x7f9a50069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:47.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.644+0000 7f9a6ae9a700 1 --2- 172.21.15.157:0/4258911176 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9a50066f60 0x7f9a50069410 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c00c010 tx=0x7f9a5c018000 comp rx=0 tx=0).stop 2023-10-06T04:09:47.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.644+0000 7f9a6ae9a700 1 -- 172.21.15.157:0/4258911176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a6406a270 msgr2=0x7f9a64145560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:47.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.644+0000 7f9a6ae9a700 1 --2- 172.21.15.157:0/4258911176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a6406a270 0x7f9a64145560 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f9a5800ecb0 tx=0x7f9a580091a0 comp rx=0 tx=0).stop 2023-10-06T04:09:47.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.644+0000 7f9a6ae9a700 1 -- 172.21.15.157:0/4258911176 shutdown_connections 2023-10-06T04:09:47.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.644+0000 7f9a6ae9a700 1 --2- 172.21.15.157:0/4258911176 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9a50066f60 0x7f9a50069410 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:47.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.644+0000 7f9a6ae9a700 1 --2- 172.21.15.157:0/4258911176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a64069ea0 0x7f9a64158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:47.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.644+0000 7f9a6ae9a700 1 --2- 172.21.15.157:0/4258911176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a6406a860 0x7f9a64145aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:47.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.644+0000 7f9a6ae9a700 1 --2- 172.21.15.157:0/4258911176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a6406a270 0x7f9a64145560 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:47.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.644+0000 7f9a6ae9a700 1 -- 172.21.15.157:0/4258911176 >> 172.21.15.157:0/4258911176 conn(0x7f9a640c8b20 msgr2=0x7f9a6414f870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:47.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.644+0000 7f9a6ae9a700 1 -- 172.21.15.157:0/4258911176 shutdown_connections 2023-10-06T04:09:47.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.644+0000 7f9a6ae9a700 1 -- 172.21.15.157:0/4258911176 wait complete. 2023-10-06T04:09:47.647 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 86 2023-10-06T04:09:47.658 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2023-10-06T04:09:47.658 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-10-06T04:09:47.658 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:09:47.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.772+0000 7f7e977a6700 1 Processor -- start 2023-10-06T04:09:47.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.773+0000 7f7e977a6700 1 -- start start 2023-10-06T04:09:47.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.773+0000 7f7e977a6700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7e9014cf90 0x7f7e9006a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:47.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.773+0000 7f7e977a6700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7e900c2450 0x7f7e9006ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:47.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.773+0000 7f7e977a6700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e900ca020 con 0x7f7e9006d300 2023-10-06T04:09:47.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.773+0000 7f7e977a6700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e9006dd30 con 0x7f7e900c2450 2023-10-06T04:09:47.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.773+0000 7f7e977a6700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e900c6c00 con 0x7f7e9014cf90 2023-10-06T04:09:47.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.774+0000 7f7e95d43700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7e900c2450 0x7f7e9006ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:47.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.774+0000 7f7e95d43700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7e900c2450 0x7f7e9006ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56356/0 (socket says 172.21.15.157:56356) 2023-10-06T04:09:47.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.774+0000 7f7e95d43700 1 -- 172.21.15.157:0/1576071822 learned_addr learned my addr 172.21.15.157:0/1576071822 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:47.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.774+0000 7f7e95542700 1 --2- 172.21.15.157:0/1576071822 >> v2:172.21.15.175:3300/0 conn(0x7f7e9014cf90 0x7f7e9006a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:47.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.774+0000 7f7e95d43700 1 -- 172.21.15.157:0/1576071822 >> v2:172.21.15.175:3300/0 conn(0x7f7e9014cf90 msgr2=0x7f7e9006a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:47.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.774+0000 7f7e95d43700 1 --2- 172.21.15.157:0/1576071822 >> v2:172.21.15.175:3300/0 conn(0x7f7e9014cf90 0x7f7e9006a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:47.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.774+0000 7f7e95d43700 1 -- 172.21.15.157:0/1576071822 >> v1:172.21.15.175:6789/0 conn(0x7f7e9006d300 legacy=0x7f7e900c1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:47.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.774+0000 7f7e95d43700 1 -- 172.21.15.157:0/1576071822 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e900c9410 con 0x7f7e900c2450 2023-10-06T04:09:47.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.774+0000 7f7e95d43700 1 --2- 172.21.15.157:0/1576071822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e900c2450 0x7f7e9006ab00 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f7e9006b100 tx=0x7f7e88009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66ebf7568004761 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:47.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.774+0000 7f7e87fff700 1 -- 172.21.15.157:0/1576071822 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8800e070 con 0x7f7e900c2450 2023-10-06T04:09:47.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.774+0000 7f7e87fff700 1 -- 172.21.15.157:0/1576071822 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8800bae0 con 0x7f7e900c2450 2023-10-06T04:09:47.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.774+0000 7f7e87fff700 1 -- 172.21.15.157:0/1576071822 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e880092e0 con 0x7f7e900c2450 2023-10-06T04:09:47.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.775+0000 7f7e977a6700 1 -- 172.21.15.157:0/1576071822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e900c2450 msgr2=0x7f7e9006ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:47.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.775+0000 7f7e977a6700 1 --2- 172.21.15.157:0/1576071822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e900c2450 0x7f7e9006ab00 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f7e9006b100 tx=0x7f7e88009840 comp rx=0 tx=0).stop 2023-10-06T04:09:47.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.775+0000 7f7e977a6700 1 -- 172.21.15.157:0/1576071822 shutdown_connections 2023-10-06T04:09:47.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.775+0000 7f7e977a6700 1 --2- 172.21.15.157:0/1576071822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e900c2450 0x7f7e9006ab00 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:47.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.775+0000 7f7e977a6700 1 --2- 172.21.15.157:0/1576071822 >> v2:172.21.15.175:3300/0 conn(0x7f7e9014cf90 0x7f7e9006a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:47.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.775+0000 7f7e977a6700 1 -- 172.21.15.157:0/1576071822 >> 172.21.15.157:0/1576071822 conn(0x7f7e9006f3e0 msgr2=0x7f7e900631f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:47.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.775+0000 7f7e977a6700 1 -- 172.21.15.157:0/1576071822 shutdown_connections 2023-10-06T04:09:47.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.775+0000 7f7e977a6700 1 -- 172.21.15.157:0/1576071822 wait complete. 2023-10-06T04:09:47.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.776+0000 7f7e977a6700 1 Processor -- start 2023-10-06T04:09:47.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.776+0000 7f7e977a6700 1 -- start start 2023-10-06T04:09:47.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.777+0000 7f7e977a6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e9006d300 0x7f7e90149e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:47.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.777+0000 7f7e977a6700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e900c2450 0x7f7e9014dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:47.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.777+0000 7f7e977a6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e9014cf90 0x7f7e90067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:47.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.777+0000 7f7e977a6700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9014fd70 con 0x7f7e9006d300 2023-10-06T04:09:47.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.777+0000 7f7e977a6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e901628c0 con 0x7f7e9014cf90 2023-10-06T04:09:47.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.777+0000 7f7e977a6700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e90162d90 con 0x7f7e900c2450 2023-10-06T04:09:47.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.777+0000 7f7e95542700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e9006d300 0x7f7e90149e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:47.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.777+0000 7f7e95542700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e9006d300 0x7f7e90149e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56370/0 (socket says 172.21.15.157:56370) 2023-10-06T04:09:47.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.777+0000 7f7e95542700 1 -- 172.21.15.157:0/759376889 learned_addr learned my addr 172.21.15.157:0/759376889 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:47.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.777+0000 7f7e95d43700 1 --2- 172.21.15.157:0/759376889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e9014cf90 0x7f7e90067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:47.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.777+0000 7f7e94d41700 1 --2- 172.21.15.157:0/759376889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e900c2450 0x7f7e9014dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:47.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.777+0000 7f7e95542700 1 -- 172.21.15.157:0/759376889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e900c2450 msgr2=0x7f7e9014dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:47.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.777+0000 7f7e95542700 1 --2- 172.21.15.157:0/759376889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e900c2450 0x7f7e9014dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:47.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.777+0000 7f7e95542700 1 -- 172.21.15.157:0/759376889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e9014cf90 msgr2=0x7f7e90067f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:47.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.777+0000 7f7e95542700 1 --2- 172.21.15.157:0/759376889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e9014cf90 0x7f7e90067f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:47.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.777+0000 7f7e95542700 1 -- 172.21.15.157:0/759376889 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e88009ae0 con 0x7f7e9006d300 2023-10-06T04:09:47.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.777+0000 7f7e94d41700 1 --2- 172.21.15.157:0/759376889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e900c2450 0x7f7e9014dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:09:47.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.777+0000 7f7e95542700 1 --2- 172.21.15.157:0/759376889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e9006d300 0x7f7e90149e40 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00d890 tx=0x7f7e8c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:47.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.777+0000 7f7e857fa700 1 -- 172.21.15.157:0/759376889 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c0125f0 con 0x7f7e9006d300 2023-10-06T04:09:47.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.777+0000 7f7e857fa700 1 -- 172.21.15.157:0/759376889 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8c012750 con 0x7f7e9006d300 2023-10-06T04:09:47.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.777+0000 7f7e977a6700 1 -- 172.21.15.157:0/759376889 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e901549c0 con 0x7f7e9006d300 2023-10-06T04:09:47.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.777+0000 7f7e857fa700 1 -- 172.21.15.157:0/759376889 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c012b10 con 0x7f7e9006d300 2023-10-06T04:09:47.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.778+0000 7f7e977a6700 1 -- 172.21.15.157:0/759376889 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e90154e50 con 0x7f7e9006d300 2023-10-06T04:09:47.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.778+0000 7f7e977a6700 1 -- 172.21.15.157:0/759376889 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e90154ca0 con 0x7f7e9006d300 2023-10-06T04:09:47.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.779+0000 7f7e857fa700 1 -- 172.21.15.157:0/759376889 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7e8c01e460 con 0x7f7e9006d300 2023-10-06T04:09:47.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.779+0000 7f7e857fa700 1 --2- 172.21.15.157:0/759376889 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7e74066f30 0x7f7e740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:47.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.779+0000 7f7e857fa700 1 -- 172.21.15.157:0/759376889 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f7e8c084ec0 con 0x7f7e9006d300 2023-10-06T04:09:47.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.779+0000 7f7e94d41700 1 --2- 172.21.15.157:0/759376889 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7e74066f30 0x7f7e740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:47.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.780+0000 7f7e94d41700 1 --2- 172.21.15.157:0/759376889 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7e74066f30 0x7f7e740693e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7e80007fd0 tx=0x7f7e80009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:47.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:47.783+0000 7f7e857fa700 1 -- 172.21.15.157:0/759376889 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7e8c051d40 con 0x7f7e9006d300 2023-10-06T04:09:48.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:48.010+0000 7f7e977a6700 1 -- 172.21.15.157:0/759376889 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e90162b10 con 0x7f7e9006d300 2023-10-06T04:09:48.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:48.011+0000 7f7e857fa700 1 -- 172.21.15.157:0/759376889 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+5389 (secure 0 0 0) 0x7f7e8c055360 con 0x7f7e9006d300 2023-10-06T04:09:48.013 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:09:48.013 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":86,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:09:46.762378+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6270,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:active","state_seq":19,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6270":{"gid":6270,"name":"e","rank":0,"incarnation":86,"state":"up:replay","state_seq":1,"addr":"172.21.15.157:6837/1036901172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1036901172},{"type":"v1","addr":"172.21.15.157:6837","nonce":1036901172}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:09:48.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:48.013+0000 7f7e977a6700 1 -- 172.21.15.157:0/759376889 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7e74066f30 msgr2=0x7f7e740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:48.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:48.013+0000 7f7e977a6700 1 --2- 172.21.15.157:0/759376889 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7e74066f30 0x7f7e740693e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7e80007fd0 tx=0x7f7e80009040 comp rx=0 tx=0).stop 2023-10-06T04:09:48.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:48.013+0000 7f7e977a6700 1 -- 172.21.15.157:0/759376889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e9006d300 msgr2=0x7f7e90149e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:48.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:48.013+0000 7f7e977a6700 1 --2- 172.21.15.157:0/759376889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e9006d300 0x7f7e90149e40 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00d890 tx=0x7f7e8c008040 comp rx=0 tx=0).stop 2023-10-06T04:09:48.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:48.013+0000 7f7e977a6700 1 -- 172.21.15.157:0/759376889 shutdown_connections 2023-10-06T04:09:48.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:48.013+0000 7f7e977a6700 1 --2- 172.21.15.157:0/759376889 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f7e74066f30 0x7f7e740693e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:48.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:48.013+0000 7f7e977a6700 1 --2- 172.21.15.157:0/759376889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e9014cf90 0x7f7e90067f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:48.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:48.013+0000 7f7e977a6700 1 --2- 172.21.15.157:0/759376889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e9006d300 0x7f7e90149e40 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:48.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:48.013+0000 7f7e977a6700 1 --2- 172.21.15.157:0/759376889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e900c2450 0x7f7e9014dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:48.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:48.013+0000 7f7e977a6700 1 -- 172.21.15.157:0/759376889 >> 172.21.15.157:0/759376889 conn(0x7f7e9006f3e0 msgr2=0x7f7e900631f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:48.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:48.013+0000 7f7e977a6700 1 -- 172.21.15.157:0/759376889 shutdown_connections 2023-10-06T04:09:48.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:48.013+0000 7f7e977a6700 1 -- 172.21.15.157:0/759376889 wait complete. 2023-10-06T04:09:48.016 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 86 2023-10-06T04:09:48.028 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 86, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-06T03:57:10.718782+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 86, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_5812": { "addr": "172.21.15.175:6837/1436765859", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6836", "nonce": 1436765859, "type": "v2" }, { "addr": "172.21.15.175:6837", "nonce": 1436765859, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0 ], "features": 4540138320759226367, "flags": 0, "gid": 5812, "incarnation": 78, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:active", "state_seq": 19 }, "gid_6270": { "addr": "172.21.15.157:6837/1036901172", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6836", "nonce": 1036901172, "type": "v2" }, { "addr": "172.21.15.157:6837", "nonce": 1036901172, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 6270, "incarnation": 86, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 53, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2023-10-06T04:09:46.762378+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2, 3, 4 ], "tableserver": 0, "up": { "mds_0": 6270, "mds_1": 5812 } } } ], "standbys": [ { "addr": "172.21.15.175:6835/1799968248", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6834", "nonce": 1799968248, "type": "v2" }, { "addr": "172.21.15.175:6835", "nonce": 1799968248, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 57, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5126, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.175:6839/745804271", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6838", "nonce": 745804271, "type": "v2" }, { "addr": "172.21.15.175:6839", "nonce": 745804271, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 42, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5164, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.157:6835/2461751318", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6833", "nonce": 2461751318, "type": "v2" }, { "addr": "172.21.15.157:6835", "nonce": 2461751318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 34, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5268, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-06T04:09:50.029 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:09:50.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.156+0000 7f6adf7d3700 1 Processor -- start 2023-10-06T04:09:50.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.156+0000 7f6adf7d3700 1 -- start start 2023-10-06T04:09:50.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.157+0000 7f6adf7d3700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6ad806dc40 0x7f6ad814fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:50.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.157+0000 7f6adf7d3700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ad806c1f0 0x7f6ad8150240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:50.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.157+0000 7f6adf7d3700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ad80c9780 con 0x7f6ad806cbb0 2023-10-06T04:09:50.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.157+0000 7f6adf7d3700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ad80c8c40 con 0x7f6ad806dc40 2023-10-06T04:09:50.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.157+0000 7f6adf7d3700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad806e0c0 con 0x7f6ad806c1f0 2023-10-06T04:09:50.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.157+0000 7f6adcd6e700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6ad806dc40 0x7f6ad814fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:50.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.157+0000 7f6adcd6e700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6ad806dc40 0x7f6ad814fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56378/0 (socket says 172.21.15.157:56378) 2023-10-06T04:09:50.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.157+0000 7f6adcd6e700 1 -- 172.21.15.157:0/1242847041 learned_addr learned my addr 172.21.15.157:0/1242847041 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:50.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.157+0000 7f6addd70700 1 --2- 172.21.15.157:0/1242847041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ad806c1f0 0x7f6ad8150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:50.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.157+0000 7f6adcd6e700 1 -- 172.21.15.157:0/1242847041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ad806c1f0 msgr2=0x7f6ad8150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:50.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.157+0000 7f6adcd6e700 1 --2- 172.21.15.157:0/1242847041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ad806c1f0 0x7f6ad8150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:50.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.157+0000 7f6adcd6e700 1 -- 172.21.15.157:0/1242847041 >> v1:172.21.15.175:6789/0 conn(0x7f6ad806cbb0 legacy=0x7f6ad806d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:50.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.157+0000 7f6adcd6e700 1 -- 172.21.15.157:0/1242847041 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad80c8e50 con 0x7f6ad806dc40 2023-10-06T04:09:50.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.157+0000 7f6adcd6e700 1 --2- 172.21.15.157:0/1242847041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ad806dc40 0x7f6ad814fd00 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80099b0 tx=0x7f6ac8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ff8350080e3b4a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:50.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.158+0000 7f6acffff700 1 -- 172.21.15.157:0/1242847041 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac800b8f0 con 0x7f6ad806dc40 2023-10-06T04:09:50.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.158+0000 7f6acffff700 1 -- 172.21.15.157:0/1242847041 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac800ba50 con 0x7f6ad806dc40 2023-10-06T04:09:50.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.158+0000 7f6acffff700 1 -- 172.21.15.157:0/1242847041 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac800bdf0 con 0x7f6ad806dc40 2023-10-06T04:09:50.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.158+0000 7f6adf7d3700 1 -- 172.21.15.157:0/1242847041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ad806dc40 msgr2=0x7f6ad814fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:50.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.158+0000 7f6adf7d3700 1 --2- 172.21.15.157:0/1242847041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ad806dc40 0x7f6ad814fd00 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80099b0 tx=0x7f6ac8009250 comp rx=0 tx=0).stop 2023-10-06T04:09:50.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.158+0000 7f6adf7d3700 1 -- 172.21.15.157:0/1242847041 shutdown_connections 2023-10-06T04:09:50.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.158+0000 7f6adf7d3700 1 --2- 172.21.15.157:0/1242847041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ad806c1f0 0x7f6ad8150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:50.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.158+0000 7f6adf7d3700 1 --2- 172.21.15.157:0/1242847041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ad806dc40 0x7f6ad814fd00 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:50.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.158+0000 7f6adf7d3700 1 -- 172.21.15.157:0/1242847041 >> 172.21.15.157:0/1242847041 conn(0x7f6ad806f230 msgr2=0x7f6ad806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:50.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.158+0000 7f6adf7d3700 1 -- 172.21.15.157:0/1242847041 shutdown_connections 2023-10-06T04:09:50.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.158+0000 7f6adf7d3700 1 -- 172.21.15.157:0/1242847041 wait complete. 2023-10-06T04:09:50.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.159+0000 7f6adf7d3700 1 Processor -- start 2023-10-06T04:09:50.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.159+0000 7f6adf7d3700 1 -- start start 2023-10-06T04:09:50.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.160+0000 7f6adf7d3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ad806c1f0 0x7f6ad8152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:50.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.160+0000 7f6adf7d3700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ad806cbb0 0x7f6ad8150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:50.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.160+0000 7f6adf7d3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ad81514c0 0x7f6ad8169a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:50.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.160+0000 7f6adf7d3700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad81503d0 con 0x7f6ad81514c0 2023-10-06T04:09:50.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.160+0000 7f6adf7d3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad81480c0 con 0x7f6ad806c1f0 2023-10-06T04:09:50.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.160+0000 7f6adf7d3700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad8150850 con 0x7f6ad806cbb0 2023-10-06T04:09:50.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.160+0000 7f6addd70700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ad81514c0 0x7f6ad8169a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:50.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.160+0000 7f6addd70700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ad81514c0 0x7f6ad8169a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56384/0 (socket says 172.21.15.157:56384) 2023-10-06T04:09:50.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.160+0000 7f6addd70700 1 -- 172.21.15.157:0/2645241675 learned_addr learned my addr 172.21.15.157:0/2645241675 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:50.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.160+0000 7f6add56f700 1 --2- 172.21.15.157:0/2645241675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ad806c1f0 0x7f6ad8152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:50.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.160+0000 7f6adcd6e700 1 --2- 172.21.15.157:0/2645241675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ad806cbb0 0x7f6ad8150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:50.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.161+0000 7f6addd70700 1 -- 172.21.15.157:0/2645241675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ad806cbb0 msgr2=0x7f6ad8150f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:50.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.161+0000 7f6addd70700 1 --2- 172.21.15.157:0/2645241675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ad806cbb0 0x7f6ad8150f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:50.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.161+0000 7f6addd70700 1 -- 172.21.15.157:0/2645241675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ad806c1f0 msgr2=0x7f6ad8152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:50.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.161+0000 7f6addd70700 1 --2- 172.21.15.157:0/2645241675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ad806c1f0 0x7f6ad8152ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:50.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.161+0000 7f6addd70700 1 -- 172.21.15.157:0/2645241675 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac80094f0 con 0x7f6ad81514c0 2023-10-06T04:09:50.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.161+0000 7f6addd70700 1 --2- 172.21.15.157:0/2645241675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ad81514c0 0x7f6ad8169a70 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009fa0 tx=0x7f6ad4009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:50.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.161+0000 7f6acd7fa700 1 -- 172.21.15.157:0/2645241675 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad400ebc0 con 0x7f6ad81514c0 2023-10-06T04:09:50.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.161+0000 7f6acd7fa700 1 -- 172.21.15.157:0/2645241675 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad400ed20 con 0x7f6ad81514c0 2023-10-06T04:09:50.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.161+0000 7f6adf7d3700 1 -- 172.21.15.157:0/2645241675 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad8150af0 con 0x7f6ad81514c0 2023-10-06T04:09:50.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.161+0000 7f6acd7fa700 1 -- 172.21.15.157:0/2645241675 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4014670 con 0x7f6ad81514c0 2023-10-06T04:09:50.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.161+0000 7f6adf7d3700 1 -- 172.21.15.157:0/2645241675 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad8145e90 con 0x7f6ad81514c0 2023-10-06T04:09:50.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.162+0000 7f6adf7d3700 1 -- 172.21.15.157:0/2645241675 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ad8089ca0 con 0x7f6ad81514c0 2023-10-06T04:09:50.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.162+0000 7f6acd7fa700 1 -- 172.21.15.157:0/2645241675 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ad40149e0 con 0x7f6ad81514c0 2023-10-06T04:09:50.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.163+0000 7f6acd7fa700 1 --2- 172.21.15.157:0/2645241675 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6abc066fb0 0x7f6abc069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:50.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.163+0000 7f6acd7fa700 1 -- 172.21.15.157:0/2645241675 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f6ad4085020 con 0x7f6ad81514c0 2023-10-06T04:09:50.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.163+0000 7f6add56f700 1 --2- 172.21.15.157:0/2645241675 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6abc066fb0 0x7f6abc069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:50.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.164+0000 7f6add56f700 1 --2- 172.21.15.157:0/2645241675 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6abc066fb0 0x7f6abc069460 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0003a80 tx=0x7f6ad0007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:50.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.167+0000 7f6acd7fa700 1 -- 172.21.15.157:0/2645241675 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ad4051e00 con 0x7f6ad81514c0 2023-10-06T04:09:50.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.393+0000 7f6adf7d3700 1 -- 172.21.15.157:0/2645241675 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ad80c9130 con 0x7f6ad81514c0 2023-10-06T04:09:50.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.393+0000 7f6acd7fa700 1 -- 172.21.15.157:0/2645241675 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+5389 (secure 0 0 0) 0x7f6ad4055420 con 0x7f6ad81514c0 2023-10-06T04:09:50.395 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:09:50.395 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":86,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:09:46.762378+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6270,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:active","state_seq":19,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6270":{"gid":6270,"name":"e","rank":0,"incarnation":86,"state":"up:replay","state_seq":1,"addr":"172.21.15.157:6837/1036901172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1036901172},{"type":"v1","addr":"172.21.15.157:6837","nonce":1036901172}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:09:50.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.395+0000 7f6adf7d3700 1 -- 172.21.15.157:0/2645241675 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6abc066fb0 msgr2=0x7f6abc069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:50.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.395+0000 7f6adf7d3700 1 --2- 172.21.15.157:0/2645241675 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6abc066fb0 0x7f6abc069460 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0003a80 tx=0x7f6ad0007ec0 comp rx=0 tx=0).stop 2023-10-06T04:09:50.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.395+0000 7f6adf7d3700 1 -- 172.21.15.157:0/2645241675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ad81514c0 msgr2=0x7f6ad8169a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:50.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.395+0000 7f6adf7d3700 1 --2- 172.21.15.157:0/2645241675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ad81514c0 0x7f6ad8169a70 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009fa0 tx=0x7f6ad4009900 comp rx=0 tx=0).stop 2023-10-06T04:09:50.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.395+0000 7f6adf7d3700 1 -- 172.21.15.157:0/2645241675 shutdown_connections 2023-10-06T04:09:50.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.395+0000 7f6adf7d3700 1 --2- 172.21.15.157:0/2645241675 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6abc066fb0 0x7f6abc069460 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:50.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.395+0000 7f6adf7d3700 1 --2- 172.21.15.157:0/2645241675 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ad806c1f0 0x7f6ad8152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:50.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.395+0000 7f6adf7d3700 1 --2- 172.21.15.157:0/2645241675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ad81514c0 0x7f6ad8169a70 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:50.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.395+0000 7f6adf7d3700 1 --2- 172.21.15.157:0/2645241675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ad806cbb0 0x7f6ad8150f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:50.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.395+0000 7f6adf7d3700 1 -- 172.21.15.157:0/2645241675 >> 172.21.15.157:0/2645241675 conn(0x7f6ad806f230 msgr2=0x7f6ad80c76b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:50.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.395+0000 7f6adf7d3700 1 -- 172.21.15.157:0/2645241675 shutdown_connections 2023-10-06T04:09:50.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:50.395+0000 7f6adf7d3700 1 -- 172.21.15.157:0/2645241675 wait complete. 2023-10-06T04:09:50.398 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 86 2023-10-06T04:09:50.409 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:09:52.410 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:09:52.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.536+0000 7f8b0d884700 1 Processor -- start 2023-10-06T04:09:52.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.536+0000 7f8b0d884700 1 -- start start 2023-10-06T04:09:52.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.537+0000 7f8b0d884700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8b080c5db0 0x7f8b08152e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:52.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.537+0000 7f8b0d884700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8b08156b80 0x7f8b08153350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:52.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.537+0000 7f8b0d884700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b080c3270 con 0x7f8b080bc3c0 2023-10-06T04:09:52.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.537+0000 7f8b0d884700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b08157070 con 0x7f8b08156b80 2023-10-06T04:09:52.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.537+0000 7f8b0d884700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b0806dc20 con 0x7f8b080c5db0 2023-10-06T04:09:52.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.537+0000 7f8b077fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8b08156b80 0x7f8b08153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:52.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.537+0000 7f8b077fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8b08156b80 0x7f8b08153350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56386/0 (socket says 172.21.15.157:56386) 2023-10-06T04:09:52.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.537+0000 7f8b077fe700 1 -- 172.21.15.157:0/1580413267 learned_addr learned my addr 172.21.15.157:0/1580413267 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:52.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.538+0000 7f8b067fc700 1 --2- 172.21.15.157:0/1580413267 >> v2:172.21.15.175:3300/0 conn(0x7f8b080c5db0 0x7f8b08152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:52.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.538+0000 7f8b077fe700 1 -- 172.21.15.157:0/1580413267 >> v2:172.21.15.175:3300/0 conn(0x7f8b080c5db0 msgr2=0x7f8b08152e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:52.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.538+0000 7f8b077fe700 1 --2- 172.21.15.157:0/1580413267 >> v2:172.21.15.175:3300/0 conn(0x7f8b080c5db0 0x7f8b08152e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:52.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.538+0000 7f8b077fe700 1 -- 172.21.15.157:0/1580413267 >> v1:172.21.15.175:6789/0 conn(0x7f8b080bc3c0 legacy=0x7f8b0806a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:52.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.538+0000 7f8b077fe700 1 -- 172.21.15.157:0/1580413267 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b080c3510 con 0x7f8b08156b80 2023-10-06T04:09:52.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.538+0000 7f8b067fc700 1 --2- 172.21.15.157:0/1580413267 >> v2:172.21.15.175:3300/0 conn(0x7f8b080c5db0 0x7f8b08152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:09:52.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.538+0000 7f8b077fe700 1 --2- 172.21.15.157:0/1580413267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b08156b80 0x7f8b08153350 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8b08153cb0 tx=0x7f8b00009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ef3c85c12015cc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:52.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.538+0000 7f8b05ffb700 1 -- 172.21.15.157:0/1580413267 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0000e070 con 0x7f8b08156b80 2023-10-06T04:09:52.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.538+0000 7f8b05ffb700 1 -- 172.21.15.157:0/1580413267 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0000bae0 con 0x7f8b08156b80 2023-10-06T04:09:52.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.538+0000 7f8b05ffb700 1 -- 172.21.15.157:0/1580413267 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b000092e0 con 0x7f8b08156b80 2023-10-06T04:09:52.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.539+0000 7f8b0d884700 1 -- 172.21.15.157:0/1580413267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b08156b80 msgr2=0x7f8b08153350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:52.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.539+0000 7f8b0d884700 1 --2- 172.21.15.157:0/1580413267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b08156b80 0x7f8b08153350 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8b08153cb0 tx=0x7f8b00009840 comp rx=0 tx=0).stop 2023-10-06T04:09:52.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.539+0000 7f8b0d884700 1 -- 172.21.15.157:0/1580413267 shutdown_connections 2023-10-06T04:09:52.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.539+0000 7f8b0d884700 1 --2- 172.21.15.157:0/1580413267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b08156b80 0x7f8b08153350 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:52.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.539+0000 7f8b0d884700 1 --2- 172.21.15.157:0/1580413267 >> v2:172.21.15.175:3300/0 conn(0x7f8b080c5db0 0x7f8b08152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:52.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.539+0000 7f8b0d884700 1 -- 172.21.15.157:0/1580413267 >> 172.21.15.157:0/1580413267 conn(0x7f8b0806b510 msgr2=0x7f8b0806b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:52.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.539+0000 7f8b0d884700 1 -- 172.21.15.157:0/1580413267 shutdown_connections 2023-10-06T04:09:52.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.539+0000 7f8b0d884700 1 -- 172.21.15.157:0/1580413267 wait complete. 2023-10-06T04:09:52.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.540+0000 7f8b0d884700 1 Processor -- start 2023-10-06T04:09:52.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.540+0000 7f8b0d884700 1 -- start start 2023-10-06T04:09:52.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.541+0000 7f8b0d884700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b080bc3c0 0x7f8b0814e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:52.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.541+0000 7f8b0d884700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b080c5db0 0x7f8b080bb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:52.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.541+0000 7f8b0d884700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b08156b80 0x7f8b0814c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:52.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.541+0000 7f8b0d884700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b08151c20 con 0x7f8b08156b80 2023-10-06T04:09:52.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.541+0000 7f8b0d884700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b08169d60 con 0x7f8b080bc3c0 2023-10-06T04:09:52.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.541+0000 7f8b0d884700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b08153fe0 con 0x7f8b080c5db0 2023-10-06T04:09:52.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.541+0000 7f8b077fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b08156b80 0x7f8b0814c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:52.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.541+0000 7f8b077fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b08156b80 0x7f8b0814c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56402/0 (socket says 172.21.15.157:56402) 2023-10-06T04:09:52.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.541+0000 7f8b077fe700 1 -- 172.21.15.157:0/791758707 learned_addr learned my addr 172.21.15.157:0/791758707 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:52.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.541+0000 7f8b06ffd700 1 --2- 172.21.15.157:0/791758707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b080bc3c0 0x7f8b0814e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:52.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.541+0000 7f8b067fc700 1 --2- 172.21.15.157:0/791758707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b080c5db0 0x7f8b080bb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:52.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.541+0000 7f8b077fe700 1 -- 172.21.15.157:0/791758707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b080c5db0 msgr2=0x7f8b080bb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:52.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.541+0000 7f8b077fe700 1 --2- 172.21.15.157:0/791758707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b080c5db0 0x7f8b080bb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:52.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.541+0000 7f8b077fe700 1 -- 172.21.15.157:0/791758707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b080bc3c0 msgr2=0x7f8b0814e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:52.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.541+0000 7f8b077fe700 1 --2- 172.21.15.157:0/791758707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b080bc3c0 0x7f8b0814e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:52.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.541+0000 7f8b077fe700 1 -- 172.21.15.157:0/791758707 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b00009ae0 con 0x7f8b08156b80 2023-10-06T04:09:52.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.542+0000 7f8b077fe700 1 --2- 172.21.15.157:0/791758707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b08156b80 0x7f8b0814c4e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f8b0806afe0 tx=0x7f8b00018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:52.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.542+0000 7f8aeeffd700 1 -- 172.21.15.157:0/791758707 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b000032f0 con 0x7f8b08156b80 2023-10-06T04:09:52.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.542+0000 7f8aeeffd700 1 -- 172.21.15.157:0/791758707 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b00008ce0 con 0x7f8b08156b80 2023-10-06T04:09:52.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.542+0000 7f8b0d884700 1 -- 172.21.15.157:0/791758707 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b08155b90 con 0x7f8b08156b80 2023-10-06T04:09:52.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.542+0000 7f8aeeffd700 1 -- 172.21.15.157:0/791758707 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0001c660 con 0x7f8b08156b80 2023-10-06T04:09:52.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.542+0000 7f8b0d884700 1 -- 172.21.15.157:0/791758707 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b08155e50 con 0x7f8b08156b80 2023-10-06T04:09:52.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.543+0000 7f8b0d884700 1 -- 172.21.15.157:0/791758707 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b080c45d0 con 0x7f8b08156b80 2023-10-06T04:09:52.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.544+0000 7f8aeeffd700 1 -- 172.21.15.157:0/791758707 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8b00003040 con 0x7f8b08156b80 2023-10-06T04:09:52.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.544+0000 7f8aeeffd700 1 --2- 172.21.15.157:0/791758707 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8af4066f30 0x7f8af40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:52.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.544+0000 7f8aeeffd700 1 -- 172.21.15.157:0/791758707 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f8b00086a10 con 0x7f8b08156b80 2023-10-06T04:09:52.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.544+0000 7f8b06ffd700 1 --2- 172.21.15.157:0/791758707 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8af4066f30 0x7f8af40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:52.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.545+0000 7f8b06ffd700 1 --2- 172.21.15.157:0/791758707 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8af4066f30 0x7f8af40693e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8af8003a80 tx=0x7f8af8007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:52.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.548+0000 7f8aeeffd700 1 -- 172.21.15.157:0/791758707 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b00053910 con 0x7f8b08156b80 2023-10-06T04:09:52.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.784+0000 7f8b0d884700 1 -- 172.21.15.157:0/791758707 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b080c48b0 con 0x7f8b08156b80 2023-10-06T04:09:52.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.785+0000 7f8aeeffd700 1 -- 172.21.15.157:0/791758707 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+5389 (secure 0 0 0) 0x7f8b00056f30 con 0x7f8b08156b80 2023-10-06T04:09:52.787 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:09:52.787 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":86,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:09:46.762378+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6270,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:active","state_seq":19,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6270":{"gid":6270,"name":"e","rank":0,"incarnation":86,"state":"up:replay","state_seq":1,"addr":"172.21.15.157:6837/1036901172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1036901172},{"type":"v1","addr":"172.21.15.157:6837","nonce":1036901172}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:09:52.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.787+0000 7f8b0d884700 1 -- 172.21.15.157:0/791758707 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8af4066f30 msgr2=0x7f8af40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:52.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.787+0000 7f8b0d884700 1 --2- 172.21.15.157:0/791758707 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8af4066f30 0x7f8af40693e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8af8003a80 tx=0x7f8af8007ec0 comp rx=0 tx=0).stop 2023-10-06T04:09:52.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.787+0000 7f8b0d884700 1 -- 172.21.15.157:0/791758707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b08156b80 msgr2=0x7f8b0814c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:52.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.787+0000 7f8b0d884700 1 --2- 172.21.15.157:0/791758707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b08156b80 0x7f8b0814c4e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f8b0806afe0 tx=0x7f8b00018000 comp rx=0 tx=0).stop 2023-10-06T04:09:52.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.787+0000 7f8b0d884700 1 -- 172.21.15.157:0/791758707 shutdown_connections 2023-10-06T04:09:52.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.787+0000 7f8b0d884700 1 --2- 172.21.15.157:0/791758707 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f8af4066f30 0x7f8af40693e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:52.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.787+0000 7f8b0d884700 1 --2- 172.21.15.157:0/791758707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b080bc3c0 0x7f8b0814e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:52.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.787+0000 7f8b0d884700 1 --2- 172.21.15.157:0/791758707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b08156b80 0x7f8b0814c4e0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:52.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.787+0000 7f8b0d884700 1 --2- 172.21.15.157:0/791758707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b080c5db0 0x7f8b080bb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:52.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.787+0000 7f8b0d884700 1 -- 172.21.15.157:0/791758707 >> 172.21.15.157:0/791758707 conn(0x7f8b0806b510 msgr2=0x7f8b0806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:52.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.788+0000 7f8b0d884700 1 -- 172.21.15.157:0/791758707 shutdown_connections 2023-10-06T04:09:52.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:52.788+0000 7f8b0d884700 1 -- 172.21.15.157:0/791758707 wait complete. 2023-10-06T04:09:52.791 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 86 2023-10-06T04:09:52.801 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:09:54.802 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:09:54.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.916+0000 7fb313b23700 1 Processor -- start 2023-10-06T04:09:54.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.916+0000 7fb313b23700 1 -- start start 2023-10-06T04:09:54.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.917+0000 7fb313b23700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb30c06cbb0 0x7fb30c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:54.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.917+0000 7fb313b23700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb30c06c1f0 0x7fb30c06da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:54.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.917+0000 7fb313b23700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb30c0c9780 con 0x7fb30c06cf80 2023-10-06T04:09:54.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.917+0000 7fb313b23700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb30c0c8c40 con 0x7fb30c06c1f0 2023-10-06T04:09:54.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.917+0000 7fb313b23700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb30c06e0c0 con 0x7fb30c06cbb0 2023-10-06T04:09:54.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.917+0000 7fb3110be700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb30c06c1f0 0x7fb30c06da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:54.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.917+0000 7fb3110be700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb30c06c1f0 0x7fb30c06da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53030/0 (socket says 172.21.15.157:53030) 2023-10-06T04:09:54.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.917+0000 7fb3120c0700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb30c06cf80 0x7fb30c14fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:56046/0 (socket says 172.21.15.157:56046) 2023-10-06T04:09:54.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.917+0000 7fb3110be700 1 -- 172.21.15.157:0/2775383366 learned_addr learned my addr 172.21.15.157:0/2775383366 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:54.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.917+0000 7fb3118bf700 1 --2- 172.21.15.157:0/2775383366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb30c06cbb0 0x7fb30c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:54.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.917+0000 7fb3110be700 1 -- 172.21.15.157:0/2775383366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb30c06cbb0 msgr2=0x7fb30c06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:54.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.917+0000 7fb3110be700 1 --2- 172.21.15.157:0/2775383366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb30c06cbb0 0x7fb30c06d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:54.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.917+0000 7fb3110be700 1 -- 172.21.15.157:0/2775383366 >> v1:172.21.15.157:6789/0 conn(0x7fb30c06cf80 legacy=0x7fb30c14fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:54.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.917+0000 7fb3108bd700 1 -- 172.21.15.157:0/2775383366 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1970072726 0 0) 0x7fb30c0c9780 con 0x7fb30c06cf80 2023-10-06T04:09:54.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.917+0000 7fb3110be700 1 -- 172.21.15.157:0/2775383366 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb30c0c8e50 con 0x7fb30c06c1f0 2023-10-06T04:09:54.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.917+0000 7fb3110be700 1 --2- 172.21.15.157:0/2775383366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb30c06c1f0 0x7fb30c06da70 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc0099e0 tx=0x7fb2fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2e01dcf04475b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:54.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.918+0000 7fb3108bd700 1 -- 172.21.15.157:0/2775383366 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc00b8b0 con 0x7fb30c06c1f0 2023-10-06T04:09:54.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.918+0000 7fb3108bd700 1 -- 172.21.15.157:0/2775383366 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2fc00ba10 con 0x7fb30c06c1f0 2023-10-06T04:09:54.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.918+0000 7fb3108bd700 1 -- 172.21.15.157:0/2775383366 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc00bdb0 con 0x7fb30c06c1f0 2023-10-06T04:09:54.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.918+0000 7fb313b23700 1 -- 172.21.15.157:0/2775383366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb30c06c1f0 msgr2=0x7fb30c06da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:54.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.918+0000 7fb313b23700 1 --2- 172.21.15.157:0/2775383366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb30c06c1f0 0x7fb30c06da70 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc0099e0 tx=0x7fb2fc009250 comp rx=0 tx=0).stop 2023-10-06T04:09:54.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.918+0000 7fb313b23700 1 -- 172.21.15.157:0/2775383366 shutdown_connections 2023-10-06T04:09:54.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.918+0000 7fb313b23700 1 --2- 172.21.15.157:0/2775383366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb30c06cbb0 0x7fb30c06d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:54.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.918+0000 7fb313b23700 1 --2- 172.21.15.157:0/2775383366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb30c06c1f0 0x7fb30c06da70 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:54.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.918+0000 7fb313b23700 1 -- 172.21.15.157:0/2775383366 >> 172.21.15.157:0/2775383366 conn(0x7fb30c06f230 msgr2=0x7fb30c06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:54.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.918+0000 7fb313b23700 1 -- 172.21.15.157:0/2775383366 shutdown_connections 2023-10-06T04:09:54.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.918+0000 7fb313b23700 1 -- 172.21.15.157:0/2775383366 wait complete. 2023-10-06T04:09:54.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.919+0000 7fb313b23700 1 Processor -- start 2023-10-06T04:09:54.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.919+0000 7fb313b23700 1 -- start start 2023-10-06T04:09:54.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.920+0000 7fb313b23700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb30c06c1f0 0x7fb30c156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:54.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.920+0000 7fb313b23700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb30c06cbb0 0x7fb30c158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:54.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.920+0000 7fb313b23700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb30c06cf80 0x7fb30c1591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:54.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.920+0000 7fb313b23700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb30c16aac0 con 0x7fb30c06c1f0 2023-10-06T04:09:54.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.920+0000 7fb313b23700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb30c1540c0 con 0x7fb30c06cbb0 2023-10-06T04:09:54.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.920+0000 7fb313b23700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb30c16a560 con 0x7fb30c06cf80 2023-10-06T04:09:54.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.920+0000 7fb3118bf700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb30c06c1f0 0x7fb30c156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:54.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.920+0000 7fb3118bf700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb30c06c1f0 0x7fb30c156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53044/0 (socket says 172.21.15.157:53044) 2023-10-06T04:09:54.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.920+0000 7fb3118bf700 1 -- 172.21.15.157:0/140028031 learned_addr learned my addr 172.21.15.157:0/140028031 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:54.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.920+0000 7fb3120c0700 1 --2- 172.21.15.157:0/140028031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb30c06cf80 0x7fb30c1591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:54.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.920+0000 7fb3110be700 1 --2- 172.21.15.157:0/140028031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb30c06cbb0 0x7fb30c158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:54.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.920+0000 7fb3118bf700 1 -- 172.21.15.157:0/140028031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb30c06cf80 msgr2=0x7fb30c1591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:54.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.920+0000 7fb3118bf700 1 --2- 172.21.15.157:0/140028031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb30c06cf80 0x7fb30c1591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:54.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.920+0000 7fb3118bf700 1 -- 172.21.15.157:0/140028031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb30c06cbb0 msgr2=0x7fb30c158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:54.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.920+0000 7fb3118bf700 1 --2- 172.21.15.157:0/140028031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb30c06cbb0 0x7fb30c158ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:54.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.920+0000 7fb3118bf700 1 -- 172.21.15.157:0/140028031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2fc0094f0 con 0x7fb30c06c1f0 2023-10-06T04:09:54.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.920+0000 7fb3110be700 1 --2- 172.21.15.157:0/140028031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb30c06cbb0 0x7fb30c158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:09:54.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.920+0000 7fb3118bf700 1 --2- 172.21.15.157:0/140028031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb30c06c1f0 0x7fb30c156630 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb304009f20 tx=0x7fb30400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:54.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.921+0000 7fb301ffb700 1 -- 172.21.15.157:0/140028031 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30400cc80 con 0x7fb30c06c1f0 2023-10-06T04:09:54.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.921+0000 7fb301ffb700 1 -- 172.21.15.157:0/140028031 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30400f040 con 0x7fb30c06c1f0 2023-10-06T04:09:54.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.921+0000 7fb313b23700 1 -- 172.21.15.157:0/140028031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb30c0aeee0 con 0x7fb30c06c1f0 2023-10-06T04:09:54.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.921+0000 7fb301ffb700 1 -- 172.21.15.157:0/140028031 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb304014480 con 0x7fb30c06c1f0 2023-10-06T04:09:54.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.921+0000 7fb313b23700 1 -- 172.21.15.157:0/140028031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb30c157400 con 0x7fb30c06c1f0 2023-10-06T04:09:54.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.922+0000 7fb313b23700 1 -- 172.21.15.157:0/140028031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb30c089ca0 con 0x7fb30c06c1f0 2023-10-06T04:09:54.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.922+0000 7fb301ffb700 1 -- 172.21.15.157:0/140028031 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb3040147f0 con 0x7fb30c06c1f0 2023-10-06T04:09:54.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.922+0000 7fb301ffb700 1 --2- 172.21.15.157:0/140028031 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb2f0066f60 0x7fb2f0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:54.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.923+0000 7fb301ffb700 1 -- 172.21.15.157:0/140028031 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7fb304085350 con 0x7fb30c06c1f0 2023-10-06T04:09:54.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.923+0000 7fb3110be700 1 --2- 172.21.15.157:0/140028031 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb2f0066f60 0x7fb2f0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:54.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.923+0000 7fb3110be700 1 --2- 172.21.15.157:0/140028031 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb2f0066f60 0x7fb2f0069410 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc0099e0 tx=0x7fb2fc017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:54.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:54.927+0000 7fb301ffb700 1 -- 172.21.15.157:0/140028031 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3040521d0 con 0x7fb30c06c1f0 2023-10-06T04:09:55.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:55.154+0000 7fb313b23700 1 -- 172.21.15.157:0/140028031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb30c0c9130 con 0x7fb30c06c1f0 2023-10-06T04:09:55.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:55.154+0000 7fb301ffb700 1 -- 172.21.15.157:0/140028031 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+5391 (secure 0 0 0) 0x7fb3040557f0 con 0x7fb30c06c1f0 2023-10-06T04:09:55.157 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:09:55.157 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":87,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:09:55.008132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6270,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:active","state_seq":19,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6270":{"gid":6270,"name":"e","rank":0,"incarnation":86,"state":"up:resolve","state_seq":13,"addr":"172.21.15.157:6837/1036901172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1036901172},{"type":"v1","addr":"172.21.15.157:6837","nonce":1036901172}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:09:55.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:55.156+0000 7fb313b23700 1 -- 172.21.15.157:0/140028031 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb2f0066f60 msgr2=0x7fb2f0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:55.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:55.156+0000 7fb313b23700 1 --2- 172.21.15.157:0/140028031 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb2f0066f60 0x7fb2f0069410 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc0099e0 tx=0x7fb2fc017000 comp rx=0 tx=0).stop 2023-10-06T04:09:55.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:55.156+0000 7fb313b23700 1 -- 172.21.15.157:0/140028031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb30c06c1f0 msgr2=0x7fb30c156630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:55.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:55.156+0000 7fb313b23700 1 --2- 172.21.15.157:0/140028031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb30c06c1f0 0x7fb30c156630 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb304009f20 tx=0x7fb30400b040 comp rx=0 tx=0).stop 2023-10-06T04:09:55.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:55.156+0000 7fb313b23700 1 -- 172.21.15.157:0/140028031 shutdown_connections 2023-10-06T04:09:55.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:55.156+0000 7fb313b23700 1 --2- 172.21.15.157:0/140028031 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb2f0066f60 0x7fb2f0069410 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:55.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:55.156+0000 7fb313b23700 1 --2- 172.21.15.157:0/140028031 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb30c06cf80 0x7fb30c1591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:55.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:55.156+0000 7fb313b23700 1 --2- 172.21.15.157:0/140028031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb30c06c1f0 0x7fb30c156630 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:55.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:55.156+0000 7fb313b23700 1 --2- 172.21.15.157:0/140028031 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb30c06cbb0 0x7fb30c158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:55.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:55.156+0000 7fb313b23700 1 -- 172.21.15.157:0/140028031 >> 172.21.15.157:0/140028031 conn(0x7fb30c06f230 msgr2=0x7fb30c150e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:55.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:55.156+0000 7fb313b23700 1 -- 172.21.15.157:0/140028031 shutdown_connections 2023-10-06T04:09:55.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:55.156+0000 7fb313b23700 1 -- 172.21.15.157:0/140028031 wait complete. 2023-10-06T04:09:55.160 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 87 2023-10-06T04:09:55.170 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:09:57.171 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:09:57.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.295+0000 7f3c37efa700 1 Processor -- start 2023-10-06T04:09:57.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.295+0000 7f3c37efa700 1 -- start start 2023-10-06T04:09:57.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.296+0000 7f3c37efa700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c3006c810 0x7f3c30150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:57.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.296+0000 7f3c37efa700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c300bf5e0 con 0x7f3c3006c440 2023-10-06T04:09:57.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.296+0000 7f3c37efa700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c30158010 con 0x7f3c300c4e30 2023-10-06T04:09:57.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.296+0000 7f3c37efa700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c30157d30 con 0x7f3c3006c810 2023-10-06T04:09:57.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.296+0000 7f3c35c96700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f3c3006c440 0x7f3c30069130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:56062/0 (socket says 172.21.15.157:56062) 2023-10-06T04:09:57.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.296+0000 7f3c35c96700 1 -- 172.21.15.157:0/2042778197 learned_addr learned my addr 172.21.15.157:0/2042778197 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:09:57.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.296+0000 7f3c35495700 1 --2- 172.21.15.157:0/2042778197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c3006c810 0x7f3c30150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:57.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.296+0000 7f3c34c94700 1 -- 172.21.15.157:0/2042778197 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 991901869 0 0) 0x7f3c300bf5e0 con 0x7f3c3006c440 2023-10-06T04:09:57.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.296+0000 7f3c34c94700 1 -- 172.21.15.157:0/2042778197 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c1c003570 con 0x7f3c3006c440 2023-10-06T04:09:57.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.297+0000 7f3c34c94700 1 -- 172.21.15.157:0/2042778197 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 932566827 0 0) 0x7f3c1c003570 con 0x7f3c3006c440 2023-10-06T04:09:57.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.297+0000 7f3c34c94700 1 -- 172.21.15.157:0/2042778197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c3006c810 msgr2=0x7f3c30150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:57.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.297+0000 7f3c34c94700 1 --2- 172.21.15.157:0/2042778197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c3006c810 0x7f3c30150ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:57.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.297+0000 7f3c34c94700 1 -- 172.21.15.157:0/2042778197 >> v1:172.21.15.175:6789/0 conn(0x7f3c300c4e30 legacy=0x7f3c3015e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:57.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.297+0000 7f3c34c94700 1 -- 172.21.15.157:0/2042778197 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c300c1090 con 0x7f3c3006c440 2023-10-06T04:09:57.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.297+0000 7f3c34c94700 1 -- 172.21.15.157:0/2042778197 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f3c280017f0 con 0x7f3c3006c440 2023-10-06T04:09:57.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.297+0000 7f3c34c94700 1 -- 172.21.15.157:0/2042778197 >> v1:172.21.15.157:6789/0 conn(0x7f3c3006c440 legacy=0x7f3c30069130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:57.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.297+0000 7f3c34c94700 1 --2- 172.21.15.157:0/2042778197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c3006c810 0x7f3c1c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:57.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.297+0000 7f3c34c94700 1 --2- 172.21.15.157:0/2042778197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c1c004010 0x7f3c1c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:57.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.297+0000 7f3c34c94700 1 --2- 172.21.15.157:0/2042778197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c1c004d50 0x7f3c1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:57.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.297+0000 7f3c34c94700 1 -- 172.21.15.157:0/2042778197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c30157d30 con 0x7f3c1c004010 2023-10-06T04:09:57.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.297+0000 7f3c34c94700 1 -- 172.21.15.157:0/2042778197 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1c00bbf0 con 0x7f3c1c004d50 2023-10-06T04:09:57.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.297+0000 7f3c34c94700 1 -- 172.21.15.157:0/2042778197 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1c00bdb0 con 0x7f3c3006c810 2023-10-06T04:09:57.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.297+0000 7f3c35495700 1 --2- 172.21.15.157:0/2042778197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c1c004010 0x7f3c1c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:57.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.297+0000 7f3c35c96700 1 --2- 172.21.15.157:0/2042778197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c3006c810 0x7f3c1c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:57.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.297+0000 7f3c36497700 1 --2- 172.21.15.157:0/2042778197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c1c004d50 0x7f3c1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:57.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.297+0000 7f3c35495700 1 -- 172.21.15.157:0/2042778197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c3006c810 msgr2=0x7f3c1c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:57.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.297+0000 7f3c35495700 1 --2- 172.21.15.157:0/2042778197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c3006c810 0x7f3c1c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:57.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.297+0000 7f3c35495700 1 -- 172.21.15.157:0/2042778197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c1c004d50 msgr2=0x7f3c1c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:57.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.297+0000 7f3c35495700 1 --2- 172.21.15.157:0/2042778197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c1c004d50 0x7f3c1c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:57.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.297+0000 7f3c35495700 1 -- 172.21.15.157:0/2042778197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c1c003570 con 0x7f3c1c004010 2023-10-06T04:09:57.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.297+0000 7f3c35495700 1 --2- 172.21.15.157:0/2042778197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c1c004010 0x7f3c1c004810 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3c20006f00 tx=0x7f3c200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28b6c4154b6484d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:57.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.297+0000 7f3c34c94700 1 -- 172.21.15.157:0/2042778197 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2000e710 con 0x7f3c1c004010 2023-10-06T04:09:57.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.297+0000 7f3c34c94700 1 -- 172.21.15.157:0/2042778197 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2000e870 con 0x7f3c1c004010 2023-10-06T04:09:57.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.297+0000 7f3c34c94700 1 -- 172.21.15.157:0/2042778197 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2000ec30 con 0x7f3c1c004010 2023-10-06T04:09:57.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.298+0000 7f3c37efa700 1 -- 172.21.15.157:0/2042778197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c1c004010 msgr2=0x7f3c1c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:57.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.298+0000 7f3c37efa700 1 --2- 172.21.15.157:0/2042778197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c1c004010 0x7f3c1c004810 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3c20006f00 tx=0x7f3c200091a0 comp rx=0 tx=0).stop 2023-10-06T04:09:57.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.298+0000 7f3c36497700 1 -- 172.21.15.157:0/2042778197 reap_dead start 2023-10-06T04:09:57.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.298+0000 7f3c37efa700 1 -- 172.21.15.157:0/2042778197 shutdown_connections 2023-10-06T04:09:57.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.298+0000 7f3c37efa700 1 -- 172.21.15.157:0/2042778197 >> 172.21.15.157:0/2042778197 conn(0x7f3c30068370 msgr2=0x7f3c30068770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:57.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.298+0000 7f3c37efa700 1 -- 172.21.15.157:0/2042778197 shutdown_connections 2023-10-06T04:09:57.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.298+0000 7f3c37efa700 1 -- 172.21.15.157:0/2042778197 wait complete. 2023-10-06T04:09:57.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.299+0000 7f3c37efa700 1 Processor -- start 2023-10-06T04:09:57.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.299+0000 7f3c37efa700 1 -- start start 2023-10-06T04:09:57.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.300+0000 7f3c37efa700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c1c004010 0x7f3c300c3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:57.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.300+0000 7f3c37efa700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c30165780 0x7f3c30164ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:57.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.300+0000 7f3c37efa700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c300c6d80 0x7f3c300abc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:57.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.300+0000 7f3c37efa700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c30154b70 con 0x7f3c30165780 2023-10-06T04:09:57.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.300+0000 7f3c37efa700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c300c4010 con 0x7f3c300c6d80 2023-10-06T04:09:57.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.300+0000 7f3c37efa700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c300c2c70 con 0x7f3c1c004010 2023-10-06T04:09:57.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.300+0000 7f3c35495700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c30165780 0x7f3c30164ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:57.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.300+0000 7f3c35495700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c30165780 0x7f3c30164ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53066/0 (socket says 172.21.15.157:53066) 2023-10-06T04:09:57.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.300+0000 7f3c35495700 1 -- 172.21.15.157:0/2445338664 learned_addr learned my addr 172.21.15.157:0/2445338664 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:57.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.300+0000 7f3c35c96700 1 --2- 172.21.15.157:0/2445338664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c1c004010 0x7f3c300c3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:57.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.300+0000 7f3c36497700 1 --2- 172.21.15.157:0/2445338664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c300c6d80 0x7f3c300abc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:57.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.300+0000 7f3c35495700 1 -- 172.21.15.157:0/2445338664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c1c004010 msgr2=0x7f3c300c3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:57.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.300+0000 7f3c35495700 1 --2- 172.21.15.157:0/2445338664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c1c004010 0x7f3c300c3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:57.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.300+0000 7f3c35495700 1 -- 172.21.15.157:0/2445338664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c300c6d80 msgr2=0x7f3c300abc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:57.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.300+0000 7f3c35495700 1 --2- 172.21.15.157:0/2445338664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c300c6d80 0x7f3c300abc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:57.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.300+0000 7f3c35495700 1 -- 172.21.15.157:0/2445338664 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c301651e0 con 0x7f3c30165780 2023-10-06T04:09:57.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.300+0000 7f3c36497700 1 --2- 172.21.15.157:0/2445338664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c300c6d80 0x7f3c300abc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:09:57.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.300+0000 7f3c35495700 1 --2- 172.21.15.157:0/2445338664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c30165780 0x7f3c30164ca0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000e4b0 tx=0x7f3c2001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:57.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.301+0000 7f3c25ffb700 1 -- 172.21.15.157:0/2445338664 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c20023070 con 0x7f3c30165780 2023-10-06T04:09:57.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.301+0000 7f3c25ffb700 1 -- 172.21.15.157:0/2445338664 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2000e9e0 con 0x7f3c30165780 2023-10-06T04:09:57.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.301+0000 7f3c37efa700 1 -- 172.21.15.157:0/2445338664 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c30165b90 con 0x7f3c30165780 2023-10-06T04:09:57.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.301+0000 7f3c25ffb700 1 -- 172.21.15.157:0/2445338664 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2001e030 con 0x7f3c30165780 2023-10-06T04:09:57.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.301+0000 7f3c37efa700 1 -- 172.21.15.157:0/2445338664 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c300c3d30 con 0x7f3c30165780 2023-10-06T04:09:57.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.302+0000 7f3c37efa700 1 -- 172.21.15.157:0/2445338664 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c3006a8f0 con 0x7f3c30165780 2023-10-06T04:09:57.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.302+0000 7f3c25ffb700 1 -- 172.21.15.157:0/2445338664 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3c20013470 con 0x7f3c30165780 2023-10-06T04:09:57.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.302+0000 7f3c25ffb700 1 --2- 172.21.15.157:0/2445338664 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c14066f60 0x7f3c14069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:57.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.303+0000 7f3c25ffb700 1 -- 172.21.15.157:0/2445338664 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f3c200860e0 con 0x7f3c30165780 2023-10-06T04:09:57.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.303+0000 7f3c35c96700 1 --2- 172.21.15.157:0/2445338664 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c14066f60 0x7f3c14069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:57.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.303+0000 7f3c35c96700 1 --2- 172.21.15.157:0/2445338664 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c14066f60 0x7f3c14069410 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3c28002060 tx=0x7f3c28006040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:57.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.307+0000 7f3c25ffb700 1 -- 172.21.15.157:0/2445338664 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c20052ec0 con 0x7f3c30165780 2023-10-06T04:09:57.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.532+0000 7f3c37efa700 1 -- 172.21.15.157:0/2445338664 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c300c2db0 con 0x7f3c30165780 2023-10-06T04:09:57.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.533+0000 7f3c25ffb700 1 -- 172.21.15.157:0/2445338664 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+5391 (secure 0 0 0) 0x7f3c200564e0 con 0x7f3c30165780 2023-10-06T04:09:57.536 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:09:57.536 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":87,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:09:55.008132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6270,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:active","state_seq":19,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6270":{"gid":6270,"name":"e","rank":0,"incarnation":86,"state":"up:resolve","state_seq":13,"addr":"172.21.15.157:6837/1036901172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1036901172},{"type":"v1","addr":"172.21.15.157:6837","nonce":1036901172}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:09:57.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.535+0000 7f3c37efa700 1 -- 172.21.15.157:0/2445338664 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c14066f60 msgr2=0x7f3c14069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:57.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.535+0000 7f3c37efa700 1 --2- 172.21.15.157:0/2445338664 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c14066f60 0x7f3c14069410 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3c28002060 tx=0x7f3c28006040 comp rx=0 tx=0).stop 2023-10-06T04:09:57.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.535+0000 7f3c37efa700 1 -- 172.21.15.157:0/2445338664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c30165780 msgr2=0x7f3c30164ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:57.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.535+0000 7f3c37efa700 1 --2- 172.21.15.157:0/2445338664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c30165780 0x7f3c30164ca0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000e4b0 tx=0x7f3c2001a000 comp rx=0 tx=0).stop 2023-10-06T04:09:57.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.535+0000 7f3c37efa700 1 -- 172.21.15.157:0/2445338664 shutdown_connections 2023-10-06T04:09:57.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.535+0000 7f3c37efa700 1 --2- 172.21.15.157:0/2445338664 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3c14066f60 0x7f3c14069410 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:57.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.535+0000 7f3c37efa700 1 --2- 172.21.15.157:0/2445338664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c300c6d80 0x7f3c300abc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:57.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.535+0000 7f3c37efa700 1 --2- 172.21.15.157:0/2445338664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c1c004010 0x7f3c300c3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:57.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.535+0000 7f3c37efa700 1 --2- 172.21.15.157:0/2445338664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c30165780 0x7f3c30164ca0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:57.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.535+0000 7f3c37efa700 1 -- 172.21.15.157:0/2445338664 >> 172.21.15.157:0/2445338664 conn(0x7f3c30068370 msgr2=0x7f3c300c96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:57.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.535+0000 7f3c37efa700 1 -- 172.21.15.157:0/2445338664 shutdown_connections 2023-10-06T04:09:57.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:57.535+0000 7f3c37efa700 1 -- 172.21.15.157:0/2445338664 wait complete. 2023-10-06T04:09:57.539 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 87 2023-10-06T04:09:57.549 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:09:59.550 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:09:59.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.674+0000 7f2680699700 1 Processor -- start 2023-10-06T04:09:59.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.674+0000 7f2680699700 1 -- start start 2023-10-06T04:09:59.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.675+0000 7f2680699700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2678068d10 0x7f267806a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:59.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.675+0000 7f2680699700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f267806d460 0x7f267806ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:59.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.675+0000 7f2680699700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26780bed40 con 0x7f26780be970 2023-10-06T04:09:59.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.675+0000 7f2680699700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2678157010 con 0x7f2678068d10 2023-10-06T04:09:59.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.675+0000 7f2680699700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f267806f9f0 con 0x7f267806d460 2023-10-06T04:09:59.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.675+0000 7f267dc34700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2678068d10 0x7f267806a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:59.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.676+0000 7f267dc34700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2678068d10 0x7f267806a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53070/0 (socket says 172.21.15.157:53070) 2023-10-06T04:09:59.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.676+0000 7f267dc34700 1 -- 172.21.15.157:0/1796043104 learned_addr learned my addr 172.21.15.157:0/1796043104 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:59.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.676+0000 7f267ec36700 1 --2- 172.21.15.157:0/1796043104 >> v2:172.21.15.175:3300/0 conn(0x7f267806d460 0x7f267806ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:59.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.676+0000 7f267dc34700 1 -- 172.21.15.157:0/1796043104 >> v2:172.21.15.175:3300/0 conn(0x7f267806d460 msgr2=0x7f267806ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:09:59.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.676+0000 7f267dc34700 1 --2- 172.21.15.157:0/1796043104 >> v2:172.21.15.175:3300/0 conn(0x7f267806d460 0x7f267806ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:59.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.676+0000 7f267dc34700 1 -- 172.21.15.157:0/1796043104 >> v1:172.21.15.175:6789/0 conn(0x7f26780be970 legacy=0x7f267806df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:59.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.676+0000 7f267dc34700 1 -- 172.21.15.157:0/1796043104 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26781574a0 con 0x7f2678068d10 2023-10-06T04:09:59.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.676+0000 7f267dc34700 1 --2- 172.21.15.157:0/1796043104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2678068d10 0x7f267806a600 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f26780692f0 tx=0x7f266800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40515747b72e19bd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:59.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.676+0000 7f267d433700 1 -- 172.21.15.157:0/1796043104 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266800dba0 con 0x7f2678068d10 2023-10-06T04:09:59.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.676+0000 7f267d433700 1 -- 172.21.15.157:0/1796043104 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266800dd00 con 0x7f2678068d10 2023-10-06T04:09:59.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.676+0000 7f267d433700 1 -- 172.21.15.157:0/1796043104 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26680196d0 con 0x7f2678068d10 2023-10-06T04:09:59.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.677+0000 7f2680699700 1 -- 172.21.15.157:0/1796043104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2678068d10 msgr2=0x7f267806a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:59.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.677+0000 7f2680699700 1 --2- 172.21.15.157:0/1796043104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2678068d10 0x7f267806a600 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f26780692f0 tx=0x7f266800b4f0 comp rx=0 tx=0).stop 2023-10-06T04:09:59.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.677+0000 7f2680699700 1 -- 172.21.15.157:0/1796043104 shutdown_connections 2023-10-06T04:09:59.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.677+0000 7f2680699700 1 --2- 172.21.15.157:0/1796043104 >> v2:172.21.15.175:3300/0 conn(0x7f267806d460 0x7f267806ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:59.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.677+0000 7f2680699700 1 --2- 172.21.15.157:0/1796043104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2678068d10 0x7f267806a600 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:59.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.677+0000 7f2680699700 1 -- 172.21.15.157:0/1796043104 >> 172.21.15.157:0/1796043104 conn(0x7f26780c8d70 msgr2=0x7f26780c4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:59.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.677+0000 7f2680699700 1 -- 172.21.15.157:0/1796043104 shutdown_connections 2023-10-06T04:09:59.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.677+0000 7f2680699700 1 -- 172.21.15.157:0/1796043104 wait complete. 2023-10-06T04:09:59.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.678+0000 7f2680699700 1 Processor -- start 2023-10-06T04:09:59.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.678+0000 7f2680699700 1 -- start start 2023-10-06T04:09:59.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.679+0000 7f2680699700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2678068d10 0x7f2678163270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:59.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.679+0000 7f2680699700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f267806d460 0x7f2678165530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:59.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.679+0000 7f2680699700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26780be970 0x7f2678165a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:59.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.679+0000 7f2680699700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267814e760 con 0x7f2678068d10 2023-10-06T04:09:59.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.679+0000 7f2680699700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26780c63b0 con 0x7f267806d460 2023-10-06T04:09:59.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.679+0000 7f2680699700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2678155f90 con 0x7f26780be970 2023-10-06T04:09:59.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.679+0000 7f267e435700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2678068d10 0x7f2678163270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:59.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.679+0000 7f267e435700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2678068d10 0x7f2678163270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53074/0 (socket says 172.21.15.157:53074) 2023-10-06T04:09:59.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.679+0000 7f267e435700 1 -- 172.21.15.157:0/3737600027 learned_addr learned my addr 172.21.15.157:0/3737600027 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:09:59.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.679+0000 7f267dc34700 1 --2- 172.21.15.157:0/3737600027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f267806d460 0x7f2678165530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:59.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.679+0000 7f267ec36700 1 --2- 172.21.15.157:0/3737600027 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26780be970 0x7f2678165a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:59.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.679+0000 7f267e435700 1 -- 172.21.15.157:0/3737600027 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26780be970 msgr2=0x7f2678165a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:59.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.679+0000 7f267e435700 1 --2- 172.21.15.157:0/3737600027 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26780be970 0x7f2678165a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:59.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.679+0000 7f267e435700 1 -- 172.21.15.157:0/3737600027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f267806d460 msgr2=0x7f2678165530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:59.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.679+0000 7f267e435700 1 --2- 172.21.15.157:0/3737600027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f267806d460 0x7f2678165530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:59.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.679+0000 7f267e435700 1 -- 172.21.15.157:0/3737600027 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266800b790 con 0x7f2678068d10 2023-10-06T04:09:59.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.679+0000 7f267ec36700 1 --2- 172.21.15.157:0/3737600027 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26780be970 0x7f2678165a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:09:59.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.679+0000 7f267e435700 1 --2- 172.21.15.157:0/3737600027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2678068d10 0x7f2678163270 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f267400cb40 tx=0x7f26740017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:59.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.679+0000 7f266e7fc700 1 -- 172.21.15.157:0/3737600027 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267400a7a0 con 0x7f2678068d10 2023-10-06T04:09:59.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.679+0000 7f266e7fc700 1 -- 172.21.15.157:0/3737600027 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267400a920 con 0x7f2678068d10 2023-10-06T04:09:59.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.679+0000 7f266e7fc700 1 -- 172.21.15.157:0/3737600027 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267400ace0 con 0x7f2678068d10 2023-10-06T04:09:59.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.679+0000 7f2680699700 1 -- 172.21.15.157:0/3737600027 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2678155410 con 0x7f2678068d10 2023-10-06T04:09:59.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.679+0000 7f2680699700 1 -- 172.21.15.157:0/3737600027 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26781561e0 con 0x7f2678068d10 2023-10-06T04:09:59.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.680+0000 7f2680699700 1 -- 172.21.15.157:0/3737600027 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26780716b0 con 0x7f2678068d10 2023-10-06T04:09:59.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.681+0000 7f266e7fc700 1 -- 172.21.15.157:0/3737600027 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2674003030 con 0x7f2678068d10 2023-10-06T04:09:59.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.681+0000 7f266e7fc700 1 --2- 172.21.15.157:0/3737600027 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f265c066f30 0x7f265c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:09:59.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.681+0000 7f266e7fc700 1 -- 172.21.15.157:0/3737600027 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f26740849e0 con 0x7f2678068d10 2023-10-06T04:09:59.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.681+0000 7f267dc34700 1 --2- 172.21.15.157:0/3737600027 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f265c066f30 0x7f265c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:09:59.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.682+0000 7f267dc34700 1 --2- 172.21.15.157:0/3737600027 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f265c066f30 0x7f265c0693e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f26780692f0 tx=0x7f266801b000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:09:59.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.686+0000 7f266e7fc700 1 -- 172.21.15.157:0/3737600027 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2674051860 con 0x7f2678068d10 2023-10-06T04:09:59.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.909+0000 7f2680699700 1 -- 172.21.15.157:0/3737600027 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26780c9170 con 0x7f2678068d10 2023-10-06T04:09:59.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.910+0000 7f266e7fc700 1 -- 172.21.15.157:0/3737600027 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+5391 (secure 0 0 0) 0x7f2674054e80 con 0x7f2678068d10 2023-10-06T04:09:59.914 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:09:59.914 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":87,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:09:55.008132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6270,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:active","state_seq":19,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6270":{"gid":6270,"name":"e","rank":0,"incarnation":86,"state":"up:resolve","state_seq":13,"addr":"172.21.15.157:6837/1036901172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1036901172},{"type":"v1","addr":"172.21.15.157:6837","nonce":1036901172}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:09:59.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.912+0000 7f2680699700 1 -- 172.21.15.157:0/3737600027 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f265c066f30 msgr2=0x7f265c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:59.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.912+0000 7f2680699700 1 --2- 172.21.15.157:0/3737600027 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f265c066f30 0x7f265c0693e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f26780692f0 tx=0x7f266801b000 comp rx=0 tx=0).stop 2023-10-06T04:09:59.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.912+0000 7f2680699700 1 -- 172.21.15.157:0/3737600027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2678068d10 msgr2=0x7f2678163270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:09:59.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.912+0000 7f2680699700 1 --2- 172.21.15.157:0/3737600027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2678068d10 0x7f2678163270 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f267400cb40 tx=0x7f26740017b0 comp rx=0 tx=0).stop 2023-10-06T04:09:59.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.912+0000 7f2680699700 1 -- 172.21.15.157:0/3737600027 shutdown_connections 2023-10-06T04:09:59.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.912+0000 7f2680699700 1 --2- 172.21.15.157:0/3737600027 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f265c066f30 0x7f265c0693e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:59.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.912+0000 7f2680699700 1 --2- 172.21.15.157:0/3737600027 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26780be970 0x7f2678165a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:59.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.912+0000 7f2680699700 1 --2- 172.21.15.157:0/3737600027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2678068d10 0x7f2678163270 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:59.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.912+0000 7f2680699700 1 --2- 172.21.15.157:0/3737600027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f267806d460 0x7f2678165530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:09:59.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.912+0000 7f2680699700 1 -- 172.21.15.157:0/3737600027 >> 172.21.15.157:0/3737600027 conn(0x7f26780c8d70 msgr2=0x7f26780c3640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:09:59.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.912+0000 7f2680699700 1 -- 172.21.15.157:0/3737600027 shutdown_connections 2023-10-06T04:09:59.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:09:59.912+0000 7f2680699700 1 -- 172.21.15.157:0/3737600027 wait complete. 2023-10-06T04:09:59.917 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 87 2023-10-06T04:09:59.927 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 87, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-06T03:57:10.718782+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 87, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_5812": { "addr": "172.21.15.175:6837/1436765859", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6836", "nonce": 1436765859, "type": "v2" }, { "addr": "172.21.15.175:6837", "nonce": 1436765859, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0 ], "features": 4540138320759226367, "flags": 0, "gid": 5812, "incarnation": 78, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:active", "state_seq": 19 }, "gid_6270": { "addr": "172.21.15.157:6837/1036901172", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6836", "nonce": 1036901172, "type": "v2" }, { "addr": "172.21.15.157:6837", "nonce": 1036901172, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 6270, "incarnation": 86, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:resolve", "state_seq": 13 } }, "last_failure": 0, "last_failure_osd_epoch": 53, "max_file_size": 1099511627776, "max_mds": 2, "metadata_pool": 3, "modified": "2023-10-06T04:09:55.008132+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2, 3, 4 ], "tableserver": 0, "up": { "mds_0": 6270, "mds_1": 5812 } } } ], "standbys": [ { "addr": "172.21.15.175:6835/1799968248", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6834", "nonce": 1799968248, "type": "v2" }, { "addr": "172.21.15.175:6835", "nonce": 1799968248, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 57, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5126, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.175:6839/745804271", "addrs": { "addrvec": [ { "addr": "172.21.15.175:6838", "nonce": 745804271, "type": "v2" }, { "addr": "172.21.15.175:6839", "nonce": 745804271, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 42, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5164, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.157:6835/2461751318", "addrs": { "addrvec": [ { "addr": "172.21.15.157:6833", "nonce": 2461751318, "type": "v2" }, { "addr": "172.21.15.157:6835", "nonce": 2461751318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 34, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5268, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-06T04:10:01.929 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:10:02.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.059+0000 7fe32bb93700 1 Processor -- start 2023-10-06T04:10:02.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.060+0000 7fe32bb93700 1 -- start start 2023-10-06T04:10:02.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.061+0000 7fe32bb93700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe32415e160 0x7fe3240c5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:02.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.061+0000 7fe32bb93700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe32415e530 0x7fe324069410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:02.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.061+0000 7fe32bb93700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe32406fc90 con 0x7fe324069950 2023-10-06T04:10:02.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.061+0000 7fe32bb93700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe324155130 con 0x7fe32415e530 2023-10-06T04:10:02.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.061+0000 7fe32bb93700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe32406f800 con 0x7fe32415e160 2023-10-06T04:10:02.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.061+0000 7fe32912e700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe32415e530 0x7fe324069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:02.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.061+0000 7fe32912e700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe32415e530 0x7fe324069410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53080/0 (socket says 172.21.15.157:53080) 2023-10-06T04:10:02.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.061+0000 7fe32912e700 1 -- 172.21.15.157:0/3054852503 learned_addr learned my addr 172.21.15.157:0/3054852503 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:10:02.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.061+0000 7fe32992f700 1 --2- 172.21.15.157:0/3054852503 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe32415e160 0x7fe3240c5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:02.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.061+0000 7fe32912e700 1 -- 172.21.15.157:0/3054852503 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe32415e160 msgr2=0x7fe3240c5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:10:02.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.061+0000 7fe32912e700 1 --2- 172.21.15.157:0/3054852503 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe32415e160 0x7fe3240c5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:02.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.061+0000 7fe32912e700 1 -- 172.21.15.157:0/3054852503 >> v1:172.21.15.175:6789/0 conn(0x7fe324069950 legacy=0x7fe3240ad930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:02.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.061+0000 7fe32912e700 1 -- 172.21.15.157:0/3054852503 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe32406e950 con 0x7fe32415e530 2023-10-06T04:10:02.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.061+0000 7fe32912e700 1 --2- 172.21.15.157:0/3054852503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe32415e530 0x7fe324069410 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fe3140099e0 tx=0x7fe314009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505cedbbc8854560 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:02.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.061+0000 7fe32892d700 1 -- 172.21.15.157:0/3054852503 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31400b820 con 0x7fe32415e530 2023-10-06T04:10:02.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.062+0000 7fe32892d700 1 -- 172.21.15.157:0/3054852503 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe31400b980 con 0x7fe32415e530 2023-10-06T04:10:02.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.062+0000 7fe32892d700 1 -- 172.21.15.157:0/3054852503 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31400bd20 con 0x7fe32415e530 2023-10-06T04:10:02.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.062+0000 7fe32bb93700 1 -- 172.21.15.157:0/3054852503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe32415e530 msgr2=0x7fe324069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:02.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.062+0000 7fe32bb93700 1 --2- 172.21.15.157:0/3054852503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe32415e530 0x7fe324069410 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fe3140099e0 tx=0x7fe314009250 comp rx=0 tx=0).stop 2023-10-06T04:10:02.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.062+0000 7fe32bb93700 1 -- 172.21.15.157:0/3054852503 shutdown_connections 2023-10-06T04:10:02.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.062+0000 7fe32bb93700 1 --2- 172.21.15.157:0/3054852503 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe32415e160 0x7fe3240c5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:02.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.062+0000 7fe32bb93700 1 --2- 172.21.15.157:0/3054852503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe32415e530 0x7fe324069410 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:02.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.062+0000 7fe32bb93700 1 -- 172.21.15.157:0/3054852503 >> 172.21.15.157:0/3054852503 conn(0x7fe3240c8d70 msgr2=0x7fe3240c4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:02.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.062+0000 7fe32bb93700 1 -- 172.21.15.157:0/3054852503 shutdown_connections 2023-10-06T04:10:02.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.062+0000 7fe32bb93700 1 -- 172.21.15.157:0/3054852503 wait complete. 2023-10-06T04:10:02.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.063+0000 7fe32bb93700 1 Processor -- start 2023-10-06T04:10:02.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.063+0000 7fe32bb93700 1 -- start start 2023-10-06T04:10:02.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.064+0000 7fe32bb93700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe324069950 0x7fe32416bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:02.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.064+0000 7fe32bb93700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe32415e160 0x7fe32416e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:02.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.064+0000 7fe32bb93700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe32415e530 0x7fe32416e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:02.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.064+0000 7fe32bb93700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32416ed00 con 0x7fe32415e160 2023-10-06T04:10:02.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.064+0000 7fe32bb93700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3240c1950 con 0x7fe324069950 2023-10-06T04:10:02.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.064+0000 7fe32bb93700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3240c0a30 con 0x7fe32415e530 2023-10-06T04:10:02.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.064+0000 7fe32912e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe32415e160 0x7fe32416e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:02.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.064+0000 7fe32912e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe32415e160 0x7fe32416e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53084/0 (socket says 172.21.15.157:53084) 2023-10-06T04:10:02.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.064+0000 7fe32912e700 1 -- 172.21.15.157:0/1498800907 learned_addr learned my addr 172.21.15.157:0/1498800907 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:10:02.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.064+0000 7fe32a130700 1 --2- 172.21.15.157:0/1498800907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe32415e530 0x7fe32416e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:02.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.064+0000 7fe32992f700 1 --2- 172.21.15.157:0/1498800907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe324069950 0x7fe32416bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:02.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.064+0000 7fe32912e700 1 -- 172.21.15.157:0/1498800907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe32415e530 msgr2=0x7fe32416e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:02.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.064+0000 7fe32912e700 1 --2- 172.21.15.157:0/1498800907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe32415e530 0x7fe32416e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:02.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.064+0000 7fe32912e700 1 -- 172.21.15.157:0/1498800907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe324069950 msgr2=0x7fe32416bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:02.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.064+0000 7fe32912e700 1 --2- 172.21.15.157:0/1498800907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe324069950 0x7fe32416bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:02.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.064+0000 7fe32912e700 1 -- 172.21.15.157:0/1498800907 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3140094f0 con 0x7fe32415e160 2023-10-06T04:10:02.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.064+0000 7fe32912e700 1 --2- 172.21.15.157:0/1498800907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe32415e160 0x7fe32416e280 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fe314006fd0 tx=0x7fe314017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:02.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.065+0000 7fe319ffb700 1 -- 172.21.15.157:0/1498800907 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe314011d30 con 0x7fe32415e160 2023-10-06T04:10:02.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.065+0000 7fe319ffb700 1 -- 172.21.15.157:0/1498800907 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe314024440 con 0x7fe32415e160 2023-10-06T04:10:02.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.065+0000 7fe32bb93700 1 -- 172.21.15.157:0/1498800907 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3240c0ec0 con 0x7fe32415e160 2023-10-06T04:10:02.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.065+0000 7fe319ffb700 1 -- 172.21.15.157:0/1498800907 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe314024800 con 0x7fe32415e160 2023-10-06T04:10:02.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.065+0000 7fe32bb93700 1 -- 172.21.15.157:0/1498800907 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3240bfe20 con 0x7fe32415e160 2023-10-06T04:10:02.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.066+0000 7fe32bb93700 1 -- 172.21.15.157:0/1498800907 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3240c0be0 con 0x7fe32415e160 2023-10-06T04:10:02.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.066+0000 7fe319ffb700 1 -- 172.21.15.157:0/1498800907 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe31402d440 con 0x7fe32415e160 2023-10-06T04:10:02.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.066+0000 7fe319ffb700 1 --2- 172.21.15.157:0/1498800907 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe308066f30 0x7fe3080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:02.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.067+0000 7fe319ffb700 1 -- 172.21.15.157:0/1498800907 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7fe3140861c0 con 0x7fe32415e160 2023-10-06T04:10:02.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.067+0000 7fe32992f700 1 --2- 172.21.15.157:0/1498800907 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe308066f30 0x7fe3080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:02.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.067+0000 7fe32992f700 1 --2- 172.21.15.157:0/1498800907 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe308066f30 0x7fe3080693e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe32000f570 tx=0x7fe32000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:02.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.071+0000 7fe319ffb700 1 -- 172.21.15.157:0/1498800907 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe3240c0be0 con 0x7fe32415e160 2023-10-06T04:10:02.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.301+0000 7fe32bb93700 1 -- 172.21.15.157:0/1498800907 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe324064780 con 0x7fe32415e160 2023-10-06T04:10:02.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.302+0000 7fe319ffb700 1 -- 172.21.15.157:0/1498800907 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+5393 (secure 0 0 0) 0x7fe314053040 con 0x7fe32415e160 2023-10-06T04:10:02.304 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:10:02.305 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:10:01.034717+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6270,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:active","state_seq":19,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6270":{"gid":6270,"name":"e","rank":0,"incarnation":86,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.157:6837/1036901172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1036901172},{"type":"v1","addr":"172.21.15.157:6837","nonce":1036901172}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:10:02.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.304+0000 7fe32bb93700 1 -- 172.21.15.157:0/1498800907 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe308066f30 msgr2=0x7fe3080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:02.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.304+0000 7fe32bb93700 1 --2- 172.21.15.157:0/1498800907 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe308066f30 0x7fe3080693e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe32000f570 tx=0x7fe32000b040 comp rx=0 tx=0).stop 2023-10-06T04:10:02.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.304+0000 7fe32bb93700 1 -- 172.21.15.157:0/1498800907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe32415e160 msgr2=0x7fe32416e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:02.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.304+0000 7fe32bb93700 1 --2- 172.21.15.157:0/1498800907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe32415e160 0x7fe32416e280 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fe314006fd0 tx=0x7fe314017000 comp rx=0 tx=0).stop 2023-10-06T04:10:02.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.304+0000 7fe32bb93700 1 -- 172.21.15.157:0/1498800907 shutdown_connections 2023-10-06T04:10:02.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.304+0000 7fe32bb93700 1 --2- 172.21.15.157:0/1498800907 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fe308066f30 0x7fe3080693e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:02.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.304+0000 7fe32bb93700 1 --2- 172.21.15.157:0/1498800907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe324069950 0x7fe32416bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:02.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.304+0000 7fe32bb93700 1 --2- 172.21.15.157:0/1498800907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe32415e530 0x7fe32416e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:02.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.304+0000 7fe32bb93700 1 --2- 172.21.15.157:0/1498800907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe32415e160 0x7fe32416e280 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:02.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.304+0000 7fe32bb93700 1 -- 172.21.15.157:0/1498800907 >> 172.21.15.157:0/1498800907 conn(0x7fe3240c8d70 msgr2=0x7fe324154660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:02.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.304+0000 7fe32bb93700 1 -- 172.21.15.157:0/1498800907 shutdown_connections 2023-10-06T04:10:02.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:02.304+0000 7fe32bb93700 1 -- 172.21.15.157:0/1498800907 wait complete. 2023-10-06T04:10:02.308 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 88 2023-10-06T04:10:02.319 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:10:04.320 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:10:04.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.445+0000 7fa745c9b700 1 Processor -- start 2023-10-06T04:10:04.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.445+0000 7fa745c9b700 1 -- start start 2023-10-06T04:10:04.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.446+0000 7fa745c9b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa74006dc40 0x7fa74014fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:04.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.446+0000 7fa745c9b700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa74006c1f0 0x7fa740150240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:04.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.446+0000 7fa745c9b700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7400c9780 con 0x7fa74006cbb0 2023-10-06T04:10:04.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.446+0000 7fa745c9b700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7400c8c40 con 0x7fa74006c1f0 2023-10-06T04:10:04.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.446+0000 7fa745c9b700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa74006e0c0 con 0x7fa74006dc40 2023-10-06T04:10:04.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.446+0000 7fa73ffff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa74006c1f0 0x7fa740150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:04.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.446+0000 7fa73ffff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa74006c1f0 0x7fa740150240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54414/0 (socket says 172.21.15.157:54414) 2023-10-06T04:10:04.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.446+0000 7fa73ffff700 1 -- 172.21.15.157:0/876138100 learned_addr learned my addr 172.21.15.157:0/876138100 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:10:04.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.446+0000 7fa73f7fe700 1 --1- 172.21.15.157:0/876138100 >> v1:172.21.15.157:6789/0 conn(0x7fa74006cbb0 0x7fa74006d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:36300/0 (socket says 172.21.15.157:36300) 2023-10-06T04:10:04.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.446+0000 7fa73effd700 1 --2- 172.21.15.157:0/876138100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa74006dc40 0x7fa74014fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:04.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.447+0000 7fa73ffff700 1 -- 172.21.15.157:0/876138100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa74006dc40 msgr2=0x7fa74014fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:10:04.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.447+0000 7fa73ffff700 1 --2- 172.21.15.157:0/876138100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa74006dc40 0x7fa74014fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:04.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.447+0000 7fa73ffff700 1 -- 172.21.15.157:0/876138100 >> v1:172.21.15.157:6789/0 conn(0x7fa74006cbb0 legacy=0x7fa74006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:04.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.447+0000 7fa73ffff700 1 -- 172.21.15.157:0/876138100 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7400c8e50 con 0x7fa74006c1f0 2023-10-06T04:10:04.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.447+0000 7fa73e7fc700 1 -- 172.21.15.157:0/876138100 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2486890777 0 0) 0x7fa7400c9780 con 0x7fa74006cbb0 2023-10-06T04:10:04.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.447+0000 7fa73ffff700 1 --2- 172.21.15.157:0/876138100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa74006c1f0 0x7fa740150240 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa7380099b0 tx=0x7fa738009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afbee192fa9e2ef4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:04.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.447+0000 7fa73e7fc700 1 -- 172.21.15.157:0/876138100 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73800b8f0 con 0x7fa74006c1f0 2023-10-06T04:10:04.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.447+0000 7fa73e7fc700 1 -- 172.21.15.157:0/876138100 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa73800ba50 con 0x7fa74006c1f0 2023-10-06T04:10:04.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.447+0000 7fa73e7fc700 1 -- 172.21.15.157:0/876138100 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73800bdf0 con 0x7fa74006c1f0 2023-10-06T04:10:04.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.447+0000 7fa745c9b700 1 -- 172.21.15.157:0/876138100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa74006c1f0 msgr2=0x7fa740150240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:04.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.447+0000 7fa745c9b700 1 --2- 172.21.15.157:0/876138100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa74006c1f0 0x7fa740150240 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa7380099b0 tx=0x7fa738009250 comp rx=0 tx=0).stop 2023-10-06T04:10:04.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.447+0000 7fa745c9b700 1 -- 172.21.15.157:0/876138100 shutdown_connections 2023-10-06T04:10:04.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.447+0000 7fa745c9b700 1 --2- 172.21.15.157:0/876138100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa74006c1f0 0x7fa740150240 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:04.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.447+0000 7fa745c9b700 1 --2- 172.21.15.157:0/876138100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa74006dc40 0x7fa74014fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:04.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.447+0000 7fa745c9b700 1 -- 172.21.15.157:0/876138100 >> 172.21.15.157:0/876138100 conn(0x7fa74006f230 msgr2=0x7fa74006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:04.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.448+0000 7fa745c9b700 1 -- 172.21.15.157:0/876138100 shutdown_connections 2023-10-06T04:10:04.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.448+0000 7fa745c9b700 1 -- 172.21.15.157:0/876138100 wait complete. 2023-10-06T04:10:04.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.449+0000 7fa745c9b700 1 Processor -- start 2023-10-06T04:10:04.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.449+0000 7fa745c9b700 1 -- start start 2023-10-06T04:10:04.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.449+0000 7fa745c9b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa74006c1f0 0x7fa740152830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:04.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.449+0000 7fa745c9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa74006cbb0 0x7fa740152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:04.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.449+0000 7fa745c9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa740153580 0x7fa740167f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:04.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.449+0000 7fa745c9b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa740167ce0 con 0x7fa740153580 2023-10-06T04:10:04.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.449+0000 7fa745c9b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74014e2d0 con 0x7fa74006cbb0 2023-10-06T04:10:04.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.449+0000 7fa745c9b700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa740148590 con 0x7fa74006c1f0 2023-10-06T04:10:04.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.450+0000 7fa73ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa740153580 0x7fa740167f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:04.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.450+0000 7fa73ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa740153580 0x7fa740167f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54420/0 (socket says 172.21.15.157:54420) 2023-10-06T04:10:04.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.450+0000 7fa73ffff700 1 -- 172.21.15.157:0/1727032794 learned_addr learned my addr 172.21.15.157:0/1727032794 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:10:04.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.450+0000 7fa73f7fe700 1 --2- 172.21.15.157:0/1727032794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa74006c1f0 0x7fa740152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:04.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.450+0000 7fa73ffff700 1 -- 172.21.15.157:0/1727032794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa74006c1f0 msgr2=0x7fa740152830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:04.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.450+0000 7fa73ffff700 1 --2- 172.21.15.157:0/1727032794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa74006c1f0 0x7fa740152830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:04.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.450+0000 7fa73effd700 1 --2- 172.21.15.157:0/1727032794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa74006cbb0 0x7fa740152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:04.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.450+0000 7fa73ffff700 1 -- 172.21.15.157:0/1727032794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa74006cbb0 msgr2=0x7fa740152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:04.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.450+0000 7fa73ffff700 1 --2- 172.21.15.157:0/1727032794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa74006cbb0 0x7fa740152d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:04.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.450+0000 7fa73ffff700 1 -- 172.21.15.157:0/1727032794 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7380094f0 con 0x7fa740153580 2023-10-06T04:10:04.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.450+0000 7fa73ffff700 1 --2- 172.21.15.157:0/1727032794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa740153580 0x7fa740167f60 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa73800b3c0 tx=0x7fa738017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:04.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.450+0000 7fa7277fe700 1 -- 172.21.15.157:0/1727032794 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73801a630 con 0x7fa740153580 2023-10-06T04:10:04.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.450+0000 7fa7277fe700 1 -- 172.21.15.157:0/1727032794 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa73801a7d0 con 0x7fa740153580 2023-10-06T04:10:04.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.450+0000 7fa745c9b700 1 -- 172.21.15.157:0/1727032794 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa740148870 con 0x7fa740153580 2023-10-06T04:10:04.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.450+0000 7fa7277fe700 1 -- 172.21.15.157:0/1727032794 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73801ab90 con 0x7fa740153580 2023-10-06T04:10:04.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.450+0000 7fa745c9b700 1 -- 172.21.15.157:0/1727032794 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa740146ab0 con 0x7fa740153580 2023-10-06T04:10:04.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.451+0000 7fa745c9b700 1 -- 172.21.15.157:0/1727032794 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa74014e760 con 0x7fa740153580 2023-10-06T04:10:04.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.452+0000 7fa7277fe700 1 -- 172.21.15.157:0/1727032794 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa7380233e0 con 0x7fa740153580 2023-10-06T04:10:04.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.452+0000 7fa7277fe700 1 --2- 172.21.15.157:0/1727032794 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa72c066f30 0x7fa72c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:04.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.452+0000 7fa7277fe700 1 -- 172.21.15.157:0/1727032794 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7fa7380864e0 con 0x7fa740153580 2023-10-06T04:10:04.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.452+0000 7fa73f7fe700 1 --2- 172.21.15.157:0/1727032794 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa72c066f30 0x7fa72c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:04.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.453+0000 7fa73f7fe700 1 --2- 172.21.15.157:0/1727032794 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa72c066f30 0x7fa72c0693e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa73000d270 tx=0x7fa730009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:04.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.456+0000 7fa7277fe700 1 -- 172.21.15.157:0/1727032794 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa74014e760 con 0x7fa740153580 2023-10-06T04:10:04.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.686+0000 7fa745c9b700 1 -- 172.21.15.157:0/1727032794 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa74014e760 con 0x7fa740153580 2023-10-06T04:10:04.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.687+0000 7fa7277fe700 1 -- 172.21.15.157:0/1727032794 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+5393 (secure 0 0 0) 0x7fa74014e760 con 0x7fa740153580 2023-10-06T04:10:04.690 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:10:04.690 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:10:01.034717+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6270,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:active","state_seq":19,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6270":{"gid":6270,"name":"e","rank":0,"incarnation":86,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.157:6837/1036901172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1036901172},{"type":"v1","addr":"172.21.15.157:6837","nonce":1036901172}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:10:04.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.689+0000 7fa745c9b700 1 -- 172.21.15.157:0/1727032794 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa72c066f30 msgr2=0x7fa72c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:04.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.689+0000 7fa745c9b700 1 --2- 172.21.15.157:0/1727032794 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa72c066f30 0x7fa72c0693e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa73000d270 tx=0x7fa730009040 comp rx=0 tx=0).stop 2023-10-06T04:10:04.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.689+0000 7fa745c9b700 1 -- 172.21.15.157:0/1727032794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa740153580 msgr2=0x7fa740167f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:04.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.689+0000 7fa745c9b700 1 --2- 172.21.15.157:0/1727032794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa740153580 0x7fa740167f60 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa73800b3c0 tx=0x7fa738017000 comp rx=0 tx=0).stop 2023-10-06T04:10:04.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.689+0000 7fa745c9b700 1 -- 172.21.15.157:0/1727032794 shutdown_connections 2023-10-06T04:10:04.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.689+0000 7fa745c9b700 1 --2- 172.21.15.157:0/1727032794 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa72c066f30 0x7fa72c0693e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:04.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.689+0000 7fa745c9b700 1 --2- 172.21.15.157:0/1727032794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa740153580 0x7fa740167f60 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:04.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.689+0000 7fa745c9b700 1 --2- 172.21.15.157:0/1727032794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa74006c1f0 0x7fa740152830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:04.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.689+0000 7fa745c9b700 1 --2- 172.21.15.157:0/1727032794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa74006cbb0 0x7fa740152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:04.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.689+0000 7fa745c9b700 1 -- 172.21.15.157:0/1727032794 >> 172.21.15.157:0/1727032794 conn(0x7fa74006f230 msgr2=0x7fa74014fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:04.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.689+0000 7fa745c9b700 1 -- 172.21.15.157:0/1727032794 shutdown_connections 2023-10-06T04:10:04.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:04.689+0000 7fa745c9b700 1 -- 172.21.15.157:0/1727032794 wait complete. 2023-10-06T04:10:04.693 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 88 2023-10-06T04:10:04.703 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:10:06.705 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:10:06.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.831+0000 7fa846422700 1 Processor -- start 2023-10-06T04:10:06.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.832+0000 7fa846422700 1 -- start start 2023-10-06T04:10:06.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.832+0000 7fa846422700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa84006cbb0 0x7fa84006d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:06.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.832+0000 7fa846422700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa84006c1f0 0x7fa84006da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:06.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.832+0000 7fa846422700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8400c9780 con 0x7fa84006cf80 2023-10-06T04:10:06.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.832+0000 7fa846422700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8400c8c40 con 0x7fa84006c1f0 2023-10-06T04:10:06.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.832+0000 7fa846422700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84006e0c0 con 0x7fa84006cbb0 2023-10-06T04:10:06.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.833+0000 7fa83f7fe700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa84006c1f0 0x7fa84006da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:06.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.833+0000 7fa83ffff700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa84006cbb0 0x7fa84006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:06.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.833+0000 7fa8449bf700 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fa84006cf80 0x7fa84014fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.157:43316/0 (socket says 172.21.15.157:43316) 2023-10-06T04:10:06.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.833+0000 7fa83f7fe700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa84006c1f0 0x7fa84006da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:57192/0 (socket says 172.21.15.157:57192) 2023-10-06T04:10:06.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.833+0000 7fa8449bf700 1 -- 172.21.15.157:0/2304965157 learned_addr learned my addr 172.21.15.157:0/2304965157 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:10:06.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.833+0000 7fa83f7fe700 1 -- 172.21.15.157:0/2304965157 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa84006cbb0 msgr2=0x7fa84006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:10:06.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.833+0000 7fa83f7fe700 1 --2- 172.21.15.157:0/2304965157 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa84006cbb0 0x7fa84006d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:06.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.833+0000 7fa83f7fe700 1 -- 172.21.15.157:0/2304965157 >> v1:172.21.15.175:6789/0 conn(0x7fa84006cf80 legacy=0x7fa84014fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:06.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.833+0000 7fa83f7fe700 1 -- 172.21.15.157:0/2304965157 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8400c8e50 con 0x7fa84006c1f0 2023-10-06T04:10:06.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.833+0000 7fa83ffff700 1 --2- 172.21.15.157:0/2304965157 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa84006cbb0 0x7fa84006d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:10:06.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.833+0000 7fa83f7fe700 1 --2- 172.21.15.157:0/2304965157 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa84006c1f0 0x7fa84006da70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa834009910 tx=0x7fa8340091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfa3a686a38809ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:06.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.833+0000 7fa83effd700 1 -- 172.21.15.157:0/2304965157 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83400b6d0 con 0x7fa84006c1f0 2023-10-06T04:10:06.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.833+0000 7fa83effd700 1 -- 172.21.15.157:0/2304965157 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83400b830 con 0x7fa84006c1f0 2023-10-06T04:10:06.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.833+0000 7fa83effd700 1 -- 172.21.15.157:0/2304965157 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83400bbf0 con 0x7fa84006c1f0 2023-10-06T04:10:06.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.834+0000 7fa846422700 1 -- 172.21.15.157:0/2304965157 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa84006c1f0 msgr2=0x7fa84006da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:06.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.834+0000 7fa846422700 1 --2- 172.21.15.157:0/2304965157 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa84006c1f0 0x7fa84006da70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa834009910 tx=0x7fa8340091a0 comp rx=0 tx=0).stop 2023-10-06T04:10:06.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.834+0000 7fa846422700 1 -- 172.21.15.157:0/2304965157 shutdown_connections 2023-10-06T04:10:06.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.834+0000 7fa846422700 1 --2- 172.21.15.157:0/2304965157 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa84006cbb0 0x7fa84006d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:06.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.834+0000 7fa846422700 1 --2- 172.21.15.157:0/2304965157 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa84006c1f0 0x7fa84006da70 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:06.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.834+0000 7fa846422700 1 -- 172.21.15.157:0/2304965157 >> 172.21.15.157:0/2304965157 conn(0x7fa84006f230 msgr2=0x7fa84006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:06.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.834+0000 7fa846422700 1 -- 172.21.15.157:0/2304965157 shutdown_connections 2023-10-06T04:10:06.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.834+0000 7fa846422700 1 -- 172.21.15.157:0/2304965157 wait complete. 2023-10-06T04:10:06.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.835+0000 7fa846422700 1 Processor -- start 2023-10-06T04:10:06.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.835+0000 7fa846422700 1 -- start start 2023-10-06T04:10:06.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.836+0000 7fa846422700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa84006c1f0 0x7fa84014bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:06.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.836+0000 7fa846422700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84006cbb0 0x7fa8400c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:06.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.836+0000 7fa846422700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa84006cf80 0x7fa8400c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:06.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.836+0000 7fa846422700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8401663f0 con 0x7fa84006cbb0 2023-10-06T04:10:06.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.836+0000 7fa846422700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8401666d0 con 0x7fa84006cf80 2023-10-06T04:10:06.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.836+0000 7fa846422700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8401669b0 con 0x7fa84006c1f0 2023-10-06T04:10:06.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.836+0000 7fa83f7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84006cbb0 0x7fa8400c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:06.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.836+0000 7fa83f7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84006cbb0 0x7fa8400c60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54430/0 (socket says 172.21.15.157:54430) 2023-10-06T04:10:06.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.836+0000 7fa83f7fe700 1 -- 172.21.15.157:0/1223375089 learned_addr learned my addr 172.21.15.157:0/1223375089 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:10:06.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.836+0000 7fa83ffff700 1 --2- 172.21.15.157:0/1223375089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa84006c1f0 0x7fa84014bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:06.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.836+0000 7fa8449bf700 1 --2- 172.21.15.157:0/1223375089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa84006cf80 0x7fa8400c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:06.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.836+0000 7fa83f7fe700 1 -- 172.21.15.157:0/1223375089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa84006c1f0 msgr2=0x7fa84014bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:06.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.836+0000 7fa83f7fe700 1 --2- 172.21.15.157:0/1223375089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa84006c1f0 0x7fa84014bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:06.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.836+0000 7fa83f7fe700 1 -- 172.21.15.157:0/1223375089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa84006cf80 msgr2=0x7fa8400c3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:06.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.836+0000 7fa83f7fe700 1 --2- 172.21.15.157:0/1223375089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa84006cf80 0x7fa8400c3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:06.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.836+0000 7fa83f7fe700 1 -- 172.21.15.157:0/1223375089 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa830009420 con 0x7fa84006cbb0 2023-10-06T04:10:06.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.836+0000 7fa83ffff700 1 --2- 172.21.15.157:0/1223375089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa84006c1f0 0x7fa84014bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:10:06.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.837+0000 7fa83f7fe700 1 --2- 172.21.15.157:0/1223375089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84006cbb0 0x7fa8400c60f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa8400c3960 tx=0x7fa834017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:06.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.837+0000 7fa827fff700 1 -- 172.21.15.157:0/1223375089 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83401f070 con 0x7fa84006cbb0 2023-10-06T04:10:06.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.837+0000 7fa827fff700 1 -- 172.21.15.157:0/1223375089 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa834000c90 con 0x7fa84006cbb0 2023-10-06T04:10:06.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.837+0000 7fa846422700 1 -- 172.21.15.157:0/1223375089 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa834009420 con 0x7fa84006cbb0 2023-10-06T04:10:06.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.837+0000 7fa827fff700 1 -- 172.21.15.157:0/1223375089 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83400bbd0 con 0x7fa84006cbb0 2023-10-06T04:10:06.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.837+0000 7fa846422700 1 -- 172.21.15.157:0/1223375089 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa840166f70 con 0x7fa84006cbb0 2023-10-06T04:10:06.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.838+0000 7fa846422700 1 -- 172.21.15.157:0/1223375089 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8400c3a20 con 0x7fa84006cbb0 2023-10-06T04:10:06.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.838+0000 7fa827fff700 1 -- 172.21.15.157:0/1223375089 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa834023070 con 0x7fa84006cbb0 2023-10-06T04:10:06.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.839+0000 7fa827fff700 1 --2- 172.21.15.157:0/1223375089 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa82c067000 0x7fa82c0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:06.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.839+0000 7fa827fff700 1 -- 172.21.15.157:0/1223375089 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7fa834085ee0 con 0x7fa84006cbb0 2023-10-06T04:10:06.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.839+0000 7fa83ffff700 1 --2- 172.21.15.157:0/1223375089 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa82c067000 0x7fa82c0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:06.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.839+0000 7fa83ffff700 1 --2- 172.21.15.157:0/1223375089 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa82c067000 0x7fa82c0694b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa8300093f0 tx=0x7fa830008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:06.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:06.843+0000 7fa827fff700 1 -- 172.21.15.157:0/1223375089 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa8400c3a20 con 0x7fa84006cbb0 2023-10-06T04:10:07.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:07.069+0000 7fa846422700 1 -- 172.21.15.157:0/1223375089 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa840167280 con 0x7fa84006cbb0 2023-10-06T04:10:07.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:07.070+0000 7fa827fff700 1 -- 172.21.15.157:0/1223375089 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+5390 (secure 0 0 0) 0x7fa834052dc0 con 0x7fa84006cbb0 2023-10-06T04:10:07.071 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:10:07.072 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":89,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:10:06.055493+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6270,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:active","state_seq":19,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6270":{"gid":6270,"name":"e","rank":0,"incarnation":86,"state":"up:rejoin","state_seq":17,"addr":"172.21.15.157:6837/1036901172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1036901172},{"type":"v1","addr":"172.21.15.157:6837","nonce":1036901172}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:10:07.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:07.072+0000 7fa846422700 1 -- 172.21.15.157:0/1223375089 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa82c067000 msgr2=0x7fa82c0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:07.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:07.072+0000 7fa846422700 1 --2- 172.21.15.157:0/1223375089 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa82c067000 0x7fa82c0694b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa8300093f0 tx=0x7fa830008040 comp rx=0 tx=0).stop 2023-10-06T04:10:07.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:07.072+0000 7fa846422700 1 -- 172.21.15.157:0/1223375089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84006cbb0 msgr2=0x7fa8400c60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:07.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:07.072+0000 7fa846422700 1 --2- 172.21.15.157:0/1223375089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84006cbb0 0x7fa8400c60f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa8400c3960 tx=0x7fa834017000 comp rx=0 tx=0).stop 2023-10-06T04:10:07.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:07.072+0000 7fa846422700 1 -- 172.21.15.157:0/1223375089 shutdown_connections 2023-10-06T04:10:07.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:07.072+0000 7fa846422700 1 --2- 172.21.15.157:0/1223375089 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa82c067000 0x7fa82c0694b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:07.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:07.072+0000 7fa846422700 1 --2- 172.21.15.157:0/1223375089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa84006cf80 0x7fa8400c3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:07.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:07.072+0000 7fa846422700 1 --2- 172.21.15.157:0/1223375089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa84006c1f0 0x7fa84014bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:07.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:07.072+0000 7fa846422700 1 --2- 172.21.15.157:0/1223375089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa84006cbb0 0x7fa8400c60f0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:07.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:07.072+0000 7fa846422700 1 -- 172.21.15.157:0/1223375089 >> 172.21.15.157:0/1223375089 conn(0x7fa84006f230 msgr2=0x7fa84014e670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:07.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:07.072+0000 7fa846422700 1 -- 172.21.15.157:0/1223375089 shutdown_connections 2023-10-06T04:10:07.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:07.072+0000 7fa846422700 1 -- 172.21.15.157:0/1223375089 wait complete. 2023-10-06T04:10:07.075 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 89 2023-10-06T04:10:07.085 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:10:09.086 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:10:09.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.214+0000 7f1c32f91700 1 Processor -- start 2023-10-06T04:10:09.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.214+0000 7f1c32f91700 1 -- start start 2023-10-06T04:10:09.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.214+0000 7f1c32f91700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1c2c06c440 0x7f1c2c069130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:09.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.214+0000 7f1c32f91700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c2c06c810 0x7f1c2c1569e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:09.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.214+0000 7f1c32f91700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c2c155f20 con 0x7f1c2c0c5fc0 2023-10-06T04:10:09.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.214+0000 7f1c32f91700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c2c06f310 con 0x7f1c2c06c440 2023-10-06T04:10:09.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.214+0000 7f1c32f91700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c06d4a0 con 0x7f1c2c06c810 2023-10-06T04:10:09.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.215+0000 7f1c30d2d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1c2c06c440 0x7f1c2c069130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:09.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.215+0000 7f1c30d2d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1c2c06c440 0x7f1c2c069130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54442/0 (socket says 172.21.15.157:54442) 2023-10-06T04:10:09.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.215+0000 7f1c30d2d700 1 -- 172.21.15.157:0/1855708748 learned_addr learned my addr 172.21.15.157:0/1855708748 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:10:09.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.215+0000 7f1c3152e700 1 --2- 172.21.15.157:0/1855708748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c2c06c810 0x7f1c2c1569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:09.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.215+0000 7f1c30d2d700 1 -- 172.21.15.157:0/1855708748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c2c06c810 msgr2=0x7f1c2c1569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:10:09.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.215+0000 7f1c30d2d700 1 --2- 172.21.15.157:0/1855708748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c2c06c810 0x7f1c2c1569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:09.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.215+0000 7f1c30d2d700 1 -- 172.21.15.157:0/1855708748 >> v1:172.21.15.175:6789/0 conn(0x7f1c2c0c5fc0 legacy=0x7f1c2c152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:09.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.215+0000 7f1c30d2d700 1 -- 172.21.15.157:0/1855708748 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2c06d780 con 0x7f1c2c06c440 2023-10-06T04:10:09.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.215+0000 7f1c3152e700 1 --2- 172.21.15.157:0/1855708748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c2c06c810 0x7f1c2c1569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:10:09.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.215+0000 7f1c30d2d700 1 --2- 172.21.15.157:0/1855708748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c2c06c440 0x7f1c2c069130 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1c280099b0 tx=0x7f1c28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c86355dec51854 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:09.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.215+0000 7f1c237fe700 1 -- 172.21.15.157:0/1855708748 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2800b860 con 0x7f1c2c06c440 2023-10-06T04:10:09.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.215+0000 7f1c237fe700 1 -- 172.21.15.157:0/1855708748 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c2800b9c0 con 0x7f1c2c06c440 2023-10-06T04:10:09.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.215+0000 7f1c237fe700 1 -- 172.21.15.157:0/1855708748 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2800bd60 con 0x7f1c2c06c440 2023-10-06T04:10:09.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.216+0000 7f1c32f91700 1 -- 172.21.15.157:0/1855708748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c2c06c440 msgr2=0x7f1c2c069130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:09.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.216+0000 7f1c32f91700 1 --2- 172.21.15.157:0/1855708748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c2c06c440 0x7f1c2c069130 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1c280099b0 tx=0x7f1c28009250 comp rx=0 tx=0).stop 2023-10-06T04:10:09.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.216+0000 7f1c32f91700 1 -- 172.21.15.157:0/1855708748 shutdown_connections 2023-10-06T04:10:09.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.216+0000 7f1c32f91700 1 --2- 172.21.15.157:0/1855708748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c2c06c810 0x7f1c2c1569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:09.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.216+0000 7f1c32f91700 1 --2- 172.21.15.157:0/1855708748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c2c06c440 0x7f1c2c069130 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:09.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.216+0000 7f1c32f91700 1 -- 172.21.15.157:0/1855708748 >> 172.21.15.157:0/1855708748 conn(0x7f1c2c068370 msgr2=0x7f1c2c068770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:09.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.216+0000 7f1c32f91700 1 -- 172.21.15.157:0/1855708748 shutdown_connections 2023-10-06T04:10:09.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.216+0000 7f1c32f91700 1 -- 172.21.15.157:0/1855708748 wait complete. 2023-10-06T04:10:09.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.217+0000 7f1c32f91700 1 Processor -- start 2023-10-06T04:10:09.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.217+0000 7f1c32f91700 1 -- start start 2023-10-06T04:10:09.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.218+0000 7f1c32f91700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c2c06c440 0x7f1c2c169d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:09.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.218+0000 7f1c32f91700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c2c06c810 0x7f1c2c14d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:09.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.218+0000 7f1c32f91700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c2c0c5fc0 0x7f1c2c14de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:09.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.218+0000 7f1c32f91700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c0adc10 con 0x7f1c2c06c810 2023-10-06T04:10:09.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.218+0000 7f1c32f91700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c167fa0 con 0x7f1c2c0c5fc0 2023-10-06T04:10:09.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.218+0000 7f1c32f91700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c0bf530 con 0x7f1c2c06c440 2023-10-06T04:10:09.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.218+0000 7f1c23fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c2c06c810 0x7f1c2c14d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:09.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.218+0000 7f1c23fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c2c06c810 0x7f1c2c14d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54454/0 (socket says 172.21.15.157:54454) 2023-10-06T04:10:09.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.218+0000 7f1c23fff700 1 -- 172.21.15.157:0/259895298 learned_addr learned my addr 172.21.15.157:0/259895298 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:10:09.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.218+0000 7f1c30d2d700 1 --2- 172.21.15.157:0/259895298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c2c06c440 0x7f1c2c169d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:09.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.218+0000 7f1c3152e700 1 --2- 172.21.15.157:0/259895298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c2c0c5fc0 0x7f1c2c14de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:09.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.218+0000 7f1c23fff700 1 -- 172.21.15.157:0/259895298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c2c06c440 msgr2=0x7f1c2c169d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:09.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.218+0000 7f1c23fff700 1 --2- 172.21.15.157:0/259895298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c2c06c440 0x7f1c2c169d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:09.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.218+0000 7f1c23fff700 1 -- 172.21.15.157:0/259895298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c2c0c5fc0 msgr2=0x7f1c2c14de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:09.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.218+0000 7f1c23fff700 1 --2- 172.21.15.157:0/259895298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c2c0c5fc0 0x7f1c2c14de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:09.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.218+0000 7f1c23fff700 1 -- 172.21.15.157:0/259895298 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c280094f0 con 0x7f1c2c06c810 2023-10-06T04:10:09.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.218+0000 7f1c30d2d700 1 --2- 172.21.15.157:0/259895298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c2c06c440 0x7f1c2c169d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:10:09.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.218+0000 7f1c3152e700 1 --2- 172.21.15.157:0/259895298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c2c0c5fc0 0x7f1c2c14de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:10:09.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.218+0000 7f1c23fff700 1 --2- 172.21.15.157:0/259895298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c2c06c810 0x7f1c2c14d950 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c00c590 tx=0x7f1c1c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:09.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.218+0000 7f1c20ff9700 1 -- 172.21.15.157:0/259895298 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c011590 con 0x7f1c2c06c810 2023-10-06T04:10:09.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.218+0000 7f1c20ff9700 1 -- 172.21.15.157:0/259895298 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1c0116f0 con 0x7f1c2c06c810 2023-10-06T04:10:09.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.218+0000 7f1c32f91700 1 -- 172.21.15.157:0/259895298 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c2c0ad950 con 0x7f1c2c06c810 2023-10-06T04:10:09.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.218+0000 7f1c20ff9700 1 -- 172.21.15.157:0/259895298 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c011ab0 con 0x7f1c2c06c810 2023-10-06T04:10:09.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.218+0000 7f1c32f91700 1 -- 172.21.15.157:0/259895298 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c2c167c30 con 0x7f1c2c06c810 2023-10-06T04:10:09.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.219+0000 7f1c32f91700 1 -- 172.21.15.157:0/259895298 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c2c167a90 con 0x7f1c2c06c810 2023-10-06T04:10:09.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.220+0000 7f1c20ff9700 1 -- 172.21.15.157:0/259895298 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1c1c01b3b0 con 0x7f1c2c06c810 2023-10-06T04:10:09.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.220+0000 7f1c20ff9700 1 --2- 172.21.15.157:0/259895298 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1c18066f30 0x7f1c180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:09.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.220+0000 7f1c20ff9700 1 -- 172.21.15.157:0/259895298 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f1c1c083e70 con 0x7f1c2c06c810 2023-10-06T04:10:09.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.220+0000 7f1c30d2d700 1 --2- 172.21.15.157:0/259895298 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1c18066f30 0x7f1c180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:09.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.221+0000 7f1c30d2d700 1 --2- 172.21.15.157:0/259895298 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1c18066f30 0x7f1c180693e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1c280099b0 tx=0x7f1c28017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:09.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.224+0000 7f1c20ff9700 1 -- 172.21.15.157:0/259895298 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c2c167a90 con 0x7f1c2c06c810 2023-10-06T04:10:09.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.449+0000 7f1c32f91700 1 -- 172.21.15.157:0/259895298 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c2c167a90 con 0x7f1c2c06c810 2023-10-06T04:10:09.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.449+0000 7f1c20ff9700 1 -- 172.21.15.157:0/259895298 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+5390 (secure 0 0 0) 0x7f1c2c167a90 con 0x7f1c2c06c810 2023-10-06T04:10:09.451 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:10:09.451 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":89,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:10:06.055493+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6270,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:active","state_seq":19,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6270":{"gid":6270,"name":"e","rank":0,"incarnation":86,"state":"up:rejoin","state_seq":17,"addr":"172.21.15.157:6837/1036901172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1036901172},{"type":"v1","addr":"172.21.15.157:6837","nonce":1036901172}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:10:09.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.451+0000 7f1c32f91700 1 -- 172.21.15.157:0/259895298 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1c18066f30 msgr2=0x7f1c180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:09.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.451+0000 7f1c32f91700 1 --2- 172.21.15.157:0/259895298 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1c18066f30 0x7f1c180693e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1c280099b0 tx=0x7f1c28017000 comp rx=0 tx=0).stop 2023-10-06T04:10:09.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.451+0000 7f1c32f91700 1 -- 172.21.15.157:0/259895298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c2c06c810 msgr2=0x7f1c2c14d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:09.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.451+0000 7f1c32f91700 1 --2- 172.21.15.157:0/259895298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c2c06c810 0x7f1c2c14d950 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c00c590 tx=0x7f1c1c00c950 comp rx=0 tx=0).stop 2023-10-06T04:10:09.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.451+0000 7f1c32f91700 1 -- 172.21.15.157:0/259895298 shutdown_connections 2023-10-06T04:10:09.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.451+0000 7f1c32f91700 1 --2- 172.21.15.157:0/259895298 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1c18066f30 0x7f1c180693e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:09.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.451+0000 7f1c32f91700 1 --2- 172.21.15.157:0/259895298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c2c0c5fc0 0x7f1c2c14de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:09.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.451+0000 7f1c32f91700 1 --2- 172.21.15.157:0/259895298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c2c06c440 0x7f1c2c169d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:09.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.451+0000 7f1c32f91700 1 --2- 172.21.15.157:0/259895298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c2c06c810 0x7f1c2c14d950 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:09.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.451+0000 7f1c32f91700 1 -- 172.21.15.157:0/259895298 >> 172.21.15.157:0/259895298 conn(0x7f1c2c068370 msgr2=0x7f1c2c0cacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:09.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.451+0000 7f1c32f91700 1 -- 172.21.15.157:0/259895298 shutdown_connections 2023-10-06T04:10:09.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:09.452+0000 7f1c32f91700 1 -- 172.21.15.157:0/259895298 wait complete. 2023-10-06T04:10:09.455 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 89 2023-10-06T04:10:09.465 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-10-06T04:10:11.466 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:10:11.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.582+0000 7f16324da700 1 Processor -- start 2023-10-06T04:10:11.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.582+0000 7f16324da700 1 -- start start 2023-10-06T04:10:11.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.583+0000 7f16324da700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f162c06dc40 0x7f162c14fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:11.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.583+0000 7f16324da700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f162c06c1f0 0x7f162c150240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:11.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.583+0000 7f16324da700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f162c0c9780 con 0x7f162c06cbb0 2023-10-06T04:10:11.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.583+0000 7f16324da700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f162c0c8c40 con 0x7f162c06dc40 2023-10-06T04:10:11.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.583+0000 7f16324da700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f162c06e0c0 con 0x7f162c06c1f0 2023-10-06T04:10:11.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.583+0000 7f162b7fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f162c06dc40 0x7f162c14fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:11.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.583+0000 7f162b7fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f162c06dc40 0x7f162c14fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54460/0 (socket says 172.21.15.157:54460) 2023-10-06T04:10:11.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.583+0000 7f162b7fe700 1 -- 172.21.15.157:0/1767974468 learned_addr learned my addr 172.21.15.157:0/1767974468 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:10:11.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.583+0000 7f1630a77700 1 --2- 172.21.15.157:0/1767974468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f162c06c1f0 0x7f162c150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:11.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.584+0000 7f162b7fe700 1 -- 172.21.15.157:0/1767974468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f162c06c1f0 msgr2=0x7f162c150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:10:11.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.584+0000 7f162b7fe700 1 --2- 172.21.15.157:0/1767974468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f162c06c1f0 0x7f162c150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:11.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.584+0000 7f162b7fe700 1 -- 172.21.15.157:0/1767974468 >> v1:172.21.15.175:6789/0 conn(0x7f162c06cbb0 legacy=0x7f162c06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:10:11.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.584+0000 7f162b7fe700 1 -- 172.21.15.157:0/1767974468 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f162c0c8e50 con 0x7f162c06dc40 2023-10-06T04:10:11.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.584+0000 7f162b7fe700 1 --2- 172.21.15.157:0/1767974468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f162c06dc40 0x7f162c14fd00 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f16200099b0 tx=0x7f1620009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=885b9fdccd4c62a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:11.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.584+0000 7f162affd700 1 -- 172.21.15.157:0/1767974468 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162000b8b0 con 0x7f162c06dc40 2023-10-06T04:10:11.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.584+0000 7f162affd700 1 -- 172.21.15.157:0/1767974468 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162000ba10 con 0x7f162c06dc40 2023-10-06T04:10:11.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.584+0000 7f162affd700 1 -- 172.21.15.157:0/1767974468 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162000bdb0 con 0x7f162c06dc40 2023-10-06T04:10:11.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.584+0000 7f16324da700 1 -- 172.21.15.157:0/1767974468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f162c06dc40 msgr2=0x7f162c14fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:11.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.584+0000 7f16324da700 1 --2- 172.21.15.157:0/1767974468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f162c06dc40 0x7f162c14fd00 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f16200099b0 tx=0x7f1620009250 comp rx=0 tx=0).stop 2023-10-06T04:10:11.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.584+0000 7f16324da700 1 -- 172.21.15.157:0/1767974468 shutdown_connections 2023-10-06T04:10:11.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.584+0000 7f16324da700 1 --2- 172.21.15.157:0/1767974468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f162c06c1f0 0x7f162c150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:11.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.584+0000 7f16324da700 1 --2- 172.21.15.157:0/1767974468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f162c06dc40 0x7f162c14fd00 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:11.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.584+0000 7f16324da700 1 -- 172.21.15.157:0/1767974468 >> 172.21.15.157:0/1767974468 conn(0x7f162c06f230 msgr2=0x7f162c06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:11.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.585+0000 7f16324da700 1 -- 172.21.15.157:0/1767974468 shutdown_connections 2023-10-06T04:10:11.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.585+0000 7f16324da700 1 -- 172.21.15.157:0/1767974468 wait complete. 2023-10-06T04:10:11.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.586+0000 7f16324da700 1 Processor -- start 2023-10-06T04:10:11.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.586+0000 7f16324da700 1 -- start start 2023-10-06T04:10:11.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.586+0000 7f16324da700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f162c06c1f0 0x7f162c0c5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:11.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.586+0000 7f16324da700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f162c06cbb0 0x7f162c0c7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:11.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.586+0000 7f16324da700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f162c0c51c0 0x7f162c0c3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:11.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.586+0000 7f16324da700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162c16a950 con 0x7f162c0c51c0 2023-10-06T04:10:11.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.586+0000 7f16324da700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162c154c60 con 0x7f162c06c1f0 2023-10-06T04:10:11.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.586+0000 7f16324da700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f162c154ec0 con 0x7f162c06cbb0 2023-10-06T04:10:11.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.587+0000 7f1630a77700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f162c0c51c0 0x7f162c0c3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:11.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.587+0000 7f1630a77700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f162c0c51c0 0x7f162c0c3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54468/0 (socket says 172.21.15.157:54468) 2023-10-06T04:10:11.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.587+0000 7f1630a77700 1 -- 172.21.15.157:0/2178669024 learned_addr learned my addr 172.21.15.157:0/2178669024 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:10:11.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.587+0000 7f162b7fe700 1 --2- 172.21.15.157:0/2178669024 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f162c06cbb0 0x7f162c0c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:11.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.587+0000 7f162bfff700 1 --2- 172.21.15.157:0/2178669024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f162c06c1f0 0x7f162c0c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:11.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.587+0000 7f1630a77700 1 -- 172.21.15.157:0/2178669024 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f162c06cbb0 msgr2=0x7f162c0c7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:11.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.587+0000 7f1630a77700 1 --2- 172.21.15.157:0/2178669024 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f162c06cbb0 0x7f162c0c7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:11.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.587+0000 7f1630a77700 1 -- 172.21.15.157:0/2178669024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f162c06c1f0 msgr2=0x7f162c0c5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:11.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.587+0000 7f1630a77700 1 --2- 172.21.15.157:0/2178669024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f162c06c1f0 0x7f162c0c5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:11.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.587+0000 7f1630a77700 1 -- 172.21.15.157:0/2178669024 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16200094f0 con 0x7f162c0c51c0 2023-10-06T04:10:11.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.587+0000 7f1630a77700 1 --2- 172.21.15.157:0/2178669024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f162c0c51c0 0x7f162c0c3c70 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f1624009fa0 tx=0x7f1624009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:11.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.587+0000 7f1613fff700 1 -- 172.21.15.157:0/2178669024 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162400ed20 con 0x7f162c0c51c0 2023-10-06T04:10:11.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.587+0000 7f1613fff700 1 -- 172.21.15.157:0/2178669024 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162400ee80 con 0x7f162c0c51c0 2023-10-06T04:10:11.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.587+0000 7f16324da700 1 -- 172.21.15.157:0/2178669024 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f162c169a60 con 0x7f162c0c51c0 2023-10-06T04:10:11.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.587+0000 7f1613fff700 1 -- 172.21.15.157:0/2178669024 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1624014670 con 0x7f162c0c51c0 2023-10-06T04:10:11.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.587+0000 7f16324da700 1 -- 172.21.15.157:0/2178669024 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f162c16a020 con 0x7f162c0c51c0 2023-10-06T04:10:11.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.588+0000 7f16324da700 1 -- 172.21.15.157:0/2178669024 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f162c0c3990 con 0x7f162c0c51c0 2023-10-06T04:10:11.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.589+0000 7f1613fff700 1 -- 172.21.15.157:0/2178669024 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f16240149e0 con 0x7f162c0c51c0 2023-10-06T04:10:11.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.589+0000 7f1613fff700 1 --2- 172.21.15.157:0/2178669024 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1618066f30 0x7f16180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:11.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.589+0000 7f1613fff700 1 -- 172.21.15.157:0/2178669024 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f16240850c0 con 0x7f162c0c51c0 2023-10-06T04:10:11.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.589+0000 7f162bfff700 1 --2- 172.21.15.157:0/2178669024 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1618066f30 0x7f16180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:11.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.590+0000 7f162bfff700 1 --2- 172.21.15.157:0/2178669024 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1618066f30 0x7f16180693e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f161c0027e0 tx=0x7f161c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:11.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.593+0000 7f1613fff700 1 -- 172.21.15.157:0/2178669024 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f162c0c3990 con 0x7f162c0c51c0 2023-10-06T04:10:11.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.821+0000 7f16324da700 1 -- 172.21.15.157:0/2178669024 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f162c0c3990 con 0x7f162c0c51c0 2023-10-06T04:10:11.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.821+0000 7f1613fff700 1 -- 172.21.15.157:0/2178669024 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+5390 (secure 0 0 0) 0x7f162c0c3990 con 0x7f162c0c51c0 2023-10-06T04:10:11.823 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:10:11.823 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":90,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:10:11.225384+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6270,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:active","state_seq":19,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6270":{"gid":6270,"name":"e","rank":0,"incarnation":86,"state":"up:active","state_seq":19,"addr":"172.21.15.157:6837/1036901172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1036901172},{"type":"v1","addr":"172.21.15.157:6837","nonce":1036901172}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:10:11.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.823+0000 7f16324da700 1 -- 172.21.15.157:0/2178669024 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1618066f30 msgr2=0x7f16180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:11.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.823+0000 7f16324da700 1 --2- 172.21.15.157:0/2178669024 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1618066f30 0x7f16180693e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f161c0027e0 tx=0x7f161c009040 comp rx=0 tx=0).stop 2023-10-06T04:10:11.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.823+0000 7f16324da700 1 -- 172.21.15.157:0/2178669024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f162c0c51c0 msgr2=0x7f162c0c3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:11.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.823+0000 7f16324da700 1 --2- 172.21.15.157:0/2178669024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f162c0c51c0 0x7f162c0c3c70 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f1624009fa0 tx=0x7f1624009900 comp rx=0 tx=0).stop 2023-10-06T04:10:11.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.823+0000 7f16324da700 1 -- 172.21.15.157:0/2178669024 shutdown_connections 2023-10-06T04:10:11.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.823+0000 7f16324da700 1 --2- 172.21.15.157:0/2178669024 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1618066f30 0x7f16180693e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:11.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.823+0000 7f16324da700 1 --2- 172.21.15.157:0/2178669024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f162c06c1f0 0x7f162c0c5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:11.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.823+0000 7f16324da700 1 --2- 172.21.15.157:0/2178669024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f162c0c51c0 0x7f162c0c3c70 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:11.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.823+0000 7f16324da700 1 --2- 172.21.15.157:0/2178669024 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f162c06cbb0 0x7f162c0c7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:11.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.823+0000 7f16324da700 1 -- 172.21.15.157:0/2178669024 >> 172.21.15.157:0/2178669024 conn(0x7f162c06f230 msgr2=0x7f162c14d780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:11.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.823+0000 7f16324da700 1 -- 172.21.15.157:0/2178669024 shutdown_connections 2023-10-06T04:10:11.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:11.823+0000 7f16324da700 1 -- 172.21.15.157:0/2178669024 wait complete. 2023-10-06T04:10:11.827 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 90 2023-10-06T04:10:11.837 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e has gained rank=0, replacing gid=5799 2023-10-06T04:10:11.837 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 8 secs before reviving mds.c 2023-10-06T04:10:16.206 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:10:16.209 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:10:18.928 INFO:tasks.ceph.mds.e.smithi157.stderr:2023-10-06T04:10:18.927+0000 7f963b145700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T04:10:18.929 INFO:tasks.ceph.mds.e.smithi157.stderr:2023-10-06T04:10:18.927+0000 7f963b145700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-10-06T04:10:19.838 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2023-10-06T04:10:19.838 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T04:10:19.838 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T04:10:19.841 INFO:tasks.ceph.mds.c:Started 2023-10-06T04:10:19.841 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:10:19.924 INFO:tasks.ceph.mds.c.smithi157.stdout:starting mds.c at 2023-10-06T04:10:19.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.970+0000 7f2f22eaa700 1 Processor -- start 2023-10-06T04:10:19.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.970+0000 7f2f22eaa700 1 -- start start 2023-10-06T04:10:19.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.971+0000 7f2f22eaa700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2f1c06a780 0x7f2f1c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:19.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.971+0000 7f2f22eaa700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2f1c06e340 0x7f2f1c0c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:19.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.971+0000 7f2f22eaa700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f1c06e840 con 0x7f2f1c06ab50 2023-10-06T04:10:19.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.971+0000 7f2f22eaa700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f1c153970 con 0x7f2f1c06a780 2023-10-06T04:10:19.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.971+0000 7f2f22eaa700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f1c157100 con 0x7f2f1c06e340 2023-10-06T04:10:19.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.971+0000 7f2f20c46700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2f1c06a780 0x7f2f1c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:19.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.971+0000 7f2f21447700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f2f1c06ab50 0x7f2f1c0c2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58036/0 (socket says 172.21.15.157:58036) 2023-10-06T04:10:19.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.971+0000 7f2f21447700 1 -- 172.21.15.157:0/2940481662 learned_addr learned my addr 172.21.15.157:0/2940481662 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:10:19.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.971+0000 7f2f1bfff700 1 --2- 172.21.15.157:0/2940481662 >> v2:172.21.15.175:3300/0 conn(0x7f2f1c06e340 0x7f2f1c0c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:19.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.971+0000 7f2f20c46700 1 -- 172.21.15.157:0/2940481662 >> v2:172.21.15.175:3300/0 conn(0x7f2f1c06e340 msgr2=0x7f2f1c0c5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:10:19.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.971+0000 7f2f20c46700 1 --2- 172.21.15.157:0/2940481662 >> v2:172.21.15.175:3300/0 conn(0x7f2f1c06e340 0x7f2f1c0c5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:19.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.971+0000 7f2f20c46700 1 -- 172.21.15.157:0/2940481662 >> v1:172.21.15.157:6789/0 conn(0x7f2f1c06ab50 legacy=0x7f2f1c0c2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:19.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.971+0000 7f2f20c46700 1 -- 172.21.15.157:0/2940481662 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1c152fd0 con 0x7f2f1c06a780 2023-10-06T04:10:19.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.971+0000 7f2f1b7fe700 1 -- 172.21.15.157:0/2940481662 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3649653133 0 0) 0x7f2f1c06e840 con 0x7f2f1c06ab50 2023-10-06T04:10:19.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.971+0000 7f2f20c46700 1 --2- 172.21.15.157:0/2940481662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f1c06a780 0x7f2f1c156bc0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c0c12f0 tx=0x7f2f100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d84f970f41643c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:19.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.971+0000 7f2f1b7fe700 1 -- 172.21.15.157:0/2940481662 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f10009e80 con 0x7f2f1c06a780 2023-10-06T04:10:19.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.972+0000 7f2f1b7fe700 1 -- 172.21.15.157:0/2940481662 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1000b8b0 con 0x7f2f1c06a780 2023-10-06T04:10:19.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.972+0000 7f2f1b7fe700 1 -- 172.21.15.157:0/2940481662 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1000bc30 con 0x7f2f1c06a780 2023-10-06T04:10:19.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.972+0000 7f2f22eaa700 1 -- 172.21.15.157:0/2940481662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f1c06a780 msgr2=0x7f2f1c156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:19.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.972+0000 7f2f22eaa700 1 --2- 172.21.15.157:0/2940481662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f1c06a780 0x7f2f1c156bc0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c0c12f0 tx=0x7f2f100091a0 comp rx=0 tx=0).stop 2023-10-06T04:10:19.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.972+0000 7f2f22eaa700 1 -- 172.21.15.157:0/2940481662 shutdown_connections 2023-10-06T04:10:19.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.972+0000 7f2f22eaa700 1 --2- 172.21.15.157:0/2940481662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f1c06a780 0x7f2f1c156bc0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:19.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.972+0000 7f2f22eaa700 1 --2- 172.21.15.157:0/2940481662 >> v2:172.21.15.175:3300/0 conn(0x7f2f1c06e340 0x7f2f1c0c5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:19.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.972+0000 7f2f22eaa700 1 -- 172.21.15.157:0/2940481662 >> 172.21.15.157:0/2940481662 conn(0x7f2f1c06b510 msgr2=0x7f2f1c06b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:19.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.972+0000 7f2f22eaa700 1 -- 172.21.15.157:0/2940481662 shutdown_connections 2023-10-06T04:10:19.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.972+0000 7f2f22eaa700 1 -- 172.21.15.157:0/2940481662 wait complete. 2023-10-06T04:10:19.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.973+0000 7f2f22eaa700 1 Processor -- start 2023-10-06T04:10:19.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.973+0000 7f2f22eaa700 1 -- start start 2023-10-06T04:10:19.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.974+0000 7f2f22eaa700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f1c06a780 0x7f2f1c0c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:19.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.974+0000 7f2f22eaa700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f1c06ab50 0x7f2f1c155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:19.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.974+0000 7f2f22eaa700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f1c06e340 0x7f2f1c155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:19.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.974+0000 7f2f22eaa700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c163380 con 0x7f2f1c06a780 2023-10-06T04:10:19.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.974+0000 7f2f22eaa700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c0c9110 con 0x7f2f1c06ab50 2023-10-06T04:10:19.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.974+0000 7f2f22eaa700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c0c7580 con 0x7f2f1c06e340 2023-10-06T04:10:19.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.974+0000 7f2f20c46700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f1c06a780 0x7f2f1c0c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:19.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.974+0000 7f2f1bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f1c06ab50 0x7f2f1c155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:19.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.974+0000 7f2f20c46700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f1c06a780 0x7f2f1c0c7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51354/0 (socket says 172.21.15.157:51354) 2023-10-06T04:10:19.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.974+0000 7f2f1bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f1c06ab50 0x7f2f1c155830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:51984/0 (socket says 172.21.15.157:51984) 2023-10-06T04:10:19.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.974+0000 7f2f20c46700 1 -- 172.21.15.157:0/670956038 learned_addr learned my addr 172.21.15.157:0/670956038 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:10:19.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.974+0000 7f2f21447700 1 --2- 172.21.15.157:0/670956038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f1c06e340 0x7f2f1c155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:19.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.974+0000 7f2f1bfff700 1 -- 172.21.15.157:0/670956038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f1c06e340 msgr2=0x7f2f1c155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:19.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.974+0000 7f2f1bfff700 1 --2- 172.21.15.157:0/670956038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f1c06e340 0x7f2f1c155d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:19.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.974+0000 7f2f1bfff700 1 -- 172.21.15.157:0/670956038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f1c06a780 msgr2=0x7f2f1c0c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:19.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.974+0000 7f2f1bfff700 1 --2- 172.21.15.157:0/670956038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f1c06a780 0x7f2f1c0c7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:19.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.974+0000 7f2f1bfff700 1 -- 172.21.15.157:0/670956038 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f10009420 con 0x7f2f1c06ab50 2023-10-06T04:10:19.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.974+0000 7f2f20c46700 1 --2- 172.21.15.157:0/670956038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f1c06a780 0x7f2f1c0c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:10:19.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.975+0000 7f2f1bfff700 1 --2- 172.21.15.157:0/670956038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f1c06ab50 0x7f2f1c155830 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2f0800d890 tx=0x7f2f08008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:19.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.975+0000 7f2f18ff9700 1 -- 172.21.15.157:0/670956038 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f080125f0 con 0x7f2f1c06ab50 2023-10-06T04:10:19.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.975+0000 7f2f18ff9700 1 -- 172.21.15.157:0/670956038 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f08012750 con 0x7f2f1c06ab50 2023-10-06T04:10:19.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.975+0000 7f2f22eaa700 1 -- 172.21.15.157:0/670956038 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f1c163810 con 0x7f2f1c06ab50 2023-10-06T04:10:19.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.975+0000 7f2f18ff9700 1 -- 172.21.15.157:0/670956038 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f08012af0 con 0x7f2f1c06ab50 2023-10-06T04:10:19.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.975+0000 7f2f22eaa700 1 -- 172.21.15.157:0/670956038 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f1c0c8e30 con 0x7f2f1c06ab50 2023-10-06T04:10:19.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.976+0000 7f2f22eaa700 1 -- 172.21.15.157:0/670956038 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f1c0692b0 con 0x7f2f1c06ab50 2023-10-06T04:10:19.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.976+0000 7f2f18ff9700 1 -- 172.21.15.157:0/670956038 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2f0801e460 con 0x7f2f1c06ab50 2023-10-06T04:10:19.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.976+0000 7f2f18ff9700 1 --2- 172.21.15.157:0/670956038 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2f00066f30 0x7f2f000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:19.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.977+0000 7f2f18ff9700 1 -- 172.21.15.157:0/670956038 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f2f08084d50 con 0x7f2f1c06ab50 2023-10-06T04:10:19.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.977+0000 7f2f20c46700 1 --2- 172.21.15.157:0/670956038 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2f00066f30 0x7f2f000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:19.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.977+0000 7f2f20c46700 1 --2- 172.21.15.157:0/670956038 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2f00066f30 0x7f2f000693e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2f100093c0 tx=0x7f2f10017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:19.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:19.981+0000 7f2f18ff9700 1 -- 172.21.15.157:0/670956038 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2f08088030 con 0x7f2f1c06ab50 2023-10-06T04:10:20.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:20.206+0000 7f2f22eaa700 1 -- 172.21.15.157:0/670956038 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f1c1532b0 con 0x7f2f1c06ab50 2023-10-06T04:10:20.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:20.207+0000 7f2f18ff9700 1 -- 172.21.15.157:0/670956038 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+5390 (secure 0 0 0) 0x7f2f08055660 con 0x7f2f1c06ab50 2023-10-06T04:10:20.208 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:10:20.209 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":90,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:10:11.225384+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6270,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:active","state_seq":19,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6270":{"gid":6270,"name":"e","rank":0,"incarnation":86,"state":"up:active","state_seq":19,"addr":"172.21.15.157:6837/1036901172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1036901172},{"type":"v1","addr":"172.21.15.157:6837","nonce":1036901172}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:10:20.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:20.209+0000 7f2f22eaa700 1 -- 172.21.15.157:0/670956038 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2f00066f30 msgr2=0x7f2f000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:20.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:20.209+0000 7f2f22eaa700 1 --2- 172.21.15.157:0/670956038 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2f00066f30 0x7f2f000693e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2f100093c0 tx=0x7f2f10017000 comp rx=0 tx=0).stop 2023-10-06T04:10:20.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:20.209+0000 7f2f22eaa700 1 -- 172.21.15.157:0/670956038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f1c06ab50 msgr2=0x7f2f1c155830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:20.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:20.209+0000 7f2f22eaa700 1 --2- 172.21.15.157:0/670956038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f1c06ab50 0x7f2f1c155830 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2f0800d890 tx=0x7f2f08008040 comp rx=0 tx=0).stop 2023-10-06T04:10:20.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:20.209+0000 7f2f22eaa700 1 -- 172.21.15.157:0/670956038 shutdown_connections 2023-10-06T04:10:20.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:20.209+0000 7f2f22eaa700 1 --2- 172.21.15.157:0/670956038 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2f00066f30 0x7f2f000693e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:20.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:20.209+0000 7f2f22eaa700 1 --2- 172.21.15.157:0/670956038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f1c06e340 0x7f2f1c155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:20.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:20.209+0000 7f2f22eaa700 1 --2- 172.21.15.157:0/670956038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f1c06a780 0x7f2f1c0c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:20.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:20.209+0000 7f2f22eaa700 1 --2- 172.21.15.157:0/670956038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f1c06ab50 0x7f2f1c155830 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:20.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:20.209+0000 7f2f22eaa700 1 -- 172.21.15.157:0/670956038 >> 172.21.15.157:0/670956038 conn(0x7f2f1c06b510 msgr2=0x7f2f1c1514c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:20.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:20.209+0000 7f2f22eaa700 1 -- 172.21.15.157:0/670956038 shutdown_connections 2023-10-06T04:10:20.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:20.209+0000 7f2f22eaa700 1 -- 172.21.15.157:0/670956038 wait complete. 2023-10-06T04:10:20.212 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 90 2023-10-06T04:10:20.223 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2023-10-06T04:10:22.224 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:10:22.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.339+0000 7f0edab9f700 1 Processor -- start 2023-10-06T04:10:22.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.339+0000 7f0edab9f700 1 -- start start 2023-10-06T04:10:22.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.340+0000 7f0edab9f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ed40b1e40 0x7f0ed4157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:22.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.340+0000 7f0edab9f700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed406d8d0 con 0x7f0ed406c2d0 2023-10-06T04:10:22.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.340+0000 7f0edab9f700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed40af8f0 con 0x7f0ed40b04d0 2023-10-06T04:10:22.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.340+0000 7f0edab9f700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed415a690 con 0x7f0ed40b1e40 2023-10-06T04:10:22.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.340+0000 7f0eda39e700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f0ed406c2d0 0x7f0ed4156ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58044/0 (socket says 172.21.15.157:58044) 2023-10-06T04:10:22.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.340+0000 7f0eda39e700 1 -- 172.21.15.157:0/137914183 learned_addr learned my addr 172.21.15.157:0/137914183 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:10:22.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.340+0000 7f0ed9b9d700 1 --2- 172.21.15.157:0/137914183 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ed40b1e40 0x7f0ed4157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:22.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.340+0000 7f0ed8b9b700 1 -- 172.21.15.157:0/137914183 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3865132164 0 0) 0x7f0ed406d8d0 con 0x7f0ed406c2d0 2023-10-06T04:10:22.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.340+0000 7f0ed8b9b700 1 -- 172.21.15.157:0/137914183 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ebc003570 con 0x7f0ed406c2d0 2023-10-06T04:10:22.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.340+0000 7f0ed8b9b700 1 -- 172.21.15.157:0/137914183 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4071995614 0 0) 0x7f0ed40af8f0 con 0x7f0ed40b04d0 2023-10-06T04:10:22.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.340+0000 7f0ed8b9b700 1 -- 172.21.15.157:0/137914183 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ed406d8d0 con 0x7f0ed40b04d0 2023-10-06T04:10:22.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.341+0000 7f0ed9b9d700 1 -- 172.21.15.157:0/137914183 >> v1:172.21.15.175:6789/0 conn(0x7f0ed40b04d0 legacy=0x7f0ed41563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:22.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.341+0000 7f0ed9b9d700 1 -- 172.21.15.157:0/137914183 >> v1:172.21.15.157:6789/0 conn(0x7f0ed406c2d0 legacy=0x7f0ed4156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:22.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.341+0000 7f0ed9b9d700 1 -- 172.21.15.157:0/137914183 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed41571d0 con 0x7f0ed40b1e40 2023-10-06T04:10:22.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.341+0000 7f0ed9b9d700 1 --2- 172.21.15.157:0/137914183 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ed40b1e40 0x7f0ed4157750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4009e90 tx=0x7f0ec4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31eda3bd897c03ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:22.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.341+0000 7f0ed8b9b700 1 -- 172.21.15.157:0/137914183 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec400e070 con 0x7f0ed40b1e40 2023-10-06T04:10:22.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.341+0000 7f0ed8b9b700 1 -- 172.21.15.157:0/137914183 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec400bb80 con 0x7f0ed40b1e40 2023-10-06T04:10:22.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.341+0000 7f0ed8b9b700 1 -- 172.21.15.157:0/137914183 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec4012430 con 0x7f0ed40b1e40 2023-10-06T04:10:22.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.341+0000 7f0edab9f700 1 -- 172.21.15.157:0/137914183 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ed40b1e40 msgr2=0x7f0ed4157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:22.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.341+0000 7f0edab9f700 1 --2- 172.21.15.157:0/137914183 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ed40b1e40 0x7f0ed4157750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4009e90 tx=0x7f0ec4009780 comp rx=0 tx=0).stop 2023-10-06T04:10:22.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.341+0000 7f0edab9f700 1 -- 172.21.15.157:0/137914183 shutdown_connections 2023-10-06T04:10:22.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.341+0000 7f0edab9f700 1 --2- 172.21.15.157:0/137914183 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ed40b1e40 0x7f0ed4157750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:22.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.341+0000 7f0edab9f700 1 -- 172.21.15.157:0/137914183 >> 172.21.15.157:0/137914183 conn(0x7f0ed40cc540 msgr2=0x7f0ed40cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:22.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.341+0000 7f0edab9f700 1 -- 172.21.15.157:0/137914183 shutdown_connections 2023-10-06T04:10:22.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.341+0000 7f0edab9f700 1 -- 172.21.15.157:0/137914183 wait complete. 2023-10-06T04:10:22.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.342+0000 7f0edab9f700 1 Processor -- start 2023-10-06T04:10:22.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.343+0000 7f0edab9f700 1 -- start start 2023-10-06T04:10:22.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.343+0000 7f0edab9f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ed406c2d0 0x7f0ed41527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:22.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.343+0000 7f0edab9f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ed40b04d0 0x7f0ed414f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:22.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.343+0000 7f0edab9f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ed40b1e40 0x7f0ed414fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:22.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.343+0000 7f0edab9f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed40cd010 con 0x7f0ed40b1e40 2023-10-06T04:10:22.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.343+0000 7f0edab9f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed4153d40 con 0x7f0ed40b04d0 2023-10-06T04:10:22.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.343+0000 7f0edab9f700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed4152360 con 0x7f0ed406c2d0 2023-10-06T04:10:22.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.343+0000 7f0eda39e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ed40b1e40 0x7f0ed414fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:22.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.343+0000 7f0eda39e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ed40b1e40 0x7f0ed414fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51362/0 (socket says 172.21.15.157:51362) 2023-10-06T04:10:22.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.343+0000 7f0ed9b9d700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ed406c2d0 0x7f0ed41527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:22.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.343+0000 7f0eda39e700 1 -- 172.21.15.157:0/1987226518 learned_addr learned my addr 172.21.15.157:0/1987226518 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:10:22.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.343+0000 7f0ed939c700 1 --2- 172.21.15.157:0/1987226518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ed40b04d0 0x7f0ed414f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:22.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.344+0000 7f0ed9b9d700 1 -- 172.21.15.157:0/1987226518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ed40b04d0 msgr2=0x7f0ed414f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:22.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.344+0000 7f0ed9b9d700 1 --2- 172.21.15.157:0/1987226518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ed40b04d0 0x7f0ed414f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:22.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.344+0000 7f0ed9b9d700 1 -- 172.21.15.157:0/1987226518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ed40b1e40 msgr2=0x7f0ed414fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:22.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.344+0000 7f0ed9b9d700 1 --2- 172.21.15.157:0/1987226518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ed40b1e40 0x7f0ed414fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:22.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.344+0000 7f0ed9b9d700 1 -- 172.21.15.157:0/1987226518 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec4009a20 con 0x7f0ed406c2d0 2023-10-06T04:10:22.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.344+0000 7f0eda39e700 1 --2- 172.21.15.157:0/1987226518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ed40b1e40 0x7f0ed414fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:10:22.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.344+0000 7f0ed9b9d700 1 --2- 172.21.15.157:0/1987226518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ed406c2d0 0x7f0ed41527f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4016000 tx=0x7f0ec4019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:22.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.344+0000 7f0ed1ffb700 1 -- 172.21.15.157:0/1987226518 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec400be40 con 0x7f0ed406c2d0 2023-10-06T04:10:22.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.344+0000 7f0ed1ffb700 1 -- 172.21.15.157:0/1987226518 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec4025430 con 0x7f0ed406c2d0 2023-10-06T04:10:22.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.344+0000 7f0edab9f700 1 -- 172.21.15.157:0/1987226518 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed4166070 con 0x7f0ed406c2d0 2023-10-06T04:10:22.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.344+0000 7f0ed1ffb700 1 -- 172.21.15.157:0/1987226518 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec40257f0 con 0x7f0ed406c2d0 2023-10-06T04:10:22.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.344+0000 7f0edab9f700 1 -- 172.21.15.157:0/1987226518 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed415c4f0 con 0x7f0ed406c2d0 2023-10-06T04:10:22.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.345+0000 7f0edab9f700 1 -- 172.21.15.157:0/1987226518 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed4088f90 con 0x7f0ed406c2d0 2023-10-06T04:10:22.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.346+0000 7f0ed1ffb700 1 -- 172.21.15.157:0/1987226518 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0ec401c400 con 0x7f0ed406c2d0 2023-10-06T04:10:22.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.346+0000 7f0ed1ffb700 1 --2- 172.21.15.157:0/1987226518 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0ec0066f60 0x7f0ec0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:22.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.346+0000 7f0ed1ffb700 1 -- 172.21.15.157:0/1987226518 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f0ec4086440 con 0x7f0ed406c2d0 2023-10-06T04:10:22.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.346+0000 7f0ed939c700 1 --2- 172.21.15.157:0/1987226518 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0ec0066f60 0x7f0ec0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:22.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.347+0000 7f0ed939c700 1 --2- 172.21.15.157:0/1987226518 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0ec0066f60 0x7f0ec0069410 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0ed40aee90 tx=0x7f0ec8007d50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:22.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.351+0000 7f0ed1ffb700 1 -- 172.21.15.157:0/1987226518 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0ec4056d30 con 0x7f0ed406c2d0 2023-10-06T04:10:22.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.582+0000 7f0edab9f700 1 -- 172.21.15.157:0/1987226518 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ed406ccc0 con 0x7f0ed406c2d0 2023-10-06T04:10:22.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.583+0000 7f0ed1ffb700 1 -- 172.21.15.157:0/1987226518 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+6147 (secure 0 0 0) 0x7f0ec40568c0 con 0x7f0ed406c2d0 2023-10-06T04:10:22.584 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:10:22.584 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":91,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6417,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/608486467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":608486467},{"type":"v1","addr":"172.21.15.157:6834","nonce":608486467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:10:11.225384+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6270,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:active","state_seq":19,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6270":{"gid":6270,"name":"e","rank":0,"incarnation":86,"state":"up:active","state_seq":19,"addr":"172.21.15.157:6837/1036901172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1036901172},{"type":"v1","addr":"172.21.15.157:6837","nonce":1036901172}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:10:22.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.585+0000 7f0edab9f700 1 -- 172.21.15.157:0/1987226518 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0ec0066f60 msgr2=0x7f0ec0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:22.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.585+0000 7f0edab9f700 1 --2- 172.21.15.157:0/1987226518 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0ec0066f60 0x7f0ec0069410 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0ed40aee90 tx=0x7f0ec8007d50 comp rx=0 tx=0).stop 2023-10-06T04:10:22.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.585+0000 7f0edab9f700 1 -- 172.21.15.157:0/1987226518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ed406c2d0 msgr2=0x7f0ed41527f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:22.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.585+0000 7f0edab9f700 1 --2- 172.21.15.157:0/1987226518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ed406c2d0 0x7f0ed41527f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4016000 tx=0x7f0ec4019000 comp rx=0 tx=0).stop 2023-10-06T04:10:22.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.585+0000 7f0edab9f700 1 -- 172.21.15.157:0/1987226518 shutdown_connections 2023-10-06T04:10:22.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.585+0000 7f0edab9f700 1 --2- 172.21.15.157:0/1987226518 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f0ec0066f60 0x7f0ec0069410 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:22.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.585+0000 7f0edab9f700 1 --2- 172.21.15.157:0/1987226518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ed40b1e40 0x7f0ed414fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:22.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.585+0000 7f0edab9f700 1 --2- 172.21.15.157:0/1987226518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ed406c2d0 0x7f0ed41527f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:22.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.585+0000 7f0edab9f700 1 --2- 172.21.15.157:0/1987226518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ed40b04d0 0x7f0ed414f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:22.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.585+0000 7f0edab9f700 1 -- 172.21.15.157:0/1987226518 >> 172.21.15.157:0/1987226518 conn(0x7f0ed40cc540 msgr2=0x7f0ed40afa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:22.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.585+0000 7f0edab9f700 1 -- 172.21.15.157:0/1987226518 shutdown_connections 2023-10-06T04:10:22.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:22.585+0000 7f0edab9f700 1 -- 172.21.15.157:0/1987226518 wait complete. 2023-10-06T04:10:22.588 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 91 2023-10-06T04:10:22.598 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2023-10-06T04:10:22.598 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 97 secs before thrashing 2023-10-06T04:10:46.305 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:10:46.307 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:10:53.923 INFO:tasks.workunit.client.0.smithi157.stderr:Updating files: 2% (268/12065) Updating files: 3% (362/12065) Updating files: 3% (407/12065) Updating files: 4% (483/12065) Updating files: 4% (528/12065) Updating files: 5% (604/12065) Updating files: 5% (688/12065) Updating files: 6% (724/12065) Updating files: 6% (754/12065) Updating files: 6% (795/12065) Updating files: 7% (845/12065) Updating files: 7% (861/12065) Updating files: 8% (966/12065) Updating files: 8% (975/12065) Updating files: 9% (1086/12065) Updating files: 9% (1153/12065) Updating files: 10% (1207/12065) Updating files: 10% (1267/12065) Updating files: 11% (1328/12065) Updating files: 11% (1397/12065) Updating files: 11% (1424/12065) Updating files: 12% (1448/12065) Updating files: 12% (1568/12065) Updating files: 13% (1569/12065) Updating files: 14% (1690/12065) Updating files: 15% (1810/12065) Updating files: 15% (1814/12065) Updating files: 16% (1931/12065) Updating files: 16% (1947/12065) Updating files: 17% (2052/12065) Updating files: 17% (2063/12065) Updating files: 17% (2157/12065) Updating files: 18% (2172/12065) Updating files: 19% (2293/12065) Updating files: 19% (2407/12065) Updating files: 19% (2410/12065) Updating files: 19% (2411/12065) Updating files: 20% (2413/12065) Updating files: 20% (2435/12065) Updating files: 21% (2534/12065) Updating files: 22% (2655/12065) Updating files: 22% (2728/12065) Updating files: 23% (2775/12065) Updating files: 24% (2896/12065) Updating files: 24% (2991/12065) Updating files: 25% (3017/12065) Updating files: 25% (3032/12065) Updating files: 26% (3137/12065) Updating files: 26% (3239/12065) Updating files: 27% (3258/12065) Updating files: 28% (3379/12065) Updating files: 28% (3394/12065) Updating files: 28% (3443/12065) Updating files: 28% (3470/12065) Updating files: 29% (3499/12065) Updating files: 29% (3589/12065) Updating files: 30% (3620/12065) Updating files: 30% (3698/12065) Updating files: 31% (3741/12065) Updating files: 31% (3790/12065) Updating files: 32% (3861/12065) Updating files: 32% (3954/12065) Updating files: 32% (3973/12065) Updating files: 33% (3982/12065) Updating files: 33% (4084/12065) Updating files: 34% (4103/12065) Updating files: 34% (4130/12065) Updating files: 35% (4223/12065) Updating files: 35% (4280/12065) Updating files: 36% (4344/12065) Updating files: 37% (4465/12065) Updating files: 38% (4585/12065) Updating files: 38% (4646/12065) Updating files: 38% (4692/12065) Updating files: 39% (4706/12065) Updating files: 39% (4753/12065) Updating files: 40% (4826/12065) Updating files: 40% (4934/12065) Updating files: 40% (4941/12065) Updating files: 41% (4947/12065) Updating files: 41% (4971/12065) Updating files: 41% (5024/12065) Updating files: 42% (5068/12065) Updating files: 42% (5125/12065) Updating files: 43% (5188/12065) Updating files: 44% (5309/12065) Updating files: 44% (5325/12065) Updating files: 44% (5348/12065) Updating files: 44% (5412/12065) Updating files: 44% (5415/12065) Updating files: 45% (5430/12065) Updating files: 45% (5548/12065) Updating files: 46% (5550/12065) Updating files: 46% (5638/12065) Updating files: 47% (5671/12065) Updating files: 47% (5706/12065) Updating files: 47% (5728/12065) Updating files: 48% (5792/12065) Updating files: 48% (5793/12065) Updating files: 48% (5858/12065) Updating files: 49% (5912/12065) Updating files: 49% (5932/12065) Updating files: 50% (6033/12065) Updating files: 50% (6096/12065) Updating files: 51% (6154/12065) Updating files: 51% (6255/12065) Updating files: 52% (6274/12065) Updating files: 52% (6311/12065) Updating files: 53% (6395/12065) Updating files: 53% (6457/12065) Updating files: 54% (6516/12065) Updating files: 55% (6636/12065) Updating files: 56% (6757/12065) Updating files: 57% (6878/12065) Updating files: 57% (6943/12065) Updating files: 57% (6951/12065) Updating files: 58% (6998/12065) Updating files: 58% (7024/12065) Updating files: 59% (7119/12065) Updating files: 59% (7130/12065) Updating files: 60% (7239/12065) Updating files: 61% (7360/12065) Updating files: 61% (7383/12065) Updating files: 61% (7402/12065) Updating files: 62% (7481/12065) Updating files: 62% (7587/12065) Updating files: 63% (7601/12065) Updating files: 64% (7722/12065) Updating files: 65% (7843/12065) Updating files: 65% (7856/12065) Updating files: 65% (7940/12065) Updating files: 66% (7963/12065) Updating files: 66% (8068/12065) Updating files: 67% (8084/12065) Updating files: 67% (8096/12065) Updating files: 67% (8174/12065) Updating files: 68% (8205/12065) Updating files: 68% (8273/12065) Updating files: 68% (8282/12065) Updating files: 69% (8325/12065) Updating files: 70% (8446/12065) Updating files: 70% (8462/12065) Updating files: 70% (8486/12065) Updating files: 71% (8567/12065) Updating files: 71% (8639/12065) Updating files: 72% (8687/12065) Updating files: 73% (8808/12065) Updating files: 73% (8817/12065) Updating files: 74% (8929/12065) Updating files: 74% (8934/12065) Updating files: 74% (8969/12065) Updating files: 75% (9049/12065) Updating files: 75% (9065/12065) Updating files: 75% (9159/12065) Updating files: 76% (9170/12065) Updating files: 77% (9291/12065) Updating files: 78% (9411/12065) Updating files: 78% (9475/12065) Updating files: 79% (9532/12065) Updating files: 80% (9652/12065) Updating files: 80% (9701/12065) Updating files: 81% (9773/12065) Updating files: 81% (9790/12065) Updating files: 82% (9894/12065) Updating files: 83% (10014/12065) Updating files: 84% (10135/12065) Updating files: 85% (10256/12065) Updating files: 85% (10259/12065) Updating files: 86% (10376/12065) Updating files: 86% (10394/12065) Updating files: 87% (10497/12065) Updating files: 88% (10618/12065) Updating files: 88% (10670/12065) Updating files: 88% (10727/12065) Updating files: 89% (10738/12065) Updating files: 89% (10741/12065) Updating files: 89% (10784/12065) Updating files: 89% (10841/12065) Updating files: 90% (10859/12065) Updating files: 91% (10980/12065) Updating files: 92% (11100/12065) Updating files: 92% (11170/12065) Updating files: 93% (11221/12065) Updating files: 94% (11342/12065) Updating files: 95% (11462/12065) Updating files: 96% (11583/12065) Updating files: 96% (11596/12065) Updating files: 96% (11617/12065) Updating files: 96% (11632/12065) Updating files: 97% (11704/12065) Updating files: 98% (11824/12065) Updating files: 98% (11841/12065) Updating files: 99% (11945/12065) Updating files: 99% (11956/12065) Updating files: 99% (12033/12065) Updating files: 100% (12065/12065) Updating files: 100% (12065/12065), done. 2023-10-06T04:10:53.964 DEBUG:teuthology.orchestra.run:got remote process result: 128 2023-10-06T04:10:53.964 INFO:tasks.workunit:Stopping ['fs/snaps'] on client.0... 2023-10-06T04:10:53.964 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-10-06T04:10:54.232 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi157 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh' 2023-10-06T04:10:54.304 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=31663c36722e41cf9a6e95b258d87ac8 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi157 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh' 2023-10-06T04:10:54.306 DEBUG:teuthology.run_tasks:Unwinding manager mds_thrash 2023-10-06T04:10:54.317 INFO:tasks.mds_thrash:joining mds_thrasher 2023-10-06T04:10:54.320 INFO:tasks.mds_thrash.fs.[cephfs]:stat['max_mds'] = 1 2023-10-06T04:10:54.320 INFO:tasks.mds_thrash.fs.[cephfs]:stat['deactivate'] = 0 2023-10-06T04:10:54.320 INFO:tasks.mds_thrash.fs.[cephfs]:stat['kill'] = 8 2023-10-06T04:10:54.320 INFO:tasks.mds_thrash:done joining 2023-10-06T04:10:54.321 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-06T04:10:54.329 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-06T04:10:54.330 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T04:10:54.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:10:54.330 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T04:10:55.516 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-10-06T04:10:55.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:10:55.516 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T04:10:55.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:10:55.531 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-10-06T04:10:55.586 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T04:10:55.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:10:55.587 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T04:10:55.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:10:55.672 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2023-10-06T04:10:55.745 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-06T04:10:55.780 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-10-06T04:10:55.780 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:10:55.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.895+0000 7f3d906d7700 1 Processor -- start 2023-10-06T04:10:55.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.895+0000 7f3d906d7700 1 -- start start 2023-10-06T04:10:55.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.896+0000 7f3d906d7700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3d8806ade0 0x7f3d880694a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:55.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.896+0000 7f3d906d7700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3d880c4570 0x7f3d880699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:55.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.896+0000 7f3d906d7700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d880c5680 con 0x7f3d880ad4c0 2023-10-06T04:10:55.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.896+0000 7f3d906d7700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d88158440 con 0x7f3d8806ade0 2023-10-06T04:10:55.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.896+0000 7f3d906d7700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d8814e940 con 0x7f3d880c4570 2023-10-06T04:10:55.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.896+0000 7f3d8dc72700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3d8806ade0 0x7f3d880694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:55.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.896+0000 7f3d8dc72700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3d8806ade0 0x7f3d880694a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51528/0 (socket says 172.21.15.157:51528) 2023-10-06T04:10:55.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.896+0000 7f3d8dc72700 1 -- 172.21.15.157:0/760768685 learned_addr learned my addr 172.21.15.157:0/760768685 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:10:55.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.897+0000 7f3d8ec74700 1 --2- 172.21.15.157:0/760768685 >> v2:172.21.15.175:3300/0 conn(0x7f3d880c4570 0x7f3d880699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:55.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.897+0000 7f3d8dc72700 1 -- 172.21.15.157:0/760768685 >> v2:172.21.15.175:3300/0 conn(0x7f3d880c4570 msgr2=0x7f3d880699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:10:55.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.897+0000 7f3d8dc72700 1 --2- 172.21.15.157:0/760768685 >> v2:172.21.15.175:3300/0 conn(0x7f3d880c4570 0x7f3d880699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:55.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.897+0000 7f3d8dc72700 1 -- 172.21.15.157:0/760768685 >> v1:172.21.15.175:6789/0 conn(0x7f3d880ad4c0 legacy=0x7f3d880ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:55.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.897+0000 7f3d8dc72700 1 -- 172.21.15.157:0/760768685 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d880c5920 con 0x7f3d8806ade0 2023-10-06T04:10:55.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.897+0000 7f3d8ec74700 1 --2- 172.21.15.157:0/760768685 >> v2:172.21.15.175:3300/0 conn(0x7f3d880c4570 0x7f3d880699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:10:55.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.897+0000 7f3d8dc72700 1 --2- 172.21.15.157:0/760768685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d8806ade0 0x7f3d880694a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3d88158340 tx=0x7f3d78009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5996ab20006251 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:55.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.897+0000 7f3d8d471700 1 -- 172.21.15.157:0/760768685 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7800b820 con 0x7f3d8806ade0 2023-10-06T04:10:55.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.897+0000 7f3d8d471700 1 -- 172.21.15.157:0/760768685 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7800b980 con 0x7f3d8806ade0 2023-10-06T04:10:55.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.897+0000 7f3d8d471700 1 -- 172.21.15.157:0/760768685 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7800bd20 con 0x7f3d8806ade0 2023-10-06T04:10:55.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.898+0000 7f3d906d7700 1 -- 172.21.15.157:0/760768685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d8806ade0 msgr2=0x7f3d880694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:55.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.898+0000 7f3d906d7700 1 --2- 172.21.15.157:0/760768685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d8806ade0 0x7f3d880694a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3d88158340 tx=0x7f3d78009250 comp rx=0 tx=0).stop 2023-10-06T04:10:55.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.898+0000 7f3d906d7700 1 -- 172.21.15.157:0/760768685 shutdown_connections 2023-10-06T04:10:55.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.898+0000 7f3d906d7700 1 --2- 172.21.15.157:0/760768685 >> v2:172.21.15.175:3300/0 conn(0x7f3d880c4570 0x7f3d880699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:55.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.898+0000 7f3d906d7700 1 --2- 172.21.15.157:0/760768685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d8806ade0 0x7f3d880694a0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:55.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.898+0000 7f3d906d7700 1 -- 172.21.15.157:0/760768685 >> 172.21.15.157:0/760768685 conn(0x7f3d880c8d70 msgr2=0x7f3d880c4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:55.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.898+0000 7f3d906d7700 1 -- 172.21.15.157:0/760768685 shutdown_connections 2023-10-06T04:10:55.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.898+0000 7f3d906d7700 1 -- 172.21.15.157:0/760768685 wait complete. 2023-10-06T04:10:55.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.899+0000 7f3d906d7700 1 Processor -- start 2023-10-06T04:10:55.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.899+0000 7f3d906d7700 1 -- start start 2023-10-06T04:10:55.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.899+0000 7f3d906d7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d8806ade0 0x7f3d881674d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:55.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.899+0000 7f3d906d7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d880ad4c0 0x7f3d880c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:55.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.899+0000 7f3d906d7700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d880c4570 0x7f3d8814f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:55.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.899+0000 7f3d906d7700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8806f2a0 con 0x7f3d880ad4c0 2023-10-06T04:10:55.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.899+0000 7f3d906d7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d88167a10 con 0x7f3d8806ade0 2023-10-06T04:10:55.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.899+0000 7f3d906d7700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8814f220 con 0x7f3d880c4570 2023-10-06T04:10:55.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.900+0000 7f3d8dc72700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d880ad4c0 0x7f3d880c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:55.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.900+0000 7f3d8dc72700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d880ad4c0 0x7f3d880c72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51530/0 (socket says 172.21.15.157:51530) 2023-10-06T04:10:55.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.900+0000 7f3d8dc72700 1 -- 172.21.15.157:0/1266395636 learned_addr learned my addr 172.21.15.157:0/1266395636 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:10:55.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.900+0000 7f3d8ec74700 1 --2- 172.21.15.157:0/1266395636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d880c4570 0x7f3d8814f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:55.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.900+0000 7f3d8e473700 1 --2- 172.21.15.157:0/1266395636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d8806ade0 0x7f3d881674d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:55.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.900+0000 7f3d8dc72700 1 -- 172.21.15.157:0/1266395636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d880c4570 msgr2=0x7f3d8814f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:55.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.900+0000 7f3d8dc72700 1 --2- 172.21.15.157:0/1266395636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d880c4570 0x7f3d8814f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:55.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.900+0000 7f3d8dc72700 1 -- 172.21.15.157:0/1266395636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d8806ade0 msgr2=0x7f3d881674d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:55.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.900+0000 7f3d8dc72700 1 --2- 172.21.15.157:0/1266395636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d8806ade0 0x7f3d881674d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:55.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.900+0000 7f3d8dc72700 1 -- 172.21.15.157:0/1266395636 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d780094f0 con 0x7f3d880ad4c0 2023-10-06T04:10:55.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.900+0000 7f3d8dc72700 1 --2- 172.21.15.157:0/1266395636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d880ad4c0 0x7f3d880c72e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f3d780090b0 tx=0x7f3d78017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:55.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.900+0000 7f3d7e7fc700 1 -- 172.21.15.157:0/1266395636 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7800bd00 con 0x7f3d880ad4c0 2023-10-06T04:10:55.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.900+0000 7f3d7e7fc700 1 -- 172.21.15.157:0/1266395636 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7801f070 con 0x7f3d880ad4c0 2023-10-06T04:10:55.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.900+0000 7f3d906d7700 1 -- 172.21.15.157:0/1266395636 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d8806ece0 con 0x7f3d880ad4c0 2023-10-06T04:10:55.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.900+0000 7f3d7e7fc700 1 -- 172.21.15.157:0/1266395636 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d780266b0 con 0x7f3d880ad4c0 2023-10-06T04:10:55.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.900+0000 7f3d906d7700 1 -- 172.21.15.157:0/1266395636 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d8814ebd0 con 0x7f3d880ad4c0 2023-10-06T04:10:55.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.901+0000 7f3d906d7700 1 -- 172.21.15.157:0/1266395636 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d8814f3d0 con 0x7f3d880ad4c0 2023-10-06T04:10:55.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.902+0000 7f3d7e7fc700 1 -- 172.21.15.157:0/1266395636 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d78026a60 con 0x7f3d880ad4c0 2023-10-06T04:10:55.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.902+0000 7f3d7e7fc700 1 --2- 172.21.15.157:0/1266395636 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3d6c066f60 0x7f3d6c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:55.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.902+0000 7f3d7e7fc700 1 -- 172.21.15.157:0/1266395636 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f3d78021070 con 0x7f3d880ad4c0 2023-10-06T04:10:55.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.902+0000 7f3d8e473700 1 --2- 172.21.15.157:0/1266395636 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3d6c066f60 0x7f3d6c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:55.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.902+0000 7f3d8e473700 1 --2- 172.21.15.157:0/1266395636 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3d6c066f60 0x7f3d6c069410 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3d84007fd0 tx=0x7f3d84001630 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:55.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:55.906+0000 7f3d7e7fc700 1 -- 172.21.15.157:0/1266395636 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d8814f3d0 con 0x7f3d880ad4c0 2023-10-06T04:10:56.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.143+0000 7f3d906d7700 1 -- 172.21.15.157:0/1266395636 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d8814f3d0 con 0x7f3d880ad4c0 2023-10-06T04:10:56.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.144+0000 7f3d7e7fc700 1 -- 172.21.15.157:0/1266395636 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+17955 (secure 0 0 0) 0x7f3d8814f3d0 con 0x7f3d880ad4c0 2023-10-06T04:10:56.147 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:10:56.147 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":53,"fsid":"28664a7a-fe30-4efe-be5a-6abf9318b700","created":"2023-10-06T03:56:51.887538+0000","modified":"2023-10-06T04:09:46.762195+0000","last_up_change":"2023-10-06T03:57:02.299440+0000","last_in_change":"2023-10-06T03:56:55.141505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:57:03.455121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:57:03.905661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:57:08.377186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:57:09.709582+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":505,"snap_epoch":48,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f677cac-e9f4-4608-acac-718b072ce984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":90640},{"type":"v1","addr":"172.21.15.157:6809","nonce":90640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":90640},{"type":"v1","addr":"172.21.15.157:6811","nonce":90640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":90640},{"type":"v1","addr":"172.21.15.157:6815","nonce":90640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":90640},{"type":"v1","addr":"172.21.15.157:6813","nonce":90640}]},"public_addr":"172.21.15.157:6809/90640","cluster_addr":"172.21.15.157:6811/90640","heartbeat_back_addr":"172.21.15.157:6815/90640","heartbeat_front_addr":"172.21.15.157:6813/90640","state":["exists","up"]},{"osd":1,"uuid":"7d771892-bb91-4e9b-9610-051861c11247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":90654},{"type":"v1","addr":"172.21.15.157:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":90654},{"type":"v1","addr":"172.21.15.157:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":90654},{"type":"v1","addr":"172.21.15.157:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":90654},{"type":"v1","addr":"172.21.15.157:6805","nonce":90654}]},"public_addr":"172.21.15.157:6801/90654","cluster_addr":"172.21.15.157:6803/90654","heartbeat_back_addr":"172.21.15.157:6807/90654","heartbeat_front_addr":"172.21.15.157:6805/90654","state":["exists","up"]},{"osd":2,"uuid":"5c1f7d4e-d185-47cf-ab6c-453b4e3009c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":90657},{"type":"v1","addr":"172.21.15.157:6825","nonce":90657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":90657},{"type":"v1","addr":"172.21.15.157:6827","nonce":90657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":90657},{"type":"v1","addr":"172.21.15.157:6831","nonce":90657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":90657},{"type":"v1","addr":"172.21.15.157:6829","nonce":90657}]},"public_addr":"172.21.15.157:6825/90657","cluster_addr":"172.21.15.157:6827/90657","heartbeat_back_addr":"172.21.15.157:6831/90657","heartbeat_front_addr":"172.21.15.157:6829/90657","state":["exists","up"]},{"osd":3,"uuid":"f6a1e825-43c7-4afd-8daf-e51d45afc75e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":90641},{"type":"v1","addr":"172.21.15.157:6817","nonce":90641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":90641},{"type":"v1","addr":"172.21.15.157:6819","nonce":90641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":90641},{"type":"v1","addr":"172.21.15.157:6823","nonce":90641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":90641},{"type":"v1","addr":"172.21.15.157:6821","nonce":90641}]},"public_addr":"172.21.15.157:6817/90641","cluster_addr":"172.21.15.157:6819/90641","heartbeat_back_addr":"172.21.15.157:6823/90641","heartbeat_front_addr":"172.21.15.157:6821/90641","state":["exists","up"]},{"osd":4,"uuid":"8ad6e3d8-9a31-4f85-9410-417b7ac5da97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":91261},{"type":"v1","addr":"172.21.15.175:6803","nonce":91261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":91261},{"type":"v1","addr":"172.21.15.175:6806","nonce":91261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":91261},{"type":"v1","addr":"172.21.15.175:6814","nonce":91261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":91261},{"type":"v1","addr":"172.21.15.175:6810","nonce":91261}]},"public_addr":"172.21.15.175:6803/91261","cluster_addr":"172.21.15.175:6806/91261","heartbeat_back_addr":"172.21.15.175:6814/91261","heartbeat_front_addr":"172.21.15.175:6810/91261","state":["exists","up"]},{"osd":5,"uuid":"f68aca2a-f182-4c00-8e21-241d09c02ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":91262},{"type":"v1","addr":"172.21.15.175:6817","nonce":91262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":91262},{"type":"v1","addr":"172.21.15.175:6819","nonce":91262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":91262},{"type":"v1","addr":"172.21.15.175:6823","nonce":91262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":91262},{"type":"v1","addr":"172.21.15.175:6821","nonce":91262}]},"public_addr":"172.21.15.175:6817/91262","cluster_addr":"172.21.15.175:6819/91262","heartbeat_back_addr":"172.21.15.175:6823/91262","heartbeat_front_addr":"172.21.15.175:6821/91262","state":["exists","up"]},{"osd":6,"uuid":"d867cd89-3430-40ca-862f-d2bea3de9d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":91263},{"type":"v1","addr":"172.21.15.175:6801","nonce":91263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6805","nonce":91263},{"type":"v1","addr":"172.21.15.175:6807","nonce":91263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6813","nonce":91263},{"type":"v1","addr":"172.21.15.175:6815","nonce":91263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":91263},{"type":"v1","addr":"172.21.15.175:6811","nonce":91263}]},"public_addr":"172.21.15.175:6801/91263","cluster_addr":"172.21.15.175:6807/91263","heartbeat_back_addr":"172.21.15.175:6815/91263","heartbeat_front_addr":"172.21.15.175:6811/91263","state":["exists","up"]},{"osd":7,"uuid":"930cd464-79f1-47cb-b4ff-188ccde74fff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":91306},{"type":"v1","addr":"172.21.15.175:6825","nonce":91306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":91306},{"type":"v1","addr":"172.21.15.175:6827","nonce":91306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":91306},{"type":"v1","addr":"172.21.15.175:6831","nonce":91306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":91306},{"type":"v1","addr":"172.21.15.175:6829","nonce":91306}]},"public_addr":"172.21.15.175:6825/91306","cluster_addr":"172.21.15.175:6827/91306","heartbeat_back_addr":"172.21.15.175:6831/91306","heartbeat_front_addr":"172.21.15.175:6829/91306","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.290673+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.056390+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.292903+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.275861+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.332095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.327453+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.317585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.309330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.24","mappings":[{"from":1,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6834/765157057":"2023-10-07T04:09:46.762128+0000","172.21.15.175:6836/2388164729":"2023-10-07T04:07:31.736793+0000","172.21.15.157:6837/4188652695":"2023-10-07T04:05:26.713520+0000","172.21.15.175:6839/4177493830":"2023-10-07T03:59:06.641459+0000","172.21.15.175:6838/4177493830":"2023-10-07T03:59:06.641459+0000","172.21.15.157:6832/1179753185":"2023-10-07T04:04:26.701047+0000","172.21.15.157:6832/765157057":"2023-10-07T04:09:46.762128+0000","172.21.15.157:6833/3932336008":"2023-10-07T04:01:01.662287+0000","172.21.15.157:6837/3065749287":"2023-10-07T04:08:36.748456+0000","172.21.15.175:6837/2388164729":"2023-10-07T04:07:31.736793+0000","172.21.15.157:6836/4188652695":"2023-10-07T04:05:26.713520+0000","172.21.15.157:6835/3932336008":"2023-10-07T04:01:01.662287+0000","172.21.15.157:6836/3065749287":"2023-10-07T04:08:36.748456+0000","172.21.15.157:6834/1179753185":"2023-10-07T04:04:26.701047+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:10:56.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.146+0000 7f3d906d7700 1 -- 172.21.15.157:0/1266395636 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3d6c066f60 msgr2=0x7f3d6c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:56.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.146+0000 7f3d906d7700 1 --2- 172.21.15.157:0/1266395636 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3d6c066f60 0x7f3d6c069410 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3d84007fd0 tx=0x7f3d84001630 comp rx=0 tx=0).stop 2023-10-06T04:10:56.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.146+0000 7f3d906d7700 1 -- 172.21.15.157:0/1266395636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d880ad4c0 msgr2=0x7f3d880c72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:56.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.146+0000 7f3d906d7700 1 --2- 172.21.15.157:0/1266395636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d880ad4c0 0x7f3d880c72e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f3d780090b0 tx=0x7f3d78017000 comp rx=0 tx=0).stop 2023-10-06T04:10:56.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.146+0000 7f3d906d7700 1 -- 172.21.15.157:0/1266395636 shutdown_connections 2023-10-06T04:10:56.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.146+0000 7f3d906d7700 1 --2- 172.21.15.157:0/1266395636 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f3d6c066f60 0x7f3d6c069410 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:56.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.146+0000 7f3d906d7700 1 --2- 172.21.15.157:0/1266395636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d8806ade0 0x7f3d881674d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:56.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.146+0000 7f3d906d7700 1 --2- 172.21.15.157:0/1266395636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d880c4570 0x7f3d8814f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:56.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.146+0000 7f3d906d7700 1 --2- 172.21.15.157:0/1266395636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d880ad4c0 0x7f3d880c72e0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:56.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.146+0000 7f3d906d7700 1 -- 172.21.15.157:0/1266395636 >> 172.21.15.157:0/1266395636 conn(0x7f3d880c8d70 msgr2=0x7f3d880bd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:56.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.146+0000 7f3d906d7700 1 -- 172.21.15.157:0/1266395636 shutdown_connections 2023-10-06T04:10:56.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.146+0000 7f3d906d7700 1 -- 172.21.15.157:0/1266395636 wait complete. 2023-10-06T04:10:56.161 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T04:10:56.162 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:10:56.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.274+0000 7f2bc759e700 1 Processor -- start 2023-10-06T04:10:56.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.274+0000 7f2bc759e700 1 -- start start 2023-10-06T04:10:56.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.275+0000 7f2bc759e700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bc80b1e40 0x7f2bc8157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:56.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.275+0000 7f2bc759e700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2bc80b04d0 0x7f2bc806dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:56.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.275+0000 7f2bc759e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2bc806c2d0 0x7f2bc81563b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:56.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.275+0000 7f2bc759e700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bc806d8d0 con 0x7f2bc80b04d0 2023-10-06T04:10:56.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.275+0000 7f2bc759e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bc80af8f0 con 0x7f2bc806c2d0 2023-10-06T04:10:56.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.275+0000 7f2bc759e700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc806ccc0 con 0x7f2bc80b1e40 2023-10-06T04:10:56.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.275+0000 7f2bbdd9b700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2bc80b04d0 0x7f2bc806dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:56.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.275+0000 7f2bbdd9b700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2bc80b04d0 0x7f2bc806dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51538/0 (socket says 172.21.15.157:51538) 2023-10-06T04:10:56.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.275+0000 7f2bbdd9b700 1 -- 172.21.15.157:0/3789716602 learned_addr learned my addr 172.21.15.157:0/3789716602 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:10:56.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.275+0000 7f2bc6d9d700 1 --2- 172.21.15.157:0/3789716602 >> v2:172.21.15.175:3300/0 conn(0x7f2bc806c2d0 0x7f2bc81563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:56.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.275+0000 7f2bc659c700 1 --2- 172.21.15.157:0/3789716602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bc80b1e40 0x7f2bc8157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:56.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.275+0000 7f2bbdd9b700 1 -- 172.21.15.157:0/3789716602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bc80b1e40 msgr2=0x7f2bc8157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:10:56.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.275+0000 7f2bbdd9b700 1 --2- 172.21.15.157:0/3789716602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bc80b1e40 0x7f2bc8157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:56.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.275+0000 7f2bbdd9b700 1 -- 172.21.15.157:0/3789716602 >> v2:172.21.15.175:3300/0 conn(0x7f2bc806c2d0 msgr2=0x7f2bc81563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:10:56.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.275+0000 7f2bbdd9b700 1 --2- 172.21.15.157:0/3789716602 >> v2:172.21.15.175:3300/0 conn(0x7f2bc806c2d0 0x7f2bc81563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:56.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.275+0000 7f2bbdd9b700 1 -- 172.21.15.157:0/3789716602 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc806cfa0 con 0x7f2bc80b04d0 2023-10-06T04:10:56.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.275+0000 7f2bc659c700 1 --2- 172.21.15.157:0/3789716602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bc80b1e40 0x7f2bc8157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:10:56.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.275+0000 7f2bbdd9b700 1 --2- 172.21.15.157:0/3789716602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc80b04d0 0x7f2bc806dd60 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f2bc808a550 tx=0x7f2bb0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18d591847765ff6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:56.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.276+0000 7f2bc5d9b700 1 -- 172.21.15.157:0/3789716602 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb000b860 con 0x7f2bc80b04d0 2023-10-06T04:10:56.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.276+0000 7f2bc5d9b700 1 -- 172.21.15.157:0/3789716602 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb000b9c0 con 0x7f2bc80b04d0 2023-10-06T04:10:56.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.276+0000 7f2bc5d9b700 1 -- 172.21.15.157:0/3789716602 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb000bd80 con 0x7f2bc80b04d0 2023-10-06T04:10:56.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.276+0000 7f2bc759e700 1 -- 172.21.15.157:0/3789716602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc80b04d0 msgr2=0x7f2bc806dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:56.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.276+0000 7f2bc759e700 1 --2- 172.21.15.157:0/3789716602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc80b04d0 0x7f2bc806dd60 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f2bc808a550 tx=0x7f2bb0009250 comp rx=0 tx=0).stop 2023-10-06T04:10:56.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.276+0000 7f2bc759e700 1 -- 172.21.15.157:0/3789716602 shutdown_connections 2023-10-06T04:10:56.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.276+0000 7f2bc759e700 1 --2- 172.21.15.157:0/3789716602 >> v2:172.21.15.175:3300/0 conn(0x7f2bc806c2d0 0x7f2bc81563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:56.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.276+0000 7f2bc759e700 1 --2- 172.21.15.157:0/3789716602 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bc80b1e40 0x7f2bc8157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:56.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.276+0000 7f2bc759e700 1 --2- 172.21.15.157:0/3789716602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc80b04d0 0x7f2bc806dd60 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:56.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.276+0000 7f2bc759e700 1 -- 172.21.15.157:0/3789716602 >> 172.21.15.157:0/3789716602 conn(0x7f2bc80cc540 msgr2=0x7f2bc80cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:56.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.276+0000 7f2bc759e700 1 -- 172.21.15.157:0/3789716602 shutdown_connections 2023-10-06T04:10:56.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.276+0000 7f2bc759e700 1 -- 172.21.15.157:0/3789716602 wait complete. 2023-10-06T04:10:56.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.277+0000 7f2bc759e700 1 Processor -- start 2023-10-06T04:10:56.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.277+0000 7f2bc759e700 1 -- start start 2023-10-06T04:10:56.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.278+0000 7f2bc759e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc806c2d0 0x7f2bc81668b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:56.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.278+0000 7f2bc759e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bc80b04d0 0x7f2bc81540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:56.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.278+0000 7f2bc759e700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bc80b1e40 0x7f2bc814d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:56.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.278+0000 7f2bc759e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc8165ea0 con 0x7f2bc806c2d0 2023-10-06T04:10:56.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.278+0000 7f2bc759e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc8166420 con 0x7f2bc80b04d0 2023-10-06T04:10:56.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.278+0000 7f2bc759e700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc8166df0 con 0x7f2bc80b1e40 2023-10-06T04:10:56.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.278+0000 7f2bc659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc806c2d0 0x7f2bc81668b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:56.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.278+0000 7f2bc659c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc806c2d0 0x7f2bc81668b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51554/0 (socket says 172.21.15.157:51554) 2023-10-06T04:10:56.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.278+0000 7f2bbdd9b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bc80b04d0 0x7f2bc81540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:56.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.278+0000 7f2bc659c700 1 -- 172.21.15.157:0/1155820957 learned_addr learned my addr 172.21.15.157:0/1155820957 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:10:56.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.278+0000 7f2bc6d9d700 1 --2- 172.21.15.157:0/1155820957 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bc80b1e40 0x7f2bc814d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:56.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.279+0000 7f2bc659c700 1 -- 172.21.15.157:0/1155820957 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bc80b1e40 msgr2=0x7f2bc814d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:56.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.279+0000 7f2bc659c700 1 --2- 172.21.15.157:0/1155820957 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bc80b1e40 0x7f2bc814d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:56.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.279+0000 7f2bc659c700 1 -- 172.21.15.157:0/1155820957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bc80b04d0 msgr2=0x7f2bc81540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:56.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.279+0000 7f2bc659c700 1 --2- 172.21.15.157:0/1155820957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bc80b04d0 0x7f2bc81540b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:56.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.279+0000 7f2bc659c700 1 -- 172.21.15.157:0/1155820957 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb00094f0 con 0x7f2bc806c2d0 2023-10-06T04:10:56.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.279+0000 7f2bc659c700 1 --2- 172.21.15.157:0/1155820957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc806c2d0 0x7f2bc81668b0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800c860 tx=0x7f2bb8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:56.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.279+0000 7f2bbeffd700 1 -- 172.21.15.157:0/1155820957 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8011600 con 0x7f2bc806c2d0 2023-10-06T04:10:56.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.279+0000 7f2bbeffd700 1 -- 172.21.15.157:0/1155820957 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb8011760 con 0x7f2bc806c2d0 2023-10-06T04:10:56.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.279+0000 7f2bc759e700 1 -- 172.21.15.157:0/1155820957 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc815c0b0 con 0x7f2bc806c2d0 2023-10-06T04:10:56.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.279+0000 7f2bbeffd700 1 -- 172.21.15.157:0/1155820957 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8011b20 con 0x7f2bc806c2d0 2023-10-06T04:10:56.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.279+0000 7f2bc759e700 1 -- 172.21.15.157:0/1155820957 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc815be10 con 0x7f2bc806c2d0 2023-10-06T04:10:56.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.280+0000 7f2bc759e700 1 -- 172.21.15.157:0/1155820957 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc8166700 con 0x7f2bc806c2d0 2023-10-06T04:10:56.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.280+0000 7f2bbeffd700 1 -- 172.21.15.157:0/1155820957 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2bb801a400 con 0x7f2bc806c2d0 2023-10-06T04:10:56.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.281+0000 7f2bbeffd700 1 --2- 172.21.15.157:0/1155820957 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2ba8066f60 0x7f2ba8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:56.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.281+0000 7f2bbeffd700 1 -- 172.21.15.157:0/1155820957 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f2bb8084c00 con 0x7f2bc806c2d0 2023-10-06T04:10:56.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.281+0000 7f2bbdd9b700 1 --2- 172.21.15.157:0/1155820957 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2ba8066f60 0x7f2ba8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:56.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.281+0000 7f2bbdd9b700 1 --2- 172.21.15.157:0/1155820957 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2ba8066f60 0x7f2ba8069410 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2bc808a550 tx=0x7f2bb0017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:56.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.287+0000 7f2bbeffd700 1 -- 172.21.15.157:0/1155820957 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2bb808b080 con 0x7f2bc806c2d0 2023-10-06T04:10:56.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.474+0000 7f2bc759e700 1 -- 172.21.15.157:0/1155820957 --> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2bc8088f90 con 0x7f2ba8066f60 2023-10-06T04:10:56.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.484+0000 7f2bbeffd700 1 -- 172.21.15.157:0/1155820957 <== mgr.4107 v2:172.21.15.175:6832/90661 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+316198 (secure 0 0 0) 0x7f2bc8088f90 con 0x7f2ba8066f60 2023-10-06T04:10:56.490 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:10:56.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.486+0000 7f2bc759e700 1 -- 172.21.15.157:0/1155820957 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2ba8066f60 msgr2=0x7f2ba8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:56.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.486+0000 7f2bc759e700 1 --2- 172.21.15.157:0/1155820957 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2ba8066f60 0x7f2ba8069410 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2bc808a550 tx=0x7f2bb0017000 comp rx=0 tx=0).stop 2023-10-06T04:10:56.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.486+0000 7f2bc759e700 1 -- 172.21.15.157:0/1155820957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc806c2d0 msgr2=0x7f2bc81668b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:56.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.486+0000 7f2bc759e700 1 --2- 172.21.15.157:0/1155820957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc806c2d0 0x7f2bc81668b0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800c860 tx=0x7f2bb8008040 comp rx=0 tx=0).stop 2023-10-06T04:10:56.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.487+0000 7f2bc759e700 1 -- 172.21.15.157:0/1155820957 shutdown_connections 2023-10-06T04:10:56.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.487+0000 7f2bc759e700 1 --2- 172.21.15.157:0/1155820957 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f2ba8066f60 0x7f2ba8069410 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:56.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.487+0000 7f2bc759e700 1 --2- 172.21.15.157:0/1155820957 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bc80b1e40 0x7f2bc814d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:56.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.487+0000 7f2bc759e700 1 --2- 172.21.15.157:0/1155820957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc806c2d0 0x7f2bc81668b0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:56.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.487+0000 7f2bc759e700 1 --2- 172.21.15.157:0/1155820957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bc80b04d0 0x7f2bc81540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:56.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.487+0000 7f2bc759e700 1 -- 172.21.15.157:0/1155820957 >> 172.21.15.157:0/1155820957 conn(0x7f2bc80cc540 msgr2=0x7f2bc8158470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:56.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.487+0000 7f2bc759e700 1 -- 172.21.15.157:0/1155820957 shutdown_connections 2023-10-06T04:10:56.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.487+0000 7f2bc759e700 1 -- 172.21.15.157:0/1155820957 wait complete. 2023-10-06T04:10:56.495 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-10-06T04:10:56.501 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2023-10-06T04:10:55.645520+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1079653394,"num_objects":10047,"num_object_clones":0,"num_object_copies":20094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10047,"num_whiteouts":0,"num_read":2914,"num_read_kb":634649,"num_write":27342,"num_write_kb":1234647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":122,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27621,"ondisk_log_size":27621,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":4139684,"kb_used_data":1720420,"kb_used_omap":0,"kb_used_meta":2419200,"kb_avail":750835036,"statfs":{"total":773094113280,"available":768855076864,"internally_reserved":0,"allocated":1761710080,"data_stored":2160230755,"data_compressed":200974452,"data_compressed_allocated":228696064,"data_compressed_original":681320448,"omap_allocated":0,"internal_metadata":2477260800},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":170,"apply_latency_ms":170,"commit_latency_ns":170000000,"apply_latency_ns":170000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":117952278,"num_objects":3304,"num_object_clones":0,"num_object_copies":6608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3419,"num_write_kb":116453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004532"},"pg_stats":[{"pgid":"4.3f","version":"53'315","reported_seq":2411,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.016174+0000","last_change":"2023-10-06T04:05:03.151059+0000","last_active":"2023-10-06T04:10:48.016174+0000","last_peered":"2023-10-06T04:10:48.016174+0000","last_clean":"2023-10-06T04:10:48.016174+0000","last_became_active":"2023-10-06T03:57:11.371756+0000","last_became_peered":"2023-10-06T03:57:11.371756+0000","last_unstale":"2023-10-06T04:10:48.016174+0000","last_undegraded":"2023-10-06T04:10:48.016174+0000","last_fullsized":"2023-10-06T04:10:48.016174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:50.086528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001094783,"stat_sum":{"num_bytes":10567600,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":29,"num_read_kb":15,"num_write":301,"num_write_kb":10442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"53'337","reported_seq":2437,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.046233+0000","last_change":"2023-10-06T04:05:02.973547+0000","last_active":"2023-10-06T04:10:48.046233+0000","last_peered":"2023-10-06T04:10:48.046233+0000","last_clean":"2023-10-06T04:10:48.046233+0000","last_became_active":"2023-10-06T03:57:11.370070+0000","last_became_peered":"2023-10-06T03:57:11.370070+0000","last_unstale":"2023-10-06T04:10:48.046233+0000","last_undegraded":"2023-10-06T04:10:48.046233+0000","last_fullsized":"2023-10-06T04:10:48.046233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:00.917607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021773819999999998,"stat_sum":{"num_bytes":6441631,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":33,"num_read_kb":16,"num_write":327,"num_write_kb":6463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"53'333","reported_seq":2437,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.050065+0000","last_change":"2023-10-06T04:05:02.723796+0000","last_active":"2023-10-06T04:10:48.050065+0000","last_peered":"2023-10-06T04:10:48.050065+0000","last_clean":"2023-10-06T04:10:48.050065+0000","last_became_active":"2023-10-06T03:57:11.366012+0000","last_became_peered":"2023-10-06T03:57:11.366012+0000","last_unstale":"2023-10-06T04:10:48.050065+0000","last_undegraded":"2023-10-06T04:10:48.050065+0000","last_fullsized":"2023-10-06T04:10:48.050065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:45.802287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002722286,"stat_sum":{"num_bytes":14390705,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":40,"num_read_kb":20499,"num_write":313,"num_write_kb":22357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"53'294","reported_seq":2393,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.819279+0000","last_change":"2023-10-06T04:05:03.543119+0000","last_active":"2023-10-06T04:10:47.819279+0000","last_peered":"2023-10-06T04:10:47.819279+0000","last_clean":"2023-10-06T04:10:47.819279+0000","last_became_active":"2023-10-06T03:57:11.365673+0000","last_became_peered":"2023-10-06T03:57:11.365673+0000","last_unstale":"2023-10-06T04:10:47.819279+0000","last_undegraded":"2023-10-06T04:10:47.819279+0000","last_fullsized":"2023-10-06T04:10:47.819279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:40.953382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001947182,"stat_sum":{"num_bytes":14739251,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":32,"num_read_kb":7963,"num_write":286,"num_write_kb":14550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"53'323","reported_seq":2421,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.102698+0000","last_change":"2023-10-06T04:05:03.366556+0000","last_active":"2023-10-06T04:10:48.102698+0000","last_peered":"2023-10-06T04:10:48.102698+0000","last_clean":"2023-10-06T04:10:48.102698+0000","last_became_active":"2023-10-06T03:57:11.361302+0000","last_became_peered":"2023-10-06T03:57:11.361302+0000","last_unstale":"2023-10-06T04:10:48.102698+0000","last_undegraded":"2023-10-06T04:10:48.102698+0000","last_fullsized":"2023-10-06T04:10:48.102698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:57.552493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001966644,"stat_sum":{"num_bytes":2126805,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":31,"num_read_kb":18,"num_write":316,"num_write_kb":2215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"53'311","reported_seq":2412,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.048068+0000","last_change":"2023-10-06T04:05:02.723270+0000","last_active":"2023-10-06T04:10:48.048068+0000","last_peered":"2023-10-06T04:10:48.048068+0000","last_clean":"2023-10-06T04:10:48.048068+0000","last_became_active":"2023-10-06T03:57:11.368775+0000","last_became_peered":"2023-10-06T03:57:11.368775+0000","last_unstale":"2023-10-06T04:10:48.048068+0000","last_undegraded":"2023-10-06T04:10:48.048068+0000","last_fullsized":"2023-10-06T04:10:48.048068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:22.368117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002640482,"stat_sum":{"num_bytes":22420575,"num_objects":141,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":141,"num_whiteouts":0,"num_read":34,"num_read_kb":13627,"num_write":301,"num_write_kb":22056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"53'368","reported_seq":2467,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.038708+0000","last_change":"2023-10-06T04:05:03.343818+0000","last_active":"2023-10-06T04:10:48.038708+0000","last_peered":"2023-10-06T04:10:48.038708+0000","last_clean":"2023-10-06T04:10:48.038708+0000","last_became_active":"2023-10-06T03:57:11.377786+0000","last_became_peered":"2023-10-06T03:57:11.377786+0000","last_unstale":"2023-10-06T04:10:48.038708+0000","last_undegraded":"2023-10-06T04:10:48.038708+0000","last_fullsized":"2023-10-06T04:10:48.038708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:12:54.218593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093960500000000004,"stat_sum":{"num_bytes":18362974,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":32,"num_read_kb":8208,"num_write":354,"num_write_kb":18122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"53'303","reported_seq":2401,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.925102+0000","last_change":"2023-10-06T04:05:02.972640+0000","last_active":"2023-10-06T04:10:47.925102+0000","last_peered":"2023-10-06T04:10:47.925102+0000","last_clean":"2023-10-06T04:10:47.925102+0000","last_became_active":"2023-10-06T03:57:11.373765+0000","last_became_peered":"2023-10-06T03:57:11.373765+0000","last_unstale":"2023-10-06T04:10:47.925102+0000","last_undegraded":"2023-10-06T04:10:47.925102+0000","last_fullsized":"2023-10-06T04:10:47.925102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:05.838007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001873881,"stat_sum":{"num_bytes":10907247,"num_objects":139,"num_object_clones":0,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":0,"num_read":31,"num_read_kb":4107,"num_write":297,"num_write_kb":10730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"53'308","reported_seq":2415,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.091138+0000","last_change":"2023-10-06T04:05:03.367332+0000","last_active":"2023-10-06T04:10:48.091138+0000","last_peered":"2023-10-06T04:10:48.091138+0000","last_clean":"2023-10-06T04:10:48.091138+0000","last_became_active":"2023-10-06T03:57:11.368004+0000","last_became_peered":"2023-10-06T03:57:11.368004+0000","last_unstale":"2023-10-06T04:10:48.091138+0000","last_undegraded":"2023-10-06T04:10:48.091138+0000","last_fullsized":"2023-10-06T04:10:48.091138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:16.760049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024398710000000001,"stat_sum":{"num_bytes":13562133,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":41,"num_read_kb":4114,"num_write":303,"num_write_kb":17498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"53'324","reported_seq":2421,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.936467+0000","last_change":"2023-10-06T04:05:02.973579+0000","last_active":"2023-10-06T04:10:47.936467+0000","last_peered":"2023-10-06T04:10:47.936467+0000","last_clean":"2023-10-06T04:10:47.936467+0000","last_became_active":"2023-10-06T03:57:11.379107+0000","last_became_peered":"2023-10-06T03:57:11.379107+0000","last_unstale":"2023-10-06T04:10:47.936467+0000","last_undegraded":"2023-10-06T04:10:47.936467+0000","last_fullsized":"2023-10-06T04:10:47.936467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:18.373103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002039126,"stat_sum":{"num_bytes":9755625,"num_objects":142,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":30,"num_read_kb":4112,"num_write":320,"num_write_kb":9702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"53'355","reported_seq":2461,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.053956+0000","last_change":"2023-10-06T04:05:02.724475+0000","last_active":"2023-10-06T04:10:48.053956+0000","last_peered":"2023-10-06T04:10:48.053956+0000","last_clean":"2023-10-06T04:10:48.053956+0000","last_became_active":"2023-10-06T03:57:11.368256+0000","last_became_peered":"2023-10-06T03:57:11.368256+0000","last_unstale":"2023-10-06T04:10:48.053956+0000","last_undegraded":"2023-10-06T04:10:48.053956+0000","last_fullsized":"2023-10-06T04:10:48.053956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:50.736367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029138929999999999,"stat_sum":{"num_bytes":31118399,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":39,"num_read_kb":4114,"num_write":348,"num_write_kb":30570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"53'305","reported_seq":2402,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.932349+0000","last_change":"2023-10-06T04:05:03.151565+0000","last_active":"2023-10-06T04:10:47.932349+0000","last_peered":"2023-10-06T04:10:47.932349+0000","last_clean":"2023-10-06T04:10:47.932349+0000","last_became_active":"2023-10-06T03:57:11.368425+0000","last_became_peered":"2023-10-06T03:57:11.368425+0000","last_unstale":"2023-10-06T04:10:47.932349+0000","last_undegraded":"2023-10-06T04:10:47.932349+0000","last_fullsized":"2023-10-06T04:10:47.932349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:06.004423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014231059999999999,"stat_sum":{"num_bytes":10394744,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":30,"num_read_kb":4105,"num_write":299,"num_write_kb":14405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"53'322","reported_seq":2425,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.059723+0000","last_change":"2023-10-06T04:05:03.150960+0000","last_active":"2023-10-06T04:10:48.059723+0000","last_peered":"2023-10-06T04:10:48.059723+0000","last_clean":"2023-10-06T04:10:48.059723+0000","last_became_active":"2023-10-06T03:57:11.367079+0000","last_became_peered":"2023-10-06T03:57:11.367079+0000","last_unstale":"2023-10-06T04:10:48.059723+0000","last_undegraded":"2023-10-06T04:10:48.059723+0000","last_fullsized":"2023-10-06T04:10:48.059723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:09:54.331206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001042765,"stat_sum":{"num_bytes":14339914,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":37,"num_read_kb":12308,"num_write":311,"num_write_kb":22299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"53'317","reported_seq":2420,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.803651+0000","last_change":"2023-10-06T04:05:02.973393+0000","last_active":"2023-10-06T04:10:47.803651+0000","last_peered":"2023-10-06T04:10:47.803651+0000","last_clean":"2023-10-06T04:10:47.803651+0000","last_became_active":"2023-10-06T03:57:11.373216+0000","last_became_peered":"2023-10-06T03:57:11.373216+0000","last_unstale":"2023-10-06T04:10:47.803651+0000","last_undegraded":"2023-10-06T04:10:47.803651+0000","last_fullsized":"2023-10-06T04:10:47.803651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:56.083203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022456289999999999,"stat_sum":{"num_bytes":5695017,"num_objects":137,"num_object_clones":0,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":0,"num_read":36,"num_read_kb":18,"num_write":313,"num_write_kb":9820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"53'324","reported_seq":2421,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.038672+0000","last_change":"2023-10-06T04:05:03.343999+0000","last_active":"2023-10-06T04:10:48.038672+0000","last_peered":"2023-10-06T04:10:48.038672+0000","last_clean":"2023-10-06T04:10:48.038672+0000","last_became_active":"2023-10-06T03:57:11.375689+0000","last_became_peered":"2023-10-06T03:57:11.375689+0000","last_unstale":"2023-10-06T04:10:48.038672+0000","last_undegraded":"2023-10-06T04:10:48.038672+0000","last_fullsized":"2023-10-06T04:10:48.038672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:42.953739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092625599999999998,"stat_sum":{"num_bytes":15101939,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":316,"num_write_kb":14917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"53'331","reported_seq":2433,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.045818+0000","last_change":"2023-10-06T04:05:02.722778+0000","last_active":"2023-10-06T04:10:48.045818+0000","last_peered":"2023-10-06T04:10:48.045818+0000","last_clean":"2023-10-06T04:10:48.045818+0000","last_became_active":"2023-10-06T03:57:11.366543+0000","last_became_peered":"2023-10-06T03:57:11.366543+0000","last_unstale":"2023-10-06T04:10:48.045818+0000","last_undegraded":"2023-10-06T04:10:48.045818+0000","last_fullsized":"2023-10-06T04:10:48.045818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:16.212967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024276609999999998,"stat_sum":{"num_bytes":19805304,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":35,"num_read_kb":19,"num_write":327,"num_write_kb":22717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"53'342","reported_seq":2442,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.091750+0000","last_change":"2023-10-06T04:05:03.367754+0000","last_active":"2023-10-06T04:10:48.091750+0000","last_peered":"2023-10-06T04:10:48.091750+0000","last_clean":"2023-10-06T04:10:48.091750+0000","last_became_active":"2023-10-06T03:57:11.366767+0000","last_became_peered":"2023-10-06T03:57:11.366767+0000","last_unstale":"2023-10-06T04:10:48.091750+0000","last_undegraded":"2023-10-06T04:10:48.091750+0000","last_fullsized":"2023-10-06T04:10:48.091750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:43.113575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002777833,"stat_sum":{"num_bytes":14662095,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":34,"num_read_kb":8213,"num_write":330,"num_write_kb":18582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"53'343","reported_seq":2453,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.060004+0000","last_change":"2023-10-06T04:05:02.723910+0000","last_active":"2023-10-06T04:10:48.060004+0000","last_peered":"2023-10-06T04:10:48.060004+0000","last_clean":"2023-10-06T04:10:48.060004+0000","last_became_active":"2023-10-06T03:57:11.362377+0000","last_became_peered":"2023-10-06T03:57:11.362377+0000","last_unstale":"2023-10-06T04:10:48.060004+0000","last_undegraded":"2023-10-06T04:10:48.060004+0000","last_fullsized":"2023-10-06T04:10:48.060004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:40.600383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027606900000000001,"stat_sum":{"num_bytes":10867734,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":43,"num_read_kb":26,"num_write":329,"num_write_kb":10783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"53'340","reported_seq":2446,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.826978+0000","last_change":"2023-10-06T04:05:03.151735+0000","last_active":"2023-10-06T04:10:47.826978+0000","last_peered":"2023-10-06T04:10:47.826978+0000","last_clean":"2023-10-06T04:10:47.826978+0000","last_became_active":"2023-10-06T03:57:11.363792+0000","last_became_peered":"2023-10-06T03:57:11.363792+0000","last_unstale":"2023-10-06T04:10:47.826978+0000","last_undegraded":"2023-10-06T04:10:47.826978+0000","last_fullsized":"2023-10-06T04:10:47.826978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:49.185450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014611940000000001,"stat_sum":{"num_bytes":14638716,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":40,"num_read_kb":8209,"num_write":327,"num_write_kb":18563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"53'345","reported_seq":2448,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.037717+0000","last_change":"2023-10-06T04:05:02.724524+0000","last_active":"2023-10-06T04:10:48.037717+0000","last_peered":"2023-10-06T04:10:48.037717+0000","last_clean":"2023-10-06T04:10:48.037717+0000","last_became_active":"2023-10-06T03:57:11.364726+0000","last_became_peered":"2023-10-06T03:57:11.364726+0000","last_unstale":"2023-10-06T04:10:48.037717+0000","last_undegraded":"2023-10-06T04:10:48.037717+0000","last_fullsized":"2023-10-06T04:10:48.037717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:56.866733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029763670000000002,"stat_sum":{"num_bytes":20613561,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":37,"num_read_kb":12308,"num_write":341,"num_write_kb":28448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"53'320","reported_seq":2429,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.103111+0000","last_change":"2023-10-06T04:05:03.344002+0000","last_active":"2023-10-06T04:10:48.103111+0000","last_peered":"2023-10-06T04:10:48.103111+0000","last_clean":"2023-10-06T04:10:48.103111+0000","last_became_active":"2023-10-06T03:57:11.375668+0000","last_became_peered":"2023-10-06T03:57:11.375668+0000","last_unstale":"2023-10-06T04:10:48.103111+0000","last_undegraded":"2023-10-06T04:10:48.103111+0000","last_fullsized":"2023-10-06T04:10:48.103111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:53.810875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010285800000000001,"stat_sum":{"num_bytes":18956719,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":42,"num_read_kb":8215,"num_write":314,"num_write_kb":18647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"53'337","reported_seq":2438,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.891519+0000","last_change":"2023-10-06T04:05:03.366839+0000","last_active":"2023-10-06T04:10:47.891519+0000","last_peered":"2023-10-06T04:10:47.891519+0000","last_clean":"2023-10-06T04:10:47.891519+0000","last_became_active":"2023-10-06T03:57:11.365149+0000","last_became_peered":"2023-10-06T03:57:11.365149+0000","last_unstale":"2023-10-06T04:10:47.891519+0000","last_undegraded":"2023-10-06T04:10:47.891519+0000","last_fullsized":"2023-10-06T04:10:47.891519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:20.911710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020974879999999998,"stat_sum":{"num_bytes":5856786,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":34,"num_read_kb":17,"num_write":329,"num_write_kb":9988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"53'337","reported_seq":2445,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.097397+0000","last_change":"2023-10-06T04:05:02.722763+0000","last_active":"2023-10-06T04:10:48.097397+0000","last_peered":"2023-10-06T04:10:48.097397+0000","last_clean":"2023-10-06T04:10:48.097397+0000","last_became_active":"2023-10-06T03:57:11.362128+0000","last_became_peered":"2023-10-06T03:57:11.362128+0000","last_unstale":"2023-10-06T04:10:48.097397+0000","last_undegraded":"2023-10-06T04:10:48.097397+0000","last_fullsized":"2023-10-06T04:10:48.097397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:53.744674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025174860000000002,"stat_sum":{"num_bytes":9975853,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":42,"num_read_kb":8214,"num_write":313,"num_write_kb":13997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"53'363","reported_seq":2477,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.777222+0000","last_change":"2023-10-06T04:05:03.344184+0000","last_active":"2023-10-06T04:10:47.777222+0000","last_peered":"2023-10-06T04:10:47.777222+0000","last_clean":"2023-10-06T04:10:47.777222+0000","last_became_active":"2023-10-06T03:57:11.375702+0000","last_became_peered":"2023-10-06T03:57:11.375702+0000","last_unstale":"2023-10-06T04:10:47.777222+0000","last_undegraded":"2023-10-06T04:10:47.777222+0000","last_fullsized":"2023-10-06T04:10:47.777222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:31.204231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010337009999999999,"stat_sum":{"num_bytes":19963417,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":47,"num_read_kb":12104,"num_write":344,"num_write_kb":19673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"53'313","reported_seq":2409,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.117818+0000","last_change":"2023-10-06T04:05:02.718513+0000","last_active":"2023-10-06T04:10:48.117818+0000","last_peered":"2023-10-06T04:10:48.117818+0000","last_clean":"2023-10-06T04:10:48.117818+0000","last_became_active":"2023-10-06T03:57:11.374842+0000","last_became_peered":"2023-10-06T03:57:11.374842+0000","last_unstale":"2023-10-06T04:10:48.117818+0000","last_undegraded":"2023-10-06T04:10:48.117818+0000","last_fullsized":"2023-10-06T04:10:48.117818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:54.438974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001332342,"stat_sum":{"num_bytes":5720212,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":29,"num_read_kb":4115,"num_write":303,"num_write_kb":5752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"53'363","reported_seq":2463,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.116692+0000","last_change":"2023-10-06T04:05:02.718726+0000","last_active":"2023-10-06T04:10:48.116692+0000","last_peered":"2023-10-06T04:10:48.116692+0000","last_clean":"2023-10-06T04:10:48.116692+0000","last_became_active":"2023-10-06T03:57:11.361349+0000","last_became_peered":"2023-10-06T03:57:11.361349+0000","last_unstale":"2023-10-06T04:10:48.116692+0000","last_undegraded":"2023-10-06T04:10:48.116692+0000","last_fullsized":"2023-10-06T04:10:48.116692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:30.646781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014394589999999999,"stat_sum":{"num_bytes":22991444,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":33,"num_read_kb":4115,"num_write":351,"num_write_kb":22637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"53'300","reported_seq":2405,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.871145+0000","last_change":"2023-10-06T04:05:02.972918+0000","last_active":"2023-10-06T04:10:47.871145+0000","last_peered":"2023-10-06T04:10:47.871145+0000","last_clean":"2023-10-06T04:10:47.871145+0000","last_became_active":"2023-10-06T03:57:11.375846+0000","last_became_peered":"2023-10-06T03:57:11.375846+0000","last_unstale":"2023-10-06T04:10:47.871145+0000","last_undegraded":"2023-10-06T04:10:47.871145+0000","last_fullsized":"2023-10-06T04:10:47.871145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:28.080614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018480599999999999,"stat_sum":{"num_bytes":26260340,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":38,"num_read_kb":12302,"num_write":293,"num_write_kb":25795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"53'362","reported_seq":417,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:45.966579+0000","last_change":"2023-10-06T03:57:10.348317+0000","last_active":"2023-10-06T04:10:45.966579+0000","last_peered":"2023-10-06T04:10:45.966579+0000","last_clean":"2023-10-06T04:10:45.966579+0000","last_became_active":"2023-10-06T03:57:10.348078+0000","last_became_peered":"2023-10-06T03:57:10.348078+0000","last_unstale":"2023-10-06T04:10:45.966579+0000","last_undegraded":"2023-10-06T04:10:45.966579+0000","last_fullsized":"2023-10-06T04:10:45.966579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:55.058324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12621090,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":13,"num_read_kb":12332,"num_write":358,"num_write_kb":12535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"53'317","reported_seq":2422,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.966557+0000","last_change":"2023-10-06T04:05:02.722997+0000","last_active":"2023-10-06T04:10:47.966557+0000","last_peered":"2023-10-06T04:10:47.966557+0000","last_clean":"2023-10-06T04:10:47.966557+0000","last_became_active":"2023-10-06T03:57:11.370245+0000","last_became_peered":"2023-10-06T03:57:11.370245+0000","last_unstale":"2023-10-06T04:10:47.966557+0000","last_undegraded":"2023-10-06T04:10:47.966557+0000","last_fullsized":"2023-10-06T04:10:47.966557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:47:18.189772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002348424,"stat_sum":{"num_bytes":22206576,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":38,"num_read_kb":9495,"num_write":309,"num_write_kb":25946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"53'89","reported_seq":168,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:42.933757+0000","last_change":"2023-10-06T03:57:10.345401+0000","last_active":"2023-10-06T04:10:42.933757+0000","last_peered":"2023-10-06T04:10:42.933757+0000","last_clean":"2023-10-06T04:10:42.933757+0000","last_became_active":"2023-10-06T03:57:10.345150+0000","last_became_peered":"2023-10-06T03:57:10.345150+0000","last_unstale":"2023-10-06T04:10:42.933757+0000","last_undegraded":"2023-10-06T04:10:42.933757+0000","last_fullsized":"2023-10-06T04:10:42.933757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:25.758193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4312676,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":146,"num_write":89,"num_write_kb":4275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"53'333","reported_seq":2426,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.998937+0000","last_change":"2023-10-06T04:05:03.367558+0000","last_active":"2023-10-06T04:10:47.998937+0000","last_peered":"2023-10-06T04:10:47.998937+0000","last_clean":"2023-10-06T04:10:47.998937+0000","last_became_active":"2023-10-06T03:57:11.375790+0000","last_became_peered":"2023-10-06T03:57:11.375790+0000","last_unstale":"2023-10-06T04:10:47.998937+0000","last_undegraded":"2023-10-06T04:10:47.998937+0000","last_fullsized":"2023-10-06T04:10:47.998937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:33.775431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025999780000000002,"stat_sum":{"num_bytes":19057467,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":26,"num_read_kb":12300,"num_write":323,"num_write_kb":18781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"53'1073","reported_seq":1122,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:52.526921+0000","last_change":"2023-10-06T03:57:10.347562+0000","last_active":"2023-10-06T04:09:52.526921+0000","last_peered":"2023-10-06T04:09:52.526921+0000","last_clean":"2023-10-06T04:09:52.526921+0000","last_became_active":"2023-10-06T03:57:10.347294+0000","last_became_peered":"2023-10-06T03:57:10.347294+0000","last_unstale":"2023-10-06T04:09:52.526921+0000","last_undegraded":"2023-10-06T04:09:52.526921+0000","last_fullsized":"2023-10-06T04:09:52.526921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":1073,"log_dups_size":0,"ondisk_log_size":1073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:25.090992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583466,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":7,"num_read_kb":16784,"num_write":1067,"num_write_kb":17022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"53'349","reported_seq":2451,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.789556+0000","last_change":"2023-10-06T04:05:03.543192+0000","last_active":"2023-10-06T04:10:47.789556+0000","last_peered":"2023-10-06T04:10:47.789556+0000","last_clean":"2023-10-06T04:10:47.789556+0000","last_became_active":"2023-10-06T03:57:11.371758+0000","last_became_peered":"2023-10-06T03:57:11.371758+0000","last_unstale":"2023-10-06T04:10:47.789556+0000","last_undegraded":"2023-10-06T04:10:47.789556+0000","last_fullsized":"2023-10-06T04:10:47.789556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:10.922157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017888349999999999,"stat_sum":{"num_bytes":16828923,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":35,"num_read_kb":4115,"num_write":344,"num_write_kb":16563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"53'47","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:36.261131+0000","last_change":"2023-10-06T03:57:10.347406+0000","last_active":"2023-10-06T04:10:36.261131+0000","last_peered":"2023-10-06T04:10:36.261131+0000","last_clean":"2023-10-06T04:10:36.261131+0000","last_became_active":"2023-10-06T03:57:10.347151+0000","last_became_peered":"2023-10-06T03:57:10.347151+0000","last_unstale":"2023-10-06T04:10:36.261131+0000","last_undegraded":"2023-10-06T04:10:36.261131+0000","last_fullsized":"2023-10-06T04:10:36.261131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:20.427620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":50,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"53'312","reported_seq":2415,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.139381+0000","last_change":"2023-10-06T04:05:02.721771+0000","last_active":"2023-10-06T04:10:48.139381+0000","last_peered":"2023-10-06T04:10:48.139381+0000","last_clean":"2023-10-06T04:10:48.139381+0000","last_became_active":"2023-10-06T03:57:11.369938+0000","last_became_peered":"2023-10-06T03:57:11.369938+0000","last_unstale":"2023-10-06T04:10:48.139381+0000","last_undegraded":"2023-10-06T04:10:48.139381+0000","last_fullsized":"2023-10-06T04:10:48.139381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:07.105832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018579569999999999,"stat_sum":{"num_bytes":31421368,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":37,"num_read_kb":12302,"num_write":304,"num_write_kb":31090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"53'50","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:46.318091+0000","last_change":"2023-10-06T03:57:10.346054+0000","last_active":"2023-10-06T04:10:46.318091+0000","last_peered":"2023-10-06T04:10:46.318091+0000","last_clean":"2023-10-06T04:10:46.318091+0000","last_became_active":"2023-10-06T03:57:10.345721+0000","last_became_peered":"2023-10-06T03:57:10.345721+0000","last_unstale":"2023-10-06T04:10:46.318091+0000","last_undegraded":"2023-10-06T04:10:46.318091+0000","last_fullsized":"2023-10-06T04:10:46.318091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:54.481304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":15,"num_write":48,"num_write_kb":4121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"53'355","reported_seq":2455,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.868827+0000","last_change":"2023-10-06T04:05:03.151517+0000","last_active":"2023-10-06T04:10:47.868827+0000","last_peered":"2023-10-06T04:10:47.868827+0000","last_clean":"2023-10-06T04:10:47.868827+0000","last_became_active":"2023-10-06T03:57:59.255587+0000","last_became_peered":"2023-10-06T03:57:59.255587+0000","last_unstale":"2023-10-06T04:10:47.868827+0000","last_undegraded":"2023-10-06T04:10:47.868827+0000","last_fullsized":"2023-10-06T04:10:47.868827+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:08.261023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001510024,"stat_sum":{"num_bytes":1780876,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":38,"num_read_kb":8209,"num_write":338,"num_write_kb":10112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"53'63","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:42.932445+0000","last_change":"2023-10-06T03:57:10.340531+0000","last_active":"2023-10-06T04:10:42.932445+0000","last_peered":"2023-10-06T04:10:42.932445+0000","last_clean":"2023-10-06T04:10:42.932445+0000","last_became_active":"2023-10-06T03:57:10.340232+0000","last_became_peered":"2023-10-06T03:57:10.340232+0000","last_unstale":"2023-10-06T04:10:42.932445+0000","last_undegraded":"2023-10-06T04:10:42.932445+0000","last_fullsized":"2023-10-06T04:10:42.932445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:43:26.332055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":66,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"53'339","reported_seq":2443,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.117177+0000","last_change":"2023-10-06T04:05:02.718927+0000","last_active":"2023-10-06T04:10:48.117177+0000","last_peered":"2023-10-06T04:10:48.117177+0000","last_clean":"2023-10-06T04:10:48.117177+0000","last_became_active":"2023-10-06T03:57:11.374892+0000","last_became_peered":"2023-10-06T03:57:11.374892+0000","last_unstale":"2023-10-06T04:10:48.117177+0000","last_undegraded":"2023-10-06T04:10:48.117177+0000","last_fullsized":"2023-10-06T04:10:48.117177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:51.198745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00156207,"stat_sum":{"num_bytes":17752359,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":38,"num_read_kb":8211,"num_write":326,"num_write_kb":25695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"53'670","reported_seq":727,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:31.373637+0000","last_change":"2023-10-06T03:57:10.346758+0000","last_active":"2023-10-06T04:10:31.373637+0000","last_peered":"2023-10-06T04:10:31.373637+0000","last_clean":"2023-10-06T04:10:31.373637+0000","last_became_active":"2023-10-06T03:57:10.346430+0000","last_became_peered":"2023-10-06T03:57:10.346430+0000","last_unstale":"2023-10-06T04:10:31.373637+0000","last_undegraded":"2023-10-06T04:10:31.373637+0000","last_fullsized":"2023-10-06T04:10:31.373637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:10.809242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389252,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":30,"num_read_kb":9887,"num_write":668,"num_write_kb":8554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"53'323","reported_seq":2428,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.032035+0000","last_change":"2023-10-06T04:05:02.723446+0000","last_active":"2023-10-06T04:10:48.032035+0000","last_peered":"2023-10-06T04:10:48.032035+0000","last_clean":"2023-10-06T04:10:48.032035+0000","last_became_active":"2023-10-06T03:57:11.367565+0000","last_became_peered":"2023-10-06T03:57:11.367565+0000","last_unstale":"2023-10-06T04:10:48.032035+0000","last_undegraded":"2023-10-06T04:10:48.032035+0000","last_fullsized":"2023-10-06T04:10:48.032035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:14.409116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027762680000000001,"stat_sum":{"num_bytes":10222697,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":38,"num_read_kb":23,"num_write":314,"num_write_kb":10145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"53'179","reported_seq":235,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:16.251094+0000","last_change":"2023-10-06T03:57:10.351062+0000","last_active":"2023-10-06T04:10:16.251094+0000","last_peered":"2023-10-06T04:10:16.251094+0000","last_clean":"2023-10-06T04:10:16.251094+0000","last_became_active":"2023-10-06T03:57:10.350795+0000","last_became_peered":"2023-10-06T03:57:10.350795+0000","last_unstale":"2023-10-06T04:10:16.251094+0000","last_undegraded":"2023-10-06T04:10:16.251094+0000","last_fullsized":"2023-10-06T04:10:16.251094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:54:45.714518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388676,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":30,"num_read_kb":20509,"num_write":184,"num_write_kb":8306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"53'276","reported_seq":2375,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.006307+0000","last_change":"2023-10-06T04:05:02.722868+0000","last_active":"2023-10-06T04:10:48.006307+0000","last_peered":"2023-10-06T04:10:48.006307+0000","last_clean":"2023-10-06T04:10:48.006307+0000","last_became_active":"2023-10-06T03:57:11.367254+0000","last_became_peered":"2023-10-06T03:57:11.367254+0000","last_unstale":"2023-10-06T04:10:48.006307+0000","last_undegraded":"2023-10-06T04:10:48.006307+0000","last_fullsized":"2023-10-06T04:10:48.006307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:04.706485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027822569999999998,"stat_sum":{"num_bytes":13638644,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":32,"num_read_kb":10530,"num_write":271,"num_write_kb":17563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"53'105","reported_seq":157,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:23.094836+0000","last_change":"2023-10-06T03:57:10.338125+0000","last_active":"2023-10-06T04:10:23.094836+0000","last_peered":"2023-10-06T04:10:23.094836+0000","last_clean":"2023-10-06T04:10:23.094836+0000","last_became_active":"2023-10-06T03:57:10.337798+0000","last_became_peered":"2023-10-06T03:57:10.337798+0000","last_unstale":"2023-10-06T04:10:23.094836+0000","last_undegraded":"2023-10-06T04:10:23.094836+0000","last_fullsized":"2023-10-06T04:10:23.094836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:00.482657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":9753,"num_write":104,"num_write_kb":4165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"53'329","reported_seq":2433,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.868704+0000","last_change":"2023-10-06T04:05:02.973084+0000","last_active":"2023-10-06T04:10:47.868704+0000","last_peered":"2023-10-06T04:10:47.868704+0000","last_clean":"2023-10-06T04:10:47.868704+0000","last_became_active":"2023-10-06T03:57:11.376223+0000","last_became_peered":"2023-10-06T03:57:11.376223+0000","last_unstale":"2023-10-06T04:10:47.868704+0000","last_undegraded":"2023-10-06T04:10:47.868704+0000","last_fullsized":"2023-10-06T04:10:47.868704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:04:37.408871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018161119999999999,"stat_sum":{"num_bytes":34899441,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":37,"num_read_kb":11570,"num_write":325,"num_write_kb":34246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"53'97","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:36.263633+0000","last_change":"2023-10-06T03:57:10.350511+0000","last_active":"2023-10-06T04:10:36.263633+0000","last_peered":"2023-10-06T04:10:36.263633+0000","last_clean":"2023-10-06T04:10:36.263633+0000","last_became_active":"2023-10-06T03:57:10.350219+0000","last_became_peered":"2023-10-06T03:57:10.350219+0000","last_unstale":"2023-10-06T04:10:36.263633+0000","last_undegraded":"2023-10-06T04:10:36.263633+0000","last_fullsized":"2023-10-06T04:10:36.263633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:56.043506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96,"num_write_kb":8294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"53'359","reported_seq":2468,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.062653+0000","last_change":"2023-10-06T04:05:03.367352+0000","last_active":"2023-10-06T04:10:48.062653+0000","last_peered":"2023-10-06T04:10:48.062653+0000","last_clean":"2023-10-06T04:10:48.062653+0000","last_became_active":"2023-10-06T03:57:11.362443+0000","last_became_peered":"2023-10-06T03:57:11.362443+0000","last_unstale":"2023-10-06T04:10:48.062653+0000","last_undegraded":"2023-10-06T04:10:48.062653+0000","last_fullsized":"2023-10-06T04:10:48.062653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:21:09.143085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002448839,"stat_sum":{"num_bytes":22757129,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":42,"num_read_kb":8288,"num_write":351,"num_write_kb":22409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"53'71","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:36.258463+0000","last_change":"2023-10-06T03:57:10.346259+0000","last_active":"2023-10-06T04:10:36.258463+0000","last_peered":"2023-10-06T04:10:36.258463+0000","last_clean":"2023-10-06T04:10:36.258463+0000","last_became_active":"2023-10-06T03:57:10.345424+0000","last_became_peered":"2023-10-06T03:57:10.345424+0000","last_unstale":"2023-10-06T04:10:36.258463+0000","last_undegraded":"2023-10-06T04:10:36.258463+0000","last_fullsized":"2023-10-06T04:10:36.258463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:19.278206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":33,"num_read_kb":33,"num_write":72,"num_write_kb":4158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"53'304","reported_seq":2409,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.152945+0000","last_change":"2023-10-06T04:05:02.723050+0000","last_active":"2023-10-06T04:10:48.152945+0000","last_peered":"2023-10-06T04:10:48.152945+0000","last_clean":"2023-10-06T04:10:48.152945+0000","last_became_active":"2023-10-06T03:57:11.375824+0000","last_became_peered":"2023-10-06T03:57:11.375824+0000","last_unstale":"2023-10-06T04:10:48.152945+0000","last_undegraded":"2023-10-06T04:10:48.152945+0000","last_fullsized":"2023-10-06T04:10:48.152945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:24:26.156866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023200299999999998,"stat_sum":{"num_bytes":9687154,"num_objects":142,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":38,"num_read_kb":8212,"num_write":296,"num_write_kb":9609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"53'250","reported_seq":295,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:23.097110+0000","last_change":"2023-10-06T03:57:10.350408+0000","last_active":"2023-10-06T04:10:23.097110+0000","last_peered":"2023-10-06T04:10:23.097110+0000","last_clean":"2023-10-06T04:10:23.097110+0000","last_became_active":"2023-10-06T03:57:10.350135+0000","last_became_peered":"2023-10-06T03:57:10.350135+0000","last_unstale":"2023-10-06T04:10:23.097110+0000","last_undegraded":"2023-10-06T04:10:23.097110+0000","last_fullsized":"2023-10-06T04:10:23.097110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:43.522111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":4098,"num_write":280,"num_write_kb":4293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"53'321","reported_seq":2420,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.862961+0000","last_change":"2023-10-06T04:05:02.972356+0000","last_active":"2023-10-06T04:10:47.862961+0000","last_peered":"2023-10-06T04:10:47.862961+0000","last_clean":"2023-10-06T04:10:47.862961+0000","last_became_active":"2023-10-06T03:57:11.376236+0000","last_became_peered":"2023-10-06T03:57:11.376236+0000","last_unstale":"2023-10-06T04:10:47.862961+0000","last_undegraded":"2023-10-06T04:10:47.862961+0000","last_fullsized":"2023-10-06T04:10:47.862961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:33.854818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015573080000000001,"stat_sum":{"num_bytes":22874532,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":32,"num_read_kb":10317,"num_write":312,"num_write_kb":26593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"50'92","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:52.580642+0000","last_change":"2023-10-06T03:57:10.345799+0000","last_active":"2023-10-06T04:09:52.580642+0000","last_peered":"2023-10-06T04:09:52.580642+0000","last_clean":"2023-10-06T04:09:52.580642+0000","last_became_active":"2023-10-06T03:57:10.345510+0000","last_became_peered":"2023-10-06T03:57:10.345510+0000","last_unstale":"2023-10-06T04:09:52.580642+0000","last_undegraded":"2023-10-06T04:09:52.580642+0000","last_fullsized":"2023-10-06T04:09:52.580642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:09.567314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":4108,"num_write":90,"num_write_kb":4140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"53'342","reported_seq":2446,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.072619+0000","last_change":"2023-10-06T04:05:02.721917+0000","last_active":"2023-10-06T04:10:48.072619+0000","last_peered":"2023-10-06T04:10:48.072619+0000","last_clean":"2023-10-06T04:10:48.072619+0000","last_became_active":"2023-10-06T03:57:11.368375+0000","last_became_peered":"2023-10-06T03:57:11.368375+0000","last_unstale":"2023-10-06T04:10:48.072619+0000","last_undegraded":"2023-10-06T04:10:48.072619+0000","last_fullsized":"2023-10-06T04:10:48.072619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:27.125523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001632912,"stat_sum":{"num_bytes":5549566,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":37,"num_read_kb":4115,"num_write":335,"num_write_kb":5549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"53'164","reported_seq":212,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:22.809255+0000","last_change":"2023-10-06T03:57:10.346945+0000","last_active":"2023-10-06T04:10:22.809255+0000","last_peered":"2023-10-06T04:10:22.809255+0000","last_clean":"2023-10-06T04:10:22.809255+0000","last_became_active":"2023-10-06T03:57:10.346689+0000","last_became_peered":"2023-10-06T03:57:10.346689+0000","last_unstale":"2023-10-06T04:10:22.809255+0000","last_undegraded":"2023-10-06T04:10:22.809255+0000","last_fullsized":"2023-10-06T04:10:22.809255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:43.420455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":12290,"num_write":163,"num_write_kb":8336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"53'349","reported_seq":2450,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.892654+0000","last_change":"2023-10-06T04:05:03.542774+0000","last_active":"2023-10-06T04:10:47.892654+0000","last_peered":"2023-10-06T04:10:47.892654+0000","last_clean":"2023-10-06T04:10:47.892654+0000","last_became_active":"2023-10-06T03:57:11.371795+0000","last_became_peered":"2023-10-06T03:57:11.371795+0000","last_unstale":"2023-10-06T04:10:47.892654+0000","last_undegraded":"2023-10-06T04:10:47.892654+0000","last_fullsized":"2023-10-06T04:10:47.892654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:00.324478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016252980000000001,"stat_sum":{"num_bytes":5725462,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":34,"num_read_kb":13,"num_write":343,"num_write_kb":5770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"53'54","reported_seq":96,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:48.081196+0000","last_change":"2023-10-06T03:57:10.348592+0000","last_active":"2023-10-06T04:09:48.081196+0000","last_peered":"2023-10-06T04:09:48.081196+0000","last_clean":"2023-10-06T04:09:48.081196+0000","last_became_active":"2023-10-06T03:57:10.348291+0000","last_became_peered":"2023-10-06T03:57:10.348291+0000","last_unstale":"2023-10-06T04:09:48.081196+0000","last_undegraded":"2023-10-06T04:09:48.081196+0000","last_fullsized":"2023-10-06T04:09:48.081196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:00.660348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":57,"num_write_kb":4195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"53'308","reported_seq":2408,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.055840+0000","last_change":"2023-10-06T04:05:03.366405+0000","last_active":"2023-10-06T04:10:48.055840+0000","last_peered":"2023-10-06T04:10:48.055840+0000","last_clean":"2023-10-06T04:10:48.055840+0000","last_became_active":"2023-10-06T03:57:11.365077+0000","last_became_peered":"2023-10-06T03:57:11.365077+0000","last_unstale":"2023-10-06T04:10:48.055840+0000","last_undegraded":"2023-10-06T04:10:48.055840+0000","last_fullsized":"2023-10-06T04:10:48.055840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:25.199391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019352270000000001,"stat_sum":{"num_bytes":15027095,"num_objects":146,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":146,"num_whiteouts":0,"num_read":33,"num_read_kb":4109,"num_write":302,"num_write_kb":14833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"52'278","reported_seq":329,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:51.781455+0000","last_change":"2023-10-06T03:57:10.348418+0000","last_active":"2023-10-06T04:09:51.781455+0000","last_peered":"2023-10-06T04:09:51.781455+0000","last_clean":"2023-10-06T04:09:51.781455+0000","last_became_active":"2023-10-06T03:57:10.348055+0000","last_became_peered":"2023-10-06T03:57:10.348055+0000","last_unstale":"2023-10-06T04:09:51.781455+0000","last_undegraded":"2023-10-06T04:09:51.781455+0000","last_fullsized":"2023-10-06T04:09:51.781455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:26.102022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":17,"num_read_kb":12301,"num_write":272,"num_write_kb":8316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"53'342","reported_seq":2437,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.070418+0000","last_change":"2023-10-06T04:05:02.724303+0000","last_active":"2023-10-06T04:10:48.070418+0000","last_peered":"2023-10-06T04:10:48.070418+0000","last_clean":"2023-10-06T04:10:48.070418+0000","last_became_active":"2023-10-06T03:57:11.370073+0000","last_became_peered":"2023-10-06T03:57:11.370073+0000","last_unstale":"2023-10-06T04:10:48.070418+0000","last_undegraded":"2023-10-06T04:10:48.070418+0000","last_fullsized":"2023-10-06T04:10:48.070418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:58.417951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027454179999999999,"stat_sum":{"num_bytes":18012929,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":28,"num_read_kb":4110,"num_write":339,"num_write_kb":17768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"53'112","reported_seq":159,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:46.295859+0000","last_change":"2023-10-06T03:57:10.352966+0000","last_active":"2023-10-06T04:10:46.295859+0000","last_peered":"2023-10-06T04:10:46.295859+0000","last_clean":"2023-10-06T04:10:46.295859+0000","last_became_active":"2023-10-06T03:57:10.352653+0000","last_became_peered":"2023-10-06T03:57:10.352653+0000","last_unstale":"2023-10-06T04:10:46.295859+0000","last_undegraded":"2023-10-06T04:10:46.295859+0000","last_fullsized":"2023-10-06T04:10:46.295859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:32.092557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":8,"num_read_kb":4217,"num_write":204,"num_write_kb":4775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"53'338","reported_seq":2437,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.039511+0000","last_change":"2023-10-06T04:05:02.724113+0000","last_active":"2023-10-06T04:10:48.039511+0000","last_peered":"2023-10-06T04:10:48.039511+0000","last_clean":"2023-10-06T04:10:48.039511+0000","last_became_active":"2023-10-06T03:57:11.363830+0000","last_became_peered":"2023-10-06T03:57:11.363830+0000","last_unstale":"2023-10-06T04:10:48.039511+0000","last_undegraded":"2023-10-06T04:10:48.039511+0000","last_fullsized":"2023-10-06T04:10:48.039511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:04.084019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027698559999999998,"stat_sum":{"num_bytes":22850224,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":32,"num_read_kb":4115,"num_write":329,"num_write_kb":26589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"53'32","reported_seq":82,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:50.963938+0000","last_change":"2023-10-06T03:57:10.349421+0000","last_active":"2023-10-06T04:10:50.963938+0000","last_peered":"2023-10-06T04:10:50.963938+0000","last_clean":"2023-10-06T04:10:50.963938+0000","last_became_active":"2023-10-06T03:57:10.349114+0000","last_became_peered":"2023-10-06T03:57:10.349114+0000","last_unstale":"2023-10-06T04:10:50.963938+0000","last_undegraded":"2023-10-06T04:10:50.963938+0000","last_fullsized":"2023-10-06T04:10:50.963938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:08.209791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2605665,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":19,"num_read_kb":18,"num_write":36,"num_write_kb":2597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"53'278","reported_seq":2370,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.045140+0000","last_change":"2023-10-06T04:05:02.724241+0000","last_active":"2023-10-06T04:10:48.045140+0000","last_peered":"2023-10-06T04:10:48.045140+0000","last_clean":"2023-10-06T04:10:48.045140+0000","last_became_active":"2023-10-06T03:57:11.367257+0000","last_became_peered":"2023-10-06T03:57:11.367257+0000","last_unstale":"2023-10-06T04:10:48.045140+0000","last_undegraded":"2023-10-06T04:10:48.045140+0000","last_fullsized":"2023-10-06T04:10:48.045140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:43.628819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002803224,"stat_sum":{"num_bytes":18462781,"num_objects":138,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":138,"num_whiteouts":0,"num_read":26,"num_read_kb":4106,"num_write":268,"num_write_kb":22268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"53'53","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:52.265701+0000","last_change":"2023-10-06T03:57:10.347649+0000","last_active":"2023-10-06T04:09:52.265701+0000","last_peered":"2023-10-06T04:09:52.265701+0000","last_clean":"2023-10-06T04:09:52.265701+0000","last_became_active":"2023-10-06T03:57:10.347433+0000","last_became_peered":"2023-10-06T03:57:10.347433+0000","last_unstale":"2023-10-06T04:09:52.265701+0000","last_undegraded":"2023-10-06T04:09:52.265701+0000","last_fullsized":"2023-10-06T04:09:52.265701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:55.951494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":11,"num_read_kb":8201,"num_write":54,"num_write_kb":4137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"53'320","reported_seq":2421,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.122912+0000","last_change":"2023-10-06T04:05:02.718759+0000","last_active":"2023-10-06T04:10:48.122912+0000","last_peered":"2023-10-06T04:10:48.122912+0000","last_clean":"2023-10-06T04:10:48.122912+0000","last_became_active":"2023-10-06T03:57:11.369825+0000","last_became_peered":"2023-10-06T03:57:11.369825+0000","last_unstale":"2023-10-06T04:10:48.122912+0000","last_undegraded":"2023-10-06T04:10:48.122912+0000","last_fullsized":"2023-10-06T04:10:48.122912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:43.415929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014204689999999999,"stat_sum":{"num_bytes":11370958,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":35,"num_read_kb":8210,"num_write":305,"num_write_kb":15362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:37.659432+0000","last_change":"2023-10-06T03:57:05.663312+0000","last_active":"2023-10-06T04:08:37.659432+0000","last_peered":"2023-10-06T04:08:37.659432+0000","last_clean":"2023-10-06T04:08:37.659432+0000","last_became_active":"2023-10-06T03:57:05.663046+0000","last_became_peered":"2023-10-06T03:57:05.663046+0000","last_unstale":"2023-10-06T04:08:37.659432+0000","last_undegraded":"2023-10-06T04:08:37.659432+0000","last_fullsized":"2023-10-06T04:08:37.659432+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:31.174359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"53'33","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:51.495313+0000","last_change":"2023-10-06T03:57:10.348930+0000","last_active":"2023-10-06T04:10:51.495313+0000","last_peered":"2023-10-06T04:10:51.495313+0000","last_clean":"2023-10-06T04:10:51.495313+0000","last_became_active":"2023-10-06T03:57:10.348608+0000","last_became_peered":"2023-10-06T03:57:10.348608+0000","last_unstale":"2023-10-06T04:10:51.495313+0000","last_undegraded":"2023-10-06T04:10:51.495313+0000","last_fullsized":"2023-10-06T04:10:51.495313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:00:09.283718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2070221,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":34,"num_write_kb":2062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"53'287","reported_seq":2383,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.110309+0000","last_change":"2023-10-06T04:05:02.718555+0000","last_active":"2023-10-06T04:10:48.110309+0000","last_peered":"2023-10-06T04:10:48.110309+0000","last_clean":"2023-10-06T04:10:48.110309+0000","last_became_active":"2023-10-06T03:57:11.370847+0000","last_became_peered":"2023-10-06T03:57:11.370847+0000","last_unstale":"2023-10-06T04:10:48.110309+0000","last_undegraded":"2023-10-06T04:10:48.110309+0000","last_fullsized":"2023-10-06T04:10:48.110309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:38.111171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013685990000000001,"stat_sum":{"num_bytes":14898272,"num_objects":129,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":129,"num_whiteouts":0,"num_read":29,"num_read_kb":12301,"num_write":279,"num_write_kb":18790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:37.658819+0000","last_change":"2023-10-06T03:57:05.318834+0000","last_active":"2023-10-06T04:08:37.658819+0000","last_peered":"2023-10-06T04:08:37.658819+0000","last_clean":"2023-10-06T04:08:37.658819+0000","last_became_active":"2023-10-06T03:57:05.318572+0000","last_became_peered":"2023-10-06T03:57:05.318572+0000","last_unstale":"2023-10-06T04:08:37.658819+0000","last_undegraded":"2023-10-06T04:08:37.658819+0000","last_fullsized":"2023-10-06T04:08:37.658819+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:21.951095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"53'10","reported_seq":60,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:46.769558+0000","last_change":"2023-10-06T03:57:10.348556+0000","last_active":"2023-10-06T04:09:46.769558+0000","last_peered":"2023-10-06T04:09:46.769558+0000","last_clean":"2023-10-06T04:09:46.769558+0000","last_became_active":"2023-10-06T03:57:10.348271+0000","last_became_peered":"2023-10-06T03:57:10.348271+0000","last_unstale":"2023-10-06T04:09:46.769558+0000","last_undegraded":"2023-10-06T04:09:46.769558+0000","last_fullsized":"2023-10-06T04:09:46.769558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:58.470989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":26,"num_write":12,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"53'320","reported_seq":2420,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.012974+0000","last_change":"2023-10-06T04:05:02.719009+0000","last_active":"2023-10-06T04:10:48.012974+0000","last_peered":"2023-10-06T04:10:48.012974+0000","last_clean":"2023-10-06T04:10:48.012974+0000","last_became_active":"2023-10-06T03:57:11.365006+0000","last_became_peered":"2023-10-06T03:57:11.365006+0000","last_unstale":"2023-10-06T04:10:48.012974+0000","last_undegraded":"2023-10-06T04:10:48.012974+0000","last_fullsized":"2023-10-06T04:10:48.012974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:33.005015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015961910000000001,"stat_sum":{"num_bytes":5694351,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":33,"num_read_kb":4109,"num_write":313,"num_write_kb":5725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:37.658115+0000","last_change":"2023-10-06T03:57:05.314695+0000","last_active":"2023-10-06T04:08:37.658115+0000","last_peered":"2023-10-06T04:08:37.658115+0000","last_clean":"2023-10-06T04:08:37.658115+0000","last_became_active":"2023-10-06T03:57:05.314408+0000","last_became_peered":"2023-10-06T03:57:05.314408+0000","last_unstale":"2023-10-06T04:08:37.658115+0000","last_undegraded":"2023-10-06T04:08:37.658115+0000","last_fullsized":"2023-10-06T04:08:37.658115+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:19.042145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"53'359","reported_seq":405,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:16.606829+0000","last_change":"2023-10-06T03:57:10.346078+0000","last_active":"2023-10-06T04:10:16.606829+0000","last_peered":"2023-10-06T04:10:16.606829+0000","last_clean":"2023-10-06T04:10:16.606829+0000","last_became_active":"2023-10-06T03:57:10.345769+0000","last_became_peered":"2023-10-06T03:57:10.345769+0000","last_unstale":"2023-10-06T04:10:16.606829+0000","last_undegraded":"2023-10-06T04:10:16.606829+0000","last_fullsized":"2023-10-06T04:10:16.606829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:16.630681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":352,"num_write_kb":12528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"53'315","reported_seq":2421,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.123122+0000","last_change":"2023-10-06T04:05:02.718334+0000","last_active":"2023-10-06T04:10:48.123122+0000","last_peered":"2023-10-06T04:10:48.123122+0000","last_clean":"2023-10-06T04:10:48.123122+0000","last_became_active":"2023-10-06T03:57:11.370076+0000","last_became_peered":"2023-10-06T03:57:11.370076+0000","last_unstale":"2023-10-06T04:10:48.123122+0000","last_undegraded":"2023-10-06T04:10:48.123122+0000","last_fullsized":"2023-10-06T04:10:48.123122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:18:59.517155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011772849999999999,"stat_sum":{"num_bytes":18508624,"num_objects":141,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":141,"num_whiteouts":0,"num_read":39,"num_read_kb":4112,"num_write":305,"num_write_kb":22326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:37.658686+0000","last_change":"2023-10-06T03:57:05.318778+0000","last_active":"2023-10-06T04:08:37.658686+0000","last_peered":"2023-10-06T04:08:37.658686+0000","last_clean":"2023-10-06T04:08:37.658686+0000","last_became_active":"2023-10-06T03:57:05.318456+0000","last_became_peered":"2023-10-06T03:57:05.318456+0000","last_unstale":"2023-10-06T04:08:37.658686+0000","last_undegraded":"2023-10-06T04:08:37.658686+0000","last_fullsized":"2023-10-06T04:08:37.658686+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:39.769241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":195,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:36.756245+0000","last_change":"2023-10-06T03:57:05.664245+0000","last_active":"2023-10-06T04:08:36.756245+0000","last_peered":"2023-10-06T04:08:36.756245+0000","last_clean":"2023-10-06T04:08:36.756245+0000","last_became_active":"2023-10-06T03:57:05.663426+0000","last_became_peered":"2023-10-06T03:57:05.663426+0000","last_unstale":"2023-10-06T04:08:36.756245+0000","last_undegraded":"2023-10-06T04:08:36.756245+0000","last_fullsized":"2023-10-06T04:08:36.756245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:29.322599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"53'719","reported_seq":770,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:52.436682+0000","last_change":"2023-10-06T03:57:10.351059+0000","last_active":"2023-10-06T04:09:52.436682+0000","last_peered":"2023-10-06T04:09:52.436682+0000","last_clean":"2023-10-06T04:09:52.436682+0000","last_became_active":"2023-10-06T03:57:10.348879+0000","last_became_peered":"2023-10-06T03:57:10.348879+0000","last_unstale":"2023-10-06T04:09:52.436682+0000","last_undegraded":"2023-10-06T04:09:52.436682+0000","last_fullsized":"2023-10-06T04:09:52.436682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:35.764148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9487985,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":20487,"num_write":715,"num_write_kb":13783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"53'332","reported_seq":2435,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.889473+0000","last_change":"2023-10-06T04:05:03.542987+0000","last_active":"2023-10-06T04:10:47.889473+0000","last_peered":"2023-10-06T04:10:47.889473+0000","last_clean":"2023-10-06T04:10:47.889473+0000","last_became_active":"2023-10-06T03:57:11.373452+0000","last_became_peered":"2023-10-06T03:57:11.373452+0000","last_unstale":"2023-10-06T04:10:47.889473+0000","last_undegraded":"2023-10-06T04:10:47.889473+0000","last_fullsized":"2023-10-06T04:10:47.889473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:20.524065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018153620000000001,"stat_sum":{"num_bytes":5826606,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":36,"num_read_kb":2733,"num_write":325,"num_write_kb":9961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:36.756325+0000","last_change":"2023-10-06T03:57:05.318790+0000","last_active":"2023-10-06T04:08:36.756325+0000","last_peered":"2023-10-06T04:08:36.756325+0000","last_clean":"2023-10-06T04:08:36.756325+0000","last_became_active":"2023-10-06T03:57:05.318531+0000","last_became_peered":"2023-10-06T03:57:05.318531+0000","last_unstale":"2023-10-06T04:08:36.756325+0000","last_undegraded":"2023-10-06T04:08:36.756325+0000","last_fullsized":"2023-10-06T04:08:36.756325+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:12.876998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"52'540","reported_seq":703,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:46.766101+0000","last_change":"2023-10-06T03:57:10.343328+0000","last_active":"2023-10-06T04:09:46.766101+0000","last_peered":"2023-10-06T04:09:46.766101+0000","last_clean":"2023-10-06T04:09:46.766101+0000","last_became_active":"2023-10-06T03:57:10.343051+0000","last_became_peered":"2023-10-06T03:57:10.343051+0000","last_unstale":"2023-10-06T04:09:46.766101+0000","last_undegraded":"2023-10-06T04:09:46.766101+0000","last_fullsized":"2023-10-06T04:09:46.766101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:27.722475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4571457,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":122,"num_read_kb":3989,"num_write":534,"num_write_kb":4895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"53'333","reported_seq":2437,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.058057+0000","last_change":"2023-10-06T04:05:03.367824+0000","last_active":"2023-10-06T04:10:48.058057+0000","last_peered":"2023-10-06T04:10:48.058057+0000","last_clean":"2023-10-06T04:10:48.058057+0000","last_became_active":"2023-10-06T03:57:11.373264+0000","last_became_peered":"2023-10-06T03:57:11.373264+0000","last_unstale":"2023-10-06T04:10:48.058057+0000","last_undegraded":"2023-10-06T04:10:48.058057+0000","last_fullsized":"2023-10-06T04:10:48.058057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:15.950245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026908990000000001,"stat_sum":{"num_bytes":10595820,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":37,"num_read_kb":4121,"num_write":327,"num_write_kb":10522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:36.758524+0000","last_change":"2023-10-06T03:57:05.663770+0000","last_active":"2023-10-06T04:08:36.758524+0000","last_peered":"2023-10-06T04:08:36.758524+0000","last_clean":"2023-10-06T04:08:36.758524+0000","last_became_active":"2023-10-06T03:57:05.663501+0000","last_became_peered":"2023-10-06T03:57:05.663501+0000","last_unstale":"2023-10-06T04:08:36.758524+0000","last_undegraded":"2023-10-06T04:08:36.758524+0000","last_fullsized":"2023-10-06T04:08:36.758524+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:09.806915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"52'5","reported_seq":47,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:09:30.961755+0000","last_change":"2023-10-06T03:57:10.346585+0000","last_active":"2023-10-06T04:09:30.961755+0000","last_peered":"2023-10-06T04:09:30.961755+0000","last_clean":"2023-10-06T04:09:30.961755+0000","last_became_active":"2023-10-06T03:57:10.345743+0000","last_became_peered":"2023-10-06T03:57:10.345743+0000","last_unstale":"2023-10-06T04:09:30.961755+0000","last_undegraded":"2023-10-06T04:09:30.961755+0000","last_fullsized":"2023-10-06T04:09:30.961755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:54.589702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20578,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"53'338","reported_seq":2432,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.051399+0000","last_change":"2023-10-06T04:05:03.366736+0000","last_active":"2023-10-06T04:10:48.051399+0000","last_peered":"2023-10-06T04:10:48.051399+0000","last_clean":"2023-10-06T04:10:48.051399+0000","last_became_active":"2023-10-06T03:57:11.373543+0000","last_became_peered":"2023-10-06T03:57:11.373543+0000","last_unstale":"2023-10-06T04:10:48.051399+0000","last_undegraded":"2023-10-06T04:10:48.051399+0000","last_fullsized":"2023-10-06T04:10:48.051399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:15.152354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020558070000000002,"stat_sum":{"num_bytes":6468011,"num_objects":146,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":146,"num_whiteouts":0,"num_read":27,"num_read_kb":14,"num_write":331,"num_write_kb":6443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"53'14","reported_seq":63,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:36.203548+0000","last_change":"2023-10-06T03:57:10.344618+0000","last_active":"2023-10-06T04:10:36.203548+0000","last_peered":"2023-10-06T04:10:36.203548+0000","last_clean":"2023-10-06T04:10:36.203548+0000","last_became_active":"2023-10-06T03:57:10.344324+0000","last_became_peered":"2023-10-06T03:57:10.344324+0000","last_unstale":"2023-10-06T04:10:36.203548+0000","last_undegraded":"2023-10-06T04:10:36.203548+0000","last_fullsized":"2023-10-06T04:10:36.203548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:06.062331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":22,"num_read_kb":21,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:37.659459+0000","last_change":"2023-10-06T03:57:05.317973+0000","last_active":"2023-10-06T04:08:37.659459+0000","last_peered":"2023-10-06T04:08:37.659459+0000","last_clean":"2023-10-06T04:08:37.659459+0000","last_became_active":"2023-10-06T03:57:05.317715+0000","last_became_peered":"2023-10-06T03:57:05.317715+0000","last_unstale":"2023-10-06T04:08:37.659459+0000","last_undegraded":"2023-10-06T04:08:37.659459+0000","last_fullsized":"2023-10-06T04:08:37.659459+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:37:51.858909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"53'307","reported_seq":2406,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.140157+0000","last_change":"2023-10-06T04:05:02.721255+0000","last_active":"2023-10-06T04:10:48.140157+0000","last_peered":"2023-10-06T04:10:48.140157+0000","last_clean":"2023-10-06T04:10:48.140157+0000","last_became_active":"2023-10-06T03:57:11.369445+0000","last_became_peered":"2023-10-06T03:57:11.369445+0000","last_unstale":"2023-10-06T04:10:48.140157+0000","last_undegraded":"2023-10-06T04:10:48.140157+0000","last_fullsized":"2023-10-06T04:10:48.140157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:52.999421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015173949999999999,"stat_sum":{"num_bytes":11679199,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":32,"num_read_kb":17,"num_write":302,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:37.573513+0000","last_change":"2023-10-06T03:57:05.320847+0000","last_active":"2023-10-06T04:08:37.573513+0000","last_peered":"2023-10-06T04:08:37.573513+0000","last_clean":"2023-10-06T04:08:37.573513+0000","last_became_active":"2023-10-06T03:57:05.320583+0000","last_became_peered":"2023-10-06T03:57:05.320583+0000","last_unstale":"2023-10-06T04:08:37.573513+0000","last_undegraded":"2023-10-06T04:08:37.573513+0000","last_fullsized":"2023-10-06T04:08:37.573513+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:12.772449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"53'92","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.058132+0000","last_change":"2023-10-06T03:57:10.341132+0000","last_active":"2023-10-06T04:10:48.058132+0000","last_peered":"2023-10-06T04:10:48.058132+0000","last_clean":"2023-10-06T04:10:48.058132+0000","last_became_active":"2023-10-06T03:57:10.339424+0000","last_became_peered":"2023-10-06T03:57:10.339424+0000","last_unstale":"2023-10-06T04:10:48.058132+0000","last_undegraded":"2023-10-06T04:10:48.058132+0000","last_fullsized":"2023-10-06T04:10:48.058132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:39.675711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":19,"num_write":219,"num_write_kb":1011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"53'316","reported_seq":2411,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.032289+0000","last_change":"2023-10-06T04:05:02.722365+0000","last_active":"2023-10-06T04:10:48.032289+0000","last_peered":"2023-10-06T04:10:48.032289+0000","last_clean":"2023-10-06T04:10:48.032289+0000","last_became_active":"2023-10-06T03:57:11.372215+0000","last_became_peered":"2023-10-06T03:57:11.372215+0000","last_unstale":"2023-10-06T04:10:48.032289+0000","last_undegraded":"2023-10-06T04:10:48.032289+0000","last_fullsized":"2023-10-06T04:10:48.032289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:23.559480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020199100000000002,"stat_sum":{"num_bytes":14547344,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":28,"num_read_kb":4106,"num_write":306,"num_write_kb":14296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"53'204","reported_seq":257,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:44.658005+0000","last_change":"2023-10-06T03:57:10.344051+0000","last_active":"2023-10-06T04:10:44.658005+0000","last_peered":"2023-10-06T04:10:44.658005+0000","last_clean":"2023-10-06T04:10:44.658005+0000","last_became_active":"2023-10-06T03:57:10.343865+0000","last_became_peered":"2023-10-06T03:57:10.343865+0000","last_unstale":"2023-10-06T04:10:44.658005+0000","last_undegraded":"2023-10-06T04:10:44.658005+0000","last_fullsized":"2023-10-06T04:10:44.658005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:32.522470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":11,"num_read_kb":32771,"num_write":206,"num_write_kb":12488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"53'331","reported_seq":2431,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.948651+0000","last_change":"2023-10-06T04:05:03.151764+0000","last_active":"2023-10-06T04:10:47.948651+0000","last_peered":"2023-10-06T04:10:47.948651+0000","last_clean":"2023-10-06T04:10:47.948651+0000","last_became_active":"2023-10-06T03:57:11.369681+0000","last_became_peered":"2023-10-06T03:57:11.369681+0000","last_unstale":"2023-10-06T04:10:47.948651+0000","last_undegraded":"2023-10-06T04:10:47.948651+0000","last_fullsized":"2023-10-06T04:10:47.948651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:53.858179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013151409999999999,"stat_sum":{"num_bytes":9592740,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":33,"num_read_kb":14,"num_write":324,"num_write_kb":13639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"50'100","reported_seq":149,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:52.634119+0000","last_change":"2023-10-06T03:57:10.349666+0000","last_active":"2023-10-06T04:09:52.634119+0000","last_peered":"2023-10-06T04:09:52.634119+0000","last_clean":"2023-10-06T04:09:52.634119+0000","last_became_active":"2023-10-06T03:57:10.349320+0000","last_became_peered":"2023-10-06T03:57:10.349320+0000","last_unstale":"2023-10-06T04:09:52.634119+0000","last_undegraded":"2023-10-06T04:09:52.634119+0000","last_fullsized":"2023-10-06T04:09:52.634119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:45:49.709519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194858,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":4110,"num_write":99,"num_write_kb":4143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"53'317","reported_seq":2411,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.096552+0000","last_change":"2023-10-06T04:05:02.723622+0000","last_active":"2023-10-06T04:10:48.096552+0000","last_peered":"2023-10-06T04:10:48.096552+0000","last_clean":"2023-10-06T04:10:48.096552+0000","last_became_active":"2023-10-06T03:57:11.370229+0000","last_became_peered":"2023-10-06T03:57:11.370229+0000","last_unstale":"2023-10-06T04:10:48.096552+0000","last_undegraded":"2023-10-06T04:10:48.096552+0000","last_fullsized":"2023-10-06T04:10:48.096552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:26.174331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027851170000000001,"stat_sum":{"num_bytes":11772234,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":27,"num_read_kb":17,"num_write":309,"num_write_kb":11655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"53'238","reported_seq":299,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:23.094832+0000","last_change":"2023-10-06T03:57:10.350263+0000","last_active":"2023-10-06T04:10:23.094832+0000","last_peered":"2023-10-06T04:10:23.094832+0000","last_clean":"2023-10-06T04:10:23.094832+0000","last_became_active":"2023-10-06T03:57:10.350083+0000","last_became_peered":"2023-10-06T03:57:10.350083+0000","last_unstale":"2023-10-06T04:10:23.094832+0000","last_undegraded":"2023-10-06T04:10:23.094832+0000","last_fullsized":"2023-10-06T04:10:23.094832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:05.433032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":36,"num_read_kb":45128,"num_write":264,"num_write_kb":12512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"53'333","reported_seq":2430,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.062537+0000","last_change":"2023-10-06T04:05:02.724048+0000","last_active":"2023-10-06T04:10:48.062537+0000","last_peered":"2023-10-06T04:10:48.062537+0000","last_clean":"2023-10-06T04:10:48.062537+0000","last_became_active":"2023-10-06T03:57:11.369123+0000","last_became_peered":"2023-10-06T03:57:11.369123+0000","last_unstale":"2023-10-06T04:10:48.062537+0000","last_undegraded":"2023-10-06T04:10:48.062537+0000","last_fullsized":"2023-10-06T04:10:48.062537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:28.139036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028613789999999998,"stat_sum":{"num_bytes":15016756,"num_objects":144,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":0,"num_read":30,"num_read_kb":4112,"num_write":318,"num_write_kb":14827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"52'181","reported_seq":225,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:47.407717+0000","last_change":"2023-10-06T03:57:10.347049+0000","last_active":"2023-10-06T04:09:47.407717+0000","last_peered":"2023-10-06T04:09:47.407717+0000","last_clean":"2023-10-06T04:09:47.407717+0000","last_became_active":"2023-10-06T03:57:10.344583+0000","last_became_peered":"2023-10-06T03:57:10.344583+0000","last_unstale":"2023-10-06T04:09:47.407717+0000","last_undegraded":"2023-10-06T04:09:47.407717+0000","last_fullsized":"2023-10-06T04:09:47.407717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:32.280257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":177,"num_write_kb":8286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"53'324","reported_seq":2428,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.088100+0000","last_change":"2023-10-06T04:05:02.721502+0000","last_active":"2023-10-06T04:10:48.088100+0000","last_peered":"2023-10-06T04:10:48.088100+0000","last_clean":"2023-10-06T04:10:48.088100+0000","last_became_active":"2023-10-06T03:57:11.366500+0000","last_became_peered":"2023-10-06T03:57:11.366500+0000","last_unstale":"2023-10-06T04:10:48.088100+0000","last_undegraded":"2023-10-06T04:10:48.088100+0000","last_fullsized":"2023-10-06T04:10:48.088100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:24.714603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016878310000000001,"stat_sum":{"num_bytes":6012647,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":38,"num_read_kb":4113,"num_write":317,"num_write_kb":10135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"50'170","reported_seq":220,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:51.778904+0000","last_change":"2023-10-06T03:57:10.348170+0000","last_active":"2023-10-06T04:09:51.778904+0000","last_peered":"2023-10-06T04:09:51.778904+0000","last_clean":"2023-10-06T04:09:51.778904+0000","last_became_active":"2023-10-06T03:57:10.347897+0000","last_became_peered":"2023-10-06T03:57:10.347897+0000","last_unstale":"2023-10-06T04:09:51.778904+0000","last_undegraded":"2023-10-06T04:09:51.778904+0000","last_fullsized":"2023-10-06T04:09:51.778904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:40.756246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7191702,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":12,"num_read_kb":8202,"num_write":168,"num_write_kb":7111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"53'386","reported_seq":2487,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.032330+0000","last_change":"2023-10-06T04:05:02.721899+0000","last_active":"2023-10-06T04:10:48.032330+0000","last_peered":"2023-10-06T04:10:48.032330+0000","last_clean":"2023-10-06T04:10:48.032330+0000","last_became_active":"2023-10-06T03:57:11.364150+0000","last_became_peered":"2023-10-06T03:57:11.364150+0000","last_unstale":"2023-10-06T04:10:48.032330+0000","last_undegraded":"2023-10-06T04:10:48.032330+0000","last_fullsized":"2023-10-06T04:10:48.032330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:11.038080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001670408,"stat_sum":{"num_bytes":10918213,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":35,"num_read_kb":7178,"num_write":378,"num_write_kb":19053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"53'345","reported_seq":2466,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.795523+0000","last_change":"2023-10-06T04:05:03.542896+0000","last_active":"2023-10-06T04:10:47.795523+0000","last_peered":"2023-10-06T04:10:47.795523+0000","last_clean":"2023-10-06T04:10:47.795523+0000","last_became_active":"2023-10-06T03:57:11.366524+0000","last_became_peered":"2023-10-06T03:57:11.366524+0000","last_unstale":"2023-10-06T04:10:47.795523+0000","last_undegraded":"2023-10-06T04:10:47.795523+0000","last_fullsized":"2023-10-06T04:10:47.795523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:33.044858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017816290000000001,"stat_sum":{"num_bytes":22531000,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":54,"num_read_kb":16279,"num_write":337,"num_write_kb":22178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"53'349","reported_seq":2453,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.079994+0000","last_change":"2023-10-06T04:05:02.721559+0000","last_active":"2023-10-06T04:10:48.079994+0000","last_peered":"2023-10-06T04:10:48.079994+0000","last_clean":"2023-10-06T04:10:48.079994+0000","last_became_active":"2023-10-06T03:57:11.370848+0000","last_became_peered":"2023-10-06T03:57:11.370848+0000","last_unstale":"2023-10-06T04:10:48.079994+0000","last_undegraded":"2023-10-06T04:10:48.079994+0000","last_fullsized":"2023-10-06T04:10:48.079994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:15.919090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017239289999999999,"stat_sum":{"num_bytes":12150184,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":37,"num_read_kb":8210,"num_write":341,"num_write_kb":16096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"53'351","reported_seq":2457,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.032069+0000","last_change":"2023-10-06T04:05:02.723715+0000","last_active":"2023-10-06T04:10:48.032069+0000","last_peered":"2023-10-06T04:10:48.032069+0000","last_clean":"2023-10-06T04:10:48.032069+0000","last_became_active":"2023-10-06T03:57:11.363875+0000","last_became_peered":"2023-10-06T03:57:11.363875+0000","last_unstale":"2023-10-06T04:10:48.032069+0000","last_undegraded":"2023-10-06T04:10:48.032069+0000","last_fullsized":"2023-10-06T04:10:48.032069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:29.236344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028142470000000002,"stat_sum":{"num_bytes":14810267,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":39,"num_read_kb":4117,"num_write":340,"num_write_kb":14623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"53'337","reported_seq":2445,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.153308+0000","last_change":"2023-10-06T04:05:02.722555+0000","last_active":"2023-10-06T04:10:48.153308+0000","last_peered":"2023-10-06T04:10:48.153308+0000","last_clean":"2023-10-06T04:10:48.153308+0000","last_became_active":"2023-10-06T03:57:11.377126+0000","last_became_peered":"2023-10-06T03:57:11.377126+0000","last_unstale":"2023-10-06T04:10:48.153308+0000","last_undegraded":"2023-10-06T04:10:48.153308+0000","last_fullsized":"2023-10-06T04:10:48.153308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:05.041737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023666569999999999,"stat_sum":{"num_bytes":6069899,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":41,"num_read_kb":18,"num_write":330,"num_write_kb":6093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"53'310","reported_seq":2407,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.768352+0000","last_change":"2023-10-06T04:05:02.972860+0000","last_active":"2023-10-06T04:10:47.768352+0000","last_peered":"2023-10-06T04:10:47.768352+0000","last_clean":"2023-10-06T04:10:47.768352+0000","last_became_active":"2023-10-06T03:57:59.256086+0000","last_became_peered":"2023-10-06T03:57:59.256086+0000","last_unstale":"2023-10-06T04:10:47.768352+0000","last_undegraded":"2023-10-06T04:10:47.768352+0000","last_fullsized":"2023-10-06T04:10:47.768352+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:54.263909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018573859999999999,"stat_sum":{"num_bytes":11282404,"num_objects":135,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":0,"num_read":33,"num_read_kb":11479,"num_write":302,"num_write_kb":14403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":918737492,"num_objects":9845,"num_object_clones":0,"num_object_copies":19690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9845,"num_whiteouts":0,"num_read":2243,"num_read_kb":380500,"num_write":20424,"num_write_kb":1040919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1675485184,"data_stored":1835934888,"data_compressed":157714556,"data_compressed_allocated":174235648,"data_compressed_original":386113536,"omap_allocated":0,"internal_metadata":0},"log_size":20998,"ondisk_log_size":20998,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":160325534,"num_objects":200,"num_object_clones":0,"num_object_copies":400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":200,"num_whiteouts":0,"num_read":555,"num_read_kb":254049,"num_write":6759,"num_write_kb":191312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":122,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81215488,"data_stored":320651068,"data_compressed":43195038,"data_compressed_allocated":54370304,"data_compressed_original":294060032,"omap_allocated":0,"internal_metadata":0},"log_size":6506,"ondisk_log_size":6506,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64858,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575018,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506268,"kb_used_data":208980,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93865572,"statfs":{"total":96636764160,"available":96118345728,"internally_reserved":0,"allocated":213995520,"data_stored":258704768,"data_compressed":22101424,"data_compressed_allocated":25088000,"data_compressed_original":75468800,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.55400000000000005,"15min":0.54500000000000004},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28399999999999997},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.88900000000000001},"last":0.50600000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52200000000000002,"15min":0.52100000000000002},"min":{"1min":0.38400000000000001,"5min":0.312,"15min":0.28899999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.78000000000000003},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54300000000000004,"15min":0.52300000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78300000000000003,"5min":1.25,"15min":1.25},"last":0.48199999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54800000000000004,"15min":0.54900000000000004},"min":{"1min":0.33400000000000002,"5min":0.29799999999999999,"15min":0.28399999999999997},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57599999999999996,"15min":0.60699999999999998},"min":{"1min":0.34300000000000003,"5min":0.33700000000000002,"15min":0.33100000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":7.3739999999999997},"last":0.56299999999999994}]},{"osd":3,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.318},"max":{"1min":0.80600000000000005,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.55400000000000005,"15min":0.55900000000000005},"min":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72899999999999998,"5min":0.78000000000000003,"15min":0.85299999999999998},"last":0.45000000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.44,"15min":0.46400000000000002},"min":{"1min":0.30099999999999999,"5min":0.27400000000000002,"15min":0.251},"max":{"1min":0.64000000000000001,"5min":0.73599999999999999,"15min":1.0149999999999999},"last":0.433},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.34300000000000003,"5min":0.33500000000000002,"15min":0.28299999999999997},"max":{"1min":0.68600000000000005,"5min":0.80200000000000005,"15min":0.90700000000000003},"last":0.53700000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.495,"15min":0.51100000000000001},"min":{"1min":0.316,"5min":0.29999999999999999,"15min":0.27300000000000002},"max":{"1min":0.74199999999999999,"5min":0.82999999999999996,"15min":0.86099999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54600000000000004,"15min":0.56299999999999994},"min":{"1min":0.40699999999999997,"5min":0.317,"15min":0.28000000000000003},"max":{"1min":0.73699999999999999,"5min":0.84899999999999998,"15min":1.377},"last":0.46800000000000003}]},{"osd":6,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.51100000000000001,"15min":0.56200000000000006},"min":{"1min":0.34100000000000003,"5min":0.30599999999999999,"15min":0.29999999999999999},"max":{"1min":0.77100000000000002,"5min":0.77600000000000002,"15min":1.3420000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60499999999999998,"15min":0.59099999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.30199999999999999},"max":{"1min":0.84599999999999997,"5min":0.88200000000000001,"15min":0.93700000000000006},"last":0.61699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575017,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473876,"kb_used_data":164300,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":93897964,"statfs":{"total":96636764160,"available":96151515136,"internally_reserved":0,"allocated":168243200,"data_stored":187372207,"data_compressed":15745085,"data_compressed_allocated":17555456,"data_compressed_original":42110976,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54800000000000004,"15min":0.53200000000000003},"min":{"1min":0.28799999999999998,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.70399999999999996,"5min":2.613,"15min":2.613},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.58399999999999996,"15min":0.55800000000000005},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.33300000000000002},"max":{"1min":0.72299999999999998,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":0.68999999999999995}]},{"osd":1,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53400000000000003,"15min":0.59599999999999997},"min":{"1min":0.28899999999999998,"5min":0.28499999999999998,"15min":0.27200000000000002},"max":{"1min":0.71599999999999997,"5min":2.3889999999999998,"15min":9.5519999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.59399999999999997,"15min":0.60199999999999998},"min":{"1min":0.33400000000000002,"5min":0.315,"15min":0.313},"max":{"1min":0.75800000000000001,"5min":6.8019999999999996,"15min":6.8019999999999996},"last":0.66600000000000004}]},{"osd":2,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.625,"15min":0.67300000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.29799999999999999},"max":{"1min":0.71799999999999997,"5min":5.7590000000000003,"15min":9.6769999999999996},"last":0.627},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.623,"15min":0.67600000000000005},"min":{"1min":0.42199999999999999,"5min":0.33200000000000002,"15min":0.31},"max":{"1min":0.75,"5min":5.71,"15min":9.7309999999999999},"last":0.70199999999999996}]},{"osd":3,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.54900000000000004,"15min":0.57499999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.29299999999999998},"max":{"1min":0.73899999999999999,"5min":0.99099999999999999,"15min":5.7359999999999998},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.59699999999999998,"15min":0.61299999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.308},"max":{"1min":0.68799999999999994,"5min":1.0149999999999999,"15min":5.7009999999999996},"last":0.54700000000000004}]},{"osd":4,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.48099999999999998,"15min":0.49399999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.63300000000000001,"5min":0.78600000000000003,"15min":1.0129999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54100000000000004,"15min":0.54900000000000004},"min":{"1min":0.32600000000000001,"5min":0.313,"15min":0.30399999999999999},"max":{"1min":0.70299999999999996,"5min":0.79800000000000004,"15min":1.026},"last":0.39800000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.49299999999999999,"15min":0.50800000000000001},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.77800000000000002,"5min":0.78200000000000003,"15min":0.995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.66500000000000004,"5min":0.78500000000000003,"15min":0.83699999999999997},"last":0.65100000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.34899999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.77000000000000002,"5min":0.84199999999999997,"15min":0.98299999999999998},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56499999999999995,"15min":0.57799999999999996},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80500000000000005,"5min":0.84699999999999998,"15min":0.90800000000000003},"last":0.52200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575017,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576968,"kb_used_data":275584,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":93794872,"statfs":{"total":96636764160,"available":96045948928,"internally_reserved":0,"allocated":282198016,"data_stored":333666787,"data_compressed":32289943,"data_compressed_allocated":36245504,"data_compressed_original":96591872,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.91800000000000004,"15min":0.52000000000000002},"min":{"1min":0.22500000000000001,"5min":0.20799999999999999,"15min":0.192},"max":{"1min":0.56999999999999995,"5min":57.700000000000003,"15min":57.700000000000003},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.65200000000000002,"15min":0.441},"min":{"1min":0.23899999999999999,"5min":0.23300000000000001,"15min":0.186},"max":{"1min":0.64000000000000001,"5min":28.629999999999999,"15min":28.629999999999999},"last":0.32600000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.97899999999999998,"15min":0.57199999999999995},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.19600000000000001},"max":{"1min":0.59899999999999998,"5min":58.795000000000002,"15min":58.795000000000002},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.67600000000000005,"15min":0.48999999999999999},"min":{"1min":0.32200000000000001,"5min":0.29599999999999999,"15min":0.23000000000000001},"max":{"1min":0.63400000000000001,"5min":25.722000000000001,"15min":25.722000000000001},"last":0.378}]},{"osd":3,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.59799999999999998,"15min":0.443},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.216},"max":{"1min":0.58499999999999996,"5min":22.920999999999999,"15min":22.920999999999999},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.439,"5min":0.63400000000000001,"15min":0.46800000000000003},"min":{"1min":0.29099999999999998,"5min":0.27300000000000002,"15min":0.22600000000000001},"max":{"1min":0.59799999999999998,"5min":22.797999999999998,"15min":22.797999999999998},"last":0.39200000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":1.2370000000000001,"15min":0.82399999999999995},"min":{"1min":0.443,"5min":0.38500000000000001,"15min":0.34499999999999997},"max":{"1min":1.0149999999999999,"5min":59.018999999999998,"15min":59.018999999999998},"last":0.624},{"interface":"front","average":{"1min":0.70999999999999996,"5min":1.26,"15min":0.83499999999999996},"min":{"1min":0.50700000000000001,"5min":0.47399999999999998,"15min":0.316},"max":{"1min":1.0609999999999999,"5min":59.046999999999997,"15min":59.046999999999997},"last":0.65200000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":1.21,"15min":0.79300000000000004},"min":{"1min":0.48599999999999999,"5min":0.40200000000000002,"15min":0.34399999999999997},"max":{"1min":0.90300000000000002,"5min":58.987000000000002,"15min":58.987000000000002},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65700000000000003,"15min":0.59599999999999997},"min":{"1min":0.44900000000000001,"5min":0.42599999999999999,"15min":0.29399999999999998},"max":{"1min":0.97699999999999998,"5min":6.2140000000000004,"15min":6.2140000000000004},"last":0.53700000000000003}]},{"osd":6,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65200000000000002,"15min":0.60399999999999998},"min":{"1min":0.44,"5min":0.41999999999999998,"15min":0.36599999999999999},"max":{"1min":0.95599999999999996,"5min":4.6020000000000003,"15min":4.6020000000000003},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":1.3,"15min":0.85999999999999999},"min":{"1min":0.49099999999999999,"5min":0.42799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94799999999999995,"5min":58.911999999999999,"15min":58.911999999999999},"last":0.61199999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":1.2210000000000001,"15min":0.81100000000000005},"min":{"1min":0.45800000000000002,"5min":0.40100000000000002,"15min":0.38700000000000001},"max":{"1min":1.1000000000000001,"5min":58.953000000000003,"15min":58.953000000000003},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.65900000000000003,"15min":0.62},"min":{"1min":0.46700000000000003,"5min":0.38100000000000001,"15min":0.34399999999999997},"max":{"1min":0.83199999999999996,"5min":4.5570000000000004,"15min":4.5570000000000004},"last":0.55800000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575017,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495552,"kb_used_data":190072,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":93876288,"statfs":{"total":96636764160,"available":96129318912,"internally_reserved":0,"allocated":194633728,"data_stored":251414553,"data_compressed":26053766,"data_compressed_allocated":29863936,"data_compressed_original":92942336,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.34100000000000003,"15min":0.34899999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.19500000000000001},"max":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":6.9539999999999997},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.35599999999999998,"15min":0.371},"min":{"1min":0.22700000000000001,"5min":0.215,"15min":0.191},"max":{"1min":0.55700000000000005,"5min":0.59499999999999997,"15min":8.0519999999999996},"last":0.27200000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.36199999999999999,"15min":0.35899999999999999},"min":{"1min":0.24299999999999999,"5min":0.24199999999999999,"15min":0.214},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":1.0589999999999999},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.38500000000000001,"15min":0.376},"min":{"1min":0.28299999999999997,"5min":0.26100000000000001,"15min":0.20799999999999999},"max":{"1min":0.626,"5min":0.626,"15min":2.0489999999999999},"last":0.36599999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.38600000000000001,"15min":0.41799999999999998},"min":{"1min":0.23300000000000001,"5min":0.23100000000000001,"15min":0.223},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":9.391},"last":0.311},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.42599999999999999,"15min":0.45600000000000002},"min":{"1min":0.31900000000000001,"5min":0.29099999999999998,"15min":0.27800000000000002},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":9.5239999999999991},"last":0.38100000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.64900000000000002,"15min":0.63400000000000001},"min":{"1min":0.35599999999999998,"5min":0.34000000000000002,"15min":0.29199999999999998},"max":{"1min":1.026,"5min":1.026,"15min":5.7699999999999996},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66000000000000003,"15min":0.63600000000000001},"min":{"1min":0.45400000000000001,"5min":0.41499999999999998,"15min":0.29499999999999998},"max":{"1min":0.997,"5min":0.997,"15min":2.7320000000000002},"last":0.57599999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.45000000000000001,"5min":0.41399999999999998,"15min":0.33800000000000002},"max":{"1min":0.89900000000000002,"5min":0.90500000000000003,"15min":5.5069999999999997},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63100000000000001,"15min":0.60499999999999998},"min":{"1min":0.436,"5min":0.38500000000000001,"15min":0.34799999999999998},"max":{"1min":0.872,"5min":0.878,"15min":1.05},"last":0.53800000000000003}]},{"osd":6,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.626,"15min":0.64700000000000002},"min":{"1min":0.42599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":6.9139999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64800000000000002,"15min":0.65500000000000003},"min":{"1min":0.46700000000000003,"5min":0.42199999999999999,"15min":0.379},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":6.3780000000000001},"last":0.44}]},{"osd":7,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61099999999999999,"15min":0.623},"min":{"1min":0.38500000000000001,"5min":0.377,"15min":0.36699999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":5.9989999999999997},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.627,"15min":0.61099999999999999},"min":{"1min":0.443,"5min":0.40100000000000002,"15min":0.39700000000000002},"max":{"1min":0.83999999999999997,"5min":0.86499999999999999,"15min":2.6930000000000001},"last":0.52400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575017,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":437036,"kb_used_data":139748,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93934804,"statfs":{"total":96636764160,"available":96189239296,"internally_reserved":0,"allocated":143101952,"data_stored":173375985,"data_compressed":18899047,"data_compressed_allocated":21323776,"data_compressed_original":57229312,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.34300000000000003,"15min":0.38400000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.19},"max":{"1min":0.51000000000000001,"5min":0.54300000000000004,"15min":14.968999999999999},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.376,"15min":0.41299999999999998},"min":{"1min":0.24099999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.52100000000000002,"5min":0.59199999999999997,"15min":14.746},"last":0.34000000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.37,"15min":0.52200000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.21099999999999999},"max":{"1min":0.53300000000000003,"5min":0.54100000000000004,"15min":50.677999999999997},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.45800000000000002,"15min":0.60099999999999998},"min":{"1min":0.34599999999999997,"5min":0.28599999999999998,"15min":0.26000000000000001},"max":{"1min":0.57599999999999996,"5min":0.66200000000000003,"15min":50.615000000000002},"last":0.35999999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.37,"15min":0.45900000000000002},"min":{"1min":0.25600000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.48999999999999999,"5min":0.61299999999999999,"15min":17.440999999999999},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.44,"5min":0.44700000000000001,"15min":0.59899999999999998},"min":{"1min":0.32400000000000001,"5min":0.29099999999999998,"15min":0.23499999999999999},"max":{"1min":0.65500000000000003,"5min":0.88900000000000001,"15min":50.424999999999997},"last":0.307}]},{"osd":4,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59499999999999997,"15min":0.64400000000000002},"min":{"1min":0.41099999999999998,"5min":0.34200000000000003,"15min":0.29699999999999999},"max":{"1min":0.76800000000000002,"5min":0.84799999999999998,"15min":14.802},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60399999999999998,"15min":0.67500000000000004},"min":{"1min":0.42199999999999999,"5min":0.39400000000000002,"15min":0.36499999999999999},"max":{"1min":0.746,"5min":0.88700000000000001,"15min":14.621},"last":0.432}]},{"osd":5,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60399999999999998,"15min":0.63200000000000001},"min":{"1min":0.48399999999999999,"5min":0.40100000000000002,"15min":0.35199999999999998},"max":{"1min":0.75700000000000001,"5min":0.84399999999999997,"15min":14.792999999999999},"last":0.499},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60899999999999999,"15min":0.76000000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.371},"max":{"1min":0.72499999999999998,"5min":0.92600000000000005,"15min":50.537999999999997},"last":0.57699999999999996}]},{"osd":6,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63700000000000001,"15min":0.68700000000000006},"min":{"1min":0.44700000000000001,"5min":0.40699999999999997,"15min":0.39100000000000001},"max":{"1min":0.78700000000000003,"5min":1.216,"15min":23.388000000000002},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64300000000000002,"15min":0.69999999999999996},"min":{"1min":0.50600000000000001,"5min":0.40400000000000003,"15min":0.39100000000000001},"max":{"1min":0.75,"5min":0.86899999999999999,"15min":14.856},"last":0.51900000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.627,"15min":0.67900000000000005},"min":{"1min":0.45600000000000002,"5min":0.40999999999999998,"15min":0.377},"max":{"1min":0.80900000000000005,"5min":0.95899999999999996,"15min":14.693},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63500000000000001,"15min":0.67400000000000004},"min":{"1min":0.45800000000000002,"5min":0.44600000000000001,"15min":0.434},"max":{"1min":0.79600000000000004,"5min":0.95499999999999996,"15min":10.349},"last":0.53600000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575017,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584240,"kb_used_data":274664,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":93787600,"statfs":{"total":96636764160,"available":96038502400,"internally_reserved":0,"allocated":281255936,"data_stored":341245364,"data_compressed":32144254,"data_compressed_allocated":36605952,"data_compressed_original":105447424,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.34399999999999997,"15min":0.35999999999999999},"min":{"1min":0.24299999999999999,"5min":0.216,"15min":0.186},"max":{"1min":0.5,"5min":0.58699999999999997,"15min":7.0999999999999996},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.35499999999999998,"15min":0.41699999999999998},"min":{"1min":0.27200000000000002,"5min":0.221,"15min":0.218},"max":{"1min":0.622,"5min":0.622,"15min":14.257},"last":0.36299999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.34999999999999998,"15min":0.35399999999999998},"min":{"1min":0.23100000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":3.1760000000000002},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.40899999999999997,"15min":0.39600000000000002},"min":{"1min":0.29999999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.61499999999999999,"5min":0.68200000000000005,"15min":5.2599999999999998},"last":0.308}]},{"osd":2,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.39800000000000002,"15min":0.45900000000000002},"min":{"1min":0.316,"5min":0.25700000000000001,"15min":0.23499999999999999},"max":{"1min":0.57599999999999996,"5min":0.747,"15min":13.138},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.42699999999999999,"15min":0.44500000000000001},"min":{"1min":0.32800000000000001,"5min":0.28299999999999997,"15min":0.27200000000000002},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":7.181},"last":0.40000000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.627,"15min":0.68100000000000005},"min":{"1min":0.46800000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":13.068},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.65600000000000003,"15min":0.67500000000000004},"min":{"1min":0.57399999999999995,"5min":0.434,"15min":0.39800000000000002},"max":{"1min":0.878,"5min":0.89100000000000001,"15min":6.4299999999999997},"last":0.73599999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.47799999999999998,"5min":0.35999999999999999,"15min":0.32700000000000001},"max":{"1min":0.89000000000000001,"5min":0.95599999999999996,"15min":3.105},"last":0.436},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.59799999999999998,"15min":0.60799999999999998},"min":{"1min":0.52000000000000002,"5min":0.372,"15min":0.29299999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":3.5369999999999999},"last":0.52400000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.63100000000000001,"15min":0.68799999999999994},"min":{"1min":0.54400000000000004,"5min":0.375,"15min":0.375},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":12.974},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63400000000000001,"15min":0.64100000000000001},"min":{"1min":0.56299999999999994,"5min":0.378,"15min":0.35599999999999998},"max":{"1min":0.93700000000000006,"5min":0.94499999999999995,"15min":3.0379999999999998},"last":0.501}]},{"osd":7,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.61299999999999999,"15min":0.63400000000000001},"min":{"1min":0.52900000000000003,"5min":0.41199999999999998,"15min":0.40100000000000002},"max":{"1min":0.93400000000000005,"5min":0.97199999999999998,"15min":6.2949999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.63400000000000001,"15min":0.67200000000000004},"min":{"1min":0.496,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90400000000000003,"5min":0.95399999999999996,"15min":13.032},"last":0.54200000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575018,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":580532,"kb_used_data":279148,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":93791308,"statfs":{"total":96636764160,"available":96042299392,"internally_reserved":0,"allocated":285847552,"data_stored":371153375,"data_compressed":31721463,"data_compressed_allocated":36786176,"data_compressed_original":130072576,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52900000000000003,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68100000000000005,"5min":0.747,"15min":0.79600000000000004},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.56699999999999995,"15min":0.54600000000000004},"min":{"1min":0.47699999999999998,"5min":0.35099999999999998,"15min":0.26600000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.80000000000000004},"last":0.72799999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.41199999999999998,"5min":0.34899999999999998,"15min":0.26200000000000001},"max":{"1min":0.73899999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.501,"15min":0.51100000000000001},"min":{"1min":0.35199999999999998,"5min":0.30399999999999999,"15min":0.27400000000000002},"max":{"1min":0.70399999999999996,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.40400000000000003,"5min":0.35899999999999999,"15min":0.312},"max":{"1min":0.69099999999999995,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.57599999999999996},"min":{"1min":0.441,"5min":0.36299999999999999,"15min":0.311},"max":{"1min":0.76700000000000002,"5min":1.413,"15min":1.413},"last":0.70699999999999996}]},{"osd":3,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.59099999999999997,"15min":0.55100000000000005},"min":{"1min":0.38400000000000001,"5min":0.32100000000000001,"15min":0.314},"max":{"1min":0.65800000000000003,"5min":6.7000000000000002,"15min":6.7000000000000002},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52900000000000003,"15min":0.54500000000000004},"min":{"1min":0.36899999999999999,"5min":0.36399999999999999,"15min":0.32000000000000001},"max":{"1min":0.72299999999999998,"5min":1.208,"15min":1.208},"last":0.63}]},{"osd":5,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.44600000000000001,"15min":0.47699999999999998},"min":{"1min":0.28599999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.59299999999999997,"5min":0.77100000000000002,"15min":0.81899999999999995},"last":0.436},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52200000000000002,"15min":0.53000000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.29499999999999998},"max":{"1min":0.66800000000000004,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.48299999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.48499999999999999,"15min":0.51300000000000001},"min":{"1min":0.315,"5min":0.30099999999999999,"15min":0.28100000000000003},"max":{"1min":0.60499999999999998,"5min":1.236,"15min":1.236},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.42899999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.69199999999999995,"5min":1.381,"15min":1.381},"last":0.64700000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53200000000000003,"15min":0.54600000000000004},"min":{"1min":0.38600000000000001,"5min":0.32800000000000001,"15min":0.30599999999999999},"max":{"1min":0.66700000000000004,"5min":0.71799999999999997,"15min":0.83499999999999996},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.45900000000000002,"5min":0.39900000000000002,"15min":0.32600000000000001},"max":{"1min":0.73599999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.57899999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575017,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485212,"kb_used_data":187924,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93886628,"statfs":{"total":96636764160,"available":96139907072,"internally_reserved":0,"allocated":192434176,"data_stored":243297716,"data_compressed":22019470,"data_compressed_allocated":25227264,"data_compressed_original":81457152,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.50700000000000001,"15min":0.55100000000000005},"min":{"1min":0.34399999999999997,"5min":0.29999999999999999,"15min":0.23300000000000001},"max":{"1min":0.60199999999999998,"5min":1.0189999999999999,"15min":11.167},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.53800000000000003,"15min":0.57699999999999996},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.221},"max":{"1min":0.71899999999999997,"5min":1.0449999999999999,"15min":9.8659999999999997},"last":0.626}]},{"osd":1,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.64100000000000001,"5min":1.0780000000000001,"15min":1.1120000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55100000000000005,"15min":0.54000000000000004},"min":{"1min":0.35599999999999998,"5min":0.29999999999999999,"15min":0.28799999999999998},"max":{"1min":0.73999999999999999,"5min":1.028,"15min":2.524},"last":0.65600000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.52700000000000002,"15min":0.54000000000000004},"min":{"1min":0.39400000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.68999999999999995,"5min":0.98499999999999999,"15min":1.0389999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70399999999999996,"5min":0.999,"15min":1.1439999999999999},"last":0.53300000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.77300000000000002,"5min":0.97299999999999998,"15min":2.9359999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60999999999999999,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.377,"15min":0.28999999999999998},"max":{"1min":0.77500000000000002,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.67700000000000005}]},{"osd":4,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.45900000000000002,"15min":0.47999999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.26000000000000001},"max":{"1min":0.71099999999999997,"5min":0.73499999999999999,"15min":1.1619999999999999},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57699999999999996,"15min":0.54200000000000004},"min":{"1min":0.33700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.75600000000000001,"5min":0.78900000000000003,"15min":1.1950000000000001},"last":0.39300000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46899999999999997,"15min":0.503},"min":{"1min":0.35699999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65900000000000003,"5min":0.78300000000000003,"15min":0.94099999999999995},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56499999999999995,"15min":0.58099999999999996},"min":{"1min":0.41899999999999998,"5min":0.38400000000000001,"15min":0.32900000000000001},"max":{"1min":0.72599999999999998,"5min":0.84999999999999998,"15min":0.97899999999999998},"last":0.51600000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.53700000000000003},"min":{"1min":0.38700000000000001,"5min":0.33500000000000002,"15min":0.28699999999999998},"max":{"1min":0.81999999999999995,"5min":0.84899999999999998,"15min":1.1830000000000001},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.438,"5min":0.375,"15min":0.311},"max":{"1min":0.79900000000000004,"5min":0.81299999999999994,"15min":1.29},"last":0.40899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13369344,"data_stored":49512965,"data_compressed":6345911,"data_compressed_allocated":8089600,"data_compressed_original":44290048,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7880704,"data_stored":33731560,"data_compressed":4481765,"data_compressed_allocated":5713920,"data_compressed_original":31571968,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3530752,"data_stored":15209155,"data_compressed":2097150,"data_compressed_allocated":2711552,"data_compressed_original":14401536,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12541952,"data_stored":45232784,"data_compressed":5646645,"data_compressed_allocated":7327744,"data_compressed_original":40050688,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20615168,"data_stored":82714194,"data_compressed":11662389,"data_compressed_allocated":14422016,"data_compressed_original":76578816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10690560,"data_stored":43917090,"data_compressed":5801730,"data_compressed_allocated":7286784,"data_compressed_original":40554496,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1855488,"data_stored":8388720,"data_compressed":1228027,"data_compressed_allocated":1515520,"data_compressed_original":8056832,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":10731520,"data_stored":41944600,"data_compressed":5931421,"data_compressed_allocated":7303168,"data_compressed_original":38555648,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":180592640,"data_stored":201004046,"data_compressed":19675426,"data_compressed_allocated":21729280,"data_compressed_original":48078848,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":273707008,"data_stored":299627100,"data_compressed":27808178,"data_compressed_allocated":30531584,"data_compressed_original":65019904,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":138960896,"data_stored":157858703,"data_compressed":16801897,"data_compressed_allocated":18612224,"data_compressed_original":42827776,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":268103680,"data_stored":295704453,"data_compressed":26497609,"data_compressed_allocated":29278208,"data_compressed_original":65396736,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":264622080,"data_stored":288131054,"data_compressed":20059074,"data_compressed_allocated":22364160,"data_compressed_original":53493760,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":181133312,"data_stored":199072499,"data_compressed":16217740,"data_compressed_allocated":17940480,"data_compressed_original":40902656,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":165777408,"data_stored":178675360,"data_compressed":14517058,"data_compressed_allocated":16039936,"data_compressed_original":34054144,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":202588160,"data_stored":215861673,"data_compressed":16137574,"data_compressed_allocated":17739776,"data_compressed_original":36339712,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:10:56.511 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:10:56.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.655+0000 7f43cabb9700 1 Processor -- start 2023-10-06T04:10:56.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.655+0000 7f43cabb9700 1 -- start start 2023-10-06T04:10:56.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.656+0000 7f43cabb9700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f43c406a780 0x7f43c4156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:56.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.656+0000 7f43cabb9700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f43c406ab50 0x7f43c40c2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:56.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.656+0000 7f43cabb9700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43c406e840 con 0x7f43c406e340 2023-10-06T04:10:56.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.656+0000 7f43cabb9700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43c4153970 con 0x7f43c406a780 2023-10-06T04:10:56.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.656+0000 7f43cabb9700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43c4157100 con 0x7f43c406ab50 2023-10-06T04:10:56.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.656+0000 7f43c8955700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f43c406a780 0x7f43c4156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:56.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.656+0000 7f43bbfff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f43c406e340 0x7f43c40c5bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:35330/0 (socket says 172.21.15.157:35330) 2023-10-06T04:10:56.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.656+0000 7f43bbfff700 1 -- 172.21.15.157:0/1266070449 learned_addr learned my addr 172.21.15.157:0/1266070449 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:10:56.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.657+0000 7f43c9156700 1 --2- 172.21.15.157:0/1266070449 >> v2:172.21.15.175:3300/0 conn(0x7f43c406ab50 0x7f43c40c2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:56.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.657+0000 7f43c8955700 1 -- 172.21.15.157:0/1266070449 >> v2:172.21.15.175:3300/0 conn(0x7f43c406ab50 msgr2=0x7f43c40c2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:10:56.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.657+0000 7f43c8955700 1 --2- 172.21.15.157:0/1266070449 >> v2:172.21.15.175:3300/0 conn(0x7f43c406ab50 0x7f43c40c2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:56.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.657+0000 7f43c8955700 1 -- 172.21.15.157:0/1266070449 >> v1:172.21.15.157:6789/0 conn(0x7f43c406e340 legacy=0x7f43c40c5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:56.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.657+0000 7f43bb7fe700 1 -- 172.21.15.157:0/1266070449 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3833762342 0 0) 0x7f43c406e840 con 0x7f43c406e340 2023-10-06T04:10:56.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.657+0000 7f43c8955700 1 -- 172.21.15.157:0/1266070449 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c4152fd0 con 0x7f43c406a780 2023-10-06T04:10:56.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.657+0000 7f43c8955700 1 --2- 172.21.15.157:0/1266070449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43c406a780 0x7f43c4156bc0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f43c40c0ca0 tx=0x7f43c0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9cb98fa4425e837 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:56.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.657+0000 7f43bb7fe700 1 -- 172.21.15.157:0/1266070449 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c000bc80 con 0x7f43c406a780 2023-10-06T04:10:56.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.657+0000 7f43bb7fe700 1 -- 172.21.15.157:0/1266070449 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c000bde0 con 0x7f43c406a780 2023-10-06T04:10:56.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.657+0000 7f43bb7fe700 1 -- 172.21.15.157:0/1266070449 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c00116b0 con 0x7f43c406a780 2023-10-06T04:10:56.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.658+0000 7f43cabb9700 1 -- 172.21.15.157:0/1266070449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43c406a780 msgr2=0x7f43c4156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:56.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.658+0000 7f43cabb9700 1 --2- 172.21.15.157:0/1266070449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43c406a780 0x7f43c4156bc0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f43c40c0ca0 tx=0x7f43c0009780 comp rx=0 tx=0).stop 2023-10-06T04:10:56.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.658+0000 7f43cabb9700 1 -- 172.21.15.157:0/1266070449 shutdown_connections 2023-10-06T04:10:56.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.658+0000 7f43cabb9700 1 --2- 172.21.15.157:0/1266070449 >> v2:172.21.15.175:3300/0 conn(0x7f43c406ab50 0x7f43c40c2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:56.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.658+0000 7f43cabb9700 1 --2- 172.21.15.157:0/1266070449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43c406a780 0x7f43c4156bc0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:56.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.658+0000 7f43cabb9700 1 -- 172.21.15.157:0/1266070449 >> 172.21.15.157:0/1266070449 conn(0x7f43c406b510 msgr2=0x7f43c406b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:56.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.658+0000 7f43cabb9700 1 -- 172.21.15.157:0/1266070449 shutdown_connections 2023-10-06T04:10:56.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.658+0000 7f43cabb9700 1 -- 172.21.15.157:0/1266070449 wait complete. 2023-10-06T04:10:56.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.659+0000 7f43cabb9700 1 Processor -- start 2023-10-06T04:10:56.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.659+0000 7f43cabb9700 1 -- start start 2023-10-06T04:10:56.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.659+0000 7f43cabb9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43c406a780 0x7f43c414ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:56.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.659+0000 7f43cabb9700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43c406ab50 0x7f43c406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:56.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.660+0000 7f43cabb9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43c406e340 0x7f43c4153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:56.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.660+0000 7f43cabb9700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c4147560 con 0x7f43c406e340 2023-10-06T04:10:56.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.660+0000 7f43cabb9700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c4146f50 con 0x7f43c406a780 2023-10-06T04:10:56.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.660+0000 7f43cabb9700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c414e8a0 con 0x7f43c406ab50 2023-10-06T04:10:56.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.660+0000 7f43c9156700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43c406e340 0x7f43c4153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:56.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.660+0000 7f43c9156700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43c406e340 0x7f43c4153df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51570/0 (socket says 172.21.15.157:51570) 2023-10-06T04:10:56.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.660+0000 7f43c9156700 1 -- 172.21.15.157:0/1632578860 learned_addr learned my addr 172.21.15.157:0/1632578860 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:10:56.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.660+0000 7f43c8955700 1 --2- 172.21.15.157:0/1632578860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43c406a780 0x7f43c414ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:56.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.660+0000 7f43bbfff700 1 --2- 172.21.15.157:0/1632578860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43c406ab50 0x7f43c406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:56.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.660+0000 7f43c9156700 1 -- 172.21.15.157:0/1632578860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43c406ab50 msgr2=0x7f43c406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:56.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.660+0000 7f43c9156700 1 --2- 172.21.15.157:0/1632578860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43c406ab50 0x7f43c406d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:56.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.660+0000 7f43c9156700 1 -- 172.21.15.157:0/1632578860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43c406a780 msgr2=0x7f43c414ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:56.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.660+0000 7f43c9156700 1 --2- 172.21.15.157:0/1632578860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43c406a780 0x7f43c414ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:56.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.660+0000 7f43c9156700 1 -- 172.21.15.157:0/1632578860 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c0009a20 con 0x7f43c406e340 2023-10-06T04:10:56.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.660+0000 7f43c9156700 1 --2- 172.21.15.157:0/1632578860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43c406e340 0x7f43c4153df0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00d8f0 tx=0x7f43bc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:56.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.660+0000 7f43b8ff9700 1 -- 172.21.15.157:0/1632578860 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc0125f0 con 0x7f43c406e340 2023-10-06T04:10:56.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.660+0000 7f43b8ff9700 1 -- 172.21.15.157:0/1632578860 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43bc012770 con 0x7f43c406e340 2023-10-06T04:10:56.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.660+0000 7f43cabb9700 1 -- 172.21.15.157:0/1632578860 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c41504f0 con 0x7f43c406e340 2023-10-06T04:10:56.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.660+0000 7f43b8ff9700 1 -- 172.21.15.157:0/1632578860 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc012b30 con 0x7f43c406e340 2023-10-06T04:10:56.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.660+0000 7f43cabb9700 1 -- 172.21.15.157:0/1632578860 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c414e5c0 con 0x7f43c406e340 2023-10-06T04:10:56.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.661+0000 7f43cabb9700 1 -- 172.21.15.157:0/1632578860 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c4147090 con 0x7f43c406e340 2023-10-06T04:10:56.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.662+0000 7f43b8ff9700 1 -- 172.21.15.157:0/1632578860 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f43bc01e460 con 0x7f43c406e340 2023-10-06T04:10:56.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.662+0000 7f43b8ff9700 1 --2- 172.21.15.157:0/1632578860 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f43b0066f30 0x7f43b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:56.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.662+0000 7f43b8ff9700 1 -- 172.21.15.157:0/1632578860 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f43bc084ef0 con 0x7f43c406e340 2023-10-06T04:10:56.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.662+0000 7f43c8955700 1 --2- 172.21.15.157:0/1632578860 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f43b0066f30 0x7f43b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:56.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.662+0000 7f43c8955700 1 --2- 172.21.15.157:0/1632578860 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f43b0066f30 0x7f43b00693e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f43c00099f0 tx=0x7f43c001b000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:56.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.667+0000 7f43b8ff9700 1 -- 172.21.15.157:0/1632578860 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43c4147090 con 0x7f43c406e340 2023-10-06T04:10:56.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.851+0000 7f43cabb9700 1 -- 172.21.15.157:0/1632578860 --> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f43c4147090 con 0x7f43b0066f30 2023-10-06T04:10:56.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.862+0000 7f43b8ff9700 1 -- 172.21.15.157:0/1632578860 <== mgr.4107 v2:172.21.15.175:6832/90661 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+316198 (secure 0 0 0) 0x7f43c4147090 con 0x7f43b0066f30 2023-10-06T04:10:56.868 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:10:56.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.866+0000 7f43cabb9700 1 -- 172.21.15.157:0/1632578860 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f43b0066f30 msgr2=0x7f43b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:56.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.866+0000 7f43cabb9700 1 --2- 172.21.15.157:0/1632578860 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f43b0066f30 0x7f43b00693e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f43c00099f0 tx=0x7f43c001b000 comp rx=0 tx=0).stop 2023-10-06T04:10:56.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.866+0000 7f43cabb9700 1 -- 172.21.15.157:0/1632578860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43c406e340 msgr2=0x7f43c4153df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:56.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.866+0000 7f43cabb9700 1 --2- 172.21.15.157:0/1632578860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43c406e340 0x7f43c4153df0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00d8f0 tx=0x7f43bc008040 comp rx=0 tx=0).stop 2023-10-06T04:10:56.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.866+0000 7f43cabb9700 1 -- 172.21.15.157:0/1632578860 shutdown_connections 2023-10-06T04:10:56.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.866+0000 7f43cabb9700 1 --2- 172.21.15.157:0/1632578860 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f43b0066f30 0x7f43b00693e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:56.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.866+0000 7f43cabb9700 1 --2- 172.21.15.157:0/1632578860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43c406a780 0x7f43c414ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:56.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.866+0000 7f43cabb9700 1 --2- 172.21.15.157:0/1632578860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43c406e340 0x7f43c4153df0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:56.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.866+0000 7f43cabb9700 1 --2- 172.21.15.157:0/1632578860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43c406ab50 0x7f43c406d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:56.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.866+0000 7f43cabb9700 1 -- 172.21.15.157:0/1632578860 >> 172.21.15.157:0/1632578860 conn(0x7f43c406b510 msgr2=0x7f43c4151500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:56.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.866+0000 7f43cabb9700 1 -- 172.21.15.157:0/1632578860 shutdown_connections 2023-10-06T04:10:56.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:56.866+0000 7f43cabb9700 1 -- 172.21.15.157:0/1632578860 wait complete. 2023-10-06T04:10:56.872 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-10-06T04:10:56.884 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2023-10-06T04:10:55.645520+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1079653394,"num_objects":10047,"num_object_clones":0,"num_object_copies":20094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10047,"num_whiteouts":0,"num_read":2914,"num_read_kb":634649,"num_write":27342,"num_write_kb":1234647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":122,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27621,"ondisk_log_size":27621,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":4139684,"kb_used_data":1720420,"kb_used_omap":0,"kb_used_meta":2419200,"kb_avail":750835036,"statfs":{"total":773094113280,"available":768855076864,"internally_reserved":0,"allocated":1761710080,"data_stored":2160230755,"data_compressed":200974452,"data_compressed_allocated":228696064,"data_compressed_original":681320448,"omap_allocated":0,"internal_metadata":2477260800},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":170,"apply_latency_ms":170,"commit_latency_ns":170000000,"apply_latency_ns":170000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":117952278,"num_objects":3304,"num_object_clones":0,"num_object_copies":6608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3419,"num_write_kb":116453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004532"},"pg_stats":[{"pgid":"4.3f","version":"53'315","reported_seq":2411,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.016174+0000","last_change":"2023-10-06T04:05:03.151059+0000","last_active":"2023-10-06T04:10:48.016174+0000","last_peered":"2023-10-06T04:10:48.016174+0000","last_clean":"2023-10-06T04:10:48.016174+0000","last_became_active":"2023-10-06T03:57:11.371756+0000","last_became_peered":"2023-10-06T03:57:11.371756+0000","last_unstale":"2023-10-06T04:10:48.016174+0000","last_undegraded":"2023-10-06T04:10:48.016174+0000","last_fullsized":"2023-10-06T04:10:48.016174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:50.086528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001094783,"stat_sum":{"num_bytes":10567600,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":29,"num_read_kb":15,"num_write":301,"num_write_kb":10442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"53'337","reported_seq":2437,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.046233+0000","last_change":"2023-10-06T04:05:02.973547+0000","last_active":"2023-10-06T04:10:48.046233+0000","last_peered":"2023-10-06T04:10:48.046233+0000","last_clean":"2023-10-06T04:10:48.046233+0000","last_became_active":"2023-10-06T03:57:11.370070+0000","last_became_peered":"2023-10-06T03:57:11.370070+0000","last_unstale":"2023-10-06T04:10:48.046233+0000","last_undegraded":"2023-10-06T04:10:48.046233+0000","last_fullsized":"2023-10-06T04:10:48.046233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:00.917607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021773819999999998,"stat_sum":{"num_bytes":6441631,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":33,"num_read_kb":16,"num_write":327,"num_write_kb":6463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"53'333","reported_seq":2437,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.050065+0000","last_change":"2023-10-06T04:05:02.723796+0000","last_active":"2023-10-06T04:10:48.050065+0000","last_peered":"2023-10-06T04:10:48.050065+0000","last_clean":"2023-10-06T04:10:48.050065+0000","last_became_active":"2023-10-06T03:57:11.366012+0000","last_became_peered":"2023-10-06T03:57:11.366012+0000","last_unstale":"2023-10-06T04:10:48.050065+0000","last_undegraded":"2023-10-06T04:10:48.050065+0000","last_fullsized":"2023-10-06T04:10:48.050065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:45.802287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002722286,"stat_sum":{"num_bytes":14390705,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":40,"num_read_kb":20499,"num_write":313,"num_write_kb":22357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"53'294","reported_seq":2393,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.819279+0000","last_change":"2023-10-06T04:05:03.543119+0000","last_active":"2023-10-06T04:10:47.819279+0000","last_peered":"2023-10-06T04:10:47.819279+0000","last_clean":"2023-10-06T04:10:47.819279+0000","last_became_active":"2023-10-06T03:57:11.365673+0000","last_became_peered":"2023-10-06T03:57:11.365673+0000","last_unstale":"2023-10-06T04:10:47.819279+0000","last_undegraded":"2023-10-06T04:10:47.819279+0000","last_fullsized":"2023-10-06T04:10:47.819279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:40.953382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001947182,"stat_sum":{"num_bytes":14739251,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":32,"num_read_kb":7963,"num_write":286,"num_write_kb":14550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"53'323","reported_seq":2421,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.102698+0000","last_change":"2023-10-06T04:05:03.366556+0000","last_active":"2023-10-06T04:10:48.102698+0000","last_peered":"2023-10-06T04:10:48.102698+0000","last_clean":"2023-10-06T04:10:48.102698+0000","last_became_active":"2023-10-06T03:57:11.361302+0000","last_became_peered":"2023-10-06T03:57:11.361302+0000","last_unstale":"2023-10-06T04:10:48.102698+0000","last_undegraded":"2023-10-06T04:10:48.102698+0000","last_fullsized":"2023-10-06T04:10:48.102698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:57.552493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001966644,"stat_sum":{"num_bytes":2126805,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":31,"num_read_kb":18,"num_write":316,"num_write_kb":2215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"53'311","reported_seq":2412,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.048068+0000","last_change":"2023-10-06T04:05:02.723270+0000","last_active":"2023-10-06T04:10:48.048068+0000","last_peered":"2023-10-06T04:10:48.048068+0000","last_clean":"2023-10-06T04:10:48.048068+0000","last_became_active":"2023-10-06T03:57:11.368775+0000","last_became_peered":"2023-10-06T03:57:11.368775+0000","last_unstale":"2023-10-06T04:10:48.048068+0000","last_undegraded":"2023-10-06T04:10:48.048068+0000","last_fullsized":"2023-10-06T04:10:48.048068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:22.368117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002640482,"stat_sum":{"num_bytes":22420575,"num_objects":141,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":141,"num_whiteouts":0,"num_read":34,"num_read_kb":13627,"num_write":301,"num_write_kb":22056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"53'368","reported_seq":2467,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.038708+0000","last_change":"2023-10-06T04:05:03.343818+0000","last_active":"2023-10-06T04:10:48.038708+0000","last_peered":"2023-10-06T04:10:48.038708+0000","last_clean":"2023-10-06T04:10:48.038708+0000","last_became_active":"2023-10-06T03:57:11.377786+0000","last_became_peered":"2023-10-06T03:57:11.377786+0000","last_unstale":"2023-10-06T04:10:48.038708+0000","last_undegraded":"2023-10-06T04:10:48.038708+0000","last_fullsized":"2023-10-06T04:10:48.038708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:12:54.218593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093960500000000004,"stat_sum":{"num_bytes":18362974,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":32,"num_read_kb":8208,"num_write":354,"num_write_kb":18122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"53'303","reported_seq":2401,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.925102+0000","last_change":"2023-10-06T04:05:02.972640+0000","last_active":"2023-10-06T04:10:47.925102+0000","last_peered":"2023-10-06T04:10:47.925102+0000","last_clean":"2023-10-06T04:10:47.925102+0000","last_became_active":"2023-10-06T03:57:11.373765+0000","last_became_peered":"2023-10-06T03:57:11.373765+0000","last_unstale":"2023-10-06T04:10:47.925102+0000","last_undegraded":"2023-10-06T04:10:47.925102+0000","last_fullsized":"2023-10-06T04:10:47.925102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:05.838007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001873881,"stat_sum":{"num_bytes":10907247,"num_objects":139,"num_object_clones":0,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":0,"num_read":31,"num_read_kb":4107,"num_write":297,"num_write_kb":10730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"53'308","reported_seq":2415,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.091138+0000","last_change":"2023-10-06T04:05:03.367332+0000","last_active":"2023-10-06T04:10:48.091138+0000","last_peered":"2023-10-06T04:10:48.091138+0000","last_clean":"2023-10-06T04:10:48.091138+0000","last_became_active":"2023-10-06T03:57:11.368004+0000","last_became_peered":"2023-10-06T03:57:11.368004+0000","last_unstale":"2023-10-06T04:10:48.091138+0000","last_undegraded":"2023-10-06T04:10:48.091138+0000","last_fullsized":"2023-10-06T04:10:48.091138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:16.760049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024398710000000001,"stat_sum":{"num_bytes":13562133,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":41,"num_read_kb":4114,"num_write":303,"num_write_kb":17498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"53'324","reported_seq":2421,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.936467+0000","last_change":"2023-10-06T04:05:02.973579+0000","last_active":"2023-10-06T04:10:47.936467+0000","last_peered":"2023-10-06T04:10:47.936467+0000","last_clean":"2023-10-06T04:10:47.936467+0000","last_became_active":"2023-10-06T03:57:11.379107+0000","last_became_peered":"2023-10-06T03:57:11.379107+0000","last_unstale":"2023-10-06T04:10:47.936467+0000","last_undegraded":"2023-10-06T04:10:47.936467+0000","last_fullsized":"2023-10-06T04:10:47.936467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:18.373103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002039126,"stat_sum":{"num_bytes":9755625,"num_objects":142,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":30,"num_read_kb":4112,"num_write":320,"num_write_kb":9702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"53'355","reported_seq":2461,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.053956+0000","last_change":"2023-10-06T04:05:02.724475+0000","last_active":"2023-10-06T04:10:48.053956+0000","last_peered":"2023-10-06T04:10:48.053956+0000","last_clean":"2023-10-06T04:10:48.053956+0000","last_became_active":"2023-10-06T03:57:11.368256+0000","last_became_peered":"2023-10-06T03:57:11.368256+0000","last_unstale":"2023-10-06T04:10:48.053956+0000","last_undegraded":"2023-10-06T04:10:48.053956+0000","last_fullsized":"2023-10-06T04:10:48.053956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:50.736367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029138929999999999,"stat_sum":{"num_bytes":31118399,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":39,"num_read_kb":4114,"num_write":348,"num_write_kb":30570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"53'305","reported_seq":2402,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.932349+0000","last_change":"2023-10-06T04:05:03.151565+0000","last_active":"2023-10-06T04:10:47.932349+0000","last_peered":"2023-10-06T04:10:47.932349+0000","last_clean":"2023-10-06T04:10:47.932349+0000","last_became_active":"2023-10-06T03:57:11.368425+0000","last_became_peered":"2023-10-06T03:57:11.368425+0000","last_unstale":"2023-10-06T04:10:47.932349+0000","last_undegraded":"2023-10-06T04:10:47.932349+0000","last_fullsized":"2023-10-06T04:10:47.932349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:06.004423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014231059999999999,"stat_sum":{"num_bytes":10394744,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":30,"num_read_kb":4105,"num_write":299,"num_write_kb":14405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"53'322","reported_seq":2425,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.059723+0000","last_change":"2023-10-06T04:05:03.150960+0000","last_active":"2023-10-06T04:10:48.059723+0000","last_peered":"2023-10-06T04:10:48.059723+0000","last_clean":"2023-10-06T04:10:48.059723+0000","last_became_active":"2023-10-06T03:57:11.367079+0000","last_became_peered":"2023-10-06T03:57:11.367079+0000","last_unstale":"2023-10-06T04:10:48.059723+0000","last_undegraded":"2023-10-06T04:10:48.059723+0000","last_fullsized":"2023-10-06T04:10:48.059723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:09:54.331206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001042765,"stat_sum":{"num_bytes":14339914,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":37,"num_read_kb":12308,"num_write":311,"num_write_kb":22299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"53'317","reported_seq":2420,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.803651+0000","last_change":"2023-10-06T04:05:02.973393+0000","last_active":"2023-10-06T04:10:47.803651+0000","last_peered":"2023-10-06T04:10:47.803651+0000","last_clean":"2023-10-06T04:10:47.803651+0000","last_became_active":"2023-10-06T03:57:11.373216+0000","last_became_peered":"2023-10-06T03:57:11.373216+0000","last_unstale":"2023-10-06T04:10:47.803651+0000","last_undegraded":"2023-10-06T04:10:47.803651+0000","last_fullsized":"2023-10-06T04:10:47.803651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:56.083203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022456289999999999,"stat_sum":{"num_bytes":5695017,"num_objects":137,"num_object_clones":0,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":0,"num_read":36,"num_read_kb":18,"num_write":313,"num_write_kb":9820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"53'324","reported_seq":2421,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.038672+0000","last_change":"2023-10-06T04:05:03.343999+0000","last_active":"2023-10-06T04:10:48.038672+0000","last_peered":"2023-10-06T04:10:48.038672+0000","last_clean":"2023-10-06T04:10:48.038672+0000","last_became_active":"2023-10-06T03:57:11.375689+0000","last_became_peered":"2023-10-06T03:57:11.375689+0000","last_unstale":"2023-10-06T04:10:48.038672+0000","last_undegraded":"2023-10-06T04:10:48.038672+0000","last_fullsized":"2023-10-06T04:10:48.038672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:42.953739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092625599999999998,"stat_sum":{"num_bytes":15101939,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":316,"num_write_kb":14917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"53'331","reported_seq":2433,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.045818+0000","last_change":"2023-10-06T04:05:02.722778+0000","last_active":"2023-10-06T04:10:48.045818+0000","last_peered":"2023-10-06T04:10:48.045818+0000","last_clean":"2023-10-06T04:10:48.045818+0000","last_became_active":"2023-10-06T03:57:11.366543+0000","last_became_peered":"2023-10-06T03:57:11.366543+0000","last_unstale":"2023-10-06T04:10:48.045818+0000","last_undegraded":"2023-10-06T04:10:48.045818+0000","last_fullsized":"2023-10-06T04:10:48.045818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:16.212967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024276609999999998,"stat_sum":{"num_bytes":19805304,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":35,"num_read_kb":19,"num_write":327,"num_write_kb":22717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"53'342","reported_seq":2442,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.091750+0000","last_change":"2023-10-06T04:05:03.367754+0000","last_active":"2023-10-06T04:10:48.091750+0000","last_peered":"2023-10-06T04:10:48.091750+0000","last_clean":"2023-10-06T04:10:48.091750+0000","last_became_active":"2023-10-06T03:57:11.366767+0000","last_became_peered":"2023-10-06T03:57:11.366767+0000","last_unstale":"2023-10-06T04:10:48.091750+0000","last_undegraded":"2023-10-06T04:10:48.091750+0000","last_fullsized":"2023-10-06T04:10:48.091750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:43.113575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002777833,"stat_sum":{"num_bytes":14662095,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":34,"num_read_kb":8213,"num_write":330,"num_write_kb":18582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"53'343","reported_seq":2453,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.060004+0000","last_change":"2023-10-06T04:05:02.723910+0000","last_active":"2023-10-06T04:10:48.060004+0000","last_peered":"2023-10-06T04:10:48.060004+0000","last_clean":"2023-10-06T04:10:48.060004+0000","last_became_active":"2023-10-06T03:57:11.362377+0000","last_became_peered":"2023-10-06T03:57:11.362377+0000","last_unstale":"2023-10-06T04:10:48.060004+0000","last_undegraded":"2023-10-06T04:10:48.060004+0000","last_fullsized":"2023-10-06T04:10:48.060004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:40.600383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027606900000000001,"stat_sum":{"num_bytes":10867734,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":43,"num_read_kb":26,"num_write":329,"num_write_kb":10783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"53'340","reported_seq":2446,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.826978+0000","last_change":"2023-10-06T04:05:03.151735+0000","last_active":"2023-10-06T04:10:47.826978+0000","last_peered":"2023-10-06T04:10:47.826978+0000","last_clean":"2023-10-06T04:10:47.826978+0000","last_became_active":"2023-10-06T03:57:11.363792+0000","last_became_peered":"2023-10-06T03:57:11.363792+0000","last_unstale":"2023-10-06T04:10:47.826978+0000","last_undegraded":"2023-10-06T04:10:47.826978+0000","last_fullsized":"2023-10-06T04:10:47.826978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:49.185450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014611940000000001,"stat_sum":{"num_bytes":14638716,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":40,"num_read_kb":8209,"num_write":327,"num_write_kb":18563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"53'345","reported_seq":2448,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.037717+0000","last_change":"2023-10-06T04:05:02.724524+0000","last_active":"2023-10-06T04:10:48.037717+0000","last_peered":"2023-10-06T04:10:48.037717+0000","last_clean":"2023-10-06T04:10:48.037717+0000","last_became_active":"2023-10-06T03:57:11.364726+0000","last_became_peered":"2023-10-06T03:57:11.364726+0000","last_unstale":"2023-10-06T04:10:48.037717+0000","last_undegraded":"2023-10-06T04:10:48.037717+0000","last_fullsized":"2023-10-06T04:10:48.037717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:56.866733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029763670000000002,"stat_sum":{"num_bytes":20613561,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":37,"num_read_kb":12308,"num_write":341,"num_write_kb":28448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"53'320","reported_seq":2429,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.103111+0000","last_change":"2023-10-06T04:05:03.344002+0000","last_active":"2023-10-06T04:10:48.103111+0000","last_peered":"2023-10-06T04:10:48.103111+0000","last_clean":"2023-10-06T04:10:48.103111+0000","last_became_active":"2023-10-06T03:57:11.375668+0000","last_became_peered":"2023-10-06T03:57:11.375668+0000","last_unstale":"2023-10-06T04:10:48.103111+0000","last_undegraded":"2023-10-06T04:10:48.103111+0000","last_fullsized":"2023-10-06T04:10:48.103111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:53.810875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010285800000000001,"stat_sum":{"num_bytes":18956719,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":42,"num_read_kb":8215,"num_write":314,"num_write_kb":18647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"53'337","reported_seq":2438,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.891519+0000","last_change":"2023-10-06T04:05:03.366839+0000","last_active":"2023-10-06T04:10:47.891519+0000","last_peered":"2023-10-06T04:10:47.891519+0000","last_clean":"2023-10-06T04:10:47.891519+0000","last_became_active":"2023-10-06T03:57:11.365149+0000","last_became_peered":"2023-10-06T03:57:11.365149+0000","last_unstale":"2023-10-06T04:10:47.891519+0000","last_undegraded":"2023-10-06T04:10:47.891519+0000","last_fullsized":"2023-10-06T04:10:47.891519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:20.911710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020974879999999998,"stat_sum":{"num_bytes":5856786,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":34,"num_read_kb":17,"num_write":329,"num_write_kb":9988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"53'337","reported_seq":2445,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.097397+0000","last_change":"2023-10-06T04:05:02.722763+0000","last_active":"2023-10-06T04:10:48.097397+0000","last_peered":"2023-10-06T04:10:48.097397+0000","last_clean":"2023-10-06T04:10:48.097397+0000","last_became_active":"2023-10-06T03:57:11.362128+0000","last_became_peered":"2023-10-06T03:57:11.362128+0000","last_unstale":"2023-10-06T04:10:48.097397+0000","last_undegraded":"2023-10-06T04:10:48.097397+0000","last_fullsized":"2023-10-06T04:10:48.097397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:53.744674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025174860000000002,"stat_sum":{"num_bytes":9975853,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":42,"num_read_kb":8214,"num_write":313,"num_write_kb":13997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"53'363","reported_seq":2477,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.777222+0000","last_change":"2023-10-06T04:05:03.344184+0000","last_active":"2023-10-06T04:10:47.777222+0000","last_peered":"2023-10-06T04:10:47.777222+0000","last_clean":"2023-10-06T04:10:47.777222+0000","last_became_active":"2023-10-06T03:57:11.375702+0000","last_became_peered":"2023-10-06T03:57:11.375702+0000","last_unstale":"2023-10-06T04:10:47.777222+0000","last_undegraded":"2023-10-06T04:10:47.777222+0000","last_fullsized":"2023-10-06T04:10:47.777222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:31.204231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010337009999999999,"stat_sum":{"num_bytes":19963417,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":47,"num_read_kb":12104,"num_write":344,"num_write_kb":19673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"53'313","reported_seq":2409,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.117818+0000","last_change":"2023-10-06T04:05:02.718513+0000","last_active":"2023-10-06T04:10:48.117818+0000","last_peered":"2023-10-06T04:10:48.117818+0000","last_clean":"2023-10-06T04:10:48.117818+0000","last_became_active":"2023-10-06T03:57:11.374842+0000","last_became_peered":"2023-10-06T03:57:11.374842+0000","last_unstale":"2023-10-06T04:10:48.117818+0000","last_undegraded":"2023-10-06T04:10:48.117818+0000","last_fullsized":"2023-10-06T04:10:48.117818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:54.438974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001332342,"stat_sum":{"num_bytes":5720212,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":29,"num_read_kb":4115,"num_write":303,"num_write_kb":5752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"53'363","reported_seq":2463,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.116692+0000","last_change":"2023-10-06T04:05:02.718726+0000","last_active":"2023-10-06T04:10:48.116692+0000","last_peered":"2023-10-06T04:10:48.116692+0000","last_clean":"2023-10-06T04:10:48.116692+0000","last_became_active":"2023-10-06T03:57:11.361349+0000","last_became_peered":"2023-10-06T03:57:11.361349+0000","last_unstale":"2023-10-06T04:10:48.116692+0000","last_undegraded":"2023-10-06T04:10:48.116692+0000","last_fullsized":"2023-10-06T04:10:48.116692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:30.646781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014394589999999999,"stat_sum":{"num_bytes":22991444,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":33,"num_read_kb":4115,"num_write":351,"num_write_kb":22637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"53'300","reported_seq":2405,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.871145+0000","last_change":"2023-10-06T04:05:02.972918+0000","last_active":"2023-10-06T04:10:47.871145+0000","last_peered":"2023-10-06T04:10:47.871145+0000","last_clean":"2023-10-06T04:10:47.871145+0000","last_became_active":"2023-10-06T03:57:11.375846+0000","last_became_peered":"2023-10-06T03:57:11.375846+0000","last_unstale":"2023-10-06T04:10:47.871145+0000","last_undegraded":"2023-10-06T04:10:47.871145+0000","last_fullsized":"2023-10-06T04:10:47.871145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:28.080614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018480599999999999,"stat_sum":{"num_bytes":26260340,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":38,"num_read_kb":12302,"num_write":293,"num_write_kb":25795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"53'362","reported_seq":417,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:45.966579+0000","last_change":"2023-10-06T03:57:10.348317+0000","last_active":"2023-10-06T04:10:45.966579+0000","last_peered":"2023-10-06T04:10:45.966579+0000","last_clean":"2023-10-06T04:10:45.966579+0000","last_became_active":"2023-10-06T03:57:10.348078+0000","last_became_peered":"2023-10-06T03:57:10.348078+0000","last_unstale":"2023-10-06T04:10:45.966579+0000","last_undegraded":"2023-10-06T04:10:45.966579+0000","last_fullsized":"2023-10-06T04:10:45.966579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:55.058324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12621090,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":13,"num_read_kb":12332,"num_write":358,"num_write_kb":12535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"53'317","reported_seq":2422,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.966557+0000","last_change":"2023-10-06T04:05:02.722997+0000","last_active":"2023-10-06T04:10:47.966557+0000","last_peered":"2023-10-06T04:10:47.966557+0000","last_clean":"2023-10-06T04:10:47.966557+0000","last_became_active":"2023-10-06T03:57:11.370245+0000","last_became_peered":"2023-10-06T03:57:11.370245+0000","last_unstale":"2023-10-06T04:10:47.966557+0000","last_undegraded":"2023-10-06T04:10:47.966557+0000","last_fullsized":"2023-10-06T04:10:47.966557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:47:18.189772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002348424,"stat_sum":{"num_bytes":22206576,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":38,"num_read_kb":9495,"num_write":309,"num_write_kb":25946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"53'89","reported_seq":168,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:42.933757+0000","last_change":"2023-10-06T03:57:10.345401+0000","last_active":"2023-10-06T04:10:42.933757+0000","last_peered":"2023-10-06T04:10:42.933757+0000","last_clean":"2023-10-06T04:10:42.933757+0000","last_became_active":"2023-10-06T03:57:10.345150+0000","last_became_peered":"2023-10-06T03:57:10.345150+0000","last_unstale":"2023-10-06T04:10:42.933757+0000","last_undegraded":"2023-10-06T04:10:42.933757+0000","last_fullsized":"2023-10-06T04:10:42.933757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:25.758193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4312676,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":146,"num_write":89,"num_write_kb":4275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"53'333","reported_seq":2426,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.998937+0000","last_change":"2023-10-06T04:05:03.367558+0000","last_active":"2023-10-06T04:10:47.998937+0000","last_peered":"2023-10-06T04:10:47.998937+0000","last_clean":"2023-10-06T04:10:47.998937+0000","last_became_active":"2023-10-06T03:57:11.375790+0000","last_became_peered":"2023-10-06T03:57:11.375790+0000","last_unstale":"2023-10-06T04:10:47.998937+0000","last_undegraded":"2023-10-06T04:10:47.998937+0000","last_fullsized":"2023-10-06T04:10:47.998937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:33.775431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025999780000000002,"stat_sum":{"num_bytes":19057467,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":26,"num_read_kb":12300,"num_write":323,"num_write_kb":18781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"53'1073","reported_seq":1122,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:52.526921+0000","last_change":"2023-10-06T03:57:10.347562+0000","last_active":"2023-10-06T04:09:52.526921+0000","last_peered":"2023-10-06T04:09:52.526921+0000","last_clean":"2023-10-06T04:09:52.526921+0000","last_became_active":"2023-10-06T03:57:10.347294+0000","last_became_peered":"2023-10-06T03:57:10.347294+0000","last_unstale":"2023-10-06T04:09:52.526921+0000","last_undegraded":"2023-10-06T04:09:52.526921+0000","last_fullsized":"2023-10-06T04:09:52.526921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":1073,"log_dups_size":0,"ondisk_log_size":1073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:25.090992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583466,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":7,"num_read_kb":16784,"num_write":1067,"num_write_kb":17022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"53'349","reported_seq":2451,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.789556+0000","last_change":"2023-10-06T04:05:03.543192+0000","last_active":"2023-10-06T04:10:47.789556+0000","last_peered":"2023-10-06T04:10:47.789556+0000","last_clean":"2023-10-06T04:10:47.789556+0000","last_became_active":"2023-10-06T03:57:11.371758+0000","last_became_peered":"2023-10-06T03:57:11.371758+0000","last_unstale":"2023-10-06T04:10:47.789556+0000","last_undegraded":"2023-10-06T04:10:47.789556+0000","last_fullsized":"2023-10-06T04:10:47.789556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:10.922157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017888349999999999,"stat_sum":{"num_bytes":16828923,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":35,"num_read_kb":4115,"num_write":344,"num_write_kb":16563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"53'47","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:36.261131+0000","last_change":"2023-10-06T03:57:10.347406+0000","last_active":"2023-10-06T04:10:36.261131+0000","last_peered":"2023-10-06T04:10:36.261131+0000","last_clean":"2023-10-06T04:10:36.261131+0000","last_became_active":"2023-10-06T03:57:10.347151+0000","last_became_peered":"2023-10-06T03:57:10.347151+0000","last_unstale":"2023-10-06T04:10:36.261131+0000","last_undegraded":"2023-10-06T04:10:36.261131+0000","last_fullsized":"2023-10-06T04:10:36.261131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:20.427620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":50,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"53'312","reported_seq":2415,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.139381+0000","last_change":"2023-10-06T04:05:02.721771+0000","last_active":"2023-10-06T04:10:48.139381+0000","last_peered":"2023-10-06T04:10:48.139381+0000","last_clean":"2023-10-06T04:10:48.139381+0000","last_became_active":"2023-10-06T03:57:11.369938+0000","last_became_peered":"2023-10-06T03:57:11.369938+0000","last_unstale":"2023-10-06T04:10:48.139381+0000","last_undegraded":"2023-10-06T04:10:48.139381+0000","last_fullsized":"2023-10-06T04:10:48.139381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:07.105832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018579569999999999,"stat_sum":{"num_bytes":31421368,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":37,"num_read_kb":12302,"num_write":304,"num_write_kb":31090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"53'50","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:46.318091+0000","last_change":"2023-10-06T03:57:10.346054+0000","last_active":"2023-10-06T04:10:46.318091+0000","last_peered":"2023-10-06T04:10:46.318091+0000","last_clean":"2023-10-06T04:10:46.318091+0000","last_became_active":"2023-10-06T03:57:10.345721+0000","last_became_peered":"2023-10-06T03:57:10.345721+0000","last_unstale":"2023-10-06T04:10:46.318091+0000","last_undegraded":"2023-10-06T04:10:46.318091+0000","last_fullsized":"2023-10-06T04:10:46.318091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:54.481304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":15,"num_write":48,"num_write_kb":4121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"53'355","reported_seq":2455,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.868827+0000","last_change":"2023-10-06T04:05:03.151517+0000","last_active":"2023-10-06T04:10:47.868827+0000","last_peered":"2023-10-06T04:10:47.868827+0000","last_clean":"2023-10-06T04:10:47.868827+0000","last_became_active":"2023-10-06T03:57:59.255587+0000","last_became_peered":"2023-10-06T03:57:59.255587+0000","last_unstale":"2023-10-06T04:10:47.868827+0000","last_undegraded":"2023-10-06T04:10:47.868827+0000","last_fullsized":"2023-10-06T04:10:47.868827+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:08.261023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001510024,"stat_sum":{"num_bytes":1780876,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":38,"num_read_kb":8209,"num_write":338,"num_write_kb":10112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"53'63","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:42.932445+0000","last_change":"2023-10-06T03:57:10.340531+0000","last_active":"2023-10-06T04:10:42.932445+0000","last_peered":"2023-10-06T04:10:42.932445+0000","last_clean":"2023-10-06T04:10:42.932445+0000","last_became_active":"2023-10-06T03:57:10.340232+0000","last_became_peered":"2023-10-06T03:57:10.340232+0000","last_unstale":"2023-10-06T04:10:42.932445+0000","last_undegraded":"2023-10-06T04:10:42.932445+0000","last_fullsized":"2023-10-06T04:10:42.932445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:43:26.332055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":66,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"53'339","reported_seq":2443,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.117177+0000","last_change":"2023-10-06T04:05:02.718927+0000","last_active":"2023-10-06T04:10:48.117177+0000","last_peered":"2023-10-06T04:10:48.117177+0000","last_clean":"2023-10-06T04:10:48.117177+0000","last_became_active":"2023-10-06T03:57:11.374892+0000","last_became_peered":"2023-10-06T03:57:11.374892+0000","last_unstale":"2023-10-06T04:10:48.117177+0000","last_undegraded":"2023-10-06T04:10:48.117177+0000","last_fullsized":"2023-10-06T04:10:48.117177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:51.198745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00156207,"stat_sum":{"num_bytes":17752359,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":38,"num_read_kb":8211,"num_write":326,"num_write_kb":25695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"53'670","reported_seq":727,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:31.373637+0000","last_change":"2023-10-06T03:57:10.346758+0000","last_active":"2023-10-06T04:10:31.373637+0000","last_peered":"2023-10-06T04:10:31.373637+0000","last_clean":"2023-10-06T04:10:31.373637+0000","last_became_active":"2023-10-06T03:57:10.346430+0000","last_became_peered":"2023-10-06T03:57:10.346430+0000","last_unstale":"2023-10-06T04:10:31.373637+0000","last_undegraded":"2023-10-06T04:10:31.373637+0000","last_fullsized":"2023-10-06T04:10:31.373637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:10.809242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389252,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":30,"num_read_kb":9887,"num_write":668,"num_write_kb":8554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"53'323","reported_seq":2428,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.032035+0000","last_change":"2023-10-06T04:05:02.723446+0000","last_active":"2023-10-06T04:10:48.032035+0000","last_peered":"2023-10-06T04:10:48.032035+0000","last_clean":"2023-10-06T04:10:48.032035+0000","last_became_active":"2023-10-06T03:57:11.367565+0000","last_became_peered":"2023-10-06T03:57:11.367565+0000","last_unstale":"2023-10-06T04:10:48.032035+0000","last_undegraded":"2023-10-06T04:10:48.032035+0000","last_fullsized":"2023-10-06T04:10:48.032035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:14.409116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027762680000000001,"stat_sum":{"num_bytes":10222697,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":38,"num_read_kb":23,"num_write":314,"num_write_kb":10145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"53'179","reported_seq":235,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:16.251094+0000","last_change":"2023-10-06T03:57:10.351062+0000","last_active":"2023-10-06T04:10:16.251094+0000","last_peered":"2023-10-06T04:10:16.251094+0000","last_clean":"2023-10-06T04:10:16.251094+0000","last_became_active":"2023-10-06T03:57:10.350795+0000","last_became_peered":"2023-10-06T03:57:10.350795+0000","last_unstale":"2023-10-06T04:10:16.251094+0000","last_undegraded":"2023-10-06T04:10:16.251094+0000","last_fullsized":"2023-10-06T04:10:16.251094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:54:45.714518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388676,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":30,"num_read_kb":20509,"num_write":184,"num_write_kb":8306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"53'276","reported_seq":2375,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.006307+0000","last_change":"2023-10-06T04:05:02.722868+0000","last_active":"2023-10-06T04:10:48.006307+0000","last_peered":"2023-10-06T04:10:48.006307+0000","last_clean":"2023-10-06T04:10:48.006307+0000","last_became_active":"2023-10-06T03:57:11.367254+0000","last_became_peered":"2023-10-06T03:57:11.367254+0000","last_unstale":"2023-10-06T04:10:48.006307+0000","last_undegraded":"2023-10-06T04:10:48.006307+0000","last_fullsized":"2023-10-06T04:10:48.006307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:04.706485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027822569999999998,"stat_sum":{"num_bytes":13638644,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":32,"num_read_kb":10530,"num_write":271,"num_write_kb":17563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"53'105","reported_seq":157,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:23.094836+0000","last_change":"2023-10-06T03:57:10.338125+0000","last_active":"2023-10-06T04:10:23.094836+0000","last_peered":"2023-10-06T04:10:23.094836+0000","last_clean":"2023-10-06T04:10:23.094836+0000","last_became_active":"2023-10-06T03:57:10.337798+0000","last_became_peered":"2023-10-06T03:57:10.337798+0000","last_unstale":"2023-10-06T04:10:23.094836+0000","last_undegraded":"2023-10-06T04:10:23.094836+0000","last_fullsized":"2023-10-06T04:10:23.094836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:00.482657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":9753,"num_write":104,"num_write_kb":4165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"53'329","reported_seq":2433,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.868704+0000","last_change":"2023-10-06T04:05:02.973084+0000","last_active":"2023-10-06T04:10:47.868704+0000","last_peered":"2023-10-06T04:10:47.868704+0000","last_clean":"2023-10-06T04:10:47.868704+0000","last_became_active":"2023-10-06T03:57:11.376223+0000","last_became_peered":"2023-10-06T03:57:11.376223+0000","last_unstale":"2023-10-06T04:10:47.868704+0000","last_undegraded":"2023-10-06T04:10:47.868704+0000","last_fullsized":"2023-10-06T04:10:47.868704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:04:37.408871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018161119999999999,"stat_sum":{"num_bytes":34899441,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":37,"num_read_kb":11570,"num_write":325,"num_write_kb":34246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"53'97","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:36.263633+0000","last_change":"2023-10-06T03:57:10.350511+0000","last_active":"2023-10-06T04:10:36.263633+0000","last_peered":"2023-10-06T04:10:36.263633+0000","last_clean":"2023-10-06T04:10:36.263633+0000","last_became_active":"2023-10-06T03:57:10.350219+0000","last_became_peered":"2023-10-06T03:57:10.350219+0000","last_unstale":"2023-10-06T04:10:36.263633+0000","last_undegraded":"2023-10-06T04:10:36.263633+0000","last_fullsized":"2023-10-06T04:10:36.263633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:56.043506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96,"num_write_kb":8294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"53'359","reported_seq":2468,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.062653+0000","last_change":"2023-10-06T04:05:03.367352+0000","last_active":"2023-10-06T04:10:48.062653+0000","last_peered":"2023-10-06T04:10:48.062653+0000","last_clean":"2023-10-06T04:10:48.062653+0000","last_became_active":"2023-10-06T03:57:11.362443+0000","last_became_peered":"2023-10-06T03:57:11.362443+0000","last_unstale":"2023-10-06T04:10:48.062653+0000","last_undegraded":"2023-10-06T04:10:48.062653+0000","last_fullsized":"2023-10-06T04:10:48.062653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:21:09.143085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002448839,"stat_sum":{"num_bytes":22757129,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":42,"num_read_kb":8288,"num_write":351,"num_write_kb":22409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"53'71","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:36.258463+0000","last_change":"2023-10-06T03:57:10.346259+0000","last_active":"2023-10-06T04:10:36.258463+0000","last_peered":"2023-10-06T04:10:36.258463+0000","last_clean":"2023-10-06T04:10:36.258463+0000","last_became_active":"2023-10-06T03:57:10.345424+0000","last_became_peered":"2023-10-06T03:57:10.345424+0000","last_unstale":"2023-10-06T04:10:36.258463+0000","last_undegraded":"2023-10-06T04:10:36.258463+0000","last_fullsized":"2023-10-06T04:10:36.258463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:19.278206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":33,"num_read_kb":33,"num_write":72,"num_write_kb":4158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"53'304","reported_seq":2409,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.152945+0000","last_change":"2023-10-06T04:05:02.723050+0000","last_active":"2023-10-06T04:10:48.152945+0000","last_peered":"2023-10-06T04:10:48.152945+0000","last_clean":"2023-10-06T04:10:48.152945+0000","last_became_active":"2023-10-06T03:57:11.375824+0000","last_became_peered":"2023-10-06T03:57:11.375824+0000","last_unstale":"2023-10-06T04:10:48.152945+0000","last_undegraded":"2023-10-06T04:10:48.152945+0000","last_fullsized":"2023-10-06T04:10:48.152945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:24:26.156866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023200299999999998,"stat_sum":{"num_bytes":9687154,"num_objects":142,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":38,"num_read_kb":8212,"num_write":296,"num_write_kb":9609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"53'250","reported_seq":295,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:23.097110+0000","last_change":"2023-10-06T03:57:10.350408+0000","last_active":"2023-10-06T04:10:23.097110+0000","last_peered":"2023-10-06T04:10:23.097110+0000","last_clean":"2023-10-06T04:10:23.097110+0000","last_became_active":"2023-10-06T03:57:10.350135+0000","last_became_peered":"2023-10-06T03:57:10.350135+0000","last_unstale":"2023-10-06T04:10:23.097110+0000","last_undegraded":"2023-10-06T04:10:23.097110+0000","last_fullsized":"2023-10-06T04:10:23.097110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:43.522111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":4098,"num_write":280,"num_write_kb":4293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"53'321","reported_seq":2420,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.862961+0000","last_change":"2023-10-06T04:05:02.972356+0000","last_active":"2023-10-06T04:10:47.862961+0000","last_peered":"2023-10-06T04:10:47.862961+0000","last_clean":"2023-10-06T04:10:47.862961+0000","last_became_active":"2023-10-06T03:57:11.376236+0000","last_became_peered":"2023-10-06T03:57:11.376236+0000","last_unstale":"2023-10-06T04:10:47.862961+0000","last_undegraded":"2023-10-06T04:10:47.862961+0000","last_fullsized":"2023-10-06T04:10:47.862961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:33.854818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015573080000000001,"stat_sum":{"num_bytes":22874532,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":32,"num_read_kb":10317,"num_write":312,"num_write_kb":26593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"50'92","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:52.580642+0000","last_change":"2023-10-06T03:57:10.345799+0000","last_active":"2023-10-06T04:09:52.580642+0000","last_peered":"2023-10-06T04:09:52.580642+0000","last_clean":"2023-10-06T04:09:52.580642+0000","last_became_active":"2023-10-06T03:57:10.345510+0000","last_became_peered":"2023-10-06T03:57:10.345510+0000","last_unstale":"2023-10-06T04:09:52.580642+0000","last_undegraded":"2023-10-06T04:09:52.580642+0000","last_fullsized":"2023-10-06T04:09:52.580642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:09.567314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":4108,"num_write":90,"num_write_kb":4140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"53'342","reported_seq":2446,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.072619+0000","last_change":"2023-10-06T04:05:02.721917+0000","last_active":"2023-10-06T04:10:48.072619+0000","last_peered":"2023-10-06T04:10:48.072619+0000","last_clean":"2023-10-06T04:10:48.072619+0000","last_became_active":"2023-10-06T03:57:11.368375+0000","last_became_peered":"2023-10-06T03:57:11.368375+0000","last_unstale":"2023-10-06T04:10:48.072619+0000","last_undegraded":"2023-10-06T04:10:48.072619+0000","last_fullsized":"2023-10-06T04:10:48.072619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:27.125523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001632912,"stat_sum":{"num_bytes":5549566,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":37,"num_read_kb":4115,"num_write":335,"num_write_kb":5549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"53'164","reported_seq":212,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:22.809255+0000","last_change":"2023-10-06T03:57:10.346945+0000","last_active":"2023-10-06T04:10:22.809255+0000","last_peered":"2023-10-06T04:10:22.809255+0000","last_clean":"2023-10-06T04:10:22.809255+0000","last_became_active":"2023-10-06T03:57:10.346689+0000","last_became_peered":"2023-10-06T03:57:10.346689+0000","last_unstale":"2023-10-06T04:10:22.809255+0000","last_undegraded":"2023-10-06T04:10:22.809255+0000","last_fullsized":"2023-10-06T04:10:22.809255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:43.420455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":12290,"num_write":163,"num_write_kb":8336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"53'349","reported_seq":2450,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.892654+0000","last_change":"2023-10-06T04:05:03.542774+0000","last_active":"2023-10-06T04:10:47.892654+0000","last_peered":"2023-10-06T04:10:47.892654+0000","last_clean":"2023-10-06T04:10:47.892654+0000","last_became_active":"2023-10-06T03:57:11.371795+0000","last_became_peered":"2023-10-06T03:57:11.371795+0000","last_unstale":"2023-10-06T04:10:47.892654+0000","last_undegraded":"2023-10-06T04:10:47.892654+0000","last_fullsized":"2023-10-06T04:10:47.892654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:00.324478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016252980000000001,"stat_sum":{"num_bytes":5725462,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":34,"num_read_kb":13,"num_write":343,"num_write_kb":5770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"53'54","reported_seq":96,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:48.081196+0000","last_change":"2023-10-06T03:57:10.348592+0000","last_active":"2023-10-06T04:09:48.081196+0000","last_peered":"2023-10-06T04:09:48.081196+0000","last_clean":"2023-10-06T04:09:48.081196+0000","last_became_active":"2023-10-06T03:57:10.348291+0000","last_became_peered":"2023-10-06T03:57:10.348291+0000","last_unstale":"2023-10-06T04:09:48.081196+0000","last_undegraded":"2023-10-06T04:09:48.081196+0000","last_fullsized":"2023-10-06T04:09:48.081196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:00.660348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":57,"num_write_kb":4195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"53'308","reported_seq":2408,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.055840+0000","last_change":"2023-10-06T04:05:03.366405+0000","last_active":"2023-10-06T04:10:48.055840+0000","last_peered":"2023-10-06T04:10:48.055840+0000","last_clean":"2023-10-06T04:10:48.055840+0000","last_became_active":"2023-10-06T03:57:11.365077+0000","last_became_peered":"2023-10-06T03:57:11.365077+0000","last_unstale":"2023-10-06T04:10:48.055840+0000","last_undegraded":"2023-10-06T04:10:48.055840+0000","last_fullsized":"2023-10-06T04:10:48.055840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:25.199391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019352270000000001,"stat_sum":{"num_bytes":15027095,"num_objects":146,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":146,"num_whiteouts":0,"num_read":33,"num_read_kb":4109,"num_write":302,"num_write_kb":14833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"52'278","reported_seq":329,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:51.781455+0000","last_change":"2023-10-06T03:57:10.348418+0000","last_active":"2023-10-06T04:09:51.781455+0000","last_peered":"2023-10-06T04:09:51.781455+0000","last_clean":"2023-10-06T04:09:51.781455+0000","last_became_active":"2023-10-06T03:57:10.348055+0000","last_became_peered":"2023-10-06T03:57:10.348055+0000","last_unstale":"2023-10-06T04:09:51.781455+0000","last_undegraded":"2023-10-06T04:09:51.781455+0000","last_fullsized":"2023-10-06T04:09:51.781455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:26.102022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":17,"num_read_kb":12301,"num_write":272,"num_write_kb":8316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"53'342","reported_seq":2437,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.070418+0000","last_change":"2023-10-06T04:05:02.724303+0000","last_active":"2023-10-06T04:10:48.070418+0000","last_peered":"2023-10-06T04:10:48.070418+0000","last_clean":"2023-10-06T04:10:48.070418+0000","last_became_active":"2023-10-06T03:57:11.370073+0000","last_became_peered":"2023-10-06T03:57:11.370073+0000","last_unstale":"2023-10-06T04:10:48.070418+0000","last_undegraded":"2023-10-06T04:10:48.070418+0000","last_fullsized":"2023-10-06T04:10:48.070418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:58.417951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027454179999999999,"stat_sum":{"num_bytes":18012929,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":28,"num_read_kb":4110,"num_write":339,"num_write_kb":17768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"53'112","reported_seq":159,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:46.295859+0000","last_change":"2023-10-06T03:57:10.352966+0000","last_active":"2023-10-06T04:10:46.295859+0000","last_peered":"2023-10-06T04:10:46.295859+0000","last_clean":"2023-10-06T04:10:46.295859+0000","last_became_active":"2023-10-06T03:57:10.352653+0000","last_became_peered":"2023-10-06T03:57:10.352653+0000","last_unstale":"2023-10-06T04:10:46.295859+0000","last_undegraded":"2023-10-06T04:10:46.295859+0000","last_fullsized":"2023-10-06T04:10:46.295859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:32.092557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":8,"num_read_kb":4217,"num_write":204,"num_write_kb":4775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"53'338","reported_seq":2437,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.039511+0000","last_change":"2023-10-06T04:05:02.724113+0000","last_active":"2023-10-06T04:10:48.039511+0000","last_peered":"2023-10-06T04:10:48.039511+0000","last_clean":"2023-10-06T04:10:48.039511+0000","last_became_active":"2023-10-06T03:57:11.363830+0000","last_became_peered":"2023-10-06T03:57:11.363830+0000","last_unstale":"2023-10-06T04:10:48.039511+0000","last_undegraded":"2023-10-06T04:10:48.039511+0000","last_fullsized":"2023-10-06T04:10:48.039511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:04.084019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027698559999999998,"stat_sum":{"num_bytes":22850224,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":32,"num_read_kb":4115,"num_write":329,"num_write_kb":26589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"53'32","reported_seq":82,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:50.963938+0000","last_change":"2023-10-06T03:57:10.349421+0000","last_active":"2023-10-06T04:10:50.963938+0000","last_peered":"2023-10-06T04:10:50.963938+0000","last_clean":"2023-10-06T04:10:50.963938+0000","last_became_active":"2023-10-06T03:57:10.349114+0000","last_became_peered":"2023-10-06T03:57:10.349114+0000","last_unstale":"2023-10-06T04:10:50.963938+0000","last_undegraded":"2023-10-06T04:10:50.963938+0000","last_fullsized":"2023-10-06T04:10:50.963938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:08.209791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2605665,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":19,"num_read_kb":18,"num_write":36,"num_write_kb":2597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"53'278","reported_seq":2370,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.045140+0000","last_change":"2023-10-06T04:05:02.724241+0000","last_active":"2023-10-06T04:10:48.045140+0000","last_peered":"2023-10-06T04:10:48.045140+0000","last_clean":"2023-10-06T04:10:48.045140+0000","last_became_active":"2023-10-06T03:57:11.367257+0000","last_became_peered":"2023-10-06T03:57:11.367257+0000","last_unstale":"2023-10-06T04:10:48.045140+0000","last_undegraded":"2023-10-06T04:10:48.045140+0000","last_fullsized":"2023-10-06T04:10:48.045140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:43.628819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002803224,"stat_sum":{"num_bytes":18462781,"num_objects":138,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":138,"num_whiteouts":0,"num_read":26,"num_read_kb":4106,"num_write":268,"num_write_kb":22268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"53'53","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:52.265701+0000","last_change":"2023-10-06T03:57:10.347649+0000","last_active":"2023-10-06T04:09:52.265701+0000","last_peered":"2023-10-06T04:09:52.265701+0000","last_clean":"2023-10-06T04:09:52.265701+0000","last_became_active":"2023-10-06T03:57:10.347433+0000","last_became_peered":"2023-10-06T03:57:10.347433+0000","last_unstale":"2023-10-06T04:09:52.265701+0000","last_undegraded":"2023-10-06T04:09:52.265701+0000","last_fullsized":"2023-10-06T04:09:52.265701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:55.951494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":11,"num_read_kb":8201,"num_write":54,"num_write_kb":4137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"53'320","reported_seq":2421,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.122912+0000","last_change":"2023-10-06T04:05:02.718759+0000","last_active":"2023-10-06T04:10:48.122912+0000","last_peered":"2023-10-06T04:10:48.122912+0000","last_clean":"2023-10-06T04:10:48.122912+0000","last_became_active":"2023-10-06T03:57:11.369825+0000","last_became_peered":"2023-10-06T03:57:11.369825+0000","last_unstale":"2023-10-06T04:10:48.122912+0000","last_undegraded":"2023-10-06T04:10:48.122912+0000","last_fullsized":"2023-10-06T04:10:48.122912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:43.415929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014204689999999999,"stat_sum":{"num_bytes":11370958,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":35,"num_read_kb":8210,"num_write":305,"num_write_kb":15362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:37.659432+0000","last_change":"2023-10-06T03:57:05.663312+0000","last_active":"2023-10-06T04:08:37.659432+0000","last_peered":"2023-10-06T04:08:37.659432+0000","last_clean":"2023-10-06T04:08:37.659432+0000","last_became_active":"2023-10-06T03:57:05.663046+0000","last_became_peered":"2023-10-06T03:57:05.663046+0000","last_unstale":"2023-10-06T04:08:37.659432+0000","last_undegraded":"2023-10-06T04:08:37.659432+0000","last_fullsized":"2023-10-06T04:08:37.659432+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:31.174359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"53'33","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:51.495313+0000","last_change":"2023-10-06T03:57:10.348930+0000","last_active":"2023-10-06T04:10:51.495313+0000","last_peered":"2023-10-06T04:10:51.495313+0000","last_clean":"2023-10-06T04:10:51.495313+0000","last_became_active":"2023-10-06T03:57:10.348608+0000","last_became_peered":"2023-10-06T03:57:10.348608+0000","last_unstale":"2023-10-06T04:10:51.495313+0000","last_undegraded":"2023-10-06T04:10:51.495313+0000","last_fullsized":"2023-10-06T04:10:51.495313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:00:09.283718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2070221,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":34,"num_write_kb":2062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"53'287","reported_seq":2383,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.110309+0000","last_change":"2023-10-06T04:05:02.718555+0000","last_active":"2023-10-06T04:10:48.110309+0000","last_peered":"2023-10-06T04:10:48.110309+0000","last_clean":"2023-10-06T04:10:48.110309+0000","last_became_active":"2023-10-06T03:57:11.370847+0000","last_became_peered":"2023-10-06T03:57:11.370847+0000","last_unstale":"2023-10-06T04:10:48.110309+0000","last_undegraded":"2023-10-06T04:10:48.110309+0000","last_fullsized":"2023-10-06T04:10:48.110309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:38.111171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013685990000000001,"stat_sum":{"num_bytes":14898272,"num_objects":129,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":129,"num_whiteouts":0,"num_read":29,"num_read_kb":12301,"num_write":279,"num_write_kb":18790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:37.658819+0000","last_change":"2023-10-06T03:57:05.318834+0000","last_active":"2023-10-06T04:08:37.658819+0000","last_peered":"2023-10-06T04:08:37.658819+0000","last_clean":"2023-10-06T04:08:37.658819+0000","last_became_active":"2023-10-06T03:57:05.318572+0000","last_became_peered":"2023-10-06T03:57:05.318572+0000","last_unstale":"2023-10-06T04:08:37.658819+0000","last_undegraded":"2023-10-06T04:08:37.658819+0000","last_fullsized":"2023-10-06T04:08:37.658819+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:21.951095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"53'10","reported_seq":60,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:46.769558+0000","last_change":"2023-10-06T03:57:10.348556+0000","last_active":"2023-10-06T04:09:46.769558+0000","last_peered":"2023-10-06T04:09:46.769558+0000","last_clean":"2023-10-06T04:09:46.769558+0000","last_became_active":"2023-10-06T03:57:10.348271+0000","last_became_peered":"2023-10-06T03:57:10.348271+0000","last_unstale":"2023-10-06T04:09:46.769558+0000","last_undegraded":"2023-10-06T04:09:46.769558+0000","last_fullsized":"2023-10-06T04:09:46.769558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:58.470989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":26,"num_write":12,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"53'320","reported_seq":2420,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.012974+0000","last_change":"2023-10-06T04:05:02.719009+0000","last_active":"2023-10-06T04:10:48.012974+0000","last_peered":"2023-10-06T04:10:48.012974+0000","last_clean":"2023-10-06T04:10:48.012974+0000","last_became_active":"2023-10-06T03:57:11.365006+0000","last_became_peered":"2023-10-06T03:57:11.365006+0000","last_unstale":"2023-10-06T04:10:48.012974+0000","last_undegraded":"2023-10-06T04:10:48.012974+0000","last_fullsized":"2023-10-06T04:10:48.012974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:33.005015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015961910000000001,"stat_sum":{"num_bytes":5694351,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":33,"num_read_kb":4109,"num_write":313,"num_write_kb":5725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:37.658115+0000","last_change":"2023-10-06T03:57:05.314695+0000","last_active":"2023-10-06T04:08:37.658115+0000","last_peered":"2023-10-06T04:08:37.658115+0000","last_clean":"2023-10-06T04:08:37.658115+0000","last_became_active":"2023-10-06T03:57:05.314408+0000","last_became_peered":"2023-10-06T03:57:05.314408+0000","last_unstale":"2023-10-06T04:08:37.658115+0000","last_undegraded":"2023-10-06T04:08:37.658115+0000","last_fullsized":"2023-10-06T04:08:37.658115+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:19.042145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"53'359","reported_seq":405,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:16.606829+0000","last_change":"2023-10-06T03:57:10.346078+0000","last_active":"2023-10-06T04:10:16.606829+0000","last_peered":"2023-10-06T04:10:16.606829+0000","last_clean":"2023-10-06T04:10:16.606829+0000","last_became_active":"2023-10-06T03:57:10.345769+0000","last_became_peered":"2023-10-06T03:57:10.345769+0000","last_unstale":"2023-10-06T04:10:16.606829+0000","last_undegraded":"2023-10-06T04:10:16.606829+0000","last_fullsized":"2023-10-06T04:10:16.606829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:16.630681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":352,"num_write_kb":12528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"53'315","reported_seq":2421,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.123122+0000","last_change":"2023-10-06T04:05:02.718334+0000","last_active":"2023-10-06T04:10:48.123122+0000","last_peered":"2023-10-06T04:10:48.123122+0000","last_clean":"2023-10-06T04:10:48.123122+0000","last_became_active":"2023-10-06T03:57:11.370076+0000","last_became_peered":"2023-10-06T03:57:11.370076+0000","last_unstale":"2023-10-06T04:10:48.123122+0000","last_undegraded":"2023-10-06T04:10:48.123122+0000","last_fullsized":"2023-10-06T04:10:48.123122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:18:59.517155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011772849999999999,"stat_sum":{"num_bytes":18508624,"num_objects":141,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":141,"num_whiteouts":0,"num_read":39,"num_read_kb":4112,"num_write":305,"num_write_kb":22326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:37.658686+0000","last_change":"2023-10-06T03:57:05.318778+0000","last_active":"2023-10-06T04:08:37.658686+0000","last_peered":"2023-10-06T04:08:37.658686+0000","last_clean":"2023-10-06T04:08:37.658686+0000","last_became_active":"2023-10-06T03:57:05.318456+0000","last_became_peered":"2023-10-06T03:57:05.318456+0000","last_unstale":"2023-10-06T04:08:37.658686+0000","last_undegraded":"2023-10-06T04:08:37.658686+0000","last_fullsized":"2023-10-06T04:08:37.658686+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:39.769241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":195,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:36.756245+0000","last_change":"2023-10-06T03:57:05.664245+0000","last_active":"2023-10-06T04:08:36.756245+0000","last_peered":"2023-10-06T04:08:36.756245+0000","last_clean":"2023-10-06T04:08:36.756245+0000","last_became_active":"2023-10-06T03:57:05.663426+0000","last_became_peered":"2023-10-06T03:57:05.663426+0000","last_unstale":"2023-10-06T04:08:36.756245+0000","last_undegraded":"2023-10-06T04:08:36.756245+0000","last_fullsized":"2023-10-06T04:08:36.756245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:29.322599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"53'719","reported_seq":770,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:52.436682+0000","last_change":"2023-10-06T03:57:10.351059+0000","last_active":"2023-10-06T04:09:52.436682+0000","last_peered":"2023-10-06T04:09:52.436682+0000","last_clean":"2023-10-06T04:09:52.436682+0000","last_became_active":"2023-10-06T03:57:10.348879+0000","last_became_peered":"2023-10-06T03:57:10.348879+0000","last_unstale":"2023-10-06T04:09:52.436682+0000","last_undegraded":"2023-10-06T04:09:52.436682+0000","last_fullsized":"2023-10-06T04:09:52.436682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:35.764148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9487985,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":20487,"num_write":715,"num_write_kb":13783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"53'332","reported_seq":2435,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.889473+0000","last_change":"2023-10-06T04:05:03.542987+0000","last_active":"2023-10-06T04:10:47.889473+0000","last_peered":"2023-10-06T04:10:47.889473+0000","last_clean":"2023-10-06T04:10:47.889473+0000","last_became_active":"2023-10-06T03:57:11.373452+0000","last_became_peered":"2023-10-06T03:57:11.373452+0000","last_unstale":"2023-10-06T04:10:47.889473+0000","last_undegraded":"2023-10-06T04:10:47.889473+0000","last_fullsized":"2023-10-06T04:10:47.889473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:20.524065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018153620000000001,"stat_sum":{"num_bytes":5826606,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":36,"num_read_kb":2733,"num_write":325,"num_write_kb":9961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:36.756325+0000","last_change":"2023-10-06T03:57:05.318790+0000","last_active":"2023-10-06T04:08:36.756325+0000","last_peered":"2023-10-06T04:08:36.756325+0000","last_clean":"2023-10-06T04:08:36.756325+0000","last_became_active":"2023-10-06T03:57:05.318531+0000","last_became_peered":"2023-10-06T03:57:05.318531+0000","last_unstale":"2023-10-06T04:08:36.756325+0000","last_undegraded":"2023-10-06T04:08:36.756325+0000","last_fullsized":"2023-10-06T04:08:36.756325+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:12.876998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"52'540","reported_seq":703,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:46.766101+0000","last_change":"2023-10-06T03:57:10.343328+0000","last_active":"2023-10-06T04:09:46.766101+0000","last_peered":"2023-10-06T04:09:46.766101+0000","last_clean":"2023-10-06T04:09:46.766101+0000","last_became_active":"2023-10-06T03:57:10.343051+0000","last_became_peered":"2023-10-06T03:57:10.343051+0000","last_unstale":"2023-10-06T04:09:46.766101+0000","last_undegraded":"2023-10-06T04:09:46.766101+0000","last_fullsized":"2023-10-06T04:09:46.766101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:27.722475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4571457,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":122,"num_read_kb":3989,"num_write":534,"num_write_kb":4895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"53'333","reported_seq":2437,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.058057+0000","last_change":"2023-10-06T04:05:03.367824+0000","last_active":"2023-10-06T04:10:48.058057+0000","last_peered":"2023-10-06T04:10:48.058057+0000","last_clean":"2023-10-06T04:10:48.058057+0000","last_became_active":"2023-10-06T03:57:11.373264+0000","last_became_peered":"2023-10-06T03:57:11.373264+0000","last_unstale":"2023-10-06T04:10:48.058057+0000","last_undegraded":"2023-10-06T04:10:48.058057+0000","last_fullsized":"2023-10-06T04:10:48.058057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:15.950245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026908990000000001,"stat_sum":{"num_bytes":10595820,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":37,"num_read_kb":4121,"num_write":327,"num_write_kb":10522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:36.758524+0000","last_change":"2023-10-06T03:57:05.663770+0000","last_active":"2023-10-06T04:08:36.758524+0000","last_peered":"2023-10-06T04:08:36.758524+0000","last_clean":"2023-10-06T04:08:36.758524+0000","last_became_active":"2023-10-06T03:57:05.663501+0000","last_became_peered":"2023-10-06T03:57:05.663501+0000","last_unstale":"2023-10-06T04:08:36.758524+0000","last_undegraded":"2023-10-06T04:08:36.758524+0000","last_fullsized":"2023-10-06T04:08:36.758524+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:09.806915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"52'5","reported_seq":47,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:09:30.961755+0000","last_change":"2023-10-06T03:57:10.346585+0000","last_active":"2023-10-06T04:09:30.961755+0000","last_peered":"2023-10-06T04:09:30.961755+0000","last_clean":"2023-10-06T04:09:30.961755+0000","last_became_active":"2023-10-06T03:57:10.345743+0000","last_became_peered":"2023-10-06T03:57:10.345743+0000","last_unstale":"2023-10-06T04:09:30.961755+0000","last_undegraded":"2023-10-06T04:09:30.961755+0000","last_fullsized":"2023-10-06T04:09:30.961755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:54.589702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20578,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"53'338","reported_seq":2432,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.051399+0000","last_change":"2023-10-06T04:05:03.366736+0000","last_active":"2023-10-06T04:10:48.051399+0000","last_peered":"2023-10-06T04:10:48.051399+0000","last_clean":"2023-10-06T04:10:48.051399+0000","last_became_active":"2023-10-06T03:57:11.373543+0000","last_became_peered":"2023-10-06T03:57:11.373543+0000","last_unstale":"2023-10-06T04:10:48.051399+0000","last_undegraded":"2023-10-06T04:10:48.051399+0000","last_fullsized":"2023-10-06T04:10:48.051399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:15.152354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020558070000000002,"stat_sum":{"num_bytes":6468011,"num_objects":146,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":146,"num_whiteouts":0,"num_read":27,"num_read_kb":14,"num_write":331,"num_write_kb":6443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"53'14","reported_seq":63,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:36.203548+0000","last_change":"2023-10-06T03:57:10.344618+0000","last_active":"2023-10-06T04:10:36.203548+0000","last_peered":"2023-10-06T04:10:36.203548+0000","last_clean":"2023-10-06T04:10:36.203548+0000","last_became_active":"2023-10-06T03:57:10.344324+0000","last_became_peered":"2023-10-06T03:57:10.344324+0000","last_unstale":"2023-10-06T04:10:36.203548+0000","last_undegraded":"2023-10-06T04:10:36.203548+0000","last_fullsized":"2023-10-06T04:10:36.203548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:06.062331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":22,"num_read_kb":21,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:37.659459+0000","last_change":"2023-10-06T03:57:05.317973+0000","last_active":"2023-10-06T04:08:37.659459+0000","last_peered":"2023-10-06T04:08:37.659459+0000","last_clean":"2023-10-06T04:08:37.659459+0000","last_became_active":"2023-10-06T03:57:05.317715+0000","last_became_peered":"2023-10-06T03:57:05.317715+0000","last_unstale":"2023-10-06T04:08:37.659459+0000","last_undegraded":"2023-10-06T04:08:37.659459+0000","last_fullsized":"2023-10-06T04:08:37.659459+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:37:51.858909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"53'307","reported_seq":2406,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.140157+0000","last_change":"2023-10-06T04:05:02.721255+0000","last_active":"2023-10-06T04:10:48.140157+0000","last_peered":"2023-10-06T04:10:48.140157+0000","last_clean":"2023-10-06T04:10:48.140157+0000","last_became_active":"2023-10-06T03:57:11.369445+0000","last_became_peered":"2023-10-06T03:57:11.369445+0000","last_unstale":"2023-10-06T04:10:48.140157+0000","last_undegraded":"2023-10-06T04:10:48.140157+0000","last_fullsized":"2023-10-06T04:10:48.140157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:52.999421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015173949999999999,"stat_sum":{"num_bytes":11679199,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":32,"num_read_kb":17,"num_write":302,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:37.573513+0000","last_change":"2023-10-06T03:57:05.320847+0000","last_active":"2023-10-06T04:08:37.573513+0000","last_peered":"2023-10-06T04:08:37.573513+0000","last_clean":"2023-10-06T04:08:37.573513+0000","last_became_active":"2023-10-06T03:57:05.320583+0000","last_became_peered":"2023-10-06T03:57:05.320583+0000","last_unstale":"2023-10-06T04:08:37.573513+0000","last_undegraded":"2023-10-06T04:08:37.573513+0000","last_fullsized":"2023-10-06T04:08:37.573513+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:12.772449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"53'92","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.058132+0000","last_change":"2023-10-06T03:57:10.341132+0000","last_active":"2023-10-06T04:10:48.058132+0000","last_peered":"2023-10-06T04:10:48.058132+0000","last_clean":"2023-10-06T04:10:48.058132+0000","last_became_active":"2023-10-06T03:57:10.339424+0000","last_became_peered":"2023-10-06T03:57:10.339424+0000","last_unstale":"2023-10-06T04:10:48.058132+0000","last_undegraded":"2023-10-06T04:10:48.058132+0000","last_fullsized":"2023-10-06T04:10:48.058132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:39.675711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":19,"num_write":219,"num_write_kb":1011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"53'316","reported_seq":2411,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.032289+0000","last_change":"2023-10-06T04:05:02.722365+0000","last_active":"2023-10-06T04:10:48.032289+0000","last_peered":"2023-10-06T04:10:48.032289+0000","last_clean":"2023-10-06T04:10:48.032289+0000","last_became_active":"2023-10-06T03:57:11.372215+0000","last_became_peered":"2023-10-06T03:57:11.372215+0000","last_unstale":"2023-10-06T04:10:48.032289+0000","last_undegraded":"2023-10-06T04:10:48.032289+0000","last_fullsized":"2023-10-06T04:10:48.032289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:23.559480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020199100000000002,"stat_sum":{"num_bytes":14547344,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":28,"num_read_kb":4106,"num_write":306,"num_write_kb":14296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"53'204","reported_seq":257,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:44.658005+0000","last_change":"2023-10-06T03:57:10.344051+0000","last_active":"2023-10-06T04:10:44.658005+0000","last_peered":"2023-10-06T04:10:44.658005+0000","last_clean":"2023-10-06T04:10:44.658005+0000","last_became_active":"2023-10-06T03:57:10.343865+0000","last_became_peered":"2023-10-06T03:57:10.343865+0000","last_unstale":"2023-10-06T04:10:44.658005+0000","last_undegraded":"2023-10-06T04:10:44.658005+0000","last_fullsized":"2023-10-06T04:10:44.658005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:32.522470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":11,"num_read_kb":32771,"num_write":206,"num_write_kb":12488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"53'331","reported_seq":2431,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.948651+0000","last_change":"2023-10-06T04:05:03.151764+0000","last_active":"2023-10-06T04:10:47.948651+0000","last_peered":"2023-10-06T04:10:47.948651+0000","last_clean":"2023-10-06T04:10:47.948651+0000","last_became_active":"2023-10-06T03:57:11.369681+0000","last_became_peered":"2023-10-06T03:57:11.369681+0000","last_unstale":"2023-10-06T04:10:47.948651+0000","last_undegraded":"2023-10-06T04:10:47.948651+0000","last_fullsized":"2023-10-06T04:10:47.948651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:53.858179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013151409999999999,"stat_sum":{"num_bytes":9592740,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":33,"num_read_kb":14,"num_write":324,"num_write_kb":13639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"50'100","reported_seq":149,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:52.634119+0000","last_change":"2023-10-06T03:57:10.349666+0000","last_active":"2023-10-06T04:09:52.634119+0000","last_peered":"2023-10-06T04:09:52.634119+0000","last_clean":"2023-10-06T04:09:52.634119+0000","last_became_active":"2023-10-06T03:57:10.349320+0000","last_became_peered":"2023-10-06T03:57:10.349320+0000","last_unstale":"2023-10-06T04:09:52.634119+0000","last_undegraded":"2023-10-06T04:09:52.634119+0000","last_fullsized":"2023-10-06T04:09:52.634119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:45:49.709519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194858,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":4110,"num_write":99,"num_write_kb":4143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"53'317","reported_seq":2411,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.096552+0000","last_change":"2023-10-06T04:05:02.723622+0000","last_active":"2023-10-06T04:10:48.096552+0000","last_peered":"2023-10-06T04:10:48.096552+0000","last_clean":"2023-10-06T04:10:48.096552+0000","last_became_active":"2023-10-06T03:57:11.370229+0000","last_became_peered":"2023-10-06T03:57:11.370229+0000","last_unstale":"2023-10-06T04:10:48.096552+0000","last_undegraded":"2023-10-06T04:10:48.096552+0000","last_fullsized":"2023-10-06T04:10:48.096552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:26.174331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027851170000000001,"stat_sum":{"num_bytes":11772234,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":27,"num_read_kb":17,"num_write":309,"num_write_kb":11655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"53'238","reported_seq":299,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:23.094832+0000","last_change":"2023-10-06T03:57:10.350263+0000","last_active":"2023-10-06T04:10:23.094832+0000","last_peered":"2023-10-06T04:10:23.094832+0000","last_clean":"2023-10-06T04:10:23.094832+0000","last_became_active":"2023-10-06T03:57:10.350083+0000","last_became_peered":"2023-10-06T03:57:10.350083+0000","last_unstale":"2023-10-06T04:10:23.094832+0000","last_undegraded":"2023-10-06T04:10:23.094832+0000","last_fullsized":"2023-10-06T04:10:23.094832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:05.433032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":36,"num_read_kb":45128,"num_write":264,"num_write_kb":12512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"53'333","reported_seq":2430,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.062537+0000","last_change":"2023-10-06T04:05:02.724048+0000","last_active":"2023-10-06T04:10:48.062537+0000","last_peered":"2023-10-06T04:10:48.062537+0000","last_clean":"2023-10-06T04:10:48.062537+0000","last_became_active":"2023-10-06T03:57:11.369123+0000","last_became_peered":"2023-10-06T03:57:11.369123+0000","last_unstale":"2023-10-06T04:10:48.062537+0000","last_undegraded":"2023-10-06T04:10:48.062537+0000","last_fullsized":"2023-10-06T04:10:48.062537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:28.139036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028613789999999998,"stat_sum":{"num_bytes":15016756,"num_objects":144,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":0,"num_read":30,"num_read_kb":4112,"num_write":318,"num_write_kb":14827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"52'181","reported_seq":225,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:47.407717+0000","last_change":"2023-10-06T03:57:10.347049+0000","last_active":"2023-10-06T04:09:47.407717+0000","last_peered":"2023-10-06T04:09:47.407717+0000","last_clean":"2023-10-06T04:09:47.407717+0000","last_became_active":"2023-10-06T03:57:10.344583+0000","last_became_peered":"2023-10-06T03:57:10.344583+0000","last_unstale":"2023-10-06T04:09:47.407717+0000","last_undegraded":"2023-10-06T04:09:47.407717+0000","last_fullsized":"2023-10-06T04:09:47.407717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:32.280257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":177,"num_write_kb":8286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"53'324","reported_seq":2428,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.088100+0000","last_change":"2023-10-06T04:05:02.721502+0000","last_active":"2023-10-06T04:10:48.088100+0000","last_peered":"2023-10-06T04:10:48.088100+0000","last_clean":"2023-10-06T04:10:48.088100+0000","last_became_active":"2023-10-06T03:57:11.366500+0000","last_became_peered":"2023-10-06T03:57:11.366500+0000","last_unstale":"2023-10-06T04:10:48.088100+0000","last_undegraded":"2023-10-06T04:10:48.088100+0000","last_fullsized":"2023-10-06T04:10:48.088100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:24.714603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016878310000000001,"stat_sum":{"num_bytes":6012647,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":38,"num_read_kb":4113,"num_write":317,"num_write_kb":10135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"50'170","reported_seq":220,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:51.778904+0000","last_change":"2023-10-06T03:57:10.348170+0000","last_active":"2023-10-06T04:09:51.778904+0000","last_peered":"2023-10-06T04:09:51.778904+0000","last_clean":"2023-10-06T04:09:51.778904+0000","last_became_active":"2023-10-06T03:57:10.347897+0000","last_became_peered":"2023-10-06T03:57:10.347897+0000","last_unstale":"2023-10-06T04:09:51.778904+0000","last_undegraded":"2023-10-06T04:09:51.778904+0000","last_fullsized":"2023-10-06T04:09:51.778904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:40.756246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7191702,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":12,"num_read_kb":8202,"num_write":168,"num_write_kb":7111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"53'386","reported_seq":2487,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.032330+0000","last_change":"2023-10-06T04:05:02.721899+0000","last_active":"2023-10-06T04:10:48.032330+0000","last_peered":"2023-10-06T04:10:48.032330+0000","last_clean":"2023-10-06T04:10:48.032330+0000","last_became_active":"2023-10-06T03:57:11.364150+0000","last_became_peered":"2023-10-06T03:57:11.364150+0000","last_unstale":"2023-10-06T04:10:48.032330+0000","last_undegraded":"2023-10-06T04:10:48.032330+0000","last_fullsized":"2023-10-06T04:10:48.032330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:11.038080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001670408,"stat_sum":{"num_bytes":10918213,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":35,"num_read_kb":7178,"num_write":378,"num_write_kb":19053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"53'345","reported_seq":2466,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.795523+0000","last_change":"2023-10-06T04:05:03.542896+0000","last_active":"2023-10-06T04:10:47.795523+0000","last_peered":"2023-10-06T04:10:47.795523+0000","last_clean":"2023-10-06T04:10:47.795523+0000","last_became_active":"2023-10-06T03:57:11.366524+0000","last_became_peered":"2023-10-06T03:57:11.366524+0000","last_unstale":"2023-10-06T04:10:47.795523+0000","last_undegraded":"2023-10-06T04:10:47.795523+0000","last_fullsized":"2023-10-06T04:10:47.795523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:33.044858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017816290000000001,"stat_sum":{"num_bytes":22531000,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":54,"num_read_kb":16279,"num_write":337,"num_write_kb":22178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"53'349","reported_seq":2453,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.079994+0000","last_change":"2023-10-06T04:05:02.721559+0000","last_active":"2023-10-06T04:10:48.079994+0000","last_peered":"2023-10-06T04:10:48.079994+0000","last_clean":"2023-10-06T04:10:48.079994+0000","last_became_active":"2023-10-06T03:57:11.370848+0000","last_became_peered":"2023-10-06T03:57:11.370848+0000","last_unstale":"2023-10-06T04:10:48.079994+0000","last_undegraded":"2023-10-06T04:10:48.079994+0000","last_fullsized":"2023-10-06T04:10:48.079994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:15.919090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017239289999999999,"stat_sum":{"num_bytes":12150184,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":37,"num_read_kb":8210,"num_write":341,"num_write_kb":16096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"53'351","reported_seq":2457,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.032069+0000","last_change":"2023-10-06T04:05:02.723715+0000","last_active":"2023-10-06T04:10:48.032069+0000","last_peered":"2023-10-06T04:10:48.032069+0000","last_clean":"2023-10-06T04:10:48.032069+0000","last_became_active":"2023-10-06T03:57:11.363875+0000","last_became_peered":"2023-10-06T03:57:11.363875+0000","last_unstale":"2023-10-06T04:10:48.032069+0000","last_undegraded":"2023-10-06T04:10:48.032069+0000","last_fullsized":"2023-10-06T04:10:48.032069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:29.236344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028142470000000002,"stat_sum":{"num_bytes":14810267,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":39,"num_read_kb":4117,"num_write":340,"num_write_kb":14623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"53'337","reported_seq":2445,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.153308+0000","last_change":"2023-10-06T04:05:02.722555+0000","last_active":"2023-10-06T04:10:48.153308+0000","last_peered":"2023-10-06T04:10:48.153308+0000","last_clean":"2023-10-06T04:10:48.153308+0000","last_became_active":"2023-10-06T03:57:11.377126+0000","last_became_peered":"2023-10-06T03:57:11.377126+0000","last_unstale":"2023-10-06T04:10:48.153308+0000","last_undegraded":"2023-10-06T04:10:48.153308+0000","last_fullsized":"2023-10-06T04:10:48.153308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:05.041737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023666569999999999,"stat_sum":{"num_bytes":6069899,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":41,"num_read_kb":18,"num_write":330,"num_write_kb":6093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"53'310","reported_seq":2407,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.768352+0000","last_change":"2023-10-06T04:05:02.972860+0000","last_active":"2023-10-06T04:10:47.768352+0000","last_peered":"2023-10-06T04:10:47.768352+0000","last_clean":"2023-10-06T04:10:47.768352+0000","last_became_active":"2023-10-06T03:57:59.256086+0000","last_became_peered":"2023-10-06T03:57:59.256086+0000","last_unstale":"2023-10-06T04:10:47.768352+0000","last_undegraded":"2023-10-06T04:10:47.768352+0000","last_fullsized":"2023-10-06T04:10:47.768352+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:54.263909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018573859999999999,"stat_sum":{"num_bytes":11282404,"num_objects":135,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":0,"num_read":33,"num_read_kb":11479,"num_write":302,"num_write_kb":14403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":918737492,"num_objects":9845,"num_object_clones":0,"num_object_copies":19690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9845,"num_whiteouts":0,"num_read":2243,"num_read_kb":380500,"num_write":20424,"num_write_kb":1040919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1675485184,"data_stored":1835934888,"data_compressed":157714556,"data_compressed_allocated":174235648,"data_compressed_original":386113536,"omap_allocated":0,"internal_metadata":0},"log_size":20998,"ondisk_log_size":20998,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":160325534,"num_objects":200,"num_object_clones":0,"num_object_copies":400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":200,"num_whiteouts":0,"num_read":555,"num_read_kb":254049,"num_write":6759,"num_write_kb":191312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":122,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81215488,"data_stored":320651068,"data_compressed":43195038,"data_compressed_allocated":54370304,"data_compressed_original":294060032,"omap_allocated":0,"internal_metadata":0},"log_size":6506,"ondisk_log_size":6506,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64858,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575018,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506268,"kb_used_data":208980,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93865572,"statfs":{"total":96636764160,"available":96118345728,"internally_reserved":0,"allocated":213995520,"data_stored":258704768,"data_compressed":22101424,"data_compressed_allocated":25088000,"data_compressed_original":75468800,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.55400000000000005,"15min":0.54500000000000004},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28399999999999997},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.88900000000000001},"last":0.50600000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52200000000000002,"15min":0.52100000000000002},"min":{"1min":0.38400000000000001,"5min":0.312,"15min":0.28899999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.78000000000000003},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54300000000000004,"15min":0.52300000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78300000000000003,"5min":1.25,"15min":1.25},"last":0.48199999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54800000000000004,"15min":0.54900000000000004},"min":{"1min":0.33400000000000002,"5min":0.29799999999999999,"15min":0.28399999999999997},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57599999999999996,"15min":0.60699999999999998},"min":{"1min":0.34300000000000003,"5min":0.33700000000000002,"15min":0.33100000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":7.3739999999999997},"last":0.56299999999999994}]},{"osd":3,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.318},"max":{"1min":0.80600000000000005,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.55400000000000005,"15min":0.55900000000000005},"min":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72899999999999998,"5min":0.78000000000000003,"15min":0.85299999999999998},"last":0.45000000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.44,"15min":0.46400000000000002},"min":{"1min":0.30099999999999999,"5min":0.27400000000000002,"15min":0.251},"max":{"1min":0.64000000000000001,"5min":0.73599999999999999,"15min":1.0149999999999999},"last":0.433},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.34300000000000003,"5min":0.33500000000000002,"15min":0.28299999999999997},"max":{"1min":0.68600000000000005,"5min":0.80200000000000005,"15min":0.90700000000000003},"last":0.53700000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.495,"15min":0.51100000000000001},"min":{"1min":0.316,"5min":0.29999999999999999,"15min":0.27300000000000002},"max":{"1min":0.74199999999999999,"5min":0.82999999999999996,"15min":0.86099999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54600000000000004,"15min":0.56299999999999994},"min":{"1min":0.40699999999999997,"5min":0.317,"15min":0.28000000000000003},"max":{"1min":0.73699999999999999,"5min":0.84899999999999998,"15min":1.377},"last":0.46800000000000003}]},{"osd":6,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.51100000000000001,"15min":0.56200000000000006},"min":{"1min":0.34100000000000003,"5min":0.30599999999999999,"15min":0.29999999999999999},"max":{"1min":0.77100000000000002,"5min":0.77600000000000002,"15min":1.3420000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60499999999999998,"15min":0.59099999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.30199999999999999},"max":{"1min":0.84599999999999997,"5min":0.88200000000000001,"15min":0.93700000000000006},"last":0.61699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575017,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473876,"kb_used_data":164300,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":93897964,"statfs":{"total":96636764160,"available":96151515136,"internally_reserved":0,"allocated":168243200,"data_stored":187372207,"data_compressed":15745085,"data_compressed_allocated":17555456,"data_compressed_original":42110976,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54800000000000004,"15min":0.53200000000000003},"min":{"1min":0.28799999999999998,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.70399999999999996,"5min":2.613,"15min":2.613},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.58399999999999996,"15min":0.55800000000000005},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.33300000000000002},"max":{"1min":0.72299999999999998,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":0.68999999999999995}]},{"osd":1,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53400000000000003,"15min":0.59599999999999997},"min":{"1min":0.28899999999999998,"5min":0.28499999999999998,"15min":0.27200000000000002},"max":{"1min":0.71599999999999997,"5min":2.3889999999999998,"15min":9.5519999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.59399999999999997,"15min":0.60199999999999998},"min":{"1min":0.33400000000000002,"5min":0.315,"15min":0.313},"max":{"1min":0.75800000000000001,"5min":6.8019999999999996,"15min":6.8019999999999996},"last":0.66600000000000004}]},{"osd":2,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.625,"15min":0.67300000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.29799999999999999},"max":{"1min":0.71799999999999997,"5min":5.7590000000000003,"15min":9.6769999999999996},"last":0.627},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.623,"15min":0.67600000000000005},"min":{"1min":0.42199999999999999,"5min":0.33200000000000002,"15min":0.31},"max":{"1min":0.75,"5min":5.71,"15min":9.7309999999999999},"last":0.70199999999999996}]},{"osd":3,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.54900000000000004,"15min":0.57499999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.29299999999999998},"max":{"1min":0.73899999999999999,"5min":0.99099999999999999,"15min":5.7359999999999998},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.59699999999999998,"15min":0.61299999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.308},"max":{"1min":0.68799999999999994,"5min":1.0149999999999999,"15min":5.7009999999999996},"last":0.54700000000000004}]},{"osd":4,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.48099999999999998,"15min":0.49399999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.63300000000000001,"5min":0.78600000000000003,"15min":1.0129999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54100000000000004,"15min":0.54900000000000004},"min":{"1min":0.32600000000000001,"5min":0.313,"15min":0.30399999999999999},"max":{"1min":0.70299999999999996,"5min":0.79800000000000004,"15min":1.026},"last":0.39800000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.49299999999999999,"15min":0.50800000000000001},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.77800000000000002,"5min":0.78200000000000003,"15min":0.995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.66500000000000004,"5min":0.78500000000000003,"15min":0.83699999999999997},"last":0.65100000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.34899999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.77000000000000002,"5min":0.84199999999999997,"15min":0.98299999999999998},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56499999999999995,"15min":0.57799999999999996},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80500000000000005,"5min":0.84699999999999998,"15min":0.90800000000000003},"last":0.52200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575017,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576968,"kb_used_data":275584,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":93794872,"statfs":{"total":96636764160,"available":96045948928,"internally_reserved":0,"allocated":282198016,"data_stored":333666787,"data_compressed":32289943,"data_compressed_allocated":36245504,"data_compressed_original":96591872,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.91800000000000004,"15min":0.52000000000000002},"min":{"1min":0.22500000000000001,"5min":0.20799999999999999,"15min":0.192},"max":{"1min":0.56999999999999995,"5min":57.700000000000003,"15min":57.700000000000003},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.65200000000000002,"15min":0.441},"min":{"1min":0.23899999999999999,"5min":0.23300000000000001,"15min":0.186},"max":{"1min":0.64000000000000001,"5min":28.629999999999999,"15min":28.629999999999999},"last":0.32600000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.97899999999999998,"15min":0.57199999999999995},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.19600000000000001},"max":{"1min":0.59899999999999998,"5min":58.795000000000002,"15min":58.795000000000002},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.67600000000000005,"15min":0.48999999999999999},"min":{"1min":0.32200000000000001,"5min":0.29599999999999999,"15min":0.23000000000000001},"max":{"1min":0.63400000000000001,"5min":25.722000000000001,"15min":25.722000000000001},"last":0.378}]},{"osd":3,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.59799999999999998,"15min":0.443},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.216},"max":{"1min":0.58499999999999996,"5min":22.920999999999999,"15min":22.920999999999999},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.439,"5min":0.63400000000000001,"15min":0.46800000000000003},"min":{"1min":0.29099999999999998,"5min":0.27300000000000002,"15min":0.22600000000000001},"max":{"1min":0.59799999999999998,"5min":22.797999999999998,"15min":22.797999999999998},"last":0.39200000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":1.2370000000000001,"15min":0.82399999999999995},"min":{"1min":0.443,"5min":0.38500000000000001,"15min":0.34499999999999997},"max":{"1min":1.0149999999999999,"5min":59.018999999999998,"15min":59.018999999999998},"last":0.624},{"interface":"front","average":{"1min":0.70999999999999996,"5min":1.26,"15min":0.83499999999999996},"min":{"1min":0.50700000000000001,"5min":0.47399999999999998,"15min":0.316},"max":{"1min":1.0609999999999999,"5min":59.046999999999997,"15min":59.046999999999997},"last":0.65200000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":1.21,"15min":0.79300000000000004},"min":{"1min":0.48599999999999999,"5min":0.40200000000000002,"15min":0.34399999999999997},"max":{"1min":0.90300000000000002,"5min":58.987000000000002,"15min":58.987000000000002},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65700000000000003,"15min":0.59599999999999997},"min":{"1min":0.44900000000000001,"5min":0.42599999999999999,"15min":0.29399999999999998},"max":{"1min":0.97699999999999998,"5min":6.2140000000000004,"15min":6.2140000000000004},"last":0.53700000000000003}]},{"osd":6,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65200000000000002,"15min":0.60399999999999998},"min":{"1min":0.44,"5min":0.41999999999999998,"15min":0.36599999999999999},"max":{"1min":0.95599999999999996,"5min":4.6020000000000003,"15min":4.6020000000000003},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":1.3,"15min":0.85999999999999999},"min":{"1min":0.49099999999999999,"5min":0.42799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94799999999999995,"5min":58.911999999999999,"15min":58.911999999999999},"last":0.61199999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":1.2210000000000001,"15min":0.81100000000000005},"min":{"1min":0.45800000000000002,"5min":0.40100000000000002,"15min":0.38700000000000001},"max":{"1min":1.1000000000000001,"5min":58.953000000000003,"15min":58.953000000000003},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.65900000000000003,"15min":0.62},"min":{"1min":0.46700000000000003,"5min":0.38100000000000001,"15min":0.34399999999999997},"max":{"1min":0.83199999999999996,"5min":4.5570000000000004,"15min":4.5570000000000004},"last":0.55800000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575017,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495552,"kb_used_data":190072,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":93876288,"statfs":{"total":96636764160,"available":96129318912,"internally_reserved":0,"allocated":194633728,"data_stored":251414553,"data_compressed":26053766,"data_compressed_allocated":29863936,"data_compressed_original":92942336,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.34100000000000003,"15min":0.34899999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.19500000000000001},"max":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":6.9539999999999997},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.35599999999999998,"15min":0.371},"min":{"1min":0.22700000000000001,"5min":0.215,"15min":0.191},"max":{"1min":0.55700000000000005,"5min":0.59499999999999997,"15min":8.0519999999999996},"last":0.27200000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.36199999999999999,"15min":0.35899999999999999},"min":{"1min":0.24299999999999999,"5min":0.24199999999999999,"15min":0.214},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":1.0589999999999999},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.38500000000000001,"15min":0.376},"min":{"1min":0.28299999999999997,"5min":0.26100000000000001,"15min":0.20799999999999999},"max":{"1min":0.626,"5min":0.626,"15min":2.0489999999999999},"last":0.36599999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.38600000000000001,"15min":0.41799999999999998},"min":{"1min":0.23300000000000001,"5min":0.23100000000000001,"15min":0.223},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":9.391},"last":0.311},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.42599999999999999,"15min":0.45600000000000002},"min":{"1min":0.31900000000000001,"5min":0.29099999999999998,"15min":0.27800000000000002},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":9.5239999999999991},"last":0.38100000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.64900000000000002,"15min":0.63400000000000001},"min":{"1min":0.35599999999999998,"5min":0.34000000000000002,"15min":0.29199999999999998},"max":{"1min":1.026,"5min":1.026,"15min":5.7699999999999996},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66000000000000003,"15min":0.63600000000000001},"min":{"1min":0.45400000000000001,"5min":0.41499999999999998,"15min":0.29499999999999998},"max":{"1min":0.997,"5min":0.997,"15min":2.7320000000000002},"last":0.57599999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.45000000000000001,"5min":0.41399999999999998,"15min":0.33800000000000002},"max":{"1min":0.89900000000000002,"5min":0.90500000000000003,"15min":5.5069999999999997},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63100000000000001,"15min":0.60499999999999998},"min":{"1min":0.436,"5min":0.38500000000000001,"15min":0.34799999999999998},"max":{"1min":0.872,"5min":0.878,"15min":1.05},"last":0.53800000000000003}]},{"osd":6,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.626,"15min":0.64700000000000002},"min":{"1min":0.42599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":6.9139999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64800000000000002,"15min":0.65500000000000003},"min":{"1min":0.46700000000000003,"5min":0.42199999999999999,"15min":0.379},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":6.3780000000000001},"last":0.44}]},{"osd":7,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61099999999999999,"15min":0.623},"min":{"1min":0.38500000000000001,"5min":0.377,"15min":0.36699999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":5.9989999999999997},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.627,"15min":0.61099999999999999},"min":{"1min":0.443,"5min":0.40100000000000002,"15min":0.39700000000000002},"max":{"1min":0.83999999999999997,"5min":0.86499999999999999,"15min":2.6930000000000001},"last":0.52400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575017,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":437036,"kb_used_data":139748,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93934804,"statfs":{"total":96636764160,"available":96189239296,"internally_reserved":0,"allocated":143101952,"data_stored":173375985,"data_compressed":18899047,"data_compressed_allocated":21323776,"data_compressed_original":57229312,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.34300000000000003,"15min":0.38400000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.19},"max":{"1min":0.51000000000000001,"5min":0.54300000000000004,"15min":14.968999999999999},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.376,"15min":0.41299999999999998},"min":{"1min":0.24099999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.52100000000000002,"5min":0.59199999999999997,"15min":14.746},"last":0.34000000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.37,"15min":0.52200000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.21099999999999999},"max":{"1min":0.53300000000000003,"5min":0.54100000000000004,"15min":50.677999999999997},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.45800000000000002,"15min":0.60099999999999998},"min":{"1min":0.34599999999999997,"5min":0.28599999999999998,"15min":0.26000000000000001},"max":{"1min":0.57599999999999996,"5min":0.66200000000000003,"15min":50.615000000000002},"last":0.35999999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.37,"15min":0.45900000000000002},"min":{"1min":0.25600000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.48999999999999999,"5min":0.61299999999999999,"15min":17.440999999999999},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.44,"5min":0.44700000000000001,"15min":0.59899999999999998},"min":{"1min":0.32400000000000001,"5min":0.29099999999999998,"15min":0.23499999999999999},"max":{"1min":0.65500000000000003,"5min":0.88900000000000001,"15min":50.424999999999997},"last":0.307}]},{"osd":4,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59499999999999997,"15min":0.64400000000000002},"min":{"1min":0.41099999999999998,"5min":0.34200000000000003,"15min":0.29699999999999999},"max":{"1min":0.76800000000000002,"5min":0.84799999999999998,"15min":14.802},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60399999999999998,"15min":0.67500000000000004},"min":{"1min":0.42199999999999999,"5min":0.39400000000000002,"15min":0.36499999999999999},"max":{"1min":0.746,"5min":0.88700000000000001,"15min":14.621},"last":0.432}]},{"osd":5,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60399999999999998,"15min":0.63200000000000001},"min":{"1min":0.48399999999999999,"5min":0.40100000000000002,"15min":0.35199999999999998},"max":{"1min":0.75700000000000001,"5min":0.84399999999999997,"15min":14.792999999999999},"last":0.499},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60899999999999999,"15min":0.76000000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.371},"max":{"1min":0.72499999999999998,"5min":0.92600000000000005,"15min":50.537999999999997},"last":0.57699999999999996}]},{"osd":6,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63700000000000001,"15min":0.68700000000000006},"min":{"1min":0.44700000000000001,"5min":0.40699999999999997,"15min":0.39100000000000001},"max":{"1min":0.78700000000000003,"5min":1.216,"15min":23.388000000000002},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64300000000000002,"15min":0.69999999999999996},"min":{"1min":0.50600000000000001,"5min":0.40400000000000003,"15min":0.39100000000000001},"max":{"1min":0.75,"5min":0.86899999999999999,"15min":14.856},"last":0.51900000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.627,"15min":0.67900000000000005},"min":{"1min":0.45600000000000002,"5min":0.40999999999999998,"15min":0.377},"max":{"1min":0.80900000000000005,"5min":0.95899999999999996,"15min":14.693},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63500000000000001,"15min":0.67400000000000004},"min":{"1min":0.45800000000000002,"5min":0.44600000000000001,"15min":0.434},"max":{"1min":0.79600000000000004,"5min":0.95499999999999996,"15min":10.349},"last":0.53600000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575017,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584240,"kb_used_data":274664,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":93787600,"statfs":{"total":96636764160,"available":96038502400,"internally_reserved":0,"allocated":281255936,"data_stored":341245364,"data_compressed":32144254,"data_compressed_allocated":36605952,"data_compressed_original":105447424,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.34399999999999997,"15min":0.35999999999999999},"min":{"1min":0.24299999999999999,"5min":0.216,"15min":0.186},"max":{"1min":0.5,"5min":0.58699999999999997,"15min":7.0999999999999996},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.35499999999999998,"15min":0.41699999999999998},"min":{"1min":0.27200000000000002,"5min":0.221,"15min":0.218},"max":{"1min":0.622,"5min":0.622,"15min":14.257},"last":0.36299999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.34999999999999998,"15min":0.35399999999999998},"min":{"1min":0.23100000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":3.1760000000000002},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.40899999999999997,"15min":0.39600000000000002},"min":{"1min":0.29999999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.61499999999999999,"5min":0.68200000000000005,"15min":5.2599999999999998},"last":0.308}]},{"osd":2,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.39800000000000002,"15min":0.45900000000000002},"min":{"1min":0.316,"5min":0.25700000000000001,"15min":0.23499999999999999},"max":{"1min":0.57599999999999996,"5min":0.747,"15min":13.138},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.42699999999999999,"15min":0.44500000000000001},"min":{"1min":0.32800000000000001,"5min":0.28299999999999997,"15min":0.27200000000000002},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":7.181},"last":0.40000000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.627,"15min":0.68100000000000005},"min":{"1min":0.46800000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":13.068},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.65600000000000003,"15min":0.67500000000000004},"min":{"1min":0.57399999999999995,"5min":0.434,"15min":0.39800000000000002},"max":{"1min":0.878,"5min":0.89100000000000001,"15min":6.4299999999999997},"last":0.73599999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.47799999999999998,"5min":0.35999999999999999,"15min":0.32700000000000001},"max":{"1min":0.89000000000000001,"5min":0.95599999999999996,"15min":3.105},"last":0.436},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.59799999999999998,"15min":0.60799999999999998},"min":{"1min":0.52000000000000002,"5min":0.372,"15min":0.29299999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":3.5369999999999999},"last":0.52400000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.63100000000000001,"15min":0.68799999999999994},"min":{"1min":0.54400000000000004,"5min":0.375,"15min":0.375},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":12.974},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63400000000000001,"15min":0.64100000000000001},"min":{"1min":0.56299999999999994,"5min":0.378,"15min":0.35599999999999998},"max":{"1min":0.93700000000000006,"5min":0.94499999999999995,"15min":3.0379999999999998},"last":0.501}]},{"osd":7,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.61299999999999999,"15min":0.63400000000000001},"min":{"1min":0.52900000000000003,"5min":0.41199999999999998,"15min":0.40100000000000002},"max":{"1min":0.93400000000000005,"5min":0.97199999999999998,"15min":6.2949999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.63400000000000001,"15min":0.67200000000000004},"min":{"1min":0.496,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90400000000000003,"5min":0.95399999999999996,"15min":13.032},"last":0.54200000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575018,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":580532,"kb_used_data":279148,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":93791308,"statfs":{"total":96636764160,"available":96042299392,"internally_reserved":0,"allocated":285847552,"data_stored":371153375,"data_compressed":31721463,"data_compressed_allocated":36786176,"data_compressed_original":130072576,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52900000000000003,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68100000000000005,"5min":0.747,"15min":0.79600000000000004},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.56699999999999995,"15min":0.54600000000000004},"min":{"1min":0.47699999999999998,"5min":0.35099999999999998,"15min":0.26600000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.80000000000000004},"last":0.72799999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.41199999999999998,"5min":0.34899999999999998,"15min":0.26200000000000001},"max":{"1min":0.73899999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.501,"15min":0.51100000000000001},"min":{"1min":0.35199999999999998,"5min":0.30399999999999999,"15min":0.27400000000000002},"max":{"1min":0.70399999999999996,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.40400000000000003,"5min":0.35899999999999999,"15min":0.312},"max":{"1min":0.69099999999999995,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.57599999999999996},"min":{"1min":0.441,"5min":0.36299999999999999,"15min":0.311},"max":{"1min":0.76700000000000002,"5min":1.413,"15min":1.413},"last":0.70699999999999996}]},{"osd":3,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.59099999999999997,"15min":0.55100000000000005},"min":{"1min":0.38400000000000001,"5min":0.32100000000000001,"15min":0.314},"max":{"1min":0.65800000000000003,"5min":6.7000000000000002,"15min":6.7000000000000002},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52900000000000003,"15min":0.54500000000000004},"min":{"1min":0.36899999999999999,"5min":0.36399999999999999,"15min":0.32000000000000001},"max":{"1min":0.72299999999999998,"5min":1.208,"15min":1.208},"last":0.63}]},{"osd":5,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.44600000000000001,"15min":0.47699999999999998},"min":{"1min":0.28599999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.59299999999999997,"5min":0.77100000000000002,"15min":0.81899999999999995},"last":0.436},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52200000000000002,"15min":0.53000000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.29499999999999998},"max":{"1min":0.66800000000000004,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.48299999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.48499999999999999,"15min":0.51300000000000001},"min":{"1min":0.315,"5min":0.30099999999999999,"15min":0.28100000000000003},"max":{"1min":0.60499999999999998,"5min":1.236,"15min":1.236},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.42899999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.69199999999999995,"5min":1.381,"15min":1.381},"last":0.64700000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53200000000000003,"15min":0.54600000000000004},"min":{"1min":0.38600000000000001,"5min":0.32800000000000001,"15min":0.30599999999999999},"max":{"1min":0.66700000000000004,"5min":0.71799999999999997,"15min":0.83499999999999996},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.45900000000000002,"5min":0.39900000000000002,"15min":0.32600000000000001},"max":{"1min":0.73599999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.57899999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575017,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485212,"kb_used_data":187924,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93886628,"statfs":{"total":96636764160,"available":96139907072,"internally_reserved":0,"allocated":192434176,"data_stored":243297716,"data_compressed":22019470,"data_compressed_allocated":25227264,"data_compressed_original":81457152,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.50700000000000001,"15min":0.55100000000000005},"min":{"1min":0.34399999999999997,"5min":0.29999999999999999,"15min":0.23300000000000001},"max":{"1min":0.60199999999999998,"5min":1.0189999999999999,"15min":11.167},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.53800000000000003,"15min":0.57699999999999996},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.221},"max":{"1min":0.71899999999999997,"5min":1.0449999999999999,"15min":9.8659999999999997},"last":0.626}]},{"osd":1,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.64100000000000001,"5min":1.0780000000000001,"15min":1.1120000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55100000000000005,"15min":0.54000000000000004},"min":{"1min":0.35599999999999998,"5min":0.29999999999999999,"15min":0.28799999999999998},"max":{"1min":0.73999999999999999,"5min":1.028,"15min":2.524},"last":0.65600000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.52700000000000002,"15min":0.54000000000000004},"min":{"1min":0.39400000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.68999999999999995,"5min":0.98499999999999999,"15min":1.0389999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70399999999999996,"5min":0.999,"15min":1.1439999999999999},"last":0.53300000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.77300000000000002,"5min":0.97299999999999998,"15min":2.9359999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60999999999999999,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.377,"15min":0.28999999999999998},"max":{"1min":0.77500000000000002,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.67700000000000005}]},{"osd":4,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.45900000000000002,"15min":0.47999999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.26000000000000001},"max":{"1min":0.71099999999999997,"5min":0.73499999999999999,"15min":1.1619999999999999},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57699999999999996,"15min":0.54200000000000004},"min":{"1min":0.33700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.75600000000000001,"5min":0.78900000000000003,"15min":1.1950000000000001},"last":0.39300000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46899999999999997,"15min":0.503},"min":{"1min":0.35699999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65900000000000003,"5min":0.78300000000000003,"15min":0.94099999999999995},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56499999999999995,"15min":0.58099999999999996},"min":{"1min":0.41899999999999998,"5min":0.38400000000000001,"15min":0.32900000000000001},"max":{"1min":0.72599999999999998,"5min":0.84999999999999998,"15min":0.97899999999999998},"last":0.51600000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.53700000000000003},"min":{"1min":0.38700000000000001,"5min":0.33500000000000002,"15min":0.28699999999999998},"max":{"1min":0.81999999999999995,"5min":0.84899999999999998,"15min":1.1830000000000001},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.438,"5min":0.375,"15min":0.311},"max":{"1min":0.79900000000000004,"5min":0.81299999999999994,"15min":1.29},"last":0.40899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13369344,"data_stored":49512965,"data_compressed":6345911,"data_compressed_allocated":8089600,"data_compressed_original":44290048,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7880704,"data_stored":33731560,"data_compressed":4481765,"data_compressed_allocated":5713920,"data_compressed_original":31571968,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3530752,"data_stored":15209155,"data_compressed":2097150,"data_compressed_allocated":2711552,"data_compressed_original":14401536,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12541952,"data_stored":45232784,"data_compressed":5646645,"data_compressed_allocated":7327744,"data_compressed_original":40050688,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20615168,"data_stored":82714194,"data_compressed":11662389,"data_compressed_allocated":14422016,"data_compressed_original":76578816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10690560,"data_stored":43917090,"data_compressed":5801730,"data_compressed_allocated":7286784,"data_compressed_original":40554496,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1855488,"data_stored":8388720,"data_compressed":1228027,"data_compressed_allocated":1515520,"data_compressed_original":8056832,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":10731520,"data_stored":41944600,"data_compressed":5931421,"data_compressed_allocated":7303168,"data_compressed_original":38555648,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":180592640,"data_stored":201004046,"data_compressed":19675426,"data_compressed_allocated":21729280,"data_compressed_original":48078848,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":273707008,"data_stored":299627100,"data_compressed":27808178,"data_compressed_allocated":30531584,"data_compressed_original":65019904,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":138960896,"data_stored":157858703,"data_compressed":16801897,"data_compressed_allocated":18612224,"data_compressed_original":42827776,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":268103680,"data_stored":295704453,"data_compressed":26497609,"data_compressed_allocated":29278208,"data_compressed_original":65396736,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":264622080,"data_stored":288131054,"data_compressed":20059074,"data_compressed_allocated":22364160,"data_compressed_original":53493760,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":181133312,"data_stored":199072499,"data_compressed":16217740,"data_compressed_allocated":17940480,"data_compressed_original":40902656,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":165777408,"data_stored":178675360,"data_compressed":14517058,"data_compressed_allocated":16039936,"data_compressed_original":34054144,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":202588160,"data_stored":215861673,"data_compressed":16137574,"data_compressed_allocated":17739776,"data_compressed_original":36339712,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:10:56.891 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T04:10:56.892 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:10:57.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.037+0000 7f9a0028e700 1 Processor -- start 2023-10-06T04:10:57.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.037+0000 7f9a0028e700 1 -- start start 2023-10-06T04:10:57.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.038+0000 7f9a0028e700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f99f806c440 0x7f99f8069130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:57.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.038+0000 7f9a0028e700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99f80c4e30 0x7f99f8069670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:57.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.038+0000 7f9a0028e700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99f80bf5e0 con 0x7f99f806c810 2023-10-06T04:10:57.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.038+0000 7f9a0028e700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f8158010 con 0x7f99f806c440 2023-10-06T04:10:57.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.038+0000 7f9a0028e700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f8157d30 con 0x7f99f80c4e30 2023-10-06T04:10:57.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.038+0000 7f99fd829700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f99f806c810 0x7f99f8150ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:35332/0 (socket says 172.21.15.157:35332) 2023-10-06T04:10:57.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.038+0000 7f99fd829700 1 -- 172.21.15.157:0/331168243 learned_addr learned my addr 172.21.15.157:0/331168243 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:10:57.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.038+0000 7f99fe02a700 1 --2- 172.21.15.157:0/331168243 >> v2:172.21.15.175:3300/0 conn(0x7f99f806c440 0x7f99f8069130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:57.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.038+0000 7f99fe82b700 1 --2- 172.21.15.157:0/331168243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99f80c4e30 0x7f99f8069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:57.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.039+0000 7f99fd028700 1 -- 172.21.15.157:0/331168243 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3061990752 0 0) 0x7f99f80bf5e0 con 0x7f99f806c810 2023-10-06T04:10:57.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.039+0000 7f99fd028700 1 -- 172.21.15.157:0/331168243 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99e4003570 con 0x7f99f806c810 2023-10-06T04:10:57.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.039+0000 7f99fd028700 1 -- 172.21.15.157:0/331168243 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4071298551 0 0) 0x7f99e4003570 con 0x7f99f806c810 2023-10-06T04:10:57.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.039+0000 7f99fd028700 1 -- 172.21.15.157:0/331168243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99f80c4e30 msgr2=0x7f99f8069670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:10:57.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.039+0000 7f99fd028700 1 --2- 172.21.15.157:0/331168243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99f80c4e30 0x7f99f8069670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:57.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.039+0000 7f99fd028700 1 -- 172.21.15.157:0/331168243 >> v2:172.21.15.175:3300/0 conn(0x7f99f806c440 msgr2=0x7f99f8069130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:10:57.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.039+0000 7f99fd028700 1 --2- 172.21.15.157:0/331168243 >> v2:172.21.15.175:3300/0 conn(0x7f99f806c440 0x7f99f8069130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:57.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.039+0000 7f99fd028700 1 -- 172.21.15.157:0/331168243 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f80c1090 con 0x7f99f806c810 2023-10-06T04:10:57.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.039+0000 7f99fd028700 1 -- 172.21.15.157:0/331168243 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f99e8001470 con 0x7f99f806c810 2023-10-06T04:10:57.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.039+0000 7f99fd028700 1 -- 172.21.15.157:0/331168243 >> v1:172.21.15.157:6789/0 conn(0x7f99f806c810 legacy=0x7f99f8150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:57.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.039+0000 7f99fd028700 1 --2- 172.21.15.157:0/331168243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99e4003a60 0x7f99e4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:57.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.039+0000 7f99fd028700 1 --2- 172.21.15.157:0/331168243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e4004810 0x7f99e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:57.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.039+0000 7f99fd028700 1 --2- 172.21.15.157:0/331168243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99f80c4e30 0x7f99e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:57.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.039+0000 7f99fd028700 1 -- 172.21.15.157:0/331168243 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f8158010 con 0x7f99e4003a60 2023-10-06T04:10:57.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.039+0000 7f99fd028700 1 -- 172.21.15.157:0/331168243 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f8157d30 con 0x7f99e4004810 2023-10-06T04:10:57.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.039+0000 7f99fd028700 1 -- 172.21.15.157:0/331168243 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e400ba60 con 0x7f99f80c4e30 2023-10-06T04:10:57.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.039+0000 7f99fe02a700 1 --2- 172.21.15.157:0/331168243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99e4003a60 0x7f99e4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:57.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.039+0000 7f99fd829700 1 --2- 172.21.15.157:0/331168243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e4004810 0x7f99e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:57.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.039+0000 7f99fe82b700 1 --2- 172.21.15.157:0/331168243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99f80c4e30 0x7f99e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:57.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.039+0000 7f99fe02a700 1 -- 172.21.15.157:0/331168243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99f80c4e30 msgr2=0x7f99e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:10:57.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.039+0000 7f99fe02a700 1 --2- 172.21.15.157:0/331168243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99f80c4e30 0x7f99e40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:57.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.039+0000 7f99fe02a700 1 -- 172.21.15.157:0/331168243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e4004810 msgr2=0x7f99e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:10:57.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.039+0000 7f99fe02a700 1 --2- 172.21.15.157:0/331168243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e4004810 0x7f99e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:57.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.039+0000 7f99fe02a700 1 -- 172.21.15.157:0/331168243 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e4003570 con 0x7f99e4003a60 2023-10-06T04:10:57.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.039+0000 7f99fe02a700 1 --2- 172.21.15.157:0/331168243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99e4003a60 0x7f99e4003e30 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009c00 tx=0x7f99f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df463d0dfb8f458f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:57.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.040+0000 7f99fd028700 1 -- 172.21.15.157:0/331168243 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f400e710 con 0x7f99e4003a60 2023-10-06T04:10:57.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.040+0000 7f99fd028700 1 -- 172.21.15.157:0/331168243 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f400e870 con 0x7f99e4003a60 2023-10-06T04:10:57.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.040+0000 7f99fd028700 1 -- 172.21.15.157:0/331168243 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f400ec30 con 0x7f99e4003a60 2023-10-06T04:10:57.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.040+0000 7f9a0028e700 1 -- 172.21.15.157:0/331168243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99e4003a60 msgr2=0x7f99e4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:57.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.040+0000 7f9a0028e700 1 --2- 172.21.15.157:0/331168243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99e4003a60 0x7f99e4003e30 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009c00 tx=0x7f99f40091a0 comp rx=0 tx=0).stop 2023-10-06T04:10:57.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.040+0000 7f99fe82b700 1 -- 172.21.15.157:0/331168243 reap_dead start 2023-10-06T04:10:57.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.040+0000 7f9a0028e700 1 -- 172.21.15.157:0/331168243 shutdown_connections 2023-10-06T04:10:57.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.040+0000 7f9a0028e700 1 -- 172.21.15.157:0/331168243 >> 172.21.15.157:0/331168243 conn(0x7f99f8068370 msgr2=0x7f99f8068770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:57.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.040+0000 7f9a0028e700 1 -- 172.21.15.157:0/331168243 shutdown_connections 2023-10-06T04:10:57.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.040+0000 7f9a0028e700 1 -- 172.21.15.157:0/331168243 wait complete. 2023-10-06T04:10:57.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.041+0000 7f9a0028e700 1 Processor -- start 2023-10-06T04:10:57.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.041+0000 7f9a0028e700 1 -- start start 2023-10-06T04:10:57.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.042+0000 7f9a0028e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e4003a60 0x7f99f8166d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:57.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.042+0000 7f9a0028e700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99f80c22e0 0x7f99f80c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:57.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.042+0000 7f9a0028e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99f806eed0 0x7f99f81698d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:57.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.042+0000 7f9a0028e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f80c1e50 con 0x7f99f806eed0 2023-10-06T04:10:57.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.042+0000 7f9a0028e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f806ecf0 con 0x7f99e4003a60 2023-10-06T04:10:57.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.042+0000 7f9a0028e700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f80c15b0 con 0x7f99f80c22e0 2023-10-06T04:10:57.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.042+0000 7f99fe82b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99f806eed0 0x7f99f81698d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:57.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.042+0000 7f99fe02a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e4003a60 0x7f99f8166d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:57.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.042+0000 7f99fe82b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99f806eed0 0x7f99f81698d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51590/0 (socket says 172.21.15.157:51590) 2023-10-06T04:10:57.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.042+0000 7f99fd829700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99f80c22e0 0x7f99f80c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:57.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.042+0000 7f99fe82b700 1 -- 172.21.15.157:0/948258799 learned_addr learned my addr 172.21.15.157:0/948258799 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:10:57.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.042+0000 7f99fe82b700 1 -- 172.21.15.157:0/948258799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99f80c22e0 msgr2=0x7f99f80c3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:57.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.042+0000 7f99fe82b700 1 --2- 172.21.15.157:0/948258799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99f80c22e0 0x7f99f80c3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:57.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.042+0000 7f99fe82b700 1 -- 172.21.15.157:0/948258799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e4003a60 msgr2=0x7f99f8166d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:57.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.042+0000 7f99fe82b700 1 --2- 172.21.15.157:0/948258799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e4003a60 0x7f99f8166d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:57.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.042+0000 7f99fe82b700 1 -- 172.21.15.157:0/948258799 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f8151300 con 0x7f99f806eed0 2023-10-06T04:10:57.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.042+0000 7f99fd829700 1 --2- 172.21.15.157:0/948258799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99f80c22e0 0x7f99f80c3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:10:57.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.042+0000 7f99fe02a700 1 --2- 172.21.15.157:0/948258799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e4003a60 0x7f99f8166d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:10:57.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.043+0000 7f99fe82b700 1 --2- 172.21.15.157:0/948258799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99f806eed0 0x7f99f81698d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f99f806ade0 tx=0x7f99f000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:57.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.043+0000 7f99ee7fc700 1 -- 172.21.15.157:0/948258799 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f000e040 con 0x7f99f806eed0 2023-10-06T04:10:57.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.043+0000 7f99ee7fc700 1 -- 172.21.15.157:0/948258799 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f000f040 con 0x7f99f806eed0 2023-10-06T04:10:57.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.043+0000 7f9a0028e700 1 -- 172.21.15.157:0/948258799 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f80aaa10 con 0x7f99f806eed0 2023-10-06T04:10:57.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.043+0000 7f99ee7fc700 1 -- 172.21.15.157:0/948258799 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f000fe20 con 0x7f99f806eed0 2023-10-06T04:10:57.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.043+0000 7f9a0028e700 1 -- 172.21.15.157:0/948258799 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f80c1800 con 0x7f99f806eed0 2023-10-06T04:10:57.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.044+0000 7f9a0028e700 1 -- 172.21.15.157:0/948258799 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f80c2000 con 0x7f99f806eed0 2023-10-06T04:10:57.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.045+0000 7f99ee7fc700 1 -- 172.21.15.157:0/948258799 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f99f0015640 con 0x7f99f806eed0 2023-10-06T04:10:57.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.045+0000 7f99ee7fc700 1 --2- 172.21.15.157:0/948258799 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f99dc066f60 0x7f99dc069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:57.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.045+0000 7f99ee7fc700 1 -- 172.21.15.157:0/948258799 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f99f00858f0 con 0x7f99f806eed0 2023-10-06T04:10:57.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.045+0000 7f99fe02a700 1 --2- 172.21.15.157:0/948258799 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f99dc066f60 0x7f99dc069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:57.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.046+0000 7f99fe02a700 1 --2- 172.21.15.157:0/948258799 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f99dc066f60 0x7f99dc069410 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f99f40091e0 tx=0x7f99f401a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:57.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.049+0000 7f99ee7fc700 1 -- 172.21.15.157:0/948258799 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f99f00528b0 con 0x7f99f806eed0 2023-10-06T04:10:57.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.227+0000 7f9a0028e700 1 -- 172.21.15.157:0/948258799 --> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f99f80c2000 con 0x7f99dc066f60 2023-10-06T04:10:57.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.237+0000 7f99ee7fc700 1 -- 172.21.15.157:0/948258799 <== mgr.4107 v2:172.21.15.175:6832/90661 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+316198 (secure 0 0 0) 0x7f99f80c2000 con 0x7f99dc066f60 2023-10-06T04:10:57.243 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:10:57.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.239+0000 7f9a0028e700 1 -- 172.21.15.157:0/948258799 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f99dc066f60 msgr2=0x7f99dc069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:57.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.239+0000 7f9a0028e700 1 --2- 172.21.15.157:0/948258799 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f99dc066f60 0x7f99dc069410 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f99f40091e0 tx=0x7f99f401a000 comp rx=0 tx=0).stop 2023-10-06T04:10:57.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.239+0000 7f9a0028e700 1 -- 172.21.15.157:0/948258799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99f806eed0 msgr2=0x7f99f81698d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:57.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.239+0000 7f9a0028e700 1 --2- 172.21.15.157:0/948258799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99f806eed0 0x7f99f81698d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f99f806ade0 tx=0x7f99f000b040 comp rx=0 tx=0).stop 2023-10-06T04:10:57.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.239+0000 7f9a0028e700 1 -- 172.21.15.157:0/948258799 shutdown_connections 2023-10-06T04:10:57.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.239+0000 7f9a0028e700 1 --2- 172.21.15.157:0/948258799 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f99dc066f60 0x7f99dc069410 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:57.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.240+0000 7f9a0028e700 1 --2- 172.21.15.157:0/948258799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99e4003a60 0x7f99f8166d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:57.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.240+0000 7f9a0028e700 1 --2- 172.21.15.157:0/948258799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99f806eed0 0x7f99f81698d0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:57.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.240+0000 7f9a0028e700 1 --2- 172.21.15.157:0/948258799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99f80c22e0 0x7f99f80c3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:57.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.240+0000 7f9a0028e700 1 -- 172.21.15.157:0/948258799 >> 172.21.15.157:0/948258799 conn(0x7f99f8068370 msgr2=0x7f99f8070940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:57.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.240+0000 7f9a0028e700 1 -- 172.21.15.157:0/948258799 shutdown_connections 2023-10-06T04:10:57.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.240+0000 7f9a0028e700 1 -- 172.21.15.157:0/948258799 wait complete. 2023-10-06T04:10:57.247 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-10-06T04:10:57.254 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2023-10-06T04:10:55.645520+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1079653394,"num_objects":10047,"num_object_clones":0,"num_object_copies":20094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10047,"num_whiteouts":0,"num_read":2914,"num_read_kb":634649,"num_write":27342,"num_write_kb":1234647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":122,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27621,"ondisk_log_size":27621,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":4139684,"kb_used_data":1720420,"kb_used_omap":0,"kb_used_meta":2419200,"kb_avail":750835036,"statfs":{"total":773094113280,"available":768855076864,"internally_reserved":0,"allocated":1761710080,"data_stored":2160230755,"data_compressed":200974452,"data_compressed_allocated":228696064,"data_compressed_original":681320448,"omap_allocated":0,"internal_metadata":2477260800},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":170,"apply_latency_ms":170,"commit_latency_ns":170000000,"apply_latency_ns":170000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":117952278,"num_objects":3304,"num_object_clones":0,"num_object_copies":6608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3419,"num_write_kb":116453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004532"},"pg_stats":[{"pgid":"4.3f","version":"53'315","reported_seq":2411,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.016174+0000","last_change":"2023-10-06T04:05:03.151059+0000","last_active":"2023-10-06T04:10:48.016174+0000","last_peered":"2023-10-06T04:10:48.016174+0000","last_clean":"2023-10-06T04:10:48.016174+0000","last_became_active":"2023-10-06T03:57:11.371756+0000","last_became_peered":"2023-10-06T03:57:11.371756+0000","last_unstale":"2023-10-06T04:10:48.016174+0000","last_undegraded":"2023-10-06T04:10:48.016174+0000","last_fullsized":"2023-10-06T04:10:48.016174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:50.086528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001094783,"stat_sum":{"num_bytes":10567600,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":29,"num_read_kb":15,"num_write":301,"num_write_kb":10442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"53'337","reported_seq":2437,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.046233+0000","last_change":"2023-10-06T04:05:02.973547+0000","last_active":"2023-10-06T04:10:48.046233+0000","last_peered":"2023-10-06T04:10:48.046233+0000","last_clean":"2023-10-06T04:10:48.046233+0000","last_became_active":"2023-10-06T03:57:11.370070+0000","last_became_peered":"2023-10-06T03:57:11.370070+0000","last_unstale":"2023-10-06T04:10:48.046233+0000","last_undegraded":"2023-10-06T04:10:48.046233+0000","last_fullsized":"2023-10-06T04:10:48.046233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:00.917607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021773819999999998,"stat_sum":{"num_bytes":6441631,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":33,"num_read_kb":16,"num_write":327,"num_write_kb":6463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"53'333","reported_seq":2437,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.050065+0000","last_change":"2023-10-06T04:05:02.723796+0000","last_active":"2023-10-06T04:10:48.050065+0000","last_peered":"2023-10-06T04:10:48.050065+0000","last_clean":"2023-10-06T04:10:48.050065+0000","last_became_active":"2023-10-06T03:57:11.366012+0000","last_became_peered":"2023-10-06T03:57:11.366012+0000","last_unstale":"2023-10-06T04:10:48.050065+0000","last_undegraded":"2023-10-06T04:10:48.050065+0000","last_fullsized":"2023-10-06T04:10:48.050065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:45.802287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002722286,"stat_sum":{"num_bytes":14390705,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":40,"num_read_kb":20499,"num_write":313,"num_write_kb":22357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"53'294","reported_seq":2393,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.819279+0000","last_change":"2023-10-06T04:05:03.543119+0000","last_active":"2023-10-06T04:10:47.819279+0000","last_peered":"2023-10-06T04:10:47.819279+0000","last_clean":"2023-10-06T04:10:47.819279+0000","last_became_active":"2023-10-06T03:57:11.365673+0000","last_became_peered":"2023-10-06T03:57:11.365673+0000","last_unstale":"2023-10-06T04:10:47.819279+0000","last_undegraded":"2023-10-06T04:10:47.819279+0000","last_fullsized":"2023-10-06T04:10:47.819279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:40.953382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001947182,"stat_sum":{"num_bytes":14739251,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":32,"num_read_kb":7963,"num_write":286,"num_write_kb":14550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"53'323","reported_seq":2421,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.102698+0000","last_change":"2023-10-06T04:05:03.366556+0000","last_active":"2023-10-06T04:10:48.102698+0000","last_peered":"2023-10-06T04:10:48.102698+0000","last_clean":"2023-10-06T04:10:48.102698+0000","last_became_active":"2023-10-06T03:57:11.361302+0000","last_became_peered":"2023-10-06T03:57:11.361302+0000","last_unstale":"2023-10-06T04:10:48.102698+0000","last_undegraded":"2023-10-06T04:10:48.102698+0000","last_fullsized":"2023-10-06T04:10:48.102698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:57.552493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001966644,"stat_sum":{"num_bytes":2126805,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":31,"num_read_kb":18,"num_write":316,"num_write_kb":2215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"53'311","reported_seq":2412,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.048068+0000","last_change":"2023-10-06T04:05:02.723270+0000","last_active":"2023-10-06T04:10:48.048068+0000","last_peered":"2023-10-06T04:10:48.048068+0000","last_clean":"2023-10-06T04:10:48.048068+0000","last_became_active":"2023-10-06T03:57:11.368775+0000","last_became_peered":"2023-10-06T03:57:11.368775+0000","last_unstale":"2023-10-06T04:10:48.048068+0000","last_undegraded":"2023-10-06T04:10:48.048068+0000","last_fullsized":"2023-10-06T04:10:48.048068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:22.368117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002640482,"stat_sum":{"num_bytes":22420575,"num_objects":141,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":141,"num_whiteouts":0,"num_read":34,"num_read_kb":13627,"num_write":301,"num_write_kb":22056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"53'368","reported_seq":2467,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.038708+0000","last_change":"2023-10-06T04:05:03.343818+0000","last_active":"2023-10-06T04:10:48.038708+0000","last_peered":"2023-10-06T04:10:48.038708+0000","last_clean":"2023-10-06T04:10:48.038708+0000","last_became_active":"2023-10-06T03:57:11.377786+0000","last_became_peered":"2023-10-06T03:57:11.377786+0000","last_unstale":"2023-10-06T04:10:48.038708+0000","last_undegraded":"2023-10-06T04:10:48.038708+0000","last_fullsized":"2023-10-06T04:10:48.038708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:12:54.218593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093960500000000004,"stat_sum":{"num_bytes":18362974,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":32,"num_read_kb":8208,"num_write":354,"num_write_kb":18122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"53'303","reported_seq":2401,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.925102+0000","last_change":"2023-10-06T04:05:02.972640+0000","last_active":"2023-10-06T04:10:47.925102+0000","last_peered":"2023-10-06T04:10:47.925102+0000","last_clean":"2023-10-06T04:10:47.925102+0000","last_became_active":"2023-10-06T03:57:11.373765+0000","last_became_peered":"2023-10-06T03:57:11.373765+0000","last_unstale":"2023-10-06T04:10:47.925102+0000","last_undegraded":"2023-10-06T04:10:47.925102+0000","last_fullsized":"2023-10-06T04:10:47.925102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:05.838007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001873881,"stat_sum":{"num_bytes":10907247,"num_objects":139,"num_object_clones":0,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":0,"num_read":31,"num_read_kb":4107,"num_write":297,"num_write_kb":10730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"53'308","reported_seq":2415,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.091138+0000","last_change":"2023-10-06T04:05:03.367332+0000","last_active":"2023-10-06T04:10:48.091138+0000","last_peered":"2023-10-06T04:10:48.091138+0000","last_clean":"2023-10-06T04:10:48.091138+0000","last_became_active":"2023-10-06T03:57:11.368004+0000","last_became_peered":"2023-10-06T03:57:11.368004+0000","last_unstale":"2023-10-06T04:10:48.091138+0000","last_undegraded":"2023-10-06T04:10:48.091138+0000","last_fullsized":"2023-10-06T04:10:48.091138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:16.760049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024398710000000001,"stat_sum":{"num_bytes":13562133,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":41,"num_read_kb":4114,"num_write":303,"num_write_kb":17498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"53'324","reported_seq":2421,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.936467+0000","last_change":"2023-10-06T04:05:02.973579+0000","last_active":"2023-10-06T04:10:47.936467+0000","last_peered":"2023-10-06T04:10:47.936467+0000","last_clean":"2023-10-06T04:10:47.936467+0000","last_became_active":"2023-10-06T03:57:11.379107+0000","last_became_peered":"2023-10-06T03:57:11.379107+0000","last_unstale":"2023-10-06T04:10:47.936467+0000","last_undegraded":"2023-10-06T04:10:47.936467+0000","last_fullsized":"2023-10-06T04:10:47.936467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:18.373103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002039126,"stat_sum":{"num_bytes":9755625,"num_objects":142,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":30,"num_read_kb":4112,"num_write":320,"num_write_kb":9702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"53'355","reported_seq":2461,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.053956+0000","last_change":"2023-10-06T04:05:02.724475+0000","last_active":"2023-10-06T04:10:48.053956+0000","last_peered":"2023-10-06T04:10:48.053956+0000","last_clean":"2023-10-06T04:10:48.053956+0000","last_became_active":"2023-10-06T03:57:11.368256+0000","last_became_peered":"2023-10-06T03:57:11.368256+0000","last_unstale":"2023-10-06T04:10:48.053956+0000","last_undegraded":"2023-10-06T04:10:48.053956+0000","last_fullsized":"2023-10-06T04:10:48.053956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:50.736367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029138929999999999,"stat_sum":{"num_bytes":31118399,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":39,"num_read_kb":4114,"num_write":348,"num_write_kb":30570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"53'305","reported_seq":2402,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.932349+0000","last_change":"2023-10-06T04:05:03.151565+0000","last_active":"2023-10-06T04:10:47.932349+0000","last_peered":"2023-10-06T04:10:47.932349+0000","last_clean":"2023-10-06T04:10:47.932349+0000","last_became_active":"2023-10-06T03:57:11.368425+0000","last_became_peered":"2023-10-06T03:57:11.368425+0000","last_unstale":"2023-10-06T04:10:47.932349+0000","last_undegraded":"2023-10-06T04:10:47.932349+0000","last_fullsized":"2023-10-06T04:10:47.932349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:06.004423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014231059999999999,"stat_sum":{"num_bytes":10394744,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":30,"num_read_kb":4105,"num_write":299,"num_write_kb":14405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"53'322","reported_seq":2425,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.059723+0000","last_change":"2023-10-06T04:05:03.150960+0000","last_active":"2023-10-06T04:10:48.059723+0000","last_peered":"2023-10-06T04:10:48.059723+0000","last_clean":"2023-10-06T04:10:48.059723+0000","last_became_active":"2023-10-06T03:57:11.367079+0000","last_became_peered":"2023-10-06T03:57:11.367079+0000","last_unstale":"2023-10-06T04:10:48.059723+0000","last_undegraded":"2023-10-06T04:10:48.059723+0000","last_fullsized":"2023-10-06T04:10:48.059723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:09:54.331206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001042765,"stat_sum":{"num_bytes":14339914,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":37,"num_read_kb":12308,"num_write":311,"num_write_kb":22299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"53'317","reported_seq":2420,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.803651+0000","last_change":"2023-10-06T04:05:02.973393+0000","last_active":"2023-10-06T04:10:47.803651+0000","last_peered":"2023-10-06T04:10:47.803651+0000","last_clean":"2023-10-06T04:10:47.803651+0000","last_became_active":"2023-10-06T03:57:11.373216+0000","last_became_peered":"2023-10-06T03:57:11.373216+0000","last_unstale":"2023-10-06T04:10:47.803651+0000","last_undegraded":"2023-10-06T04:10:47.803651+0000","last_fullsized":"2023-10-06T04:10:47.803651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:56.083203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022456289999999999,"stat_sum":{"num_bytes":5695017,"num_objects":137,"num_object_clones":0,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":0,"num_read":36,"num_read_kb":18,"num_write":313,"num_write_kb":9820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"53'324","reported_seq":2421,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.038672+0000","last_change":"2023-10-06T04:05:03.343999+0000","last_active":"2023-10-06T04:10:48.038672+0000","last_peered":"2023-10-06T04:10:48.038672+0000","last_clean":"2023-10-06T04:10:48.038672+0000","last_became_active":"2023-10-06T03:57:11.375689+0000","last_became_peered":"2023-10-06T03:57:11.375689+0000","last_unstale":"2023-10-06T04:10:48.038672+0000","last_undegraded":"2023-10-06T04:10:48.038672+0000","last_fullsized":"2023-10-06T04:10:48.038672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:42.953739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092625599999999998,"stat_sum":{"num_bytes":15101939,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":316,"num_write_kb":14917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"53'331","reported_seq":2433,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.045818+0000","last_change":"2023-10-06T04:05:02.722778+0000","last_active":"2023-10-06T04:10:48.045818+0000","last_peered":"2023-10-06T04:10:48.045818+0000","last_clean":"2023-10-06T04:10:48.045818+0000","last_became_active":"2023-10-06T03:57:11.366543+0000","last_became_peered":"2023-10-06T03:57:11.366543+0000","last_unstale":"2023-10-06T04:10:48.045818+0000","last_undegraded":"2023-10-06T04:10:48.045818+0000","last_fullsized":"2023-10-06T04:10:48.045818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:16.212967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024276609999999998,"stat_sum":{"num_bytes":19805304,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":35,"num_read_kb":19,"num_write":327,"num_write_kb":22717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"53'342","reported_seq":2442,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.091750+0000","last_change":"2023-10-06T04:05:03.367754+0000","last_active":"2023-10-06T04:10:48.091750+0000","last_peered":"2023-10-06T04:10:48.091750+0000","last_clean":"2023-10-06T04:10:48.091750+0000","last_became_active":"2023-10-06T03:57:11.366767+0000","last_became_peered":"2023-10-06T03:57:11.366767+0000","last_unstale":"2023-10-06T04:10:48.091750+0000","last_undegraded":"2023-10-06T04:10:48.091750+0000","last_fullsized":"2023-10-06T04:10:48.091750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:43.113575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002777833,"stat_sum":{"num_bytes":14662095,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":34,"num_read_kb":8213,"num_write":330,"num_write_kb":18582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"53'343","reported_seq":2453,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.060004+0000","last_change":"2023-10-06T04:05:02.723910+0000","last_active":"2023-10-06T04:10:48.060004+0000","last_peered":"2023-10-06T04:10:48.060004+0000","last_clean":"2023-10-06T04:10:48.060004+0000","last_became_active":"2023-10-06T03:57:11.362377+0000","last_became_peered":"2023-10-06T03:57:11.362377+0000","last_unstale":"2023-10-06T04:10:48.060004+0000","last_undegraded":"2023-10-06T04:10:48.060004+0000","last_fullsized":"2023-10-06T04:10:48.060004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:40.600383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027606900000000001,"stat_sum":{"num_bytes":10867734,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":43,"num_read_kb":26,"num_write":329,"num_write_kb":10783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"53'340","reported_seq":2446,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.826978+0000","last_change":"2023-10-06T04:05:03.151735+0000","last_active":"2023-10-06T04:10:47.826978+0000","last_peered":"2023-10-06T04:10:47.826978+0000","last_clean":"2023-10-06T04:10:47.826978+0000","last_became_active":"2023-10-06T03:57:11.363792+0000","last_became_peered":"2023-10-06T03:57:11.363792+0000","last_unstale":"2023-10-06T04:10:47.826978+0000","last_undegraded":"2023-10-06T04:10:47.826978+0000","last_fullsized":"2023-10-06T04:10:47.826978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:49.185450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014611940000000001,"stat_sum":{"num_bytes":14638716,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":40,"num_read_kb":8209,"num_write":327,"num_write_kb":18563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"53'345","reported_seq":2448,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.037717+0000","last_change":"2023-10-06T04:05:02.724524+0000","last_active":"2023-10-06T04:10:48.037717+0000","last_peered":"2023-10-06T04:10:48.037717+0000","last_clean":"2023-10-06T04:10:48.037717+0000","last_became_active":"2023-10-06T03:57:11.364726+0000","last_became_peered":"2023-10-06T03:57:11.364726+0000","last_unstale":"2023-10-06T04:10:48.037717+0000","last_undegraded":"2023-10-06T04:10:48.037717+0000","last_fullsized":"2023-10-06T04:10:48.037717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:56.866733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029763670000000002,"stat_sum":{"num_bytes":20613561,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":37,"num_read_kb":12308,"num_write":341,"num_write_kb":28448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"53'320","reported_seq":2429,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.103111+0000","last_change":"2023-10-06T04:05:03.344002+0000","last_active":"2023-10-06T04:10:48.103111+0000","last_peered":"2023-10-06T04:10:48.103111+0000","last_clean":"2023-10-06T04:10:48.103111+0000","last_became_active":"2023-10-06T03:57:11.375668+0000","last_became_peered":"2023-10-06T03:57:11.375668+0000","last_unstale":"2023-10-06T04:10:48.103111+0000","last_undegraded":"2023-10-06T04:10:48.103111+0000","last_fullsized":"2023-10-06T04:10:48.103111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:53.810875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010285800000000001,"stat_sum":{"num_bytes":18956719,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":42,"num_read_kb":8215,"num_write":314,"num_write_kb":18647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"53'337","reported_seq":2438,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.891519+0000","last_change":"2023-10-06T04:05:03.366839+0000","last_active":"2023-10-06T04:10:47.891519+0000","last_peered":"2023-10-06T04:10:47.891519+0000","last_clean":"2023-10-06T04:10:47.891519+0000","last_became_active":"2023-10-06T03:57:11.365149+0000","last_became_peered":"2023-10-06T03:57:11.365149+0000","last_unstale":"2023-10-06T04:10:47.891519+0000","last_undegraded":"2023-10-06T04:10:47.891519+0000","last_fullsized":"2023-10-06T04:10:47.891519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:20.911710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020974879999999998,"stat_sum":{"num_bytes":5856786,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":34,"num_read_kb":17,"num_write":329,"num_write_kb":9988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"53'337","reported_seq":2445,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.097397+0000","last_change":"2023-10-06T04:05:02.722763+0000","last_active":"2023-10-06T04:10:48.097397+0000","last_peered":"2023-10-06T04:10:48.097397+0000","last_clean":"2023-10-06T04:10:48.097397+0000","last_became_active":"2023-10-06T03:57:11.362128+0000","last_became_peered":"2023-10-06T03:57:11.362128+0000","last_unstale":"2023-10-06T04:10:48.097397+0000","last_undegraded":"2023-10-06T04:10:48.097397+0000","last_fullsized":"2023-10-06T04:10:48.097397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:53.744674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025174860000000002,"stat_sum":{"num_bytes":9975853,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":42,"num_read_kb":8214,"num_write":313,"num_write_kb":13997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"53'363","reported_seq":2477,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.777222+0000","last_change":"2023-10-06T04:05:03.344184+0000","last_active":"2023-10-06T04:10:47.777222+0000","last_peered":"2023-10-06T04:10:47.777222+0000","last_clean":"2023-10-06T04:10:47.777222+0000","last_became_active":"2023-10-06T03:57:11.375702+0000","last_became_peered":"2023-10-06T03:57:11.375702+0000","last_unstale":"2023-10-06T04:10:47.777222+0000","last_undegraded":"2023-10-06T04:10:47.777222+0000","last_fullsized":"2023-10-06T04:10:47.777222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:31.204231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010337009999999999,"stat_sum":{"num_bytes":19963417,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":47,"num_read_kb":12104,"num_write":344,"num_write_kb":19673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"53'313","reported_seq":2409,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.117818+0000","last_change":"2023-10-06T04:05:02.718513+0000","last_active":"2023-10-06T04:10:48.117818+0000","last_peered":"2023-10-06T04:10:48.117818+0000","last_clean":"2023-10-06T04:10:48.117818+0000","last_became_active":"2023-10-06T03:57:11.374842+0000","last_became_peered":"2023-10-06T03:57:11.374842+0000","last_unstale":"2023-10-06T04:10:48.117818+0000","last_undegraded":"2023-10-06T04:10:48.117818+0000","last_fullsized":"2023-10-06T04:10:48.117818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:54.438974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001332342,"stat_sum":{"num_bytes":5720212,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":29,"num_read_kb":4115,"num_write":303,"num_write_kb":5752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"53'363","reported_seq":2463,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.116692+0000","last_change":"2023-10-06T04:05:02.718726+0000","last_active":"2023-10-06T04:10:48.116692+0000","last_peered":"2023-10-06T04:10:48.116692+0000","last_clean":"2023-10-06T04:10:48.116692+0000","last_became_active":"2023-10-06T03:57:11.361349+0000","last_became_peered":"2023-10-06T03:57:11.361349+0000","last_unstale":"2023-10-06T04:10:48.116692+0000","last_undegraded":"2023-10-06T04:10:48.116692+0000","last_fullsized":"2023-10-06T04:10:48.116692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:30.646781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014394589999999999,"stat_sum":{"num_bytes":22991444,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":33,"num_read_kb":4115,"num_write":351,"num_write_kb":22637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"53'300","reported_seq":2405,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.871145+0000","last_change":"2023-10-06T04:05:02.972918+0000","last_active":"2023-10-06T04:10:47.871145+0000","last_peered":"2023-10-06T04:10:47.871145+0000","last_clean":"2023-10-06T04:10:47.871145+0000","last_became_active":"2023-10-06T03:57:11.375846+0000","last_became_peered":"2023-10-06T03:57:11.375846+0000","last_unstale":"2023-10-06T04:10:47.871145+0000","last_undegraded":"2023-10-06T04:10:47.871145+0000","last_fullsized":"2023-10-06T04:10:47.871145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:28.080614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018480599999999999,"stat_sum":{"num_bytes":26260340,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":38,"num_read_kb":12302,"num_write":293,"num_write_kb":25795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"53'362","reported_seq":417,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:45.966579+0000","last_change":"2023-10-06T03:57:10.348317+0000","last_active":"2023-10-06T04:10:45.966579+0000","last_peered":"2023-10-06T04:10:45.966579+0000","last_clean":"2023-10-06T04:10:45.966579+0000","last_became_active":"2023-10-06T03:57:10.348078+0000","last_became_peered":"2023-10-06T03:57:10.348078+0000","last_unstale":"2023-10-06T04:10:45.966579+0000","last_undegraded":"2023-10-06T04:10:45.966579+0000","last_fullsized":"2023-10-06T04:10:45.966579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:55.058324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12621090,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":13,"num_read_kb":12332,"num_write":358,"num_write_kb":12535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"53'317","reported_seq":2422,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.966557+0000","last_change":"2023-10-06T04:05:02.722997+0000","last_active":"2023-10-06T04:10:47.966557+0000","last_peered":"2023-10-06T04:10:47.966557+0000","last_clean":"2023-10-06T04:10:47.966557+0000","last_became_active":"2023-10-06T03:57:11.370245+0000","last_became_peered":"2023-10-06T03:57:11.370245+0000","last_unstale":"2023-10-06T04:10:47.966557+0000","last_undegraded":"2023-10-06T04:10:47.966557+0000","last_fullsized":"2023-10-06T04:10:47.966557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:47:18.189772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002348424,"stat_sum":{"num_bytes":22206576,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":38,"num_read_kb":9495,"num_write":309,"num_write_kb":25946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"53'89","reported_seq":168,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:42.933757+0000","last_change":"2023-10-06T03:57:10.345401+0000","last_active":"2023-10-06T04:10:42.933757+0000","last_peered":"2023-10-06T04:10:42.933757+0000","last_clean":"2023-10-06T04:10:42.933757+0000","last_became_active":"2023-10-06T03:57:10.345150+0000","last_became_peered":"2023-10-06T03:57:10.345150+0000","last_unstale":"2023-10-06T04:10:42.933757+0000","last_undegraded":"2023-10-06T04:10:42.933757+0000","last_fullsized":"2023-10-06T04:10:42.933757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:25.758193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4312676,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":146,"num_write":89,"num_write_kb":4275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"53'333","reported_seq":2426,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.998937+0000","last_change":"2023-10-06T04:05:03.367558+0000","last_active":"2023-10-06T04:10:47.998937+0000","last_peered":"2023-10-06T04:10:47.998937+0000","last_clean":"2023-10-06T04:10:47.998937+0000","last_became_active":"2023-10-06T03:57:11.375790+0000","last_became_peered":"2023-10-06T03:57:11.375790+0000","last_unstale":"2023-10-06T04:10:47.998937+0000","last_undegraded":"2023-10-06T04:10:47.998937+0000","last_fullsized":"2023-10-06T04:10:47.998937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:33.775431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025999780000000002,"stat_sum":{"num_bytes":19057467,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":26,"num_read_kb":12300,"num_write":323,"num_write_kb":18781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"53'1073","reported_seq":1122,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:52.526921+0000","last_change":"2023-10-06T03:57:10.347562+0000","last_active":"2023-10-06T04:09:52.526921+0000","last_peered":"2023-10-06T04:09:52.526921+0000","last_clean":"2023-10-06T04:09:52.526921+0000","last_became_active":"2023-10-06T03:57:10.347294+0000","last_became_peered":"2023-10-06T03:57:10.347294+0000","last_unstale":"2023-10-06T04:09:52.526921+0000","last_undegraded":"2023-10-06T04:09:52.526921+0000","last_fullsized":"2023-10-06T04:09:52.526921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":1073,"log_dups_size":0,"ondisk_log_size":1073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:25.090992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583466,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":7,"num_read_kb":16784,"num_write":1067,"num_write_kb":17022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"53'349","reported_seq":2451,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.789556+0000","last_change":"2023-10-06T04:05:03.543192+0000","last_active":"2023-10-06T04:10:47.789556+0000","last_peered":"2023-10-06T04:10:47.789556+0000","last_clean":"2023-10-06T04:10:47.789556+0000","last_became_active":"2023-10-06T03:57:11.371758+0000","last_became_peered":"2023-10-06T03:57:11.371758+0000","last_unstale":"2023-10-06T04:10:47.789556+0000","last_undegraded":"2023-10-06T04:10:47.789556+0000","last_fullsized":"2023-10-06T04:10:47.789556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:10.922157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017888349999999999,"stat_sum":{"num_bytes":16828923,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":35,"num_read_kb":4115,"num_write":344,"num_write_kb":16563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"53'47","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:36.261131+0000","last_change":"2023-10-06T03:57:10.347406+0000","last_active":"2023-10-06T04:10:36.261131+0000","last_peered":"2023-10-06T04:10:36.261131+0000","last_clean":"2023-10-06T04:10:36.261131+0000","last_became_active":"2023-10-06T03:57:10.347151+0000","last_became_peered":"2023-10-06T03:57:10.347151+0000","last_unstale":"2023-10-06T04:10:36.261131+0000","last_undegraded":"2023-10-06T04:10:36.261131+0000","last_fullsized":"2023-10-06T04:10:36.261131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:20.427620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":50,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"53'312","reported_seq":2415,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.139381+0000","last_change":"2023-10-06T04:05:02.721771+0000","last_active":"2023-10-06T04:10:48.139381+0000","last_peered":"2023-10-06T04:10:48.139381+0000","last_clean":"2023-10-06T04:10:48.139381+0000","last_became_active":"2023-10-06T03:57:11.369938+0000","last_became_peered":"2023-10-06T03:57:11.369938+0000","last_unstale":"2023-10-06T04:10:48.139381+0000","last_undegraded":"2023-10-06T04:10:48.139381+0000","last_fullsized":"2023-10-06T04:10:48.139381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:07.105832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018579569999999999,"stat_sum":{"num_bytes":31421368,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":37,"num_read_kb":12302,"num_write":304,"num_write_kb":31090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"53'50","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:46.318091+0000","last_change":"2023-10-06T03:57:10.346054+0000","last_active":"2023-10-06T04:10:46.318091+0000","last_peered":"2023-10-06T04:10:46.318091+0000","last_clean":"2023-10-06T04:10:46.318091+0000","last_became_active":"2023-10-06T03:57:10.345721+0000","last_became_peered":"2023-10-06T03:57:10.345721+0000","last_unstale":"2023-10-06T04:10:46.318091+0000","last_undegraded":"2023-10-06T04:10:46.318091+0000","last_fullsized":"2023-10-06T04:10:46.318091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:54.481304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":15,"num_write":48,"num_write_kb":4121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"53'355","reported_seq":2455,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.868827+0000","last_change":"2023-10-06T04:05:03.151517+0000","last_active":"2023-10-06T04:10:47.868827+0000","last_peered":"2023-10-06T04:10:47.868827+0000","last_clean":"2023-10-06T04:10:47.868827+0000","last_became_active":"2023-10-06T03:57:59.255587+0000","last_became_peered":"2023-10-06T03:57:59.255587+0000","last_unstale":"2023-10-06T04:10:47.868827+0000","last_undegraded":"2023-10-06T04:10:47.868827+0000","last_fullsized":"2023-10-06T04:10:47.868827+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:08.261023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001510024,"stat_sum":{"num_bytes":1780876,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":38,"num_read_kb":8209,"num_write":338,"num_write_kb":10112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"53'63","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:42.932445+0000","last_change":"2023-10-06T03:57:10.340531+0000","last_active":"2023-10-06T04:10:42.932445+0000","last_peered":"2023-10-06T04:10:42.932445+0000","last_clean":"2023-10-06T04:10:42.932445+0000","last_became_active":"2023-10-06T03:57:10.340232+0000","last_became_peered":"2023-10-06T03:57:10.340232+0000","last_unstale":"2023-10-06T04:10:42.932445+0000","last_undegraded":"2023-10-06T04:10:42.932445+0000","last_fullsized":"2023-10-06T04:10:42.932445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:43:26.332055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":66,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"53'339","reported_seq":2443,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.117177+0000","last_change":"2023-10-06T04:05:02.718927+0000","last_active":"2023-10-06T04:10:48.117177+0000","last_peered":"2023-10-06T04:10:48.117177+0000","last_clean":"2023-10-06T04:10:48.117177+0000","last_became_active":"2023-10-06T03:57:11.374892+0000","last_became_peered":"2023-10-06T03:57:11.374892+0000","last_unstale":"2023-10-06T04:10:48.117177+0000","last_undegraded":"2023-10-06T04:10:48.117177+0000","last_fullsized":"2023-10-06T04:10:48.117177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:51.198745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00156207,"stat_sum":{"num_bytes":17752359,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":38,"num_read_kb":8211,"num_write":326,"num_write_kb":25695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"53'670","reported_seq":727,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:31.373637+0000","last_change":"2023-10-06T03:57:10.346758+0000","last_active":"2023-10-06T04:10:31.373637+0000","last_peered":"2023-10-06T04:10:31.373637+0000","last_clean":"2023-10-06T04:10:31.373637+0000","last_became_active":"2023-10-06T03:57:10.346430+0000","last_became_peered":"2023-10-06T03:57:10.346430+0000","last_unstale":"2023-10-06T04:10:31.373637+0000","last_undegraded":"2023-10-06T04:10:31.373637+0000","last_fullsized":"2023-10-06T04:10:31.373637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:10.809242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389252,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":30,"num_read_kb":9887,"num_write":668,"num_write_kb":8554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"53'323","reported_seq":2428,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.032035+0000","last_change":"2023-10-06T04:05:02.723446+0000","last_active":"2023-10-06T04:10:48.032035+0000","last_peered":"2023-10-06T04:10:48.032035+0000","last_clean":"2023-10-06T04:10:48.032035+0000","last_became_active":"2023-10-06T03:57:11.367565+0000","last_became_peered":"2023-10-06T03:57:11.367565+0000","last_unstale":"2023-10-06T04:10:48.032035+0000","last_undegraded":"2023-10-06T04:10:48.032035+0000","last_fullsized":"2023-10-06T04:10:48.032035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:14.409116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027762680000000001,"stat_sum":{"num_bytes":10222697,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":38,"num_read_kb":23,"num_write":314,"num_write_kb":10145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"53'179","reported_seq":235,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:16.251094+0000","last_change":"2023-10-06T03:57:10.351062+0000","last_active":"2023-10-06T04:10:16.251094+0000","last_peered":"2023-10-06T04:10:16.251094+0000","last_clean":"2023-10-06T04:10:16.251094+0000","last_became_active":"2023-10-06T03:57:10.350795+0000","last_became_peered":"2023-10-06T03:57:10.350795+0000","last_unstale":"2023-10-06T04:10:16.251094+0000","last_undegraded":"2023-10-06T04:10:16.251094+0000","last_fullsized":"2023-10-06T04:10:16.251094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:54:45.714518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388676,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":30,"num_read_kb":20509,"num_write":184,"num_write_kb":8306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"53'276","reported_seq":2375,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.006307+0000","last_change":"2023-10-06T04:05:02.722868+0000","last_active":"2023-10-06T04:10:48.006307+0000","last_peered":"2023-10-06T04:10:48.006307+0000","last_clean":"2023-10-06T04:10:48.006307+0000","last_became_active":"2023-10-06T03:57:11.367254+0000","last_became_peered":"2023-10-06T03:57:11.367254+0000","last_unstale":"2023-10-06T04:10:48.006307+0000","last_undegraded":"2023-10-06T04:10:48.006307+0000","last_fullsized":"2023-10-06T04:10:48.006307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:04.706485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027822569999999998,"stat_sum":{"num_bytes":13638644,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":32,"num_read_kb":10530,"num_write":271,"num_write_kb":17563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"53'105","reported_seq":157,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:23.094836+0000","last_change":"2023-10-06T03:57:10.338125+0000","last_active":"2023-10-06T04:10:23.094836+0000","last_peered":"2023-10-06T04:10:23.094836+0000","last_clean":"2023-10-06T04:10:23.094836+0000","last_became_active":"2023-10-06T03:57:10.337798+0000","last_became_peered":"2023-10-06T03:57:10.337798+0000","last_unstale":"2023-10-06T04:10:23.094836+0000","last_undegraded":"2023-10-06T04:10:23.094836+0000","last_fullsized":"2023-10-06T04:10:23.094836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:00.482657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":9753,"num_write":104,"num_write_kb":4165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"53'329","reported_seq":2433,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.868704+0000","last_change":"2023-10-06T04:05:02.973084+0000","last_active":"2023-10-06T04:10:47.868704+0000","last_peered":"2023-10-06T04:10:47.868704+0000","last_clean":"2023-10-06T04:10:47.868704+0000","last_became_active":"2023-10-06T03:57:11.376223+0000","last_became_peered":"2023-10-06T03:57:11.376223+0000","last_unstale":"2023-10-06T04:10:47.868704+0000","last_undegraded":"2023-10-06T04:10:47.868704+0000","last_fullsized":"2023-10-06T04:10:47.868704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:04:37.408871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018161119999999999,"stat_sum":{"num_bytes":34899441,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":37,"num_read_kb":11570,"num_write":325,"num_write_kb":34246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"53'97","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:36.263633+0000","last_change":"2023-10-06T03:57:10.350511+0000","last_active":"2023-10-06T04:10:36.263633+0000","last_peered":"2023-10-06T04:10:36.263633+0000","last_clean":"2023-10-06T04:10:36.263633+0000","last_became_active":"2023-10-06T03:57:10.350219+0000","last_became_peered":"2023-10-06T03:57:10.350219+0000","last_unstale":"2023-10-06T04:10:36.263633+0000","last_undegraded":"2023-10-06T04:10:36.263633+0000","last_fullsized":"2023-10-06T04:10:36.263633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:56.043506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96,"num_write_kb":8294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"53'359","reported_seq":2468,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.062653+0000","last_change":"2023-10-06T04:05:03.367352+0000","last_active":"2023-10-06T04:10:48.062653+0000","last_peered":"2023-10-06T04:10:48.062653+0000","last_clean":"2023-10-06T04:10:48.062653+0000","last_became_active":"2023-10-06T03:57:11.362443+0000","last_became_peered":"2023-10-06T03:57:11.362443+0000","last_unstale":"2023-10-06T04:10:48.062653+0000","last_undegraded":"2023-10-06T04:10:48.062653+0000","last_fullsized":"2023-10-06T04:10:48.062653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:21:09.143085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002448839,"stat_sum":{"num_bytes":22757129,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":42,"num_read_kb":8288,"num_write":351,"num_write_kb":22409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"53'71","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:36.258463+0000","last_change":"2023-10-06T03:57:10.346259+0000","last_active":"2023-10-06T04:10:36.258463+0000","last_peered":"2023-10-06T04:10:36.258463+0000","last_clean":"2023-10-06T04:10:36.258463+0000","last_became_active":"2023-10-06T03:57:10.345424+0000","last_became_peered":"2023-10-06T03:57:10.345424+0000","last_unstale":"2023-10-06T04:10:36.258463+0000","last_undegraded":"2023-10-06T04:10:36.258463+0000","last_fullsized":"2023-10-06T04:10:36.258463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:19.278206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":33,"num_read_kb":33,"num_write":72,"num_write_kb":4158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"53'304","reported_seq":2409,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.152945+0000","last_change":"2023-10-06T04:05:02.723050+0000","last_active":"2023-10-06T04:10:48.152945+0000","last_peered":"2023-10-06T04:10:48.152945+0000","last_clean":"2023-10-06T04:10:48.152945+0000","last_became_active":"2023-10-06T03:57:11.375824+0000","last_became_peered":"2023-10-06T03:57:11.375824+0000","last_unstale":"2023-10-06T04:10:48.152945+0000","last_undegraded":"2023-10-06T04:10:48.152945+0000","last_fullsized":"2023-10-06T04:10:48.152945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:24:26.156866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023200299999999998,"stat_sum":{"num_bytes":9687154,"num_objects":142,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":38,"num_read_kb":8212,"num_write":296,"num_write_kb":9609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"53'250","reported_seq":295,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:23.097110+0000","last_change":"2023-10-06T03:57:10.350408+0000","last_active":"2023-10-06T04:10:23.097110+0000","last_peered":"2023-10-06T04:10:23.097110+0000","last_clean":"2023-10-06T04:10:23.097110+0000","last_became_active":"2023-10-06T03:57:10.350135+0000","last_became_peered":"2023-10-06T03:57:10.350135+0000","last_unstale":"2023-10-06T04:10:23.097110+0000","last_undegraded":"2023-10-06T04:10:23.097110+0000","last_fullsized":"2023-10-06T04:10:23.097110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:43.522111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":4098,"num_write":280,"num_write_kb":4293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"53'321","reported_seq":2420,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.862961+0000","last_change":"2023-10-06T04:05:02.972356+0000","last_active":"2023-10-06T04:10:47.862961+0000","last_peered":"2023-10-06T04:10:47.862961+0000","last_clean":"2023-10-06T04:10:47.862961+0000","last_became_active":"2023-10-06T03:57:11.376236+0000","last_became_peered":"2023-10-06T03:57:11.376236+0000","last_unstale":"2023-10-06T04:10:47.862961+0000","last_undegraded":"2023-10-06T04:10:47.862961+0000","last_fullsized":"2023-10-06T04:10:47.862961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:33.854818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015573080000000001,"stat_sum":{"num_bytes":22874532,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":32,"num_read_kb":10317,"num_write":312,"num_write_kb":26593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"50'92","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:52.580642+0000","last_change":"2023-10-06T03:57:10.345799+0000","last_active":"2023-10-06T04:09:52.580642+0000","last_peered":"2023-10-06T04:09:52.580642+0000","last_clean":"2023-10-06T04:09:52.580642+0000","last_became_active":"2023-10-06T03:57:10.345510+0000","last_became_peered":"2023-10-06T03:57:10.345510+0000","last_unstale":"2023-10-06T04:09:52.580642+0000","last_undegraded":"2023-10-06T04:09:52.580642+0000","last_fullsized":"2023-10-06T04:09:52.580642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:09.567314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":4108,"num_write":90,"num_write_kb":4140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"53'342","reported_seq":2446,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.072619+0000","last_change":"2023-10-06T04:05:02.721917+0000","last_active":"2023-10-06T04:10:48.072619+0000","last_peered":"2023-10-06T04:10:48.072619+0000","last_clean":"2023-10-06T04:10:48.072619+0000","last_became_active":"2023-10-06T03:57:11.368375+0000","last_became_peered":"2023-10-06T03:57:11.368375+0000","last_unstale":"2023-10-06T04:10:48.072619+0000","last_undegraded":"2023-10-06T04:10:48.072619+0000","last_fullsized":"2023-10-06T04:10:48.072619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:27.125523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001632912,"stat_sum":{"num_bytes":5549566,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":37,"num_read_kb":4115,"num_write":335,"num_write_kb":5549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"53'164","reported_seq":212,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:22.809255+0000","last_change":"2023-10-06T03:57:10.346945+0000","last_active":"2023-10-06T04:10:22.809255+0000","last_peered":"2023-10-06T04:10:22.809255+0000","last_clean":"2023-10-06T04:10:22.809255+0000","last_became_active":"2023-10-06T03:57:10.346689+0000","last_became_peered":"2023-10-06T03:57:10.346689+0000","last_unstale":"2023-10-06T04:10:22.809255+0000","last_undegraded":"2023-10-06T04:10:22.809255+0000","last_fullsized":"2023-10-06T04:10:22.809255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:43.420455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":12290,"num_write":163,"num_write_kb":8336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"53'349","reported_seq":2450,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.892654+0000","last_change":"2023-10-06T04:05:03.542774+0000","last_active":"2023-10-06T04:10:47.892654+0000","last_peered":"2023-10-06T04:10:47.892654+0000","last_clean":"2023-10-06T04:10:47.892654+0000","last_became_active":"2023-10-06T03:57:11.371795+0000","last_became_peered":"2023-10-06T03:57:11.371795+0000","last_unstale":"2023-10-06T04:10:47.892654+0000","last_undegraded":"2023-10-06T04:10:47.892654+0000","last_fullsized":"2023-10-06T04:10:47.892654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:00.324478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016252980000000001,"stat_sum":{"num_bytes":5725462,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":34,"num_read_kb":13,"num_write":343,"num_write_kb":5770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"53'54","reported_seq":96,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:48.081196+0000","last_change":"2023-10-06T03:57:10.348592+0000","last_active":"2023-10-06T04:09:48.081196+0000","last_peered":"2023-10-06T04:09:48.081196+0000","last_clean":"2023-10-06T04:09:48.081196+0000","last_became_active":"2023-10-06T03:57:10.348291+0000","last_became_peered":"2023-10-06T03:57:10.348291+0000","last_unstale":"2023-10-06T04:09:48.081196+0000","last_undegraded":"2023-10-06T04:09:48.081196+0000","last_fullsized":"2023-10-06T04:09:48.081196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:00.660348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":57,"num_write_kb":4195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"53'308","reported_seq":2408,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.055840+0000","last_change":"2023-10-06T04:05:03.366405+0000","last_active":"2023-10-06T04:10:48.055840+0000","last_peered":"2023-10-06T04:10:48.055840+0000","last_clean":"2023-10-06T04:10:48.055840+0000","last_became_active":"2023-10-06T03:57:11.365077+0000","last_became_peered":"2023-10-06T03:57:11.365077+0000","last_unstale":"2023-10-06T04:10:48.055840+0000","last_undegraded":"2023-10-06T04:10:48.055840+0000","last_fullsized":"2023-10-06T04:10:48.055840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:25.199391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019352270000000001,"stat_sum":{"num_bytes":15027095,"num_objects":146,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":146,"num_whiteouts":0,"num_read":33,"num_read_kb":4109,"num_write":302,"num_write_kb":14833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"52'278","reported_seq":329,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:51.781455+0000","last_change":"2023-10-06T03:57:10.348418+0000","last_active":"2023-10-06T04:09:51.781455+0000","last_peered":"2023-10-06T04:09:51.781455+0000","last_clean":"2023-10-06T04:09:51.781455+0000","last_became_active":"2023-10-06T03:57:10.348055+0000","last_became_peered":"2023-10-06T03:57:10.348055+0000","last_unstale":"2023-10-06T04:09:51.781455+0000","last_undegraded":"2023-10-06T04:09:51.781455+0000","last_fullsized":"2023-10-06T04:09:51.781455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:26.102022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":17,"num_read_kb":12301,"num_write":272,"num_write_kb":8316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"53'342","reported_seq":2437,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.070418+0000","last_change":"2023-10-06T04:05:02.724303+0000","last_active":"2023-10-06T04:10:48.070418+0000","last_peered":"2023-10-06T04:10:48.070418+0000","last_clean":"2023-10-06T04:10:48.070418+0000","last_became_active":"2023-10-06T03:57:11.370073+0000","last_became_peered":"2023-10-06T03:57:11.370073+0000","last_unstale":"2023-10-06T04:10:48.070418+0000","last_undegraded":"2023-10-06T04:10:48.070418+0000","last_fullsized":"2023-10-06T04:10:48.070418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:58.417951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027454179999999999,"stat_sum":{"num_bytes":18012929,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":28,"num_read_kb":4110,"num_write":339,"num_write_kb":17768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"53'112","reported_seq":159,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:46.295859+0000","last_change":"2023-10-06T03:57:10.352966+0000","last_active":"2023-10-06T04:10:46.295859+0000","last_peered":"2023-10-06T04:10:46.295859+0000","last_clean":"2023-10-06T04:10:46.295859+0000","last_became_active":"2023-10-06T03:57:10.352653+0000","last_became_peered":"2023-10-06T03:57:10.352653+0000","last_unstale":"2023-10-06T04:10:46.295859+0000","last_undegraded":"2023-10-06T04:10:46.295859+0000","last_fullsized":"2023-10-06T04:10:46.295859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:32.092557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":8,"num_read_kb":4217,"num_write":204,"num_write_kb":4775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"53'338","reported_seq":2437,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.039511+0000","last_change":"2023-10-06T04:05:02.724113+0000","last_active":"2023-10-06T04:10:48.039511+0000","last_peered":"2023-10-06T04:10:48.039511+0000","last_clean":"2023-10-06T04:10:48.039511+0000","last_became_active":"2023-10-06T03:57:11.363830+0000","last_became_peered":"2023-10-06T03:57:11.363830+0000","last_unstale":"2023-10-06T04:10:48.039511+0000","last_undegraded":"2023-10-06T04:10:48.039511+0000","last_fullsized":"2023-10-06T04:10:48.039511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:04.084019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027698559999999998,"stat_sum":{"num_bytes":22850224,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":32,"num_read_kb":4115,"num_write":329,"num_write_kb":26589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"53'32","reported_seq":82,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:50.963938+0000","last_change":"2023-10-06T03:57:10.349421+0000","last_active":"2023-10-06T04:10:50.963938+0000","last_peered":"2023-10-06T04:10:50.963938+0000","last_clean":"2023-10-06T04:10:50.963938+0000","last_became_active":"2023-10-06T03:57:10.349114+0000","last_became_peered":"2023-10-06T03:57:10.349114+0000","last_unstale":"2023-10-06T04:10:50.963938+0000","last_undegraded":"2023-10-06T04:10:50.963938+0000","last_fullsized":"2023-10-06T04:10:50.963938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:08.209791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2605665,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":19,"num_read_kb":18,"num_write":36,"num_write_kb":2597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"53'278","reported_seq":2370,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.045140+0000","last_change":"2023-10-06T04:05:02.724241+0000","last_active":"2023-10-06T04:10:48.045140+0000","last_peered":"2023-10-06T04:10:48.045140+0000","last_clean":"2023-10-06T04:10:48.045140+0000","last_became_active":"2023-10-06T03:57:11.367257+0000","last_became_peered":"2023-10-06T03:57:11.367257+0000","last_unstale":"2023-10-06T04:10:48.045140+0000","last_undegraded":"2023-10-06T04:10:48.045140+0000","last_fullsized":"2023-10-06T04:10:48.045140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:43.628819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002803224,"stat_sum":{"num_bytes":18462781,"num_objects":138,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":138,"num_whiteouts":0,"num_read":26,"num_read_kb":4106,"num_write":268,"num_write_kb":22268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"53'53","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:52.265701+0000","last_change":"2023-10-06T03:57:10.347649+0000","last_active":"2023-10-06T04:09:52.265701+0000","last_peered":"2023-10-06T04:09:52.265701+0000","last_clean":"2023-10-06T04:09:52.265701+0000","last_became_active":"2023-10-06T03:57:10.347433+0000","last_became_peered":"2023-10-06T03:57:10.347433+0000","last_unstale":"2023-10-06T04:09:52.265701+0000","last_undegraded":"2023-10-06T04:09:52.265701+0000","last_fullsized":"2023-10-06T04:09:52.265701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:55.951494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":11,"num_read_kb":8201,"num_write":54,"num_write_kb":4137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"53'320","reported_seq":2421,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.122912+0000","last_change":"2023-10-06T04:05:02.718759+0000","last_active":"2023-10-06T04:10:48.122912+0000","last_peered":"2023-10-06T04:10:48.122912+0000","last_clean":"2023-10-06T04:10:48.122912+0000","last_became_active":"2023-10-06T03:57:11.369825+0000","last_became_peered":"2023-10-06T03:57:11.369825+0000","last_unstale":"2023-10-06T04:10:48.122912+0000","last_undegraded":"2023-10-06T04:10:48.122912+0000","last_fullsized":"2023-10-06T04:10:48.122912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:43.415929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014204689999999999,"stat_sum":{"num_bytes":11370958,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":35,"num_read_kb":8210,"num_write":305,"num_write_kb":15362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:37.659432+0000","last_change":"2023-10-06T03:57:05.663312+0000","last_active":"2023-10-06T04:08:37.659432+0000","last_peered":"2023-10-06T04:08:37.659432+0000","last_clean":"2023-10-06T04:08:37.659432+0000","last_became_active":"2023-10-06T03:57:05.663046+0000","last_became_peered":"2023-10-06T03:57:05.663046+0000","last_unstale":"2023-10-06T04:08:37.659432+0000","last_undegraded":"2023-10-06T04:08:37.659432+0000","last_fullsized":"2023-10-06T04:08:37.659432+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:31.174359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"53'33","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:51.495313+0000","last_change":"2023-10-06T03:57:10.348930+0000","last_active":"2023-10-06T04:10:51.495313+0000","last_peered":"2023-10-06T04:10:51.495313+0000","last_clean":"2023-10-06T04:10:51.495313+0000","last_became_active":"2023-10-06T03:57:10.348608+0000","last_became_peered":"2023-10-06T03:57:10.348608+0000","last_unstale":"2023-10-06T04:10:51.495313+0000","last_undegraded":"2023-10-06T04:10:51.495313+0000","last_fullsized":"2023-10-06T04:10:51.495313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:00:09.283718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2070221,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":34,"num_write_kb":2062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"53'287","reported_seq":2383,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.110309+0000","last_change":"2023-10-06T04:05:02.718555+0000","last_active":"2023-10-06T04:10:48.110309+0000","last_peered":"2023-10-06T04:10:48.110309+0000","last_clean":"2023-10-06T04:10:48.110309+0000","last_became_active":"2023-10-06T03:57:11.370847+0000","last_became_peered":"2023-10-06T03:57:11.370847+0000","last_unstale":"2023-10-06T04:10:48.110309+0000","last_undegraded":"2023-10-06T04:10:48.110309+0000","last_fullsized":"2023-10-06T04:10:48.110309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:38.111171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013685990000000001,"stat_sum":{"num_bytes":14898272,"num_objects":129,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":129,"num_whiteouts":0,"num_read":29,"num_read_kb":12301,"num_write":279,"num_write_kb":18790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:37.658819+0000","last_change":"2023-10-06T03:57:05.318834+0000","last_active":"2023-10-06T04:08:37.658819+0000","last_peered":"2023-10-06T04:08:37.658819+0000","last_clean":"2023-10-06T04:08:37.658819+0000","last_became_active":"2023-10-06T03:57:05.318572+0000","last_became_peered":"2023-10-06T03:57:05.318572+0000","last_unstale":"2023-10-06T04:08:37.658819+0000","last_undegraded":"2023-10-06T04:08:37.658819+0000","last_fullsized":"2023-10-06T04:08:37.658819+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:21.951095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"53'10","reported_seq":60,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:46.769558+0000","last_change":"2023-10-06T03:57:10.348556+0000","last_active":"2023-10-06T04:09:46.769558+0000","last_peered":"2023-10-06T04:09:46.769558+0000","last_clean":"2023-10-06T04:09:46.769558+0000","last_became_active":"2023-10-06T03:57:10.348271+0000","last_became_peered":"2023-10-06T03:57:10.348271+0000","last_unstale":"2023-10-06T04:09:46.769558+0000","last_undegraded":"2023-10-06T04:09:46.769558+0000","last_fullsized":"2023-10-06T04:09:46.769558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:58.470989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":26,"num_write":12,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"53'320","reported_seq":2420,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.012974+0000","last_change":"2023-10-06T04:05:02.719009+0000","last_active":"2023-10-06T04:10:48.012974+0000","last_peered":"2023-10-06T04:10:48.012974+0000","last_clean":"2023-10-06T04:10:48.012974+0000","last_became_active":"2023-10-06T03:57:11.365006+0000","last_became_peered":"2023-10-06T03:57:11.365006+0000","last_unstale":"2023-10-06T04:10:48.012974+0000","last_undegraded":"2023-10-06T04:10:48.012974+0000","last_fullsized":"2023-10-06T04:10:48.012974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:33.005015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015961910000000001,"stat_sum":{"num_bytes":5694351,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":33,"num_read_kb":4109,"num_write":313,"num_write_kb":5725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:37.658115+0000","last_change":"2023-10-06T03:57:05.314695+0000","last_active":"2023-10-06T04:08:37.658115+0000","last_peered":"2023-10-06T04:08:37.658115+0000","last_clean":"2023-10-06T04:08:37.658115+0000","last_became_active":"2023-10-06T03:57:05.314408+0000","last_became_peered":"2023-10-06T03:57:05.314408+0000","last_unstale":"2023-10-06T04:08:37.658115+0000","last_undegraded":"2023-10-06T04:08:37.658115+0000","last_fullsized":"2023-10-06T04:08:37.658115+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:19.042145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"53'359","reported_seq":405,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:16.606829+0000","last_change":"2023-10-06T03:57:10.346078+0000","last_active":"2023-10-06T04:10:16.606829+0000","last_peered":"2023-10-06T04:10:16.606829+0000","last_clean":"2023-10-06T04:10:16.606829+0000","last_became_active":"2023-10-06T03:57:10.345769+0000","last_became_peered":"2023-10-06T03:57:10.345769+0000","last_unstale":"2023-10-06T04:10:16.606829+0000","last_undegraded":"2023-10-06T04:10:16.606829+0000","last_fullsized":"2023-10-06T04:10:16.606829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:16.630681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":352,"num_write_kb":12528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"53'315","reported_seq":2421,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.123122+0000","last_change":"2023-10-06T04:05:02.718334+0000","last_active":"2023-10-06T04:10:48.123122+0000","last_peered":"2023-10-06T04:10:48.123122+0000","last_clean":"2023-10-06T04:10:48.123122+0000","last_became_active":"2023-10-06T03:57:11.370076+0000","last_became_peered":"2023-10-06T03:57:11.370076+0000","last_unstale":"2023-10-06T04:10:48.123122+0000","last_undegraded":"2023-10-06T04:10:48.123122+0000","last_fullsized":"2023-10-06T04:10:48.123122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:18:59.517155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011772849999999999,"stat_sum":{"num_bytes":18508624,"num_objects":141,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":141,"num_whiteouts":0,"num_read":39,"num_read_kb":4112,"num_write":305,"num_write_kb":22326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:37.658686+0000","last_change":"2023-10-06T03:57:05.318778+0000","last_active":"2023-10-06T04:08:37.658686+0000","last_peered":"2023-10-06T04:08:37.658686+0000","last_clean":"2023-10-06T04:08:37.658686+0000","last_became_active":"2023-10-06T03:57:05.318456+0000","last_became_peered":"2023-10-06T03:57:05.318456+0000","last_unstale":"2023-10-06T04:08:37.658686+0000","last_undegraded":"2023-10-06T04:08:37.658686+0000","last_fullsized":"2023-10-06T04:08:37.658686+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:39.769241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":195,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:36.756245+0000","last_change":"2023-10-06T03:57:05.664245+0000","last_active":"2023-10-06T04:08:36.756245+0000","last_peered":"2023-10-06T04:08:36.756245+0000","last_clean":"2023-10-06T04:08:36.756245+0000","last_became_active":"2023-10-06T03:57:05.663426+0000","last_became_peered":"2023-10-06T03:57:05.663426+0000","last_unstale":"2023-10-06T04:08:36.756245+0000","last_undegraded":"2023-10-06T04:08:36.756245+0000","last_fullsized":"2023-10-06T04:08:36.756245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:29.322599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"53'719","reported_seq":770,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:52.436682+0000","last_change":"2023-10-06T03:57:10.351059+0000","last_active":"2023-10-06T04:09:52.436682+0000","last_peered":"2023-10-06T04:09:52.436682+0000","last_clean":"2023-10-06T04:09:52.436682+0000","last_became_active":"2023-10-06T03:57:10.348879+0000","last_became_peered":"2023-10-06T03:57:10.348879+0000","last_unstale":"2023-10-06T04:09:52.436682+0000","last_undegraded":"2023-10-06T04:09:52.436682+0000","last_fullsized":"2023-10-06T04:09:52.436682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:35.764148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9487985,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":20487,"num_write":715,"num_write_kb":13783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"53'332","reported_seq":2435,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.889473+0000","last_change":"2023-10-06T04:05:03.542987+0000","last_active":"2023-10-06T04:10:47.889473+0000","last_peered":"2023-10-06T04:10:47.889473+0000","last_clean":"2023-10-06T04:10:47.889473+0000","last_became_active":"2023-10-06T03:57:11.373452+0000","last_became_peered":"2023-10-06T03:57:11.373452+0000","last_unstale":"2023-10-06T04:10:47.889473+0000","last_undegraded":"2023-10-06T04:10:47.889473+0000","last_fullsized":"2023-10-06T04:10:47.889473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:20.524065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018153620000000001,"stat_sum":{"num_bytes":5826606,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":36,"num_read_kb":2733,"num_write":325,"num_write_kb":9961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:36.756325+0000","last_change":"2023-10-06T03:57:05.318790+0000","last_active":"2023-10-06T04:08:36.756325+0000","last_peered":"2023-10-06T04:08:36.756325+0000","last_clean":"2023-10-06T04:08:36.756325+0000","last_became_active":"2023-10-06T03:57:05.318531+0000","last_became_peered":"2023-10-06T03:57:05.318531+0000","last_unstale":"2023-10-06T04:08:36.756325+0000","last_undegraded":"2023-10-06T04:08:36.756325+0000","last_fullsized":"2023-10-06T04:08:36.756325+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:12.876998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"52'540","reported_seq":703,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:46.766101+0000","last_change":"2023-10-06T03:57:10.343328+0000","last_active":"2023-10-06T04:09:46.766101+0000","last_peered":"2023-10-06T04:09:46.766101+0000","last_clean":"2023-10-06T04:09:46.766101+0000","last_became_active":"2023-10-06T03:57:10.343051+0000","last_became_peered":"2023-10-06T03:57:10.343051+0000","last_unstale":"2023-10-06T04:09:46.766101+0000","last_undegraded":"2023-10-06T04:09:46.766101+0000","last_fullsized":"2023-10-06T04:09:46.766101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:27.722475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4571457,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":122,"num_read_kb":3989,"num_write":534,"num_write_kb":4895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"53'333","reported_seq":2437,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.058057+0000","last_change":"2023-10-06T04:05:03.367824+0000","last_active":"2023-10-06T04:10:48.058057+0000","last_peered":"2023-10-06T04:10:48.058057+0000","last_clean":"2023-10-06T04:10:48.058057+0000","last_became_active":"2023-10-06T03:57:11.373264+0000","last_became_peered":"2023-10-06T03:57:11.373264+0000","last_unstale":"2023-10-06T04:10:48.058057+0000","last_undegraded":"2023-10-06T04:10:48.058057+0000","last_fullsized":"2023-10-06T04:10:48.058057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:15.950245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026908990000000001,"stat_sum":{"num_bytes":10595820,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":37,"num_read_kb":4121,"num_write":327,"num_write_kb":10522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:36.758524+0000","last_change":"2023-10-06T03:57:05.663770+0000","last_active":"2023-10-06T04:08:36.758524+0000","last_peered":"2023-10-06T04:08:36.758524+0000","last_clean":"2023-10-06T04:08:36.758524+0000","last_became_active":"2023-10-06T03:57:05.663501+0000","last_became_peered":"2023-10-06T03:57:05.663501+0000","last_unstale":"2023-10-06T04:08:36.758524+0000","last_undegraded":"2023-10-06T04:08:36.758524+0000","last_fullsized":"2023-10-06T04:08:36.758524+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:09.806915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"52'5","reported_seq":47,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:09:30.961755+0000","last_change":"2023-10-06T03:57:10.346585+0000","last_active":"2023-10-06T04:09:30.961755+0000","last_peered":"2023-10-06T04:09:30.961755+0000","last_clean":"2023-10-06T04:09:30.961755+0000","last_became_active":"2023-10-06T03:57:10.345743+0000","last_became_peered":"2023-10-06T03:57:10.345743+0000","last_unstale":"2023-10-06T04:09:30.961755+0000","last_undegraded":"2023-10-06T04:09:30.961755+0000","last_fullsized":"2023-10-06T04:09:30.961755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:54.589702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20578,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"53'338","reported_seq":2432,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.051399+0000","last_change":"2023-10-06T04:05:03.366736+0000","last_active":"2023-10-06T04:10:48.051399+0000","last_peered":"2023-10-06T04:10:48.051399+0000","last_clean":"2023-10-06T04:10:48.051399+0000","last_became_active":"2023-10-06T03:57:11.373543+0000","last_became_peered":"2023-10-06T03:57:11.373543+0000","last_unstale":"2023-10-06T04:10:48.051399+0000","last_undegraded":"2023-10-06T04:10:48.051399+0000","last_fullsized":"2023-10-06T04:10:48.051399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:15.152354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020558070000000002,"stat_sum":{"num_bytes":6468011,"num_objects":146,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":146,"num_whiteouts":0,"num_read":27,"num_read_kb":14,"num_write":331,"num_write_kb":6443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"53'14","reported_seq":63,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:36.203548+0000","last_change":"2023-10-06T03:57:10.344618+0000","last_active":"2023-10-06T04:10:36.203548+0000","last_peered":"2023-10-06T04:10:36.203548+0000","last_clean":"2023-10-06T04:10:36.203548+0000","last_became_active":"2023-10-06T03:57:10.344324+0000","last_became_peered":"2023-10-06T03:57:10.344324+0000","last_unstale":"2023-10-06T04:10:36.203548+0000","last_undegraded":"2023-10-06T04:10:36.203548+0000","last_fullsized":"2023-10-06T04:10:36.203548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:06.062331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":22,"num_read_kb":21,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:37.659459+0000","last_change":"2023-10-06T03:57:05.317973+0000","last_active":"2023-10-06T04:08:37.659459+0000","last_peered":"2023-10-06T04:08:37.659459+0000","last_clean":"2023-10-06T04:08:37.659459+0000","last_became_active":"2023-10-06T03:57:05.317715+0000","last_became_peered":"2023-10-06T03:57:05.317715+0000","last_unstale":"2023-10-06T04:08:37.659459+0000","last_undegraded":"2023-10-06T04:08:37.659459+0000","last_fullsized":"2023-10-06T04:08:37.659459+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:37:51.858909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"53'307","reported_seq":2406,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.140157+0000","last_change":"2023-10-06T04:05:02.721255+0000","last_active":"2023-10-06T04:10:48.140157+0000","last_peered":"2023-10-06T04:10:48.140157+0000","last_clean":"2023-10-06T04:10:48.140157+0000","last_became_active":"2023-10-06T03:57:11.369445+0000","last_became_peered":"2023-10-06T03:57:11.369445+0000","last_unstale":"2023-10-06T04:10:48.140157+0000","last_undegraded":"2023-10-06T04:10:48.140157+0000","last_fullsized":"2023-10-06T04:10:48.140157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:52.999421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015173949999999999,"stat_sum":{"num_bytes":11679199,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":32,"num_read_kb":17,"num_write":302,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:37.573513+0000","last_change":"2023-10-06T03:57:05.320847+0000","last_active":"2023-10-06T04:08:37.573513+0000","last_peered":"2023-10-06T04:08:37.573513+0000","last_clean":"2023-10-06T04:08:37.573513+0000","last_became_active":"2023-10-06T03:57:05.320583+0000","last_became_peered":"2023-10-06T03:57:05.320583+0000","last_unstale":"2023-10-06T04:08:37.573513+0000","last_undegraded":"2023-10-06T04:08:37.573513+0000","last_fullsized":"2023-10-06T04:08:37.573513+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:12.772449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"53'92","reported_seq":136,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.058132+0000","last_change":"2023-10-06T03:57:10.341132+0000","last_active":"2023-10-06T04:10:48.058132+0000","last_peered":"2023-10-06T04:10:48.058132+0000","last_clean":"2023-10-06T04:10:48.058132+0000","last_became_active":"2023-10-06T03:57:10.339424+0000","last_became_peered":"2023-10-06T03:57:10.339424+0000","last_unstale":"2023-10-06T04:10:48.058132+0000","last_undegraded":"2023-10-06T04:10:48.058132+0000","last_fullsized":"2023-10-06T04:10:48.058132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:39.675711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":19,"num_write":219,"num_write_kb":1011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"53'316","reported_seq":2411,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.032289+0000","last_change":"2023-10-06T04:05:02.722365+0000","last_active":"2023-10-06T04:10:48.032289+0000","last_peered":"2023-10-06T04:10:48.032289+0000","last_clean":"2023-10-06T04:10:48.032289+0000","last_became_active":"2023-10-06T03:57:11.372215+0000","last_became_peered":"2023-10-06T03:57:11.372215+0000","last_unstale":"2023-10-06T04:10:48.032289+0000","last_undegraded":"2023-10-06T04:10:48.032289+0000","last_fullsized":"2023-10-06T04:10:48.032289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:23.559480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020199100000000002,"stat_sum":{"num_bytes":14547344,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":28,"num_read_kb":4106,"num_write":306,"num_write_kb":14296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"53'204","reported_seq":257,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:44.658005+0000","last_change":"2023-10-06T03:57:10.344051+0000","last_active":"2023-10-06T04:10:44.658005+0000","last_peered":"2023-10-06T04:10:44.658005+0000","last_clean":"2023-10-06T04:10:44.658005+0000","last_became_active":"2023-10-06T03:57:10.343865+0000","last_became_peered":"2023-10-06T03:57:10.343865+0000","last_unstale":"2023-10-06T04:10:44.658005+0000","last_undegraded":"2023-10-06T04:10:44.658005+0000","last_fullsized":"2023-10-06T04:10:44.658005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:32.522470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":11,"num_read_kb":32771,"num_write":206,"num_write_kb":12488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"53'331","reported_seq":2431,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.948651+0000","last_change":"2023-10-06T04:05:03.151764+0000","last_active":"2023-10-06T04:10:47.948651+0000","last_peered":"2023-10-06T04:10:47.948651+0000","last_clean":"2023-10-06T04:10:47.948651+0000","last_became_active":"2023-10-06T03:57:11.369681+0000","last_became_peered":"2023-10-06T03:57:11.369681+0000","last_unstale":"2023-10-06T04:10:47.948651+0000","last_undegraded":"2023-10-06T04:10:47.948651+0000","last_fullsized":"2023-10-06T04:10:47.948651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:53.858179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013151409999999999,"stat_sum":{"num_bytes":9592740,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":33,"num_read_kb":14,"num_write":324,"num_write_kb":13639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"50'100","reported_seq":149,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:52.634119+0000","last_change":"2023-10-06T03:57:10.349666+0000","last_active":"2023-10-06T04:09:52.634119+0000","last_peered":"2023-10-06T04:09:52.634119+0000","last_clean":"2023-10-06T04:09:52.634119+0000","last_became_active":"2023-10-06T03:57:10.349320+0000","last_became_peered":"2023-10-06T03:57:10.349320+0000","last_unstale":"2023-10-06T04:09:52.634119+0000","last_undegraded":"2023-10-06T04:09:52.634119+0000","last_fullsized":"2023-10-06T04:09:52.634119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:45:49.709519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194858,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":4110,"num_write":99,"num_write_kb":4143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"53'317","reported_seq":2411,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.096552+0000","last_change":"2023-10-06T04:05:02.723622+0000","last_active":"2023-10-06T04:10:48.096552+0000","last_peered":"2023-10-06T04:10:48.096552+0000","last_clean":"2023-10-06T04:10:48.096552+0000","last_became_active":"2023-10-06T03:57:11.370229+0000","last_became_peered":"2023-10-06T03:57:11.370229+0000","last_unstale":"2023-10-06T04:10:48.096552+0000","last_undegraded":"2023-10-06T04:10:48.096552+0000","last_fullsized":"2023-10-06T04:10:48.096552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:26.174331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027851170000000001,"stat_sum":{"num_bytes":11772234,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":27,"num_read_kb":17,"num_write":309,"num_write_kb":11655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"53'238","reported_seq":299,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:23.094832+0000","last_change":"2023-10-06T03:57:10.350263+0000","last_active":"2023-10-06T04:10:23.094832+0000","last_peered":"2023-10-06T04:10:23.094832+0000","last_clean":"2023-10-06T04:10:23.094832+0000","last_became_active":"2023-10-06T03:57:10.350083+0000","last_became_peered":"2023-10-06T03:57:10.350083+0000","last_unstale":"2023-10-06T04:10:23.094832+0000","last_undegraded":"2023-10-06T04:10:23.094832+0000","last_fullsized":"2023-10-06T04:10:23.094832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:05.433032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":36,"num_read_kb":45128,"num_write":264,"num_write_kb":12512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"53'333","reported_seq":2430,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.062537+0000","last_change":"2023-10-06T04:05:02.724048+0000","last_active":"2023-10-06T04:10:48.062537+0000","last_peered":"2023-10-06T04:10:48.062537+0000","last_clean":"2023-10-06T04:10:48.062537+0000","last_became_active":"2023-10-06T03:57:11.369123+0000","last_became_peered":"2023-10-06T03:57:11.369123+0000","last_unstale":"2023-10-06T04:10:48.062537+0000","last_undegraded":"2023-10-06T04:10:48.062537+0000","last_fullsized":"2023-10-06T04:10:48.062537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:28.139036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028613789999999998,"stat_sum":{"num_bytes":15016756,"num_objects":144,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":0,"num_read":30,"num_read_kb":4112,"num_write":318,"num_write_kb":14827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"52'181","reported_seq":225,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:47.407717+0000","last_change":"2023-10-06T03:57:10.347049+0000","last_active":"2023-10-06T04:09:47.407717+0000","last_peered":"2023-10-06T04:09:47.407717+0000","last_clean":"2023-10-06T04:09:47.407717+0000","last_became_active":"2023-10-06T03:57:10.344583+0000","last_became_peered":"2023-10-06T03:57:10.344583+0000","last_unstale":"2023-10-06T04:09:47.407717+0000","last_undegraded":"2023-10-06T04:09:47.407717+0000","last_fullsized":"2023-10-06T04:09:47.407717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:32.280257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":177,"num_write_kb":8286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"53'324","reported_seq":2428,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.088100+0000","last_change":"2023-10-06T04:05:02.721502+0000","last_active":"2023-10-06T04:10:48.088100+0000","last_peered":"2023-10-06T04:10:48.088100+0000","last_clean":"2023-10-06T04:10:48.088100+0000","last_became_active":"2023-10-06T03:57:11.366500+0000","last_became_peered":"2023-10-06T03:57:11.366500+0000","last_unstale":"2023-10-06T04:10:48.088100+0000","last_undegraded":"2023-10-06T04:10:48.088100+0000","last_fullsized":"2023-10-06T04:10:48.088100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:24.714603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016878310000000001,"stat_sum":{"num_bytes":6012647,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":38,"num_read_kb":4113,"num_write":317,"num_write_kb":10135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"50'170","reported_seq":220,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:51.778904+0000","last_change":"2023-10-06T03:57:10.348170+0000","last_active":"2023-10-06T04:09:51.778904+0000","last_peered":"2023-10-06T04:09:51.778904+0000","last_clean":"2023-10-06T04:09:51.778904+0000","last_became_active":"2023-10-06T03:57:10.347897+0000","last_became_peered":"2023-10-06T03:57:10.347897+0000","last_unstale":"2023-10-06T04:09:51.778904+0000","last_undegraded":"2023-10-06T04:09:51.778904+0000","last_fullsized":"2023-10-06T04:09:51.778904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:40.756246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7191702,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":12,"num_read_kb":8202,"num_write":168,"num_write_kb":7111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"53'386","reported_seq":2487,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.032330+0000","last_change":"2023-10-06T04:05:02.721899+0000","last_active":"2023-10-06T04:10:48.032330+0000","last_peered":"2023-10-06T04:10:48.032330+0000","last_clean":"2023-10-06T04:10:48.032330+0000","last_became_active":"2023-10-06T03:57:11.364150+0000","last_became_peered":"2023-10-06T03:57:11.364150+0000","last_unstale":"2023-10-06T04:10:48.032330+0000","last_undegraded":"2023-10-06T04:10:48.032330+0000","last_fullsized":"2023-10-06T04:10:48.032330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:11.038080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001670408,"stat_sum":{"num_bytes":10918213,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":35,"num_read_kb":7178,"num_write":378,"num_write_kb":19053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"53'345","reported_seq":2466,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.795523+0000","last_change":"2023-10-06T04:05:03.542896+0000","last_active":"2023-10-06T04:10:47.795523+0000","last_peered":"2023-10-06T04:10:47.795523+0000","last_clean":"2023-10-06T04:10:47.795523+0000","last_became_active":"2023-10-06T03:57:11.366524+0000","last_became_peered":"2023-10-06T03:57:11.366524+0000","last_unstale":"2023-10-06T04:10:47.795523+0000","last_undegraded":"2023-10-06T04:10:47.795523+0000","last_fullsized":"2023-10-06T04:10:47.795523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:33.044858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017816290000000001,"stat_sum":{"num_bytes":22531000,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":54,"num_read_kb":16279,"num_write":337,"num_write_kb":22178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"53'349","reported_seq":2453,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.079994+0000","last_change":"2023-10-06T04:05:02.721559+0000","last_active":"2023-10-06T04:10:48.079994+0000","last_peered":"2023-10-06T04:10:48.079994+0000","last_clean":"2023-10-06T04:10:48.079994+0000","last_became_active":"2023-10-06T03:57:11.370848+0000","last_became_peered":"2023-10-06T03:57:11.370848+0000","last_unstale":"2023-10-06T04:10:48.079994+0000","last_undegraded":"2023-10-06T04:10:48.079994+0000","last_fullsized":"2023-10-06T04:10:48.079994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:15.919090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017239289999999999,"stat_sum":{"num_bytes":12150184,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":37,"num_read_kb":8210,"num_write":341,"num_write_kb":16096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"53'351","reported_seq":2457,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.032069+0000","last_change":"2023-10-06T04:05:02.723715+0000","last_active":"2023-10-06T04:10:48.032069+0000","last_peered":"2023-10-06T04:10:48.032069+0000","last_clean":"2023-10-06T04:10:48.032069+0000","last_became_active":"2023-10-06T03:57:11.363875+0000","last_became_peered":"2023-10-06T03:57:11.363875+0000","last_unstale":"2023-10-06T04:10:48.032069+0000","last_undegraded":"2023-10-06T04:10:48.032069+0000","last_fullsized":"2023-10-06T04:10:48.032069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:29.236344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028142470000000002,"stat_sum":{"num_bytes":14810267,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":39,"num_read_kb":4117,"num_write":340,"num_write_kb":14623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"53'337","reported_seq":2445,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:48.153308+0000","last_change":"2023-10-06T04:05:02.722555+0000","last_active":"2023-10-06T04:10:48.153308+0000","last_peered":"2023-10-06T04:10:48.153308+0000","last_clean":"2023-10-06T04:10:48.153308+0000","last_became_active":"2023-10-06T03:57:11.377126+0000","last_became_peered":"2023-10-06T03:57:11.377126+0000","last_unstale":"2023-10-06T04:10:48.153308+0000","last_undegraded":"2023-10-06T04:10:48.153308+0000","last_fullsized":"2023-10-06T04:10:48.153308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:05.041737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023666569999999999,"stat_sum":{"num_bytes":6069899,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":41,"num_read_kb":18,"num_write":330,"num_write_kb":6093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"53'310","reported_seq":2407,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:47.768352+0000","last_change":"2023-10-06T04:05:02.972860+0000","last_active":"2023-10-06T04:10:47.768352+0000","last_peered":"2023-10-06T04:10:47.768352+0000","last_clean":"2023-10-06T04:10:47.768352+0000","last_became_active":"2023-10-06T03:57:59.256086+0000","last_became_peered":"2023-10-06T03:57:59.256086+0000","last_unstale":"2023-10-06T04:10:47.768352+0000","last_undegraded":"2023-10-06T04:10:47.768352+0000","last_fullsized":"2023-10-06T04:10:47.768352+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:54.263909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018573859999999999,"stat_sum":{"num_bytes":11282404,"num_objects":135,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":0,"num_read":33,"num_read_kb":11479,"num_write":302,"num_write_kb":14403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":918737492,"num_objects":9845,"num_object_clones":0,"num_object_copies":19690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9845,"num_whiteouts":0,"num_read":2243,"num_read_kb":380500,"num_write":20424,"num_write_kb":1040919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1675485184,"data_stored":1835934888,"data_compressed":157714556,"data_compressed_allocated":174235648,"data_compressed_original":386113536,"omap_allocated":0,"internal_metadata":0},"log_size":20998,"ondisk_log_size":20998,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":160325534,"num_objects":200,"num_object_clones":0,"num_object_copies":400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":200,"num_whiteouts":0,"num_read":555,"num_read_kb":254049,"num_write":6759,"num_write_kb":191312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":122,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81215488,"data_stored":320651068,"data_compressed":43195038,"data_compressed_allocated":54370304,"data_compressed_original":294060032,"omap_allocated":0,"internal_metadata":0},"log_size":6506,"ondisk_log_size":6506,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64858,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575018,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506268,"kb_used_data":208980,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93865572,"statfs":{"total":96636764160,"available":96118345728,"internally_reserved":0,"allocated":213995520,"data_stored":258704768,"data_compressed":22101424,"data_compressed_allocated":25088000,"data_compressed_original":75468800,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.55400000000000005,"15min":0.54500000000000004},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28399999999999997},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.88900000000000001},"last":0.50600000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52200000000000002,"15min":0.52100000000000002},"min":{"1min":0.38400000000000001,"5min":0.312,"15min":0.28899999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.78000000000000003},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54300000000000004,"15min":0.52300000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78300000000000003,"5min":1.25,"15min":1.25},"last":0.48199999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54800000000000004,"15min":0.54900000000000004},"min":{"1min":0.33400000000000002,"5min":0.29799999999999999,"15min":0.28399999999999997},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57599999999999996,"15min":0.60699999999999998},"min":{"1min":0.34300000000000003,"5min":0.33700000000000002,"15min":0.33100000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":7.3739999999999997},"last":0.56299999999999994}]},{"osd":3,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.318},"max":{"1min":0.80600000000000005,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.55400000000000005,"15min":0.55900000000000005},"min":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72899999999999998,"5min":0.78000000000000003,"15min":0.85299999999999998},"last":0.45000000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.44,"15min":0.46400000000000002},"min":{"1min":0.30099999999999999,"5min":0.27400000000000002,"15min":0.251},"max":{"1min":0.64000000000000001,"5min":0.73599999999999999,"15min":1.0149999999999999},"last":0.433},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.34300000000000003,"5min":0.33500000000000002,"15min":0.28299999999999997},"max":{"1min":0.68600000000000005,"5min":0.80200000000000005,"15min":0.90700000000000003},"last":0.53700000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.495,"15min":0.51100000000000001},"min":{"1min":0.316,"5min":0.29999999999999999,"15min":0.27300000000000002},"max":{"1min":0.74199999999999999,"5min":0.82999999999999996,"15min":0.86099999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54600000000000004,"15min":0.56299999999999994},"min":{"1min":0.40699999999999997,"5min":0.317,"15min":0.28000000000000003},"max":{"1min":0.73699999999999999,"5min":0.84899999999999998,"15min":1.377},"last":0.46800000000000003}]},{"osd":6,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.51100000000000001,"15min":0.56200000000000006},"min":{"1min":0.34100000000000003,"5min":0.30599999999999999,"15min":0.29999999999999999},"max":{"1min":0.77100000000000002,"5min":0.77600000000000002,"15min":1.3420000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60499999999999998,"15min":0.59099999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.30199999999999999},"max":{"1min":0.84599999999999997,"5min":0.88200000000000001,"15min":0.93700000000000006},"last":0.61699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575017,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473876,"kb_used_data":164300,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":93897964,"statfs":{"total":96636764160,"available":96151515136,"internally_reserved":0,"allocated":168243200,"data_stored":187372207,"data_compressed":15745085,"data_compressed_allocated":17555456,"data_compressed_original":42110976,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54800000000000004,"15min":0.53200000000000003},"min":{"1min":0.28799999999999998,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.70399999999999996,"5min":2.613,"15min":2.613},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.58399999999999996,"15min":0.55800000000000005},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.33300000000000002},"max":{"1min":0.72299999999999998,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":0.68999999999999995}]},{"osd":1,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53400000000000003,"15min":0.59599999999999997},"min":{"1min":0.28899999999999998,"5min":0.28499999999999998,"15min":0.27200000000000002},"max":{"1min":0.71599999999999997,"5min":2.3889999999999998,"15min":9.5519999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.59399999999999997,"15min":0.60199999999999998},"min":{"1min":0.33400000000000002,"5min":0.315,"15min":0.313},"max":{"1min":0.75800000000000001,"5min":6.8019999999999996,"15min":6.8019999999999996},"last":0.66600000000000004}]},{"osd":2,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.625,"15min":0.67300000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.29799999999999999},"max":{"1min":0.71799999999999997,"5min":5.7590000000000003,"15min":9.6769999999999996},"last":0.627},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.623,"15min":0.67600000000000005},"min":{"1min":0.42199999999999999,"5min":0.33200000000000002,"15min":0.31},"max":{"1min":0.75,"5min":5.71,"15min":9.7309999999999999},"last":0.70199999999999996}]},{"osd":3,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.54900000000000004,"15min":0.57499999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.29299999999999998},"max":{"1min":0.73899999999999999,"5min":0.99099999999999999,"15min":5.7359999999999998},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.59699999999999998,"15min":0.61299999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.308},"max":{"1min":0.68799999999999994,"5min":1.0149999999999999,"15min":5.7009999999999996},"last":0.54700000000000004}]},{"osd":4,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.48099999999999998,"15min":0.49399999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.63300000000000001,"5min":0.78600000000000003,"15min":1.0129999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54100000000000004,"15min":0.54900000000000004},"min":{"1min":0.32600000000000001,"5min":0.313,"15min":0.30399999999999999},"max":{"1min":0.70299999999999996,"5min":0.79800000000000004,"15min":1.026},"last":0.39800000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.49299999999999999,"15min":0.50800000000000001},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.77800000000000002,"5min":0.78200000000000003,"15min":0.995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.66500000000000004,"5min":0.78500000000000003,"15min":0.83699999999999997},"last":0.65100000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.34899999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.77000000000000002,"5min":0.84199999999999997,"15min":0.98299999999999998},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56499999999999995,"15min":0.57799999999999996},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80500000000000005,"5min":0.84699999999999998,"15min":0.90800000000000003},"last":0.52200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575017,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576968,"kb_used_data":275584,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":93794872,"statfs":{"total":96636764160,"available":96045948928,"internally_reserved":0,"allocated":282198016,"data_stored":333666787,"data_compressed":32289943,"data_compressed_allocated":36245504,"data_compressed_original":96591872,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.91800000000000004,"15min":0.52000000000000002},"min":{"1min":0.22500000000000001,"5min":0.20799999999999999,"15min":0.192},"max":{"1min":0.56999999999999995,"5min":57.700000000000003,"15min":57.700000000000003},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.65200000000000002,"15min":0.441},"min":{"1min":0.23899999999999999,"5min":0.23300000000000001,"15min":0.186},"max":{"1min":0.64000000000000001,"5min":28.629999999999999,"15min":28.629999999999999},"last":0.32600000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.97899999999999998,"15min":0.57199999999999995},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.19600000000000001},"max":{"1min":0.59899999999999998,"5min":58.795000000000002,"15min":58.795000000000002},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.67600000000000005,"15min":0.48999999999999999},"min":{"1min":0.32200000000000001,"5min":0.29599999999999999,"15min":0.23000000000000001},"max":{"1min":0.63400000000000001,"5min":25.722000000000001,"15min":25.722000000000001},"last":0.378}]},{"osd":3,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.59799999999999998,"15min":0.443},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.216},"max":{"1min":0.58499999999999996,"5min":22.920999999999999,"15min":22.920999999999999},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.439,"5min":0.63400000000000001,"15min":0.46800000000000003},"min":{"1min":0.29099999999999998,"5min":0.27300000000000002,"15min":0.22600000000000001},"max":{"1min":0.59799999999999998,"5min":22.797999999999998,"15min":22.797999999999998},"last":0.39200000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":1.2370000000000001,"15min":0.82399999999999995},"min":{"1min":0.443,"5min":0.38500000000000001,"15min":0.34499999999999997},"max":{"1min":1.0149999999999999,"5min":59.018999999999998,"15min":59.018999999999998},"last":0.624},{"interface":"front","average":{"1min":0.70999999999999996,"5min":1.26,"15min":0.83499999999999996},"min":{"1min":0.50700000000000001,"5min":0.47399999999999998,"15min":0.316},"max":{"1min":1.0609999999999999,"5min":59.046999999999997,"15min":59.046999999999997},"last":0.65200000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":1.21,"15min":0.79300000000000004},"min":{"1min":0.48599999999999999,"5min":0.40200000000000002,"15min":0.34399999999999997},"max":{"1min":0.90300000000000002,"5min":58.987000000000002,"15min":58.987000000000002},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65700000000000003,"15min":0.59599999999999997},"min":{"1min":0.44900000000000001,"5min":0.42599999999999999,"15min":0.29399999999999998},"max":{"1min":0.97699999999999998,"5min":6.2140000000000004,"15min":6.2140000000000004},"last":0.53700000000000003}]},{"osd":6,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65200000000000002,"15min":0.60399999999999998},"min":{"1min":0.44,"5min":0.41999999999999998,"15min":0.36599999999999999},"max":{"1min":0.95599999999999996,"5min":4.6020000000000003,"15min":4.6020000000000003},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":1.3,"15min":0.85999999999999999},"min":{"1min":0.49099999999999999,"5min":0.42799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94799999999999995,"5min":58.911999999999999,"15min":58.911999999999999},"last":0.61199999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":1.2210000000000001,"15min":0.81100000000000005},"min":{"1min":0.45800000000000002,"5min":0.40100000000000002,"15min":0.38700000000000001},"max":{"1min":1.1000000000000001,"5min":58.953000000000003,"15min":58.953000000000003},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.65900000000000003,"15min":0.62},"min":{"1min":0.46700000000000003,"5min":0.38100000000000001,"15min":0.34399999999999997},"max":{"1min":0.83199999999999996,"5min":4.5570000000000004,"15min":4.5570000000000004},"last":0.55800000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575017,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495552,"kb_used_data":190072,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":93876288,"statfs":{"total":96636764160,"available":96129318912,"internally_reserved":0,"allocated":194633728,"data_stored":251414553,"data_compressed":26053766,"data_compressed_allocated":29863936,"data_compressed_original":92942336,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.34100000000000003,"15min":0.34899999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.19500000000000001},"max":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":6.9539999999999997},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.35599999999999998,"15min":0.371},"min":{"1min":0.22700000000000001,"5min":0.215,"15min":0.191},"max":{"1min":0.55700000000000005,"5min":0.59499999999999997,"15min":8.0519999999999996},"last":0.27200000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.36199999999999999,"15min":0.35899999999999999},"min":{"1min":0.24299999999999999,"5min":0.24199999999999999,"15min":0.214},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":1.0589999999999999},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.38500000000000001,"15min":0.376},"min":{"1min":0.28299999999999997,"5min":0.26100000000000001,"15min":0.20799999999999999},"max":{"1min":0.626,"5min":0.626,"15min":2.0489999999999999},"last":0.36599999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.38600000000000001,"15min":0.41799999999999998},"min":{"1min":0.23300000000000001,"5min":0.23100000000000001,"15min":0.223},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":9.391},"last":0.311},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.42599999999999999,"15min":0.45600000000000002},"min":{"1min":0.31900000000000001,"5min":0.29099999999999998,"15min":0.27800000000000002},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":9.5239999999999991},"last":0.38100000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.64900000000000002,"15min":0.63400000000000001},"min":{"1min":0.35599999999999998,"5min":0.34000000000000002,"15min":0.29199999999999998},"max":{"1min":1.026,"5min":1.026,"15min":5.7699999999999996},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66000000000000003,"15min":0.63600000000000001},"min":{"1min":0.45400000000000001,"5min":0.41499999999999998,"15min":0.29499999999999998},"max":{"1min":0.997,"5min":0.997,"15min":2.7320000000000002},"last":0.57599999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.45000000000000001,"5min":0.41399999999999998,"15min":0.33800000000000002},"max":{"1min":0.89900000000000002,"5min":0.90500000000000003,"15min":5.5069999999999997},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63100000000000001,"15min":0.60499999999999998},"min":{"1min":0.436,"5min":0.38500000000000001,"15min":0.34799999999999998},"max":{"1min":0.872,"5min":0.878,"15min":1.05},"last":0.53800000000000003}]},{"osd":6,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.626,"15min":0.64700000000000002},"min":{"1min":0.42599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":6.9139999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64800000000000002,"15min":0.65500000000000003},"min":{"1min":0.46700000000000003,"5min":0.42199999999999999,"15min":0.379},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":6.3780000000000001},"last":0.44}]},{"osd":7,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61099999999999999,"15min":0.623},"min":{"1min":0.38500000000000001,"5min":0.377,"15min":0.36699999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":5.9989999999999997},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.627,"15min":0.61099999999999999},"min":{"1min":0.443,"5min":0.40100000000000002,"15min":0.39700000000000002},"max":{"1min":0.83999999999999997,"5min":0.86499999999999999,"15min":2.6930000000000001},"last":0.52400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575017,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":437036,"kb_used_data":139748,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93934804,"statfs":{"total":96636764160,"available":96189239296,"internally_reserved":0,"allocated":143101952,"data_stored":173375985,"data_compressed":18899047,"data_compressed_allocated":21323776,"data_compressed_original":57229312,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.34300000000000003,"15min":0.38400000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.19},"max":{"1min":0.51000000000000001,"5min":0.54300000000000004,"15min":14.968999999999999},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.376,"15min":0.41299999999999998},"min":{"1min":0.24099999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.52100000000000002,"5min":0.59199999999999997,"15min":14.746},"last":0.34000000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.37,"15min":0.52200000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.21099999999999999},"max":{"1min":0.53300000000000003,"5min":0.54100000000000004,"15min":50.677999999999997},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.45800000000000002,"15min":0.60099999999999998},"min":{"1min":0.34599999999999997,"5min":0.28599999999999998,"15min":0.26000000000000001},"max":{"1min":0.57599999999999996,"5min":0.66200000000000003,"15min":50.615000000000002},"last":0.35999999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.37,"15min":0.45900000000000002},"min":{"1min":0.25600000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.48999999999999999,"5min":0.61299999999999999,"15min":17.440999999999999},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.44,"5min":0.44700000000000001,"15min":0.59899999999999998},"min":{"1min":0.32400000000000001,"5min":0.29099999999999998,"15min":0.23499999999999999},"max":{"1min":0.65500000000000003,"5min":0.88900000000000001,"15min":50.424999999999997},"last":0.307}]},{"osd":4,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59499999999999997,"15min":0.64400000000000002},"min":{"1min":0.41099999999999998,"5min":0.34200000000000003,"15min":0.29699999999999999},"max":{"1min":0.76800000000000002,"5min":0.84799999999999998,"15min":14.802},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60399999999999998,"15min":0.67500000000000004},"min":{"1min":0.42199999999999999,"5min":0.39400000000000002,"15min":0.36499999999999999},"max":{"1min":0.746,"5min":0.88700000000000001,"15min":14.621},"last":0.432}]},{"osd":5,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60399999999999998,"15min":0.63200000000000001},"min":{"1min":0.48399999999999999,"5min":0.40100000000000002,"15min":0.35199999999999998},"max":{"1min":0.75700000000000001,"5min":0.84399999999999997,"15min":14.792999999999999},"last":0.499},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60899999999999999,"15min":0.76000000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.371},"max":{"1min":0.72499999999999998,"5min":0.92600000000000005,"15min":50.537999999999997},"last":0.57699999999999996}]},{"osd":6,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63700000000000001,"15min":0.68700000000000006},"min":{"1min":0.44700000000000001,"5min":0.40699999999999997,"15min":0.39100000000000001},"max":{"1min":0.78700000000000003,"5min":1.216,"15min":23.388000000000002},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64300000000000002,"15min":0.69999999999999996},"min":{"1min":0.50600000000000001,"5min":0.40400000000000003,"15min":0.39100000000000001},"max":{"1min":0.75,"5min":0.86899999999999999,"15min":14.856},"last":0.51900000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.627,"15min":0.67900000000000005},"min":{"1min":0.45600000000000002,"5min":0.40999999999999998,"15min":0.377},"max":{"1min":0.80900000000000005,"5min":0.95899999999999996,"15min":14.693},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63500000000000001,"15min":0.67400000000000004},"min":{"1min":0.45800000000000002,"5min":0.44600000000000001,"15min":0.434},"max":{"1min":0.79600000000000004,"5min":0.95499999999999996,"15min":10.349},"last":0.53600000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575017,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584240,"kb_used_data":274664,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":93787600,"statfs":{"total":96636764160,"available":96038502400,"internally_reserved":0,"allocated":281255936,"data_stored":341245364,"data_compressed":32144254,"data_compressed_allocated":36605952,"data_compressed_original":105447424,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.34399999999999997,"15min":0.35999999999999999},"min":{"1min":0.24299999999999999,"5min":0.216,"15min":0.186},"max":{"1min":0.5,"5min":0.58699999999999997,"15min":7.0999999999999996},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.35499999999999998,"15min":0.41699999999999998},"min":{"1min":0.27200000000000002,"5min":0.221,"15min":0.218},"max":{"1min":0.622,"5min":0.622,"15min":14.257},"last":0.36299999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.34999999999999998,"15min":0.35399999999999998},"min":{"1min":0.23100000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":3.1760000000000002},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.40899999999999997,"15min":0.39600000000000002},"min":{"1min":0.29999999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.61499999999999999,"5min":0.68200000000000005,"15min":5.2599999999999998},"last":0.308}]},{"osd":2,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.39800000000000002,"15min":0.45900000000000002},"min":{"1min":0.316,"5min":0.25700000000000001,"15min":0.23499999999999999},"max":{"1min":0.57599999999999996,"5min":0.747,"15min":13.138},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.42699999999999999,"15min":0.44500000000000001},"min":{"1min":0.32800000000000001,"5min":0.28299999999999997,"15min":0.27200000000000002},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":7.181},"last":0.40000000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.627,"15min":0.68100000000000005},"min":{"1min":0.46800000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":13.068},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.65600000000000003,"15min":0.67500000000000004},"min":{"1min":0.57399999999999995,"5min":0.434,"15min":0.39800000000000002},"max":{"1min":0.878,"5min":0.89100000000000001,"15min":6.4299999999999997},"last":0.73599999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.47799999999999998,"5min":0.35999999999999999,"15min":0.32700000000000001},"max":{"1min":0.89000000000000001,"5min":0.95599999999999996,"15min":3.105},"last":0.436},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.59799999999999998,"15min":0.60799999999999998},"min":{"1min":0.52000000000000002,"5min":0.372,"15min":0.29299999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":3.5369999999999999},"last":0.52400000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.63100000000000001,"15min":0.68799999999999994},"min":{"1min":0.54400000000000004,"5min":0.375,"15min":0.375},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":12.974},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63400000000000001,"15min":0.64100000000000001},"min":{"1min":0.56299999999999994,"5min":0.378,"15min":0.35599999999999998},"max":{"1min":0.93700000000000006,"5min":0.94499999999999995,"15min":3.0379999999999998},"last":0.501}]},{"osd":7,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.61299999999999999,"15min":0.63400000000000001},"min":{"1min":0.52900000000000003,"5min":0.41199999999999998,"15min":0.40100000000000002},"max":{"1min":0.93400000000000005,"5min":0.97199999999999998,"15min":6.2949999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.63400000000000001,"15min":0.67200000000000004},"min":{"1min":0.496,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90400000000000003,"5min":0.95399999999999996,"15min":13.032},"last":0.54200000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575018,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":580532,"kb_used_data":279148,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":93791308,"statfs":{"total":96636764160,"available":96042299392,"internally_reserved":0,"allocated":285847552,"data_stored":371153375,"data_compressed":31721463,"data_compressed_allocated":36786176,"data_compressed_original":130072576,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52900000000000003,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68100000000000005,"5min":0.747,"15min":0.79600000000000004},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.56699999999999995,"15min":0.54600000000000004},"min":{"1min":0.47699999999999998,"5min":0.35099999999999998,"15min":0.26600000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.80000000000000004},"last":0.72799999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.41199999999999998,"5min":0.34899999999999998,"15min":0.26200000000000001},"max":{"1min":0.73899999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.501,"15min":0.51100000000000001},"min":{"1min":0.35199999999999998,"5min":0.30399999999999999,"15min":0.27400000000000002},"max":{"1min":0.70399999999999996,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.40400000000000003,"5min":0.35899999999999999,"15min":0.312},"max":{"1min":0.69099999999999995,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.57599999999999996},"min":{"1min":0.441,"5min":0.36299999999999999,"15min":0.311},"max":{"1min":0.76700000000000002,"5min":1.413,"15min":1.413},"last":0.70699999999999996}]},{"osd":3,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.59099999999999997,"15min":0.55100000000000005},"min":{"1min":0.38400000000000001,"5min":0.32100000000000001,"15min":0.314},"max":{"1min":0.65800000000000003,"5min":6.7000000000000002,"15min":6.7000000000000002},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52900000000000003,"15min":0.54500000000000004},"min":{"1min":0.36899999999999999,"5min":0.36399999999999999,"15min":0.32000000000000001},"max":{"1min":0.72299999999999998,"5min":1.208,"15min":1.208},"last":0.63}]},{"osd":5,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.44600000000000001,"15min":0.47699999999999998},"min":{"1min":0.28599999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.59299999999999997,"5min":0.77100000000000002,"15min":0.81899999999999995},"last":0.436},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52200000000000002,"15min":0.53000000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.29499999999999998},"max":{"1min":0.66800000000000004,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.48299999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.48499999999999999,"15min":0.51300000000000001},"min":{"1min":0.315,"5min":0.30099999999999999,"15min":0.28100000000000003},"max":{"1min":0.60499999999999998,"5min":1.236,"15min":1.236},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.42899999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.69199999999999995,"5min":1.381,"15min":1.381},"last":0.64700000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53200000000000003,"15min":0.54600000000000004},"min":{"1min":0.38600000000000001,"5min":0.32800000000000001,"15min":0.30599999999999999},"max":{"1min":0.66700000000000004,"5min":0.71799999999999997,"15min":0.83499999999999996},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.45900000000000002,"5min":0.39900000000000002,"15min":0.32600000000000001},"max":{"1min":0.73599999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.57899999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575017,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485212,"kb_used_data":187924,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93886628,"statfs":{"total":96636764160,"available":96139907072,"internally_reserved":0,"allocated":192434176,"data_stored":243297716,"data_compressed":22019470,"data_compressed_allocated":25227264,"data_compressed_original":81457152,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.50700000000000001,"15min":0.55100000000000005},"min":{"1min":0.34399999999999997,"5min":0.29999999999999999,"15min":0.23300000000000001},"max":{"1min":0.60199999999999998,"5min":1.0189999999999999,"15min":11.167},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.53800000000000003,"15min":0.57699999999999996},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.221},"max":{"1min":0.71899999999999997,"5min":1.0449999999999999,"15min":9.8659999999999997},"last":0.626}]},{"osd":1,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.64100000000000001,"5min":1.0780000000000001,"15min":1.1120000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55100000000000005,"15min":0.54000000000000004},"min":{"1min":0.35599999999999998,"5min":0.29999999999999999,"15min":0.28799999999999998},"max":{"1min":0.73999999999999999,"5min":1.028,"15min":2.524},"last":0.65600000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.52700000000000002,"15min":0.54000000000000004},"min":{"1min":0.39400000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.68999999999999995,"5min":0.98499999999999999,"15min":1.0389999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70399999999999996,"5min":0.999,"15min":1.1439999999999999},"last":0.53300000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.77300000000000002,"5min":0.97299999999999998,"15min":2.9359999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60999999999999999,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.377,"15min":0.28999999999999998},"max":{"1min":0.77500000000000002,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.67700000000000005}]},{"osd":4,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.45900000000000002,"15min":0.47999999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.26000000000000001},"max":{"1min":0.71099999999999997,"5min":0.73499999999999999,"15min":1.1619999999999999},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57699999999999996,"15min":0.54200000000000004},"min":{"1min":0.33700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.75600000000000001,"5min":0.78900000000000003,"15min":1.1950000000000001},"last":0.39300000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46899999999999997,"15min":0.503},"min":{"1min":0.35699999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65900000000000003,"5min":0.78300000000000003,"15min":0.94099999999999995},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56499999999999995,"15min":0.58099999999999996},"min":{"1min":0.41899999999999998,"5min":0.38400000000000001,"15min":0.32900000000000001},"max":{"1min":0.72599999999999998,"5min":0.84999999999999998,"15min":0.97899999999999998},"last":0.51600000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.53700000000000003},"min":{"1min":0.38700000000000001,"5min":0.33500000000000002,"15min":0.28699999999999998},"max":{"1min":0.81999999999999995,"5min":0.84899999999999998,"15min":1.1830000000000001},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.438,"5min":0.375,"15min":0.311},"max":{"1min":0.79900000000000004,"5min":0.81299999999999994,"15min":1.29},"last":0.40899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13369344,"data_stored":49512965,"data_compressed":6345911,"data_compressed_allocated":8089600,"data_compressed_original":44290048,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7880704,"data_stored":33731560,"data_compressed":4481765,"data_compressed_allocated":5713920,"data_compressed_original":31571968,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3530752,"data_stored":15209155,"data_compressed":2097150,"data_compressed_allocated":2711552,"data_compressed_original":14401536,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12541952,"data_stored":45232784,"data_compressed":5646645,"data_compressed_allocated":7327744,"data_compressed_original":40050688,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20615168,"data_stored":82714194,"data_compressed":11662389,"data_compressed_allocated":14422016,"data_compressed_original":76578816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10690560,"data_stored":43917090,"data_compressed":5801730,"data_compressed_allocated":7286784,"data_compressed_original":40554496,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1855488,"data_stored":8388720,"data_compressed":1228027,"data_compressed_allocated":1515520,"data_compressed_original":8056832,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":10731520,"data_stored":41944600,"data_compressed":5931421,"data_compressed_allocated":7303168,"data_compressed_original":38555648,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":180592640,"data_stored":201004046,"data_compressed":19675426,"data_compressed_allocated":21729280,"data_compressed_original":48078848,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":273707008,"data_stored":299627100,"data_compressed":27808178,"data_compressed_allocated":30531584,"data_compressed_original":65019904,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":138960896,"data_stored":157858703,"data_compressed":16801897,"data_compressed_allocated":18612224,"data_compressed_original":42827776,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":268103680,"data_stored":295704453,"data_compressed":26497609,"data_compressed_allocated":29278208,"data_compressed_original":65396736,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":264622080,"data_stored":288131054,"data_compressed":20059074,"data_compressed_allocated":22364160,"data_compressed_original":53493760,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":181133312,"data_stored":199072499,"data_compressed":16217740,"data_compressed_allocated":17940480,"data_compressed_original":40902656,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":165777408,"data_stored":178675360,"data_compressed":14517058,"data_compressed_allocated":16039936,"data_compressed_original":34054144,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":202588160,"data_stored":215861673,"data_compressed":16137574,"data_compressed_allocated":17739776,"data_compressed_original":36339712,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:10:57.592 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:10:57.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.706+0000 7fddb537d700 1 Processor -- start 2023-10-06T04:10:57.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.706+0000 7fddb537d700 1 -- start start 2023-10-06T04:10:57.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.707+0000 7fddb537d700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fddb00b1e40 0x7fddb0157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:57.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.707+0000 7fddb537d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fddb006c2d0 0x7fddb006dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:57.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.707+0000 7fddb537d700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddb006d8d0 con 0x7fddb00b04d0 2023-10-06T04:10:57.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.707+0000 7fddb537d700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddb00af8f0 con 0x7fddb006c2d0 2023-10-06T04:10:57.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.707+0000 7fddb537d700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddb015a690 con 0x7fddb00b1e40 2023-10-06T04:10:57.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.707+0000 7fddb4b7c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fddb006c2d0 0x7fddb006dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:57.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.707+0000 7fddaf7fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fddb00b04d0 0x7fddb01563b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:35344/0 (socket says 172.21.15.157:35344) 2023-10-06T04:10:57.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.707+0000 7fddaf7fe700 1 -- 172.21.15.157:0/3825447490 learned_addr learned my addr 172.21.15.157:0/3825447490 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:10:57.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.707+0000 7fddaffff700 1 --2- 172.21.15.157:0/3825447490 >> v2:172.21.15.175:3300/0 conn(0x7fddb00b1e40 0x7fddb0157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:57.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.707+0000 7fddb4b7c700 1 -- 172.21.15.157:0/3825447490 >> v2:172.21.15.175:3300/0 conn(0x7fddb00b1e40 msgr2=0x7fddb0157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:10:57.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.707+0000 7fddb4b7c700 1 --2- 172.21.15.157:0/3825447490 >> v2:172.21.15.175:3300/0 conn(0x7fddb00b1e40 0x7fddb0157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:57.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.707+0000 7fddb4b7c700 1 -- 172.21.15.157:0/3825447490 >> v1:172.21.15.157:6789/0 conn(0x7fddb00b04d0 legacy=0x7fddb01563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:57.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.707+0000 7fddaeffd700 1 -- 172.21.15.157:0/3825447490 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 841229304 0 0) 0x7fddb006d8d0 con 0x7fddb00b04d0 2023-10-06T04:10:57.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.707+0000 7fddb4b7c700 1 -- 172.21.15.157:0/3825447490 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb006ccc0 con 0x7fddb006c2d0 2023-10-06T04:10:57.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.707+0000 7fddaffff700 1 --2- 172.21.15.157:0/3825447490 >> v2:172.21.15.175:3300/0 conn(0x7fddb00b1e40 0x7fddb0157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:10:57.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.708+0000 7fddb4b7c700 1 --2- 172.21.15.157:0/3825447490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddb006c2d0 0x7fddb006dd60 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fddb00cc450 tx=0x7fdda8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5755d94e2d7a0b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:57.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.708+0000 7fddaeffd700 1 -- 172.21.15.157:0/3825447490 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda800e070 con 0x7fddb006c2d0 2023-10-06T04:10:57.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.708+0000 7fddaeffd700 1 -- 172.21.15.157:0/3825447490 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdda800bc30 con 0x7fddb006c2d0 2023-10-06T04:10:57.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.708+0000 7fddaeffd700 1 -- 172.21.15.157:0/3825447490 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda8012430 con 0x7fddb006c2d0 2023-10-06T04:10:57.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.708+0000 7fddb537d700 1 -- 172.21.15.157:0/3825447490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddb006c2d0 msgr2=0x7fddb006dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:57.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.708+0000 7fddb537d700 1 --2- 172.21.15.157:0/3825447490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddb006c2d0 0x7fddb006dd60 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fddb00cc450 tx=0x7fdda8009780 comp rx=0 tx=0).stop 2023-10-06T04:10:57.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.708+0000 7fddb537d700 1 -- 172.21.15.157:0/3825447490 shutdown_connections 2023-10-06T04:10:57.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.708+0000 7fddb537d700 1 --2- 172.21.15.157:0/3825447490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddb006c2d0 0x7fddb006dd60 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:57.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.708+0000 7fddb537d700 1 --2- 172.21.15.157:0/3825447490 >> v2:172.21.15.175:3300/0 conn(0x7fddb00b1e40 0x7fddb0157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:57.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.708+0000 7fddb537d700 1 -- 172.21.15.157:0/3825447490 >> 172.21.15.157:0/3825447490 conn(0x7fddb00cc540 msgr2=0x7fddb00cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:57.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.708+0000 7fddb537d700 1 -- 172.21.15.157:0/3825447490 shutdown_connections 2023-10-06T04:10:57.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.708+0000 7fddb537d700 1 -- 172.21.15.157:0/3825447490 wait complete. 2023-10-06T04:10:57.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.708+0000 7fddb537d700 1 Processor -- start 2023-10-06T04:10:57.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.709+0000 7fddb537d700 1 -- start start 2023-10-06T04:10:57.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.709+0000 7fddb537d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddb006c2d0 0x7fddb00c1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:57.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.709+0000 7fddb537d700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddb00b04d0 0x7fddb00c2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:57.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.709+0000 7fddb537d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddb00b1e40 0x7fddb0168f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:57.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.709+0000 7fddb537d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb00c79b0 con 0x7fddb00b1e40 2023-10-06T04:10:57.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.709+0000 7fddb537d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb00c1950 con 0x7fddb006c2d0 2023-10-06T04:10:57.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.709+0000 7fddb537d700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb00c76d0 con 0x7fddb00b04d0 2023-10-06T04:10:57.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.709+0000 7fddb4b7c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddb00b1e40 0x7fddb0168f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:57.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.709+0000 7fddb4b7c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddb00b1e40 0x7fddb0168f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51608/0 (socket says 172.21.15.157:51608) 2023-10-06T04:10:57.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.709+0000 7fddb4b7c700 1 -- 172.21.15.157:0/1525616424 learned_addr learned my addr 172.21.15.157:0/1525616424 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:10:57.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.709+0000 7fddaffff700 1 --2- 172.21.15.157:0/1525616424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddb006c2d0 0x7fddb00c1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:58.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.709+0000 7fddaf7fe700 1 --2- 172.21.15.157:0/1525616424 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddb00b04d0 0x7fddb00c2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:58.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.710+0000 7fddb4b7c700 1 -- 172.21.15.157:0/1525616424 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddb00b04d0 msgr2=0x7fddb00c2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:58.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.710+0000 7fddb4b7c700 1 --2- 172.21.15.157:0/1525616424 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddb00b04d0 0x7fddb00c2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:58.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.710+0000 7fddb4b7c700 1 -- 172.21.15.157:0/1525616424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddb006c2d0 msgr2=0x7fddb00c1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:58.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.710+0000 7fddb4b7c700 1 --2- 172.21.15.157:0/1525616424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddb006c2d0 0x7fddb00c1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:58.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.710+0000 7fddb4b7c700 1 -- 172.21.15.157:0/1525616424 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda8009a20 con 0x7fddb00b1e40 2023-10-06T04:10:58.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.710+0000 7fddaf7fe700 1 --2- 172.21.15.157:0/1525616424 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddb00b04d0 0x7fddb00c2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:10:58.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.710+0000 7fddb4b7c700 1 --2- 172.21.15.157:0/1525616424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddb00b1e40 0x7fddb0168f10 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fdda800b6c0 tx=0x7fdda8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:58.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.710+0000 7fdd97fff700 1 -- 172.21.15.157:0/1525616424 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda8007000 con 0x7fddb00b1e40 2023-10-06T04:10:58.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.710+0000 7fdd97fff700 1 -- 172.21.15.157:0/1525616424 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdda80263f0 con 0x7fddb00b1e40 2023-10-06T04:10:58.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.710+0000 7fddb537d700 1 -- 172.21.15.157:0/1525616424 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb00bff30 con 0x7fddb00b1e40 2023-10-06T04:10:58.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.710+0000 7fdd97fff700 1 -- 172.21.15.157:0/1525616424 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda80267b0 con 0x7fddb00b1e40 2023-10-06T04:10:58.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.710+0000 7fddb537d700 1 -- 172.21.15.157:0/1525616424 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddb00c0210 con 0x7fddb00b1e40 2023-10-06T04:10:58.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.712+0000 7fddb537d700 1 -- 172.21.15.157:0/1525616424 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb00c1b10 con 0x7fddb00b1e40 2023-10-06T04:10:58.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.712+0000 7fdd97fff700 1 -- 172.21.15.157:0/1525616424 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdda8026be0 con 0x7fddb00b1e40 2023-10-06T04:10:58.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.713+0000 7fdd97fff700 1 --2- 172.21.15.157:0/1525616424 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdd9c066f60 0x7fdd9c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:58.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.713+0000 7fdd97fff700 1 -- 172.21.15.157:0/1525616424 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7fdda80864f0 con 0x7fddb00b1e40 2023-10-06T04:10:58.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.713+0000 7fddaffff700 1 --2- 172.21.15.157:0/1525616424 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdd9c066f60 0x7fdd9c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:58.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.713+0000 7fddaffff700 1 --2- 172.21.15.157:0/1525616424 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdd9c066f60 0x7fdd9c069410 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdda000adc0 tx=0x7fdda0006f90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:58.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.717+0000 7fdd97fff700 1 -- 172.21.15.157:0/1525616424 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdda8053370 con 0x7fddb00b1e40 2023-10-06T04:10:58.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.905+0000 7fddb537d700 1 -- 172.21.15.157:0/1525616424 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fddb0088f90 con 0x7fddb00b1e40 2023-10-06T04:10:58.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.906+0000 7fdd97fff700 1 -- 172.21.15.157:0/1525616424 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+17955 (secure 0 0 0) 0x7fdda8056990 con 0x7fddb00b1e40 2023-10-06T04:10:58.124 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:10:58.125 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":53,"fsid":"28664a7a-fe30-4efe-be5a-6abf9318b700","created":"2023-10-06T03:56:51.887538+0000","modified":"2023-10-06T04:09:46.762195+0000","last_up_change":"2023-10-06T03:57:02.299440+0000","last_in_change":"2023-10-06T03:56:55.141505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:57:03.455121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:57:03.905661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:57:08.377186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:57:09.709582+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":505,"snap_epoch":48,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f677cac-e9f4-4608-acac-718b072ce984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":90640},{"type":"v1","addr":"172.21.15.157:6809","nonce":90640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":90640},{"type":"v1","addr":"172.21.15.157:6811","nonce":90640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":90640},{"type":"v1","addr":"172.21.15.157:6815","nonce":90640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":90640},{"type":"v1","addr":"172.21.15.157:6813","nonce":90640}]},"public_addr":"172.21.15.157:6809/90640","cluster_addr":"172.21.15.157:6811/90640","heartbeat_back_addr":"172.21.15.157:6815/90640","heartbeat_front_addr":"172.21.15.157:6813/90640","state":["exists","up"]},{"osd":1,"uuid":"7d771892-bb91-4e9b-9610-051861c11247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":90654},{"type":"v1","addr":"172.21.15.157:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":90654},{"type":"v1","addr":"172.21.15.157:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":90654},{"type":"v1","addr":"172.21.15.157:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":90654},{"type":"v1","addr":"172.21.15.157:6805","nonce":90654}]},"public_addr":"172.21.15.157:6801/90654","cluster_addr":"172.21.15.157:6803/90654","heartbeat_back_addr":"172.21.15.157:6807/90654","heartbeat_front_addr":"172.21.15.157:6805/90654","state":["exists","up"]},{"osd":2,"uuid":"5c1f7d4e-d185-47cf-ab6c-453b4e3009c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":90657},{"type":"v1","addr":"172.21.15.157:6825","nonce":90657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":90657},{"type":"v1","addr":"172.21.15.157:6827","nonce":90657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":90657},{"type":"v1","addr":"172.21.15.157:6831","nonce":90657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":90657},{"type":"v1","addr":"172.21.15.157:6829","nonce":90657}]},"public_addr":"172.21.15.157:6825/90657","cluster_addr":"172.21.15.157:6827/90657","heartbeat_back_addr":"172.21.15.157:6831/90657","heartbeat_front_addr":"172.21.15.157:6829/90657","state":["exists","up"]},{"osd":3,"uuid":"f6a1e825-43c7-4afd-8daf-e51d45afc75e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":90641},{"type":"v1","addr":"172.21.15.157:6817","nonce":90641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":90641},{"type":"v1","addr":"172.21.15.157:6819","nonce":90641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":90641},{"type":"v1","addr":"172.21.15.157:6823","nonce":90641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":90641},{"type":"v1","addr":"172.21.15.157:6821","nonce":90641}]},"public_addr":"172.21.15.157:6817/90641","cluster_addr":"172.21.15.157:6819/90641","heartbeat_back_addr":"172.21.15.157:6823/90641","heartbeat_front_addr":"172.21.15.157:6821/90641","state":["exists","up"]},{"osd":4,"uuid":"8ad6e3d8-9a31-4f85-9410-417b7ac5da97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":91261},{"type":"v1","addr":"172.21.15.175:6803","nonce":91261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":91261},{"type":"v1","addr":"172.21.15.175:6806","nonce":91261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":91261},{"type":"v1","addr":"172.21.15.175:6814","nonce":91261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":91261},{"type":"v1","addr":"172.21.15.175:6810","nonce":91261}]},"public_addr":"172.21.15.175:6803/91261","cluster_addr":"172.21.15.175:6806/91261","heartbeat_back_addr":"172.21.15.175:6814/91261","heartbeat_front_addr":"172.21.15.175:6810/91261","state":["exists","up"]},{"osd":5,"uuid":"f68aca2a-f182-4c00-8e21-241d09c02ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":91262},{"type":"v1","addr":"172.21.15.175:6817","nonce":91262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":91262},{"type":"v1","addr":"172.21.15.175:6819","nonce":91262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":91262},{"type":"v1","addr":"172.21.15.175:6823","nonce":91262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":91262},{"type":"v1","addr":"172.21.15.175:6821","nonce":91262}]},"public_addr":"172.21.15.175:6817/91262","cluster_addr":"172.21.15.175:6819/91262","heartbeat_back_addr":"172.21.15.175:6823/91262","heartbeat_front_addr":"172.21.15.175:6821/91262","state":["exists","up"]},{"osd":6,"uuid":"d867cd89-3430-40ca-862f-d2bea3de9d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":91263},{"type":"v1","addr":"172.21.15.175:6801","nonce":91263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6805","nonce":91263},{"type":"v1","addr":"172.21.15.175:6807","nonce":91263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6813","nonce":91263},{"type":"v1","addr":"172.21.15.175:6815","nonce":91263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":91263},{"type":"v1","addr":"172.21.15.175:6811","nonce":91263}]},"public_addr":"172.21.15.175:6801/91263","cluster_addr":"172.21.15.175:6807/91263","heartbeat_back_addr":"172.21.15.175:6815/91263","heartbeat_front_addr":"172.21.15.175:6811/91263","state":["exists","up"]},{"osd":7,"uuid":"930cd464-79f1-47cb-b4ff-188ccde74fff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":91306},{"type":"v1","addr":"172.21.15.175:6825","nonce":91306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":91306},{"type":"v1","addr":"172.21.15.175:6827","nonce":91306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":91306},{"type":"v1","addr":"172.21.15.175:6831","nonce":91306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":91306},{"type":"v1","addr":"172.21.15.175:6829","nonce":91306}]},"public_addr":"172.21.15.175:6825/91306","cluster_addr":"172.21.15.175:6827/91306","heartbeat_back_addr":"172.21.15.175:6831/91306","heartbeat_front_addr":"172.21.15.175:6829/91306","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.290673+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.056390+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.292903+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.275861+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.332095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.327453+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.317585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.309330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.24","mappings":[{"from":1,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6834/765157057":"2023-10-07T04:09:46.762128+0000","172.21.15.175:6836/2388164729":"2023-10-07T04:07:31.736793+0000","172.21.15.157:6837/4188652695":"2023-10-07T04:05:26.713520+0000","172.21.15.175:6839/4177493830":"2023-10-07T03:59:06.641459+0000","172.21.15.175:6838/4177493830":"2023-10-07T03:59:06.641459+0000","172.21.15.157:6832/1179753185":"2023-10-07T04:04:26.701047+0000","172.21.15.157:6832/765157057":"2023-10-07T04:09:46.762128+0000","172.21.15.157:6833/3932336008":"2023-10-07T04:01:01.662287+0000","172.21.15.157:6837/3065749287":"2023-10-07T04:08:36.748456+0000","172.21.15.175:6837/2388164729":"2023-10-07T04:07:31.736793+0000","172.21.15.157:6836/4188652695":"2023-10-07T04:05:26.713520+0000","172.21.15.157:6835/3932336008":"2023-10-07T04:01:01.662287+0000","172.21.15.157:6836/3065749287":"2023-10-07T04:08:36.748456+0000","172.21.15.157:6834/1179753185":"2023-10-07T04:04:26.701047+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:10:58.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.908+0000 7fddb537d700 1 -- 172.21.15.157:0/1525616424 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdd9c066f60 msgr2=0x7fdd9c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:58.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.908+0000 7fddb537d700 1 --2- 172.21.15.157:0/1525616424 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdd9c066f60 0x7fdd9c069410 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdda000adc0 tx=0x7fdda0006f90 comp rx=0 tx=0).stop 2023-10-06T04:10:58.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.908+0000 7fddb537d700 1 -- 172.21.15.157:0/1525616424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddb00b1e40 msgr2=0x7fddb0168f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:58.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.909+0000 7fddb537d700 1 --2- 172.21.15.157:0/1525616424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddb00b1e40 0x7fddb0168f10 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fdda800b6c0 tx=0x7fdda8019000 comp rx=0 tx=0).stop 2023-10-06T04:10:58.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.909+0000 7fddb537d700 1 -- 172.21.15.157:0/1525616424 shutdown_connections 2023-10-06T04:10:58.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.909+0000 7fddb537d700 1 --2- 172.21.15.157:0/1525616424 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fdd9c066f60 0x7fdd9c069410 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:58.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.909+0000 7fddb537d700 1 --2- 172.21.15.157:0/1525616424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddb006c2d0 0x7fddb00c1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:58.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.909+0000 7fddb537d700 1 --2- 172.21.15.157:0/1525616424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fddb00b1e40 0x7fddb0168f10 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:58.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.909+0000 7fddb537d700 1 --2- 172.21.15.157:0/1525616424 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddb00b04d0 0x7fddb00c2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:58.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.909+0000 7fddb537d700 1 -- 172.21.15.157:0/1525616424 >> 172.21.15.157:0/1525616424 conn(0x7fddb00cc540 msgr2=0x7fddb00ccb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:58.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.909+0000 7fddb537d700 1 -- 172.21.15.157:0/1525616424 shutdown_connections 2023-10-06T04:10:58.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:57.909+0000 7fddb537d700 1 -- 172.21.15.157:0/1525616424 wait complete. 2023-10-06T04:10:59.130 INFO:tasks.ceph:Scrubbing osd.0 2023-10-06T04:10:59.130 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-06T04:10:59.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.260+0000 7f476afa5700 1 Processor -- start 2023-10-06T04:10:59.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.260+0000 7f476afa5700 1 -- start start 2023-10-06T04:10:59.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.261+0000 7f476afa5700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f47640c8b30 0x7f4764155f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.261+0000 7f476afa5700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47640c7040 con 0x7f47641504b0 2023-10-06T04:10:59.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.261+0000 7f476afa5700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47640af4c0 con 0x7f47641500e0 2023-10-06T04:10:59.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.261+0000 7f476afa5700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47640c9000 con 0x7f47640c8b30 2023-10-06T04:10:59.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.261+0000 7f4763fff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f47641504b0 0x7f47640b1c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:35356/0 (socket says 172.21.15.157:35356) 2023-10-06T04:10:59.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.261+0000 7f4763fff700 1 -- 172.21.15.157:0/2254566268 learned_addr learned my addr 172.21.15.157:0/2254566268 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:10:59.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.261+0000 7f4769542700 1 --2- 172.21.15.157:0/2254566268 >> v2:172.21.15.175:3300/0 conn(0x7f47640c8b30 0x7f4764155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.261+0000 7f47637fe700 1 -- 172.21.15.157:0/2254566268 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2106800388 0 0) 0x7f47640c7040 con 0x7f47641504b0 2023-10-06T04:10:59.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.261+0000 7f47637fe700 1 -- 172.21.15.157:0/2254566268 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4754003570 con 0x7f47641504b0 2023-10-06T04:10:59.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.261+0000 7f47637fe700 1 -- 172.21.15.157:0/2254566268 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1891673408 0 0) 0x7f4754003570 con 0x7f47641504b0 2023-10-06T04:10:59.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.261+0000 7f47637fe700 1 -- 172.21.15.157:0/2254566268 >> v2:172.21.15.175:3300/0 conn(0x7f47640c8b30 msgr2=0x7f4764155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:10:59.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.261+0000 7f47637fe700 1 --2- 172.21.15.157:0/2254566268 >> v2:172.21.15.175:3300/0 conn(0x7f47640c8b30 0x7f4764155f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.261+0000 7f47637fe700 1 -- 172.21.15.157:0/2254566268 >> v1:172.21.15.175:6789/0 conn(0x7f47641500e0 legacy=0x7f47640c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:59.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.261+0000 7f47637fe700 1 -- 172.21.15.157:0/2254566268 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47640c92e0 con 0x7f47641504b0 2023-10-06T04:10:59.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.261+0000 7f47637fe700 1 -- 172.21.15.157:0/2254566268 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f4750001470 con 0x7f47641504b0 2023-10-06T04:10:59.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.262+0000 7f47637fe700 1 -- 172.21.15.157:0/2254566268 >> v1:172.21.15.157:6789/0 conn(0x7f47641504b0 legacy=0x7f47640b1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:59.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.262+0000 7f47637fe700 1 --2- 172.21.15.157:0/2254566268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4754003a60 0x7f4754003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.262+0000 7f47637fe700 1 --2- 172.21.15.157:0/2254566268 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4754004810 0x7f4754004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.262+0000 7f47637fe700 1 --2- 172.21.15.157:0/2254566268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47540051c0 0x7f475400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.262+0000 7f47637fe700 1 -- 172.21.15.157:0/2254566268 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47640c9000 con 0x7f4754003a60 2023-10-06T04:10:59.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.262+0000 7f47637fe700 1 -- 172.21.15.157:0/2254566268 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475400c0d0 con 0x7f47540051c0 2023-10-06T04:10:59.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.262+0000 7f47637fe700 1 -- 172.21.15.157:0/2254566268 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f475400c290 con 0x7f4754004810 2023-10-06T04:10:59.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.262+0000 7f4768d41700 1 --2- 172.21.15.157:0/2254566268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4754003a60 0x7f4754003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.262+0000 7f4763fff700 1 --2- 172.21.15.157:0/2254566268 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4754004810 0x7f4754004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.262+0000 7f4769542700 1 --2- 172.21.15.157:0/2254566268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47540051c0 0x7f475400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.262+0000 7f4768d41700 1 -- 172.21.15.157:0/2254566268 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4754004810 msgr2=0x7f4754004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:10:59.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.262+0000 7f4768d41700 1 --2- 172.21.15.157:0/2254566268 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4754004810 0x7f4754004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.262+0000 7f4768d41700 1 -- 172.21.15.157:0/2254566268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47540051c0 msgr2=0x7f475400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:10:59.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.262+0000 7f4768d41700 1 --2- 172.21.15.157:0/2254566268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47540051c0 0x7f475400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.262+0000 7f4768d41700 1 -- 172.21.15.157:0/2254566268 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4754003570 con 0x7f4754003a60 2023-10-06T04:10:59.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.262+0000 7f4769542700 1 -- 172.21.15.157:0/2254566268 reap_dead start 2023-10-06T04:10:59.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.262+0000 7f4768d41700 1 --2- 172.21.15.157:0/2254566268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4754003a60 0x7f4754003e70 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f47580018b0 tx=0x7f4758007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27813e7d0e56b2f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:59.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.262+0000 7f47637fe700 1 -- 172.21.15.157:0/2254566268 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4758012420 con 0x7f4754003a60 2023-10-06T04:10:59.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.262+0000 7f47637fe700 1 -- 172.21.15.157:0/2254566268 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47580125a0 con 0x7f4754003a60 2023-10-06T04:10:59.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.262+0000 7f47637fe700 1 -- 172.21.15.157:0/2254566268 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4758012960 con 0x7f4754003a60 2023-10-06T04:10:59.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.263+0000 7f476afa5700 1 -- 172.21.15.157:0/2254566268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4754003a60 msgr2=0x7f4754003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:59.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.263+0000 7f476afa5700 1 --2- 172.21.15.157:0/2254566268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4754003a60 0x7f4754003e70 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f47580018b0 tx=0x7f4758007d30 comp rx=0 tx=0).stop 2023-10-06T04:10:59.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.263+0000 7f476afa5700 1 -- 172.21.15.157:0/2254566268 shutdown_connections 2023-10-06T04:10:59.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.263+0000 7f476afa5700 1 --2- 172.21.15.157:0/2254566268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4754003a60 0x7f4754003e70 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.263+0000 7f476afa5700 1 -- 172.21.15.157:0/2254566268 >> 172.21.15.157:0/2254566268 conn(0x7f476406f230 msgr2=0x7f476406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:59.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.263+0000 7f476afa5700 1 -- 172.21.15.157:0/2254566268 shutdown_connections 2023-10-06T04:10:59.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.263+0000 7f476afa5700 1 -- 172.21.15.157:0/2254566268 wait complete. 2023-10-06T04:10:59.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.264+0000 7f476afa5700 1 Processor -- start 2023-10-06T04:10:59.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.264+0000 7f476afa5700 1 -- start start 2023-10-06T04:10:59.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.265+0000 7f476afa5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4754003a60 0x7f47640c4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.265+0000 7f476afa5700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47640c43c0 0x7f47641559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.265+0000 7f476afa5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f476414b120 0x7f4764148830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.265+0000 7f476afa5700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47640c5530 con 0x7f476414b120 2023-10-06T04:10:59.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.265+0000 7f476afa5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4764155520 con 0x7f4754003a60 2023-10-06T04:10:59.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.265+0000 7f476afa5700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47640c52c0 con 0x7f47640c43c0 2023-10-06T04:10:59.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.265+0000 7f4769542700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f476414b120 0x7f4764148830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.265+0000 7f4768d41700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4754003a60 0x7f47640c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.265+0000 7f4769542700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f476414b120 0x7f4764148830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51634/0 (socket says 172.21.15.157:51634) 2023-10-06T04:10:59.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.265+0000 7f4769542700 1 -- 172.21.15.157:0/2309355960 learned_addr learned my addr 172.21.15.157:0/2309355960 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:10:59.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.265+0000 7f4763fff700 1 --2- 172.21.15.157:0/2309355960 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47640c43c0 0x7f47641559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.265+0000 7f4769542700 1 -- 172.21.15.157:0/2309355960 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47640c43c0 msgr2=0x7f47641559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:59.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.265+0000 7f4769542700 1 --2- 172.21.15.157:0/2309355960 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47640c43c0 0x7f47641559b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.265+0000 7f4769542700 1 -- 172.21.15.157:0/2309355960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4754003a60 msgr2=0x7f47640c4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:59.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.265+0000 7f4769542700 1 --2- 172.21.15.157:0/2309355960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4754003a60 0x7f47640c4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.265+0000 7f4769542700 1 -- 172.21.15.157:0/2309355960 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476406d990 con 0x7f476414b120 2023-10-06T04:10:59.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.265+0000 7f4768d41700 1 --2- 172.21.15.157:0/2309355960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4754003a60 0x7f47640c4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:10:59.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.265+0000 7f4769542700 1 --2- 172.21.15.157:0/2309355960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f476414b120 0x7f4764148830 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f47640b1b90 tx=0x7f475c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:59.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.265+0000 7f4760ff9700 1 -- 172.21.15.157:0/2309355960 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c00e040 con 0x7f476414b120 2023-10-06T04:10:59.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.265+0000 7f4760ff9700 1 -- 172.21.15.157:0/2309355960 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f475c00f040 con 0x7f476414b120 2023-10-06T04:10:59.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.266+0000 7f4760ff9700 1 -- 172.21.15.157:0/2309355960 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c00fdc0 con 0x7f476414b120 2023-10-06T04:10:59.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.266+0000 7f476afa5700 1 -- 172.21.15.157:0/2309355960 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4764154ed0 con 0x7f476414b120 2023-10-06T04:10:59.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.266+0000 7f476afa5700 1 -- 172.21.15.157:0/2309355960 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4764150be0 con 0x7f476414b120 2023-10-06T04:10:59.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.266+0000 7f476afa5700 1 -- 172.21.15.157:0/2309355960 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f47640af050 con 0x7f476414b120 2023-10-06T04:10:59.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.267+0000 7f4760ff9700 1 -- 172.21.15.157:0/2309355960 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f475c015640 con 0x7f476414b120 2023-10-06T04:10:59.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.267+0000 7f4760ff9700 1 --2- 172.21.15.157:0/2309355960 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4748066f30 0x7f47480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.267+0000 7f4760ff9700 1 -- 172.21.15.157:0/2309355960 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f475c085b30 con 0x7f476414b120 2023-10-06T04:10:59.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.268+0000 7f4760ff9700 1 --2- 172.21.15.157:0/2309355960 >> [v2:172.21.15.157:6808/90640,v1:172.21.15.157:6809/90640] conn(0x7f474806fd90 0x7f47480721c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.268+0000 7f4760ff9700 1 -- 172.21.15.157:0/2309355960 --> [v2:172.21.15.157:6808/90640,v1:172.21.15.157:6809/90640] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4748072890 con 0x7f474806fd90 2023-10-06T04:10:59.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.268+0000 7f4760ff9700 1 -- 172.21.15.157:0/2309355960 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f475c085e20 con 0x7f476414b120 2023-10-06T04:10:59.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.268+0000 7f4768d41700 1 --2- 172.21.15.157:0/2309355960 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4748066f30 0x7f47480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.268+0000 7f4763fff700 1 --2- 172.21.15.157:0/2309355960 >> [v2:172.21.15.157:6808/90640,v1:172.21.15.157:6809/90640] conn(0x7f474806fd90 0x7f47480721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.268+0000 7f4768d41700 1 --2- 172.21.15.157:0/2309355960 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4748066f30 0x7f47480693e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4758012ad0 tx=0x7f4758015000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:59.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.268+0000 7f4763fff700 1 --2- 172.21.15.157:0/2309355960 >> [v2:172.21.15.157:6808/90640,v1:172.21.15.157:6809/90640] conn(0x7f474806fd90 0x7f47480721c0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:59.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.269+0000 7f4760ff9700 1 -- 172.21.15.157:0/2309355960 <== osd.0 v2:172.21.15.157:6808/90640 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f4748072890 con 0x7f474806fd90 2023-10-06T04:10:59.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.282+0000 7f476afa5700 1 -- 172.21.15.157:0/2309355960 --> [v2:172.21.15.157:6808/90640,v1:172.21.15.157:6809/90640] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f47640c4790 con 0x7f474806fd90 2023-10-06T04:10:59.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.283+0000 7f4760ff9700 1 -- 172.21.15.157:0/2309355960 <== osd.0 v2:172.21.15.157:6808/90640 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f47640c4790 con 0x7f474806fd90 2023-10-06T04:10:59.285 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-10-06T04:10:59.285 INFO:teuthology.orchestra.run.smithi157.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T04:10:59.285 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-10-06T04:10:59.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.283+0000 7f476afa5700 1 -- 172.21.15.157:0/2309355960 >> [v2:172.21.15.157:6808/90640,v1:172.21.15.157:6809/90640] conn(0x7f474806fd90 msgr2=0x7f47480721c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:59.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.283+0000 7f476afa5700 1 --2- 172.21.15.157:0/2309355960 >> [v2:172.21.15.157:6808/90640,v1:172.21.15.157:6809/90640] conn(0x7f474806fd90 0x7f47480721c0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.283+0000 7f476afa5700 1 -- 172.21.15.157:0/2309355960 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4748066f30 msgr2=0x7f47480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:59.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.283+0000 7f476afa5700 1 --2- 172.21.15.157:0/2309355960 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f4748066f30 0x7f47480693e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4758012ad0 tx=0x7f4758015000 comp rx=0 tx=0).stop 2023-10-06T04:10:59.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.283+0000 7f476afa5700 1 -- 172.21.15.157:0/2309355960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f476414b120 msgr2=0x7f4764148830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:59.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.283+0000 7f476afa5700 1 --2- 172.21.15.157:0/2309355960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f476414b120 0x7f4764148830 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f47640b1b90 tx=0x7f475c00b040 comp rx=0 tx=0).stop 2023-10-06T04:10:59.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.283+0000 7f4769542700 1 -- 172.21.15.157:0/2309355960 reap_dead start 2023-10-06T04:10:59.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.283+0000 7f476afa5700 1 -- 172.21.15.157:0/2309355960 shutdown_connections 2023-10-06T04:10:59.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.283+0000 7f476afa5700 1 -- 172.21.15.157:0/2309355960 >> 172.21.15.157:0/2309355960 conn(0x7f476406f230 msgr2=0x7f47640c76b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:59.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.283+0000 7f476afa5700 1 -- 172.21.15.157:0/2309355960 shutdown_connections 2023-10-06T04:10:59.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.283+0000 7f476afa5700 1 -- 172.21.15.157:0/2309355960 wait complete. 2023-10-06T04:10:59.297 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-06T04:10:59.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.411+0000 7f06f3a5b700 1 Processor -- start 2023-10-06T04:10:59.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.411+0000 7f06f3a5b700 1 -- start start 2023-10-06T04:10:59.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.412+0000 7f06f3a5b700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f06ec06cbb0 0x7f06ec06d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.412+0000 7f06f3a5b700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f06ec06c1f0 0x7f06ec06da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.412+0000 7f06f3a5b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06ec06cf80 0x7f06ec14fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.412+0000 7f06f3a5b700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06ec0c8c40 con 0x7f06ec06c1f0 2023-10-06T04:10:59.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.412+0000 7f06f3a5b700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06ec0c9780 con 0x7f06ec06cbb0 2023-10-06T04:10:59.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.412+0000 7f06f3a5b700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06ec06e0c0 con 0x7f06ec06cf80 2023-10-06T04:10:59.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.412+0000 7f06f0ff6700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f06ec06c1f0 0x7f06ec06da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.412+0000 7f06f0ff6700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f06ec06c1f0 0x7f06ec06da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51640/0 (socket says 172.21.15.157:51640) 2023-10-06T04:10:59.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.412+0000 7f06f0ff6700 1 -- 172.21.15.157:0/1003232488 learned_addr learned my addr 172.21.15.157:0/1003232488 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:10:59.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.412+0000 7f06f17f7700 1 --2- 172.21.15.157:0/1003232488 >> v2:172.21.15.175:3300/0 conn(0x7f06ec06cbb0 0x7f06ec06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.412+0000 7f06f1ff8700 1 --2- 172.21.15.157:0/1003232488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06ec06cf80 0x7f06ec14fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.413+0000 7f06f0ff6700 1 -- 172.21.15.157:0/1003232488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06ec06cf80 msgr2=0x7f06ec14fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:10:59.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.413+0000 7f06f0ff6700 1 --2- 172.21.15.157:0/1003232488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06ec06cf80 0x7f06ec14fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.413+0000 7f06f0ff6700 1 -- 172.21.15.157:0/1003232488 >> v2:172.21.15.175:3300/0 conn(0x7f06ec06cbb0 msgr2=0x7f06ec06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:10:59.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.413+0000 7f06f0ff6700 1 --2- 172.21.15.157:0/1003232488 >> v2:172.21.15.175:3300/0 conn(0x7f06ec06cbb0 0x7f06ec06d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.413+0000 7f06f0ff6700 1 -- 172.21.15.157:0/1003232488 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ec0c8e50 con 0x7f06ec06c1f0 2023-10-06T04:10:59.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.413+0000 7f06f1ff8700 1 --2- 172.21.15.157:0/1003232488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06ec06cf80 0x7f06ec14fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:10:59.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.413+0000 7f06f17f7700 1 --2- 172.21.15.157:0/1003232488 >> v2:172.21.15.175:3300/0 conn(0x7f06ec06cbb0 0x7f06ec06d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:10:59.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.413+0000 7f06f0ff6700 1 --2- 172.21.15.157:0/1003232488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06ec06c1f0 0x7f06ec06da70 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f06ec06fd70 tx=0x7f06dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b226ec69ea3d5892 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:59.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.413+0000 7f06e3fff700 1 -- 172.21.15.157:0/1003232488 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06dc00b8f0 con 0x7f06ec06c1f0 2023-10-06T04:10:59.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.413+0000 7f06e3fff700 1 -- 172.21.15.157:0/1003232488 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06dc00ba50 con 0x7f06ec06c1f0 2023-10-06T04:10:59.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.413+0000 7f06e3fff700 1 -- 172.21.15.157:0/1003232488 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06dc00bdf0 con 0x7f06ec06c1f0 2023-10-06T04:10:59.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.414+0000 7f06f3a5b700 1 -- 172.21.15.157:0/1003232488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06ec06c1f0 msgr2=0x7f06ec06da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:59.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.414+0000 7f06f3a5b700 1 --2- 172.21.15.157:0/1003232488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06ec06c1f0 0x7f06ec06da70 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f06ec06fd70 tx=0x7f06dc009250 comp rx=0 tx=0).stop 2023-10-06T04:10:59.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.414+0000 7f06f3a5b700 1 -- 172.21.15.157:0/1003232488 shutdown_connections 2023-10-06T04:10:59.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.414+0000 7f06f3a5b700 1 --2- 172.21.15.157:0/1003232488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06ec06cf80 0x7f06ec14fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.414+0000 7f06f3a5b700 1 --2- 172.21.15.157:0/1003232488 >> v2:172.21.15.175:3300/0 conn(0x7f06ec06cbb0 0x7f06ec06d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.414+0000 7f06f3a5b700 1 --2- 172.21.15.157:0/1003232488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06ec06c1f0 0x7f06ec06da70 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.414+0000 7f06f3a5b700 1 -- 172.21.15.157:0/1003232488 >> 172.21.15.157:0/1003232488 conn(0x7f06ec06f230 msgr2=0x7f06ec06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:59.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.414+0000 7f06f3a5b700 1 -- 172.21.15.157:0/1003232488 shutdown_connections 2023-10-06T04:10:59.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.414+0000 7f06f3a5b700 1 -- 172.21.15.157:0/1003232488 wait complete. 2023-10-06T04:10:59.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.415+0000 7f06f3a5b700 1 Processor -- start 2023-10-06T04:10:59.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.415+0000 7f06f3a5b700 1 -- start start 2023-10-06T04:10:59.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.416+0000 7f06f3a5b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06ec06c1f0 0x7f06ec14e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.416+0000 7f06f3a5b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06ec06cbb0 0x7f06ec167420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.416+0000 7f06f3a5b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06ec06cf80 0x7f06ec0c22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.416+0000 7f06f3a5b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ec154d90 con 0x7f06ec06c1f0 2023-10-06T04:10:59.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.416+0000 7f06f3a5b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ec155ed0 con 0x7f06ec06cf80 2023-10-06T04:10:59.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.416+0000 7f06f3a5b700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06ec156360 con 0x7f06ec06cbb0 2023-10-06T04:10:59.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.416+0000 7f06f17f7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06ec06c1f0 0x7f06ec14e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.416+0000 7f06f17f7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06ec06c1f0 0x7f06ec14e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51652/0 (socket says 172.21.15.157:51652) 2023-10-06T04:10:59.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.416+0000 7f06f17f7700 1 -- 172.21.15.157:0/4289995826 learned_addr learned my addr 172.21.15.157:0/4289995826 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:10:59.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.416+0000 7f06f0ff6700 1 --2- 172.21.15.157:0/4289995826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06ec06cbb0 0x7f06ec167420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.416+0000 7f06f1ff8700 1 --2- 172.21.15.157:0/4289995826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06ec06cf80 0x7f06ec0c22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.416+0000 7f06f17f7700 1 -- 172.21.15.157:0/4289995826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06ec06cbb0 msgr2=0x7f06ec167420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:59.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.416+0000 7f06f17f7700 1 --2- 172.21.15.157:0/4289995826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06ec06cbb0 0x7f06ec167420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.416+0000 7f06f17f7700 1 -- 172.21.15.157:0/4289995826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06ec06cf80 msgr2=0x7f06ec0c22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:59.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.416+0000 7f06f17f7700 1 --2- 172.21.15.157:0/4289995826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06ec06cf80 0x7f06ec0c22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.416+0000 7f06f17f7700 1 -- 172.21.15.157:0/4289995826 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06dc0094f0 con 0x7f06ec06c1f0 2023-10-06T04:10:59.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.416+0000 7f06f17f7700 1 --2- 172.21.15.157:0/4289995826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06ec06c1f0 0x7f06ec14e020 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f06e800e900 tx=0x7f06e8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:59.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.416+0000 7f06e17fa700 1 -- 172.21.15.157:0/4289995826 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e800eea0 con 0x7f06ec06c1f0 2023-10-06T04:10:59.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.416+0000 7f06e17fa700 1 -- 172.21.15.157:0/4289995826 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e800c5e0 con 0x7f06ec06c1f0 2023-10-06T04:10:59.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.416+0000 7f06e17fa700 1 -- 172.21.15.157:0/4289995826 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e800c980 con 0x7f06ec06c1f0 2023-10-06T04:10:59.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.416+0000 7f06f3a5b700 1 -- 172.21.15.157:0/4289995826 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06ec154fe0 con 0x7f06ec06c1f0 2023-10-06T04:10:59.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.416+0000 7f06f3a5b700 1 -- 172.21.15.157:0/4289995826 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06ec1555a0 con 0x7f06ec06c1f0 2023-10-06T04:10:59.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.417+0000 7f06f3a5b700 1 -- 172.21.15.157:0/4289995826 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06ec1561b0 con 0x7f06ec06c1f0 2023-10-06T04:10:59.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.418+0000 7f06e17fa700 1 -- 172.21.15.157:0/4289995826 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f06e800cd30 con 0x7f06ec06c1f0 2023-10-06T04:10:59.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.418+0000 7f06e17fa700 1 --2- 172.21.15.157:0/4289995826 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f06d0066f60 0x7f06d0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.418+0000 7f06e17fa700 1 -- 172.21.15.157:0/4289995826 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f06e8084740 con 0x7f06ec06c1f0 2023-10-06T04:10:59.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.418+0000 7f06f0ff6700 1 --2- 172.21.15.157:0/4289995826 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f06d0066f60 0x7f06d0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.419+0000 7f06f0ff6700 1 --2- 172.21.15.157:0/4289995826 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f06d0066f60 0x7f06d0069410 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f06ec06fd70 tx=0x7f06dc017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:59.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.422+0000 7f06e17fa700 1 -- 172.21.15.157:0/4289995826 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f06e8055050 con 0x7f06ec06c1f0 2023-10-06T04:10:59.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.605+0000 7f06f3a5b700 1 -- 172.21.15.157:0/4289995826 --> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f06ec156010 con 0x7f06d0066f60 2023-10-06T04:10:59.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.606+0000 7f06e17fa700 1 -- 172.21.15.157:0/4289995826 <== mgr.4107 v2:172.21.15.175:6832/90661 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f06dc00bbc0 con 0x7f06d0066f60 2023-10-06T04:10:59.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.608+0000 7f06f3a5b700 1 -- 172.21.15.157:0/4289995826 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f06d0066f60 msgr2=0x7f06d0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:59.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.608+0000 7f06f3a5b700 1 --2- 172.21.15.157:0/4289995826 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f06d0066f60 0x7f06d0069410 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f06ec06fd70 tx=0x7f06dc017000 comp rx=0 tx=0).stop 2023-10-06T04:10:59.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.608+0000 7f06f3a5b700 1 -- 172.21.15.157:0/4289995826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06ec06c1f0 msgr2=0x7f06ec14e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:59.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.608+0000 7f06f3a5b700 1 --2- 172.21.15.157:0/4289995826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06ec06c1f0 0x7f06ec14e020 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f06e800e900 tx=0x7f06e8008040 comp rx=0 tx=0).stop 2023-10-06T04:10:59.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.608+0000 7f06f3a5b700 1 -- 172.21.15.157:0/4289995826 shutdown_connections 2023-10-06T04:10:59.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.608+0000 7f06f3a5b700 1 --2- 172.21.15.157:0/4289995826 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f06d0066f60 0x7f06d0069410 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.608+0000 7f06f3a5b700 1 --2- 172.21.15.157:0/4289995826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06ec06cf80 0x7f06ec0c22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.608+0000 7f06f3a5b700 1 --2- 172.21.15.157:0/4289995826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06ec06c1f0 0x7f06ec14e020 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.608+0000 7f06f3a5b700 1 --2- 172.21.15.157:0/4289995826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06ec06cbb0 0x7f06ec167420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.608+0000 7f06f3a5b700 1 -- 172.21.15.157:0/4289995826 >> 172.21.15.157:0/4289995826 conn(0x7f06ec06f230 msgr2=0x7f06ec0af630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:59.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.608+0000 7f06f3a5b700 1 -- 172.21.15.157:0/4289995826 shutdown_connections 2023-10-06T04:10:59.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.608+0000 7f06f3a5b700 1 -- 172.21.15.157:0/4289995826 wait complete. 2023-10-06T04:10:59.612 INFO:teuthology.orchestra.run.smithi157.stderr:instructed osd(s) 0 to deep-scrub 2023-10-06T04:10:59.623 INFO:tasks.ceph:Scrubbing osd.1 2023-10-06T04:10:59.623 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-06T04:10:59.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.740+0000 7f74c6d85700 1 Processor -- start 2023-10-06T04:10:59.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.740+0000 7f74c6d85700 1 -- start start 2023-10-06T04:10:59.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.741+0000 7f74c6d85700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74c01500e0 0x7f74c0155f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.741+0000 7f74c6d85700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f74c01504b0 0x7f74c00c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.741+0000 7f74c6d85700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f74c00c8b30 0x7f74c00b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.741+0000 7f74c6d85700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c00c7040 con 0x7f74c01504b0 2023-10-06T04:10:59.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.741+0000 7f74c6d85700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c00af4c0 con 0x7f74c00c8b30 2023-10-06T04:10:59.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.741+0000 7f74c6d85700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c00c9000 con 0x7f74c01500e0 2023-10-06T04:10:59.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.741+0000 7f74bffff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f74c01504b0 0x7f74c00c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.741+0000 7f74bffff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f74c01504b0 0x7f74c00c77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51664/0 (socket says 172.21.15.157:51664) 2023-10-06T04:10:59.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.741+0000 7f74bffff700 1 -- 172.21.15.157:0/871410072 learned_addr learned my addr 172.21.15.157:0/871410072 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:10:59.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.741+0000 7f74c4b21700 1 --2- 172.21.15.157:0/871410072 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74c01500e0 0x7f74c0155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.741+0000 7f74c5322700 1 --2- 172.21.15.157:0/871410072 >> v2:172.21.15.175:3300/0 conn(0x7f74c00c8b30 0x7f74c00b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.741+0000 7f74bffff700 1 -- 172.21.15.157:0/871410072 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74c01500e0 msgr2=0x7f74c0155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:10:59.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.741+0000 7f74bffff700 1 --2- 172.21.15.157:0/871410072 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74c01500e0 0x7f74c0155f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.741+0000 7f74bffff700 1 -- 172.21.15.157:0/871410072 >> v2:172.21.15.175:3300/0 conn(0x7f74c00c8b30 msgr2=0x7f74c00b1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:10:59.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.741+0000 7f74bffff700 1 --2- 172.21.15.157:0/871410072 >> v2:172.21.15.175:3300/0 conn(0x7f74c00c8b30 0x7f74c00b1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.742+0000 7f74bffff700 1 -- 172.21.15.157:0/871410072 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c00c92e0 con 0x7f74c01504b0 2023-10-06T04:10:59.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.742+0000 7f74c5322700 1 --2- 172.21.15.157:0/871410072 >> v2:172.21.15.175:3300/0 conn(0x7f74c00c8b30 0x7f74c00b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:10:59.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.742+0000 7f74bffff700 1 --2- 172.21.15.157:0/871410072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74c01504b0 0x7f74c00c77b0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f74c008a550 tx=0x7f74b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509b2fbfb64c2522 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:59.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.742+0000 7f74bf7fe700 1 -- 172.21.15.157:0/871410072 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b4009e70 con 0x7f74c01504b0 2023-10-06T04:10:59.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.742+0000 7f74bf7fe700 1 -- 172.21.15.157:0/871410072 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b400b860 con 0x7f74c01504b0 2023-10-06T04:10:59.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.742+0000 7f74bf7fe700 1 -- 172.21.15.157:0/871410072 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b400bc00 con 0x7f74c01504b0 2023-10-06T04:10:59.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.742+0000 7f74c6d85700 1 -- 172.21.15.157:0/871410072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74c01504b0 msgr2=0x7f74c00c77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:59.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.742+0000 7f74c6d85700 1 --2- 172.21.15.157:0/871410072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74c01504b0 0x7f74c00c77b0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f74c008a550 tx=0x7f74b4009250 comp rx=0 tx=0).stop 2023-10-06T04:10:59.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.742+0000 7f74c6d85700 1 -- 172.21.15.157:0/871410072 shutdown_connections 2023-10-06T04:10:59.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.742+0000 7f74c6d85700 1 --2- 172.21.15.157:0/871410072 >> v2:172.21.15.175:3300/0 conn(0x7f74c00c8b30 0x7f74c00b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.742+0000 7f74c6d85700 1 --2- 172.21.15.157:0/871410072 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74c01500e0 0x7f74c0155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.742+0000 7f74c6d85700 1 --2- 172.21.15.157:0/871410072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74c01504b0 0x7f74c00c77b0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.742+0000 7f74c6d85700 1 -- 172.21.15.157:0/871410072 >> 172.21.15.157:0/871410072 conn(0x7f74c006f230 msgr2=0x7f74c006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:59.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.742+0000 7f74c6d85700 1 -- 172.21.15.157:0/871410072 shutdown_connections 2023-10-06T04:10:59.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.742+0000 7f74c6d85700 1 -- 172.21.15.157:0/871410072 wait complete. 2023-10-06T04:10:59.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.743+0000 7f74c6d85700 1 Processor -- start 2023-10-06T04:10:59.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.744+0000 7f74c6d85700 1 -- start start 2023-10-06T04:10:59.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.744+0000 7f74c6d85700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74c00c8b30 0x7f74c01661a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.744+0000 7f74c6d85700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74c01500e0 0x7f74c00cb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.744+0000 7f74c6d85700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74c01504b0 0x7f74c00cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.744+0000 7f74c6d85700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c0166e10 con 0x7f74c01504b0 2023-10-06T04:10:59.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.744+0000 7f74c6d85700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c00cc880 con 0x7f74c00c8b30 2023-10-06T04:10:59.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.744+0000 7f74c6d85700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c0166b70 con 0x7f74c01500e0 2023-10-06T04:10:59.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.744+0000 7f74c5322700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74c01504b0 0x7f74c00cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.744+0000 7f74c5322700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74c01504b0 0x7f74c00cb6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51666/0 (socket says 172.21.15.157:51666) 2023-10-06T04:10:59.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.744+0000 7f74c5322700 1 -- 172.21.15.157:0/1372419428 learned_addr learned my addr 172.21.15.157:0/1372419428 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:10:59.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.745+0000 7f74c4b21700 1 --2- 172.21.15.157:0/1372419428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74c00c8b30 0x7f74c01661a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.745+0000 7f74bffff700 1 --2- 172.21.15.157:0/1372419428 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74c01500e0 0x7f74c00cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.745+0000 7f74c5322700 1 -- 172.21.15.157:0/1372419428 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74c01500e0 msgr2=0x7f74c00cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:59.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.745+0000 7f74c5322700 1 --2- 172.21.15.157:0/1372419428 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74c01500e0 0x7f74c00cb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.745+0000 7f74c5322700 1 -- 172.21.15.157:0/1372419428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74c00c8b30 msgr2=0x7f74c01661a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:59.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.745+0000 7f74c5322700 1 --2- 172.21.15.157:0/1372419428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74c00c8b30 0x7f74c01661a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.745+0000 7f74c5322700 1 -- 172.21.15.157:0/1372419428 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b40094f0 con 0x7f74c01504b0 2023-10-06T04:10:59.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.745+0000 7f74c5322700 1 --2- 172.21.15.157:0/1372419428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74c01504b0 0x7f74c00cb6d0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f74b800e8a0 tx=0x7f74b8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:59.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.745+0000 7f74bcff9700 1 -- 172.21.15.157:0/1372419428 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8013410 con 0x7f74c01504b0 2023-10-06T04:10:59.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.745+0000 7f74bcff9700 1 -- 172.21.15.157:0/1372419428 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b8013570 con 0x7f74c01504b0 2023-10-06T04:10:59.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.745+0000 7f74c6d85700 1 -- 172.21.15.157:0/1372419428 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c00ca6b0 con 0x7f74c01504b0 2023-10-06T04:10:59.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.745+0000 7f74bcff9700 1 -- 172.21.15.157:0/1372419428 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b80138f0 con 0x7f74c01504b0 2023-10-06T04:10:59.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.745+0000 7f74c6d85700 1 -- 172.21.15.157:0/1372419428 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c00cac70 con 0x7f74c01504b0 2023-10-06T04:10:59.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.746+0000 7f74c6d85700 1 -- 172.21.15.157:0/1372419428 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f74c0089ca0 con 0x7f74c01504b0 2023-10-06T04:10:59.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.746+0000 7f74bcff9700 1 -- 172.21.15.157:0/1372419428 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f74b8013ce0 con 0x7f74c01504b0 2023-10-06T04:10:59.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.747+0000 7f74bcff9700 1 --2- 172.21.15.157:0/1372419428 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f74ac066f30 0x7f74ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.747+0000 7f74bcff9700 1 -- 172.21.15.157:0/1372419428 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f74b8084530 con 0x7f74c01504b0 2023-10-06T04:10:59.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.747+0000 7f74bcff9700 1 --2- 172.21.15.157:0/1372419428 >> [v2:172.21.15.157:6800/90654,v1:172.21.15.157:6801/90654] conn(0x7f74ac06fd90 0x7f74ac0721c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.747+0000 7f74bcff9700 1 -- 172.21.15.157:0/1372419428 --> [v2:172.21.15.157:6800/90654,v1:172.21.15.157:6801/90654] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f74ac072890 con 0x7f74ac06fd90 2023-10-06T04:10:59.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.747+0000 7f74bcff9700 1 -- 172.21.15.157:0/1372419428 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74b8086b70 con 0x7f74c01504b0 2023-10-06T04:10:59.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.747+0000 7f74c4b21700 1 --2- 172.21.15.157:0/1372419428 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f74ac066f30 0x7f74ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.747+0000 7f74bffff700 1 --2- 172.21.15.157:0/1372419428 >> [v2:172.21.15.157:6800/90654,v1:172.21.15.157:6801/90654] conn(0x7f74ac06fd90 0x7f74ac0721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.747+0000 7f74c4b21700 1 --2- 172.21.15.157:0/1372419428 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f74ac066f30 0x7f74ac0693e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f74b000f570 tx=0x7f74b000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:59.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.748+0000 7f74bffff700 1 --2- 172.21.15.157:0/1372419428 >> [v2:172.21.15.157:6800/90654,v1:172.21.15.157:6801/90654] conn(0x7f74ac06fd90 0x7f74ac0721c0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:59.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.748+0000 7f74bcff9700 1 -- 172.21.15.157:0/1372419428 <== osd.1 v2:172.21.15.157:6800/90654 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f74ac072890 con 0x7f74ac06fd90 2023-10-06T04:10:59.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.762+0000 7f74c6d85700 1 -- 172.21.15.157:0/1372419428 --> [v2:172.21.15.157:6800/90654,v1:172.21.15.157:6801/90654] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f74c00c95c0 con 0x7f74ac06fd90 2023-10-06T04:10:59.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.762+0000 7f74bcff9700 1 -- 172.21.15.157:0/1372419428 <== osd.1 v2:172.21.15.157:6800/90654 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f74c00c95c0 con 0x7f74ac06fd90 2023-10-06T04:10:59.764 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-10-06T04:10:59.764 INFO:teuthology.orchestra.run.smithi157.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T04:10:59.764 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-10-06T04:10:59.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.763+0000 7f74c6d85700 1 -- 172.21.15.157:0/1372419428 >> [v2:172.21.15.157:6800/90654,v1:172.21.15.157:6801/90654] conn(0x7f74ac06fd90 msgr2=0x7f74ac0721c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:59.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.763+0000 7f74c6d85700 1 --2- 172.21.15.157:0/1372419428 >> [v2:172.21.15.157:6800/90654,v1:172.21.15.157:6801/90654] conn(0x7f74ac06fd90 0x7f74ac0721c0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.763+0000 7f74c6d85700 1 -- 172.21.15.157:0/1372419428 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f74ac066f30 msgr2=0x7f74ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:59.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.763+0000 7f74c6d85700 1 --2- 172.21.15.157:0/1372419428 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f74ac066f30 0x7f74ac0693e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f74b000f570 tx=0x7f74b000b040 comp rx=0 tx=0).stop 2023-10-06T04:10:59.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.763+0000 7f74c6d85700 1 -- 172.21.15.157:0/1372419428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74c01504b0 msgr2=0x7f74c00cb6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:59.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.763+0000 7f74c6d85700 1 --2- 172.21.15.157:0/1372419428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74c01504b0 0x7f74c00cb6d0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f74b800e8a0 tx=0x7f74b8008040 comp rx=0 tx=0).stop 2023-10-06T04:10:59.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.763+0000 7f74c5322700 1 -- 172.21.15.157:0/1372419428 reap_dead start 2023-10-06T04:10:59.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.763+0000 7f74c6d85700 1 -- 172.21.15.157:0/1372419428 shutdown_connections 2023-10-06T04:10:59.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.763+0000 7f74c6d85700 1 -- 172.21.15.157:0/1372419428 >> 172.21.15.157:0/1372419428 conn(0x7f74c006f230 msgr2=0x7f74c014d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:59.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.763+0000 7f74c6d85700 1 -- 172.21.15.157:0/1372419428 shutdown_connections 2023-10-06T04:10:59.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.763+0000 7f74c6d85700 1 -- 172.21.15.157:0/1372419428 wait complete. 2023-10-06T04:10:59.776 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-06T04:10:59.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.891+0000 7ff78a7e8700 1 Processor -- start 2023-10-06T04:10:59.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.891+0000 7ff78a7e8700 1 -- start start 2023-10-06T04:10:59.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.892+0000 7ff78a7e8700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff7840c5fc0 0x7ff784069130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.892+0000 7ff78a7e8700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff78406c810 0x7ff7841569e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.892+0000 7ff78a7e8700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff784155f20 con 0x7ff78406c440 2023-10-06T04:10:59.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.892+0000 7ff78a7e8700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff78406f310 con 0x7ff7840c5fc0 2023-10-06T04:10:59.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.892+0000 7ff78a7e8700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78406d4a0 con 0x7ff78406c810 2023-10-06T04:10:59.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.892+0000 7ff7837fe700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff7840c5fc0 0x7ff784069130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.892+0000 7ff788d85700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff78406c810 0x7ff7841569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.892+0000 7ff7837fe700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff7840c5fc0 0x7ff784069130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:44718/0 (socket says 172.21.15.157:44718) 2023-10-06T04:10:59.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.892+0000 7ff7837fe700 1 -- 172.21.15.157:0/3029741065 learned_addr learned my addr 172.21.15.157:0/3029741065 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:10:59.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.892+0000 7ff783fff700 1 --1- 172.21.15.157:0/3029741065 >> v1:172.21.15.175:6789/0 conn(0x7ff78406c440 0x7ff784152e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.157:36180/0 (socket says 172.21.15.157:36180) 2023-10-06T04:10:59.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.893+0000 7ff7837fe700 1 -- 172.21.15.157:0/3029741065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff78406c810 msgr2=0x7ff7841569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:10:59.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.893+0000 7ff7837fe700 1 --2- 172.21.15.157:0/3029741065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff78406c810 0x7ff7841569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.893+0000 7ff7837fe700 1 -- 172.21.15.157:0/3029741065 >> v1:172.21.15.175:6789/0 conn(0x7ff78406c440 legacy=0x7ff784152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:59.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.893+0000 7ff7837fe700 1 -- 172.21.15.157:0/3029741065 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78406d780 con 0x7ff7840c5fc0 2023-10-06T04:10:59.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.893+0000 7ff788d85700 1 --2- 172.21.15.157:0/3029741065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff78406c810 0x7ff7841569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:10:59.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.893+0000 7ff7837fe700 1 --2- 172.21.15.157:0/3029741065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7840c5fc0 0x7ff784069130 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff7780099b0 tx=0x7ff778009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e22c6bfa9c1c200b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:59.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.893+0000 7ff782ffd700 1 -- 172.21.15.157:0/3029741065 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff778009ea0 con 0x7ff7840c5fc0 2023-10-06T04:10:59.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.893+0000 7ff782ffd700 1 -- 172.21.15.157:0/3029741065 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77800b860 con 0x7ff7840c5fc0 2023-10-06T04:10:59.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.893+0000 7ff782ffd700 1 -- 172.21.15.157:0/3029741065 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77800bc00 con 0x7ff7840c5fc0 2023-10-06T04:10:59.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.894+0000 7ff78a7e8700 1 -- 172.21.15.157:0/3029741065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7840c5fc0 msgr2=0x7ff784069130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:59.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.894+0000 7ff78a7e8700 1 --2- 172.21.15.157:0/3029741065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7840c5fc0 0x7ff784069130 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff7780099b0 tx=0x7ff778009250 comp rx=0 tx=0).stop 2023-10-06T04:10:59.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.894+0000 7ff78a7e8700 1 -- 172.21.15.157:0/3029741065 shutdown_connections 2023-10-06T04:10:59.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.894+0000 7ff78a7e8700 1 --2- 172.21.15.157:0/3029741065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff78406c810 0x7ff7841569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.894+0000 7ff78a7e8700 1 --2- 172.21.15.157:0/3029741065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7840c5fc0 0x7ff784069130 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.894+0000 7ff78a7e8700 1 -- 172.21.15.157:0/3029741065 >> 172.21.15.157:0/3029741065 conn(0x7ff784068370 msgr2=0x7ff784068770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:10:59.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.894+0000 7ff78a7e8700 1 -- 172.21.15.157:0/3029741065 shutdown_connections 2023-10-06T04:10:59.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.894+0000 7ff78a7e8700 1 -- 172.21.15.157:0/3029741065 wait complete. 2023-10-06T04:10:59.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.895+0000 7ff78a7e8700 1 Processor -- start 2023-10-06T04:10:59.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.895+0000 7ff78a7e8700 1 -- start start 2023-10-06T04:10:59.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.895+0000 7ff78a7e8700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff78406c440 0x7ff7840c4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.895+0000 7ff78a7e8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff78406c810 0x7ff7840af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.896+0000 7ff78a7e8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7840c5fc0 0x7ff7840af800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.896+0000 7ff78a7e8700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7840c2b00 con 0x7ff78406c810 2023-10-06T04:10:59.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.896+0000 7ff78a7e8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7841552a0 con 0x7ff7840c5fc0 2023-10-06T04:10:59.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.896+0000 7ff78a7e8700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7840bfb30 con 0x7ff78406c440 2023-10-06T04:10:59.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.896+0000 7ff7837fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff78406c810 0x7ff7840af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.896+0000 7ff7837fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff78406c810 0x7ff7840af2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51674/0 (socket says 172.21.15.157:51674) 2023-10-06T04:10:59.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.896+0000 7ff7837fe700 1 -- 172.21.15.157:0/3303053696 learned_addr learned my addr 172.21.15.157:0/3303053696 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:10:59.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.896+0000 7ff788d85700 1 --2- 172.21.15.157:0/3303053696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7840c5fc0 0x7ff7840af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.896+0000 7ff783fff700 1 --2- 172.21.15.157:0/3303053696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff78406c440 0x7ff7840c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.896+0000 7ff7837fe700 1 -- 172.21.15.157:0/3303053696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff78406c440 msgr2=0x7ff7840c4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:59.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.896+0000 7ff7837fe700 1 --2- 172.21.15.157:0/3303053696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff78406c440 0x7ff7840c4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.896+0000 7ff7837fe700 1 -- 172.21.15.157:0/3303053696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7840c5fc0 msgr2=0x7ff7840af800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:10:59.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.896+0000 7ff7837fe700 1 --2- 172.21.15.157:0/3303053696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7840c5fc0 0x7ff7840af800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:10:59.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.896+0000 7ff7837fe700 1 -- 172.21.15.157:0/3303053696 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77c009a20 con 0x7ff78406c810 2023-10-06T04:10:59.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.896+0000 7ff7837fe700 1 --2- 172.21.15.157:0/3303053696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff78406c810 0x7ff7840af2c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff778006fd0 tx=0x7ff778017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:59.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.896+0000 7ff76bfff700 1 -- 172.21.15.157:0/3303053696 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77800bbe0 con 0x7ff78406c810 2023-10-06T04:10:59.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.896+0000 7ff76bfff700 1 -- 172.21.15.157:0/3303053696 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff778011420 con 0x7ff78406c810 2023-10-06T04:10:59.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.896+0000 7ff78a7e8700 1 -- 172.21.15.157:0/3303053696 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7780094f0 con 0x7ff78406c810 2023-10-06T04:10:59.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.896+0000 7ff76bfff700 1 -- 172.21.15.157:0/3303053696 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7780117c0 con 0x7ff78406c810 2023-10-06T04:10:59.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.896+0000 7ff78a7e8700 1 -- 172.21.15.157:0/3303053696 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7840aee30 con 0x7ff78406c810 2023-10-06T04:10:59.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.897+0000 7ff78a7e8700 1 -- 172.21.15.157:0/3303053696 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7840aefc0 con 0x7ff78406c810 2023-10-06T04:10:59.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.898+0000 7ff76bfff700 1 -- 172.21.15.157:0/3303053696 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff778011bb0 con 0x7ff78406c810 2023-10-06T04:10:59.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.898+0000 7ff76bfff700 1 --2- 172.21.15.157:0/3303053696 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff770066f60 0x7ff770069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:10:59.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.898+0000 7ff76bfff700 1 -- 172.21.15.157:0/3303053696 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7ff778086280 con 0x7ff78406c810 2023-10-06T04:10:59.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.898+0000 7ff783fff700 1 --2- 172.21.15.157:0/3303053696 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff770066f60 0x7ff770069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:10:59.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.899+0000 7ff783fff700 1 --2- 172.21.15.157:0/3303053696 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff770066f60 0x7ff770069410 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff774007120 tx=0x7ff774000bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:10:59.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:10:59.902+0000 7ff76bfff700 1 -- 172.21.15.157:0/3303053696 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff778053100 con 0x7ff78406c810 2023-10-06T04:11:00.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.087+0000 7ff78a7e8700 1 -- 172.21.15.157:0/3303053696 --> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7841561e0 con 0x7ff770066f60 2023-10-06T04:11:00.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.088+0000 7ff76bfff700 1 -- 172.21.15.157:0/3303053696 <== mgr.4107 v2:172.21.15.175:6832/90661 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff7841561e0 con 0x7ff770066f60 2023-10-06T04:11:00.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.090+0000 7ff78a7e8700 1 -- 172.21.15.157:0/3303053696 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff770066f60 msgr2=0x7ff770069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:00.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.090+0000 7ff78a7e8700 1 --2- 172.21.15.157:0/3303053696 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff770066f60 0x7ff770069410 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff774007120 tx=0x7ff774000bc0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.090+0000 7ff78a7e8700 1 -- 172.21.15.157:0/3303053696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff78406c810 msgr2=0x7ff7840af2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:00.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.090+0000 7ff78a7e8700 1 --2- 172.21.15.157:0/3303053696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff78406c810 0x7ff7840af2c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff778006fd0 tx=0x7ff778017000 comp rx=0 tx=0).stop 2023-10-06T04:11:00.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.090+0000 7ff78a7e8700 1 -- 172.21.15.157:0/3303053696 shutdown_connections 2023-10-06T04:11:00.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.090+0000 7ff78a7e8700 1 --2- 172.21.15.157:0/3303053696 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff770066f60 0x7ff770069410 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.090+0000 7ff78a7e8700 1 --2- 172.21.15.157:0/3303053696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7840c5fc0 0x7ff7840af800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.090+0000 7ff78a7e8700 1 --2- 172.21.15.157:0/3303053696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff78406c440 0x7ff7840c4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.090+0000 7ff78a7e8700 1 --2- 172.21.15.157:0/3303053696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff78406c810 0x7ff7840af2c0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.090+0000 7ff78a7e8700 1 -- 172.21.15.157:0/3303053696 >> 172.21.15.157:0/3303053696 conn(0x7ff784068370 msgr2=0x7ff7840c63f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:00.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.090+0000 7ff78a7e8700 1 -- 172.21.15.157:0/3303053696 shutdown_connections 2023-10-06T04:11:00.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.090+0000 7ff78a7e8700 1 -- 172.21.15.157:0/3303053696 wait complete. 2023-10-06T04:11:00.093 INFO:teuthology.orchestra.run.smithi157.stderr:instructed osd(s) 1 to deep-scrub 2023-10-06T04:11:00.103 INFO:tasks.ceph:Scrubbing osd.2 2023-10-06T04:11:00.104 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-06T04:11:00.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.221+0000 7f85e1979700 1 Processor -- start 2023-10-06T04:11:00.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.221+0000 7f85e1979700 1 -- start start 2023-10-06T04:11:00.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.221+0000 7f85e1979700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f85dc06a860 0x7f85dc0631f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:00.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.221+0000 7f85e1979700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85dc06a270 0x7f85dc156870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:00.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.221+0000 7f85e1979700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85dc0c48f0 con 0x7f85dc069ea0 2023-10-06T04:11:00.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.221+0000 7f85e1979700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85dc156db0 con 0x7f85dc06a860 2023-10-06T04:11:00.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.221+0000 7f85e1979700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc0c3d20 con 0x7f85dc06a270 2023-10-06T04:11:00.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.221+0000 7f85d27fc700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f85dc06a860 0x7f85dc0631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:00.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.221+0000 7f85daffd700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f85dc069ea0 0x7f85dc0c4d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:35368/0 (socket says 172.21.15.157:35368) 2023-10-06T04:11:00.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.222+0000 7f85daffd700 1 -- 172.21.15.157:0/3342858560 learned_addr learned my addr 172.21.15.157:0/3342858560 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:11:00.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.222+0000 7f85db7fe700 1 --2- 172.21.15.157:0/3342858560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85dc06a270 0x7f85dc156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:00.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.222+0000 7f85d27fc700 1 -- 172.21.15.157:0/3342858560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85dc06a270 msgr2=0x7f85dc156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:11:00.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.222+0000 7f85d27fc700 1 --2- 172.21.15.157:0/3342858560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85dc06a270 0x7f85dc156870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.222+0000 7f85d27fc700 1 -- 172.21.15.157:0/3342858560 >> v1:172.21.15.157:6789/0 conn(0x7f85dc069ea0 legacy=0x7f85dc0c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:00.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.222+0000 7f85d27fc700 1 -- 172.21.15.157:0/3342858560 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85dc063840 con 0x7f85dc06a860 2023-10-06T04:11:00.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.222+0000 7f85da7fc700 1 -- 172.21.15.157:0/3342858560 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2771206427 0 0) 0x7f85dc0c48f0 con 0x7f85dc069ea0 2023-10-06T04:11:00.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.222+0000 7f85db7fe700 1 --2- 172.21.15.157:0/3342858560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85dc06a270 0x7f85dc156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:11:00.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.222+0000 7f85d27fc700 1 --2- 172.21.15.157:0/3342858560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85dc06a860 0x7f85dc0631f0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f85c40098e0 tx=0x7f85c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90482abca2f9b49 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:00.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.222+0000 7f85da7fc700 1 -- 172.21.15.157:0/3342858560 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c4009e80 con 0x7f85dc06a860 2023-10-06T04:11:00.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.222+0000 7f85da7fc700 1 -- 172.21.15.157:0/3342858560 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c400b8f0 con 0x7f85dc06a860 2023-10-06T04:11:00.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.222+0000 7f85da7fc700 1 -- 172.21.15.157:0/3342858560 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c400bc70 con 0x7f85dc06a860 2023-10-06T04:11:00.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.223+0000 7f85e1979700 1 -- 172.21.15.157:0/3342858560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85dc06a860 msgr2=0x7f85dc0631f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:00.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.223+0000 7f85e1979700 1 --2- 172.21.15.157:0/3342858560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85dc06a860 0x7f85dc0631f0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f85c40098e0 tx=0x7f85c40091a0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.223+0000 7f85e1979700 1 -- 172.21.15.157:0/3342858560 shutdown_connections 2023-10-06T04:11:00.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.223+0000 7f85e1979700 1 --2- 172.21.15.157:0/3342858560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85dc06a270 0x7f85dc156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.223+0000 7f85e1979700 1 --2- 172.21.15.157:0/3342858560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85dc06a860 0x7f85dc0631f0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.223+0000 7f85e1979700 1 -- 172.21.15.157:0/3342858560 >> 172.21.15.157:0/3342858560 conn(0x7f85dc0c8b20 msgr2=0x7f85dc0c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:00.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.223+0000 7f85e1979700 1 -- 172.21.15.157:0/3342858560 shutdown_connections 2023-10-06T04:11:00.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.223+0000 7f85e1979700 1 -- 172.21.15.157:0/3342858560 wait complete. 2023-10-06T04:11:00.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.224+0000 7f85e1979700 1 Processor -- start 2023-10-06T04:11:00.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.224+0000 7f85e1979700 1 -- start start 2023-10-06T04:11:00.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.225+0000 7f85e1979700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85dc069ea0 0x7f85dc169710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:00.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.225+0000 7f85e1979700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85dc06a270 0x7f85dc14fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:00.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.225+0000 7f85e1979700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85dc06a860 0x7f85dc13b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:00.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.225+0000 7f85e1979700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc146e70 con 0x7f85dc06a860 2023-10-06T04:11:00.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.225+0000 7f85e1979700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc156fc0 con 0x7f85dc069ea0 2023-10-06T04:11:00.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.225+0000 7f85e1979700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc169c50 con 0x7f85dc06a270 2023-10-06T04:11:00.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.225+0000 7f85db7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85dc06a860 0x7f85dc13b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:00.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.225+0000 7f85db7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85dc06a860 0x7f85dc13b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51698/0 (socket says 172.21.15.157:51698) 2023-10-06T04:11:00.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.225+0000 7f85db7fe700 1 -- 172.21.15.157:0/3801076485 learned_addr learned my addr 172.21.15.157:0/3801076485 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:11:00.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.225+0000 7f85d27fc700 1 --2- 172.21.15.157:0/3801076485 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85dc06a270 0x7f85dc14fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:00.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.225+0000 7f85daffd700 1 --2- 172.21.15.157:0/3801076485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85dc069ea0 0x7f85dc169710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:00.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.225+0000 7f85db7fe700 1 -- 172.21.15.157:0/3801076485 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85dc06a270 msgr2=0x7f85dc14fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:00.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.225+0000 7f85db7fe700 1 --2- 172.21.15.157:0/3801076485 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85dc06a270 0x7f85dc14fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.225+0000 7f85db7fe700 1 -- 172.21.15.157:0/3801076485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85dc069ea0 msgr2=0x7f85dc169710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:00.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.225+0000 7f85db7fe700 1 --2- 172.21.15.157:0/3801076485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85dc069ea0 0x7f85dc169710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.225+0000 7f85db7fe700 1 -- 172.21.15.157:0/3801076485 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c4009420 con 0x7f85dc06a860 2023-10-06T04:11:00.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.225+0000 7f85db7fe700 1 --2- 172.21.15.157:0/3801076485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85dc06a860 0x7f85dc13b400 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f85d400a380 tx=0x7f85d4007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:00.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.225+0000 7f85d37fe700 1 -- 172.21.15.157:0/3801076485 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d4015440 con 0x7f85dc06a860 2023-10-06T04:11:00.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.226+0000 7f85d37fe700 1 -- 172.21.15.157:0/3801076485 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d40155a0 con 0x7f85dc06a860 2023-10-06T04:11:00.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.226+0000 7f85e1979700 1 -- 172.21.15.157:0/3801076485 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85dc1473c0 con 0x7f85dc06a860 2023-10-06T04:11:00.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.226+0000 7f85d37fe700 1 -- 172.21.15.157:0/3801076485 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d4015960 con 0x7f85dc06a860 2023-10-06T04:11:00.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.226+0000 7f85e1979700 1 -- 172.21.15.157:0/3801076485 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85dc0c2d50 con 0x7f85dc06a860 2023-10-06T04:11:00.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.226+0000 7f85e1979700 1 -- 172.21.15.157:0/3801076485 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f85dc0642a0 con 0x7f85dc06a860 2023-10-06T04:11:00.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.227+0000 7f85d37fe700 1 -- 172.21.15.157:0/3801076485 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f85d4015d50 con 0x7f85dc06a860 2023-10-06T04:11:00.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.227+0000 7f85d37fe700 1 --2- 172.21.15.157:0/3801076485 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f85bc066f30 0x7f85bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:00.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.227+0000 7f85d37fe700 1 -- 172.21.15.157:0/3801076485 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f85d4085880 con 0x7f85dc06a860 2023-10-06T04:11:00.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.227+0000 7f85daffd700 1 --2- 172.21.15.157:0/3801076485 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f85bc066f30 0x7f85bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:00.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.227+0000 7f85d37fe700 1 --2- 172.21.15.157:0/3801076485 >> [v2:172.21.15.157:6824/90657,v1:172.21.15.157:6825/90657] conn(0x7f85bc06fd90 0x7f85bc0721c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:00.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.227+0000 7f85d37fe700 1 -- 172.21.15.157:0/3801076485 --> [v2:172.21.15.157:6824/90657,v1:172.21.15.157:6825/90657] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f85bc072890 con 0x7f85bc06fd90 2023-10-06T04:11:00.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.227+0000 7f85d37fe700 1 -- 172.21.15.157:0/3801076485 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85d4085c50 con 0x7f85dc06a860 2023-10-06T04:11:00.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.228+0000 7f85d27fc700 1 --2- 172.21.15.157:0/3801076485 >> [v2:172.21.15.157:6824/90657,v1:172.21.15.157:6825/90657] conn(0x7f85bc06fd90 0x7f85bc0721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:00.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.228+0000 7f85daffd700 1 --2- 172.21.15.157:0/3801076485 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f85bc066f30 0x7f85bc0693e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f85cc007fd0 tx=0x7f85cc007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:00.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.228+0000 7f85d27fc700 1 --2- 172.21.15.157:0/3801076485 >> [v2:172.21.15.157:6824/90657,v1:172.21.15.157:6825/90657] conn(0x7f85bc06fd90 0x7f85bc0721c0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:00.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.228+0000 7f85d37fe700 1 -- 172.21.15.157:0/3801076485 <== osd.2 v2:172.21.15.157:6824/90657 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f85bc072890 con 0x7f85bc06fd90 2023-10-06T04:11:00.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.242+0000 7f85e1979700 1 -- 172.21.15.157:0/3801076485 --> [v2:172.21.15.157:6824/90657,v1:172.21.15.157:6825/90657] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f85dc0c3fc0 con 0x7f85bc06fd90 2023-10-06T04:11:00.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.242+0000 7f85d37fe700 1 -- 172.21.15.157:0/3801076485 <== osd.2 v2:172.21.15.157:6824/90657 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f85dc0c3fc0 con 0x7f85bc06fd90 2023-10-06T04:11:00.244 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-10-06T04:11:00.244 INFO:teuthology.orchestra.run.smithi157.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T04:11:00.245 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-10-06T04:11:00.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.242+0000 7f85e1979700 1 -- 172.21.15.157:0/3801076485 >> [v2:172.21.15.157:6824/90657,v1:172.21.15.157:6825/90657] conn(0x7f85bc06fd90 msgr2=0x7f85bc0721c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:00.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.242+0000 7f85e1979700 1 --2- 172.21.15.157:0/3801076485 >> [v2:172.21.15.157:6824/90657,v1:172.21.15.157:6825/90657] conn(0x7f85bc06fd90 0x7f85bc0721c0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.243+0000 7f85e1979700 1 -- 172.21.15.157:0/3801076485 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f85bc066f30 msgr2=0x7f85bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:00.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.243+0000 7f85e1979700 1 --2- 172.21.15.157:0/3801076485 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f85bc066f30 0x7f85bc0693e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f85cc007fd0 tx=0x7f85cc007d30 comp rx=0 tx=0).stop 2023-10-06T04:11:00.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.243+0000 7f85e1979700 1 -- 172.21.15.157:0/3801076485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85dc06a860 msgr2=0x7f85dc13b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:00.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.243+0000 7f85e1979700 1 --2- 172.21.15.157:0/3801076485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85dc06a860 0x7f85dc13b400 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f85d400a380 tx=0x7f85d4007900 comp rx=0 tx=0).stop 2023-10-06T04:11:00.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.243+0000 7f85db7fe700 1 -- 172.21.15.157:0/3801076485 reap_dead start 2023-10-06T04:11:00.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.243+0000 7f85e1979700 1 -- 172.21.15.157:0/3801076485 shutdown_connections 2023-10-06T04:11:00.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.243+0000 7f85e1979700 1 -- 172.21.15.157:0/3801076485 >> 172.21.15.157:0/3801076485 conn(0x7f85dc0c8b20 msgr2=0x7f85dc06d520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:00.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.243+0000 7f85e1979700 1 -- 172.21.15.157:0/3801076485 shutdown_connections 2023-10-06T04:11:00.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.243+0000 7f85e1979700 1 -- 172.21.15.157:0/3801076485 wait complete. 2023-10-06T04:11:00.256 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-06T04:11:00.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.371+0000 7f6cac171700 1 Processor -- start 2023-10-06T04:11:00.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.371+0000 7f6cac171700 1 -- start start 2023-10-06T04:11:00.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.372+0000 7f6cac171700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ca406dc40 0x7f6ca414fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:00.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.372+0000 7f6cac171700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca40c9780 con 0x7f6ca406cbb0 2023-10-06T04:11:00.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.372+0000 7f6cac171700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca406e0c0 con 0x7f6ca406c1f0 2023-10-06T04:11:00.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.372+0000 7f6cac171700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca40c8c40 con 0x7f6ca406dc40 2023-10-06T04:11:00.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.372+0000 7f6ca9f0d700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f6ca406cbb0 0x7f6ca406d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:35370/0 (socket says 172.21.15.157:35370) 2023-10-06T04:11:00.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.372+0000 7f6ca9f0d700 1 -- 172.21.15.157:0/1893075467 learned_addr learned my addr 172.21.15.157:0/1893075467 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:11:00.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.372+0000 7f6ca970c700 1 --2- 172.21.15.157:0/1893075467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ca406dc40 0x7f6ca414fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:00.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.373+0000 7f6ca8f0b700 1 -- 172.21.15.157:0/1893075467 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2880380245 0 0) 0x7f6ca40c9780 con 0x7f6ca406cbb0 2023-10-06T04:11:00.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.373+0000 7f6ca8f0b700 1 -- 172.21.15.157:0/1893075467 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c90003570 con 0x7f6ca406cbb0 2023-10-06T04:11:00.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.373+0000 7f6ca8f0b700 1 -- 172.21.15.157:0/1893075467 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3649966629 0 0) 0x7f6ca406e0c0 con 0x7f6ca406c1f0 2023-10-06T04:11:00.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.373+0000 7f6ca8f0b700 1 -- 172.21.15.157:0/1893075467 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ca40c9780 con 0x7f6ca406c1f0 2023-10-06T04:11:00.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.373+0000 7f6ca970c700 1 -- 172.21.15.157:0/1893075467 >> v1:172.21.15.175:6789/0 conn(0x7f6ca406c1f0 legacy=0x7f6ca4155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:00.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.373+0000 7f6ca8f0b700 1 -- 172.21.15.157:0/1893075467 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3455685360 0 0) 0x7f6c90003570 con 0x7f6ca406cbb0 2023-10-06T04:11:00.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.373+0000 7f6ca970c700 1 -- 172.21.15.157:0/1893075467 >> v1:172.21.15.157:6789/0 conn(0x7f6ca406cbb0 legacy=0x7f6ca406d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:00.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.373+0000 7f6ca970c700 1 -- 172.21.15.157:0/1893075467 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca40c8e50 con 0x7f6ca406dc40 2023-10-06T04:11:00.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.373+0000 7f6ca970c700 1 --2- 172.21.15.157:0/1893075467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ca406dc40 0x7f6ca414fd00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6c94009910 tx=0x7f6c940091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbc41e93a25a672b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:00.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.373+0000 7f6ca8f0b700 1 -- 172.21.15.157:0/1893075467 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0001470 con 0x7f6ca406dc40 2023-10-06T04:11:00.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.373+0000 7f6ca8f0b700 1 -- 172.21.15.157:0/1893075467 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c94011420 con 0x7f6ca406dc40 2023-10-06T04:11:00.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.373+0000 7f6ca8f0b700 1 -- 172.21.15.157:0/1893075467 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c940117c0 con 0x7f6ca406dc40 2023-10-06T04:11:00.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.374+0000 7f6cac171700 1 -- 172.21.15.157:0/1893075467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ca406dc40 msgr2=0x7f6ca414fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:00.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.374+0000 7f6cac171700 1 --2- 172.21.15.157:0/1893075467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ca406dc40 0x7f6ca414fd00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6c94009910 tx=0x7f6c940091a0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.374+0000 7f6cac171700 1 -- 172.21.15.157:0/1893075467 shutdown_connections 2023-10-06T04:11:00.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.374+0000 7f6cac171700 1 --2- 172.21.15.157:0/1893075467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ca406dc40 0x7f6ca414fd00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.374+0000 7f6cac171700 1 -- 172.21.15.157:0/1893075467 >> 172.21.15.157:0/1893075467 conn(0x7f6ca406f230 msgr2=0x7f6ca406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:00.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.374+0000 7f6cac171700 1 -- 172.21.15.157:0/1893075467 shutdown_connections 2023-10-06T04:11:00.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.374+0000 7f6cac171700 1 -- 172.21.15.157:0/1893075467 wait complete. 2023-10-06T04:11:00.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.375+0000 7f6cac171700 1 Processor -- start 2023-10-06T04:11:00.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.375+0000 7f6cac171700 1 -- start start 2023-10-06T04:11:00.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.375+0000 7f6cac171700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ca406c1f0 0x7f6ca4152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:00.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.375+0000 7f6cac171700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ca406cbb0 0x7f6ca40afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:00.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.375+0000 7f6cac171700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ca416a050 0x7f6ca416a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:00.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.376+0000 7f6cac171700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca4166750 con 0x7f6ca406cbb0 2023-10-06T04:11:00.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.376+0000 7f6cac171700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca4148920 con 0x7f6ca406c1f0 2023-10-06T04:11:00.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.376+0000 7f6cac171700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca4166fb0 con 0x7f6ca416a050 2023-10-06T04:11:00.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.376+0000 7f6ca970c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ca406cbb0 0x7f6ca40afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:00.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.376+0000 7f6ca970c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ca406cbb0 0x7f6ca40afb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51704/0 (socket says 172.21.15.157:51704) 2023-10-06T04:11:00.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.376+0000 7f6ca970c700 1 -- 172.21.15.157:0/1869147107 learned_addr learned my addr 172.21.15.157:0/1869147107 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:11:00.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.376+0000 7f6ca9f0d700 1 --2- 172.21.15.157:0/1869147107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ca406c1f0 0x7f6ca4152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:00.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.376+0000 7f6caa70e700 1 --2- 172.21.15.157:0/1869147107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ca416a050 0x7f6ca416a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:00.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.376+0000 7f6ca970c700 1 -- 172.21.15.157:0/1869147107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ca416a050 msgr2=0x7f6ca416a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:00.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.376+0000 7f6ca970c700 1 --2- 172.21.15.157:0/1869147107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ca416a050 0x7f6ca416a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.376+0000 7f6ca970c700 1 -- 172.21.15.157:0/1869147107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ca406c1f0 msgr2=0x7f6ca4152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:00.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.376+0000 7f6ca970c700 1 --2- 172.21.15.157:0/1869147107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ca406c1f0 0x7f6ca4152310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.376+0000 7f6ca970c700 1 -- 172.21.15.157:0/1869147107 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c94009420 con 0x7f6ca406cbb0 2023-10-06T04:11:00.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.376+0000 7f6ca9f0d700 1 --2- 172.21.15.157:0/1869147107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ca406c1f0 0x7f6ca4152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:11:00.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.376+0000 7f6ca970c700 1 --2- 172.21.15.157:0/1869147107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ca406cbb0 0x7f6ca40afb60 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f6c940030c0 tx=0x7f6c94016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:00.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.376+0000 7f6c99ffb700 1 -- 172.21.15.157:0/1869147107 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9401cc50 con 0x7f6ca406cbb0 2023-10-06T04:11:00.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.376+0000 7f6c99ffb700 1 -- 172.21.15.157:0/1869147107 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9401cdb0 con 0x7f6ca406cbb0 2023-10-06T04:11:00.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.376+0000 7f6c99ffb700 1 -- 172.21.15.157:0/1869147107 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9401a6a0 con 0x7f6ca406cbb0 2023-10-06T04:11:00.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.376+0000 7f6cac171700 1 -- 172.21.15.157:0/1869147107 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca4148b50 con 0x7f6ca406cbb0 2023-10-06T04:11:00.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.376+0000 7f6cac171700 1 -- 172.21.15.157:0/1869147107 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca4166960 con 0x7f6ca406cbb0 2023-10-06T04:11:00.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.377+0000 7f6cac171700 1 -- 172.21.15.157:0/1869147107 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca4089ca0 con 0x7f6ca406cbb0 2023-10-06T04:11:00.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.378+0000 7f6c99ffb700 1 -- 172.21.15.157:0/1869147107 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6c9401aa90 con 0x7f6ca406cbb0 2023-10-06T04:11:00.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.378+0000 7f6c99ffb700 1 --2- 172.21.15.157:0/1869147107 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6c88066f60 0x7f6c88069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:00.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.378+0000 7f6c99ffb700 1 -- 172.21.15.157:0/1869147107 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f6c940862d0 con 0x7f6ca406cbb0 2023-10-06T04:11:00.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.378+0000 7f6ca9f0d700 1 --2- 172.21.15.157:0/1869147107 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6c88066f60 0x7f6c88069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:00.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.379+0000 7f6ca9f0d700 1 --2- 172.21.15.157:0/1869147107 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6c88066f60 0x7f6c88069410 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6ca00011e0 tx=0x7f6ca0000bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:00.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.382+0000 7f6c99ffb700 1 -- 172.21.15.157:0/1869147107 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c94053150 con 0x7f6ca406cbb0 2023-10-06T04:11:00.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.563+0000 7f6cac171700 1 -- 172.21.15.157:0/1869147107 --> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ca40c9130 con 0x7f6c88066f60 2023-10-06T04:11:00.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.564+0000 7f6c99ffb700 1 -- 172.21.15.157:0/1869147107 <== mgr.4107 v2:172.21.15.175:6832/90661 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6ca40c9130 con 0x7f6c88066f60 2023-10-06T04:11:00.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.566+0000 7f6cac171700 1 -- 172.21.15.157:0/1869147107 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6c88066f60 msgr2=0x7f6c88069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:00.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.566+0000 7f6cac171700 1 --2- 172.21.15.157:0/1869147107 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6c88066f60 0x7f6c88069410 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6ca00011e0 tx=0x7f6ca0000bc0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.566+0000 7f6cac171700 1 -- 172.21.15.157:0/1869147107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ca406cbb0 msgr2=0x7f6ca40afb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:00.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.566+0000 7f6cac171700 1 --2- 172.21.15.157:0/1869147107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ca406cbb0 0x7f6ca40afb60 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f6c940030c0 tx=0x7f6c94016000 comp rx=0 tx=0).stop 2023-10-06T04:11:00.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.566+0000 7f6cac171700 1 -- 172.21.15.157:0/1869147107 shutdown_connections 2023-10-06T04:11:00.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.566+0000 7f6cac171700 1 --2- 172.21.15.157:0/1869147107 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f6c88066f60 0x7f6c88069410 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.566+0000 7f6cac171700 1 --2- 172.21.15.157:0/1869147107 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ca406c1f0 0x7f6ca4152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.566+0000 7f6cac171700 1 --2- 172.21.15.157:0/1869147107 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ca416a050 0x7f6ca416a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.566+0000 7f6cac171700 1 --2- 172.21.15.157:0/1869147107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ca406cbb0 0x7f6ca40afb60 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.566+0000 7f6cac171700 1 -- 172.21.15.157:0/1869147107 >> 172.21.15.157:0/1869147107 conn(0x7f6ca406f230 msgr2=0x7f6ca414fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:00.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.566+0000 7f6cac171700 1 -- 172.21.15.157:0/1869147107 shutdown_connections 2023-10-06T04:11:00.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.566+0000 7f6cac171700 1 -- 172.21.15.157:0/1869147107 wait complete. 2023-10-06T04:11:00.570 INFO:teuthology.orchestra.run.smithi157.stderr:instructed osd(s) 2 to deep-scrub 2023-10-06T04:11:00.579 INFO:tasks.ceph:Scrubbing osd.3 2023-10-06T04:11:00.580 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-06T04:11:00.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.697+0000 7ff1ad513700 1 Processor -- start 2023-10-06T04:11:00.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.697+0000 7ff1ad513700 1 -- start start 2023-10-06T04:11:00.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.698+0000 7ff1ad513700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1a814ffc0 0x7ff1a8159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:00.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.698+0000 7ff1ad513700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff1a8067530 0x7ff1a8067960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:00.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.698+0000 7ff1ad513700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff1a806cd70 0x7ff1a806d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:00.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.698+0000 7ff1ad513700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1a80c4e50 con 0x7ff1a806cd70 2023-10-06T04:11:00.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.698+0000 7ff1ad513700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1a815a0b0 con 0x7ff1a8067530 2023-10-06T04:11:00.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.698+0000 7ff1ad513700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a806d710 con 0x7ff1a814ffc0 2023-10-06T04:11:00.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.698+0000 7ff1acd12700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff1a806cd70 0x7ff1a806d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:00.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.698+0000 7ff1acd12700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff1a806cd70 0x7ff1a806d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51712/0 (socket says 172.21.15.157:51712) 2023-10-06T04:11:00.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.698+0000 7ff1acd12700 1 -- 172.21.15.157:0/2317807832 learned_addr learned my addr 172.21.15.157:0/2317807832 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:11:00.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.698+0000 7ff1a77fe700 1 --2- 172.21.15.157:0/2317807832 >> v2:172.21.15.175:3300/0 conn(0x7ff1a8067530 0x7ff1a8067960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:00.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.698+0000 7ff1a7fff700 1 --2- 172.21.15.157:0/2317807832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1a814ffc0 0x7ff1a8159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:00.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.698+0000 7ff1acd12700 1 -- 172.21.15.157:0/2317807832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1a814ffc0 msgr2=0x7ff1a8159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:11:00.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.698+0000 7ff1acd12700 1 --2- 172.21.15.157:0/2317807832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1a814ffc0 0x7ff1a8159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.698+0000 7ff1acd12700 1 -- 172.21.15.157:0/2317807832 >> v2:172.21.15.175:3300/0 conn(0x7ff1a8067530 msgr2=0x7ff1a8067960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:11:00.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.698+0000 7ff1acd12700 1 --2- 172.21.15.157:0/2317807832 >> v2:172.21.15.175:3300/0 conn(0x7ff1a8067530 0x7ff1a8067960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.698+0000 7ff1acd12700 1 -- 172.21.15.157:0/2317807832 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a806c8e0 con 0x7ff1a806cd70 2023-10-06T04:11:00.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.698+0000 7ff1a7fff700 1 --2- 172.21.15.157:0/2317807832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1a814ffc0 0x7ff1a8159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:11:00.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.699+0000 7ff1acd12700 1 --2- 172.21.15.157:0/2317807832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1a806cd70 0x7ff1a806d1d0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8069370 tx=0x7ff1a0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fda374e7f62e32b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:00.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.699+0000 7ff1a6ffd700 1 -- 172.21.15.157:0/2317807832 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a000e070 con 0x7ff1a806cd70 2023-10-06T04:11:00.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.699+0000 7ff1a6ffd700 1 -- 172.21.15.157:0/2317807832 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a000ba50 con 0x7ff1a806cd70 2023-10-06T04:11:00.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.699+0000 7ff1a6ffd700 1 -- 172.21.15.157:0/2317807832 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a00092e0 con 0x7ff1a806cd70 2023-10-06T04:11:00.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.699+0000 7ff1ad513700 1 -- 172.21.15.157:0/2317807832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1a806cd70 msgr2=0x7ff1a806d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:00.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.699+0000 7ff1ad513700 1 --2- 172.21.15.157:0/2317807832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1a806cd70 0x7ff1a806d1d0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8069370 tx=0x7ff1a0009840 comp rx=0 tx=0).stop 2023-10-06T04:11:00.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.699+0000 7ff1ad513700 1 -- 172.21.15.157:0/2317807832 shutdown_connections 2023-10-06T04:11:00.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.699+0000 7ff1ad513700 1 --2- 172.21.15.157:0/2317807832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1a806cd70 0x7ff1a806d1d0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.699+0000 7ff1ad513700 1 --2- 172.21.15.157:0/2317807832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1a814ffc0 0x7ff1a8159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.699+0000 7ff1ad513700 1 --2- 172.21.15.157:0/2317807832 >> v2:172.21.15.175:3300/0 conn(0x7ff1a8067530 0x7ff1a8067960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.699+0000 7ff1ad513700 1 -- 172.21.15.157:0/2317807832 >> 172.21.15.157:0/2317807832 conn(0x7ff1a80c8990 msgr2=0x7ff1a80c8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:00.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.699+0000 7ff1ad513700 1 -- 172.21.15.157:0/2317807832 shutdown_connections 2023-10-06T04:11:00.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.699+0000 7ff1ad513700 1 -- 172.21.15.157:0/2317807832 wait complete. 2023-10-06T04:11:00.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.700+0000 7ff1ad513700 1 Processor -- start 2023-10-06T04:11:00.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.701+0000 7ff1ad513700 1 -- start start 2023-10-06T04:11:00.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.701+0000 7ff1ad513700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1a8067530 0x7ff1a8165d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:00.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.701+0000 7ff1ad513700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1a806cd70 0x7ff1a8150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:00.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.701+0000 7ff1ad513700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1a814ffc0 0x7ff1a8152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:00.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.701+0000 7ff1ad513700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a8151a80 con 0x7ff1a8067530 2023-10-06T04:11:00.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.701+0000 7ff1ad513700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a8150b90 con 0x7ff1a814ffc0 2023-10-06T04:11:00.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.701+0000 7ff1ad513700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a8150e70 con 0x7ff1a806cd70 2023-10-06T04:11:00.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.701+0000 7ff1a7fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1a8067530 0x7ff1a8165d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:00.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.701+0000 7ff1a7fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1a8067530 0x7ff1a8165d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51718/0 (socket says 172.21.15.157:51718) 2023-10-06T04:11:00.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.701+0000 7ff1a7fff700 1 -- 172.21.15.157:0/275904889 learned_addr learned my addr 172.21.15.157:0/275904889 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:11:00.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.701+0000 7ff1a77fe700 1 --2- 172.21.15.157:0/275904889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1a806cd70 0x7ff1a8150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:00.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.701+0000 7ff1acd12700 1 --2- 172.21.15.157:0/275904889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1a814ffc0 0x7ff1a8152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:00.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.702+0000 7ff1a7fff700 1 -- 172.21.15.157:0/275904889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1a806cd70 msgr2=0x7ff1a8150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:00.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.702+0000 7ff1a7fff700 1 --2- 172.21.15.157:0/275904889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1a806cd70 0x7ff1a8150550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.702+0000 7ff1a7fff700 1 -- 172.21.15.157:0/275904889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1a814ffc0 msgr2=0x7ff1a8152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:00.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.702+0000 7ff1a7fff700 1 --2- 172.21.15.157:0/275904889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1a814ffc0 0x7ff1a8152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.702+0000 7ff1a7fff700 1 -- 172.21.15.157:0/275904889 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a0009ae0 con 0x7ff1a8067530 2023-10-06T04:11:00.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.702+0000 7ff1a7fff700 1 --2- 172.21.15.157:0/275904889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1a8067530 0x7ff1a8165d60 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7ff19800c850 tx=0x7ff198008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:00.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.702+0000 7ff18ffff700 1 -- 172.21.15.157:0/275904889 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198007650 con 0x7ff1a8067530 2023-10-06T04:11:00.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.702+0000 7ff18ffff700 1 -- 172.21.15.157:0/275904889 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1980077b0 con 0x7ff1a8067530 2023-10-06T04:11:00.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.702+0000 7ff1ad513700 1 -- 172.21.15.157:0/275904889 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a8151150 con 0x7ff1a8067530 2023-10-06T04:11:00.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.702+0000 7ff18ffff700 1 -- 172.21.15.157:0/275904889 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198007b50 con 0x7ff1a8067530 2023-10-06T04:11:00.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.702+0000 7ff1ad513700 1 -- 172.21.15.157:0/275904889 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a8151710 con 0x7ff1a8067530 2023-10-06T04:11:00.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.703+0000 7ff1ad513700 1 -- 172.21.15.157:0/275904889 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff1a80c8780 con 0x7ff1a8067530 2023-10-06T04:11:00.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.703+0000 7ff18ffff700 1 -- 172.21.15.157:0/275904889 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff1980113f0 con 0x7ff1a8067530 2023-10-06T04:11:00.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.704+0000 7ff18ffff700 1 --2- 172.21.15.157:0/275904889 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff194066f30 0x7ff1940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:00.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.704+0000 7ff18ffff700 1 -- 172.21.15.157:0/275904889 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7ff198084920 con 0x7ff1a8067530 2023-10-06T04:11:00.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.704+0000 7ff1a77fe700 1 --2- 172.21.15.157:0/275904889 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff194066f30 0x7ff1940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:00.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.704+0000 7ff18ffff700 1 --2- 172.21.15.157:0/275904889 >> [v2:172.21.15.157:6816/90641,v1:172.21.15.157:6817/90641] conn(0x7ff19406fd90 0x7ff1940721c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:00.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.704+0000 7ff18ffff700 1 -- 172.21.15.157:0/275904889 --> [v2:172.21.15.157:6816/90641,v1:172.21.15.157:6817/90641] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff194072890 con 0x7ff19406fd90 2023-10-06T04:11:00.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.704+0000 7ff18ffff700 1 -- 172.21.15.157:0/275904889 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff198089240 con 0x7ff1a8067530 2023-10-06T04:11:00.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.704+0000 7ff1acd12700 1 --2- 172.21.15.157:0/275904889 >> [v2:172.21.15.157:6816/90641,v1:172.21.15.157:6817/90641] conn(0x7ff19406fd90 0x7ff1940721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:00.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.704+0000 7ff1a77fe700 1 --2- 172.21.15.157:0/275904889 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff194066f30 0x7ff1940693e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8160690 tx=0x7ff19c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:00.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.704+0000 7ff1acd12700 1 --2- 172.21.15.157:0/275904889 >> [v2:172.21.15.157:6816/90641,v1:172.21.15.157:6817/90641] conn(0x7ff19406fd90 0x7ff1940721c0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:00.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.705+0000 7ff18ffff700 1 -- 172.21.15.157:0/275904889 <== osd.3 v2:172.21.15.157:6816/90641 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7ff194072890 con 0x7ff19406fd90 2023-10-06T04:11:00.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.718+0000 7ff1ad513700 1 -- 172.21.15.157:0/275904889 --> [v2:172.21.15.157:6816/90641,v1:172.21.15.157:6817/90641] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff1a8068aa0 con 0x7ff19406fd90 2023-10-06T04:11:00.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.718+0000 7ff18ffff700 1 -- 172.21.15.157:0/275904889 <== osd.3 v2:172.21.15.157:6816/90641 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff1a8068aa0 con 0x7ff19406fd90 2023-10-06T04:11:00.720 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-10-06T04:11:00.720 INFO:teuthology.orchestra.run.smithi157.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T04:11:00.720 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-10-06T04:11:00.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.719+0000 7ff1ad513700 1 -- 172.21.15.157:0/275904889 >> [v2:172.21.15.157:6816/90641,v1:172.21.15.157:6817/90641] conn(0x7ff19406fd90 msgr2=0x7ff1940721c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:00.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.719+0000 7ff1ad513700 1 --2- 172.21.15.157:0/275904889 >> [v2:172.21.15.157:6816/90641,v1:172.21.15.157:6817/90641] conn(0x7ff19406fd90 0x7ff1940721c0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.719+0000 7ff1ad513700 1 -- 172.21.15.157:0/275904889 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff194066f30 msgr2=0x7ff1940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:00.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.719+0000 7ff1ad513700 1 --2- 172.21.15.157:0/275904889 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff194066f30 0x7ff1940693e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8160690 tx=0x7ff19c008040 comp rx=0 tx=0).stop 2023-10-06T04:11:00.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.719+0000 7ff1ad513700 1 -- 172.21.15.157:0/275904889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1a8067530 msgr2=0x7ff1a8165d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:00.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.719+0000 7ff1ad513700 1 --2- 172.21.15.157:0/275904889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1a8067530 0x7ff1a8165d60 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7ff19800c850 tx=0x7ff198008040 comp rx=0 tx=0).stop 2023-10-06T04:11:00.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.719+0000 7ff1acd12700 1 -- 172.21.15.157:0/275904889 reap_dead start 2023-10-06T04:11:00.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.719+0000 7ff1ad513700 1 -- 172.21.15.157:0/275904889 shutdown_connections 2023-10-06T04:11:00.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.719+0000 7ff1ad513700 1 -- 172.21.15.157:0/275904889 >> 172.21.15.157:0/275904889 conn(0x7ff1a80c8990 msgr2=0x7ff1a814c390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:00.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.719+0000 7ff1ad513700 1 -- 172.21.15.157:0/275904889 shutdown_connections 2023-10-06T04:11:00.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.719+0000 7ff1ad513700 1 -- 172.21.15.157:0/275904889 wait complete. 2023-10-06T04:11:00.732 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-06T04:11:00.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.846+0000 7fa3bc923700 1 Processor -- start 2023-10-06T04:11:00.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.846+0000 7fa3bc923700 1 -- start start 2023-10-06T04:11:00.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.847+0000 7fa3bc923700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa3b806cbb0 0x7fa3b806d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:00.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.847+0000 7fa3bc923700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3b806c1f0 0x7fa3b806da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:00.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.847+0000 7fa3bc923700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa3b806cf80 0x7fa3b814fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:00.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.847+0000 7fa3bc923700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3b80c8c40 con 0x7fa3b806cf80 2023-10-06T04:11:00.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.847+0000 7fa3bc923700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3b80c9780 con 0x7fa3b806cbb0 2023-10-06T04:11:00.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.847+0000 7fa3bc923700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b806e0c0 con 0x7fa3b806c1f0 2023-10-06T04:11:00.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.847+0000 7fa3b6d9d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa3b806cf80 0x7fa3b814fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:00.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.847+0000 7fa3b6d9d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa3b806cf80 0x7fa3b814fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51722/0 (socket says 172.21.15.157:51722) 2023-10-06T04:11:00.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.847+0000 7fa3b6d9d700 1 -- 172.21.15.157:0/4013070169 learned_addr learned my addr 172.21.15.157:0/4013070169 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:11:00.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.847+0000 7fa3b659c700 1 --2- 172.21.15.157:0/4013070169 >> v2:172.21.15.175:3300/0 conn(0x7fa3b806cbb0 0x7fa3b806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:00.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.847+0000 7fa3b5d9b700 1 --2- 172.21.15.157:0/4013070169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3b806c1f0 0x7fa3b806da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:00.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.847+0000 7fa3b6d9d700 1 -- 172.21.15.157:0/4013070169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3b806c1f0 msgr2=0x7fa3b806da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:11:00.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.847+0000 7fa3b6d9d700 1 --2- 172.21.15.157:0/4013070169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3b806c1f0 0x7fa3b806da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.847+0000 7fa3b6d9d700 1 -- 172.21.15.157:0/4013070169 >> v2:172.21.15.175:3300/0 conn(0x7fa3b806cbb0 msgr2=0x7fa3b806d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:11:00.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.847+0000 7fa3b6d9d700 1 --2- 172.21.15.157:0/4013070169 >> v2:172.21.15.175:3300/0 conn(0x7fa3b806cbb0 0x7fa3b806d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.847+0000 7fa3b6d9d700 1 -- 172.21.15.157:0/4013070169 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b80c8e50 con 0x7fa3b806cf80 2023-10-06T04:11:00.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.847+0000 7fa3b6d9d700 1 --2- 172.21.15.157:0/4013070169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3b806cf80 0x7fa3b814fd00 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009f20 tx=0x7fa3b0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e65b8c8373255a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:00.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.847+0000 7fa3b559a700 1 -- 172.21.15.157:0/4013070169 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b000e070 con 0x7fa3b806cf80 2023-10-06T04:11:00.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.847+0000 7fa3b559a700 1 -- 172.21.15.157:0/4013070169 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b000bad0 con 0x7fa3b806cf80 2023-10-06T04:11:00.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.847+0000 7fa3b559a700 1 -- 172.21.15.157:0/4013070169 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b00092e0 con 0x7fa3b806cf80 2023-10-06T04:11:00.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.848+0000 7fa3bc923700 1 -- 172.21.15.157:0/4013070169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3b806cf80 msgr2=0x7fa3b814fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:00.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.848+0000 7fa3bc923700 1 --2- 172.21.15.157:0/4013070169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3b806cf80 0x7fa3b814fd00 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009f20 tx=0x7fa3b0009840 comp rx=0 tx=0).stop 2023-10-06T04:11:00.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.848+0000 7fa3bc923700 1 -- 172.21.15.157:0/4013070169 shutdown_connections 2023-10-06T04:11:00.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.848+0000 7fa3bc923700 1 --2- 172.21.15.157:0/4013070169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3b806cf80 0x7fa3b814fd00 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.848+0000 7fa3bc923700 1 --2- 172.21.15.157:0/4013070169 >> v2:172.21.15.175:3300/0 conn(0x7fa3b806cbb0 0x7fa3b806d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.848+0000 7fa3bc923700 1 --2- 172.21.15.157:0/4013070169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3b806c1f0 0x7fa3b806da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.848+0000 7fa3bc923700 1 -- 172.21.15.157:0/4013070169 >> 172.21.15.157:0/4013070169 conn(0x7fa3b806f230 msgr2=0x7fa3b806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:00.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.848+0000 7fa3bc923700 1 -- 172.21.15.157:0/4013070169 shutdown_connections 2023-10-06T04:11:00.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.848+0000 7fa3bc923700 1 -- 172.21.15.157:0/4013070169 wait complete. 2023-10-06T04:11:00.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.849+0000 7fa3bc923700 1 Processor -- start 2023-10-06T04:11:00.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.849+0000 7fa3bc923700 1 -- start start 2023-10-06T04:11:00.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.850+0000 7fa3bc923700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3b806c1f0 0x7fa3b8146ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:00.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.850+0000 7fa3bc923700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3b806cbb0 0x7fa3b814b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:00.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.850+0000 7fa3bc923700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3b806cf80 0x7fa3b8169740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:00.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.850+0000 7fa3bc923700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b8156ce0 con 0x7fa3b806cf80 2023-10-06T04:11:00.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.850+0000 7fa3bc923700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b8146160 con 0x7fa3b806c1f0 2023-10-06T04:11:00.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.850+0000 7fa3bc923700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b814bea0 con 0x7fa3b806cbb0 2023-10-06T04:11:00.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.850+0000 7fa3b6d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3b806cf80 0x7fa3b8169740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:00.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.850+0000 7fa3b6d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3b806cf80 0x7fa3b8169740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51730/0 (socket says 172.21.15.157:51730) 2023-10-06T04:11:00.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.850+0000 7fa3b659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3b806c1f0 0x7fa3b8146ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:00.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.850+0000 7fa3b6d9d700 1 -- 172.21.15.157:0/1329029207 learned_addr learned my addr 172.21.15.157:0/1329029207 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:11:00.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.850+0000 7fa3b5d9b700 1 --2- 172.21.15.157:0/1329029207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3b806cbb0 0x7fa3b814b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:00.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.850+0000 7fa3b6d9d700 1 -- 172.21.15.157:0/1329029207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3b806cbb0 msgr2=0x7fa3b814b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:00.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.850+0000 7fa3b6d9d700 1 --2- 172.21.15.157:0/1329029207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3b806cbb0 0x7fa3b814b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.850+0000 7fa3b6d9d700 1 -- 172.21.15.157:0/1329029207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3b806c1f0 msgr2=0x7fa3b8146ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:00.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.850+0000 7fa3b6d9d700 1 --2- 172.21.15.157:0/1329029207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3b806c1f0 0x7fa3b8146ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:00.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.850+0000 7fa3b6d9d700 1 -- 172.21.15.157:0/1329029207 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b0009ae0 con 0x7fa3b806cf80 2023-10-06T04:11:00.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.851+0000 7fa3b6d9d700 1 --2- 172.21.15.157:0/1329029207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3b806cf80 0x7fa3b8169740 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80c3960 tx=0x7fa3b0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:00.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.851+0000 7fa3a67fc700 1 -- 172.21.15.157:0/1329029207 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b00032f0 con 0x7fa3b806cf80 2023-10-06T04:11:00.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.851+0000 7fa3a67fc700 1 -- 172.21.15.157:0/1329029207 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b0008c60 con 0x7fa3b806cf80 2023-10-06T04:11:00.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.851+0000 7fa3bc923700 1 -- 172.21.15.157:0/1329029207 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b8146ff0 con 0x7fa3b806cf80 2023-10-06T04:11:00.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.851+0000 7fa3a67fc700 1 -- 172.21.15.157:0/1329029207 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b001c410 con 0x7fa3b806cf80 2023-10-06T04:11:00.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.851+0000 7fa3bc923700 1 -- 172.21.15.157:0/1329029207 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b816a400 con 0x7fa3b806cf80 2023-10-06T04:11:00.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.852+0000 7fa3bc923700 1 -- 172.21.15.157:0/1329029207 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3b8089ca0 con 0x7fa3b806cf80 2023-10-06T04:11:00.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.853+0000 7fa3a67fc700 1 -- 172.21.15.157:0/1329029207 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa3b0003040 con 0x7fa3b806cf80 2023-10-06T04:11:00.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.853+0000 7fa3a67fc700 1 --2- 172.21.15.157:0/1329029207 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa398066f30 0x7fa3980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:00.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.853+0000 7fa3a67fc700 1 -- 172.21.15.157:0/1329029207 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7fa3b0086060 con 0x7fa3b806cf80 2023-10-06T04:11:00.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.853+0000 7fa3b659c700 1 --2- 172.21.15.157:0/1329029207 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa398066f30 0x7fa3980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:00.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.854+0000 7fa3b659c700 1 --2- 172.21.15.157:0/1329029207 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa398066f30 0x7fa3980693e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8006fd0 tx=0x7fa3a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:00.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:00.857+0000 7fa3a67fc700 1 -- 172.21.15.157:0/1329029207 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa3b0052ee0 con 0x7fa3b806cf80 2023-10-06T04:11:01.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.044+0000 7fa3bc923700 1 -- 172.21.15.157:0/1329029207 --> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3b81462a0 con 0x7fa398066f30 2023-10-06T04:11:01.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.045+0000 7fa3a67fc700 1 -- 172.21.15.157:0/1329029207 <== mgr.4107 v2:172.21.15.175:6832/90661 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa3b81462a0 con 0x7fa398066f30 2023-10-06T04:11:01.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.047+0000 7fa3bc923700 1 -- 172.21.15.157:0/1329029207 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa398066f30 msgr2=0x7fa3980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:01.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.047+0000 7fa3bc923700 1 --2- 172.21.15.157:0/1329029207 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa398066f30 0x7fa3980693e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8006fd0 tx=0x7fa3a8008040 comp rx=0 tx=0).stop 2023-10-06T04:11:01.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.047+0000 7fa3bc923700 1 -- 172.21.15.157:0/1329029207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3b806cf80 msgr2=0x7fa3b8169740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:01.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.047+0000 7fa3bc923700 1 --2- 172.21.15.157:0/1329029207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3b806cf80 0x7fa3b8169740 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80c3960 tx=0x7fa3b0018000 comp rx=0 tx=0).stop 2023-10-06T04:11:01.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.047+0000 7fa3bc923700 1 -- 172.21.15.157:0/1329029207 shutdown_connections 2023-10-06T04:11:01.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.047+0000 7fa3bc923700 1 --2- 172.21.15.157:0/1329029207 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa398066f30 0x7fa3980693e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.047+0000 7fa3bc923700 1 --2- 172.21.15.157:0/1329029207 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3b806c1f0 0x7fa3b8146ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.047+0000 7fa3bc923700 1 --2- 172.21.15.157:0/1329029207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3b806cf80 0x7fa3b8169740 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.047+0000 7fa3bc923700 1 --2- 172.21.15.157:0/1329029207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3b806cbb0 0x7fa3b814b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.047+0000 7fa3bc923700 1 -- 172.21.15.157:0/1329029207 >> 172.21.15.157:0/1329029207 conn(0x7fa3b806f230 msgr2=0x7fa3b814e960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:01.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.047+0000 7fa3bc923700 1 -- 172.21.15.157:0/1329029207 shutdown_connections 2023-10-06T04:11:01.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.047+0000 7fa3bc923700 1 -- 172.21.15.157:0/1329029207 wait complete. 2023-10-06T04:11:01.053 INFO:teuthology.orchestra.run.smithi157.stderr:instructed osd(s) 3 to deep-scrub 2023-10-06T04:11:01.061 INFO:tasks.ceph:Scrubbing osd.4 2023-10-06T04:11:01.061 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-06T04:11:01.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.178+0000 7fb31ea62700 1 Processor -- start 2023-10-06T04:11:01.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.178+0000 7fb31ea62700 1 -- start start 2023-10-06T04:11:01.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.179+0000 7fb31ea62700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb3181504b0 0x7fb318155f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:01.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.179+0000 7fb31ea62700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb3180c8b30 0x7fb3180b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:01.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.179+0000 7fb31ea62700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3180c7040 con 0x7fb3181500e0 2023-10-06T04:11:01.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.179+0000 7fb31ea62700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3180af4c0 con 0x7fb3181504b0 2023-10-06T04:11:01.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.179+0000 7fb31ea62700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3180c9000 con 0x7fb3180c8b30 2023-10-06T04:11:01.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.179+0000 7fb3177fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb3181504b0 0x7fb318155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:01.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.179+0000 7fb3177fe700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb3181504b0 0x7fb318155f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51744/0 (socket says 172.21.15.157:51744) 2023-10-06T04:11:01.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.179+0000 7fb3177fe700 1 -- 172.21.15.157:0/2875291415 learned_addr learned my addr 172.21.15.157:0/2875291415 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:11:01.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.179+0000 7fb31cfff700 1 --2- 172.21.15.157:0/2875291415 >> v2:172.21.15.175:3300/0 conn(0x7fb3180c8b30 0x7fb3180b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:01.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.179+0000 7fb3177fe700 1 -- 172.21.15.157:0/2875291415 >> v2:172.21.15.175:3300/0 conn(0x7fb3180c8b30 msgr2=0x7fb3180b1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:11:01.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.179+0000 7fb3177fe700 1 --2- 172.21.15.157:0/2875291415 >> v2:172.21.15.175:3300/0 conn(0x7fb3180c8b30 0x7fb3180b1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.179+0000 7fb3177fe700 1 -- 172.21.15.157:0/2875291415 >> v1:172.21.15.175:6789/0 conn(0x7fb3181500e0 legacy=0x7fb3180c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:11:01.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.179+0000 7fb3177fe700 1 -- 172.21.15.157:0/2875291415 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3180c92e0 con 0x7fb3181504b0 2023-10-06T04:11:01.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.180+0000 7fb3177fe700 1 --2- 172.21.15.157:0/2875291415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3181504b0 0x7fb318155f60 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb3180c3630 tx=0x7fb30c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d65cb077652dca8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:01.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.180+0000 7fb316ffd700 1 -- 172.21.15.157:0/2875291415 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c00b860 con 0x7fb3181504b0 2023-10-06T04:11:01.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.180+0000 7fb316ffd700 1 -- 172.21.15.157:0/2875291415 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30c00b9c0 con 0x7fb3181504b0 2023-10-06T04:11:01.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.180+0000 7fb316ffd700 1 -- 172.21.15.157:0/2875291415 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c00bd60 con 0x7fb3181504b0 2023-10-06T04:11:01.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.180+0000 7fb31ea62700 1 -- 172.21.15.157:0/2875291415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3181504b0 msgr2=0x7fb318155f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:01.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.180+0000 7fb31ea62700 1 --2- 172.21.15.157:0/2875291415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3181504b0 0x7fb318155f60 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb3180c3630 tx=0x7fb30c009250 comp rx=0 tx=0).stop 2023-10-06T04:11:01.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.180+0000 7fb31ea62700 1 -- 172.21.15.157:0/2875291415 shutdown_connections 2023-10-06T04:11:01.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.180+0000 7fb31ea62700 1 --2- 172.21.15.157:0/2875291415 >> v2:172.21.15.175:3300/0 conn(0x7fb3180c8b30 0x7fb3180b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.180+0000 7fb31ea62700 1 --2- 172.21.15.157:0/2875291415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3181504b0 0x7fb318155f60 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.180+0000 7fb31ea62700 1 -- 172.21.15.157:0/2875291415 >> 172.21.15.157:0/2875291415 conn(0x7fb31806f230 msgr2=0x7fb31806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:01.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.180+0000 7fb31ea62700 1 -- 172.21.15.157:0/2875291415 shutdown_connections 2023-10-06T04:11:01.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.180+0000 7fb31ea62700 1 -- 172.21.15.157:0/2875291415 wait complete. 2023-10-06T04:11:01.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.181+0000 7fb31ea62700 1 Processor -- start 2023-10-06T04:11:01.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.182+0000 7fb31ea62700 1 -- start start 2023-10-06T04:11:01.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.182+0000 7fb31ea62700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3180c8b30 0x7fb318151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:01.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.182+0000 7fb31ea62700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3181500e0 0x7fb3180ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:01.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.182+0000 7fb31ea62700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3181504b0 0x7fb318147780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:01.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.182+0000 7fb31ea62700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb318147cc0 con 0x7fb3180c8b30 2023-10-06T04:11:01.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.182+0000 7fb31ea62700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3180c73f0 con 0x7fb3181500e0 2023-10-06T04:11:01.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.182+0000 7fb31ea62700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb318145f30 con 0x7fb3181504b0 2023-10-06T04:11:01.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.182+0000 7fb317fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3180c8b30 0x7fb318151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:01.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.182+0000 7fb317fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3180c8b30 0x7fb318151360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51760/0 (socket says 172.21.15.157:51760) 2023-10-06T04:11:01.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.182+0000 7fb317fff700 1 -- 172.21.15.157:0/3702325066 learned_addr learned my addr 172.21.15.157:0/3702325066 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:11:01.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.183+0000 7fb3177fe700 1 --2- 172.21.15.157:0/3702325066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3181500e0 0x7fb3180ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:01.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.183+0000 7fb31cfff700 1 --2- 172.21.15.157:0/3702325066 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3181504b0 0x7fb318147780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:01.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.183+0000 7fb317fff700 1 -- 172.21.15.157:0/3702325066 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3181504b0 msgr2=0x7fb318147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:01.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.183+0000 7fb317fff700 1 --2- 172.21.15.157:0/3702325066 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3181504b0 0x7fb318147780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.183+0000 7fb317fff700 1 -- 172.21.15.157:0/3702325066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3181500e0 msgr2=0x7fb3180ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:01.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.183+0000 7fb317fff700 1 --2- 172.21.15.157:0/3702325066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3181500e0 0x7fb3180ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.183+0000 7fb317fff700 1 -- 172.21.15.157:0/3702325066 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb30c0094f0 con 0x7fb3180c8b30 2023-10-06T04:11:01.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.183+0000 7fb3177fe700 1 --2- 172.21.15.157:0/3702325066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3181500e0 0x7fb3180ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:11:01.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.183+0000 7fb317fff700 1 --2- 172.21.15.157:0/3702325066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3180c8b30 0x7fb318151360 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb30800c5c0 tx=0x7fb30800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:01.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.183+0000 7fb2fffff700 1 -- 172.21.15.157:0/3702325066 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb308011590 con 0x7fb3180c8b30 2023-10-06T04:11:01.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.183+0000 7fb2fffff700 1 -- 172.21.15.157:0/3702325066 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb308011710 con 0x7fb3180c8b30 2023-10-06T04:11:01.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.183+0000 7fb31ea62700 1 -- 172.21.15.157:0/3702325066 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3181472f0 con 0x7fb3180c8b30 2023-10-06T04:11:01.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.183+0000 7fb2fffff700 1 -- 172.21.15.157:0/3702325066 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb308011ad0 con 0x7fb3180c8b30 2023-10-06T04:11:01.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.183+0000 7fb31ea62700 1 -- 172.21.15.157:0/3702325066 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb318145c50 con 0x7fb3180c8b30 2023-10-06T04:11:01.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.184+0000 7fb31ea62700 1 -- 172.21.15.157:0/3702325066 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb318089ca0 con 0x7fb3180c8b30 2023-10-06T04:11:01.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.184+0000 7fb2fffff700 1 -- 172.21.15.157:0/3702325066 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb308012420 con 0x7fb3180c8b30 2023-10-06T04:11:01.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.185+0000 7fb2fffff700 1 --2- 172.21.15.157:0/3702325066 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb304066f60 0x7fb304069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:01.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.185+0000 7fb2fffff700 1 -- 172.21.15.157:0/3702325066 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7fb308083c90 con 0x7fb3180c8b30 2023-10-06T04:11:01.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.185+0000 7fb2fffff700 1 --2- 172.21.15.157:0/3702325066 >> [v2:172.21.15.175:6800/91261,v1:172.21.15.175:6803/91261] conn(0x7fb30406fdc0 0x7fb3040721f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:01.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.185+0000 7fb2fffff700 1 -- 172.21.15.157:0/3702325066 --> [v2:172.21.15.175:6800/91261,v1:172.21.15.175:6803/91261] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb3040728c0 con 0x7fb30406fdc0 2023-10-06T04:11:01.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.185+0000 7fb2fffff700 1 -- 172.21.15.157:0/3702325066 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3080862d0 con 0x7fb3180c8b30 2023-10-06T04:11:01.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.185+0000 7fb3177fe700 1 --2- 172.21.15.157:0/3702325066 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb304066f60 0x7fb304069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:01.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.185+0000 7fb31cfff700 1 --2- 172.21.15.157:0/3702325066 >> [v2:172.21.15.175:6800/91261,v1:172.21.15.175:6803/91261] conn(0x7fb30406fdc0 0x7fb3040721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:01.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.185+0000 7fb3177fe700 1 --2- 172.21.15.157:0/3702325066 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb304066f60 0x7fb304069410 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb3180c3630 tx=0x7fb30c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:01.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.186+0000 7fb31cfff700 1 --2- 172.21.15.157:0/3702325066 >> [v2:172.21.15.175:6800/91261,v1:172.21.15.175:6803/91261] conn(0x7fb30406fdc0 0x7fb3040721f0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:01.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.186+0000 7fb2fffff700 1 -- 172.21.15.157:0/3702325066 <== osd.4 v2:172.21.15.175:6800/91261 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fb3040728c0 con 0x7fb30406fdc0 2023-10-06T04:11:01.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.200+0000 7fb31ea62700 1 -- 172.21.15.157:0/3702325066 --> [v2:172.21.15.175:6800/91261,v1:172.21.15.175:6803/91261] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb3180c95c0 con 0x7fb30406fdc0 2023-10-06T04:11:01.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.200+0000 7fb2fffff700 1 -- 172.21.15.157:0/3702325066 <== osd.4 v2:172.21.15.175:6800/91261 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb3180c95c0 con 0x7fb30406fdc0 2023-10-06T04:11:01.202 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-10-06T04:11:01.202 INFO:teuthology.orchestra.run.smithi157.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T04:11:01.202 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-10-06T04:11:01.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.201+0000 7fb31ea62700 1 -- 172.21.15.157:0/3702325066 >> [v2:172.21.15.175:6800/91261,v1:172.21.15.175:6803/91261] conn(0x7fb30406fdc0 msgr2=0x7fb3040721f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:01.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.201+0000 7fb31ea62700 1 --2- 172.21.15.157:0/3702325066 >> [v2:172.21.15.175:6800/91261,v1:172.21.15.175:6803/91261] conn(0x7fb30406fdc0 0x7fb3040721f0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.201+0000 7fb31ea62700 1 -- 172.21.15.157:0/3702325066 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb304066f60 msgr2=0x7fb304069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:01.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.201+0000 7fb31ea62700 1 --2- 172.21.15.157:0/3702325066 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fb304066f60 0x7fb304069410 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb3180c3630 tx=0x7fb30c017000 comp rx=0 tx=0).stop 2023-10-06T04:11:01.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.201+0000 7fb31ea62700 1 -- 172.21.15.157:0/3702325066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3180c8b30 msgr2=0x7fb318151360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:01.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.201+0000 7fb31ea62700 1 --2- 172.21.15.157:0/3702325066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3180c8b30 0x7fb318151360 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb30800c5c0 tx=0x7fb30800c980 comp rx=0 tx=0).stop 2023-10-06T04:11:01.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.201+0000 7fb31cfff700 1 -- 172.21.15.157:0/3702325066 reap_dead start 2023-10-06T04:11:01.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.201+0000 7fb31ea62700 1 -- 172.21.15.157:0/3702325066 shutdown_connections 2023-10-06T04:11:01.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.201+0000 7fb31ea62700 1 -- 172.21.15.157:0/3702325066 >> 172.21.15.157:0/3702325066 conn(0x7fb31806f230 msgr2=0x7fb31814bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:01.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.201+0000 7fb31ea62700 1 -- 172.21.15.157:0/3702325066 shutdown_connections 2023-10-06T04:11:01.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.201+0000 7fb31ea62700 1 -- 172.21.15.157:0/3702325066 wait complete. 2023-10-06T04:11:01.214 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-06T04:11:01.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.335+0000 7f74c0861700 1 Processor -- start 2023-10-06T04:11:01.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.335+0000 7f74c0861700 1 -- start start 2023-10-06T04:11:01.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.336+0000 7f74c0861700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f74bc0c9450 0x7f74bc06a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:01.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.336+0000 7f74c0861700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74bc06af50 con 0x7f74bc14d530 2023-10-06T04:11:01.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.336+0000 7f74c0861700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74bc153160 con 0x7f74bc0bfad0 2023-10-06T04:11:01.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.336+0000 7f74c0861700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74bc152f50 con 0x7f74bc0c9450 2023-10-06T04:11:01.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.336+0000 7f74bad9d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f74bc0c9450 0x7f74bc06a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:01.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.336+0000 7f74bb59e700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f74bc14d530 0x7f74bc06b3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:35380/0 (socket says 172.21.15.157:35380) 2023-10-06T04:11:01.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.336+0000 7f74bb59e700 1 -- 172.21.15.157:0/1630265471 learned_addr learned my addr 172.21.15.157:0/1630265471 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:11:01.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.336+0000 7f74bad9d700 1 -- 172.21.15.157:0/1630265471 >> v1:172.21.15.175:6789/0 conn(0x7f74bc0bfad0 legacy=0x7f74bc14ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:01.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.336+0000 7f74bad9d700 1 -- 172.21.15.157:0/1630265471 >> v1:172.21.15.157:6789/0 conn(0x7f74bc14d530 legacy=0x7f74bc06b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:01.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.336+0000 7f74bad9d700 1 -- 172.21.15.157:0/1630265471 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74bc153440 con 0x7f74bc0c9450 2023-10-06T04:11:01.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.336+0000 7f74b9d9b700 1 -- 172.21.15.157:0/1630265471 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4236229465 0 0) 0x7f74bc06af50 con 0x7f74bc14d530 2023-10-06T04:11:01.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.337+0000 7f74bad9d700 1 --2- 172.21.15.157:0/1630265471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74bc0c9450 0x7f74bc06a330 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0098e0 tx=0x7f74ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9246e6809074c43b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:01.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.337+0000 7f74b9d9b700 1 -- 172.21.15.157:0/1630265471 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac009e80 con 0x7f74bc0c9450 2023-10-06T04:11:01.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.337+0000 7f74b9d9b700 1 -- 172.21.15.157:0/1630265471 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74ac00b860 con 0x7f74bc0c9450 2023-10-06T04:11:01.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.337+0000 7f74b9d9b700 1 -- 172.21.15.157:0/1630265471 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac00bbe0 con 0x7f74bc0c9450 2023-10-06T04:11:01.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.337+0000 7f74c0861700 1 -- 172.21.15.157:0/1630265471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74bc0c9450 msgr2=0x7f74bc06a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:01.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.337+0000 7f74c0861700 1 --2- 172.21.15.157:0/1630265471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74bc0c9450 0x7f74bc06a330 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0098e0 tx=0x7f74ac0091a0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.337+0000 7f74c0861700 1 -- 172.21.15.157:0/1630265471 shutdown_connections 2023-10-06T04:11:01.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.337+0000 7f74c0861700 1 --2- 172.21.15.157:0/1630265471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74bc0c9450 0x7f74bc06a330 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.337+0000 7f74c0861700 1 -- 172.21.15.157:0/1630265471 >> 172.21.15.157:0/1630265471 conn(0x7f74bc068020 msgr2=0x7f74bc068420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:01.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.337+0000 7f74c0861700 1 -- 172.21.15.157:0/1630265471 shutdown_connections 2023-10-06T04:11:01.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.337+0000 7f74c0861700 1 -- 172.21.15.157:0/1630265471 wait complete. 2023-10-06T04:11:01.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.339+0000 7f74c0861700 1 Processor -- start 2023-10-06T04:11:01.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.339+0000 7f74c0861700 1 -- start start 2023-10-06T04:11:01.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.339+0000 7f74c0861700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74bc0bfad0 0x7f74bc143b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:01.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.339+0000 7f74c0861700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74bc0c9450 0x7f74bc145130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:01.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.339+0000 7f74c0861700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74bc14d530 0x7f74bc14dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:01.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.339+0000 7f74c0861700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc144070 con 0x7f74bc14d530 2023-10-06T04:11:01.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.339+0000 7f74c0861700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc14ac80 con 0x7f74bc0c9450 2023-10-06T04:11:01.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.339+0000 7f74c0861700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc0ab7f0 con 0x7f74bc0bfad0 2023-10-06T04:11:01.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.340+0000 7f74ba59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74bc0c9450 0x7f74bc145130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:01.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.340+0000 7f74bad9d700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74bc0bfad0 0x7f74bc143b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:01.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.340+0000 7f74bb59e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74bc14d530 0x7f74bc14dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:01.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.340+0000 7f74bad9d700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74bc0bfad0 0x7f74bc143b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:50590/0 (socket says 172.21.15.157:50590) 2023-10-06T04:11:01.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.340+0000 7f74bad9d700 1 -- 172.21.15.157:0/2028858080 learned_addr learned my addr 172.21.15.157:0/2028858080 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:11:01.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.340+0000 7f74ba59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74bc0c9450 0x7f74bc145130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:44808/0 (socket says 172.21.15.157:44808) 2023-10-06T04:11:01.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.340+0000 7f74bb59e700 1 -- 172.21.15.157:0/2028858080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74bc0bfad0 msgr2=0x7f74bc143b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:01.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.340+0000 7f74bb59e700 1 --2- 172.21.15.157:0/2028858080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74bc0bfad0 0x7f74bc143b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.340+0000 7f74bb59e700 1 -- 172.21.15.157:0/2028858080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74bc0c9450 msgr2=0x7f74bc145130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:01.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.340+0000 7f74bb59e700 1 --2- 172.21.15.157:0/2028858080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74bc0c9450 0x7f74bc145130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.340+0000 7f74bb59e700 1 -- 172.21.15.157:0/2028858080 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ac009420 con 0x7f74bc14d530 2023-10-06T04:11:01.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.340+0000 7f74ba59c700 1 --2- 172.21.15.157:0/2028858080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74bc0c9450 0x7f74bc145130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:11:01.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.340+0000 7f74bb59e700 1 --2- 172.21.15.157:0/2028858080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74bc14d530 0x7f74bc14dae0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f74bc0befe0 tx=0x7f74b4001900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:01.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.340+0000 7f74a2ffd700 1 -- 172.21.15.157:0/2028858080 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b400a7e0 con 0x7f74bc14d530 2023-10-06T04:11:01.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.340+0000 7f74a2ffd700 1 -- 172.21.15.157:0/2028858080 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b400a940 con 0x7f74bc14d530 2023-10-06T04:11:01.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.340+0000 7f74a2ffd700 1 -- 172.21.15.157:0/2028858080 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b400ad00 con 0x7f74bc14d530 2023-10-06T04:11:01.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.340+0000 7f74c0861700 1 -- 172.21.15.157:0/2028858080 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74bc0abad0 con 0x7f74bc14d530 2023-10-06T04:11:01.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.341+0000 7f74c0861700 1 -- 172.21.15.157:0/2028858080 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74bc1528c0 con 0x7f74bc14d530 2023-10-06T04:11:01.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.342+0000 7f74a2ffd700 1 -- 172.21.15.157:0/2028858080 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f74b4007300 con 0x7f74bc14d530 2023-10-06T04:11:01.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.342+0000 7f74a2ffd700 1 --2- 172.21.15.157:0/2028858080 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f74a8066f30 0x7f74a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:01.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.342+0000 7f74a2ffd700 1 -- 172.21.15.157:0/2028858080 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f74b4084b50 con 0x7f74bc14d530 2023-10-06T04:11:01.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.343+0000 7f74bad9d700 1 --2- 172.21.15.157:0/2028858080 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f74a8066f30 0x7f74a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:01.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.343+0000 7f74c0861700 1 -- 172.21.15.157:0/2028858080 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74bc152ae0 con 0x7f74bc14d530 2023-10-06T04:11:01.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.343+0000 7f74bad9d700 1 --2- 172.21.15.157:0/2028858080 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f74a8066f30 0x7f74a80693e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0091e0 tx=0x7f74ac017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:01.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.347+0000 7f74a2ffd700 1 -- 172.21.15.157:0/2028858080 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f74b40519c0 con 0x7f74bc14d530 2023-10-06T04:11:01.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.534+0000 7f74c0861700 1 -- 172.21.15.157:0/2028858080 --> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f74bc152460 con 0x7f74a8066f30 2023-10-06T04:11:01.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.534+0000 7f74a2ffd700 1 -- 172.21.15.157:0/2028858080 <== mgr.4107 v2:172.21.15.175:6832/90661 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f74ac00ba60 con 0x7f74a8066f30 2023-10-06T04:11:01.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.536+0000 7f74c0861700 1 -- 172.21.15.157:0/2028858080 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f74a8066f30 msgr2=0x7f74a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:01.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.536+0000 7f74c0861700 1 --2- 172.21.15.157:0/2028858080 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f74a8066f30 0x7f74a80693e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0091e0 tx=0x7f74ac017000 comp rx=0 tx=0).stop 2023-10-06T04:11:01.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.536+0000 7f74c0861700 1 -- 172.21.15.157:0/2028858080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74bc14d530 msgr2=0x7f74bc14dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:01.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.536+0000 7f74c0861700 1 --2- 172.21.15.157:0/2028858080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74bc14d530 0x7f74bc14dae0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f74bc0befe0 tx=0x7f74b4001900 comp rx=0 tx=0).stop 2023-10-06T04:11:01.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.536+0000 7f74c0861700 1 -- 172.21.15.157:0/2028858080 shutdown_connections 2023-10-06T04:11:01.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.536+0000 7f74c0861700 1 --2- 172.21.15.157:0/2028858080 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f74a8066f30 0x7f74a80693e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.536+0000 7f74c0861700 1 --2- 172.21.15.157:0/2028858080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74bc14d530 0x7f74bc14dae0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.536+0000 7f74c0861700 1 --2- 172.21.15.157:0/2028858080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74bc0bfad0 0x7f74bc143b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.536+0000 7f74c0861700 1 --2- 172.21.15.157:0/2028858080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74bc0c9450 0x7f74bc145130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.536+0000 7f74c0861700 1 -- 172.21.15.157:0/2028858080 >> 172.21.15.157:0/2028858080 conn(0x7f74bc068020 msgr2=0x7f74bc14a100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:01.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.536+0000 7f74c0861700 1 -- 172.21.15.157:0/2028858080 shutdown_connections 2023-10-06T04:11:01.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.537+0000 7f74c0861700 1 -- 172.21.15.157:0/2028858080 wait complete. 2023-10-06T04:11:01.540 INFO:teuthology.orchestra.run.smithi157.stderr:instructed osd(s) 4 to deep-scrub 2023-10-06T04:11:01.550 INFO:tasks.ceph:Scrubbing osd.5 2023-10-06T04:11:01.550 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-06T04:11:01.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.672+0000 7fd534d42700 1 Processor -- start 2023-10-06T04:11:01.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.672+0000 7fd534d42700 1 -- start start 2023-10-06T04:11:01.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.673+0000 7fd534d42700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5301500e0 0x7fd530155f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:01.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.673+0000 7fd534d42700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd5301504b0 0x7fd5300c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:01.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.673+0000 7fd534d42700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd5300c8b30 0x7fd5300b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:01.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.673+0000 7fd534d42700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5300c7040 con 0x7fd5300c8b30 2023-10-06T04:11:01.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.673+0000 7fd534d42700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5300af4c0 con 0x7fd5301504b0 2023-10-06T04:11:01.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.673+0000 7fd534d42700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300c9000 con 0x7fd5301500e0 2023-10-06T04:11:01.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.673+0000 7fd52ed9d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd5300c8b30 0x7fd5300b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:01.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.673+0000 7fd52ed9d700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd5300c8b30 0x7fd5300b1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51784/0 (socket says 172.21.15.157:51784) 2023-10-06T04:11:01.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.673+0000 7fd52ed9d700 1 -- 172.21.15.157:0/554816403 learned_addr learned my addr 172.21.15.157:0/554816403 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:11:01.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.673+0000 7fd52dd9b700 1 --2- 172.21.15.157:0/554816403 >> v2:172.21.15.175:3300/0 conn(0x7fd5301504b0 0x7fd5300c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:01.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.673+0000 7fd52e59c700 1 --2- 172.21.15.157:0/554816403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5301500e0 0x7fd530155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:01.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.674+0000 7fd52ed9d700 1 -- 172.21.15.157:0/554816403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5301500e0 msgr2=0x7fd530155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:11:01.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.674+0000 7fd52ed9d700 1 --2- 172.21.15.157:0/554816403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5301500e0 0x7fd530155f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.674+0000 7fd52ed9d700 1 -- 172.21.15.157:0/554816403 >> v2:172.21.15.175:3300/0 conn(0x7fd5301504b0 msgr2=0x7fd5300c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:11:01.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.674+0000 7fd52ed9d700 1 --2- 172.21.15.157:0/554816403 >> v2:172.21.15.175:3300/0 conn(0x7fd5301504b0 0x7fd5300c77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.674+0000 7fd52ed9d700 1 -- 172.21.15.157:0/554816403 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5300c92e0 con 0x7fd5300c8b30 2023-10-06T04:11:01.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.674+0000 7fd52e59c700 1 --2- 172.21.15.157:0/554816403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5301500e0 0x7fd530155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:11:01.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.674+0000 7fd52ed9d700 1 --2- 172.21.15.157:0/554816403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5300c8b30 0x7fd5300b1c30 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd53006fdd0 tx=0x7fd528009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4102833b141b575 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:01.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.674+0000 7fd52d59a700 1 -- 172.21.15.157:0/554816403 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528009e70 con 0x7fd5300c8b30 2023-10-06T04:11:01.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.674+0000 7fd52d59a700 1 -- 172.21.15.157:0/554816403 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52800b860 con 0x7fd5300c8b30 2023-10-06T04:11:01.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.674+0000 7fd52d59a700 1 -- 172.21.15.157:0/554816403 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52800bc00 con 0x7fd5300c8b30 2023-10-06T04:11:01.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.674+0000 7fd534d42700 1 -- 172.21.15.157:0/554816403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5300c8b30 msgr2=0x7fd5300b1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:01.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.674+0000 7fd534d42700 1 --2- 172.21.15.157:0/554816403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5300c8b30 0x7fd5300b1c30 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd53006fdd0 tx=0x7fd528009250 comp rx=0 tx=0).stop 2023-10-06T04:11:01.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.675+0000 7fd534d42700 1 -- 172.21.15.157:0/554816403 shutdown_connections 2023-10-06T04:11:01.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.675+0000 7fd534d42700 1 --2- 172.21.15.157:0/554816403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5300c8b30 0x7fd5300b1c30 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.675+0000 7fd534d42700 1 --2- 172.21.15.157:0/554816403 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5301500e0 0x7fd530155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.675+0000 7fd534d42700 1 --2- 172.21.15.157:0/554816403 >> v2:172.21.15.175:3300/0 conn(0x7fd5301504b0 0x7fd5300c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.675+0000 7fd534d42700 1 -- 172.21.15.157:0/554816403 >> 172.21.15.157:0/554816403 conn(0x7fd53006f230 msgr2=0x7fd53006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:01.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.675+0000 7fd534d42700 1 -- 172.21.15.157:0/554816403 shutdown_connections 2023-10-06T04:11:01.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.675+0000 7fd534d42700 1 -- 172.21.15.157:0/554816403 wait complete. 2023-10-06T04:11:01.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.676+0000 7fd534d42700 1 Processor -- start 2023-10-06T04:11:01.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.676+0000 7fd534d42700 1 -- start start 2023-10-06T04:11:01.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.676+0000 7fd534d42700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5300c8b30 0x7fd5300af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:01.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.676+0000 7fd534d42700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5301500e0 0x7fd5301508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:01.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.676+0000 7fd534d42700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5301504b0 0x7fd530149470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:01.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.676+0000 7fd534d42700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd530166590 con 0x7fd5300c8b30 2023-10-06T04:11:01.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.676+0000 7fd534d42700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd530166a20 con 0x7fd5301500e0 2023-10-06T04:11:01.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.676+0000 7fd534d42700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd53014eb20 con 0x7fd5301504b0 2023-10-06T04:11:01.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.676+0000 7fd52e59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5300c8b30 0x7fd5300af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:01.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.676+0000 7fd52e59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5300c8b30 0x7fd5300af930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51792/0 (socket says 172.21.15.157:51792) 2023-10-06T04:11:01.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.676+0000 7fd52e59c700 1 -- 172.21.15.157:0/364809786 learned_addr learned my addr 172.21.15.157:0/364809786 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:11:01.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.677+0000 7fd52dd9b700 1 --2- 172.21.15.157:0/364809786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5301500e0 0x7fd5301508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:01.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.677+0000 7fd52ed9d700 1 --2- 172.21.15.157:0/364809786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5301504b0 0x7fd530149470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:01.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.677+0000 7fd52e59c700 1 -- 172.21.15.157:0/364809786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5301504b0 msgr2=0x7fd530149470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:01.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.677+0000 7fd52e59c700 1 --2- 172.21.15.157:0/364809786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5301504b0 0x7fd530149470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.677+0000 7fd52e59c700 1 -- 172.21.15.157:0/364809786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5301500e0 msgr2=0x7fd5301508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:01.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.677+0000 7fd52e59c700 1 --2- 172.21.15.157:0/364809786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5301500e0 0x7fd5301508c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.677+0000 7fd52e59c700 1 -- 172.21.15.157:0/364809786 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5280094f0 con 0x7fd5300c8b30 2023-10-06T04:11:01.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.677+0000 7fd52e59c700 1 --2- 172.21.15.157:0/364809786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5300c8b30 0x7fd5300af930 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd52000d1d0 tx=0x7fd520009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:01.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.677+0000 7fd51e7fc700 1 -- 172.21.15.157:0/364809786 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52000de70 con 0x7fd5300c8b30 2023-10-06T04:11:01.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.677+0000 7fd51e7fc700 1 -- 172.21.15.157:0/364809786 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52000f040 con 0x7fd5300c8b30 2023-10-06T04:11:01.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.677+0000 7fd534d42700 1 -- 172.21.15.157:0/364809786 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd530153670 con 0x7fd5300c8b30 2023-10-06T04:11:01.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.677+0000 7fd51e7fc700 1 -- 172.21.15.157:0/364809786 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd520015670 con 0x7fd5300c8b30 2023-10-06T04:11:01.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.677+0000 7fd534d42700 1 -- 172.21.15.157:0/364809786 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd530165f40 con 0x7fd5300c8b30 2023-10-06T04:11:01.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.678+0000 7fd534d42700 1 -- 172.21.15.157:0/364809786 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd530089ca0 con 0x7fd5300c8b30 2023-10-06T04:11:01.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.678+0000 7fd51e7fc700 1 -- 172.21.15.157:0/364809786 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd520015a20 con 0x7fd5300c8b30 2023-10-06T04:11:01.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.679+0000 7fd51e7fc700 1 --2- 172.21.15.157:0/364809786 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd510066f30 0x7fd5100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:01.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.679+0000 7fd51e7fc700 1 -- 172.21.15.157:0/364809786 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7fd520084df0 con 0x7fd5300c8b30 2023-10-06T04:11:01.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.679+0000 7fd51e7fc700 1 --2- 172.21.15.157:0/364809786 >> [v2:172.21.15.175:6816/91262,v1:172.21.15.175:6817/91262] conn(0x7fd51006fd90 0x7fd5100721c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:01.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.679+0000 7fd51e7fc700 1 -- 172.21.15.157:0/364809786 --> [v2:172.21.15.175:6816/91262,v1:172.21.15.175:6817/91262] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd510072890 con 0x7fd51006fd90 2023-10-06T04:11:01.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.679+0000 7fd52dd9b700 1 --2- 172.21.15.157:0/364809786 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd510066f30 0x7fd5100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:01.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.679+0000 7fd51e7fc700 1 -- 172.21.15.157:0/364809786 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5200873d0 con 0x7fd5300c8b30 2023-10-06T04:11:01.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.679+0000 7fd52ed9d700 1 --2- 172.21.15.157:0/364809786 >> [v2:172.21.15.175:6816/91262,v1:172.21.15.175:6817/91262] conn(0x7fd51006fd90 0x7fd5100721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:01.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.680+0000 7fd52dd9b700 1 --2- 172.21.15.157:0/364809786 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd510066f30 0x7fd5100693e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd53006c4a0 tx=0x7fd524008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:01.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.680+0000 7fd52ed9d700 1 --2- 172.21.15.157:0/364809786 >> [v2:172.21.15.175:6816/91262,v1:172.21.15.175:6817/91262] conn(0x7fd51006fd90 0x7fd5100721c0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:01.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.680+0000 7fd51e7fc700 1 -- 172.21.15.157:0/364809786 <== osd.5 v2:172.21.15.175:6816/91262 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fd510072890 con 0x7fd51006fd90 2023-10-06T04:11:01.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.694+0000 7fd534d42700 1 -- 172.21.15.157:0/364809786 --> [v2:172.21.15.175:6816/91262,v1:172.21.15.175:6817/91262] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd5300c95c0 con 0x7fd51006fd90 2023-10-06T04:11:01.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.694+0000 7fd51e7fc700 1 -- 172.21.15.157:0/364809786 <== osd.5 v2:172.21.15.175:6816/91262 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fd5300c95c0 con 0x7fd51006fd90 2023-10-06T04:11:01.696 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-10-06T04:11:01.696 INFO:teuthology.orchestra.run.smithi157.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T04:11:01.696 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-10-06T04:11:01.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.695+0000 7fd534d42700 1 -- 172.21.15.157:0/364809786 >> [v2:172.21.15.175:6816/91262,v1:172.21.15.175:6817/91262] conn(0x7fd51006fd90 msgr2=0x7fd5100721c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:01.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.695+0000 7fd534d42700 1 --2- 172.21.15.157:0/364809786 >> [v2:172.21.15.175:6816/91262,v1:172.21.15.175:6817/91262] conn(0x7fd51006fd90 0x7fd5100721c0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.695+0000 7fd534d42700 1 -- 172.21.15.157:0/364809786 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd510066f30 msgr2=0x7fd5100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:01.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.695+0000 7fd534d42700 1 --2- 172.21.15.157:0/364809786 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fd510066f30 0x7fd5100693e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd53006c4a0 tx=0x7fd524008040 comp rx=0 tx=0).stop 2023-10-06T04:11:01.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.695+0000 7fd534d42700 1 -- 172.21.15.157:0/364809786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5300c8b30 msgr2=0x7fd5300af930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:01.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.695+0000 7fd534d42700 1 --2- 172.21.15.157:0/364809786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5300c8b30 0x7fd5300af930 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd52000d1d0 tx=0x7fd520009a90 comp rx=0 tx=0).stop 2023-10-06T04:11:01.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.695+0000 7fd52ed9d700 1 -- 172.21.15.157:0/364809786 reap_dead start 2023-10-06T04:11:01.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.695+0000 7fd534d42700 1 -- 172.21.15.157:0/364809786 shutdown_connections 2023-10-06T04:11:01.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.695+0000 7fd534d42700 1 -- 172.21.15.157:0/364809786 >> 172.21.15.157:0/364809786 conn(0x7fd53006f230 msgr2=0x7fd53014e260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:01.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.695+0000 7fd534d42700 1 -- 172.21.15.157:0/364809786 shutdown_connections 2023-10-06T04:11:01.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.695+0000 7fd534d42700 1 -- 172.21.15.157:0/364809786 wait complete. 2023-10-06T04:11:01.708 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-06T04:11:01.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.824+0000 7fbbb2aa0700 1 Processor -- start 2023-10-06T04:11:01.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.824+0000 7fbbb2aa0700 1 -- start start 2023-10-06T04:11:01.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.825+0000 7fbbb2aa0700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbac0b04d0 0x7fbbac157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:01.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.825+0000 7fbbb2aa0700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbbac06c2d0 0x7fbbac06dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:01.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.825+0000 7fbbb2aa0700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbac06d8d0 con 0x7fbbac0b1e40 2023-10-06T04:11:01.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.825+0000 7fbbb2aa0700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbac0af8f0 con 0x7fbbac06c2d0 2023-10-06T04:11:01.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.825+0000 7fbbb2aa0700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac15a690 con 0x7fbbac0b04d0 2023-10-06T04:11:01.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.825+0000 7fbbb229f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbbac06c2d0 0x7fbbac06dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:01.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.825+0000 7fbbb129d700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbac0b04d0 0x7fbbac157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:01.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.825+0000 7fbbb129d700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbac0b04d0 0x7fbbac157750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:50620/0 (socket says 172.21.15.157:50620) 2023-10-06T04:11:01.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.825+0000 7fbbb129d700 1 -- 172.21.15.157:0/1211140920 learned_addr learned my addr 172.21.15.157:0/1211140920 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:11:01.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.825+0000 7fbbb229f700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbbac06c2d0 0x7fbbac06dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:44836/0 (socket says 172.21.15.157:44836) 2023-10-06T04:11:01.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.826+0000 7fbbb129d700 1 -- 172.21.15.157:0/1211140920 >> v2:172.21.15.175:3300/0 conn(0x7fbbac06c2d0 msgr2=0x7fbbac06dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:11:01.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.826+0000 7fbbb129d700 1 --2- 172.21.15.157:0/1211140920 >> v2:172.21.15.175:3300/0 conn(0x7fbbac06c2d0 0x7fbbac06dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.826+0000 7fbbb129d700 1 -- 172.21.15.157:0/1211140920 >> v1:172.21.15.175:6789/0 conn(0x7fbbac0b1e40 legacy=0x7fbbac1563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:01.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.826+0000 7fbbb129d700 1 -- 172.21.15.157:0/1211140920 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbac06ccc0 con 0x7fbbac0b04d0 2023-10-06T04:11:01.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.826+0000 7fbbb129d700 1 --2- 172.21.15.157:0/1211140920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbac0b04d0 0x7fbbac157750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0099b0 tx=0x7fbb9c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a43ed2201fb1d2ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:01.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.826+0000 7fbbb0a9c700 1 -- 172.21.15.157:0/1211140920 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c009ea0 con 0x7fbbac0b04d0 2023-10-06T04:11:01.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.826+0000 7fbbb0a9c700 1 -- 172.21.15.157:0/1211140920 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9c00b860 con 0x7fbbac0b04d0 2023-10-06T04:11:01.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.826+0000 7fbbb0a9c700 1 -- 172.21.15.157:0/1211140920 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c00bc20 con 0x7fbbac0b04d0 2023-10-06T04:11:01.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.826+0000 7fbbb2aa0700 1 -- 172.21.15.157:0/1211140920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbac0b04d0 msgr2=0x7fbbac157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:01.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.826+0000 7fbbb2aa0700 1 --2- 172.21.15.157:0/1211140920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbac0b04d0 0x7fbbac157750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0099b0 tx=0x7fbb9c009250 comp rx=0 tx=0).stop 2023-10-06T04:11:01.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.827+0000 7fbbb2aa0700 1 -- 172.21.15.157:0/1211140920 shutdown_connections 2023-10-06T04:11:01.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.827+0000 7fbbb2aa0700 1 --2- 172.21.15.157:0/1211140920 >> v2:172.21.15.175:3300/0 conn(0x7fbbac06c2d0 0x7fbbac06dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.827+0000 7fbbb2aa0700 1 --2- 172.21.15.157:0/1211140920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbac0b04d0 0x7fbbac157750 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.827+0000 7fbbb2aa0700 1 -- 172.21.15.157:0/1211140920 >> 172.21.15.157:0/1211140920 conn(0x7fbbac0cc540 msgr2=0x7fbbac0cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:01.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.827+0000 7fbbb2aa0700 1 -- 172.21.15.157:0/1211140920 shutdown_connections 2023-10-06T04:11:01.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.827+0000 7fbbb2aa0700 1 -- 172.21.15.157:0/1211140920 wait complete. 2023-10-06T04:11:01.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.828+0000 7fbbb2aa0700 1 Processor -- start 2023-10-06T04:11:01.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.828+0000 7fbbb2aa0700 1 -- start start 2023-10-06T04:11:01.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.828+0000 7fbbb2aa0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbac06c2d0 0x7fbbac166320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:01.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.828+0000 7fbbb2aa0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbac0b04d0 0x7fbbac166860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:01.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.828+0000 7fbbb2aa0700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbac0b1e40 0x7fbbac152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:01.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.828+0000 7fbbb2aa0700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac0c43e0 con 0x7fbbac06c2d0 2023-10-06T04:11:01.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.828+0000 7fbbb2aa0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac14e9b0 con 0x7fbbac0b04d0 2023-10-06T04:11:01.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.828+0000 7fbbb2aa0700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac166ec0 con 0x7fbbac0b1e40 2023-10-06T04:11:01.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.829+0000 7fbbb1a9e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbac06c2d0 0x7fbbac166320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:01.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.829+0000 7fbbb1a9e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbac06c2d0 0x7fbbac166320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51800/0 (socket says 172.21.15.157:51800) 2023-10-06T04:11:01.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.829+0000 7fbbb1a9e700 1 -- 172.21.15.157:0/2146836342 learned_addr learned my addr 172.21.15.157:0/2146836342 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:11:01.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.829+0000 7fbbb129d700 1 --2- 172.21.15.157:0/2146836342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbac0b04d0 0x7fbbac166860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:01.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.829+0000 7fbbb229f700 1 --2- 172.21.15.157:0/2146836342 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbac0b1e40 0x7fbbac152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:01.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.829+0000 7fbbb1a9e700 1 -- 172.21.15.157:0/2146836342 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbac0b1e40 msgr2=0x7fbbac152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:01.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.829+0000 7fbbb1a9e700 1 --2- 172.21.15.157:0/2146836342 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbac0b1e40 0x7fbbac152ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.829+0000 7fbbb1a9e700 1 -- 172.21.15.157:0/2146836342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbac0b04d0 msgr2=0x7fbbac166860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:01.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.829+0000 7fbbb1a9e700 1 --2- 172.21.15.157:0/2146836342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbac0b04d0 0x7fbbac166860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:01.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.829+0000 7fbbb1a9e700 1 -- 172.21.15.157:0/2146836342 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb9c0094f0 con 0x7fbbac06c2d0 2023-10-06T04:11:01.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.829+0000 7fbbb1a9e700 1 --2- 172.21.15.157:0/2146836342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbac06c2d0 0x7fbbac166320 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fbba800cb40 tx=0x7fbba8001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:01.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.829+0000 7fbba1ffb700 1 -- 172.21.15.157:0/2146836342 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba8002060 con 0x7fbbac06c2d0 2023-10-06T04:11:01.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.829+0000 7fbba1ffb700 1 -- 172.21.15.157:0/2146836342 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba800a870 con 0x7fbbac06c2d0 2023-10-06T04:11:01.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.829+0000 7fbbb2aa0700 1 -- 172.21.15.157:0/2146836342 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbac151000 con 0x7fbbac06c2d0 2023-10-06T04:11:01.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.829+0000 7fbba1ffb700 1 -- 172.21.15.157:0/2146836342 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba800ac30 con 0x7fbbac06c2d0 2023-10-06T04:11:01.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.829+0000 7fbbb2aa0700 1 -- 172.21.15.157:0/2146836342 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbac150d10 con 0x7fbbac06c2d0 2023-10-06T04:11:01.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.830+0000 7fbbb2aa0700 1 -- 172.21.15.157:0/2146836342 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbac088f90 con 0x7fbbac06c2d0 2023-10-06T04:11:01.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.831+0000 7fbba1ffb700 1 -- 172.21.15.157:0/2146836342 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbba8003030 con 0x7fbbac06c2d0 2023-10-06T04:11:01.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.831+0000 7fbba1ffb700 1 --2- 172.21.15.157:0/2146836342 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbb90066f30 0x7fbb900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:01.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.831+0000 7fbba1ffb700 1 -- 172.21.15.157:0/2146836342 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7fbba80848d0 con 0x7fbbac06c2d0 2023-10-06T04:11:01.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.831+0000 7fbbb129d700 1 --2- 172.21.15.157:0/2146836342 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbb90066f30 0x7fbb900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:01.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.832+0000 7fbbb129d700 1 --2- 172.21.15.157:0/2146836342 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbb90066f30 0x7fbb900693e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0099b0 tx=0x7fbb9c017000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:01.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:01.835+0000 7fbba1ffb700 1 -- 172.21.15.157:0/2146836342 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbba8051750 con 0x7fbbac06c2d0 2023-10-06T04:11:02.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.019+0000 7fbbb2aa0700 1 -- 172.21.15.157:0/2146836342 --> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbac06cfa0 con 0x7fbb90066f30 2023-10-06T04:11:02.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.020+0000 7fbba1ffb700 1 -- 172.21.15.157:0/2146836342 <== mgr.4107 v2:172.21.15.175:6832/90661 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbbac06cfa0 con 0x7fbb90066f30 2023-10-06T04:11:02.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.022+0000 7fbbb2aa0700 1 -- 172.21.15.157:0/2146836342 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbb90066f30 msgr2=0x7fbb900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:02.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.022+0000 7fbbb2aa0700 1 --2- 172.21.15.157:0/2146836342 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbb90066f30 0x7fbb900693e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0099b0 tx=0x7fbb9c017000 comp rx=0 tx=0).stop 2023-10-06T04:11:02.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.022+0000 7fbbb2aa0700 1 -- 172.21.15.157:0/2146836342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbac06c2d0 msgr2=0x7fbbac166320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:02.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.022+0000 7fbbb2aa0700 1 --2- 172.21.15.157:0/2146836342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbac06c2d0 0x7fbbac166320 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fbba800cb40 tx=0x7fbba8001770 comp rx=0 tx=0).stop 2023-10-06T04:11:02.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.022+0000 7fbbb2aa0700 1 -- 172.21.15.157:0/2146836342 shutdown_connections 2023-10-06T04:11:02.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.022+0000 7fbbb2aa0700 1 --2- 172.21.15.157:0/2146836342 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbb90066f30 0x7fbb900693e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.022+0000 7fbbb2aa0700 1 --2- 172.21.15.157:0/2146836342 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbac0b1e40 0x7fbbac152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.022+0000 7fbbb2aa0700 1 --2- 172.21.15.157:0/2146836342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbac06c2d0 0x7fbbac166320 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.022+0000 7fbbb2aa0700 1 --2- 172.21.15.157:0/2146836342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbac0b04d0 0x7fbbac166860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.022+0000 7fbbb2aa0700 1 -- 172.21.15.157:0/2146836342 >> 172.21.15.157:0/2146836342 conn(0x7fbbac0cc540 msgr2=0x7fbbac0c3820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:02.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.022+0000 7fbbb2aa0700 1 -- 172.21.15.157:0/2146836342 shutdown_connections 2023-10-06T04:11:02.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.022+0000 7fbbb2aa0700 1 -- 172.21.15.157:0/2146836342 wait complete. 2023-10-06T04:11:02.025 INFO:teuthology.orchestra.run.smithi157.stderr:instructed osd(s) 5 to deep-scrub 2023-10-06T04:11:02.035 INFO:tasks.ceph:Scrubbing osd.6 2023-10-06T04:11:02.035 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-06T04:11:02.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.152+0000 7f62211d4700 1 Processor -- start 2023-10-06T04:11:02.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.152+0000 7f62211d4700 1 -- start start 2023-10-06T04:11:02.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.153+0000 7f62211d4700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f621c06a860 0x7f621c0631f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:02.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.153+0000 7f62211d4700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f621c06a270 0x7f621c156870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:02.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.153+0000 7f62211d4700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f621c0c48f0 con 0x7f621c069ea0 2023-10-06T04:11:02.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.153+0000 7f62211d4700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f621c156db0 con 0x7f621c06a860 2023-10-06T04:11:02.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.153+0000 7f62211d4700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f621c0c3d20 con 0x7f621c06a270 2023-10-06T04:11:02.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.154+0000 7f621259c700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f621c06a860 0x7f621c0631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:02.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.154+0000 7f621b59e700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f621c06a270 0x7f621c156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:02.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.154+0000 7f621259c700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f621c06a860 0x7f621c0631f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:44852/0 (socket says 172.21.15.157:44852) 2023-10-06T04:11:02.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.154+0000 7f621259c700 1 -- 172.21.15.157:0/579958790 learned_addr learned my addr 172.21.15.157:0/579958790 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:11:02.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.154+0000 7f621ad9d700 1 --1- 172.21.15.157:0/579958790 >> v1:172.21.15.175:6789/0 conn(0x7f621c069ea0 0x7f621c0c4d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.157:36224/0 (socket says 172.21.15.157:36224) 2023-10-06T04:11:02.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.154+0000 7f621b59e700 1 -- 172.21.15.157:0/579958790 >> v2:172.21.15.175:3300/0 conn(0x7f621c06a860 msgr2=0x7f621c0631f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:11:02.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.154+0000 7f621b59e700 1 --2- 172.21.15.157:0/579958790 >> v2:172.21.15.175:3300/0 conn(0x7f621c06a860 0x7f621c0631f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.154+0000 7f621b59e700 1 -- 172.21.15.157:0/579958790 >> v1:172.21.15.175:6789/0 conn(0x7f621c069ea0 legacy=0x7f621c0c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:02.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.154+0000 7f621b59e700 1 -- 172.21.15.157:0/579958790 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621c063840 con 0x7f621c06a270 2023-10-06T04:11:02.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.154+0000 7f621a59c700 1 -- 172.21.15.157:0/579958790 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2199798603 0 0) 0x7f621c0c48f0 con 0x7f621c069ea0 2023-10-06T04:11:02.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.154+0000 7f621259c700 1 --2- 172.21.15.157:0/579958790 >> v2:172.21.15.175:3300/0 conn(0x7f621c06a860 0x7f621c0631f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:11:02.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.154+0000 7f621b59e700 1 --2- 172.21.15.157:0/579958790 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f621c06a270 0x7f621c156870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6214009e60 tx=0x7f6214009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff5068e0f2ed4c10 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:02.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.154+0000 7f621a59c700 1 -- 172.21.15.157:0/579958790 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621400e070 con 0x7f621c06a270 2023-10-06T04:11:02.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.154+0000 7f621a59c700 1 -- 172.21.15.157:0/579958790 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621400bbf0 con 0x7f621c06a270 2023-10-06T04:11:02.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.154+0000 7f621a59c700 1 -- 172.21.15.157:0/579958790 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6214012430 con 0x7f621c06a270 2023-10-06T04:11:02.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.155+0000 7f62211d4700 1 -- 172.21.15.157:0/579958790 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f621c06a270 msgr2=0x7f621c156870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:02.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.155+0000 7f62211d4700 1 --2- 172.21.15.157:0/579958790 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f621c06a270 0x7f621c156870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6214009e60 tx=0x7f6214009780 comp rx=0 tx=0).stop 2023-10-06T04:11:02.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.155+0000 7f62211d4700 1 -- 172.21.15.157:0/579958790 shutdown_connections 2023-10-06T04:11:02.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.155+0000 7f62211d4700 1 --2- 172.21.15.157:0/579958790 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f621c06a270 0x7f621c156870 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.155+0000 7f62211d4700 1 --2- 172.21.15.157:0/579958790 >> v2:172.21.15.175:3300/0 conn(0x7f621c06a860 0x7f621c0631f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.155+0000 7f62211d4700 1 -- 172.21.15.157:0/579958790 >> 172.21.15.157:0/579958790 conn(0x7f621c0c8b20 msgr2=0x7f621c0c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:02.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.155+0000 7f62211d4700 1 -- 172.21.15.157:0/579958790 shutdown_connections 2023-10-06T04:11:02.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.155+0000 7f62211d4700 1 -- 172.21.15.157:0/579958790 wait complete. 2023-10-06T04:11:02.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.156+0000 7f62211d4700 1 Processor -- start 2023-10-06T04:11:02.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.156+0000 7f62211d4700 1 -- start start 2023-10-06T04:11:02.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.157+0000 7f62211d4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f621c069ea0 0x7f621c140350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:02.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.157+0000 7f62211d4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f621c06a270 0x7f621c14f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:02.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.157+0000 7f62211d4700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f621c06a860 0x7f621c14fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:02.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.157+0000 7f62211d4700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c0c82c0 con 0x7f621c069ea0 2023-10-06T04:11:02.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.157+0000 7f62211d4700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c13fc60 con 0x7f621c06a270 2023-10-06T04:11:02.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.157+0000 7f62211d4700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f621c165ec0 con 0x7f621c06a860 2023-10-06T04:11:02.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.157+0000 7f621ad9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f621c069ea0 0x7f621c140350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:02.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.157+0000 7f621ad9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f621c069ea0 0x7f621c140350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51808/0 (socket says 172.21.15.157:51808) 2023-10-06T04:11:02.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.157+0000 7f621ad9d700 1 -- 172.21.15.157:0/3435469191 learned_addr learned my addr 172.21.15.157:0/3435469191 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:11:02.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.157+0000 7f621b59e700 1 --2- 172.21.15.157:0/3435469191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f621c06a860 0x7f621c14fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:02.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.157+0000 7f621259c700 1 --2- 172.21.15.157:0/3435469191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f621c06a270 0x7f621c14f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:02.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.157+0000 7f621ad9d700 1 -- 172.21.15.157:0/3435469191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f621c06a860 msgr2=0x7f621c14fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:02.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.157+0000 7f621ad9d700 1 --2- 172.21.15.157:0/3435469191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f621c06a860 0x7f621c14fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.157+0000 7f621ad9d700 1 -- 172.21.15.157:0/3435469191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f621c06a270 msgr2=0x7f621c14f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:02.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.157+0000 7f621ad9d700 1 --2- 172.21.15.157:0/3435469191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f621c06a270 0x7f621c14f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.157+0000 7f621ad9d700 1 -- 172.21.15.157:0/3435469191 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62040094f0 con 0x7f621c069ea0 2023-10-06T04:11:02.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.158+0000 7f621ad9d700 1 --2- 172.21.15.157:0/3435469191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f621c069ea0 0x7f621c140350 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f620c0011e0 tx=0x7f620c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:02.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.158+0000 7f62137fe700 1 -- 172.21.15.157:0/3435469191 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c005a70 con 0x7f621c069ea0 2023-10-06T04:11:02.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.158+0000 7f62137fe700 1 -- 172.21.15.157:0/3435469191 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f620c005bd0 con 0x7f621c069ea0 2023-10-06T04:11:02.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.158+0000 7f62211d4700 1 -- 172.21.15.157:0/3435469191 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6214009a20 con 0x7f621c069ea0 2023-10-06T04:11:02.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.158+0000 7f62137fe700 1 -- 172.21.15.157:0/3435469191 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c0070d0 con 0x7f621c069ea0 2023-10-06T04:11:02.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.158+0000 7f62211d4700 1 -- 172.21.15.157:0/3435469191 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f621c166180 con 0x7f621c069ea0 2023-10-06T04:11:02.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.158+0000 7f62211d4700 1 -- 172.21.15.157:0/3435469191 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f621c0642a0 con 0x7f621c069ea0 2023-10-06T04:11:02.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.159+0000 7f62137fe700 1 -- 172.21.15.157:0/3435469191 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f620c007480 con 0x7f621c069ea0 2023-10-06T04:11:02.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.159+0000 7f62137fe700 1 --2- 172.21.15.157:0/3435469191 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f61fc066f30 0x7f61fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:02.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.160+0000 7f62137fe700 1 -- 172.21.15.157:0/3435469191 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f620c084520 con 0x7f621c069ea0 2023-10-06T04:11:02.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.160+0000 7f62137fe700 1 --2- 172.21.15.157:0/3435469191 >> [v2:172.21.15.175:6802/91263,v1:172.21.15.175:6801/91263] conn(0x7f61fc06fd90 0x7f61fc0721c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:02.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.160+0000 7f62137fe700 1 -- 172.21.15.157:0/3435469191 --> [v2:172.21.15.175:6802/91263,v1:172.21.15.175:6801/91263] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f61fc072890 con 0x7f61fc06fd90 2023-10-06T04:11:02.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.160+0000 7f62137fe700 1 -- 172.21.15.157:0/3435469191 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f620c0848f0 con 0x7f621c069ea0 2023-10-06T04:11:02.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.160+0000 7f621259c700 1 --2- 172.21.15.157:0/3435469191 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f61fc066f30 0x7f61fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:02.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.160+0000 7f621b59e700 1 --2- 172.21.15.157:0/3435469191 >> [v2:172.21.15.175:6802/91263,v1:172.21.15.175:6801/91263] conn(0x7f61fc06fd90 0x7f61fc0721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:02.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.160+0000 7f621259c700 1 --2- 172.21.15.157:0/3435469191 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f61fc066f30 0x7f61fc0693e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f621c150600 tx=0x7f6204009250 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:02.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.160+0000 7f621b59e700 1 --2- 172.21.15.157:0/3435469191 >> [v2:172.21.15.175:6802/91263,v1:172.21.15.175:6801/91263] conn(0x7f61fc06fd90 0x7f61fc0721c0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:02.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.161+0000 7f62137fe700 1 -- 172.21.15.157:0/3435469191 <== osd.6 v2:172.21.15.175:6802/91263 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f61fc072890 con 0x7f61fc06fd90 2023-10-06T04:11:02.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.174+0000 7f62211d4700 1 -- 172.21.15.157:0/3435469191 --> [v2:172.21.15.175:6802/91263,v1:172.21.15.175:6801/91263] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f621c0c3fc0 con 0x7f61fc06fd90 2023-10-06T04:11:02.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.175+0000 7f62137fe700 1 -- 172.21.15.157:0/3435469191 <== osd.6 v2:172.21.15.175:6802/91263 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f621c0c3fc0 con 0x7f61fc06fd90 2023-10-06T04:11:02.177 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-10-06T04:11:02.177 INFO:teuthology.orchestra.run.smithi157.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T04:11:02.177 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-10-06T04:11:02.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.175+0000 7f62211d4700 1 -- 172.21.15.157:0/3435469191 >> [v2:172.21.15.175:6802/91263,v1:172.21.15.175:6801/91263] conn(0x7f61fc06fd90 msgr2=0x7f61fc0721c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:02.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.175+0000 7f62211d4700 1 --2- 172.21.15.157:0/3435469191 >> [v2:172.21.15.175:6802/91263,v1:172.21.15.175:6801/91263] conn(0x7f61fc06fd90 0x7f61fc0721c0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.176+0000 7f62211d4700 1 -- 172.21.15.157:0/3435469191 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f61fc066f30 msgr2=0x7f61fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:02.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.176+0000 7f62211d4700 1 --2- 172.21.15.157:0/3435469191 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f61fc066f30 0x7f61fc0693e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f621c150600 tx=0x7f6204009250 comp rx=0 tx=0).stop 2023-10-06T04:11:02.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.176+0000 7f62211d4700 1 -- 172.21.15.157:0/3435469191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f621c069ea0 msgr2=0x7f621c140350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:02.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.176+0000 7f62211d4700 1 --2- 172.21.15.157:0/3435469191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f621c069ea0 0x7f621c140350 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f620c0011e0 tx=0x7f620c007e10 comp rx=0 tx=0).stop 2023-10-06T04:11:02.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.176+0000 7f621b59e700 1 -- 172.21.15.157:0/3435469191 reap_dead start 2023-10-06T04:11:02.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.176+0000 7f62211d4700 1 -- 172.21.15.157:0/3435469191 shutdown_connections 2023-10-06T04:11:02.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.176+0000 7f62211d4700 1 -- 172.21.15.157:0/3435469191 >> 172.21.15.157:0/3435469191 conn(0x7f621c0c8b20 msgr2=0x7f621c1418a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:02.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.176+0000 7f62211d4700 1 -- 172.21.15.157:0/3435469191 shutdown_connections 2023-10-06T04:11:02.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.176+0000 7f62211d4700 1 -- 172.21.15.157:0/3435469191 wait complete. 2023-10-06T04:11:02.189 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-06T04:11:02.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.305+0000 7f5121618700 1 Processor -- start 2023-10-06T04:11:02.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.305+0000 7f5121618700 1 -- start start 2023-10-06T04:11:02.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.305+0000 7f5121618700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f511c06cbb0 0x7f511c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:02.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.305+0000 7f5121618700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f511c06c1f0 0x7f511c06da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:02.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.305+0000 7f5121618700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f511c0c9780 con 0x7f511c06cf80 2023-10-06T04:11:02.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.305+0000 7f5121618700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f511c0c8c40 con 0x7f511c06cbb0 2023-10-06T04:11:02.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.305+0000 7f5121618700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f511c06e0c0 con 0x7f511c06c1f0 2023-10-06T04:11:02.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.305+0000 7f511affd700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f511c06cbb0 0x7f511c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:02.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.305+0000 7f511b7fe700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f511c06cf80 0x7f511c14fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:35384/0 (socket says 172.21.15.157:35384) 2023-10-06T04:11:02.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.305+0000 7f511b7fe700 1 -- 172.21.15.157:0/104536541 learned_addr learned my addr 172.21.15.157:0/104536541 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:11:02.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.306+0000 7f511a7fc700 1 --2- 172.21.15.157:0/104536541 >> v2:172.21.15.175:3300/0 conn(0x7f511c06c1f0 0x7f511c06da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:02.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.306+0000 7f511affd700 1 -- 172.21.15.157:0/104536541 >> v2:172.21.15.175:3300/0 conn(0x7f511c06c1f0 msgr2=0x7f511c06da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:11:02.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.306+0000 7f511affd700 1 --2- 172.21.15.157:0/104536541 >> v2:172.21.15.175:3300/0 conn(0x7f511c06c1f0 0x7f511c06da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.306+0000 7f511affd700 1 -- 172.21.15.157:0/104536541 >> v1:172.21.15.157:6789/0 conn(0x7f511c06cf80 legacy=0x7f511c14fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:02.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.306+0000 7f511affd700 1 -- 172.21.15.157:0/104536541 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511c0c8e50 con 0x7f511c06cbb0 2023-10-06T04:11:02.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.306+0000 7f5119ffb700 1 -- 172.21.15.157:0/104536541 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3873760726 0 0) 0x7f511c0c9780 con 0x7f511c06cf80 2023-10-06T04:11:02.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.306+0000 7f511affd700 1 --2- 172.21.15.157:0/104536541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f511c06cbb0 0x7f511c06d530 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f511c157070 tx=0x7f510c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fcb4396f45999fa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:02.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.306+0000 7f5119ffb700 1 -- 172.21.15.157:0/104536541 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c00e070 con 0x7f511c06cbb0 2023-10-06T04:11:02.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.306+0000 7f5119ffb700 1 -- 172.21.15.157:0/104536541 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510c00bbf0 con 0x7f511c06cbb0 2023-10-06T04:11:02.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.306+0000 7f5119ffb700 1 -- 172.21.15.157:0/104536541 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c012430 con 0x7f511c06cbb0 2023-10-06T04:11:02.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.307+0000 7f5121618700 1 -- 172.21.15.157:0/104536541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f511c06cbb0 msgr2=0x7f511c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:02.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.307+0000 7f5121618700 1 --2- 172.21.15.157:0/104536541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f511c06cbb0 0x7f511c06d530 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f511c157070 tx=0x7f510c009780 comp rx=0 tx=0).stop 2023-10-06T04:11:02.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.307+0000 7f5121618700 1 -- 172.21.15.157:0/104536541 shutdown_connections 2023-10-06T04:11:02.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.307+0000 7f5121618700 1 --2- 172.21.15.157:0/104536541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f511c06cbb0 0x7f511c06d530 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.307+0000 7f5121618700 1 --2- 172.21.15.157:0/104536541 >> v2:172.21.15.175:3300/0 conn(0x7f511c06c1f0 0x7f511c06da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.307+0000 7f5121618700 1 -- 172.21.15.157:0/104536541 >> 172.21.15.157:0/104536541 conn(0x7f511c06f230 msgr2=0x7f511c06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:02.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.307+0000 7f5121618700 1 -- 172.21.15.157:0/104536541 shutdown_connections 2023-10-06T04:11:02.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.307+0000 7f5121618700 1 -- 172.21.15.157:0/104536541 wait complete. 2023-10-06T04:11:02.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.308+0000 7f5121618700 1 Processor -- start 2023-10-06T04:11:02.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.308+0000 7f5121618700 1 -- start start 2023-10-06T04:11:02.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.309+0000 7f5121618700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f511c06c1f0 0x7f511c0bfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:02.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.309+0000 7f5121618700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f511c06cbb0 0x7f511c0c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:02.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.309+0000 7f5121618700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f511c06cf80 0x7f511c147ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:02.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.309+0000 7f5121618700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c0c01b0 con 0x7f511c06c1f0 2023-10-06T04:11:02.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.309+0000 7f5121618700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c1667d0 con 0x7f511c06cf80 2023-10-06T04:11:02.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.309+0000 7f5121618700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f511c0c03b0 con 0x7f511c06cbb0 2023-10-06T04:11:02.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.309+0000 7f511affd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f511c06c1f0 0x7f511c0bfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:02.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.309+0000 7f511affd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f511c06c1f0 0x7f511c0bfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51834/0 (socket says 172.21.15.157:51834) 2023-10-06T04:11:02.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.309+0000 7f511affd700 1 -- 172.21.15.157:0/2598084491 learned_addr learned my addr 172.21.15.157:0/2598084491 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:11:02.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.309+0000 7f511b7fe700 1 --2- 172.21.15.157:0/2598084491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f511c06cf80 0x7f511c147ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:02.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.309+0000 7f511a7fc700 1 --2- 172.21.15.157:0/2598084491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f511c06cbb0 0x7f511c0c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:02.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.309+0000 7f511affd700 1 -- 172.21.15.157:0/2598084491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f511c06cbb0 msgr2=0x7f511c0c2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:02.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.309+0000 7f511affd700 1 --2- 172.21.15.157:0/2598084491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f511c06cbb0 0x7f511c0c2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.309+0000 7f511affd700 1 -- 172.21.15.157:0/2598084491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f511c06cf80 msgr2=0x7f511c147ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:02.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.309+0000 7f511affd700 1 --2- 172.21.15.157:0/2598084491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f511c06cf80 0x7f511c147ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.309+0000 7f511affd700 1 -- 172.21.15.157:0/2598084491 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f510c009a20 con 0x7f511c06c1f0 2023-10-06T04:11:02.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.309+0000 7f511b7fe700 1 --2- 172.21.15.157:0/2598084491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f511c06cf80 0x7f511c147ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:11:02.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.309+0000 7f511affd700 1 --2- 172.21.15.157:0/2598084491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f511c06c1f0 0x7f511c0bfc70 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f511c0c3960 tx=0x7f510c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:02.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.309+0000 7f5102ffd700 1 -- 172.21.15.157:0/2598084491 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c012ea0 con 0x7f511c06c1f0 2023-10-06T04:11:02.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.309+0000 7f5102ffd700 1 -- 172.21.15.157:0/2598084491 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510c027470 con 0x7f511c06c1f0 2023-10-06T04:11:02.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.309+0000 7f5121618700 1 -- 172.21.15.157:0/2598084491 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f511c155420 con 0x7f511c06c1f0 2023-10-06T04:11:02.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.309+0000 7f5102ffd700 1 -- 172.21.15.157:0/2598084491 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c027810 con 0x7f511c06c1f0 2023-10-06T04:11:02.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.309+0000 7f5121618700 1 -- 172.21.15.157:0/2598084491 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f511c0c32e0 con 0x7f511c06c1f0 2023-10-06T04:11:02.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.310+0000 7f5121618700 1 -- 172.21.15.157:0/2598084491 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f511c089ca0 con 0x7f511c06c1f0 2023-10-06T04:11:02.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.311+0000 7f5102ffd700 1 -- 172.21.15.157:0/2598084491 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f510c024070 con 0x7f511c06c1f0 2023-10-06T04:11:02.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.311+0000 7f5102ffd700 1 --2- 172.21.15.157:0/2598084491 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5108066f30 0x7f51080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:02.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.311+0000 7f5102ffd700 1 -- 172.21.15.157:0/2598084491 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f510c0870c0 con 0x7f511c06c1f0 2023-10-06T04:11:02.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.311+0000 7f511a7fc700 1 --2- 172.21.15.157:0/2598084491 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5108066f30 0x7f51080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:02.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.312+0000 7f511a7fc700 1 --2- 172.21.15.157:0/2598084491 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5108066f30 0x7f51080693e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f511c0aef00 tx=0x7f5110008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:02.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.315+0000 7f5102ffd700 1 -- 172.21.15.157:0/2598084491 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f510c053f40 con 0x7f511c06c1f0 2023-10-06T04:11:02.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.505+0000 7f5121618700 1 -- 172.21.15.157:0/2598084491 --> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f511c0c9130 con 0x7f5108066f30 2023-10-06T04:11:02.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.505+0000 7f5102ffd700 1 -- 172.21.15.157:0/2598084491 <== mgr.4107 v2:172.21.15.175:6832/90661 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f511c0c9130 con 0x7f5108066f30 2023-10-06T04:11:02.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.507+0000 7f5121618700 1 -- 172.21.15.157:0/2598084491 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5108066f30 msgr2=0x7f51080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:02.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.507+0000 7f5121618700 1 --2- 172.21.15.157:0/2598084491 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5108066f30 0x7f51080693e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f511c0aef00 tx=0x7f5110008040 comp rx=0 tx=0).stop 2023-10-06T04:11:02.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.507+0000 7f5121618700 1 -- 172.21.15.157:0/2598084491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f511c06c1f0 msgr2=0x7f511c0bfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:02.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.507+0000 7f5121618700 1 --2- 172.21.15.157:0/2598084491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f511c06c1f0 0x7f511c0bfc70 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f511c0c3960 tx=0x7f510c019000 comp rx=0 tx=0).stop 2023-10-06T04:11:02.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.507+0000 7f5121618700 1 -- 172.21.15.157:0/2598084491 shutdown_connections 2023-10-06T04:11:02.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.507+0000 7f5121618700 1 --2- 172.21.15.157:0/2598084491 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5108066f30 0x7f51080693e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.507+0000 7f5121618700 1 --2- 172.21.15.157:0/2598084491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f511c06cf80 0x7f511c147ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.507+0000 7f5121618700 1 --2- 172.21.15.157:0/2598084491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f511c06c1f0 0x7f511c0bfc70 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.507+0000 7f5121618700 1 --2- 172.21.15.157:0/2598084491 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f511c06cbb0 0x7f511c0c2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.507+0000 7f5121618700 1 -- 172.21.15.157:0/2598084491 >> 172.21.15.157:0/2598084491 conn(0x7f511c06f230 msgr2=0x7f511c06d530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:02.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.507+0000 7f5121618700 1 -- 172.21.15.157:0/2598084491 shutdown_connections 2023-10-06T04:11:02.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.507+0000 7f5121618700 1 -- 172.21.15.157:0/2598084491 wait complete. 2023-10-06T04:11:02.511 INFO:teuthology.orchestra.run.smithi157.stderr:instructed osd(s) 6 to deep-scrub 2023-10-06T04:11:02.521 INFO:tasks.ceph:Scrubbing osd.7 2023-10-06T04:11:02.521 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-06T04:11:02.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.641+0000 7fbfa11e5700 1 Processor -- start 2023-10-06T04:11:02.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.641+0000 7fbfa11e5700 1 -- start start 2023-10-06T04:11:02.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.642+0000 7fbfa11e5700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf9c0caa00 0x7fbf9c153e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:02.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.642+0000 7fbfa11e5700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbf9c0c3e50 0x7fbf9c0c42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:02.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.642+0000 7fbfa11e5700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf9c0c39c0 con 0x7fbf9c0ca630 2023-10-06T04:11:02.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.642+0000 7fbfa11e5700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf9c154340 con 0x7fbf9c0c3e50 2023-10-06T04:11:02.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.642+0000 7fbfa11e5700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c068b80 con 0x7fbf9c0caa00 2023-10-06T04:11:02.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.642+0000 7fbf9b59e700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbf9c0c3e50 0x7fbf9c0c42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:02.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.642+0000 7fbf9b59e700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbf9c0c3e50 0x7fbf9c0c42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51848/0 (socket says 172.21.15.157:51848) 2023-10-06T04:11:02.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.642+0000 7fbf9b59e700 1 -- 172.21.15.157:0/313120729 learned_addr learned my addr 172.21.15.157:0/313120729 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:11:02.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.642+0000 7fbf9a59c700 1 --2- 172.21.15.157:0/313120729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf9c0caa00 0x7fbf9c153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:02.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.642+0000 7fbf9b59e700 1 -- 172.21.15.157:0/313120729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf9c0caa00 msgr2=0x7fbf9c153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:11:02.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.642+0000 7fbf9b59e700 1 --2- 172.21.15.157:0/313120729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf9c0caa00 0x7fbf9c153e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.642+0000 7fbf9b59e700 1 -- 172.21.15.157:0/313120729 >> v1:172.21.15.175:6789/0 conn(0x7fbf9c0ca630 legacy=0x7fbf9c0c18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:11:02.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.642+0000 7fbf9b59e700 1 -- 172.21.15.157:0/313120729 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9c0c47f0 con 0x7fbf9c0c3e50 2023-10-06T04:11:02.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.642+0000 7fbf9a59c700 1 --2- 172.21.15.157:0/313120729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf9c0caa00 0x7fbf9c153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:11:02.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.643+0000 7fbf9b59e700 1 --2- 172.21.15.157:0/313120729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf9c0c3e50 0x7fbf9c0c42b0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009f20 tx=0x7fbf94009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6104e1570aba53e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:02.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.643+0000 7fbf99d9b700 1 -- 172.21.15.157:0/313120729 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9400e070 con 0x7fbf9c0c3e50 2023-10-06T04:11:02.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.643+0000 7fbf99d9b700 1 -- 172.21.15.157:0/313120729 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf9400bad0 con 0x7fbf9c0c3e50 2023-10-06T04:11:02.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.643+0000 7fbf99d9b700 1 -- 172.21.15.157:0/313120729 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf940092e0 con 0x7fbf9c0c3e50 2023-10-06T04:11:02.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.643+0000 7fbfa11e5700 1 -- 172.21.15.157:0/313120729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf9c0c3e50 msgr2=0x7fbf9c0c42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:02.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.643+0000 7fbfa11e5700 1 --2- 172.21.15.157:0/313120729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf9c0c3e50 0x7fbf9c0c42b0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009f20 tx=0x7fbf94009840 comp rx=0 tx=0).stop 2023-10-06T04:11:02.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.643+0000 7fbfa11e5700 1 -- 172.21.15.157:0/313120729 shutdown_connections 2023-10-06T04:11:02.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.643+0000 7fbfa11e5700 1 --2- 172.21.15.157:0/313120729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf9c0c3e50 0x7fbf9c0c42b0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.643+0000 7fbfa11e5700 1 --2- 172.21.15.157:0/313120729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf9c0caa00 0x7fbf9c153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.643+0000 7fbfa11e5700 1 -- 172.21.15.157:0/313120729 >> 172.21.15.157:0/313120729 conn(0x7fbf9c0692b0 msgr2=0x7fbf9c0696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:02.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.643+0000 7fbfa11e5700 1 -- 172.21.15.157:0/313120729 shutdown_connections 2023-10-06T04:11:02.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.643+0000 7fbfa11e5700 1 -- 172.21.15.157:0/313120729 wait complete. 2023-10-06T04:11:02.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.645+0000 7fbfa11e5700 1 Processor -- start 2023-10-06T04:11:02.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.645+0000 7fbfa11e5700 1 -- start start 2023-10-06T04:11:02.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.645+0000 7fbfa11e5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf9c0c3e50 0x7fbf9c163d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:02.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.645+0000 7fbfa11e5700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf9c0ca630 0x7fbf9c161a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:02.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.645+0000 7fbfa11e5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf9c0caa00 0x7fbf9c1467d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:02.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.645+0000 7fbfa11e5700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c0c6620 con 0x7fbf9c0caa00 2023-10-06T04:11:02.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.645+0000 7fbfa11e5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c0c5d10 con 0x7fbf9c0c3e50 2023-10-06T04:11:02.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.645+0000 7fbfa11e5700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c1646b0 con 0x7fbf9c0ca630 2023-10-06T04:11:02.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.645+0000 7fbf9b59e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf9c0caa00 0x7fbf9c1467d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:02.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.645+0000 7fbf9b59e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf9c0caa00 0x7fbf9c1467d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51850/0 (socket says 172.21.15.157:51850) 2023-10-06T04:11:02.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.645+0000 7fbf9b59e700 1 -- 172.21.15.157:0/3792317539 learned_addr learned my addr 172.21.15.157:0/3792317539 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:11:02.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.645+0000 7fbf9a59c700 1 --2- 172.21.15.157:0/3792317539 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf9c0ca630 0x7fbf9c161a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:02.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.645+0000 7fbf9ad9d700 1 --2- 172.21.15.157:0/3792317539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf9c0c3e50 0x7fbf9c163d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:02.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.646+0000 7fbf9b59e700 1 -- 172.21.15.157:0/3792317539 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf9c0ca630 msgr2=0x7fbf9c161a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:02.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.646+0000 7fbf9b59e700 1 --2- 172.21.15.157:0/3792317539 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf9c0ca630 0x7fbf9c161a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.646+0000 7fbf9b59e700 1 -- 172.21.15.157:0/3792317539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf9c0c3e50 msgr2=0x7fbf9c163d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:02.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.646+0000 7fbf9b59e700 1 --2- 172.21.15.157:0/3792317539 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf9c0c3e50 0x7fbf9c163d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.646+0000 7fbf9b59e700 1 -- 172.21.15.157:0/3792317539 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf94009ae0 con 0x7fbf9c0caa00 2023-10-06T04:11:02.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.646+0000 7fbf9b59e700 1 --2- 172.21.15.157:0/3792317539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf9c0caa00 0x7fbf9c1467d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c069e60 tx=0x7fbf94017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:02.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.646+0000 7fbf82ffd700 1 -- 172.21.15.157:0/3792317539 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94012c80 con 0x7fbf9c0caa00 2023-10-06T04:11:02.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.646+0000 7fbf82ffd700 1 -- 172.21.15.157:0/3792317539 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf94012de0 con 0x7fbf9c0caa00 2023-10-06T04:11:02.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.646+0000 7fbfa11e5700 1 -- 172.21.15.157:0/3792317539 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf9c0c5fd0 con 0x7fbf9c0caa00 2023-10-06T04:11:02.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.646+0000 7fbf82ffd700 1 -- 172.21.15.157:0/3792317539 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9401d670 con 0x7fbf9c0caa00 2023-10-06T04:11:02.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.646+0000 7fbfa11e5700 1 -- 172.21.15.157:0/3792317539 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf9c0c6ab0 con 0x7fbf9c0caa00 2023-10-06T04:11:02.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.647+0000 7fbfa11e5700 1 -- 172.21.15.157:0/3792317539 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbf9c06a8f0 con 0x7fbf9c0caa00 2023-10-06T04:11:02.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.647+0000 7fbf82ffd700 1 -- 172.21.15.157:0/3792317539 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbf9401da60 con 0x7fbf9c0caa00 2023-10-06T04:11:02.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.648+0000 7fbf82ffd700 1 --2- 172.21.15.157:0/3792317539 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbf88066f60 0x7fbf88069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:02.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.648+0000 7fbf82ffd700 1 -- 172.21.15.157:0/3792317539 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7fbf94086220 con 0x7fbf9c0caa00 2023-10-06T04:11:02.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.648+0000 7fbf82ffd700 1 --2- 172.21.15.157:0/3792317539 >> [v2:172.21.15.175:6824/91306,v1:172.21.15.175:6825/91306] conn(0x7fbf8806fdc0 0x7fbf880721f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:02.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.648+0000 7fbf82ffd700 1 -- 172.21.15.157:0/3792317539 --> [v2:172.21.15.175:6824/91306,v1:172.21.15.175:6825/91306] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbf880728c0 con 0x7fbf8806fdc0 2023-10-06T04:11:02.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.648+0000 7fbf82ffd700 1 -- 172.21.15.157:0/3792317539 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf94088860 con 0x7fbf9c0caa00 2023-10-06T04:11:02.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.648+0000 7fbf9ad9d700 1 --2- 172.21.15.157:0/3792317539 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbf88066f60 0x7fbf88069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:02.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.648+0000 7fbf9a59c700 1 --2- 172.21.15.157:0/3792317539 >> [v2:172.21.15.175:6824/91306,v1:172.21.15.175:6825/91306] conn(0x7fbf8806fdc0 0x7fbf880721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:02.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.648+0000 7fbf9ad9d700 1 --2- 172.21.15.157:0/3792317539 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbf88066f60 0x7fbf88069410 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c003a30 tx=0x7fbf8c007ea0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:02.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.648+0000 7fbf9a59c700 1 --2- 172.21.15.157:0/3792317539 >> [v2:172.21.15.175:6824/91306,v1:172.21.15.175:6825/91306] conn(0x7fbf8806fdc0 0x7fbf880721f0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:02.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.649+0000 7fbf82ffd700 1 -- 172.21.15.157:0/3792317539 <== osd.7 v2:172.21.15.175:6824/91306 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fbf880728c0 con 0x7fbf8806fdc0 2023-10-06T04:11:02.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.663+0000 7fbfa11e5700 1 -- 172.21.15.157:0/3792317539 --> [v2:172.21.15.175:6824/91306,v1:172.21.15.175:6825/91306] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbf9c0c6470 con 0x7fbf8806fdc0 2023-10-06T04:11:02.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.663+0000 7fbf82ffd700 1 -- 172.21.15.157:0/3792317539 <== osd.7 v2:172.21.15.175:6824/91306 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fbf9c0c6470 con 0x7fbf8806fdc0 2023-10-06T04:11:02.665 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-10-06T04:11:02.665 INFO:teuthology.orchestra.run.smithi157.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T04:11:02.665 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-10-06T04:11:02.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.664+0000 7fbfa11e5700 1 -- 172.21.15.157:0/3792317539 >> [v2:172.21.15.175:6824/91306,v1:172.21.15.175:6825/91306] conn(0x7fbf8806fdc0 msgr2=0x7fbf880721f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:02.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.664+0000 7fbfa11e5700 1 --2- 172.21.15.157:0/3792317539 >> [v2:172.21.15.175:6824/91306,v1:172.21.15.175:6825/91306] conn(0x7fbf8806fdc0 0x7fbf880721f0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.664+0000 7fbfa11e5700 1 -- 172.21.15.157:0/3792317539 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbf88066f60 msgr2=0x7fbf88069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:02.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.664+0000 7fbfa11e5700 1 --2- 172.21.15.157:0/3792317539 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fbf88066f60 0x7fbf88069410 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c003a30 tx=0x7fbf8c007ea0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.664+0000 7fbfa11e5700 1 -- 172.21.15.157:0/3792317539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf9c0caa00 msgr2=0x7fbf9c1467d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:02.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.664+0000 7fbfa11e5700 1 --2- 172.21.15.157:0/3792317539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf9c0caa00 0x7fbf9c1467d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c069e60 tx=0x7fbf94017000 comp rx=0 tx=0).stop 2023-10-06T04:11:02.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.664+0000 7fbf9b59e700 1 -- 172.21.15.157:0/3792317539 reap_dead start 2023-10-06T04:11:02.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.664+0000 7fbfa11e5700 1 -- 172.21.15.157:0/3792317539 shutdown_connections 2023-10-06T04:11:02.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.664+0000 7fbfa11e5700 1 -- 172.21.15.157:0/3792317539 >> 172.21.15.157:0/3792317539 conn(0x7fbf9c0692b0 msgr2=0x7fbf9c0c9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:02.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.664+0000 7fbfa11e5700 1 -- 172.21.15.157:0/3792317539 shutdown_connections 2023-10-06T04:11:02.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.664+0000 7fbfa11e5700 1 -- 172.21.15.157:0/3792317539 wait complete. 2023-10-06T04:11:02.677 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-06T04:11:02.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.791+0000 7f9ebdc31700 1 Processor -- start 2023-10-06T04:11:02.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.791+0000 7f9ebdc31700 1 -- start start 2023-10-06T04:11:02.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.792+0000 7f9ebdc31700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb806cbb0 0x7f9eb806d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:02.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.792+0000 7f9ebdc31700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9eb806c1f0 0x7f9eb806da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:02.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.792+0000 7f9ebdc31700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9eb806cf80 0x7f9eb814fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:02.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.792+0000 7f9ebdc31700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eb80c8c40 con 0x7f9eb806cf80 2023-10-06T04:11:02.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.792+0000 7f9ebdc31700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eb80c9780 con 0x7f9eb806c1f0 2023-10-06T04:11:02.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.792+0000 7f9ebdc31700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb806e0c0 con 0x7f9eb806cbb0 2023-10-06T04:11:02.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.792+0000 7f9eb7fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9eb806cf80 0x7f9eb814fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:02.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.792+0000 7f9eb7fff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9eb806cf80 0x7f9eb814fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51856/0 (socket says 172.21.15.157:51856) 2023-10-06T04:11:02.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.792+0000 7f9eb7fff700 1 -- 172.21.15.157:0/2619945781 learned_addr learned my addr 172.21.15.157:0/2619945781 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:11:02.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.792+0000 7f9eb6ffd700 1 --2- 172.21.15.157:0/2619945781 >> v2:172.21.15.175:3300/0 conn(0x7f9eb806c1f0 0x7f9eb806da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:02.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.792+0000 7f9eb77fe700 1 --2- 172.21.15.157:0/2619945781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb806cbb0 0x7f9eb806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:02.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.792+0000 7f9eb7fff700 1 -- 172.21.15.157:0/2619945781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb806cbb0 msgr2=0x7f9eb806d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:11:02.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.792+0000 7f9eb7fff700 1 --2- 172.21.15.157:0/2619945781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb806cbb0 0x7f9eb806d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.793+0000 7f9eb7fff700 1 -- 172.21.15.157:0/2619945781 >> v2:172.21.15.175:3300/0 conn(0x7f9eb806c1f0 msgr2=0x7f9eb806da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:11:02.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.793+0000 7f9eb7fff700 1 --2- 172.21.15.157:0/2619945781 >> v2:172.21.15.175:3300/0 conn(0x7f9eb806c1f0 0x7f9eb806da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.793+0000 7f9eb7fff700 1 -- 172.21.15.157:0/2619945781 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb80c8e50 con 0x7f9eb806cf80 2023-10-06T04:11:02.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.793+0000 7f9eb6ffd700 1 --2- 172.21.15.157:0/2619945781 >> v2:172.21.15.175:3300/0 conn(0x7f9eb806c1f0 0x7f9eb806da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:11:02.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.793+0000 7f9eb7fff700 1 --2- 172.21.15.157:0/2619945781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb806cf80 0x7f9eb814fd00 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f9eb00099b0 tx=0x7f9eb0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a75944270e4080 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:02.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.793+0000 7f9eb67fc700 1 -- 172.21.15.157:0/2619945781 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb000b8b0 con 0x7f9eb806cf80 2023-10-06T04:11:02.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.793+0000 7f9eb67fc700 1 -- 172.21.15.157:0/2619945781 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb000ba10 con 0x7f9eb806cf80 2023-10-06T04:11:02.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.793+0000 7f9eb67fc700 1 -- 172.21.15.157:0/2619945781 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb000bdb0 con 0x7f9eb806cf80 2023-10-06T04:11:02.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.793+0000 7f9ebdc31700 1 -- 172.21.15.157:0/2619945781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb806cf80 msgr2=0x7f9eb814fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:02.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.793+0000 7f9ebdc31700 1 --2- 172.21.15.157:0/2619945781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb806cf80 0x7f9eb814fd00 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f9eb00099b0 tx=0x7f9eb0009250 comp rx=0 tx=0).stop 2023-10-06T04:11:02.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.793+0000 7f9ebdc31700 1 -- 172.21.15.157:0/2619945781 shutdown_connections 2023-10-06T04:11:02.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.793+0000 7f9ebdc31700 1 --2- 172.21.15.157:0/2619945781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb806cf80 0x7f9eb814fd00 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.793+0000 7f9ebdc31700 1 --2- 172.21.15.157:0/2619945781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb806cbb0 0x7f9eb806d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.793+0000 7f9ebdc31700 1 --2- 172.21.15.157:0/2619945781 >> v2:172.21.15.175:3300/0 conn(0x7f9eb806c1f0 0x7f9eb806da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.793+0000 7f9ebdc31700 1 -- 172.21.15.157:0/2619945781 >> 172.21.15.157:0/2619945781 conn(0x7f9eb806f230 msgr2=0x7f9eb806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:02.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.793+0000 7f9ebdc31700 1 -- 172.21.15.157:0/2619945781 shutdown_connections 2023-10-06T04:11:02.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.793+0000 7f9ebdc31700 1 -- 172.21.15.157:0/2619945781 wait complete. 2023-10-06T04:11:02.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.795+0000 7f9ebdc31700 1 Processor -- start 2023-10-06T04:11:02.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.795+0000 7f9ebdc31700 1 -- start start 2023-10-06T04:11:02.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.795+0000 7f9ebdc31700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb806c1f0 0x7f9eb814fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:02.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.795+0000 7f9ebdc31700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb806cbb0 0x7f9eb8166cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:02.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.795+0000 7f9ebdc31700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eb806cf80 0x7f9eb8146640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:02.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.795+0000 7f9ebdc31700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb80c68f0 con 0x7f9eb806cbb0 2023-10-06T04:11:02.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.795+0000 7f9ebdc31700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb8150670 con 0x7f9eb806cf80 2023-10-06T04:11:02.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.795+0000 7f9ebdc31700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb81513d0 con 0x7f9eb806c1f0 2023-10-06T04:11:02.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.795+0000 7f9eb6ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb806cbb0 0x7f9eb8166cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:02.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.795+0000 7f9eb6ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb806cbb0 0x7f9eb8166cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51870/0 (socket says 172.21.15.157:51870) 2023-10-06T04:11:02.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.795+0000 7f9eb6ffd700 1 -- 172.21.15.157:0/1448186567 learned_addr learned my addr 172.21.15.157:0/1448186567 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:11:02.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.796+0000 7f9eb7fff700 1 --2- 172.21.15.157:0/1448186567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eb806cf80 0x7f9eb8146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:02.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.796+0000 7f9eb77fe700 1 --2- 172.21.15.157:0/1448186567 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb806c1f0 0x7f9eb814fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:02.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.796+0000 7f9eb6ffd700 1 -- 172.21.15.157:0/1448186567 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb806c1f0 msgr2=0x7f9eb814fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:02.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.796+0000 7f9eb6ffd700 1 --2- 172.21.15.157:0/1448186567 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb806c1f0 0x7f9eb814fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.796+0000 7f9eb6ffd700 1 -- 172.21.15.157:0/1448186567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eb806cf80 msgr2=0x7f9eb8146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:02.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.796+0000 7f9eb6ffd700 1 --2- 172.21.15.157:0/1448186567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eb806cf80 0x7f9eb8146640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:02.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.796+0000 7f9eb6ffd700 1 -- 172.21.15.157:0/1448186567 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb00094f0 con 0x7f9eb806cbb0 2023-10-06T04:11:02.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.796+0000 7f9eb7fff700 1 --2- 172.21.15.157:0/1448186567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eb806cf80 0x7f9eb8146640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:11:02.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.796+0000 7f9eb6ffd700 1 --2- 172.21.15.157:0/1448186567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb806cbb0 0x7f9eb8166cf0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f9eac00e8d0 tx=0x7f9eac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:02.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.796+0000 7f9e9f7fe700 1 -- 172.21.15.157:0/1448186567 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac00ee70 con 0x7f9eb806cbb0 2023-10-06T04:11:02.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.796+0000 7f9e9f7fe700 1 -- 172.21.15.157:0/1448186567 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eac00c5e0 con 0x7f9eb806cbb0 2023-10-06T04:11:02.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.796+0000 7f9ebdc31700 1 -- 172.21.15.157:0/1448186567 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb8167230 con 0x7f9eb806cbb0 2023-10-06T04:11:02.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.796+0000 7f9e9f7fe700 1 -- 172.21.15.157:0/1448186567 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac00c980 con 0x7f9eb806cbb0 2023-10-06T04:11:02.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.796+0000 7f9ebdc31700 1 -- 172.21.15.157:0/1448186567 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb8150aa0 con 0x7f9eb806cbb0 2023-10-06T04:11:02.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.797+0000 7f9ebdc31700 1 -- 172.21.15.157:0/1448186567 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb8089ca0 con 0x7f9eb806cbb0 2023-10-06T04:11:02.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.797+0000 7f9e9f7fe700 1 -- 172.21.15.157:0/1448186567 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9eac00cd30 con 0x7f9eb806cbb0 2023-10-06T04:11:02.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.798+0000 7f9e9f7fe700 1 --2- 172.21.15.157:0/1448186567 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9ea4066f30 0x7f9ea40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:02.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.798+0000 7f9e9f7fe700 1 -- 172.21.15.157:0/1448186567 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f9eac084770 con 0x7f9eb806cbb0 2023-10-06T04:11:02.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.798+0000 7f9eb77fe700 1 --2- 172.21.15.157:0/1448186567 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9ea4066f30 0x7f9ea40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:02.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.798+0000 7f9eb77fe700 1 --2- 172.21.15.157:0/1448186567 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9ea4066f30 0x7f9ea40693e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8007460 tx=0x7f9ea800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:02.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.802+0000 7f9e9f7fe700 1 -- 172.21.15.157:0/1448186567 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9eac0515f0 con 0x7f9eb806cbb0 2023-10-06T04:11:02.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.997+0000 7f9ebdc31700 1 -- 172.21.15.157:0/1448186567 --> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f9eb80c9130 con 0x7f9ea4066f30 2023-10-06T04:11:02.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.997+0000 7f9e9f7fe700 1 -- 172.21.15.157:0/1448186567 <== mgr.4107 v2:172.21.15.175:6832/90661 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9eb80c9130 con 0x7f9ea4066f30 2023-10-06T04:11:03.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.999+0000 7f9ebdc31700 1 -- 172.21.15.157:0/1448186567 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9ea4066f30 msgr2=0x7f9ea40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:03.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.999+0000 7f9ebdc31700 1 --2- 172.21.15.157:0/1448186567 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9ea4066f30 0x7f9ea40693e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8007460 tx=0x7f9ea800b040 comp rx=0 tx=0).stop 2023-10-06T04:11:03.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.999+0000 7f9ebdc31700 1 -- 172.21.15.157:0/1448186567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb806cbb0 msgr2=0x7f9eb8166cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:03.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.999+0000 7f9ebdc31700 1 --2- 172.21.15.157:0/1448186567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb806cbb0 0x7f9eb8166cf0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f9eac00e8d0 tx=0x7f9eac008040 comp rx=0 tx=0).stop 2023-10-06T04:11:03.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.999+0000 7f9ebdc31700 1 -- 172.21.15.157:0/1448186567 shutdown_connections 2023-10-06T04:11:03.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.999+0000 7f9ebdc31700 1 --2- 172.21.15.157:0/1448186567 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f9ea4066f30 0x7f9ea40693e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:03.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.999+0000 7f9ebdc31700 1 --2- 172.21.15.157:0/1448186567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9eb806cf80 0x7f9eb8146640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:03.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.999+0000 7f9ebdc31700 1 --2- 172.21.15.157:0/1448186567 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9eb806c1f0 0x7f9eb814fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:03.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.999+0000 7f9ebdc31700 1 --2- 172.21.15.157:0/1448186567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9eb806cbb0 0x7f9eb8166cf0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:03.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:02.999+0000 7f9ebdc31700 1 -- 172.21.15.157:0/1448186567 >> 172.21.15.157:0/1448186567 conn(0x7f9eb806f230 msgr2=0x7f9eb814c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:03.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.000+0000 7f9ebdc31700 1 -- 172.21.15.157:0/1448186567 shutdown_connections 2023-10-06T04:11:03.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.000+0000 7f9ebdc31700 1 -- 172.21.15.157:0/1448186567 wait complete. 2023-10-06T04:11:03.003 INFO:teuthology.orchestra.run.smithi157.stderr:instructed osd(s) 7 to deep-scrub 2023-10-06T04:11:03.013 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:11:03.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.126+0000 7f521adc4700 1 Processor -- start 2023-10-06T04:11:03.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.126+0000 7f521adc4700 1 -- start start 2023-10-06T04:11:03.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.127+0000 7f521adc4700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f52140b1e40 0x7f5214157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:03.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.127+0000 7f521adc4700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52140b04d0 0x7f521406dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:03.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.127+0000 7f521adc4700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f521406c2d0 0x7f52141563b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:03.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.127+0000 7f521adc4700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f521406d8d0 con 0x7f521406c2d0 2023-10-06T04:11:03.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.127+0000 7f521adc4700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52140af8f0 con 0x7f52140b1e40 2023-10-06T04:11:03.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.127+0000 7f521adc4700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f521406ccc0 con 0x7f52140b04d0 2023-10-06T04:11:03.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.127+0000 7f521a5c3700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f521406c2d0 0x7f52141563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:03.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.127+0000 7f521a5c3700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f521406c2d0 0x7f52141563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51884/0 (socket says 172.21.15.157:51884) 2023-10-06T04:11:03.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.127+0000 7f521a5c3700 1 -- 172.21.15.157:0/3125671297 learned_addr learned my addr 172.21.15.157:0/3125671297 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:11:03.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.127+0000 7f52195c1700 1 --2- 172.21.15.157:0/3125671297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52140b04d0 0x7f521406dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:03.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.127+0000 7f5219dc2700 1 --2- 172.21.15.157:0/3125671297 >> v2:172.21.15.175:3300/0 conn(0x7f52140b1e40 0x7f5214157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:03.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.127+0000 7f521a5c3700 1 -- 172.21.15.157:0/3125671297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52140b04d0 msgr2=0x7f521406dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:11:03.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.127+0000 7f521a5c3700 1 --2- 172.21.15.157:0/3125671297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52140b04d0 0x7f521406dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:03.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.127+0000 7f521a5c3700 1 -- 172.21.15.157:0/3125671297 >> v2:172.21.15.175:3300/0 conn(0x7f52140b1e40 msgr2=0x7f5214157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:11:03.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.127+0000 7f521a5c3700 1 --2- 172.21.15.157:0/3125671297 >> v2:172.21.15.175:3300/0 conn(0x7f52140b1e40 0x7f5214157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:03.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.127+0000 7f521a5c3700 1 -- 172.21.15.157:0/3125671297 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521406cfa0 con 0x7f521406c2d0 2023-10-06T04:11:03.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.127+0000 7f5219dc2700 1 --2- 172.21.15.157:0/3125671297 >> v2:172.21.15.175:3300/0 conn(0x7f52140b1e40 0x7f5214157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:11:03.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.127+0000 7f521a5c3700 1 --2- 172.21.15.157:0/3125671297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f521406c2d0 0x7f52141563b0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f52140cc450 tx=0x7f5210009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a234a2b7d27ff20 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:03.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.128+0000 7f5218dc0700 1 -- 172.21.15.157:0/3125671297 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521000e070 con 0x7f521406c2d0 2023-10-06T04:11:03.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.128+0000 7f5218dc0700 1 -- 172.21.15.157:0/3125671297 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521000ba50 con 0x7f521406c2d0 2023-10-06T04:11:03.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.128+0000 7f5218dc0700 1 -- 172.21.15.157:0/3125671297 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52100092e0 con 0x7f521406c2d0 2023-10-06T04:11:03.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.128+0000 7f521adc4700 1 -- 172.21.15.157:0/3125671297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f521406c2d0 msgr2=0x7f52141563b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:03.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.128+0000 7f521adc4700 1 --2- 172.21.15.157:0/3125671297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f521406c2d0 0x7f52141563b0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f52140cc450 tx=0x7f5210009840 comp rx=0 tx=0).stop 2023-10-06T04:11:03.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.128+0000 7f521adc4700 1 -- 172.21.15.157:0/3125671297 shutdown_connections 2023-10-06T04:11:03.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.128+0000 7f521adc4700 1 --2- 172.21.15.157:0/3125671297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f521406c2d0 0x7f52141563b0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:03.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.128+0000 7f521adc4700 1 --2- 172.21.15.157:0/3125671297 >> v2:172.21.15.175:3300/0 conn(0x7f52140b1e40 0x7f5214157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:03.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.128+0000 7f521adc4700 1 --2- 172.21.15.157:0/3125671297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52140b04d0 0x7f521406dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:03.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.128+0000 7f521adc4700 1 -- 172.21.15.157:0/3125671297 >> 172.21.15.157:0/3125671297 conn(0x7f52140cc540 msgr2=0x7f52140cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:03.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.128+0000 7f521adc4700 1 -- 172.21.15.157:0/3125671297 shutdown_connections 2023-10-06T04:11:03.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.128+0000 7f521adc4700 1 -- 172.21.15.157:0/3125671297 wait complete. 2023-10-06T04:11:03.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.129+0000 7f521adc4700 1 Processor -- start 2023-10-06T04:11:03.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.129+0000 7f521adc4700 1 -- start start 2023-10-06T04:11:03.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.130+0000 7f521adc4700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f521406c2d0 0x7f52140a7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:03.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.130+0000 7f521adc4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52140b04d0 0x7f52140cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:03.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.130+0000 7f52195c1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52140b04d0 0x7f52140cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:03.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.130+0000 7f52195c1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52140b04d0 0x7f52140cb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51890/0 (socket says 172.21.15.157:51890) 2023-10-06T04:11:03.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.130+0000 7f521adc4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52140b1e40 0x7f5214156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:03.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.130+0000 7f5219dc2700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f521406c2d0 0x7f52140a7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:03.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.130+0000 7f52195c1700 1 -- 172.21.15.157:0/2363869001 learned_addr learned my addr 172.21.15.157:0/2363869001 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:11:03.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.130+0000 7f521adc4700 1 -- 172.21.15.157:0/2363869001 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521415c070 con 0x7f52140b04d0 2023-10-06T04:11:03.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.130+0000 7f521adc4700 1 -- 172.21.15.157:0/2363869001 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52140c21b0 con 0x7f52140b1e40 2023-10-06T04:11:03.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.130+0000 7f521adc4700 1 -- 172.21.15.157:0/2363869001 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f521415bdd0 con 0x7f521406c2d0 2023-10-06T04:11:03.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.130+0000 7f521a5c3700 1 --2- 172.21.15.157:0/2363869001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52140b1e40 0x7f5214156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:03.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.130+0000 7f52195c1700 1 -- 172.21.15.157:0/2363869001 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f521406c2d0 msgr2=0x7f52140a7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:03.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.130+0000 7f52195c1700 1 --2- 172.21.15.157:0/2363869001 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f521406c2d0 0x7f52140a7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:03.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.130+0000 7f52195c1700 1 -- 172.21.15.157:0/2363869001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52140b1e40 msgr2=0x7f5214156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:03.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.130+0000 7f52195c1700 1 --2- 172.21.15.157:0/2363869001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52140b1e40 0x7f5214156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:03.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.130+0000 7f52195c1700 1 -- 172.21.15.157:0/2363869001 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5210009ae0 con 0x7f52140b04d0 2023-10-06T04:11:03.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.131+0000 7f52195c1700 1 --2- 172.21.15.157:0/2363869001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52140b04d0 0x7f52140cb9b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f520400d8c0 tx=0x7f5204008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:03.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.131+0000 7f5209ffb700 1 -- 172.21.15.157:0/2363869001 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52040125f0 con 0x7f52140b04d0 2023-10-06T04:11:03.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.131+0000 7f5209ffb700 1 -- 172.21.15.157:0/2363869001 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5204012750 con 0x7f52140b04d0 2023-10-06T04:11:03.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.131+0000 7f521adc4700 1 -- 172.21.15.157:0/2363869001 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52140c18c0 con 0x7f52140b04d0 2023-10-06T04:11:03.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.131+0000 7f5209ffb700 1 -- 172.21.15.157:0/2363869001 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5204012af0 con 0x7f52140b04d0 2023-10-06T04:11:03.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.131+0000 7f521adc4700 1 -- 172.21.15.157:0/2363869001 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52140c1b60 con 0x7f52140b04d0 2023-10-06T04:11:03.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.132+0000 7f521adc4700 1 -- 172.21.15.157:0/2363869001 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5214088f90 con 0x7f52140b04d0 2023-10-06T04:11:03.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.132+0000 7f5209ffb700 1 -- 172.21.15.157:0/2363869001 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f520401e460 con 0x7f52140b04d0 2023-10-06T04:11:03.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.132+0000 7f5209ffb700 1 --2- 172.21.15.157:0/2363869001 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f51f8066f60 0x7f51f8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:03.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.132+0000 7f5209ffb700 1 -- 172.21.15.157:0/2363869001 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f5204084e20 con 0x7f52140b04d0 2023-10-06T04:11:03.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.133+0000 7f5219dc2700 1 --2- 172.21.15.157:0/2363869001 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f51f8066f60 0x7f51f8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:03.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.133+0000 7f5219dc2700 1 --2- 172.21.15.157:0/2363869001 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f51f8066f60 0x7f51f8069410 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f520c00adc0 tx=0x7f520c006f90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:03.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.137+0000 7f5209ffb700 1 -- 172.21.15.157:0/2363869001 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5204055730 con 0x7f52140b04d0 2023-10-06T04:11:03.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.315+0000 7f521adc4700 1 -- 172.21.15.157:0/2363869001 --> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f521406d280 con 0x7f51f8066f60 2023-10-06T04:11:03.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.323+0000 7f5209ffb700 1 -- 172.21.15.157:0/2363869001 <== mgr.4107 v2:172.21.15.175:6832/90661 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+316200 (secure 0 0 0) 0x7f521406d280 con 0x7f51f8066f60 2023-10-06T04:11:03.329 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:11:03.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.326+0000 7f521adc4700 1 -- 172.21.15.157:0/2363869001 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f51f8066f60 msgr2=0x7f51f8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:03.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.326+0000 7f521adc4700 1 --2- 172.21.15.157:0/2363869001 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f51f8066f60 0x7f51f8069410 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f520c00adc0 tx=0x7f520c006f90 comp rx=0 tx=0).stop 2023-10-06T04:11:03.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.326+0000 7f521adc4700 1 -- 172.21.15.157:0/2363869001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52140b04d0 msgr2=0x7f52140cb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:03.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.326+0000 7f521adc4700 1 --2- 172.21.15.157:0/2363869001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52140b04d0 0x7f52140cb9b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f520400d8c0 tx=0x7f5204008040 comp rx=0 tx=0).stop 2023-10-06T04:11:03.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.326+0000 7f521adc4700 1 -- 172.21.15.157:0/2363869001 shutdown_connections 2023-10-06T04:11:03.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.326+0000 7f521adc4700 1 --2- 172.21.15.157:0/2363869001 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f51f8066f60 0x7f51f8069410 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:03.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.326+0000 7f521adc4700 1 --2- 172.21.15.157:0/2363869001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52140b1e40 0x7f5214156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:03.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.326+0000 7f521adc4700 1 --2- 172.21.15.157:0/2363869001 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f521406c2d0 0x7f52140a7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:03.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.326+0000 7f521adc4700 1 --2- 172.21.15.157:0/2363869001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52140b04d0 0x7f52140cb9b0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:03.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.327+0000 7f521adc4700 1 -- 172.21.15.157:0/2363869001 >> 172.21.15.157:0/2363869001 conn(0x7f52140cc540 msgr2=0x7f52140cc910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:03.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.327+0000 7f521adc4700 1 -- 172.21.15.157:0/2363869001 shutdown_connections 2023-10-06T04:11:03.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:03.327+0000 7f521adc4700 1 -- 172.21.15.157:0/2363869001 wait complete. 2023-10-06T04:11:03.333 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-10-06T04:11:03.341 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":466,"stamp":"2023-10-06T04:11:01.647115+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1091080968,"num_objects":10491,"num_object_clones":0,"num_object_copies":20982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10491,"num_whiteouts":0,"num_read":2914,"num_read_kb":634649,"num_write":27917,"num_write_kb":1246104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":122,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":29196,"ondisk_log_size":29196,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":4149728,"kb_used_data":1730464,"kb_used_omap":0,"kb_used_meta":2419200,"kb_avail":750824992,"statfs":{"total":773094113280,"available":768844791808,"internally_reserved":0,"allocated":1771995136,"data_stored":2182618599,"data_compressed":205433751,"data_compressed_allocated":234086400,"data_compressed_original":700964864,"omap_allocated":0,"internal_metadata":2477260800},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,37,0,26],"upper_bound":64},"perf_stat":{"commit_latency_ms":72,"apply_latency_ms":72,"commit_latency_ns":72000000,"apply_latency_ns":72000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":121205826,"num_objects":3140,"num_object_clones":0,"num_object_copies":6280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3328,"num_write_kb":119406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003345"},"pg_stats":[{"pgid":"4.3f","version":"53'338","reported_seq":2434,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.142502+0000","last_change":"2023-10-06T04:05:03.151059+0000","last_active":"2023-10-06T04:10:55.142502+0000","last_peered":"2023-10-06T04:10:55.142502+0000","last_clean":"2023-10-06T04:10:55.142502+0000","last_became_active":"2023-10-06T03:57:11.371756+0000","last_became_peered":"2023-10-06T03:57:11.371756+0000","last_unstale":"2023-10-06T04:10:55.142502+0000","last_undegraded":"2023-10-06T04:10:55.142502+0000","last_fullsized":"2023-10-06T04:10:55.142502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:50.086528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001094783,"stat_sum":{"num_bytes":10615219,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":29,"num_read_kb":15,"num_write":308,"num_write_kb":10492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"53'354","reported_seq":2454,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.129106+0000","last_change":"2023-10-06T04:05:02.973547+0000","last_active":"2023-10-06T04:10:55.129106+0000","last_peered":"2023-10-06T04:10:55.129106+0000","last_clean":"2023-10-06T04:10:55.129106+0000","last_became_active":"2023-10-06T03:57:11.370070+0000","last_became_peered":"2023-10-06T03:57:11.370070+0000","last_unstale":"2023-10-06T04:10:55.129106+0000","last_undegraded":"2023-10-06T04:10:55.129106+0000","last_fullsized":"2023-10-06T04:10:55.129106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:00.917607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021773819999999998,"stat_sum":{"num_bytes":6466808,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":33,"num_read_kb":16,"num_write":334,"num_write_kb":6491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"53'356","reported_seq":2460,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.122316+0000","last_change":"2023-10-06T04:05:02.723796+0000","last_active":"2023-10-06T04:10:55.122316+0000","last_peered":"2023-10-06T04:10:55.122316+0000","last_clean":"2023-10-06T04:10:55.122316+0000","last_became_active":"2023-10-06T03:57:11.366012+0000","last_became_peered":"2023-10-06T03:57:11.366012+0000","last_unstale":"2023-10-06T04:10:55.122316+0000","last_undegraded":"2023-10-06T04:10:55.122316+0000","last_fullsized":"2023-10-06T04:10:55.122316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:45.802287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002722286,"stat_sum":{"num_bytes":15834623,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":40,"num_read_kb":20499,"num_write":321,"num_write_kb":23772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"53'317","reported_seq":2416,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.117161+0000","last_change":"2023-10-06T04:05:03.543119+0000","last_active":"2023-10-06T04:10:55.117161+0000","last_peered":"2023-10-06T04:10:55.117161+0000","last_clean":"2023-10-06T04:10:55.117161+0000","last_became_active":"2023-10-06T03:57:11.365673+0000","last_became_peered":"2023-10-06T03:57:11.365673+0000","last_unstale":"2023-10-06T04:10:55.117161+0000","last_undegraded":"2023-10-06T04:10:55.117161+0000","last_fullsized":"2023-10-06T04:10:55.117161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:40.953382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001947182,"stat_sum":{"num_bytes":14764610,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":32,"num_read_kb":7963,"num_write":291,"num_write_kb":14577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"53'342","reported_seq":2440,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.141091+0000","last_change":"2023-10-06T04:05:03.366556+0000","last_active":"2023-10-06T04:10:55.141091+0000","last_peered":"2023-10-06T04:10:55.141091+0000","last_clean":"2023-10-06T04:10:55.141091+0000","last_became_active":"2023-10-06T03:57:11.361302+0000","last_became_peered":"2023-10-06T03:57:11.361302+0000","last_unstale":"2023-10-06T04:10:55.141091+0000","last_undegraded":"2023-10-06T04:10:55.141091+0000","last_fullsized":"2023-10-06T04:10:55.141091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:57.552493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001966644,"stat_sum":{"num_bytes":2193993,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":31,"num_read_kb":18,"num_write":324,"num_write_kb":2284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"53'328","reported_seq":2429,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.128484+0000","last_change":"2023-10-06T04:05:02.723270+0000","last_active":"2023-10-06T04:10:55.128484+0000","last_peered":"2023-10-06T04:10:55.128484+0000","last_clean":"2023-10-06T04:10:55.128484+0000","last_became_active":"2023-10-06T03:57:11.368775+0000","last_became_peered":"2023-10-06T03:57:11.368775+0000","last_unstale":"2023-10-06T04:10:55.128484+0000","last_undegraded":"2023-10-06T04:10:55.128484+0000","last_fullsized":"2023-10-06T04:10:55.128484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:22.368117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002640482,"stat_sum":{"num_bytes":22431099,"num_objects":144,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":0,"num_read":34,"num_read_kb":13627,"num_write":304,"num_write_kb":22068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"53'386","reported_seq":2485,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.121780+0000","last_change":"2023-10-06T04:05:03.343818+0000","last_active":"2023-10-06T04:10:55.121780+0000","last_peered":"2023-10-06T04:10:55.121780+0000","last_clean":"2023-10-06T04:10:55.121780+0000","last_became_active":"2023-10-06T03:57:11.377786+0000","last_became_peered":"2023-10-06T03:57:11.377786+0000","last_unstale":"2023-10-06T04:10:55.121780+0000","last_undegraded":"2023-10-06T04:10:55.121780+0000","last_fullsized":"2023-10-06T04:10:55.121780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:12:54.218593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093960500000000004,"stat_sum":{"num_bytes":18398941,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":32,"num_read_kb":8208,"num_write":357,"num_write_kb":18159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"53'332","reported_seq":2430,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.132073+0000","last_change":"2023-10-06T04:05:02.972640+0000","last_active":"2023-10-06T04:10:55.132073+0000","last_peered":"2023-10-06T04:10:55.132073+0000","last_clean":"2023-10-06T04:10:55.132073+0000","last_became_active":"2023-10-06T03:57:11.373765+0000","last_became_peered":"2023-10-06T03:57:11.373765+0000","last_unstale":"2023-10-06T04:10:55.132073+0000","last_undegraded":"2023-10-06T04:10:55.132073+0000","last_fullsized":"2023-10-06T04:10:55.132073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:21:05.838007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001873881,"stat_sum":{"num_bytes":10960737,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":31,"num_read_kb":4107,"num_write":305,"num_write_kb":10786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"53'330","reported_seq":2437,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.145767+0000","last_change":"2023-10-06T04:05:03.367332+0000","last_active":"2023-10-06T04:10:55.145767+0000","last_peered":"2023-10-06T04:10:55.145767+0000","last_clean":"2023-10-06T04:10:55.145767+0000","last_became_active":"2023-10-06T03:57:11.368004+0000","last_became_peered":"2023-10-06T03:57:11.368004+0000","last_unstale":"2023-10-06T04:10:55.145767+0000","last_undegraded":"2023-10-06T04:10:55.145767+0000","last_fullsized":"2023-10-06T04:10:55.145767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:16.760049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024398710000000001,"stat_sum":{"num_bytes":13584161,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":41,"num_read_kb":4114,"num_write":308,"num_write_kb":17522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"53'352","reported_seq":2449,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.135808+0000","last_change":"2023-10-06T04:05:02.973579+0000","last_active":"2023-10-06T04:10:55.135808+0000","last_peered":"2023-10-06T04:10:55.135808+0000","last_clean":"2023-10-06T04:10:55.135808+0000","last_became_active":"2023-10-06T03:57:11.379107+0000","last_became_peered":"2023-10-06T03:57:11.379107+0000","last_unstale":"2023-10-06T04:10:55.135808+0000","last_undegraded":"2023-10-06T04:10:55.135808+0000","last_fullsized":"2023-10-06T04:10:55.135808+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:18.373103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002039126,"stat_sum":{"num_bytes":9784235,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":30,"num_read_kb":4112,"num_write":328,"num_write_kb":9733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"53'376","reported_seq":2482,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.123159+0000","last_change":"2023-10-06T04:05:02.724475+0000","last_active":"2023-10-06T04:10:55.123159+0000","last_peered":"2023-10-06T04:10:55.123159+0000","last_clean":"2023-10-06T04:10:55.123159+0000","last_became_active":"2023-10-06T03:57:11.368256+0000","last_became_peered":"2023-10-06T03:57:11.368256+0000","last_unstale":"2023-10-06T04:10:55.123159+0000","last_undegraded":"2023-10-06T04:10:55.123159+0000","last_fullsized":"2023-10-06T04:10:55.123159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:50.736367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029138929999999999,"stat_sum":{"num_bytes":31134351,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":39,"num_read_kb":4114,"num_write":352,"num_write_kb":30587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"53'328","reported_seq":2425,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.122630+0000","last_change":"2023-10-06T04:05:03.151565+0000","last_active":"2023-10-06T04:10:55.122630+0000","last_peered":"2023-10-06T04:10:55.122630+0000","last_clean":"2023-10-06T04:10:55.122630+0000","last_became_active":"2023-10-06T03:57:11.368425+0000","last_became_peered":"2023-10-06T03:57:11.368425+0000","last_unstale":"2023-10-06T04:10:55.122630+0000","last_undegraded":"2023-10-06T04:10:55.122630+0000","last_fullsized":"2023-10-06T04:10:55.122630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:06.004423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014231059999999999,"stat_sum":{"num_bytes":10422851,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":30,"num_read_kb":4105,"num_write":305,"num_write_kb":14436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"53'343","reported_seq":2446,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.142516+0000","last_change":"2023-10-06T04:05:03.150960+0000","last_active":"2023-10-06T04:10:55.142516+0000","last_peered":"2023-10-06T04:10:55.142516+0000","last_clean":"2023-10-06T04:10:55.142516+0000","last_became_active":"2023-10-06T03:57:11.367079+0000","last_became_peered":"2023-10-06T03:57:11.367079+0000","last_unstale":"2023-10-06T04:10:55.142516+0000","last_undegraded":"2023-10-06T04:10:55.142516+0000","last_fullsized":"2023-10-06T04:10:55.142516+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:09:54.331206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001042765,"stat_sum":{"num_bytes":14365712,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":37,"num_read_kb":12308,"num_write":314,"num_write_kb":22325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"53'346","reported_seq":2449,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.127894+0000","last_change":"2023-10-06T04:05:02.973393+0000","last_active":"2023-10-06T04:10:55.127894+0000","last_peered":"2023-10-06T04:10:55.127894+0000","last_clean":"2023-10-06T04:10:55.127894+0000","last_became_active":"2023-10-06T03:57:11.373216+0000","last_became_peered":"2023-10-06T03:57:11.373216+0000","last_unstale":"2023-10-06T04:10:55.127894+0000","last_undegraded":"2023-10-06T04:10:55.127894+0000","last_fullsized":"2023-10-06T04:10:55.127894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:56.083203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022456289999999999,"stat_sum":{"num_bytes":5765763,"num_objects":144,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":0,"num_read":36,"num_read_kb":18,"num_write":320,"num_write_kb":9892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"53'348","reported_seq":2445,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.121683+0000","last_change":"2023-10-06T04:05:03.343999+0000","last_active":"2023-10-06T04:10:55.121683+0000","last_peered":"2023-10-06T04:10:55.121683+0000","last_clean":"2023-10-06T04:10:55.121683+0000","last_became_active":"2023-10-06T03:57:11.375689+0000","last_became_peered":"2023-10-06T03:57:11.375689+0000","last_unstale":"2023-10-06T04:10:55.121683+0000","last_undegraded":"2023-10-06T04:10:55.121683+0000","last_fullsized":"2023-10-06T04:10:55.121683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:42.953739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092625599999999998,"stat_sum":{"num_bytes":15107134,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":322,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"53'359","reported_seq":2461,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.135744+0000","last_change":"2023-10-06T04:05:02.722778+0000","last_active":"2023-10-06T04:10:55.135744+0000","last_peered":"2023-10-06T04:10:55.135744+0000","last_clean":"2023-10-06T04:10:55.135744+0000","last_became_active":"2023-10-06T03:57:11.366543+0000","last_became_peered":"2023-10-06T03:57:11.366543+0000","last_unstale":"2023-10-06T04:10:55.135744+0000","last_undegraded":"2023-10-06T04:10:55.135744+0000","last_fullsized":"2023-10-06T04:10:55.135744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:16.212967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024276609999999998,"stat_sum":{"num_bytes":19833591,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":35,"num_read_kb":19,"num_write":334,"num_write_kb":22749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"53'358","reported_seq":2458,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.145337+0000","last_change":"2023-10-06T04:05:03.367754+0000","last_active":"2023-10-06T04:10:55.145337+0000","last_peered":"2023-10-06T04:10:55.145337+0000","last_clean":"2023-10-06T04:10:55.145337+0000","last_became_active":"2023-10-06T03:57:11.366767+0000","last_became_peered":"2023-10-06T03:57:11.366767+0000","last_unstale":"2023-10-06T04:10:55.145337+0000","last_undegraded":"2023-10-06T04:10:55.145337+0000","last_fullsized":"2023-10-06T04:10:55.145337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:43.113575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002777833,"stat_sum":{"num_bytes":14683528,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":34,"num_read_kb":8213,"num_write":334,"num_write_kb":18604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"53'370","reported_seq":2480,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.128296+0000","last_change":"2023-10-06T04:05:02.723910+0000","last_active":"2023-10-06T04:10:55.128296+0000","last_peered":"2023-10-06T04:10:55.128296+0000","last_clean":"2023-10-06T04:10:55.128296+0000","last_became_active":"2023-10-06T03:57:11.362377+0000","last_became_peered":"2023-10-06T03:57:11.362377+0000","last_unstale":"2023-10-06T04:10:55.128296+0000","last_undegraded":"2023-10-06T04:10:55.128296+0000","last_fullsized":"2023-10-06T04:10:55.128296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:40.600383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027606900000000001,"stat_sum":{"num_bytes":10886042,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":43,"num_read_kb":26,"num_write":338,"num_write_kb":10805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"53'368","reported_seq":2474,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.123874+0000","last_change":"2023-10-06T04:05:03.151735+0000","last_active":"2023-10-06T04:10:55.123874+0000","last_peered":"2023-10-06T04:10:55.123874+0000","last_clean":"2023-10-06T04:10:55.123874+0000","last_became_active":"2023-10-06T03:57:11.363792+0000","last_became_peered":"2023-10-06T03:57:11.363792+0000","last_unstale":"2023-10-06T04:10:55.123874+0000","last_undegraded":"2023-10-06T04:10:55.123874+0000","last_fullsized":"2023-10-06T04:10:55.123874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:22:49.185450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014611940000000001,"stat_sum":{"num_bytes":14666818,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":40,"num_read_kb":8209,"num_write":335,"num_write_kb":18594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"53'367","reported_seq":2470,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.130639+0000","last_change":"2023-10-06T04:05:02.724524+0000","last_active":"2023-10-06T04:10:55.130639+0000","last_peered":"2023-10-06T04:10:55.130639+0000","last_clean":"2023-10-06T04:10:55.130639+0000","last_became_active":"2023-10-06T03:57:11.364726+0000","last_became_peered":"2023-10-06T03:57:11.364726+0000","last_unstale":"2023-10-06T04:10:55.130639+0000","last_undegraded":"2023-10-06T04:10:55.130639+0000","last_fullsized":"2023-10-06T04:10:55.130639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:56.866733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029763670000000002,"stat_sum":{"num_bytes":20706667,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":37,"num_read_kb":12308,"num_write":348,"num_write_kb":28542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"53'348","reported_seq":2457,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.142219+0000","last_change":"2023-10-06T04:05:03.344002+0000","last_active":"2023-10-06T04:10:55.142219+0000","last_peered":"2023-10-06T04:10:55.142219+0000","last_clean":"2023-10-06T04:10:55.142219+0000","last_became_active":"2023-10-06T03:57:11.375668+0000","last_became_peered":"2023-10-06T03:57:11.375668+0000","last_unstale":"2023-10-06T04:10:55.142219+0000","last_undegraded":"2023-10-06T04:10:55.142219+0000","last_fullsized":"2023-10-06T04:10:55.142219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:53.810875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010285800000000001,"stat_sum":{"num_bytes":19005599,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":42,"num_read_kb":8215,"num_write":326,"num_write_kb":18702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"53'363","reported_seq":2464,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.135011+0000","last_change":"2023-10-06T04:05:03.366839+0000","last_active":"2023-10-06T04:10:55.135011+0000","last_peered":"2023-10-06T04:10:55.135011+0000","last_clean":"2023-10-06T04:10:55.135011+0000","last_became_active":"2023-10-06T03:57:11.365149+0000","last_became_peered":"2023-10-06T03:57:11.365149+0000","last_unstale":"2023-10-06T04:10:55.135011+0000","last_undegraded":"2023-10-06T04:10:55.135011+0000","last_fullsized":"2023-10-06T04:10:55.135011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:20.911710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020974879999999998,"stat_sum":{"num_bytes":5860015,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":34,"num_read_kb":17,"num_write":331,"num_write_kb":9992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"53'364","reported_seq":2472,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.139880+0000","last_change":"2023-10-06T04:05:02.722763+0000","last_active":"2023-10-06T04:10:55.139880+0000","last_peered":"2023-10-06T04:10:55.139880+0000","last_clean":"2023-10-06T04:10:55.139880+0000","last_became_active":"2023-10-06T03:57:11.362128+0000","last_became_peered":"2023-10-06T03:57:11.362128+0000","last_unstale":"2023-10-06T04:10:55.139880+0000","last_undegraded":"2023-10-06T04:10:55.139880+0000","last_fullsized":"2023-10-06T04:10:55.139880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:53.744674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025174860000000002,"stat_sum":{"num_bytes":10052289,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":42,"num_read_kb":8214,"num_write":324,"num_write_kb":14077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"53'376","reported_seq":2490,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.141916+0000","last_change":"2023-10-06T04:05:03.344184+0000","last_active":"2023-10-06T04:10:55.141916+0000","last_peered":"2023-10-06T04:10:55.141916+0000","last_clean":"2023-10-06T04:10:55.141916+0000","last_became_active":"2023-10-06T03:57:11.375702+0000","last_became_peered":"2023-10-06T03:57:11.375702+0000","last_unstale":"2023-10-06T04:10:55.141916+0000","last_undegraded":"2023-10-06T04:10:55.141916+0000","last_fullsized":"2023-10-06T04:10:55.141916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:31.204231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010337009999999999,"stat_sum":{"num_bytes":19987419,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":47,"num_read_kb":12104,"num_write":347,"num_write_kb":19698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"53'323","reported_seq":2419,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.118449+0000","last_change":"2023-10-06T04:05:02.718513+0000","last_active":"2023-10-06T04:10:55.118449+0000","last_peered":"2023-10-06T04:10:55.118449+0000","last_clean":"2023-10-06T04:10:55.118449+0000","last_became_active":"2023-10-06T03:57:11.374842+0000","last_became_peered":"2023-10-06T03:57:11.374842+0000","last_unstale":"2023-10-06T04:10:55.118449+0000","last_undegraded":"2023-10-06T04:10:55.118449+0000","last_fullsized":"2023-10-06T04:10:55.118449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:54.438974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001332342,"stat_sum":{"num_bytes":5811346,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":29,"num_read_kb":4115,"num_write":308,"num_write_kb":5844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"53'393","reported_seq":2493,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.125824+0000","last_change":"2023-10-06T04:05:02.718726+0000","last_active":"2023-10-06T04:10:55.125824+0000","last_peered":"2023-10-06T04:10:55.125824+0000","last_clean":"2023-10-06T04:10:55.125824+0000","last_became_active":"2023-10-06T03:57:11.361349+0000","last_became_peered":"2023-10-06T03:57:11.361349+0000","last_unstale":"2023-10-06T04:10:55.125824+0000","last_undegraded":"2023-10-06T04:10:55.125824+0000","last_fullsized":"2023-10-06T04:10:55.125824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:30.646781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014394589999999999,"stat_sum":{"num_bytes":23015133,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":33,"num_read_kb":4115,"num_write":357,"num_write_kb":22663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"53'328","reported_seq":2433,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.131224+0000","last_change":"2023-10-06T04:05:02.972918+0000","last_active":"2023-10-06T04:10:55.131224+0000","last_peered":"2023-10-06T04:10:55.131224+0000","last_clean":"2023-10-06T04:10:55.131224+0000","last_became_active":"2023-10-06T03:57:11.375846+0000","last_became_peered":"2023-10-06T03:57:11.375846+0000","last_unstale":"2023-10-06T04:10:55.131224+0000","last_undegraded":"2023-10-06T04:10:55.131224+0000","last_fullsized":"2023-10-06T04:10:55.131224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:28.080614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018480599999999999,"stat_sum":{"num_bytes":26318621,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":38,"num_read_kb":12302,"num_write":306,"num_write_kb":25858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"53'411","reported_seq":466,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:56.864866+0000","last_change":"2023-10-06T03:57:10.348317+0000","last_active":"2023-10-06T04:10:56.864866+0000","last_peered":"2023-10-06T04:10:56.864866+0000","last_clean":"2023-10-06T04:10:56.864866+0000","last_became_active":"2023-10-06T03:57:10.348078+0000","last_became_peered":"2023-10-06T03:57:10.348078+0000","last_unstale":"2023-10-06T04:10:56.864866+0000","last_undegraded":"2023-10-06T04:10:56.864866+0000","last_fullsized":"2023-10-06T04:10:56.864866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:55.058324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15284152,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":13,"num_read_kb":12332,"num_write":406,"num_write_kb":15159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"53'349","reported_seq":2454,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.139734+0000","last_change":"2023-10-06T04:05:02.722997+0000","last_active":"2023-10-06T04:10:55.139734+0000","last_peered":"2023-10-06T04:10:55.139734+0000","last_clean":"2023-10-06T04:10:55.139734+0000","last_became_active":"2023-10-06T03:57:11.370245+0000","last_became_peered":"2023-10-06T03:57:11.370245+0000","last_unstale":"2023-10-06T04:10:55.139734+0000","last_undegraded":"2023-10-06T04:10:55.139734+0000","last_fullsized":"2023-10-06T04:10:55.139734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:47:18.189772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002348424,"stat_sum":{"num_bytes":22265950,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":38,"num_read_kb":9495,"num_write":317,"num_write_kb":26009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"53'89","reported_seq":168,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:42.933757+0000","last_change":"2023-10-06T03:57:10.345401+0000","last_active":"2023-10-06T04:10:42.933757+0000","last_peered":"2023-10-06T04:10:42.933757+0000","last_clean":"2023-10-06T04:10:42.933757+0000","last_became_active":"2023-10-06T03:57:10.345150+0000","last_became_peered":"2023-10-06T03:57:10.345150+0000","last_unstale":"2023-10-06T04:10:42.933757+0000","last_undegraded":"2023-10-06T04:10:42.933757+0000","last_fullsized":"2023-10-06T04:10:42.933757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:25.758193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4312676,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":146,"num_write":89,"num_write_kb":4275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"53'350","reported_seq":2443,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.135097+0000","last_change":"2023-10-06T04:05:03.367558+0000","last_active":"2023-10-06T04:10:55.135097+0000","last_peered":"2023-10-06T04:10:55.135097+0000","last_clean":"2023-10-06T04:10:55.135097+0000","last_became_active":"2023-10-06T03:57:11.375790+0000","last_became_peered":"2023-10-06T03:57:11.375790+0000","last_unstale":"2023-10-06T04:10:55.135097+0000","last_undegraded":"2023-10-06T04:10:55.135097+0000","last_fullsized":"2023-10-06T04:10:55.135097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:33.775431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025999780000000002,"stat_sum":{"num_bytes":19085062,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":26,"num_read_kb":12300,"num_write":331,"num_write_kb":18812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"53'1073","reported_seq":1122,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:52.526921+0000","last_change":"2023-10-06T03:57:10.347562+0000","last_active":"2023-10-06T04:09:52.526921+0000","last_peered":"2023-10-06T04:09:52.526921+0000","last_clean":"2023-10-06T04:09:52.526921+0000","last_became_active":"2023-10-06T03:57:10.347294+0000","last_became_peered":"2023-10-06T03:57:10.347294+0000","last_unstale":"2023-10-06T04:09:52.526921+0000","last_undegraded":"2023-10-06T04:09:52.526921+0000","last_fullsized":"2023-10-06T04:09:52.526921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":1073,"log_dups_size":0,"ondisk_log_size":1073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:25.090992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583466,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":7,"num_read_kb":16784,"num_write":1067,"num_write_kb":17022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"53'365","reported_seq":2467,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.113224+0000","last_change":"2023-10-06T04:05:03.543192+0000","last_active":"2023-10-06T04:10:55.113224+0000","last_peered":"2023-10-06T04:10:55.113224+0000","last_clean":"2023-10-06T04:10:55.113224+0000","last_became_active":"2023-10-06T03:57:11.371758+0000","last_became_peered":"2023-10-06T03:57:11.371758+0000","last_unstale":"2023-10-06T04:10:55.113224+0000","last_undegraded":"2023-10-06T04:10:55.113224+0000","last_fullsized":"2023-10-06T04:10:55.113224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:10.922157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017888349999999999,"stat_sum":{"num_bytes":16888108,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":35,"num_read_kb":4115,"num_write":349,"num_write_kb":16624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"53'47","reported_seq":95,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:36.261131+0000","last_change":"2023-10-06T03:57:10.347406+0000","last_active":"2023-10-06T04:10:36.261131+0000","last_peered":"2023-10-06T04:10:36.261131+0000","last_clean":"2023-10-06T04:10:36.261131+0000","last_became_active":"2023-10-06T03:57:10.347151+0000","last_became_peered":"2023-10-06T03:57:10.347151+0000","last_unstale":"2023-10-06T04:10:36.261131+0000","last_undegraded":"2023-10-06T04:10:36.261131+0000","last_fullsized":"2023-10-06T04:10:36.261131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:20.427620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":50,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"53'332","reported_seq":2435,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.145432+0000","last_change":"2023-10-06T04:05:02.721771+0000","last_active":"2023-10-06T04:10:55.145432+0000","last_peered":"2023-10-06T04:10:55.145432+0000","last_clean":"2023-10-06T04:10:55.145432+0000","last_became_active":"2023-10-06T03:57:11.369938+0000","last_became_peered":"2023-10-06T03:57:11.369938+0000","last_unstale":"2023-10-06T04:10:55.145432+0000","last_undegraded":"2023-10-06T04:10:55.145432+0000","last_fullsized":"2023-10-06T04:10:55.145432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:07.105832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018579569999999999,"stat_sum":{"num_bytes":31490698,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":37,"num_read_kb":12302,"num_write":314,"num_write_kb":31163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"53'50","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:46.318091+0000","last_change":"2023-10-06T03:57:10.346054+0000","last_active":"2023-10-06T04:10:46.318091+0000","last_peered":"2023-10-06T04:10:46.318091+0000","last_clean":"2023-10-06T04:10:46.318091+0000","last_became_active":"2023-10-06T03:57:10.345721+0000","last_became_peered":"2023-10-06T03:57:10.345721+0000","last_unstale":"2023-10-06T04:10:46.318091+0000","last_undegraded":"2023-10-06T04:10:46.318091+0000","last_fullsized":"2023-10-06T04:10:46.318091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:54.481304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":15,"num_write":48,"num_write_kb":4121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"53'374","reported_seq":2474,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.124059+0000","last_change":"2023-10-06T04:05:03.151517+0000","last_active":"2023-10-06T04:10:55.124059+0000","last_peered":"2023-10-06T04:10:55.124059+0000","last_clean":"2023-10-06T04:10:55.124059+0000","last_became_active":"2023-10-06T03:57:59.255587+0000","last_became_peered":"2023-10-06T03:57:59.255587+0000","last_unstale":"2023-10-06T04:10:55.124059+0000","last_undegraded":"2023-10-06T04:10:55.124059+0000","last_fullsized":"2023-10-06T04:10:55.124059+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:08.261023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001510024,"stat_sum":{"num_bytes":1845822,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":38,"num_read_kb":8209,"num_write":345,"num_write_kb":10179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"53'63","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:42.932445+0000","last_change":"2023-10-06T03:57:10.340531+0000","last_active":"2023-10-06T04:10:42.932445+0000","last_peered":"2023-10-06T04:10:42.932445+0000","last_clean":"2023-10-06T04:10:42.932445+0000","last_became_active":"2023-10-06T03:57:10.340232+0000","last_became_peered":"2023-10-06T03:57:10.340232+0000","last_unstale":"2023-10-06T04:10:42.932445+0000","last_undegraded":"2023-10-06T04:10:42.932445+0000","last_fullsized":"2023-10-06T04:10:42.932445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:43:26.332055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":66,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"53'364","reported_seq":2468,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.130771+0000","last_change":"2023-10-06T04:05:02.718927+0000","last_active":"2023-10-06T04:10:55.130771+0000","last_peered":"2023-10-06T04:10:55.130771+0000","last_clean":"2023-10-06T04:10:55.130771+0000","last_became_active":"2023-10-06T03:57:11.374892+0000","last_became_peered":"2023-10-06T03:57:11.374892+0000","last_unstale":"2023-10-06T04:10:55.130771+0000","last_undegraded":"2023-10-06T04:10:55.130771+0000","last_fullsized":"2023-10-06T04:10:55.130771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:51.198745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00156207,"stat_sum":{"num_bytes":17801071,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":38,"num_read_kb":8211,"num_write":336,"num_write_kb":25748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"53'671","reported_seq":728,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:41.284434+0000","last_change":"2023-10-06T03:57:10.346758+0000","last_active":"2023-10-06T04:10:41.284434+0000","last_peered":"2023-10-06T04:10:41.284434+0000","last_clean":"2023-10-06T04:10:41.284434+0000","last_became_active":"2023-10-06T03:57:10.346430+0000","last_became_peered":"2023-10-06T03:57:10.346430+0000","last_unstale":"2023-10-06T04:10:41.284434+0000","last_undegraded":"2023-10-06T04:10:41.284434+0000","last_fullsized":"2023-10-06T04:10:41.284434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:10.809242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389252,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":30,"num_read_kb":9887,"num_write":669,"num_write_kb":8555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"53'354","reported_seq":2459,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.130514+0000","last_change":"2023-10-06T04:05:02.723446+0000","last_active":"2023-10-06T04:10:55.130514+0000","last_peered":"2023-10-06T04:10:55.130514+0000","last_clean":"2023-10-06T04:10:55.130514+0000","last_became_active":"2023-10-06T03:57:11.367565+0000","last_became_peered":"2023-10-06T03:57:11.367565+0000","last_unstale":"2023-10-06T04:10:55.130514+0000","last_undegraded":"2023-10-06T04:10:55.130514+0000","last_fullsized":"2023-10-06T04:10:55.130514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:14.409116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027762680000000001,"stat_sum":{"num_bytes":10311229,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":38,"num_read_kb":23,"num_write":322,"num_write_kb":10236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"53'179","reported_seq":235,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:16.251094+0000","last_change":"2023-10-06T03:57:10.351062+0000","last_active":"2023-10-06T04:10:16.251094+0000","last_peered":"2023-10-06T04:10:16.251094+0000","last_clean":"2023-10-06T04:10:16.251094+0000","last_became_active":"2023-10-06T03:57:10.350795+0000","last_became_peered":"2023-10-06T03:57:10.350795+0000","last_unstale":"2023-10-06T04:10:16.251094+0000","last_undegraded":"2023-10-06T04:10:16.251094+0000","last_fullsized":"2023-10-06T04:10:16.251094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:54:45.714518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388676,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":30,"num_read_kb":20509,"num_write":184,"num_write_kb":8306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"53'303","reported_seq":2402,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.142731+0000","last_change":"2023-10-06T04:05:02.722868+0000","last_active":"2023-10-06T04:10:55.142731+0000","last_peered":"2023-10-06T04:10:55.142731+0000","last_clean":"2023-10-06T04:10:55.142731+0000","last_became_active":"2023-10-06T03:57:11.367254+0000","last_became_peered":"2023-10-06T03:57:11.367254+0000","last_unstale":"2023-10-06T04:10:55.142731+0000","last_undegraded":"2023-10-06T04:10:55.142731+0000","last_fullsized":"2023-10-06T04:10:55.142731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:04.706485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027822569999999998,"stat_sum":{"num_bytes":13704260,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":32,"num_read_kb":10530,"num_write":279,"num_write_kb":17631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"53'105","reported_seq":157,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:23.094836+0000","last_change":"2023-10-06T03:57:10.338125+0000","last_active":"2023-10-06T04:10:23.094836+0000","last_peered":"2023-10-06T04:10:23.094836+0000","last_clean":"2023-10-06T04:10:23.094836+0000","last_became_active":"2023-10-06T03:57:10.337798+0000","last_became_peered":"2023-10-06T03:57:10.337798+0000","last_unstale":"2023-10-06T04:10:23.094836+0000","last_undegraded":"2023-10-06T04:10:23.094836+0000","last_fullsized":"2023-10-06T04:10:23.094836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:00.482657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":9753,"num_write":104,"num_write_kb":4165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"53'356","reported_seq":2460,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.131034+0000","last_change":"2023-10-06T04:05:02.973084+0000","last_active":"2023-10-06T04:10:55.131034+0000","last_peered":"2023-10-06T04:10:55.131034+0000","last_clean":"2023-10-06T04:10:55.131034+0000","last_became_active":"2023-10-06T03:57:11.376223+0000","last_became_peered":"2023-10-06T03:57:11.376223+0000","last_unstale":"2023-10-06T04:10:55.131034+0000","last_undegraded":"2023-10-06T04:10:55.131034+0000","last_fullsized":"2023-10-06T04:10:55.131034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:04:37.408871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018161119999999999,"stat_sum":{"num_bytes":34940720,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":37,"num_read_kb":11570,"num_write":336,"num_write_kb":34292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"53'99","reported_seq":141,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:56.861368+0000","last_change":"2023-10-06T03:57:10.350511+0000","last_active":"2023-10-06T04:10:56.861368+0000","last_peered":"2023-10-06T04:10:56.861368+0000","last_clean":"2023-10-06T04:10:56.861368+0000","last_became_active":"2023-10-06T03:57:10.350219+0000","last_became_peered":"2023-10-06T03:57:10.350219+0000","last_unstale":"2023-10-06T04:10:56.861368+0000","last_undegraded":"2023-10-06T04:10:56.861368+0000","last_fullsized":"2023-10-06T04:10:56.861368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:56.043506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4377301,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98,"num_write_kb":8473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"53'381","reported_seq":2490,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.140966+0000","last_change":"2023-10-06T04:05:03.367352+0000","last_active":"2023-10-06T04:10:55.140966+0000","last_peered":"2023-10-06T04:10:55.140966+0000","last_clean":"2023-10-06T04:10:55.140966+0000","last_became_active":"2023-10-06T03:57:11.362443+0000","last_became_peered":"2023-10-06T03:57:11.362443+0000","last_unstale":"2023-10-06T04:10:55.140966+0000","last_undegraded":"2023-10-06T04:10:55.140966+0000","last_fullsized":"2023-10-06T04:10:55.140966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:21:09.143085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002448839,"stat_sum":{"num_bytes":22978165,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":42,"num_read_kb":8288,"num_write":356,"num_write_kb":22628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"53'71","reported_seq":124,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:36.258463+0000","last_change":"2023-10-06T03:57:10.346259+0000","last_active":"2023-10-06T04:10:36.258463+0000","last_peered":"2023-10-06T04:10:36.258463+0000","last_clean":"2023-10-06T04:10:36.258463+0000","last_became_active":"2023-10-06T03:57:10.345424+0000","last_became_peered":"2023-10-06T03:57:10.345424+0000","last_unstale":"2023-10-06T04:10:36.258463+0000","last_undegraded":"2023-10-06T04:10:36.258463+0000","last_fullsized":"2023-10-06T04:10:36.258463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:19.278206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":33,"num_read_kb":33,"num_write":72,"num_write_kb":4158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"53'333","reported_seq":2438,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.144094+0000","last_change":"2023-10-06T04:05:02.723050+0000","last_active":"2023-10-06T04:10:55.144094+0000","last_peered":"2023-10-06T04:10:55.144094+0000","last_clean":"2023-10-06T04:10:55.144094+0000","last_became_active":"2023-10-06T03:57:11.375824+0000","last_became_peered":"2023-10-06T03:57:11.375824+0000","last_unstale":"2023-10-06T04:10:55.144094+0000","last_undegraded":"2023-10-06T04:10:55.144094+0000","last_fullsized":"2023-10-06T04:10:55.144094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:24:26.156866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023200299999999998,"stat_sum":{"num_bytes":9857160,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":38,"num_read_kb":8212,"num_write":304,"num_write_kb":9779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"53'250","reported_seq":295,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:23.097110+0000","last_change":"2023-10-06T03:57:10.350408+0000","last_active":"2023-10-06T04:10:23.097110+0000","last_peered":"2023-10-06T04:10:23.097110+0000","last_clean":"2023-10-06T04:10:23.097110+0000","last_became_active":"2023-10-06T03:57:10.350135+0000","last_became_peered":"2023-10-06T03:57:10.350135+0000","last_unstale":"2023-10-06T04:10:23.097110+0000","last_undegraded":"2023-10-06T04:10:23.097110+0000","last_fullsized":"2023-10-06T04:10:23.097110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:43.522111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":4098,"num_write":280,"num_write_kb":4293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"53'341","reported_seq":2440,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.131425+0000","last_change":"2023-10-06T04:05:02.972356+0000","last_active":"2023-10-06T04:10:55.131425+0000","last_peered":"2023-10-06T04:10:55.131425+0000","last_clean":"2023-10-06T04:10:55.131425+0000","last_became_active":"2023-10-06T03:57:11.376236+0000","last_became_peered":"2023-10-06T03:57:11.376236+0000","last_unstale":"2023-10-06T04:10:55.131425+0000","last_undegraded":"2023-10-06T04:10:55.131425+0000","last_fullsized":"2023-10-06T04:10:55.131425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:33.854818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015573080000000001,"stat_sum":{"num_bytes":22935537,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":32,"num_read_kb":10317,"num_write":317,"num_write_kb":26655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"50'92","reported_seq":139,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:52.580642+0000","last_change":"2023-10-06T03:57:10.345799+0000","last_active":"2023-10-06T04:09:52.580642+0000","last_peered":"2023-10-06T04:09:52.580642+0000","last_clean":"2023-10-06T04:09:52.580642+0000","last_became_active":"2023-10-06T03:57:10.345510+0000","last_became_peered":"2023-10-06T03:57:10.345510+0000","last_unstale":"2023-10-06T04:09:52.580642+0000","last_undegraded":"2023-10-06T04:09:52.580642+0000","last_fullsized":"2023-10-06T04:09:52.580642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:09.567314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":4108,"num_write":90,"num_write_kb":4140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"53'359","reported_seq":2463,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.047920+0000","last_change":"2023-10-06T04:05:02.721917+0000","last_active":"2023-10-06T04:10:55.047920+0000","last_peered":"2023-10-06T04:10:55.047920+0000","last_clean":"2023-10-06T04:10:55.047920+0000","last_became_active":"2023-10-06T03:57:11.368375+0000","last_became_peered":"2023-10-06T03:57:11.368375+0000","last_unstale":"2023-10-06T04:10:55.047920+0000","last_undegraded":"2023-10-06T04:10:55.047920+0000","last_fullsized":"2023-10-06T04:10:55.047920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:27.125523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001632912,"stat_sum":{"num_bytes":5657073,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":37,"num_read_kb":4115,"num_write":343,"num_write_kb":5657,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"53'164","reported_seq":212,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:22.809255+0000","last_change":"2023-10-06T03:57:10.346945+0000","last_active":"2023-10-06T04:10:22.809255+0000","last_peered":"2023-10-06T04:10:22.809255+0000","last_clean":"2023-10-06T04:10:22.809255+0000","last_became_active":"2023-10-06T03:57:10.346689+0000","last_became_peered":"2023-10-06T03:57:10.346689+0000","last_unstale":"2023-10-06T04:10:22.809255+0000","last_undegraded":"2023-10-06T04:10:22.809255+0000","last_fullsized":"2023-10-06T04:10:22.809255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:43.420455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":12290,"num_write":163,"num_write_kb":8336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"53'375","reported_seq":2476,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.119832+0000","last_change":"2023-10-06T04:05:03.542774+0000","last_active":"2023-10-06T04:10:55.119832+0000","last_peered":"2023-10-06T04:10:55.119832+0000","last_clean":"2023-10-06T04:10:55.119832+0000","last_became_active":"2023-10-06T03:57:11.371795+0000","last_became_peered":"2023-10-06T03:57:11.371795+0000","last_unstale":"2023-10-06T04:10:55.119832+0000","last_undegraded":"2023-10-06T04:10:55.119832+0000","last_fullsized":"2023-10-06T04:10:55.119832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:00.324478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016252980000000001,"stat_sum":{"num_bytes":5774867,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":34,"num_read_kb":13,"num_write":355,"num_write_kb":5825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"53'54","reported_seq":96,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:48.081196+0000","last_change":"2023-10-06T03:57:10.348592+0000","last_active":"2023-10-06T04:09:48.081196+0000","last_peered":"2023-10-06T04:09:48.081196+0000","last_clean":"2023-10-06T04:09:48.081196+0000","last_became_active":"2023-10-06T03:57:10.348291+0000","last_became_peered":"2023-10-06T03:57:10.348291+0000","last_unstale":"2023-10-06T04:09:48.081196+0000","last_undegraded":"2023-10-06T04:09:48.081196+0000","last_fullsized":"2023-10-06T04:09:48.081196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:00.660348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":57,"num_write_kb":4195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"53'328","reported_seq":2428,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.134970+0000","last_change":"2023-10-06T04:05:03.366405+0000","last_active":"2023-10-06T04:10:55.134970+0000","last_peered":"2023-10-06T04:10:55.134970+0000","last_clean":"2023-10-06T04:10:55.134970+0000","last_became_active":"2023-10-06T03:57:11.365077+0000","last_became_peered":"2023-10-06T03:57:11.365077+0000","last_unstale":"2023-10-06T04:10:55.134970+0000","last_undegraded":"2023-10-06T04:10:55.134970+0000","last_fullsized":"2023-10-06T04:10:55.134970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:25.199391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019352270000000001,"stat_sum":{"num_bytes":15068515,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":33,"num_read_kb":4109,"num_write":308,"num_write_kb":14877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"52'278","reported_seq":329,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:51.781455+0000","last_change":"2023-10-06T03:57:10.348418+0000","last_active":"2023-10-06T04:09:51.781455+0000","last_peered":"2023-10-06T04:09:51.781455+0000","last_clean":"2023-10-06T04:09:51.781455+0000","last_became_active":"2023-10-06T03:57:10.348055+0000","last_became_peered":"2023-10-06T03:57:10.348055+0000","last_unstale":"2023-10-06T04:09:51.781455+0000","last_undegraded":"2023-10-06T04:09:51.781455+0000","last_fullsized":"2023-10-06T04:09:51.781455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:26.102022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":17,"num_read_kb":12301,"num_write":272,"num_write_kb":8316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"53'366","reported_seq":2461,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.135948+0000","last_change":"2023-10-06T04:05:02.724303+0000","last_active":"2023-10-06T04:10:55.135948+0000","last_peered":"2023-10-06T04:10:55.135948+0000","last_clean":"2023-10-06T04:10:55.135948+0000","last_became_active":"2023-10-06T03:57:11.370073+0000","last_became_peered":"2023-10-06T03:57:11.370073+0000","last_unstale":"2023-10-06T04:10:55.135948+0000","last_undegraded":"2023-10-06T04:10:55.135948+0000","last_fullsized":"2023-10-06T04:10:55.135948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:58.417951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027454179999999999,"stat_sum":{"num_bytes":18058404,"num_objects":185,"num_object_clones":0,"num_object_copies":370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":185,"num_whiteouts":0,"num_read":28,"num_read_kb":4110,"num_write":345,"num_write_kb":17816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"53'113","reported_seq":160,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:50.963705+0000","last_change":"2023-10-06T03:57:10.352966+0000","last_active":"2023-10-06T04:10:50.963705+0000","last_peered":"2023-10-06T04:10:50.963705+0000","last_clean":"2023-10-06T04:10:50.963705+0000","last_became_active":"2023-10-06T03:57:10.352653+0000","last_became_peered":"2023-10-06T03:57:10.352653+0000","last_unstale":"2023-10-06T04:10:50.963705+0000","last_undegraded":"2023-10-06T04:10:50.963705+0000","last_fullsized":"2023-10-06T04:10:50.963705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:32.092557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":8,"num_read_kb":4217,"num_write":206,"num_write_kb":4775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"53'358","reported_seq":2457,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.128937+0000","last_change":"2023-10-06T04:05:02.724113+0000","last_active":"2023-10-06T04:10:55.128937+0000","last_peered":"2023-10-06T04:10:55.128937+0000","last_clean":"2023-10-06T04:10:55.128937+0000","last_became_active":"2023-10-06T03:57:11.363830+0000","last_became_peered":"2023-10-06T03:57:11.363830+0000","last_unstale":"2023-10-06T04:10:55.128937+0000","last_undegraded":"2023-10-06T04:10:55.128937+0000","last_fullsized":"2023-10-06T04:10:55.128937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:04.084019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027698559999999998,"stat_sum":{"num_bytes":22869017,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":32,"num_read_kb":4115,"num_write":333,"num_write_kb":26610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"53'106","reported_seq":156,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:56.861116+0000","last_change":"2023-10-06T03:57:10.349421+0000","last_active":"2023-10-06T04:10:56.861116+0000","last_peered":"2023-10-06T04:10:56.861116+0000","last_clean":"2023-10-06T04:10:56.861116+0000","last_became_active":"2023-10-06T03:57:10.349114+0000","last_became_peered":"2023-10-06T03:57:10.349114+0000","last_unstale":"2023-10-06T04:10:56.861116+0000","last_undegraded":"2023-10-06T04:10:56.861116+0000","last_fullsized":"2023-10-06T04:10:56.861116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:08.209791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":19,"num_read_kb":18,"num_write":109,"num_write_kb":4188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"53'291","reported_seq":2383,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.128027+0000","last_change":"2023-10-06T04:05:02.724241+0000","last_active":"2023-10-06T04:10:55.128027+0000","last_peered":"2023-10-06T04:10:55.128027+0000","last_clean":"2023-10-06T04:10:55.128027+0000","last_became_active":"2023-10-06T03:57:11.367257+0000","last_became_peered":"2023-10-06T03:57:11.367257+0000","last_unstale":"2023-10-06T04:10:55.128027+0000","last_undegraded":"2023-10-06T04:10:55.128027+0000","last_fullsized":"2023-10-06T04:10:55.128027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:43.628819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002803224,"stat_sum":{"num_bytes":18486267,"num_objects":142,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":26,"num_read_kb":4106,"num_write":272,"num_write_kb":22293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"53'53","reported_seq":100,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:52.265701+0000","last_change":"2023-10-06T03:57:10.347649+0000","last_active":"2023-10-06T04:09:52.265701+0000","last_peered":"2023-10-06T04:09:52.265701+0000","last_clean":"2023-10-06T04:09:52.265701+0000","last_became_active":"2023-10-06T03:57:10.347433+0000","last_became_peered":"2023-10-06T03:57:10.347433+0000","last_unstale":"2023-10-06T04:09:52.265701+0000","last_undegraded":"2023-10-06T04:09:52.265701+0000","last_fullsized":"2023-10-06T04:09:52.265701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:55.951494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":11,"num_read_kb":8201,"num_write":54,"num_write_kb":4137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"53'338","reported_seq":2439,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.142199+0000","last_change":"2023-10-06T04:05:02.718759+0000","last_active":"2023-10-06T04:10:55.142199+0000","last_peered":"2023-10-06T04:10:55.142199+0000","last_clean":"2023-10-06T04:10:55.142199+0000","last_became_active":"2023-10-06T03:57:11.369825+0000","last_became_peered":"2023-10-06T03:57:11.369825+0000","last_unstale":"2023-10-06T04:10:55.142199+0000","last_undegraded":"2023-10-06T04:10:55.142199+0000","last_fullsized":"2023-10-06T04:10:55.142199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:43.415929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014204689999999999,"stat_sum":{"num_bytes":11391239,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":35,"num_read_kb":8210,"num_write":309,"num_write_kb":15384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:37.659432+0000","last_change":"2023-10-06T03:57:05.663312+0000","last_active":"2023-10-06T04:08:37.659432+0000","last_peered":"2023-10-06T04:08:37.659432+0000","last_clean":"2023-10-06T04:08:37.659432+0000","last_became_active":"2023-10-06T03:57:05.663046+0000","last_became_peered":"2023-10-06T03:57:05.663046+0000","last_unstale":"2023-10-06T04:08:37.659432+0000","last_undegraded":"2023-10-06T04:08:37.659432+0000","last_fullsized":"2023-10-06T04:08:37.659432+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:31.174359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"53'38","reported_seq":85,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.051899+0000","last_change":"2023-10-06T03:57:10.348930+0000","last_active":"2023-10-06T04:10:55.051899+0000","last_peered":"2023-10-06T04:10:55.051899+0000","last_clean":"2023-10-06T04:10:55.051899+0000","last_became_active":"2023-10-06T03:57:10.348608+0000","last_became_peered":"2023-10-06T03:57:10.348608+0000","last_unstale":"2023-10-06T04:10:55.051899+0000","last_undegraded":"2023-10-06T04:10:55.051899+0000","last_fullsized":"2023-10-06T04:10:55.051899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:00:09.283718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194416,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":39,"num_write_kb":4139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"53'310","reported_seq":2406,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.130362+0000","last_change":"2023-10-06T04:05:02.718555+0000","last_active":"2023-10-06T04:10:55.130362+0000","last_peered":"2023-10-06T04:10:55.130362+0000","last_clean":"2023-10-06T04:10:55.130362+0000","last_became_active":"2023-10-06T03:57:11.370847+0000","last_became_peered":"2023-10-06T03:57:11.370847+0000","last_unstale":"2023-10-06T04:10:55.130362+0000","last_undegraded":"2023-10-06T04:10:55.130362+0000","last_fullsized":"2023-10-06T04:10:55.130362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:38.111171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013685990000000001,"stat_sum":{"num_bytes":14926709,"num_objects":139,"num_object_clones":0,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":0,"num_read":29,"num_read_kb":12301,"num_write":289,"num_write_kb":18822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:37.658819+0000","last_change":"2023-10-06T03:57:05.318834+0000","last_active":"2023-10-06T04:08:37.658819+0000","last_peered":"2023-10-06T04:08:37.658819+0000","last_clean":"2023-10-06T04:08:37.658819+0000","last_became_active":"2023-10-06T03:57:05.318572+0000","last_became_peered":"2023-10-06T03:57:05.318572+0000","last_unstale":"2023-10-06T04:08:37.658819+0000","last_undegraded":"2023-10-06T04:08:37.658819+0000","last_fullsized":"2023-10-06T04:08:37.658819+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:21.951095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"53'10","reported_seq":60,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:46.769558+0000","last_change":"2023-10-06T03:57:10.348556+0000","last_active":"2023-10-06T04:09:46.769558+0000","last_peered":"2023-10-06T04:09:46.769558+0000","last_clean":"2023-10-06T04:09:46.769558+0000","last_became_active":"2023-10-06T03:57:10.348271+0000","last_became_peered":"2023-10-06T03:57:10.348271+0000","last_unstale":"2023-10-06T04:09:46.769558+0000","last_undegraded":"2023-10-06T04:09:46.769558+0000","last_fullsized":"2023-10-06T04:09:46.769558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:58.470989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":26,"num_write":12,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"53'349","reported_seq":2449,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.125846+0000","last_change":"2023-10-06T04:05:02.719009+0000","last_active":"2023-10-06T04:10:55.125846+0000","last_peered":"2023-10-06T04:10:55.125846+0000","last_clean":"2023-10-06T04:10:55.125846+0000","last_became_active":"2023-10-06T03:57:11.365006+0000","last_became_peered":"2023-10-06T03:57:11.365006+0000","last_unstale":"2023-10-06T04:10:55.125846+0000","last_undegraded":"2023-10-06T04:10:55.125846+0000","last_fullsized":"2023-10-06T04:10:55.125846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:33.005015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015961910000000001,"stat_sum":{"num_bytes":5739830,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":33,"num_read_kb":4109,"num_write":322,"num_write_kb":5775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:37.658115+0000","last_change":"2023-10-06T03:57:05.314695+0000","last_active":"2023-10-06T04:08:37.658115+0000","last_peered":"2023-10-06T04:08:37.658115+0000","last_clean":"2023-10-06T04:08:37.658115+0000","last_became_active":"2023-10-06T03:57:05.314408+0000","last_became_peered":"2023-10-06T03:57:05.314408+0000","last_unstale":"2023-10-06T04:08:37.658115+0000","last_undegraded":"2023-10-06T04:08:37.658115+0000","last_fullsized":"2023-10-06T04:08:37.658115+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:19.042145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"53'359","reported_seq":405,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:16.606829+0000","last_change":"2023-10-06T03:57:10.346078+0000","last_active":"2023-10-06T04:10:16.606829+0000","last_peered":"2023-10-06T04:10:16.606829+0000","last_clean":"2023-10-06T04:10:16.606829+0000","last_became_active":"2023-10-06T03:57:10.345769+0000","last_became_peered":"2023-10-06T03:57:10.345769+0000","last_unstale":"2023-10-06T04:10:16.606829+0000","last_undegraded":"2023-10-06T04:10:16.606829+0000","last_fullsized":"2023-10-06T04:10:16.606829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:16.630681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":352,"num_write_kb":12528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"53'331","reported_seq":2437,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.129401+0000","last_change":"2023-10-06T04:05:02.718334+0000","last_active":"2023-10-06T04:10:55.129401+0000","last_peered":"2023-10-06T04:10:55.129401+0000","last_clean":"2023-10-06T04:10:55.129401+0000","last_became_active":"2023-10-06T03:57:11.370076+0000","last_became_peered":"2023-10-06T03:57:11.370076+0000","last_unstale":"2023-10-06T04:10:55.129401+0000","last_undegraded":"2023-10-06T04:10:55.129401+0000","last_fullsized":"2023-10-06T04:10:55.129401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:18:59.517155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011772849999999999,"stat_sum":{"num_bytes":18548829,"num_objects":146,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":146,"num_whiteouts":0,"num_read":39,"num_read_kb":4112,"num_write":310,"num_write_kb":22369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:37.658686+0000","last_change":"2023-10-06T03:57:05.318778+0000","last_active":"2023-10-06T04:08:37.658686+0000","last_peered":"2023-10-06T04:08:37.658686+0000","last_clean":"2023-10-06T04:08:37.658686+0000","last_became_active":"2023-10-06T03:57:05.318456+0000","last_became_peered":"2023-10-06T03:57:05.318456+0000","last_unstale":"2023-10-06T04:08:37.658686+0000","last_undegraded":"2023-10-06T04:08:37.658686+0000","last_fullsized":"2023-10-06T04:08:37.658686+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:39.769241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":195,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:36.756245+0000","last_change":"2023-10-06T03:57:05.664245+0000","last_active":"2023-10-06T04:08:36.756245+0000","last_peered":"2023-10-06T04:08:36.756245+0000","last_clean":"2023-10-06T04:08:36.756245+0000","last_became_active":"2023-10-06T03:57:05.663426+0000","last_became_peered":"2023-10-06T03:57:05.663426+0000","last_unstale":"2023-10-06T04:08:36.756245+0000","last_undegraded":"2023-10-06T04:08:36.756245+0000","last_fullsized":"2023-10-06T04:08:36.756245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:29.322599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"53'719","reported_seq":770,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:52.436682+0000","last_change":"2023-10-06T03:57:10.351059+0000","last_active":"2023-10-06T04:09:52.436682+0000","last_peered":"2023-10-06T04:09:52.436682+0000","last_clean":"2023-10-06T04:09:52.436682+0000","last_became_active":"2023-10-06T03:57:10.348879+0000","last_became_peered":"2023-10-06T03:57:10.348879+0000","last_unstale":"2023-10-06T04:09:52.436682+0000","last_undegraded":"2023-10-06T04:09:52.436682+0000","last_fullsized":"2023-10-06T04:09:52.436682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:35.764148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9487985,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":20487,"num_write":715,"num_write_kb":13783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"53'349","reported_seq":2452,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.124152+0000","last_change":"2023-10-06T04:05:03.542987+0000","last_active":"2023-10-06T04:10:55.124152+0000","last_peered":"2023-10-06T04:10:55.124152+0000","last_clean":"2023-10-06T04:10:55.124152+0000","last_became_active":"2023-10-06T03:57:11.373452+0000","last_became_peered":"2023-10-06T03:57:11.373452+0000","last_unstale":"2023-10-06T04:10:55.124152+0000","last_undegraded":"2023-10-06T04:10:55.124152+0000","last_fullsized":"2023-10-06T04:10:55.124152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:20.524065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018153620000000001,"stat_sum":{"num_bytes":5937326,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":36,"num_read_kb":2733,"num_write":331,"num_write_kb":10073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:36.756325+0000","last_change":"2023-10-06T03:57:05.318790+0000","last_active":"2023-10-06T04:08:36.756325+0000","last_peered":"2023-10-06T04:08:36.756325+0000","last_clean":"2023-10-06T04:08:36.756325+0000","last_became_active":"2023-10-06T03:57:05.318531+0000","last_became_peered":"2023-10-06T03:57:05.318531+0000","last_unstale":"2023-10-06T04:08:36.756325+0000","last_undegraded":"2023-10-06T04:08:36.756325+0000","last_fullsized":"2023-10-06T04:08:36.756325+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:12.876998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"52'540","reported_seq":703,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:46.766101+0000","last_change":"2023-10-06T03:57:10.343328+0000","last_active":"2023-10-06T04:09:46.766101+0000","last_peered":"2023-10-06T04:09:46.766101+0000","last_clean":"2023-10-06T04:09:46.766101+0000","last_became_active":"2023-10-06T03:57:10.343051+0000","last_became_peered":"2023-10-06T03:57:10.343051+0000","last_unstale":"2023-10-06T04:09:46.766101+0000","last_undegraded":"2023-10-06T04:09:46.766101+0000","last_fullsized":"2023-10-06T04:09:46.766101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:27.722475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4571457,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":122,"num_read_kb":3989,"num_write":534,"num_write_kb":4895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"53'352","reported_seq":2456,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.135325+0000","last_change":"2023-10-06T04:05:03.367824+0000","last_active":"2023-10-06T04:10:55.135325+0000","last_peered":"2023-10-06T04:10:55.135325+0000","last_clean":"2023-10-06T04:10:55.135325+0000","last_became_active":"2023-10-06T03:57:11.373264+0000","last_became_peered":"2023-10-06T03:57:11.373264+0000","last_unstale":"2023-10-06T04:10:55.135325+0000","last_undegraded":"2023-10-06T04:10:55.135325+0000","last_fullsized":"2023-10-06T04:10:55.135325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:15.950245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026908990000000001,"stat_sum":{"num_bytes":10658865,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":37,"num_read_kb":4121,"num_write":334,"num_write_kb":10588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:36.758524+0000","last_change":"2023-10-06T03:57:05.663770+0000","last_active":"2023-10-06T04:08:36.758524+0000","last_peered":"2023-10-06T04:08:36.758524+0000","last_clean":"2023-10-06T04:08:36.758524+0000","last_became_active":"2023-10-06T03:57:05.663501+0000","last_became_peered":"2023-10-06T03:57:05.663501+0000","last_unstale":"2023-10-06T04:08:36.758524+0000","last_undegraded":"2023-10-06T04:08:36.758524+0000","last_fullsized":"2023-10-06T04:08:36.758524+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:09.806915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"52'5","reported_seq":47,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:09:30.961755+0000","last_change":"2023-10-06T03:57:10.346585+0000","last_active":"2023-10-06T04:09:30.961755+0000","last_peered":"2023-10-06T04:09:30.961755+0000","last_clean":"2023-10-06T04:09:30.961755+0000","last_became_active":"2023-10-06T03:57:10.345743+0000","last_became_peered":"2023-10-06T03:57:10.345743+0000","last_unstale":"2023-10-06T04:09:30.961755+0000","last_undegraded":"2023-10-06T04:09:30.961755+0000","last_fullsized":"2023-10-06T04:09:30.961755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:54.589702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20578,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"53'365","reported_seq":2459,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.134869+0000","last_change":"2023-10-06T04:05:03.366736+0000","last_active":"2023-10-06T04:10:55.134869+0000","last_peered":"2023-10-06T04:10:55.134869+0000","last_clean":"2023-10-06T04:10:55.134869+0000","last_became_active":"2023-10-06T03:57:11.373543+0000","last_became_peered":"2023-10-06T03:57:11.373543+0000","last_unstale":"2023-10-06T04:10:55.134869+0000","last_undegraded":"2023-10-06T04:10:55.134869+0000","last_fullsized":"2023-10-06T04:10:55.134869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:15.152354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020558070000000002,"stat_sum":{"num_bytes":6540215,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":27,"num_read_kb":14,"num_write":340,"num_write_kb":6519,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"53'14","reported_seq":63,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:36.203548+0000","last_change":"2023-10-06T03:57:10.344618+0000","last_active":"2023-10-06T04:10:36.203548+0000","last_peered":"2023-10-06T04:10:36.203548+0000","last_clean":"2023-10-06T04:10:36.203548+0000","last_became_active":"2023-10-06T03:57:10.344324+0000","last_became_peered":"2023-10-06T03:57:10.344324+0000","last_unstale":"2023-10-06T04:10:36.203548+0000","last_undegraded":"2023-10-06T04:10:36.203548+0000","last_fullsized":"2023-10-06T04:10:36.203548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:06.062331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":22,"num_read_kb":21,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:37.659459+0000","last_change":"2023-10-06T03:57:05.317973+0000","last_active":"2023-10-06T04:08:37.659459+0000","last_peered":"2023-10-06T04:08:37.659459+0000","last_clean":"2023-10-06T04:08:37.659459+0000","last_became_active":"2023-10-06T03:57:05.317715+0000","last_became_peered":"2023-10-06T03:57:05.317715+0000","last_unstale":"2023-10-06T04:08:37.659459+0000","last_undegraded":"2023-10-06T04:08:37.659459+0000","last_fullsized":"2023-10-06T04:08:37.659459+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:37:51.858909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"53'331","reported_seq":2430,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.145918+0000","last_change":"2023-10-06T04:05:02.721255+0000","last_active":"2023-10-06T04:10:55.145918+0000","last_peered":"2023-10-06T04:10:55.145918+0000","last_clean":"2023-10-06T04:10:55.145918+0000","last_became_active":"2023-10-06T03:57:11.369445+0000","last_became_peered":"2023-10-06T03:57:11.369445+0000","last_unstale":"2023-10-06T04:10:55.145918+0000","last_undegraded":"2023-10-06T04:10:55.145918+0000","last_fullsized":"2023-10-06T04:10:55.145918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:52.999421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015173949999999999,"stat_sum":{"num_bytes":11743957,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":32,"num_read_kb":17,"num_write":311,"num_write_kb":11603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":45,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T04:08:37.573513+0000","last_change":"2023-10-06T03:57:05.320847+0000","last_active":"2023-10-06T04:08:37.573513+0000","last_peered":"2023-10-06T04:08:37.573513+0000","last_clean":"2023-10-06T04:08:37.573513+0000","last_became_active":"2023-10-06T03:57:05.320583+0000","last_became_peered":"2023-10-06T03:57:05.320583+0000","last_unstale":"2023-10-06T04:08:37.573513+0000","last_undegraded":"2023-10-06T04:08:37.573513+0000","last_fullsized":"2023-10-06T04:08:37.573513+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:04.302319+0000","last_clean_scrub_stamp":"2023-10-06T03:57:04.302319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:12.772449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"53'93","reported_seq":137,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:51.294477+0000","last_change":"2023-10-06T03:57:10.341132+0000","last_active":"2023-10-06T04:10:51.294477+0000","last_peered":"2023-10-06T04:10:51.294477+0000","last_clean":"2023-10-06T04:10:51.294477+0000","last_became_active":"2023-10-06T03:57:10.339424+0000","last_became_peered":"2023-10-06T03:57:10.339424+0000","last_unstale":"2023-10-06T04:10:51.294477+0000","last_undegraded":"2023-10-06T04:10:51.294477+0000","last_fullsized":"2023-10-06T04:10:51.294477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:39.675711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":19,"num_write":221,"num_write_kb":1011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"53'335","reported_seq":2430,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.121289+0000","last_change":"2023-10-06T04:05:02.722365+0000","last_active":"2023-10-06T04:10:55.121289+0000","last_peered":"2023-10-06T04:10:55.121289+0000","last_clean":"2023-10-06T04:10:55.121289+0000","last_became_active":"2023-10-06T03:57:11.372215+0000","last_became_peered":"2023-10-06T03:57:11.372215+0000","last_unstale":"2023-10-06T04:10:55.121289+0000","last_undegraded":"2023-10-06T04:10:55.121289+0000","last_fullsized":"2023-10-06T04:10:55.121289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:23.559480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020199100000000002,"stat_sum":{"num_bytes":14603524,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":28,"num_read_kb":4106,"num_write":312,"num_write_kb":14355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"53'204","reported_seq":257,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:44.658005+0000","last_change":"2023-10-06T03:57:10.344051+0000","last_active":"2023-10-06T04:10:44.658005+0000","last_peered":"2023-10-06T04:10:44.658005+0000","last_clean":"2023-10-06T04:10:44.658005+0000","last_became_active":"2023-10-06T03:57:10.343865+0000","last_became_peered":"2023-10-06T03:57:10.343865+0000","last_unstale":"2023-10-06T04:10:44.658005+0000","last_undegraded":"2023-10-06T04:10:44.658005+0000","last_fullsized":"2023-10-06T04:10:44.658005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:32.522470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":11,"num_read_kb":32771,"num_write":206,"num_write_kb":12488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"53'357","reported_seq":2457,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.123318+0000","last_change":"2023-10-06T04:05:03.151764+0000","last_active":"2023-10-06T04:10:55.123318+0000","last_peered":"2023-10-06T04:10:55.123318+0000","last_clean":"2023-10-06T04:10:55.123318+0000","last_became_active":"2023-10-06T03:57:11.369681+0000","last_became_peered":"2023-10-06T03:57:11.369681+0000","last_unstale":"2023-10-06T04:10:55.123318+0000","last_undegraded":"2023-10-06T04:10:55.123318+0000","last_fullsized":"2023-10-06T04:10:55.123318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:53.858179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013151409999999999,"stat_sum":{"num_bytes":9655954,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":33,"num_read_kb":14,"num_write":334,"num_write_kb":13707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"50'100","reported_seq":149,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:52.634119+0000","last_change":"2023-10-06T03:57:10.349666+0000","last_active":"2023-10-06T04:09:52.634119+0000","last_peered":"2023-10-06T04:09:52.634119+0000","last_clean":"2023-10-06T04:09:52.634119+0000","last_became_active":"2023-10-06T03:57:10.349320+0000","last_became_peered":"2023-10-06T03:57:10.349320+0000","last_unstale":"2023-10-06T04:09:52.634119+0000","last_undegraded":"2023-10-06T04:09:52.634119+0000","last_fullsized":"2023-10-06T04:09:52.634119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:45:49.709519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194858,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":4110,"num_write":99,"num_write_kb":4143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"53'342","reported_seq":2436,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.141134+0000","last_change":"2023-10-06T04:05:02.723622+0000","last_active":"2023-10-06T04:10:55.141134+0000","last_peered":"2023-10-06T04:10:55.141134+0000","last_clean":"2023-10-06T04:10:55.141134+0000","last_became_active":"2023-10-06T03:57:11.370229+0000","last_became_peered":"2023-10-06T03:57:11.370229+0000","last_unstale":"2023-10-06T04:10:55.141134+0000","last_undegraded":"2023-10-06T04:10:55.141134+0000","last_fullsized":"2023-10-06T04:10:55.141134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:26.174331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027851170000000001,"stat_sum":{"num_bytes":11865278,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":27,"num_read_kb":17,"num_write":315,"num_write_kb":11749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"53'238","reported_seq":299,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:23.094832+0000","last_change":"2023-10-06T03:57:10.350263+0000","last_active":"2023-10-06T04:10:23.094832+0000","last_peered":"2023-10-06T04:10:23.094832+0000","last_clean":"2023-10-06T04:10:23.094832+0000","last_became_active":"2023-10-06T03:57:10.350083+0000","last_became_peered":"2023-10-06T03:57:10.350083+0000","last_unstale":"2023-10-06T04:10:23.094832+0000","last_undegraded":"2023-10-06T04:10:23.094832+0000","last_fullsized":"2023-10-06T04:10:23.094832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:05.433032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":36,"num_read_kb":45128,"num_write":264,"num_write_kb":12512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"53'350","reported_seq":2447,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.140477+0000","last_change":"2023-10-06T04:05:02.724048+0000","last_active":"2023-10-06T04:10:55.140477+0000","last_peered":"2023-10-06T04:10:55.140477+0000","last_clean":"2023-10-06T04:10:55.140477+0000","last_became_active":"2023-10-06T03:57:11.369123+0000","last_became_peered":"2023-10-06T03:57:11.369123+0000","last_unstale":"2023-10-06T04:10:55.140477+0000","last_undegraded":"2023-10-06T04:10:55.140477+0000","last_fullsized":"2023-10-06T04:10:55.140477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:28.139036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028613789999999998,"stat_sum":{"num_bytes":15066743,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":30,"num_read_kb":4112,"num_write":322,"num_write_kb":14878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"52'181","reported_seq":225,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:47.407717+0000","last_change":"2023-10-06T03:57:10.347049+0000","last_active":"2023-10-06T04:09:47.407717+0000","last_peered":"2023-10-06T04:09:47.407717+0000","last_clean":"2023-10-06T04:09:47.407717+0000","last_became_active":"2023-10-06T03:57:10.344583+0000","last_became_peered":"2023-10-06T03:57:10.344583+0000","last_unstale":"2023-10-06T04:09:47.407717+0000","last_undegraded":"2023-10-06T04:09:47.407717+0000","last_fullsized":"2023-10-06T04:09:47.407717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:32.280257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":177,"num_write_kb":8286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"53'352","reported_seq":2456,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.144798+0000","last_change":"2023-10-06T04:05:02.721502+0000","last_active":"2023-10-06T04:10:55.144798+0000","last_peered":"2023-10-06T04:10:55.144798+0000","last_clean":"2023-10-06T04:10:55.144798+0000","last_became_active":"2023-10-06T03:57:11.366500+0000","last_became_peered":"2023-10-06T03:57:11.366500+0000","last_unstale":"2023-10-06T04:10:55.144798+0000","last_undegraded":"2023-10-06T04:10:55.144798+0000","last_fullsized":"2023-10-06T04:10:55.144798+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:24.714603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016878310000000001,"stat_sum":{"num_bytes":6166988,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":38,"num_read_kb":4113,"num_write":326,"num_write_kb":10290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"50'170","reported_seq":220,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:09:51.778904+0000","last_change":"2023-10-06T03:57:10.348170+0000","last_active":"2023-10-06T04:09:51.778904+0000","last_peered":"2023-10-06T04:09:51.778904+0000","last_clean":"2023-10-06T04:09:51.778904+0000","last_became_active":"2023-10-06T03:57:10.347897+0000","last_became_peered":"2023-10-06T03:57:10.347897+0000","last_unstale":"2023-10-06T04:09:51.778904+0000","last_undegraded":"2023-10-06T04:09:51.778904+0000","last_fullsized":"2023-10-06T04:09:51.778904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:40.756246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7191702,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":12,"num_read_kb":8202,"num_write":168,"num_write_kb":7111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"53'410","reported_seq":2511,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.131095+0000","last_change":"2023-10-06T04:05:02.721899+0000","last_active":"2023-10-06T04:10:55.131095+0000","last_peered":"2023-10-06T04:10:55.131095+0000","last_clean":"2023-10-06T04:10:55.131095+0000","last_became_active":"2023-10-06T03:57:11.364150+0000","last_became_peered":"2023-10-06T03:57:11.364150+0000","last_unstale":"2023-10-06T04:10:55.131095+0000","last_undegraded":"2023-10-06T04:10:55.131095+0000","last_fullsized":"2023-10-06T04:10:55.131095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:06:11.038080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001670408,"stat_sum":{"num_bytes":10944544,"num_objects":176,"num_object_clones":0,"num_object_copies":352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":176,"num_whiteouts":0,"num_read":35,"num_read_kb":7178,"num_write":386,"num_write_kb":19084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"53'363","reported_seq":2484,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.125070+0000","last_change":"2023-10-06T04:05:03.542896+0000","last_active":"2023-10-06T04:10:55.125070+0000","last_peered":"2023-10-06T04:10:55.125070+0000","last_clean":"2023-10-06T04:10:55.125070+0000","last_became_active":"2023-10-06T03:57:11.366524+0000","last_became_peered":"2023-10-06T03:57:11.366524+0000","last_unstale":"2023-10-06T04:10:55.125070+0000","last_undegraded":"2023-10-06T04:10:55.125070+0000","last_fullsized":"2023-10-06T04:10:55.125070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:13:33.044858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017816290000000001,"stat_sum":{"num_bytes":22594948,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":54,"num_read_kb":16279,"num_write":343,"num_write_kb":22243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"53'380","reported_seq":2484,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.144038+0000","last_change":"2023-10-06T04:05:02.721559+0000","last_active":"2023-10-06T04:10:55.144038+0000","last_peered":"2023-10-06T04:10:55.144038+0000","last_clean":"2023-10-06T04:10:55.144038+0000","last_became_active":"2023-10-06T03:57:11.370848+0000","last_became_peered":"2023-10-06T03:57:11.370848+0000","last_unstale":"2023-10-06T04:10:55.144038+0000","last_undegraded":"2023-10-06T04:10:55.144038+0000","last_fullsized":"2023-10-06T04:10:55.144038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:15.919090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017239289999999999,"stat_sum":{"num_bytes":12161208,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":37,"num_read_kb":8210,"num_write":347,"num_write_kb":16110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"53'371","reported_seq":2477,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.120689+0000","last_change":"2023-10-06T04:05:02.723715+0000","last_active":"2023-10-06T04:10:55.120689+0000","last_peered":"2023-10-06T04:10:55.120689+0000","last_clean":"2023-10-06T04:10:55.120689+0000","last_became_active":"2023-10-06T03:57:11.363875+0000","last_became_peered":"2023-10-06T03:57:11.363875+0000","last_unstale":"2023-10-06T04:10:55.120689+0000","last_undegraded":"2023-10-06T04:10:55.120689+0000","last_fullsized":"2023-10-06T04:10:55.120689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:29.236344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028142470000000002,"stat_sum":{"num_bytes":14948732,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":39,"num_read_kb":4117,"num_write":348,"num_write_kb":14761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"53'357","reported_seq":2465,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.143994+0000","last_change":"2023-10-06T04:05:02.722555+0000","last_active":"2023-10-06T04:10:55.143994+0000","last_peered":"2023-10-06T04:10:55.143994+0000","last_clean":"2023-10-06T04:10:55.143994+0000","last_became_active":"2023-10-06T03:57:11.377126+0000","last_became_peered":"2023-10-06T03:57:11.377126+0000","last_unstale":"2023-10-06T04:10:55.143994+0000","last_undegraded":"2023-10-06T04:10:55.143994+0000","last_fullsized":"2023-10-06T04:10:55.143994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:05.041737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023666569999999999,"stat_sum":{"num_bytes":6085663,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":41,"num_read_kb":18,"num_write":335,"num_write_kb":6110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"53'326","reported_seq":2423,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.124012+0000","last_change":"2023-10-06T04:05:02.972860+0000","last_active":"2023-10-06T04:10:55.124012+0000","last_peered":"2023-10-06T04:10:55.124012+0000","last_clean":"2023-10-06T04:10:55.124012+0000","last_became_active":"2023-10-06T03:57:59.256086+0000","last_became_peered":"2023-10-06T03:57:59.256086+0000","last_unstale":"2023-10-06T04:10:55.124012+0000","last_undegraded":"2023-10-06T04:10:55.124012+0000","last_fullsized":"2023-10-06T04:10:55.124012+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:54.263909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018573859999999999,"stat_sum":{"num_bytes":11350301,"num_objects":142,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":33,"num_read_kb":11479,"num_write":309,"num_write_kb":14473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":923606083,"num_objects":10287,"num_object_clones":0,"num_object_copies":20574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10287,"num_whiteouts":0,"num_read":2243,"num_read_kb":380500,"num_write":20866,"num_write_kb":1045904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1683152896,"data_stored":1845672070,"data_compressed":160523048,"data_compressed_allocated":177569792,"data_compressed_original":393674752,"omap_allocated":0,"internal_metadata":0},"log_size":22440,"ondisk_log_size":22440,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":166884517,"num_objects":202,"num_object_clones":0,"num_object_copies":404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":202,"num_whiteouts":0,"num_read":555,"num_read_kb":254049,"num_write":6892,"num_write_kb":197784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":122,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":83959808,"data_stored":333769034,"data_compressed":44919486,"data_compressed_allocated":56524800,"data_compressed_original":306585600,"omap_allocated":0,"internal_metadata":0},"log_size":6639,"ondisk_log_size":6639,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64858,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575019,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506960,"kb_used_data":209672,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93864880,"statfs":{"total":96636764160,"available":96117637120,"internally_reserved":0,"allocated":214704128,"data_stored":259573340,"data_compressed":22277630,"data_compressed_allocated":25305088,"data_compressed_original":76083200,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.55400000000000005,"15min":0.54500000000000004},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28399999999999997},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.88900000000000001},"last":0.47399999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52200000000000002,"15min":0.52100000000000002},"min":{"1min":0.38400000000000001,"5min":0.312,"15min":0.28899999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.78000000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54300000000000004,"15min":0.52300000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78300000000000003,"5min":1.25,"15min":1.25},"last":0.58999999999999997}]},{"osd":2,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54800000000000004,"15min":0.54900000000000004},"min":{"1min":0.33400000000000002,"5min":0.29799999999999999,"15min":0.28399999999999997},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.438},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57599999999999996,"15min":0.60699999999999998},"min":{"1min":0.34300000000000003,"5min":0.33700000000000002,"15min":0.33100000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":7.3739999999999997},"last":0.57499999999999996}]},{"osd":3,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.318},"max":{"1min":0.80600000000000005,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.55400000000000005,"15min":0.55900000000000005},"min":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72899999999999998,"5min":0.78000000000000003,"15min":0.85299999999999998},"last":0.62}]},{"osd":4,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.44,"15min":0.46400000000000002},"min":{"1min":0.30099999999999999,"5min":0.27400000000000002,"15min":0.251},"max":{"1min":0.64000000000000001,"5min":0.73599999999999999,"15min":1.0149999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.34300000000000003,"5min":0.33500000000000002,"15min":0.28299999999999997},"max":{"1min":0.68600000000000005,"5min":0.80200000000000005,"15min":0.90700000000000003},"last":0.60499999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.495,"15min":0.51100000000000001},"min":{"1min":0.316,"5min":0.29999999999999999,"15min":0.27300000000000002},"max":{"1min":0.74199999999999999,"5min":0.82999999999999996,"15min":0.86099999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54600000000000004,"15min":0.56299999999999994},"min":{"1min":0.40699999999999997,"5min":0.317,"15min":0.28000000000000003},"max":{"1min":0.73699999999999999,"5min":0.84899999999999998,"15min":1.377},"last":0.64500000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.51100000000000001,"15min":0.56200000000000006},"min":{"1min":0.34100000000000003,"5min":0.30599999999999999,"15min":0.29999999999999999},"max":{"1min":0.77100000000000002,"5min":0.77600000000000002,"15min":1.3420000000000001},"last":0.628},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60499999999999998,"15min":0.59099999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.30199999999999999},"max":{"1min":0.84599999999999997,"5min":0.88200000000000001,"15min":0.93700000000000006},"last":0.67300000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575018,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":474560,"kb_used_data":164984,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":93897280,"statfs":{"total":96636764160,"available":96150814720,"internally_reserved":0,"allocated":168943616,"data_stored":188218692,"data_compressed":15936082,"data_compressed_allocated":17805312,"data_compressed_original":42737664,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54800000000000004,"15min":0.53200000000000003},"min":{"1min":0.28799999999999998,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.70399999999999996,"5min":2.613,"15min":2.613},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.58399999999999996,"15min":0.55800000000000005},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.33300000000000002},"max":{"1min":0.72299999999999998,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":0.44}]},{"osd":1,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53400000000000003,"15min":0.59599999999999997},"min":{"1min":0.28899999999999998,"5min":0.28499999999999998,"15min":0.27200000000000002},"max":{"1min":0.71599999999999997,"5min":2.3889999999999998,"15min":9.5519999999999996},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.59399999999999997,"15min":0.60199999999999998},"min":{"1min":0.33400000000000002,"5min":0.315,"15min":0.313},"max":{"1min":0.75800000000000001,"5min":6.8019999999999996,"15min":6.8019999999999996},"last":0.42899999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.625,"15min":0.67300000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.29799999999999999},"max":{"1min":0.71799999999999997,"5min":5.7590000000000003,"15min":9.6769999999999996},"last":0.495},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.623,"15min":0.67600000000000005},"min":{"1min":0.42199999999999999,"5min":0.33200000000000002,"15min":0.31},"max":{"1min":0.75,"5min":5.71,"15min":9.7309999999999999},"last":0.46700000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.54900000000000004,"15min":0.57499999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.29299999999999998},"max":{"1min":0.73899999999999999,"5min":0.99099999999999999,"15min":5.7359999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.59699999999999998,"15min":0.61299999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.308},"max":{"1min":0.68799999999999994,"5min":1.0149999999999999,"15min":5.7009999999999996},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.48099999999999998,"15min":0.49399999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.63300000000000001,"5min":0.78600000000000003,"15min":1.0129999999999999},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54100000000000004,"15min":0.54900000000000004},"min":{"1min":0.32600000000000001,"5min":0.313,"15min":0.30399999999999999},"max":{"1min":0.70299999999999996,"5min":0.79800000000000004,"15min":1.026},"last":0.45400000000000001}]},{"osd":5,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.49299999999999999,"15min":0.50800000000000001},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.77800000000000002,"5min":0.78200000000000003,"15min":0.995},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.66500000000000004,"5min":0.78500000000000003,"15min":0.83699999999999997},"last":0.53900000000000003}]},{"osd":7,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.34899999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.77000000000000002,"5min":0.84199999999999997,"15min":0.98299999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56499999999999995,"15min":0.57799999999999996},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80500000000000005,"5min":0.84699999999999998,"15min":0.90800000000000003},"last":0.52000000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575018,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":578436,"kb_used_data":277052,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":93793404,"statfs":{"total":96636764160,"available":96044445696,"internally_reserved":0,"allocated":283701248,"data_stored":337243241,"data_compressed":32816762,"data_compressed_allocated":36933632,"data_compressed_original":99717120,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.91800000000000004,"15min":0.52000000000000002},"min":{"1min":0.22500000000000001,"5min":0.20799999999999999,"15min":0.192},"max":{"1min":0.56999999999999995,"5min":57.700000000000003,"15min":57.700000000000003},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.65200000000000002,"15min":0.441},"min":{"1min":0.23899999999999999,"5min":0.23300000000000001,"15min":0.186},"max":{"1min":0.64000000000000001,"5min":28.629999999999999,"15min":28.629999999999999},"last":0.28899999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.97899999999999998,"15min":0.57199999999999995},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.19600000000000001},"max":{"1min":0.59899999999999998,"5min":58.795000000000002,"15min":58.795000000000002},"last":0.315},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.67600000000000005,"15min":0.48999999999999999},"min":{"1min":0.32200000000000001,"5min":0.29599999999999999,"15min":0.23000000000000001},"max":{"1min":0.63400000000000001,"5min":25.722000000000001,"15min":25.722000000000001},"last":0.32800000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.59799999999999998,"15min":0.443},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.216},"max":{"1min":0.58499999999999996,"5min":22.920999999999999,"15min":22.920999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.63400000000000001,"15min":0.46800000000000003},"min":{"1min":0.29099999999999998,"5min":0.27300000000000002,"15min":0.22600000000000001},"max":{"1min":0.59799999999999998,"5min":22.797999999999998,"15min":22.797999999999998},"last":0.36799999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":1.2370000000000001,"15min":0.82399999999999995},"min":{"1min":0.443,"5min":0.38500000000000001,"15min":0.34499999999999997},"max":{"1min":1.0149999999999999,"5min":59.018999999999998,"15min":59.018999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":1.26,"15min":0.83499999999999996},"min":{"1min":0.50700000000000001,"5min":0.47399999999999998,"15min":0.316},"max":{"1min":1.0609999999999999,"5min":59.046999999999997,"15min":59.046999999999997},"last":0.52900000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":1.21,"15min":0.79300000000000004},"min":{"1min":0.48599999999999999,"5min":0.40200000000000002,"15min":0.34399999999999997},"max":{"1min":0.90300000000000002,"5min":58.987000000000002,"15min":58.987000000000002},"last":0.497},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65700000000000003,"15min":0.59599999999999997},"min":{"1min":0.44900000000000001,"5min":0.42599999999999999,"15min":0.29399999999999998},"max":{"1min":0.97699999999999998,"5min":6.2140000000000004,"15min":6.2140000000000004},"last":0.58899999999999997}]},{"osd":6,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65200000000000002,"15min":0.60399999999999998},"min":{"1min":0.44,"5min":0.41999999999999998,"15min":0.36599999999999999},"max":{"1min":0.95599999999999996,"5min":4.6020000000000003,"15min":4.6020000000000003},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":1.3,"15min":0.85999999999999999},"min":{"1min":0.49099999999999999,"5min":0.42799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94799999999999995,"5min":58.911999999999999,"15min":58.911999999999999},"last":0.57099999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":1.2210000000000001,"15min":0.81100000000000005},"min":{"1min":0.45800000000000002,"5min":0.40100000000000002,"15min":0.38700000000000001},"max":{"1min":1.1000000000000001,"5min":58.953000000000003,"15min":58.953000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.65900000000000003,"15min":0.62},"min":{"1min":0.46700000000000003,"5min":0.38100000000000001,"15min":0.34399999999999997},"max":{"1min":0.83199999999999996,"5min":4.5570000000000004,"15min":4.5570000000000004},"last":0.54800000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575018,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":496884,"kb_used_data":191404,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":93874956,"statfs":{"total":96636764160,"available":96127954944,"internally_reserved":0,"allocated":195997696,"data_stored":253566865,"data_compressed":26834878,"data_compressed_allocated":30744576,"data_compressed_original":94838784,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.34100000000000003,"15min":0.34899999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.19500000000000001},"max":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":6.9539999999999997},"last":0.442},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.35599999999999998,"15min":0.371},"min":{"1min":0.22700000000000001,"5min":0.215,"15min":0.191},"max":{"1min":0.55700000000000005,"5min":0.59499999999999997,"15min":8.0519999999999996},"last":0.36699999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.36199999999999999,"15min":0.35899999999999999},"min":{"1min":0.24299999999999999,"5min":0.24199999999999999,"15min":0.214},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":1.0589999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.38500000000000001,"15min":0.376},"min":{"1min":0.28299999999999997,"5min":0.26100000000000001,"15min":0.20799999999999999},"max":{"1min":0.626,"5min":0.626,"15min":2.0489999999999999},"last":0.42299999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.38600000000000001,"15min":0.41799999999999998},"min":{"1min":0.23300000000000001,"5min":0.23100000000000001,"15min":0.223},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":9.391},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.42599999999999999,"15min":0.45600000000000002},"min":{"1min":0.31900000000000001,"5min":0.29099999999999998,"15min":0.27800000000000002},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":9.5239999999999991},"last":0.51200000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.64900000000000002,"15min":0.63400000000000001},"min":{"1min":0.35599999999999998,"5min":0.34000000000000002,"15min":0.29199999999999998},"max":{"1min":1.026,"5min":1.026,"15min":5.7699999999999996},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66000000000000003,"15min":0.63600000000000001},"min":{"1min":0.45400000000000001,"5min":0.41499999999999998,"15min":0.29499999999999998},"max":{"1min":0.997,"5min":0.997,"15min":2.7320000000000002},"last":0.81699999999999995}]},{"osd":5,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.45000000000000001,"5min":0.41399999999999998,"15min":0.33800000000000002},"max":{"1min":0.89900000000000002,"5min":0.90500000000000003,"15min":5.5069999999999997},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63100000000000001,"15min":0.60499999999999998},"min":{"1min":0.436,"5min":0.38500000000000001,"15min":0.34799999999999998},"max":{"1min":0.872,"5min":0.878,"15min":1.05},"last":0.84399999999999997}]},{"osd":6,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.626,"15min":0.64700000000000002},"min":{"1min":0.42599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":6.9139999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64800000000000002,"15min":0.65500000000000003},"min":{"1min":0.46700000000000003,"5min":0.42199999999999999,"15min":0.379},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":6.3780000000000001},"last":0.44}]},{"osd":7,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61099999999999999,"15min":0.623},"min":{"1min":0.38500000000000001,"5min":0.377,"15min":0.36699999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":5.9989999999999997},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.627,"15min":0.61099999999999999},"min":{"1min":0.443,"5min":0.40100000000000002,"15min":0.39700000000000002},"max":{"1min":0.83999999999999997,"5min":0.86499999999999999,"15min":2.6930000000000001},"last":0.88200000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575018,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":438052,"kb_used_data":140764,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93933788,"statfs":{"total":96636764160,"available":96188198912,"internally_reserved":0,"allocated":144142336,"data_stored":175617804,"data_compressed":19256061,"data_compressed_allocated":21770240,"data_compressed_original":59097088,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,37,0,26],"upper_bound":64},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.34300000000000003,"15min":0.38400000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.19},"max":{"1min":0.51000000000000001,"5min":0.54300000000000004,"15min":14.968999999999999},"last":11.335000000000001},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.376,"15min":0.41299999999999998},"min":{"1min":0.24099999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.52100000000000002,"5min":0.59199999999999997,"15min":14.746},"last":10.484999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.37,"15min":0.52200000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.21099999999999999},"max":{"1min":0.53300000000000003,"5min":0.54100000000000004,"15min":50.677999999999997},"last":5.3890000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.45800000000000002,"15min":0.60099999999999998},"min":{"1min":0.34599999999999997,"5min":0.28599999999999998,"15min":0.26000000000000001},"max":{"1min":0.57599999999999996,"5min":0.66200000000000003,"15min":50.615000000000002},"last":5.4260000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.37,"15min":0.45900000000000002},"min":{"1min":0.25600000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.48999999999999999,"5min":0.61299999999999999,"15min":17.440999999999999},"last":6.5179999999999998},{"interface":"front","average":{"1min":0.44,"5min":0.44700000000000001,"15min":0.59899999999999998},"min":{"1min":0.32400000000000001,"5min":0.29099999999999998,"15min":0.23499999999999999},"max":{"1min":0.65500000000000003,"5min":0.88900000000000001,"15min":50.424999999999997},"last":8.1750000000000007}]},{"osd":4,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59499999999999997,"15min":0.64400000000000002},"min":{"1min":0.41099999999999998,"5min":0.34200000000000003,"15min":0.29699999999999999},"max":{"1min":0.76800000000000002,"5min":0.84799999999999998,"15min":14.802},"last":14.308},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60399999999999998,"15min":0.67500000000000004},"min":{"1min":0.42199999999999999,"5min":0.39400000000000002,"15min":0.36499999999999999},"max":{"1min":0.746,"5min":0.88700000000000001,"15min":14.621},"last":2.2749999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60399999999999998,"15min":0.63200000000000001},"min":{"1min":0.48399999999999999,"5min":0.40100000000000002,"15min":0.35199999999999998},"max":{"1min":0.75700000000000001,"5min":0.84399999999999997,"15min":14.792999999999999},"last":2.2450000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60899999999999999,"15min":0.76000000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.371},"max":{"1min":0.72499999999999998,"5min":0.92600000000000005,"15min":50.537999999999997},"last":9.1630000000000003}]},{"osd":6,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63700000000000001,"15min":0.68700000000000006},"min":{"1min":0.44700000000000001,"5min":0.40699999999999997,"15min":0.39100000000000001},"max":{"1min":0.78700000000000003,"5min":1.216,"15min":23.388000000000002},"last":15.627000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64300000000000002,"15min":0.69999999999999996},"min":{"1min":0.50600000000000001,"5min":0.40400000000000003,"15min":0.39100000000000001},"max":{"1min":0.75,"5min":0.86899999999999999,"15min":14.856},"last":11.372}]},{"osd":7,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.627,"15min":0.67900000000000005},"min":{"1min":0.45600000000000002,"5min":0.40999999999999998,"15min":0.377},"max":{"1min":0.80900000000000005,"5min":0.95899999999999996,"15min":14.693},"last":1.7569999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63500000000000001,"15min":0.67400000000000004},"min":{"1min":0.45800000000000002,"5min":0.44600000000000001,"15min":0.434},"max":{"1min":0.79600000000000004,"5min":0.95499999999999996,"15min":10.349},"last":2.2090000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575018,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585668,"kb_used_data":276092,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":93786172,"statfs":{"total":96636764160,"available":96037040128,"internally_reserved":0,"allocated":282718208,"data_stored":344742979,"data_compressed":32796809,"data_compressed_allocated":37396480,"data_compressed_original":108593152,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.34399999999999997,"15min":0.35999999999999999},"min":{"1min":0.24299999999999999,"5min":0.216,"15min":0.186},"max":{"1min":0.5,"5min":0.58699999999999997,"15min":7.0999999999999996},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.35499999999999998,"15min":0.41699999999999998},"min":{"1min":0.27200000000000002,"5min":0.221,"15min":0.218},"max":{"1min":0.622,"5min":0.622,"15min":14.257},"last":0.25900000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.34999999999999998,"15min":0.35399999999999998},"min":{"1min":0.23100000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":3.1760000000000002},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.40899999999999997,"15min":0.39600000000000002},"min":{"1min":0.29999999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.61499999999999999,"5min":0.68200000000000005,"15min":5.2599999999999998},"last":0.32400000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.39800000000000002,"15min":0.45900000000000002},"min":{"1min":0.316,"5min":0.25700000000000001,"15min":0.23499999999999999},"max":{"1min":0.57599999999999996,"5min":0.747,"15min":13.138},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.42699999999999999,"15min":0.44500000000000001},"min":{"1min":0.32800000000000001,"5min":0.28299999999999997,"15min":0.27200000000000002},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":7.181},"last":0.379}]},{"osd":4,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.627,"15min":0.68100000000000005},"min":{"1min":0.46800000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":13.068},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.65600000000000003,"15min":0.67500000000000004},"min":{"1min":0.57399999999999995,"5min":0.434,"15min":0.39800000000000002},"max":{"1min":0.878,"5min":0.89100000000000001,"15min":6.4299999999999997},"last":0.53600000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.47799999999999998,"5min":0.35999999999999999,"15min":0.32700000000000001},"max":{"1min":0.89000000000000001,"5min":0.95599999999999996,"15min":3.105},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.59799999999999998,"15min":0.60799999999999998},"min":{"1min":0.52000000000000002,"5min":0.372,"15min":0.29299999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":3.5369999999999999},"last":0.41399999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.63100000000000001,"15min":0.68799999999999994},"min":{"1min":0.54400000000000004,"5min":0.375,"15min":0.375},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":12.974},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63400000000000001,"15min":0.64100000000000001},"min":{"1min":0.56299999999999994,"5min":0.378,"15min":0.35599999999999998},"max":{"1min":0.93700000000000006,"5min":0.94499999999999995,"15min":3.0379999999999998},"last":0.439}]},{"osd":7,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.61299999999999999,"15min":0.63400000000000001},"min":{"1min":0.52900000000000003,"5min":0.41199999999999998,"15min":0.40100000000000002},"max":{"1min":0.93400000000000005,"5min":0.97199999999999998,"15min":6.2949999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.63400000000000001,"15min":0.67200000000000004},"min":{"1min":0.496,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90400000000000003,"5min":0.95399999999999996,"15min":13.032},"last":0.36899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575019,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":582900,"kb_used_data":281516,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":93788940,"statfs":{"total":96636764160,"available":96039874560,"internally_reserved":0,"allocated":288272384,"data_stored":377571071,"data_compressed":33050990,"data_compressed_allocated":38346752,"data_compressed_original":135929856,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52900000000000003,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68100000000000005,"5min":0.747,"15min":0.79600000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.56699999999999995,"15min":0.54600000000000004},"min":{"1min":0.47699999999999998,"5min":0.35099999999999998,"15min":0.26600000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.80000000000000004},"last":0.46000000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.41199999999999998,"5min":0.34899999999999998,"15min":0.26200000000000001},"max":{"1min":0.73899999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.501,"15min":0.51100000000000001},"min":{"1min":0.35199999999999998,"5min":0.30399999999999999,"15min":0.27400000000000002},"max":{"1min":0.70399999999999996,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.35299999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.40400000000000003,"5min":0.35899999999999999,"15min":0.312},"max":{"1min":0.69099999999999995,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.57599999999999996},"min":{"1min":0.441,"5min":0.36299999999999999,"15min":0.311},"max":{"1min":0.76700000000000002,"5min":1.413,"15min":1.413},"last":0.42399999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.59099999999999997,"15min":0.55100000000000005},"min":{"1min":0.38400000000000001,"5min":0.32100000000000001,"15min":0.314},"max":{"1min":0.65800000000000003,"5min":6.7000000000000002,"15min":6.7000000000000002},"last":0.504},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52900000000000003,"15min":0.54500000000000004},"min":{"1min":0.36899999999999999,"5min":0.36399999999999999,"15min":0.32000000000000001},"max":{"1min":0.72299999999999998,"5min":1.208,"15min":1.208},"last":0.47799999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.44600000000000001,"15min":0.47699999999999998},"min":{"1min":0.28599999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.59299999999999997,"5min":0.77100000000000002,"15min":0.81899999999999995},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52200000000000002,"15min":0.53000000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.29499999999999998},"max":{"1min":0.66800000000000004,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.38900000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.48499999999999999,"15min":0.51300000000000001},"min":{"1min":0.315,"5min":0.30099999999999999,"15min":0.28100000000000003},"max":{"1min":0.60499999999999998,"5min":1.236,"15min":1.236},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.42899999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.69199999999999995,"5min":1.381,"15min":1.381},"last":0.53600000000000003}]},{"osd":7,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53200000000000003,"15min":0.54600000000000004},"min":{"1min":0.38600000000000001,"5min":0.32800000000000001,"15min":0.30599999999999999},"max":{"1min":0.66700000000000004,"5min":0.71799999999999997,"15min":0.83499999999999996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.45900000000000002,"5min":0.39900000000000002,"15min":0.32600000000000001},"max":{"1min":0.73599999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.52000000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575018,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486268,"kb_used_data":188980,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93885572,"statfs":{"total":96636764160,"available":96138825728,"internally_reserved":0,"allocated":193515520,"data_stored":246084607,"data_compressed":22464539,"data_compressed_allocated":25784320,"data_compressed_original":83968000,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.50700000000000001,"15min":0.55100000000000005},"min":{"1min":0.34399999999999997,"5min":0.29999999999999999,"15min":0.23300000000000001},"max":{"1min":0.60199999999999998,"5min":1.0189999999999999,"15min":11.167},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.53800000000000003,"15min":0.57699999999999996},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.221},"max":{"1min":0.71899999999999997,"5min":1.0449999999999999,"15min":9.8659999999999997},"last":0.443}]},{"osd":1,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.64100000000000001,"5min":1.0780000000000001,"15min":1.1120000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55100000000000005,"15min":0.54000000000000004},"min":{"1min":0.35599999999999998,"5min":0.29999999999999999,"15min":0.28799999999999998},"max":{"1min":0.73999999999999999,"5min":1.028,"15min":2.524},"last":0.41199999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.52700000000000002,"15min":0.54000000000000004},"min":{"1min":0.39400000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.68999999999999995,"5min":0.98499999999999999,"15min":1.0389999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70399999999999996,"5min":0.999,"15min":1.1439999999999999},"last":0.56499999999999995}]},{"osd":3,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.77300000000000002,"5min":0.97299999999999998,"15min":2.9359999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60999999999999999,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.377,"15min":0.28999999999999998},"max":{"1min":0.77500000000000002,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.55300000000000005}]},{"osd":4,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.45900000000000002,"15min":0.47999999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.26000000000000001},"max":{"1min":0.71099999999999997,"5min":0.73499999999999999,"15min":1.1619999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57699999999999996,"15min":0.54200000000000004},"min":{"1min":0.33700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.75600000000000001,"5min":0.78900000000000003,"15min":1.1950000000000001},"last":0.53400000000000003}]},{"osd":6,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46899999999999997,"15min":0.503},"min":{"1min":0.35699999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65900000000000003,"5min":0.78300000000000003,"15min":0.94099999999999995},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56499999999999995,"15min":0.58099999999999996},"min":{"1min":0.41899999999999998,"5min":0.38400000000000001,"15min":0.32900000000000001},"max":{"1min":0.72599999999999998,"5min":0.84999999999999998,"15min":0.97899999999999998},"last":0.51200000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.53700000000000003},"min":{"1min":0.38700000000000001,"5min":0.33500000000000002,"15min":0.28699999999999998},"max":{"1min":0.81999999999999995,"5min":0.84899999999999998,"15min":1.1830000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.438,"5min":0.375,"15min":0.311},"max":{"1min":0.79900000000000004,"5min":0.81299999999999994,"15min":1.29},"last":0.58199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13369344,"data_stored":49512965,"data_compressed":6345911,"data_compressed_allocated":8089600,"data_compressed_original":44290048,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8429568,"data_stored":36394622,"data_compressed":4808050,"data_compressed_allocated":6139904,"data_compressed_original":34115584,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3923968,"data_stored":16797884,"data_compressed":2299517,"data_compressed_allocated":2953216,"data_compressed_original":15835136,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12926976,"data_stored":47356979,"data_compressed":5951348,"data_compressed_allocated":7700480,"data_compressed_original":42160128,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":21602304,"data_stored":87148982,"data_compressed":12219910,"data_compressed_allocated":15126528,"data_compressed_original":80732160,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":11075584,"data_stored":46041285,"data_compressed":6106433,"data_compressed_allocated":7659520,"data_compressed_original":42663936,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1855488,"data_stored":8388720,"data_compressed":1228027,"data_compressed_allocated":1515520,"data_compressed_original":8056832,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":10776576,"data_stored":42127597,"data_compressed":5960290,"data_compressed_allocated":7340032,"data_compressed_original":38731776,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":181956608,"data_stored":203156358,"data_compressed":20456538,"data_compressed_allocated":22609920,"data_compressed_original":49975296,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":274661376,"data_stored":300540492,"data_compressed":28008712,"data_compressed_allocated":30793728,"data_compressed_original":65601536,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139608064,"data_stored":158511793,"data_compressed":16956544,"data_compressed_allocated":18817024,"data_compressed_original":43261952,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":269180928,"data_stored":297077873,"data_compressed":26845461,"data_compressed_allocated":29696000,"data_compressed_original":66433024,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":266186752,"data_stored":290581266,"data_compressed":20904721,"data_compressed_allocated":23318528,"data_compressed_original":55640064,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":181829632,"data_stored":199735195,"data_compressed":16358106,"data_compressed_allocated":18124800,"data_compressed_original":41304064,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":166477824,"data_stored":179521845,"data_compressed":14708055,"data_compressed_allocated":16289792,"data_compressed_original":34680832,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":203251712,"data_stored":216547248,"data_compressed":16284911,"data_compressed_allocated":17920000,"data_compressed_original":36777984,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:11:03.351 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.351 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.351 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.351 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.352 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.352 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.352 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.352 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.352 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.353 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.353 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.353 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.353 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.353 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.354 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.354 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.354 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.354 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.354 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.354 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.355 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.355 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.355 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.355 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.355 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.355 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.356 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.356 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.356 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.356 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.356 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.356 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.357 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.357 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.357 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.357 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.357 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.358 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.358 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.358 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.358 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.358 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.359 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.359 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.359 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.359 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.359 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.360 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.360 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.360 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.360 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.360 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.360 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.361 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.361 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.361 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.361 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.361 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.362 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.362 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.362 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.362 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.362 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.362 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.363 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.363 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-06T03:57:04.302319+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=4, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.363 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.363 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.363 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-06T03:57:04.302319+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=4, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.363 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.364 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.364 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-06T03:57:04.302319+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=4, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.364 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.364 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.364 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-06T03:57:04.302319+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=4, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.365 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-06T03:57:04.302319+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=4, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.365 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.365 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.365 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-06T03:57:04.302319+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=4, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.365 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.365 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.366 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-06T03:57:04.302319+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=4, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.366 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.366 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.366 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.366 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-06T03:57:04.302319+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=4, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.367 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.367 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-06T03:57:04.302319+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=4, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.367 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.367 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.367 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.367 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.368 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.368 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.368 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.368 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.368 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.368 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.369 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.369 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.369 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.369 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.369 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.370 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.370 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:03.370 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T04:11:16.364 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:11:16.367 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:11:23.371 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:11:23.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.525+0000 7fea44175700 1 Processor -- start 2023-10-06T04:11:23.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.525+0000 7fea44175700 1 -- start start 2023-10-06T04:11:23.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.526+0000 7fea44175700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fea3c06dc40 0x7fea3c14fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:23.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.526+0000 7fea44175700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea3c06c1f0 0x7fea3c150240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:23.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.526+0000 7fea44175700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea3c0c9780 con 0x7fea3c06cbb0 2023-10-06T04:11:23.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.526+0000 7fea44175700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea3c0c8c40 con 0x7fea3c06dc40 2023-10-06T04:11:23.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.526+0000 7fea44175700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c06e0c0 con 0x7fea3c06c1f0 2023-10-06T04:11:23.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.526+0000 7fea41710700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fea3c06dc40 0x7fea3c14fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:23.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.526+0000 7fea41710700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fea3c06dc40 0x7fea3c14fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39240/0 (socket says 172.21.15.157:39240) 2023-10-06T04:11:23.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.526+0000 7fea41710700 1 -- 172.21.15.157:0/746303362 learned_addr learned my addr 172.21.15.157:0/746303362 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:11:23.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.526+0000 7fea42712700 1 --2- 172.21.15.157:0/746303362 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea3c06c1f0 0x7fea3c150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:23.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.526+0000 7fea41710700 1 -- 172.21.15.157:0/746303362 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea3c06c1f0 msgr2=0x7fea3c150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:11:23.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.526+0000 7fea41710700 1 --2- 172.21.15.157:0/746303362 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea3c06c1f0 0x7fea3c150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:23.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.526+0000 7fea41710700 1 -- 172.21.15.157:0/746303362 >> v1:172.21.15.175:6789/0 conn(0x7fea3c06cbb0 legacy=0x7fea3c06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:23.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.526+0000 7fea41710700 1 -- 172.21.15.157:0/746303362 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3c0c8e50 con 0x7fea3c06dc40 2023-10-06T04:11:23.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.526+0000 7fea41710700 1 --2- 172.21.15.157:0/746303362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea3c06dc40 0x7fea3c14fd00 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0099b0 tx=0x7fea2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63090d8ac28d775d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:23.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.526+0000 7fea40f0f700 1 -- 172.21.15.157:0/746303362 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c00b8f0 con 0x7fea3c06dc40 2023-10-06T04:11:23.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.526+0000 7fea40f0f700 1 -- 172.21.15.157:0/746303362 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2c00ba50 con 0x7fea3c06dc40 2023-10-06T04:11:23.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.526+0000 7fea40f0f700 1 -- 172.21.15.157:0/746303362 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c00bdf0 con 0x7fea3c06dc40 2023-10-06T04:11:23.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.527+0000 7fea44175700 1 -- 172.21.15.157:0/746303362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea3c06dc40 msgr2=0x7fea3c14fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:23.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.527+0000 7fea44175700 1 --2- 172.21.15.157:0/746303362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea3c06dc40 0x7fea3c14fd00 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0099b0 tx=0x7fea2c009250 comp rx=0 tx=0).stop 2023-10-06T04:11:23.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.527+0000 7fea44175700 1 -- 172.21.15.157:0/746303362 shutdown_connections 2023-10-06T04:11:23.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.527+0000 7fea44175700 1 --2- 172.21.15.157:0/746303362 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea3c06c1f0 0x7fea3c150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:23.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.527+0000 7fea44175700 1 --2- 172.21.15.157:0/746303362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea3c06dc40 0x7fea3c14fd00 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:23.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.527+0000 7fea44175700 1 -- 172.21.15.157:0/746303362 >> 172.21.15.157:0/746303362 conn(0x7fea3c06f230 msgr2=0x7fea3c06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:23.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.527+0000 7fea44175700 1 -- 172.21.15.157:0/746303362 shutdown_connections 2023-10-06T04:11:23.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.527+0000 7fea44175700 1 -- 172.21.15.157:0/746303362 wait complete. 2023-10-06T04:11:23.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.528+0000 7fea44175700 1 Processor -- start 2023-10-06T04:11:23.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.528+0000 7fea44175700 1 -- start start 2023-10-06T04:11:23.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.529+0000 7fea44175700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea3c06c1f0 0x7fea3c0c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:23.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.529+0000 7fea44175700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea3c06cbb0 0x7fea3c148a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:23.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.529+0000 7fea44175700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea3c0c4350 0x7fea3c148f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:23.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.529+0000 7fea44175700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c1485f0 con 0x7fea3c0c4350 2023-10-06T04:11:23.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.529+0000 7fea44175700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c166b10 con 0x7fea3c06cbb0 2023-10-06T04:11:23.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.529+0000 7fea44175700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c166df0 con 0x7fea3c06c1f0 2023-10-06T04:11:23.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.529+0000 7fea42712700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea3c0c4350 0x7fea3c148f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:23.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.529+0000 7fea42712700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea3c0c4350 0x7fea3c148f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39252/0 (socket says 172.21.15.157:39252) 2023-10-06T04:11:23.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.529+0000 7fea42712700 1 -- 172.21.15.157:0/1157926336 learned_addr learned my addr 172.21.15.157:0/1157926336 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:11:23.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.529+0000 7fea41710700 1 --2- 172.21.15.157:0/1157926336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea3c06cbb0 0x7fea3c148a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:23.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.529+0000 7fea41f11700 1 --2- 172.21.15.157:0/1157926336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea3c06c1f0 0x7fea3c0c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:23.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.529+0000 7fea42712700 1 -- 172.21.15.157:0/1157926336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea3c06c1f0 msgr2=0x7fea3c0c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:23.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.529+0000 7fea42712700 1 --2- 172.21.15.157:0/1157926336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea3c06c1f0 0x7fea3c0c5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:23.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.529+0000 7fea42712700 1 -- 172.21.15.157:0/1157926336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea3c06cbb0 msgr2=0x7fea3c148a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:23.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.529+0000 7fea42712700 1 --2- 172.21.15.157:0/1157926336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea3c06cbb0 0x7fea3c148a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:23.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.529+0000 7fea42712700 1 -- 172.21.15.157:0/1157926336 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2c0094f0 con 0x7fea3c0c4350 2023-10-06T04:11:23.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.529+0000 7fea42712700 1 --2- 172.21.15.157:0/1157926336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea3c0c4350 0x7fea3c148f80 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fea34009fa0 tx=0x7fea34009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:23.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.529+0000 7fea31ffb700 1 -- 172.21.15.157:0/1157926336 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3400ec50 con 0x7fea3c0c4350 2023-10-06T04:11:23.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.530+0000 7fea31ffb700 1 -- 172.21.15.157:0/1157926336 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3400edb0 con 0x7fea3c0c4350 2023-10-06T04:11:23.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.530+0000 7fea44175700 1 -- 172.21.15.157:0/1157926336 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea3c1670d0 con 0x7fea3c0c4350 2023-10-06T04:11:23.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.530+0000 7fea31ffb700 1 -- 172.21.15.157:0/1157926336 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea34014670 con 0x7fea3c0c4350 2023-10-06T04:11:23.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.530+0000 7fea44175700 1 -- 172.21.15.157:0/1157926336 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea3c166800 con 0x7fea3c0c4350 2023-10-06T04:11:23.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.531+0000 7fea44175700 1 -- 172.21.15.157:0/1157926336 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea3c148760 con 0x7fea3c0c4350 2023-10-06T04:11:23.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.531+0000 7fea31ffb700 1 -- 172.21.15.157:0/1157926336 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fea340149e0 con 0x7fea3c0c4350 2023-10-06T04:11:23.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.532+0000 7fea31ffb700 1 --2- 172.21.15.157:0/1157926336 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fea20066f60 0x7fea20069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:23.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.532+0000 7fea31ffb700 1 -- 172.21.15.157:0/1157926336 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7fea34084fe0 con 0x7fea3c0c4350 2023-10-06T04:11:23.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.532+0000 7fea41f11700 1 --2- 172.21.15.157:0/1157926336 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fea20066f60 0x7fea20069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:23.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.532+0000 7fea41f11700 1 --2- 172.21.15.157:0/1157926336 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fea20066f60 0x7fea20069410 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fea38003a80 tx=0x7fea38007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:23.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.536+0000 7fea31ffb700 1 -- 172.21.15.157:0/1157926336 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fea3c148760 con 0x7fea3c0c4350 2023-10-06T04:11:23.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.715+0000 7fea44175700 1 -- 172.21.15.157:0/1157926336 --> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fea3c148730 con 0x7fea20066f60 2023-10-06T04:11:23.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.724+0000 7fea31ffb700 1 -- 172.21.15.157:0/1157926336 <== mgr.4107 v2:172.21.15.175:6832/90661 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+315848 (secure 0 0 0) 0x7fea3c148730 con 0x7fea20066f60 2023-10-06T04:11:23.730 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:11:23.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.728+0000 7fea44175700 1 -- 172.21.15.157:0/1157926336 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fea20066f60 msgr2=0x7fea20069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:23.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.728+0000 7fea44175700 1 --2- 172.21.15.157:0/1157926336 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fea20066f60 0x7fea20069410 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fea38003a80 tx=0x7fea38007ec0 comp rx=0 tx=0).stop 2023-10-06T04:11:23.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.728+0000 7fea44175700 1 -- 172.21.15.157:0/1157926336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea3c0c4350 msgr2=0x7fea3c148f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:23.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.728+0000 7fea44175700 1 --2- 172.21.15.157:0/1157926336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea3c0c4350 0x7fea3c148f80 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fea34009fa0 tx=0x7fea34009900 comp rx=0 tx=0).stop 2023-10-06T04:11:23.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.728+0000 7fea44175700 1 -- 172.21.15.157:0/1157926336 shutdown_connections 2023-10-06T04:11:23.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.728+0000 7fea44175700 1 --2- 172.21.15.157:0/1157926336 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fea20066f60 0x7fea20069410 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:23.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.728+0000 7fea44175700 1 --2- 172.21.15.157:0/1157926336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea3c0c4350 0x7fea3c148f80 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:23.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.728+0000 7fea44175700 1 --2- 172.21.15.157:0/1157926336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea3c06c1f0 0x7fea3c0c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:23.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.728+0000 7fea44175700 1 --2- 172.21.15.157:0/1157926336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea3c06cbb0 0x7fea3c148a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:23.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.728+0000 7fea44175700 1 -- 172.21.15.157:0/1157926336 >> 172.21.15.157:0/1157926336 conn(0x7fea3c06f230 msgr2=0x7fea3c0af240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:23.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.728+0000 7fea44175700 1 -- 172.21.15.157:0/1157926336 shutdown_connections 2023-10-06T04:11:23.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:23.728+0000 7fea44175700 1 -- 172.21.15.157:0/1157926336 wait complete. 2023-10-06T04:11:23.734 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-10-06T04:11:23.741 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":477,"stamp":"2023-10-06T04:11:23.652691+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1092139978,"num_objects":10491,"num_object_clones":0,"num_object_copies":20982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10491,"num_whiteouts":0,"num_read":2914,"num_read_kb":634649,"num_write":27986,"num_write_kb":1247170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":122,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":536325,"num_omap_keys":945,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32135,"ondisk_log_size":32135,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":4142380,"kb_used_data":1731308,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":750832340,"statfs":{"total":773094113280,"available":768852316160,"internally_reserved":0,"allocated":1772859392,"data_stored":2185203923,"data_compressed":205777110,"data_compressed_allocated":234545152,"data_compressed_original":703152128,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":228371,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":449109,"num_omap_keys":791,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003065"},"pg_stats":[{"pgid":"4.3f","version":"53'449","reported_seq":2562,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:09.544941+0000","last_change":"2023-10-06T04:11:09.544918+0000","last_active":"2023-10-06T04:11:09.544941+0000","last_peered":"2023-10-06T04:11:09.544941+0000","last_clean":"2023-10-06T04:11:09.544941+0000","last_became_active":"2023-10-06T03:57:11.371756+0000","last_became_peered":"2023-10-06T03:57:11.371756+0000","last_unstale":"2023-10-06T04:11:09.544941+0000","last_undegraded":"2023-10-06T04:11:09.544941+0000","last_fullsized":"2023-10-06T04:11:09.544941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'449","last_scrub_stamp":"2023-10-06T04:11:09.544846+0000","last_deep_scrub":"53'449","last_deep_scrub_stamp":"2023-10-06T04:11:09.544846+0000","last_clean_scrub_stamp":"2023-10-06T04:11:09.544846+0000","objects_scrubbed":169,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:19:26.736027+0000","scrub_duration":0.096486541999999995,"objects_trimmed":0,"snaptrim_duration":0.001094783,"stat_sum":{"num_bytes":10615219,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":29,"num_read_kb":15,"num_write":308,"num_write_kb":10492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"53'354","reported_seq":2455,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.130359+0000","last_change":"2023-10-06T04:05:02.973547+0000","last_active":"2023-10-06T04:10:55.130359+0000","last_peered":"2023-10-06T04:10:55.130359+0000","last_clean":"2023-10-06T04:10:55.130359+0000","last_became_active":"2023-10-06T03:57:11.370070+0000","last_became_peered":"2023-10-06T03:57:11.370070+0000","last_unstale":"2023-10-06T04:10:55.130359+0000","last_undegraded":"2023-10-06T04:10:55.130359+0000","last_fullsized":"2023-10-06T04:10:55.130359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021773819999999998,"stat_sum":{"num_bytes":6466808,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":33,"num_read_kb":16,"num_write":334,"num_write_kb":6491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"53'356","reported_seq":2461,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.122379+0000","last_change":"2023-10-06T04:05:02.723796+0000","last_active":"2023-10-06T04:10:55.122379+0000","last_peered":"2023-10-06T04:10:55.122379+0000","last_clean":"2023-10-06T04:10:55.122379+0000","last_became_active":"2023-10-06T03:57:11.366012+0000","last_became_peered":"2023-10-06T03:57:11.366012+0000","last_unstale":"2023-10-06T04:10:55.122379+0000","last_undegraded":"2023-10-06T04:10:55.122379+0000","last_fullsized":"2023-10-06T04:10:55.122379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002722286,"stat_sum":{"num_bytes":15834623,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":40,"num_read_kb":20499,"num_write":321,"num_write_kb":23772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"53'431","reported_seq":2547,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:10.152318+0000","last_change":"2023-10-06T04:11:10.152270+0000","last_active":"2023-10-06T04:11:10.152318+0000","last_peered":"2023-10-06T04:11:10.152318+0000","last_clean":"2023-10-06T04:11:10.152318+0000","last_became_active":"2023-10-06T03:57:11.365673+0000","last_became_peered":"2023-10-06T03:57:11.365673+0000","last_unstale":"2023-10-06T04:11:10.152318+0000","last_undegraded":"2023-10-06T04:11:10.152318+0000","last_fullsized":"2023-10-06T04:11:10.152318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'431","last_scrub_stamp":"2023-10-06T04:11:10.152150+0000","last_deep_scrub":"53'431","last_deep_scrub_stamp":"2023-10-06T04:11:10.152150+0000","last_clean_scrub_stamp":"2023-10-06T04:11:10.152150+0000","objects_scrubbed":157,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:40.609815+0000","scrub_duration":0.112734037,"objects_trimmed":0,"snaptrim_duration":0.001947182,"stat_sum":{"num_bytes":14764610,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":32,"num_read_kb":7963,"num_write":291,"num_write_kb":14577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"53'342","reported_seq":2441,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.141196+0000","last_change":"2023-10-06T04:05:03.366556+0000","last_active":"2023-10-06T04:10:55.141196+0000","last_peered":"2023-10-06T04:10:55.141196+0000","last_clean":"2023-10-06T04:10:55.141196+0000","last_became_active":"2023-10-06T03:57:11.361302+0000","last_became_peered":"2023-10-06T03:57:11.361302+0000","last_unstale":"2023-10-06T04:10:55.141196+0000","last_undegraded":"2023-10-06T04:10:55.141196+0000","last_fullsized":"2023-10-06T04:10:55.141196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001966644,"stat_sum":{"num_bytes":2193993,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":31,"num_read_kb":18,"num_write":324,"num_write_kb":2284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"53'328","reported_seq":2430,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.135779+0000","last_change":"2023-10-06T04:05:02.723270+0000","last_active":"2023-10-06T04:10:55.135779+0000","last_peered":"2023-10-06T04:10:55.135779+0000","last_clean":"2023-10-06T04:10:55.135779+0000","last_became_active":"2023-10-06T03:57:11.368775+0000","last_became_peered":"2023-10-06T03:57:11.368775+0000","last_unstale":"2023-10-06T04:10:55.135779+0000","last_undegraded":"2023-10-06T04:10:55.135779+0000","last_fullsized":"2023-10-06T04:10:55.135779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002640482,"stat_sum":{"num_bytes":22431099,"num_objects":144,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":0,"num_read":34,"num_read_kb":13627,"num_write":304,"num_write_kb":22068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"53'386","reported_seq":2486,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.121956+0000","last_change":"2023-10-06T04:05:03.343818+0000","last_active":"2023-10-06T04:10:55.121956+0000","last_peered":"2023-10-06T04:10:55.121956+0000","last_clean":"2023-10-06T04:10:55.121956+0000","last_became_active":"2023-10-06T03:57:11.377786+0000","last_became_peered":"2023-10-06T03:57:11.377786+0000","last_unstale":"2023-10-06T04:10:55.121956+0000","last_undegraded":"2023-10-06T04:10:55.121956+0000","last_fullsized":"2023-10-06T04:10:55.121956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093960500000000004,"stat_sum":{"num_bytes":18398941,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":32,"num_read_kb":8208,"num_write":357,"num_write_kb":18159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"53'332","reported_seq":2431,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.132184+0000","last_change":"2023-10-06T04:05:02.972640+0000","last_active":"2023-10-06T04:10:55.132184+0000","last_peered":"2023-10-06T04:10:55.132184+0000","last_clean":"2023-10-06T04:10:55.132184+0000","last_became_active":"2023-10-06T03:57:11.373765+0000","last_became_peered":"2023-10-06T03:57:11.373765+0000","last_unstale":"2023-10-06T04:10:55.132184+0000","last_undegraded":"2023-10-06T04:10:55.132184+0000","last_fullsized":"2023-10-06T04:10:55.132184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001873881,"stat_sum":{"num_bytes":10960737,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":31,"num_read_kb":4107,"num_write":305,"num_write_kb":10786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"53'330","reported_seq":2438,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.145891+0000","last_change":"2023-10-06T04:05:03.367332+0000","last_active":"2023-10-06T04:10:55.145891+0000","last_peered":"2023-10-06T04:10:55.145891+0000","last_clean":"2023-10-06T04:10:55.145891+0000","last_became_active":"2023-10-06T03:57:11.368004+0000","last_became_peered":"2023-10-06T03:57:11.368004+0000","last_unstale":"2023-10-06T04:10:55.145891+0000","last_undegraded":"2023-10-06T04:10:55.145891+0000","last_fullsized":"2023-10-06T04:10:55.145891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024398710000000001,"stat_sum":{"num_bytes":13584161,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":41,"num_read_kb":4114,"num_write":308,"num_write_kb":17522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"53'352","reported_seq":2450,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.135881+0000","last_change":"2023-10-06T04:05:02.973579+0000","last_active":"2023-10-06T04:10:55.135881+0000","last_peered":"2023-10-06T04:10:55.135881+0000","last_clean":"2023-10-06T04:10:55.135881+0000","last_became_active":"2023-10-06T03:57:11.379107+0000","last_became_peered":"2023-10-06T03:57:11.379107+0000","last_unstale":"2023-10-06T04:10:55.135881+0000","last_undegraded":"2023-10-06T04:10:55.135881+0000","last_fullsized":"2023-10-06T04:10:55.135881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002039126,"stat_sum":{"num_bytes":9784235,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":30,"num_read_kb":4112,"num_write":328,"num_write_kb":9733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"53'376","reported_seq":2483,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.123250+0000","last_change":"2023-10-06T04:05:02.724475+0000","last_active":"2023-10-06T04:10:55.123250+0000","last_peered":"2023-10-06T04:10:55.123250+0000","last_clean":"2023-10-06T04:10:55.123250+0000","last_became_active":"2023-10-06T03:57:11.368256+0000","last_became_peered":"2023-10-06T03:57:11.368256+0000","last_unstale":"2023-10-06T04:10:55.123250+0000","last_undegraded":"2023-10-06T04:10:55.123250+0000","last_fullsized":"2023-10-06T04:10:55.123250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029138929999999999,"stat_sum":{"num_bytes":31134351,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":39,"num_read_kb":4114,"num_write":352,"num_write_kb":30587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"53'483","reported_seq":2597,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:21.563546+0000","last_change":"2023-10-06T04:11:21.563509+0000","last_active":"2023-10-06T04:11:21.563546+0000","last_peered":"2023-10-06T04:11:21.563546+0000","last_clean":"2023-10-06T04:11:21.563546+0000","last_became_active":"2023-10-06T03:57:11.368425+0000","last_became_peered":"2023-10-06T03:57:11.368425+0000","last_unstale":"2023-10-06T04:11:21.563546+0000","last_undegraded":"2023-10-06T04:11:21.563546+0000","last_fullsized":"2023-10-06T04:11:21.563546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'483","last_scrub_stamp":"2023-10-06T04:11:21.563396+0000","last_deep_scrub":"53'483","last_deep_scrub_stamp":"2023-10-06T04:11:21.563396+0000","last_clean_scrub_stamp":"2023-10-06T04:11:21.563396+0000","objects_scrubbed":161,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:51.078052+0000","scrub_duration":0.131329159,"objects_trimmed":0,"snaptrim_duration":0.0014231059999999999,"stat_sum":{"num_bytes":10422851,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":30,"num_read_kb":4105,"num_write":305,"num_write_kb":14436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"53'513","reported_seq":2633,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:19.515410+0000","last_change":"2023-10-06T04:11:19.515377+0000","last_active":"2023-10-06T04:11:19.515410+0000","last_peered":"2023-10-06T04:11:19.515410+0000","last_clean":"2023-10-06T04:11:19.515410+0000","last_became_active":"2023-10-06T03:57:11.367079+0000","last_became_peered":"2023-10-06T03:57:11.367079+0000","last_unstale":"2023-10-06T04:11:19.515410+0000","last_undegraded":"2023-10-06T04:11:19.515410+0000","last_fullsized":"2023-10-06T04:11:19.515410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'513","last_scrub_stamp":"2023-10-06T04:11:19.515270+0000","last_deep_scrub":"53'513","last_deep_scrub_stamp":"2023-10-06T04:11:19.515270+0000","last_clean_scrub_stamp":"2023-10-06T04:11:19.515270+0000","objects_scrubbed":173,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:26.862858+0000","scrub_duration":0.13944414499999999,"objects_trimmed":0,"snaptrim_duration":0.001042765,"stat_sum":{"num_bytes":14365712,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":37,"num_read_kb":12308,"num_write":314,"num_write_kb":22325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"53'346","reported_seq":2450,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.128029+0000","last_change":"2023-10-06T04:05:02.973393+0000","last_active":"2023-10-06T04:10:55.128029+0000","last_peered":"2023-10-06T04:10:55.128029+0000","last_clean":"2023-10-06T04:10:55.128029+0000","last_became_active":"2023-10-06T03:57:11.373216+0000","last_became_peered":"2023-10-06T03:57:11.373216+0000","last_unstale":"2023-10-06T04:10:55.128029+0000","last_undegraded":"2023-10-06T04:10:55.128029+0000","last_fullsized":"2023-10-06T04:10:55.128029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022456289999999999,"stat_sum":{"num_bytes":5765763,"num_objects":144,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":0,"num_read":36,"num_read_kb":18,"num_write":320,"num_write_kb":9892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"53'348","reported_seq":2446,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.121868+0000","last_change":"2023-10-06T04:05:03.343999+0000","last_active":"2023-10-06T04:10:55.121868+0000","last_peered":"2023-10-06T04:10:55.121868+0000","last_clean":"2023-10-06T04:10:55.121868+0000","last_became_active":"2023-10-06T03:57:11.375689+0000","last_became_peered":"2023-10-06T03:57:11.375689+0000","last_unstale":"2023-10-06T04:10:55.121868+0000","last_undegraded":"2023-10-06T04:10:55.121868+0000","last_fullsized":"2023-10-06T04:10:55.121868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092625599999999998,"stat_sum":{"num_bytes":15107134,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":322,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"53'458","reported_seq":2577,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:02.990894+0000","last_change":"2023-10-06T04:11:02.990870+0000","last_active":"2023-10-06T04:11:02.990894+0000","last_peered":"2023-10-06T04:11:02.990894+0000","last_clean":"2023-10-06T04:11:02.990894+0000","last_became_active":"2023-10-06T03:57:11.366543+0000","last_became_peered":"2023-10-06T03:57:11.366543+0000","last_unstale":"2023-10-06T04:11:02.990894+0000","last_undegraded":"2023-10-06T04:11:02.990894+0000","last_fullsized":"2023-10-06T04:11:02.990894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'458","last_scrub_stamp":"2023-10-06T04:11:02.990798+0000","last_deep_scrub":"53'458","last_deep_scrub_stamp":"2023-10-06T04:11:02.990798+0000","last_clean_scrub_stamp":"2023-10-06T04:11:02.990798+0000","objects_scrubbed":155,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:39.255270+0000","scrub_duration":0.116930864,"objects_trimmed":0,"snaptrim_duration":0.0024276609999999998,"stat_sum":{"num_bytes":19833591,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":35,"num_read_kb":19,"num_write":334,"num_write_kb":22749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"53'358","reported_seq":2459,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.145544+0000","last_change":"2023-10-06T04:05:03.367754+0000","last_active":"2023-10-06T04:10:55.145544+0000","last_peered":"2023-10-06T04:10:55.145544+0000","last_clean":"2023-10-06T04:10:55.145544+0000","last_became_active":"2023-10-06T03:57:11.366767+0000","last_became_peered":"2023-10-06T03:57:11.366767+0000","last_unstale":"2023-10-06T04:10:55.145544+0000","last_undegraded":"2023-10-06T04:10:55.145544+0000","last_fullsized":"2023-10-06T04:10:55.145544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002777833,"stat_sum":{"num_bytes":14683528,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":34,"num_read_kb":8213,"num_write":334,"num_write_kb":18604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"53'370","reported_seq":2481,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.128378+0000","last_change":"2023-10-06T04:05:02.723910+0000","last_active":"2023-10-06T04:10:55.128378+0000","last_peered":"2023-10-06T04:10:55.128378+0000","last_clean":"2023-10-06T04:10:55.128378+0000","last_became_active":"2023-10-06T03:57:11.362377+0000","last_became_peered":"2023-10-06T03:57:11.362377+0000","last_unstale":"2023-10-06T04:10:55.128378+0000","last_undegraded":"2023-10-06T04:10:55.128378+0000","last_fullsized":"2023-10-06T04:10:55.128378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027606900000000001,"stat_sum":{"num_bytes":10886042,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":43,"num_read_kb":26,"num_write":338,"num_write_kb":10805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"53'458","reported_seq":2581,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:13.608185+0000","last_change":"2023-10-06T04:11:13.608149+0000","last_active":"2023-10-06T04:11:13.608185+0000","last_peered":"2023-10-06T04:11:13.608185+0000","last_clean":"2023-10-06T04:11:13.608185+0000","last_became_active":"2023-10-06T03:57:11.363792+0000","last_became_peered":"2023-10-06T03:57:11.363792+0000","last_unstale":"2023-10-06T04:11:13.608185+0000","last_undegraded":"2023-10-06T04:11:13.608185+0000","last_fullsized":"2023-10-06T04:11:13.608185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'458","last_scrub_stamp":"2023-10-06T04:11:13.608057+0000","last_deep_scrub":"53'458","last_deep_scrub_stamp":"2023-10-06T04:11:13.608057+0000","last_clean_scrub_stamp":"2023-10-06T04:11:13.608057+0000","objects_scrubbed":159,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:44.566660+0000","scrub_duration":0.104721544,"objects_trimmed":0,"snaptrim_duration":0.0014611940000000001,"stat_sum":{"num_bytes":14666818,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":40,"num_read_kb":8209,"num_write":335,"num_write_kb":18594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"53'367","reported_seq":2471,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.131549+0000","last_change":"2023-10-06T04:05:02.724524+0000","last_active":"2023-10-06T04:10:55.131549+0000","last_peered":"2023-10-06T04:10:55.131549+0000","last_clean":"2023-10-06T04:10:55.131549+0000","last_became_active":"2023-10-06T03:57:11.364726+0000","last_became_peered":"2023-10-06T03:57:11.364726+0000","last_unstale":"2023-10-06T04:10:55.131549+0000","last_undegraded":"2023-10-06T04:10:55.131549+0000","last_fullsized":"2023-10-06T04:10:55.131549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029763670000000002,"stat_sum":{"num_bytes":20706667,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":37,"num_read_kb":12308,"num_write":348,"num_write_kb":28542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"53'348","reported_seq":2458,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.347235+0000","last_change":"2023-10-06T04:05:03.344002+0000","last_active":"2023-10-06T04:10:55.347235+0000","last_peered":"2023-10-06T04:10:55.347235+0000","last_clean":"2023-10-06T04:10:55.347235+0000","last_became_active":"2023-10-06T03:57:11.375668+0000","last_became_peered":"2023-10-06T03:57:11.375668+0000","last_unstale":"2023-10-06T04:10:55.347235+0000","last_undegraded":"2023-10-06T04:10:55.347235+0000","last_fullsized":"2023-10-06T04:10:55.347235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010285800000000001,"stat_sum":{"num_bytes":19005599,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":42,"num_read_kb":8215,"num_write":326,"num_write_kb":18702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"53'363","reported_seq":2465,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.135066+0000","last_change":"2023-10-06T04:05:03.366839+0000","last_active":"2023-10-06T04:10:55.135066+0000","last_peered":"2023-10-06T04:10:55.135066+0000","last_clean":"2023-10-06T04:10:55.135066+0000","last_became_active":"2023-10-06T03:57:11.365149+0000","last_became_peered":"2023-10-06T03:57:11.365149+0000","last_unstale":"2023-10-06T04:10:55.135066+0000","last_undegraded":"2023-10-06T04:10:55.135066+0000","last_fullsized":"2023-10-06T04:10:55.135066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020974879999999998,"stat_sum":{"num_bytes":5860015,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":34,"num_read_kb":17,"num_write":331,"num_write_kb":9992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"53'517","reported_seq":2642,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:22.576927+0000","last_change":"2023-10-06T04:11:22.576904+0000","last_active":"2023-10-06T04:11:22.576927+0000","last_peered":"2023-10-06T04:11:22.576927+0000","last_clean":"2023-10-06T04:11:22.576927+0000","last_became_active":"2023-10-06T03:57:11.362128+0000","last_became_peered":"2023-10-06T03:57:11.362128+0000","last_unstale":"2023-10-06T04:11:22.576927+0000","last_undegraded":"2023-10-06T04:11:22.576927+0000","last_fullsized":"2023-10-06T04:11:22.576927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'517","last_scrub_stamp":"2023-10-06T04:11:22.576836+0000","last_deep_scrub":"53'517","last_deep_scrub_stamp":"2023-10-06T04:11:22.576836+0000","last_clean_scrub_stamp":"2023-10-06T04:11:22.576836+0000","objects_scrubbed":164,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:20.021257+0000","scrub_duration":0.10803399599999999,"objects_trimmed":0,"snaptrim_duration":0.0025174860000000002,"stat_sum":{"num_bytes":10052289,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":42,"num_read_kb":8214,"num_write":324,"num_write_kb":14077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"53'376","reported_seq":2491,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.144206+0000","last_change":"2023-10-06T04:05:03.344184+0000","last_active":"2023-10-06T04:10:55.144206+0000","last_peered":"2023-10-06T04:10:55.144206+0000","last_clean":"2023-10-06T04:10:55.144206+0000","last_became_active":"2023-10-06T03:57:11.375702+0000","last_became_peered":"2023-10-06T03:57:11.375702+0000","last_unstale":"2023-10-06T04:10:55.144206+0000","last_undegraded":"2023-10-06T04:10:55.144206+0000","last_fullsized":"2023-10-06T04:10:55.144206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010337009999999999,"stat_sum":{"num_bytes":19987419,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":47,"num_read_kb":12104,"num_write":347,"num_write_kb":19698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"53'323","reported_seq":2420,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.120450+0000","last_change":"2023-10-06T04:05:02.718513+0000","last_active":"2023-10-06T04:10:55.120450+0000","last_peered":"2023-10-06T04:10:55.120450+0000","last_clean":"2023-10-06T04:10:55.120450+0000","last_became_active":"2023-10-06T03:57:11.374842+0000","last_became_peered":"2023-10-06T03:57:11.374842+0000","last_unstale":"2023-10-06T04:10:55.120450+0000","last_undegraded":"2023-10-06T04:10:55.120450+0000","last_fullsized":"2023-10-06T04:10:55.120450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001332342,"stat_sum":{"num_bytes":5811346,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":29,"num_read_kb":4115,"num_write":308,"num_write_kb":5844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"53'393","reported_seq":2494,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.125945+0000","last_change":"2023-10-06T04:05:02.718726+0000","last_active":"2023-10-06T04:10:55.125945+0000","last_peered":"2023-10-06T04:10:55.125945+0000","last_clean":"2023-10-06T04:10:55.125945+0000","last_became_active":"2023-10-06T03:57:11.361349+0000","last_became_peered":"2023-10-06T03:57:11.361349+0000","last_unstale":"2023-10-06T04:10:55.125945+0000","last_undegraded":"2023-10-06T04:10:55.125945+0000","last_fullsized":"2023-10-06T04:10:55.125945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014394589999999999,"stat_sum":{"num_bytes":23015133,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":33,"num_read_kb":4115,"num_write":357,"num_write_kb":22663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"53'328","reported_seq":2434,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.131318+0000","last_change":"2023-10-06T04:05:02.972918+0000","last_active":"2023-10-06T04:10:55.131318+0000","last_peered":"2023-10-06T04:10:55.131318+0000","last_clean":"2023-10-06T04:10:55.131318+0000","last_became_active":"2023-10-06T03:57:11.375846+0000","last_became_peered":"2023-10-06T03:57:11.375846+0000","last_unstale":"2023-10-06T04:10:55.131318+0000","last_undegraded":"2023-10-06T04:10:55.131318+0000","last_fullsized":"2023-10-06T04:10:55.131318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018480599999999999,"stat_sum":{"num_bytes":26318621,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":38,"num_read_kb":12302,"num_write":306,"num_write_kb":25858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"53'450","reported_seq":506,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:16.288922+0000","last_change":"2023-10-06T03:57:10.348317+0000","last_active":"2023-10-06T04:11:16.288922+0000","last_peered":"2023-10-06T04:11:16.288922+0000","last_clean":"2023-10-06T04:11:16.288922+0000","last_became_active":"2023-10-06T03:57:10.348078+0000","last_became_peered":"2023-10-06T03:57:10.348078+0000","last_unstale":"2023-10-06T04:11:16.288922+0000","last_undegraded":"2023-10-06T04:11:16.288922+0000","last_fullsized":"2023-10-06T04:11:16.288922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15812275,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":13,"num_read_kb":12332,"num_write":445,"num_write_kb":15691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"53'477","reported_seq":2599,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:17.713507+0000","last_change":"2023-10-06T04:11:17.713485+0000","last_active":"2023-10-06T04:11:17.713507+0000","last_peered":"2023-10-06T04:11:17.713507+0000","last_clean":"2023-10-06T04:11:17.713507+0000","last_became_active":"2023-10-06T03:57:11.370245+0000","last_became_peered":"2023-10-06T03:57:11.370245+0000","last_unstale":"2023-10-06T04:11:17.713507+0000","last_undegraded":"2023-10-06T04:11:17.713507+0000","last_fullsized":"2023-10-06T04:11:17.713507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'477","last_scrub_stamp":"2023-10-06T04:11:17.713414+0000","last_deep_scrub":"53'477","last_deep_scrub_stamp":"2023-10-06T04:11:17.713414+0000","last_clean_scrub_stamp":"2023-10-06T04:11:17.713414+0000","objects_scrubbed":162,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:09.132306+0000","scrub_duration":0.116172479,"objects_trimmed":0,"snaptrim_duration":0.002348424,"stat_sum":{"num_bytes":22265950,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":38,"num_read_kb":9495,"num_write":317,"num_write_kb":26009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"53'95","reported_seq":185,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:19.796378+0000","last_change":"2023-10-06T04:11:19.796356+0000","last_active":"2023-10-06T04:11:19.796378+0000","last_peered":"2023-10-06T04:11:19.796378+0000","last_clean":"2023-10-06T04:11:19.796378+0000","last_became_active":"2023-10-06T03:57:10.345150+0000","last_became_peered":"2023-10-06T03:57:10.345150+0000","last_unstale":"2023-10-06T04:11:19.796378+0000","last_undegraded":"2023-10-06T04:11:19.796378+0000","last_fullsized":"2023-10-06T04:11:19.796378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'95","last_scrub_stamp":"2023-10-06T04:11:19.796288+0000","last_deep_scrub":"53'95","last_deep_scrub_stamp":"2023-10-06T04:11:19.796288+0000","last_clean_scrub_stamp":"2023-10-06T04:11:19.796288+0000","objects_scrubbed":6,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:50.621259+0000","scrub_duration":0.0099644750000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4312676,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":146,"num_write":89,"num_write_kb":4275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":27214,"num_omap_keys":50,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"53'350","reported_seq":2444,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.135384+0000","last_change":"2023-10-06T04:05:03.367558+0000","last_active":"2023-10-06T04:10:55.135384+0000","last_peered":"2023-10-06T04:10:55.135384+0000","last_clean":"2023-10-06T04:10:55.135384+0000","last_became_active":"2023-10-06T03:57:11.375790+0000","last_became_peered":"2023-10-06T03:57:11.375790+0000","last_unstale":"2023-10-06T04:10:55.135384+0000","last_undegraded":"2023-10-06T04:10:55.135384+0000","last_fullsized":"2023-10-06T04:10:55.135384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025999780000000002,"stat_sum":{"num_bytes":19085062,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":26,"num_read_kb":12300,"num_write":331,"num_write_kb":18812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"53'1080","reported_seq":1140,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:10.608190+0000","last_change":"2023-10-06T04:11:10.608166+0000","last_active":"2023-10-06T04:11:10.608190+0000","last_peered":"2023-10-06T04:11:10.608190+0000","last_clean":"2023-10-06T04:11:10.608190+0000","last_became_active":"2023-10-06T03:57:10.347294+0000","last_became_peered":"2023-10-06T03:57:10.347294+0000","last_unstale":"2023-10-06T04:11:10.608190+0000","last_undegraded":"2023-10-06T04:11:10.608190+0000","last_fullsized":"2023-10-06T04:11:10.608190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1080","last_scrub_stamp":"2023-10-06T04:11:10.608093+0000","last_deep_scrub":"53'1080","last_deep_scrub_stamp":"2023-10-06T04:11:10.608093+0000","last_clean_scrub_stamp":"2023-10-06T04:11:10.608093+0000","objects_scrubbed":8,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:54.093395+0000","scrub_duration":0.023937142000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583466,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":7,"num_read_kb":16784,"num_write":1067,"num_write_kb":17022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30102,"num_omap_keys":52,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"53'457","reported_seq":2576,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:16.190235+0000","last_change":"2023-10-06T04:11:16.190201+0000","last_active":"2023-10-06T04:11:16.190235+0000","last_peered":"2023-10-06T04:11:16.190235+0000","last_clean":"2023-10-06T04:11:16.190235+0000","last_became_active":"2023-10-06T03:57:11.371758+0000","last_became_peered":"2023-10-06T03:57:11.371758+0000","last_unstale":"2023-10-06T04:11:16.190235+0000","last_undegraded":"2023-10-06T04:11:16.190235+0000","last_fullsized":"2023-10-06T04:11:16.190235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'457","last_scrub_stamp":"2023-10-06T04:11:16.190090+0000","last_deep_scrub":"53'457","last_deep_scrub_stamp":"2023-10-06T04:11:16.190090+0000","last_clean_scrub_stamp":"2023-10-06T04:11:16.190090+0000","objects_scrubbed":153,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:59.896644+0000","scrub_duration":0.12942683099999999,"objects_trimmed":0,"snaptrim_duration":0.0017888349999999999,"stat_sum":{"num_bytes":16888108,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":35,"num_read_kb":4115,"num_write":349,"num_write_kb":16624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"53'53","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:02.285823+0000","last_change":"2023-10-06T04:11:02.285761+0000","last_active":"2023-10-06T04:11:02.285823+0000","last_peered":"2023-10-06T04:11:02.285823+0000","last_clean":"2023-10-06T04:11:02.285823+0000","last_became_active":"2023-10-06T03:57:10.347151+0000","last_became_peered":"2023-10-06T03:57:10.347151+0000","last_unstale":"2023-10-06T04:11:02.285823+0000","last_undegraded":"2023-10-06T04:11:02.285823+0000","last_fullsized":"2023-10-06T04:11:02.285823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'52","last_scrub_stamp":"2023-10-06T04:11:02.285649+0000","last_deep_scrub":"53'52","last_deep_scrub_stamp":"2023-10-06T04:11:02.285649+0000","last_clean_scrub_stamp":"2023-10-06T04:11:02.285649+0000","objects_scrubbed":7,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:44.621007+0000","scrub_duration":0.0062605860000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":51,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":37322,"num_omap_keys":65,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"53'494","reported_seq":2614,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:21.604835+0000","last_change":"2023-10-06T04:11:21.604798+0000","last_active":"2023-10-06T04:11:21.604835+0000","last_peered":"2023-10-06T04:11:21.604835+0000","last_clean":"2023-10-06T04:11:21.604835+0000","last_became_active":"2023-10-06T03:57:11.369938+0000","last_became_peered":"2023-10-06T03:57:11.369938+0000","last_unstale":"2023-10-06T04:11:21.604835+0000","last_undegraded":"2023-10-06T04:11:21.604835+0000","last_fullsized":"2023-10-06T04:11:21.604835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'494","last_scrub_stamp":"2023-10-06T04:11:21.604691+0000","last_deep_scrub":"53'494","last_deep_scrub_stamp":"2023-10-06T04:11:21.604691+0000","last_clean_scrub_stamp":"2023-10-06T04:11:21.604691+0000","objects_scrubbed":172,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:46.110446+0000","scrub_duration":0.142007095,"objects_trimmed":0,"snaptrim_duration":0.0018579569999999999,"stat_sum":{"num_bytes":31490698,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":37,"num_read_kb":12302,"num_write":314,"num_write_kb":31163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"53'53","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:14.740559+0000","last_change":"2023-10-06T04:11:14.740529+0000","last_active":"2023-10-06T04:11:14.740559+0000","last_peered":"2023-10-06T04:11:14.740559+0000","last_clean":"2023-10-06T04:11:14.740559+0000","last_became_active":"2023-10-06T03:57:10.345721+0000","last_became_peered":"2023-10-06T03:57:10.345721+0000","last_unstale":"2023-10-06T04:11:14.740559+0000","last_undegraded":"2023-10-06T04:11:14.740559+0000","last_fullsized":"2023-10-06T04:11:14.740559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'53","last_scrub_stamp":"2023-10-06T04:11:14.740442+0000","last_deep_scrub":"53'53","last_deep_scrub_stamp":"2023-10-06T04:11:14.740442+0000","last_clean_scrub_stamp":"2023-10-06T04:11:14.740442+0000","objects_scrubbed":4,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:37.780654+0000","scrub_duration":0.013645548,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":15,"num_write":48,"num_write_kb":4121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9078,"num_omap_keys":16,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"53'374","reported_seq":2475,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.124153+0000","last_change":"2023-10-06T04:05:03.151517+0000","last_active":"2023-10-06T04:10:55.124153+0000","last_peered":"2023-10-06T04:10:55.124153+0000","last_clean":"2023-10-06T04:10:55.124153+0000","last_became_active":"2023-10-06T03:57:59.255587+0000","last_became_peered":"2023-10-06T03:57:59.255587+0000","last_unstale":"2023-10-06T04:10:55.124153+0000","last_undegraded":"2023-10-06T04:10:55.124153+0000","last_fullsized":"2023-10-06T04:10:55.124153+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001510024,"stat_sum":{"num_bytes":1845822,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":38,"num_read_kb":8209,"num_write":345,"num_write_kb":10179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"53'69","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:21.778145+0000","last_change":"2023-10-06T04:11:21.778119+0000","last_active":"2023-10-06T04:11:21.778145+0000","last_peered":"2023-10-06T04:11:21.778145+0000","last_clean":"2023-10-06T04:11:21.778145+0000","last_became_active":"2023-10-06T03:57:10.340232+0000","last_became_peered":"2023-10-06T03:57:10.340232+0000","last_unstale":"2023-10-06T04:11:21.778145+0000","last_undegraded":"2023-10-06T04:11:21.778145+0000","last_fullsized":"2023-10-06T04:11:21.778145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'69","last_scrub_stamp":"2023-10-06T04:11:21.778042+0000","last_deep_scrub":"53'69","last_deep_scrub_stamp":"2023-10-06T04:11:21.778042+0000","last_clean_scrub_stamp":"2023-10-06T04:11:21.778042+0000","objects_scrubbed":6,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:35:39.233114+0000","scrub_duration":0.011501937,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":66,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24992,"num_omap_keys":46,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"53'364","reported_seq":2469,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.130871+0000","last_change":"2023-10-06T04:05:02.718927+0000","last_active":"2023-10-06T04:10:55.130871+0000","last_peered":"2023-10-06T04:10:55.130871+0000","last_clean":"2023-10-06T04:10:55.130871+0000","last_became_active":"2023-10-06T03:57:11.374892+0000","last_became_peered":"2023-10-06T03:57:11.374892+0000","last_unstale":"2023-10-06T04:10:55.130871+0000","last_undegraded":"2023-10-06T04:10:55.130871+0000","last_fullsized":"2023-10-06T04:10:55.130871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00156207,"stat_sum":{"num_bytes":17801071,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":38,"num_read_kb":8211,"num_write":336,"num_write_kb":25748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"53'680","reported_seq":748,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:19.900780+0000","last_change":"2023-10-06T04:11:19.900740+0000","last_active":"2023-10-06T04:11:19.900780+0000","last_peered":"2023-10-06T04:11:19.900780+0000","last_clean":"2023-10-06T04:11:19.900780+0000","last_became_active":"2023-10-06T03:57:10.346430+0000","last_became_peered":"2023-10-06T03:57:10.346430+0000","last_unstale":"2023-10-06T04:11:19.900780+0000","last_undegraded":"2023-10-06T04:11:19.900780+0000","last_fullsized":"2023-10-06T04:11:19.900780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'680","last_scrub_stamp":"2023-10-06T04:11:19.900666+0000","last_deep_scrub":"53'680","last_deep_scrub_stamp":"2023-10-06T04:11:19.900666+0000","last_clean_scrub_stamp":"2023-10-06T04:11:19.900666+0000","objects_scrubbed":9,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:55.003315+0000","scrub_duration":0.024294989999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389252,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":30,"num_read_kb":9887,"num_write":670,"num_write_kb":8556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":23944,"num_omap_keys":41,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"53'354","reported_seq":2460,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.130611+0000","last_change":"2023-10-06T04:05:02.723446+0000","last_active":"2023-10-06T04:10:55.130611+0000","last_peered":"2023-10-06T04:10:55.130611+0000","last_clean":"2023-10-06T04:10:55.130611+0000","last_became_active":"2023-10-06T03:57:11.367565+0000","last_became_peered":"2023-10-06T03:57:11.367565+0000","last_unstale":"2023-10-06T04:10:55.130611+0000","last_undegraded":"2023-10-06T04:10:55.130611+0000","last_fullsized":"2023-10-06T04:10:55.130611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027762680000000001,"stat_sum":{"num_bytes":10311229,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":38,"num_read_kb":23,"num_write":322,"num_write_kb":10236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"53'188","reported_seq":255,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:11.762952+0000","last_change":"2023-10-06T04:11:11.762926+0000","last_active":"2023-10-06T04:11:11.762952+0000","last_peered":"2023-10-06T04:11:11.762952+0000","last_clean":"2023-10-06T04:11:11.762952+0000","last_became_active":"2023-10-06T03:57:10.350795+0000","last_became_peered":"2023-10-06T03:57:10.350795+0000","last_unstale":"2023-10-06T04:11:11.762952+0000","last_undegraded":"2023-10-06T04:11:11.762952+0000","last_fullsized":"2023-10-06T04:11:11.762952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'188","last_scrub_stamp":"2023-10-06T04:11:11.762854+0000","last_deep_scrub":"53'188","last_deep_scrub_stamp":"2023-10-06T04:11:11.762854+0000","last_clean_scrub_stamp":"2023-10-06T04:11:11.762854+0000","objects_scrubbed":9,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:37.116115+0000","scrub_duration":0.020924005999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388676,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":30,"num_read_kb":20509,"num_write":184,"num_write_kb":8306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":18896,"num_omap_keys":34,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"53'406","reported_seq":2522,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:14.748333+0000","last_change":"2023-10-06T04:11:14.748308+0000","last_active":"2023-10-06T04:11:14.748333+0000","last_peered":"2023-10-06T04:11:14.748333+0000","last_clean":"2023-10-06T04:11:14.748333+0000","last_became_active":"2023-10-06T03:57:11.367254+0000","last_became_peered":"2023-10-06T03:57:11.367254+0000","last_unstale":"2023-10-06T04:11:14.748333+0000","last_undegraded":"2023-10-06T04:11:14.748333+0000","last_fullsized":"2023-10-06T04:11:14.748333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'406","last_scrub_stamp":"2023-10-06T04:11:14.748233+0000","last_deep_scrub":"53'406","last_deep_scrub_stamp":"2023-10-06T04:11:14.748233+0000","last_clean_scrub_stamp":"2023-10-06T04:11:14.748233+0000","objects_scrubbed":165,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:29.852108+0000","scrub_duration":0.096980769999999994,"objects_trimmed":0,"snaptrim_duration":0.0027822569999999998,"stat_sum":{"num_bytes":13704260,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":32,"num_read_kb":10530,"num_write":279,"num_write_kb":17631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"53'111","reported_seq":174,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:04.279658+0000","last_change":"2023-10-06T04:11:04.279631+0000","last_active":"2023-10-06T04:11:04.279658+0000","last_peered":"2023-10-06T04:11:04.279658+0000","last_clean":"2023-10-06T04:11:04.279658+0000","last_became_active":"2023-10-06T03:57:10.337798+0000","last_became_peered":"2023-10-06T03:57:10.337798+0000","last_unstale":"2023-10-06T04:11:04.279658+0000","last_undegraded":"2023-10-06T04:11:04.279658+0000","last_fullsized":"2023-10-06T04:11:04.279658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'111","last_scrub_stamp":"2023-10-06T04:11:04.279550+0000","last_deep_scrub":"53'111","last_deep_scrub_stamp":"2023-10-06T04:11:04.279550+0000","last_clean_scrub_stamp":"2023-10-06T04:11:04.279550+0000","objects_scrubbed":6,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:39.866223+0000","scrub_duration":0.015622552,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":9753,"num_write":104,"num_write_kb":4165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":14386,"num_omap_keys":26,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"53'356","reported_seq":2461,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.131127+0000","last_change":"2023-10-06T04:05:02.973084+0000","last_active":"2023-10-06T04:10:55.131127+0000","last_peered":"2023-10-06T04:10:55.131127+0000","last_clean":"2023-10-06T04:10:55.131127+0000","last_became_active":"2023-10-06T03:57:11.376223+0000","last_became_peered":"2023-10-06T03:57:11.376223+0000","last_unstale":"2023-10-06T04:10:55.131127+0000","last_undegraded":"2023-10-06T04:10:55.131127+0000","last_fullsized":"2023-10-06T04:10:55.131127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018161119999999999,"stat_sum":{"num_bytes":34940720,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":37,"num_read_kb":11570,"num_write":336,"num_write_kb":34292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"53'133","reported_seq":186,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:15.963837+0000","last_change":"2023-10-06T04:11:10.713437+0000","last_active":"2023-10-06T04:11:15.963837+0000","last_peered":"2023-10-06T04:11:15.963837+0000","last_clean":"2023-10-06T04:11:15.963837+0000","last_became_active":"2023-10-06T03:57:10.350219+0000","last_became_peered":"2023-10-06T03:57:10.350219+0000","last_unstale":"2023-10-06T04:11:15.963837+0000","last_undegraded":"2023-10-06T04:11:15.963837+0000","last_fullsized":"2023-10-06T04:11:15.963837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'126","last_scrub_stamp":"2023-10-06T04:11:10.713339+0000","last_deep_scrub":"53'126","last_deep_scrub_stamp":"2023-10-06T04:11:10.713339+0000","last_clean_scrub_stamp":"2023-10-06T04:11:10.713339+0000","objects_scrubbed":8,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:04.717440+0000","scrub_duration":0.020180555999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4908188,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":125,"num_write_kb":9005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":51900,"num_omap_keys":89,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"53'381","reported_seq":2491,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.141067+0000","last_change":"2023-10-06T04:05:03.367352+0000","last_active":"2023-10-06T04:10:55.141067+0000","last_peered":"2023-10-06T04:10:55.141067+0000","last_clean":"2023-10-06T04:10:55.141067+0000","last_became_active":"2023-10-06T03:57:11.362443+0000","last_became_peered":"2023-10-06T03:57:11.362443+0000","last_unstale":"2023-10-06T04:10:55.141067+0000","last_undegraded":"2023-10-06T04:10:55.141067+0000","last_fullsized":"2023-10-06T04:10:55.141067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002448839,"stat_sum":{"num_bytes":22978165,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":42,"num_read_kb":8288,"num_write":356,"num_write_kb":22628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"53'83","reported_seq":147,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:08.057909+0000","last_change":"2023-10-06T04:11:08.057878+0000","last_active":"2023-10-06T04:11:08.057909+0000","last_peered":"2023-10-06T04:11:08.057909+0000","last_clean":"2023-10-06T04:11:08.057909+0000","last_became_active":"2023-10-06T03:57:10.345424+0000","last_became_peered":"2023-10-06T03:57:10.345424+0000","last_unstale":"2023-10-06T04:11:08.057909+0000","last_undegraded":"2023-10-06T04:11:08.057909+0000","last_fullsized":"2023-10-06T04:11:08.057909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'83","last_scrub_stamp":"2023-10-06T04:11:08.057782+0000","last_deep_scrub":"53'83","last_deep_scrub_stamp":"2023-10-06T04:11:08.057782+0000","last_clean_scrub_stamp":"2023-10-06T04:11:08.057782+0000","objects_scrubbed":12,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:41:48.011373+0000","scrub_duration":0.0096812489999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":33,"num_read_kb":33,"num_write":72,"num_write_kb":4158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31156,"num_omap_keys":54,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"53'419","reported_seq":2541,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:12.728194+0000","last_change":"2023-10-06T04:11:12.728172+0000","last_active":"2023-10-06T04:11:12.728194+0000","last_peered":"2023-10-06T04:11:12.728194+0000","last_clean":"2023-10-06T04:11:12.728194+0000","last_became_active":"2023-10-06T03:57:11.375824+0000","last_became_peered":"2023-10-06T03:57:11.375824+0000","last_unstale":"2023-10-06T04:11:12.728194+0000","last_undegraded":"2023-10-06T04:11:12.728194+0000","last_fullsized":"2023-10-06T04:11:12.728194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'419","last_scrub_stamp":"2023-10-06T04:11:12.728105+0000","last_deep_scrub":"53'419","last_deep_scrub_stamp":"2023-10-06T04:11:12.728105+0000","last_clean_scrub_stamp":"2023-10-06T04:11:12.728105+0000","objects_scrubbed":150,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:57.027413+0000","scrub_duration":0.083998824,"objects_trimmed":0,"snaptrim_duration":0.0023200299999999998,"stat_sum":{"num_bytes":9857160,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":38,"num_read_kb":8212,"num_write":304,"num_write_kb":9779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"53'256","reported_seq":312,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:20.763949+0000","last_change":"2023-10-06T04:11:20.763919+0000","last_active":"2023-10-06T04:11:20.763949+0000","last_peered":"2023-10-06T04:11:20.763949+0000","last_clean":"2023-10-06T04:11:20.763949+0000","last_became_active":"2023-10-06T03:57:10.350135+0000","last_became_peered":"2023-10-06T03:57:10.350135+0000","last_unstale":"2023-10-06T04:11:20.763949+0000","last_undegraded":"2023-10-06T04:11:20.763949+0000","last_fullsized":"2023-10-06T04:11:20.763949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'256","last_scrub_stamp":"2023-10-06T04:11:20.763831+0000","last_deep_scrub":"53'256","last_deep_scrub_stamp":"2023-10-06T04:11:20.763831+0000","last_clean_scrub_stamp":"2023-10-06T04:11:20.763831+0000","objects_scrubbed":6,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:26.714475+0000","scrub_duration":0.01459685,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":4098,"num_write":280,"num_write_kb":4293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":28994,"num_omap_keys":51,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"53'341","reported_seq":2441,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.131548+0000","last_change":"2023-10-06T04:05:02.972356+0000","last_active":"2023-10-06T04:10:55.131548+0000","last_peered":"2023-10-06T04:10:55.131548+0000","last_clean":"2023-10-06T04:10:55.131548+0000","last_became_active":"2023-10-06T03:57:11.376236+0000","last_became_peered":"2023-10-06T03:57:11.376236+0000","last_unstale":"2023-10-06T04:10:55.131548+0000","last_undegraded":"2023-10-06T04:10:55.131548+0000","last_fullsized":"2023-10-06T04:10:55.131548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015573080000000001,"stat_sum":{"num_bytes":22935537,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":32,"num_read_kb":10317,"num_write":317,"num_write_kb":26655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"53'95","reported_seq":153,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:18.759540+0000","last_change":"2023-10-06T04:11:18.759511+0000","last_active":"2023-10-06T04:11:18.759540+0000","last_peered":"2023-10-06T04:11:18.759540+0000","last_clean":"2023-10-06T04:11:18.759540+0000","last_became_active":"2023-10-06T03:57:10.345510+0000","last_became_peered":"2023-10-06T03:57:10.345510+0000","last_unstale":"2023-10-06T04:11:18.759540+0000","last_undegraded":"2023-10-06T04:11:18.759540+0000","last_fullsized":"2023-10-06T04:11:18.759540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'95","last_scrub_stamp":"2023-10-06T04:11:18.759410+0000","last_deep_scrub":"53'95","last_deep_scrub_stamp":"2023-10-06T04:11:18.759410+0000","last_clean_scrub_stamp":"2023-10-06T04:11:18.759410+0000","objects_scrubbed":3,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:29.676193+0000","scrub_duration":0.014631524999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":4108,"num_write":90,"num_write_kb":4140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"53'359","reported_seq":2464,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.122851+0000","last_change":"2023-10-06T04:05:02.721917+0000","last_active":"2023-10-06T04:10:55.122851+0000","last_peered":"2023-10-06T04:10:55.122851+0000","last_clean":"2023-10-06T04:10:55.122851+0000","last_became_active":"2023-10-06T03:57:11.368375+0000","last_became_peered":"2023-10-06T03:57:11.368375+0000","last_unstale":"2023-10-06T04:10:55.122851+0000","last_undegraded":"2023-10-06T04:10:55.122851+0000","last_fullsized":"2023-10-06T04:10:55.122851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001632912,"stat_sum":{"num_bytes":5657073,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":37,"num_read_kb":4115,"num_write":343,"num_write_kb":5657,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"53'164","reported_seq":216,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:16.734834+0000","last_change":"2023-10-06T04:11:16.734834+0000","last_active":"2023-10-06T04:11:16.734834+0000","last_peered":"2023-10-06T04:11:16.734834+0000","last_clean":"2023-10-06T04:11:16.734834+0000","last_became_active":"2023-10-06T03:57:10.346689+0000","last_became_peered":"2023-10-06T03:57:10.346689+0000","last_unstale":"2023-10-06T04:11:16.734834+0000","last_undegraded":"2023-10-06T04:11:16.734834+0000","last_fullsized":"2023-10-06T04:11:16.734834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":12290,"num_write":163,"num_write_kb":8336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"53'471","reported_seq":2589,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:13.079557+0000","last_change":"2023-10-06T04:11:13.079535+0000","last_active":"2023-10-06T04:11:13.079557+0000","last_peered":"2023-10-06T04:11:13.079557+0000","last_clean":"2023-10-06T04:11:13.079557+0000","last_became_active":"2023-10-06T03:57:11.371795+0000","last_became_peered":"2023-10-06T03:57:11.371795+0000","last_unstale":"2023-10-06T04:11:13.079557+0000","last_undegraded":"2023-10-06T04:11:13.079557+0000","last_fullsized":"2023-10-06T04:11:13.079557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'471","last_scrub_stamp":"2023-10-06T04:11:13.079467+0000","last_deep_scrub":"53'471","last_deep_scrub_stamp":"2023-10-06T04:11:13.079467+0000","last_clean_scrub_stamp":"2023-10-06T04:11:13.079467+0000","objects_scrubbed":169,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:17.052127+0000","scrub_duration":0.080011451999999997,"objects_trimmed":0,"snaptrim_duration":0.0016252980000000001,"stat_sum":{"num_bytes":5774867,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":34,"num_read_kb":13,"num_write":355,"num_write_kb":5825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"53'59","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:15.755241+0000","last_change":"2023-10-06T04:11:15.755204+0000","last_active":"2023-10-06T04:11:15.755241+0000","last_peered":"2023-10-06T04:11:15.755241+0000","last_clean":"2023-10-06T04:11:15.755241+0000","last_became_active":"2023-10-06T03:57:10.348291+0000","last_became_peered":"2023-10-06T03:57:10.348291+0000","last_unstale":"2023-10-06T04:11:15.755241+0000","last_undegraded":"2023-10-06T04:11:15.755241+0000","last_fullsized":"2023-10-06T04:11:15.755241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'59","last_scrub_stamp":"2023-10-06T04:11:15.755103+0000","last_deep_scrub":"53'59","last_deep_scrub_stamp":"2023-10-06T04:11:15.755103+0000","last_clean_scrub_stamp":"2023-10-06T04:11:15.755103+0000","objects_scrubbed":5,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:27.967550+0000","scrub_duration":0.0073040559999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":57,"num_write_kb":4195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":20734,"num_omap_keys":37,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"53'328","reported_seq":2429,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.135091+0000","last_change":"2023-10-06T04:05:03.366405+0000","last_active":"2023-10-06T04:10:55.135091+0000","last_peered":"2023-10-06T04:10:55.135091+0000","last_clean":"2023-10-06T04:10:55.135091+0000","last_became_active":"2023-10-06T03:57:11.365077+0000","last_became_peered":"2023-10-06T03:57:11.365077+0000","last_unstale":"2023-10-06T04:10:55.135091+0000","last_undegraded":"2023-10-06T04:10:55.135091+0000","last_fullsized":"2023-10-06T04:10:55.135091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019352270000000001,"stat_sum":{"num_bytes":15068515,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":33,"num_read_kb":4109,"num_write":308,"num_write_kb":14877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"53'283","reported_seq":345,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:01.636104+0000","last_change":"2023-10-06T04:11:01.636081+0000","last_active":"2023-10-06T04:11:01.636104+0000","last_peered":"2023-10-06T04:11:01.636104+0000","last_clean":"2023-10-06T04:11:01.636104+0000","last_became_active":"2023-10-06T03:57:10.348055+0000","last_became_peered":"2023-10-06T03:57:10.348055+0000","last_unstale":"2023-10-06T04:11:01.636104+0000","last_undegraded":"2023-10-06T04:11:01.636104+0000","last_fullsized":"2023-10-06T04:11:01.636104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'283","last_scrub_stamp":"2023-10-06T04:11:01.636008+0000","last_deep_scrub":"53'283","last_deep_scrub_stamp":"2023-10-06T04:11:01.636008+0000","last_clean_scrub_stamp":"2023-10-06T04:11:01.636008+0000","objects_scrubbed":5,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:11.936218+0000","scrub_duration":0.017511342999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":17,"num_read_kb":12301,"num_write":272,"num_write_kb":8316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"53'496","reported_seq":2609,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:11.948075+0000","last_change":"2023-10-06T04:11:11.948042+0000","last_active":"2023-10-06T04:11:11.948075+0000","last_peered":"2023-10-06T04:11:11.948075+0000","last_clean":"2023-10-06T04:11:11.948075+0000","last_became_active":"2023-10-06T03:57:11.370073+0000","last_became_peered":"2023-10-06T03:57:11.370073+0000","last_unstale":"2023-10-06T04:11:11.948075+0000","last_undegraded":"2023-10-06T04:11:11.948075+0000","last_fullsized":"2023-10-06T04:11:11.948075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'496","last_scrub_stamp":"2023-10-06T04:11:11.947965+0000","last_deep_scrub":"53'496","last_deep_scrub_stamp":"2023-10-06T04:11:11.947965+0000","last_clean_scrub_stamp":"2023-10-06T04:11:11.947965+0000","objects_scrubbed":185,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:02.104713+0000","scrub_duration":0.114303379,"objects_trimmed":0,"snaptrim_duration":0.0027454179999999999,"stat_sum":{"num_bytes":18058404,"num_objects":185,"num_object_clones":0,"num_object_copies":370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":185,"num_whiteouts":0,"num_read":28,"num_read_kb":4110,"num_write":345,"num_write_kb":17816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"53'118","reported_seq":176,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:09.093158+0000","last_change":"2023-10-06T04:11:09.093116+0000","last_active":"2023-10-06T04:11:09.093158+0000","last_peered":"2023-10-06T04:11:09.093158+0000","last_clean":"2023-10-06T04:11:09.093158+0000","last_became_active":"2023-10-06T03:57:10.352653+0000","last_became_peered":"2023-10-06T03:57:10.352653+0000","last_unstale":"2023-10-06T04:11:09.093158+0000","last_undegraded":"2023-10-06T04:11:09.093158+0000","last_fullsized":"2023-10-06T04:11:09.093158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'118","last_scrub_stamp":"2023-10-06T04:11:09.092974+0000","last_deep_scrub":"53'118","last_deep_scrub_stamp":"2023-10-06T04:11:09.092974+0000","last_clean_scrub_stamp":"2023-10-06T04:11:09.092974+0000","objects_scrubbed":6,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:03.839458+0000","scrub_duration":0.016085988999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":8,"num_read_kb":4217,"num_write":207,"num_write_kb":4775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":18688,"num_omap_keys":32,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"53'358","reported_seq":2458,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.129017+0000","last_change":"2023-10-06T04:05:02.724113+0000","last_active":"2023-10-06T04:10:55.129017+0000","last_peered":"2023-10-06T04:10:55.129017+0000","last_clean":"2023-10-06T04:10:55.129017+0000","last_became_active":"2023-10-06T03:57:11.363830+0000","last_became_peered":"2023-10-06T03:57:11.363830+0000","last_unstale":"2023-10-06T04:10:55.129017+0000","last_undegraded":"2023-10-06T04:10:55.129017+0000","last_fullsized":"2023-10-06T04:10:55.129017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027698559999999998,"stat_sum":{"num_bytes":22869017,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":32,"num_read_kb":4115,"num_write":333,"num_write_kb":26610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"53'113","reported_seq":174,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:14.850284+0000","last_change":"2023-10-06T04:11:14.850257+0000","last_active":"2023-10-06T04:11:14.850284+0000","last_peered":"2023-10-06T04:11:14.850284+0000","last_clean":"2023-10-06T04:11:14.850284+0000","last_became_active":"2023-10-06T03:57:10.349114+0000","last_became_peered":"2023-10-06T03:57:10.349114+0000","last_unstale":"2023-10-06T04:11:14.850284+0000","last_undegraded":"2023-10-06T04:11:14.850284+0000","last_fullsized":"2023-10-06T04:11:14.850284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'113","last_scrub_stamp":"2023-10-06T04:11:14.850170+0000","last_deep_scrub":"53'113","last_deep_scrub_stamp":"2023-10-06T04:11:14.850170+0000","last_clean_scrub_stamp":"2023-10-06T04:11:14.850170+0000","objects_scrubbed":8,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:59.463057+0000","scrub_duration":0.012734528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":19,"num_read_kb":18,"num_write":109,"num_write_kb":4188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":33482,"num_omap_keys":60,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"53'393","reported_seq":2501,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:07.012391+0000","last_change":"2023-10-06T04:11:07.012369+0000","last_active":"2023-10-06T04:11:07.012391+0000","last_peered":"2023-10-06T04:11:07.012391+0000","last_clean":"2023-10-06T04:11:07.012391+0000","last_became_active":"2023-10-06T03:57:11.367257+0000","last_became_peered":"2023-10-06T03:57:11.367257+0000","last_unstale":"2023-10-06T04:11:07.012391+0000","last_undegraded":"2023-10-06T04:11:07.012391+0000","last_fullsized":"2023-10-06T04:11:07.012391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'393","last_scrub_stamp":"2023-10-06T04:11:07.012293+0000","last_deep_scrub":"53'393","last_deep_scrub_stamp":"2023-10-06T04:11:07.012293+0000","last_clean_scrub_stamp":"2023-10-06T04:11:07.012293+0000","objects_scrubbed":142,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:54.463356+0000","scrub_duration":0.119522168,"objects_trimmed":0,"snaptrim_duration":0.002803224,"stat_sum":{"num_bytes":18486267,"num_objects":142,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":26,"num_read_kb":4106,"num_write":272,"num_write_kb":22293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"53'56","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:02.635387+0000","last_change":"2023-10-06T04:11:02.635352+0000","last_active":"2023-10-06T04:11:02.635387+0000","last_peered":"2023-10-06T04:11:02.635387+0000","last_clean":"2023-10-06T04:11:02.635387+0000","last_became_active":"2023-10-06T03:57:10.347433+0000","last_became_peered":"2023-10-06T03:57:10.347433+0000","last_unstale":"2023-10-06T04:11:02.635387+0000","last_undegraded":"2023-10-06T04:11:02.635387+0000","last_fullsized":"2023-10-06T04:11:02.635387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'56","last_scrub_stamp":"2023-10-06T04:11:02.635240+0000","last_deep_scrub":"53'56","last_deep_scrub_stamp":"2023-10-06T04:11:02.635240+0000","last_clean_scrub_stamp":"2023-10-06T04:11:02.635240+0000","objects_scrubbed":4,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:08.757110+0000","scrub_duration":0.010252476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":11,"num_read_kb":8201,"num_write":54,"num_write_kb":4137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17420,"num_omap_keys":31,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"53'338","reported_seq":2440,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.142293+0000","last_change":"2023-10-06T04:05:02.718759+0000","last_active":"2023-10-06T04:10:55.142293+0000","last_peered":"2023-10-06T04:10:55.142293+0000","last_clean":"2023-10-06T04:10:55.142293+0000","last_became_active":"2023-10-06T03:57:11.369825+0000","last_became_peered":"2023-10-06T03:57:11.369825+0000","last_unstale":"2023-10-06T04:10:55.142293+0000","last_undegraded":"2023-10-06T04:10:55.142293+0000","last_fullsized":"2023-10-06T04:10:55.142293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014204689999999999,"stat_sum":{"num_bytes":11391239,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":35,"num_read_kb":8210,"num_write":309,"num_write_kb":15384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:09.870140+0000","last_change":"2023-10-06T04:11:09.870107+0000","last_active":"2023-10-06T04:11:09.870140+0000","last_peered":"2023-10-06T04:11:09.870140+0000","last_clean":"2023-10-06T04:11:09.870140+0000","last_became_active":"2023-10-06T03:57:05.663046+0000","last_became_peered":"2023-10-06T03:57:05.663046+0000","last_unstale":"2023-10-06T04:11:09.870140+0000","last_undegraded":"2023-10-06T04:11:09.870140+0000","last_fullsized":"2023-10-06T04:11:09.870140+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:11:09.869990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:11:09.869990+0000","last_clean_scrub_stamp":"2023-10-06T04:11:09.869990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:07.282698+0000","scrub_duration":0.001426486,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"53'43","reported_seq":101,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:12.801160+0000","last_change":"2023-10-06T04:11:12.801137+0000","last_active":"2023-10-06T04:11:12.801160+0000","last_peered":"2023-10-06T04:11:12.801160+0000","last_clean":"2023-10-06T04:11:12.801160+0000","last_became_active":"2023-10-06T03:57:10.348608+0000","last_became_peered":"2023-10-06T03:57:10.348608+0000","last_unstale":"2023-10-06T04:11:12.801160+0000","last_undegraded":"2023-10-06T04:11:12.801160+0000","last_fullsized":"2023-10-06T04:11:12.801160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'43","last_scrub_stamp":"2023-10-06T04:11:12.801071+0000","last_deep_scrub":"53'43","last_deep_scrub_stamp":"2023-10-06T04:11:12.801071+0000","last_clean_scrub_stamp":"2023-10-06T04:11:12.801071+0000","objects_scrubbed":6,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:46.340363+0000","scrub_duration":0.010232031000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194416,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":39,"num_write_kb":4139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":21608,"num_omap_keys":37,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"53'310","reported_seq":2407,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.130450+0000","last_change":"2023-10-06T04:05:02.718555+0000","last_active":"2023-10-06T04:10:55.130450+0000","last_peered":"2023-10-06T04:10:55.130450+0000","last_clean":"2023-10-06T04:10:55.130450+0000","last_became_active":"2023-10-06T03:57:11.370847+0000","last_became_peered":"2023-10-06T03:57:11.370847+0000","last_unstale":"2023-10-06T04:10:55.130450+0000","last_undegraded":"2023-10-06T04:10:55.130450+0000","last_fullsized":"2023-10-06T04:10:55.130450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013685990000000001,"stat_sum":{"num_bytes":14926709,"num_objects":139,"num_object_clones":0,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":0,"num_read":29,"num_read_kb":12301,"num_write":289,"num_write_kb":18822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:02.948635+0000","last_change":"2023-10-06T04:11:02.948611+0000","last_active":"2023-10-06T04:11:02.948635+0000","last_peered":"2023-10-06T04:11:02.948635+0000","last_clean":"2023-10-06T04:11:02.948635+0000","last_became_active":"2023-10-06T03:57:05.318572+0000","last_became_peered":"2023-10-06T03:57:05.318572+0000","last_unstale":"2023-10-06T04:11:02.948635+0000","last_undegraded":"2023-10-06T04:11:02.948635+0000","last_fullsized":"2023-10-06T04:11:02.948635+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:11:02.948529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:11:02.948529+0000","last_clean_scrub_stamp":"2023-10-06T04:11:02.948529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:16.383610+0000","scrub_duration":0.0014570340000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"53'16","reported_seq":77,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:15.846630+0000","last_change":"2023-10-06T04:11:15.846599+0000","last_active":"2023-10-06T04:11:15.846630+0000","last_peered":"2023-10-06T04:11:15.846630+0000","last_clean":"2023-10-06T04:11:15.846630+0000","last_became_active":"2023-10-06T03:57:10.348271+0000","last_became_peered":"2023-10-06T03:57:10.348271+0000","last_unstale":"2023-10-06T04:11:15.846630+0000","last_undegraded":"2023-10-06T04:11:15.846630+0000","last_fullsized":"2023-10-06T04:11:15.846630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2023-10-06T04:11:15.846508+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2023-10-06T04:11:15.846508+0000","last_clean_scrub_stamp":"2023-10-06T04:11:15.846508+0000","objects_scrubbed":6,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:43.610039+0000","scrub_duration":0.011518349000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":26,"num_write":12,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17144,"num_omap_keys":31,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"53'349","reported_seq":2450,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.125933+0000","last_change":"2023-10-06T04:05:02.719009+0000","last_active":"2023-10-06T04:10:55.125933+0000","last_peered":"2023-10-06T04:10:55.125933+0000","last_clean":"2023-10-06T04:10:55.125933+0000","last_became_active":"2023-10-06T03:57:11.365006+0000","last_became_peered":"2023-10-06T03:57:11.365006+0000","last_unstale":"2023-10-06T04:10:55.125933+0000","last_undegraded":"2023-10-06T04:10:55.125933+0000","last_fullsized":"2023-10-06T04:10:55.125933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015961910000000001,"stat_sum":{"num_bytes":5739830,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":33,"num_read_kb":4109,"num_write":322,"num_write_kb":5775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:06.413155+0000","last_change":"2023-10-06T04:11:06.413102+0000","last_active":"2023-10-06T04:11:06.413155+0000","last_peered":"2023-10-06T04:11:06.413155+0000","last_clean":"2023-10-06T04:11:06.413155+0000","last_became_active":"2023-10-06T03:57:05.314408+0000","last_became_peered":"2023-10-06T03:57:05.314408+0000","last_unstale":"2023-10-06T04:11:06.413155+0000","last_undegraded":"2023-10-06T04:11:06.413155+0000","last_fullsized":"2023-10-06T04:11:06.413155+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:11:06.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:11:06.412975+0000","last_clean_scrub_stamp":"2023-10-06T04:11:06.412975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:42:44.133543+0000","scrub_duration":0.0018344570000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"53'365","reported_seq":422,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:11.809107+0000","last_change":"2023-10-06T04:11:11.809083+0000","last_active":"2023-10-06T04:11:11.809107+0000","last_peered":"2023-10-06T04:11:11.809107+0000","last_clean":"2023-10-06T04:11:11.809107+0000","last_became_active":"2023-10-06T03:57:10.345769+0000","last_became_peered":"2023-10-06T03:57:10.345769+0000","last_unstale":"2023-10-06T04:11:11.809107+0000","last_undegraded":"2023-10-06T04:11:11.809107+0000","last_fullsized":"2023-10-06T04:11:11.809107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'365","last_scrub_stamp":"2023-10-06T04:11:11.809003+0000","last_deep_scrub":"53'365","last_deep_scrub_stamp":"2023-10-06T04:11:11.809003+0000","last_clean_scrub_stamp":"2023-10-06T04:11:11.809003+0000","objects_scrubbed":6,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:05.207133+0000","scrub_duration":0.020844386999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":352,"num_write_kb":12528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":20402,"num_omap_keys":35,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"53'331","reported_seq":2438,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.129495+0000","last_change":"2023-10-06T04:05:02.718334+0000","last_active":"2023-10-06T04:10:55.129495+0000","last_peered":"2023-10-06T04:10:55.129495+0000","last_clean":"2023-10-06T04:10:55.129495+0000","last_became_active":"2023-10-06T03:57:11.370076+0000","last_became_peered":"2023-10-06T03:57:11.370076+0000","last_unstale":"2023-10-06T04:10:55.129495+0000","last_undegraded":"2023-10-06T04:10:55.129495+0000","last_fullsized":"2023-10-06T04:10:55.129495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011772849999999999,"stat_sum":{"num_bytes":18548829,"num_objects":146,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":146,"num_whiteouts":0,"num_read":39,"num_read_kb":4112,"num_write":310,"num_write_kb":22369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:08.939772+0000","last_change":"2023-10-06T04:11:08.939745+0000","last_active":"2023-10-06T04:11:08.939772+0000","last_peered":"2023-10-06T04:11:08.939772+0000","last_clean":"2023-10-06T04:11:08.939772+0000","last_became_active":"2023-10-06T03:57:05.318456+0000","last_became_peered":"2023-10-06T03:57:05.318456+0000","last_unstale":"2023-10-06T04:11:08.939772+0000","last_undegraded":"2023-10-06T04:11:08.939772+0000","last_fullsized":"2023-10-06T04:11:08.939772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:11:08.939596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:11:08.939596+0000","last_clean_scrub_stamp":"2023-10-06T04:11:08.939596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:34.690483+0000","scrub_duration":0.0022435490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"53'119","reported_seq":208,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:03.558037+0000","last_change":"2023-10-06T04:11:03.558012+0000","last_active":"2023-10-06T04:11:03.558037+0000","last_peered":"2023-10-06T04:11:03.558037+0000","last_clean":"2023-10-06T04:11:03.558037+0000","last_became_active":"2023-10-06T03:57:05.663426+0000","last_became_peered":"2023-10-06T03:57:05.663426+0000","last_unstale":"2023-10-06T04:11:03.558037+0000","last_undegraded":"2023-10-06T04:11:03.558037+0000","last_fullsized":"2023-10-06T04:11:03.558037+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'119","last_scrub_stamp":"2023-10-06T04:11:03.557930+0000","last_deep_scrub":"53'119","last_deep_scrub_stamp":"2023-10-06T04:11:03.557930+0000","last_clean_scrub_stamp":"2023-10-06T04:11:03.557930+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:55:23.076482+0000","scrub_duration":0.0036399879999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"53'724","reported_seq":786,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:03.652644+0000","last_change":"2023-10-06T04:11:03.652622+0000","last_active":"2023-10-06T04:11:03.652644+0000","last_peered":"2023-10-06T04:11:03.652644+0000","last_clean":"2023-10-06T04:11:03.652644+0000","last_became_active":"2023-10-06T03:57:10.348879+0000","last_became_peered":"2023-10-06T03:57:10.348879+0000","last_unstale":"2023-10-06T04:11:03.652644+0000","last_undegraded":"2023-10-06T04:11:03.652644+0000","last_fullsized":"2023-10-06T04:11:03.652644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'724","last_scrub_stamp":"2023-10-06T04:11:03.652510+0000","last_deep_scrub":"53'724","last_deep_scrub_stamp":"2023-10-06T04:11:03.652510+0000","last_clean_scrub_stamp":"2023-10-06T04:11:03.652510+0000","objects_scrubbed":5,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:03.918796+0000","scrub_duration":0.024355216999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9487985,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":20487,"num_write":715,"num_write_kb":13783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9328,"num_omap_keys":17,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"53'450","reported_seq":2570,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:15.162025+0000","last_change":"2023-10-06T04:11:15.162001+0000","last_active":"2023-10-06T04:11:15.162025+0000","last_peered":"2023-10-06T04:11:15.162025+0000","last_clean":"2023-10-06T04:11:15.162025+0000","last_became_active":"2023-10-06T03:57:11.373452+0000","last_became_peered":"2023-10-06T03:57:11.373452+0000","last_unstale":"2023-10-06T04:11:15.162025+0000","last_undegraded":"2023-10-06T04:11:15.162025+0000","last_fullsized":"2023-10-06T04:11:15.162025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'450","last_scrub_stamp":"2023-10-06T04:11:15.161925+0000","last_deep_scrub":"53'450","last_deep_scrub_stamp":"2023-10-06T04:11:15.161925+0000","last_clean_scrub_stamp":"2023-10-06T04:11:15.161925+0000","objects_scrubbed":165,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:13.301939+0000","scrub_duration":0.119156151,"objects_trimmed":0,"snaptrim_duration":0.0018153620000000001,"stat_sum":{"num_bytes":5937326,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":36,"num_read_kb":2733,"num_write":331,"num_write_kb":10073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:07.651843+0000","last_change":"2023-10-06T04:11:07.651815+0000","last_active":"2023-10-06T04:11:07.651843+0000","last_peered":"2023-10-06T04:11:07.651843+0000","last_clean":"2023-10-06T04:11:07.651843+0000","last_became_active":"2023-10-06T03:57:05.318531+0000","last_became_peered":"2023-10-06T03:57:05.318531+0000","last_unstale":"2023-10-06T04:11:07.651843+0000","last_undegraded":"2023-10-06T04:11:07.651843+0000","last_fullsized":"2023-10-06T04:11:07.651843+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:11:07.651712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:11:07.651712+0000","last_clean_scrub_stamp":"2023-10-06T04:11:07.651712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:16.562485+0000","scrub_duration":0.0020165349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"53'545","reported_seq":719,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:16.314725+0000","last_change":"2023-10-06T04:11:16.314684+0000","last_active":"2023-10-06T04:11:16.314725+0000","last_peered":"2023-10-06T04:11:16.314725+0000","last_clean":"2023-10-06T04:11:16.314725+0000","last_became_active":"2023-10-06T03:57:10.343051+0000","last_became_peered":"2023-10-06T03:57:10.343051+0000","last_unstale":"2023-10-06T04:11:16.314725+0000","last_undegraded":"2023-10-06T04:11:16.314725+0000","last_fullsized":"2023-10-06T04:11:16.314725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'545","last_scrub_stamp":"2023-10-06T04:11:16.314572+0000","last_deep_scrub":"53'545","last_deep_scrub_stamp":"2023-10-06T04:11:16.314572+0000","last_clean_scrub_stamp":"2023-10-06T04:11:16.314572+0000","objects_scrubbed":5,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:43.647007+0000","scrub_duration":0.014138608,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4571457,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":122,"num_read_kb":3989,"num_write":534,"num_write_kb":4895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"53'513","reported_seq":2634,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:22.415858+0000","last_change":"2023-10-06T04:11:22.415833+0000","last_active":"2023-10-06T04:11:22.415858+0000","last_peered":"2023-10-06T04:11:22.415858+0000","last_clean":"2023-10-06T04:11:22.415858+0000","last_became_active":"2023-10-06T03:57:11.373264+0000","last_became_peered":"2023-10-06T03:57:11.373264+0000","last_unstale":"2023-10-06T04:11:22.415858+0000","last_undegraded":"2023-10-06T04:11:22.415858+0000","last_fullsized":"2023-10-06T04:11:22.415858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'513","last_scrub_stamp":"2023-10-06T04:11:22.415754+0000","last_deep_scrub":"53'513","last_deep_scrub_stamp":"2023-10-06T04:11:22.415754+0000","last_clean_scrub_stamp":"2023-10-06T04:11:22.415754+0000","objects_scrubbed":168,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:30:17.810664+0000","scrub_duration":0.11956710500000001,"objects_trimmed":0,"snaptrim_duration":0.0026908990000000001,"stat_sum":{"num_bytes":10658865,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":37,"num_read_kb":4121,"num_write":334,"num_write_kb":10588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:04.597831+0000","last_change":"2023-10-06T04:11:04.597785+0000","last_active":"2023-10-06T04:11:04.597831+0000","last_peered":"2023-10-06T04:11:04.597831+0000","last_clean":"2023-10-06T04:11:04.597831+0000","last_became_active":"2023-10-06T03:57:05.663501+0000","last_became_peered":"2023-10-06T03:57:05.663501+0000","last_unstale":"2023-10-06T04:11:04.597831+0000","last_undegraded":"2023-10-06T04:11:04.597831+0000","last_fullsized":"2023-10-06T04:11:04.597831+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:11:04.597657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:11:04.597657+0000","last_clean_scrub_stamp":"2023-10-06T04:11:04.597657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:09.111900+0000","scrub_duration":0.0020988000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"53'7","reported_seq":60,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:13.746940+0000","last_change":"2023-10-06T04:11:13.746917+0000","last_active":"2023-10-06T04:11:13.746940+0000","last_peered":"2023-10-06T04:11:13.746940+0000","last_clean":"2023-10-06T04:11:13.746940+0000","last_became_active":"2023-10-06T03:57:10.345743+0000","last_became_peered":"2023-10-06T03:57:10.345743+0000","last_unstale":"2023-10-06T04:11:13.746940+0000","last_undegraded":"2023-10-06T04:11:13.746940+0000","last_fullsized":"2023-10-06T04:11:13.746940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7","last_scrub_stamp":"2023-10-06T04:11:13.746842+0000","last_deep_scrub":"53'7","last_deep_scrub_stamp":"2023-10-06T04:11:13.746842+0000","last_clean_scrub_stamp":"2023-10-06T04:11:13.746842+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:47:15.026107+0000","scrub_duration":0.0026376030000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20578,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"53'511","reported_seq":2622,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:19.489280+0000","last_change":"2023-10-06T04:11:19.489257+0000","last_active":"2023-10-06T04:11:19.489280+0000","last_peered":"2023-10-06T04:11:19.489280+0000","last_clean":"2023-10-06T04:11:19.489280+0000","last_became_active":"2023-10-06T03:57:11.373543+0000","last_became_peered":"2023-10-06T03:57:11.373543+0000","last_unstale":"2023-10-06T04:11:19.489280+0000","last_undegraded":"2023-10-06T04:11:19.489280+0000","last_fullsized":"2023-10-06T04:11:19.489280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'511","last_scrub_stamp":"2023-10-06T04:11:19.489194+0000","last_deep_scrub":"53'511","last_deep_scrub_stamp":"2023-10-06T04:11:19.489194+0000","last_clean_scrub_stamp":"2023-10-06T04:11:19.489194+0000","objects_scrubbed":155,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:28.970043+0000","scrub_duration":0.13727003800000001,"objects_trimmed":0,"snaptrim_duration":0.0020558070000000002,"stat_sum":{"num_bytes":6540215,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":27,"num_read_kb":14,"num_write":340,"num_write_kb":6519,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"53'18","reported_seq":78,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:04.624957+0000","last_change":"2023-10-06T04:11:04.624929+0000","last_active":"2023-10-06T04:11:04.624957+0000","last_peered":"2023-10-06T04:11:04.624957+0000","last_clean":"2023-10-06T04:11:04.624957+0000","last_became_active":"2023-10-06T03:57:10.344324+0000","last_became_peered":"2023-10-06T03:57:10.344324+0000","last_unstale":"2023-10-06T04:11:04.624957+0000","last_undegraded":"2023-10-06T04:11:04.624957+0000","last_fullsized":"2023-10-06T04:11:04.624957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'18","last_scrub_stamp":"2023-10-06T04:11:04.624797+0000","last_deep_scrub":"53'18","last_deep_scrub_stamp":"2023-10-06T04:11:04.624797+0000","last_clean_scrub_stamp":"2023-10-06T04:11:04.624797+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:02.674217+0000","scrub_duration":0.0092723449999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":22,"num_read_kb":21,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:01.951088+0000","last_change":"2023-10-06T04:11:01.951066+0000","last_active":"2023-10-06T04:11:01.951088+0000","last_peered":"2023-10-06T04:11:01.951088+0000","last_clean":"2023-10-06T04:11:01.951088+0000","last_became_active":"2023-10-06T03:57:05.317715+0000","last_became_peered":"2023-10-06T03:57:05.317715+0000","last_unstale":"2023-10-06T04:11:01.951088+0000","last_undegraded":"2023-10-06T04:11:01.951088+0000","last_fullsized":"2023-10-06T04:11:01.951088+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:11:01.950987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:11:01.950987+0000","last_clean_scrub_stamp":"2023-10-06T04:11:01.950987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:51.601489+0000","scrub_duration":0.0016264490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"53'331","reported_seq":2431,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.146028+0000","last_change":"2023-10-06T04:05:02.721255+0000","last_active":"2023-10-06T04:10:55.146028+0000","last_peered":"2023-10-06T04:10:55.146028+0000","last_clean":"2023-10-06T04:10:55.146028+0000","last_became_active":"2023-10-06T03:57:11.369445+0000","last_became_peered":"2023-10-06T03:57:11.369445+0000","last_unstale":"2023-10-06T04:10:55.146028+0000","last_undegraded":"2023-10-06T04:10:55.146028+0000","last_fullsized":"2023-10-06T04:10:55.146028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015173949999999999,"stat_sum":{"num_bytes":11743957,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":32,"num_read_kb":17,"num_write":311,"num_write_kb":11603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:02.976078+0000","last_change":"2023-10-06T04:11:02.976046+0000","last_active":"2023-10-06T04:11:02.976078+0000","last_peered":"2023-10-06T04:11:02.976078+0000","last_clean":"2023-10-06T04:11:02.976078+0000","last_became_active":"2023-10-06T03:57:05.320583+0000","last_became_peered":"2023-10-06T03:57:05.320583+0000","last_unstale":"2023-10-06T04:11:02.976078+0000","last_undegraded":"2023-10-06T04:11:02.976078+0000","last_fullsized":"2023-10-06T04:11:02.976078+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:11:02.975967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:11:02.975967+0000","last_clean_scrub_stamp":"2023-10-06T04:11:02.975967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:09.044661+0000","scrub_duration":0.001182069,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"53'96","reported_seq":151,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:18.309985+0000","last_change":"2023-10-06T04:11:18.309956+0000","last_active":"2023-10-06T04:11:18.309985+0000","last_peered":"2023-10-06T04:11:18.309985+0000","last_clean":"2023-10-06T04:11:18.309985+0000","last_became_active":"2023-10-06T03:57:10.339424+0000","last_became_peered":"2023-10-06T03:57:10.339424+0000","last_unstale":"2023-10-06T04:11:18.309985+0000","last_undegraded":"2023-10-06T04:11:18.309985+0000","last_fullsized":"2023-10-06T04:11:18.309985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'96","last_scrub_stamp":"2023-10-06T04:11:18.309874+0000","last_deep_scrub":"53'96","last_deep_scrub_stamp":"2023-10-06T04:11:18.309874+0000","last_clean_scrub_stamp":"2023-10-06T04:11:18.309874+0000","objects_scrubbed":4,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:47.802237+0000","scrub_duration":0.006477053,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":19,"num_write":221,"num_write_kb":1011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"53'335","reported_seq":2431,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.121376+0000","last_change":"2023-10-06T04:05:02.722365+0000","last_active":"2023-10-06T04:10:55.121376+0000","last_peered":"2023-10-06T04:10:55.121376+0000","last_clean":"2023-10-06T04:10:55.121376+0000","last_became_active":"2023-10-06T03:57:11.372215+0000","last_became_peered":"2023-10-06T03:57:11.372215+0000","last_unstale":"2023-10-06T04:10:55.121376+0000","last_undegraded":"2023-10-06T04:10:55.121376+0000","last_fullsized":"2023-10-06T04:10:55.121376+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020199100000000002,"stat_sum":{"num_bytes":14603524,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":28,"num_read_kb":4106,"num_write":312,"num_write_kb":14355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"53'211","reported_seq":275,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:14.290761+0000","last_change":"2023-10-06T04:11:14.290711+0000","last_active":"2023-10-06T04:11:14.290761+0000","last_peered":"2023-10-06T04:11:14.290761+0000","last_clean":"2023-10-06T04:11:14.290761+0000","last_became_active":"2023-10-06T03:57:10.343865+0000","last_became_peered":"2023-10-06T03:57:10.343865+0000","last_unstale":"2023-10-06T04:11:14.290761+0000","last_undegraded":"2023-10-06T04:11:14.290761+0000","last_fullsized":"2023-10-06T04:11:14.290761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'211","last_scrub_stamp":"2023-10-06T04:11:14.290584+0000","last_deep_scrub":"53'211","last_deep_scrub_stamp":"2023-10-06T04:11:14.290584+0000","last_clean_scrub_stamp":"2023-10-06T04:11:14.290584+0000","objects_scrubbed":9,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:30.042251+0000","scrub_duration":0.028391895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":11,"num_read_kb":32771,"num_write":206,"num_write_kb":12488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":51091,"num_omap_keys":90,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"53'505","reported_seq":2622,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:20.497794+0000","last_change":"2023-10-06T04:11:20.497772+0000","last_active":"2023-10-06T04:11:20.497794+0000","last_peered":"2023-10-06T04:11:20.497794+0000","last_clean":"2023-10-06T04:11:20.497794+0000","last_became_active":"2023-10-06T03:57:11.369681+0000","last_became_peered":"2023-10-06T03:57:11.369681+0000","last_unstale":"2023-10-06T04:11:20.497794+0000","last_undegraded":"2023-10-06T04:11:20.497794+0000","last_fullsized":"2023-10-06T04:11:20.497794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'505","last_scrub_stamp":"2023-10-06T04:11:20.497703+0000","last_deep_scrub":"53'505","last_deep_scrub_stamp":"2023-10-06T04:11:20.497703+0000","last_clean_scrub_stamp":"2023-10-06T04:11:20.497703+0000","objects_scrubbed":158,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:42.201624+0000","scrub_duration":0.099109163,"objects_trimmed":0,"snaptrim_duration":0.0013151409999999999,"stat_sum":{"num_bytes":9655954,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":33,"num_read_kb":14,"num_write":334,"num_write_kb":13707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"53'104","reported_seq":164,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:17.710929+0000","last_change":"2023-10-06T04:11:17.710901+0000","last_active":"2023-10-06T04:11:17.710929+0000","last_peered":"2023-10-06T04:11:17.710929+0000","last_clean":"2023-10-06T04:11:17.710929+0000","last_became_active":"2023-10-06T03:57:10.349320+0000","last_became_peered":"2023-10-06T03:57:10.349320+0000","last_unstale":"2023-10-06T04:11:17.710929+0000","last_undegraded":"2023-10-06T04:11:17.710929+0000","last_fullsized":"2023-10-06T04:11:17.710929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'104","last_scrub_stamp":"2023-10-06T04:11:17.710816+0000","last_deep_scrub":"53'104","last_deep_scrub_stamp":"2023-10-06T04:11:17.710816+0000","last_clean_scrub_stamp":"2023-10-06T04:11:17.710816+0000","objects_scrubbed":4,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:46.926120+0000","scrub_duration":0.01386691,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194858,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":4110,"num_write":99,"num_write_kb":4143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"53'441","reported_seq":2552,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:10.921994+0000","last_change":"2023-10-06T04:11:10.921969+0000","last_active":"2023-10-06T04:11:10.921994+0000","last_peered":"2023-10-06T04:11:10.921994+0000","last_clean":"2023-10-06T04:11:10.921994+0000","last_became_active":"2023-10-06T03:57:11.370229+0000","last_became_peered":"2023-10-06T03:57:11.370229+0000","last_unstale":"2023-10-06T04:11:10.921994+0000","last_undegraded":"2023-10-06T04:11:10.921994+0000","last_fullsized":"2023-10-06T04:11:10.921994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'441","last_scrub_stamp":"2023-10-06T04:11:10.921900+0000","last_deep_scrub":"53'441","last_deep_scrub_stamp":"2023-10-06T04:11:10.921900+0000","last_clean_scrub_stamp":"2023-10-06T04:11:10.921900+0000","objects_scrubbed":155,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:16.456865+0000","scrub_duration":0.100086199,"objects_trimmed":0,"snaptrim_duration":0.0027851170000000001,"stat_sum":{"num_bytes":11865278,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":27,"num_read_kb":17,"num_write":315,"num_write_kb":11749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"53'250","reported_seq":322,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:16.743648+0000","last_change":"2023-10-06T04:11:16.743623+0000","last_active":"2023-10-06T04:11:16.743648+0000","last_peered":"2023-10-06T04:11:16.743648+0000","last_clean":"2023-10-06T04:11:16.743648+0000","last_became_active":"2023-10-06T03:57:10.350083+0000","last_became_peered":"2023-10-06T03:57:10.350083+0000","last_unstale":"2023-10-06T04:11:16.743648+0000","last_undegraded":"2023-10-06T04:11:16.743648+0000","last_fullsized":"2023-10-06T04:11:16.743648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'250","last_scrub_stamp":"2023-10-06T04:11:16.743549+0000","last_deep_scrub":"53'250","last_deep_scrub_stamp":"2023-10-06T04:11:16.743549+0000","last_clean_scrub_stamp":"2023-10-06T04:11:16.743549+0000","objects_scrubbed":12,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:57.567358+0000","scrub_duration":0.028247550999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":36,"num_read_kb":45128,"num_write":264,"num_write_kb":12512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19684,"num_omap_keys":36,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"53'350","reported_seq":2448,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.140594+0000","last_change":"2023-10-06T04:05:02.724048+0000","last_active":"2023-10-06T04:10:55.140594+0000","last_peered":"2023-10-06T04:10:55.140594+0000","last_clean":"2023-10-06T04:10:55.140594+0000","last_became_active":"2023-10-06T03:57:11.369123+0000","last_became_peered":"2023-10-06T03:57:11.369123+0000","last_unstale":"2023-10-06T04:10:55.140594+0000","last_undegraded":"2023-10-06T04:10:55.140594+0000","last_fullsized":"2023-10-06T04:10:55.140594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028613789999999998,"stat_sum":{"num_bytes":15066743,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":30,"num_read_kb":4112,"num_write":322,"num_write_kb":14878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"53'183","reported_seq":238,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:06.608805+0000","last_change":"2023-10-06T04:11:06.608778+0000","last_active":"2023-10-06T04:11:06.608805+0000","last_peered":"2023-10-06T04:11:06.608805+0000","last_clean":"2023-10-06T04:11:06.608805+0000","last_became_active":"2023-10-06T03:57:10.344583+0000","last_became_peered":"2023-10-06T03:57:10.344583+0000","last_unstale":"2023-10-06T04:11:06.608805+0000","last_undegraded":"2023-10-06T04:11:06.608805+0000","last_fullsized":"2023-10-06T04:11:06.608805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'183","last_scrub_stamp":"2023-10-06T04:11:06.608691+0000","last_deep_scrub":"53'183","last_deep_scrub_stamp":"2023-10-06T04:11:06.608691+0000","last_clean_scrub_stamp":"2023-10-06T04:11:06.608691+0000","objects_scrubbed":2,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:46:02.740119+0000","scrub_duration":0.010859541,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":177,"num_write_kb":8286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8760,"num_omap_keys":15,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"53'352","reported_seq":2457,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.145062+0000","last_change":"2023-10-06T04:05:02.721502+0000","last_active":"2023-10-06T04:10:55.145062+0000","last_peered":"2023-10-06T04:10:55.145062+0000","last_clean":"2023-10-06T04:10:55.145062+0000","last_became_active":"2023-10-06T03:57:11.366500+0000","last_became_peered":"2023-10-06T03:57:11.366500+0000","last_unstale":"2023-10-06T04:10:55.145062+0000","last_undegraded":"2023-10-06T04:10:55.145062+0000","last_fullsized":"2023-10-06T04:10:55.145062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016878310000000001,"stat_sum":{"num_bytes":6166988,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":38,"num_read_kb":4113,"num_write":326,"num_write_kb":10290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"53'178","reported_seq":239,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:09.230941+0000","last_change":"2023-10-06T04:11:09.230916+0000","last_active":"2023-10-06T04:11:09.230941+0000","last_peered":"2023-10-06T04:11:09.230941+0000","last_clean":"2023-10-06T04:11:09.230941+0000","last_became_active":"2023-10-06T03:57:10.347897+0000","last_became_peered":"2023-10-06T03:57:10.347897+0000","last_unstale":"2023-10-06T04:11:09.230941+0000","last_undegraded":"2023-10-06T04:11:09.230941+0000","last_fullsized":"2023-10-06T04:11:09.230941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'178","last_scrub_stamp":"2023-10-06T04:11:09.230838+0000","last_deep_scrub":"53'178","last_deep_scrub_stamp":"2023-10-06T04:11:09.230838+0000","last_clean_scrub_stamp":"2023-10-06T04:11:09.230838+0000","objects_scrubbed":8,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:21.538557+0000","scrub_duration":0.018787900999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7191702,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":12,"num_read_kb":8202,"num_write":168,"num_write_kb":7111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"53'410","reported_seq":2512,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.131194+0000","last_change":"2023-10-06T04:05:02.721899+0000","last_active":"2023-10-06T04:10:55.131194+0000","last_peered":"2023-10-06T04:10:55.131194+0000","last_clean":"2023-10-06T04:10:55.131194+0000","last_became_active":"2023-10-06T03:57:11.364150+0000","last_became_peered":"2023-10-06T03:57:11.364150+0000","last_unstale":"2023-10-06T04:10:55.131194+0000","last_undegraded":"2023-10-06T04:10:55.131194+0000","last_fullsized":"2023-10-06T04:10:55.131194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001670408,"stat_sum":{"num_bytes":10944544,"num_objects":176,"num_object_clones":0,"num_object_copies":352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":176,"num_whiteouts":0,"num_read":35,"num_read_kb":7178,"num_write":386,"num_write_kb":19084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"53'466","reported_seq":2604,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:11.117345+0000","last_change":"2023-10-06T04:11:11.117321+0000","last_active":"2023-10-06T04:11:11.117345+0000","last_peered":"2023-10-06T04:11:11.117345+0000","last_clean":"2023-10-06T04:11:11.117345+0000","last_became_active":"2023-10-06T03:57:11.366524+0000","last_became_peered":"2023-10-06T03:57:11.366524+0000","last_unstale":"2023-10-06T04:11:11.117345+0000","last_undegraded":"2023-10-06T04:11:11.117345+0000","last_fullsized":"2023-10-06T04:11:11.117345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'466","last_scrub_stamp":"2023-10-06T04:11:11.117253+0000","last_deep_scrub":"53'466","last_deep_scrub_stamp":"2023-10-06T04:11:11.117253+0000","last_clean_scrub_stamp":"2023-10-06T04:11:11.117253+0000","objects_scrubbed":151,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:09.926514+0000","scrub_duration":0.109297006,"objects_trimmed":0,"snaptrim_duration":0.0017816290000000001,"stat_sum":{"num_bytes":22594948,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":54,"num_read_kb":16279,"num_write":343,"num_write_kb":22243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"53'380","reported_seq":2485,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.144474+0000","last_change":"2023-10-06T04:05:02.721559+0000","last_active":"2023-10-06T04:10:55.144474+0000","last_peered":"2023-10-06T04:10:55.144474+0000","last_clean":"2023-10-06T04:10:55.144474+0000","last_became_active":"2023-10-06T03:57:11.370848+0000","last_became_peered":"2023-10-06T03:57:11.370848+0000","last_unstale":"2023-10-06T04:10:55.144474+0000","last_undegraded":"2023-10-06T04:10:55.144474+0000","last_fullsized":"2023-10-06T04:10:55.144474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017239289999999999,"stat_sum":{"num_bytes":12161208,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":37,"num_read_kb":8210,"num_write":347,"num_write_kb":16110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"53'371","reported_seq":2478,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.130278+0000","last_change":"2023-10-06T04:05:02.723715+0000","last_active":"2023-10-06T04:10:55.130278+0000","last_peered":"2023-10-06T04:10:55.130278+0000","last_clean":"2023-10-06T04:10:55.130278+0000","last_became_active":"2023-10-06T03:57:11.363875+0000","last_became_peered":"2023-10-06T03:57:11.363875+0000","last_unstale":"2023-10-06T04:10:55.130278+0000","last_undegraded":"2023-10-06T04:10:55.130278+0000","last_fullsized":"2023-10-06T04:10:55.130278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028142470000000002,"stat_sum":{"num_bytes":14948732,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":39,"num_read_kb":4117,"num_write":348,"num_write_kb":14761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"53'505","reported_seq":2630,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:19.658777+0000","last_change":"2023-10-06T04:11:19.658754+0000","last_active":"2023-10-06T04:11:19.658777+0000","last_peered":"2023-10-06T04:11:19.658777+0000","last_clean":"2023-10-06T04:11:19.658777+0000","last_became_active":"2023-10-06T03:57:11.377126+0000","last_became_peered":"2023-10-06T03:57:11.377126+0000","last_unstale":"2023-10-06T04:11:19.658777+0000","last_undegraded":"2023-10-06T04:11:19.658777+0000","last_fullsized":"2023-10-06T04:11:19.658777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'505","last_scrub_stamp":"2023-10-06T04:11:19.658678+0000","last_deep_scrub":"53'505","last_deep_scrub_stamp":"2023-10-06T04:11:19.658678+0000","last_clean_scrub_stamp":"2023-10-06T04:11:19.658678+0000","objects_scrubbed":153,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:16.651951+0000","scrub_duration":0.108743853,"objects_trimmed":0,"snaptrim_duration":0.0023666569999999999,"stat_sum":{"num_bytes":6085663,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":41,"num_read_kb":18,"num_write":335,"num_write_kb":6110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"53'326","reported_seq":2424,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.124114+0000","last_change":"2023-10-06T04:05:02.972860+0000","last_active":"2023-10-06T04:10:55.124114+0000","last_peered":"2023-10-06T04:10:55.124114+0000","last_clean":"2023-10-06T04:10:55.124114+0000","last_became_active":"2023-10-06T03:57:59.256086+0000","last_became_peered":"2023-10-06T03:57:59.256086+0000","last_unstale":"2023-10-06T04:10:55.124114+0000","last_undegraded":"2023-10-06T04:10:55.124114+0000","last_fullsized":"2023-10-06T04:10:55.124114+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018573859999999999,"stat_sum":{"num_bytes":11350301,"num_objects":142,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":33,"num_read_kb":11479,"num_write":309,"num_write_kb":14473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":923606083,"num_objects":10287,"num_object_clones":0,"num_object_copies":20574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10287,"num_whiteouts":0,"num_read":2243,"num_read_kb":380500,"num_write":20866,"num_write_kb":1045904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1683152896,"data_stored":1845672070,"data_compressed":160523048,"data_compressed_allocated":177569792,"data_compressed_original":393674752,"omap_allocated":0,"internal_metadata":0},"log_size":25137,"ondisk_log_size":25137,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":167943527,"num_objects":202,"num_object_clones":0,"num_object_copies":404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":202,"num_whiteouts":0,"num_read":555,"num_read_kb":254049,"num_write":6961,"num_write_kb":198850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":122,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":536325,"num_omap_keys":945,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":84697088,"data_stored":335887054,"data_compressed":45189204,"data_compressed_allocated":56885248,"data_compressed_original":308330496,"omap_allocated":0,"internal_metadata":0},"log_size":6879,"ondisk_log_size":6879,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64858,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575024,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":507132,"kb_used_data":209844,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93864708,"statfs":{"total":96636764160,"available":96117460992,"internally_reserved":0,"allocated":214880256,"data_stored":260104227,"data_compressed":22363353,"data_compressed_allocated":25411584,"data_compressed_original":76546048,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.55400000000000005,"15min":0.54500000000000004},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.626},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28399999999999997},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.88900000000000001},"last":0.59599999999999997}]},{"osd":1,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52200000000000002,"15min":0.52100000000000002},"min":{"1min":0.38400000000000001,"5min":0.312,"15min":0.28899999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.78000000000000003},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54300000000000004,"15min":0.52300000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78300000000000003,"5min":1.25,"15min":1.25},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54800000000000004,"15min":0.54900000000000004},"min":{"1min":0.33400000000000002,"5min":0.29799999999999999,"15min":0.28399999999999997},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57599999999999996,"15min":0.60699999999999998},"min":{"1min":0.34300000000000003,"5min":0.33700000000000002,"15min":0.33100000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":7.3739999999999997},"last":0.64700000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.318},"max":{"1min":0.80600000000000005,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.55400000000000005,"15min":0.55900000000000005},"min":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72899999999999998,"5min":0.78000000000000003,"15min":0.85299999999999998},"last":0.53500000000000003}]},{"osd":4,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.44,"15min":0.46400000000000002},"min":{"1min":0.30099999999999999,"5min":0.27400000000000002,"15min":0.251},"max":{"1min":0.64000000000000001,"5min":0.73599999999999999,"15min":1.0149999999999999},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.34300000000000003,"5min":0.33500000000000002,"15min":0.28299999999999997},"max":{"1min":0.68600000000000005,"5min":0.80200000000000005,"15min":0.90700000000000003},"last":0.48199999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.495,"15min":0.51100000000000001},"min":{"1min":0.316,"5min":0.29999999999999999,"15min":0.27300000000000002},"max":{"1min":0.74199999999999999,"5min":0.82999999999999996,"15min":0.86099999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54600000000000004,"15min":0.56299999999999994},"min":{"1min":0.40699999999999997,"5min":0.317,"15min":0.28000000000000003},"max":{"1min":0.73699999999999999,"5min":0.84899999999999998,"15min":1.377},"last":0.55700000000000005}]},{"osd":6,"last update":"Fri Oct 6 04:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.51100000000000001,"15min":0.56200000000000006},"min":{"1min":0.34100000000000003,"5min":0.30599999999999999,"15min":0.29999999999999999},"max":{"1min":0.77100000000000002,"5min":0.77600000000000002,"15min":1.3420000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60499999999999998,"15min":0.59099999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.30199999999999999},"max":{"1min":0.84599999999999997,"5min":0.88200000000000001,"15min":0.93700000000000006},"last":0.45600000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575023,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":474560,"kb_used_data":164984,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":93897280,"statfs":{"total":96636764160,"available":96150814720,"internally_reserved":0,"allocated":168943616,"data_stored":188218692,"data_compressed":15936082,"data_compressed_allocated":17805312,"data_compressed_original":42737664,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54800000000000004,"15min":0.53200000000000003},"min":{"1min":0.28799999999999998,"5min":0.26400000000000001,"15min":0.253},"max":{"1min":0.70399999999999996,"5min":2.613,"15min":2.613},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.58399999999999996,"15min":0.55800000000000005},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.33300000000000002},"max":{"1min":0.72299999999999998,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":0.99299999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53400000000000003,"15min":0.59599999999999997},"min":{"1min":0.28899999999999998,"5min":0.28499999999999998,"15min":0.27200000000000002},"max":{"1min":0.71599999999999997,"5min":2.3889999999999998,"15min":9.5519999999999996},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.59399999999999997,"15min":0.60199999999999998},"min":{"1min":0.33400000000000002,"5min":0.315,"15min":0.313},"max":{"1min":0.75800000000000001,"5min":6.8019999999999996,"15min":6.8019999999999996},"last":0.58699999999999997}]},{"osd":2,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.625,"15min":0.67300000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.29799999999999999},"max":{"1min":0.71799999999999997,"5min":5.7590000000000003,"15min":9.6769999999999996},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.623,"15min":0.67600000000000005},"min":{"1min":0.42199999999999999,"5min":0.33200000000000002,"15min":0.31},"max":{"1min":0.75,"5min":5.71,"15min":9.7309999999999999},"last":0.65800000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.54900000000000004,"15min":0.57499999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.29299999999999998},"max":{"1min":0.73899999999999999,"5min":0.99099999999999999,"15min":5.7359999999999998},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.59699999999999998,"15min":0.61299999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.308},"max":{"1min":0.68799999999999994,"5min":1.0149999999999999,"15min":5.7009999999999996},"last":0.55900000000000005}]},{"osd":4,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.48099999999999998,"15min":0.49399999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.63300000000000001,"5min":0.78600000000000003,"15min":1.0129999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54100000000000004,"15min":0.54900000000000004},"min":{"1min":0.32600000000000001,"5min":0.313,"15min":0.30399999999999999},"max":{"1min":0.70299999999999996,"5min":0.79800000000000004,"15min":1.026},"last":0.41999999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.49299999999999999,"15min":0.50800000000000001},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.77800000000000002,"5min":0.78200000000000003,"15min":0.995},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.66500000000000004,"5min":0.78500000000000003,"15min":0.83699999999999997},"last":0.36699999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.34899999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.77000000000000002,"5min":0.84199999999999997,"15min":0.98299999999999998},"last":0.443},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56499999999999995,"15min":0.57799999999999996},"min":{"1min":0.376,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80500000000000005,"5min":0.84699999999999998,"15min":0.90800000000000003},"last":0.35399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575023,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":578624,"kb_used_data":277240,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":93793216,"statfs":{"total":96636764160,"available":96044253184,"internally_reserved":0,"allocated":283893760,"data_stored":337771364,"data_compressed":32865898,"data_compressed_allocated":37007360,"data_compressed_original":100126720,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.91800000000000004,"15min":0.52000000000000002},"min":{"1min":0.22500000000000001,"5min":0.20799999999999999,"15min":0.192},"max":{"1min":0.56999999999999995,"5min":57.700000000000003,"15min":57.700000000000003},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.65200000000000002,"15min":0.441},"min":{"1min":0.23899999999999999,"5min":0.23300000000000001,"15min":0.186},"max":{"1min":0.64000000000000001,"5min":28.629999999999999,"15min":28.629999999999999},"last":0.372}]},{"osd":2,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.97899999999999998,"15min":0.57199999999999995},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.19600000000000001},"max":{"1min":0.59899999999999998,"5min":58.795000000000002,"15min":58.795000000000002},"last":0.499},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.67600000000000005,"15min":0.48999999999999999},"min":{"1min":0.32200000000000001,"5min":0.29599999999999999,"15min":0.23000000000000001},"max":{"1min":0.63400000000000001,"5min":25.722000000000001,"15min":25.722000000000001},"last":0.46100000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.59799999999999998,"15min":0.443},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.216},"max":{"1min":0.58499999999999996,"5min":22.920999999999999,"15min":22.920999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.63400000000000001,"15min":0.46800000000000003},"min":{"1min":0.29099999999999998,"5min":0.27300000000000002,"15min":0.22600000000000001},"max":{"1min":0.59799999999999998,"5min":22.797999999999998,"15min":22.797999999999998},"last":0.434}]},{"osd":4,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":1.2370000000000001,"15min":0.82399999999999995},"min":{"1min":0.443,"5min":0.38500000000000001,"15min":0.34499999999999997},"max":{"1min":1.0149999999999999,"5min":59.018999999999998,"15min":59.018999999999998},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":1.26,"15min":0.83499999999999996},"min":{"1min":0.50700000000000001,"5min":0.47399999999999998,"15min":0.316},"max":{"1min":1.0609999999999999,"5min":59.046999999999997,"15min":59.046999999999997},"last":0.93600000000000005}]},{"osd":5,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":1.21,"15min":0.79300000000000004},"min":{"1min":0.48599999999999999,"5min":0.40200000000000002,"15min":0.34399999999999997},"max":{"1min":0.90300000000000002,"5min":58.987000000000002,"15min":58.987000000000002},"last":0.874},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65700000000000003,"15min":0.59599999999999997},"min":{"1min":0.44900000000000001,"5min":0.42599999999999999,"15min":0.29399999999999998},"max":{"1min":0.97699999999999998,"5min":6.2140000000000004,"15min":6.2140000000000004},"last":0.85799999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65200000000000002,"15min":0.60399999999999998},"min":{"1min":0.44,"5min":0.41999999999999998,"15min":0.36599999999999999},"max":{"1min":0.95599999999999996,"5min":4.6020000000000003,"15min":4.6020000000000003},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":1.3,"15min":0.85999999999999999},"min":{"1min":0.49099999999999999,"5min":0.42799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94799999999999995,"5min":58.911999999999999,"15min":58.911999999999999},"last":0.91200000000000003}]},{"osd":7,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":1.2210000000000001,"15min":0.81100000000000005},"min":{"1min":0.45800000000000002,"5min":0.40100000000000002,"15min":0.38700000000000001},"max":{"1min":1.1000000000000001,"5min":58.953000000000003,"15min":58.953000000000003},"last":0.749},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.65900000000000003,"15min":0.62},"min":{"1min":0.46700000000000003,"5min":0.38100000000000001,"15min":0.34399999999999997},"max":{"1min":0.83199999999999996,"5min":4.5570000000000004,"15min":4.5570000000000004},"last":0.84399999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575023,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":496884,"kb_used_data":191404,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":93874956,"statfs":{"total":96636764160,"available":96127954944,"internally_reserved":0,"allocated":195997696,"data_stored":253566865,"data_compressed":26834878,"data_compressed_allocated":30744576,"data_compressed_original":94838784,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.34100000000000003,"15min":0.34899999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.19500000000000001},"max":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":6.9539999999999997},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.35599999999999998,"15min":0.371},"min":{"1min":0.22700000000000001,"5min":0.215,"15min":0.191},"max":{"1min":0.55700000000000005,"5min":0.59499999999999997,"15min":8.0519999999999996},"last":0.31}]},{"osd":2,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.36199999999999999,"15min":0.35899999999999999},"min":{"1min":0.24299999999999999,"5min":0.24199999999999999,"15min":0.214},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":1.0589999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.38500000000000001,"15min":0.376},"min":{"1min":0.28299999999999997,"5min":0.26100000000000001,"15min":0.20799999999999999},"max":{"1min":0.626,"5min":0.626,"15min":2.0489999999999999},"last":0.39400000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.38600000000000001,"15min":0.41799999999999998},"min":{"1min":0.23300000000000001,"5min":0.23100000000000001,"15min":0.223},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":9.391},"last":0.437},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.42599999999999999,"15min":0.45600000000000002},"min":{"1min":0.31900000000000001,"5min":0.29099999999999998,"15min":0.27800000000000002},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":9.5239999999999991},"last":0.55100000000000005}]},{"osd":4,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.64900000000000002,"15min":0.63400000000000001},"min":{"1min":0.35599999999999998,"5min":0.34000000000000002,"15min":0.29199999999999998},"max":{"1min":1.026,"5min":1.026,"15min":5.7699999999999996},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66000000000000003,"15min":0.63600000000000001},"min":{"1min":0.45400000000000001,"5min":0.41499999999999998,"15min":0.29499999999999998},"max":{"1min":0.997,"5min":0.997,"15min":2.7320000000000002},"last":0.50700000000000001}]},{"osd":5,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.45000000000000001,"5min":0.41399999999999998,"15min":0.33800000000000002},"max":{"1min":0.89900000000000002,"5min":0.90500000000000003,"15min":5.5069999999999997},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63100000000000001,"15min":0.60499999999999998},"min":{"1min":0.436,"5min":0.38500000000000001,"15min":0.34799999999999998},"max":{"1min":0.872,"5min":0.878,"15min":1.05},"last":0.46800000000000003}]},{"osd":6,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.626,"15min":0.64700000000000002},"min":{"1min":0.42599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":6.9139999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64800000000000002,"15min":0.65500000000000003},"min":{"1min":0.46700000000000003,"5min":0.42199999999999999,"15min":0.379},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":6.3780000000000001},"last":0.51900000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:10:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61099999999999999,"15min":0.623},"min":{"1min":0.38500000000000001,"5min":0.377,"15min":0.36699999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":5.9989999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.627,"15min":0.61099999999999999},"min":{"1min":0.443,"5min":0.40100000000000002,"15min":0.39700000000000002},"max":{"1min":0.83999999999999997,"5min":0.86499999999999999,"15min":2.6930000000000001},"last":0.58799999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575023,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442148,"kb_used_data":140764,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":93929692,"statfs":{"total":96636764160,"available":96184004608,"internally_reserved":0,"allocated":144142336,"data_stored":175617804,"data_compressed":19256061,"data_compressed_allocated":21770240,"data_compressed_original":59097088,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.34300000000000003,"15min":0.38400000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.19},"max":{"1min":0.51000000000000001,"5min":0.54300000000000004,"15min":14.968999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.376,"15min":0.41299999999999998},"min":{"1min":0.24099999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.52100000000000002,"5min":0.59199999999999997,"15min":14.746},"last":0.35499999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.37,"15min":0.52200000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.21099999999999999},"max":{"1min":0.53300000000000003,"5min":0.54100000000000004,"15min":50.677999999999997},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.45800000000000002,"15min":0.60099999999999998},"min":{"1min":0.34599999999999997,"5min":0.28599999999999998,"15min":0.26000000000000001},"max":{"1min":0.57599999999999996,"5min":0.66200000000000003,"15min":50.615000000000002},"last":0.48299999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.37,"15min":0.45900000000000002},"min":{"1min":0.25600000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.48999999999999999,"5min":0.61299999999999999,"15min":17.440999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.44700000000000001,"15min":0.59899999999999998},"min":{"1min":0.32400000000000001,"5min":0.29099999999999998,"15min":0.23499999999999999},"max":{"1min":0.65500000000000003,"5min":0.88900000000000001,"15min":50.424999999999997},"last":0.45800000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59499999999999997,"15min":0.64400000000000002},"min":{"1min":0.41099999999999998,"5min":0.34200000000000003,"15min":0.29699999999999999},"max":{"1min":0.76800000000000002,"5min":0.84799999999999998,"15min":14.802},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60399999999999998,"15min":0.67500000000000004},"min":{"1min":0.42199999999999999,"5min":0.39400000000000002,"15min":0.36499999999999999},"max":{"1min":0.746,"5min":0.88700000000000001,"15min":14.621},"last":0.52700000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60399999999999998,"15min":0.63200000000000001},"min":{"1min":0.48399999999999999,"5min":0.40100000000000002,"15min":0.35199999999999998},"max":{"1min":0.75700000000000001,"5min":0.84399999999999997,"15min":14.792999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60899999999999999,"15min":0.76000000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.371},"max":{"1min":0.72499999999999998,"5min":0.92600000000000005,"15min":50.537999999999997},"last":0.46899999999999997}]},{"osd":6,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63700000000000001,"15min":0.68700000000000006},"min":{"1min":0.44700000000000001,"5min":0.40699999999999997,"15min":0.39100000000000001},"max":{"1min":0.78700000000000003,"5min":1.216,"15min":23.388000000000002},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64300000000000002,"15min":0.69999999999999996},"min":{"1min":0.50600000000000001,"5min":0.40400000000000003,"15min":0.39100000000000001},"max":{"1min":0.75,"5min":0.86899999999999999,"15min":14.856},"last":0.56699999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.627,"15min":0.67900000000000005},"min":{"1min":0.45600000000000002,"5min":0.40999999999999998,"15min":0.377},"max":{"1min":0.80900000000000005,"5min":0.95899999999999996,"15min":14.693},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63500000000000001,"15min":0.67400000000000004},"min":{"1min":0.45800000000000002,"5min":0.44600000000000001,"15min":0.434},"max":{"1min":0.79600000000000004,"5min":0.95499999999999996,"15min":10.349},"last":0.55600000000000005}]}]},{"osd":3,"up_from":13,"seq":55834575023,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573380,"kb_used_data":276092,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93798460,"statfs":{"total":96636764160,"available":96049623040,"internally_reserved":0,"allocated":282718208,"data_stored":344742979,"data_compressed":32796809,"data_compressed_allocated":37396480,"data_compressed_original":108593152,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.34399999999999997,"15min":0.35999999999999999},"min":{"1min":0.24299999999999999,"5min":0.216,"15min":0.186},"max":{"1min":0.5,"5min":0.58699999999999997,"15min":7.0999999999999996},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.35499999999999998,"15min":0.41699999999999998},"min":{"1min":0.27200000000000002,"5min":0.221,"15min":0.218},"max":{"1min":0.622,"5min":0.622,"15min":14.257},"last":0.38700000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.34999999999999998,"15min":0.35399999999999998},"min":{"1min":0.23100000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":3.1760000000000002},"last":0.375},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.40899999999999997,"15min":0.39600000000000002},"min":{"1min":0.29999999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.61499999999999999,"5min":0.68200000000000005,"15min":5.2599999999999998},"last":0.434}]},{"osd":2,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.39800000000000002,"15min":0.45900000000000002},"min":{"1min":0.316,"5min":0.25700000000000001,"15min":0.23499999999999999},"max":{"1min":0.57599999999999996,"5min":0.747,"15min":13.138},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.42699999999999999,"15min":0.44500000000000001},"min":{"1min":0.32800000000000001,"5min":0.28299999999999997,"15min":0.27200000000000002},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":7.181},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.627,"15min":0.68100000000000005},"min":{"1min":0.46800000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":13.068},"last":0.747},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.65600000000000003,"15min":0.67500000000000004},"min":{"1min":0.57399999999999995,"5min":0.434,"15min":0.39800000000000002},"max":{"1min":0.878,"5min":0.89100000000000001,"15min":6.4299999999999997},"last":0.81499999999999995}]},{"osd":5,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.47799999999999998,"5min":0.35999999999999999,"15min":0.32700000000000001},"max":{"1min":0.89000000000000001,"5min":0.95599999999999996,"15min":3.105},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.59799999999999998,"15min":0.60799999999999998},"min":{"1min":0.52000000000000002,"5min":0.372,"15min":0.29299999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":3.5369999999999999},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.63100000000000001,"15min":0.68799999999999994},"min":{"1min":0.54400000000000004,"5min":0.375,"15min":0.375},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":12.974},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63400000000000001,"15min":0.64100000000000001},"min":{"1min":0.56299999999999994,"5min":0.378,"15min":0.35599999999999998},"max":{"1min":0.93700000000000006,"5min":0.94499999999999995,"15min":3.0379999999999998},"last":0.68899999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:10:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.61299999999999999,"15min":0.63400000000000001},"min":{"1min":0.52900000000000003,"5min":0.41199999999999998,"15min":0.40100000000000002},"max":{"1min":0.93400000000000005,"5min":0.97199999999999998,"15min":6.2949999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.63400000000000001,"15min":0.67200000000000004},"min":{"1min":0.496,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90400000000000003,"5min":0.95399999999999996,"15min":13.032},"last":0.73099999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575024,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":583384,"kb_used_data":282000,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":93788456,"statfs":{"total":96636764160,"available":96039378944,"internally_reserved":0,"allocated":288768000,"data_stored":379097385,"data_compressed":33259490,"data_compressed_allocated":38625280,"data_compressed_original":137244672,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52900000000000003,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68100000000000005,"5min":0.747,"15min":0.79600000000000004},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.56699999999999995,"15min":0.54600000000000004},"min":{"1min":0.47699999999999998,"5min":0.35099999999999998,"15min":0.26600000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.80000000000000004},"last":0.66900000000000004}]},{"osd":1,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.41199999999999998,"5min":0.34899999999999998,"15min":0.26200000000000001},"max":{"1min":0.73899999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.501,"15min":0.51100000000000001},"min":{"1min":0.35199999999999998,"5min":0.30399999999999999,"15min":0.27400000000000002},"max":{"1min":0.70399999999999996,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.55500000000000005}]},{"osd":2,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.40400000000000003,"5min":0.35899999999999999,"15min":0.312},"max":{"1min":0.69099999999999995,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.57599999999999996},"min":{"1min":0.441,"5min":0.36299999999999999,"15min":0.311},"max":{"1min":0.76700000000000002,"5min":1.413,"15min":1.413},"last":0.627}]},{"osd":3,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.59099999999999997,"15min":0.55100000000000005},"min":{"1min":0.38400000000000001,"5min":0.32100000000000001,"15min":0.314},"max":{"1min":0.65800000000000003,"5min":6.7000000000000002,"15min":6.7000000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52900000000000003,"15min":0.54500000000000004},"min":{"1min":0.36899999999999999,"5min":0.36399999999999999,"15min":0.32000000000000001},"max":{"1min":0.72299999999999998,"5min":1.208,"15min":1.208},"last":0.5}]},{"osd":5,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.44600000000000001,"15min":0.47699999999999998},"min":{"1min":0.28599999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.59299999999999997,"5min":0.77100000000000002,"15min":0.81899999999999995},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52200000000000002,"15min":0.53000000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.29499999999999998},"max":{"1min":0.66800000000000004,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.57399999999999995}]},{"osd":6,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.48499999999999999,"15min":0.51300000000000001},"min":{"1min":0.315,"5min":0.30099999999999999,"15min":0.28100000000000003},"max":{"1min":0.60499999999999998,"5min":1.236,"15min":1.236},"last":0.433},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.42899999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.69199999999999995,"5min":1.381,"15min":1.381},"last":0.60499999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:10:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53200000000000003,"15min":0.54600000000000004},"min":{"1min":0.38600000000000001,"5min":0.32800000000000001,"15min":0.30599999999999999},"max":{"1min":0.66700000000000004,"5min":0.71799999999999997,"15min":0.83499999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.45900000000000002,"5min":0.39900000000000002,"15min":0.32600000000000001},"max":{"1min":0.73599999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.64800000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575023,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486268,"kb_used_data":188980,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93885572,"statfs":{"total":96636764160,"available":96138825728,"internally_reserved":0,"allocated":193515520,"data_stored":246084607,"data_compressed":22464539,"data_compressed_allocated":25784320,"data_compressed_original":83968000,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.50700000000000001,"15min":0.55100000000000005},"min":{"1min":0.34399999999999997,"5min":0.29999999999999999,"15min":0.23300000000000001},"max":{"1min":0.60199999999999998,"5min":1.0189999999999999,"15min":11.167},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.53800000000000003,"15min":0.57699999999999996},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.221},"max":{"1min":0.71899999999999997,"5min":1.0449999999999999,"15min":9.8659999999999997},"last":0.70899999999999996}]},{"osd":1,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.64100000000000001,"5min":1.0780000000000001,"15min":1.1120000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55100000000000005,"15min":0.54000000000000004},"min":{"1min":0.35599999999999998,"5min":0.29999999999999999,"15min":0.28799999999999998},"max":{"1min":0.73999999999999999,"5min":1.028,"15min":2.524},"last":0.65500000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.52700000000000002,"15min":0.54000000000000004},"min":{"1min":0.39400000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.68999999999999995,"5min":0.98499999999999999,"15min":1.0389999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70399999999999996,"5min":0.999,"15min":1.1439999999999999},"last":0.625}]},{"osd":3,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.77300000000000002,"5min":0.97299999999999998,"15min":2.9359999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60999999999999999,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.377,"15min":0.28999999999999998},"max":{"1min":0.77500000000000002,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.68500000000000005}]},{"osd":4,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.45900000000000002,"15min":0.47999999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.26000000000000001},"max":{"1min":0.71099999999999997,"5min":0.73499999999999999,"15min":1.1619999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57699999999999996,"15min":0.54200000000000004},"min":{"1min":0.33700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.75600000000000001,"5min":0.78900000000000003,"15min":1.1950000000000001},"last":0.55900000000000005}]},{"osd":6,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46899999999999997,"15min":0.503},"min":{"1min":0.35699999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65900000000000003,"5min":0.78300000000000003,"15min":0.94099999999999995},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56499999999999995,"15min":0.58099999999999996},"min":{"1min":0.41899999999999998,"5min":0.38400000000000001,"15min":0.32900000000000001},"max":{"1min":0.72599999999999998,"5min":0.84999999999999998,"15min":0.97899999999999998},"last":0.59499999999999997}]},{"osd":7,"last update":"Fri Oct 6 04:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.53700000000000003},"min":{"1min":0.38700000000000001,"5min":0.33500000000000002,"15min":0.28699999999999998},"max":{"1min":0.81999999999999995,"5min":0.84899999999999998,"15min":1.1830000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.438,"5min":0.375,"15min":0.311},"max":{"1min":0.79900000000000004,"5min":0.81299999999999994,"15min":1.29},"last":0.58199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13369344,"data_stored":49512965,"data_compressed":6345911,"data_compressed_allocated":8089600,"data_compressed_original":44290048,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8622080,"data_stored":36922745,"data_compressed":4857186,"data_compressed_allocated":6213632,"data_compressed_original":34525184,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3923968,"data_stored":16797884,"data_compressed":2299517,"data_compressed_allocated":2953216,"data_compressed_original":15835136,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12926976,"data_stored":47356979,"data_compressed":5951348,"data_compressed_allocated":7700480,"data_compressed_original":42160128,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":21970944,"data_stored":88207992,"data_compressed":12354769,"data_compressed_allocated":15306752,"data_compressed_original":81604608,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":11075584,"data_stored":46041285,"data_compressed":6106433,"data_compressed_allocated":7659520,"data_compressed_original":42663936,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1855488,"data_stored":8388720,"data_compressed":1228027,"data_compressed_allocated":1515520,"data_compressed_original":8056832,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":10952704,"data_stored":42658484,"data_compressed":6046013,"data_compressed_allocated":7446528,"data_compressed_original":39194624,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":181956608,"data_stored":203156358,"data_compressed":20456538,"data_compressed_allocated":22609920,"data_compressed_original":49975296,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":274661376,"data_stored":300540492,"data_compressed":28008712,"data_compressed_allocated":30793728,"data_compressed_original":65601536,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139608064,"data_stored":158511793,"data_compressed":16956544,"data_compressed_allocated":18817024,"data_compressed_original":43261952,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":269180928,"data_stored":297077873,"data_compressed":26845461,"data_compressed_allocated":29696000,"data_compressed_original":66433024,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":266186752,"data_stored":290581266,"data_compressed":20904721,"data_compressed_allocated":23318528,"data_compressed_original":55640064,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":181829632,"data_stored":199735195,"data_compressed":16358106,"data_compressed_allocated":18124800,"data_compressed_original":41304064,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":166477824,"data_stored":179521845,"data_compressed":14708055,"data_compressed_allocated":16289792,"data_compressed_original":34680832,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":203251712,"data_stored":216547248,"data_compressed":16284911,"data_compressed_allocated":17920000,"data_compressed_original":36777984,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:11:23.749 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.750 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.750 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.750 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.750 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.750 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.751 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.751 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.751 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.751 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.751 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.752 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.752 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.752 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.752 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.752 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.752 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.753 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.753 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.753 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.754 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.754 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.754 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.754 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.754 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.755 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.755 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.755 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.755 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.755 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.756 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.756 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.756 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.756 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.757 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.757 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.757 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.757 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.758 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.758 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.758 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.758 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.758 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.759 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:23.759 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T04:11:43.760 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:11:43.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.886+0000 7eff84d63700 1 Processor -- start 2023-10-06T04:11:43.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.886+0000 7eff84d63700 1 -- start start 2023-10-06T04:11:43.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.887+0000 7eff84d63700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7eff8014dcd0 0x7eff80153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:43.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.887+0000 7eff84d63700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7eff8014e0a0 0x7eff800c18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:43.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.887+0000 7eff84d63700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff800c5770 con 0x7eff800c5c00 2023-10-06T04:11:43.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.887+0000 7eff84d63700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff80154c80 con 0x7eff8014e0a0 2023-10-06T04:11:43.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.887+0000 7eff84d63700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff80154200 con 0x7eff8014dcd0 2023-10-06T04:11:43.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.887+0000 7eff7dd9b700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7eff8014e0a0 0x7eff800c18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:43.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.887+0000 7eff7dd9b700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7eff8014e0a0 0x7eff800c18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49392/0 (socket says 172.21.15.157:49392) 2023-10-06T04:11:43.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.888+0000 7eff7dd9b700 1 -- 172.21.15.157:0/3970017485 learned_addr learned my addr 172.21.15.157:0/3970017485 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:11:43.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.888+0000 7eff7e59c700 1 --2- 172.21.15.157:0/3970017485 >> v2:172.21.15.175:3300/0 conn(0x7eff8014dcd0 0x7eff80153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:43.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.888+0000 7eff7dd9b700 1 -- 172.21.15.157:0/3970017485 >> v2:172.21.15.175:3300/0 conn(0x7eff8014dcd0 msgr2=0x7eff80153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:11:43.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.888+0000 7eff7dd9b700 1 --2- 172.21.15.157:0/3970017485 >> v2:172.21.15.175:3300/0 conn(0x7eff8014dcd0 0x7eff80153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:43.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.888+0000 7eff7dd9b700 1 -- 172.21.15.157:0/3970017485 >> v1:172.21.15.175:6789/0 conn(0x7eff800c5c00 legacy=0x7eff800ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:43.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.888+0000 7eff7dd9b700 1 -- 172.21.15.157:0/3970017485 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff801543e0 con 0x7eff8014e0a0 2023-10-06T04:11:43.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.888+0000 7eff7dd9b700 1 --2- 172.21.15.157:0/3970017485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff8014e0a0 0x7eff800c18c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7eff8006cd20 tx=0x7eff68009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14ca256b73c97ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:43.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.888+0000 7eff7d59a700 1 -- 172.21.15.157:0/3970017485 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff68009e70 con 0x7eff8014e0a0 2023-10-06T04:11:43.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.888+0000 7eff7d59a700 1 -- 172.21.15.157:0/3970017485 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff6800b8f0 con 0x7eff8014e0a0 2023-10-06T04:11:43.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.888+0000 7eff7d59a700 1 -- 172.21.15.157:0/3970017485 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6800bc90 con 0x7eff8014e0a0 2023-10-06T04:11:43.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.889+0000 7eff84d63700 1 -- 172.21.15.157:0/3970017485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff8014e0a0 msgr2=0x7eff800c18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:43.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.889+0000 7eff84d63700 1 --2- 172.21.15.157:0/3970017485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff8014e0a0 0x7eff800c18c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7eff8006cd20 tx=0x7eff68009250 comp rx=0 tx=0).stop 2023-10-06T04:11:43.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.889+0000 7eff84d63700 1 -- 172.21.15.157:0/3970017485 shutdown_connections 2023-10-06T04:11:43.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.889+0000 7eff84d63700 1 --2- 172.21.15.157:0/3970017485 >> v2:172.21.15.175:3300/0 conn(0x7eff8014dcd0 0x7eff80153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:43.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.889+0000 7eff84d63700 1 --2- 172.21.15.157:0/3970017485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff8014e0a0 0x7eff800c18c0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:43.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.889+0000 7eff84d63700 1 -- 172.21.15.157:0/3970017485 >> 172.21.15.157:0/3970017485 conn(0x7eff800692b0 msgr2=0x7eff800696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:43.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.889+0000 7eff84d63700 1 -- 172.21.15.157:0/3970017485 shutdown_connections 2023-10-06T04:11:43.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.889+0000 7eff84d63700 1 -- 172.21.15.157:0/3970017485 wait complete. 2023-10-06T04:11:43.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.890+0000 7eff84d63700 1 Processor -- start 2023-10-06T04:11:43.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.890+0000 7eff84d63700 1 -- start start 2023-10-06T04:11:43.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.890+0000 7eff84d63700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff800c5c00 0x7eff800c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:43.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.890+0000 7eff84d63700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff8014dcd0 0x7eff800ca390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:43.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.890+0000 7eff84d63700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff8014e0a0 0x7eff80167420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:43.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.890+0000 7eff84d63700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff80155510 con 0x7eff8014dcd0 2023-10-06T04:11:43.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.890+0000 7eff84d63700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8014e860 con 0x7eff8014e0a0 2023-10-06T04:11:43.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.890+0000 7eff84d63700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff800c11e0 con 0x7eff800c5c00 2023-10-06T04:11:43.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.891+0000 7eff7dd9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff8014dcd0 0x7eff800ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:43.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.891+0000 7eff7dd9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff8014dcd0 0x7eff800ca390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49398/0 (socket says 172.21.15.157:49398) 2023-10-06T04:11:43.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.891+0000 7eff7dd9b700 1 -- 172.21.15.157:0/3878520030 learned_addr learned my addr 172.21.15.157:0/3878520030 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:11:43.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.891+0000 7eff7ed9d700 1 --2- 172.21.15.157:0/3878520030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff8014e0a0 0x7eff80167420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:43.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.891+0000 7eff7e59c700 1 --2- 172.21.15.157:0/3878520030 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff800c5c00 0x7eff800c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:43.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.891+0000 7eff7dd9b700 1 -- 172.21.15.157:0/3878520030 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff800c5c00 msgr2=0x7eff800c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:43.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.891+0000 7eff7dd9b700 1 --2- 172.21.15.157:0/3878520030 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff800c5c00 0x7eff800c9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:43.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.891+0000 7eff7dd9b700 1 -- 172.21.15.157:0/3878520030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff8014e0a0 msgr2=0x7eff80167420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:43.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.891+0000 7eff7dd9b700 1 --2- 172.21.15.157:0/3878520030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff8014e0a0 0x7eff80167420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:43.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.891+0000 7eff7dd9b700 1 -- 172.21.15.157:0/3878520030 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff680094f0 con 0x7eff8014dcd0 2023-10-06T04:11:43.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.891+0000 7eff7ed9d700 1 --2- 172.21.15.157:0/3878520030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff8014e0a0 0x7eff80167420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:11:43.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.891+0000 7eff7dd9b700 1 --2- 172.21.15.157:0/3878520030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff8014dcd0 0x7eff800ca390 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7eff8006ce20 tx=0x7eff68017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:43.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.891+0000 7eff767fc700 1 -- 172.21.15.157:0/3878520030 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff68024440 con 0x7eff8014dcd0 2023-10-06T04:11:43.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.891+0000 7eff767fc700 1 -- 172.21.15.157:0/3878520030 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff680245a0 con 0x7eff8014dcd0 2023-10-06T04:11:43.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.891+0000 7eff84d63700 1 -- 172.21.15.157:0/3878520030 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff800c0f00 con 0x7eff8014dcd0 2023-10-06T04:11:43.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.891+0000 7eff767fc700 1 -- 172.21.15.157:0/3878520030 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff68024960 con 0x7eff8014dcd0 2023-10-06T04:11:43.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.891+0000 7eff84d63700 1 -- 172.21.15.157:0/3878520030 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff8014ea70 con 0x7eff8014dcd0 2023-10-06T04:11:43.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.892+0000 7eff84d63700 1 -- 172.21.15.157:0/3878520030 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff800c9ca0 con 0x7eff8014dcd0 2023-10-06T04:11:43.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.893+0000 7eff767fc700 1 -- 172.21.15.157:0/3878520030 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7eff68024d50 con 0x7eff8014dcd0 2023-10-06T04:11:43.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.893+0000 7eff767fc700 1 --2- 172.21.15.157:0/3878520030 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7eff60066f30 0x7eff600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:11:43.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.893+0000 7eff767fc700 1 -- 172.21.15.157:0/3878520030 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7eff68086430 con 0x7eff8014dcd0 2023-10-06T04:11:43.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.893+0000 7eff7e59c700 1 --2- 172.21.15.157:0/3878520030 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7eff60066f30 0x7eff600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:11:43.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.893+0000 7eff7e59c700 1 --2- 172.21.15.157:0/3878520030 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7eff60066f30 0x7eff600693e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7eff700075a0 tx=0x7eff7000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:11:43.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:43.897+0000 7eff767fc700 1 -- 172.21.15.157:0/3878520030 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff68053330 con 0x7eff8014dcd0 2023-10-06T04:11:44.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:44.077+0000 7eff84d63700 1 -- 172.21.15.157:0/3878520030 --> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7eff800c1320 con 0x7eff60066f30 2023-10-06T04:11:44.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:44.086+0000 7eff767fc700 1 -- 172.21.15.157:0/3878520030 <== mgr.4107 v2:172.21.15.175:6832/90661 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+317445 (secure 0 0 0) 0x7eff70007e70 con 0x7eff60066f30 2023-10-06T04:11:44.092 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:11:44.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:44.089+0000 7eff84d63700 1 -- 172.21.15.157:0/3878520030 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7eff60066f30 msgr2=0x7eff600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:44.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:44.089+0000 7eff84d63700 1 --2- 172.21.15.157:0/3878520030 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7eff60066f30 0x7eff600693e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7eff700075a0 tx=0x7eff7000b040 comp rx=0 tx=0).stop 2023-10-06T04:11:44.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:44.090+0000 7eff84d63700 1 -- 172.21.15.157:0/3878520030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff8014dcd0 msgr2=0x7eff800ca390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:11:44.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:44.090+0000 7eff84d63700 1 --2- 172.21.15.157:0/3878520030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff8014dcd0 0x7eff800ca390 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7eff8006ce20 tx=0x7eff68017000 comp rx=0 tx=0).stop 2023-10-06T04:11:44.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:44.090+0000 7eff84d63700 1 -- 172.21.15.157:0/3878520030 shutdown_connections 2023-10-06T04:11:44.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:44.090+0000 7eff84d63700 1 --2- 172.21.15.157:0/3878520030 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7eff60066f30 0x7eff600693e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:44.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:44.090+0000 7eff84d63700 1 --2- 172.21.15.157:0/3878520030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff8014e0a0 0x7eff80167420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:44.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:44.090+0000 7eff84d63700 1 --2- 172.21.15.157:0/3878520030 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff800c5c00 0x7eff800c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:44.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:44.090+0000 7eff84d63700 1 --2- 172.21.15.157:0/3878520030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff8014dcd0 0x7eff800ca390 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:11:44.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:44.090+0000 7eff84d63700 1 -- 172.21.15.157:0/3878520030 >> 172.21.15.157:0/3878520030 conn(0x7eff800692b0 msgr2=0x7eff800c17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:11:44.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:44.090+0000 7eff84d63700 1 -- 172.21.15.157:0/3878520030 shutdown_connections 2023-10-06T04:11:44.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:11:44.090+0000 7eff84d63700 1 -- 172.21.15.157:0/3878520030 wait complete. 2023-10-06T04:11:44.097 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-10-06T04:11:44.104 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":487,"stamp":"2023-10-06T04:11:43.657468+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1092256967,"num_objects":10491,"num_object_clones":0,"num_object_copies":20982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10491,"num_whiteouts":0,"num_read":2914,"num_read_kb":634649,"num_write":27991,"num_write_kb":1247287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":122,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":566645,"num_omap_keys":999,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37295,"ondisk_log_size":37295,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":4171100,"kb_used_data":1731356,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":750803620,"statfs":{"total":773094113280,"available":768822906880,"internally_reserved":0,"allocated":1772908544,"data_stored":2185437901,"data_compressed":205801822,"data_compressed_allocated":234577920,"data_compressed_original":703365120,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":574,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002317"},"pg_stats":[{"pgid":"4.3f","version":"53'449","reported_seq":2562,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:09.544941+0000","last_change":"2023-10-06T04:11:09.544918+0000","last_active":"2023-10-06T04:11:09.544941+0000","last_peered":"2023-10-06T04:11:09.544941+0000","last_clean":"2023-10-06T04:11:09.544941+0000","last_became_active":"2023-10-06T03:57:11.371756+0000","last_became_peered":"2023-10-06T03:57:11.371756+0000","last_unstale":"2023-10-06T04:11:09.544941+0000","last_undegraded":"2023-10-06T04:11:09.544941+0000","last_fullsized":"2023-10-06T04:11:09.544941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'449","last_scrub_stamp":"2023-10-06T04:11:09.544846+0000","last_deep_scrub":"53'449","last_deep_scrub_stamp":"2023-10-06T04:11:09.544846+0000","last_clean_scrub_stamp":"2023-10-06T04:11:09.544846+0000","objects_scrubbed":169,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:19:26.736027+0000","scrub_duration":0.096486541999999995,"objects_trimmed":0,"snaptrim_duration":0.001094783,"stat_sum":{"num_bytes":10615219,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":29,"num_read_kb":15,"num_write":308,"num_write_kb":10492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"53'532","reported_seq":2650,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:41.704323+0000","last_change":"2023-10-06T04:11:41.704302+0000","last_active":"2023-10-06T04:11:41.704323+0000","last_peered":"2023-10-06T04:11:41.704323+0000","last_clean":"2023-10-06T04:11:41.704323+0000","last_became_active":"2023-10-06T03:57:11.370070+0000","last_became_peered":"2023-10-06T03:57:11.370070+0000","last_unstale":"2023-10-06T04:11:41.704323+0000","last_undegraded":"2023-10-06T04:11:41.704323+0000","last_fullsized":"2023-10-06T04:11:41.704323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'532","last_scrub_stamp":"2023-10-06T04:11:41.704234+0000","last_deep_scrub":"53'532","last_deep_scrub_stamp":"2023-10-06T04:11:41.704234+0000","last_clean_scrub_stamp":"2023-10-06T04:11:41.704234+0000","objects_scrubbed":178,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:21.310968+0000","scrub_duration":0.11469288599999999,"objects_trimmed":0,"snaptrim_duration":0.0021773819999999998,"stat_sum":{"num_bytes":6466808,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":33,"num_read_kb":16,"num_write":334,"num_write_kb":6491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"53'356","reported_seq":2461,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.122379+0000","last_change":"2023-10-06T04:05:02.723796+0000","last_active":"2023-10-06T04:10:55.122379+0000","last_peered":"2023-10-06T04:10:55.122379+0000","last_clean":"2023-10-06T04:10:55.122379+0000","last_became_active":"2023-10-06T03:57:11.366012+0000","last_became_peered":"2023-10-06T03:57:11.366012+0000","last_unstale":"2023-10-06T04:10:55.122379+0000","last_undegraded":"2023-10-06T04:10:55.122379+0000","last_fullsized":"2023-10-06T04:10:55.122379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002722286,"stat_sum":{"num_bytes":15834623,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":40,"num_read_kb":20499,"num_write":321,"num_write_kb":23772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"53'431","reported_seq":2547,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:10.152318+0000","last_change":"2023-10-06T04:11:10.152270+0000","last_active":"2023-10-06T04:11:10.152318+0000","last_peered":"2023-10-06T04:11:10.152318+0000","last_clean":"2023-10-06T04:11:10.152318+0000","last_became_active":"2023-10-06T03:57:11.365673+0000","last_became_peered":"2023-10-06T03:57:11.365673+0000","last_unstale":"2023-10-06T04:11:10.152318+0000","last_undegraded":"2023-10-06T04:11:10.152318+0000","last_fullsized":"2023-10-06T04:11:10.152318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'431","last_scrub_stamp":"2023-10-06T04:11:10.152150+0000","last_deep_scrub":"53'431","last_deep_scrub_stamp":"2023-10-06T04:11:10.152150+0000","last_clean_scrub_stamp":"2023-10-06T04:11:10.152150+0000","objects_scrubbed":157,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:40.609815+0000","scrub_duration":0.112734037,"objects_trimmed":0,"snaptrim_duration":0.001947182,"stat_sum":{"num_bytes":14764610,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":32,"num_read_kb":7963,"num_write":291,"num_write_kb":14577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"53'495","reported_seq":2610,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:37.529840+0000","last_change":"2023-10-06T04:11:37.529810+0000","last_active":"2023-10-06T04:11:37.529840+0000","last_peered":"2023-10-06T04:11:37.529840+0000","last_clean":"2023-10-06T04:11:37.529840+0000","last_became_active":"2023-10-06T03:57:11.361302+0000","last_became_peered":"2023-10-06T03:57:11.361302+0000","last_unstale":"2023-10-06T04:11:37.529840+0000","last_undegraded":"2023-10-06T04:11:37.529840+0000","last_fullsized":"2023-10-06T04:11:37.529840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'495","last_scrub_stamp":"2023-10-06T04:11:37.529742+0000","last_deep_scrub":"53'495","last_deep_scrub_stamp":"2023-10-06T04:11:37.529742+0000","last_clean_scrub_stamp":"2023-10-06T04:11:37.529742+0000","objects_scrubbed":153,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:25.180275+0000","scrub_duration":0.095122445999999999,"objects_trimmed":0,"snaptrim_duration":0.001966644,"stat_sum":{"num_bytes":2193993,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":31,"num_read_kb":18,"num_write":324,"num_write_kb":2284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"53'328","reported_seq":2430,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.135779+0000","last_change":"2023-10-06T04:05:02.723270+0000","last_active":"2023-10-06T04:10:55.135779+0000","last_peered":"2023-10-06T04:10:55.135779+0000","last_clean":"2023-10-06T04:10:55.135779+0000","last_became_active":"2023-10-06T03:57:11.368775+0000","last_became_peered":"2023-10-06T03:57:11.368775+0000","last_unstale":"2023-10-06T04:10:55.135779+0000","last_undegraded":"2023-10-06T04:10:55.135779+0000","last_fullsized":"2023-10-06T04:10:55.135779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002640482,"stat_sum":{"num_bytes":22431099,"num_objects":144,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":0,"num_read":34,"num_read_kb":13627,"num_write":304,"num_write_kb":22068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"53'559","reported_seq":2675,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:41.143847+0000","last_change":"2023-10-06T04:11:41.143824+0000","last_active":"2023-10-06T04:11:41.143847+0000","last_peered":"2023-10-06T04:11:41.143847+0000","last_clean":"2023-10-06T04:11:41.143847+0000","last_became_active":"2023-10-06T03:57:11.377786+0000","last_became_peered":"2023-10-06T03:57:11.377786+0000","last_unstale":"2023-10-06T04:11:41.143847+0000","last_undegraded":"2023-10-06T04:11:41.143847+0000","last_fullsized":"2023-10-06T04:11:41.143847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'559","last_scrub_stamp":"2023-10-06T04:11:41.143745+0000","last_deep_scrub":"53'559","last_deep_scrub_stamp":"2023-10-06T04:11:41.143745+0000","last_clean_scrub_stamp":"2023-10-06T04:11:41.143745+0000","objects_scrubbed":173,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:15:18.950590+0000","scrub_duration":0.15119919300000001,"objects_trimmed":0,"snaptrim_duration":0.00093960500000000004,"stat_sum":{"num_bytes":18398941,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":32,"num_read_kb":8208,"num_write":357,"num_write_kb":18159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"53'479","reported_seq":2593,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:35.684950+0000","last_change":"2023-10-06T04:11:35.684928+0000","last_active":"2023-10-06T04:11:35.684950+0000","last_peered":"2023-10-06T04:11:35.684950+0000","last_clean":"2023-10-06T04:11:35.684950+0000","last_became_active":"2023-10-06T03:57:11.373765+0000","last_became_peered":"2023-10-06T03:57:11.373765+0000","last_unstale":"2023-10-06T04:11:35.684950+0000","last_undegraded":"2023-10-06T04:11:35.684950+0000","last_fullsized":"2023-10-06T04:11:35.684950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'479","last_scrub_stamp":"2023-10-06T04:11:35.684846+0000","last_deep_scrub":"53'479","last_deep_scrub_stamp":"2023-10-06T04:11:35.684846+0000","last_clean_scrub_stamp":"2023-10-06T04:11:35.684846+0000","objects_scrubbed":147,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:13.767095+0000","scrub_duration":0.117065961,"objects_trimmed":0,"snaptrim_duration":0.001873881,"stat_sum":{"num_bytes":10960737,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":31,"num_read_kb":4107,"num_write":305,"num_write_kb":10786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"53'484","reported_seq":2608,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:25.493681+0000","last_change":"2023-10-06T04:11:25.493658+0000","last_active":"2023-10-06T04:11:25.493681+0000","last_peered":"2023-10-06T04:11:25.493681+0000","last_clean":"2023-10-06T04:11:25.493681+0000","last_became_active":"2023-10-06T03:57:11.368004+0000","last_became_peered":"2023-10-06T03:57:11.368004+0000","last_unstale":"2023-10-06T04:11:25.493681+0000","last_undegraded":"2023-10-06T04:11:25.493681+0000","last_fullsized":"2023-10-06T04:11:25.493681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'484","last_scrub_stamp":"2023-10-06T04:11:25.493587+0000","last_deep_scrub":"53'484","last_deep_scrub_stamp":"2023-10-06T04:11:25.493587+0000","last_clean_scrub_stamp":"2023-10-06T04:11:25.493587+0000","objects_scrubbed":154,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:04.307189+0000","scrub_duration":0.10856695700000001,"objects_trimmed":0,"snaptrim_duration":0.0024398710000000001,"stat_sum":{"num_bytes":13584161,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":41,"num_read_kb":4114,"num_write":308,"num_write_kb":17522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"53'502","reported_seq":2616,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:39.671416+0000","last_change":"2023-10-06T04:11:39.671389+0000","last_active":"2023-10-06T04:11:39.671416+0000","last_peered":"2023-10-06T04:11:39.671416+0000","last_clean":"2023-10-06T04:11:39.671416+0000","last_became_active":"2023-10-06T03:57:11.379107+0000","last_became_peered":"2023-10-06T03:57:11.379107+0000","last_unstale":"2023-10-06T04:11:39.671416+0000","last_undegraded":"2023-10-06T04:11:39.671416+0000","last_fullsized":"2023-10-06T04:11:39.671416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'502","last_scrub_stamp":"2023-10-06T04:11:39.671319+0000","last_deep_scrub":"53'502","last_deep_scrub_stamp":"2023-10-06T04:11:39.671319+0000","last_clean_scrub_stamp":"2023-10-06T04:11:39.671319+0000","objects_scrubbed":150,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:09:58.519738+0000","scrub_duration":0.10696708100000001,"objects_trimmed":0,"snaptrim_duration":0.002039126,"stat_sum":{"num_bytes":9784235,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":30,"num_read_kb":4112,"num_write":328,"num_write_kb":9733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"53'376","reported_seq":2483,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.123250+0000","last_change":"2023-10-06T04:05:02.724475+0000","last_active":"2023-10-06T04:10:55.123250+0000","last_peered":"2023-10-06T04:10:55.123250+0000","last_clean":"2023-10-06T04:10:55.123250+0000","last_became_active":"2023-10-06T03:57:11.368256+0000","last_became_peered":"2023-10-06T03:57:11.368256+0000","last_unstale":"2023-10-06T04:10:55.123250+0000","last_undegraded":"2023-10-06T04:10:55.123250+0000","last_fullsized":"2023-10-06T04:10:55.123250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029138929999999999,"stat_sum":{"num_bytes":31134351,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":39,"num_read_kb":4114,"num_write":352,"num_write_kb":30587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"53'483","reported_seq":2597,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:21.563546+0000","last_change":"2023-10-06T04:11:21.563509+0000","last_active":"2023-10-06T04:11:21.563546+0000","last_peered":"2023-10-06T04:11:21.563546+0000","last_clean":"2023-10-06T04:11:21.563546+0000","last_became_active":"2023-10-06T03:57:11.368425+0000","last_became_peered":"2023-10-06T03:57:11.368425+0000","last_unstale":"2023-10-06T04:11:21.563546+0000","last_undegraded":"2023-10-06T04:11:21.563546+0000","last_fullsized":"2023-10-06T04:11:21.563546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'483","last_scrub_stamp":"2023-10-06T04:11:21.563396+0000","last_deep_scrub":"53'483","last_deep_scrub_stamp":"2023-10-06T04:11:21.563396+0000","last_clean_scrub_stamp":"2023-10-06T04:11:21.563396+0000","objects_scrubbed":161,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:51.078052+0000","scrub_duration":0.131329159,"objects_trimmed":0,"snaptrim_duration":0.0014231059999999999,"stat_sum":{"num_bytes":10422851,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":30,"num_read_kb":4105,"num_write":305,"num_write_kb":14436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"53'513","reported_seq":2633,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:19.515410+0000","last_change":"2023-10-06T04:11:19.515377+0000","last_active":"2023-10-06T04:11:19.515410+0000","last_peered":"2023-10-06T04:11:19.515410+0000","last_clean":"2023-10-06T04:11:19.515410+0000","last_became_active":"2023-10-06T03:57:11.367079+0000","last_became_peered":"2023-10-06T03:57:11.367079+0000","last_unstale":"2023-10-06T04:11:19.515410+0000","last_undegraded":"2023-10-06T04:11:19.515410+0000","last_fullsized":"2023-10-06T04:11:19.515410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'513","last_scrub_stamp":"2023-10-06T04:11:19.515270+0000","last_deep_scrub":"53'513","last_deep_scrub_stamp":"2023-10-06T04:11:19.515270+0000","last_clean_scrub_stamp":"2023-10-06T04:11:19.515270+0000","objects_scrubbed":173,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:26.862858+0000","scrub_duration":0.13944414499999999,"objects_trimmed":0,"snaptrim_duration":0.001042765,"stat_sum":{"num_bytes":14365712,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":37,"num_read_kb":12308,"num_write":314,"num_write_kb":22325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"53'490","reported_seq":2609,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:26.721067+0000","last_change":"2023-10-06T04:11:26.721045+0000","last_active":"2023-10-06T04:11:26.721067+0000","last_peered":"2023-10-06T04:11:26.721067+0000","last_clean":"2023-10-06T04:11:26.721067+0000","last_became_active":"2023-10-06T03:57:11.373216+0000","last_became_peered":"2023-10-06T03:57:11.373216+0000","last_unstale":"2023-10-06T04:11:26.721067+0000","last_undegraded":"2023-10-06T04:11:26.721067+0000","last_fullsized":"2023-10-06T04:11:26.721067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'490","last_scrub_stamp":"2023-10-06T04:11:26.720977+0000","last_deep_scrub":"53'490","last_deep_scrub_stamp":"2023-10-06T04:11:26.720977+0000","last_clean_scrub_stamp":"2023-10-06T04:11:26.720977+0000","objects_scrubbed":144,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:11.472062+0000","scrub_duration":0.10190239399999999,"objects_trimmed":0,"snaptrim_duration":0.0022456289999999999,"stat_sum":{"num_bytes":5765763,"num_objects":144,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":0,"num_read":36,"num_read_kb":18,"num_write":320,"num_write_kb":9892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"53'516","reported_seq":2630,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:26.988802+0000","last_change":"2023-10-06T04:11:26.988779+0000","last_active":"2023-10-06T04:11:26.988802+0000","last_peered":"2023-10-06T04:11:26.988802+0000","last_clean":"2023-10-06T04:11:26.988802+0000","last_became_active":"2023-10-06T03:57:11.375689+0000","last_became_peered":"2023-10-06T03:57:11.375689+0000","last_unstale":"2023-10-06T04:11:26.988802+0000","last_undegraded":"2023-10-06T04:11:26.988802+0000","last_fullsized":"2023-10-06T04:11:26.988802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'516","last_scrub_stamp":"2023-10-06T04:11:26.988706+0000","last_deep_scrub":"53'516","last_deep_scrub_stamp":"2023-10-06T04:11:26.988706+0000","last_clean_scrub_stamp":"2023-10-06T04:11:26.988706+0000","objects_scrubbed":168,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:32.963008+0000","scrub_duration":0.12428009,"objects_trimmed":0,"snaptrim_duration":0.00092625599999999998,"stat_sum":{"num_bytes":15107134,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":322,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"53'458","reported_seq":2577,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:02.990894+0000","last_change":"2023-10-06T04:11:02.990870+0000","last_active":"2023-10-06T04:11:02.990894+0000","last_peered":"2023-10-06T04:11:02.990894+0000","last_clean":"2023-10-06T04:11:02.990894+0000","last_became_active":"2023-10-06T03:57:11.366543+0000","last_became_peered":"2023-10-06T03:57:11.366543+0000","last_unstale":"2023-10-06T04:11:02.990894+0000","last_undegraded":"2023-10-06T04:11:02.990894+0000","last_fullsized":"2023-10-06T04:11:02.990894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'458","last_scrub_stamp":"2023-10-06T04:11:02.990798+0000","last_deep_scrub":"53'458","last_deep_scrub_stamp":"2023-10-06T04:11:02.990798+0000","last_clean_scrub_stamp":"2023-10-06T04:11:02.990798+0000","objects_scrubbed":155,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:39.255270+0000","scrub_duration":0.116930864,"objects_trimmed":0,"snaptrim_duration":0.0024276609999999998,"stat_sum":{"num_bytes":19833591,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":35,"num_read_kb":19,"num_write":334,"num_write_kb":22749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"53'512","reported_seq":2629,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:23.466557+0000","last_change":"2023-10-06T04:11:23.466513+0000","last_active":"2023-10-06T04:11:23.466557+0000","last_peered":"2023-10-06T04:11:23.466557+0000","last_clean":"2023-10-06T04:11:23.466557+0000","last_became_active":"2023-10-06T03:57:11.366767+0000","last_became_peered":"2023-10-06T03:57:11.366767+0000","last_unstale":"2023-10-06T04:11:23.466557+0000","last_undegraded":"2023-10-06T04:11:23.466557+0000","last_fullsized":"2023-10-06T04:11:23.466557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'512","last_scrub_stamp":"2023-10-06T04:11:23.466398+0000","last_deep_scrub":"53'512","last_deep_scrub_stamp":"2023-10-06T04:11:23.466398+0000","last_clean_scrub_stamp":"2023-10-06T04:11:23.466398+0000","objects_scrubbed":158,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:34.859597+0000","scrub_duration":0.14575623900000001,"objects_trimmed":0,"snaptrim_duration":0.002777833,"stat_sum":{"num_bytes":14683528,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":34,"num_read_kb":8213,"num_write":334,"num_write_kb":18604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"53'370","reported_seq":2481,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.128378+0000","last_change":"2023-10-06T04:05:02.723910+0000","last_active":"2023-10-06T04:10:55.128378+0000","last_peered":"2023-10-06T04:10:55.128378+0000","last_clean":"2023-10-06T04:10:55.128378+0000","last_became_active":"2023-10-06T03:57:11.362377+0000","last_became_peered":"2023-10-06T03:57:11.362377+0000","last_unstale":"2023-10-06T04:10:55.128378+0000","last_undegraded":"2023-10-06T04:10:55.128378+0000","last_fullsized":"2023-10-06T04:10:55.128378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027606900000000001,"stat_sum":{"num_bytes":10886042,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":43,"num_read_kb":26,"num_write":338,"num_write_kb":10805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"53'458","reported_seq":2581,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:13.608185+0000","last_change":"2023-10-06T04:11:13.608149+0000","last_active":"2023-10-06T04:11:13.608185+0000","last_peered":"2023-10-06T04:11:13.608185+0000","last_clean":"2023-10-06T04:11:13.608185+0000","last_became_active":"2023-10-06T03:57:11.363792+0000","last_became_peered":"2023-10-06T03:57:11.363792+0000","last_unstale":"2023-10-06T04:11:13.608185+0000","last_undegraded":"2023-10-06T04:11:13.608185+0000","last_fullsized":"2023-10-06T04:11:13.608185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'458","last_scrub_stamp":"2023-10-06T04:11:13.608057+0000","last_deep_scrub":"53'458","last_deep_scrub_stamp":"2023-10-06T04:11:13.608057+0000","last_clean_scrub_stamp":"2023-10-06T04:11:13.608057+0000","objects_scrubbed":159,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:44.566660+0000","scrub_duration":0.104721544,"objects_trimmed":0,"snaptrim_duration":0.0014611940000000001,"stat_sum":{"num_bytes":14666818,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":40,"num_read_kb":8209,"num_write":335,"num_write_kb":18594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"53'367","reported_seq":2471,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.131549+0000","last_change":"2023-10-06T04:05:02.724524+0000","last_active":"2023-10-06T04:10:55.131549+0000","last_peered":"2023-10-06T04:10:55.131549+0000","last_clean":"2023-10-06T04:10:55.131549+0000","last_became_active":"2023-10-06T03:57:11.364726+0000","last_became_peered":"2023-10-06T03:57:11.364726+0000","last_unstale":"2023-10-06T04:10:55.131549+0000","last_undegraded":"2023-10-06T04:10:55.131549+0000","last_fullsized":"2023-10-06T04:10:55.131549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029763670000000002,"stat_sum":{"num_bytes":20706667,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":37,"num_read_kb":12308,"num_write":348,"num_write_kb":28542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"53'507","reported_seq":2633,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:38.133749+0000","last_change":"2023-10-06T04:11:38.133716+0000","last_active":"2023-10-06T04:11:38.133749+0000","last_peered":"2023-10-06T04:11:38.133749+0000","last_clean":"2023-10-06T04:11:38.133749+0000","last_became_active":"2023-10-06T03:57:11.375668+0000","last_became_peered":"2023-10-06T03:57:11.375668+0000","last_unstale":"2023-10-06T04:11:38.133749+0000","last_undegraded":"2023-10-06T04:11:38.133749+0000","last_fullsized":"2023-10-06T04:11:38.133749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'507","last_scrub_stamp":"2023-10-06T04:11:38.133638+0000","last_deep_scrub":"53'507","last_deep_scrub_stamp":"2023-10-06T04:11:38.133638+0000","last_clean_scrub_stamp":"2023-10-06T04:11:38.133638+0000","objects_scrubbed":159,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:36.224532+0000","scrub_duration":0.20192412500000001,"objects_trimmed":0,"snaptrim_duration":0.0010285800000000001,"stat_sum":{"num_bytes":19005599,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":42,"num_read_kb":8215,"num_write":326,"num_write_kb":18702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"53'528","reported_seq":2646,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:24.447665+0000","last_change":"2023-10-06T04:11:24.447630+0000","last_active":"2023-10-06T04:11:24.447665+0000","last_peered":"2023-10-06T04:11:24.447665+0000","last_clean":"2023-10-06T04:11:24.447665+0000","last_became_active":"2023-10-06T03:57:11.365149+0000","last_became_peered":"2023-10-06T03:57:11.365149+0000","last_unstale":"2023-10-06T04:11:24.447665+0000","last_undegraded":"2023-10-06T04:11:24.447665+0000","last_fullsized":"2023-10-06T04:11:24.447665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'528","last_scrub_stamp":"2023-10-06T04:11:24.447558+0000","last_deep_scrub":"53'528","last_deep_scrub_stamp":"2023-10-06T04:11:24.447558+0000","last_clean_scrub_stamp":"2023-10-06T04:11:24.447558+0000","objects_scrubbed":167,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:44.722933+0000","scrub_duration":0.104800032,"objects_trimmed":0,"snaptrim_duration":0.0020974879999999998,"stat_sum":{"num_bytes":5860015,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":34,"num_read_kb":17,"num_write":331,"num_write_kb":9992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"53'517","reported_seq":2642,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:22.576927+0000","last_change":"2023-10-06T04:11:22.576904+0000","last_active":"2023-10-06T04:11:22.576927+0000","last_peered":"2023-10-06T04:11:22.576927+0000","last_clean":"2023-10-06T04:11:22.576927+0000","last_became_active":"2023-10-06T03:57:11.362128+0000","last_became_peered":"2023-10-06T03:57:11.362128+0000","last_unstale":"2023-10-06T04:11:22.576927+0000","last_undegraded":"2023-10-06T04:11:22.576927+0000","last_fullsized":"2023-10-06T04:11:22.576927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'517","last_scrub_stamp":"2023-10-06T04:11:22.576836+0000","last_deep_scrub":"53'517","last_deep_scrub_stamp":"2023-10-06T04:11:22.576836+0000","last_clean_scrub_stamp":"2023-10-06T04:11:22.576836+0000","objects_scrubbed":164,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:20.021257+0000","scrub_duration":0.10803399599999999,"objects_trimmed":0,"snaptrim_duration":0.0025174860000000002,"stat_sum":{"num_bytes":10052289,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":42,"num_read_kb":8214,"num_write":324,"num_write_kb":14077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"53'551","reported_seq":2683,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:40.166837+0000","last_change":"2023-10-06T04:11:40.166804+0000","last_active":"2023-10-06T04:11:40.166837+0000","last_peered":"2023-10-06T04:11:40.166837+0000","last_clean":"2023-10-06T04:11:40.166837+0000","last_became_active":"2023-10-06T03:57:11.375702+0000","last_became_peered":"2023-10-06T03:57:11.375702+0000","last_unstale":"2023-10-06T04:11:40.166837+0000","last_undegraded":"2023-10-06T04:11:40.166837+0000","last_fullsized":"2023-10-06T04:11:40.166837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'551","last_scrub_stamp":"2023-10-06T04:11:40.166664+0000","last_deep_scrub":"53'551","last_deep_scrub_stamp":"2023-10-06T04:11:40.166664+0000","last_clean_scrub_stamp":"2023-10-06T04:11:40.166664+0000","objects_scrubbed":175,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:54.238456+0000","scrub_duration":0.155582093,"objects_trimmed":0,"snaptrim_duration":0.0010337009999999999,"stat_sum":{"num_bytes":19987419,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":47,"num_read_kb":12104,"num_write":347,"num_write_kb":19698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"53'485","reported_seq":2598,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:40.945819+0000","last_change":"2023-10-06T04:11:40.945793+0000","last_active":"2023-10-06T04:11:40.945819+0000","last_peered":"2023-10-06T04:11:40.945819+0000","last_clean":"2023-10-06T04:11:40.945819+0000","last_became_active":"2023-10-06T03:57:11.374842+0000","last_became_peered":"2023-10-06T03:57:11.374842+0000","last_unstale":"2023-10-06T04:11:40.945819+0000","last_undegraded":"2023-10-06T04:11:40.945819+0000","last_fullsized":"2023-10-06T04:11:40.945819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'485","last_scrub_stamp":"2023-10-06T04:11:40.945726+0000","last_deep_scrub":"53'485","last_deep_scrub_stamp":"2023-10-06T04:11:40.945726+0000","last_clean_scrub_stamp":"2023-10-06T04:11:40.945726+0000","objects_scrubbed":162,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:15.336651+0000","scrub_duration":0.114962457,"objects_trimmed":0,"snaptrim_duration":0.001332342,"stat_sum":{"num_bytes":5811346,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":29,"num_read_kb":4115,"num_write":308,"num_write_kb":5844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"53'564","reported_seq":2681,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:36.940922+0000","last_change":"2023-10-06T04:11:36.940898+0000","last_active":"2023-10-06T04:11:36.940922+0000","last_peered":"2023-10-06T04:11:36.940922+0000","last_clean":"2023-10-06T04:11:36.940922+0000","last_became_active":"2023-10-06T03:57:11.361349+0000","last_became_peered":"2023-10-06T03:57:11.361349+0000","last_unstale":"2023-10-06T04:11:36.940922+0000","last_undegraded":"2023-10-06T04:11:36.940922+0000","last_fullsized":"2023-10-06T04:11:36.940922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'564","last_scrub_stamp":"2023-10-06T04:11:36.940826+0000","last_deep_scrub":"53'564","last_deep_scrub_stamp":"2023-10-06T04:11:36.940826+0000","last_clean_scrub_stamp":"2023-10-06T04:11:36.940826+0000","objects_scrubbed":171,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:06.680252+0000","scrub_duration":0.13662444100000001,"objects_trimmed":0,"snaptrim_duration":0.0014394589999999999,"stat_sum":{"num_bytes":23015133,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":33,"num_read_kb":4115,"num_write":357,"num_write_kb":22663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"53'328","reported_seq":2434,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.131318+0000","last_change":"2023-10-06T04:05:02.972918+0000","last_active":"2023-10-06T04:10:55.131318+0000","last_peered":"2023-10-06T04:10:55.131318+0000","last_clean":"2023-10-06T04:10:55.131318+0000","last_became_active":"2023-10-06T03:57:11.375846+0000","last_became_peered":"2023-10-06T03:57:11.375846+0000","last_unstale":"2023-10-06T04:10:55.131318+0000","last_undegraded":"2023-10-06T04:10:55.131318+0000","last_fullsized":"2023-10-06T04:10:55.131318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018480599999999999,"stat_sum":{"num_bytes":26318621,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":38,"num_read_kb":12302,"num_write":306,"num_write_kb":25858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"53'462","reported_seq":528,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:25.963693+0000","last_change":"2023-10-06T04:11:25.905654+0000","last_active":"2023-10-06T04:11:25.963693+0000","last_peered":"2023-10-06T04:11:25.963693+0000","last_clean":"2023-10-06T04:11:25.963693+0000","last_became_active":"2023-10-06T03:57:10.348078+0000","last_became_peered":"2023-10-06T03:57:10.348078+0000","last_unstale":"2023-10-06T04:11:25.963693+0000","last_undegraded":"2023-10-06T04:11:25.963693+0000","last_fullsized":"2023-10-06T04:11:25.963693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'460","last_scrub_stamp":"2023-10-06T04:11:25.905554+0000","last_deep_scrub":"53'460","last_deep_scrub_stamp":"2023-10-06T04:11:25.905554+0000","last_clean_scrub_stamp":"2023-10-06T04:11:25.905554+0000","objects_scrubbed":11,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:22.245460+0000","scrub_duration":0.035131687000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15927542,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":13,"num_read_kb":12332,"num_write":447,"num_write_kb":15804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30320,"num_omap_keys":54,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"53'477","reported_seq":2599,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:17.713507+0000","last_change":"2023-10-06T04:11:17.713485+0000","last_active":"2023-10-06T04:11:17.713507+0000","last_peered":"2023-10-06T04:11:17.713507+0000","last_clean":"2023-10-06T04:11:17.713507+0000","last_became_active":"2023-10-06T03:57:11.370245+0000","last_became_peered":"2023-10-06T03:57:11.370245+0000","last_unstale":"2023-10-06T04:11:17.713507+0000","last_undegraded":"2023-10-06T04:11:17.713507+0000","last_fullsized":"2023-10-06T04:11:17.713507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'477","last_scrub_stamp":"2023-10-06T04:11:17.713414+0000","last_deep_scrub":"53'477","last_deep_scrub_stamp":"2023-10-06T04:11:17.713414+0000","last_clean_scrub_stamp":"2023-10-06T04:11:17.713414+0000","objects_scrubbed":162,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:09.132306+0000","scrub_duration":0.116172479,"objects_trimmed":0,"snaptrim_duration":0.002348424,"stat_sum":{"num_bytes":22265950,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":38,"num_read_kb":9495,"num_write":317,"num_write_kb":26009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"53'95","reported_seq":185,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:19.796378+0000","last_change":"2023-10-06T04:11:19.796356+0000","last_active":"2023-10-06T04:11:19.796378+0000","last_peered":"2023-10-06T04:11:19.796378+0000","last_clean":"2023-10-06T04:11:19.796378+0000","last_became_active":"2023-10-06T03:57:10.345150+0000","last_became_peered":"2023-10-06T03:57:10.345150+0000","last_unstale":"2023-10-06T04:11:19.796378+0000","last_undegraded":"2023-10-06T04:11:19.796378+0000","last_fullsized":"2023-10-06T04:11:19.796378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'95","last_scrub_stamp":"2023-10-06T04:11:19.796288+0000","last_deep_scrub":"53'95","last_deep_scrub_stamp":"2023-10-06T04:11:19.796288+0000","last_clean_scrub_stamp":"2023-10-06T04:11:19.796288+0000","objects_scrubbed":6,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:50.621259+0000","scrub_duration":0.0099644750000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4312676,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":146,"num_write":89,"num_write_kb":4275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":27214,"num_omap_keys":50,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"53'509","reported_seq":2619,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:33.482977+0000","last_change":"2023-10-06T04:11:33.482954+0000","last_active":"2023-10-06T04:11:33.482977+0000","last_peered":"2023-10-06T04:11:33.482977+0000","last_clean":"2023-10-06T04:11:33.482977+0000","last_became_active":"2023-10-06T03:57:11.375790+0000","last_became_peered":"2023-10-06T03:57:11.375790+0000","last_unstale":"2023-10-06T04:11:33.482977+0000","last_undegraded":"2023-10-06T04:11:33.482977+0000","last_fullsized":"2023-10-06T04:11:33.482977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'509","last_scrub_stamp":"2023-10-06T04:11:33.482884+0000","last_deep_scrub":"53'509","last_deep_scrub_stamp":"2023-10-06T04:11:33.482884+0000","last_clean_scrub_stamp":"2023-10-06T04:11:33.482884+0000","objects_scrubbed":159,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:48.231531+0000","scrub_duration":0.127308381,"objects_trimmed":0,"snaptrim_duration":0.0025999780000000002,"stat_sum":{"num_bytes":19085062,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":26,"num_read_kb":12300,"num_write":331,"num_write_kb":18812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"53'1080","reported_seq":1140,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:10.608190+0000","last_change":"2023-10-06T04:11:10.608166+0000","last_active":"2023-10-06T04:11:10.608190+0000","last_peered":"2023-10-06T04:11:10.608190+0000","last_clean":"2023-10-06T04:11:10.608190+0000","last_became_active":"2023-10-06T03:57:10.347294+0000","last_became_peered":"2023-10-06T03:57:10.347294+0000","last_unstale":"2023-10-06T04:11:10.608190+0000","last_undegraded":"2023-10-06T04:11:10.608190+0000","last_fullsized":"2023-10-06T04:11:10.608190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1080","last_scrub_stamp":"2023-10-06T04:11:10.608093+0000","last_deep_scrub":"53'1080","last_deep_scrub_stamp":"2023-10-06T04:11:10.608093+0000","last_clean_scrub_stamp":"2023-10-06T04:11:10.608093+0000","objects_scrubbed":8,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:54.093395+0000","scrub_duration":0.023937142000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583466,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":7,"num_read_kb":16784,"num_write":1067,"num_write_kb":17022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30102,"num_omap_keys":52,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"53'457","reported_seq":2576,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:16.190235+0000","last_change":"2023-10-06T04:11:16.190201+0000","last_active":"2023-10-06T04:11:16.190235+0000","last_peered":"2023-10-06T04:11:16.190235+0000","last_clean":"2023-10-06T04:11:16.190235+0000","last_became_active":"2023-10-06T03:57:11.371758+0000","last_became_peered":"2023-10-06T03:57:11.371758+0000","last_unstale":"2023-10-06T04:11:16.190235+0000","last_undegraded":"2023-10-06T04:11:16.190235+0000","last_fullsized":"2023-10-06T04:11:16.190235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'457","last_scrub_stamp":"2023-10-06T04:11:16.190090+0000","last_deep_scrub":"53'457","last_deep_scrub_stamp":"2023-10-06T04:11:16.190090+0000","last_clean_scrub_stamp":"2023-10-06T04:11:16.190090+0000","objects_scrubbed":153,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:59.896644+0000","scrub_duration":0.12942683099999999,"objects_trimmed":0,"snaptrim_duration":0.0017888349999999999,"stat_sum":{"num_bytes":16888108,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":35,"num_read_kb":4115,"num_write":349,"num_write_kb":16624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"53'54","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:05.959897+0000","last_change":"2023-10-06T04:11:02.285761+0000","last_active":"2023-10-06T04:11:05.959897+0000","last_peered":"2023-10-06T04:11:05.959897+0000","last_clean":"2023-10-06T04:11:05.959897+0000","last_became_active":"2023-10-06T03:57:10.347151+0000","last_became_peered":"2023-10-06T03:57:10.347151+0000","last_unstale":"2023-10-06T04:11:05.959897+0000","last_undegraded":"2023-10-06T04:11:05.959897+0000","last_fullsized":"2023-10-06T04:11:05.959897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'52","last_scrub_stamp":"2023-10-06T04:11:02.285649+0000","last_deep_scrub":"53'52","last_deep_scrub_stamp":"2023-10-06T04:11:02.285649+0000","last_clean_scrub_stamp":"2023-10-06T04:11:02.285649+0000","objects_scrubbed":7,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:44.621007+0000","scrub_duration":0.0062605860000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":52,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":37322,"num_omap_keys":65,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"53'494","reported_seq":2614,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:21.604835+0000","last_change":"2023-10-06T04:11:21.604798+0000","last_active":"2023-10-06T04:11:21.604835+0000","last_peered":"2023-10-06T04:11:21.604835+0000","last_clean":"2023-10-06T04:11:21.604835+0000","last_became_active":"2023-10-06T03:57:11.369938+0000","last_became_peered":"2023-10-06T03:57:11.369938+0000","last_unstale":"2023-10-06T04:11:21.604835+0000","last_undegraded":"2023-10-06T04:11:21.604835+0000","last_fullsized":"2023-10-06T04:11:21.604835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'494","last_scrub_stamp":"2023-10-06T04:11:21.604691+0000","last_deep_scrub":"53'494","last_deep_scrub_stamp":"2023-10-06T04:11:21.604691+0000","last_clean_scrub_stamp":"2023-10-06T04:11:21.604691+0000","objects_scrubbed":172,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:46.110446+0000","scrub_duration":0.142007095,"objects_trimmed":0,"snaptrim_duration":0.0018579569999999999,"stat_sum":{"num_bytes":31490698,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":37,"num_read_kb":12302,"num_write":314,"num_write_kb":31163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"53'53","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:14.740559+0000","last_change":"2023-10-06T04:11:14.740529+0000","last_active":"2023-10-06T04:11:14.740559+0000","last_peered":"2023-10-06T04:11:14.740559+0000","last_clean":"2023-10-06T04:11:14.740559+0000","last_became_active":"2023-10-06T03:57:10.345721+0000","last_became_peered":"2023-10-06T03:57:10.345721+0000","last_unstale":"2023-10-06T04:11:14.740559+0000","last_undegraded":"2023-10-06T04:11:14.740559+0000","last_fullsized":"2023-10-06T04:11:14.740559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'53","last_scrub_stamp":"2023-10-06T04:11:14.740442+0000","last_deep_scrub":"53'53","last_deep_scrub_stamp":"2023-10-06T04:11:14.740442+0000","last_clean_scrub_stamp":"2023-10-06T04:11:14.740442+0000","objects_scrubbed":4,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:37.780654+0000","scrub_duration":0.013645548,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":15,"num_write":48,"num_write_kb":4121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9078,"num_omap_keys":16,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"53'542","reported_seq":2660,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:23.510740+0000","last_change":"2023-10-06T04:11:23.510701+0000","last_active":"2023-10-06T04:11:23.510740+0000","last_peered":"2023-10-06T04:11:23.510740+0000","last_clean":"2023-10-06T04:11:23.510740+0000","last_became_active":"2023-10-06T03:57:59.255587+0000","last_became_peered":"2023-10-06T03:57:59.255587+0000","last_unstale":"2023-10-06T04:11:23.510740+0000","last_undegraded":"2023-10-06T04:11:23.510740+0000","last_fullsized":"2023-10-06T04:11:23.510740+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'542","last_scrub_stamp":"2023-10-06T04:11:23.510570+0000","last_deep_scrub":"53'542","last_deep_scrub_stamp":"2023-10-06T04:11:23.510570+0000","last_clean_scrub_stamp":"2023-10-06T04:11:23.510570+0000","objects_scrubbed":175,"log_size":542,"log_dups_size":0,"ondisk_log_size":542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:07:13.627016+0000","scrub_duration":0.14334406899999999,"objects_trimmed":0,"snaptrim_duration":0.001510024,"stat_sum":{"num_bytes":1845822,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":38,"num_read_kb":8209,"num_write":345,"num_write_kb":10179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"53'69","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:21.778145+0000","last_change":"2023-10-06T04:11:21.778119+0000","last_active":"2023-10-06T04:11:21.778145+0000","last_peered":"2023-10-06T04:11:21.778145+0000","last_clean":"2023-10-06T04:11:21.778145+0000","last_became_active":"2023-10-06T03:57:10.340232+0000","last_became_peered":"2023-10-06T03:57:10.340232+0000","last_unstale":"2023-10-06T04:11:21.778145+0000","last_undegraded":"2023-10-06T04:11:21.778145+0000","last_fullsized":"2023-10-06T04:11:21.778145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'69","last_scrub_stamp":"2023-10-06T04:11:21.778042+0000","last_deep_scrub":"53'69","last_deep_scrub_stamp":"2023-10-06T04:11:21.778042+0000","last_clean_scrub_stamp":"2023-10-06T04:11:21.778042+0000","objects_scrubbed":6,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:35:39.233114+0000","scrub_duration":0.011501937,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":66,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24992,"num_omap_keys":46,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"53'528","reported_seq":2649,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:37.973900+0000","last_change":"2023-10-06T04:11:37.973866+0000","last_active":"2023-10-06T04:11:37.973900+0000","last_peered":"2023-10-06T04:11:37.973900+0000","last_clean":"2023-10-06T04:11:37.973900+0000","last_became_active":"2023-10-06T03:57:11.374892+0000","last_became_peered":"2023-10-06T03:57:11.374892+0000","last_unstale":"2023-10-06T04:11:37.973900+0000","last_undegraded":"2023-10-06T04:11:37.973900+0000","last_fullsized":"2023-10-06T04:11:37.973900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'528","last_scrub_stamp":"2023-10-06T04:11:37.973741+0000","last_deep_scrub":"53'528","last_deep_scrub_stamp":"2023-10-06T04:11:37.973741+0000","last_clean_scrub_stamp":"2023-10-06T04:11:37.973741+0000","objects_scrubbed":164,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:25.227153+0000","scrub_duration":0.14730583799999999,"objects_trimmed":0,"snaptrim_duration":0.00156207,"stat_sum":{"num_bytes":17801071,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":38,"num_read_kb":8211,"num_write":336,"num_write_kb":25748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"53'680","reported_seq":748,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:19.900780+0000","last_change":"2023-10-06T04:11:19.900740+0000","last_active":"2023-10-06T04:11:19.900780+0000","last_peered":"2023-10-06T04:11:19.900780+0000","last_clean":"2023-10-06T04:11:19.900780+0000","last_became_active":"2023-10-06T03:57:10.346430+0000","last_became_peered":"2023-10-06T03:57:10.346430+0000","last_unstale":"2023-10-06T04:11:19.900780+0000","last_undegraded":"2023-10-06T04:11:19.900780+0000","last_fullsized":"2023-10-06T04:11:19.900780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'680","last_scrub_stamp":"2023-10-06T04:11:19.900666+0000","last_deep_scrub":"53'680","last_deep_scrub_stamp":"2023-10-06T04:11:19.900666+0000","last_clean_scrub_stamp":"2023-10-06T04:11:19.900666+0000","objects_scrubbed":9,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:55.003315+0000","scrub_duration":0.024294989999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389252,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":30,"num_read_kb":9887,"num_write":670,"num_write_kb":8556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":23944,"num_omap_keys":41,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"53'354","reported_seq":2460,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.130611+0000","last_change":"2023-10-06T04:05:02.723446+0000","last_active":"2023-10-06T04:10:55.130611+0000","last_peered":"2023-10-06T04:10:55.130611+0000","last_clean":"2023-10-06T04:10:55.130611+0000","last_became_active":"2023-10-06T03:57:11.367565+0000","last_became_peered":"2023-10-06T03:57:11.367565+0000","last_unstale":"2023-10-06T04:10:55.130611+0000","last_undegraded":"2023-10-06T04:10:55.130611+0000","last_fullsized":"2023-10-06T04:10:55.130611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027762680000000001,"stat_sum":{"num_bytes":10311229,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":38,"num_read_kb":23,"num_write":322,"num_write_kb":10236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"53'188","reported_seq":255,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:11.762952+0000","last_change":"2023-10-06T04:11:11.762926+0000","last_active":"2023-10-06T04:11:11.762952+0000","last_peered":"2023-10-06T04:11:11.762952+0000","last_clean":"2023-10-06T04:11:11.762952+0000","last_became_active":"2023-10-06T03:57:10.350795+0000","last_became_peered":"2023-10-06T03:57:10.350795+0000","last_unstale":"2023-10-06T04:11:11.762952+0000","last_undegraded":"2023-10-06T04:11:11.762952+0000","last_fullsized":"2023-10-06T04:11:11.762952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'188","last_scrub_stamp":"2023-10-06T04:11:11.762854+0000","last_deep_scrub":"53'188","last_deep_scrub_stamp":"2023-10-06T04:11:11.762854+0000","last_clean_scrub_stamp":"2023-10-06T04:11:11.762854+0000","objects_scrubbed":9,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:37.116115+0000","scrub_duration":0.020924005999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388676,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":30,"num_read_kb":20509,"num_write":184,"num_write_kb":8306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":18896,"num_omap_keys":34,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"53'406","reported_seq":2522,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:14.748333+0000","last_change":"2023-10-06T04:11:14.748308+0000","last_active":"2023-10-06T04:11:14.748333+0000","last_peered":"2023-10-06T04:11:14.748333+0000","last_clean":"2023-10-06T04:11:14.748333+0000","last_became_active":"2023-10-06T03:57:11.367254+0000","last_became_peered":"2023-10-06T03:57:11.367254+0000","last_unstale":"2023-10-06T04:11:14.748333+0000","last_undegraded":"2023-10-06T04:11:14.748333+0000","last_fullsized":"2023-10-06T04:11:14.748333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'406","last_scrub_stamp":"2023-10-06T04:11:14.748233+0000","last_deep_scrub":"53'406","last_deep_scrub_stamp":"2023-10-06T04:11:14.748233+0000","last_clean_scrub_stamp":"2023-10-06T04:11:14.748233+0000","objects_scrubbed":165,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:29.852108+0000","scrub_duration":0.096980769999999994,"objects_trimmed":0,"snaptrim_duration":0.0027822569999999998,"stat_sum":{"num_bytes":13704260,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":32,"num_read_kb":10530,"num_write":279,"num_write_kb":17631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"53'111","reported_seq":174,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:04.279658+0000","last_change":"2023-10-06T04:11:04.279631+0000","last_active":"2023-10-06T04:11:04.279658+0000","last_peered":"2023-10-06T04:11:04.279658+0000","last_clean":"2023-10-06T04:11:04.279658+0000","last_became_active":"2023-10-06T03:57:10.337798+0000","last_became_peered":"2023-10-06T03:57:10.337798+0000","last_unstale":"2023-10-06T04:11:04.279658+0000","last_undegraded":"2023-10-06T04:11:04.279658+0000","last_fullsized":"2023-10-06T04:11:04.279658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'111","last_scrub_stamp":"2023-10-06T04:11:04.279550+0000","last_deep_scrub":"53'111","last_deep_scrub_stamp":"2023-10-06T04:11:04.279550+0000","last_clean_scrub_stamp":"2023-10-06T04:11:04.279550+0000","objects_scrubbed":6,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:39.866223+0000","scrub_duration":0.015622552,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":9753,"num_write":104,"num_write_kb":4165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":14386,"num_omap_keys":26,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"53'356","reported_seq":2461,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.131127+0000","last_change":"2023-10-06T04:05:02.973084+0000","last_active":"2023-10-06T04:10:55.131127+0000","last_peered":"2023-10-06T04:10:55.131127+0000","last_clean":"2023-10-06T04:10:55.131127+0000","last_became_active":"2023-10-06T03:57:11.376223+0000","last_became_peered":"2023-10-06T03:57:11.376223+0000","last_unstale":"2023-10-06T04:10:55.131127+0000","last_undegraded":"2023-10-06T04:10:55.131127+0000","last_fullsized":"2023-10-06T04:10:55.131127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018161119999999999,"stat_sum":{"num_bytes":34940720,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":37,"num_read_kb":11570,"num_write":336,"num_write_kb":34292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"53'135","reported_seq":188,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:25.960222+0000","last_change":"2023-10-06T04:11:10.713437+0000","last_active":"2023-10-06T04:11:25.960222+0000","last_peered":"2023-10-06T04:11:25.960222+0000","last_clean":"2023-10-06T04:11:25.960222+0000","last_became_active":"2023-10-06T03:57:10.350219+0000","last_became_peered":"2023-10-06T03:57:10.350219+0000","last_unstale":"2023-10-06T04:11:25.960222+0000","last_undegraded":"2023-10-06T04:11:25.960222+0000","last_fullsized":"2023-10-06T04:11:25.960222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'126","last_scrub_stamp":"2023-10-06T04:11:10.713339+0000","last_deep_scrub":"53'126","last_deep_scrub_stamp":"2023-10-06T04:11:10.713339+0000","last_clean_scrub_stamp":"2023-10-06T04:11:10.713339+0000","objects_scrubbed":8,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:04.717440+0000","scrub_duration":0.020180555999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4909910,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":9008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":51900,"num_omap_keys":89,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"53'535","reported_seq":2664,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:38.507456+0000","last_change":"2023-10-06T04:11:38.507435+0000","last_active":"2023-10-06T04:11:38.507456+0000","last_peered":"2023-10-06T04:11:38.507456+0000","last_clean":"2023-10-06T04:11:38.507456+0000","last_became_active":"2023-10-06T03:57:11.362443+0000","last_became_peered":"2023-10-06T03:57:11.362443+0000","last_unstale":"2023-10-06T04:11:38.507456+0000","last_undegraded":"2023-10-06T04:11:38.507456+0000","last_fullsized":"2023-10-06T04:11:38.507456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'535","last_scrub_stamp":"2023-10-06T04:11:38.507361+0000","last_deep_scrub":"53'535","last_deep_scrub_stamp":"2023-10-06T04:11:38.507361+0000","last_clean_scrub_stamp":"2023-10-06T04:11:38.507361+0000","objects_scrubbed":154,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:06:02.993776+0000","scrub_duration":0.11848410199999999,"objects_trimmed":0,"snaptrim_duration":0.002448839,"stat_sum":{"num_bytes":22978165,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":42,"num_read_kb":8288,"num_write":356,"num_write_kb":22628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"53'83","reported_seq":147,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:08.057909+0000","last_change":"2023-10-06T04:11:08.057878+0000","last_active":"2023-10-06T04:11:08.057909+0000","last_peered":"2023-10-06T04:11:08.057909+0000","last_clean":"2023-10-06T04:11:08.057909+0000","last_became_active":"2023-10-06T03:57:10.345424+0000","last_became_peered":"2023-10-06T03:57:10.345424+0000","last_unstale":"2023-10-06T04:11:08.057909+0000","last_undegraded":"2023-10-06T04:11:08.057909+0000","last_fullsized":"2023-10-06T04:11:08.057909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'83","last_scrub_stamp":"2023-10-06T04:11:08.057782+0000","last_deep_scrub":"53'83","last_deep_scrub_stamp":"2023-10-06T04:11:08.057782+0000","last_clean_scrub_stamp":"2023-10-06T04:11:08.057782+0000","objects_scrubbed":12,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:41:48.011373+0000","scrub_duration":0.0096812489999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":33,"num_read_kb":33,"num_write":72,"num_write_kb":4158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31156,"num_omap_keys":54,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"53'419","reported_seq":2541,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:12.728194+0000","last_change":"2023-10-06T04:11:12.728172+0000","last_active":"2023-10-06T04:11:12.728194+0000","last_peered":"2023-10-06T04:11:12.728194+0000","last_clean":"2023-10-06T04:11:12.728194+0000","last_became_active":"2023-10-06T03:57:11.375824+0000","last_became_peered":"2023-10-06T03:57:11.375824+0000","last_unstale":"2023-10-06T04:11:12.728194+0000","last_undegraded":"2023-10-06T04:11:12.728194+0000","last_fullsized":"2023-10-06T04:11:12.728194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'419","last_scrub_stamp":"2023-10-06T04:11:12.728105+0000","last_deep_scrub":"53'419","last_deep_scrub_stamp":"2023-10-06T04:11:12.728105+0000","last_clean_scrub_stamp":"2023-10-06T04:11:12.728105+0000","objects_scrubbed":150,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:57.027413+0000","scrub_duration":0.083998824,"objects_trimmed":0,"snaptrim_duration":0.0023200299999999998,"stat_sum":{"num_bytes":9857160,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":38,"num_read_kb":8212,"num_write":304,"num_write_kb":9779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"53'256","reported_seq":312,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:20.763949+0000","last_change":"2023-10-06T04:11:20.763919+0000","last_active":"2023-10-06T04:11:20.763949+0000","last_peered":"2023-10-06T04:11:20.763949+0000","last_clean":"2023-10-06T04:11:20.763949+0000","last_became_active":"2023-10-06T03:57:10.350135+0000","last_became_peered":"2023-10-06T03:57:10.350135+0000","last_unstale":"2023-10-06T04:11:20.763949+0000","last_undegraded":"2023-10-06T04:11:20.763949+0000","last_fullsized":"2023-10-06T04:11:20.763949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'256","last_scrub_stamp":"2023-10-06T04:11:20.763831+0000","last_deep_scrub":"53'256","last_deep_scrub_stamp":"2023-10-06T04:11:20.763831+0000","last_clean_scrub_stamp":"2023-10-06T04:11:20.763831+0000","objects_scrubbed":6,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:26.714475+0000","scrub_duration":0.01459685,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":4098,"num_write":280,"num_write_kb":4293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":28994,"num_omap_keys":51,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"53'511","reported_seq":2627,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:30.698925+0000","last_change":"2023-10-06T04:11:30.698890+0000","last_active":"2023-10-06T04:11:30.698925+0000","last_peered":"2023-10-06T04:11:30.698925+0000","last_clean":"2023-10-06T04:11:30.698925+0000","last_became_active":"2023-10-06T03:57:11.376236+0000","last_became_peered":"2023-10-06T03:57:11.376236+0000","last_unstale":"2023-10-06T04:11:30.698925+0000","last_undegraded":"2023-10-06T04:11:30.698925+0000","last_fullsized":"2023-10-06T04:11:30.698925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'511","last_scrub_stamp":"2023-10-06T04:11:30.698814+0000","last_deep_scrub":"53'511","last_deep_scrub_stamp":"2023-10-06T04:11:30.698814+0000","last_clean_scrub_stamp":"2023-10-06T04:11:30.698814+0000","objects_scrubbed":170,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:58.790004+0000","scrub_duration":0.157753385,"objects_trimmed":0,"snaptrim_duration":0.0015573080000000001,"stat_sum":{"num_bytes":22935537,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":32,"num_read_kb":10317,"num_write":317,"num_write_kb":26655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"53'95","reported_seq":153,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:18.759540+0000","last_change":"2023-10-06T04:11:18.759511+0000","last_active":"2023-10-06T04:11:18.759540+0000","last_peered":"2023-10-06T04:11:18.759540+0000","last_clean":"2023-10-06T04:11:18.759540+0000","last_became_active":"2023-10-06T03:57:10.345510+0000","last_became_peered":"2023-10-06T03:57:10.345510+0000","last_unstale":"2023-10-06T04:11:18.759540+0000","last_undegraded":"2023-10-06T04:11:18.759540+0000","last_fullsized":"2023-10-06T04:11:18.759540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'95","last_scrub_stamp":"2023-10-06T04:11:18.759410+0000","last_deep_scrub":"53'95","last_deep_scrub_stamp":"2023-10-06T04:11:18.759410+0000","last_clean_scrub_stamp":"2023-10-06T04:11:18.759410+0000","objects_scrubbed":3,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:29.676193+0000","scrub_duration":0.014631524999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":4108,"num_write":90,"num_write_kb":4140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"53'359","reported_seq":2464,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.122851+0000","last_change":"2023-10-06T04:05:02.721917+0000","last_active":"2023-10-06T04:10:55.122851+0000","last_peered":"2023-10-06T04:10:55.122851+0000","last_clean":"2023-10-06T04:10:55.122851+0000","last_became_active":"2023-10-06T03:57:11.368375+0000","last_became_peered":"2023-10-06T03:57:11.368375+0000","last_unstale":"2023-10-06T04:10:55.122851+0000","last_undegraded":"2023-10-06T04:10:55.122851+0000","last_fullsized":"2023-10-06T04:10:55.122851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001632912,"stat_sum":{"num_bytes":5657073,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":37,"num_read_kb":4115,"num_write":343,"num_write_kb":5657,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"53'164","reported_seq":216,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:16.734834+0000","last_change":"2023-10-06T04:11:16.734834+0000","last_active":"2023-10-06T04:11:16.734834+0000","last_peered":"2023-10-06T04:11:16.734834+0000","last_clean":"2023-10-06T04:11:16.734834+0000","last_became_active":"2023-10-06T03:57:10.346689+0000","last_became_peered":"2023-10-06T03:57:10.346689+0000","last_unstale":"2023-10-06T04:11:16.734834+0000","last_undegraded":"2023-10-06T04:11:16.734834+0000","last_fullsized":"2023-10-06T04:11:16.734834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:09.323709+0000","last_clean_scrub_stamp":"2023-10-06T03:57:09.323709+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":12290,"num_write":163,"num_write_kb":8336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"53'471","reported_seq":2589,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:13.079557+0000","last_change":"2023-10-06T04:11:13.079535+0000","last_active":"2023-10-06T04:11:13.079557+0000","last_peered":"2023-10-06T04:11:13.079557+0000","last_clean":"2023-10-06T04:11:13.079557+0000","last_became_active":"2023-10-06T03:57:11.371795+0000","last_became_peered":"2023-10-06T03:57:11.371795+0000","last_unstale":"2023-10-06T04:11:13.079557+0000","last_undegraded":"2023-10-06T04:11:13.079557+0000","last_fullsized":"2023-10-06T04:11:13.079557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'471","last_scrub_stamp":"2023-10-06T04:11:13.079467+0000","last_deep_scrub":"53'471","last_deep_scrub_stamp":"2023-10-06T04:11:13.079467+0000","last_clean_scrub_stamp":"2023-10-06T04:11:13.079467+0000","objects_scrubbed":169,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:17.052127+0000","scrub_duration":0.080011451999999997,"objects_trimmed":0,"snaptrim_duration":0.0016252980000000001,"stat_sum":{"num_bytes":5774867,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":34,"num_read_kb":13,"num_write":355,"num_write_kb":5825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"53'59","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:15.755241+0000","last_change":"2023-10-06T04:11:15.755204+0000","last_active":"2023-10-06T04:11:15.755241+0000","last_peered":"2023-10-06T04:11:15.755241+0000","last_clean":"2023-10-06T04:11:15.755241+0000","last_became_active":"2023-10-06T03:57:10.348291+0000","last_became_peered":"2023-10-06T03:57:10.348291+0000","last_unstale":"2023-10-06T04:11:15.755241+0000","last_undegraded":"2023-10-06T04:11:15.755241+0000","last_fullsized":"2023-10-06T04:11:15.755241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'59","last_scrub_stamp":"2023-10-06T04:11:15.755103+0000","last_deep_scrub":"53'59","last_deep_scrub_stamp":"2023-10-06T04:11:15.755103+0000","last_clean_scrub_stamp":"2023-10-06T04:11:15.755103+0000","objects_scrubbed":5,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:27.967550+0000","scrub_duration":0.0073040559999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":57,"num_write_kb":4195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":20734,"num_omap_keys":37,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"53'480","reported_seq":2597,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:30.531095+0000","last_change":"2023-10-06T04:11:30.531072+0000","last_active":"2023-10-06T04:11:30.531095+0000","last_peered":"2023-10-06T04:11:30.531095+0000","last_clean":"2023-10-06T04:11:30.531095+0000","last_became_active":"2023-10-06T03:57:11.365077+0000","last_became_peered":"2023-10-06T03:57:11.365077+0000","last_unstale":"2023-10-06T04:11:30.531095+0000","last_undegraded":"2023-10-06T04:11:30.531095+0000","last_fullsized":"2023-10-06T04:11:30.531095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'480","last_scrub_stamp":"2023-10-06T04:11:30.531002+0000","last_deep_scrub":"53'480","last_deep_scrub_stamp":"2023-10-06T04:11:30.531002+0000","last_clean_scrub_stamp":"2023-10-06T04:11:30.531002+0000","objects_scrubbed":152,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:08.827420+0000","scrub_duration":0.107887435,"objects_trimmed":0,"snaptrim_duration":0.0019352270000000001,"stat_sum":{"num_bytes":15068515,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":33,"num_read_kb":4109,"num_write":308,"num_write_kb":14877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"53'283","reported_seq":345,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:01.636104+0000","last_change":"2023-10-06T04:11:01.636081+0000","last_active":"2023-10-06T04:11:01.636104+0000","last_peered":"2023-10-06T04:11:01.636104+0000","last_clean":"2023-10-06T04:11:01.636104+0000","last_became_active":"2023-10-06T03:57:10.348055+0000","last_became_peered":"2023-10-06T03:57:10.348055+0000","last_unstale":"2023-10-06T04:11:01.636104+0000","last_undegraded":"2023-10-06T04:11:01.636104+0000","last_fullsized":"2023-10-06T04:11:01.636104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'283","last_scrub_stamp":"2023-10-06T04:11:01.636008+0000","last_deep_scrub":"53'283","last_deep_scrub_stamp":"2023-10-06T04:11:01.636008+0000","last_clean_scrub_stamp":"2023-10-06T04:11:01.636008+0000","objects_scrubbed":5,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:11.936218+0000","scrub_duration":0.017511342999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":17,"num_read_kb":12301,"num_write":272,"num_write_kb":8316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"53'496","reported_seq":2609,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:11.948075+0000","last_change":"2023-10-06T04:11:11.948042+0000","last_active":"2023-10-06T04:11:11.948075+0000","last_peered":"2023-10-06T04:11:11.948075+0000","last_clean":"2023-10-06T04:11:11.948075+0000","last_became_active":"2023-10-06T03:57:11.370073+0000","last_became_peered":"2023-10-06T03:57:11.370073+0000","last_unstale":"2023-10-06T04:11:11.948075+0000","last_undegraded":"2023-10-06T04:11:11.948075+0000","last_fullsized":"2023-10-06T04:11:11.948075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'496","last_scrub_stamp":"2023-10-06T04:11:11.947965+0000","last_deep_scrub":"53'496","last_deep_scrub_stamp":"2023-10-06T04:11:11.947965+0000","last_clean_scrub_stamp":"2023-10-06T04:11:11.947965+0000","objects_scrubbed":185,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:02.104713+0000","scrub_duration":0.114303379,"objects_trimmed":0,"snaptrim_duration":0.0027454179999999999,"stat_sum":{"num_bytes":18058404,"num_objects":185,"num_object_clones":0,"num_object_copies":370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":185,"num_whiteouts":0,"num_read":28,"num_read_kb":4110,"num_write":345,"num_write_kb":17816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"53'118","reported_seq":176,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:09.093158+0000","last_change":"2023-10-06T04:11:09.093116+0000","last_active":"2023-10-06T04:11:09.093158+0000","last_peered":"2023-10-06T04:11:09.093158+0000","last_clean":"2023-10-06T04:11:09.093158+0000","last_became_active":"2023-10-06T03:57:10.352653+0000","last_became_peered":"2023-10-06T03:57:10.352653+0000","last_unstale":"2023-10-06T04:11:09.093158+0000","last_undegraded":"2023-10-06T04:11:09.093158+0000","last_fullsized":"2023-10-06T04:11:09.093158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'118","last_scrub_stamp":"2023-10-06T04:11:09.092974+0000","last_deep_scrub":"53'118","last_deep_scrub_stamp":"2023-10-06T04:11:09.092974+0000","last_clean_scrub_stamp":"2023-10-06T04:11:09.092974+0000","objects_scrubbed":6,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:03.839458+0000","scrub_duration":0.016085988999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":8,"num_read_kb":4217,"num_write":207,"num_write_kb":4775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":18688,"num_omap_keys":32,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"53'520","reported_seq":2636,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:35.299212+0000","last_change":"2023-10-06T04:11:35.299190+0000","last_active":"2023-10-06T04:11:35.299212+0000","last_peered":"2023-10-06T04:11:35.299212+0000","last_clean":"2023-10-06T04:11:35.299212+0000","last_became_active":"2023-10-06T03:57:11.363830+0000","last_became_peered":"2023-10-06T03:57:11.363830+0000","last_unstale":"2023-10-06T04:11:35.299212+0000","last_undegraded":"2023-10-06T04:11:35.299212+0000","last_fullsized":"2023-10-06T04:11:35.299212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'520","last_scrub_stamp":"2023-10-06T04:11:35.299124+0000","last_deep_scrub":"53'520","last_deep_scrub_stamp":"2023-10-06T04:11:35.299124+0000","last_clean_scrub_stamp":"2023-10-06T04:11:35.299124+0000","objects_scrubbed":162,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:32.020653+0000","scrub_duration":0.348993265,"objects_trimmed":0,"snaptrim_duration":0.0027698559999999998,"stat_sum":{"num_bytes":22869017,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":32,"num_read_kb":4115,"num_write":333,"num_write_kb":26610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"53'113","reported_seq":174,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:14.850284+0000","last_change":"2023-10-06T04:11:14.850257+0000","last_active":"2023-10-06T04:11:14.850284+0000","last_peered":"2023-10-06T04:11:14.850284+0000","last_clean":"2023-10-06T04:11:14.850284+0000","last_became_active":"2023-10-06T03:57:10.349114+0000","last_became_peered":"2023-10-06T03:57:10.349114+0000","last_unstale":"2023-10-06T04:11:14.850284+0000","last_undegraded":"2023-10-06T04:11:14.850284+0000","last_fullsized":"2023-10-06T04:11:14.850284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'113","last_scrub_stamp":"2023-10-06T04:11:14.850170+0000","last_deep_scrub":"53'113","last_deep_scrub_stamp":"2023-10-06T04:11:14.850170+0000","last_clean_scrub_stamp":"2023-10-06T04:11:14.850170+0000","objects_scrubbed":8,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:59.463057+0000","scrub_duration":0.012734528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":19,"num_read_kb":18,"num_write":109,"num_write_kb":4188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":33482,"num_omap_keys":60,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"53'393","reported_seq":2501,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:07.012391+0000","last_change":"2023-10-06T04:11:07.012369+0000","last_active":"2023-10-06T04:11:07.012391+0000","last_peered":"2023-10-06T04:11:07.012391+0000","last_clean":"2023-10-06T04:11:07.012391+0000","last_became_active":"2023-10-06T03:57:11.367257+0000","last_became_peered":"2023-10-06T03:57:11.367257+0000","last_unstale":"2023-10-06T04:11:07.012391+0000","last_undegraded":"2023-10-06T04:11:07.012391+0000","last_fullsized":"2023-10-06T04:11:07.012391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'393","last_scrub_stamp":"2023-10-06T04:11:07.012293+0000","last_deep_scrub":"53'393","last_deep_scrub_stamp":"2023-10-06T04:11:07.012293+0000","last_clean_scrub_stamp":"2023-10-06T04:11:07.012293+0000","objects_scrubbed":142,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:54.463356+0000","scrub_duration":0.119522168,"objects_trimmed":0,"snaptrim_duration":0.002803224,"stat_sum":{"num_bytes":18486267,"num_objects":142,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":26,"num_read_kb":4106,"num_write":272,"num_write_kb":22293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"53'56","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:02.635387+0000","last_change":"2023-10-06T04:11:02.635352+0000","last_active":"2023-10-06T04:11:02.635387+0000","last_peered":"2023-10-06T04:11:02.635387+0000","last_clean":"2023-10-06T04:11:02.635387+0000","last_became_active":"2023-10-06T03:57:10.347433+0000","last_became_peered":"2023-10-06T03:57:10.347433+0000","last_unstale":"2023-10-06T04:11:02.635387+0000","last_undegraded":"2023-10-06T04:11:02.635387+0000","last_fullsized":"2023-10-06T04:11:02.635387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'56","last_scrub_stamp":"2023-10-06T04:11:02.635240+0000","last_deep_scrub":"53'56","last_deep_scrub_stamp":"2023-10-06T04:11:02.635240+0000","last_clean_scrub_stamp":"2023-10-06T04:11:02.635240+0000","objects_scrubbed":4,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:08.757110+0000","scrub_duration":0.010252476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":11,"num_read_kb":8201,"num_write":54,"num_write_kb":4137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17420,"num_omap_keys":31,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"53'493","reported_seq":2611,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:23.921966+0000","last_change":"2023-10-06T04:11:23.921926+0000","last_active":"2023-10-06T04:11:23.921966+0000","last_peered":"2023-10-06T04:11:23.921966+0000","last_clean":"2023-10-06T04:11:23.921966+0000","last_became_active":"2023-10-06T03:57:11.369825+0000","last_became_peered":"2023-10-06T03:57:11.369825+0000","last_unstale":"2023-10-06T04:11:23.921966+0000","last_undegraded":"2023-10-06T04:11:23.921966+0000","last_fullsized":"2023-10-06T04:11:23.921966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'493","last_scrub_stamp":"2023-10-06T04:11:23.921850+0000","last_deep_scrub":"53'493","last_deep_scrub_stamp":"2023-10-06T04:11:23.921850+0000","last_clean_scrub_stamp":"2023-10-06T04:11:23.921850+0000","objects_scrubbed":159,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:01:43.526753+0000","scrub_duration":0.099932071999999997,"objects_trimmed":0,"snaptrim_duration":0.0014204689999999999,"stat_sum":{"num_bytes":11391239,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":35,"num_read_kb":8210,"num_write":309,"num_write_kb":15384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:09.870140+0000","last_change":"2023-10-06T04:11:09.870107+0000","last_active":"2023-10-06T04:11:09.870140+0000","last_peered":"2023-10-06T04:11:09.870140+0000","last_clean":"2023-10-06T04:11:09.870140+0000","last_became_active":"2023-10-06T03:57:05.663046+0000","last_became_peered":"2023-10-06T03:57:05.663046+0000","last_unstale":"2023-10-06T04:11:09.870140+0000","last_undegraded":"2023-10-06T04:11:09.870140+0000","last_fullsized":"2023-10-06T04:11:09.870140+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:11:09.869990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:11:09.869990+0000","last_clean_scrub_stamp":"2023-10-06T04:11:09.869990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:07.282698+0000","scrub_duration":0.001426486,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"53'43","reported_seq":101,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:12.801160+0000","last_change":"2023-10-06T04:11:12.801137+0000","last_active":"2023-10-06T04:11:12.801160+0000","last_peered":"2023-10-06T04:11:12.801160+0000","last_clean":"2023-10-06T04:11:12.801160+0000","last_became_active":"2023-10-06T03:57:10.348608+0000","last_became_peered":"2023-10-06T03:57:10.348608+0000","last_unstale":"2023-10-06T04:11:12.801160+0000","last_undegraded":"2023-10-06T04:11:12.801160+0000","last_fullsized":"2023-10-06T04:11:12.801160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'43","last_scrub_stamp":"2023-10-06T04:11:12.801071+0000","last_deep_scrub":"53'43","last_deep_scrub_stamp":"2023-10-06T04:11:12.801071+0000","last_clean_scrub_stamp":"2023-10-06T04:11:12.801071+0000","objects_scrubbed":6,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:46.340363+0000","scrub_duration":0.010232031000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194416,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":39,"num_write_kb":4139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":21608,"num_omap_keys":37,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"53'449","reported_seq":2561,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:24.911859+0000","last_change":"2023-10-06T04:11:24.911837+0000","last_active":"2023-10-06T04:11:24.911859+0000","last_peered":"2023-10-06T04:11:24.911859+0000","last_clean":"2023-10-06T04:11:24.911859+0000","last_became_active":"2023-10-06T03:57:11.370847+0000","last_became_peered":"2023-10-06T03:57:11.370847+0000","last_unstale":"2023-10-06T04:11:24.911859+0000","last_undegraded":"2023-10-06T04:11:24.911859+0000","last_fullsized":"2023-10-06T04:11:24.911859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'449","last_scrub_stamp":"2023-10-06T04:11:24.911765+0000","last_deep_scrub":"53'449","last_deep_scrub_stamp":"2023-10-06T04:11:24.911765+0000","last_clean_scrub_stamp":"2023-10-06T04:11:24.911765+0000","objects_scrubbed":139,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:18:37.604695+0000","scrub_duration":0.10052712699999999,"objects_trimmed":0,"snaptrim_duration":0.0013685990000000001,"stat_sum":{"num_bytes":14926709,"num_objects":139,"num_object_clones":0,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":0,"num_read":29,"num_read_kb":12301,"num_write":289,"num_write_kb":18822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:02.948635+0000","last_change":"2023-10-06T04:11:02.948611+0000","last_active":"2023-10-06T04:11:02.948635+0000","last_peered":"2023-10-06T04:11:02.948635+0000","last_clean":"2023-10-06T04:11:02.948635+0000","last_became_active":"2023-10-06T03:57:05.318572+0000","last_became_peered":"2023-10-06T03:57:05.318572+0000","last_unstale":"2023-10-06T04:11:02.948635+0000","last_undegraded":"2023-10-06T04:11:02.948635+0000","last_fullsized":"2023-10-06T04:11:02.948635+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:11:02.948529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:11:02.948529+0000","last_clean_scrub_stamp":"2023-10-06T04:11:02.948529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:16.383610+0000","scrub_duration":0.0014570340000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"53'16","reported_seq":77,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:15.846630+0000","last_change":"2023-10-06T04:11:15.846599+0000","last_active":"2023-10-06T04:11:15.846630+0000","last_peered":"2023-10-06T04:11:15.846630+0000","last_clean":"2023-10-06T04:11:15.846630+0000","last_became_active":"2023-10-06T03:57:10.348271+0000","last_became_peered":"2023-10-06T03:57:10.348271+0000","last_unstale":"2023-10-06T04:11:15.846630+0000","last_undegraded":"2023-10-06T04:11:15.846630+0000","last_fullsized":"2023-10-06T04:11:15.846630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2023-10-06T04:11:15.846508+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2023-10-06T04:11:15.846508+0000","last_clean_scrub_stamp":"2023-10-06T04:11:15.846508+0000","objects_scrubbed":6,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:43.610039+0000","scrub_duration":0.011518349000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":26,"num_write":12,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17144,"num_omap_keys":31,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"53'511","reported_seq":2628,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:26.853504+0000","last_change":"2023-10-06T04:11:26.853469+0000","last_active":"2023-10-06T04:11:26.853504+0000","last_peered":"2023-10-06T04:11:26.853504+0000","last_clean":"2023-10-06T04:11:26.853504+0000","last_became_active":"2023-10-06T03:57:11.365006+0000","last_became_peered":"2023-10-06T03:57:11.365006+0000","last_unstale":"2023-10-06T04:11:26.853504+0000","last_undegraded":"2023-10-06T04:11:26.853504+0000","last_fullsized":"2023-10-06T04:11:26.853504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'511","last_scrub_stamp":"2023-10-06T04:11:26.853393+0000","last_deep_scrub":"53'511","last_deep_scrub_stamp":"2023-10-06T04:11:26.853393+0000","last_clean_scrub_stamp":"2023-10-06T04:11:26.853393+0000","objects_scrubbed":162,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:26:41.224552+0000","scrub_duration":0.103410242,"objects_trimmed":0,"snaptrim_duration":0.0015961910000000001,"stat_sum":{"num_bytes":5739830,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":33,"num_read_kb":4109,"num_write":322,"num_write_kb":5775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:06.413155+0000","last_change":"2023-10-06T04:11:06.413102+0000","last_active":"2023-10-06T04:11:06.413155+0000","last_peered":"2023-10-06T04:11:06.413155+0000","last_clean":"2023-10-06T04:11:06.413155+0000","last_became_active":"2023-10-06T03:57:05.314408+0000","last_became_peered":"2023-10-06T03:57:05.314408+0000","last_unstale":"2023-10-06T04:11:06.413155+0000","last_undegraded":"2023-10-06T04:11:06.413155+0000","last_fullsized":"2023-10-06T04:11:06.413155+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:11:06.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:11:06.412975+0000","last_clean_scrub_stamp":"2023-10-06T04:11:06.412975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:42:44.133543+0000","scrub_duration":0.0018344570000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"53'365","reported_seq":422,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:11.809107+0000","last_change":"2023-10-06T04:11:11.809083+0000","last_active":"2023-10-06T04:11:11.809107+0000","last_peered":"2023-10-06T04:11:11.809107+0000","last_clean":"2023-10-06T04:11:11.809107+0000","last_became_active":"2023-10-06T03:57:10.345769+0000","last_became_peered":"2023-10-06T03:57:10.345769+0000","last_unstale":"2023-10-06T04:11:11.809107+0000","last_undegraded":"2023-10-06T04:11:11.809107+0000","last_fullsized":"2023-10-06T04:11:11.809107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'365","last_scrub_stamp":"2023-10-06T04:11:11.809003+0000","last_deep_scrub":"53'365","last_deep_scrub_stamp":"2023-10-06T04:11:11.809003+0000","last_clean_scrub_stamp":"2023-10-06T04:11:11.809003+0000","objects_scrubbed":6,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:05.207133+0000","scrub_duration":0.020844386999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":352,"num_write_kb":12528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":20402,"num_omap_keys":35,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"53'477","reported_seq":2599,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:29.906931+0000","last_change":"2023-10-06T04:11:29.906907+0000","last_active":"2023-10-06T04:11:29.906931+0000","last_peered":"2023-10-06T04:11:29.906931+0000","last_clean":"2023-10-06T04:11:29.906931+0000","last_became_active":"2023-10-06T03:57:11.370076+0000","last_became_peered":"2023-10-06T03:57:11.370076+0000","last_unstale":"2023-10-06T04:11:29.906931+0000","last_undegraded":"2023-10-06T04:11:29.906931+0000","last_fullsized":"2023-10-06T04:11:29.906931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'477","last_scrub_stamp":"2023-10-06T04:11:29.906836+0000","last_deep_scrub":"53'477","last_deep_scrub_stamp":"2023-10-06T04:11:29.906836+0000","last_clean_scrub_stamp":"2023-10-06T04:11:29.906836+0000","objects_scrubbed":146,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:08:33.902528+0000","scrub_duration":0.12060051500000001,"objects_trimmed":0,"snaptrim_duration":0.0011772849999999999,"stat_sum":{"num_bytes":18548829,"num_objects":146,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":146,"num_whiteouts":0,"num_read":39,"num_read_kb":4112,"num_write":310,"num_write_kb":22369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:08.939772+0000","last_change":"2023-10-06T04:11:08.939745+0000","last_active":"2023-10-06T04:11:08.939772+0000","last_peered":"2023-10-06T04:11:08.939772+0000","last_clean":"2023-10-06T04:11:08.939772+0000","last_became_active":"2023-10-06T03:57:05.318456+0000","last_became_peered":"2023-10-06T03:57:05.318456+0000","last_unstale":"2023-10-06T04:11:08.939772+0000","last_undegraded":"2023-10-06T04:11:08.939772+0000","last_fullsized":"2023-10-06T04:11:08.939772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:11:08.939596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:11:08.939596+0000","last_clean_scrub_stamp":"2023-10-06T04:11:08.939596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:34.690483+0000","scrub_duration":0.0022435490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"53'119","reported_seq":208,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:03.558037+0000","last_change":"2023-10-06T04:11:03.558012+0000","last_active":"2023-10-06T04:11:03.558037+0000","last_peered":"2023-10-06T04:11:03.558037+0000","last_clean":"2023-10-06T04:11:03.558037+0000","last_became_active":"2023-10-06T03:57:05.663426+0000","last_became_peered":"2023-10-06T03:57:05.663426+0000","last_unstale":"2023-10-06T04:11:03.558037+0000","last_undegraded":"2023-10-06T04:11:03.558037+0000","last_fullsized":"2023-10-06T04:11:03.558037+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'119","last_scrub_stamp":"2023-10-06T04:11:03.557930+0000","last_deep_scrub":"53'119","last_deep_scrub_stamp":"2023-10-06T04:11:03.557930+0000","last_clean_scrub_stamp":"2023-10-06T04:11:03.557930+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:55:23.076482+0000","scrub_duration":0.0036399879999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"53'724","reported_seq":786,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:03.652644+0000","last_change":"2023-10-06T04:11:03.652622+0000","last_active":"2023-10-06T04:11:03.652644+0000","last_peered":"2023-10-06T04:11:03.652644+0000","last_clean":"2023-10-06T04:11:03.652644+0000","last_became_active":"2023-10-06T03:57:10.348879+0000","last_became_peered":"2023-10-06T03:57:10.348879+0000","last_unstale":"2023-10-06T04:11:03.652644+0000","last_undegraded":"2023-10-06T04:11:03.652644+0000","last_fullsized":"2023-10-06T04:11:03.652644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'724","last_scrub_stamp":"2023-10-06T04:11:03.652510+0000","last_deep_scrub":"53'724","last_deep_scrub_stamp":"2023-10-06T04:11:03.652510+0000","last_clean_scrub_stamp":"2023-10-06T04:11:03.652510+0000","objects_scrubbed":5,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:03.918796+0000","scrub_duration":0.024355216999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9487985,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":20487,"num_write":715,"num_write_kb":13783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9328,"num_omap_keys":17,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"53'450","reported_seq":2570,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:15.162025+0000","last_change":"2023-10-06T04:11:15.162001+0000","last_active":"2023-10-06T04:11:15.162025+0000","last_peered":"2023-10-06T04:11:15.162025+0000","last_clean":"2023-10-06T04:11:15.162025+0000","last_became_active":"2023-10-06T03:57:11.373452+0000","last_became_peered":"2023-10-06T03:57:11.373452+0000","last_unstale":"2023-10-06T04:11:15.162025+0000","last_undegraded":"2023-10-06T04:11:15.162025+0000","last_fullsized":"2023-10-06T04:11:15.162025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'450","last_scrub_stamp":"2023-10-06T04:11:15.161925+0000","last_deep_scrub":"53'450","last_deep_scrub_stamp":"2023-10-06T04:11:15.161925+0000","last_clean_scrub_stamp":"2023-10-06T04:11:15.161925+0000","objects_scrubbed":165,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:13.301939+0000","scrub_duration":0.119156151,"objects_trimmed":0,"snaptrim_duration":0.0018153620000000001,"stat_sum":{"num_bytes":5937326,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":36,"num_read_kb":2733,"num_write":331,"num_write_kb":10073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:07.651843+0000","last_change":"2023-10-06T04:11:07.651815+0000","last_active":"2023-10-06T04:11:07.651843+0000","last_peered":"2023-10-06T04:11:07.651843+0000","last_clean":"2023-10-06T04:11:07.651843+0000","last_became_active":"2023-10-06T03:57:05.318531+0000","last_became_peered":"2023-10-06T03:57:05.318531+0000","last_unstale":"2023-10-06T04:11:07.651843+0000","last_undegraded":"2023-10-06T04:11:07.651843+0000","last_fullsized":"2023-10-06T04:11:07.651843+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:11:07.651712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:11:07.651712+0000","last_clean_scrub_stamp":"2023-10-06T04:11:07.651712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:16.562485+0000","scrub_duration":0.0020165349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"53'545","reported_seq":719,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:16.314725+0000","last_change":"2023-10-06T04:11:16.314684+0000","last_active":"2023-10-06T04:11:16.314725+0000","last_peered":"2023-10-06T04:11:16.314725+0000","last_clean":"2023-10-06T04:11:16.314725+0000","last_became_active":"2023-10-06T03:57:10.343051+0000","last_became_peered":"2023-10-06T03:57:10.343051+0000","last_unstale":"2023-10-06T04:11:16.314725+0000","last_undegraded":"2023-10-06T04:11:16.314725+0000","last_fullsized":"2023-10-06T04:11:16.314725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'545","last_scrub_stamp":"2023-10-06T04:11:16.314572+0000","last_deep_scrub":"53'545","last_deep_scrub_stamp":"2023-10-06T04:11:16.314572+0000","last_clean_scrub_stamp":"2023-10-06T04:11:16.314572+0000","objects_scrubbed":5,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:43.647007+0000","scrub_duration":0.014138608,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4571457,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":122,"num_read_kb":3989,"num_write":534,"num_write_kb":4895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"53'513","reported_seq":2634,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:22.415858+0000","last_change":"2023-10-06T04:11:22.415833+0000","last_active":"2023-10-06T04:11:22.415858+0000","last_peered":"2023-10-06T04:11:22.415858+0000","last_clean":"2023-10-06T04:11:22.415858+0000","last_became_active":"2023-10-06T03:57:11.373264+0000","last_became_peered":"2023-10-06T03:57:11.373264+0000","last_unstale":"2023-10-06T04:11:22.415858+0000","last_undegraded":"2023-10-06T04:11:22.415858+0000","last_fullsized":"2023-10-06T04:11:22.415858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'513","last_scrub_stamp":"2023-10-06T04:11:22.415754+0000","last_deep_scrub":"53'513","last_deep_scrub_stamp":"2023-10-06T04:11:22.415754+0000","last_clean_scrub_stamp":"2023-10-06T04:11:22.415754+0000","objects_scrubbed":168,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:30:17.810664+0000","scrub_duration":0.11956710500000001,"objects_trimmed":0,"snaptrim_duration":0.0026908990000000001,"stat_sum":{"num_bytes":10658865,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":37,"num_read_kb":4121,"num_write":334,"num_write_kb":10588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:04.597831+0000","last_change":"2023-10-06T04:11:04.597785+0000","last_active":"2023-10-06T04:11:04.597831+0000","last_peered":"2023-10-06T04:11:04.597831+0000","last_clean":"2023-10-06T04:11:04.597831+0000","last_became_active":"2023-10-06T03:57:05.663501+0000","last_became_peered":"2023-10-06T03:57:05.663501+0000","last_unstale":"2023-10-06T04:11:04.597831+0000","last_undegraded":"2023-10-06T04:11:04.597831+0000","last_fullsized":"2023-10-06T04:11:04.597831+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:11:04.597657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:11:04.597657+0000","last_clean_scrub_stamp":"2023-10-06T04:11:04.597657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:09.111900+0000","scrub_duration":0.0020988000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"53'7","reported_seq":60,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:13.746940+0000","last_change":"2023-10-06T04:11:13.746917+0000","last_active":"2023-10-06T04:11:13.746940+0000","last_peered":"2023-10-06T04:11:13.746940+0000","last_clean":"2023-10-06T04:11:13.746940+0000","last_became_active":"2023-10-06T03:57:10.345743+0000","last_became_peered":"2023-10-06T03:57:10.345743+0000","last_unstale":"2023-10-06T04:11:13.746940+0000","last_undegraded":"2023-10-06T04:11:13.746940+0000","last_fullsized":"2023-10-06T04:11:13.746940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7","last_scrub_stamp":"2023-10-06T04:11:13.746842+0000","last_deep_scrub":"53'7","last_deep_scrub_stamp":"2023-10-06T04:11:13.746842+0000","last_clean_scrub_stamp":"2023-10-06T04:11:13.746842+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:47:15.026107+0000","scrub_duration":0.0026376030000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20578,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"53'511","reported_seq":2622,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:19.489280+0000","last_change":"2023-10-06T04:11:19.489257+0000","last_active":"2023-10-06T04:11:19.489280+0000","last_peered":"2023-10-06T04:11:19.489280+0000","last_clean":"2023-10-06T04:11:19.489280+0000","last_became_active":"2023-10-06T03:57:11.373543+0000","last_became_peered":"2023-10-06T03:57:11.373543+0000","last_unstale":"2023-10-06T04:11:19.489280+0000","last_undegraded":"2023-10-06T04:11:19.489280+0000","last_fullsized":"2023-10-06T04:11:19.489280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'511","last_scrub_stamp":"2023-10-06T04:11:19.489194+0000","last_deep_scrub":"53'511","last_deep_scrub_stamp":"2023-10-06T04:11:19.489194+0000","last_clean_scrub_stamp":"2023-10-06T04:11:19.489194+0000","objects_scrubbed":155,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:28.970043+0000","scrub_duration":0.13727003800000001,"objects_trimmed":0,"snaptrim_duration":0.0020558070000000002,"stat_sum":{"num_bytes":6540215,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":27,"num_read_kb":14,"num_write":340,"num_write_kb":6519,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"53'18","reported_seq":78,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:04.624957+0000","last_change":"2023-10-06T04:11:04.624929+0000","last_active":"2023-10-06T04:11:04.624957+0000","last_peered":"2023-10-06T04:11:04.624957+0000","last_clean":"2023-10-06T04:11:04.624957+0000","last_became_active":"2023-10-06T03:57:10.344324+0000","last_became_peered":"2023-10-06T03:57:10.344324+0000","last_unstale":"2023-10-06T04:11:04.624957+0000","last_undegraded":"2023-10-06T04:11:04.624957+0000","last_fullsized":"2023-10-06T04:11:04.624957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'18","last_scrub_stamp":"2023-10-06T04:11:04.624797+0000","last_deep_scrub":"53'18","last_deep_scrub_stamp":"2023-10-06T04:11:04.624797+0000","last_clean_scrub_stamp":"2023-10-06T04:11:04.624797+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:02.674217+0000","scrub_duration":0.0092723449999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":22,"num_read_kb":21,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:01.951088+0000","last_change":"2023-10-06T04:11:01.951066+0000","last_active":"2023-10-06T04:11:01.951088+0000","last_peered":"2023-10-06T04:11:01.951088+0000","last_clean":"2023-10-06T04:11:01.951088+0000","last_became_active":"2023-10-06T03:57:05.317715+0000","last_became_peered":"2023-10-06T03:57:05.317715+0000","last_unstale":"2023-10-06T04:11:01.951088+0000","last_undegraded":"2023-10-06T04:11:01.951088+0000","last_fullsized":"2023-10-06T04:11:01.951088+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:11:01.950987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:11:01.950987+0000","last_clean_scrub_stamp":"2023-10-06T04:11:01.950987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:51.601489+0000","scrub_duration":0.0016264490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"53'501","reported_seq":2617,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:26.514674+0000","last_change":"2023-10-06T04:11:26.514649+0000","last_active":"2023-10-06T04:11:26.514674+0000","last_peered":"2023-10-06T04:11:26.514674+0000","last_clean":"2023-10-06T04:11:26.514674+0000","last_became_active":"2023-10-06T03:57:11.369445+0000","last_became_peered":"2023-10-06T03:57:11.369445+0000","last_unstale":"2023-10-06T04:11:26.514674+0000","last_undegraded":"2023-10-06T04:11:26.514674+0000","last_fullsized":"2023-10-06T04:11:26.514674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'501","last_scrub_stamp":"2023-10-06T04:11:26.514580+0000","last_deep_scrub":"53'501","last_deep_scrub_stamp":"2023-10-06T04:11:26.514580+0000","last_clean_scrub_stamp":"2023-10-06T04:11:26.514580+0000","objects_scrubbed":170,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:54:27.080939+0000","scrub_duration":0.121460004,"objects_trimmed":0,"snaptrim_duration":0.0015173949999999999,"stat_sum":{"num_bytes":11743957,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":32,"num_read_kb":17,"num_write":311,"num_write_kb":11603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:02.976078+0000","last_change":"2023-10-06T04:11:02.976046+0000","last_active":"2023-10-06T04:11:02.976078+0000","last_peered":"2023-10-06T04:11:02.976078+0000","last_clean":"2023-10-06T04:11:02.976078+0000","last_became_active":"2023-10-06T03:57:05.320583+0000","last_became_peered":"2023-10-06T03:57:05.320583+0000","last_unstale":"2023-10-06T04:11:02.976078+0000","last_undegraded":"2023-10-06T04:11:02.976078+0000","last_fullsized":"2023-10-06T04:11:02.976078+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:11:02.975967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:11:02.975967+0000","last_clean_scrub_stamp":"2023-10-06T04:11:02.975967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:09.044661+0000","scrub_duration":0.001182069,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"53'96","reported_seq":151,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:18.309985+0000","last_change":"2023-10-06T04:11:18.309956+0000","last_active":"2023-10-06T04:11:18.309985+0000","last_peered":"2023-10-06T04:11:18.309985+0000","last_clean":"2023-10-06T04:11:18.309985+0000","last_became_active":"2023-10-06T03:57:10.339424+0000","last_became_peered":"2023-10-06T03:57:10.339424+0000","last_unstale":"2023-10-06T04:11:18.309985+0000","last_undegraded":"2023-10-06T04:11:18.309985+0000","last_fullsized":"2023-10-06T04:11:18.309985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'96","last_scrub_stamp":"2023-10-06T04:11:18.309874+0000","last_deep_scrub":"53'96","last_deep_scrub_stamp":"2023-10-06T04:11:18.309874+0000","last_clean_scrub_stamp":"2023-10-06T04:11:18.309874+0000","objects_scrubbed":4,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:47.802237+0000","scrub_duration":0.006477053,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":19,"num_write":221,"num_write_kb":1011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"53'495","reported_seq":2607,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:31.043852+0000","last_change":"2023-10-06T04:11:31.043829+0000","last_active":"2023-10-06T04:11:31.043852+0000","last_peered":"2023-10-06T04:11:31.043852+0000","last_clean":"2023-10-06T04:11:31.043852+0000","last_became_active":"2023-10-06T03:57:11.372215+0000","last_became_peered":"2023-10-06T03:57:11.372215+0000","last_unstale":"2023-10-06T04:11:31.043852+0000","last_undegraded":"2023-10-06T04:11:31.043852+0000","last_fullsized":"2023-10-06T04:11:31.043852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'495","last_scrub_stamp":"2023-10-06T04:11:31.043759+0000","last_deep_scrub":"53'495","last_deep_scrub_stamp":"2023-10-06T04:11:31.043759+0000","last_clean_scrub_stamp":"2023-10-06T04:11:31.043759+0000","objects_scrubbed":160,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:24.464374+0000","scrub_duration":0.134907318,"objects_trimmed":0,"snaptrim_duration":0.0020199100000000002,"stat_sum":{"num_bytes":14603524,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":28,"num_read_kb":4106,"num_write":312,"num_write_kb":14355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"53'211","reported_seq":275,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:14.290761+0000","last_change":"2023-10-06T04:11:14.290711+0000","last_active":"2023-10-06T04:11:14.290761+0000","last_peered":"2023-10-06T04:11:14.290761+0000","last_clean":"2023-10-06T04:11:14.290761+0000","last_became_active":"2023-10-06T03:57:10.343865+0000","last_became_peered":"2023-10-06T03:57:10.343865+0000","last_unstale":"2023-10-06T04:11:14.290761+0000","last_undegraded":"2023-10-06T04:11:14.290761+0000","last_fullsized":"2023-10-06T04:11:14.290761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'211","last_scrub_stamp":"2023-10-06T04:11:14.290584+0000","last_deep_scrub":"53'211","last_deep_scrub_stamp":"2023-10-06T04:11:14.290584+0000","last_clean_scrub_stamp":"2023-10-06T04:11:14.290584+0000","objects_scrubbed":9,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:30.042251+0000","scrub_duration":0.028391895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":11,"num_read_kb":32771,"num_write":206,"num_write_kb":12488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":51091,"num_omap_keys":90,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"53'505","reported_seq":2622,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:20.497794+0000","last_change":"2023-10-06T04:11:20.497772+0000","last_active":"2023-10-06T04:11:20.497794+0000","last_peered":"2023-10-06T04:11:20.497794+0000","last_clean":"2023-10-06T04:11:20.497794+0000","last_became_active":"2023-10-06T03:57:11.369681+0000","last_became_peered":"2023-10-06T03:57:11.369681+0000","last_unstale":"2023-10-06T04:11:20.497794+0000","last_undegraded":"2023-10-06T04:11:20.497794+0000","last_fullsized":"2023-10-06T04:11:20.497794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'505","last_scrub_stamp":"2023-10-06T04:11:20.497703+0000","last_deep_scrub":"53'505","last_deep_scrub_stamp":"2023-10-06T04:11:20.497703+0000","last_clean_scrub_stamp":"2023-10-06T04:11:20.497703+0000","objects_scrubbed":158,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:42.201624+0000","scrub_duration":0.099109163,"objects_trimmed":0,"snaptrim_duration":0.0013151409999999999,"stat_sum":{"num_bytes":9655954,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":33,"num_read_kb":14,"num_write":334,"num_write_kb":13707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"53'104","reported_seq":164,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:17.710929+0000","last_change":"2023-10-06T04:11:17.710901+0000","last_active":"2023-10-06T04:11:17.710929+0000","last_peered":"2023-10-06T04:11:17.710929+0000","last_clean":"2023-10-06T04:11:17.710929+0000","last_became_active":"2023-10-06T03:57:10.349320+0000","last_became_peered":"2023-10-06T03:57:10.349320+0000","last_unstale":"2023-10-06T04:11:17.710929+0000","last_undegraded":"2023-10-06T04:11:17.710929+0000","last_fullsized":"2023-10-06T04:11:17.710929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'104","last_scrub_stamp":"2023-10-06T04:11:17.710816+0000","last_deep_scrub":"53'104","last_deep_scrub_stamp":"2023-10-06T04:11:17.710816+0000","last_clean_scrub_stamp":"2023-10-06T04:11:17.710816+0000","objects_scrubbed":4,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:46.926120+0000","scrub_duration":0.01386691,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194858,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":4110,"num_write":99,"num_write_kb":4143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"53'441","reported_seq":2552,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:10.921994+0000","last_change":"2023-10-06T04:11:10.921969+0000","last_active":"2023-10-06T04:11:10.921994+0000","last_peered":"2023-10-06T04:11:10.921994+0000","last_clean":"2023-10-06T04:11:10.921994+0000","last_became_active":"2023-10-06T03:57:11.370229+0000","last_became_peered":"2023-10-06T03:57:11.370229+0000","last_unstale":"2023-10-06T04:11:10.921994+0000","last_undegraded":"2023-10-06T04:11:10.921994+0000","last_fullsized":"2023-10-06T04:11:10.921994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'441","last_scrub_stamp":"2023-10-06T04:11:10.921900+0000","last_deep_scrub":"53'441","last_deep_scrub_stamp":"2023-10-06T04:11:10.921900+0000","last_clean_scrub_stamp":"2023-10-06T04:11:10.921900+0000","objects_scrubbed":155,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:16.456865+0000","scrub_duration":0.100086199,"objects_trimmed":0,"snaptrim_duration":0.0027851170000000001,"stat_sum":{"num_bytes":11865278,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":27,"num_read_kb":17,"num_write":315,"num_write_kb":11749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"53'250","reported_seq":322,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:16.743648+0000","last_change":"2023-10-06T04:11:16.743623+0000","last_active":"2023-10-06T04:11:16.743648+0000","last_peered":"2023-10-06T04:11:16.743648+0000","last_clean":"2023-10-06T04:11:16.743648+0000","last_became_active":"2023-10-06T03:57:10.350083+0000","last_became_peered":"2023-10-06T03:57:10.350083+0000","last_unstale":"2023-10-06T04:11:16.743648+0000","last_undegraded":"2023-10-06T04:11:16.743648+0000","last_fullsized":"2023-10-06T04:11:16.743648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'250","last_scrub_stamp":"2023-10-06T04:11:16.743549+0000","last_deep_scrub":"53'250","last_deep_scrub_stamp":"2023-10-06T04:11:16.743549+0000","last_clean_scrub_stamp":"2023-10-06T04:11:16.743549+0000","objects_scrubbed":12,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:57.567358+0000","scrub_duration":0.028247550999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":36,"num_read_kb":45128,"num_write":264,"num_write_kb":12512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19684,"num_omap_keys":36,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"53'498","reported_seq":2611,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:33.054545+0000","last_change":"2023-10-06T04:11:33.054522+0000","last_active":"2023-10-06T04:11:33.054545+0000","last_peered":"2023-10-06T04:11:33.054545+0000","last_clean":"2023-10-06T04:11:33.054545+0000","last_became_active":"2023-10-06T03:57:11.369123+0000","last_became_peered":"2023-10-06T03:57:11.369123+0000","last_unstale":"2023-10-06T04:11:33.054545+0000","last_undegraded":"2023-10-06T04:11:33.054545+0000","last_fullsized":"2023-10-06T04:11:33.054545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'498","last_scrub_stamp":"2023-10-06T04:11:33.054447+0000","last_deep_scrub":"53'498","last_deep_scrub_stamp":"2023-10-06T04:11:33.054447+0000","last_clean_scrub_stamp":"2023-10-06T04:11:33.054447+0000","objects_scrubbed":148,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:40.005920+0000","scrub_duration":0.12269728000000001,"objects_trimmed":0,"snaptrim_duration":0.0028613789999999998,"stat_sum":{"num_bytes":15066743,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":30,"num_read_kb":4112,"num_write":322,"num_write_kb":14878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"53'183","reported_seq":238,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:06.608805+0000","last_change":"2023-10-06T04:11:06.608778+0000","last_active":"2023-10-06T04:11:06.608805+0000","last_peered":"2023-10-06T04:11:06.608805+0000","last_clean":"2023-10-06T04:11:06.608805+0000","last_became_active":"2023-10-06T03:57:10.344583+0000","last_became_peered":"2023-10-06T03:57:10.344583+0000","last_unstale":"2023-10-06T04:11:06.608805+0000","last_undegraded":"2023-10-06T04:11:06.608805+0000","last_fullsized":"2023-10-06T04:11:06.608805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'183","last_scrub_stamp":"2023-10-06T04:11:06.608691+0000","last_deep_scrub":"53'183","last_deep_scrub_stamp":"2023-10-06T04:11:06.608691+0000","last_clean_scrub_stamp":"2023-10-06T04:11:06.608691+0000","objects_scrubbed":2,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:46:02.740119+0000","scrub_duration":0.010859541,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":177,"num_write_kb":8286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8760,"num_omap_keys":15,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"53'516","reported_seq":2637,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:27.453717+0000","last_change":"2023-10-06T04:11:27.453693+0000","last_active":"2023-10-06T04:11:27.453717+0000","last_peered":"2023-10-06T04:11:27.453717+0000","last_clean":"2023-10-06T04:11:27.453717+0000","last_became_active":"2023-10-06T03:57:11.366500+0000","last_became_peered":"2023-10-06T03:57:11.366500+0000","last_unstale":"2023-10-06T04:11:27.453717+0000","last_undegraded":"2023-10-06T04:11:27.453717+0000","last_fullsized":"2023-10-06T04:11:27.453717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'516","last_scrub_stamp":"2023-10-06T04:11:27.453623+0000","last_deep_scrub":"53'516","last_deep_scrub_stamp":"2023-10-06T04:11:27.453623+0000","last_clean_scrub_stamp":"2023-10-06T04:11:27.453623+0000","objects_scrubbed":164,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:46.741288+0000","scrub_duration":0.102849519,"objects_trimmed":0,"snaptrim_duration":0.0016878310000000001,"stat_sum":{"num_bytes":6166988,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":38,"num_read_kb":4113,"num_write":326,"num_write_kb":10290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"53'178","reported_seq":239,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:09.230941+0000","last_change":"2023-10-06T04:11:09.230916+0000","last_active":"2023-10-06T04:11:09.230941+0000","last_peered":"2023-10-06T04:11:09.230941+0000","last_clean":"2023-10-06T04:11:09.230941+0000","last_became_active":"2023-10-06T03:57:10.347897+0000","last_became_peered":"2023-10-06T03:57:10.347897+0000","last_unstale":"2023-10-06T04:11:09.230941+0000","last_undegraded":"2023-10-06T04:11:09.230941+0000","last_fullsized":"2023-10-06T04:11:09.230941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'178","last_scrub_stamp":"2023-10-06T04:11:09.230838+0000","last_deep_scrub":"53'178","last_deep_scrub_stamp":"2023-10-06T04:11:09.230838+0000","last_clean_scrub_stamp":"2023-10-06T04:11:09.230838+0000","objects_scrubbed":8,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:21.538557+0000","scrub_duration":0.018787900999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7191702,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":12,"num_read_kb":8202,"num_write":168,"num_write_kb":7111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"53'586","reported_seq":2705,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:29.079404+0000","last_change":"2023-10-06T04:11:29.079377+0000","last_active":"2023-10-06T04:11:29.079404+0000","last_peered":"2023-10-06T04:11:29.079404+0000","last_clean":"2023-10-06T04:11:29.079404+0000","last_became_active":"2023-10-06T03:57:11.364150+0000","last_became_peered":"2023-10-06T03:57:11.364150+0000","last_unstale":"2023-10-06T04:11:29.079404+0000","last_undegraded":"2023-10-06T04:11:29.079404+0000","last_fullsized":"2023-10-06T04:11:29.079404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'586","last_scrub_stamp":"2023-10-06T04:11:29.079310+0000","last_deep_scrub":"53'586","last_deep_scrub_stamp":"2023-10-06T04:11:29.079310+0000","last_clean_scrub_stamp":"2023-10-06T04:11:29.079310+0000","objects_scrubbed":176,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:35.809878+0000","scrub_duration":0.15600001299999999,"objects_trimmed":0,"snaptrim_duration":0.001670408,"stat_sum":{"num_bytes":10944544,"num_objects":176,"num_object_clones":0,"num_object_copies":352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":176,"num_whiteouts":0,"num_read":35,"num_read_kb":7178,"num_write":386,"num_write_kb":19084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"53'466","reported_seq":2604,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:11.117345+0000","last_change":"2023-10-06T04:11:11.117321+0000","last_active":"2023-10-06T04:11:11.117345+0000","last_peered":"2023-10-06T04:11:11.117345+0000","last_clean":"2023-10-06T04:11:11.117345+0000","last_became_active":"2023-10-06T03:57:11.366524+0000","last_became_peered":"2023-10-06T03:57:11.366524+0000","last_unstale":"2023-10-06T04:11:11.117345+0000","last_undegraded":"2023-10-06T04:11:11.117345+0000","last_fullsized":"2023-10-06T04:11:11.117345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'466","last_scrub_stamp":"2023-10-06T04:11:11.117253+0000","last_deep_scrub":"53'466","last_deep_scrub_stamp":"2023-10-06T04:11:11.117253+0000","last_clean_scrub_stamp":"2023-10-06T04:11:11.117253+0000","objects_scrubbed":151,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:09.926514+0000","scrub_duration":0.109297006,"objects_trimmed":0,"snaptrim_duration":0.0017816290000000001,"stat_sum":{"num_bytes":22594948,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":54,"num_read_kb":16279,"num_write":343,"num_write_kb":22243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"53'540","reported_seq":2661,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:29.427696+0000","last_change":"2023-10-06T04:11:29.427672+0000","last_active":"2023-10-06T04:11:29.427696+0000","last_peered":"2023-10-06T04:11:29.427696+0000","last_clean":"2023-10-06T04:11:29.427696+0000","last_became_active":"2023-10-06T03:57:11.370848+0000","last_became_peered":"2023-10-06T03:57:11.370848+0000","last_unstale":"2023-10-06T04:11:29.427696+0000","last_undegraded":"2023-10-06T04:11:29.427696+0000","last_fullsized":"2023-10-06T04:11:29.427696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'540","last_scrub_stamp":"2023-10-06T04:11:29.427597+0000","last_deep_scrub":"53'540","last_deep_scrub_stamp":"2023-10-06T04:11:29.427597+0000","last_clean_scrub_stamp":"2023-10-06T04:11:29.427597+0000","objects_scrubbed":160,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:00.100427+0000","scrub_duration":0.11843801499999999,"objects_trimmed":0,"snaptrim_duration":0.0017239289999999999,"stat_sum":{"num_bytes":12161208,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":37,"num_read_kb":8210,"num_write":347,"num_write_kb":16110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"53'554","reported_seq":2678,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:36.149409+0000","last_change":"2023-10-06T04:11:36.149381+0000","last_active":"2023-10-06T04:11:36.149409+0000","last_peered":"2023-10-06T04:11:36.149409+0000","last_clean":"2023-10-06T04:11:36.149409+0000","last_became_active":"2023-10-06T03:57:11.363875+0000","last_became_peered":"2023-10-06T03:57:11.363875+0000","last_unstale":"2023-10-06T04:11:36.149409+0000","last_undegraded":"2023-10-06T04:11:36.149409+0000","last_fullsized":"2023-10-06T04:11:36.149409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'554","last_scrub_stamp":"2023-10-06T04:11:36.149308+0000","last_deep_scrub":"53'554","last_deep_scrub_stamp":"2023-10-06T04:11:36.149308+0000","last_clean_scrub_stamp":"2023-10-06T04:11:36.149308+0000","objects_scrubbed":183,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:46:05.831956+0000","scrub_duration":0.167688535,"objects_trimmed":0,"snaptrim_duration":0.0028142470000000002,"stat_sum":{"num_bytes":14948732,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":39,"num_read_kb":4117,"num_write":348,"num_write_kb":14761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"53'505","reported_seq":2630,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:19.658777+0000","last_change":"2023-10-06T04:11:19.658754+0000","last_active":"2023-10-06T04:11:19.658777+0000","last_peered":"2023-10-06T04:11:19.658777+0000","last_clean":"2023-10-06T04:11:19.658777+0000","last_became_active":"2023-10-06T03:57:11.377126+0000","last_became_peered":"2023-10-06T03:57:11.377126+0000","last_unstale":"2023-10-06T04:11:19.658777+0000","last_undegraded":"2023-10-06T04:11:19.658777+0000","last_fullsized":"2023-10-06T04:11:19.658777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'505","last_scrub_stamp":"2023-10-06T04:11:19.658678+0000","last_deep_scrub":"53'505","last_deep_scrub_stamp":"2023-10-06T04:11:19.658678+0000","last_clean_scrub_stamp":"2023-10-06T04:11:19.658678+0000","objects_scrubbed":153,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:16.651951+0000","scrub_duration":0.108743853,"objects_trimmed":0,"snaptrim_duration":0.0023666569999999999,"stat_sum":{"num_bytes":6085663,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":41,"num_read_kb":18,"num_write":335,"num_write_kb":6110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"53'326","reported_seq":2424,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:10:55.124114+0000","last_change":"2023-10-06T04:05:02.972860+0000","last_active":"2023-10-06T04:10:55.124114+0000","last_peered":"2023-10-06T04:10:55.124114+0000","last_clean":"2023-10-06T04:10:55.124114+0000","last_became_active":"2023-10-06T03:57:59.256086+0000","last_became_peered":"2023-10-06T03:57:59.256086+0000","last_unstale":"2023-10-06T04:10:55.124114+0000","last_undegraded":"2023-10-06T04:10:55.124114+0000","last_fullsized":"2023-10-06T04:10:55.124114+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:57:10.325696+0000","last_clean_scrub_stamp":"2023-10-06T03:57:10.325696+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018573859999999999,"stat_sum":{"num_bytes":11350301,"num_objects":142,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":33,"num_read_kb":11479,"num_write":309,"num_write_kb":14473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":923606083,"num_objects":10287,"num_object_clones":0,"num_object_copies":20574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10287,"num_whiteouts":0,"num_read":2243,"num_read_kb":380500,"num_write":20866,"num_write_kb":1045904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1683152896,"data_stored":1845672070,"data_compressed":160523048,"data_compressed_allocated":177569792,"data_compressed_original":393674752,"omap_allocated":0,"internal_metadata":0},"log_size":30282,"ondisk_log_size":30282,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":168060516,"num_objects":202,"num_object_clones":0,"num_object_copies":404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":202,"num_whiteouts":0,"num_read":555,"num_read_kb":254049,"num_write":6966,"num_write_kb":198967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":122,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":566645,"num_omap_keys":999,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":84746240,"data_stored":336121032,"data_compressed":45213916,"data_compressed_allocated":56918016,"data_compressed_original":308543488,"omap_allocated":0,"internal_metadata":0},"log_size":6894,"ondisk_log_size":6894,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64858,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575028,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511228,"kb_used_data":209844,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":93860612,"statfs":{"total":96636764160,"available":96113266688,"internally_reserved":0,"allocated":214880256,"data_stored":260105949,"data_compressed":22363353,"data_compressed_allocated":25411584,"data_compressed_original":76546048,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.53800000000000003,"15min":0.54500000000000004},"min":{"1min":0.29499999999999998,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.81200000000000006,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.54400000000000004,"15min":0.56599999999999995},"min":{"1min":0.33000000000000002,"5min":0.28599999999999998,"15min":0.28399999999999997},"max":{"1min":0.73099999999999998,"5min":0.80500000000000005,"15min":0.88900000000000001},"last":0.39000000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51400000000000001,"15min":0.52200000000000002},"min":{"1min":0.36299999999999999,"5min":0.312,"15min":0.28899999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54500000000000004,"15min":0.52700000000000002},"min":{"1min":0.35199999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.82699999999999996,"5min":1.25,"15min":1.25},"last":0.40100000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.57499999999999996,"15min":0.55900000000000005},"min":{"1min":0.377,"5min":0.29799999999999999,"15min":0.28399999999999997},"max":{"1min":3.6850000000000001,"5min":3.6850000000000001,"15min":3.6850000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.58599999999999997,"15min":0.61099999999999999},"min":{"1min":0.376,"5min":0.33700000000000002,"15min":0.33100000000000002},"max":{"1min":3.0179999999999998,"5min":3.0179999999999998,"15min":7.3739999999999997},"last":0.41699999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32200000000000001,"15min":0.318},"max":{"1min":0.66000000000000003,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.55800000000000005,"15min":0.56000000000000005},"min":{"1min":0.39500000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.32900000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.44700000000000001,"15min":0.45900000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.749,"5min":0.749,"15min":1.0149999999999999},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28299999999999997},"max":{"1min":0.70699999999999996,"5min":0.80200000000000005,"15min":0.90700000000000003},"last":0.28299999999999997}]},{"osd":5,"last update":"Fri Oct 6 04:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.502,"15min":0.50800000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.27300000000000002},"max":{"1min":0.72199999999999998,"5min":0.82999999999999996,"15min":0.86099999999999999},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55600000000000005,"15min":0.55900000000000005},"min":{"1min":0.33800000000000002,"5min":0.317,"15min":0.28000000000000003},"max":{"1min":0.78600000000000003,"5min":0.84899999999999998,"15min":1.377},"last":0.34000000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51700000000000002,"15min":0.55400000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.29999999999999999},"max":{"1min":0.69199999999999995,"5min":0.77600000000000002,"15min":1.3420000000000001},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60699999999999998,"15min":0.58999999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.30199999999999999},"max":{"1min":0.81999999999999995,"5min":0.88200000000000001,"15min":0.93700000000000006},"last":0.372}]}]},{"osd":6,"up_from":13,"seq":55834575027,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":474560,"kb_used_data":164984,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":93897280,"statfs":{"total":96636764160,"available":96150814720,"internally_reserved":0,"allocated":168943616,"data_stored":188218692,"data_compressed":15936082,"data_compressed_allocated":17805312,"data_compressed_original":42737664,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54200000000000004,"15min":0.53100000000000003},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.98099999999999998,"5min":2.613,"15min":2.613},"last":0.433},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.58899999999999997,"15min":0.56000000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.99299999999999999,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":0.38}]},{"osd":1,"last update":"Fri Oct 6 04:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.53700000000000003,"15min":0.56899999999999995},"min":{"1min":0.33700000000000002,"5min":0.28499999999999998,"15min":0.27200000000000002},"max":{"1min":0.79200000000000004,"5min":2.3889999999999998,"15min":9.5519999999999996},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.59499999999999997,"15min":0.58399999999999996},"min":{"1min":0.37,"5min":0.315,"15min":0.313},"max":{"1min":0.66600000000000004,"5min":6.8019999999999996,"15min":6.8019999999999996},"last":0.38700000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.61699999999999999,"15min":0.64200000000000002},"min":{"1min":0.377,"5min":0.32600000000000001,"15min":0.29799999999999999},"max":{"1min":0.67000000000000004,"5min":5.7590000000000003,"15min":9.6769999999999996},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.63200000000000001,"15min":0.64800000000000002},"min":{"1min":0.33600000000000002,"5min":0.33200000000000002,"15min":0.31},"max":{"1min":0.89800000000000002,"5min":5.71,"15min":9.7309999999999999},"last":0.33600000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.53300000000000003,"15min":0.55600000000000005},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.61699999999999999,"5min":0.99099999999999999,"15min":5.7359999999999998},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.57599999999999996,"15min":0.59299999999999997},"min":{"1min":0.317,"5min":0.311,"15min":0.308},"max":{"1min":0.76100000000000001,"5min":1.0149999999999999,"15min":5.7009999999999996},"last":0.41299999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.47899999999999998,"15min":0.48499999999999999},"min":{"1min":0.27200000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.60999999999999999,"5min":0.78600000000000003,"15min":1.0129999999999999},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.52500000000000002,"15min":0.54000000000000004},"min":{"1min":0.35299999999999998,"5min":0.313,"15min":0.30399999999999999},"max":{"1min":0.55600000000000005,"5min":0.79800000000000004,"15min":1.026},"last":0.39600000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.48999999999999999,"15min":0.501},"min":{"1min":0.28899999999999998,"5min":0.247,"15min":0.247},"max":{"1min":0.59499999999999997,"5min":0.78200000000000003,"15min":0.995},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.53000000000000003,"15min":0.53500000000000003},"min":{"1min":0.32100000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65100000000000002,"5min":0.78500000000000003,"15min":0.83699999999999997},"last":0.35699999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.53800000000000003,"15min":0.53300000000000003},"min":{"1min":0.32100000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.72299999999999998,"5min":0.84199999999999997,"15min":0.98299999999999998},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.55600000000000005,"15min":0.56799999999999995},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.58899999999999997,"5min":0.84699999999999998,"15min":0.90800000000000003},"last":0.42399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575027,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":590936,"kb_used_data":277264,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":93780904,"statfs":{"total":96636764160,"available":96031645696,"internally_reserved":0,"allocated":283918336,"data_stored":337886631,"data_compressed":32878254,"data_compressed_allocated":37023744,"data_compressed_original":100233216,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:11:29 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.92300000000000004,"15min":0.52400000000000002},"min":{"1min":0.20899999999999999,"5min":0.20799999999999999,"15min":0.192},"max":{"1min":0.97599999999999998,"5min":57.700000000000003,"15min":57.700000000000003},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.64800000000000002,"15min":0.443},"min":{"1min":0.224,"5min":0.224,"15min":0.186},"max":{"1min":0.495,"5min":28.629999999999999,"15min":28.629999999999999},"last":0.314}]},{"osd":2,"last update":"Fri Oct 6 04:11:29 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.97799999999999998,"15min":0.57399999999999995},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.19600000000000001},"max":{"1min":0.59799999999999998,"5min":58.795000000000002,"15min":58.795000000000002},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.67400000000000004,"15min":0.49099999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.23000000000000001},"max":{"1min":0.64800000000000002,"5min":25.722000000000001,"15min":25.722000000000001},"last":0.35299999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:11:29 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.59999999999999998,"15min":0.442},"min":{"1min":0.246,"5min":0.23999999999999999,"15min":0.216},"max":{"1min":0.498,"5min":22.920999999999999,"15min":22.920999999999999},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.63600000000000001,"15min":0.47199999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.22600000000000001},"max":{"1min":0.59099999999999997,"5min":22.797999999999998,"15min":22.797999999999998},"last":0.32800000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:11:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":1.2310000000000001,"15min":0.82699999999999996},"min":{"1min":0.40200000000000002,"5min":0.38500000000000001,"15min":0.34499999999999997},"max":{"1min":0.96499999999999997,"5min":59.018999999999998,"15min":59.018999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.626,"5min":1.252,"15min":0.83899999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.316},"max":{"1min":0.93600000000000005,"5min":59.046999999999997,"15min":59.046999999999997},"last":0.60199999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:11:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":1.1950000000000001,"15min":0.79300000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.874,"5min":58.987000000000002,"15min":58.987000000000002},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.65600000000000003,"15min":0.59799999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.29399999999999998},"max":{"1min":0.85799999999999998,"5min":6.2140000000000004,"15min":6.2140000000000004},"last":0.54400000000000004}]},{"osd":6,"last update":"Fri Oct 6 04:11:29 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.65700000000000003,"15min":0.60899999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.36599999999999999},"max":{"1min":0.90500000000000003,"5min":4.6020000000000003,"15min":4.6020000000000003},"last":0.63},{"interface":"front","average":{"1min":0.628,"5min":1.2949999999999999,"15min":0.86199999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.36799999999999999},"max":{"1min":0.91200000000000003,"5min":58.911999999999999,"15min":58.911999999999999},"last":0.58899999999999997}]},{"osd":7,"last update":"Fri Oct 6 04:11:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":1.216,"15min":0.81399999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88100000000000001,"5min":58.953000000000003,"15min":58.953000000000003},"last":45.987000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.65500000000000003,"15min":0.61799999999999999},"min":{"1min":0.40200000000000002,"5min":0.38100000000000001,"15min":0.34399999999999997},"max":{"1min":0.84399999999999997,"5min":4.5570000000000004,"15min":4.5570000000000004},"last":46}]}]},{"osd":0,"up_from":13,"seq":55834575027,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488692,"kb_used_data":191404,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93883148,"statfs":{"total":96636764160,"available":96136343552,"internally_reserved":0,"allocated":195997696,"data_stored":253566865,"data_compressed":26834878,"data_compressed_allocated":30744576,"data_compressed_original":94838784,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33800000000000002,"15min":0.35099999999999998},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19500000000000001},"max":{"1min":0.44800000000000001,"5min":0.52900000000000003,"15min":6.9539999999999997},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.34899999999999998,"15min":0.371},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.191},"max":{"1min":0.46300000000000002,"5min":0.59499999999999997,"15min":8.0519999999999996},"last":0.23699999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36799999999999999,"15min":0.36099999999999999},"min":{"1min":0.25900000000000001,"5min":0.24299999999999999,"15min":0.214},"max":{"1min":0.53000000000000003,"5min":0.59099999999999997,"15min":1.0589999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.371,"5min":0.38400000000000001,"15min":0.378},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.20799999999999999},"max":{"1min":0.496,"5min":0.626,"15min":2.0489999999999999},"last":0.28100000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.39300000000000002,"15min":0.42099999999999999},"min":{"1min":0.27500000000000002,"5min":0.23100000000000001,"15min":0.223},"max":{"1min":0.56799999999999995,"5min":0.60699999999999998,"15min":9.391},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.42799999999999999,"15min":0.45700000000000002},"min":{"1min":0.32900000000000001,"5min":0.311,"15min":0.27800000000000002},"max":{"1min":0.58899999999999997,"5min":0.64400000000000002,"15min":9.5239999999999991},"last":0.50900000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.64700000000000002,"15min":0.63700000000000001},"min":{"1min":0.35599999999999998,"5min":0.34000000000000002,"15min":0.29199999999999998},"max":{"1min":0.90800000000000003,"5min":1.026,"15min":5.7699999999999996},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.65400000000000003,"15min":0.63600000000000001},"min":{"1min":0.48399999999999999,"5min":0.41499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81699999999999995,"5min":0.997,"15min":2.7320000000000002},"last":0.40799999999999997}]},{"osd":5,"last update":"Fri Oct 6 04:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60499999999999998,"15min":0.61599999999999999},"min":{"1min":0.45000000000000001,"5min":0.41399999999999998,"15min":0.33800000000000002},"max":{"1min":0.753,"5min":0.90500000000000003,"15min":5.5069999999999997},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.63400000000000001,"15min":0.60799999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.34799999999999998},"max":{"1min":0.84399999999999997,"5min":0.878,"15min":1.05},"last":0.499}]},{"osd":6,"last update":"Fri Oct 6 04:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.62,"15min":0.64700000000000002},"min":{"1min":0.38900000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77800000000000002,"5min":0.96499999999999997,"15min":6.9139999999999997},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63900000000000001,"15min":0.65400000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.379},"max":{"1min":0.83099999999999996,"5min":0.93000000000000005,"15min":6.3780000000000001},"last":0.53800000000000003}]},{"osd":7,"last update":"Fri Oct 6 04:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60399999999999998,"15min":0.621},"min":{"1min":0.39300000000000002,"5min":0.377,"15min":0.36699999999999999},"max":{"1min":0.84899999999999998,"5min":0.97799999999999998,"15min":5.9989999999999997},"last":0.439},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.624,"15min":0.61299999999999999},"min":{"1min":0.42499999999999999,"5min":0.40500000000000003,"15min":0.39700000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":2.6930000000000001},"last":0.46700000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575027,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446244,"kb_used_data":140764,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":93925596,"statfs":{"total":96636764160,"available":96179810304,"internally_reserved":0,"allocated":144142336,"data_stored":175617804,"data_compressed":19256061,"data_compressed_allocated":21770240,"data_compressed_original":59097088,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:11:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.45600000000000002,"15min":0.42199999999999999},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.19},"max":{"1min":11.335000000000001,"5min":11.335000000000001,"15min":14.968999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.46999999999999997,"15min":0.44700000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.21299999999999999},"max":{"1min":10.484999999999999,"5min":10.484999999999999,"15min":14.746},"last":0.35399999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.37,"15min":0.52200000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.21099999999999999},"max":{"1min":0.53300000000000003,"5min":0.54100000000000004,"15min":50.677999999999997},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.439,"5min":0.45800000000000002,"15min":0.60099999999999998},"min":{"1min":0.34599999999999997,"5min":0.28599999999999998,"15min":0.26000000000000001},"max":{"1min":0.57599999999999996,"5min":0.66200000000000003,"15min":50.615000000000002},"last":0.42599999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:11:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.433,"15min":0.47899999999999998},"min":{"1min":0.26000000000000001,"5min":0.23599999999999999,"15min":0.22900000000000001},"max":{"1min":6.5179999999999998,"5min":6.5179999999999998,"15min":17.440999999999999},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.52400000000000002,"15min":0.63},"min":{"1min":0.307,"5min":0.29099999999999998,"15min":0.23499999999999999},"max":{"1min":8.1750000000000007,"5min":8.1750000000000007,"15min":50.424999999999997},"last":0.40600000000000003}]},{"osd":4,"last update":"Fri Oct 6 04:11:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":0.72899999999999998,"15min":0.69099999999999995},"min":{"1min":0.35799999999999998,"5min":0.34200000000000003,"15min":0.29699999999999999},"max":{"1min":14.308,"5min":14.308,"15min":14.802},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61399999999999999,"15min":0.67700000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":14.621},"last":0.48699999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:11:34 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.61399999999999999,"15min":0.63800000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.35199999999999998},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":14.792999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.69799999999999995,"15min":0.79000000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.371},"max":{"1min":9.1630000000000003,"5min":9.1630000000000003,"15min":50.537999999999997},"last":0.496}]},{"osd":6,"last update":"Fri Oct 6 04:11:34 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":0.78300000000000003,"15min":0.73699999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":15.627000000000001,"5min":15.627000000000001,"15min":23.388000000000002},"last":0.58399999999999996},{"interface":"front","average":{"1min":1.1539999999999999,"5min":0.751,"15min":0.73699999999999999},"min":{"1min":0.439,"5min":0.42499999999999999,"15min":0.39100000000000001},"max":{"1min":11.372,"5min":11.372,"15min":14.856},"last":0.55500000000000005}]},{"osd":7,"last update":"Fri Oct 6 04:11:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.64500000000000002,"15min":0.68500000000000005},"min":{"1min":0.44600000000000001,"5min":0.40999999999999998,"15min":0.377},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":14.693},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.64800000000000002,"15min":0.67800000000000005},"min":{"1min":0.45900000000000002,"5min":0.44600000000000001,"15min":0.434},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":10.349},"last":0.51500000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575027,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":581572,"kb_used_data":276092,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":93790268,"statfs":{"total":96636764160,"available":96041234432,"internally_reserved":0,"allocated":282718208,"data_stored":344742979,"data_compressed":32796809,"data_compressed_allocated":37396480,"data_compressed_original":108593152,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:11:30 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.34000000000000002,"15min":0.35999999999999999},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.186},"max":{"1min":0.439,"5min":0.58699999999999997,"15min":7.0999999999999996},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.35399999999999998,"15min":0.41699999999999998},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.56499999999999995,"5min":0.622,"15min":14.257},"last":0.30399999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:11:30 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.34699999999999998,"15min":0.35399999999999998},"min":{"1min":0.222,"5min":0.219,"15min":0.20799999999999999},"max":{"1min":0.432,"5min":0.54600000000000004,"15min":3.1760000000000002},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.40200000000000002,"15min":0.39700000000000002},"min":{"1min":0.27300000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.48299999999999998,"5min":0.68200000000000005,"15min":5.2599999999999998},"last":0.33700000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:11:30 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.38500000000000001,"15min":0.45700000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23499999999999999},"max":{"1min":0.55100000000000005,"5min":0.59299999999999997,"15min":13.138},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.42299999999999999,"15min":0.44500000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.27200000000000002},"max":{"1min":0.66100000000000003,"5min":0.67300000000000004,"15min":7.181},"last":0.35899999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:11:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61199999999999999,"15min":0.68000000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75800000000000001,"5min":0.92000000000000004,"15min":13.068},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64800000000000002,"15min":0.67700000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.39800000000000002},"max":{"1min":0.81499999999999995,"5min":0.89100000000000001,"15min":6.4299999999999997},"last":0.622}]},{"osd":5,"last update":"Fri Oct 6 04:11:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.57499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39800000000000002,"5min":0.35999999999999999,"15min":0.32700000000000001},"max":{"1min":0.75600000000000001,"5min":0.95599999999999996,"15min":3.105},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58399999999999996,"15min":0.60399999999999998},"min":{"1min":0.40600000000000003,"5min":0.372,"15min":0.29299999999999998},"max":{"1min":0.77200000000000002,"5min":0.85499999999999998,"15min":3.5369999999999999},"last":0.47099999999999997}]},{"osd":6,"last update":"Fri Oct 6 04:11:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.627,"15min":0.68799999999999994},"min":{"1min":0.41599999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.84499999999999997,"5min":0.96099999999999997,"15min":12.974},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.622,"15min":0.63900000000000001},"min":{"1min":0.439,"5min":0.39300000000000002,"15min":0.35599999999999998},"max":{"1min":0.68899999999999995,"5min":0.94499999999999995,"15min":3.0379999999999998},"last":0.66600000000000004}]},{"osd":7,"last update":"Fri Oct 6 04:11:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.61099999999999999,"15min":0.63500000000000001},"min":{"1min":0.44600000000000001,"5min":0.41199999999999998,"15min":0.40100000000000002},"max":{"1min":0.78400000000000003,"5min":0.97199999999999998,"15min":6.2949999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.61899999999999999,"15min":0.67000000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73099999999999998,"5min":0.95399999999999996,"15min":13.032},"last":0.63100000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575028,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587504,"kb_used_data":282024,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":93784336,"statfs":{"total":96636764160,"available":96035160064,"internally_reserved":0,"allocated":288792576,"data_stored":379214374,"data_compressed":33271846,"data_compressed_allocated":38641664,"data_compressed_original":137351168,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:11:32 2023","interfaces":[{"interface":"back","average":{"1min":3.661,"5min":1.149,"15min":0.73999999999999999},"min":{"1min":0.34499999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":54.295999999999999,"5min":54.295999999999999,"15min":54.295999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":3.2909999999999999,"5min":1.1200000000000001,"15min":0.72999999999999998},"min":{"1min":0.46000000000000002,"5min":0.35099999999999998,"15min":0.26600000000000001},"max":{"1min":46.795000000000002,"5min":46.795000000000002,"15min":46.795000000000002},"last":0.46800000000000003}]},{"osd":1,"last update":"Fri Oct 6 04:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.59099999999999997,"15min":0.54600000000000004},"min":{"1min":0.41699999999999998,"5min":0.34899999999999998,"15min":0.26200000000000001},"max":{"1min":3.621,"5min":3.621,"15min":3.621},"last":0.28000000000000003},{"interface":"front","average":{"1min":3.6629999999999998,"5min":1.135,"15min":0.71999999999999997},"min":{"1min":0.35299999999999998,"5min":0.30399999999999999,"15min":0.27400000000000002},"max":{"1min":54.832999999999998,"5min":54.832999999999998,"15min":54.832999999999998},"last":0.35499999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.55000000000000004,"15min":0.53600000000000003},"min":{"1min":0.378,"5min":0.35899999999999999,"15min":0.312},"max":{"1min":0.68799999999999994,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.42399999999999999,"5min":0.36499999999999999,"15min":0.311},"max":{"1min":0.70699999999999996,"5min":1.413,"15min":1.413},"last":0.49399999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.59099999999999997,"15min":0.55000000000000004},"min":{"1min":0.36099999999999999,"5min":0.32100000000000001,"15min":0.314},"max":{"1min":1.6459999999999999,"5min":6.7000000000000002,"15min":6.7000000000000002},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.52200000000000002,"15min":0.53900000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.32000000000000001},"max":{"1min":0.64600000000000002,"5min":1.208,"15min":1.208},"last":0.34100000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.45400000000000001,"15min":0.47799999999999998},"min":{"1min":0.26400000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.52600000000000002,"15min":0.53100000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.436}]},{"osd":6,"last update":"Fri Oct 6 04:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.46899999999999997,"15min":0.505},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.28100000000000003},"max":{"1min":0.61699999999999999,"5min":1.236,"15min":1.236},"last":0.376},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.376,"5min":0.37,"15min":0.37},"max":{"1min":0.84499999999999997,"5min":1.381,"15min":1.381},"last":0.53500000000000003}]},{"osd":7,"last update":"Fri Oct 6 04:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.53400000000000003,"15min":0.54400000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.30599999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60599999999999998,"15min":0.60299999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.32600000000000001},"max":{"1min":0.80000000000000004,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.51700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575027,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490364,"kb_used_data":188980,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":93881476,"statfs":{"total":96636764160,"available":96134631424,"internally_reserved":0,"allocated":193515520,"data_stored":246084607,"data_compressed":22464539,"data_compressed_allocated":25784320,"data_compressed_original":83968000,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.502,"15min":0.55600000000000005},"min":{"1min":0.32400000000000001,"5min":0.29999999999999999,"15min":0.23300000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":11.167},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52600000000000002,"15min":0.57699999999999996},"min":{"1min":0.34799999999999998,"5min":0.23599999999999999,"15min":0.221},"max":{"1min":0.82899999999999996,"5min":0.94799999999999995,"15min":9.8659999999999997},"last":0.64500000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53100000000000003,"15min":0.54000000000000004},"min":{"1min":0.40799999999999997,"5min":0.222,"15min":0.222},"max":{"1min":0.78700000000000003,"5min":0.80800000000000005,"15min":1.1120000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.30199999999999999,"5min":0.29999999999999999,"15min":0.28799999999999998},"max":{"1min":0.73399999999999999,"5min":0.875,"15min":2.524},"last":0.53500000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.52100000000000002,"15min":0.54100000000000004},"min":{"1min":0.34200000000000003,"5min":0.316,"15min":0.316},"max":{"1min":0.69599999999999995,"5min":0.85899999999999999,"15min":1.0389999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54500000000000004,"15min":0.55600000000000005},"min":{"1min":0.32300000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68200000000000005,"5min":0.76600000000000001,"15min":1.1439999999999999},"last":0.58599999999999997}]},{"osd":3,"last update":"Fri Oct 6 04:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56100000000000005,"15min":0.56899999999999995},"min":{"1min":0.45700000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":2.9359999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61599999999999999,"15min":0.59599999999999997},"min":{"1min":0.45300000000000001,"5min":0.377,"15min":0.28999999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.0620000000000001},"last":0.71199999999999997}]},{"osd":4,"last update":"Fri Oct 6 04:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.44900000000000001,"15min":0.47699999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":1.1619999999999999},"last":0.434},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56799999999999995,"15min":0.54500000000000004},"min":{"1min":0.36799999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":1.1950000000000001},"last":0.48799999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46899999999999997,"15min":0.497},"min":{"1min":0.30099999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70599999999999996,"5min":0.78300000000000003,"15min":0.94099999999999995},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56000000000000005,"15min":0.57899999999999996},"min":{"1min":0.39000000000000001,"5min":0.38400000000000001,"15min":0.32900000000000001},"max":{"1min":0.751,"5min":0.84999999999999998,"15min":0.97899999999999998},"last":0.51200000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.52500000000000002,"15min":0.53500000000000003},"min":{"1min":0.35099999999999998,"5min":0.33500000000000002,"15min":0.28699999999999998},"max":{"1min":0.81100000000000005,"5min":0.84899999999999998,"15min":1.1830000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58099999999999996,"15min":0.58599999999999997},"min":{"1min":0.376,"5min":0.375,"15min":0.311},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":1.29},"last":0.38400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13369344,"data_stored":49512965,"data_compressed":6345911,"data_compressed_allocated":8089600,"data_compressed_original":44290048,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8646656,"data_stored":37038012,"data_compressed":4869542,"data_compressed_allocated":6230016,"data_compressed_original":34631680,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3923968,"data_stored":16797884,"data_compressed":2299517,"data_compressed_allocated":2953216,"data_compressed_original":15835136,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12926976,"data_stored":47356979,"data_compressed":5951348,"data_compressed_allocated":7700480,"data_compressed_original":42160128,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":21995520,"data_stored":88324981,"data_compressed":12367125,"data_compressed_allocated":15323136,"data_compressed_original":81711104,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":11075584,"data_stored":46041285,"data_compressed":6106433,"data_compressed_allocated":7659520,"data_compressed_original":42663936,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1855488,"data_stored":8388720,"data_compressed":1228027,"data_compressed_allocated":1515520,"data_compressed_original":8056832,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":10952704,"data_stored":42660206,"data_compressed":6046013,"data_compressed_allocated":7446528,"data_compressed_original":39194624,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":181956608,"data_stored":203156358,"data_compressed":20456538,"data_compressed_allocated":22609920,"data_compressed_original":49975296,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":274661376,"data_stored":300540492,"data_compressed":28008712,"data_compressed_allocated":30793728,"data_compressed_original":65601536,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139608064,"data_stored":158511793,"data_compressed":16956544,"data_compressed_allocated":18817024,"data_compressed_original":43261952,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":269180928,"data_stored":297077873,"data_compressed":26845461,"data_compressed_allocated":29696000,"data_compressed_original":66433024,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":266186752,"data_stored":290581266,"data_compressed":20904721,"data_compressed_allocated":23318528,"data_compressed_original":55640064,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":181829632,"data_stored":199735195,"data_compressed":16358106,"data_compressed_allocated":18124800,"data_compressed_original":41304064,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":166477824,"data_stored":179521845,"data_compressed":14708055,"data_compressed_allocated":16289792,"data_compressed_original":34680832,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":203251712,"data_stored":216547248,"data_compressed":16284911,"data_compressed_allocated":17920000,"data_compressed_original":36777984,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:11:44.112 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:44.112 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:44.112 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:44.112 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:44.113 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:44.113 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:44.113 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:44.113 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:44.114 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:44.114 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-10-06T03:57:09.323709+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:44.115 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2023-10-06T03:57:10.325696+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=57, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=10, tm_sec=58, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:11:44.115 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T04:11:46.718 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:11:46.720 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:12:04.115 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:12:04.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.243+0000 7f03eec40700 1 Processor -- start 2023-10-06T04:12:04.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.243+0000 7f03eec40700 1 -- start start 2023-10-06T04:12:04.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.243+0000 7f03eec40700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f03e80b1d40 0x7f03e8159df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:04.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.243+0000 7f03eec40700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f03e80b2110 0x7f03e80af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:04.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.243+0000 7f03eec40700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03e806c1f0 0x7f03e814fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:04.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.243+0000 7f03eec40700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e806ced0 con 0x7f03e80b2110 2023-10-06T04:12:04.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.243+0000 7f03eec40700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e806cc10 con 0x7f03e80b1d40 2023-10-06T04:12:04.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.243+0000 7f03eec40700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e806e570 con 0x7f03e806c1f0 2023-10-06T04:12:04.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.243+0000 7f03dffff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f03e80b2110 0x7f03e80af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:04.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.243+0000 7f03dffff700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f03e80b2110 0x7f03e80af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56086/0 (socket says 172.21.15.157:56086) 2023-10-06T04:12:04.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.244+0000 7f03dffff700 1 -- 172.21.15.157:0/1521504139 learned_addr learned my addr 172.21.15.157:0/1521504139 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:12:04.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.244+0000 7f03ec9dc700 1 --2- 172.21.15.157:0/1521504139 >> v2:172.21.15.175:3300/0 conn(0x7f03e80b1d40 0x7f03e8159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:04.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.244+0000 7f03ed1dd700 1 --2- 172.21.15.157:0/1521504139 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03e806c1f0 0x7f03e814fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:04.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.244+0000 7f03dffff700 1 -- 172.21.15.157:0/1521504139 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03e806c1f0 msgr2=0x7f03e814fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:12:04.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.244+0000 7f03dffff700 1 --2- 172.21.15.157:0/1521504139 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03e806c1f0 0x7f03e814fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:04.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.244+0000 7f03dffff700 1 -- 172.21.15.157:0/1521504139 >> v2:172.21.15.175:3300/0 conn(0x7f03e80b1d40 msgr2=0x7f03e8159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:12:04.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.244+0000 7f03dffff700 1 --2- 172.21.15.157:0/1521504139 >> v2:172.21.15.175:3300/0 conn(0x7f03e80b1d40 0x7f03e8159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:04.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.244+0000 7f03dffff700 1 -- 172.21.15.157:0/1521504139 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e806d7b0 con 0x7f03e80b2110 2023-10-06T04:12:04.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.244+0000 7f03ec9dc700 1 --2- 172.21.15.157:0/1521504139 >> v2:172.21.15.175:3300/0 conn(0x7f03e80b1d40 0x7f03e8159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:12:04.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.244+0000 7f03dffff700 1 --2- 172.21.15.157:0/1521504139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03e80b2110 0x7f03e80af400 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f03e8089e90 tx=0x7f03d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f05027a07b8cefa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:12:04.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.244+0000 7f03df7fe700 1 -- 172.21.15.157:0/1521504139 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d800b8f0 con 0x7f03e80b2110 2023-10-06T04:12:04.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.244+0000 7f03df7fe700 1 -- 172.21.15.157:0/1521504139 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d800ba50 con 0x7f03e80b2110 2023-10-06T04:12:04.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.244+0000 7f03df7fe700 1 -- 172.21.15.157:0/1521504139 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d800bdf0 con 0x7f03e80b2110 2023-10-06T04:12:04.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.245+0000 7f03eec40700 1 -- 172.21.15.157:0/1521504139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03e80b2110 msgr2=0x7f03e80af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:04.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.245+0000 7f03eec40700 1 --2- 172.21.15.157:0/1521504139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03e80b2110 0x7f03e80af400 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f03e8089e90 tx=0x7f03d8009250 comp rx=0 tx=0).stop 2023-10-06T04:12:04.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.245+0000 7f03eec40700 1 -- 172.21.15.157:0/1521504139 shutdown_connections 2023-10-06T04:12:04.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.245+0000 7f03eec40700 1 --2- 172.21.15.157:0/1521504139 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03e806c1f0 0x7f03e814fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:04.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.245+0000 7f03eec40700 1 --2- 172.21.15.157:0/1521504139 >> v2:172.21.15.175:3300/0 conn(0x7f03e80b1d40 0x7f03e8159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:04.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.245+0000 7f03eec40700 1 --2- 172.21.15.157:0/1521504139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03e80b2110 0x7f03e80af400 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:04.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.245+0000 7f03eec40700 1 -- 172.21.15.157:0/1521504139 >> 172.21.15.157:0/1521504139 conn(0x7f03e80cc6e0 msgr2=0x7f03e80ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:12:04.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.245+0000 7f03eec40700 1 -- 172.21.15.157:0/1521504139 shutdown_connections 2023-10-06T04:12:04.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.245+0000 7f03eec40700 1 -- 172.21.15.157:0/1521504139 wait complete. 2023-10-06T04:12:04.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.246+0000 7f03eec40700 1 Processor -- start 2023-10-06T04:12:04.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.246+0000 7f03eec40700 1 -- start start 2023-10-06T04:12:04.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.247+0000 7f03eec40700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03e806c1f0 0x7f03e8153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:04.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.247+0000 7f03eec40700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03e80b1d40 0x7f03e8155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:04.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.247+0000 7f03eec40700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03e80b2110 0x7f03e8156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:04.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.247+0000 7f03eec40700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e8166ea0 con 0x7f03e80b1d40 2023-10-06T04:12:04.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.247+0000 7f03eec40700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e8154eb0 con 0x7f03e80b2110 2023-10-06T04:12:04.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.247+0000 7f03eec40700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e8155190 con 0x7f03e806c1f0 2023-10-06T04:12:04.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.247+0000 7f03dffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03e80b1d40 0x7f03e8155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:04.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.247+0000 7f03dffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03e80b1d40 0x7f03e8155f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56092/0 (socket says 172.21.15.157:56092) 2023-10-06T04:12:04.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.247+0000 7f03dffff700 1 -- 172.21.15.157:0/3991126455 learned_addr learned my addr 172.21.15.157:0/3991126455 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:12:04.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.247+0000 7f03ec9dc700 1 --2- 172.21.15.157:0/3991126455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03e806c1f0 0x7f03e8153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:04.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.247+0000 7f03ed1dd700 1 --2- 172.21.15.157:0/3991126455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03e80b2110 0x7f03e8156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:04.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.247+0000 7f03dffff700 1 -- 172.21.15.157:0/3991126455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03e806c1f0 msgr2=0x7f03e8153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:04.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.247+0000 7f03dffff700 1 --2- 172.21.15.157:0/3991126455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03e806c1f0 0x7f03e8153020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:04.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.247+0000 7f03dffff700 1 -- 172.21.15.157:0/3991126455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03e80b2110 msgr2=0x7f03e8156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:04.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.247+0000 7f03dffff700 1 --2- 172.21.15.157:0/3991126455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03e80b2110 0x7f03e8156490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:04.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.247+0000 7f03dffff700 1 -- 172.21.15.157:0/3991126455 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d80094f0 con 0x7f03e80b1d40 2023-10-06T04:12:04.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.247+0000 7f03ed1dd700 1 --2- 172.21.15.157:0/3991126455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03e80b2110 0x7f03e8156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:12:04.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.247+0000 7f03ec9dc700 1 --2- 172.21.15.157:0/3991126455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03e806c1f0 0x7f03e8153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:12:04.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.247+0000 7f03dffff700 1 --2- 172.21.15.157:0/3991126455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03e80b1d40 0x7f03e8155f50 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f03d8009290 tx=0x7f03d8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:12:04.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.248+0000 7f03dcff9700 1 -- 172.21.15.157:0/3991126455 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d8011e60 con 0x7f03e80b1d40 2023-10-06T04:12:04.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.248+0000 7f03dcff9700 1 -- 172.21.15.157:0/3991126455 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d801f070 con 0x7f03e80b1d40 2023-10-06T04:12:04.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.248+0000 7f03eec40700 1 -- 172.21.15.157:0/3991126455 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e8155470 con 0x7f03e80b1d40 2023-10-06T04:12:04.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.248+0000 7f03dcff9700 1 -- 172.21.15.157:0/3991126455 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d801a890 con 0x7f03e80b1d40 2023-10-06T04:12:04.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.248+0000 7f03eec40700 1 -- 172.21.15.157:0/3991126455 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e8155a30 con 0x7f03e80b1d40 2023-10-06T04:12:04.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.249+0000 7f03eec40700 1 -- 172.21.15.157:0/3991126455 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e80896c0 con 0x7f03e80b1d40 2023-10-06T04:12:04.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.249+0000 7f03dcff9700 1 -- 172.21.15.157:0/3991126455 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f03d801ac00 con 0x7f03e80b1d40 2023-10-06T04:12:04.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.249+0000 7f03dcff9700 1 --2- 172.21.15.157:0/3991126455 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f03d4066f30 0x7f03d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:04.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.249+0000 7f03dcff9700 1 -- 172.21.15.157:0/3991126455 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f03d8021070 con 0x7f03e80b1d40 2023-10-06T04:12:04.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.250+0000 7f03ec9dc700 1 --2- 172.21.15.157:0/3991126455 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f03d4066f30 0x7f03d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:04.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.250+0000 7f03ec9dc700 1 --2- 172.21.15.157:0/3991126455 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f03d4066f30 0x7f03d40693e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f03e4006f00 tx=0x7f03e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:12:04.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.254+0000 7f03dcff9700 1 -- 172.21.15.157:0/3991126455 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f03d80537a0 con 0x7f03e80b1d40 2023-10-06T04:12:04.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.433+0000 7f03eec40700 1 -- 172.21.15.157:0/3991126455 --> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f03e806da90 con 0x7f03d4066f30 2023-10-06T04:12:04.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.443+0000 7f03dcff9700 1 -- 172.21.15.157:0/3991126455 <== mgr.4107 v2:172.21.15.175:6832/90661 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+317999 (secure 0 0 0) 0x7f03e806da90 con 0x7f03d4066f30 2023-10-06T04:12:04.448 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:12:04.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.446+0000 7f03eec40700 1 -- 172.21.15.157:0/3991126455 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f03d4066f30 msgr2=0x7f03d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:04.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.446+0000 7f03eec40700 1 --2- 172.21.15.157:0/3991126455 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f03d4066f30 0x7f03d40693e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f03e4006f00 tx=0x7f03e4008040 comp rx=0 tx=0).stop 2023-10-06T04:12:04.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.446+0000 7f03eec40700 1 -- 172.21.15.157:0/3991126455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03e80b1d40 msgr2=0x7f03e8155f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:04.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.446+0000 7f03eec40700 1 --2- 172.21.15.157:0/3991126455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03e80b1d40 0x7f03e8155f50 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f03d8009290 tx=0x7f03d8017000 comp rx=0 tx=0).stop 2023-10-06T04:12:04.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.446+0000 7f03eec40700 1 -- 172.21.15.157:0/3991126455 shutdown_connections 2023-10-06T04:12:04.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.446+0000 7f03eec40700 1 --2- 172.21.15.157:0/3991126455 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f03d4066f30 0x7f03d40693e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:04.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.446+0000 7f03eec40700 1 --2- 172.21.15.157:0/3991126455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03e80b2110 0x7f03e8156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:04.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.446+0000 7f03eec40700 1 --2- 172.21.15.157:0/3991126455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03e806c1f0 0x7f03e8153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:04.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.446+0000 7f03eec40700 1 --2- 172.21.15.157:0/3991126455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03e80b1d40 0x7f03e8155f50 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:04.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.446+0000 7f03eec40700 1 -- 172.21.15.157:0/3991126455 >> 172.21.15.157:0/3991126455 conn(0x7f03e80cc6e0 msgr2=0x7f03e81576b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:12:04.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.446+0000 7f03eec40700 1 -- 172.21.15.157:0/3991126455 shutdown_connections 2023-10-06T04:12:04.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.446+0000 7f03eec40700 1 -- 172.21.15.157:0/3991126455 wait complete. 2023-10-06T04:12:04.453 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-10-06T04:12:04.459 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":497,"stamp":"2023-10-06T04:12:03.662500+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1092256967,"num_objects":10491,"num_object_clones":0,"num_object_copies":20982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10491,"num_whiteouts":0,"num_read":2914,"num_read_kb":634649,"num_write":27991,"num_write_kb":1247287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":122,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":581829,"num_omap_keys":1025,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38884,"ondisk_log_size":38884,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":4154716,"kb_used_data":1731356,"kb_used_omap":0,"kb_used_meta":2423296,"kb_avail":750820004,"statfs":{"total":773094113280,"available":768839684096,"internally_reserved":0,"allocated":1772908544,"data_stored":2185437901,"data_compressed":205801822,"data_compressed_allocated":234577920,"data_compressed_original":703365120,"omap_allocated":0,"internal_metadata":2481455104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003180"},"pg_stats":[{"pgid":"4.3f","version":"53'449","reported_seq":2562,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:09.544941+0000","last_change":"2023-10-06T04:11:09.544918+0000","last_active":"2023-10-06T04:11:09.544941+0000","last_peered":"2023-10-06T04:11:09.544941+0000","last_clean":"2023-10-06T04:11:09.544941+0000","last_became_active":"2023-10-06T03:57:11.371756+0000","last_became_peered":"2023-10-06T03:57:11.371756+0000","last_unstale":"2023-10-06T04:11:09.544941+0000","last_undegraded":"2023-10-06T04:11:09.544941+0000","last_fullsized":"2023-10-06T04:11:09.544941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'449","last_scrub_stamp":"2023-10-06T04:11:09.544846+0000","last_deep_scrub":"53'449","last_deep_scrub_stamp":"2023-10-06T04:11:09.544846+0000","last_clean_scrub_stamp":"2023-10-06T04:11:09.544846+0000","objects_scrubbed":169,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:19:26.736027+0000","scrub_duration":0.096486541999999995,"objects_trimmed":0,"snaptrim_duration":0.001094783,"stat_sum":{"num_bytes":10615219,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":29,"num_read_kb":15,"num_write":308,"num_write_kb":10492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"53'532","reported_seq":2650,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:41.704323+0000","last_change":"2023-10-06T04:11:41.704302+0000","last_active":"2023-10-06T04:11:41.704323+0000","last_peered":"2023-10-06T04:11:41.704323+0000","last_clean":"2023-10-06T04:11:41.704323+0000","last_became_active":"2023-10-06T03:57:11.370070+0000","last_became_peered":"2023-10-06T03:57:11.370070+0000","last_unstale":"2023-10-06T04:11:41.704323+0000","last_undegraded":"2023-10-06T04:11:41.704323+0000","last_fullsized":"2023-10-06T04:11:41.704323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'532","last_scrub_stamp":"2023-10-06T04:11:41.704234+0000","last_deep_scrub":"53'532","last_deep_scrub_stamp":"2023-10-06T04:11:41.704234+0000","last_clean_scrub_stamp":"2023-10-06T04:11:41.704234+0000","objects_scrubbed":178,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:21.310968+0000","scrub_duration":0.11469288599999999,"objects_trimmed":0,"snaptrim_duration":0.0021773819999999998,"stat_sum":{"num_bytes":6466808,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":33,"num_read_kb":16,"num_write":334,"num_write_kb":6491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"53'512","reported_seq":2633,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:58.363966+0000","last_change":"2023-10-06T04:11:58.363943+0000","last_active":"2023-10-06T04:11:58.363966+0000","last_peered":"2023-10-06T04:11:58.363966+0000","last_clean":"2023-10-06T04:11:58.363966+0000","last_became_active":"2023-10-06T03:57:11.366012+0000","last_became_peered":"2023-10-06T03:57:11.366012+0000","last_unstale":"2023-10-06T04:11:58.363966+0000","last_undegraded":"2023-10-06T04:11:58.363966+0000","last_fullsized":"2023-10-06T04:11:58.363966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'512","last_scrub_stamp":"2023-10-06T04:11:58.363874+0000","last_deep_scrub":"53'512","last_deep_scrub_stamp":"2023-10-06T04:11:58.363874+0000","last_clean_scrub_stamp":"2023-10-06T04:11:58.363874+0000","objects_scrubbed":156,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:01.934316+0000","scrub_duration":0.125796144,"objects_trimmed":0,"snaptrim_duration":0.002722286,"stat_sum":{"num_bytes":15834623,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":40,"num_read_kb":20499,"num_write":321,"num_write_kb":23772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"53'431","reported_seq":2547,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:10.152318+0000","last_change":"2023-10-06T04:11:10.152270+0000","last_active":"2023-10-06T04:11:10.152318+0000","last_peered":"2023-10-06T04:11:10.152318+0000","last_clean":"2023-10-06T04:11:10.152318+0000","last_became_active":"2023-10-06T03:57:11.365673+0000","last_became_peered":"2023-10-06T03:57:11.365673+0000","last_unstale":"2023-10-06T04:11:10.152318+0000","last_undegraded":"2023-10-06T04:11:10.152318+0000","last_fullsized":"2023-10-06T04:11:10.152318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'431","last_scrub_stamp":"2023-10-06T04:11:10.152150+0000","last_deep_scrub":"53'431","last_deep_scrub_stamp":"2023-10-06T04:11:10.152150+0000","last_clean_scrub_stamp":"2023-10-06T04:11:10.152150+0000","objects_scrubbed":157,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:40.609815+0000","scrub_duration":0.112734037,"objects_trimmed":0,"snaptrim_duration":0.001947182,"stat_sum":{"num_bytes":14764610,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":32,"num_read_kb":7963,"num_write":291,"num_write_kb":14577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"53'495","reported_seq":2610,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:37.529840+0000","last_change":"2023-10-06T04:11:37.529810+0000","last_active":"2023-10-06T04:11:37.529840+0000","last_peered":"2023-10-06T04:11:37.529840+0000","last_clean":"2023-10-06T04:11:37.529840+0000","last_became_active":"2023-10-06T03:57:11.361302+0000","last_became_peered":"2023-10-06T03:57:11.361302+0000","last_unstale":"2023-10-06T04:11:37.529840+0000","last_undegraded":"2023-10-06T04:11:37.529840+0000","last_fullsized":"2023-10-06T04:11:37.529840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'495","last_scrub_stamp":"2023-10-06T04:11:37.529742+0000","last_deep_scrub":"53'495","last_deep_scrub_stamp":"2023-10-06T04:11:37.529742+0000","last_clean_scrub_stamp":"2023-10-06T04:11:37.529742+0000","objects_scrubbed":153,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:25.180275+0000","scrub_duration":0.095122445999999999,"objects_trimmed":0,"snaptrim_duration":0.001966644,"stat_sum":{"num_bytes":2193993,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":31,"num_read_kb":18,"num_write":324,"num_write_kb":2284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"53'472","reported_seq":2589,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:50.237243+0000","last_change":"2023-10-06T04:11:50.237221+0000","last_active":"2023-10-06T04:11:50.237243+0000","last_peered":"2023-10-06T04:11:50.237243+0000","last_clean":"2023-10-06T04:11:50.237243+0000","last_became_active":"2023-10-06T03:57:11.368775+0000","last_became_peered":"2023-10-06T03:57:11.368775+0000","last_unstale":"2023-10-06T04:11:50.237243+0000","last_undegraded":"2023-10-06T04:11:50.237243+0000","last_fullsized":"2023-10-06T04:11:50.237243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'472","last_scrub_stamp":"2023-10-06T04:11:50.237148+0000","last_deep_scrub":"53'472","last_deep_scrub_stamp":"2023-10-06T04:11:50.237148+0000","last_clean_scrub_stamp":"2023-10-06T04:11:50.237148+0000","objects_scrubbed":144,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:27:11.682423+0000","scrub_duration":0.114309204,"objects_trimmed":0,"snaptrim_duration":0.002640482,"stat_sum":{"num_bytes":22431099,"num_objects":144,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":0,"num_read":34,"num_read_kb":13627,"num_write":304,"num_write_kb":22068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"53'559","reported_seq":2675,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:41.143847+0000","last_change":"2023-10-06T04:11:41.143824+0000","last_active":"2023-10-06T04:11:41.143847+0000","last_peered":"2023-10-06T04:11:41.143847+0000","last_clean":"2023-10-06T04:11:41.143847+0000","last_became_active":"2023-10-06T03:57:11.377786+0000","last_became_peered":"2023-10-06T03:57:11.377786+0000","last_unstale":"2023-10-06T04:11:41.143847+0000","last_undegraded":"2023-10-06T04:11:41.143847+0000","last_fullsized":"2023-10-06T04:11:41.143847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'559","last_scrub_stamp":"2023-10-06T04:11:41.143745+0000","last_deep_scrub":"53'559","last_deep_scrub_stamp":"2023-10-06T04:11:41.143745+0000","last_clean_scrub_stamp":"2023-10-06T04:11:41.143745+0000","objects_scrubbed":173,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:15:18.950590+0000","scrub_duration":0.15119919300000001,"objects_trimmed":0,"snaptrim_duration":0.00093960500000000004,"stat_sum":{"num_bytes":18398941,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":32,"num_read_kb":8208,"num_write":357,"num_write_kb":18159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"53'479","reported_seq":2593,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:35.684950+0000","last_change":"2023-10-06T04:11:35.684928+0000","last_active":"2023-10-06T04:11:35.684950+0000","last_peered":"2023-10-06T04:11:35.684950+0000","last_clean":"2023-10-06T04:11:35.684950+0000","last_became_active":"2023-10-06T03:57:11.373765+0000","last_became_peered":"2023-10-06T03:57:11.373765+0000","last_unstale":"2023-10-06T04:11:35.684950+0000","last_undegraded":"2023-10-06T04:11:35.684950+0000","last_fullsized":"2023-10-06T04:11:35.684950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'479","last_scrub_stamp":"2023-10-06T04:11:35.684846+0000","last_deep_scrub":"53'479","last_deep_scrub_stamp":"2023-10-06T04:11:35.684846+0000","last_clean_scrub_stamp":"2023-10-06T04:11:35.684846+0000","objects_scrubbed":147,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:13.767095+0000","scrub_duration":0.117065961,"objects_trimmed":0,"snaptrim_duration":0.001873881,"stat_sum":{"num_bytes":10960737,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":31,"num_read_kb":4107,"num_write":305,"num_write_kb":10786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"53'484","reported_seq":2608,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:25.493681+0000","last_change":"2023-10-06T04:11:25.493658+0000","last_active":"2023-10-06T04:11:25.493681+0000","last_peered":"2023-10-06T04:11:25.493681+0000","last_clean":"2023-10-06T04:11:25.493681+0000","last_became_active":"2023-10-06T03:57:11.368004+0000","last_became_peered":"2023-10-06T03:57:11.368004+0000","last_unstale":"2023-10-06T04:11:25.493681+0000","last_undegraded":"2023-10-06T04:11:25.493681+0000","last_fullsized":"2023-10-06T04:11:25.493681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'484","last_scrub_stamp":"2023-10-06T04:11:25.493587+0000","last_deep_scrub":"53'484","last_deep_scrub_stamp":"2023-10-06T04:11:25.493587+0000","last_clean_scrub_stamp":"2023-10-06T04:11:25.493587+0000","objects_scrubbed":154,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:04.307189+0000","scrub_duration":0.10856695700000001,"objects_trimmed":0,"snaptrim_duration":0.0024398710000000001,"stat_sum":{"num_bytes":13584161,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":41,"num_read_kb":4114,"num_write":308,"num_write_kb":17522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"53'502","reported_seq":2616,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:39.671416+0000","last_change":"2023-10-06T04:11:39.671389+0000","last_active":"2023-10-06T04:11:39.671416+0000","last_peered":"2023-10-06T04:11:39.671416+0000","last_clean":"2023-10-06T04:11:39.671416+0000","last_became_active":"2023-10-06T03:57:11.379107+0000","last_became_peered":"2023-10-06T03:57:11.379107+0000","last_unstale":"2023-10-06T04:11:39.671416+0000","last_undegraded":"2023-10-06T04:11:39.671416+0000","last_fullsized":"2023-10-06T04:11:39.671416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'502","last_scrub_stamp":"2023-10-06T04:11:39.671319+0000","last_deep_scrub":"53'502","last_deep_scrub_stamp":"2023-10-06T04:11:39.671319+0000","last_clean_scrub_stamp":"2023-10-06T04:11:39.671319+0000","objects_scrubbed":150,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:09:58.519738+0000","scrub_duration":0.10696708100000001,"objects_trimmed":0,"snaptrim_duration":0.002039126,"stat_sum":{"num_bytes":9784235,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":30,"num_read_kb":4112,"num_write":328,"num_write_kb":9733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"53'544","reported_seq":2667,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:49.230570+0000","last_change":"2023-10-06T04:11:49.230535+0000","last_active":"2023-10-06T04:11:49.230570+0000","last_peered":"2023-10-06T04:11:49.230570+0000","last_clean":"2023-10-06T04:11:49.230570+0000","last_became_active":"2023-10-06T03:57:11.368256+0000","last_became_peered":"2023-10-06T03:57:11.368256+0000","last_unstale":"2023-10-06T04:11:49.230570+0000","last_undegraded":"2023-10-06T04:11:49.230570+0000","last_fullsized":"2023-10-06T04:11:49.230570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'544","last_scrub_stamp":"2023-10-06T04:11:49.230426+0000","last_deep_scrub":"53'544","last_deep_scrub_stamp":"2023-10-06T04:11:49.230426+0000","last_clean_scrub_stamp":"2023-10-06T04:11:49.230426+0000","objects_scrubbed":168,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:27.314027+0000","scrub_duration":0.15375768300000001,"objects_trimmed":0,"snaptrim_duration":0.0029138929999999999,"stat_sum":{"num_bytes":31134351,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":39,"num_read_kb":4114,"num_write":352,"num_write_kb":30587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"53'483","reported_seq":2597,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:21.563546+0000","last_change":"2023-10-06T04:11:21.563509+0000","last_active":"2023-10-06T04:11:21.563546+0000","last_peered":"2023-10-06T04:11:21.563546+0000","last_clean":"2023-10-06T04:11:21.563546+0000","last_became_active":"2023-10-06T03:57:11.368425+0000","last_became_peered":"2023-10-06T03:57:11.368425+0000","last_unstale":"2023-10-06T04:11:21.563546+0000","last_undegraded":"2023-10-06T04:11:21.563546+0000","last_fullsized":"2023-10-06T04:11:21.563546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'483","last_scrub_stamp":"2023-10-06T04:11:21.563396+0000","last_deep_scrub":"53'483","last_deep_scrub_stamp":"2023-10-06T04:11:21.563396+0000","last_clean_scrub_stamp":"2023-10-06T04:11:21.563396+0000","objects_scrubbed":161,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:51.078052+0000","scrub_duration":0.131329159,"objects_trimmed":0,"snaptrim_duration":0.0014231059999999999,"stat_sum":{"num_bytes":10422851,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":30,"num_read_kb":4105,"num_write":305,"num_write_kb":14436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"53'513","reported_seq":2633,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:19.515410+0000","last_change":"2023-10-06T04:11:19.515377+0000","last_active":"2023-10-06T04:11:19.515410+0000","last_peered":"2023-10-06T04:11:19.515410+0000","last_clean":"2023-10-06T04:11:19.515410+0000","last_became_active":"2023-10-06T03:57:11.367079+0000","last_became_peered":"2023-10-06T03:57:11.367079+0000","last_unstale":"2023-10-06T04:11:19.515410+0000","last_undegraded":"2023-10-06T04:11:19.515410+0000","last_fullsized":"2023-10-06T04:11:19.515410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'513","last_scrub_stamp":"2023-10-06T04:11:19.515270+0000","last_deep_scrub":"53'513","last_deep_scrub_stamp":"2023-10-06T04:11:19.515270+0000","last_clean_scrub_stamp":"2023-10-06T04:11:19.515270+0000","objects_scrubbed":173,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:26.862858+0000","scrub_duration":0.13944414499999999,"objects_trimmed":0,"snaptrim_duration":0.001042765,"stat_sum":{"num_bytes":14365712,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":37,"num_read_kb":12308,"num_write":314,"num_write_kb":22325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"53'490","reported_seq":2609,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:26.721067+0000","last_change":"2023-10-06T04:11:26.721045+0000","last_active":"2023-10-06T04:11:26.721067+0000","last_peered":"2023-10-06T04:11:26.721067+0000","last_clean":"2023-10-06T04:11:26.721067+0000","last_became_active":"2023-10-06T03:57:11.373216+0000","last_became_peered":"2023-10-06T03:57:11.373216+0000","last_unstale":"2023-10-06T04:11:26.721067+0000","last_undegraded":"2023-10-06T04:11:26.721067+0000","last_fullsized":"2023-10-06T04:11:26.721067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'490","last_scrub_stamp":"2023-10-06T04:11:26.720977+0000","last_deep_scrub":"53'490","last_deep_scrub_stamp":"2023-10-06T04:11:26.720977+0000","last_clean_scrub_stamp":"2023-10-06T04:11:26.720977+0000","objects_scrubbed":144,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:11.472062+0000","scrub_duration":0.10190239399999999,"objects_trimmed":0,"snaptrim_duration":0.0022456289999999999,"stat_sum":{"num_bytes":5765763,"num_objects":144,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":0,"num_read":36,"num_read_kb":18,"num_write":320,"num_write_kb":9892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"53'516","reported_seq":2630,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:26.988802+0000","last_change":"2023-10-06T04:11:26.988779+0000","last_active":"2023-10-06T04:11:26.988802+0000","last_peered":"2023-10-06T04:11:26.988802+0000","last_clean":"2023-10-06T04:11:26.988802+0000","last_became_active":"2023-10-06T03:57:11.375689+0000","last_became_peered":"2023-10-06T03:57:11.375689+0000","last_unstale":"2023-10-06T04:11:26.988802+0000","last_undegraded":"2023-10-06T04:11:26.988802+0000","last_fullsized":"2023-10-06T04:11:26.988802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'516","last_scrub_stamp":"2023-10-06T04:11:26.988706+0000","last_deep_scrub":"53'516","last_deep_scrub_stamp":"2023-10-06T04:11:26.988706+0000","last_clean_scrub_stamp":"2023-10-06T04:11:26.988706+0000","objects_scrubbed":168,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:32.963008+0000","scrub_duration":0.12428009,"objects_trimmed":0,"snaptrim_duration":0.00092625599999999998,"stat_sum":{"num_bytes":15107134,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":322,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"53'458","reported_seq":2577,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:02.990894+0000","last_change":"2023-10-06T04:11:02.990870+0000","last_active":"2023-10-06T04:11:02.990894+0000","last_peered":"2023-10-06T04:11:02.990894+0000","last_clean":"2023-10-06T04:11:02.990894+0000","last_became_active":"2023-10-06T03:57:11.366543+0000","last_became_peered":"2023-10-06T03:57:11.366543+0000","last_unstale":"2023-10-06T04:11:02.990894+0000","last_undegraded":"2023-10-06T04:11:02.990894+0000","last_fullsized":"2023-10-06T04:11:02.990894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'458","last_scrub_stamp":"2023-10-06T04:11:02.990798+0000","last_deep_scrub":"53'458","last_deep_scrub_stamp":"2023-10-06T04:11:02.990798+0000","last_clean_scrub_stamp":"2023-10-06T04:11:02.990798+0000","objects_scrubbed":155,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:39.255270+0000","scrub_duration":0.116930864,"objects_trimmed":0,"snaptrim_duration":0.0024276609999999998,"stat_sum":{"num_bytes":19833591,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":35,"num_read_kb":19,"num_write":334,"num_write_kb":22749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"53'512","reported_seq":2629,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:23.466557+0000","last_change":"2023-10-06T04:11:23.466513+0000","last_active":"2023-10-06T04:11:23.466557+0000","last_peered":"2023-10-06T04:11:23.466557+0000","last_clean":"2023-10-06T04:11:23.466557+0000","last_became_active":"2023-10-06T03:57:11.366767+0000","last_became_peered":"2023-10-06T03:57:11.366767+0000","last_unstale":"2023-10-06T04:11:23.466557+0000","last_undegraded":"2023-10-06T04:11:23.466557+0000","last_fullsized":"2023-10-06T04:11:23.466557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'512","last_scrub_stamp":"2023-10-06T04:11:23.466398+0000","last_deep_scrub":"53'512","last_deep_scrub_stamp":"2023-10-06T04:11:23.466398+0000","last_clean_scrub_stamp":"2023-10-06T04:11:23.466398+0000","objects_scrubbed":158,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:34.859597+0000","scrub_duration":0.14575623900000001,"objects_trimmed":0,"snaptrim_duration":0.002777833,"stat_sum":{"num_bytes":14683528,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":34,"num_read_kb":8213,"num_write":334,"num_write_kb":18604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"53'536","reported_seq":2663,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:46.098417+0000","last_change":"2023-10-06T04:11:46.098392+0000","last_active":"2023-10-06T04:11:46.098417+0000","last_peered":"2023-10-06T04:11:46.098417+0000","last_clean":"2023-10-06T04:11:46.098417+0000","last_became_active":"2023-10-06T03:57:11.362377+0000","last_became_peered":"2023-10-06T03:57:11.362377+0000","last_unstale":"2023-10-06T04:11:46.098417+0000","last_undegraded":"2023-10-06T04:11:46.098417+0000","last_fullsized":"2023-10-06T04:11:46.098417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'536","last_scrub_stamp":"2023-10-06T04:11:46.098320+0000","last_deep_scrub":"53'536","last_deep_scrub_stamp":"2023-10-06T04:11:46.098320+0000","last_clean_scrub_stamp":"2023-10-06T04:11:46.098320+0000","objects_scrubbed":166,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:12.862470+0000","scrub_duration":0.119497831,"objects_trimmed":0,"snaptrim_duration":0.0027606900000000001,"stat_sum":{"num_bytes":10886042,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":43,"num_read_kb":26,"num_write":338,"num_write_kb":10805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"53'458","reported_seq":2581,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:13.608185+0000","last_change":"2023-10-06T04:11:13.608149+0000","last_active":"2023-10-06T04:11:13.608185+0000","last_peered":"2023-10-06T04:11:13.608185+0000","last_clean":"2023-10-06T04:11:13.608185+0000","last_became_active":"2023-10-06T03:57:11.363792+0000","last_became_peered":"2023-10-06T03:57:11.363792+0000","last_unstale":"2023-10-06T04:11:13.608185+0000","last_undegraded":"2023-10-06T04:11:13.608185+0000","last_fullsized":"2023-10-06T04:11:13.608185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'458","last_scrub_stamp":"2023-10-06T04:11:13.608057+0000","last_deep_scrub":"53'458","last_deep_scrub_stamp":"2023-10-06T04:11:13.608057+0000","last_clean_scrub_stamp":"2023-10-06T04:11:13.608057+0000","objects_scrubbed":159,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:44.566660+0000","scrub_duration":0.104721544,"objects_trimmed":0,"snaptrim_duration":0.0014611940000000001,"stat_sum":{"num_bytes":14666818,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":40,"num_read_kb":8209,"num_write":335,"num_write_kb":18594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"53'525","reported_seq":2645,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:43.099112+0000","last_change":"2023-10-06T04:11:43.099088+0000","last_active":"2023-10-06T04:11:43.099112+0000","last_peered":"2023-10-06T04:11:43.099112+0000","last_clean":"2023-10-06T04:11:43.099112+0000","last_became_active":"2023-10-06T03:57:11.364726+0000","last_became_peered":"2023-10-06T03:57:11.364726+0000","last_unstale":"2023-10-06T04:11:43.099112+0000","last_undegraded":"2023-10-06T04:11:43.099112+0000","last_fullsized":"2023-10-06T04:11:43.099112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'525","last_scrub_stamp":"2023-10-06T04:11:43.099016+0000","last_deep_scrub":"53'525","last_deep_scrub_stamp":"2023-10-06T04:11:43.099016+0000","last_clean_scrub_stamp":"2023-10-06T04:11:43.099016+0000","objects_scrubbed":158,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:27.535736+0000","scrub_duration":0.16158688099999999,"objects_trimmed":0,"snaptrim_duration":0.0029763670000000002,"stat_sum":{"num_bytes":20706667,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":37,"num_read_kb":12308,"num_write":348,"num_write_kb":28542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"53'507","reported_seq":2633,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:38.133749+0000","last_change":"2023-10-06T04:11:38.133716+0000","last_active":"2023-10-06T04:11:38.133749+0000","last_peered":"2023-10-06T04:11:38.133749+0000","last_clean":"2023-10-06T04:11:38.133749+0000","last_became_active":"2023-10-06T03:57:11.375668+0000","last_became_peered":"2023-10-06T03:57:11.375668+0000","last_unstale":"2023-10-06T04:11:38.133749+0000","last_undegraded":"2023-10-06T04:11:38.133749+0000","last_fullsized":"2023-10-06T04:11:38.133749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'507","last_scrub_stamp":"2023-10-06T04:11:38.133638+0000","last_deep_scrub":"53'507","last_deep_scrub_stamp":"2023-10-06T04:11:38.133638+0000","last_clean_scrub_stamp":"2023-10-06T04:11:38.133638+0000","objects_scrubbed":159,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:36.224532+0000","scrub_duration":0.20192412500000001,"objects_trimmed":0,"snaptrim_duration":0.0010285800000000001,"stat_sum":{"num_bytes":19005599,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":42,"num_read_kb":8215,"num_write":326,"num_write_kb":18702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"53'528","reported_seq":2646,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:24.447665+0000","last_change":"2023-10-06T04:11:24.447630+0000","last_active":"2023-10-06T04:11:24.447665+0000","last_peered":"2023-10-06T04:11:24.447665+0000","last_clean":"2023-10-06T04:11:24.447665+0000","last_became_active":"2023-10-06T03:57:11.365149+0000","last_became_peered":"2023-10-06T03:57:11.365149+0000","last_unstale":"2023-10-06T04:11:24.447665+0000","last_undegraded":"2023-10-06T04:11:24.447665+0000","last_fullsized":"2023-10-06T04:11:24.447665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'528","last_scrub_stamp":"2023-10-06T04:11:24.447558+0000","last_deep_scrub":"53'528","last_deep_scrub_stamp":"2023-10-06T04:11:24.447558+0000","last_clean_scrub_stamp":"2023-10-06T04:11:24.447558+0000","objects_scrubbed":167,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:44.722933+0000","scrub_duration":0.104800032,"objects_trimmed":0,"snaptrim_duration":0.0020974879999999998,"stat_sum":{"num_bytes":5860015,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":34,"num_read_kb":17,"num_write":331,"num_write_kb":9992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"53'517","reported_seq":2642,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:22.576927+0000","last_change":"2023-10-06T04:11:22.576904+0000","last_active":"2023-10-06T04:11:22.576927+0000","last_peered":"2023-10-06T04:11:22.576927+0000","last_clean":"2023-10-06T04:11:22.576927+0000","last_became_active":"2023-10-06T03:57:11.362128+0000","last_became_peered":"2023-10-06T03:57:11.362128+0000","last_unstale":"2023-10-06T04:11:22.576927+0000","last_undegraded":"2023-10-06T04:11:22.576927+0000","last_fullsized":"2023-10-06T04:11:22.576927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'517","last_scrub_stamp":"2023-10-06T04:11:22.576836+0000","last_deep_scrub":"53'517","last_deep_scrub_stamp":"2023-10-06T04:11:22.576836+0000","last_clean_scrub_stamp":"2023-10-06T04:11:22.576836+0000","objects_scrubbed":164,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:20.021257+0000","scrub_duration":0.10803399599999999,"objects_trimmed":0,"snaptrim_duration":0.0025174860000000002,"stat_sum":{"num_bytes":10052289,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":42,"num_read_kb":8214,"num_write":324,"num_write_kb":14077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"53'551","reported_seq":2683,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:40.166837+0000","last_change":"2023-10-06T04:11:40.166804+0000","last_active":"2023-10-06T04:11:40.166837+0000","last_peered":"2023-10-06T04:11:40.166837+0000","last_clean":"2023-10-06T04:11:40.166837+0000","last_became_active":"2023-10-06T03:57:11.375702+0000","last_became_peered":"2023-10-06T03:57:11.375702+0000","last_unstale":"2023-10-06T04:11:40.166837+0000","last_undegraded":"2023-10-06T04:11:40.166837+0000","last_fullsized":"2023-10-06T04:11:40.166837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'551","last_scrub_stamp":"2023-10-06T04:11:40.166664+0000","last_deep_scrub":"53'551","last_deep_scrub_stamp":"2023-10-06T04:11:40.166664+0000","last_clean_scrub_stamp":"2023-10-06T04:11:40.166664+0000","objects_scrubbed":175,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:54.238456+0000","scrub_duration":0.155582093,"objects_trimmed":0,"snaptrim_duration":0.0010337009999999999,"stat_sum":{"num_bytes":19987419,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":47,"num_read_kb":12104,"num_write":347,"num_write_kb":19698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"53'485","reported_seq":2598,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:40.945819+0000","last_change":"2023-10-06T04:11:40.945793+0000","last_active":"2023-10-06T04:11:40.945819+0000","last_peered":"2023-10-06T04:11:40.945819+0000","last_clean":"2023-10-06T04:11:40.945819+0000","last_became_active":"2023-10-06T03:57:11.374842+0000","last_became_peered":"2023-10-06T03:57:11.374842+0000","last_unstale":"2023-10-06T04:11:40.945819+0000","last_undegraded":"2023-10-06T04:11:40.945819+0000","last_fullsized":"2023-10-06T04:11:40.945819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'485","last_scrub_stamp":"2023-10-06T04:11:40.945726+0000","last_deep_scrub":"53'485","last_deep_scrub_stamp":"2023-10-06T04:11:40.945726+0000","last_clean_scrub_stamp":"2023-10-06T04:11:40.945726+0000","objects_scrubbed":162,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:15.336651+0000","scrub_duration":0.114962457,"objects_trimmed":0,"snaptrim_duration":0.001332342,"stat_sum":{"num_bytes":5811346,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":29,"num_read_kb":4115,"num_write":308,"num_write_kb":5844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"53'564","reported_seq":2681,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:36.940922+0000","last_change":"2023-10-06T04:11:36.940898+0000","last_active":"2023-10-06T04:11:36.940922+0000","last_peered":"2023-10-06T04:11:36.940922+0000","last_clean":"2023-10-06T04:11:36.940922+0000","last_became_active":"2023-10-06T03:57:11.361349+0000","last_became_peered":"2023-10-06T03:57:11.361349+0000","last_unstale":"2023-10-06T04:11:36.940922+0000","last_undegraded":"2023-10-06T04:11:36.940922+0000","last_fullsized":"2023-10-06T04:11:36.940922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'564","last_scrub_stamp":"2023-10-06T04:11:36.940826+0000","last_deep_scrub":"53'564","last_deep_scrub_stamp":"2023-10-06T04:11:36.940826+0000","last_clean_scrub_stamp":"2023-10-06T04:11:36.940826+0000","objects_scrubbed":171,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:06.680252+0000","scrub_duration":0.13662444100000001,"objects_trimmed":0,"snaptrim_duration":0.0014394589999999999,"stat_sum":{"num_bytes":23015133,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":33,"num_read_kb":4115,"num_write":357,"num_write_kb":22663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"53'498","reported_seq":2620,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:44.731913+0000","last_change":"2023-10-06T04:11:44.731890+0000","last_active":"2023-10-06T04:11:44.731913+0000","last_peered":"2023-10-06T04:11:44.731913+0000","last_clean":"2023-10-06T04:11:44.731913+0000","last_became_active":"2023-10-06T03:57:11.375846+0000","last_became_peered":"2023-10-06T03:57:11.375846+0000","last_unstale":"2023-10-06T04:11:44.731913+0000","last_undegraded":"2023-10-06T04:11:44.731913+0000","last_fullsized":"2023-10-06T04:11:44.731913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'498","last_scrub_stamp":"2023-10-06T04:11:44.731820+0000","last_deep_scrub":"53'498","last_deep_scrub_stamp":"2023-10-06T04:11:44.731820+0000","last_clean_scrub_stamp":"2023-10-06T04:11:44.731820+0000","objects_scrubbed":170,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:45.326987+0000","scrub_duration":0.16816954100000001,"objects_trimmed":0,"snaptrim_duration":0.0018480599999999999,"stat_sum":{"num_bytes":26318621,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":38,"num_read_kb":12302,"num_write":306,"num_write_kb":25858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"53'462","reported_seq":528,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:25.963693+0000","last_change":"2023-10-06T04:11:25.905654+0000","last_active":"2023-10-06T04:11:25.963693+0000","last_peered":"2023-10-06T04:11:25.963693+0000","last_clean":"2023-10-06T04:11:25.963693+0000","last_became_active":"2023-10-06T03:57:10.348078+0000","last_became_peered":"2023-10-06T03:57:10.348078+0000","last_unstale":"2023-10-06T04:11:25.963693+0000","last_undegraded":"2023-10-06T04:11:25.963693+0000","last_fullsized":"2023-10-06T04:11:25.963693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'460","last_scrub_stamp":"2023-10-06T04:11:25.905554+0000","last_deep_scrub":"53'460","last_deep_scrub_stamp":"2023-10-06T04:11:25.905554+0000","last_clean_scrub_stamp":"2023-10-06T04:11:25.905554+0000","objects_scrubbed":11,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:22.245460+0000","scrub_duration":0.035131687000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15927542,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":13,"num_read_kb":12332,"num_write":447,"num_write_kb":15804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30320,"num_omap_keys":54,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"53'477","reported_seq":2599,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:17.713507+0000","last_change":"2023-10-06T04:11:17.713485+0000","last_active":"2023-10-06T04:11:17.713507+0000","last_peered":"2023-10-06T04:11:17.713507+0000","last_clean":"2023-10-06T04:11:17.713507+0000","last_became_active":"2023-10-06T03:57:11.370245+0000","last_became_peered":"2023-10-06T03:57:11.370245+0000","last_unstale":"2023-10-06T04:11:17.713507+0000","last_undegraded":"2023-10-06T04:11:17.713507+0000","last_fullsized":"2023-10-06T04:11:17.713507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'477","last_scrub_stamp":"2023-10-06T04:11:17.713414+0000","last_deep_scrub":"53'477","last_deep_scrub_stamp":"2023-10-06T04:11:17.713414+0000","last_clean_scrub_stamp":"2023-10-06T04:11:17.713414+0000","objects_scrubbed":162,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:09.132306+0000","scrub_duration":0.116172479,"objects_trimmed":0,"snaptrim_duration":0.002348424,"stat_sum":{"num_bytes":22265950,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":38,"num_read_kb":9495,"num_write":317,"num_write_kb":26009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"53'95","reported_seq":185,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:19.796378+0000","last_change":"2023-10-06T04:11:19.796356+0000","last_active":"2023-10-06T04:11:19.796378+0000","last_peered":"2023-10-06T04:11:19.796378+0000","last_clean":"2023-10-06T04:11:19.796378+0000","last_became_active":"2023-10-06T03:57:10.345150+0000","last_became_peered":"2023-10-06T03:57:10.345150+0000","last_unstale":"2023-10-06T04:11:19.796378+0000","last_undegraded":"2023-10-06T04:11:19.796378+0000","last_fullsized":"2023-10-06T04:11:19.796378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'95","last_scrub_stamp":"2023-10-06T04:11:19.796288+0000","last_deep_scrub":"53'95","last_deep_scrub_stamp":"2023-10-06T04:11:19.796288+0000","last_clean_scrub_stamp":"2023-10-06T04:11:19.796288+0000","objects_scrubbed":6,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:50.621259+0000","scrub_duration":0.0099644750000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4312676,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":146,"num_write":89,"num_write_kb":4275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":27214,"num_omap_keys":50,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"53'509","reported_seq":2619,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:33.482977+0000","last_change":"2023-10-06T04:11:33.482954+0000","last_active":"2023-10-06T04:11:33.482977+0000","last_peered":"2023-10-06T04:11:33.482977+0000","last_clean":"2023-10-06T04:11:33.482977+0000","last_became_active":"2023-10-06T03:57:11.375790+0000","last_became_peered":"2023-10-06T03:57:11.375790+0000","last_unstale":"2023-10-06T04:11:33.482977+0000","last_undegraded":"2023-10-06T04:11:33.482977+0000","last_fullsized":"2023-10-06T04:11:33.482977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'509","last_scrub_stamp":"2023-10-06T04:11:33.482884+0000","last_deep_scrub":"53'509","last_deep_scrub_stamp":"2023-10-06T04:11:33.482884+0000","last_clean_scrub_stamp":"2023-10-06T04:11:33.482884+0000","objects_scrubbed":159,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:48.231531+0000","scrub_duration":0.127308381,"objects_trimmed":0,"snaptrim_duration":0.0025999780000000002,"stat_sum":{"num_bytes":19085062,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":26,"num_read_kb":12300,"num_write":331,"num_write_kb":18812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"53'1080","reported_seq":1140,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:10.608190+0000","last_change":"2023-10-06T04:11:10.608166+0000","last_active":"2023-10-06T04:11:10.608190+0000","last_peered":"2023-10-06T04:11:10.608190+0000","last_clean":"2023-10-06T04:11:10.608190+0000","last_became_active":"2023-10-06T03:57:10.347294+0000","last_became_peered":"2023-10-06T03:57:10.347294+0000","last_unstale":"2023-10-06T04:11:10.608190+0000","last_undegraded":"2023-10-06T04:11:10.608190+0000","last_fullsized":"2023-10-06T04:11:10.608190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'1080","last_scrub_stamp":"2023-10-06T04:11:10.608093+0000","last_deep_scrub":"53'1080","last_deep_scrub_stamp":"2023-10-06T04:11:10.608093+0000","last_clean_scrub_stamp":"2023-10-06T04:11:10.608093+0000","objects_scrubbed":8,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:54.093395+0000","scrub_duration":0.023937142000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583466,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":7,"num_read_kb":16784,"num_write":1067,"num_write_kb":17022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30102,"num_omap_keys":52,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"53'457","reported_seq":2576,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:16.190235+0000","last_change":"2023-10-06T04:11:16.190201+0000","last_active":"2023-10-06T04:11:16.190235+0000","last_peered":"2023-10-06T04:11:16.190235+0000","last_clean":"2023-10-06T04:11:16.190235+0000","last_became_active":"2023-10-06T03:57:11.371758+0000","last_became_peered":"2023-10-06T03:57:11.371758+0000","last_unstale":"2023-10-06T04:11:16.190235+0000","last_undegraded":"2023-10-06T04:11:16.190235+0000","last_fullsized":"2023-10-06T04:11:16.190235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'457","last_scrub_stamp":"2023-10-06T04:11:16.190090+0000","last_deep_scrub":"53'457","last_deep_scrub_stamp":"2023-10-06T04:11:16.190090+0000","last_clean_scrub_stamp":"2023-10-06T04:11:16.190090+0000","objects_scrubbed":153,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:59.896644+0000","scrub_duration":0.12942683099999999,"objects_trimmed":0,"snaptrim_duration":0.0017888349999999999,"stat_sum":{"num_bytes":16888108,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":35,"num_read_kb":4115,"num_write":349,"num_write_kb":16624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"53'54","reported_seq":113,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:05.959897+0000","last_change":"2023-10-06T04:11:02.285761+0000","last_active":"2023-10-06T04:11:05.959897+0000","last_peered":"2023-10-06T04:11:05.959897+0000","last_clean":"2023-10-06T04:11:05.959897+0000","last_became_active":"2023-10-06T03:57:10.347151+0000","last_became_peered":"2023-10-06T03:57:10.347151+0000","last_unstale":"2023-10-06T04:11:05.959897+0000","last_undegraded":"2023-10-06T04:11:05.959897+0000","last_fullsized":"2023-10-06T04:11:05.959897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'52","last_scrub_stamp":"2023-10-06T04:11:02.285649+0000","last_deep_scrub":"53'52","last_deep_scrub_stamp":"2023-10-06T04:11:02.285649+0000","last_clean_scrub_stamp":"2023-10-06T04:11:02.285649+0000","objects_scrubbed":7,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:44.621007+0000","scrub_duration":0.0062605860000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":52,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":37322,"num_omap_keys":65,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"53'494","reported_seq":2614,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:21.604835+0000","last_change":"2023-10-06T04:11:21.604798+0000","last_active":"2023-10-06T04:11:21.604835+0000","last_peered":"2023-10-06T04:11:21.604835+0000","last_clean":"2023-10-06T04:11:21.604835+0000","last_became_active":"2023-10-06T03:57:11.369938+0000","last_became_peered":"2023-10-06T03:57:11.369938+0000","last_unstale":"2023-10-06T04:11:21.604835+0000","last_undegraded":"2023-10-06T04:11:21.604835+0000","last_fullsized":"2023-10-06T04:11:21.604835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'494","last_scrub_stamp":"2023-10-06T04:11:21.604691+0000","last_deep_scrub":"53'494","last_deep_scrub_stamp":"2023-10-06T04:11:21.604691+0000","last_clean_scrub_stamp":"2023-10-06T04:11:21.604691+0000","objects_scrubbed":172,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:46.110446+0000","scrub_duration":0.142007095,"objects_trimmed":0,"snaptrim_duration":0.0018579569999999999,"stat_sum":{"num_bytes":31490698,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":37,"num_read_kb":12302,"num_write":314,"num_write_kb":31163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"53'53","reported_seq":111,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:14.740559+0000","last_change":"2023-10-06T04:11:14.740529+0000","last_active":"2023-10-06T04:11:14.740559+0000","last_peered":"2023-10-06T04:11:14.740559+0000","last_clean":"2023-10-06T04:11:14.740559+0000","last_became_active":"2023-10-06T03:57:10.345721+0000","last_became_peered":"2023-10-06T03:57:10.345721+0000","last_unstale":"2023-10-06T04:11:14.740559+0000","last_undegraded":"2023-10-06T04:11:14.740559+0000","last_fullsized":"2023-10-06T04:11:14.740559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'53","last_scrub_stamp":"2023-10-06T04:11:14.740442+0000","last_deep_scrub":"53'53","last_deep_scrub_stamp":"2023-10-06T04:11:14.740442+0000","last_clean_scrub_stamp":"2023-10-06T04:11:14.740442+0000","objects_scrubbed":4,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:37.780654+0000","scrub_duration":0.013645548,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":15,"num_write":48,"num_write_kb":4121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9078,"num_omap_keys":16,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"53'542","reported_seq":2660,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:23.510740+0000","last_change":"2023-10-06T04:11:23.510701+0000","last_active":"2023-10-06T04:11:23.510740+0000","last_peered":"2023-10-06T04:11:23.510740+0000","last_clean":"2023-10-06T04:11:23.510740+0000","last_became_active":"2023-10-06T03:57:59.255587+0000","last_became_peered":"2023-10-06T03:57:59.255587+0000","last_unstale":"2023-10-06T04:11:23.510740+0000","last_undegraded":"2023-10-06T04:11:23.510740+0000","last_fullsized":"2023-10-06T04:11:23.510740+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'542","last_scrub_stamp":"2023-10-06T04:11:23.510570+0000","last_deep_scrub":"53'542","last_deep_scrub_stamp":"2023-10-06T04:11:23.510570+0000","last_clean_scrub_stamp":"2023-10-06T04:11:23.510570+0000","objects_scrubbed":175,"log_size":542,"log_dups_size":0,"ondisk_log_size":542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:07:13.627016+0000","scrub_duration":0.14334406899999999,"objects_trimmed":0,"snaptrim_duration":0.001510024,"stat_sum":{"num_bytes":1845822,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":38,"num_read_kb":8209,"num_write":345,"num_write_kb":10179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"53'69","reported_seq":129,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:21.778145+0000","last_change":"2023-10-06T04:11:21.778119+0000","last_active":"2023-10-06T04:11:21.778145+0000","last_peered":"2023-10-06T04:11:21.778145+0000","last_clean":"2023-10-06T04:11:21.778145+0000","last_became_active":"2023-10-06T03:57:10.340232+0000","last_became_peered":"2023-10-06T03:57:10.340232+0000","last_unstale":"2023-10-06T04:11:21.778145+0000","last_undegraded":"2023-10-06T04:11:21.778145+0000","last_fullsized":"2023-10-06T04:11:21.778145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'69","last_scrub_stamp":"2023-10-06T04:11:21.778042+0000","last_deep_scrub":"53'69","last_deep_scrub_stamp":"2023-10-06T04:11:21.778042+0000","last_clean_scrub_stamp":"2023-10-06T04:11:21.778042+0000","objects_scrubbed":6,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:35:39.233114+0000","scrub_duration":0.011501937,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":66,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24992,"num_omap_keys":46,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"53'528","reported_seq":2649,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:37.973900+0000","last_change":"2023-10-06T04:11:37.973866+0000","last_active":"2023-10-06T04:11:37.973900+0000","last_peered":"2023-10-06T04:11:37.973900+0000","last_clean":"2023-10-06T04:11:37.973900+0000","last_became_active":"2023-10-06T03:57:11.374892+0000","last_became_peered":"2023-10-06T03:57:11.374892+0000","last_unstale":"2023-10-06T04:11:37.973900+0000","last_undegraded":"2023-10-06T04:11:37.973900+0000","last_fullsized":"2023-10-06T04:11:37.973900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'528","last_scrub_stamp":"2023-10-06T04:11:37.973741+0000","last_deep_scrub":"53'528","last_deep_scrub_stamp":"2023-10-06T04:11:37.973741+0000","last_clean_scrub_stamp":"2023-10-06T04:11:37.973741+0000","objects_scrubbed":164,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:25.227153+0000","scrub_duration":0.14730583799999999,"objects_trimmed":0,"snaptrim_duration":0.00156207,"stat_sum":{"num_bytes":17801071,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":38,"num_read_kb":8211,"num_write":336,"num_write_kb":25748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"53'680","reported_seq":748,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:19.900780+0000","last_change":"2023-10-06T04:11:19.900740+0000","last_active":"2023-10-06T04:11:19.900780+0000","last_peered":"2023-10-06T04:11:19.900780+0000","last_clean":"2023-10-06T04:11:19.900780+0000","last_became_active":"2023-10-06T03:57:10.346430+0000","last_became_peered":"2023-10-06T03:57:10.346430+0000","last_unstale":"2023-10-06T04:11:19.900780+0000","last_undegraded":"2023-10-06T04:11:19.900780+0000","last_fullsized":"2023-10-06T04:11:19.900780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'680","last_scrub_stamp":"2023-10-06T04:11:19.900666+0000","last_deep_scrub":"53'680","last_deep_scrub_stamp":"2023-10-06T04:11:19.900666+0000","last_clean_scrub_stamp":"2023-10-06T04:11:19.900666+0000","objects_scrubbed":9,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:55.003315+0000","scrub_duration":0.024294989999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389252,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":30,"num_read_kb":9887,"num_write":670,"num_write_kb":8556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":23944,"num_omap_keys":41,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"53'502","reported_seq":2623,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:45.069162+0000","last_change":"2023-10-06T04:11:45.069138+0000","last_active":"2023-10-06T04:11:45.069162+0000","last_peered":"2023-10-06T04:11:45.069162+0000","last_clean":"2023-10-06T04:11:45.069162+0000","last_became_active":"2023-10-06T03:57:11.367565+0000","last_became_peered":"2023-10-06T03:57:11.367565+0000","last_unstale":"2023-10-06T04:11:45.069162+0000","last_undegraded":"2023-10-06T04:11:45.069162+0000","last_fullsized":"2023-10-06T04:11:45.069162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'502","last_scrub_stamp":"2023-10-06T04:11:45.069070+0000","last_deep_scrub":"53'502","last_deep_scrub_stamp":"2023-10-06T04:11:45.069070+0000","last_clean_scrub_stamp":"2023-10-06T04:11:45.069070+0000","objects_scrubbed":148,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:03.118619+0000","scrub_duration":0.11968153099999999,"objects_trimmed":0,"snaptrim_duration":0.0027762680000000001,"stat_sum":{"num_bytes":10311229,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":38,"num_read_kb":23,"num_write":322,"num_write_kb":10236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"53'188","reported_seq":255,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:11.762952+0000","last_change":"2023-10-06T04:11:11.762926+0000","last_active":"2023-10-06T04:11:11.762952+0000","last_peered":"2023-10-06T04:11:11.762952+0000","last_clean":"2023-10-06T04:11:11.762952+0000","last_became_active":"2023-10-06T03:57:10.350795+0000","last_became_peered":"2023-10-06T03:57:10.350795+0000","last_unstale":"2023-10-06T04:11:11.762952+0000","last_undegraded":"2023-10-06T04:11:11.762952+0000","last_fullsized":"2023-10-06T04:11:11.762952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'188","last_scrub_stamp":"2023-10-06T04:11:11.762854+0000","last_deep_scrub":"53'188","last_deep_scrub_stamp":"2023-10-06T04:11:11.762854+0000","last_clean_scrub_stamp":"2023-10-06T04:11:11.762854+0000","objects_scrubbed":9,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:37.116115+0000","scrub_duration":0.020924005999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388676,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":30,"num_read_kb":20509,"num_write":184,"num_write_kb":8306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":18896,"num_omap_keys":34,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"53'406","reported_seq":2522,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:14.748333+0000","last_change":"2023-10-06T04:11:14.748308+0000","last_active":"2023-10-06T04:11:14.748333+0000","last_peered":"2023-10-06T04:11:14.748333+0000","last_clean":"2023-10-06T04:11:14.748333+0000","last_became_active":"2023-10-06T03:57:11.367254+0000","last_became_peered":"2023-10-06T03:57:11.367254+0000","last_unstale":"2023-10-06T04:11:14.748333+0000","last_undegraded":"2023-10-06T04:11:14.748333+0000","last_fullsized":"2023-10-06T04:11:14.748333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'406","last_scrub_stamp":"2023-10-06T04:11:14.748233+0000","last_deep_scrub":"53'406","last_deep_scrub_stamp":"2023-10-06T04:11:14.748233+0000","last_clean_scrub_stamp":"2023-10-06T04:11:14.748233+0000","objects_scrubbed":165,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:29.852108+0000","scrub_duration":0.096980769999999994,"objects_trimmed":0,"snaptrim_duration":0.0027822569999999998,"stat_sum":{"num_bytes":13704260,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":32,"num_read_kb":10530,"num_write":279,"num_write_kb":17631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"53'111","reported_seq":174,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:04.279658+0000","last_change":"2023-10-06T04:11:04.279631+0000","last_active":"2023-10-06T04:11:04.279658+0000","last_peered":"2023-10-06T04:11:04.279658+0000","last_clean":"2023-10-06T04:11:04.279658+0000","last_became_active":"2023-10-06T03:57:10.337798+0000","last_became_peered":"2023-10-06T03:57:10.337798+0000","last_unstale":"2023-10-06T04:11:04.279658+0000","last_undegraded":"2023-10-06T04:11:04.279658+0000","last_fullsized":"2023-10-06T04:11:04.279658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'111","last_scrub_stamp":"2023-10-06T04:11:04.279550+0000","last_deep_scrub":"53'111","last_deep_scrub_stamp":"2023-10-06T04:11:04.279550+0000","last_clean_scrub_stamp":"2023-10-06T04:11:04.279550+0000","objects_scrubbed":6,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:39.866223+0000","scrub_duration":0.015622552,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":9753,"num_write":104,"num_write_kb":4165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":14386,"num_omap_keys":26,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"53'528","reported_seq":2649,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:43.776931+0000","last_change":"2023-10-06T04:11:43.776908+0000","last_active":"2023-10-06T04:11:43.776931+0000","last_peered":"2023-10-06T04:11:43.776931+0000","last_clean":"2023-10-06T04:11:43.776931+0000","last_became_active":"2023-10-06T03:57:11.376223+0000","last_became_peered":"2023-10-06T03:57:11.376223+0000","last_unstale":"2023-10-06T04:11:43.776931+0000","last_undegraded":"2023-10-06T04:11:43.776931+0000","last_fullsized":"2023-10-06T04:11:43.776931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'528","last_scrub_stamp":"2023-10-06T04:11:43.776827+0000","last_deep_scrub":"53'528","last_deep_scrub_stamp":"2023-10-06T04:11:43.776827+0000","last_clean_scrub_stamp":"2023-10-06T04:11:43.776827+0000","objects_scrubbed":172,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:46:18.373845+0000","scrub_duration":0.17515610600000001,"objects_trimmed":0,"snaptrim_duration":0.0018161119999999999,"stat_sum":{"num_bytes":34940720,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":37,"num_read_kb":11570,"num_write":336,"num_write_kb":34292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"53'135","reported_seq":188,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:25.960222+0000","last_change":"2023-10-06T04:11:10.713437+0000","last_active":"2023-10-06T04:11:25.960222+0000","last_peered":"2023-10-06T04:11:25.960222+0000","last_clean":"2023-10-06T04:11:25.960222+0000","last_became_active":"2023-10-06T03:57:10.350219+0000","last_became_peered":"2023-10-06T03:57:10.350219+0000","last_unstale":"2023-10-06T04:11:25.960222+0000","last_undegraded":"2023-10-06T04:11:25.960222+0000","last_fullsized":"2023-10-06T04:11:25.960222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'126","last_scrub_stamp":"2023-10-06T04:11:10.713339+0000","last_deep_scrub":"53'126","last_deep_scrub_stamp":"2023-10-06T04:11:10.713339+0000","last_clean_scrub_stamp":"2023-10-06T04:11:10.713339+0000","objects_scrubbed":8,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:04.717440+0000","scrub_duration":0.020180555999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4909910,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":9008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":51900,"num_omap_keys":89,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"53'535","reported_seq":2664,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:38.507456+0000","last_change":"2023-10-06T04:11:38.507435+0000","last_active":"2023-10-06T04:11:38.507456+0000","last_peered":"2023-10-06T04:11:38.507456+0000","last_clean":"2023-10-06T04:11:38.507456+0000","last_became_active":"2023-10-06T03:57:11.362443+0000","last_became_peered":"2023-10-06T03:57:11.362443+0000","last_unstale":"2023-10-06T04:11:38.507456+0000","last_undegraded":"2023-10-06T04:11:38.507456+0000","last_fullsized":"2023-10-06T04:11:38.507456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'535","last_scrub_stamp":"2023-10-06T04:11:38.507361+0000","last_deep_scrub":"53'535","last_deep_scrub_stamp":"2023-10-06T04:11:38.507361+0000","last_clean_scrub_stamp":"2023-10-06T04:11:38.507361+0000","objects_scrubbed":154,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:06:02.993776+0000","scrub_duration":0.11848410199999999,"objects_trimmed":0,"snaptrim_duration":0.002448839,"stat_sum":{"num_bytes":22978165,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":42,"num_read_kb":8288,"num_write":356,"num_write_kb":22628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"53'83","reported_seq":147,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:08.057909+0000","last_change":"2023-10-06T04:11:08.057878+0000","last_active":"2023-10-06T04:11:08.057909+0000","last_peered":"2023-10-06T04:11:08.057909+0000","last_clean":"2023-10-06T04:11:08.057909+0000","last_became_active":"2023-10-06T03:57:10.345424+0000","last_became_peered":"2023-10-06T03:57:10.345424+0000","last_unstale":"2023-10-06T04:11:08.057909+0000","last_undegraded":"2023-10-06T04:11:08.057909+0000","last_fullsized":"2023-10-06T04:11:08.057909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'83","last_scrub_stamp":"2023-10-06T04:11:08.057782+0000","last_deep_scrub":"53'83","last_deep_scrub_stamp":"2023-10-06T04:11:08.057782+0000","last_clean_scrub_stamp":"2023-10-06T04:11:08.057782+0000","objects_scrubbed":12,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:41:48.011373+0000","scrub_duration":0.0096812489999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":33,"num_read_kb":33,"num_write":72,"num_write_kb":4158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31156,"num_omap_keys":54,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"53'419","reported_seq":2541,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:12.728194+0000","last_change":"2023-10-06T04:11:12.728172+0000","last_active":"2023-10-06T04:11:12.728194+0000","last_peered":"2023-10-06T04:11:12.728194+0000","last_clean":"2023-10-06T04:11:12.728194+0000","last_became_active":"2023-10-06T03:57:11.375824+0000","last_became_peered":"2023-10-06T03:57:11.375824+0000","last_unstale":"2023-10-06T04:11:12.728194+0000","last_undegraded":"2023-10-06T04:11:12.728194+0000","last_fullsized":"2023-10-06T04:11:12.728194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'419","last_scrub_stamp":"2023-10-06T04:11:12.728105+0000","last_deep_scrub":"53'419","last_deep_scrub_stamp":"2023-10-06T04:11:12.728105+0000","last_clean_scrub_stamp":"2023-10-06T04:11:12.728105+0000","objects_scrubbed":150,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:57.027413+0000","scrub_duration":0.083998824,"objects_trimmed":0,"snaptrim_duration":0.0023200299999999998,"stat_sum":{"num_bytes":9857160,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":38,"num_read_kb":8212,"num_write":304,"num_write_kb":9779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"53'256","reported_seq":312,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:20.763949+0000","last_change":"2023-10-06T04:11:20.763919+0000","last_active":"2023-10-06T04:11:20.763949+0000","last_peered":"2023-10-06T04:11:20.763949+0000","last_clean":"2023-10-06T04:11:20.763949+0000","last_became_active":"2023-10-06T03:57:10.350135+0000","last_became_peered":"2023-10-06T03:57:10.350135+0000","last_unstale":"2023-10-06T04:11:20.763949+0000","last_undegraded":"2023-10-06T04:11:20.763949+0000","last_fullsized":"2023-10-06T04:11:20.763949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'256","last_scrub_stamp":"2023-10-06T04:11:20.763831+0000","last_deep_scrub":"53'256","last_deep_scrub_stamp":"2023-10-06T04:11:20.763831+0000","last_clean_scrub_stamp":"2023-10-06T04:11:20.763831+0000","objects_scrubbed":6,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:26.714475+0000","scrub_duration":0.01459685,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":4098,"num_write":280,"num_write_kb":4293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":28994,"num_omap_keys":51,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"53'511","reported_seq":2627,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:30.698925+0000","last_change":"2023-10-06T04:11:30.698890+0000","last_active":"2023-10-06T04:11:30.698925+0000","last_peered":"2023-10-06T04:11:30.698925+0000","last_clean":"2023-10-06T04:11:30.698925+0000","last_became_active":"2023-10-06T03:57:11.376236+0000","last_became_peered":"2023-10-06T03:57:11.376236+0000","last_unstale":"2023-10-06T04:11:30.698925+0000","last_undegraded":"2023-10-06T04:11:30.698925+0000","last_fullsized":"2023-10-06T04:11:30.698925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'511","last_scrub_stamp":"2023-10-06T04:11:30.698814+0000","last_deep_scrub":"53'511","last_deep_scrub_stamp":"2023-10-06T04:11:30.698814+0000","last_clean_scrub_stamp":"2023-10-06T04:11:30.698814+0000","objects_scrubbed":170,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:58.790004+0000","scrub_duration":0.157753385,"objects_trimmed":0,"snaptrim_duration":0.0015573080000000001,"stat_sum":{"num_bytes":22935537,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":32,"num_read_kb":10317,"num_write":317,"num_write_kb":26655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"53'95","reported_seq":153,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:18.759540+0000","last_change":"2023-10-06T04:11:18.759511+0000","last_active":"2023-10-06T04:11:18.759540+0000","last_peered":"2023-10-06T04:11:18.759540+0000","last_clean":"2023-10-06T04:11:18.759540+0000","last_became_active":"2023-10-06T03:57:10.345510+0000","last_became_peered":"2023-10-06T03:57:10.345510+0000","last_unstale":"2023-10-06T04:11:18.759540+0000","last_undegraded":"2023-10-06T04:11:18.759540+0000","last_fullsized":"2023-10-06T04:11:18.759540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'95","last_scrub_stamp":"2023-10-06T04:11:18.759410+0000","last_deep_scrub":"53'95","last_deep_scrub_stamp":"2023-10-06T04:11:18.759410+0000","last_clean_scrub_stamp":"2023-10-06T04:11:18.759410+0000","objects_scrubbed":3,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:29.676193+0000","scrub_duration":0.014631524999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":4108,"num_write":90,"num_write_kb":4140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"53'519","reported_seq":2640,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:51.273470+0000","last_change":"2023-10-06T04:11:51.273447+0000","last_active":"2023-10-06T04:11:51.273470+0000","last_peered":"2023-10-06T04:11:51.273470+0000","last_clean":"2023-10-06T04:11:51.273470+0000","last_became_active":"2023-10-06T03:57:11.368375+0000","last_became_peered":"2023-10-06T03:57:11.368375+0000","last_unstale":"2023-10-06T04:11:51.273470+0000","last_undegraded":"2023-10-06T04:11:51.273470+0000","last_fullsized":"2023-10-06T04:11:51.273470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'519","last_scrub_stamp":"2023-10-06T04:11:51.273376+0000","last_deep_scrub":"53'519","last_deep_scrub_stamp":"2023-10-06T04:11:51.273376+0000","last_clean_scrub_stamp":"2023-10-06T04:11:51.273376+0000","objects_scrubbed":160,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:24.271253+0000","scrub_duration":0.115544565,"objects_trimmed":0,"snaptrim_duration":0.001632912,"stat_sum":{"num_bytes":5657073,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":37,"num_read_kb":4115,"num_write":343,"num_write_kb":5657,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"53'169","reported_seq":231,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:45.922091+0000","last_change":"2023-10-06T04:11:45.922063+0000","last_active":"2023-10-06T04:11:45.922091+0000","last_peered":"2023-10-06T04:11:45.922091+0000","last_clean":"2023-10-06T04:11:45.922091+0000","last_became_active":"2023-10-06T03:57:10.346689+0000","last_became_peered":"2023-10-06T03:57:10.346689+0000","last_unstale":"2023-10-06T04:11:45.922091+0000","last_undegraded":"2023-10-06T04:11:45.922091+0000","last_fullsized":"2023-10-06T04:11:45.922091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'169","last_scrub_stamp":"2023-10-06T04:11:45.921982+0000","last_deep_scrub":"53'169","last_deep_scrub_stamp":"2023-10-06T04:11:45.921982+0000","last_clean_scrub_stamp":"2023-10-06T04:11:45.921982+0000","objects_scrubbed":5,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:33.120676+0000","scrub_duration":0.014444475,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":12290,"num_write":163,"num_write_kb":8336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15184,"num_omap_keys":26,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"53'471","reported_seq":2589,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:13.079557+0000","last_change":"2023-10-06T04:11:13.079535+0000","last_active":"2023-10-06T04:11:13.079557+0000","last_peered":"2023-10-06T04:11:13.079557+0000","last_clean":"2023-10-06T04:11:13.079557+0000","last_became_active":"2023-10-06T03:57:11.371795+0000","last_became_peered":"2023-10-06T03:57:11.371795+0000","last_unstale":"2023-10-06T04:11:13.079557+0000","last_undegraded":"2023-10-06T04:11:13.079557+0000","last_fullsized":"2023-10-06T04:11:13.079557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'471","last_scrub_stamp":"2023-10-06T04:11:13.079467+0000","last_deep_scrub":"53'471","last_deep_scrub_stamp":"2023-10-06T04:11:13.079467+0000","last_clean_scrub_stamp":"2023-10-06T04:11:13.079467+0000","objects_scrubbed":169,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:17.052127+0000","scrub_duration":0.080011451999999997,"objects_trimmed":0,"snaptrim_duration":0.0016252980000000001,"stat_sum":{"num_bytes":5774867,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":34,"num_read_kb":13,"num_write":355,"num_write_kb":5825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"53'59","reported_seq":112,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:15.755241+0000","last_change":"2023-10-06T04:11:15.755204+0000","last_active":"2023-10-06T04:11:15.755241+0000","last_peered":"2023-10-06T04:11:15.755241+0000","last_clean":"2023-10-06T04:11:15.755241+0000","last_became_active":"2023-10-06T03:57:10.348291+0000","last_became_peered":"2023-10-06T03:57:10.348291+0000","last_unstale":"2023-10-06T04:11:15.755241+0000","last_undegraded":"2023-10-06T04:11:15.755241+0000","last_fullsized":"2023-10-06T04:11:15.755241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'59","last_scrub_stamp":"2023-10-06T04:11:15.755103+0000","last_deep_scrub":"53'59","last_deep_scrub_stamp":"2023-10-06T04:11:15.755103+0000","last_clean_scrub_stamp":"2023-10-06T04:11:15.755103+0000","objects_scrubbed":5,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:27.967550+0000","scrub_duration":0.0073040559999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":57,"num_write_kb":4195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":20734,"num_omap_keys":37,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"53'480","reported_seq":2597,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:30.531095+0000","last_change":"2023-10-06T04:11:30.531072+0000","last_active":"2023-10-06T04:11:30.531095+0000","last_peered":"2023-10-06T04:11:30.531095+0000","last_clean":"2023-10-06T04:11:30.531095+0000","last_became_active":"2023-10-06T03:57:11.365077+0000","last_became_peered":"2023-10-06T03:57:11.365077+0000","last_unstale":"2023-10-06T04:11:30.531095+0000","last_undegraded":"2023-10-06T04:11:30.531095+0000","last_fullsized":"2023-10-06T04:11:30.531095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'480","last_scrub_stamp":"2023-10-06T04:11:30.531002+0000","last_deep_scrub":"53'480","last_deep_scrub_stamp":"2023-10-06T04:11:30.531002+0000","last_clean_scrub_stamp":"2023-10-06T04:11:30.531002+0000","objects_scrubbed":152,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:08.827420+0000","scrub_duration":0.107887435,"objects_trimmed":0,"snaptrim_duration":0.0019352270000000001,"stat_sum":{"num_bytes":15068515,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":33,"num_read_kb":4109,"num_write":308,"num_write_kb":14877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"53'283","reported_seq":345,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:01.636104+0000","last_change":"2023-10-06T04:11:01.636081+0000","last_active":"2023-10-06T04:11:01.636104+0000","last_peered":"2023-10-06T04:11:01.636104+0000","last_clean":"2023-10-06T04:11:01.636104+0000","last_became_active":"2023-10-06T03:57:10.348055+0000","last_became_peered":"2023-10-06T03:57:10.348055+0000","last_unstale":"2023-10-06T04:11:01.636104+0000","last_undegraded":"2023-10-06T04:11:01.636104+0000","last_fullsized":"2023-10-06T04:11:01.636104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'283","last_scrub_stamp":"2023-10-06T04:11:01.636008+0000","last_deep_scrub":"53'283","last_deep_scrub_stamp":"2023-10-06T04:11:01.636008+0000","last_clean_scrub_stamp":"2023-10-06T04:11:01.636008+0000","objects_scrubbed":5,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:11.936218+0000","scrub_duration":0.017511342999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":17,"num_read_kb":12301,"num_write":272,"num_write_kb":8316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"53'496","reported_seq":2609,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:11.948075+0000","last_change":"2023-10-06T04:11:11.948042+0000","last_active":"2023-10-06T04:11:11.948075+0000","last_peered":"2023-10-06T04:11:11.948075+0000","last_clean":"2023-10-06T04:11:11.948075+0000","last_became_active":"2023-10-06T03:57:11.370073+0000","last_became_peered":"2023-10-06T03:57:11.370073+0000","last_unstale":"2023-10-06T04:11:11.948075+0000","last_undegraded":"2023-10-06T04:11:11.948075+0000","last_fullsized":"2023-10-06T04:11:11.948075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'496","last_scrub_stamp":"2023-10-06T04:11:11.947965+0000","last_deep_scrub":"53'496","last_deep_scrub_stamp":"2023-10-06T04:11:11.947965+0000","last_clean_scrub_stamp":"2023-10-06T04:11:11.947965+0000","objects_scrubbed":185,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:02.104713+0000","scrub_duration":0.114303379,"objects_trimmed":0,"snaptrim_duration":0.0027454179999999999,"stat_sum":{"num_bytes":18058404,"num_objects":185,"num_object_clones":0,"num_object_copies":370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":185,"num_whiteouts":0,"num_read":28,"num_read_kb":4110,"num_write":345,"num_write_kb":17816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"53'118","reported_seq":176,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:09.093158+0000","last_change":"2023-10-06T04:11:09.093116+0000","last_active":"2023-10-06T04:11:09.093158+0000","last_peered":"2023-10-06T04:11:09.093158+0000","last_clean":"2023-10-06T04:11:09.093158+0000","last_became_active":"2023-10-06T03:57:10.352653+0000","last_became_peered":"2023-10-06T03:57:10.352653+0000","last_unstale":"2023-10-06T04:11:09.093158+0000","last_undegraded":"2023-10-06T04:11:09.093158+0000","last_fullsized":"2023-10-06T04:11:09.093158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'118","last_scrub_stamp":"2023-10-06T04:11:09.092974+0000","last_deep_scrub":"53'118","last_deep_scrub_stamp":"2023-10-06T04:11:09.092974+0000","last_clean_scrub_stamp":"2023-10-06T04:11:09.092974+0000","objects_scrubbed":6,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:03.839458+0000","scrub_duration":0.016085988999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":8,"num_read_kb":4217,"num_write":207,"num_write_kb":4775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":18688,"num_omap_keys":32,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"53'520","reported_seq":2636,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:35.299212+0000","last_change":"2023-10-06T04:11:35.299190+0000","last_active":"2023-10-06T04:11:35.299212+0000","last_peered":"2023-10-06T04:11:35.299212+0000","last_clean":"2023-10-06T04:11:35.299212+0000","last_became_active":"2023-10-06T03:57:11.363830+0000","last_became_peered":"2023-10-06T03:57:11.363830+0000","last_unstale":"2023-10-06T04:11:35.299212+0000","last_undegraded":"2023-10-06T04:11:35.299212+0000","last_fullsized":"2023-10-06T04:11:35.299212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'520","last_scrub_stamp":"2023-10-06T04:11:35.299124+0000","last_deep_scrub":"53'520","last_deep_scrub_stamp":"2023-10-06T04:11:35.299124+0000","last_clean_scrub_stamp":"2023-10-06T04:11:35.299124+0000","objects_scrubbed":162,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:32.020653+0000","scrub_duration":0.348993265,"objects_trimmed":0,"snaptrim_duration":0.0027698559999999998,"stat_sum":{"num_bytes":22869017,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":32,"num_read_kb":4115,"num_write":333,"num_write_kb":26610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"53'113","reported_seq":174,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:14.850284+0000","last_change":"2023-10-06T04:11:14.850257+0000","last_active":"2023-10-06T04:11:14.850284+0000","last_peered":"2023-10-06T04:11:14.850284+0000","last_clean":"2023-10-06T04:11:14.850284+0000","last_became_active":"2023-10-06T03:57:10.349114+0000","last_became_peered":"2023-10-06T03:57:10.349114+0000","last_unstale":"2023-10-06T04:11:14.850284+0000","last_undegraded":"2023-10-06T04:11:14.850284+0000","last_fullsized":"2023-10-06T04:11:14.850284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'113","last_scrub_stamp":"2023-10-06T04:11:14.850170+0000","last_deep_scrub":"53'113","last_deep_scrub_stamp":"2023-10-06T04:11:14.850170+0000","last_clean_scrub_stamp":"2023-10-06T04:11:14.850170+0000","objects_scrubbed":8,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:59.463057+0000","scrub_duration":0.012734528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":19,"num_read_kb":18,"num_write":109,"num_write_kb":4188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":33482,"num_omap_keys":60,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"53'393","reported_seq":2501,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:07.012391+0000","last_change":"2023-10-06T04:11:07.012369+0000","last_active":"2023-10-06T04:11:07.012391+0000","last_peered":"2023-10-06T04:11:07.012391+0000","last_clean":"2023-10-06T04:11:07.012391+0000","last_became_active":"2023-10-06T03:57:11.367257+0000","last_became_peered":"2023-10-06T03:57:11.367257+0000","last_unstale":"2023-10-06T04:11:07.012391+0000","last_undegraded":"2023-10-06T04:11:07.012391+0000","last_fullsized":"2023-10-06T04:11:07.012391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'393","last_scrub_stamp":"2023-10-06T04:11:07.012293+0000","last_deep_scrub":"53'393","last_deep_scrub_stamp":"2023-10-06T04:11:07.012293+0000","last_clean_scrub_stamp":"2023-10-06T04:11:07.012293+0000","objects_scrubbed":142,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:54.463356+0000","scrub_duration":0.119522168,"objects_trimmed":0,"snaptrim_duration":0.002803224,"stat_sum":{"num_bytes":18486267,"num_objects":142,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":26,"num_read_kb":4106,"num_write":272,"num_write_kb":22293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"53'56","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:02.635387+0000","last_change":"2023-10-06T04:11:02.635352+0000","last_active":"2023-10-06T04:11:02.635387+0000","last_peered":"2023-10-06T04:11:02.635387+0000","last_clean":"2023-10-06T04:11:02.635387+0000","last_became_active":"2023-10-06T03:57:10.347433+0000","last_became_peered":"2023-10-06T03:57:10.347433+0000","last_unstale":"2023-10-06T04:11:02.635387+0000","last_undegraded":"2023-10-06T04:11:02.635387+0000","last_fullsized":"2023-10-06T04:11:02.635387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'56","last_scrub_stamp":"2023-10-06T04:11:02.635240+0000","last_deep_scrub":"53'56","last_deep_scrub_stamp":"2023-10-06T04:11:02.635240+0000","last_clean_scrub_stamp":"2023-10-06T04:11:02.635240+0000","objects_scrubbed":4,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:08.757110+0000","scrub_duration":0.010252476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":11,"num_read_kb":8201,"num_write":54,"num_write_kb":4137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17420,"num_omap_keys":31,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"53'493","reported_seq":2611,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:23.921966+0000","last_change":"2023-10-06T04:11:23.921926+0000","last_active":"2023-10-06T04:11:23.921966+0000","last_peered":"2023-10-06T04:11:23.921966+0000","last_clean":"2023-10-06T04:11:23.921966+0000","last_became_active":"2023-10-06T03:57:11.369825+0000","last_became_peered":"2023-10-06T03:57:11.369825+0000","last_unstale":"2023-10-06T04:11:23.921966+0000","last_undegraded":"2023-10-06T04:11:23.921966+0000","last_fullsized":"2023-10-06T04:11:23.921966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'493","last_scrub_stamp":"2023-10-06T04:11:23.921850+0000","last_deep_scrub":"53'493","last_deep_scrub_stamp":"2023-10-06T04:11:23.921850+0000","last_clean_scrub_stamp":"2023-10-06T04:11:23.921850+0000","objects_scrubbed":159,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:01:43.526753+0000","scrub_duration":0.099932071999999997,"objects_trimmed":0,"snaptrim_duration":0.0014204689999999999,"stat_sum":{"num_bytes":11391239,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":35,"num_read_kb":8210,"num_write":309,"num_write_kb":15384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:09.870140+0000","last_change":"2023-10-06T04:11:09.870107+0000","last_active":"2023-10-06T04:11:09.870140+0000","last_peered":"2023-10-06T04:11:09.870140+0000","last_clean":"2023-10-06T04:11:09.870140+0000","last_became_active":"2023-10-06T03:57:05.663046+0000","last_became_peered":"2023-10-06T03:57:05.663046+0000","last_unstale":"2023-10-06T04:11:09.870140+0000","last_undegraded":"2023-10-06T04:11:09.870140+0000","last_fullsized":"2023-10-06T04:11:09.870140+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:11:09.869990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:11:09.869990+0000","last_clean_scrub_stamp":"2023-10-06T04:11:09.869990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:07.282698+0000","scrub_duration":0.001426486,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"53'43","reported_seq":101,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:12.801160+0000","last_change":"2023-10-06T04:11:12.801137+0000","last_active":"2023-10-06T04:11:12.801160+0000","last_peered":"2023-10-06T04:11:12.801160+0000","last_clean":"2023-10-06T04:11:12.801160+0000","last_became_active":"2023-10-06T03:57:10.348608+0000","last_became_peered":"2023-10-06T03:57:10.348608+0000","last_unstale":"2023-10-06T04:11:12.801160+0000","last_undegraded":"2023-10-06T04:11:12.801160+0000","last_fullsized":"2023-10-06T04:11:12.801160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'43","last_scrub_stamp":"2023-10-06T04:11:12.801071+0000","last_deep_scrub":"53'43","last_deep_scrub_stamp":"2023-10-06T04:11:12.801071+0000","last_clean_scrub_stamp":"2023-10-06T04:11:12.801071+0000","objects_scrubbed":6,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:46.340363+0000","scrub_duration":0.010232031000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194416,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":39,"num_write_kb":4139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":21608,"num_omap_keys":37,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"53'449","reported_seq":2561,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:24.911859+0000","last_change":"2023-10-06T04:11:24.911837+0000","last_active":"2023-10-06T04:11:24.911859+0000","last_peered":"2023-10-06T04:11:24.911859+0000","last_clean":"2023-10-06T04:11:24.911859+0000","last_became_active":"2023-10-06T03:57:11.370847+0000","last_became_peered":"2023-10-06T03:57:11.370847+0000","last_unstale":"2023-10-06T04:11:24.911859+0000","last_undegraded":"2023-10-06T04:11:24.911859+0000","last_fullsized":"2023-10-06T04:11:24.911859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'449","last_scrub_stamp":"2023-10-06T04:11:24.911765+0000","last_deep_scrub":"53'449","last_deep_scrub_stamp":"2023-10-06T04:11:24.911765+0000","last_clean_scrub_stamp":"2023-10-06T04:11:24.911765+0000","objects_scrubbed":139,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:18:37.604695+0000","scrub_duration":0.10052712699999999,"objects_trimmed":0,"snaptrim_duration":0.0013685990000000001,"stat_sum":{"num_bytes":14926709,"num_objects":139,"num_object_clones":0,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":0,"num_read":29,"num_read_kb":12301,"num_write":289,"num_write_kb":18822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:02.948635+0000","last_change":"2023-10-06T04:11:02.948611+0000","last_active":"2023-10-06T04:11:02.948635+0000","last_peered":"2023-10-06T04:11:02.948635+0000","last_clean":"2023-10-06T04:11:02.948635+0000","last_became_active":"2023-10-06T03:57:05.318572+0000","last_became_peered":"2023-10-06T03:57:05.318572+0000","last_unstale":"2023-10-06T04:11:02.948635+0000","last_undegraded":"2023-10-06T04:11:02.948635+0000","last_fullsized":"2023-10-06T04:11:02.948635+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:11:02.948529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:11:02.948529+0000","last_clean_scrub_stamp":"2023-10-06T04:11:02.948529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:16.383610+0000","scrub_duration":0.0014570340000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"53'16","reported_seq":77,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:15.846630+0000","last_change":"2023-10-06T04:11:15.846599+0000","last_active":"2023-10-06T04:11:15.846630+0000","last_peered":"2023-10-06T04:11:15.846630+0000","last_clean":"2023-10-06T04:11:15.846630+0000","last_became_active":"2023-10-06T03:57:10.348271+0000","last_became_peered":"2023-10-06T03:57:10.348271+0000","last_unstale":"2023-10-06T04:11:15.846630+0000","last_undegraded":"2023-10-06T04:11:15.846630+0000","last_fullsized":"2023-10-06T04:11:15.846630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'16","last_scrub_stamp":"2023-10-06T04:11:15.846508+0000","last_deep_scrub":"53'16","last_deep_scrub_stamp":"2023-10-06T04:11:15.846508+0000","last_clean_scrub_stamp":"2023-10-06T04:11:15.846508+0000","objects_scrubbed":6,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:43.610039+0000","scrub_duration":0.011518349000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":26,"num_write":12,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17144,"num_omap_keys":31,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"53'511","reported_seq":2628,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:26.853504+0000","last_change":"2023-10-06T04:11:26.853469+0000","last_active":"2023-10-06T04:11:26.853504+0000","last_peered":"2023-10-06T04:11:26.853504+0000","last_clean":"2023-10-06T04:11:26.853504+0000","last_became_active":"2023-10-06T03:57:11.365006+0000","last_became_peered":"2023-10-06T03:57:11.365006+0000","last_unstale":"2023-10-06T04:11:26.853504+0000","last_undegraded":"2023-10-06T04:11:26.853504+0000","last_fullsized":"2023-10-06T04:11:26.853504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'511","last_scrub_stamp":"2023-10-06T04:11:26.853393+0000","last_deep_scrub":"53'511","last_deep_scrub_stamp":"2023-10-06T04:11:26.853393+0000","last_clean_scrub_stamp":"2023-10-06T04:11:26.853393+0000","objects_scrubbed":162,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:26:41.224552+0000","scrub_duration":0.103410242,"objects_trimmed":0,"snaptrim_duration":0.0015961910000000001,"stat_sum":{"num_bytes":5739830,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":33,"num_read_kb":4109,"num_write":322,"num_write_kb":5775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:06.413155+0000","last_change":"2023-10-06T04:11:06.413102+0000","last_active":"2023-10-06T04:11:06.413155+0000","last_peered":"2023-10-06T04:11:06.413155+0000","last_clean":"2023-10-06T04:11:06.413155+0000","last_became_active":"2023-10-06T03:57:05.314408+0000","last_became_peered":"2023-10-06T03:57:05.314408+0000","last_unstale":"2023-10-06T04:11:06.413155+0000","last_undegraded":"2023-10-06T04:11:06.413155+0000","last_fullsized":"2023-10-06T04:11:06.413155+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:11:06.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:11:06.412975+0000","last_clean_scrub_stamp":"2023-10-06T04:11:06.412975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:42:44.133543+0000","scrub_duration":0.0018344570000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"53'365","reported_seq":422,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:11.809107+0000","last_change":"2023-10-06T04:11:11.809083+0000","last_active":"2023-10-06T04:11:11.809107+0000","last_peered":"2023-10-06T04:11:11.809107+0000","last_clean":"2023-10-06T04:11:11.809107+0000","last_became_active":"2023-10-06T03:57:10.345769+0000","last_became_peered":"2023-10-06T03:57:10.345769+0000","last_unstale":"2023-10-06T04:11:11.809107+0000","last_undegraded":"2023-10-06T04:11:11.809107+0000","last_fullsized":"2023-10-06T04:11:11.809107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'365","last_scrub_stamp":"2023-10-06T04:11:11.809003+0000","last_deep_scrub":"53'365","last_deep_scrub_stamp":"2023-10-06T04:11:11.809003+0000","last_clean_scrub_stamp":"2023-10-06T04:11:11.809003+0000","objects_scrubbed":6,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:05.207133+0000","scrub_duration":0.020844386999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":352,"num_write_kb":12528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":20402,"num_omap_keys":35,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"53'477","reported_seq":2599,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:29.906931+0000","last_change":"2023-10-06T04:11:29.906907+0000","last_active":"2023-10-06T04:11:29.906931+0000","last_peered":"2023-10-06T04:11:29.906931+0000","last_clean":"2023-10-06T04:11:29.906931+0000","last_became_active":"2023-10-06T03:57:11.370076+0000","last_became_peered":"2023-10-06T03:57:11.370076+0000","last_unstale":"2023-10-06T04:11:29.906931+0000","last_undegraded":"2023-10-06T04:11:29.906931+0000","last_fullsized":"2023-10-06T04:11:29.906931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'477","last_scrub_stamp":"2023-10-06T04:11:29.906836+0000","last_deep_scrub":"53'477","last_deep_scrub_stamp":"2023-10-06T04:11:29.906836+0000","last_clean_scrub_stamp":"2023-10-06T04:11:29.906836+0000","objects_scrubbed":146,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:08:33.902528+0000","scrub_duration":0.12060051500000001,"objects_trimmed":0,"snaptrim_duration":0.0011772849999999999,"stat_sum":{"num_bytes":18548829,"num_objects":146,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":146,"num_whiteouts":0,"num_read":39,"num_read_kb":4112,"num_write":310,"num_write_kb":22369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:08.939772+0000","last_change":"2023-10-06T04:11:08.939745+0000","last_active":"2023-10-06T04:11:08.939772+0000","last_peered":"2023-10-06T04:11:08.939772+0000","last_clean":"2023-10-06T04:11:08.939772+0000","last_became_active":"2023-10-06T03:57:05.318456+0000","last_became_peered":"2023-10-06T03:57:05.318456+0000","last_unstale":"2023-10-06T04:11:08.939772+0000","last_undegraded":"2023-10-06T04:11:08.939772+0000","last_fullsized":"2023-10-06T04:11:08.939772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:11:08.939596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:11:08.939596+0000","last_clean_scrub_stamp":"2023-10-06T04:11:08.939596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:34.690483+0000","scrub_duration":0.0022435490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"53'119","reported_seq":208,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:03.558037+0000","last_change":"2023-10-06T04:11:03.558012+0000","last_active":"2023-10-06T04:11:03.558037+0000","last_peered":"2023-10-06T04:11:03.558037+0000","last_clean":"2023-10-06T04:11:03.558037+0000","last_became_active":"2023-10-06T03:57:05.663426+0000","last_became_peered":"2023-10-06T03:57:05.663426+0000","last_unstale":"2023-10-06T04:11:03.558037+0000","last_undegraded":"2023-10-06T04:11:03.558037+0000","last_fullsized":"2023-10-06T04:11:03.558037+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'119","last_scrub_stamp":"2023-10-06T04:11:03.557930+0000","last_deep_scrub":"53'119","last_deep_scrub_stamp":"2023-10-06T04:11:03.557930+0000","last_clean_scrub_stamp":"2023-10-06T04:11:03.557930+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:55:23.076482+0000","scrub_duration":0.0036399879999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"53'724","reported_seq":786,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:03.652644+0000","last_change":"2023-10-06T04:11:03.652622+0000","last_active":"2023-10-06T04:11:03.652644+0000","last_peered":"2023-10-06T04:11:03.652644+0000","last_clean":"2023-10-06T04:11:03.652644+0000","last_became_active":"2023-10-06T03:57:10.348879+0000","last_became_peered":"2023-10-06T03:57:10.348879+0000","last_unstale":"2023-10-06T04:11:03.652644+0000","last_undegraded":"2023-10-06T04:11:03.652644+0000","last_fullsized":"2023-10-06T04:11:03.652644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'724","last_scrub_stamp":"2023-10-06T04:11:03.652510+0000","last_deep_scrub":"53'724","last_deep_scrub_stamp":"2023-10-06T04:11:03.652510+0000","last_clean_scrub_stamp":"2023-10-06T04:11:03.652510+0000","objects_scrubbed":5,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:03.918796+0000","scrub_duration":0.024355216999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9487985,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":20487,"num_write":715,"num_write_kb":13783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9328,"num_omap_keys":17,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"53'450","reported_seq":2570,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:15.162025+0000","last_change":"2023-10-06T04:11:15.162001+0000","last_active":"2023-10-06T04:11:15.162025+0000","last_peered":"2023-10-06T04:11:15.162025+0000","last_clean":"2023-10-06T04:11:15.162025+0000","last_became_active":"2023-10-06T03:57:11.373452+0000","last_became_peered":"2023-10-06T03:57:11.373452+0000","last_unstale":"2023-10-06T04:11:15.162025+0000","last_undegraded":"2023-10-06T04:11:15.162025+0000","last_fullsized":"2023-10-06T04:11:15.162025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'450","last_scrub_stamp":"2023-10-06T04:11:15.161925+0000","last_deep_scrub":"53'450","last_deep_scrub_stamp":"2023-10-06T04:11:15.161925+0000","last_clean_scrub_stamp":"2023-10-06T04:11:15.161925+0000","objects_scrubbed":165,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:13.301939+0000","scrub_duration":0.119156151,"objects_trimmed":0,"snaptrim_duration":0.0018153620000000001,"stat_sum":{"num_bytes":5937326,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":36,"num_read_kb":2733,"num_write":331,"num_write_kb":10073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:07.651843+0000","last_change":"2023-10-06T04:11:07.651815+0000","last_active":"2023-10-06T04:11:07.651843+0000","last_peered":"2023-10-06T04:11:07.651843+0000","last_clean":"2023-10-06T04:11:07.651843+0000","last_became_active":"2023-10-06T03:57:05.318531+0000","last_became_peered":"2023-10-06T03:57:05.318531+0000","last_unstale":"2023-10-06T04:11:07.651843+0000","last_undegraded":"2023-10-06T04:11:07.651843+0000","last_fullsized":"2023-10-06T04:11:07.651843+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:11:07.651712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:11:07.651712+0000","last_clean_scrub_stamp":"2023-10-06T04:11:07.651712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:16.562485+0000","scrub_duration":0.0020165349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"53'545","reported_seq":719,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:16.314725+0000","last_change":"2023-10-06T04:11:16.314684+0000","last_active":"2023-10-06T04:11:16.314725+0000","last_peered":"2023-10-06T04:11:16.314725+0000","last_clean":"2023-10-06T04:11:16.314725+0000","last_became_active":"2023-10-06T03:57:10.343051+0000","last_became_peered":"2023-10-06T03:57:10.343051+0000","last_unstale":"2023-10-06T04:11:16.314725+0000","last_undegraded":"2023-10-06T04:11:16.314725+0000","last_fullsized":"2023-10-06T04:11:16.314725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'545","last_scrub_stamp":"2023-10-06T04:11:16.314572+0000","last_deep_scrub":"53'545","last_deep_scrub_stamp":"2023-10-06T04:11:16.314572+0000","last_clean_scrub_stamp":"2023-10-06T04:11:16.314572+0000","objects_scrubbed":5,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:43.647007+0000","scrub_duration":0.014138608,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4571457,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":122,"num_read_kb":3989,"num_write":534,"num_write_kb":4895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"53'513","reported_seq":2634,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:22.415858+0000","last_change":"2023-10-06T04:11:22.415833+0000","last_active":"2023-10-06T04:11:22.415858+0000","last_peered":"2023-10-06T04:11:22.415858+0000","last_clean":"2023-10-06T04:11:22.415858+0000","last_became_active":"2023-10-06T03:57:11.373264+0000","last_became_peered":"2023-10-06T03:57:11.373264+0000","last_unstale":"2023-10-06T04:11:22.415858+0000","last_undegraded":"2023-10-06T04:11:22.415858+0000","last_fullsized":"2023-10-06T04:11:22.415858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'513","last_scrub_stamp":"2023-10-06T04:11:22.415754+0000","last_deep_scrub":"53'513","last_deep_scrub_stamp":"2023-10-06T04:11:22.415754+0000","last_clean_scrub_stamp":"2023-10-06T04:11:22.415754+0000","objects_scrubbed":168,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:30:17.810664+0000","scrub_duration":0.11956710500000001,"objects_trimmed":0,"snaptrim_duration":0.0026908990000000001,"stat_sum":{"num_bytes":10658865,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":37,"num_read_kb":4121,"num_write":334,"num_write_kb":10588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:04.597831+0000","last_change":"2023-10-06T04:11:04.597785+0000","last_active":"2023-10-06T04:11:04.597831+0000","last_peered":"2023-10-06T04:11:04.597831+0000","last_clean":"2023-10-06T04:11:04.597831+0000","last_became_active":"2023-10-06T03:57:05.663501+0000","last_became_peered":"2023-10-06T03:57:05.663501+0000","last_unstale":"2023-10-06T04:11:04.597831+0000","last_undegraded":"2023-10-06T04:11:04.597831+0000","last_fullsized":"2023-10-06T04:11:04.597831+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:11:04.597657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:11:04.597657+0000","last_clean_scrub_stamp":"2023-10-06T04:11:04.597657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:09.111900+0000","scrub_duration":0.0020988000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"53'7","reported_seq":60,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:13.746940+0000","last_change":"2023-10-06T04:11:13.746917+0000","last_active":"2023-10-06T04:11:13.746940+0000","last_peered":"2023-10-06T04:11:13.746940+0000","last_clean":"2023-10-06T04:11:13.746940+0000","last_became_active":"2023-10-06T03:57:10.345743+0000","last_became_peered":"2023-10-06T03:57:10.345743+0000","last_unstale":"2023-10-06T04:11:13.746940+0000","last_undegraded":"2023-10-06T04:11:13.746940+0000","last_fullsized":"2023-10-06T04:11:13.746940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'7","last_scrub_stamp":"2023-10-06T04:11:13.746842+0000","last_deep_scrub":"53'7","last_deep_scrub_stamp":"2023-10-06T04:11:13.746842+0000","last_clean_scrub_stamp":"2023-10-06T04:11:13.746842+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:47:15.026107+0000","scrub_duration":0.0026376030000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20578,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"53'511","reported_seq":2622,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:19.489280+0000","last_change":"2023-10-06T04:11:19.489257+0000","last_active":"2023-10-06T04:11:19.489280+0000","last_peered":"2023-10-06T04:11:19.489280+0000","last_clean":"2023-10-06T04:11:19.489280+0000","last_became_active":"2023-10-06T03:57:11.373543+0000","last_became_peered":"2023-10-06T03:57:11.373543+0000","last_unstale":"2023-10-06T04:11:19.489280+0000","last_undegraded":"2023-10-06T04:11:19.489280+0000","last_fullsized":"2023-10-06T04:11:19.489280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'511","last_scrub_stamp":"2023-10-06T04:11:19.489194+0000","last_deep_scrub":"53'511","last_deep_scrub_stamp":"2023-10-06T04:11:19.489194+0000","last_clean_scrub_stamp":"2023-10-06T04:11:19.489194+0000","objects_scrubbed":155,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:28.970043+0000","scrub_duration":0.13727003800000001,"objects_trimmed":0,"snaptrim_duration":0.0020558070000000002,"stat_sum":{"num_bytes":6540215,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":27,"num_read_kb":14,"num_write":340,"num_write_kb":6519,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"53'18","reported_seq":78,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:04.624957+0000","last_change":"2023-10-06T04:11:04.624929+0000","last_active":"2023-10-06T04:11:04.624957+0000","last_peered":"2023-10-06T04:11:04.624957+0000","last_clean":"2023-10-06T04:11:04.624957+0000","last_became_active":"2023-10-06T03:57:10.344324+0000","last_became_peered":"2023-10-06T03:57:10.344324+0000","last_unstale":"2023-10-06T04:11:04.624957+0000","last_undegraded":"2023-10-06T04:11:04.624957+0000","last_fullsized":"2023-10-06T04:11:04.624957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'18","last_scrub_stamp":"2023-10-06T04:11:04.624797+0000","last_deep_scrub":"53'18","last_deep_scrub_stamp":"2023-10-06T04:11:04.624797+0000","last_clean_scrub_stamp":"2023-10-06T04:11:04.624797+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:02.674217+0000","scrub_duration":0.0092723449999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":22,"num_read_kb":21,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:01.951088+0000","last_change":"2023-10-06T04:11:01.951066+0000","last_active":"2023-10-06T04:11:01.951088+0000","last_peered":"2023-10-06T04:11:01.951088+0000","last_clean":"2023-10-06T04:11:01.951088+0000","last_became_active":"2023-10-06T03:57:05.317715+0000","last_became_peered":"2023-10-06T03:57:05.317715+0000","last_unstale":"2023-10-06T04:11:01.951088+0000","last_undegraded":"2023-10-06T04:11:01.951088+0000","last_fullsized":"2023-10-06T04:11:01.951088+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:11:01.950987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:11:01.950987+0000","last_clean_scrub_stamp":"2023-10-06T04:11:01.950987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:51.601489+0000","scrub_duration":0.0016264490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"53'501","reported_seq":2617,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:26.514674+0000","last_change":"2023-10-06T04:11:26.514649+0000","last_active":"2023-10-06T04:11:26.514674+0000","last_peered":"2023-10-06T04:11:26.514674+0000","last_clean":"2023-10-06T04:11:26.514674+0000","last_became_active":"2023-10-06T03:57:11.369445+0000","last_became_peered":"2023-10-06T03:57:11.369445+0000","last_unstale":"2023-10-06T04:11:26.514674+0000","last_undegraded":"2023-10-06T04:11:26.514674+0000","last_fullsized":"2023-10-06T04:11:26.514674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'501","last_scrub_stamp":"2023-10-06T04:11:26.514580+0000","last_deep_scrub":"53'501","last_deep_scrub_stamp":"2023-10-06T04:11:26.514580+0000","last_clean_scrub_stamp":"2023-10-06T04:11:26.514580+0000","objects_scrubbed":170,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:54:27.080939+0000","scrub_duration":0.121460004,"objects_trimmed":0,"snaptrim_duration":0.0015173949999999999,"stat_sum":{"num_bytes":11743957,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":32,"num_read_kb":17,"num_write":311,"num_write_kb":11603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":56,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:02.976078+0000","last_change":"2023-10-06T04:11:02.976046+0000","last_active":"2023-10-06T04:11:02.976078+0000","last_peered":"2023-10-06T04:11:02.976078+0000","last_clean":"2023-10-06T04:11:02.976078+0000","last_became_active":"2023-10-06T03:57:05.320583+0000","last_became_peered":"2023-10-06T03:57:05.320583+0000","last_unstale":"2023-10-06T04:11:02.976078+0000","last_undegraded":"2023-10-06T04:11:02.976078+0000","last_fullsized":"2023-10-06T04:11:02.976078+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:11:02.975967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:11:02.975967+0000","last_clean_scrub_stamp":"2023-10-06T04:11:02.975967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:09.044661+0000","scrub_duration":0.001182069,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"53'96","reported_seq":151,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:18.309985+0000","last_change":"2023-10-06T04:11:18.309956+0000","last_active":"2023-10-06T04:11:18.309985+0000","last_peered":"2023-10-06T04:11:18.309985+0000","last_clean":"2023-10-06T04:11:18.309985+0000","last_became_active":"2023-10-06T03:57:10.339424+0000","last_became_peered":"2023-10-06T03:57:10.339424+0000","last_unstale":"2023-10-06T04:11:18.309985+0000","last_undegraded":"2023-10-06T04:11:18.309985+0000","last_fullsized":"2023-10-06T04:11:18.309985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'96","last_scrub_stamp":"2023-10-06T04:11:18.309874+0000","last_deep_scrub":"53'96","last_deep_scrub_stamp":"2023-10-06T04:11:18.309874+0000","last_clean_scrub_stamp":"2023-10-06T04:11:18.309874+0000","objects_scrubbed":4,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:47.802237+0000","scrub_duration":0.006477053,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":19,"num_write":221,"num_write_kb":1011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"53'495","reported_seq":2607,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:31.043852+0000","last_change":"2023-10-06T04:11:31.043829+0000","last_active":"2023-10-06T04:11:31.043852+0000","last_peered":"2023-10-06T04:11:31.043852+0000","last_clean":"2023-10-06T04:11:31.043852+0000","last_became_active":"2023-10-06T03:57:11.372215+0000","last_became_peered":"2023-10-06T03:57:11.372215+0000","last_unstale":"2023-10-06T04:11:31.043852+0000","last_undegraded":"2023-10-06T04:11:31.043852+0000","last_fullsized":"2023-10-06T04:11:31.043852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'495","last_scrub_stamp":"2023-10-06T04:11:31.043759+0000","last_deep_scrub":"53'495","last_deep_scrub_stamp":"2023-10-06T04:11:31.043759+0000","last_clean_scrub_stamp":"2023-10-06T04:11:31.043759+0000","objects_scrubbed":160,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:24.464374+0000","scrub_duration":0.134907318,"objects_trimmed":0,"snaptrim_duration":0.0020199100000000002,"stat_sum":{"num_bytes":14603524,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":28,"num_read_kb":4106,"num_write":312,"num_write_kb":14355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"53'211","reported_seq":275,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:14.290761+0000","last_change":"2023-10-06T04:11:14.290711+0000","last_active":"2023-10-06T04:11:14.290761+0000","last_peered":"2023-10-06T04:11:14.290761+0000","last_clean":"2023-10-06T04:11:14.290761+0000","last_became_active":"2023-10-06T03:57:10.343865+0000","last_became_peered":"2023-10-06T03:57:10.343865+0000","last_unstale":"2023-10-06T04:11:14.290761+0000","last_undegraded":"2023-10-06T04:11:14.290761+0000","last_fullsized":"2023-10-06T04:11:14.290761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'211","last_scrub_stamp":"2023-10-06T04:11:14.290584+0000","last_deep_scrub":"53'211","last_deep_scrub_stamp":"2023-10-06T04:11:14.290584+0000","last_clean_scrub_stamp":"2023-10-06T04:11:14.290584+0000","objects_scrubbed":9,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:30.042251+0000","scrub_duration":0.028391895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":11,"num_read_kb":32771,"num_write":206,"num_write_kb":12488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":51091,"num_omap_keys":90,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"53'505","reported_seq":2622,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:20.497794+0000","last_change":"2023-10-06T04:11:20.497772+0000","last_active":"2023-10-06T04:11:20.497794+0000","last_peered":"2023-10-06T04:11:20.497794+0000","last_clean":"2023-10-06T04:11:20.497794+0000","last_became_active":"2023-10-06T03:57:11.369681+0000","last_became_peered":"2023-10-06T03:57:11.369681+0000","last_unstale":"2023-10-06T04:11:20.497794+0000","last_undegraded":"2023-10-06T04:11:20.497794+0000","last_fullsized":"2023-10-06T04:11:20.497794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'505","last_scrub_stamp":"2023-10-06T04:11:20.497703+0000","last_deep_scrub":"53'505","last_deep_scrub_stamp":"2023-10-06T04:11:20.497703+0000","last_clean_scrub_stamp":"2023-10-06T04:11:20.497703+0000","objects_scrubbed":158,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:42.201624+0000","scrub_duration":0.099109163,"objects_trimmed":0,"snaptrim_duration":0.0013151409999999999,"stat_sum":{"num_bytes":9655954,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":33,"num_read_kb":14,"num_write":334,"num_write_kb":13707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"53'104","reported_seq":164,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:17.710929+0000","last_change":"2023-10-06T04:11:17.710901+0000","last_active":"2023-10-06T04:11:17.710929+0000","last_peered":"2023-10-06T04:11:17.710929+0000","last_clean":"2023-10-06T04:11:17.710929+0000","last_became_active":"2023-10-06T03:57:10.349320+0000","last_became_peered":"2023-10-06T03:57:10.349320+0000","last_unstale":"2023-10-06T04:11:17.710929+0000","last_undegraded":"2023-10-06T04:11:17.710929+0000","last_fullsized":"2023-10-06T04:11:17.710929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'104","last_scrub_stamp":"2023-10-06T04:11:17.710816+0000","last_deep_scrub":"53'104","last_deep_scrub_stamp":"2023-10-06T04:11:17.710816+0000","last_clean_scrub_stamp":"2023-10-06T04:11:17.710816+0000","objects_scrubbed":4,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:48:46.926120+0000","scrub_duration":0.01386691,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194858,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":4110,"num_write":99,"num_write_kb":4143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"53'441","reported_seq":2552,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:10.921994+0000","last_change":"2023-10-06T04:11:10.921969+0000","last_active":"2023-10-06T04:11:10.921994+0000","last_peered":"2023-10-06T04:11:10.921994+0000","last_clean":"2023-10-06T04:11:10.921994+0000","last_became_active":"2023-10-06T03:57:11.370229+0000","last_became_peered":"2023-10-06T03:57:11.370229+0000","last_unstale":"2023-10-06T04:11:10.921994+0000","last_undegraded":"2023-10-06T04:11:10.921994+0000","last_fullsized":"2023-10-06T04:11:10.921994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'441","last_scrub_stamp":"2023-10-06T04:11:10.921900+0000","last_deep_scrub":"53'441","last_deep_scrub_stamp":"2023-10-06T04:11:10.921900+0000","last_clean_scrub_stamp":"2023-10-06T04:11:10.921900+0000","objects_scrubbed":155,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:16.456865+0000","scrub_duration":0.100086199,"objects_trimmed":0,"snaptrim_duration":0.0027851170000000001,"stat_sum":{"num_bytes":11865278,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":27,"num_read_kb":17,"num_write":315,"num_write_kb":11749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"53'250","reported_seq":322,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:16.743648+0000","last_change":"2023-10-06T04:11:16.743623+0000","last_active":"2023-10-06T04:11:16.743648+0000","last_peered":"2023-10-06T04:11:16.743648+0000","last_clean":"2023-10-06T04:11:16.743648+0000","last_became_active":"2023-10-06T03:57:10.350083+0000","last_became_peered":"2023-10-06T03:57:10.350083+0000","last_unstale":"2023-10-06T04:11:16.743648+0000","last_undegraded":"2023-10-06T04:11:16.743648+0000","last_fullsized":"2023-10-06T04:11:16.743648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'250","last_scrub_stamp":"2023-10-06T04:11:16.743549+0000","last_deep_scrub":"53'250","last_deep_scrub_stamp":"2023-10-06T04:11:16.743549+0000","last_clean_scrub_stamp":"2023-10-06T04:11:16.743549+0000","objects_scrubbed":12,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:57.567358+0000","scrub_duration":0.028247550999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":36,"num_read_kb":45128,"num_write":264,"num_write_kb":12512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19684,"num_omap_keys":36,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"53'498","reported_seq":2611,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:33.054545+0000","last_change":"2023-10-06T04:11:33.054522+0000","last_active":"2023-10-06T04:11:33.054545+0000","last_peered":"2023-10-06T04:11:33.054545+0000","last_clean":"2023-10-06T04:11:33.054545+0000","last_became_active":"2023-10-06T03:57:11.369123+0000","last_became_peered":"2023-10-06T03:57:11.369123+0000","last_unstale":"2023-10-06T04:11:33.054545+0000","last_undegraded":"2023-10-06T04:11:33.054545+0000","last_fullsized":"2023-10-06T04:11:33.054545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'498","last_scrub_stamp":"2023-10-06T04:11:33.054447+0000","last_deep_scrub":"53'498","last_deep_scrub_stamp":"2023-10-06T04:11:33.054447+0000","last_clean_scrub_stamp":"2023-10-06T04:11:33.054447+0000","objects_scrubbed":148,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:40.005920+0000","scrub_duration":0.12269728000000001,"objects_trimmed":0,"snaptrim_duration":0.0028613789999999998,"stat_sum":{"num_bytes":15066743,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":30,"num_read_kb":4112,"num_write":322,"num_write_kb":14878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"53'183","reported_seq":238,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:06.608805+0000","last_change":"2023-10-06T04:11:06.608778+0000","last_active":"2023-10-06T04:11:06.608805+0000","last_peered":"2023-10-06T04:11:06.608805+0000","last_clean":"2023-10-06T04:11:06.608805+0000","last_became_active":"2023-10-06T03:57:10.344583+0000","last_became_peered":"2023-10-06T03:57:10.344583+0000","last_unstale":"2023-10-06T04:11:06.608805+0000","last_undegraded":"2023-10-06T04:11:06.608805+0000","last_fullsized":"2023-10-06T04:11:06.608805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'183","last_scrub_stamp":"2023-10-06T04:11:06.608691+0000","last_deep_scrub":"53'183","last_deep_scrub_stamp":"2023-10-06T04:11:06.608691+0000","last_clean_scrub_stamp":"2023-10-06T04:11:06.608691+0000","objects_scrubbed":2,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:46:02.740119+0000","scrub_duration":0.010859541,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":177,"num_write_kb":8286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8760,"num_omap_keys":15,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"53'516","reported_seq":2637,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:27.453717+0000","last_change":"2023-10-06T04:11:27.453693+0000","last_active":"2023-10-06T04:11:27.453717+0000","last_peered":"2023-10-06T04:11:27.453717+0000","last_clean":"2023-10-06T04:11:27.453717+0000","last_became_active":"2023-10-06T03:57:11.366500+0000","last_became_peered":"2023-10-06T03:57:11.366500+0000","last_unstale":"2023-10-06T04:11:27.453717+0000","last_undegraded":"2023-10-06T04:11:27.453717+0000","last_fullsized":"2023-10-06T04:11:27.453717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'516","last_scrub_stamp":"2023-10-06T04:11:27.453623+0000","last_deep_scrub":"53'516","last_deep_scrub_stamp":"2023-10-06T04:11:27.453623+0000","last_clean_scrub_stamp":"2023-10-06T04:11:27.453623+0000","objects_scrubbed":164,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:46.741288+0000","scrub_duration":0.102849519,"objects_trimmed":0,"snaptrim_duration":0.0016878310000000001,"stat_sum":{"num_bytes":6166988,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":38,"num_read_kb":4113,"num_write":326,"num_write_kb":10290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"53'178","reported_seq":239,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:09.230941+0000","last_change":"2023-10-06T04:11:09.230916+0000","last_active":"2023-10-06T04:11:09.230941+0000","last_peered":"2023-10-06T04:11:09.230941+0000","last_clean":"2023-10-06T04:11:09.230941+0000","last_became_active":"2023-10-06T03:57:10.347897+0000","last_became_peered":"2023-10-06T03:57:10.347897+0000","last_unstale":"2023-10-06T04:11:09.230941+0000","last_undegraded":"2023-10-06T04:11:09.230941+0000","last_fullsized":"2023-10-06T04:11:09.230941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'178","last_scrub_stamp":"2023-10-06T04:11:09.230838+0000","last_deep_scrub":"53'178","last_deep_scrub_stamp":"2023-10-06T04:11:09.230838+0000","last_clean_scrub_stamp":"2023-10-06T04:11:09.230838+0000","objects_scrubbed":8,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:21.538557+0000","scrub_duration":0.018787900999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7191702,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":12,"num_read_kb":8202,"num_write":168,"num_write_kb":7111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"53'586","reported_seq":2705,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:29.079404+0000","last_change":"2023-10-06T04:11:29.079377+0000","last_active":"2023-10-06T04:11:29.079404+0000","last_peered":"2023-10-06T04:11:29.079404+0000","last_clean":"2023-10-06T04:11:29.079404+0000","last_became_active":"2023-10-06T03:57:11.364150+0000","last_became_peered":"2023-10-06T03:57:11.364150+0000","last_unstale":"2023-10-06T04:11:29.079404+0000","last_undegraded":"2023-10-06T04:11:29.079404+0000","last_fullsized":"2023-10-06T04:11:29.079404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'586","last_scrub_stamp":"2023-10-06T04:11:29.079310+0000","last_deep_scrub":"53'586","last_deep_scrub_stamp":"2023-10-06T04:11:29.079310+0000","last_clean_scrub_stamp":"2023-10-06T04:11:29.079310+0000","objects_scrubbed":176,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:35.809878+0000","scrub_duration":0.15600001299999999,"objects_trimmed":0,"snaptrim_duration":0.001670408,"stat_sum":{"num_bytes":10944544,"num_objects":176,"num_object_clones":0,"num_object_copies":352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":176,"num_whiteouts":0,"num_read":35,"num_read_kb":7178,"num_write":386,"num_write_kb":19084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"53'466","reported_seq":2604,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:11.117345+0000","last_change":"2023-10-06T04:11:11.117321+0000","last_active":"2023-10-06T04:11:11.117345+0000","last_peered":"2023-10-06T04:11:11.117345+0000","last_clean":"2023-10-06T04:11:11.117345+0000","last_became_active":"2023-10-06T03:57:11.366524+0000","last_became_peered":"2023-10-06T03:57:11.366524+0000","last_unstale":"2023-10-06T04:11:11.117345+0000","last_undegraded":"2023-10-06T04:11:11.117345+0000","last_fullsized":"2023-10-06T04:11:11.117345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'466","last_scrub_stamp":"2023-10-06T04:11:11.117253+0000","last_deep_scrub":"53'466","last_deep_scrub_stamp":"2023-10-06T04:11:11.117253+0000","last_clean_scrub_stamp":"2023-10-06T04:11:11.117253+0000","objects_scrubbed":151,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:53:09.926514+0000","scrub_duration":0.109297006,"objects_trimmed":0,"snaptrim_duration":0.0017816290000000001,"stat_sum":{"num_bytes":22594948,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":54,"num_read_kb":16279,"num_write":343,"num_write_kb":22243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"53'540","reported_seq":2661,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:29.427696+0000","last_change":"2023-10-06T04:11:29.427672+0000","last_active":"2023-10-06T04:11:29.427696+0000","last_peered":"2023-10-06T04:11:29.427696+0000","last_clean":"2023-10-06T04:11:29.427696+0000","last_became_active":"2023-10-06T03:57:11.370848+0000","last_became_peered":"2023-10-06T03:57:11.370848+0000","last_unstale":"2023-10-06T04:11:29.427696+0000","last_undegraded":"2023-10-06T04:11:29.427696+0000","last_fullsized":"2023-10-06T04:11:29.427696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'540","last_scrub_stamp":"2023-10-06T04:11:29.427597+0000","last_deep_scrub":"53'540","last_deep_scrub_stamp":"2023-10-06T04:11:29.427597+0000","last_clean_scrub_stamp":"2023-10-06T04:11:29.427597+0000","objects_scrubbed":160,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:00.100427+0000","scrub_duration":0.11843801499999999,"objects_trimmed":0,"snaptrim_duration":0.0017239289999999999,"stat_sum":{"num_bytes":12161208,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":37,"num_read_kb":8210,"num_write":347,"num_write_kb":16110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"53'554","reported_seq":2678,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:36.149409+0000","last_change":"2023-10-06T04:11:36.149381+0000","last_active":"2023-10-06T04:11:36.149409+0000","last_peered":"2023-10-06T04:11:36.149409+0000","last_clean":"2023-10-06T04:11:36.149409+0000","last_became_active":"2023-10-06T03:57:11.363875+0000","last_became_peered":"2023-10-06T03:57:11.363875+0000","last_unstale":"2023-10-06T04:11:36.149409+0000","last_undegraded":"2023-10-06T04:11:36.149409+0000","last_fullsized":"2023-10-06T04:11:36.149409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'554","last_scrub_stamp":"2023-10-06T04:11:36.149308+0000","last_deep_scrub":"53'554","last_deep_scrub_stamp":"2023-10-06T04:11:36.149308+0000","last_clean_scrub_stamp":"2023-10-06T04:11:36.149308+0000","objects_scrubbed":183,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:46:05.831956+0000","scrub_duration":0.167688535,"objects_trimmed":0,"snaptrim_duration":0.0028142470000000002,"stat_sum":{"num_bytes":14948732,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":39,"num_read_kb":4117,"num_write":348,"num_write_kb":14761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"53'505","reported_seq":2630,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:19.658777+0000","last_change":"2023-10-06T04:11:19.658754+0000","last_active":"2023-10-06T04:11:19.658777+0000","last_peered":"2023-10-06T04:11:19.658777+0000","last_clean":"2023-10-06T04:11:19.658777+0000","last_became_active":"2023-10-06T03:57:11.377126+0000","last_became_peered":"2023-10-06T03:57:11.377126+0000","last_unstale":"2023-10-06T04:11:19.658777+0000","last_undegraded":"2023-10-06T04:11:19.658777+0000","last_fullsized":"2023-10-06T04:11:19.658777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'505","last_scrub_stamp":"2023-10-06T04:11:19.658678+0000","last_deep_scrub":"53'505","last_deep_scrub_stamp":"2023-10-06T04:11:19.658678+0000","last_clean_scrub_stamp":"2023-10-06T04:11:19.658678+0000","objects_scrubbed":153,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:16.651951+0000","scrub_duration":0.108743853,"objects_trimmed":0,"snaptrim_duration":0.0023666569999999999,"stat_sum":{"num_bytes":6085663,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":41,"num_read_kb":18,"num_write":335,"num_write_kb":6110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"53'468","reported_seq":2581,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T04:11:50.735606+0000","last_change":"2023-10-06T04:11:50.735584+0000","last_active":"2023-10-06T04:11:50.735606+0000","last_peered":"2023-10-06T04:11:50.735606+0000","last_clean":"2023-10-06T04:11:50.735606+0000","last_became_active":"2023-10-06T03:57:59.256086+0000","last_became_peered":"2023-10-06T03:57:59.256086+0000","last_unstale":"2023-10-06T04:11:50.735606+0000","last_undegraded":"2023-10-06T04:11:50.735606+0000","last_fullsized":"2023-10-06T04:11:50.735606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'468","last_scrub_stamp":"2023-10-06T04:11:50.735512+0000","last_deep_scrub":"53'468","last_deep_scrub_stamp":"2023-10-06T04:11:50.735512+0000","last_clean_scrub_stamp":"2023-10-06T04:11:50.735512+0000","objects_scrubbed":142,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:18:51.898575+0000","scrub_duration":0.118609065,"objects_trimmed":0,"snaptrim_duration":0.0018573859999999999,"stat_sum":{"num_bytes":11350301,"num_objects":142,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":33,"num_read_kb":11479,"num_write":309,"num_write_kb":14473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":923606083,"num_objects":10287,"num_object_clones":0,"num_object_copies":20574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10287,"num_whiteouts":0,"num_read":2243,"num_read_kb":380500,"num_write":20866,"num_write_kb":1045904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1683152896,"data_stored":1845672070,"data_compressed":160523048,"data_compressed_allocated":177569792,"data_compressed_original":393674752,"omap_allocated":0,"internal_metadata":0},"log_size":31866,"ondisk_log_size":31866,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":168060516,"num_objects":202,"num_object_clones":0,"num_object_copies":404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":202,"num_whiteouts":0,"num_read":555,"num_read_kb":254049,"num_write":6966,"num_write_kb":198967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":122,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":581829,"num_omap_keys":1025,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":84746240,"data_stored":336121032,"data_compressed":45213916,"data_compressed_allocated":56918016,"data_compressed_original":308543488,"omap_allocated":0,"internal_metadata":0},"log_size":6899,"ondisk_log_size":6899,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64858,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575032,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511228,"kb_used_data":209844,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":93860612,"statfs":{"total":96636764160,"available":96113266688,"internally_reserved":0,"allocated":214880256,"data_stored":260105949,"data_compressed":22363353,"data_compressed_allocated":25411584,"data_compressed_original":76546048,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.53800000000000003,"15min":0.54500000000000004},"min":{"1min":0.29499999999999998,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.81200000000000006,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.628},{"interface":"front","average":{"1min":0.503,"5min":0.54400000000000004,"15min":0.56599999999999995},"min":{"1min":0.33000000000000002,"5min":0.28599999999999998,"15min":0.28399999999999997},"max":{"1min":0.73099999999999998,"5min":0.80500000000000005,"15min":0.88900000000000001},"last":0.61599999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51400000000000001,"15min":0.52200000000000002},"min":{"1min":0.36299999999999999,"5min":0.312,"15min":0.28899999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54500000000000004,"15min":0.52700000000000002},"min":{"1min":0.35199999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.82699999999999996,"5min":1.25,"15min":1.25},"last":0.60499999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.57499999999999996,"15min":0.55900000000000005},"min":{"1min":0.377,"5min":0.29799999999999999,"15min":0.28399999999999997},"max":{"1min":3.6850000000000001,"5min":3.6850000000000001,"15min":3.6850000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.58599999999999997,"15min":0.61099999999999999},"min":{"1min":0.376,"5min":0.33700000000000002,"15min":0.33100000000000002},"max":{"1min":3.0179999999999998,"5min":3.0179999999999998,"15min":7.3739999999999997},"last":0.53300000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32200000000000001,"15min":0.318},"max":{"1min":0.66000000000000003,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.55800000000000005,"15min":0.56000000000000005},"min":{"1min":0.39500000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.57199999999999995}]},{"osd":4,"last update":"Fri Oct 6 04:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.44700000000000001,"15min":0.45900000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.749,"5min":0.749,"15min":1.0149999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28299999999999997},"max":{"1min":0.70699999999999996,"5min":0.80200000000000005,"15min":0.90700000000000003},"last":0.40500000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.502,"15min":0.50800000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.27300000000000002},"max":{"1min":0.72199999999999998,"5min":0.82999999999999996,"15min":0.86099999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55600000000000005,"15min":0.55900000000000005},"min":{"1min":0.33800000000000002,"5min":0.317,"15min":0.28000000000000003},"max":{"1min":0.78600000000000003,"5min":0.84899999999999998,"15min":1.377},"last":0.46200000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51700000000000002,"15min":0.55400000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.29999999999999999},"max":{"1min":0.69199999999999995,"5min":0.77600000000000002,"15min":1.3420000000000001},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60699999999999998,"15min":0.58999999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.30199999999999999},"max":{"1min":0.81999999999999995,"5min":0.88200000000000001,"15min":0.93700000000000006},"last":0.438}]}]},{"osd":6,"up_from":13,"seq":55834575031,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":474560,"kb_used_data":164984,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":93897280,"statfs":{"total":96636764160,"available":96150814720,"internally_reserved":0,"allocated":168943616,"data_stored":188218692,"data_compressed":15936082,"data_compressed_allocated":17805312,"data_compressed_original":42737664,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54200000000000004,"15min":0.53100000000000003},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.98099999999999998,"5min":2.613,"15min":2.613},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.58899999999999997,"15min":0.56000000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.99299999999999999,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":0.58699999999999997}]},{"osd":1,"last update":"Fri Oct 6 04:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.53700000000000003,"15min":0.56899999999999995},"min":{"1min":0.33700000000000002,"5min":0.28499999999999998,"15min":0.27200000000000002},"max":{"1min":0.79200000000000004,"5min":2.3889999999999998,"15min":9.5519999999999996},"last":0.627},{"interface":"front","average":{"1min":0.502,"5min":0.59499999999999997,"15min":0.58399999999999996},"min":{"1min":0.37,"5min":0.315,"15min":0.313},"max":{"1min":0.66600000000000004,"5min":6.8019999999999996,"15min":6.8019999999999996},"last":0.504}]},{"osd":2,"last update":"Fri Oct 6 04:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.61699999999999999,"15min":0.64200000000000002},"min":{"1min":0.377,"5min":0.32600000000000001,"15min":0.29799999999999999},"max":{"1min":0.67000000000000004,"5min":5.7590000000000003,"15min":9.6769999999999996},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.63200000000000001,"15min":0.64800000000000002},"min":{"1min":0.33600000000000002,"5min":0.33200000000000002,"15min":0.31},"max":{"1min":0.89800000000000002,"5min":5.71,"15min":9.7309999999999999},"last":0.59899999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.53300000000000003,"15min":0.55600000000000005},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.61699999999999999,"5min":0.99099999999999999,"15min":5.7359999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.5,"5min":0.57599999999999996,"15min":0.59299999999999997},"min":{"1min":0.317,"5min":0.311,"15min":0.308},"max":{"1min":0.76100000000000001,"5min":1.0149999999999999,"15min":5.7009999999999996},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.47899999999999998,"15min":0.48499999999999999},"min":{"1min":0.27200000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.60999999999999999,"5min":0.78600000000000003,"15min":1.0129999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.52500000000000002,"15min":0.54000000000000004},"min":{"1min":0.35299999999999998,"5min":0.313,"15min":0.30399999999999999},"max":{"1min":0.55600000000000005,"5min":0.79800000000000004,"15min":1.026},"last":0.61199999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.48999999999999999,"15min":0.501},"min":{"1min":0.28899999999999998,"5min":0.247,"15min":0.247},"max":{"1min":0.59499999999999997,"5min":0.78200000000000003,"15min":0.995},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.53000000000000003,"15min":0.53500000000000003},"min":{"1min":0.32100000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65100000000000002,"5min":0.78500000000000003,"15min":0.83699999999999997},"last":0.63600000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.53800000000000003,"15min":0.53300000000000003},"min":{"1min":0.32100000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.72299999999999998,"5min":0.84199999999999997,"15min":0.98299999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.55600000000000005,"15min":0.56799999999999995},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.58899999999999997,"5min":0.84699999999999998,"15min":0.90800000000000003},"last":0.65300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575031,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574552,"kb_used_data":277264,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93797288,"statfs":{"total":96636764160,"available":96048422912,"internally_reserved":0,"allocated":283918336,"data_stored":337886631,"data_compressed":32878254,"data_compressed_allocated":37023744,"data_compressed_original":100233216,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:11:29 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.92300000000000004,"15min":0.52400000000000002},"min":{"1min":0.20899999999999999,"5min":0.20799999999999999,"15min":0.192},"max":{"1min":0.97599999999999998,"5min":57.700000000000003,"15min":57.700000000000003},"last":0.435},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.64800000000000002,"15min":0.443},"min":{"1min":0.224,"5min":0.224,"15min":0.186},"max":{"1min":0.495,"5min":28.629999999999999,"15min":28.629999999999999},"last":0.375}]},{"osd":2,"last update":"Fri Oct 6 04:11:29 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.97799999999999998,"15min":0.57399999999999995},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.19600000000000001},"max":{"1min":0.59799999999999998,"5min":58.795000000000002,"15min":58.795000000000002},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.67400000000000004,"15min":0.49099999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.23000000000000001},"max":{"1min":0.64800000000000002,"5min":25.722000000000001,"15min":25.722000000000001},"last":0.46400000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:11:29 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.59999999999999998,"15min":0.442},"min":{"1min":0.246,"5min":0.23999999999999999,"15min":0.216},"max":{"1min":0.498,"5min":22.920999999999999,"15min":22.920999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.63600000000000001,"15min":0.47199999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.22600000000000001},"max":{"1min":0.59099999999999997,"5min":22.797999999999998,"15min":22.797999999999998},"last":0.41299999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:11:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":1.2310000000000001,"15min":0.82699999999999996},"min":{"1min":0.40200000000000002,"5min":0.38500000000000001,"15min":0.34499999999999997},"max":{"1min":0.96499999999999997,"5min":59.018999999999998,"15min":59.018999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.626,"5min":1.252,"15min":0.83899999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.316},"max":{"1min":0.93600000000000005,"5min":59.046999999999997,"15min":59.046999999999997},"last":0.71199999999999997}]},{"osd":5,"last update":"Fri Oct 6 04:11:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":1.1950000000000001,"15min":0.79300000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.874,"5min":58.987000000000002,"15min":58.987000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.65600000000000003,"15min":0.59799999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.29399999999999998},"max":{"1min":0.85799999999999998,"5min":6.2140000000000004,"15min":6.2140000000000004},"last":0.61199999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:11:29 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.65700000000000003,"15min":0.60899999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.36599999999999999},"max":{"1min":0.90500000000000003,"5min":4.6020000000000003,"15min":4.6020000000000003},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.628,"5min":1.2949999999999999,"15min":0.86199999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.36799999999999999},"max":{"1min":0.91200000000000003,"5min":58.911999999999999,"15min":58.911999999999999},"last":0.69799999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:11:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":1.216,"15min":0.81399999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88100000000000001,"5min":58.953000000000003,"15min":58.953000000000003},"last":0.748},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.65500000000000003,"15min":0.61799999999999999},"min":{"1min":0.40200000000000002,"5min":0.38100000000000001,"15min":0.34399999999999997},"max":{"1min":0.84399999999999997,"5min":4.5570000000000004,"15min":4.5570000000000004},"last":0.72799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575031,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488692,"kb_used_data":191404,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93883148,"statfs":{"total":96636764160,"available":96136343552,"internally_reserved":0,"allocated":195997696,"data_stored":253566865,"data_compressed":26834878,"data_compressed_allocated":30744576,"data_compressed_original":94838784,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33800000000000002,"15min":0.35099999999999998},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19500000000000001},"max":{"1min":0.44800000000000001,"5min":0.52900000000000003,"15min":6.9539999999999997},"last":0.312},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.34899999999999998,"15min":0.371},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.191},"max":{"1min":0.46300000000000002,"5min":0.59499999999999997,"15min":8.0519999999999996},"last":0.33400000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36799999999999999,"15min":0.36099999999999999},"min":{"1min":0.25900000000000001,"5min":0.24299999999999999,"15min":0.214},"max":{"1min":0.53000000000000003,"5min":0.59099999999999997,"15min":1.0589999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.371,"5min":0.38400000000000001,"15min":0.378},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.20799999999999999},"max":{"1min":0.496,"5min":0.626,"15min":2.0489999999999999},"last":0.38600000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.39300000000000002,"15min":0.42099999999999999},"min":{"1min":0.27500000000000002,"5min":0.23100000000000001,"15min":0.223},"max":{"1min":0.56799999999999995,"5min":0.60699999999999998,"15min":9.391},"last":0.437},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.42799999999999999,"15min":0.45700000000000002},"min":{"1min":0.32900000000000001,"5min":0.311,"15min":0.27800000000000002},"max":{"1min":0.58899999999999997,"5min":0.64400000000000002,"15min":9.5239999999999991},"last":0.45900000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.64700000000000002,"15min":0.63700000000000001},"min":{"1min":0.35599999999999998,"5min":0.34000000000000002,"15min":0.29199999999999998},"max":{"1min":0.90800000000000003,"5min":1.026,"15min":5.7699999999999996},"last":0.754},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.65400000000000003,"15min":0.63600000000000001},"min":{"1min":0.48399999999999999,"5min":0.41499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81699999999999995,"5min":0.997,"15min":2.7320000000000002},"last":0.79800000000000004}]},{"osd":5,"last update":"Fri Oct 6 04:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60499999999999998,"15min":0.61599999999999999},"min":{"1min":0.45000000000000001,"5min":0.41399999999999998,"15min":0.33800000000000002},"max":{"1min":0.753,"5min":0.90500000000000003,"15min":5.5069999999999997},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.627,"5min":0.63400000000000001,"15min":0.60799999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.34799999999999998},"max":{"1min":0.84399999999999997,"5min":0.878,"15min":1.05},"last":0.66800000000000004}]},{"osd":6,"last update":"Fri Oct 6 04:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.62,"15min":0.64700000000000002},"min":{"1min":0.38900000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77800000000000002,"5min":0.96499999999999997,"15min":6.9139999999999997},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63900000000000001,"15min":0.65400000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.379},"max":{"1min":0.83099999999999996,"5min":0.93000000000000005,"15min":6.3780000000000001},"last":0.63100000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60399999999999998,"15min":0.621},"min":{"1min":0.39300000000000002,"5min":0.377,"15min":0.36699999999999999},"max":{"1min":0.84899999999999998,"5min":0.97799999999999998,"15min":5.9989999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.624,"15min":0.61299999999999999},"min":{"1min":0.42499999999999999,"5min":0.40500000000000003,"15min":0.39700000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":2.6930000000000001},"last":0.78800000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575031,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446244,"kb_used_data":140764,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":93925596,"statfs":{"total":96636764160,"available":96179810304,"internally_reserved":0,"allocated":144142336,"data_stored":175617804,"data_compressed":19256061,"data_compressed_allocated":21770240,"data_compressed_original":59097088,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:11:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.45600000000000002,"15min":0.42199999999999999},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.19},"max":{"1min":11.335000000000001,"5min":11.335000000000001,"15min":14.968999999999999},"last":0.436},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.46999999999999997,"15min":0.44700000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.21299999999999999},"max":{"1min":10.484999999999999,"5min":10.484999999999999,"15min":14.746},"last":0.32600000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.42099999999999999,"15min":0.54300000000000004},"min":{"1min":0.26500000000000001,"5min":0.23999999999999999,"15min":0.21099999999999999},"max":{"1min":5.3890000000000002,"5min":5.3890000000000002,"15min":50.677999999999997},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.50900000000000001,"15min":0.623},"min":{"1min":0.33900000000000002,"5min":0.32900000000000001,"15min":0.26000000000000001},"max":{"1min":5.4260000000000002,"5min":5.4260000000000002,"15min":50.615000000000002},"last":0.45500000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:11:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.433,"15min":0.47899999999999998},"min":{"1min":0.26000000000000001,"5min":0.23599999999999999,"15min":0.22900000000000001},"max":{"1min":6.5179999999999998,"5min":6.5179999999999998,"15min":17.440999999999999},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.52400000000000002,"15min":0.63},"min":{"1min":0.307,"5min":0.29099999999999998,"15min":0.23499999999999999},"max":{"1min":8.1750000000000007,"5min":8.1750000000000007,"15min":50.424999999999997},"last":0.40300000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:11:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":0.72899999999999998,"15min":0.69099999999999995},"min":{"1min":0.35799999999999998,"5min":0.34200000000000003,"15min":0.29699999999999999},"max":{"1min":14.308,"5min":14.308,"15min":14.802},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61399999999999999,"15min":0.67700000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":14.621},"last":0.505}]},{"osd":5,"last update":"Fri Oct 6 04:11:34 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.61399999999999999,"15min":0.63800000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.35199999999999998},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":14.792999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.69799999999999995,"15min":0.79000000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.371},"max":{"1min":9.1630000000000003,"5min":9.1630000000000003,"15min":50.537999999999997},"last":0.48499999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:11:34 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":0.78300000000000003,"15min":0.73699999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":15.627000000000001,"5min":15.627000000000001,"15min":23.388000000000002},"last":0.497},{"interface":"front","average":{"1min":1.1539999999999999,"5min":0.751,"15min":0.73699999999999999},"min":{"1min":0.439,"5min":0.42499999999999999,"15min":0.39100000000000001},"max":{"1min":11.372,"5min":11.372,"15min":14.856},"last":0.58299999999999996}]},{"osd":7,"last update":"Fri Oct 6 04:11:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.64500000000000002,"15min":0.68500000000000005},"min":{"1min":0.44600000000000001,"5min":0.40999999999999998,"15min":0.377},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":14.693},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.64800000000000002,"15min":0.67800000000000005},"min":{"1min":0.45900000000000002,"5min":0.44600000000000001,"15min":0.434},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":10.349},"last":0.46999999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575031,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":581572,"kb_used_data":276092,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":93790268,"statfs":{"total":96636764160,"available":96041234432,"internally_reserved":0,"allocated":282718208,"data_stored":344742979,"data_compressed":32796809,"data_compressed_allocated":37396480,"data_compressed_original":108593152,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:11:30 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.34000000000000002,"15min":0.35999999999999999},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.186},"max":{"1min":0.439,"5min":0.58699999999999997,"15min":7.0999999999999996},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.35399999999999998,"15min":0.41699999999999998},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.56499999999999995,"5min":0.622,"15min":14.257},"last":0.38100000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:11:30 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.34699999999999998,"15min":0.35399999999999998},"min":{"1min":0.222,"5min":0.219,"15min":0.20799999999999999},"max":{"1min":0.432,"5min":0.54600000000000004,"15min":3.1760000000000002},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.40200000000000002,"15min":0.39700000000000002},"min":{"1min":0.27300000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.48299999999999998,"5min":0.68200000000000005,"15min":5.2599999999999998},"last":0.41199999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:11:30 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.38500000000000001,"15min":0.45700000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23499999999999999},"max":{"1min":0.55100000000000005,"5min":0.59299999999999997,"15min":13.138},"last":0.442},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.42299999999999999,"15min":0.44500000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.27200000000000002},"max":{"1min":0.66100000000000003,"5min":0.67300000000000004,"15min":7.181},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:11:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61199999999999999,"15min":0.68000000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75800000000000001,"5min":0.92000000000000004,"15min":13.068},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64800000000000002,"15min":0.67700000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.39800000000000002},"max":{"1min":0.81499999999999995,"5min":0.89100000000000001,"15min":6.4299999999999997},"last":0.66700000000000004}]},{"osd":5,"last update":"Fri Oct 6 04:11:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.57499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39800000000000002,"5min":0.35999999999999999,"15min":0.32700000000000001},"max":{"1min":0.75600000000000001,"5min":0.95599999999999996,"15min":3.105},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58399999999999996,"15min":0.60399999999999998},"min":{"1min":0.40600000000000003,"5min":0.372,"15min":0.29299999999999998},"max":{"1min":0.77200000000000002,"5min":0.85499999999999998,"15min":3.5369999999999999},"last":0.56699999999999995}]},{"osd":6,"last update":"Fri Oct 6 04:11:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.627,"15min":0.68799999999999994},"min":{"1min":0.41599999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.84499999999999997,"5min":0.96099999999999997,"15min":12.974},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.622,"15min":0.63900000000000001},"min":{"1min":0.439,"5min":0.39300000000000002,"15min":0.35599999999999998},"max":{"1min":0.68899999999999995,"5min":0.94499999999999995,"15min":3.0379999999999998},"last":0.67800000000000005}]},{"osd":7,"last update":"Fri Oct 6 04:11:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.61099999999999999,"15min":0.63500000000000001},"min":{"1min":0.44600000000000001,"5min":0.41199999999999998,"15min":0.40100000000000002},"max":{"1min":0.78400000000000003,"5min":0.97199999999999998,"15min":6.2949999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.61899999999999999,"15min":0.67000000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73099999999999998,"5min":0.95399999999999996,"15min":13.032},"last":0.61599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575032,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587504,"kb_used_data":282024,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":93784336,"statfs":{"total":96636764160,"available":96035160064,"internally_reserved":0,"allocated":288792576,"data_stored":379214374,"data_compressed":33271846,"data_compressed_allocated":38641664,"data_compressed_original":137351168,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:11:32 2023","interfaces":[{"interface":"back","average":{"1min":3.661,"5min":1.149,"15min":0.73999999999999999},"min":{"1min":0.34499999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":54.295999999999999,"5min":54.295999999999999,"15min":54.295999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":3.2909999999999999,"5min":1.1200000000000001,"15min":0.72999999999999998},"min":{"1min":0.46000000000000002,"5min":0.35099999999999998,"15min":0.26600000000000001},"max":{"1min":46.795000000000002,"5min":46.795000000000002,"15min":46.795000000000002},"last":0.69299999999999995}]},{"osd":1,"last update":"Fri Oct 6 04:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.59099999999999997,"15min":0.54600000000000004},"min":{"1min":0.41699999999999998,"5min":0.34899999999999998,"15min":0.26200000000000001},"max":{"1min":3.621,"5min":3.621,"15min":3.621},"last":0.71499999999999997},{"interface":"front","average":{"1min":3.6629999999999998,"5min":1.135,"15min":0.71999999999999997},"min":{"1min":0.35299999999999998,"5min":0.30399999999999999,"15min":0.27400000000000002},"max":{"1min":54.832999999999998,"5min":54.832999999999998,"15min":54.832999999999998},"last":0.55500000000000005}]},{"osd":2,"last update":"Fri Oct 6 04:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.55000000000000004,"15min":0.53600000000000003},"min":{"1min":0.378,"5min":0.35899999999999999,"15min":0.312},"max":{"1min":0.68799999999999994,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.42399999999999999,"5min":0.36499999999999999,"15min":0.311},"max":{"1min":0.70699999999999996,"5min":1.413,"15min":1.413},"last":0.755}]},{"osd":3,"last update":"Fri Oct 6 04:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.59099999999999997,"15min":0.55000000000000004},"min":{"1min":0.36099999999999999,"5min":0.32100000000000001,"15min":0.314},"max":{"1min":1.6459999999999999,"5min":6.7000000000000002,"15min":6.7000000000000002},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.52200000000000002,"15min":0.53900000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.32000000000000001},"max":{"1min":0.64600000000000002,"5min":1.208,"15min":1.208},"last":0.63}]},{"osd":5,"last update":"Fri Oct 6 04:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.45400000000000001,"15min":0.47799999999999998},"min":{"1min":0.26400000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.52600000000000002,"15min":0.53100000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.60899999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.46899999999999997,"15min":0.505},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.28100000000000003},"max":{"1min":0.61699999999999999,"5min":1.236,"15min":1.236},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.376,"5min":0.37,"15min":0.37},"max":{"1min":0.84499999999999997,"5min":1.381,"15min":1.381},"last":0.77600000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.53400000000000003,"15min":0.54400000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.30599999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60599999999999998,"15min":0.60299999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.32600000000000001},"max":{"1min":0.80000000000000004,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.73599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575031,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490364,"kb_used_data":188980,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":93881476,"statfs":{"total":96636764160,"available":96134631424,"internally_reserved":0,"allocated":193515520,"data_stored":246084607,"data_compressed":22464539,"data_compressed_allocated":25784320,"data_compressed_original":83968000,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.502,"15min":0.55600000000000005},"min":{"1min":0.32400000000000001,"5min":0.29999999999999999,"15min":0.23300000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":11.167},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52600000000000002,"15min":0.57699999999999996},"min":{"1min":0.34799999999999998,"5min":0.23599999999999999,"15min":0.221},"max":{"1min":0.82899999999999996,"5min":0.94799999999999995,"15min":9.8659999999999997},"last":0.47099999999999997}]},{"osd":1,"last update":"Fri Oct 6 04:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53100000000000003,"15min":0.54000000000000004},"min":{"1min":0.40799999999999997,"5min":0.222,"15min":0.222},"max":{"1min":0.78700000000000003,"5min":0.80800000000000005,"15min":1.1120000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.30199999999999999,"5min":0.29999999999999999,"15min":0.28799999999999998},"max":{"1min":0.73399999999999999,"5min":0.875,"15min":2.524},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Oct 6 04:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.52100000000000002,"15min":0.54100000000000004},"min":{"1min":0.34200000000000003,"5min":0.316,"15min":0.316},"max":{"1min":0.69599999999999995,"5min":0.85899999999999999,"15min":1.0389999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54500000000000004,"15min":0.55600000000000005},"min":{"1min":0.32300000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68200000000000005,"5min":0.76600000000000001,"15min":1.1439999999999999},"last":0.56499999999999995}]},{"osd":3,"last update":"Fri Oct 6 04:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56100000000000005,"15min":0.56899999999999995},"min":{"1min":0.45700000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":2.9359999999999999},"last":0.438},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61599999999999999,"15min":0.59599999999999997},"min":{"1min":0.45300000000000001,"5min":0.377,"15min":0.28999999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.0620000000000001},"last":0.51400000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.44900000000000001,"15min":0.47699999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":1.1619999999999999},"last":0.314},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56799999999999995,"15min":0.54500000000000004},"min":{"1min":0.36799999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":1.1950000000000001},"last":0.48999999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46899999999999997,"15min":0.497},"min":{"1min":0.30099999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70599999999999996,"5min":0.78300000000000003,"15min":0.94099999999999995},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56000000000000005,"15min":0.57899999999999996},"min":{"1min":0.39000000000000001,"5min":0.38400000000000001,"15min":0.32900000000000001},"max":{"1min":0.751,"5min":0.84999999999999998,"15min":0.97899999999999998},"last":0.36399999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.52500000000000002,"15min":0.53500000000000003},"min":{"1min":0.35099999999999998,"5min":0.33500000000000002,"15min":0.28699999999999998},"max":{"1min":0.81100000000000005,"5min":0.84899999999999998,"15min":1.1830000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58099999999999996,"15min":0.58599999999999997},"min":{"1min":0.376,"5min":0.375,"15min":0.311},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":1.29},"last":0.378}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32429,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13369344,"data_stored":49512965,"data_compressed":6345911,"data_compressed_allocated":8089600,"data_compressed_original":44290048,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8646656,"data_stored":37038012,"data_compressed":4869542,"data_compressed_allocated":6230016,"data_compressed_original":34631680,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3923968,"data_stored":16797884,"data_compressed":2299517,"data_compressed_allocated":2953216,"data_compressed_original":15835136,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12926976,"data_stored":47356979,"data_compressed":5951348,"data_compressed_allocated":7700480,"data_compressed_original":42160128,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":21995520,"data_stored":88324981,"data_compressed":12367125,"data_compressed_allocated":15323136,"data_compressed_original":81711104,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":11075584,"data_stored":46041285,"data_compressed":6106433,"data_compressed_allocated":7659520,"data_compressed_original":42663936,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1855488,"data_stored":8388720,"data_compressed":1228027,"data_compressed_allocated":1515520,"data_compressed_original":8056832,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":10952704,"data_stored":42660206,"data_compressed":6046013,"data_compressed_allocated":7446528,"data_compressed_original":39194624,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":181956608,"data_stored":203156358,"data_compressed":20456538,"data_compressed_allocated":22609920,"data_compressed_original":49975296,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":274661376,"data_stored":300540492,"data_compressed":28008712,"data_compressed_allocated":30793728,"data_compressed_original":65601536,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139608064,"data_stored":158511793,"data_compressed":16956544,"data_compressed_allocated":18817024,"data_compressed_original":43261952,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":269180928,"data_stored":297077873,"data_compressed":26845461,"data_compressed_allocated":29696000,"data_compressed_original":66433024,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":266186752,"data_stored":290581266,"data_compressed":20904721,"data_compressed_allocated":23318528,"data_compressed_original":55640064,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":181829632,"data_stored":199735195,"data_compressed":16358106,"data_compressed_allocated":18124800,"data_compressed_original":41304064,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":166477824,"data_stored":179521845,"data_compressed":14708055,"data_compressed_allocated":16289792,"data_compressed_original":34680832,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":203251712,"data_stored":216547248,"data_compressed":16284911,"data_compressed_allocated":17920000,"data_compressed_original":36777984,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:12:04.514 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-06T04:12:04.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.627+0000 7f5d8c663700 1 Processor -- start 2023-10-06T04:12:04.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.627+0000 7f5d8c663700 1 -- start start 2023-10-06T04:12:04.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.628+0000 7f5d8c663700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5d8406dc40 0x7f5d8414fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:04.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.628+0000 7f5d8c663700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d8406c1f0 0x7f5d84150240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:04.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.628+0000 7f5d8c663700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d840c9780 con 0x7f5d8406cbb0 2023-10-06T04:12:04.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.628+0000 7f5d8c663700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d840c8c40 con 0x7f5d8406dc40 2023-10-06T04:12:04.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.628+0000 7f5d8c663700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8406e0c0 con 0x7f5d8406c1f0 2023-10-06T04:12:04.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.628+0000 7f5d8a3ff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f5d8406cbb0 0x7f5d8406d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58942/0 (socket says 172.21.15.157:58942) 2023-10-06T04:12:04.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.628+0000 7f5d8a3ff700 1 -- 172.21.15.157:0/1079362742 learned_addr learned my addr 172.21.15.157:0/1079362742 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:12:04.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.628+0000 7f5d89bfe700 1 --2- 172.21.15.157:0/1079362742 >> v2:172.21.15.175:3300/0 conn(0x7f5d8406dc40 0x7f5d8414fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:04.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.629+0000 7f5d8ac00700 1 --2- 172.21.15.157:0/1079362742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d8406c1f0 0x7f5d84150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:04.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.629+0000 7f5d893fd700 1 -- 172.21.15.157:0/1079362742 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1924086058 0 0) 0x7f5d840c9780 con 0x7f5d8406cbb0 2023-10-06T04:12:04.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.629+0000 7f5d893fd700 1 -- 172.21.15.157:0/1079362742 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d70003570 con 0x7f5d8406cbb0 2023-10-06T04:12:04.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.629+0000 7f5d893fd700 1 -- 172.21.15.157:0/1079362742 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1287612488 0 0) 0x7f5d70003570 con 0x7f5d8406cbb0 2023-10-06T04:12:04.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.629+0000 7f5d893fd700 1 -- 172.21.15.157:0/1079362742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d8406c1f0 msgr2=0x7f5d84150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:12:04.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.629+0000 7f5d893fd700 1 --2- 172.21.15.157:0/1079362742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d8406c1f0 0x7f5d84150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:04.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.629+0000 7f5d893fd700 1 -- 172.21.15.157:0/1079362742 >> v2:172.21.15.175:3300/0 conn(0x7f5d8406dc40 msgr2=0x7f5d8414fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:12:04.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.629+0000 7f5d893fd700 1 --2- 172.21.15.157:0/1079362742 >> v2:172.21.15.175:3300/0 conn(0x7f5d8406dc40 0x7f5d8414fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:04.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.629+0000 7f5d893fd700 1 -- 172.21.15.157:0/1079362742 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d840c8e50 con 0x7f5d8406cbb0 2023-10-06T04:12:04.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.629+0000 7f5d893fd700 1 -- 172.21.15.157:0/1079362742 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f5d80001470 con 0x7f5d8406cbb0 2023-10-06T04:12:04.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.629+0000 7f5d8ac00700 1 --2- 172.21.15.157:0/1079362742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d8406c1f0 0x7f5d84150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:12:04.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.629+0000 7f5d89bfe700 1 --2- 172.21.15.157:0/1079362742 >> v2:172.21.15.175:3300/0 conn(0x7f5d8406dc40 0x7f5d8414fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:12:04.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.629+0000 7f5d893fd700 1 -- 172.21.15.157:0/1079362742 >> v1:172.21.15.157:6789/0 conn(0x7f5d8406cbb0 legacy=0x7f5d8406d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:04.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.629+0000 7f5d893fd700 1 --2- 172.21.15.157:0/1079362742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d70003a60 0x7f5d70003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:04.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.629+0000 7f5d893fd700 1 --2- 172.21.15.157:0/1079362742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d70004810 0x7f5d70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:04.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.629+0000 7f5d893fd700 1 --2- 172.21.15.157:0/1079362742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d8406c1f0 0x7f5d700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:04.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.629+0000 7f5d893fd700 1 -- 172.21.15.157:0/1079362742 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d840c8c40 con 0x7f5d70003a60 2023-10-06T04:12:04.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.629+0000 7f5d893fd700 1 -- 172.21.15.157:0/1079362742 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8406e0c0 con 0x7f5d70004810 2023-10-06T04:12:04.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.629+0000 7f5d893fd700 1 -- 172.21.15.157:0/1079362742 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7000ba60 con 0x7f5d8406c1f0 2023-10-06T04:12:04.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.629+0000 7f5d8a3ff700 1 --2- 172.21.15.157:0/1079362742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d70003a60 0x7f5d70003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:04.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.629+0000 7f5d89bfe700 1 --2- 172.21.15.157:0/1079362742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d70004810 0x7f5d70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:04.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.629+0000 7f5d8ac00700 1 --2- 172.21.15.157:0/1079362742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d8406c1f0 0x7f5d700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:04.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.629+0000 7f5d8a3ff700 1 -- 172.21.15.157:0/1079362742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d8406c1f0 msgr2=0x7f5d700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:12:04.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.629+0000 7f5d8a3ff700 1 --2- 172.21.15.157:0/1079362742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d8406c1f0 0x7f5d700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:04.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.629+0000 7f5d8a3ff700 1 -- 172.21.15.157:0/1079362742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d70004810 msgr2=0x7f5d70004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:12:04.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.629+0000 7f5d8a3ff700 1 --2- 172.21.15.157:0/1079362742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d70004810 0x7f5d70004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:04.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.629+0000 7f5d8a3ff700 1 -- 172.21.15.157:0/1079362742 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7c009a20 con 0x7f5d70003a60 2023-10-06T04:12:04.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.629+0000 7f5d8ac00700 1 --2- 172.21.15.157:0/1079362742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d8406c1f0 0x7f5d700051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:12:04.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.629+0000 7f5d8a3ff700 1 --2- 172.21.15.157:0/1079362742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d70003a60 0x7f5d70003e70 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f5d8000c3c0 tx=0x7f5d840b1020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb1b405507decaf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:12:04.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.630+0000 7f5d893fd700 1 -- 172.21.15.157:0/1079362742 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8000cc90 con 0x7f5d70003a60 2023-10-06T04:12:04.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.630+0000 7f5d893fd700 1 -- 172.21.15.157:0/1079362742 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8000ce30 con 0x7f5d70003a60 2023-10-06T04:12:04.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.630+0000 7f5d893fd700 1 -- 172.21.15.157:0/1079362742 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d80005b70 con 0x7f5d70003a60 2023-10-06T04:12:04.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.630+0000 7f5d8c663700 1 -- 172.21.15.157:0/1079362742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d70003a60 msgr2=0x7f5d70003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:04.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.630+0000 7f5d8c663700 1 --2- 172.21.15.157:0/1079362742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d70003a60 0x7f5d70003e70 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f5d8000c3c0 tx=0x7f5d840b1020 comp rx=0 tx=0).stop 2023-10-06T04:12:04.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.630+0000 7f5d8ac00700 1 -- 172.21.15.157:0/1079362742 reap_dead start 2023-10-06T04:12:04.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.630+0000 7f5d8c663700 1 -- 172.21.15.157:0/1079362742 shutdown_connections 2023-10-06T04:12:04.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.630+0000 7f5d8c663700 1 -- 172.21.15.157:0/1079362742 >> 172.21.15.157:0/1079362742 conn(0x7f5d8406f230 msgr2=0x7f5d8406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:12:04.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.630+0000 7f5d8c663700 1 -- 172.21.15.157:0/1079362742 shutdown_connections 2023-10-06T04:12:04.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.630+0000 7f5d8c663700 1 -- 172.21.15.157:0/1079362742 wait complete. 2023-10-06T04:12:04.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.631+0000 7f5d8c663700 1 Processor -- start 2023-10-06T04:12:04.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.631+0000 7f5d8c663700 1 -- start start 2023-10-06T04:12:04.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.632+0000 7f5d8c663700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d70003a60 0x7f5d84146790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:04.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.632+0000 7f5d8c663700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d84148550 0x7f5d84167170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:04.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.632+0000 7f5d8c663700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d84152020 0x7f5d840cce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:04.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.632+0000 7f5d8c663700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d84147d40 con 0x7f5d70003a60 2023-10-06T04:12:04.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.632+0000 7f5d8c663700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d84146300 con 0x7f5d84148550 2023-10-06T04:12:04.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.632+0000 7f5d8c663700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d84152490 con 0x7f5d84152020 2023-10-06T04:12:04.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.632+0000 7f5d8a3ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d70003a60 0x7f5d84146790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:04.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.632+0000 7f5d8a3ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d70003a60 0x7f5d84146790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56110/0 (socket says 172.21.15.157:56110) 2023-10-06T04:12:04.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.632+0000 7f5d8a3ff700 1 -- 172.21.15.157:0/3343906428 learned_addr learned my addr 172.21.15.157:0/3343906428 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:12:04.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.632+0000 7f5d89bfe700 1 --2- 172.21.15.157:0/3343906428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d84148550 0x7f5d84167170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:04.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.632+0000 7f5d8ac00700 1 --2- 172.21.15.157:0/3343906428 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d84152020 0x7f5d840cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:04.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.633+0000 7f5d8a3ff700 1 -- 172.21.15.157:0/3343906428 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d84152020 msgr2=0x7f5d840cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:04.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.633+0000 7f5d8a3ff700 1 --2- 172.21.15.157:0/3343906428 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d84152020 0x7f5d840cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:04.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.633+0000 7f5d8a3ff700 1 -- 172.21.15.157:0/3343906428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d84148550 msgr2=0x7f5d84167170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:04.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.633+0000 7f5d8a3ff700 1 --2- 172.21.15.157:0/3343906428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d84148550 0x7f5d84167170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:04.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.633+0000 7f5d8a3ff700 1 -- 172.21.15.157:0/3343906428 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d84147f90 con 0x7f5d70003a60 2023-10-06T04:12:04.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.633+0000 7f5d8a3ff700 1 --2- 172.21.15.157:0/3343906428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d70003a60 0x7f5d84146790 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f5d8000c9b0 tx=0x7f5d80015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:12:04.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.633+0000 7f5d7a7fc700 1 -- 172.21.15.157:0/3343906428 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8000cc90 con 0x7f5d70003a60 2023-10-06T04:12:04.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.633+0000 7f5d7a7fc700 1 -- 172.21.15.157:0/3343906428 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d800072e0 con 0x7f5d70003a60 2023-10-06T04:12:04.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.633+0000 7f5d8c663700 1 -- 172.21.15.157:0/3343906428 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d84145cb0 con 0x7f5d70003a60 2023-10-06T04:12:04.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.633+0000 7f5d7a7fc700 1 -- 172.21.15.157:0/3343906428 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d800076a0 con 0x7f5d70003a60 2023-10-06T04:12:04.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.633+0000 7f5d8c663700 1 -- 172.21.15.157:0/3343906428 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d840c3d30 con 0x7f5d70003a60 2023-10-06T04:12:04.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.634+0000 7f5d8c663700 1 -- 172.21.15.157:0/3343906428 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d84146150 con 0x7f5d70003a60 2023-10-06T04:12:04.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.634+0000 7f5d7a7fc700 1 -- 172.21.15.157:0/3343906428 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5d80019070 con 0x7f5d70003a60 2023-10-06T04:12:04.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.635+0000 7f5d7a7fc700 1 --2- 172.21.15.157:0/3343906428 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5d68066f30 0x7f5d680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:04.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.635+0000 7f5d7a7fc700 1 -- 172.21.15.157:0/3343906428 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f5d800856e0 con 0x7f5d70003a60 2023-10-06T04:12:04.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.635+0000 7f5d89bfe700 1 --2- 172.21.15.157:0/3343906428 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5d68066f30 0x7f5d680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:04.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.635+0000 7f5d89bfe700 1 --2- 172.21.15.157:0/3343906428 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5d68066f30 0x7f5d680693e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5d74006f00 tx=0x7f5d740091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:12:04.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.639+0000 7f5d7a7fc700 1 -- 172.21.15.157:0/3343906428 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5d80052560 con 0x7f5d70003a60 2023-10-06T04:12:04.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.822+0000 7f5d8c663700 1 -- 172.21.15.157:0/3343906428 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f5d840c9130 con 0x7f5d70003a60 2023-10-06T04:12:04.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.826+0000 7f5d7a7fc700 1 -- 172.21.15.157:0/3343906428 <== mon.0 v2:172.21.15.157:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f5d80052560 con 0x7f5d70003a60 2023-10-06T04:12:04.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.826+0000 7f5d7a7fc700 1 -- 172.21.15.157:0/3343906428 <== mon.0 v2:172.21.15.157:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) v1 ==== 123+0+0 (secure 0 0 0) 0x7f5d80055b80 con 0x7f5d70003a60 2023-10-06T04:12:04.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.829+0000 7f5d8c663700 1 -- 172.21.15.157:0/3343906428 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5d68066f30 msgr2=0x7f5d680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:04.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.829+0000 7f5d8c663700 1 --2- 172.21.15.157:0/3343906428 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5d68066f30 0x7f5d680693e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5d74006f00 tx=0x7f5d740091a0 comp rx=0 tx=0).stop 2023-10-06T04:12:04.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.829+0000 7f5d8c663700 1 -- 172.21.15.157:0/3343906428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d70003a60 msgr2=0x7f5d84146790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:04.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.829+0000 7f5d8c663700 1 --2- 172.21.15.157:0/3343906428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d70003a60 0x7f5d84146790 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f5d8000c9b0 tx=0x7f5d80015000 comp rx=0 tx=0).stop 2023-10-06T04:12:04.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.829+0000 7f5d8c663700 1 -- 172.21.15.157:0/3343906428 shutdown_connections 2023-10-06T04:12:04.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.829+0000 7f5d8c663700 1 --2- 172.21.15.157:0/3343906428 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f5d68066f30 0x7f5d680693e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:04.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.829+0000 7f5d8c663700 1 --2- 172.21.15.157:0/3343906428 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d84152020 0x7f5d840cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:04.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.829+0000 7f5d8c663700 1 --2- 172.21.15.157:0/3343906428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d70003a60 0x7f5d84146790 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:04.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.829+0000 7f5d8c663700 1 --2- 172.21.15.157:0/3343906428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d84148550 0x7f5d84167170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:04.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.829+0000 7f5d8c663700 1 -- 172.21.15.157:0/3343906428 >> 172.21.15.157:0/3343906428 conn(0x7f5d8406f230 msgr2=0x7f5d8414dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:12:04.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.829+0000 7f5d8c663700 1 -- 172.21.15.157:0/3343906428 shutdown_connections 2023-10-06T04:12:04.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.829+0000 7f5d8c663700 1 -- 172.21.15.157:0/3343906428 wait complete. 2023-10-06T04:12:04.842 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T04:12:04.842 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:12:04.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.955+0000 7f91fd47b700 1 Processor -- start 2023-10-06T04:12:04.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.956+0000 7f91fd47b700 1 -- start start 2023-10-06T04:12:04.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.956+0000 7f91fd47b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91f8159df0 0x7f91f814fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:04.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.956+0000 7f91fd47b700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91f806ced0 con 0x7f91f80b2110 2023-10-06T04:12:04.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.956+0000 7f91fd47b700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91f80afb10 con 0x7f91f80b1d40 2023-10-06T04:12:04.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.956+0000 7f91fd47b700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f815a270 con 0x7f91f8159df0 2023-10-06T04:12:04.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.956+0000 7f91ee7fc700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f91f80b2110 0x7f91f806c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58950/0 (socket says 172.21.15.157:58950) 2023-10-06T04:12:04.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.956+0000 7f91ee7fc700 1 -- 172.21.15.157:0/3454470236 learned_addr learned my addr 172.21.15.157:0/3454470236 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:12:04.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.957+0000 7f91f77fe700 1 --2- 172.21.15.157:0/3454470236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91f8159df0 0x7f91f814fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:04.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.957+0000 7f91f67fc700 1 -- 172.21.15.157:0/3454470236 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 882975467 0 0) 0x7f91f806ced0 con 0x7f91f80b2110 2023-10-06T04:12:04.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.957+0000 7f91f67fc700 1 -- 172.21.15.157:0/3454470236 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91e4003570 con 0x7f91f80b2110 2023-10-06T04:12:04.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.957+0000 7f91f67fc700 1 -- 172.21.15.157:0/3454470236 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1658319739 0 0) 0x7f91f80afb10 con 0x7f91f80b1d40 2023-10-06T04:12:04.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.957+0000 7f91f67fc700 1 -- 172.21.15.157:0/3454470236 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91f806ced0 con 0x7f91f80b1d40 2023-10-06T04:12:04.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.957+0000 7f91f67fc700 1 -- 172.21.15.157:0/3454470236 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3755585143 0 0) 0x7f91e4003570 con 0x7f91f80b2110 2023-10-06T04:12:04.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.957+0000 7f91f67fc700 1 -- 172.21.15.157:0/3454470236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91f8159df0 msgr2=0x7f91f814fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:12:04.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.957+0000 7f91f67fc700 1 --2- 172.21.15.157:0/3454470236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91f8159df0 0x7f91f814fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:04.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.957+0000 7f91f67fc700 1 -- 172.21.15.157:0/3454470236 >> v1:172.21.15.175:6789/0 conn(0x7f91f80b1d40 legacy=0x7f91f80af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:04.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.957+0000 7f91f67fc700 1 -- 172.21.15.157:0/3454470236 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f806cc10 con 0x7f91f80b2110 2023-10-06T04:12:04.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.957+0000 7f91f67fc700 1 -- 172.21.15.157:0/3454470236 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 484040021 0 0) 0x7f91e0001470 con 0x7f91f80b2110 2023-10-06T04:12:04.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.957+0000 7f91f67fc700 1 -- 172.21.15.157:0/3454470236 >> v1:172.21.15.157:6789/0 conn(0x7f91f80b2110 legacy=0x7f91f806c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:04.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.957+0000 7f91f67fc700 1 --2- 172.21.15.157:0/3454470236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91e4003c20 0x7f91e4004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:04.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.957+0000 7f91f67fc700 1 --2- 172.21.15.157:0/3454470236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91f8159df0 0x7f91e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:04.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.957+0000 7f91f67fc700 1 --2- 172.21.15.157:0/3454470236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91e4004d50 0x7f91e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:04.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.957+0000 7f91f67fc700 1 -- 172.21.15.157:0/3454470236 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f815a270 con 0x7f91e4004d50 2023-10-06T04:12:04.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.957+0000 7f91f67fc700 1 -- 172.21.15.157:0/3454470236 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e400bb40 con 0x7f91e4003c20 2023-10-06T04:12:04.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.957+0000 7f91f67fc700 1 -- 172.21.15.157:0/3454470236 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e400bd00 con 0x7f91f8159df0 2023-10-06T04:12:04.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.957+0000 7f91f77fe700 1 --2- 172.21.15.157:0/3454470236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91e4004d50 0x7f91e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:04.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.957+0000 7f91f6ffd700 1 --2- 172.21.15.157:0/3454470236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91e4003c20 0x7f91e4004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:04.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.957+0000 7f91ee7fc700 1 --2- 172.21.15.157:0/3454470236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91f8159df0 0x7f91e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:04.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.957+0000 7f91f77fe700 1 -- 172.21.15.157:0/3454470236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91f8159df0 msgr2=0x7f91e4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:12:04.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.957+0000 7f91f77fe700 1 --2- 172.21.15.157:0/3454470236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91f8159df0 0x7f91e4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:04.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.957+0000 7f91f77fe700 1 -- 172.21.15.157:0/3454470236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91e4003c20 msgr2=0x7f91e4004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:12:04.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.957+0000 7f91f77fe700 1 --2- 172.21.15.157:0/3454470236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91e4003c20 0x7f91e4004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:04.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.957+0000 7f91f77fe700 1 -- 172.21.15.157:0/3454470236 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e4003570 con 0x7f91e4004d50 2023-10-06T04:12:04.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.958+0000 7f91f77fe700 1 --2- 172.21.15.157:0/3454470236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91e4004d50 0x7f91e40051c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f91f0009c00 tx=0x7f91f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9622bc29ac5647b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:12:04.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.958+0000 7f91f67fc700 1 -- 172.21.15.157:0/3454470236 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f000e710 con 0x7f91e4004d50 2023-10-06T04:12:04.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.958+0000 7f91f67fc700 1 -- 172.21.15.157:0/3454470236 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f91f000e870 con 0x7f91e4004d50 2023-10-06T04:12:04.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.958+0000 7f91f67fc700 1 -- 172.21.15.157:0/3454470236 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f000ec50 con 0x7f91e4004d50 2023-10-06T04:12:04.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.958+0000 7f91fd47b700 1 -- 172.21.15.157:0/3454470236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91e4004d50 msgr2=0x7f91e40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:04.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.958+0000 7f91fd47b700 1 --2- 172.21.15.157:0/3454470236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91e4004d50 0x7f91e40051c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f91f0009c00 tx=0x7f91f00091a0 comp rx=0 tx=0).stop 2023-10-06T04:12:04.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.958+0000 7f91f77fe700 1 -- 172.21.15.157:0/3454470236 reap_dead start 2023-10-06T04:12:04.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.958+0000 7f91fd47b700 1 -- 172.21.15.157:0/3454470236 shutdown_connections 2023-10-06T04:12:04.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.958+0000 7f91fd47b700 1 -- 172.21.15.157:0/3454470236 >> 172.21.15.157:0/3454470236 conn(0x7f91f80cc6e0 msgr2=0x7f91f80ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:12:04.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.958+0000 7f91fd47b700 1 -- 172.21.15.157:0/3454470236 shutdown_connections 2023-10-06T04:12:04.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.958+0000 7f91fd47b700 1 -- 172.21.15.157:0/3454470236 wait complete. 2023-10-06T04:12:04.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.959+0000 7f91fd47b700 1 Processor -- start 2023-10-06T04:12:04.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.960+0000 7f91fd47b700 1 -- start start 2023-10-06T04:12:04.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.960+0000 7f91fd47b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91f8158e60 0x7f91f81533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:04.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.960+0000 7f91fd47b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91f8151aa0 0x7f91f8166b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:04.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.960+0000 7f91fd47b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91f81670a0 0x7f91f8169e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:04.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.960+0000 7f91fd47b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f8166450 con 0x7f91f8151aa0 2023-10-06T04:12:04.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.960+0000 7f91fd47b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f81666d0 con 0x7f91f81670a0 2023-10-06T04:12:04.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.960+0000 7f91fd47b700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f8158280 con 0x7f91f8158e60 2023-10-06T04:12:04.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.960+0000 7f91ee7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91f8151aa0 0x7f91f8166b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:04.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.960+0000 7f91ee7fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91f8151aa0 0x7f91f8166b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56128/0 (socket says 172.21.15.157:56128) 2023-10-06T04:12:04.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.960+0000 7f91ee7fc700 1 -- 172.21.15.157:0/1627755082 learned_addr learned my addr 172.21.15.157:0/1627755082 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:12:04.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.960+0000 7f91f6ffd700 1 --2- 172.21.15.157:0/1627755082 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91f8158e60 0x7f91f81533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:04.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.960+0000 7f91f77fe700 1 --2- 172.21.15.157:0/1627755082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91f81670a0 0x7f91f8169e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:04.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.961+0000 7f91ee7fc700 1 -- 172.21.15.157:0/1627755082 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91f8158e60 msgr2=0x7f91f81533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:04.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.961+0000 7f91ee7fc700 1 --2- 172.21.15.157:0/1627755082 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91f8158e60 0x7f91f81533b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:04.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.961+0000 7f91ee7fc700 1 -- 172.21.15.157:0/1627755082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91f81670a0 msgr2=0x7f91f8169e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:04.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.961+0000 7f91ee7fc700 1 --2- 172.21.15.157:0/1627755082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91f81670a0 0x7f91f8169e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:04.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.961+0000 7f91ee7fc700 1 -- 172.21.15.157:0/1627755082 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f8169a10 con 0x7f91f8151aa0 2023-10-06T04:12:04.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.961+0000 7f91ee7fc700 1 --2- 172.21.15.157:0/1627755082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91f8151aa0 0x7f91f8166b60 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f91e00015a0 tx=0x7f91e0001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:12:04.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.961+0000 7f91ef7fe700 1 -- 172.21.15.157:0/1627755082 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e0018030 con 0x7f91f8151aa0 2023-10-06T04:12:04.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.961+0000 7f91ef7fe700 1 -- 172.21.15.157:0/1627755082 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f91e0007850 con 0x7f91f8151aa0 2023-10-06T04:12:04.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.961+0000 7f91fd47b700 1 -- 172.21.15.157:0/1627755082 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f81539d0 con 0x7f91f8151aa0 2023-10-06T04:12:04.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.961+0000 7f91ef7fe700 1 -- 172.21.15.157:0/1627755082 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e000fa90 con 0x7f91f8151aa0 2023-10-06T04:12:04.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.961+0000 7f91fd47b700 1 -- 172.21.15.157:0/1627755082 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f8150b00 con 0x7f91f8151aa0 2023-10-06T04:12:04.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.962+0000 7f91fd47b700 1 -- 172.21.15.157:0/1627755082 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91f80896c0 con 0x7f91f8151aa0 2023-10-06T04:12:04.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.962+0000 7f91ef7fe700 1 -- 172.21.15.157:0/1627755082 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f91e001b070 con 0x7f91f8151aa0 2023-10-06T04:12:04.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.963+0000 7f91ef7fe700 1 --2- 172.21.15.157:0/1627755082 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f91d8066f30 0x7f91d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:04.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.963+0000 7f91ef7fe700 1 -- 172.21.15.157:0/1627755082 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7f91e0019070 con 0x7f91f8151aa0 2023-10-06T04:12:04.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.963+0000 7f91f6ffd700 1 --2- 172.21.15.157:0/1627755082 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f91d8066f30 0x7f91d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:04.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.963+0000 7f91f6ffd700 1 --2- 172.21.15.157:0/1627755082 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f91d8066f30 0x7f91d80693e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f91e80018b0 tx=0x7f91e80017a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:12:04.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:04.967+0000 7f91ef7fe700 1 -- 172.21.15.157:0/1627755082 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f91e00516e0 con 0x7f91f8151aa0 2023-10-06T04:12:05.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.195+0000 7f91fd47b700 1 -- 172.21.15.157:0/1627755082 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91f8166810 con 0x7f91f8151aa0 2023-10-06T04:12:05.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.196+0000 7f91ef7fe700 1 -- 172.21.15.157:0/1627755082 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+6148 (secure 0 0 0) 0x7f91e0054d00 con 0x7f91f8151aa0 2023-10-06T04:12:05.198 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:12:05.198 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":92,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6417,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/608486467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":608486467},{"type":"v1","addr":"172.21.15.157:6834","nonce":608486467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:10:56.835216+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6270,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:active","state_seq":19,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6270":{"gid":6270,"name":"e","rank":0,"incarnation":86,"state":"up:active","state_seq":19,"addr":"172.21.15.157:6837/1036901172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1036901172},{"type":"v1","addr":"172.21.15.157:6837","nonce":1036901172}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:12:05.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.198+0000 7f91fd47b700 1 -- 172.21.15.157:0/1627755082 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f91d8066f30 msgr2=0x7f91d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:05.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.198+0000 7f91fd47b700 1 --2- 172.21.15.157:0/1627755082 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f91d8066f30 0x7f91d80693e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f91e80018b0 tx=0x7f91e80017a0 comp rx=0 tx=0).stop 2023-10-06T04:12:05.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.198+0000 7f91fd47b700 1 -- 172.21.15.157:0/1627755082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91f8151aa0 msgr2=0x7f91f8166b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:05.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.198+0000 7f91fd47b700 1 --2- 172.21.15.157:0/1627755082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91f8151aa0 0x7f91f8166b60 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f91e00015a0 tx=0x7f91e0001e10 comp rx=0 tx=0).stop 2023-10-06T04:12:05.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.198+0000 7f91fd47b700 1 -- 172.21.15.157:0/1627755082 shutdown_connections 2023-10-06T04:12:05.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.198+0000 7f91fd47b700 1 --2- 172.21.15.157:0/1627755082 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f91d8066f30 0x7f91d80693e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:05.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.198+0000 7f91fd47b700 1 --2- 172.21.15.157:0/1627755082 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91f81670a0 0x7f91f8169e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:05.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.198+0000 7f91fd47b700 1 --2- 172.21.15.157:0/1627755082 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91f8158e60 0x7f91f81533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:05.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.198+0000 7f91fd47b700 1 --2- 172.21.15.157:0/1627755082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91f8151aa0 0x7f91f8166b60 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:05.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.198+0000 7f91fd47b700 1 -- 172.21.15.157:0/1627755082 >> 172.21.15.157:0/1627755082 conn(0x7f91f80cc6e0 msgr2=0x7f91f814fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:12:05.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.198+0000 7f91fd47b700 1 -- 172.21.15.157:0/1627755082 shutdown_connections 2023-10-06T04:12:05.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.198+0000 7f91fd47b700 1 -- 172.21.15.157:0/1627755082 wait complete. 2023-10-06T04:12:05.201 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 92 2023-10-06T04:12:05.211 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:12:05.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.324+0000 7efd8c05c700 1 Processor -- start 2023-10-06T04:12:05.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.324+0000 7efd8c05c700 1 -- start start 2023-10-06T04:12:05.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.325+0000 7efd8c05c700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7efd8406cbb0 0x7efd8406d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:05.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.325+0000 7efd8c05c700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd8406c1f0 0x7efd8406da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:05.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.325+0000 7efd8c05c700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd840c9780 con 0x7efd8406cf80 2023-10-06T04:12:05.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.325+0000 7efd8c05c700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd840c8c40 con 0x7efd8406cbb0 2023-10-06T04:12:05.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.325+0000 7efd8c05c700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8406e0c0 con 0x7efd8406c1f0 2023-10-06T04:12:05.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.326+0000 7efd895f7700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd8406c1f0 0x7efd8406da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:05.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.326+0000 7efd89df8700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7efd8406cbb0 0x7efd8406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:05.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.326+0000 7efd89df8700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7efd8406cbb0 0x7efd8406d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.157:41084/0 (socket says 172.21.15.157:41084) 2023-10-06T04:12:05.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.326+0000 7efd895f7700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd8406c1f0 0x7efd8406da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.157:34100/0 (socket says 172.21.15.157:34100) 2023-10-06T04:12:05.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.326+0000 7efd89df8700 1 -- 172.21.15.157:0/2801574003 learned_addr learned my addr 172.21.15.157:0/2801574003 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:12:05.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.326+0000 7efd89df8700 1 -- 172.21.15.157:0/2801574003 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd8406c1f0 msgr2=0x7efd8406da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:12:05.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.326+0000 7efd89df8700 1 --2- 172.21.15.157:0/2801574003 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd8406c1f0 0x7efd8406da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:05.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.326+0000 7efd89df8700 1 -- 172.21.15.157:0/2801574003 >> v1:172.21.15.175:6789/0 conn(0x7efd8406cf80 legacy=0x7efd8414fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:05.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.326+0000 7efd89df8700 1 -- 172.21.15.157:0/2801574003 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd840c8e50 con 0x7efd8406cbb0 2023-10-06T04:12:05.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.326+0000 7efd895f7700 1 --2- 172.21.15.157:0/2801574003 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd8406c1f0 0x7efd8406da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:12:05.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.326+0000 7efd89df8700 1 --2- 172.21.15.157:0/2801574003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd8406cbb0 0x7efd8406d530 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efd74009f50 tx=0x7efd74009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4405c7a89ed83c94 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:12:05.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.326+0000 7efd88df6700 1 -- 172.21.15.157:0/2801574003 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7400e070 con 0x7efd8406cbb0 2023-10-06T04:12:05.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.326+0000 7efd88df6700 1 -- 172.21.15.157:0/2801574003 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7efd7400ba50 con 0x7efd8406cbb0 2023-10-06T04:12:05.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.326+0000 7efd88df6700 1 -- 172.21.15.157:0/2801574003 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd740092e0 con 0x7efd8406cbb0 2023-10-06T04:12:05.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.327+0000 7efd8c05c700 1 -- 172.21.15.157:0/2801574003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd8406cbb0 msgr2=0x7efd8406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:05.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.327+0000 7efd8c05c700 1 --2- 172.21.15.157:0/2801574003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd8406cbb0 0x7efd8406d530 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efd74009f50 tx=0x7efd74009840 comp rx=0 tx=0).stop 2023-10-06T04:12:05.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.327+0000 7efd8c05c700 1 -- 172.21.15.157:0/2801574003 shutdown_connections 2023-10-06T04:12:05.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.327+0000 7efd8c05c700 1 --2- 172.21.15.157:0/2801574003 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd8406cbb0 0x7efd8406d530 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:05.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.327+0000 7efd8c05c700 1 --2- 172.21.15.157:0/2801574003 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd8406c1f0 0x7efd8406da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:05.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.327+0000 7efd8c05c700 1 -- 172.21.15.157:0/2801574003 >> 172.21.15.157:0/2801574003 conn(0x7efd8406f230 msgr2=0x7efd8406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:12:05.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.327+0000 7efd8c05c700 1 -- 172.21.15.157:0/2801574003 shutdown_connections 2023-10-06T04:12:05.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.327+0000 7efd8c05c700 1 -- 172.21.15.157:0/2801574003 wait complete. 2023-10-06T04:12:05.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.328+0000 7efd8c05c700 1 Processor -- start 2023-10-06T04:12:05.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.328+0000 7efd8c05c700 1 -- start start 2023-10-06T04:12:05.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.329+0000 7efd8c05c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd8406c1f0 0x7efd840c5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:05.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.329+0000 7efd8c05c700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd8406cbb0 0x7efd84153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:05.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.329+0000 7efd8c05c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd8406cf80 0x7efd84154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:05.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.329+0000 7efd8c05c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd840c5710 con 0x7efd8406cf80 2023-10-06T04:12:05.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.329+0000 7efd8c05c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd840c1030 con 0x7efd8406c1f0 2023-10-06T04:12:05.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.329+0000 7efd8c05c700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd840c4b00 con 0x7efd8406cbb0 2023-10-06T04:12:05.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.329+0000 7efd8a5f9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd8406cf80 0x7efd84154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:05.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.329+0000 7efd8a5f9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd8406cf80 0x7efd84154340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56136/0 (socket says 172.21.15.157:56136) 2023-10-06T04:12:05.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.329+0000 7efd8a5f9700 1 -- 172.21.15.157:0/2412961329 learned_addr learned my addr 172.21.15.157:0/2412961329 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:12:05.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.329+0000 7efd89df8700 1 --2- 172.21.15.157:0/2412961329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd8406c1f0 0x7efd840c5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:05.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.329+0000 7efd895f7700 1 --2- 172.21.15.157:0/2412961329 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd8406cbb0 0x7efd84153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:05.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.329+0000 7efd8a5f9700 1 -- 172.21.15.157:0/2412961329 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd8406cbb0 msgr2=0x7efd84153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:05.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.329+0000 7efd8a5f9700 1 --2- 172.21.15.157:0/2412961329 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd8406cbb0 0x7efd84153e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:05.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.329+0000 7efd8a5f9700 1 -- 172.21.15.157:0/2412961329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd8406c1f0 msgr2=0x7efd840c5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:05.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.329+0000 7efd8a5f9700 1 --2- 172.21.15.157:0/2412961329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd8406c1f0 0x7efd840c5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:05.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.329+0000 7efd8a5f9700 1 -- 172.21.15.157:0/2412961329 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd78009420 con 0x7efd8406cf80 2023-10-06T04:12:05.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.329+0000 7efd8a5f9700 1 --2- 172.21.15.157:0/2412961329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd8406cf80 0x7efd84154340 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00c560 tx=0x7efd7c007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:12:05.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.330+0000 7efd81ffb700 1 -- 172.21.15.157:0/2412961329 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c011590 con 0x7efd8406cf80 2023-10-06T04:12:05.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.330+0000 7efd81ffb700 1 -- 172.21.15.157:0/2412961329 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7efd7c0116f0 con 0x7efd8406cf80 2023-10-06T04:12:05.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.330+0000 7efd81ffb700 1 -- 172.21.15.157:0/2412961329 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c011ad0 con 0x7efd8406cf80 2023-10-06T04:12:05.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.330+0000 7efd8c05c700 1 -- 172.21.15.157:0/2412961329 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd74009ae0 con 0x7efd8406cf80 2023-10-06T04:12:05.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.330+0000 7efd8c05c700 1 -- 172.21.15.157:0/2412961329 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd840c50c0 con 0x7efd8406cf80 2023-10-06T04:12:05.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.331+0000 7efd8c05c700 1 -- 172.21.15.157:0/2412961329 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd84089ca0 con 0x7efd8406cf80 2023-10-06T04:12:05.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.331+0000 7efd81ffb700 1 -- 172.21.15.157:0/2412961329 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efd7c01b3b0 con 0x7efd8406cf80 2023-10-06T04:12:05.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.331+0000 7efd81ffb700 1 --2- 172.21.15.157:0/2412961329 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7efd70066f30 0x7efd700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:05.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.331+0000 7efd81ffb700 1 -- 172.21.15.157:0/2412961329 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7efd7c083ef0 con 0x7efd8406cf80 2023-10-06T04:12:05.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.332+0000 7efd89df8700 1 --2- 172.21.15.157:0/2412961329 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7efd70066f30 0x7efd700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:05.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.332+0000 7efd89df8700 1 --2- 172.21.15.157:0/2412961329 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7efd70066f30 0x7efd700693e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7efd74009a60 tx=0x7efd74009840 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:12:05.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.335+0000 7efd81ffb700 1 -- 172.21.15.157:0/2412961329 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd7c050cd0 con 0x7efd8406cf80 2023-10-06T04:12:05.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.558+0000 7efd8c05c700 1 -- 172.21.15.157:0/2412961329 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd840c9130 con 0x7efd8406cf80 2023-10-06T04:12:05.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.559+0000 7efd81ffb700 1 -- 172.21.15.157:0/2412961329 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+6148 (secure 0 0 0) 0x7efd7c017070 con 0x7efd8406cf80 2023-10-06T04:12:05.561 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:12:05.561 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":92,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5126,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1799968248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1799968248},{"type":"v1","addr":"172.21.15.175:6835","nonce":1799968248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5164,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/745804271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":745804271},{"type":"v1","addr":"172.21.15.175:6839","nonce":745804271}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2461751318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2461751318},{"type":"v1","addr":"172.21.15.157:6835","nonce":2461751318}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6417,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/608486467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":608486467},{"type":"v1","addr":"172.21.15.157:6834","nonce":608486467}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:57:10.718782+0000","modified":"2023-10-06T04:10:56.835216+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6270,"mds_1":5812},"failed":[],"damaged":[],"stopped":[2,3,4],"info":{"gid_5812":{"gid":5812,"name":"d","rank":1,"incarnation":78,"state":"up:active","state_seq":19,"addr":"172.21.15.175:6837/1436765859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1436765859},{"type":"v1","addr":"172.21.15.175:6837","nonce":1436765859}]},"join_fscid":-1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6270":{"gid":6270,"name":"e","rank":0,"incarnation":86,"state":"up:active","state_seq":19,"addr":"172.21.15.157:6837/1036901172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1036901172},{"type":"v1","addr":"172.21.15.157:6837","nonce":1036901172}]},"join_fscid":-1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:12:05.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.561+0000 7efd8c05c700 1 -- 172.21.15.157:0/2412961329 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7efd70066f30 msgr2=0x7efd700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:05.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.561+0000 7efd8c05c700 1 --2- 172.21.15.157:0/2412961329 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7efd70066f30 0x7efd700693e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7efd74009a60 tx=0x7efd74009840 comp rx=0 tx=0).stop 2023-10-06T04:12:05.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.561+0000 7efd8c05c700 1 -- 172.21.15.157:0/2412961329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd8406cf80 msgr2=0x7efd84154340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:05.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.561+0000 7efd8c05c700 1 --2- 172.21.15.157:0/2412961329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd8406cf80 0x7efd84154340 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00c560 tx=0x7efd7c007ef0 comp rx=0 tx=0).stop 2023-10-06T04:12:05.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.561+0000 7efd8c05c700 1 -- 172.21.15.157:0/2412961329 shutdown_connections 2023-10-06T04:12:05.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.561+0000 7efd8c05c700 1 --2- 172.21.15.157:0/2412961329 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7efd70066f30 0x7efd700693e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:05.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.561+0000 7efd8c05c700 1 --2- 172.21.15.157:0/2412961329 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd8406c1f0 0x7efd840c5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:05.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.561+0000 7efd8c05c700 1 --2- 172.21.15.157:0/2412961329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd8406cf80 0x7efd84154340 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:05.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.561+0000 7efd8c05c700 1 --2- 172.21.15.157:0/2412961329 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd8406cbb0 0x7efd84153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:05.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.561+0000 7efd8c05c700 1 -- 172.21.15.157:0/2412961329 >> 172.21.15.157:0/2412961329 conn(0x7efd8406f230 msgr2=0x7efd8414ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:12:05.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.561+0000 7efd8c05c700 1 -- 172.21.15.157:0/2412961329 shutdown_connections 2023-10-06T04:12:05.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.561+0000 7efd8c05c700 1 -- 172.21.15.157:0/2412961329 wait complete. 2023-10-06T04:12:05.565 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 92 2023-10-06T04:12:05.574 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:12:05.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.687+0000 7fa8d6871700 1 Processor -- start 2023-10-06T04:12:05.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.687+0000 7fa8d6871700 1 -- start start 2023-10-06T04:12:05.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.688+0000 7fa8d6871700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa8d006c2d0 0x7fa8d0157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:05.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.688+0000 7fa8d6871700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8d006d8d0 con 0x7fa8d00b04d0 2023-10-06T04:12:05.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.688+0000 7fa8d6871700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8d00af8f0 con 0x7fa8d00b1e40 2023-10-06T04:12:05.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.688+0000 7fa8d6871700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8d015a690 con 0x7fa8d006c2d0 2023-10-06T04:12:05.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.688+0000 7fa8d506e700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa8d00b04d0 0x7fa8d0156ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58952/0 (socket says 172.21.15.157:58952) 2023-10-06T04:12:05.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.688+0000 7fa8d506e700 1 -- 172.21.15.157:0/1598963063 learned_addr learned my addr 172.21.15.157:0/1598963063 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:12:05.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.688+0000 7fa8d6070700 1 --2- 172.21.15.157:0/1598963063 >> v2:172.21.15.175:3300/0 conn(0x7fa8d006c2d0 0x7fa8d0157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:05.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.688+0000 7fa8d486d700 1 -- 172.21.15.157:0/1598963063 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4211880066 0 0) 0x7fa8d006d8d0 con 0x7fa8d00b04d0 2023-10-06T04:12:05.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.688+0000 7fa8d486d700 1 -- 172.21.15.157:0/1598963063 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8bc003570 con 0x7fa8d00b04d0 2023-10-06T04:12:05.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.689+0000 7fa8d486d700 1 -- 172.21.15.157:0/1598963063 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2770737306 0 0) 0x7fa8bc003570 con 0x7fa8d00b04d0 2023-10-06T04:12:05.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.689+0000 7fa8d6070700 1 -- 172.21.15.157:0/1598963063 >> v1:172.21.15.175:6789/0 conn(0x7fa8d00b1e40 legacy=0x7fa8d01563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:05.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.689+0000 7fa8d6070700 1 -- 172.21.15.157:0/1598963063 >> v1:172.21.15.157:6789/0 conn(0x7fa8d00b04d0 legacy=0x7fa8d0156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:05.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.689+0000 7fa8d6070700 1 -- 172.21.15.157:0/1598963063 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d01571d0 con 0x7fa8d006c2d0 2023-10-06T04:12:05.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.689+0000 7fa8d6070700 1 --2- 172.21.15.157:0/1598963063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8d006c2d0 0x7fa8d0157750 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8009e60 tx=0x7fa8c8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=532b5219b58a656d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:12:05.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.689+0000 7fa8d486d700 1 -- 172.21.15.157:0/1598963063 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c800e070 con 0x7fa8d006c2d0 2023-10-06T04:12:05.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.689+0000 7fa8d486d700 1 -- 172.21.15.157:0/1598963063 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fa8c800b850 con 0x7fa8d006c2d0 2023-10-06T04:12:05.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.689+0000 7fa8d486d700 1 -- 172.21.15.157:0/1598963063 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0001470 con 0x7fa8d006c2d0 2023-10-06T04:12:05.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.689+0000 7fa8d6871700 1 -- 172.21.15.157:0/1598963063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8d006c2d0 msgr2=0x7fa8d0157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:05.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.689+0000 7fa8d6871700 1 --2- 172.21.15.157:0/1598963063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8d006c2d0 0x7fa8d0157750 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8009e60 tx=0x7fa8c8009780 comp rx=0 tx=0).stop 2023-10-06T04:12:05.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.689+0000 7fa8d6871700 1 -- 172.21.15.157:0/1598963063 shutdown_connections 2023-10-06T04:12:05.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.689+0000 7fa8d6871700 1 --2- 172.21.15.157:0/1598963063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8d006c2d0 0x7fa8d0157750 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:05.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.689+0000 7fa8d6871700 1 -- 172.21.15.157:0/1598963063 >> 172.21.15.157:0/1598963063 conn(0x7fa8d00cc540 msgr2=0x7fa8d00cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:12:05.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.690+0000 7fa8d6871700 1 -- 172.21.15.157:0/1598963063 shutdown_connections 2023-10-06T04:12:05.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.690+0000 7fa8d6871700 1 -- 172.21.15.157:0/1598963063 wait complete. 2023-10-06T04:12:05.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.690+0000 7fa8d6871700 1 Processor -- start 2023-10-06T04:12:05.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.691+0000 7fa8d6871700 1 -- start start 2023-10-06T04:12:05.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.691+0000 7fa8d6871700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8d006c2d0 0x7fa8d00c2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:05.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.691+0000 7fa8d6871700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8d00b04d0 0x7fa8d0167e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:05.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.691+0000 7fa8d6871700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8d00b1e40 0x7fa8d0168350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:05.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.691+0000 7fa8d6871700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d00ab050 con 0x7fa8d006c2d0 2023-10-06T04:12:05.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.691+0000 7fa8d6871700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d0166100 con 0x7fa8d00b1e40 2023-10-06T04:12:05.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.691+0000 7fa8d6871700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d00b2210 con 0x7fa8d00b04d0 2023-10-06T04:12:05.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.691+0000 7fa8d586f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8d006c2d0 0x7fa8d00c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:05.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.691+0000 7fa8d586f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8d006c2d0 0x7fa8d00c2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56140/0 (socket says 172.21.15.157:56140) 2023-10-06T04:12:05.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.691+0000 7fa8d586f700 1 -- 172.21.15.157:0/3951410700 learned_addr learned my addr 172.21.15.157:0/3951410700 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:12:05.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.691+0000 7fa8d506e700 1 --2- 172.21.15.157:0/3951410700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8d00b04d0 0x7fa8d0167e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:05.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.691+0000 7fa8d6070700 1 --2- 172.21.15.157:0/3951410700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8d00b1e40 0x7fa8d0168350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:05.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.692+0000 7fa8d586f700 1 -- 172.21.15.157:0/3951410700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8d00b04d0 msgr2=0x7fa8d0167e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:05.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.692+0000 7fa8d586f700 1 --2- 172.21.15.157:0/3951410700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8d00b04d0 0x7fa8d0167e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:05.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.692+0000 7fa8d586f700 1 -- 172.21.15.157:0/3951410700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8d00b1e40 msgr2=0x7fa8d0168350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:05.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.692+0000 7fa8d586f700 1 --2- 172.21.15.157:0/3951410700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8d00b1e40 0x7fa8d0168350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:05.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.692+0000 7fa8d586f700 1 -- 172.21.15.157:0/3951410700 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c8009a20 con 0x7fa8d006c2d0 2023-10-06T04:12:05.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.692+0000 7fa8d586f700 1 --2- 172.21.15.157:0/3951410700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8d006c2d0 0x7fa8d00c2110 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc00c810 tx=0x7fa8cc007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:12:05.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.692+0000 7fa8c5ffb700 1 -- 172.21.15.157:0/3951410700 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc001230 con 0x7fa8d006c2d0 2023-10-06T04:12:05.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.692+0000 7fa8c5ffb700 1 -- 172.21.15.157:0/3951410700 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fa8cc001e90 con 0x7fa8d006c2d0 2023-10-06T04:12:05.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.692+0000 7fa8d6871700 1 -- 172.21.15.157:0/3951410700 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d0167980 con 0x7fa8d006c2d0 2023-10-06T04:12:05.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.692+0000 7fa8c5ffb700 1 -- 172.21.15.157:0/3951410700 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc011800 con 0x7fa8d006c2d0 2023-10-06T04:12:05.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.692+0000 7fa8d6871700 1 -- 172.21.15.157:0/3951410700 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d00c3280 con 0x7fa8d006c2d0 2023-10-06T04:12:05.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.693+0000 7fa8d6871700 1 -- 172.21.15.157:0/3951410700 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8d0088f90 con 0x7fa8d006c2d0 2023-10-06T04:12:05.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.693+0000 7fa8c5ffb700 1 -- 172.21.15.157:0/3951410700 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa8cc011bf0 con 0x7fa8d006c2d0 2023-10-06T04:12:05.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.694+0000 7fa8c5ffb700 1 --2- 172.21.15.157:0/3951410700 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa8b4066f30 0x7fa8b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:05.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.694+0000 7fa8c5ffb700 1 -- 172.21.15.157:0/3951410700 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7fa8cc083c70 con 0x7fa8d006c2d0 2023-10-06T04:12:05.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.694+0000 7fa8d506e700 1 --2- 172.21.15.157:0/3951410700 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa8b4066f30 0x7fa8b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:05.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.694+0000 7fa8d506e700 1 --2- 172.21.15.157:0/3951410700 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa8b4066f30 0x7fa8b40693e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa8d00c54b0 tx=0x7fa8c0000bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:12:05.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.698+0000 7fa8c5ffb700 1 -- 172.21.15.157:0/3951410700 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa8cc050b70 con 0x7fa8d006c2d0 2023-10-06T04:12:05.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.880+0000 7fa8d6871700 1 -- 172.21.15.157:0/3951410700 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8d006ccc0 con 0x7fa8d006c2d0 2023-10-06T04:12:05.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.881+0000 7fa8c5ffb700 1 -- 172.21.15.157:0/3951410700 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+17955 (secure 0 0 0) 0x7fa8cc017070 con 0x7fa8d006c2d0 2023-10-06T04:12:05.883 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-06T04:12:05.883 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":53,"fsid":"28664a7a-fe30-4efe-be5a-6abf9318b700","created":"2023-10-06T03:56:51.887538+0000","modified":"2023-10-06T04:09:46.762195+0000","last_up_change":"2023-10-06T03:57:02.299440+0000","last_in_change":"2023-10-06T03:56:55.141505+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:57:03.455121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:57:03.905661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:57:08.377186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:57:09.709582+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":505,"snap_epoch":48,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3f677cac-e9f4-4608-acac-718b072ce984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":90640},{"type":"v1","addr":"172.21.15.157:6809","nonce":90640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":90640},{"type":"v1","addr":"172.21.15.157:6811","nonce":90640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":90640},{"type":"v1","addr":"172.21.15.157:6815","nonce":90640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":90640},{"type":"v1","addr":"172.21.15.157:6813","nonce":90640}]},"public_addr":"172.21.15.157:6809/90640","cluster_addr":"172.21.15.157:6811/90640","heartbeat_back_addr":"172.21.15.157:6815/90640","heartbeat_front_addr":"172.21.15.157:6813/90640","state":["exists","up"]},{"osd":1,"uuid":"7d771892-bb91-4e9b-9610-051861c11247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":90654},{"type":"v1","addr":"172.21.15.157:6801","nonce":90654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":90654},{"type":"v1","addr":"172.21.15.157:6803","nonce":90654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":90654},{"type":"v1","addr":"172.21.15.157:6807","nonce":90654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":90654},{"type":"v1","addr":"172.21.15.157:6805","nonce":90654}]},"public_addr":"172.21.15.157:6801/90654","cluster_addr":"172.21.15.157:6803/90654","heartbeat_back_addr":"172.21.15.157:6807/90654","heartbeat_front_addr":"172.21.15.157:6805/90654","state":["exists","up"]},{"osd":2,"uuid":"5c1f7d4e-d185-47cf-ab6c-453b4e3009c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":90657},{"type":"v1","addr":"172.21.15.157:6825","nonce":90657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":90657},{"type":"v1","addr":"172.21.15.157:6827","nonce":90657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":90657},{"type":"v1","addr":"172.21.15.157:6831","nonce":90657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":90657},{"type":"v1","addr":"172.21.15.157:6829","nonce":90657}]},"public_addr":"172.21.15.157:6825/90657","cluster_addr":"172.21.15.157:6827/90657","heartbeat_back_addr":"172.21.15.157:6831/90657","heartbeat_front_addr":"172.21.15.157:6829/90657","state":["exists","up"]},{"osd":3,"uuid":"f6a1e825-43c7-4afd-8daf-e51d45afc75e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":90641},{"type":"v1","addr":"172.21.15.157:6817","nonce":90641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":90641},{"type":"v1","addr":"172.21.15.157:6819","nonce":90641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":90641},{"type":"v1","addr":"172.21.15.157:6823","nonce":90641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":90641},{"type":"v1","addr":"172.21.15.157:6821","nonce":90641}]},"public_addr":"172.21.15.157:6817/90641","cluster_addr":"172.21.15.157:6819/90641","heartbeat_back_addr":"172.21.15.157:6823/90641","heartbeat_front_addr":"172.21.15.157:6821/90641","state":["exists","up"]},{"osd":4,"uuid":"8ad6e3d8-9a31-4f85-9410-417b7ac5da97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":91261},{"type":"v1","addr":"172.21.15.175:6803","nonce":91261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":91261},{"type":"v1","addr":"172.21.15.175:6806","nonce":91261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":91261},{"type":"v1","addr":"172.21.15.175:6814","nonce":91261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":91261},{"type":"v1","addr":"172.21.15.175:6810","nonce":91261}]},"public_addr":"172.21.15.175:6803/91261","cluster_addr":"172.21.15.175:6806/91261","heartbeat_back_addr":"172.21.15.175:6814/91261","heartbeat_front_addr":"172.21.15.175:6810/91261","state":["exists","up"]},{"osd":5,"uuid":"f68aca2a-f182-4c00-8e21-241d09c02ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":91262},{"type":"v1","addr":"172.21.15.175:6817","nonce":91262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":91262},{"type":"v1","addr":"172.21.15.175:6819","nonce":91262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":91262},{"type":"v1","addr":"172.21.15.175:6823","nonce":91262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":91262},{"type":"v1","addr":"172.21.15.175:6821","nonce":91262}]},"public_addr":"172.21.15.175:6817/91262","cluster_addr":"172.21.15.175:6819/91262","heartbeat_back_addr":"172.21.15.175:6823/91262","heartbeat_front_addr":"172.21.15.175:6821/91262","state":["exists","up"]},{"osd":6,"uuid":"d867cd89-3430-40ca-862f-d2bea3de9d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":91263},{"type":"v1","addr":"172.21.15.175:6801","nonce":91263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6805","nonce":91263},{"type":"v1","addr":"172.21.15.175:6807","nonce":91263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6813","nonce":91263},{"type":"v1","addr":"172.21.15.175:6815","nonce":91263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":91263},{"type":"v1","addr":"172.21.15.175:6811","nonce":91263}]},"public_addr":"172.21.15.175:6801/91263","cluster_addr":"172.21.15.175:6807/91263","heartbeat_back_addr":"172.21.15.175:6815/91263","heartbeat_front_addr":"172.21.15.175:6811/91263","state":["exists","up"]},{"osd":7,"uuid":"930cd464-79f1-47cb-b4ff-188ccde74fff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":91306},{"type":"v1","addr":"172.21.15.175:6825","nonce":91306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":91306},{"type":"v1","addr":"172.21.15.175:6827","nonce":91306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":91306},{"type":"v1","addr":"172.21.15.175:6831","nonce":91306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":91306},{"type":"v1","addr":"172.21.15.175:6829","nonce":91306}]},"public_addr":"172.21.15.175:6825/91306","cluster_addr":"172.21.15.175:6827/91306","heartbeat_back_addr":"172.21.15.175:6831/91306","heartbeat_front_addr":"172.21.15.175:6829/91306","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.290673+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.056390+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.292903+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.275861+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.332095+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.327453+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.317585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:57:00.309330+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.24","mappings":[{"from":1,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6834/765157057":"2023-10-07T04:09:46.762128+0000","172.21.15.175:6836/2388164729":"2023-10-07T04:07:31.736793+0000","172.21.15.157:6837/4188652695":"2023-10-07T04:05:26.713520+0000","172.21.15.175:6839/4177493830":"2023-10-07T03:59:06.641459+0000","172.21.15.175:6838/4177493830":"2023-10-07T03:59:06.641459+0000","172.21.15.157:6832/1179753185":"2023-10-07T04:04:26.701047+0000","172.21.15.157:6832/765157057":"2023-10-07T04:09:46.762128+0000","172.21.15.157:6833/3932336008":"2023-10-07T04:01:01.662287+0000","172.21.15.157:6837/3065749287":"2023-10-07T04:08:36.748456+0000","172.21.15.175:6837/2388164729":"2023-10-07T04:07:31.736793+0000","172.21.15.157:6836/4188652695":"2023-10-07T04:05:26.713520+0000","172.21.15.157:6835/3932336008":"2023-10-07T04:01:01.662287+0000","172.21.15.157:6836/3065749287":"2023-10-07T04:08:36.748456+0000","172.21.15.157:6834/1179753185":"2023-10-07T04:04:26.701047+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:12:05.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.883+0000 7fa8d6871700 1 -- 172.21.15.157:0/3951410700 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa8b4066f30 msgr2=0x7fa8b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:05.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.883+0000 7fa8d6871700 1 --2- 172.21.15.157:0/3951410700 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa8b4066f30 0x7fa8b40693e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa8d00c54b0 tx=0x7fa8c0000bc0 comp rx=0 tx=0).stop 2023-10-06T04:12:05.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.883+0000 7fa8d6871700 1 -- 172.21.15.157:0/3951410700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8d006c2d0 msgr2=0x7fa8d00c2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:05.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.883+0000 7fa8d6871700 1 --2- 172.21.15.157:0/3951410700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8d006c2d0 0x7fa8d00c2110 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc00c810 tx=0x7fa8cc007f00 comp rx=0 tx=0).stop 2023-10-06T04:12:05.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.883+0000 7fa8d6871700 1 -- 172.21.15.157:0/3951410700 shutdown_connections 2023-10-06T04:12:05.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.883+0000 7fa8d6871700 1 --2- 172.21.15.157:0/3951410700 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fa8b4066f30 0x7fa8b40693e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:05.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.883+0000 7fa8d6871700 1 --2- 172.21.15.157:0/3951410700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8d00b1e40 0x7fa8d0168350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:05.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.883+0000 7fa8d6871700 1 --2- 172.21.15.157:0/3951410700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8d006c2d0 0x7fa8d00c2110 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:05.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.883+0000 7fa8d6871700 1 --2- 172.21.15.157:0/3951410700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8d00b04d0 0x7fa8d0167e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:05.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.883+0000 7fa8d6871700 1 -- 172.21.15.157:0/3951410700 >> 172.21.15.157:0/3951410700 conn(0x7fa8d00cc540 msgr2=0x7fa8d0156860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:12:05.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.883+0000 7fa8d6871700 1 -- 172.21.15.157:0/3951410700 shutdown_connections 2023-10-06T04:12:05.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:05.883+0000 7fa8d6871700 1 -- 172.21.15.157:0/3951410700 wait complete. 2023-10-06T04:12:05.897 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T04:12:06.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.011+0000 7fad271bf700 1 Processor -- start 2023-10-06T04:12:06.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.011+0000 7fad271bf700 1 -- start start 2023-10-06T04:12:06.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.012+0000 7fad271bf700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fad20150410 0x7fad20155f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:06.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.012+0000 7fad271bf700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad200c9780 con 0x7fad2006dc40 2023-10-06T04:12:06.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.012+0000 7fad271bf700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad2006e0c0 con 0x7fad2006cbb0 2023-10-06T04:12:06.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.012+0000 7fad271bf700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad200c8c40 con 0x7fad20150410 2023-10-06T04:12:06.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.012+0000 7fad1ffff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fad2006dc40 0x7fad2014fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58956/0 (socket says 172.21.15.157:58956) 2023-10-06T04:12:06.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.012+0000 7fad1ffff700 1 -- 172.21.15.157:0/2069113901 learned_addr learned my addr 172.21.15.157:0/2069113901 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:12:06.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.012+0000 7fad2575c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fad20150410 0x7fad20155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:06.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.012+0000 7fad2575c700 1 -- 172.21.15.157:0/2069113901 >> v1:172.21.15.175:6789/0 conn(0x7fad2006cbb0 legacy=0x7fad2006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:06.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.012+0000 7fad2575c700 1 -- 172.21.15.157:0/2069113901 >> v1:172.21.15.157:6789/0 conn(0x7fad2006dc40 legacy=0x7fad2014fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:06.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.013+0000 7fad1f7fe700 1 -- 172.21.15.157:0/2069113901 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2342584684 0 0) 0x7fad200c9780 con 0x7fad2006dc40 2023-10-06T04:12:06.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.013+0000 7fad2575c700 1 -- 172.21.15.157:0/2069113901 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad200c8e50 con 0x7fad20150410 2023-10-06T04:12:06.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.013+0000 7fad2575c700 1 --2- 172.21.15.157:0/2069113901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad20150410 0x7fad20155f70 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fad180098e0 tx=0x7fad180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff79b0731ea78c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:12:06.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.013+0000 7fad1f7fe700 1 -- 172.21.15.157:0/2069113901 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18009e80 con 0x7fad20150410 2023-10-06T04:12:06.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.013+0000 7fad1f7fe700 1 -- 172.21.15.157:0/2069113901 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fad1800b8f0 con 0x7fad20150410 2023-10-06T04:12:06.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.013+0000 7fad1f7fe700 1 -- 172.21.15.157:0/2069113901 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1800bc90 con 0x7fad20150410 2023-10-06T04:12:06.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.013+0000 7fad271bf700 1 -- 172.21.15.157:0/2069113901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad20150410 msgr2=0x7fad20155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:06.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.013+0000 7fad271bf700 1 --2- 172.21.15.157:0/2069113901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad20150410 0x7fad20155f70 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fad180098e0 tx=0x7fad180091a0 comp rx=0 tx=0).stop 2023-10-06T04:12:06.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.013+0000 7fad271bf700 1 -- 172.21.15.157:0/2069113901 shutdown_connections 2023-10-06T04:12:06.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.013+0000 7fad271bf700 1 --2- 172.21.15.157:0/2069113901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad20150410 0x7fad20155f70 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:06.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.013+0000 7fad271bf700 1 -- 172.21.15.157:0/2069113901 >> 172.21.15.157:0/2069113901 conn(0x7fad2006f230 msgr2=0x7fad2006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:12:06.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.013+0000 7fad271bf700 1 -- 172.21.15.157:0/2069113901 shutdown_connections 2023-10-06T04:12:06.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.013+0000 7fad271bf700 1 -- 172.21.15.157:0/2069113901 wait complete. 2023-10-06T04:12:06.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.014+0000 7fad271bf700 1 Processor -- start 2023-10-06T04:12:06.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.015+0000 7fad271bf700 1 -- start start 2023-10-06T04:12:06.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.015+0000 7fad271bf700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad2006cbb0 0x7fad200c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:06.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.015+0000 7fad271bf700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad20150410 0x7fad200c1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:06.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.015+0000 7fad271bf700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad200c5fa0 0x7fad2014b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:06.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.015+0000 7fad271bf700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad200c5b10 con 0x7fad20150410 2023-10-06T04:12:06.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.015+0000 7fad271bf700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad200c16b0 con 0x7fad2006cbb0 2023-10-06T04:12:06.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.015+0000 7fad271bf700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad200c4960 con 0x7fad200c5fa0 2023-10-06T04:12:06.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.015+0000 7fad1ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad20150410 0x7fad200c1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:06.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.015+0000 7fad1ffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad20150410 0x7fad200c1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56166/0 (socket says 172.21.15.157:56166) 2023-10-06T04:12:06.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.015+0000 7fad1ffff700 1 -- 172.21.15.157:0/1146949183 learned_addr learned my addr 172.21.15.157:0/1146949183 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:12:06.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.015+0000 7fad24f5b700 1 --2- 172.21.15.157:0/1146949183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad2006cbb0 0x7fad200c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:06.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.016+0000 7fad2575c700 1 --2- 172.21.15.157:0/1146949183 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad200c5fa0 0x7fad2014b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:06.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.016+0000 7fad1ffff700 1 -- 172.21.15.157:0/1146949183 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad200c5fa0 msgr2=0x7fad2014b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:06.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.016+0000 7fad1ffff700 1 --2- 172.21.15.157:0/1146949183 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad200c5fa0 0x7fad2014b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:06.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.016+0000 7fad1ffff700 1 -- 172.21.15.157:0/1146949183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad2006cbb0 msgr2=0x7fad200c7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:06.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.016+0000 7fad1ffff700 1 --2- 172.21.15.157:0/1146949183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad2006cbb0 0x7fad200c7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:06.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.016+0000 7fad1ffff700 1 -- 172.21.15.157:0/1146949183 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad18009420 con 0x7fad20150410 2023-10-06T04:12:06.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.016+0000 7fad24f5b700 1 --2- 172.21.15.157:0/1146949183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad2006cbb0 0x7fad200c7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:12:06.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.016+0000 7fad1ffff700 1 --2- 172.21.15.157:0/1146949183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad20150410 0x7fad200c1170 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fad0800c810 tx=0x7fad08007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:12:06.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.016+0000 7fad1cff9700 1 -- 172.21.15.157:0/1146949183 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad08005c00 con 0x7fad20150410 2023-10-06T04:12:06.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.016+0000 7fad1cff9700 1 -- 172.21.15.157:0/1146949183 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fad08005d60 con 0x7fad20150410 2023-10-06T04:12:06.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.016+0000 7fad1cff9700 1 -- 172.21.15.157:0/1146949183 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad08007360 con 0x7fad20150410 2023-10-06T04:12:06.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.016+0000 7fad271bf700 1 -- 172.21.15.157:0/1146949183 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad200c4c20 con 0x7fad20150410 2023-10-06T04:12:06.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.016+0000 7fad271bf700 1 -- 172.21.15.157:0/1146949183 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad200c51e0 con 0x7fad20150410 2023-10-06T04:12:06.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.017+0000 7fad271bf700 1 -- 172.21.15.157:0/1146949183 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad200c5c80 con 0x7fad20150410 2023-10-06T04:12:06.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.017+0000 7fad1cff9700 1 -- 172.21.15.157:0/1146949183 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fad0801b400 con 0x7fad20150410 2023-10-06T04:12:06.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.018+0000 7fad1cff9700 1 --2- 172.21.15.157:0/1146949183 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fad00066f60 0x7fad00069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:06.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.018+0000 7fad1cff9700 1 -- 172.21.15.157:0/1146949183 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6718+0+0 (secure 0 0 0) 0x7fad08084350 con 0x7fad20150410 2023-10-06T04:12:06.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.018+0000 7fad24f5b700 1 --2- 172.21.15.157:0/1146949183 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fad00066f60 0x7fad00069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:06.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.018+0000 7fad24f5b700 1 --2- 172.21.15.157:0/1146949183 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fad00066f60 0x7fad00069410 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fad10007fd0 tx=0x7fad10009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:12:06.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.022+0000 7fad1cff9700 1 -- 172.21.15.157:0/1146949183 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad08051330 con 0x7fad20150410 2023-10-06T04:12:06.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.249+0000 7fad271bf700 1 -- 172.21.15.157:0/1146949183 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fad200c9130 con 0x7fad20150410 2023-10-06T04:12:06.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.254+0000 7fad1cff9700 1 -- 172.21.15.157:0/1146949183 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v93) v1 ==== 161+0+0 (secure 0 0 0) 0x7fad08054950 con 0x7fad20150410 2023-10-06T04:12:06.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.257+0000 7fad271bf700 1 -- 172.21.15.157:0/1146949183 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fad00066f60 msgr2=0x7fad00069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:06.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.257+0000 7fad271bf700 1 --2- 172.21.15.157:0/1146949183 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fad00066f60 0x7fad00069410 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fad10007fd0 tx=0x7fad10009040 comp rx=0 tx=0).stop 2023-10-06T04:12:06.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.257+0000 7fad271bf700 1 -- 172.21.15.157:0/1146949183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad20150410 msgr2=0x7fad200c1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:06.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.257+0000 7fad271bf700 1 --2- 172.21.15.157:0/1146949183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad20150410 0x7fad200c1170 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fad0800c810 tx=0x7fad08007d90 comp rx=0 tx=0).stop 2023-10-06T04:12:06.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.257+0000 7fad271bf700 1 -- 172.21.15.157:0/1146949183 shutdown_connections 2023-10-06T04:12:06.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.257+0000 7fad271bf700 1 --2- 172.21.15.157:0/1146949183 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7fad00066f60 0x7fad00069410 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:06.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.257+0000 7fad271bf700 1 --2- 172.21.15.157:0/1146949183 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad2006cbb0 0x7fad200c7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:06.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.257+0000 7fad271bf700 1 --2- 172.21.15.157:0/1146949183 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad200c5fa0 0x7fad2014b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:06.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.257+0000 7fad271bf700 1 --2- 172.21.15.157:0/1146949183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad20150410 0x7fad200c1170 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:06.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.257+0000 7fad271bf700 1 -- 172.21.15.157:0/1146949183 >> 172.21.15.157:0/1146949183 conn(0x7fad2006f230 msgr2=0x7fad2014e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:12:06.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.257+0000 7fad271bf700 1 -- 172.21.15.157:0/1146949183 shutdown_connections 2023-10-06T04:12:06.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.257+0000 7fad271bf700 1 -- 172.21.15.157:0/1146949183 wait complete. 2023-10-06T04:12:06.260 INFO:teuthology.orchestra.run.smithi157.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T04:12:06.271 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T04:12:06.293 INFO:tasks.ceph.mds.e.smithi157.stdout:starting mds.e at 2023-10-06T04:12:06.297 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2023-10-06T04:12:06.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.390+0000 7ff5b2762700 1 Processor -- start 2023-10-06T04:12:06.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.390+0000 7ff5b2762700 1 -- start start 2023-10-06T04:12:06.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.390+0000 7ff5b2762700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff5ac06cbb0 0x7ff5ac06d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:06.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.390+0000 7ff5b2762700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5ac06c1f0 0x7ff5ac06da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:06.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.390+0000 7ff5b2762700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5ac0c9780 con 0x7ff5ac06cf80 2023-10-06T04:12:06.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.390+0000 7ff5b2762700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5ac0c8c40 con 0x7ff5ac06cbb0 2023-10-06T04:12:06.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.390+0000 7ff5b2762700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac06e0c0 con 0x7ff5ac06c1f0 2023-10-06T04:12:06.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.391+0000 7ff5b0cff700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff5ac06cf80 0x7ff5ac14fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58980/0 (socket says 172.21.15.157:58980) 2023-10-06T04:12:06.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.391+0000 7ff5b0cff700 1 -- 172.21.15.157:0/636025936 learned_addr learned my addr 172.21.15.157:0/636025936 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:12:06.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.391+0000 7ff5abfff700 1 --2- 172.21.15.157:0/636025936 >> v2:172.21.15.175:3300/0 conn(0x7ff5ac06cbb0 0x7ff5ac06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:06.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.391+0000 7ff5ab7fe700 1 --2- 172.21.15.157:0/636025936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5ac06c1f0 0x7ff5ac06da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:06.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.391+0000 7ff5aaffd700 1 -- 172.21.15.157:0/636025936 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 249871345 0 0) 0x7ff5ac0c9780 con 0x7ff5ac06cf80 2023-10-06T04:12:06.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.391+0000 7ff5aaffd700 1 -- 172.21.15.157:0/636025936 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff594003570 con 0x7ff5ac06cf80 2023-10-06T04:12:06.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.391+0000 7ff5abfff700 1 -- 172.21.15.157:0/636025936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5ac06c1f0 msgr2=0x7ff5ac06da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:12:06.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.391+0000 7ff5abfff700 1 --2- 172.21.15.157:0/636025936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5ac06c1f0 0x7ff5ac06da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:06.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.391+0000 7ff5abfff700 1 -- 172.21.15.157:0/636025936 >> v1:172.21.15.157:6789/0 conn(0x7ff5ac06cf80 legacy=0x7ff5ac14fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:06.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.391+0000 7ff5aaffd700 1 -- 172.21.15.157:0/636025936 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2943470999 0 0) 0x7ff594003570 con 0x7ff5ac06cf80 2023-10-06T04:12:06.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.391+0000 7ff5abfff700 1 -- 172.21.15.157:0/636025936 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ac0c8e50 con 0x7ff5ac06cbb0 2023-10-06T04:12:06.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.391+0000 7ff5ab7fe700 1 --2- 172.21.15.157:0/636025936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5ac06c1f0 0x7ff5ac06da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:12:06.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.391+0000 7ff5abfff700 1 --2- 172.21.15.157:0/636025936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5ac06cbb0 0x7ff5ac06d530 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff59c009910 tx=0x7ff59c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b8751ad73eb2b9b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:12:06.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.391+0000 7ff5aaffd700 1 -- 172.21.15.157:0/636025936 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a40017f0 con 0x7ff5ac06cbb0 2023-10-06T04:12:06.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.391+0000 7ff5aaffd700 1 -- 172.21.15.157:0/636025936 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff59c011420 con 0x7ff5ac06cbb0 2023-10-06T04:12:06.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.391+0000 7ff5aaffd700 1 -- 172.21.15.157:0/636025936 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0117a0 con 0x7ff5ac06cbb0 2023-10-06T04:12:06.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.392+0000 7ff5b2762700 1 -- 172.21.15.157:0/636025936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5ac06cbb0 msgr2=0x7ff5ac06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:06.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.392+0000 7ff5b2762700 1 --2- 172.21.15.157:0/636025936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5ac06cbb0 0x7ff5ac06d530 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff59c009910 tx=0x7ff59c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:12:06.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.392+0000 7ff5b2762700 1 -- 172.21.15.157:0/636025936 shutdown_connections 2023-10-06T04:12:06.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.392+0000 7ff5b2762700 1 --2- 172.21.15.157:0/636025936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5ac06cbb0 0x7ff5ac06d530 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:06.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.392+0000 7ff5b2762700 1 --2- 172.21.15.157:0/636025936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5ac06c1f0 0x7ff5ac06da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:06.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.392+0000 7ff5b2762700 1 -- 172.21.15.157:0/636025936 >> 172.21.15.157:0/636025936 conn(0x7ff5ac06f230 msgr2=0x7ff5ac06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:12:06.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.392+0000 7ff5b2762700 1 -- 172.21.15.157:0/636025936 shutdown_connections 2023-10-06T04:12:06.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.392+0000 7ff5b2762700 1 -- 172.21.15.157:0/636025936 wait complete. 2023-10-06T04:12:06.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.393+0000 7ff5b2762700 1 Processor -- start 2023-10-06T04:12:06.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.393+0000 7ff5b2762700 1 -- start start 2023-10-06T04:12:06.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.394+0000 7ff5b2762700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5ac06c1f0 0x7ff5ac1671d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:06.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.394+0000 7ff5b2762700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5ac06cbb0 0x7ff5ac1567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:06.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.394+0000 7ff5b2762700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5ac06cf80 0x7ff5ac154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:06.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.394+0000 7ff5b2762700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac154d00 con 0x7ff5ac06cf80 2023-10-06T04:12:06.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.394+0000 7ff5b2762700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac0c5050 con 0x7ff5ac06c1f0 2023-10-06T04:12:06.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.394+0000 7ff5b2762700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac14ba20 con 0x7ff5ac06cbb0 2023-10-06T04:12:06.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.394+0000 7ff5b0cff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5ac06cf80 0x7ff5ac154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:06.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.394+0000 7ff5b0cff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5ac06cf80 0x7ff5ac154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56190/0 (socket says 172.21.15.157:56190) 2023-10-06T04:12:06.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.394+0000 7ff5b0cff700 1 -- 172.21.15.157:0/3113849808 learned_addr learned my addr 172.21.15.157:0/3113849808 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:12:06.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.394+0000 7ff5abfff700 1 --2- 172.21.15.157:0/3113849808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5ac06c1f0 0x7ff5ac1671d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:06.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.394+0000 7ff5ab7fe700 1 --2- 172.21.15.157:0/3113849808 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5ac06cbb0 0x7ff5ac1567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:06.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.394+0000 7ff5b0cff700 1 -- 172.21.15.157:0/3113849808 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5ac06cbb0 msgr2=0x7ff5ac1567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:06.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.394+0000 7ff5b0cff700 1 --2- 172.21.15.157:0/3113849808 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5ac06cbb0 0x7ff5ac1567f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:06.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.394+0000 7ff5b0cff700 1 -- 172.21.15.157:0/3113849808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5ac06c1f0 msgr2=0x7ff5ac1671d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:06.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.394+0000 7ff5b0cff700 1 --2- 172.21.15.157:0/3113849808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5ac06c1f0 0x7ff5ac1671d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:06.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.394+0000 7ff5b0cff700 1 -- 172.21.15.157:0/3113849808 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a0009420 con 0x7ff5ac06cf80 2023-10-06T04:12:06.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.395+0000 7ff5b0cff700 1 --2- 172.21.15.157:0/3113849808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5ac06cf80 0x7ff5ac154180 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7ff5a400cbf0 tx=0x7ff5a4007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:12:06.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.395+0000 7ff593fff700 1 -- 172.21.15.157:0/3113849808 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4004dc0 con 0x7ff5ac06cf80 2023-10-06T04:12:06.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.395+0000 7ff593fff700 1 -- 172.21.15.157:0/3113849808 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff5a400a660 con 0x7ff5ac06cf80 2023-10-06T04:12:06.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.395+0000 7ff5b2762700 1 -- 172.21.15.157:0/3113849808 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff59c009420 con 0x7ff5ac06cf80 2023-10-06T04:12:06.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.395+0000 7ff593fff700 1 -- 172.21.15.157:0/3113849808 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a400aa40 con 0x7ff5ac06cf80 2023-10-06T04:12:06.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.395+0000 7ff5b2762700 1 -- 172.21.15.157:0/3113849808 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5ac150150 con 0x7ff5ac06cf80 2023-10-06T04:12:06.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.396+0000 7ff5b2762700 1 -- 172.21.15.157:0/3113849808 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5ac089ca0 con 0x7ff5ac06cf80 2023-10-06T04:12:06.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.396+0000 7ff593fff700 1 -- 172.21.15.157:0/3113849808 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff5a4007150 con 0x7ff5ac06cf80 2023-10-06T04:12:06.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.396+0000 7ff593fff700 1 --2- 172.21.15.157:0/3113849808 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff598066f30 0x7ff5980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:06.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.397+0000 7ff593fff700 1 -- 172.21.15.157:0/3113849808 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6890+0+0 (secure 0 0 0) 0x7ff5a4084290 con 0x7ff5ac06cf80 2023-10-06T04:12:06.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.397+0000 7ff5abfff700 1 --2- 172.21.15.157:0/3113849808 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff598066f30 0x7ff5980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:06.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.397+0000 7ff5abfff700 1 --2- 172.21.15.157:0/3113849808 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff598066f30 0x7ff5980693e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff59c003080 tx=0x7ff59c0091a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:12:06.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.401+0000 7ff593fff700 1 -- 172.21.15.157:0/3113849808 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff5a40511c0 con 0x7ff5ac06cf80 2023-10-06T04:12:06.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:06.628+0000 7ff5b2762700 1 -- 172.21.15.157:0/3113849808 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff5ac0c9130 con 0x7ff5ac06cf80 2023-10-06T04:12:07.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.255+0000 7ff593fff700 1 -- 172.21.15.157:0/3113849808 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v94) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff5a40547e0 con 0x7ff5ac06cf80 2023-10-06T04:12:07.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.258+0000 7ff5b2762700 1 -- 172.21.15.157:0/3113849808 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff598066f30 msgr2=0x7ff5980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:07.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.258+0000 7ff5b2762700 1 --2- 172.21.15.157:0/3113849808 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff598066f30 0x7ff5980693e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff59c003080 tx=0x7ff59c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:12:07.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.258+0000 7ff5b2762700 1 -- 172.21.15.157:0/3113849808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5ac06cf80 msgr2=0x7ff5ac154180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:07.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.258+0000 7ff5b2762700 1 --2- 172.21.15.157:0/3113849808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5ac06cf80 0x7ff5ac154180 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7ff5a400cbf0 tx=0x7ff5a4007c70 comp rx=0 tx=0).stop 2023-10-06T04:12:07.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.258+0000 7ff5b2762700 1 -- 172.21.15.157:0/3113849808 shutdown_connections 2023-10-06T04:12:07.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.258+0000 7ff5b2762700 1 --2- 172.21.15.157:0/3113849808 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7ff598066f30 0x7ff5980693e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:07.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.258+0000 7ff5b2762700 1 --2- 172.21.15.157:0/3113849808 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5ac06c1f0 0x7ff5ac1671d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:07.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.258+0000 7ff5b2762700 1 --2- 172.21.15.157:0/3113849808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5ac06cf80 0x7ff5ac154180 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:07.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.258+0000 7ff5b2762700 1 --2- 172.21.15.157:0/3113849808 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff5ac06cbb0 0x7ff5ac1567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:07.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.258+0000 7ff5b2762700 1 -- 172.21.15.157:0/3113849808 >> 172.21.15.157:0/3113849808 conn(0x7ff5ac06f230 msgr2=0x7ff5ac0c7620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:12:07.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.258+0000 7ff5b2762700 1 -- 172.21.15.157:0/3113849808 shutdown_connections 2023-10-06T04:12:07.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.258+0000 7ff5b2762700 1 -- 172.21.15.157:0/3113849808 wait complete. 2023-10-06T04:12:07.274 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T04:12:07.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.440+0000 7f29f6869700 1 Processor -- start 2023-10-06T04:12:07.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.440+0000 7f29f6869700 1 -- start start 2023-10-06T04:12:07.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.441+0000 7f29f6869700 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f29f00c6290 0x7f29f00c0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:07.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.441+0000 7f29f6869700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29f0157c80 0x7f29f0067530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:07.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.441+0000 7f29f6869700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f29f0153b80 0x7f29f00631f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:07.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.441+0000 7f29f6869700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29f00bf610 con 0x7f29f0153b80 2023-10-06T04:12:07.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.441+0000 7f29f6869700 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29f0067a70 con 0x7f29f00c6290 2023-10-06T04:12:07.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.441+0000 7f29f6869700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29f00cb1c0 con 0x7f29f0157c80 2023-10-06T04:12:07.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.441+0000 7f29f6068700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f29f0153b80 0x7f29f00631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:07.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.441+0000 7f29f6068700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f29f0153b80 0x7f29f00631f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56192/0 (socket says 172.21.15.157:56192) 2023-10-06T04:12:07.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.441+0000 7f29f6068700 1 -- 172.21.15.157:0/2586026969 learned_addr learned my addr 172.21.15.157:0/2586026969 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:12:07.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.441+0000 7f29f5867700 1 --2- 172.21.15.157:0/2586026969 >> v2:172.21.15.175:3300/0 conn(0x7f29f00c6290 0x7f29f00c0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:07.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.441+0000 7f29f5066700 1 --2- 172.21.15.157:0/2586026969 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29f0157c80 0x7f29f0067530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:07.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.441+0000 7f29f6068700 1 -- 172.21.15.157:0/2586026969 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29f0157c80 msgr2=0x7f29f0067530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:12:07.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.442+0000 7f29f6068700 1 --2- 172.21.15.157:0/2586026969 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29f0157c80 0x7f29f0067530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:07.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.442+0000 7f29f6068700 1 -- 172.21.15.157:0/2586026969 >> v2:172.21.15.175:3300/0 conn(0x7f29f00c6290 msgr2=0x7f29f00c0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:12:07.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.442+0000 7f29f6068700 1 --2- 172.21.15.157:0/2586026969 >> v2:172.21.15.175:3300/0 conn(0x7f29f00c6290 0x7f29f00c0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:07.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.442+0000 7f29f6068700 1 -- 172.21.15.157:0/2586026969 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f01577f0 con 0x7f29f0153b80 2023-10-06T04:12:07.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.442+0000 7f29f5867700 1 --2- 172.21.15.157:0/2586026969 >> v2:172.21.15.175:3300/0 conn(0x7f29f00c6290 0x7f29f00c0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:12:07.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.442+0000 7f29f6068700 1 --2- 172.21.15.157:0/2586026969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29f0153b80 0x7f29f00631f0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f29f00cb0e0 tx=0x7f29e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd997ff0fadbb8b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:12:07.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.442+0000 7f29f4865700 1 -- 172.21.15.157:0/2586026969 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e800b870 con 0x7f29f0153b80 2023-10-06T04:12:07.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.442+0000 7f29f4865700 1 -- 172.21.15.157:0/2586026969 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f29e800b9d0 con 0x7f29f0153b80 2023-10-06T04:12:07.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.442+0000 7f29f4865700 1 -- 172.21.15.157:0/2586026969 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e800bd90 con 0x7f29f0153b80 2023-10-06T04:12:07.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.442+0000 7f29f6869700 1 -- 172.21.15.157:0/2586026969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29f0153b80 msgr2=0x7f29f00631f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:07.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.442+0000 7f29f6869700 1 --2- 172.21.15.157:0/2586026969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29f0153b80 0x7f29f00631f0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f29f00cb0e0 tx=0x7f29e8009250 comp rx=0 tx=0).stop 2023-10-06T04:12:07.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.442+0000 7f29f6869700 1 -- 172.21.15.157:0/2586026969 shutdown_connections 2023-10-06T04:12:07.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.442+0000 7f29f6869700 1 --2- 172.21.15.157:0/2586026969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29f0153b80 0x7f29f00631f0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:07.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.442+0000 7f29f6869700 1 --2- 172.21.15.157:0/2586026969 >> v2:172.21.15.175:3300/0 conn(0x7f29f00c6290 0x7f29f00c0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:07.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.442+0000 7f29f6869700 1 --2- 172.21.15.157:0/2586026969 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29f0157c80 0x7f29f0067530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:07.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.442+0000 7f29f6869700 1 -- 172.21.15.157:0/2586026969 >> 172.21.15.157:0/2586026969 conn(0x7f29f006c1f0 msgr2=0x7f29f006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:12:07.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.443+0000 7f29f6869700 1 -- 172.21.15.157:0/2586026969 shutdown_connections 2023-10-06T04:12:07.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.443+0000 7f29f6869700 1 -- 172.21.15.157:0/2586026969 wait complete. 2023-10-06T04:12:07.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.444+0000 7f29f6869700 1 Processor -- start 2023-10-06T04:12:07.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.444+0000 7f29f6869700 1 -- start start 2023-10-06T04:12:07.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.444+0000 7f29f6869700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29f00c6290 0x7f29f0163c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:07.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.444+0000 7f29f6869700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29f0153b80 0x7f29f00a66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:07.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.444+0000 7f29f6869700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29f0157c80 0x7f29f0167260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:07.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.444+0000 7f29f6869700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f00c2df0 con 0x7f29f0157c80 2023-10-06T04:12:07.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.444+0000 7f29f6869700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f00c9530 con 0x7f29f00c6290 2023-10-06T04:12:07.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.444+0000 7f29f6869700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29f00c9810 con 0x7f29f0153b80 2023-10-06T04:12:07.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.444+0000 7f29f6068700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29f0157c80 0x7f29f0167260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:07.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.444+0000 7f29f6068700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29f0157c80 0x7f29f0167260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56200/0 (socket says 172.21.15.157:56200) 2023-10-06T04:12:07.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.444+0000 7f29f6068700 1 -- 172.21.15.157:0/3971715682 learned_addr learned my addr 172.21.15.157:0/3971715682 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:12:07.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.444+0000 7f29f5066700 1 --2- 172.21.15.157:0/3971715682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29f0153b80 0x7f29f00a66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:07.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.444+0000 7f29f5867700 1 --2- 172.21.15.157:0/3971715682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29f00c6290 0x7f29f0163c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:07.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.445+0000 7f29f6068700 1 -- 172.21.15.157:0/3971715682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29f0153b80 msgr2=0x7f29f00a66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:07.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.445+0000 7f29f6068700 1 --2- 172.21.15.157:0/3971715682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29f0153b80 0x7f29f00a66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:07.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.445+0000 7f29f6068700 1 -- 172.21.15.157:0/3971715682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29f00c6290 msgr2=0x7f29f0163c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:07.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.445+0000 7f29f6068700 1 --2- 172.21.15.157:0/3971715682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29f00c6290 0x7f29f0163c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:07.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.445+0000 7f29f6068700 1 -- 172.21.15.157:0/3971715682 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e80094f0 con 0x7f29f0157c80 2023-10-06T04:12:07.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.445+0000 7f29f5867700 1 --2- 172.21.15.157:0/3971715682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29f00c6290 0x7f29f0163c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:12:07.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.445+0000 7f29f6068700 1 --2- 172.21.15.157:0/3971715682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29f0157c80 0x7f29f0167260 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f29e80090b0 tx=0x7f29e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:12:07.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.445+0000 7f29e5ffb700 1 -- 172.21.15.157:0/3971715682 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e800b870 con 0x7f29f0157c80 2023-10-06T04:12:07.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.445+0000 7f29e5ffb700 1 -- 172.21.15.157:0/3971715682 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f29e801f070 con 0x7f29f0157c80 2023-10-06T04:12:07.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.445+0000 7f29e5ffb700 1 -- 172.21.15.157:0/3971715682 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e801a8b0 con 0x7f29f0157c80 2023-10-06T04:12:07.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.445+0000 7f29f6869700 1 -- 172.21.15.157:0/3971715682 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29f00c9af0 con 0x7f29f0157c80 2023-10-06T04:12:07.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.445+0000 7f29f6869700 1 -- 172.21.15.157:0/3971715682 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29f00c2a80 con 0x7f29f0157c80 2023-10-06T04:12:07.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.446+0000 7f29f6869700 1 -- 172.21.15.157:0/3971715682 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29f00c2fa0 con 0x7f29f0157c80 2023-10-06T04:12:07.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.446+0000 7f29e5ffb700 1 -- 172.21.15.157:0/3971715682 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f29e801ac20 con 0x7f29f0157c80 2023-10-06T04:12:07.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.447+0000 7f29e5ffb700 1 --2- 172.21.15.157:0/3971715682 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f29d4066f60 0x7f29d4069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:07.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.447+0000 7f29e5ffb700 1 -- 172.21.15.157:0/3971715682 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6890+0+0 (secure 0 0 0) 0x7f29e8021070 con 0x7f29f0157c80 2023-10-06T04:12:07.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.447+0000 7f29f5867700 1 --2- 172.21.15.157:0/3971715682 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f29d4066f60 0x7f29d4069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:07.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.447+0000 7f29f5867700 1 --2- 172.21.15.157:0/3971715682 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f29d4066f60 0x7f29d4069410 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f29ec007640 tx=0x7f29ec009850 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:12:07.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.451+0000 7f29e5ffb700 1 -- 172.21.15.157:0/3971715682 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f29f00c2fa0 con 0x7f29f0157c80 2023-10-06T04:12:07.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:07.633+0000 7f29f6869700 1 -- 172.21.15.157:0/3971715682 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f29f00c2fa0 con 0x7f29f0157c80 2023-10-06T04:12:08.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.263+0000 7f29e5ffb700 1 -- 172.21.15.157:0/3971715682 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v55) v1 ==== 181+0+0 (secure 0 0 0) 0x7f29f00c2fa0 con 0x7f29f0157c80 2023-10-06T04:12:08.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.266+0000 7f29f6869700 1 -- 172.21.15.157:0/3971715682 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f29d4066f60 msgr2=0x7f29d4069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:08.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.266+0000 7f29f6869700 1 --2- 172.21.15.157:0/3971715682 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f29d4066f60 0x7f29d4069410 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f29ec007640 tx=0x7f29ec009850 comp rx=0 tx=0).stop 2023-10-06T04:12:08.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.266+0000 7f29f6869700 1 -- 172.21.15.157:0/3971715682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29f0157c80 msgr2=0x7f29f0167260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:08.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.266+0000 7f29f6869700 1 --2- 172.21.15.157:0/3971715682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29f0157c80 0x7f29f0167260 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f29e80090b0 tx=0x7f29e8009250 comp rx=0 tx=0).stop 2023-10-06T04:12:08.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.266+0000 7f29f6869700 1 -- 172.21.15.157:0/3971715682 shutdown_connections 2023-10-06T04:12:08.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.266+0000 7f29f6869700 1 --2- 172.21.15.157:0/3971715682 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f29d4066f60 0x7f29d4069410 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:08.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.266+0000 7f29f6869700 1 --2- 172.21.15.157:0/3971715682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29f00c6290 0x7f29f0163c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:08.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.266+0000 7f29f6869700 1 --2- 172.21.15.157:0/3971715682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29f0157c80 0x7f29f0167260 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:08.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.266+0000 7f29f6869700 1 --2- 172.21.15.157:0/3971715682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29f0153b80 0x7f29f00a66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:08.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.266+0000 7f29f6869700 1 -- 172.21.15.157:0/3971715682 >> 172.21.15.157:0/3971715682 conn(0x7f29f006c1f0 msgr2=0x7f29f00c6b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:12:08.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.266+0000 7f29f6869700 1 -- 172.21.15.157:0/3971715682 shutdown_connections 2023-10-06T04:12:08.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.266+0000 7f29f6869700 1 -- 172.21.15.157:0/3971715682 wait complete. 2023-10-06T04:12:08.270 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' removed 2023-10-06T04:12:08.281 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T04:12:08.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.418+0000 7f1b272bf700 1 Processor -- start 2023-10-06T04:12:08.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.418+0000 7f1b272bf700 1 -- start start 2023-10-06T04:12:08.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.419+0000 7f1b272bf700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1b2006eed0 0x7f1b20156a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:08.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.419+0000 7f1b272bf700 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b200c68f0 con 0x7f1b200c7490 2023-10-06T04:12:08.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.419+0000 7f1b272bf700 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b2006d9d0 con 0x7f1b200ca630 2023-10-06T04:12:08.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.419+0000 7f1b272bf700 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b200c59e0 con 0x7f1b2006eed0 2023-10-06T04:12:08.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.419+0000 7f1b2585c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1b2006eed0 0x7f1b20156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:08.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.419+0000 7f1b2485a700 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f1b200c7490 0x7f1b20156370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58984/0 (socket says 172.21.15.157:58984) 2023-10-06T04:12:08.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.419+0000 7f1b2585c700 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1b2006eed0 0x7f1b20156a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56206/0 (socket says 172.21.15.157:56206) 2023-10-06T04:12:08.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.419+0000 7f1b2485a700 1 -- 172.21.15.157:0/1879639411 learned_addr learned my addr 172.21.15.157:0/1879639411 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-10-06T04:12:08.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.420+0000 7f1b2585c700 1 -- 172.21.15.157:0/1879639411 >> v1:172.21.15.175:6789/0 conn(0x7f1b200ca630 legacy=0x7f1b200c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:12:08.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.420+0000 7f1b2585c700 1 -- 172.21.15.157:0/1879639411 >> v1:172.21.15.157:6789/0 conn(0x7f1b200c7490 legacy=0x7f1b20156370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:08.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.420+0000 7f1b2585c700 1 -- 172.21.15.157:0/1879639411 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b2006e7d0 con 0x7f1b2006eed0 2023-10-06T04:12:08.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.420+0000 7f1b17fff700 1 -- 172.21.15.157:0/1879639411 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 543886690 0 0) 0x7f1b200c68f0 con 0x7f1b200c7490 2023-10-06T04:12:08.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.420+0000 7f1b2585c700 1 --2- 172.21.15.157:0/1879639411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b2006eed0 0x7f1b20156a80 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f1b2006d340 tx=0x7f1b180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b102615963f32e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:12:08.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.420+0000 7f1b17fff700 1 -- 172.21.15.157:0/1879639411 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1800b670 con 0x7f1b2006eed0 2023-10-06T04:12:08.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.420+0000 7f1b17fff700 1 -- 172.21.15.157:0/1879639411 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f1b1800b7d0 con 0x7f1b2006eed0 2023-10-06T04:12:08.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.420+0000 7f1b17fff700 1 -- 172.21.15.157:0/1879639411 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1800bb90 con 0x7f1b2006eed0 2023-10-06T04:12:08.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.420+0000 7f1b272bf700 1 -- 172.21.15.157:0/1879639411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b2006eed0 msgr2=0x7f1b20156a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:08.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.420+0000 7f1b272bf700 1 --2- 172.21.15.157:0/1879639411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b2006eed0 0x7f1b20156a80 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f1b2006d340 tx=0x7f1b180091a0 comp rx=0 tx=0).stop 2023-10-06T04:12:08.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.420+0000 7f1b272bf700 1 -- 172.21.15.157:0/1879639411 shutdown_connections 2023-10-06T04:12:08.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.420+0000 7f1b272bf700 1 --2- 172.21.15.157:0/1879639411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b2006eed0 0x7f1b20156a80 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:08.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.420+0000 7f1b272bf700 1 -- 172.21.15.157:0/1879639411 >> 172.21.15.157:0/1879639411 conn(0x7f1b200692b0 msgr2=0x7f1b200696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:12:08.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.421+0000 7f1b272bf700 1 -- 172.21.15.157:0/1879639411 shutdown_connections 2023-10-06T04:12:08.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.421+0000 7f1b272bf700 1 -- 172.21.15.157:0/1879639411 wait complete. 2023-10-06T04:12:08.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.422+0000 7f1b272bf700 1 Processor -- start 2023-10-06T04:12:08.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.422+0000 7f1b272bf700 1 -- start start 2023-10-06T04:12:08.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.422+0000 7f1b272bf700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b2006eed0 0x7f1b2014f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:08.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.422+0000 7f1b272bf700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b200c7490 0x7f1b2014fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:08.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.422+0000 7f1b272bf700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b200ca630 0x7f1b2015bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:08.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.422+0000 7f1b272bf700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b200c4200 con 0x7f1b200c7490 2023-10-06T04:12:08.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.422+0000 7f1b272bf700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2015c600 con 0x7f1b2006eed0 2023-10-06T04:12:08.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.422+0000 7f1b272bf700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2014f750 con 0x7f1b200ca630 2023-10-06T04:12:08.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.423+0000 7f1b2485a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b200c7490 0x7f1b2014fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:08.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.423+0000 7f1b2485a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b200c7490 0x7f1b2014fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56214/0 (socket says 172.21.15.157:56214) 2023-10-06T04:12:08.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.423+0000 7f1b2485a700 1 -- 172.21.15.157:0/2010620860 learned_addr learned my addr 172.21.15.157:0/2010620860 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-10-06T04:12:08.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.423+0000 7f1b2505b700 1 --2- 172.21.15.157:0/2010620860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b2006eed0 0x7f1b2014f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:08.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.423+0000 7f1b2485a700 1 -- 172.21.15.157:0/2010620860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b200ca630 msgr2=0x7f1b2015bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:08.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.423+0000 7f1b2485a700 1 --2- 172.21.15.157:0/2010620860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b200ca630 0x7f1b2015bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:08.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.423+0000 7f1b2485a700 1 -- 172.21.15.157:0/2010620860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b2006eed0 msgr2=0x7f1b2014f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:08.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.423+0000 7f1b2485a700 1 --2- 172.21.15.157:0/2010620860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b2006eed0 0x7f1b2014f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:08.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.423+0000 7f1b2485a700 1 -- 172.21.15.157:0/2010620860 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b18009420 con 0x7f1b200c7490 2023-10-06T04:12:08.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.423+0000 7f1b2505b700 1 --2- 172.21.15.157:0/2010620860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b2006eed0 0x7f1b2014f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:12:08.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.423+0000 7f1b2485a700 1 --2- 172.21.15.157:0/2010620860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b200c7490 0x7f1b2014fb50 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000c840 tx=0x7f1b10007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:12:08.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.423+0000 7f1b157fa700 1 -- 172.21.15.157:0/2010620860 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b10005c00 con 0x7f1b200c7490 2023-10-06T04:12:08.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.423+0000 7f1b157fa700 1 -- 172.21.15.157:0/2010620860 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f1b10005d60 con 0x7f1b200c7490 2023-10-06T04:12:08.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.423+0000 7f1b272bf700 1 -- 172.21.15.157:0/2010620860 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b200c34a0 con 0x7f1b200c7490 2023-10-06T04:12:08.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.423+0000 7f1b157fa700 1 -- 172.21.15.157:0/2010620860 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b10007300 con 0x7f1b200c7490 2023-10-06T04:12:08.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.423+0000 7f1b272bf700 1 -- 172.21.15.157:0/2010620860 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b200c3d40 con 0x7f1b200c7490 2023-10-06T04:12:08.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.424+0000 7f1b272bf700 1 -- 172.21.15.157:0/2010620860 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b2006a8f0 con 0x7f1b200c7490 2023-10-06T04:12:08.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.424+0000 7f1b157fa700 1 -- 172.21.15.157:0/2010620860 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1b1001b3b0 con 0x7f1b200c7490 2023-10-06T04:12:08.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.425+0000 7f1b157fa700 1 --2- 172.21.15.157:0/2010620860 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1b04066f30 0x7f1b040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:12:08.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.425+0000 7f1b157fa700 1 -- 172.21.15.157:0/2010620860 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6452+0+0 (secure 0 0 0) 0x7f1b10083ed0 con 0x7f1b200c7490 2023-10-06T04:12:08.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.425+0000 7f1b2505b700 1 --2- 172.21.15.157:0/2010620860 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1b04066f30 0x7f1b040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:12:08.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.425+0000 7f1b2505b700 1 --2- 172.21.15.157:0/2010620860 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1b04066f30 0x7f1b040693e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c007fd0 tx=0x7f1b1c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:12:08.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.429+0000 7f1b157fa700 1 -- 172.21.15.157:0/2010620860 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b10050e30 con 0x7f1b200c7490 2023-10-06T04:12:08.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:08.612+0000 7f1b272bf700 1 -- 172.21.15.157:0/2010620860 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1b2015c740 con 0x7f1b200c7490 2023-10-06T04:12:09.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:09.267+0000 7f1b157fa700 1 -- 172.21.15.157:0/2010620860 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v56) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1b10017020 con 0x7f1b200c7490 2023-10-06T04:12:09.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:09.274+0000 7f1b272bf700 1 -- 172.21.15.157:0/2010620860 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1b04066f30 msgr2=0x7f1b040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:09.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:09.274+0000 7f1b272bf700 1 --2- 172.21.15.157:0/2010620860 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1b04066f30 0x7f1b040693e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c007fd0 tx=0x7f1b1c009040 comp rx=0 tx=0).stop 2023-10-06T04:12:09.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:09.274+0000 7f1b272bf700 1 -- 172.21.15.157:0/2010620860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b200c7490 msgr2=0x7f1b2014fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:12:09.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:09.274+0000 7f1b272bf700 1 --2- 172.21.15.157:0/2010620860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b200c7490 0x7f1b2014fb50 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000c840 tx=0x7f1b10007db0 comp rx=0 tx=0).stop 2023-10-06T04:12:09.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:09.275+0000 7f1b272bf700 1 -- 172.21.15.157:0/2010620860 shutdown_connections 2023-10-06T04:12:09.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:09.275+0000 7f1b272bf700 1 --2- 172.21.15.157:0/2010620860 >> [v2:172.21.15.175:6832/90661,v1:172.21.15.175:6833/90661] conn(0x7f1b04066f30 0x7f1b040693e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:09.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:09.275+0000 7f1b272bf700 1 --2- 172.21.15.157:0/2010620860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b2006eed0 0x7f1b2014f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:09.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:09.275+0000 7f1b272bf700 1 --2- 172.21.15.157:0/2010620860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b200ca630 0x7f1b2015bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:12:09.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:09.275+0000 7f1b272bf700 1 --2- 172.21.15.157:0/2010620860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b200c7490 0x7f1b2014fb50 secure :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000c840 tx=0x7f1b10007db0 comp rx=0 tx=0).stop 2023-10-06T04:12:09.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:09.275+0000 7f1b272bf700 1 -- 172.21.15.157:0/2010620860 >> 172.21.15.157:0/2010620860 conn(0x7f1b200692b0 msgr2=0x7f1b200caa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:12:09.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:09.276+0000 7f1b272bf700 1 -- 172.21.15.157:0/2010620860 shutdown_connections 2023-10-06T04:12:09.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-10-06T04:12:09.277+0000 7f1b272bf700 1 -- 172.21.15.157:0/2010620860 wait complete. 2023-10-06T04:12:09.281 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' removed 2023-10-06T04:12:09.299 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-06T04:12:09.299 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T04:12:09.299 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:12:09.311 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T04:12:09.311 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T04:12:09.311 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:12:09.363 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T04:12:09.363 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2023-10-06T04:12:09.363 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:12:09.415 INFO:tasks.ceph.mds.e:Stopped 2023-10-06T04:12:09.415 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T04:12:09.415 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:12:09.424 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T04:12:09.424 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T04:12:09.424 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:12:09.473 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T04:12:09.473 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2023-10-06T04:12:09.473 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:12:09.521 INFO:tasks.ceph.mds.f:Stopped 2023-10-06T04:12:09.522 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-06T04:12:09.522 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-06T04:12:09.522 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:12:09.564 INFO:tasks.ceph.osd.0:Stopped 2023-10-06T04:12:09.564 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-06T04:12:09.564 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:12:09.614 INFO:tasks.ceph.osd.1:Stopped 2023-10-06T04:12:09.615 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-06T04:12:09.615 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:12:09.653 INFO:tasks.ceph.osd.2:Stopped 2023-10-06T04:12:09.654 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-06T04:12:09.654 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:12:09.730 INFO:tasks.ceph.osd.3:Stopped 2023-10-06T04:12:09.730 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-06T04:12:09.730 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:12:09.780 INFO:tasks.ceph.osd.4:Stopped 2023-10-06T04:12:09.780 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-06T04:12:09.781 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:12:09.825 INFO:tasks.ceph.osd.5:Stopped 2023-10-06T04:12:09.825 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-06T04:12:09.825 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:12:09.866 INFO:tasks.ceph.osd.6:Stopped 2023-10-06T04:12:09.867 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-06T04:12:09.867 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:12:09.916 INFO:tasks.ceph.osd.7:Stopped 2023-10-06T04:12:09.916 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-06T04:12:09.916 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-06T04:12:09.916 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:12:09.939 INFO:tasks.ceph.mgr.y:Stopped 2023-10-06T04:12:09.940 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-06T04:12:09.940 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:12:09.963 INFO:tasks.ceph.mgr.x:Stopped 2023-10-06T04:12:09.963 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-06T04:12:09.963 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-06T04:12:09.964 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:12:09.991 INFO:tasks.ceph.mon.a:Stopped 2023-10-06T04:12:09.992 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-06T04:12:09.992 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:12:10.013 INFO:tasks.ceph.mon.b:Stopped 2023-10-06T04:12:10.013 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-06T04:12:10.013 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:12:10.063 INFO:tasks.ceph.mon.c:Stopped 2023-10-06T04:12:10.063 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-06T04:12:10.063 DEBUG:teuthology.orchestra.run.smithi157:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'Replacing daemon mds' | head -n 1 2023-10-06T04:12:10.095 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi157.front.sepia.ceph.com 2023-10-06T04:12:10.095 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-06T04:12:10.770 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi157.front.sepia.ceph.com 2023-10-06T04:12:10.770 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-06T04:12:10.876 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi157.front.sepia.ceph.com 2023-10-06T04:12:10.876 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-06T04:12:10.949 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi157.front.sepia.ceph.com 2023-10-06T04:12:10.950 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-06T04:12:11.031 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi175.front.sepia.ceph.com 2023-10-06T04:12:11.031 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-06T04:12:11.206 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi175.front.sepia.ceph.com 2023-10-06T04:12:11.206 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-06T04:12:11.298 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi175.front.sepia.ceph.com 2023-10-06T04:12:11.299 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-06T04:12:11.364 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi175.front.sepia.ceph.com 2023-10-06T04:12:11.364 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-06T04:12:11.425 INFO:tasks.ceph:Archiving mon data... 2023-10-06T04:12:11.427 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412234/data/mon.a.tgz 2023-10-06T04:12:11.427 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2023-10-06T04:12:11.443 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.xtEKfZLf2r 2023-10-06T04:12:11.443 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.xtEKfZLf2r 2023-10-06T04:12:11.581 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.xtEKfZLf2r 2023-10-06T04:12:11.663 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.xtEKfZLf2r is 350KB 2023-10-06T04:12:11.759 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.xtEKfZLf2r 2023-10-06T04:12:11.773 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412234/data/mon.b.tgz 2023-10-06T04:12:11.773 DEBUG:teuthology.orchestra.run.smithi175:> mktemp 2023-10-06T04:12:11.790 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.tNwia5ybZM 2023-10-06T04:12:11.790 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.tNwia5ybZM 2023-10-06T04:12:11.946 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.tNwia5ybZM 2023-10-06T04:12:12.027 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.tNwia5ybZM is 359KB 2023-10-06T04:12:12.123 DEBUG:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.tNwia5ybZM 2023-10-06T04:12:12.137 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412234/data/mon.c.tgz 2023-10-06T04:12:12.137 DEBUG:teuthology.orchestra.run.smithi175:> mktemp 2023-10-06T04:12:12.190 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.tWpe4kn4mv 2023-10-06T04:12:12.191 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.tWpe4kn4mv 2023-10-06T04:12:12.319 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.tWpe4kn4mv 2023-10-06T04:12:12.398 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.tWpe4kn4mv is 359KB 2023-10-06T04:12:12.492 DEBUG:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.tWpe4kn4mv 2023-10-06T04:12:12.505 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-06T04:12:12.506 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T04:12:12.508 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T04:12:12.568 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-06T04:12:12.568 DEBUG:teuthology.orchestra.run.smithi157:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T04:12:12.573 DEBUG:teuthology.orchestra.run.smithi175:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T04:12:12.596 INFO:teuthology.orchestra.run.smithi157.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T04:12:12.632 INFO:teuthology.orchestra.run.smithi175.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T04:12:12.637 INFO:tasks.ceph:Archiving crash dumps... 2023-10-06T04:12:12.639 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412234/remote/smithi157/crash 2023-10-06T04:12:12.640 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T04:12:12.667 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412234/remote/smithi175/crash 2023-10-06T04:12:12.668 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T04:12:12.701 INFO:tasks.ceph:Shutting down logrotate 2023-10-06T04:12:12.702 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:12:12.711 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:12:12.804 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T04:12:12.828 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T04:12:12.852 INFO:tasks.ceph:Compressing logs... 2023-10-06T04:12:12.852 DEBUG:teuthology.orchestra.run.smithi157:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T04:12:12.871 DEBUG:teuthology.orchestra.run.smithi175:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T04:13:06.587 INFO:tasks.ceph:Archiving logs... 2023-10-06T04:13:06.588 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412234/remote/smithi157/log 2023-10-06T04:13:06.589 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T04:13:13.223 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412234/remote/smithi175/log 2023-10-06T04:13:13.224 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T04:13:16.771 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-06T04:13:16.782 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-06T04:13:16.782 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T04:13:16.785 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T04:13:16.871 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-06T04:13:16.881 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-06T04:13:16.882 DEBUG:teuthology.orchestra.run.smithi157:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T04:13:16.884 DEBUG:teuthology.orchestra.run.smithi175:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T04:13:16.896 INFO:teuthology.orchestra.run.smithi157.stderr:bash: ntpq: command not found 2023-10-06T04:13:16.899 INFO:teuthology.orchestra.run.smithi157.stdout:210 Number of sources = 4 2023-10-06T04:13:16.901 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T04:13:16.901 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2023-10-06T04:13:16.901 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 43 +561us[ +561us] +/- 103ms 2023-10-06T04:13:16.902 INFO:teuthology.orchestra.run.smithi157.stdout:^* hv02.front.sepia.ceph.com 3 6 377 43 -433us[ -426us] +/- 56ms 2023-10-06T04:13:16.902 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 40 +188us[ +188us] +/- 63ms 2023-10-06T04:13:16.902 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T04:13:16.903 INFO:teuthology.orchestra.run.smithi175.stderr:bash: ntpq: command not found 2023-10-06T04:13:16.903 INFO:teuthology.orchestra.run.smithi175.stdout:210 Number of sources = 4 2023-10-06T04:13:16.904 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T04:13:16.904 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2023-10-06T04:13:16.904 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 45 +555us[ +565us] +/- 103ms 2023-10-06T04:13:16.904 INFO:teuthology.orchestra.run.smithi175.stdout:^* hv02.front.sepia.ceph.com 3 6 377 43 -435us[ -425us] +/- 56ms 2023-10-06T04:13:16.904 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 +183us[ +183us] +/- 63ms 2023-10-06T04:13:16.904 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T04:13:16.905 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-06T04:13:16.916 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-06T04:13:16.916 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-06T04:13:16.928 DEBUG:teuthology.orchestra.run.smithi157:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T04:13:17.037 DEBUG:teuthology.orchestra.run.smithi175:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T04:13:17.131 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T04:13:17.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:13:17.162 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T04:13:17.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:13:17.192 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-06T04:13:17.202 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-06T04:13:17.220 INFO:teuthology.task.internal:Duration was 1635.935685 seconds 2023-10-06T04:13:17.220 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-06T04:13:17.232 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-06T04:13:17.233 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T04:13:17.235 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T04:13:17.276 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T04:13:17.279 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T04:13:17.586 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-06T04:13:17.586 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi157.front.sepia.ceph.com 2023-10-06T04:13:17.586 DEBUG:teuthology.orchestra.run.smithi157:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T04:13:17.645 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi175.front.sepia.ceph.com 2023-10-06T04:13:17.646 DEBUG:teuthology.orchestra.run.smithi175:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T04:13:17.665 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-06T04:13:17.666 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T04:13:17.689 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T04:13:17.744 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-06T04:13:17.745 DEBUG:teuthology.orchestra.run.smithi157:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T04:13:17.772 DEBUG:teuthology.orchestra.run.smithi175:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T04:13:17.856 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-06T04:13:17.867 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-06T04:13:17.867 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T04:13:17.892 DEBUG:teuthology.orchestra.run.smithi175:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T04:13:17.916 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-06T04:13:17.930 DEBUG:teuthology.orchestra.run.smithi157:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T04:13:17.934 DEBUG:teuthology.orchestra.run.smithi175:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T04:13:17.956 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = core 2023-10-06T04:13:17.978 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = core 2023-10-06T04:13:17.991 DEBUG:teuthology.orchestra.run.smithi157:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T04:13:18.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:13:18.024 DEBUG:teuthology.orchestra.run.smithi175:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T04:13:18.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:13:18.044 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-06T04:13:18.055 INFO:teuthology.task.internal:Transferring archived files... 2023-10-06T04:13:18.056 DEBUG:teuthology.misc:Transferring archived files from smithi157:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412234/remote/smithi157 2023-10-06T04:13:18.056 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T04:13:18.107 DEBUG:teuthology.misc:Transferring archived files from smithi175:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412234/remote/smithi175 2023-10-06T04:13:18.108 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T04:13:18.189 INFO:teuthology.task.internal:Removing archive directory... 2023-10-06T04:13:18.189 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T04:13:18.201 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T04:13:18.243 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-06T04:13:18.261 INFO:teuthology.task.internal:Not uploading archives. 2023-10-06T04:13:18.261 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-06T04:13:18.283 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-06T04:13:18.284 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T04:13:18.299 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T04:13:18.310 INFO:teuthology.orchestra.run.smithi157.stdout: 262176 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 04:13 /home/ubuntu/cephtest 2023-10-06T04:13:18.315 INFO:teuthology.orchestra.run.smithi175.stdout: 262149 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 04:13 /home/ubuntu/cephtest 2023-10-06T04:13:18.316 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-06T04:13:18.334 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-06T04:13:18.411 INFO:teuthology.nuke:Checking targets against current locks 2023-10-06T04:13:18.440 DEBUG:teuthology.nuke:shortname: smithi157 2023-10-06T04:13:18.441 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T04:13:18.466 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412234', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 03:37:55.989510', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBqwONQQ9SRgP7wl/eGNxCKZEU6YhxyvvULbzoPkEvm/SMYB0MAzt8PvgJdbr+t7Aux/At50nJ+Dt9rZoU+QuSw='} 2023-10-06T04:13:18.470 DEBUG:teuthology.nuke:shortname: smithi175 2023-10-06T04:13:18.470 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T04:13:18.493 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412234', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 03:37:55.987883', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMD1lY8k/JMCThp+vPfiNZh2sGw9juoPiQx7FaD+ZXMGbasjgL2nRuw9URi7/tDh0qjtjGWE635fMHv52/5FaFg='} 2023-10-06T04:13:18.526 INFO:teuthology.orchestra.console.smithi157:Power off 2023-10-06T04:13:18.527 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T04:13:18.555 DEBUG:teuthology.orchestra.console.smithi157:power off output: Chassis Power Control: Down/Off 2023-10-06T04:13:18.556 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T04:13:18.575 INFO:teuthology.orchestra.console.smithi175:Power off 2023-10-06T04:13:18.576 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T04:13:18.597 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2023-10-06T04:13:18.611 DEBUG:teuthology.orchestra.console.smithi175:power off output: Chassis Power Control: Down/Off 2023-10-06T04:13:18.611 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T04:13:18.640 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is on 2023-10-06T04:13:22.598 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T04:13:22.641 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T04:13:22.712 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2023-10-06T04:13:22.753 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is on 2023-10-06T04:13:26.714 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T04:13:26.754 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T04:13:26.825 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off 2023-10-06T04:13:26.864 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is on 2023-10-06T04:13:26.926 INFO:teuthology.orchestra.console.smithi157:Power off completed 2023-10-06T04:13:30.866 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T04:13:30.977 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is off 2023-10-06T04:13:31.078 INFO:teuthology.orchestra.console.smithi175:Power off completed 2023-10-06T04:13:31.184 INFO:teuthology.run:Summary data: description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/snaps}} duration: 1635.9356851577759 failure_reason: 'Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi157 with status 128: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh''' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=31663c36722e41cf9a6e95b258d87ac8 status: fail success: false 2023-10-06T04:13:31.184 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T04:13:31.264 INFO:teuthology.run:FAIL